Forget About Bugs

Forget About Bugs

Ask my wife how forgetful I am. You’ll probably get a big sigh. Case in point; the other day we were on our way home and we were about 30 seconds away from turning onto the side street we live on. My wife said to me “don’t forget that we need to go to the grocery store.”

“Yes dear”

30 seconds later.

“Why are your turning onto our road! We need to go to the grocery store!”

Yes, I can be that forgetful. I know that I’m forgetful and so I have systems in place to help myself with this. I try to write things down and use software to remind of things when I can. I might be an extreme case, but I think we all forget stuff sometimes.  It happens in software development as well. This is why we have stories, and documentation and defects.

However, sometimes I want to forget things. Take bugs for example. Having a big bug backlog just slows us down and I don’t want to go through it every week or two and see at the things we haven’t fixed. So what can I do to make bugs more forgettable?

Well let’s consider a few ideas:

I could fix it myself. Who says only developers can fix bugs? Sometime I know how to fix it myself or can figure it out with a bit of work, so why don’t I just do that?

I could immediately work with a developer to fix it. Instead of filing a bug and sticking it in some backlog, I could chat with a developer who know that area of the code and see if we can sit down and figure out the issue and fix it on the spot.

I could work with PM to make into a story for the back log. Sometime a bug doesn’t have to be a bug and can just be prioritized with other work we are doing

I could walk away. Maybe this bug is ok to ignore.  Maybe I can just embrace my forgetfulness, look the other way, and go find something more important to fix or work on.

Of course sometimes a bug does need to be remembered, but wouldn’t it be nice to be able to forgot about them more often?

 

Photo by AbsolutVision on Unsplash

API Testing – Getting Started

API Testing – Getting Started

In many ways APIs are the glue of the internet. The modern internet is increasingly built on distributed architectures and companies want to make their platforms extensible. APIs are a crucial part of these kind of business strategies. (If you aren’t sure about API terminology, check out my API testing glossary here).  APIs are also challenging to design and test. By their very nature they are meant to be accessed programmatically and to expose underlying constructs in ways that are simple to use and understand. You might be comfortable with how to test manually or even with various types of test automation, but not really know and understand how to effectively test an API.

I have been working on testing APIs for a while now and have also put together a course that I hope will help people looking to get started with API testing.  I also want to share some material for this on my blog in the hope that I can help others along this path towards API testing skills.

Let’s start with trying to understand a bit about web services. Many web services provide APIs that can be used to drive them and so understanding a bit about what is meant by the term web services is as a good a place as any to start with in API testing. As with any term in technology, it can take on different nuances for different companies and contexts but I find that a helpful way to think about it is actually in terms of mathematical functions. Don’t worry though, no complex math is needed to understand this.

When I was first introduced to functions in grade school I learned to think of them as a black box that took in some inputs and did some kind of transformation on those inputs to then produce an output. I use this same model to think of a web service.

Let’s make it concrete and look at a real life example. Let’s say we are testing an application and we want to get a first and last name to use for a login test.  We don’t want to use the same name every time, but we still want to have somewhat realistic names.  We could find a database of names and write some code that randomly selects names from it, or we could go to a service like uinames.com and get it to generate the name. By giving the service certain inputs (like an API path and perhaps some other request parameters), we can get back a user name that we can use.

We don’t know anything about how that service does what it does. It might use python, or php or javascript to do this work. It might get the names from a database, or it might request them from other services. We don’t know. We can just think of it as a black box that will give us back certain inputs based on the inputs we give. And that’s really all we need to know at this point. This is how I think of a web service. A black box that does some action(s) for you based on the inputs you give it and then gives you an input. In this way of thinking, APIs are the way we get inputs into the service and how the service gives us back the responses it has for us.

I want to start a bit of a series on API testing where we think of it this way.  We could call this black box API testing if we really want to give a name to it. I think it is an important part of interacting with and testing the modern web and I hope you find this that this series helps you along your API testing journey.

How to Become a Great Tester

How to Become a Great Tester

I want you to think about butterflies. The beautiful feathery winged creatures that flit from one flower to another. The weather is warming up, and yesterday my son and I were marveling at the beauty of these little creatures.

Now, let’s go back a bit in the development life-cycle of a butterfly. Let’s go back to the moment a butterfly starts to emerge from the chrysalis. The story is that if you feel sorry for the butterfly and cut the chrysalis open, it won’t be able to open it’s wings.  The struggle is necessary for the blood flow. Now, as with most inspirational stories you hear, it is probably not strictly true, but I think there are still a few lessons that can be drawn from it about how to become a great tester.

Struggle is important for becoming good at your craft. You can really only learn some things through the struggle of trying to do them. Sometimes we can try to short circuit the pain of the learning process, but at the end of the day you aren’t going to be able to learn without some struggle.

This isn’t the only lesson to learn though. The other lesson we can glean comes from the fact that this story is probably not strictly true. A couple of quick google searches shows experts who say that as long as you are careful, there are ways to help a butterfly that is having trouble getting out of the cocoon. Struggle is good and helpful and shouldn’t be short circuited, but there is a time to get help.

So how do you become a great tester?

Do lots of testing and struggle through the pain of trying to figure it out. Don’t think you can just read a few blog posts and listen to few talks and then have what it takes to be a great tester. It will take time. Lots of time. It will take trying things and failing and trying again.  It will take getting stuck and getting frustrated. It will take pain and joy. It will take frustration and annoyance and aha! moments and a powerful sense of accomplishment. In the struggle you will slowly grow and become a better and better tester.

Embrace the struggle, but don’t get lost in the struggle. Don’t forget to ask for help. Find those voices that can speak to you on how to get through the struggles you face. Go to the conferences. Ask the questions. Pair up with the experienced tester. Read the books. Take the course. Get help.

Becoming a great tester takes time and persistence. It also takes help and encouragement.

Happy testing!

 

Photo by Suzanne D. Williams on Unsplash

The Bug is Gone?

The Bug is Gone?

“Now that is weird.”

I stared at my screen and puzzled over the problem. How could this happen?

It had all started with a 500 error due to a missing config. When I checked the config panel, sure enough, the config was missing. We were searching through pull requests trying to figure out what had gone wrong, when one of my co-workers said that it seemed to be there for her.

So I refreshed the page. And there it was.

How can that be? We didn’t make any code changes to the site so how can a config that wasn’t there a minute ago just appear?

But oh well right? The initial thing I was trying to do was working now, so maybe it was just some network glitch or something.  I felt the temptation to just shake my head, attribute it to the whims of the software gods, and move on. But I decided to be curious and to see if I could find out anything else.

I posted on our company’s testing slack channel asking for ideas and within minutes I had a response that led to understanding the issue. We were creating the variable too late in the process and so the site didn’t know about it the first time we tried to use it. We got a fix together and took care of the issue, but I think there are a few important lessons to learn.

Be curious – It would have been easy to shrug my shoulders and move on, but that would have led to a bunch of clients hitting weird random errors when they first started using the feature.  Not a good first impression!

It takes a team –  I might have noticed the issue, but without the help of others it would have taken a long time to understand what was going on. We can’t know everything about everything. It really does take a team to solve problems like this

Open communication – A team isn’t a team without good communication and if it wasn’t for open communication channels like Slack, it would have taken much longer to get to the bottom of an issue like this.

Intermittent bugs are still bugs –  Just because you can’t reproduce something, doesn’t mean it doesn’t matter. The only way to reproduce this bug is to create a whole new site. The problem is we create hundreds of sites for clients for each release. So even though I can’t reproduce this bug (at least not without spinning up a whole new site), it would still be a high impact bug.  The more users and usage you have, the more important intermittent bugs become. Don’t just ignore them!

Did that bug really go away? Don’t assume it too quickly!

 

Photo by Sebastian Pichler on Unsplash

Working with TestProject

Working with TestProject

Note that this is a sponsored post. Posts like this will only ever be about things that I believe and that I think you would potentially find useful. You can read about my full philosophy on sponsored posts here

I always manage to keep a full plate in life. I have many interests and responsibilities. Sometimes it can feel like life is too busy, but I really do enjoy the many different things I do. One interesting project I have been working on lately is a collaboration I have been doing with TestProject. Their goal is to provide the testing community with a community powered platform that helps with recording, developing and analyzing test automation.

I have been working with them on creating some video tutorials that explain how the tool works and highlights some of its benefits. If you want to check out some of the videos I’ve made, you can go to the TestProject page and click on the “Watch how it works” button to hear me talk about the high level overview of the product, or you can go to the videos section to find my other tutorials. There is also a YouTube playlist that has them all in one place.

As I have been putting these together, I thought I would stop and reflect a little on this process. I want to look at it from two perspectives. In the first place, I want to reflect a little bit on the tool itself and in the second place I want to talk a bit about what it has been like to work with the TestProject team

Ok. So the tool itself. I find that I learn the most about something when I teach others how to use it. In making these tutorials I have been learning a lot about TestProject and I have to say that overall I am pretty impressed by the tool. I have been making some videos, for example on the smart test recorder. UI automation is notoriously hard to do well, but I have to say that the smart test recorder really does a good job of setting you up for success. It makes creating test scripts straightforward and easy, and it also does a good job of creating the necessary levels of abstraction for you. For example it automatically saves elements for you so that you can share them between tests. It also allows you to call one test from another test, which allows you to follow the ideas of the page object model. We usually think of the page object model as something you would do in coded tests, but TestProject makes it easy to take this test design principle and put it into action in high level UI tests as well.

There are also a lot of other helpful things that TestProject does. It takes care of all the dirty work of getting the selenium and appium drivers installed and setup and is honestly the fastest and simplest way I have come across for getting you started with tools like this. I have started using TestProject to automate some of the boring things I have to do. For example, if I want to start doing some exploratory testing on a new build, I have a TestProject script I can run to setup some stuff in the way I want it to, so that I can just dive right into the part of the testing that I am currently interested in. This is something I have done with Selenium in the past, but I found it be a lot easier to get started when using TestProject.

I think this could be especially powerful if you don’t know how to code since you can use the smart test recorder to set this all up for you with very little difficulty (and without needing to know a single line of code). You can of course also use the TestProject SDK to create coded tests or even export your recorded tests to code if you wanted to dive a little deeper.

TestProject also works well for creating regression tests that you run across browsers and platforms. It instantly recognizes any devices you have attached and lets you create and run tests on applications you have on those devices as well. It also does all this in a way that is designed around collaboration. They really are seeking to provide a solution that is team based and you can see that in the way that all tests and run time environments etc. can be shared with your team.

The community driven aspect of TestProject brings me to the second point I wanted to talk about which is what it has been like working with the TestProject team. I don’t have a ton to say on this front, but I did want to mention what a great experience it has been for me. The team members I have interacted with there have really made me feel like my input and expertise is valued and important and I can see in those interactions why they have a free forever plan. They really are committed to making things better for the testing community. Just to be clear the free forever plan isn’t just a ‘give you just enough to make you want to upgrade.’ It is something that would be really valuable just on its own including things like unlimited test execution and access to all important functionality like the addons library, smart test recorder, reporting and much more. I’d encourage you to check it out and sign up.

I really think this a good tool and a welcome addition to the list of testing tools out there. It has been a lot of fun working on creating tutorials that explain what it does and I’m looking forward to continuing to learn and use this tool.

Why Create Bug Free Software?

Why Create Bug Free Software?

I’ve been talking about creating bug free software in my last two  posts. In this article I want to think about why we might want to do this.

Bug free software (as I have narrowly defined it in these posts), is valuable. If you can really deliver software that solves a customer problem in a way that they agree does not create any bugs, you have something that is incredibly valuable to them. However, it will most likely also be expensive. Figuring out what exactly the customer needs and working with them to deliver exactly that, is cost and time intensive. We all know that good testing takes time.

I think there is a place for dollar stores and other providers of low cost goods.  There is also a place for low cost providers of software solutions.  If all you need is a quick and dirty solution that doesn’t last long, a piece of software with a lot of issues that ‘bug’ you might be just fine. But if you are dealing with a long term and important problem that can be solved with software, do you really want a dollar store solution for it? Is it worth it to go back to the dollar store every two day to replace that broken item?  Or is it perhaps worth getting a higher quality item that will last and do exactly what you need it to?

I think this metaphor can be applied to software.  If your software solves a problem well, and does so in a way that doesn’t feel like a dollar store toy that breaks 2 minutes after your kid starts playing with it, you have something that is valuable.

When you have something that is valuable you can get people to give you money for it, and then you have a successful business. So caring about quality, and the effort that goes into it is a great way to create value for the business. Taking time to invest in good automation and learn good coding practices; taking the time to reach out to customers and understand usage patterns; taking the time to keep the code base understandable; taking the time to find and fix things that prevent customers from efficiently solving their problems; all of these things contribute to having valuable software.

I very often see companies getting caught up in the need to release new features and make more exciting stuff that they can market and sell.  Those things matter and are ways to add value and make money, but we need to remember that working on the quality of our existing solutions is also a way to add value.  The better we solve the customer’s problems that more valuable our software will be. And I think it is a pretty simple inference to say that the more valuable our software is, the more potential there is to make money.

So why create ‘bug free’ software?  Well in a nutshell, because it is valuable.

Let me close with one parting thought: How valuable would it be to the company to have someone (say a tester) that can demonstrably help them get closer to that bug free state?

 

Photo by Niels Steeman on Unsplash

Bug Free Software at Scale?

Bug Free Software at Scale?

In my last post I proposed that it is possible to create bug free software if the context is very limited and the number of stakeholders who determine if something is buggy is small.  In the discussion on that post in various forums we fleshed out a bit more the idea that this is limited to a snapshot in time, since new context or stakeholders could change the things that are considered to be bugs.

Can this apply at scale though? If you have a large scale software product with thousands or millions of users and stakeholders can it be bug free?

I think here the answer is no. But I think it is a softer no that I would have used before

No, you can’t simultaneously satisfy the competing demands of many different customers. But there are still some powerful techniques that we can use. If we use the same definition of a bug that we did in the last post. (i.e. something that is a threat to stakeholder value), we can actually get pretty close to a bug free state. If we understand enough about what our customers value and aggressively fix things that get in the way of this, we can make very valuable products. We might not ever get to a fully bug free state, but we can get pretty close.

I guess in way what I am saying is that a 0 bug backlog way of working is a possibility. If you are willing to put in the work. They key here is understanding that a bug involves a relationship. Bugs don’t exist without people. They don’t exist if they impact something no-one cares about or uses. If we think back to that limited scope bug free project in the last post, there could be any number of things that are in there that they programmer did not intend the software to do. But if those things do not affect the stakeholder, are they really bugs? A bug isn’t a bug, until it impacts someone that matters.

Working from this, leads to an important point about being able to achieve quality at scale. We won’t get to bug free, but we can get to high quality if we focus aggressively on fixing actual bugs. A lot of times we end up working on things that we think are bugs, or things that might be bugs, but we don’t consider the relationship aspect of a bug. Who is this bug bugging? Why does it matter?

As a testing profession we have tended to focus on what I’m going to call ‘potential bugs.’ These are issues like logic errors or crashes or really anything that gets filed in a bug report, but they are found before they impact any of the stakeholders. I’m not against finding and fixing potential bugs, but we need to remember that there is importance to finding and fixing actual bugs. Bugs that are currently bugging someone, not just bugs that might bug someone if we release them. We need to find and fix things that are actually, right now, threatening the value of our product. We need to get rid of the things that are currently bugging someone. As the proverb goes, a bird in the hand is worth two in the bush.

This also means that the goal of a nearly bug free product will lead us to prioritize the fixing of these actual bugs over creating new features. Businesses don’t always like this, especially in a highly competitive landscape where time to market for features is important, but if your goal is have a high quality product, you need to prioritize in this way.

This gets expensive. It takes time to find and fix bugs. Time that could be spent on making new features. It also add a lot of interruptions. If we prioritize fixing actual bugs over feature delivery we are going to sometime end up in a responsive mode. Remember the comments about how ‘bug free’ is a time limited snapshot? Our system and users and environments change over time and so even though we might be bug free today, that does not mean we will be so tomorrow.  Again this approach is expensive. It has real costs associated with it.

And that is what I want to talk about in the next post in this series. The interplay between the cost and value of creating ‘nearly bug free’ software. Is this cost worth it?

What do you think?  I’ll share my thoughts in the next post. Stay tuned!

 

Photo by rawpixel on Unsplash