Bug Free Software at Scale?

Bug Free Software at Scale?

In my last post I proposed that it is possible to create bug free software if the context is very limited and the number of stakeholders who determine if something is buggy is small.  In the discussion on that post in various forums we fleshed out a bit more the idea that this is limited to a snapshot in time, since new context or stakeholders could change the things that are considered to be bugs.

Can this apply at scale though? If you have a large scale software product with thousands or millions of users and stakeholders can it be bug free?

I think here the answer is no. But I think it is a softer no that I would have used before

No, you can’t simultaneously satisfy the competing demands of many different customers. But there are still some powerful techniques that we can use. If we use the same definition of a bug that we did in the last post. (i.e. something that is a threat to stakeholder value), we can actually get pretty close to a bug free state. If we understand enough about what our customers value and aggressively fix things that get in the way of this, we can make very valuable products. We might not ever get to a fully bug free state, but we can get pretty close.

I guess in way what I am saying is that a 0 bug backlog way of working is a possibility. If you are willing to put in the work. They key here is understanding that a bug involves a relationship. Bugs don’t exist without people. They don’t exist if they impact something no-one cares about or uses. If we think back to that limited scope bug free project in the last post, there could be any number of things that are in there that they programmer did not intend the software to do. But if those things do not affect the stakeholder, are they really bugs? A bug isn’t a bug, until it impacts someone that matters.

Working from this, leads to an important point about being able to achieve quality at scale. We won’t get to bug free, but we can get to high quality if we focus aggressively on fixing actual bugs. A lot of times we end up working on things that we think are bugs, or things that might be bugs, but we don’t consider the relationship aspect of a bug. Who is this bug bugging? Why does it matter?

As a testing profession we have tended to focus on what I’m going to call ‘potential bugs.’ These are issues like logic errors or crashes or really anything that gets filed in a bug report, but they are found before they impact any of the stakeholders. I’m not against finding and fixing potential bugs, but we need to remember that there is importance to finding and fixing actual bugs. Bugs that are currently bugging someone, not just bugs that might bug someone if we release them. We need to find and fix things that are actually, right now, threatening the value of our product. We need to get rid of the things that are currently bugging someone. As the proverb goes, a bird in the hand is worth two in the bush.

This also means that the goal of a nearly bug free product will lead us to prioritize the fixing of these actual bugs over creating new features. Businesses don’t always like this, especially in a highly competitive landscape where time to market for features is important, but if your goal is have a high quality product, you need to prioritize in this way.

This gets expensive. It takes time to find and fix bugs. Time that could be spent on making new features. It also add a lot of interruptions. If we prioritize fixing actual bugs over feature delivery we are going to sometime end up in a responsive mode. Remember the comments about how ‘bug free’ is a time limited snapshot? Our system and users and environments change over time and so even though we might be bug free today, that does not mean we will be so tomorrow.  Again this approach is expensive. It has real costs associated with it.

And that is what I want to talk about in the next post in this series. The interplay between the cost and value of creating ‘nearly bug free’ software. Is this cost worth it?

What do you think?  I’ll share my thoughts in the next post. Stay tuned!


Photo by rawpixel on Unsplash

How to Create Bug Free Software

How to Create Bug Free Software

In a recent post to his email list, Jonathan Stark talked about giving guarantees for your work.  He gave the example of offering a “bug free” guarantee for software development work. All the testing hairs on my arm stood up and I wanted to protest that there is no such thing as bug free software!

But then I let another testing instinct kick in – curiosity.  I started thinking about how he can do this. How can he offer a guarantee that he will deliver bug free software? Is there such a thing as bug free software?

I came to a conclusion that yes there is.  There really can be bug free software.

The context for that statement has to be just right though. If you are delivering a piece of software that has only one (or a very limited number of) stakeholder that gets to decide if something is bug free, I think you can indeed make the software bug free. There might still be logic errors in the code somewhere. There might even be things that you as a tester could find that are ‘wrong.’ But if the client agrees that the software does what they need it to in a way that is reasonable for them and if they agree that there are no issues that prevent them from getting the value they need out of it, you have indeed delivered bug free software.

I think a bug is something that threatens the value of the product. If you have a limited scope product and limited number of stakeholders, you should be able to make something that has no threats to it’s value. You would need to do this through careful consultation and iteration with the customer. They would have to tell you where they are not getting the value they need and you would need to work closely with them to remove all bugs (that matter to them) from the product, but it should be possible to do.

I want to try a more difficult thought experiment though. Take this into general purpose software with many thousands or millions of users with competing demands, skill levels and expectations. In this context can you create bug free software?

I’m going to take up that thought experiment in the next post. Stay tuned!


Photo by pan xiaozhen on Unsplash