In a recent post to his email list, Jonathan Stark talked about giving guarantees for your work.  He gave the example of offering a “bug free” guarantee for software development work. All the testing hairs on my arm stood up and I wanted to protest that there is no such thing as bug free software!

But then I let another testing instinct kick in – curiosity.  I started thinking about how he can do this. How can he offer a guarantee that he will deliver bug free software? Is there such a thing as bug free software?

I came to a conclusion that yes there is.  There really can be bug free software.

The context for that statement has to be just right though. If you are delivering a piece of software that has only one (or a very limited number of) stakeholder that gets to decide if something is bug free, I think you can indeed make the software bug free. There might still be logic errors in the code somewhere. There might even be things that you as a tester could find that are ‘wrong.’ But if the client agrees that the software does what they need it to in a way that is reasonable for them and if they agree that there are no issues that prevent them from getting the value they need out of it, you have indeed delivered bug free software.

I think a bug is something that threatens the value of the product. If you have a limited scope product and limited number of stakeholders, you should be able to make something that has no threats to it’s value. You would need to do this through careful consultation and iteration with the customer. They would have to tell you where they are not getting the value they need and you would need to work closely with them to remove all bugs (that matter to them) from the product, but it should be possible to do.

I want to try a more difficult thought experiment though. Take this into general purpose software with many thousands or millions of users with competing demands, skill levels and expectations. In this context can you create bug free software?

I’m going to take up that thought experiment in the next post. Stay tuned!

 

Photo by pan xiaozhen on Unsplash

 

Advertisements

8 thoughts on “How to Create Bug Free Software

  1. Interesting! I like your definition of a bug “something that threatens the value of the product” ☺.

    Sam

    From: offbeattesting
    Reply-To: offbeattesting
    Date: Monday, January 14, 2019 at 7:40 AM
    To: Sam Chandrashekar
    Subject: [New post] How to Create Bug Free Software

    CAUTION: This email originated from outside of D2L. Do not respond to, click links or open attachments unless you recognize the sender and know the content is safe.

    offbeattesting posted: “In a recent post to his email list, Jonathan Stark talked about giving guarantees for your work. He gave the example of offering a “bug free” guarantee for software development work. All the testing hairs on my arm stood up and I wanted to protest that t”

    Like

  2. Interesting idea! What happens if/when a problem crops up, that threatens the value of the product, after the relevant stakeholders have agreed the software is “bug-free”?

    To my mind, the risk of stating software is bug-free is that it assumes that we can know with at-or-near 100% certainty that there are no behaviours in the software, intentional or not, that have a negative impact on the value to a relevant stakeholder. Maybe in software that isn’t particularly complex, running in environments (both hardware and OS) that are relatively simple as well?

    Like

    1. Ahh! Very important question! Just because it is bug free now, doesn’t mean it will stay that way. Environments change. Business needs change. Maybe the software itself changes. I think this is essentially the regression testing problem summarized and I think it’s a really part of this discussion

      I have a couple of more posts I want to do on ‘bug free’ software and in one of them I want to talk about this problem, so guess stay tuned for more discussion on this 🙂

      Also, as an aside, I love how you phrased it: “The risk of stating software is bug free…”

      When providing a guarantee of any sort (including a bug free guarantee) you are essentially taking risk away from the client and on to yourself. This concept of risk (and it’s relation to value) is bouncing around in my head right now too and will probably come out in a future ‘bug free’ post.

      Liked by 1 person

      1. I’d argue that just because you THINK it’s bug free, doesn’t mean it actually IS. Bugs can lie undiscovered for a long time, even without any actual changes. Admittedly, they’re often edge cases, but that doesn’t make them any less of a bug, or reduce the potential impact. For example, look at the Heartbleed bug in OpenSSH – it was in the wild for about 2 years before anyone discovered it.

        Liked by 1 person

  3. Totally agree. But if we define a bug as something that impacts customer value, is it really a bug if it has not been found (by anyone)? It is a potential bug for sure, and once it has been found it is a bug, but in the meantime if everyone is happy and it is not really causing damage is it really a bug?

    I’m playing around a bit with semantics and definitions here, but I guess the point I’m trying to get it is that the only bugs that really matter are those that ‘bug’ somebody who matters.

    Now, how important it is to find potential bugs (like heartbleed), is another interesting question. I’m loving the discussion on this post in various forums as it is giving me a lot to think about. Thanks for your interactions!

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s