30 Days of Agile Testing – Zero Bug Tolerance

30 Days of Agile Testing – Zero Bug Tolerance

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

Could we do it?  Could we get to a zero bug tolerance on my team?  Well, anything is possible.  I’m sure we could, if we wanted to badly enough. I have even toyed with bringing the idea up in the past, but the problem is that there are many things I want to do and change, but there is only time for so many things. Change is hard and overloading on change will just make you fail at all the changes.  We live in an information saturated world, and one of the biggest challenges anyone faces is the challenge of filtering.  How do we filter out the less important information to find the more important?  There are so many good things to try and to do, but there are only so many hours in a day.

For now I am focused on other things and spending the time on advocating for a zero bug policy just isn’t something I see as being valuable enough (at this time) for me to put the energy it would require into it.  I am fascinated by the idea though and hope that someday we’ll get to the point where we can consider this, but for now – bigger fish to fry.

Advertisements

30 Days of Testing – Test Plan

30 Days of Testing – Test Plan

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

What does my test plan look like?  Well I’ll keep this simple.  It looks like this at the beginning of testing a new feature.

Test Idea
Initial exploration
Does it do thing A?
What about B?

And this is what it looks like part of the way through testing:

Test Idea
Initial exploration
Does it do thing A?
What about B?
Are there any issues with this type of interactions and inputs?
I wonder what happens if I X
Do E,F and G interact with this?

And then as I get close to the end it looks like…well you get the picture right?  The list keeps expanding as I learn more about the feature and as I think of things and try things. I don’t spend a lot of time planning up front. Instead I usually just start with what I have and then pause to plan and re-plan frequently along the way.  Sometime I will take more time to plan out and think through a particular type of coverage if is really important and other times I’ll just merrily go along my way letting my interactions with the product lead me. I any case, I try to keep it as simple and lightweight as possible. The plan itself has very little end value.  It is a tool to help achieve something and I like light and lean tools!

30 Days of Agile Testing – What Can’t be Automated?

30 Days of Agile Testing – What Can’t be Automated?

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

I enjoy automation, and I like to talk about how to make it better.  I do that a lot on this blog, and in fact, I’m giving a talk on that very subject tomorrow. However, today’s challenge is to think about what can’t be automated so let’s dig into that.

Usability

Yes, there are many aspects of usability that automation can help us with.  For example gathering data on your actual customer usage patterns is very helpful for this, but with current technology we still need a lot of human involvement.  If you aren’t using automation to help you with this, you are really missing out, but I think it will be a long time before we see this kind of work being fully automated.

Intuition

Sometime I just know there will be bugs what I try certain things.  How?  I’m not quite sure.  It’s probably a combination of knowing the product, past bad experiences with certain things, knowing how the developer tends to write his code, what other things have recently gone into the code, etc. Whatever it is, there is some intuition going on that I can’t easily automate.  This intuition helps guide my testing and often makes me much faster and more efficient than automation at finding problems.

Bug Advocacy

Finding a bug can be the easy part.  Demonstrating why it matters?  That’s often difficult. One of the things I notice with new testers is that often their bug reports are too factual. Do a, b and c and you get d.  The problem is there is no information about why is undesired or wrong and why we should go about fixing it.  Depending on your team dynamics, this can be an important skill to have.

Figuring out what to do

I can automate a lot of things, but notice that there is an active agent at the start of this sentence.

I.

I can automate a lot of things.  I have to decide what to automate and what to test and what other things to work on. I need to set up the goals we are trying to accomplish in the first place. Automating that job away will take some serious effort.

Communication

Again tools help us here, but we still need a lot of human to human interaction to build software and we can’t just automate that away. Sometimes it feels like we try to do that, but my ability to pull together people and resources and ideas from different areas and synthesize them into something that we as a team can use to get better at doing our jobs, is not something that can be automated.

 

30 Days of Testing – We Aren’t the Only Ones

30 Days of Testing – We Aren’t the Only Ones

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

What kind of testing do others on the team do?  Sure we as testers are tasked with the testing, but we know that others test and in fact, we actively work on helping other improve their testing.  What testing do others do?

We’ve actually had some discussions on our team recently on this in terms of how do we all communicate with each other about what testing has been done.  The developers do some testing of their code, both via adding or running unit tests and by running integration tests against new changes. Developers and product owners also do additional interactive testing at times and others, like our customer engagement team, also use and test the product.  With so many different people testing the product, how do we coordinate the testing so that we are being efficient? This falls into our test management strategy and so we use things like simple shared spreadsheets, that can track at a very basic level, some of the things that have been done.

We also do group testing sessions where we can help and observe each other testing. This helps to teach those with less testing experience on things they can do to improve their testing and also helps us generate new testing ideas. The more collaboration we have with other members of the team, the more testing we can see being done in various ways and areas.  Developers do a lot of testing.  Product owners do a lot of testing. Documentation writers do a lot of testing.  Many people are testing our products and that is a wonderful thing.  As someone who spends a lot of time thinking about good testing and how to be effective at it, I can help and encourage this.  Quality is a team sport!

30 Days of Agile Testing – Managing Testing

30 Days of Agile Testing – Managing Testing

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

How do we manage testing and are we agile about it?

I work for a great manager – he also reads this blog, so you know I have no ulterior motives in saying that. Joking aside, I do work for a great manager and we have an approach to managing testing on our team that makes a lot of sense for the context we are in. Is it capital A Agile?  Probably not, but we don’t work in a capital A kind of company.  For the place that we work it is a great process.  I meets the compromise between the realities of the world we are in and the things that are in the world we would like to be in.  It takes where we are as a company and stretches us just a bit in the right direction. So what does it look like?

The testers all belong to a testing team, but we are also each involved with development teams on particular areas of the product. We are each responsible for and encouraged to grow and develop relationships on those teams. We try to integrate into the development teams and work with them on testing feature early on in the process.  One of the things that we have done is to use group exploratory testing sessions to help us test early and test often, and give feedback to developers in lightweight ways.  As much as possible the approach has been to have a distributed group of testers working with development teams on individual features but also collaborating and sharing experience together.  We try to keep the process as lean as we can and work on close relationships between developers and testers.

This approach has worked fairly well and while it may not be Agile in a strict sense of that term, it illustrates an important agile principle which is to prioritize what works over strictly following a particular process.

30 Days of Agile Testing – Testing Toolkit

30 Days of Agile Testing – Testing Toolkit

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

What is in my toolkit?  What kind of tools do I and my team rely on and use to help us at testing?  I recently got a new laptop and so maybe I’ll just go through some of the tools and programs I installed on it in the last couple of weeks as examples of the kinds of tools I find useful.

Git

Version control systems are a part of most software development team’s tool kits. Knowing and understanding how they work and how to effectively use them can make them a powerful testing tool.  I have been spending some time in the last few weeks learning more about how to read and parse the git log and I’m working on some scripting around it so that we will be able to glean data and insights about changes to our tests and the health of our automation system.  Don’t just treat your version control system as a way to safely make changes, treat is as a way to get information.

Process Explorer

Seriously, if you are a tester and if you haven’t yet tried it, you need to try out process explorer.  As a tester you need to be able to understand what is happening with the various processes on your machine, and process explorer opens this world up for you in a way that Windows task manager can’t.

Everything

This little file search utility is another extremely useful program for a tester to have. How many times have you tried to figure out where your software saved something or if new files have been created somewhere?  This powerful, lightweight search utility has pretty much instantaneous search results and has helped me out so many times.  Another tool I would recommend every tester add to their toolkit

Notepad++

My text editor of choice is Notepad++.  I have tried a few others and I have heard a lot of good things about Sublime text or Visual Studio Code, but so far I haven’t found them compelling enough to switch.  Custom syntax highlighting.  File comparison addons. Find and replace across many files.  Regular expression search capability (if you are a regular expression kind of person).  Some automatic formatting and auto-completion options. There are a lot of powerful and useful features in this text editor and yet is pretty snappy and lightweight.  For the kinds of things I usually do, I find it to work quite well.

Python

Is this a testing tool?  Heck yes!  It’s the most powerful tool in my toolbox.  Now admittedly it’s a tool that takes some time and skill to learn, but once you’ve grasped the basics of it (and this applies to other scripting languages like Ruby), it let’s you leverage yourself in ways you never could have otherwise.  There are so many one off simple tasks a tester wants to do and that you can’t find a specific tool for, but which you can accomplish with a few lines of scripting.  For example we recently had a list of (hundreds of) files that contained specific commands in them and we wanted to run the corresponding tests for those.  There is no tool that could do the conversion, but after 15 minutes with python we had a script that could help us out with this.  Add a scripting language to your testing toolbox!

Sqlite Browser

Sqlite is the database format used in the built in database modules that come with python, and over the years I have written a few webpages that use these kinds of databases.  When trying to debug problems or understand what is happening as we add new features to the webpages, I find sqlite browser to be a very handy and powerful tool.

Phrase Express

I have been using texter for years and I’ve come to depend on auto-completion of certain strings.  With my new (Windows10) laptop, I finally got to the point where the lack of ongoing support for texter caught up with me and so I had to find a new utility.  I’ve been trying out Phrase Express and so far it has been pretty good. It isn’t as lightweight as texter and to be honest, has a lot of functionality I don’t need or use. I would have preferred to have something with less features, but now that I’ve figured it out and gotten it setup the way I need it, it does a decent job.  There are still some little quirks that I don’t like, but the value of being able to define certain keyboard shortcuts that automate some of my typing for me makes this program well worth it.

There are many more tools I use and rely on as a testers (for example I haven’t included any online tools in this list).  A tester should have  well stocked toolbox. What tools do you have in yours?

30 Days of Agile Testing – Helping the Team

30 Days of Agile Testing – Helping the Team

Note that this post is part of a series where I am ‘live blogging’ my way through the ministry of testing’s 30 days of Agile Testing challenge.

Yesterday’s challenge asked how I could make my job easier and today’s asks how I can make my team’s job easier.  I’m going to cheat and use yesterday’s answer today – Automate.

Well, only a partial cheat, because there are actually two teams I want to talk about.  For the first team – my testing team – I think the ‘automate’ answer sums it up well.  There are things I can do (and am currently looking into) that I hope will help the team get better at understanding our tests.  The fact is that many of the things I automate for my own purposes can be shared with other testers and help them out as well, and so when it comes to the testing team, I can probably help them out with my scripting skills.

When I think of my other team – the development team I’m embedded with – my primary skill set isn’t to automate things.  Everyone else on the team writes code far more often than I do and they are well able to automate the things they need to.  The ways that I help this team have little to do with creating scripts, and more to do with reducing friction in our processes.  I want to reduce the number of steps and the amount of time it takes to get code into the main branch.  I want to reduce the friction around running tests.  I want to be able to give more rapid and timely feedback. Along with building a stronger team spirit across development and testing, smoothing out our processes is one of the primary ways I can make life easier for this team.