Last week I talked about how features can sometimes be bugs.  If this is true it means that in all likelihood at least some of your product’s features are bugs and should be removed. This week I want to think about what you as a tester might be able to do about it. We sometimes have trouble convincing people that the bugs we find are actually bugs and now I’m saying we need to convince people that features are bugs?!? How in the world are we supposed to be able to do that?

I could go ahead and give you a lot of ‘do this or do that’ advice, but before we go too far down that path, let me try rephrasing the question.  “How do I convince someone to do some particular thing I see as important.”  Take away the details of the specific thing you are trying to convince someone of and we are left with something that (encouragingly) we have a lot of experience with and (maybe discouragingly) is often very hard to do.  The task here is, at one level, no different than the task of convincing someone that the bug you just found is important.

However, I don’t want to gloss over one big difficulty here and that difficulty is something we could call cultural bias.  Probably in your company (if it is anything like mine) you have a cultural bias towards having more features – that’s what makes us money right? – and so what you are trying to convince people of here is not just that something you all agree is important (fixing a bug) should be done.  What you are trying to do in this case is to convince people that what they believe to be true actually isn’t.  If you’ve been a human for a while or if you pay any attention to politics, you have probably come to realize that this is very hard to do.

Just because something is hard doesn’t mean we should give up on it, but we should be realistic about it.  This will take time, and will be a slow bumpy road.  We should also take lessons from what we see going on in the world around us.  When we see religious and political groups jousting with each other, whatever else we learn, let’s take note that yelling at each other is not the way to change someone’s mind.  As cliche as it sounds, minds are changed through relationships.

If you try to force people to do something they aren’t ready for or don’t see the need of you aren’t going to get to far.  If, however, you have built a relationship and the other party can trust you, you will find that others will give you a chance to explain yourself and will listen to what you have to say.  If they truly believe that you are looking out for their best interests they will be open to listen to and consider what you have to say.  Without a meaningful relationship it is going to be hard to change much of anything.  So look to those that have influence.  Seek them out and see how they  build real meaningful (not manipulative what-can-I-get-out-of-it) relationships. Gently advocate for the positive changes that you think need to happen.  Be ready to listen to and learn from others.  Be patient and stick with it, and over time you will see things change.  As others have said, it is sometimes surprising how little change happens in a year, but also how much can happen in a decade.  Sometimes when we are looking for a quick fix to our problem we just keep chasing after the latest rainbows and unicorns fad, and we don’t end up getting anywhere.  Some things just take good old fashioned work.  Stick with it.  Change the world (says the writer to himself). 

When Features are bugs


The old joke in software engineering is that when a tester files a bug that is too hard to fix, the developer sends it back as ‘working as intended.’  It’s a magical way of making a bug go away – we turn it into a feature instead.  If you have been a tester for any significant amount of time, I’m sure you have experienced something like this.  However, have you ever stopped to think that sometimes the opposite can be true? Sometimes things that we call features are actually bugs.

Now, you might be thinking “slow down cowboy.  If we don’t have features we don’t have a product.”  That is true of course, but consider this quote:

[high] feature count makes users confused (hence it’s a product quality issue) and it adds code complexity, making the code base less amenable to change.1 

The idea of quality is a many faceted thing and plays out in many different ways.  This quote points out two things that affect quality, but that we might not always think of as quality issues.  The first is the idea that user confusion is a quality issue.  If a user can’t find the features they want or need, your product is not valuable to them. Adding features that let one user do something, but that obscures things for other users, can be a quality issue.  Obviously it is OK to add new features, but what kind of features can cause user confusion?

Off the top of my head I can think of a few general questions that could help diagnose if a new feature falls into this category.

  • Is this feature only intended for a very small subset of our customers? – Perhaps it will hurt more customers than it helps?
  • Does this feature fit in well with what our core product offering is?  Is it a whole new type or category of thing?  Does it fit well into one of our existing menus? – If not, we may be adding unnecessary complexity and hence confusion to the product.
  • Is this feature overly similar to other features we have?  – We think by giving users more choices we will make them happier, but the truth is too much choice actually can confuse and leave people dissatisfied.2

I’m sure there are many other questions you could add to this list (go ahead- write some down!), but the point is that by taking a few minutes to think and talk about this aspect of quality, we might be able to keep our product honed, simple and less confusing and in the process we will have created a higher quality product.

A second aspect of quality that the quote above points out is that code complexity and the ease with which you can change your code are quality issues.  This is perhaps a less obvious form of quality as we are used to thinking of quality in terms of things that impact customers, but the reality is that a complex, tangled scary to modify code base is a low quality code base.  We may be able to ‘make it work’ for a  while, but eventually it will catch up with us and the quality issues at the code level with leak out to customers.  Every new feature we add increases the complexity of our code and unless we are very careful about the way we architect and add new features, it will become more and more difficult to respond to the changes our product needs.

The point of this all is that features have a cost.  They don’t just cost you something when you create them.  They continue to have costs down the road.  Maintenance costs and repair cost when things break, and monitoring costs to make sure they don’t.  When you buy a new car you know you will have to spend money on upkeep, and you know you need to check on things once in awhile.  It’s the same way with adding new features.  There are ongoing cost associated with them.  Make sure the feature you add is valuable enough to pay back those costs.  Don’t just add something in because it’s easy to do. Think about the long term costs.  To change the metaphor, don’t buy a house and then realize you can’t pay the property tax and mortgage on it.
A feature that is working as intended might still be hurting the quality of your software.  Don’t let the fact that it is working as intended stop you from pursuing a high quality product both you and your company can be proud of.


1. Quality of Qualities
2. Paradox of Choice

Long live the spreadsheet

One of the problems that most testers have faced is planning and tracking the testing that we do.  The solution that we have traditionally come up with for this is called a Test Case. Over the years we have also come up with many different test case management tools and templates.  Recently I (re)discovered one of the most powerful test case tools out there: the spreadsheet.

One of my colleagues at work has been experimenting with  distributed group exploratory testing sessions and as part of this he created a spreadsheet template (in google docs) for collaboratively tracking issues found in the group exploratory session.  Cool idea right?  Well I was working on testing a feature and I decided to use one of these spreadsheet templates to track stuff that I was finding and I was once again reminded that sometimes the most powerful tools are the simple tried and true ones.

I think many testers realize that there are flaws in the standard approach to test cases.  Creating test cases up front (that then rapidly go out of date) and trying to think about everything before hand has a lot of inefficiencies, but what do we do instead?  How do we approach test cases?  Using a simple spreadsheet has actually been working very well for me.  The way it works is we have a column for the submitter’s name,one for the description of the issue, and another to categorize the type of problem. The category column is probably the most powerful part of it.  If you have something you are confused about, just write it down and set the category to Question.  If you have UX feedback set the category to UX Feedback.  If it is a defect set the category to Defect and if it is a future testing idea, set the category to New Test Idea.

Using this approach allows for a lot of low cost and yet powerful tracking and filtering of the testing being done, and – in Google sheets at least – allows you to easily pull in relevant people by tagging them in a comment.  This approach also allows you to quickly get ideas down without causing too much of a disruption in the flow of your testing while giving you a record to come back to later and follow up on things with. It can also be setup to give you a pretty quick summary of the status of the feature as well.  So before you fork over the money for an expensive test case management tool, give the good ole spreadsheet a try.  It’s not perfect (For examples screenshots are a bit tricky), but it can be very effect at a very low cost.



TestLinks Tuesday

A few links worth pursuing on a Tuesday afternoon

Technology and the future (AOM podcast) – If you work in technology you might as well understand where it is going and how to use it right?

Quality comes in many forms This is from a couple of years ago, but still some really good stuff in here: “Developer-facing quality is a completely different thing from end-user facing quality, and is usually more important.”

The Expensive Fear of Forgetting The always on point Maaret come through with another post for pondering. How much of the paperwork that we do is because we are afraid of forgetting?  How expensive is it to not forget?

Pavlov’s in your pocket Maybe it is hard to change behavior because there are hidden rewards for the existing behavior.

Exercise, Collaboration and Software Testing

I grew up in the country and spent my summers during highschool working on a nearby vegetable farm picking produce.  Collaboration in that environment was pretty easy.  We each had  role – you drive the machine, you pick the cucumbers, you pick up the full buckets, etc. – and as long we each did our thing we didn’t need to worry too much about what others where doing.  We also got plenty of exercise – the good ‘ole days right?

But now I am a software tester, a knowledge worker, and effective knowledge work requires collaboration.  In fact, I think figuring out how to effectively collaborate is one of the biggest challenges we face in the software industry and this also effects us as testers.  For any number of reasons, from things like trying to apply manufacturing management strategies to knowledge workers, all to way to company cultures that encourage a individualistic hero mentality and many other thing in between, we struggle with the idea of collaboration.  We have a lot of learning and growing to do in this area.

Becoming a software tester also means that I spend most of my days sitting at a desk manipulating computers.  I finally realized last year that this meant I needed to go to the gym regularly, and I have been doing pretty good on that front, but just hitting up the gym for half an hour before work isn’t enough to keep me healthy.  There is plenty of research that shows the importance of getting up and walking around throughout the day, so how do we integrate more of that into our lives?

My thought for today is that maybe we can combine these two things into one. In fact, I have tried to do that in my life.  If I need to communicate with someone in the office, I will walk to their desk to talk to them.  No chats, emails or phone calls.  Yes, sometimes they are busy or can’t talk and that’s fine.  I got up and walked around which is good for me anyways so there is really nothing to lose on my end.  It also points to one of the things that is valuable in collaboration, namely that the richest communication medium is face to face conversation with someone.  We can communicate a lot of information in person that we can’t via email or other media, but there is more too it than that.  We also build relationships and rapport when we see and interact with each other in person.   This is how you build collaboration: one relationship at a time.

So get up.  Walk around.  Talk to someone.  Maybe you’ll end up with some stronger relationships and maybe you’ll even be lucky and get rid of some of that undesirable stuff that’s making it hard to button up your pants in the morning.  Worth a shot right?

Becoming a Testing Batman


What kind of tester are you?  I think we all know some Superman testers.  You know the kind I mean.  The people that can just get in there and in 30 minutes they have found dozens of bugs and flaws and have totally changed the way everyone thinks of the quality of the system.  They have some kind of innate testing superpower and seem to be able to effortlessly find issues with anything you throw at them.  But what about the rest of us?  What if we don’t have that built in magical testing superpower?  Is there hope for us?

I think so.

I think we can become Batman testers.  Batman is just an ordinary human.  No genetic mutations or innate superpowers.  Just a normal human who uses tools to extend his powers and become a superhero.  This is the key to becoming a Batman tester: tools.  Batman surrounds himself with tools and he knows how to leverage them to his advantage.  He doesn’t try to replace himself with tools so that he can stay at home and let some robots take care of Gotham.  Instead he enhances himself with his tools and gets out there and gets the job done.  As testers we should do the same thing.  We should be finding and using tools.  Not so that we can replace ourselves, but so that we can enhance our testing powers and turn them into superpowers.  Using the right set of tools in a smart way can transform who you are and what you can do.

It seems that a lot of the talk in our cultures is about how automation is going to displace humans.  People talk about how we won’t need truck drivers any more, or we won’t need testers anymore, but what often gets forgotten in this conversation is that usually the most powerful way to use technology is symbiotically.  If we create a symbiosis between the human and his tools we can leverage the human for the things humans are good at and the tools for the things the tools have been designed for and are good at and together the whole becomes more powerful than either of its parts.  Look for example at human/machine partnerships in chess where a human and  machine working together can be better than either a human or a machine on their own.  Even if we can create automation that is ‘better than’ humans (as for example we did in chess), that doesn’t automatically mean that we should leave humans out altogether.  Maybe by continuing to partner together with our tools we can become superheros.  So find tools that help you where you are weak. Learn how to use them effectively and leverage them to become a testing Batman

Co-op Students and testing

Along with the new year we have a new co-op student (or intern if your prefer).  Although I am not directly their manager, I am the one who has been assigned to take care of the co-op student training and I also end up helping out with many of their day to day questions.  One thing that training someone who is entirely new to testing has revealed to me is that good testing is actually very hard to do and perhaps even harder to train.  How do you teach someone to be a good tester and how do you approach training someone that might only be with the company for 4 months in a way that allows them to get up to speed quickly enough to be genuinely helpful to the team?

One of the things I have noticed is that personality really comes to play here.  Those that have a more inquisitive and curious personality tend to pick things up more quickly just because they will try things out.  I love it when a co-op student shows me something they tried that I never talked about or asked them to look at (even when they have ‘done it wrong’), because then I know they are learning about the product.  I’ve noticed that the more they learn based on stuff they have sought out on their own , the more quickly they pick up the product and the more effective testers they become.

Of course you can’t have a training program that is entirely dependent on your trainee’s personality (although hopefully interviews help find those that have the needed traits), so what approach do I use?  My main ‘tool’ is to use the ‘throw them into the deep end’ approach.  I get them setup with an install and have them run through a couple of demos that we have and then give them a problem that they need to figure out on their own and that our software can help solve. By having them see (a) how our software can help them with that problem  and (b) things that are still hard to do even with our software’s help, they learn a lot about the product itself and how to use it as well as why customers pay us good money for it.

I’ve also started trying paired testing sessions with them and have found this to be another very helpful training tool.  By sitting right there with them and modeling the kind of curiosity, note taking and investigative skills that are important to have, I find they pick things up much more quickly than they would have otherwise. At first it seems that this might be expensive in terms of time commitment, but I find that often I have to heavily review their testing work anyways (especially early on as they are still learning to test) and so by being right there testing with them, I am reviewing and giving (and getting) feedback all at the same time.  It has been a helpful approach that I’ve been finding to work well and that I hope to continue to expand on

So that is largely the approach I’ve migrated to with training co-op students.  It is pretty low key and informal. Maybe it is a brilliant strategy or maybe it is just a reflection of my personality, but whatever it is, it seems to be working well so far.

Work/Life balance

Coming back to work after taking a couple of weeks off over Christmas has got me thinking about the idea of work/life balance.  For those of us that are working professionals, a large percentage of our life is taken up with working as well as the time spent preparing for and traveling to work.  This Christmas I had two weeks off and I hardly thought about work at all during that time.  I usually try to confine my work to working hours and I rarely check emails or do work in the evenings, but it often still seems that work is there percolating in the back of my mind. It seems to take an extended period of time for ‘work mode’ to fully turn off.  As knowledge workers, part of being a good employee is taking care of our brains. If you want to be good at what you do, you need to work hard and sometimes put in long hours.  But you also need to know when to stop.

But it is more than that. It’s not just balancing work life with family life and hobbies or pleasures.  It’s having a holistic approach to life that lets you keep body, mind and soul all healthy.  I’m not sure that the answer lies in spending less time at work or making sure I  have unconnected time – although those kinds of things may be helpful – but I think part of the answer lies in taking life as a whole in a balanced and sustainable way.

What things can I  learn about my home life or what I  do with my hobbies that I might be able to use at work? And what work things could I take and use in my home life?

If my spouse was to pile the kind of workload on my that I regularly accept for myself at work, would I put up with it?  Maybe I  need to learn to say no (hopefully in a nice way!) to co-workers just as I  have learned (hopefully in a nice way!) to push back on some requests from my spouse.  Or what about the other side of the coin. My wife asked me on New Year’s Day what my goals where for the year ahead and I realized that all the goals (at least the ones I had articulated) where around stuff I do at work.   If I was to approach my work in the haphazard directionless way that I sometimes approach family life would my boss be ok with that?  Maybe by having some goals for what I want my family to do learn and experience that I can share with them,  I would see a more fruitful and healthy family life.

In some ways what I am saying is that I need to blur some of the lines between work and the other aspects of my life.  What I do at work is still me and still affects me.  I’m not just a machine for the time at work  when I’m ‘producing’ and I’m not just a passive consumer for those times outside of work.  We don’t just need macro balance (taking vacations etc.), but we also need micro balance (being able to come home at the end of the day without being totally exhausted).  I need to be able to both work and play at a sustainable pace throughout the year.