Usability First

The job of a tester is not just to find issues, but it is also to present the case for why those issues matter.  This is much the same thing as saying that we want to find relevant issues and see the quality of the product improve.  There is another side of the equation though.  As with most things in life, trade-offs have to be made and the trade-off here is the balance between how important and relevant an issue is and how much work it will take to fix it.  Some things may be important to fix, but may end up being too expensive to fix and so don’t get addressed.  This can be very frustrating for a tester who sees the importance of the issues found but perhaps has less exposure to the cost of fixing it.

However, something that struck me recently is that there are certain kinds of feedback that are more important to give early in the process.  Some types of issues become more and more expensive to fix as you get later in the release cycle. In my experience workflow and usability problems typically fall in this category.  Early in the development cycle of a feature many things are changing and so it is actually much easier to change workflows or usability issues.  As the feature gets more mature, other things start to depend on it being a certain way and APIs are defined and there is upgrade code in place for older versions, etc.  Making workflow changes at this point in the process is often much more painful and so the cost side of the equation goes up.  As a result, we often have these kinds of issues dismissed as not worth fixing or as a system limitation.  However, I have noticed that if we find these kinds of issues early in the cycle they are much more likely to be addressed.  The benefit side of the equation stays the same, but the cost side has gone down and so the issue gets corrected.

Based on this experience I have been wondering if I need to shift my testing approach a bit and really focus on finding workflow and usability issues early on.  Typically my approach has been to bug hunt at the early stages and try to give the developers feedback about bugs early so they can fix them early. I still think this is a good approach, but what I think I need to do is shift my search to be for certain types of bugs.  I need to look for the usability and confusing workflow issues early and leave the testing for coding and logic errors and crashes and other things that are clearly defects until later in the cycle.  These other items are things that usually have less cost increases as development progresses and so it is ok to find them later in the process.  We need to ask questions early, but we need to make sure we are asking the right kinds of questions as well.