Know Thyself (And Thy Automation Tools)

Technology can hypnotize us sometimes.  There seems to be something about working with software that makes us get caught up in solutions and artifacts and tools.  These are good things, but sometimes we forget the human element.  We forget that software isn’t made in a vacuum, it’s made by humans.  We all know humans have their flaws and biases, and so let’s just be honest and admit that those flaws and biases are going to come out in the software we create.

One fascinating theory I’ve heard is the idea that the structure of a software product tends to reflect the structure of the company (or division) that made it.  I can think of examples of this and find it to be an interesting theory, but that isn’t what I want to get into today.  I want to consider the ways in which the automation tools we use shape the way we approach software quality.

Test automation is something almost every tester will be exposed to and there are many different tools out there.  When evaluating a tool we often look at many different technical aspects.  One of the interesting things I have noticed though, as I switch to using a different test harness than I was before, is that the tool itself will shape your behavior.  This can be good or bad, but I think mostly we need to be aware of this so that we can consider if we want to make changes to address this.

For example, the previous test harness I was using, gave the automation engineer a lot of low level control over things, but the test harness I have now hides away a lot more of the details of the execution.  This shows up in something as simple as the fact that the old tool was a set of readable python scripts, while this tool is a set of compiled files.  There are a lot of benefits to the new tool such as a small learning curve and ease of use and a lot of features that are built in from the start, but the fact is this different approach will push me to behave in different ways.  With the previous harness I could get down into details pretty easily and so could target more precisely the exact thing I wanted to test. In the new harness I have a harder time getting to that level of detail and so the harness itself will push me to write tests that are more high level integration style.

This isn’t necessarily a bad thing since tests like that can be valuable, but it does mean that I should consider the fact that I might be tempted to take the easy path and create an integration test when a lower level or more detailed test might be more appropriate.  The fact that I have a tool that will allow me to very easily create tests of one sort means that I will be tempted to create those kinds of tests even when it doesn’t make sense to do so.

The very fact that certain automation tools make doing certain things easy is the whole point of using them in the first place, but we need to consider the fact that making something easy means we will probably do more of it, and that may not always be the best things for the product.  Don’t loose sight of the end goal – delivering high quality code in a timely fashion.