When Should you Automate?

When should you automate?

Well let’s think about it for a minute. What is automation good at?  Repeating the same thing over and over again.  So when should you automate? When you have something you want to repeat over and over right?

But let’s think about it a bit more.  What does automation do?  Repeats the same thing over and over… What if you want to change something?  Your automation might just lock you into something you don’t want any more.

Automation is a powerful way to add leverage. If it is done well it can allow you to get way more done in the same amount of time.  However, it can also reduce flexibility. Let’s look at an example. You’ve created a high level test that checks for the existence of a certain value in a table.  Now you want to change something in your app so that the value will be in a div instead of a table. At this point you either have to get rid of the test, re-write it completely, or not make the change all.  Your automation has made you less flexible.

So we still haven’t answered the question: When should you automate? Let’s take a stab at a couple of heuristics that can be used to help answer that question

When you know what you want

If you have confidence that things will be a certain way for a significant amount of time, it probably makes sense to do some automation.  The designs are settled on and we are fairly certain this is the way we will go for the foreseeable future. In other words what we are looking at here is an estimate of the shelf life of the automation. How confident are you that this is something we want to lock ourselves into?

When the automation is simple

Sometimes though, automation still makes sense even when we don’t know what things will look like. For example, when the automation is simple to make.  If it only takes me two minutes to automate something it doesn’t really matter to much if the design changes next week and I have to throw that test out.

ROI

In a sneaky way, what I’ve been talking about here is your return on investment. How do you know when to automate?  Well it comes down to understanding how much leverage your automation will give you and for how long, as compared to the cost of creating that automation.  If it is easy to automate something, the ROI comes more quickly and so we can create it in more uncertain environments.  If it is hard to automate something, we need to have a lot more confidence that we will be able to use that automation for the long term.

I want to take a step back from this and think about how it applies to test automation in general.  I think there is a principle here that bears some reflection.  Software creation is an inherently unstable process. We are constantly changing things in response to customer and business needs. In many cases the shelf life of test automation is going to be quite short. It won’t be long before a test fails.  If we think about this in terms of ROI doesn’t that imply that we should bias towards smaller tests?  Smaller, easier to create automated tests give a return on investment more quickly. The more difficult a test is to create and maintain the more hesitant we should be about automating it.

So when should you automate? When it gives you a good return on your investment.