Automation That Teaches

I think learning is important.  In fact I think it is probably vital to continue to learn new things throughout your career if you want to be a successful knowledge worker. In particular as a software tester one of the primary ways I add value to the team is by the things I learn about the software.  I learn where things don’t work well and where they do.  I learn about how certain approaches to developing and testing tend us toward certain outcomes.  I learn about when things go horribly wrong and try to figure out why, and I try to use all this knowledge to work with my team so that we get better at  more rapidly producing valuable software.

One of things that I spend a lot of time learning about is test automation and how to use it effectively.  I’ve spent a lot of time learning about it, but I’ve come to realize that I can also learn things from it.  My automation can teach me things. By changing my mindset, I have been able to turn my automation into a teacher. Instead of thinking about automation as something that is primarily used to make sure nothing has changed since the last time we ran it (what I like to call ‘change detection’ automation), I see automation as a powerful way to learn things about the product that I would not (or could not) otherwise know,

I still use change detection automation in many places (and I think it is helpful in those places) but even with these types of tests I have asked myself what it can teach me.  With that change in mindset I have been able to leverage my automation in new ways.  For example, I have a small tool I wrote that will perturb my tests in very specific ways.  By making minor changes to the tests and then running them and parsing through the results I can very easily gain new insights about the behavior of the system.  Another example is parsing through the logs after the fact. I have many gigs of log data from years of running tests (I have a large hard drive and don’t delete) and so I wrote a script that can parse through that data and search for some specific things, put them into a database, and allow me to do some statistical analysis on them.  By doing this I have discovered new and interesting behavior that I probably would not have ever noticed without seeing it in the aggregate.

There are other ways that I have and continue to use automation as a teacher, but I won’t share them all.  Instead I want you to think about your automation.  Do you learn anything from it?  If you shift to thinking about it as a teacher are there any simple changes or additions you can make that would turn it into a better teacher?  What can you learn from your automation?