Current Projects

Current Projects

I love creating courses and content for my fellow testers. Over the years I have learned a lot about testing in various ways, and I’m glad that I can help other testers in some small way.

I have a few cool things that I’ve been up to recently, but the big excitement for me right now has to do with Postman. I was contacted by Packt Publishing asking if I would be interested in writing a book about using Postman for API testing and development. The idea was very interesting to me, but as a I have a full time day job and spend a lot of my spare time creating courses, I didn’t think I would be able to do it and so I had to turn down the offer.

However, I love API testing and I think Postman is a great tool and the more I thought about it, the more I wanted to do it. I ended up reaching out to my boss and having a discussion about it, and we decided that I would reduce my hours and take off one day a week so that I would have some time to work on this book. I am grateful to my employer for their flexibility on this. And now? Well, I have some writing to do!

The book won’t be coming out for a while, but if you can’t wait to see some of the things that you might be able to learn, you should check out the Ultimate Postman Tutorial for API Testing that I just published on the TestProject blog. This tutorial will help you get up and running with using Postman and give you a bit of taste for what you might get in my book.

I might not have much time to post on this blog over the next few months, as many of my keystrokes will be dedicated to putting together a book that I can be proud of. I’m nervous and excited about this. Wish me luck!

Photo by Aaron Burden on Unsplash

API Mocking

API Mocking

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of things including lives and livelihoods. I want to take some of the focus off the destruction this virus is causing, and focus instead on creating something. I’m doing a series of posts on API testing as my way of fighting back against the virus. We can still go on with life even in times like this.

You might have heard about API mocking before but what exactly does that mean? No, it’s not pointing your finger at an API while laughing at it and making fun of it (as tempting as that might be sometimes). Mocking is actually an important test strategy that we can use to control how an API responds to certain calls.

Mocks for Exploring

Let’s imagine you have an application with a front end that calls certain APIs to get the data that it needs and then renders that data to the user. This is a pretty common way to build websites, but what if you want to test what happens when the API returns an error code? It can be hard to do this kind of testing because you would have to somehow trigger an error in the API. The problem is API designers want their API to return good data and so finding situations where it gives back error codes can be hard. You could try things like disconnecting the network or intercepting the network calls to trigger errors. Or you could just create a mock.

A mock API is just a locally hosted version of the API that will return preset calls for given endpoints. If you have a mock API, you can set it up so that the endpoint will return the kind of error you are trying to test for and then point the application to the mock API instead of the real one. This allows you to generate the exact situation that you want to check. 

Essentially when you are mocking out an API you are creating a (usually simplified) version of the API that you can use instead of the real one so that you can have full control over what it does. This makes it much easier to create a wide array of test scenarios that can greatly enhance your ability to explore how a front end will react to different kinds of data in an API call.

Mocks for Automation

Using mocks to help with exploratory testing isn’t the only way they are beneficial. Another way to use API mocking is to help out with test automation. Mocks can help solve a couple of test automation challenges.

One thing they can help with is flaky tests due to network problems. If the network disconnects or slows down during a test run, any network calls being performed could cause test failures that you aren’t interested in. A mock API can eliminate the need to make network calls. If you setup and host a mock API locally you can use it in your tests without needing to do any network calls.

Another potential benefit of mock APIs in test automation is that your tests can run a bit faster. If you are getting your data locally instead of sending it off to a server somewhere and waiting for a reply, your tests can run much faster, depending of course on how many network calls you are making and how slow the network is.

Problems with API mocks

However, using mocks isn’t the ultimate solution to all of the world’s testing problems. There are some challenges that can arise from using them as well. One of the biggest challenges lies in data management. You see, mock APIs aren’t the real deal. I know. Mind blown right?

Despite how obvious that statement might seem, its easy to miss what is right there in front of your face. A mock API is usually a handcrafted affair that is setup to match what the real API does. This gives you the benefits mentioned above, but it also means that the mock API is only as accurate as you have made it to be. If the data and endpoints in the mock API don’t match up with the real API, you may be missing important stuff in your testing. Getting a mock API setup just right can be a challenge in it’s own right, but even after that, we know that software continues to change which means that in all probability the real API will change. If you don’t have some mechanism in place to keep the mock API up to date, it will no longer accurately reflect the system you are testing.

Data Management

This is where data management comes into play. You need to be thinking about the data in your mock API and how you can ensure that it stays up to date. If you don’t, you will end up with problems like an out of date mock API that no longer tests what it should, or a maintenance headache that you need to deal with every time something changes.

So what can you do? Well, test data management is a complex thing on it’s own and this article is already getting long, so we can’t dive into it in detail, but let me share a couple of examples of things I’ve done in the past.

One strategy that I’ve used to reduce the maintenance work of a mock API is to create a script that can update it. This doesn’t work for every kind of API and may not always be perfect, but in my situation it was very helpful. I had a test site that was setup with the data that I wanted and when I wanted to update the mock API, I would execute the script which would make calls to the test site and update the mock API with any changes in the API. This worked well in this particular case because the API I was using was a hypermedia API which included information in the responses that my script could easily use to see if there were any new endpoints added or other changes. Not all APIs give you that kind of information and so this strategy may not work as well in other cases, but I would encourage you in general to consider cases where you might have to do manual data updates like this and to see if you can’t learn enough scripting to automates parts of it.

Another strategy I have used in the past to keep API mocks up to date, is to do some variation of contract testing. The idea of this (at least as I have implemented it), is to have some set of tests that checks the API functionality you are using in your mocks. Any changes found by these tests are changes to the API contract and let you know that you need to update your mocks.

Now, these are just a few ideas and are certainly not perfect solutions, but hopefully they give you something that you can build off of if you are thinking about data management in your API mocks. 

So should you use mock APIs? Well, it depends of course on what problem you are trying to solve, but they certainly are a powerful tool to keep in your testing toolbox.

Photo by Phil Hearing on Unsplash

Testing Third Party APIs

Testing Third Party APIs

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of things including lives and livelihoods. I want to take some of the focus off the destruction this virus is causing, and focus instead on creating something. I’m doing a series of posts on API testing as my way of fighting back against the virus. We can still go on with life even in times like this.

APIs great. They are a powerful way for different applications to work together. They allow for the easy integration of one application with another. APIs really have enabled much of the modern web. They are great.

Until.

Until the API changes. If it is an internal API, it might not be too much of problem, although in some companies the API team might be totally separate from those that use the APIs and so even internal API changes can cause problems. In fact, in bigger companies with separate API development teams, it often makes sense to treat internal APIs as if they are third party. API developers will also often version APIs so that you can continue to use old versions without breaking your workflows, but what happens when the company decides to deprecate or stop supporting old versions of an API?

Software changes are inevitable. Any piece of software that people care about and that is actively being used is going to change. Somehow though this seems to be a difficult thing for us humans to get our minds around. We want to be able to ‘set it and forget it’ but since we know software is going change we can’t do that. In broad strokes there are two strategies we can take to deal with this

Test First

The first strategy we could take is one where we just fully embrace the fact that software changes will happen and so we don’t just start using an API and hope for the best. Instead we build up a set of tests for the API that verifies that it does everything we need it to do. We then run those tests before every build and use them to ensure that the API still does what we need it to before we start using it in any of our software.

This approach obviously will take quite a bit of work since you will need to create the tests up front and run and maintain them into the future, but there are times when this is a good strategy to employ. If you are using a third party API to provide you with functionality that is core to what you application is doing, you will probably want to verify the API before you consume it. The more critical the information you are getting from the API and the more devastating the effects of failure the more careful you will want to be with verifying it ahead of time.

Monitor

There is another approach you could take though. Instead of testing the API before you even use it, you could just monitor for failures and then fix things up afterwards.

This idea can sometimes rub testers the wrong way, since we want to be able to prevent failures before they are out in the wild, but there are times when this strategy makes sense. Perhaps the API you are using doesn’t provide critical information and if it stops working you won’t have a huge impact on your clients. Or perhaps the API calls are for internal scripts that workflow that help you with your job, but if they break and those scripts don’t work for a few days it’s no big deal. There are many cases where broken functionality isn’t too big of deal as long as you can fix it up in a timely way. In those situations it makes a lot more sense to just monitor for things breaking and deal with the changes after you see something broken.

I would also remind you that monitoring doesn’t have to be a big complicated high tech thing. For example, I have a course on API testing and in that course I provide a lot of example APIs that people can use to practice their testing with. However, software changes, and so sometimes the examples that I use don’t work anymore due to changes in the API. This has happened in the course and one of the students sent me a message and asked me about why the example wasn’t working. I then went and looked into it and was able to update things to point to the correct API. The student reaching out to me is an example of monitoring. It might not be very efficient monitoring since it depends on the goodwill (or annoyance) of one of my students, but the usage of that API is still being monitored.

Monitoring doesn’t have to be high tech, but you should consider how you would actually know if it fails. Do you have to wait for a client to complain (like in my example), or do you have some internal user or process that you can rely on to let you know if it has failed? You may even want to consider monitoring software of some kind if it is important to know about breaking changes in a timely manner.

Mitigation

Regardless of which approach you take, you should have a way to mitigate the problems when an API fails to do what you want it to. Monitoring that lets you know that your API calls are failing doesn’t do you much good if you don’t know how fix it or if you don’t have some way to work around the problem until it is fixed. The same applies with testing it up front. If you don’t have a plan for how to make the changes that you need to when the API stops working you could end up holding back changes that you want because of a ‘broken’ API. Understanding what you will do if/when the API breaks is an important part of dealing with 3rd part APIs.

Mitigation is also important in another way. No matter how much testing or monitoring you do, things are going to fail in weird ways sometimes. There are some things that testing just can’t deal with. What if the service the API provides you access to goes down or stops working (or maybe even the company providing it goes bankrupt or stops giving access to it). What do you do? I guess one thing you could do is deny it – that will never happen! But if covid-19 has taught us anything it is to never say something like that! Having some kind of fall back strategy that at least informs users that something has gone wrong and preferably also gives them some kind of (perhaps reduced) functionality is good plan when possible.

Conclusion

Testing 3rd party APIs is actually a lot like testing other kinds of software. You need to think carefully about what kind of testing it makes sense to do for the given context you are in.

As to the API calls that break in my course, I have been taking a very low tech (and inefficient) way to monitoring them by waiting for students to report issues. I’m not sure I like that. Who knows how many students just skip past the broken stuff before one finally asks a question and lets me know what is going on. I would like to know sooner if the one of the APIs I’m using breaks or moves to another location. I plan to make a simple script that will ping the APIs I’m using once a day and email me if the endpoint goes away or changes. I want to make my monitoring more efficient. What about you? Are there any places in your application that you are doing your monitoring by waiting for a human to notice something? Could you maybe improve that just a little bit with a simple script?

Third party APIs have enabled a lot of the wonderful things we see on the modern web, but they have their own set of risks. What are you doing to test for and mitigate them?

Photo by Mari Helin on Unsplash

Types of APIs

Types of APIs

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of things including lives and livelihoods. I want to take some of the focus off the destruction this virus is causing, and focus instead on creating something. I’m doing a series of posts on API testing as my way of fighting back against the virus. We can still go on with life even in times like this.

In order to effectively test an API, you need to know what kind of API you are testing. Often in today’s world when we talk about APIs we are talking about REST APIs, but there are several other important API formats that you might need to test. Let’s look through them and see what we can learn about some of the major API types.

REST APIs

We’ll start with what is probably the most common type of API you’ll come across on the modern web; the RESTful API. REST stands for Representational State Transfer and refers to an architectural style that guides how you should create APIs. I won’t go into the details of the properties that a RESTful API should have (you can look them up on wikipedia here if you want), but there are a few clues that can let you know that you are probably testing a RESTful API.

RESTful APIs are based on a set of guidelines and so they do not all look the same. There is no official standard that defines the exact specifications that a response has to conform to. This means that the following are in general good clues and hints that you are dealing with a REST API, but they are not definitive. It is also worth noting that many APIs that are considered to be RESTful do not strictly follow all the REST guidelines. REST in general has more flexibility than a standards based protocol like SOAP, but this means that there can be a lot of diversity in the way REST APIs are defined and used.

How do I know if the API I’m looking at is RESTful?

So, what are some clues that you are looking at a REST API? Well in the first place, what kind of requests are typically defined? Most REST APIs will defined GET, POST, PUT, and DELETE calls with perhaps a few others. Depending on the needs of the API, it may not define all of these, but those are the common ones.

Another clue, is in the types of requests or responses that are allowed by the API. Often REST APIs will use JSON data in their responses (although the could use text or even xml). Generally speaking if the data in the responses and requests of the API is not xml, there is a good chance you are dealing with a REST based API of some sort.

REST API Example

There are many examples of REST APIs on the web, but let’s take a quick look at just one of them to help cement the idea of what RESTful API is in our minds. It’s the internet, so let’s look at cats, and since this is an article about API testing, let’s also look at http status codes (which are used by REST APIs). And of course, the internet knows that combining cats with http status codes needed to be done, so we can go to https://http.cat/ to see a cat image for each http status code.

This site provides an API and so we can call https://http.cat/200 to get the cat image for the 200 status code. For example, if we were to do this in Postman, we could create a request, set the method to GET and put in the url and click on send.

call cat API in Postman

We don’t have to specify any xml when we make the call and can just send the request directly in Postman (or another API tool) as a GET call, so we are most likely dealing with a REST API here.

SOAP APIs

Before REST there was SOAP. SOAP stands for Simple Object Access Protocol. The SOAP protocol has been around since long before Roy Fileding came up with the concept of REST APIs. It is not as widely used on the web now (especially for smaller applications), but for many years it was the default way to make APIs and so there are still many SOAP APIs around.

SOAP is an actual protocol with a W3 standards definition and so its usage is much more strictly defined than REST which is an architectural guideline as opposed to a strictly defined protocol.

If you want a little light reading, check out this document. It claims to be a:

non-normative document intended to provide an easily understandable tutorial on the features of SOAP Version 1.2

I’ll be honest, I’m not sure how well it delivers on the ‘easily understandable tutorial’ part of that statement. Looking at some of the examples in there may help you understand why REST APIs have become so popular. SOAP APIs require a highly structured xml message to be sent with the request. Being built in xml, these requests are not that easy to read for humans, and require a lot of complexity to build up. There are of course many tools like SoapUI that can help with this, but in general SOAP APIs tend to be a bit more complex to get started with. You need to know more information (like the envelope structure) in order to get started.

How do I know if the API I’m looking at is a SOAP API?

The most important rule of thumb here is: does it require you to specify structure xml in order to work? If it does, it’s a SOAP API. Since these kinds of APIs are required to follow the w3c specification, they must use xml and they must specify things like env:Envelope nodes inside of the xml. If the API you are looking at requires xml to be specified and that xml includes the Envelope node you are almost certainly dealing with a SOAP API.

SOAP API Example:

A SOAP API example is a little bit harder than just sending a GET request to an endpoint. We will use this webservice as an example SOAP API. Let’s look at the first action there, which gives us a list of continents. In order to call this API in Postman we will need to set up a few things. We of course need to create a request in Postman and then we will need to set the request method to POST and put in the url. However we can’t yet click send. Since this is a SOAP API, we need to send some xml information as well. In Postman, this means setting the body type to raw, and choosing XML from the dropdown and then putting in the xml envelope data as indicated by the documentation

SOAP API Information

For this particular API, we would also need to modify the Content-Type header (by adding a new one) at the bottom, so that is is set to application/soap+xml

Content-Type Header

As you can see, there is a lot more complexity to calling SOAP APIs. REST APIs can of course have complex bodies specified as well, but the requirement to do this in xml and the existence of the Envelope node in this, indicates that this API is indeed a SOAP API.

GraphQL APIs

SOAP came before REST and in many ways REST was designed to deal with some of the shortcomings of SOAP. Of course in software we are never done making things better and so along comes GraphQL. GraphQL is a query language and it was designed to deal with some of the situations where REST APIs have shortcomings. RESTful APIs don’t know what specific information you might be looking for and so when you call a REST API endpoint, it gives back all the information it has. This can mean that we are sending extra information that you don’t need, or it can mean that we aren’t sending all the information you need and you need to call multiple endpoints to get what you want. Either of these cases can slow things down and for big applications with many users that can become problematic. GraphQL was designed by Facebook to deal with these issues.

GraphQL is a query language for APIs, and so it requires you to specify in a query what you are looking for. With REST APIs you will usually need to know what the different endpoints are in order to find the information you are looking for, but with a GraphQL API a single ‘endpoint’ will contain most or all of the information you need and you will use queries to filter down that information to only the bits that you are interested in. This means that with GraphQL APIs you will need to know the schema or structure of the data so that you know how to properly query it, instead of needing to know what all the endpoints are.

How do I know if the API I’m looking at is a GraphQL API?

Well, if the documentation is telling you about what kinds of queries you need to write, you are almost certainly looking at a GraphQL API. In some ways a GraphQL API is similar to a SOAP API in that you need to tell the service some information about what you are interested in, but a SOAP API will always use xml and follow a strict definition in the calls, whereas GraphQL APIs will usually a bit simpler and not defined in xml. Also with GraphQL the way the schema is defined can vary from one API to another as it does not need to follow a strictly set standard.

GraphQL API Example

Let’s take a look at a real life example of calling a GraphQL API to understand it a bit better. This example will use the countries api as hosted here. There is information about the schema for it on github here and we can use that to create queries in the provided playground, but for this example let’s look at setting it up in Postman

Similar to calling a SOAP API, we will need to specify the service we want and do a POST request. We will also need to choose the GraphQL option on the body tab, and put in the query that we want

GraphQL query

As you can see in the example I’ve done, I’ve requested the name and languages of Canada. Once I have specified this information I can click Send and I get back some json with the country name and a list of the official languages. If I wanted additional information (Say the name of the capital city), I could just modify the query to include a request for that information and send it off again using the same endpoint.

Conclusion

We’ve looked at three different kinds of APIs. Each of them has it’s own strengths and weaknesses and is used in different applications. Some applications will even support more than one type of API, and as a tester you need to be ready to test any kind of API that you come across. Hopefully this article has at least helped you in that journey and given you some understanding of how the various APIs you might come across work and can be interacted with.

Photo by Markus Spiske on Unsplash

The Simplified Guide to API Testing

The Simplified Guide to API Testing

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of lives and livelihoods. However, I want to take some of the focus off the destruction this virus is causing, and focus on creating instead of destruction. For my creative act of defiance, I want to do a series of posts on API testing over the next few weeks. Consider it to be my way of fighting back against the covid-19 virus. We can still go on with life even in times like this.

When I started testing APIs, didn’t know anything about them. I’ve learned a lot over the last few years and I now feel pretty comfortable with using and testing APIs, but it wasn’t always that way. When I first started it was hard to even figure out what to do. I can still remember some of that pain I went through.

In my first year of university we had a calculus professor we nicknamed Einstein. Partly because of his hair and partly because he managed to make calculus as confusing as relativity. He would start to solve a problem on the board and then he would say ‘and therefore it is obvious that’ and write down some solution. We would all sit there and look puzzled.  What was obvious to him was not obvious to us as the students. We needed to know the intermediate steps that he had skipped in order to understand what had happened.

This is a common occurrence. As you learn things, the pain of that initial learning curve goes away and it becomes harder and harder to explain things to those who are just starting out. You start to assume things and makes leaps of logic that seem obvious to you but that are confusing to those who are new to the subject.  As I continue to progress in what I am learning about APIs, I want to get down some of my basic thoughts on API testing, hopefully before it is ‘too late’ for me to explain to those that are just starting out.

So with that long preamble, here is my attempt at a simplified guide to getting started with API testing

Figure out the endpoints

I found this to be one of the most difficult parts of API testing. How do I even know what there is to test? What things can the API do?  If you are working with a well documented public API, this isn’t much of a concern, but in my experience most API testing is done on internal APIs that support different parts of your application.  These kinds of APIs tend to be poorly documented (if they are at all). So how do you figure it out?

Well of course any available in house documentation helps, so certainly start there if you have some available.  And don’t forget that documentation can sit in places you might not expect. For example sometimes code comments or even signatures in a method can help you out. You also might also be able to find documentation in stories or requirements that point you in the direction you need to go.

Another great source of information is humans. You see there are people in your company who have created, designed, tested and/or used the API you are looking at. If you can find and talk to those people, they can be an invaluable source of information.

Code bases are another place that contain information about how APIs work. You might not know how to write code, but you can often still find out some information by looking at how an API is called. You might be able to pick up something by the urls used in the caller, or you might be able to find some unit tests that use the API and give you insights into how it was used.

Last, but certainly not least, you might want to look at network calls in the developer console to figure out what API endpoints are available.  Often API calls are sent over the network and by looking at network calls in the developer tools you can figure out a lot about how they work.

One thing you will find with all of these sources of information is that they will give you imperfect and incomplete information.  There will be things you just don’t understand and (especially at first) everything will be very unclear and foggy.  But if you stick with it and continue to ask questions and get feedback, you will suddenly realize one day that you actually get it and have a decent understand of the API.

Auth and Security

Authorization. Authentication. Security. What is it all about and how does it work? I found auth workflows hard to understand. Security is obviously very important to the way an API works, but it can significantly complicate the use of an API.  This article would get far too long if I was to dive deep on this, but I do want to point out a few hints.

First of all look at how your tools can support you. Tools like Postman offer a lot of support for making authorization and authentication easier.  They can also provide you with insight in the security of your API. Dig into those features a bit in whatever tool you use and see what you can figure out.

Another consideration is that, again, when you find yourself frustrated by it, consider it to be a learning experience. Ask questions of your coworkers (and Google).  Try and figure out a bit about how Oauth works. Get a bit of an understanding of authorization workflows.  I have found that digging into authorization really pushed me with respect to understanding stuff about how APIs and HTTP protocols worked and so it was a great way to learn a lot more about API testing

A last comment on the security front, is that many things are ‘security’ issues that are not related directly to technical security testing. Making sure that different users types can only access the information they have permissions for and making sure that all paths give back the correct information are also forms of security testing.  Remember to look at your API holistically

Learn to do by doing

It can be tempting to spend a lot of time learning about things like API testing. We can feel like we need to know what we are doing before we get started, but I really believe that the best way to learn is by doing. Just try making API calls. Figure out one endpoint and try calling it. Start with the smallest thing you know and use it. As you try to do things and get stuck and then figure it out you will learn the things you need to know for the API you are working on.  You will also learn what things you don’t know and where you need further study and so you’ll be able to find resources that can help you with the exact things you need to know.

Figure out some of the basic terminology

I have a glossary of terms I’ve put together here. This is a set of terms I had to learn. You may find it helpful, but I think it will be important to also figure out what terms you hear that you don’t understand and figure them out. It is also important to listen to how your team uses terms and to make sure you are understanding what they mean. Often different teams or companies can use the same or similar terms in different ways. Pay attention to the words and build up a vocabulary.  You will find that this makes it easier to ask questions and to understand what is going on.

Next steps

So where to do from here?

  1. Install an API testing tool (Postman perhaps)
  2. Find an endpoint on the API you are interested and make a call to it
  3. Change something in the API call and try again
  4. Start writing down what you observe and try to build a mental map of what the API does
  5. Try to use the API to answers some questions you have and follow up on figuring out how to get it to do those things
  6. Research and learn the things you need
  7. Try your hand at some API testing challenges (The automation in testing site is a great place to start).  You could also try out some public APIs (here is a list of some you could try)
  8. Ask lots of questions!

Photo by Daniil Silantev on Unsplash

Effective Testing During COVID-19

Effective Testing During COVID-19

Can one blog about something that isn’t corona virus related at this time? This is affecting us all in so many ways. I’m thankful though that I have a job that makes it easy to work from home. The company I work at is also in an industry that will likely benefit from this as we enable online education which is kind of a big thing right now. So as someone who does not have to worry about their job and income and who is at an age where the impact of the virus is unlikely to be serious if I was to get it, I feel blessed.

I have always worked from home on a regular basis (usually 1 or 2 times a week), and so the transition to full time working from home hasn’t been too difficult for me. My wife and I also started homeschooling our kids just this year, so that also hasn’t been too big of an adjustment for us either. I think the biggest thing this pandemic has given me is some perspective. When everything changes you realize what things you might have been taking for granted. Full time remote work is different than going into the office to regularly have face to face contact with your coworkers. As a team we are all working on ways to stay connected and have effective communication in a world where we can’t just pop our heads over the cubicle wall and chat.

What does one do as software tester in this world? How do you work effectively in a fully remote environment? There are a few things I have been thinking about and working on at this time. One important thing (probably for all remote workers) is organization and discipline. There are different kinds of distractions when you work from home. Your family, your pets, even your housework all distract in different ways than your co-workers. You may need to have conversations with kids and your spouse about how to protect your time so that you have the ability to dive deep on your testing sometimes.

I think organization is another important factor to consider. There may be some built in accountability at work where your co-workers or boss can see what you are doing when they walk past your desk. At home you may need to watch your schedule more and make sure you aren’t spending too much time reading the latest news about the virus (it’s probably not a healthy thing to spend too much time reading about anyways).

There are also less clearly defined lines between work life and family life when you work from home, so set some boundaries for yourself that way too. You may have needed to talk to your kids about boundaries while you are working, but don’t forget to set boundaries for when you are not working too. Just because your computer is only 5 feet away in your office, doesn’t mean you need to go do some work at 8 pm. Give yourself and your family some uninterrupted time

Those items above are probably general to anyone working from home. As a tester are there any particular challenges? One thing that may be a challenge is access to your test lab or test devices. This might be the time to investigate Sauce Labs or CrossBrowserTesting or similar services as options to get access to the devices that you need. Another challenge testers may face is in communicating with developers about issues. I will often go over to a developer’s desk to discuss, debug or demonstrate an issue I’ve found. This can be more difficult when working remotely, but certainly using tools like zoom or google hangouts to do these things virtually helps a lot.

I also firmly believe that quality is a team sport. When working from home, testers need to remember that we aren’t working off on our own, finding bugs and throwing them over the wall at the developers. I think as an industry we have been making positive strides towards integrating quality into the entire development life cycle. Let’s not let this set us back to a mindset of developers develop and then give to testers who find all the bugs and give it back to developers. This kind of ping-pong approach of bouncing things back and forth over the wall between test and dev is not an effective way to create high quality software. Quality is a team sport and so in times of ‘isolation’ we need to stay connected as teams to ensure we are focused on this together.

Talk about testing and quality as a team. Meet with developers to talk about things to consider for stories they are working on. Pair up with developers to help grow quality skills throughout the team. Stay involved in design and story definition meetings. Keep working on having quality thinking permeate the entire development process. Working as teams remotely can be challenging, but stay focused on ensuring that you are indeed working as part of a team.

Take care of yourself in these crazy times in which we live, and happy testing!

 

Photo by Dimitri Karastelev on Unsplash

Speed Limits in Software Development

Speed Limits in Software Development

I took the train into Toronto last weekend and without my car spent the weekend walking everywhere. As I walked under the highway, I was struck by how obsessed we are with speed. I could hear the thump, thump of car tires as they rushed by over my head at 100+ km/hr (traffic was good for once). As a species we have gone from being able to move at the speed our legs could carry us, to using animals to help us move faster and then eventually we started using mechanical devices like the bicycle to increase our speed. With the invention of mechanical engines we’ve added even more speed. Our cars will rush us from place to place in almost the blink of an eye. We can listen to an audio book about a pioneer family taking months to cross a part of America and we can’t even finish the audio book before all the country they covered has all rushed passed the windows of our car. What used to take months, now takes hours.

But even with all that speed, there are limits on how fast we can go. Jump on a jet plane and you can get across a country or continent pretty quickly, but it still takes time. Physics still applies. I could wax philosophical on our obsession with speed and what we loose from it – and I may yet do that at some point – but in this article I want to move us into thinking about the limits of speed in software development. It is an axiom of modern software development that faster is better, and like I said that is a philosophical discussion for another day, but how fast can we actually go? What are the limits of software development? The speed a developer can type? I don’t think anyone is advocating instantaneous deployment on each keystroke a developer does. If not that, then perhaps the limit is how fast we can get a minimum viable feature completed. By how do we know if it is viable?

You might be able to guess that I am going to argue that quality matters. What is the fastest we can move in software development? That is, of course, not quite the right question. How fast a can a car go? Well, if it is an F1 race car, pretty fast. If it is a car attempting to set a land speed record, even faster. But those things aren’t really what we are asking about. For the vast majority of us, the answer to how fast a car can go has to do with the posted speed limit and not with the physical limits of our car. Someone engaged in a police chase can go pretty fast, but it certainly wouldn’t be safe to do so. When we are trying to figure out how fast we can go in software development we aren’t trying to figure out they physical limits of it (deploy on every keystroke). We are trying to find how fast we can safely deploy.

For software development there aren’t road signs telling us a safe speed to deploy at, but perhaps we can extend the driving metaphor a bit more to help us think this through. One thing that relates to safe speed is responsiveness. A slick road makes it harder for your car to respond to changes in direction, and slow deployment makes it hard to respond to problems with your application. How easy is it to respond to issue in your application? Don’t forget that an F1 race car with new tires and perfect tuning can respond a lot better than the little commuter car you might have. We can tune our code and deployments and get better at responsiveness over time.

If the road is foggy and you can’t see where you are going when you drive, I hope you slow down. If you can’t see what is going in your application and understand how it is being used, I hope you slow down.

Let’s think a little more about the foggy road analogy though. Sometimes we think of testers as fog lights. We pierce the darkness a bit and show where there might be threats. That is helpful when we are stuck in the fog, but in software we have an option that we don’t have in car driving. We can dispel the fog. We can make our applications more observable. Fog lights are helpful but sunshine is even better.

So how fast can we go in software development? Well, in the ideal case if we know everything and have a smooth path ahead of us, pretty fast. I don’t think we can get to a land speed record since software development doesn’t often involve going in a straight line, but with a bit of work on the code and deployment process and with investment in observability and operations, I think we can go pretty fast, pretty safely. Just be careful. Don’t try to drive your homemade hot rod as if it is an F1 race car. Go fast and push the limits, but know where those limits are for your situations (and then make them better).

 

Photo by Adrien CÉSARD on Unsplash

Some Things are Worth Doing Badly

Some Things are Worth Doing Badly

The British philosopher/theologian G.K. Chesterton once said that if something is worth doing, it is worth doing badly.  Chesterton meant this as a defense of the amateur or generalist over the specialist, but I want to take this thought down a bit of different path.

If something is worth doing it is worth doing badly, because only in doing it badly will you ever do it well. Do you want to learn how to test APIs? You will have to do it badly at first.  You won’t be very good at it. But if it is worth doing, it is worth doing badly so that eventually you can get good at it.  Do you want to learn programming or scripting? You aren’t going to be very good at it when you start, but if it really is something worth doing then isn’t that ok?

We don’t like doing things badly, but the world needs more people who are willing to not be very good at something. When we stop worrying about how we look and start embracing failure as the path to learning we can get somewhere with the things we want to learn and do.

So step out and doing something worth doing. Do it badly. Get better. Do it again and again.  And don’t forget the point that Chesterton was originally making – sometimes it is in the mistakes and imperfections of the amateur ‘doing it badly’ that we get the most profoundly human insights and results.  Don’t be afraid of the mistakes you will make. You might just change the world.

Photo by Sarah Kilian on Unsplash

Deep Dive into TestProject: A Free Test Automation Platform for Web & Mobile!

Deep Dive into TestProject: A Free Test Automation Platform for Web & Mobile!

Note that this is a sponsored post. Posts like this will only ever be about things that I believe and that I think you would potentially find useful. You can read about my full philosophy on sponsored posts here

I’ve talked about TestProject and some of the great features that it has before. Since then, I have recorded a few more videos (you can watch the whole playlist here) and I wanted to talk about a few more features that TestProject has that make it a tool worth having in your testing tool box.

As the first of its kind free end-to-end test automation platform, I would certainly encourage you to check it out and see if it can help you with the testing challenges you might be facing.  I’ve already talked about how easy it is to get started with using the tool, but now I want to talk about some of the features that can help you take it to the next level. 

For example: Addons. TestProject has an addons library that can help you solve those tricky UI automation problems you might run into. Trying to find an element in a scroll-able container?  Yup, there’s an addon for that. Want to generate some random data for test inputs? Yup, there’s an addon for that. Trying to parse some strings to verify that some text is in it?  You guessed it, there’s an addon for that too. There are many publicly available addons, and if the addon you want is not there, TestProject provides you with the tools that you need to make your own. Since the platform is built around the idea of full team collaboration, you could tap into the developers on your team to help you out with building some addons that extend TestProject to have the capabilities you need. You can then choose to share those addons publicly for others to use or you can just keep them within your team if you prefer. 

Let’s also talk about the TestProject API.  I like APIs. I love how with a simple command definition you can have access to all kinds of power without needing to do any complex coding. Maybe I’m just a bit of an API nerd, but I think it is fun to play around with APIs and to see how I can enhance my testing superpowers with them. The TestProject API can help you take the tests that you have created and make them into something you can easily run in your build system and can really help you take the tool to the next level. 

One thing that every tester knows about testing is that it isn’t just about what you test, but it is also about how you communicate that testing to others. Testing is about much more than just providing information, but effective communication of what you have done is certainly an important part of any testing initiative. This is where reports in TestProject can be very helpful. There are a number of built in reports (that you can download into a pdf if you want) that can help you quickly and easily summarize what is going on with the tests over time and across different devices and browsers. 

There are also really helpful tools for debugging failing tests that are built into the reporting tools as well. I mean let’s just talk about screenshots for a minute. How would you like to have your tool automatically take a screenshot of what it was doing whenever a test steps fails?  Now all you have to do is click on the failing test, go to the step that is causing the issue and click on the screen shot to get some context as to why the test step failed. And yes, this is all automatically taken care of by TestProject right out of the box. 

As you can see, TestProject is a good tool for helping you get started with test automation but it also packs that power that is needed for advanced usage as well. With their  free forever plan there is nothing stopping you from checking it out today!

Photo by Julian Dufort on Unsplash

When DevOps makes life harder

When DevOps makes life harder

I know DevOps is the latest and greatest way to do software development and there is a lot that I like about it, but there is a one big problem with being a DevOps team.

Interruptions.

We’ve started to learn to plan for interruptions. There are going to production issues that need to be dealt with. We are going to get client escalations. Things are going to come across our plate without any warning and we need to be able to respond to them.

The problem is interruptions keep you from getting done what you were planning to get done. Features don’t get delivered and the team can start to get frustrated. Interruptions can really make the life of a devops team get harder.

Do you know what another word for interruptions is?  Bugs.

These interruptions are coming from clients that are having issues. Sometime they are just because we have changed a workflow and they don’t understand it, but to me anything that is impacting the ability of our clients to solve their problems is a bug.  These interruptions are bugs.

So let’s rephrase that statement at the top of this article. Bugs are a big problem with being on a devops team. You know I think they are a big problem with being on any team aren’t they? So what do we do about it? Do we need to spend more time finding the bugs up front so they aren’t coming in as interruptions during the next development cycle?  Sometimes yes – but sometimes these are things we probably never would have found no matter how much time we spent on it.  So what other solutions do we have?

Well we could just account for it in our plans and add a 25% interruption buffer. This doesn’t seem to be the greatest option, but there are times when it makes sense (like when we just changed a major workflow for hundreds of clients). In general though I don’t think this is going to allow us to most effectively deliver value. So what else?

Perhaps we need to make it so that the interruptions don’t hurt as much. If we can pinpoint exactly where the problem is coming from and find the solution more quickly, interruptions becomes less painful.  This mean being committed to good telemetry and good coding practices that let you quickly and easily find issues.

This issue of interruptions and bugs is not a simple one. It is one that has been with us from many years and will be with us for many more. There isn’t just one simple, quick fix answer. I’ve shared a few ideas we’ve been using or working towards on our team.  What ideas to do you have? Share in the comments below!

Photo by Jonathan Safa on Unsplash