I think performance testing is a very overloaded term. It can mean a lot of different things ranging from how fast responses come back, to how well things scale with increasing usage, to many other stress or speed related tests.
The way I approach performance testing in an API is similar to the way I approach all my testing, which is to say holistically. There are many great tools that can help with performance testing an API, but you really don’t need to go there right away.
Just the other day, I was testing an API that allowed you set a display of items as selected or not. The API calls were not the fastest, but they really weren’t that slow. That is they weren’t that slow until you looked at how long it took for the user to be able to interact with the list. You see, each item on the list made it’s own API call, and so if you had say, 100 items on the list, you would have to wait for 100 API calls to resolve before you could interact with the last item. This caused a performance issue. Not so much because the API calls were too slow, but because the way the API was being used by the client did not give the necessary responsiveness.
The point being that you can’t just find performance issues, by checking how fast API responses come back or by looking at how many calls per second you can send to the API. Those may be important things to check, but there is a holistic approach that needs to come into play with performance testing as well. How is the API used and can it support a valuable end user experience? To me this is the crucial question we need to start with and then we can use tools to help us along the way. We can create automated performance tests and use stress and load testing tools to help us keep things where they need to be in terms of the ongoing performance of different aspects of the system. We can also use tools like these to help us figure out and understand where issues are coming from, but we need to always keep in mind the end goal. The user doesn’t care about how many milliseconds it takes for your API call to return. They care about how quickly they can do what they want to do. Make sure your API performance testing is in support of that!
Photo by Marc-Olivier Jodoin on Unsplash
So you want to get started with API testing?
First of all, good for you! It is pretty hard to avoid APIs if you work in software and testing them is an important part of creating high quality software. If you haven’t done stuff like this before it can be pretty scary, but I think you will find that it can also be rewarding. There is something exciting about learning how the pieces of an application come together and being able test and explore and API can help you understand this better.
But how do you get started?
Well, try using an API. There are many, many public APIs that you can look at. Here for example, is a big list of public APIs. Find something that is pretty simple and that you find interesting. Maybe the cat facts API. Download Postman or another easy to use API tool and put in a URL and press send.
Making your first API call is pretty easy and from there, just try stuff. Try different endpoints. See what happens when you change things. Google for answers when you get stuck. I think you will pretty quickly find that you can do a few different things and are starting to get an understanding of the way APIs work.
The next step might be to find out what APIs your team has that you can test. Do the same thing. Find an endpoint you can get started with and try stuff. Look at what it can do and change things to see what happens. Search for internal documentation. Talk to developers and architects. See what you can figure out about it and how it works.
At first it will be a frustrating experience. You will feel like you have no idea what is going on and you will get stuck frequently. Just leave it for a little while and then come back again, perhaps the next day. Keep trying things. Look at it from new angles. Grow your understand of the pieces of the product. Ask lots of questions. Over time you will find that things become more and more clear and then you’ll be doing it. You will be an API tester!
Photo by Tim Huyghe on Unsplash
Security testing is a huge topic in it’s own right, but the intersection between API and security is an important one to consider. It can be overwhelming since there is a lot of specialist knowledge required. The truth is the majority of us do not have the ability to do a deep dive on security testing when we are testing APIs, but I think there are a few simple things that we can consider as testers.
One of the simplest things is to try out API paths with different credentials. For example, if you have an API path that allows you to edit something that is meant for admins, make sure you can’t access it as a ‘regular’ user. Another similar check to this is to try a different user’s credentials on endpoints that give back private information.
Another quick and dirty test that anyone should be able to do in API testing is try different API paths that give back the same information. Often there are multiple ways to access something in an API. For example you might have a direct endpoint that lets you get information for a particular resource. You might also be able to get to that information by using search functionality on another part of the API. Checking that the search functionality does not give you more information than it should based on your permission is something anyone can do without needing a lot of security training.
There are also system level issues to consider. Does the way your API is setup encourage or make it easy for people to accidentally expose data? Considering API design and how it interacts with human behaviour is another important place where you can provide security insights without needing security certifications.
There will be the need for security experts who are trained in the intricacies of technical security testing to take a look at your APIs, especially if they are public, but don’t let that stop you from digging in and doing your own security testing!
Photo by Bernard Hermant on Unsplash
I’m not even sure anymore where I found this, but I have found this image to be a helpful resource when I need to be reminded of what the various API status codes mean
On thing I would note about status codes, is that they aren’t always correct. The status codes need to be set by the API and sometimes APIs aren’t setup to give back the correct status code. Most APIs only use a few of these status codes and so you might get a generic error code instead of the specific one related to the issue you are actually hitting. It also happens on occasion that there are bugs in the API that return the incorrect status code. When testing you want to pay attention to the status codes, but don’t just blindly trust them
One other thing to keep in mind is that there is some ambiguity in these codes as well. Sometimes an API programmer has to make a choice as to which status code to return and different APIs will sometimes make slightly different choices in what they return. Once again you will want to pay close attention to how the API you are testing uses them. Does it make sense? Is it helpful to the users?
Status codes are an important part of API testing and are often one of the things we check in API automation, so hopefully this list is helpful for you!
Photo by rawpixel on Unsplash
I really enjoy testing APIs, but let’s face reality here: every job has its frustrations. One of the things that I have found frustrating in API testing is when an API is inadequately documented. There is something fun about digging around and figuring out what paths there are in an API and how it works, but some APIs make this too hard.
When I’ve worked with hypermedia APIs, I’ve found they generally include enough information in the calls themselves, that with a little domain knowledge you can figure them out fairly well. But I’ve also tested APIs that don’t have any self documentation and only partial external documentation. I’ve had to go digging through the code base and I’ve had to try and figure out who I could ask about stuff and kept banging my head against the API until I eventually figured out how it worked. Doing this kind of thing can get very frustrating.
As with any code, APIs need to be testable and when they are written without documentation and using magic numbers and other anti-patterns, test-ability drops. We don’t need every detail documented, but there should be enough there that someone new to the API can figure it out fairly quickly.
Photo by Steve Johnson on Unsplash
Most of the API testing I am doing at this point is exploratory testing. Currently the area I’m working in does not have a lot of test automation on the APIs. We have developer level unit tests in place on the actual implementation of the APIs, but we don’t have many regression tests that run the APIs externally.
The tools I currently use for exploring APIs are largely a mixture of Postman and Python. I have used Postman to get in and figure out what an API looks like and the basics of what it can do. Then when I want to dive deeper and perhaps throw some interesting probes at it, I’ll dig in with the requests module in python. I also use Postman to document the API so that we have a shared references for what it can do and so that I have something to come back to when things are changing. One other thing I do is use Postman or SoapUI to make mock servers so that I can do front end testing of the components.
Python can also help with doing front end testing. For one of our components we have a bunch of static ‘demo’ data that we can use. Of course that data can become stale over time and so we need to occasionally refresh it. I use python to automate this process. It can make the request to the API to get the new data and then do the cleanup need to make it work as static files.
I have also experimented with a few other tools and hope to flesh out a bit more automation as we move forward, but for now that is my API testing tool set.
Photo by Bernard Hermant on Unsplash
So the video I am going to share today isn’t explicitly about testing an API, but if you want to test something you should really understand it. I think this video is a great introduction to understanding APIs and thinking about how they work.
Note that this video mentions the Postman addon, but you will probably want to use the desktop version of Postman now.
Hope you enjoy it.
Photo by Jakob Owens on Unsplash