I think performance testing is a very overloaded term. It can mean a lot of different things ranging from how fast responses come back, to how well things scale with increasing usage, to many other stress or speed related tests.
The way I approach performance testing in an API is similar to the way I approach all my testing, which is to say holistically. There are many great tools that can help with performance testing an API, but you really don’t need to go there right away.
Just the other day, I was testing an API that allowed you set a display of items as selected or not. The API calls were not the fastest, but they really weren’t that slow. That is they weren’t that slow until you looked at how long it took for the user to be able to interact with the list. You see, each item on the list made it’s own API call, and so if you had say, 100 items on the list, you would have to wait for 100 API calls to resolve before you could interact with the last item. This caused a performance issue. Not so much because the API calls were too slow, but because the way the API was being used by the client did not give the necessary responsiveness.
The point being that you can’t just find performance issues, by checking how fast API responses come back or by looking at how many calls per second you can send to the API. Those may be important things to check, but there is a holistic approach that needs to come into play with performance testing as well. How is the API used and can it support a valuable end user experience? To me this is the crucial question we need to start with and then we can use tools to help us along the way. We can create automated performance tests and use stress and load testing tools to help us keep things where they need to be in terms of the ongoing performance of different aspects of the system. We can also use tools like these to help us figure out and understand where issues are coming from, but we need to always keep in mind the end goal. The user doesn’t care about how many milliseconds it takes for your API call to return. They care about how quickly they can do what they want to do. Make sure your API performance testing is in support of that!
Photo by Marc-Olivier Jodoin on Unsplash
offbeattesting 