Testing Third Party APIs

Testing Third Party APIs

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of things including lives and livelihoods. I want to take some of the focus off the destruction this virus is causing, and focus instead on creating something. I’m doing a series of posts on API testing as my way of fighting back against the virus. We can still go on with life even in times like this.

APIs great. They are a powerful way for different applications to work together. They allow for the easy integration of one application with another. APIs really have enabled much of the modern web. They are great.

Until.

Until the API changes. If it is an internal API, it might not be too much of problem, although in some companies the API team might be totally separate from those that use the APIs and so even internal API changes can cause problems. In fact, in bigger companies with separate API development teams, it often makes sense to treat internal APIs as if they are third party. API developers will also often version APIs so that you can continue to use old versions without breaking your workflows, but what happens when the company decides to deprecate or stop supporting old versions of an API?

Software changes are inevitable. Any piece of software that people care about and that is actively being used is going to change. Somehow though this seems to be a difficult thing for us humans to get our minds around. We want to be able to ‘set it and forget it’ but since we know software is going change we can’t do that. In broad strokes there are two strategies we can take to deal with this

Test First

The first strategy we could take is one where we just fully embrace the fact that software changes will happen and so we don’t just start using an API and hope for the best. Instead we build up a set of tests for the API that verifies that it does everything we need it to do. We then run those tests before every build and use them to ensure that the API still does what we need it to before we start using it in any of our software.

This approach obviously will take quite a bit of work since you will need to create the tests up front and run and maintain them into the future, but there are times when this is a good strategy to employ. If you are using a third party API to provide you with functionality that is core to what you application is doing, you will probably want to verify the API before you consume it. The more critical the information you are getting from the API and the more devastating the effects of failure the more careful you will want to be with verifying it ahead of time.

Monitor

There is another approach you could take though. Instead of testing the API before you even use it, you could just monitor for failures and then fix things up afterwards.

This idea can sometimes rub testers the wrong way, since we want to be able to prevent failures before they are out in the wild, but there are times when this strategy makes sense. Perhaps the API you are using doesn’t provide critical information and if it stops working you won’t have a huge impact on your clients. Or perhaps the API calls are for internal scripts that workflow that help you with your job, but if they break and those scripts don’t work for a few days it’s no big deal. There are many cases where broken functionality isn’t too big of deal as long as you can fix it up in a timely way. In those situations it makes a lot more sense to just monitor for things breaking and deal with the changes after you see something broken.

I would also remind you that monitoring doesn’t have to be a big complicated high tech thing. For example, I have a course on API testing and in that course I provide a lot of example APIs that people can use to practice their testing with. However, software changes, and so sometimes the examples that I use don’t work anymore due to changes in the API. This has happened in the course and one of the students sent me a message and asked me about why the example wasn’t working. I then went and looked into it and was able to update things to point to the correct API. The student reaching out to me is an example of monitoring. It might not be very efficient monitoring since it depends on the goodwill (or annoyance) of one of my students, but the usage of that API is still being monitored.

Monitoring doesn’t have to be high tech, but you should consider how you would actually know if it fails. Do you have to wait for a client to complain (like in my example), or do you have some internal user or process that you can rely on to let you know if it has failed? You may even want to consider monitoring software of some kind if it is important to know about breaking changes in a timely manner.

Mitigation

Regardless of which approach you take, you should have a way to mitigate the problems when an API fails to do what you want it to. Monitoring that lets you know that your API calls are failing doesn’t do you much good if you don’t know how fix it or if you don’t have some way to work around the problem until it is fixed. The same applies with testing it up front. If you don’t have a plan for how to make the changes that you need to when the API stops working you could end up holding back changes that you want because of a ‘broken’ API. Understanding what you will do if/when the API breaks is an important part of dealing with 3rd part APIs.

Mitigation is also important in another way. No matter how much testing or monitoring you do, things are going to fail in weird ways sometimes. There are some things that testing just can’t deal with. What if the service the API provides you access to goes down or stops working (or maybe even the company providing it goes bankrupt or stops giving access to it). What do you do? I guess one thing you could do is deny it – that will never happen! But if covid-19 has taught us anything it is to never say something like that! Having some kind of fall back strategy that at least informs users that something has gone wrong and preferably also gives them some kind of (perhaps reduced) functionality is good plan when possible.

Conclusion

Testing 3rd party APIs is actually a lot like testing other kinds of software. You need to think carefully about what kind of testing it makes sense to do for the given context you are in.

As to the API calls that break in my course, I have been taking a very low tech (and inefficient) way to monitoring them by waiting for students to report issues. I’m not sure I like that. Who knows how many students just skip past the broken stuff before one finally asks a question and lets me know what is going on. I would like to know sooner if the one of the APIs I’m using breaks or moves to another location. I plan to make a simple script that will ping the APIs I’m using once a day and email me if the endpoint goes away or changes. I want to make my monitoring more efficient. What about you? Are there any places in your application that you are doing your monitoring by waiting for a human to notice something? Could you maybe improve that just a little bit with a simple script?

Third party APIs have enabled a lot of the wonderful things we see on the modern web, but they have their own set of risks. What are you doing to test for and mitigate them?

Photo by Mari Helin on Unsplash

Types of APIs

Types of APIs

COVID-19 Note: There is a virus shutting down the world right now. It’s destroying a lot of things including lives and livelihoods. I want to take some of the focus off the destruction this virus is causing, and focus instead on creating something. I’m doing a series of posts on API testing as my way of fighting back against the virus. We can still go on with life even in times like this.

In order to effectively test an API, you need to know what kind of API you are testing. Often in today’s world when we talk about APIs we are talking about REST APIs, but there are several other important API formats that you might need to test. Let’s look through them and see what we can learn about some of the major API types.

REST APIs

We’ll start with what is probably the most common type of API you’ll come across on the modern web; the RESTful API. REST stands for Representational State Transfer and refers to an architectural style that guides how you should create APIs. I won’t go into the details of the properties that a RESTful API should have (you can look them up on wikipedia here if you want), but there are a few clues that can let you know that you are probably testing a RESTful API.

RESTful APIs are based on a set of guidelines and so they do not all look the same. There is no official standard that defines the exact specifications that a response has to conform to. This means that the following are in general good clues and hints that you are dealing with a REST API, but they are not definitive. It is also worth noting that many APIs that are considered to be RESTful do not strictly follow all the REST guidelines. REST in general has more flexibility than a standards based protocol like SOAP, but this means that there can be a lot of diversity in the way REST APIs are defined and used.

How do I know if the API I’m looking at is RESTful?

So, what are some clues that you are looking at a REST API? Well in the first place, what kind of requests are typically defined? Most REST APIs will defined GET, POST, PUT, and DELETE calls with perhaps a few others. Depending on the needs of the API, it may not define all of these, but those are the common ones.

Another clue, is in the types of requests or responses that are allowed by the API. Often REST APIs will use JSON data in their responses (although the could use text or even xml). Generally speaking if the data in the responses and requests of the API is not xml, there is a good chance you are dealing with a REST based API of some sort.

REST API Example

There are many examples of REST APIs on the web, but let’s take a quick look at just one of them to help cement the idea of what RESTful API is in our minds. It’s the internet, so let’s look at cats, and since this is an article about API testing, let’s also look at http status codes (which are used by REST APIs). And of course, the internet knows that combining cats with http status codes needed to be done, so we can go to https://http.cat/ to see a cat image for each http status code.

This site provides an API and so we can call https://http.cat/200 to get the cat image for the 200 status code. For example, if we were to do this in Postman, we could create a request, set the method to GET and put in the url and click on send.

call cat API in Postman

We don’t have to specify any xml when we make the call and can just send the request directly in Postman (or another API tool) as a GET call, so we are most likely dealing with a REST API here.

SOAP APIs

Before REST there was SOAP. SOAP stands for Simple Object Access Protocol. The SOAP protocol has been around since long before Roy Fileding came up with the concept of REST APIs. It is not as widely used on the web now (especially for smaller applications), but for many years it was the default way to make APIs and so there are still many SOAP APIs around.

SOAP is an actual protocol with a W3 standards definition and so its usage is much more strictly defined than REST which is an architectural guideline as opposed to a strictly defined protocol.

If you want a little light reading, check out this document. It claims to be a:

non-normative document intended to provide an easily understandable tutorial on the features of SOAP Version 1.2

I’ll be honest, I’m not sure how well it delivers on the ‘easily understandable tutorial’ part of that statement. Looking at some of the examples in there may help you understand why REST APIs have become so popular. SOAP APIs require a highly structured xml message to be sent with the request. Being built in xml, these requests are not that easy to read for humans, and require a lot of complexity to build up. There are of course many tools like SoapUI that can help with this, but in general SOAP APIs tend to be a bit more complex to get started with. You need to know more information (like the envelope structure) in order to get started.

How do I know if the API I’m looking at is a SOAP API?

The most important rule of thumb here is: does it require you to specify structure xml in order to work? If it does, it’s a SOAP API. Since these kinds of APIs are required to follow the w3c specification, they must use xml and they must specify things like env:Envelope nodes inside of the xml. If the API you are looking at requires xml to be specified and that xml includes the Envelope node you are almost certainly dealing with a SOAP API.

SOAP API Example:

A SOAP API example is a little bit harder than just sending a GET request to an endpoint. We will use this webservice as an example SOAP API. Let’s look at the first action there, which gives us a list of continents. In order to call this API in Postman we will need to set up a few things. We of course need to create a request in Postman and then we will need to set the request method to POST and put in the url. However we can’t yet click send. Since this is a SOAP API, we need to send some xml information as well. In Postman, this means setting the body type to raw, and choosing XML from the dropdown and then putting in the xml envelope data as indicated by the documentation

SOAP API Information

For this particular API, we would also need to modify the Content-Type header (by adding a new one) at the bottom, so that is is set to application/soap+xml

Content-Type Header

As you can see, there is a lot more complexity to calling SOAP APIs. REST APIs can of course have complex bodies specified as well, but the requirement to do this in xml and the existence of the Envelope node in this, indicates that this API is indeed a SOAP API.

GraphQL APIs

SOAP came before REST and in many ways REST was designed to deal with some of the shortcomings of SOAP. Of course in software we are never done making things better and so along comes GraphQL. GraphQL is a query language and it was designed to deal with some of the situations where REST APIs have shortcomings. RESTful APIs don’t know what specific information you might be looking for and so when you call a REST API endpoint, it gives back all the information it has. This can mean that we are sending extra information that you don’t need, or it can mean that we aren’t sending all the information you need and you need to call multiple endpoints to get what you want. Either of these cases can slow things down and for big applications with many users that can become problematic. GraphQL was designed by Facebook to deal with these issues.

GraphQL is a query language for APIs, and so it requires you to specify in a query what you are looking for. With REST APIs you will usually need to know what the different endpoints are in order to find the information you are looking for, but with a GraphQL API a single ‘endpoint’ will contain most or all of the information you need and you will use queries to filter down that information to only the bits that you are interested in. This means that with GraphQL APIs you will need to know the schema or structure of the data so that you know how to properly query it, instead of needing to know what all the endpoints are.

How do I know if the API I’m looking at is a GraphQL API?

Well, if the documentation is telling you about what kinds of queries you need to write, you are almost certainly looking at a GraphQL API. In some ways a GraphQL API is similar to a SOAP API in that you need to tell the service some information about what you are interested in, but a SOAP API will always use xml and follow a strict definition in the calls, whereas GraphQL APIs will usually a bit simpler and not defined in xml. Also with GraphQL the way the schema is defined can vary from one API to another as it does not need to follow a strictly set standard.

GraphQL API Example

Let’s take a look at a real life example of calling a GraphQL API to understand it a bit better. This example will use the countries api as hosted here. There is information about the schema for it on github here and we can use that to create queries in the provided playground, but for this example let’s look at setting it up in Postman

Similar to calling a SOAP API, we will need to specify the service we want and do a POST request. We will also need to choose the GraphQL option on the body tab, and put in the query that we want

GraphQL query

As you can see in the example I’ve done, I’ve requested the name and languages of Canada. Once I have specified this information I can click Send and I get back some json with the country name and a list of the official languages. If I wanted additional information (Say the name of the capital city), I could just modify the query to include a request for that information and send it off again using the same endpoint.

Conclusion

We’ve looked at three different kinds of APIs. Each of them has it’s own strengths and weaknesses and is used in different applications. Some applications will even support more than one type of API, and as a tester you need to be ready to test any kind of API that you come across. Hopefully this article has at least helped you in that journey and given you some understanding of how the various APIs you might come across work and can be interacted with.

Photo by Markus Spiske on Unsplash