Author Archives: offbeattesting

Today’s challenge is about explaining mocks, stubs and fakes.

I don’t know if I can pull it off.

The problem is, different teams will use these words in different ways. I don’t think it makes a lot of sense to argue about what exactly each of them means and how they are distinct from each other, but there is some value in at least understanding the general idea that these terms are meant to communicate.

In the broadest terms, when we talk about these things we are talking about something that is a substitute for a piece of production code. For example, we might have a database that stores some values. When running tests though, it might expensive to go retrieve those values many times, so instead we might make a local text file that has a few of the values we care about hard coded into it and just use that instead of the ‘real’ databases.

No matter what term we are talking about – mocking, stubbing, faking, etc. – what we are doing is using something that is not the production code to simplify in some way what we are doing for our tests.  This helps us isolate things to just the specific part of the code we are interested in and can be used in some really powerful ways.

I wouldn’t worry too much about understanding in an absolute sense what these various terms means. What matters in figuring out what they mean in your context. How do the members of your team use these terms?  What do they mean by them? How do the tools that you use talk about these things and what do they mean by it?  Understanding how the terms are used in the contexts in which you work will help you better communicate with those that you work with.

The idea of mocking, stubbing or faking something is a very powerful testing concept and well worth taking the time to learn how to do, but don’t get too caught up in the specifics of the terminology.

Photo by Jelleke Vanooteghem on Unsplash

What skills does a team need to succeed with API testing?

Let’s start with what I think is the most obvious one. Testing skills. I’ve said this before, but API testing is, well, testing. You need to approach it with all the same skills and mindset you would approach any testing task.

I think you also need someone who understand or can learn some good API testing tools. Effective API testing often involves diving deep into the API and the better you understand tools that can help you do that, the more effective you will be in your testing.

Creating a valuable API, also means you need someone who deeply understands the business or customer need for this API. The vast majority of API bugs I find are related to the API not providing the information the business needs, or providing it in ways that are difficult to find, use or understand. This does fall under general testing skills, but it is worth mentioning again as it is vitally important that APIs actually do what the business needs out of them. Sometime we can forget that when we are deep into something technical like an API, but we (should) only write these to help drive the value we are after. If they aren’t doing that, they need to be changed.

On last thing I will mention here is security. Security is important everywhere in the software development process, but I think it is especially important in APIs.  If I were trying to break into a site or service, I would start with the APIs.  They are designed to be scripted, which means you can use a lot of algorithmic and brute force attacks to try and break into them. You definitely don’t want to forget about security and authorization when testing an API.

Photo by rawpixel on Unsplash

There are so many tools out there for testing APIs, that it can be overwhelming. I use a few different tools, depending on what I’m trying to do. One thing that can be difficult in API testing is figuring out what routes are available in your API.  This is especially true if the API is poorly documented or does not including any self-documentation.

There are a number of different tools out there that will help you discover what calls your API is making, but do you want to know what my favorite tool is for this?

The developer console.

Yup, that’s right. The developer console. There are tools like Wireshark or feature built into tools like Postman that let you ‘watch’ network traffic and figure out API calls from there, but I find that often the API calls are right there in the network tab of your console. It can be quick and easy way to get the info you need to get you started.

Photo by Markus Spiske on Unsplash

Today’s challenge involves reading and sharing a blog post. I read a lot of blogs and I find it to be a great way to learn and stay current with thoughts in the industry.  I’ll actually share a couple of posts. The first one is more of a reference.  The Web API Checklist gives a lot of useful things to think about when testing and API and is a great reference

Another great resource, if you are getting started with API testing is Katrina Clokie’s API Testing Pathway.  She gives you a lot of guidance on this without specifying everything you do. I’ve found this to be a great resource and would highly recommend it to anyone interested in learning more about API testing.

What blog post or resources have stood out to you?

Photo by Autumn Mott Rodeheaver on Unsplash

Today’s challenge involves sharing publicly available APIs that you can use for testing. In putting together recent courses on API testing I have spent a bit of time looking for APIs like this. The Ministry of Testing Club has a thread on sites you can use for practicing testing which was very helpful to me. There is also another thread on the MoT Club that directly asks about useful real world APIs to test against. If you are looking to practice API testing those are great places to start.

There are a lot of good resources for practice in testing GET calls, but it is a lot harder to find ways to test POST, PUT or DELETE calls. This is of course understandable, as those calls change things in the application, meaning it is hard to keep the application consistent for each user.

In recognition of this, I have tried to create some API Testing Challenges that can be used to learn testing.  So far they are pretty basic and just at the starting stage, but if you want to try them out, you can go ahead and get them on Github here.  They are pretty rough at this point, but feel free to use them if you want.  Feedback is always appreciated as well of course!

Testing is something that it hard to learn without doing it, so I would encourage you to follow up on some of these resources.  Try to ‘test’ some of the sample APIs out there and see how much you grow as a tester!

Photo by Jordan Sanchez on Unsplash

API as a term has become synonymous with client server interactions over the internet. As such APIs use the HTTP protocol to communicate. HTTP stands for HyperText Transfer Protocol, and interestingly, when you think of API calls (things like GET, POST and PUT), they are just directly using commands built into HTTP. These commands are not something that are defined in some API standard outside of the protocol. They are actually part of the protocol itself.  This article gives a nice summary of HTTP at a high level.

If you want to dive a bit deeper into the detail of this, you can follow along with this tutorial.  If you are testing APIs, it is probably a good idea to grasp at a basic level how HTTP works.  Honestly, if you are testing anything client/server based on the web it is probably a good idea to have a basic understanding of this stuff. I must confess that I really hadn’t read much on HTTP before. I’m thankful for this challenge that pushed me to do this!

Today’s challenge is to begin reading an API testing book and share something from it by the end of the month.

I’ve downloaded this book to my kindle.  It looks like it is more about building an API than testing one, but I’m thinking that learning how to build a good API is a good step in learning how to test one.

I will share a review once I have complete the book

I recently wrote an article about how API testing is exploratory, so I won’t spend too much time on this one, but I do want to re-iterate that all testing involves exploration.

When I approach any new testing task (including API testing), I start with exploration. How else are you supposed to figure out what to do? My approach to this in an API is similar to any other area I might be exploring. I start with some piece of information I know (perhaps an API end point) and do something with it (GET the request) and then look at the results of that and see if there is anything interesting or that seems to be out of place.  I then follow up on that, making notes along the way, until that line of inquiry has died down.

I will talk to developers and other testers and consult documentation to try and find new starting points and follow them down various branches as they seem interesting, occasionally circling back as I learn more about the application.  I take streaks of learning and keep expanding on them as needed until I am satisfied that I have a reasonable understanding of the state of the feature and can articulate it to those that need to know about it.

I use tools along the way of course.  Things like Postman and Python and Curl and developer tools in the browser. Anything really that will help me answer the questions I’m curious about and give me insights into other things that might interesting. The tool set might differ in API testing, but the principles are the same. Follow your curiosity. Pay attention to what is happening. Ask lots of questions. Make notes. Grow your mental map. Try new things. In a word, Explore!

Ministry of Testing has another 30 days of testing challenge, this time around API testing.  Looks like fun, and I’ve been doing a lot of API testing lately, so I’ll follow along and see how many of them I can do over the next month.

The first one on the list is to define what API testing is. I actually see two distinct parts to API testing.  In one case you could consider it to be testing of APIs themselves, but in another case you could look at it as testing an application or service using the APIs.

I tend to think of API testing in the second sense, although of course there is overlap between the two. To me, API testing is about using the API to help you discover useful information about the product. That might sometimes be finding actual bugs in the API, but often it can mean using the API to drive testing of other things, or finding issues in the way different parts of the service work together.

Not a formal definition, but that’s the way I think about it. How do you approach it?

I’ve been doing a fairly deep dive on API testing over the last several month, both as part of the project I am currently working on and as part of some courses I am preparing. As with any specialization there is a lot of terminology that goes into it and so I thought I would put together a post that summarizes a number of definitions related to API testing.

Many of these definitions can have different meanings depending on the context you are working in and they are used by your specific team, so don’t take this as definitive. They are just the way I have explained them to myself so that I can better understand and conceptualize them. By defining these terms I am better able to wrap my mind around them and to use them to do better testing.  Hopefully they help you out too in your testing journey

REST

Acronym for REpresentation State Transfer.  This is a protocol that is based on the dissertation of a guy name Roy Fielding. Obviously there is a lot that goes into this, but in really simple terms, a RESTful API is one that consistently applies actions (verbs) like GET, POST, PUT and DELETE to resources (nouns) which are usually a url that may have some parameters.

Further Reading:

Martin Folwer’s take on using REST well

Ruben Verborgh has a good explanation on Quora

Hypermedia

People will argue about this (surprise), but some say that an API is only truly RESTful if it uses a hypermedia approach. Hypermedia means that the server tells you what resources are available for you to use. Every request to the server should tell you what other resources and actions are available to you on objects that are related to the request you just made.  Sounds confusing? You are already pretty used to it. You came on this web page and there are number of links here that allow you navigate to other places on the web. When we simplify it down, that is really all hypermedia in an API is doing. It is telling you some other links (endpoints) and actions that you can use in the API.

HATEOS

Speaking of Hypermedia, let’s just make up a big long hard to say acronym that describes it’s usage. HATEOS stands for Hypermedia As The Engine Of State and is just a way of making people argue about pronunciation saying that your REST API uses Hypermedia.

GraphQL

It’s new! It’s exciting! It shall rule the world! Ok, in reality we like to get excited about new things in the tech space, but GraphQL is just another way of specifying an API. It is a query language that helps to optimize some things in network API calls and so for applications that have high performance requirements it can be very helpful. It is a bit more complex and rigid then REST though.

Idempotent

Big word for saying that every time you perform a call you get back the same result. I’ll give you a silly example to help you remember.  Think about putting snow tires on a car. Once you have them on you have a car with snow tires. Now if you repeat that ‘request’ you will end up with the same thing. A car with snow tires. In an API this would be a PUT call.  No matter how many times you send the call (with the same parameters), it should always give you back the same result.

Safety

Speaking of idempotency, a GET call is idempotent (you get the same result every time you execute it), but it has an additional factor called safety.  Safety just means that nothing changes when you issue the command. Let’s use a silly example again. Imagine a bookshelf.  You bend your head sideways a read the title off the spine of a book. Nothing has changed and no matter how many times you do that nothing will change. This is an example of a safe call.

Verbs and Nouns

No this isn’t English class, but we do sometimes talk about nouns and verbs in APIs. Verbs are the actions that an API can do (like GET, POST, PUT, DELETE) and the nouns are what the API acts on (resources usually represented by url endpoints)

Services (micro-services)

Buzzword time. So what is a micro-service? Well, it is a service that is very small, or uh, micro sized (see what I did there?). And a service is just something that does stuff and that lets you tell it how to do stuff (usually through APIs).  See it doesn’t have to be hard! Let’s use an example to get a better handle on it.  You want to create a meme (because you are that kind of cool) but photo editing is just to passe for you.  If only you had something you could send a command to that said ‘generate a cat meme for me with these words.’  Well if you did, that would be an example of a service – memes as a service in fact. You give it some commands.  It does something for you and produces an output for you based on the commands you gave.

Micro-services is just an architectural pattern that tries to have a number of services that can each do one specific thing. These various services can then talk to each other through the defined APIs. The micro part just means that each service has a limited number of tasks that it can do. So you might break down a customer facing service into a number of micro-services that each do one particular part of the overall task at hand.

Schema

Adding this one in based on comments. A schema is used to define the structure of the data in an API. It defines things like what the API can do with various endpoints and what data it is allowed to use. If you have a well defined schema for your API it can be used to automate things like the creation of documentation and sometimes even the way  it is used. If you have defined you schema in the Open API format you can use tools like Swagger to help with this.

I have added the next few terms based on questions I have received through private messages.

Payload

When you send API requests you are trying to get a service to ‘do something.’ The service will then tell you what it did or give you what it asked for. The payload is the part of that response that is communicating directly to you. In REST APIs this is usually some JSON formatted data.

Let’s say you want to see a random picture of a cat (because you are on the internet). You go to The Cat API (yes it exists – this is the internet we are talking about) and you send an API request. You get back an JSON object with a link to a cat picture along with a few other pieces of information. That JSON is the payload.

Payload isn’t just something you get from the server though. You can also send payload to the server as well. This is known as the body of your request and is once again usually a JSON object that you send to the server. It is used to make more complex requests or when you are trying to create or modify an object on the service and you need to give it information about the object you are creating

Endpoint

An API is a communication channel. Think of making a phone call. Your phone is one end of the communication channel, but you need to somehow connect to another phone. To do this you punch in someone’s phone number. You can think of that phone number as being similar to an API endpoint. That phone number lets you complete the communication with a particular phone of your choosing.  In an API you do this with a link to the resource that you want to interact with.  The link is kind of like that phone number. It lets you ‘connect’ to a particular item on the server and interact with it.

URI

URI stand for Universal Resource Indicator and pretty much it’s just a fancy way for saying URL which in the API world is pretty much just another way of saying endpoint. Confusing? Yup. That’s why I made this glossary.

All good things must end – including this list

I’m sure there are other terms to define in API testing. If there are any you find confusing, let me know and perhaps I’ll add to this list over time. Maybe I’ll even get ambitious and make this into it’s own page on the site. We will see.

Let me know what you think of these definitions!