Getting started with the Open API Specification

Twenty, twenty-one, twenty-two. Twenty-two seconds! Twenty-two seconds to load the data for a simple table. Somehow we had repeated the same mistake we had run into before. A year or two before we were working with an API that used sperate endpoints for each cell in the table we were creating on the page. That works fine until you have a table with 100 rows and 5 or 6 columns. Now you have to send hundreds of API calls and even if each one is small on it’s own, it takes time for the calls to traverse the network. The HTTP protocol limits the number of calls you can make concurrently and so queing theory applies and the load time for the table gets really slow.

And now here we were. We were creating another table in our UI and the API that supplied it had a different endpoint for each cell in the table. How did we get here? During the design we had discussed performance and how we would need to design the API so that we wouldn’t come across this exact issue. Somehow as a team when we go caught up in the work of creating the APIs that would pull together all the data we needed, we missed this piece of the API design.

We are a well functioning, highly productive software development team. In fact, we refactored the API to work as it should have within a week, and yet we had missed this. I suspect we aren’t the only team that has faced something like this. We were building a system that would deal with a lot of complexity and present it in a simple way so that our clients wouldn’t need to deal with it. The challenge of course, is that there are so many details to keep in mind. Domain and functional requirements. Design and presentation of the UI. Security and correct user permissions. Accessibility and performance. It isn’t possible to keep this all our heads at the same time. Thing get missed.

That’s why we have testers right?

I don’t know. That doesn’t feel like the right approach to me. Sure, we can use after-the-fact testing to find those bugs, but we had already identified this bug. We knew that creating an API where each cell in a table needs to call a different endpoint is not scalable. In essence, we had to find this bug twice. Once during design and once after development. The second time we found it, the bug was expensive to fix. Could we have prevented that regression? Can we create regression tests for our designs?

This is one of the problems that API specifications attempt to solve. In reflecting on this issue, I started to wonder if we could have saved ourselves a lot of rework by encoding the things we had discussed and discovered during our design sessions. What if we wrote them down in an API definition that set out what the API should look like and do?

You can’t do much about the past, but you can learn lessons from it and use those lessons to make the future better. I’m sure our team will be writing more APIs in the future and so I want to explore some of the different API specification to see if there is something that would make sense for us to use in the future.

Open API Specification

The most common specification format is the Open API Specification (or OAS). It has in many ways become the default specification for describing how APIs work. It has very broad adoption across many companies and has a large supporting community. It is designed for RESTful APIs, which we use, and it has support for almost anything you would do with these APIs.

I’ll be honest though, I find it overwhelming to get started with. It is a lot of work to document every detail of what your API can or should do. Since we have existing APIs that do not yet have API definitions I thought I would try using the Swagger inspector. It gave me something to get me started, but it still needed a lot to be done by hand. Since at this point, my goal is more around evaluating if this makes sense for future APIs, I decided to instead look at creating a definition “from scratch”. I’d like to use a design first rather than code first approach to this.

So how hard is it to create an API defintion that follows the OpenAPI Specification?

The Tools

Well, let’s start with talking about the tools. There are a lot of Swagger tools that work with Open API Specifications. Confusingly the OAS used to be called Swagger, but once you get past that, there are still many tools out there. It’s easy to get lost in trying to figure out which tool does what, and which one you want to use. I know because I spent too much time trying to figure out which one to use. Eventually I decided that I wouldn’t use any of the custom built tools. OpenAPI defintions are just text files (either yaml or json format) and so they can be edited in any text editor.

I created a file in my trusty Visual Studio Code, installed a couple of OpenAPI formatting and linting extentions and I was finally off the races. I’m sure that if I use this more, I will eventually want tools that are built specificially for the OAS, but I’ll stick with the basics for now and see where the pain points show up as I scale. If you are new to using the OpenAPI Spec, I’d recomend taking this approach too. Not only will it force you to understand what is going on more deeply, but it will also remove a whole layer of confusion that can get in the way of just getting started.

Creating the API Defintion file

A blank file staring at you can be intimidating, but I guess the best way to get something done is to start doing it, so let’s look at what data needs to go into this file.

The First Section

So, how do you actually create an API definition with the OAS? The basics of it are straigtforward. Start with a bit of boilerplate at the top that says what version of the OpenAPI Spec you are using along with some information about the API your are creating the definition for.

openapi: 3.0.1
info:
  title: A cool API
  description: This API does cool stuff
  version: '1.0'
servers:
  - url: https://{tenantId}.domain.api.com
    variables:
      tenantId:
        default: tenant-id-guid-123

One simple thing I ran into here was that each instance that we create (and we have at least one for each of our clients) has a different base url. This includes all the testing sites. Pretty easy to take care of though. You can create a parameter in the url. You can read more about how the base urls work here if you want.

Defining an Endpoint

With the boiler-plate out of the way, the next thing to define is an endpoint. Endpoints are defined under the paths section.

paths:
  /endpoint/path/{pathParameter}:
    get:
      description: Makes the world a better place.
      responses:
        '200':
          description: You got a totally ok result.
      parameters:
        - name: pathParameter
          in: path
          required: true
          schema:
            type: integer
            format: int64
            minimum: 6606

I’m obviously redacting some stuff here, since these are internal APIs, but it’s interesting to note how easy it is to create a parameter in the path. All I needed to do was to include it in the path defition and then create a parameters section under the request to define what kind of parameter it was.

You will notice though, that the responses object only defines the response code. It says nothing about what kind of content this API should give back. That seems like it would be kind of important. Defining this opens up a whole new can of worms. Obviously the API does not always give back an identical response every time you call it, and so you can’t just copy a response and put in in there. Instead you need to specify some general rules that should apply to the response. For example, you could specify that it should be a list and that each item in the list should be a string. These kinds of rules are known as schema. The kinds of objects our API represents are pretty complex and so it takes a bit of work to create a schema for them.

In fact, as I dug into this I realized that this is actually the area that was the most intimidating to me. I’m pretty comfortable with the way RESTful APIs work and so setting up stuff with the paths was pretty intuitive. However, although I have benefited greatly from using schema that other have made to help with validation, I have not had to create schema before. I did find a good schema generation tool here that really helped, but I needed to go through and do a lot of clean up. Everything that went into creating the schema is worthy of it’s own blog post someday, so I won’t go down that rabbit trail right now.

After creating the schema, I could use it to define how the content of the response should look:

paths:
  /endpoint/path/{pathParameter}:
    get:
      description: Makes the world a better place.
      responses:
        '200':
          description: You got a totally ok result.
       content:
            application/vnd.siren+json:
              schema:
                $ref: '#/components/schemas/SchemaName'

In the content object I’ve first defined the content type. This is what is returned in the Content-Type header of the response. I then use a $ref object to specify a reference location where it should look for the schema for this response. Schema is often shared between different parts of an API, and so by moving it into it’s own section, you can reference it from multiple places without needing to create it multiple times.

Defining Security

As I said earlier, I won’t get into the details of creating the schema in this post. Instead let’s look at how to define security options. The API I’m investigating uses bearer tokens which is pretty easy to define with the OAS.

components:
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
security:
  - bearerAuth: []

And with that, I have an extremely basic, but functional schema. I’ll leave it there for now, but in future posts I will share how it went when I was creating a schema and also share how I used the definition file to actually do something useful.

Although it was a bit intimidating at first, as I worked my way throught this, it didn’t take long to feel like I could put together something useful. How did you find it getting started with using the Open API Specification?

Photo by Jon Eric Marababol on Unsplash