11436 SSO

API-First Development with Apigee-127 (and Swagger-Node)

Chapter 1: How and why we built an API to power our platform
Gregory Brail
Oct 23, 2015

This is the first in a series of posts examining the considerations that went into building an API to power our intelligent API management platform solution.

Starting in in early 2011, we set out to re-design our product for managing APIs. The project had a lot of requirements based on our experience running our product on our own and in our customers’ data centers. One of the requirements would be to make sure that everything it could do was powered by a well-designed, intuitive API.

Furthermore, the bar for this API was pretty high. We’ve made a big deal at this company about APIs (hence the name) and have been clear that the most successful APIs are ones that are designed in a clear and consistent way. We knew that this management API would be used by everyone who uses our platform, which we hoped would be tens of thousands of developers, if not more. We also knew that we had a reputation to maintain, so the design had to be pretty good.

That meant following certain patterns that we had seen used in other successful APIs, and following them in a consistent way. For instance:

  • To create a new “application,” POST to a URI named /applications
  • To get a particular application, GET from a URI named /applications/{name}
  • To delete an application, DELETE that same URI
  • To update that application, PUT to the same URI

There are other aspects to the pattern, but it was one that's well understood around the industry and which, if done consistently, would result in an intuitive API that didn’t require a huge cognitive load. Once you understood “applications,” the same patterns would work for “developers” or “messages,” and so on.

Building the API

Now the next step was to build it. We had a small but effective team located in Bangalore that was tasked with building this new platform. While they were devoted to building a great product, they didn’t have as much experience with our particular way of designing APIs as I did, nor did they feel as much passion about it. More importantly, they had many more urgent things to do than debate the finer points of the design of each API. Plus, they were 12 (and a half!) time zones away.

I felt that I had to design the API myself. I ended up doing what generations of API designers did, which is to open up a text editor and start typing. I ended up writing up little snippets like many others have done:

GET /applications Return a list of the names of the applications

GET /applications/{id} Return the application with the specified ID

POST /applications Create a new application

And so on.

The developers then did what they did, which was to turn those high-level descriptions into APIs. We were building this API in Java, so the code looked a bit like this:

@GET

@Path("/applications")

public List<String> getApplications() {

// Blah blah blah

}

@GET

@Produces({"application/json","application/xml"})

@Path("/applications/{id}")

public Application getApplication(@PathParam("id") String id) {

// Blah blah blah

}

@POST

@Produces({"application/json","application/xml"})

@Consumes({"application/json","application/xml"})

@Path("/applications")

public void createApplication(Application app) {

// Bleh bleh bleh

}

So far so good—we have an API!

The disconnect

Of course, things didn’t always work out so cleanly. For instance, a developer might end up wanting to add a method to get all the applications, but to get all the details, not just their names.

According to our API design, this would happen by adding a query parameter called “expand” to the “/applications” URI. A GET with “expand=true” would return more information than one that did not.

The developer, however, might have a different idea, and instead add a completely new URI like this:

@GET

@Path("/getApplicationsWithDetails")

public List<Applications> getApplicationsWithDetails() {

// Blah blah blah

}

(Note: Our actual developers understood our standards better than that and wouldn’t have done something that far from our design principles, but I’ve seen things like this happen elsewhere.)

As the product grew, and as the team grew, these kinds of problems kept cropping up.

We eventually grew to realize that API design is like user interface design. Just as we would not roll out a new user interface without letting our user interaction people work on it, and wouldn’t put something on the corporate web site without approval from our chief graphic designer, we would not want to put out an API without input, or even approval, from the small, passionate group of API experts on our product team.

But that didn’t happen. New APIs kept cropping up, and the PMs and others would ask questions like, “did you know about that API? Why did they decide to call it ‘audit’ rather than ‘audittrails’? Why is that a POST and not a DELETE?” And so on.

Plus, we didn’t have an effective way to decide on the API design before the code was done and pushed to production. In lots of cases, if we changed it “now” we’d have to stop-ship the whole release, or go back to customers and tell them that we made an incompatible change to an API which would break all the scripts that they wrote, not to mention our own UI and other tools.

The documentation

What made things extra difficult was that we’d now have to go back and document these APIs. Since we had built them in Java and the truth was in the code—and nowhere else—that meant the process was expensive:

  1. Either the (expensive) engineers would have to sit down and write out, in a text doc of some sort, exactly what the API did, what all the parameters were, example inputs and outputs, error conditions, and so on. Some engineers are also great writers who love to describe what they did, but that’s not everyone. Furthermore, documenting every aspect of every parameter is no fun for anyone to have to do more than once.

  2. Or the (expensive) technical writers would have to read the code and figure those things out, which slowed them down since the answers were not always obvious and required asking more questions of the engineers.

  3. Or, someone like me would have to read the code, read the docs, and do a two-way mental diff until we felt that the docs were correct enough.

But Roy Fielding said ...

The API design principles I just described are popular in the world of APIs, and in my experience represent what nearly every developer today recognizes as an “API.” The term “REST” is often used to describe such APIs, but that’s not what “REST” is really supposed to mean.

It is popular in API design circles today to advocate for “hypermedia-driven” APIs, which are constructed just like web pages, as a series of documents containing hyperlinks. The more extreme proponents of hypermedia claim that documenting such APIs is unnecessary, because a smart client will discover what they do by following links, just as a human discovers what a web app does by looking at rendered HTML pages.

While the APIs described here are not hypermedia-based and thus aren’t actually “REST,” I believe that the principles are the same. The idea that a set of hypermedia links in an API response obviates the need to create API documentation for humans just doesn’t make sense.

At this point, we knew that we could come up with something better—something that adhered to our central tenets of API design and helped us solve some of these problems.

In the next installment of this series, I’ll describe how we started to close the loop between API design and implementation, the role Swagger played, and alternative frameworks for building APIs.

Photo: webtreats/Flickr

Microservices Done Right

Next Steps

 
 

Resources Gallery

News