RESTful API design

RESTful API Design: what about attribute names?

In a recent post in this series about Pragmatic REST API design, I talked about formats - supporting multiple formats and working with JSON as the default. Check out the full series.

This time, let's talk about what happens when a response comes back.

You have an object with data attributes on it. How should you name the attributes?

Here are API responses from a few leading APIs:

Twitter

"created_at": Thu Nov 03 05:19;38 +0000 2011"

Bing

"DateTime": "2011-10-29T09:35:00Z"

Foursquare

"createdAt": 1320296464

They each use a different code convention. Although the Twitter approach is familiar to me as a Ruby on Rails developer, I think that Foursquare has the best approach.

How does the API response get back in the code? You parse the response (JSON parser); what comes back populates the Object. It looks like this

var myObject = JSON.parse(response);

If you chose the Twitter or Bing approach, your code looks like this. It's not JavaScript convention and looks weird - looks like the name of another object or class in the system, which is not correct.

timing = myObject.created_at;

timing - myObject.DateTime;

Recommendations

Use JSON as default

Follow JavaScript conventions for naming attributes

- Use medial capitalization (aka CamelCase)

- Use uppercase or lowercase depending on type of object

This results in code that looks like the following, allowing the JavaScript developer to write it in a way that makes sense for JavaScript.

"createdAt": 1320296464

timing = myObject.createdAt;

Next: What to do when your API returns something other than a resource.

RESTful API Design: support multiple formats

In the most recent post in this series about Pragmatic REST API design, I talked about pagination and partial response. Check out the full series. This time:

What about formats? Should you support only one format or multiple formats?
I recommend that you support more than one format - that you push things out in one format and accept as many formats as necessary. You can usually automate the mapping from format to format.

Here's what the syntax looks like for a few key APIs

Google Data

?alt=json

Foursquare

/venue.json

Digg*

Accept: application/json
?type=json

* the type argument, if present, overrides the Accept header

Digg allows you to specify in two ways: in a pure RESTful way in the Accept header or in the type parameter in the URL. This can be confusing - at the very least you need to document what to do if there are conflicts.

I recommend the Foursquare approach

To get the JSON format from a collection or specific element.

dogs.json

/dogs/1234.json

Developers and even casual users of any file system are familiar to this dot notation. It also requires just one additional character (the period) to get the point across.

What about defaults?

In my opinion, JSON is winning out as the default format. JSON is the closest thing we have to universal language. Even if the back end is built in Ruby on Rails, PHP, Java, Python etc., most projects probably touch JavaScript for the front-end. It also has the advantage of being terse - less verbose than XML.

Next time: what about attribute names?

RESTful API Design: can your API give developers just the information they need?

In a recent post in this series about Pragmatic REST API design, I talked about tips for versioning your APIs. Check out the full series.

This time - pagination and partial response - how do you give developers exactly the information they need?

Partial response

Take for example the Twitter API - a request for a tweet. You'll get much more than a typical twitter app often needs - including the name of person, the text of the tweet, a timestamp, how often the message was retweeted, and a lot of metadata.

Let's look at how several leading APIs handle giving developers just what they need in responses, including Google who pioneered the idea of partial response.

LinkedIn

/people:(id,first-name,last-name,industry)

This request on a person returns ID, first name, last name, and industry.

LinkedIn uses this terse :(...) syntax which isn't self-evident. Plus it's difficult for a developer to reverse engineer the meaning using a search engine. But this is the way that LinkedIn does partial selection.

Facebook

/joe.smith/friends?fields=id,name,picture

Google

?fields=title,media:group(media:thumbnail)

Google and Facebook have a similar approach, which works well.

They each have an optional parameter called fields after which you put the names of fields you want to be returned.

As you see in this example, you can also put sub-objects in responses to pull in other information from additional resources.

Add optional fields in a comma delimited list

The Google approach works extremely well.

Here's how to get just the information we need from our dogs API using this approach:

/dogs?fields=name,color,location

It's simple to read; a developer can select just the information an app needs at a given time; it cuts down on bandwidth issues, which is important for mobile apps.

The partial selection syntax can also be used to include associated resources cutting down on the number of requests needed to get the required information.

Make it easy for developers to paginate objects in a database

It's almost always a bad idea to return every resource in a database.

Let's look at how Facebook, Twitter, and LinkedIn handle pagination.

  • Facebook uses offset and limit
  • Twitter uses page and rpp (records per page)
  • LinkedIn uses start and count

Semantically, Facebook and LinkedIn do the same thing. That is, the LinkedIn start & count is used in the same way as the Facebook offset & limit.

Say you want to get records 50 through 75 from each system. You would use:

  • Facebookoffset 50 and limit 25
  • Twitter - page 3 and rpp 25 (records per page)
  • LinkedIn - start 50 and count 25

Use limit and offset

I recommend limit and offset. It is more common, well understood in leading databases, and easy for developers.

/dogs?limit=25&offset=50

What about defaults?

My loose rule of thumb for default pagination is limit=10 with offset=0.

limit=10&offset=0

The pagination defaults are of course dependent on your data size. If your resources are large, you probably want to limit it to fewer than 10; if resources are small, it can make sense to choose a larger limit.

In summary:

Support partial response by adding optional fields in a comma delimited list

Use limit and offset to make it easy for developers to paginate objects.


Next time: dealing with multiple formats.

RESTful API Design: tips for versioning

In the last post in this series about Pragmatic REST API design, I talked about designing error conditions in your APIs. Check out the full series.

This time - versioning - one of the most important considerations when designing your pragmatic API.

Never release an API without a version and make the version mandatory.

Let's see how three top API providers handle versioning.

 

Twilio uses a timestamp in the URL (note the European format).

At compilation time, the developer includes the timestamp of the application when the code was compiled. That timestamp goes in all the HTTP requests.

When the request comes into Twilio, they do a look up. Based on the timestamp they identify the API that was valid when this code was created and route accordingly.

It's a very clever and interesting approach, although I think it is a bit complex. It can be confusing to understand whether the timestamp is compilation time or the timestamp when the API was released, for example.

Salesforce.com uses v20.0, placed somewhere in the middle of the URL.

I like the use of the v. notation. However, I don't like using the .0 because it implies that the interface might be changing more frequently than it should. The logic behind an interface can change rapidly but the interface itself shouldn't change frequently.

Facebook also uses the v. notation but makes the version an optional parameter.

This is problematic because as soon as Facebook forced the API up to the next version, all the apps that didn't include the version number broke and had to be pulled back and version number added.

How to think about version numbers in a pragmatic way with REST?

Specify the version with a 'v' prefix. Move it all the way to the left in the URL so that it has the highest scope (e.g. /v1/dogs).

Use a simple ordinal number - v1, v2, and so on. Don't use the dot notation like v1.2 because it implies a granularity of versioning that doesn't work well with APIs--it's an interface not an implementation.

Make the version mandatory.

How many versions to maintain?

Next time -Giving developers just the information they need -  pagination and partial response.

Please join the API Craft conversation on Google groups.

RESTful API Design: what about errors?

In the previous posts in this series about Pragmatic REST API design, I talked about simplyfing associations, using the HTTP ? to hide complexities and optional parameters, choosing plural nouns and concrete names, and more. Check out the full series.

What about errors in the context of RESTful API best practices? Many software developers, including myself, don't always like to think about exceptions and error handling but it is a very important piece of the puzzle for any software developer, and especially for API designers.  

Why is good error design especially important for API designers?

Bottom line, it's about making your APIs intuitive and making developers successful.

First, developers learn to write code through errors.  The "test-first" concepts of the extreme programming model and the more recent "test driven development" models represent a body of best practices that have evolved because this is such an important and natural way for developers to work.

From the perspective of the developer consuming your Web API, everything at the other side of that interface is a black box. Errors therefore become a key tool providing context and visibility into how to use an API.

Secondly, in addition to when they're developing their applications, developers depend on well-designed errors at the critical times when they are troubleshooting and resolving issues after the applications they've built using your APIs are in the hands of their users.

How to think about errors in a pragmatic way with REST?

Let's take a look at how three top APIs approach it.

Facebook
No matter what happens on a Facebook request, you get back the 200 status code - everything is OK. Many error messages also push down into the HTTP response. Here they also throw an #803 error but with no information about what #803 is or how to react to it.

Twilio
Twilio does a great job aligning errors with HTTP status codes. Like Facebook, they provide a more granular error message but with a link that takes you to the documentation. Community commenting  and discussion on the documentation helps to build a body of information and adds context for developers experiencing these errors.

SimpleGeo
Provides error codes but with no additional value in the payload. 

A couple of best practices

Use HTTP status codes
Use HTTP status codes and try to map them cleanly to relevant standard-based codes.

There are over 70 HTTP status codes. However, most developers don't have all 70 memorized. So if you choose status codes that are not very common you will force application developers away from building their apps and over to wikipedia to figure out what you're trying to tell them. 

Therefore, most API providers use a small subset. For example, the Google GData API uses only 10 status codes, Netflix uses 9, and Digg, only 8.

How many status codes should you use for your API?

When you boil it down, there are really only 3 outcomes in the interaction between an app and an API:

  • Everything worked
  • The application did something wrong
  • The API did something wrong

Start by using the following 3 codes. If you need more, add them. But you shouldn't go beyond 8.

  • 200 - OK
  • 404 - Not Found
  • 500 - Internal Server Error

If you're not comfortable reducing all your error conditions to these 3, try picking among these additional 5:

  • 201 - Created
  • 304 - Not Modified
  • 400 - Bad Request
  • 401 - Unauthorized
  • 403 - Forbidden

(Check out this good Wikipedia entry for all HTTP Status codes.)

It is important that the code that is returned can be consumed and acted upon by the application's business logic - for example, in an if-then-else, or a case statement. 

Make messages returned in the payload as verbose as possible

Be verbose.

Use plain language descriptions.

Add as many hints as your API team can think of about what's causing an error.

I highly recommend you add a link in your description to more information, like Twilio does.

 

Next time - versioning your API.

Please join the API Craft conversation on Google groups.

 

RESTful API Design: simplify associations, sweep complexities under the HTTP ?

In the previous post in this series about Pragmatic REST API design, I talked about choosing plural versus singular nouns and concrete names over abstract. See RESTful API Design: plural nouns and concrete names.

This time we'll explore API design considerations when handling associations between resources and parameters like states and attributes.

Associations
Resources almost always have relationship to other resources. What's a simple way to express these relationships in a RESTful world?

Let's look again at the API we modeled in nouns are good, verbs are bad - the API that interacts with our dogs resource. Remember, we had two base URLs:

/dogs

/dogs/1234

We're using HTTP verbs to operate on the resources and collections. Our dogs belong to owners. To get all the dogs belonging to a specific owner, or to create a new dog for that owner, do a GET or a POST:

GET owners/5678/dogs

POST owners/5678/dogs

Now, the relationships can be complex. Owners have relationships with vetinerians, who have relationships with dogs, who have relationships with food, and so on. It's not uncommon to see people string these together making a URL 5 or 6 levels deep. Remember that once you have the primary key for one level, you don't need to include the levels above that because you've already got your specific object. In other words, you shouldn't need too many cases where a URL is deeper than what we have above - resource name/identifier/resource name.

Sweep complexity behind the HTTP ?
Most APIs have intricacies beyond the base level of a resource. Complexities can include many states that can be updated, changed, queried, as well as the attributes associated with a resource.

Make it simple for developers to use the base URL by putting optional states and attributes behind the HTTP question mark. To get all red dogs running in the park:

GET /dogs?color=red&state=running&location=park

In summary, keep your API intuitive by simplifying the associations between resources, and sweeping parameters and other complexities under the rug of the HTTP question mark.

Next time, what about errors?

RESTful API Design: plural nouns and concrete names

In the first post in this series, Are you a Pragmatist or a RESTafarian?, I proposed that "pragmatic REST" is a design issue.

In the second post, RESTful API Design: nouns are good, verbs are bad, I outlined some of  the API design practices that work well: Nouns in URLs are good. Verbs are bad. Try to limit your API to 2 base URLs per resource.

This time, we'll explore how to pick the nouns for your URLs.

Plural nouns are better than singular
Should you choose singular or plural nouns? You'll see popular APIs use both. Let's look at a few examples  - key resources for these businesses are expressed as either singular or plural:

Foursquare
/checkins

GroupOn
/deals

Zappos
/Product

Given that the first thing most people probably do with a RESTful API is a GET, I think it reads more easily and is more intuitive to use plural nouns. But above all, avoid a mixed model in which you use singular for some resources, plural for others. Being consistent allows developers to predict and guess the method calls as they learn to work with your API.

Concrete names are better than abstract
Achieving pure abstraction is sometimes a goal of API architects. However, that abstraction is not always meaningful for developers.

Take for example an API that accesses content in various forms  - blogs, videos, news articles, and so on.

An API that models everything at the highest level of abstraction -  as /items or /assets in our example - loses the opportunity to paint a tangible picture for a developer to know what they can do with this API. It is more compelling and useful to see the resources listed as blogs, videos, and news articles.

The level of abstraction depends on your scenario. You also want to expose a manageable number of resources. Aim for concrete naming and to keep the number of resources between 12 and 24.

In summary, an intuitive API uses plural nouns and concrete names rather than abstract.

Next time, simplifying associations and hiding complexities using the HTTP ?.

RESTful API Design: nouns are good, verbs are bad

In the first post in this series about pragmatic REST API design, I defined 'pragmatic REST' as looking at API design from the developer point of view. Now let's get into specific design practices we've seen work well.

The #1 principle in pragmatic RESTful design is: keep simple things simple.

Keep your base URL simple and intuitive

The base URL is the most important design affordance of your API.  A simple and intuitive base URL design will make using your API easy.

A design affordance is a design element that communicates how something should be used without requiring documentation.  A door handle's design should communicate whether you pull or push. But here's an example of a conflict between design affordance and documentation - not an intuitive interface!

 

My key litmus test for simple API design and pragmatic REST is:  only 2 base URLs per resource

Let's model an API around a simple object or resource (dogs) and create a RESTful API that interacts with dogs.

The first base URL is for collections; the second is for a specific element in the collection.

 

 

 


Boiling it down to this level will also force the verbs out of your base URLs.   

Keep verbs out of your base URLs

Many RESTful APIs start by using a method-driven approach to URL design. These method-based URLs sometimes contain verbs -  sometimes at the beginning, sometimes at the end.

For any resource that you model, like our dog, you can never consider one object in isolation. Rather, there are always related and interacting resources to account for - like owners, veterinarians, leashes, food, squirrels, and so on.

Think about the method calls required to address all the objects in the dogs world. The URLs for our resource might end up looking something like this:



It's a slippery slope - soon you have a huge list of URLs and no consistent pattern making it difficult for developers to learn how to use your APIs.
 
Use HTTP verbs to operate on the collections and elements

For our dog resources, we have two base URLs that use nouns as labels, and we can operate on them with HTTP verbs. Our HTTP verbs are POST, GET, PUT, and DELETE. (I think of them as mapping to the old metaphor of CRUD (Create-Read-Update-Delete).)  

With our two resources (/dogs and /dogs/1234) and the four HTTP verbs, we have a rich set of capability that's intuitive to the developer. Here is a chart that shows what I mean for our dogs.



The point is that a developer probably doesn't need the chart to grok how the API behaves. They can experiment with and learn the API without having to dig into the documentation.  

In summary: Use 2 base URLs per resource.  In your URLs - nouns are good; verbs are bad.

Next time:  how do you pick your nouns?

Please join the API Craft conversation on Google groups.

API Design: Are you a Pragmatist or a RESTafarian?

I wanted to write some posts on RESTful API design best practices we've observed over the last year.   I covered a lot of these recently in a API design presentation but want to write more detail out.

First - let's start with our overall point of view on API design.

We advocate pragmatic, not dogmatic REST. What do I mean by dogmatic?   

You might have seen discussion threads on true REST - some of them can get pretty strict and wonky. 

 @mikeschinkel sums it up well - defining the RESTafarian

Our view: approach API design from the 'outside-in' perspective.

This means we start by asking - what are we trying to achieve with an API?

The API's job is to make the developer as successful as possible.   

Why? Look at the value chain below - the developer is the lynchpin of our entire API strategy.

So design the API to maximize developer productivity and success as the primary design principle.  

This is what we call pragmatic REST.

Pragmatic REST is a design problem.

You have to get the design right, because design communicates how it will be used.

So now the question is  - what is the design with optimal benefit for the app developer?

The design asethetic for APIs is to think about design choices from the developer point of view.

This is pretty much the guiding principle for all the specific tips that we'll dig into next.

Next time:   considerations in designing your base URLs.