11436 SSO

API Design: Binary Data, Caching, Transactions, and More

May 30, 2013

After our recent discussion about hypermedia types, let’s delve deeper into some of the unforeseen elements of design, such as how to support binary data, caching, JavaScript APIs, data posting, and transactions.

Let’s begin by looking at how we accept binary data. This comes up a lot on the API-Craft Google Group because there’s no single way to do it.

How can we accept binary data?

One method to use is multipart/form-data. This is exactly what your browser does when you are filling out a form. The Content-Disposition includes the field name, with a value, and the option of assigning different Content-Types with each bit sent.

This example shows a jpeg image being sent along.


Content-Type: multipart/form-data; boundary=AaB03x 
Content-Disposition: form-data; name=“caption” 
Cool picture of my cat. 
Content-Disposition: form-data; name=“photo”; filename=“catpajamas.jpg” 
Content-Type: image/jpeg 
Content-Transfer-Encoding: binary 
…contents of catpajamas.jpg… 

The benefits of using multipart/form-data include, using only one HTTP call, employing the binary format, which is more compact than base64, and the ability to use HTTP handling tools.

Another option is to inline a Base64 encoded version of the binary data.

Inline Base64 Encoding

POST /photos
“caption”: “Cool picture of my cat.”
“photo”: “RHVkZSwgbXkgY2F0IGhhcyB0aGUgYmVzdCBwYWphbWFzLg==”

The benefits of inline Base64 encoding are that it’s quick to implement and it’s good for small files. The downside is the increased size.

The last option is a two-step process, which is good for large binary files.

Two-Step Process

POST /photos 
{ “caption”:
 “Cool picture of my cat.” 
PUT /photos/1234/data 
Content-Type: image/jpeg 
Content-Length: 240 
Content-Transfer-Encoding: binary 
…binary content… 

However you decide to submit binary data in your API, recognize the trade-offs. With that said, opt for multipart/form data.

How do we manage API design when caching?

One method is to use an expiration with a private Cache-Control header, which means only the clients can cache this, not a proxy in the middle, and a max-age in seconds. For example, 30 days (2592000).


200 OK 
Cache-Control: private, max-age=2592000 

Another method is ETags. ETags are sometimes calculated as a checksum of a file or other data elements. Some databases add a revision number on each document but you can also manage your own keys for ETags. If your resource has an ETag associated with it and you want to make sure you have the latest, do a GET with the url, include the header If-None-Match, and the ETag you received to get the latest response.


GET /dogs/1 
ETag: “a7D92kda94aisdfG” 

GET /dogs/1 
If-None-Match: “a7D92kda94aisdfG” 

A third option is using the Last-Modified header. Preceding the Last-Modified header, use the If-Modified-Since header to return newer modifications and not the original full body of a message.


GET /dogs/1 
Last-Modified: Thu, 10 Jan 2013 19:43:31 GMT 

GET /dogs/1 
If-Modified-Since: Thu, 10 Jan 2013 19:43:31 GMT 

There’s no standout method for caching, it all depends on use cases. Think from the client’s perspective, instead of just saving resources and bandwidth on the server, and help them save valuable time if they don’t need to access the server.

Do we need a JavaScript API?

Often times there’s pressure on the API to give usable hints about UI elements. This is always a mistake. You want a clean separation between the API and the consumer. If you want adoption and the ability to explain your API, it’s wise to take the time to create a JavaScript API library or SDK.

What about posting data?

Besides understanding actions and state changes, it’s also important to discuss how to serialize data. What form does your data take when it’s sent to the server?

One option is to use application/x-www-form-ulrencoded, which looks just like a query string.



Another option is to take XML (application/xml) as the request body but it tends toward verbosity.



Finally, there’s JSON (application/json).


 “breed”: “Dachshund”,
 “name”: “Hotdog”,
 “age”: 2 

We favor the application/x-www-form-ulrencoded version because it’s simple and easy.

How do we handle transactions?

One way you could do this is to model the transaction directly in your API. For this example, we can use the ideas of a shopping cart.

First, create a shopping cart with POST to get a 201 created back.

Create a Transaction

POST /carts 
201 Created 
Location: /carts/1 

Then, from the product catalog, add some items to the cart.

Add Items

POST /carts/1/items/ 
{ “productId”: “mittens123”, “quantity”: 1 } 
… 201 Created 
Location: /items/1234 

And, check out. 

Commit the Transaction

POST /carts/1 
{ “message”: “checkout” } 
200 OK 

So create a transaction, add items to it, and commit. We’ll discuss this further, including errors that may occur another time.

To join the conversation around APIs, check out the API-Craft Google Group.

Next: Ruminating over REST

Scaling Microservices