Dan Bovey

Modern HTTP Practices (Fuck the Envelope)

Modern HTTP Practices (Fuck the Envelope)

Posted on 8 November 2016

TL;DR: I made a pagination library for Laravel and Lumen that removes the envelope: Laravel LinkHeader Paginator

I put forth this vote to the developer community yesterday, which seems much more important than any other vote going on this week!

Don’t worry, I knew what I was setting myself up for. I’m going against the (small sample of) people above, and what’s recommended in a specification for creating JSON APIs which I was quickly linked to by a voter while writing this but there’s also a percentage of others who actually believe in using HTTP properly. #shotsfired

I think my favourite web blog/tutorial of all time is Vinay Sahni’s Best Practices for Designing a Pragmatic RESTful API. It’s a truly non-biased guide to designing an API and it suggests not using the envelope you’re all so clearly attached to!

Here’s the important bits:

Many APIs wrap their responses in envelopes like this:

{
  "data" : {
    "id" : 123,
    "name" : "John"
  }
}

There are a couple of justifications for doing this - it makes it easy to include additional metadata or pagination information, some REST clients don’t allow easy access to HTTP headers & JSONP requests have no access to HTTP headers. However, with standards that are being rapidly adopted like CORS and the Link header from RFC 5988, enveloping is starting to become unnecessary.

We can future proof the API by staying envelope free by default and enveloping only in exceptional cases.

Just remember when creating an API, HTTP is your envelope. When calling an endpoint such as GET /users/{id} you’d expect the returned resource to be a user, so don’t pollute the resource with unnecessary extras. Expecting the client to be dumb, not being able to access headers or status codes, is like expecting the user to be on IE6 and therefore not using HTML5.

One top-level member often included is an errors object. In the HTTP RFC, most of the HTTP Error codes advocate that you do return a description of why the error occurred but it doesn’t seem very logical to return an empty error object along with every successful response, or be such a pessimist!

If you take a bit of time out for a design process, assigning a little bit of semantics to the URI and HTTP methods adds a lot of simplicity to the overall architecture without requiring an envelope format to exchange data.

So if you haven’t closed this page yet in disgust of my blasphemy against clear-cut specifications, then hello! Welcome open minded friend, to:

LinkHeader pagination

An API that uses the Link header can return a set of ready-made links so the API consumer doesn’t have to construct links themselves. This is especially important when pagination is cursor based. Here is an example of a Link header used properly, grabbed from GitHub’s documentation:

Link: <https://api.github.com/user/repos?page=3&per_page=100>; rel="next", <https://api.github.com/user/repos?page=50&per_page=100>; rel="last"

Currently, there seems to be a lot of implementations for parsing Link headers but not for creating them. One of the most popular Javascript HTTP clients superagent supports Link headers and exposes a res.links object in responses. It’s not a very well documented feature but if you make a request to an API that uses Link, res.links.next would give you the URI of the next page.

Now I could be biased, but I think Laravel and Lumen are becoming the most popular PHP frameworks. It has the easiest pagination system known to man but it creates JSON responses like this:

{
  "total": 50,
  "per_page": 15,
  "current_page": 1,
  "last_page": 4,
  "next_page_url": "http://laravel.app?page=2",
  "prev_page_url": null,
  "from": 1,
  "to": 15,
  "data": [
    {
      // Result Object
    },
    {
      // Result Object
    }
  ]
}

The only work I’ve found that seems to be ushering in modern HTTP practices is the Dingo API package. It wraps around a lot of Laravel functionality to make building an API simple…. buuuut they closed the issue and gave some mediocre code example of how to add Link pagination.

So, I took it upon myself to build a Laravel paginator that removes the envelope and puts pagination info in the HTTP Link header! I call it the Laravel LinkHeader Paginator. (so creative)

Create the pagination with the Eloquent/DB Builder and pass it to the LengthAwarePaginator.

$items = User::where('active', 1)->paginate(20);

$paginator = new LengthAwarePaginator($items);

return $paginator->toResponse();

Which gives you a response like:

HTTP/1.1 200 OK
Content-Type: application/json
...
Link: <http://api.local/users/?page=1>; rel="current", <http://api.local/users/?page=2>; rel="last", <http://api.local/users/?page=2>; rel="next"

[
  {
    // Result Object
  },
  {
    // Result Object
  }
]

So to close up:

  • Don’t use an envelope
  • HTTP does a pretty good job if you make common sense endpoints and make full use of headers and status codes
  • Go use that thing I made if you happen to have the exact set up as me - a JSON API in Laravel or Lumen
  • Have a look at how you can parse Link in Javascript generally
  • Or use an XHR client that supports it
  • Don’t use an envelope