Unbelievable how time flies by. Someone who doesn’t know me personally, could’ve probably come to the conclusion that this little blog series has already halted before it even began in the meantime. Let me assure you that is not the case. I’m still invested in this project. Though I need to admit that things are progressing much slower than I originally had planed or hoped for.
So what is the current state of our journey? And what did I learn so far?
I started by building the most simple Rails based API I could think of. At the moment it’s only capable of listing users and showing an individual user. It doesn’t have any Hypermedia capabilities what so ever. It’s uses the Rails-API for a slimmed down version of the Rails stack, Devise for implementing HTTP Basic Authentication and the faker gem to populate the development database with some test data.
The home document
Then I added a minimal home document that lists the servers resources and how they can be interacted with. I implemented the specification as a mini DSL in Ruby, which is probably over the top, but was fun anyway.
1 2 3 4 5 6 7 8 9 10 11 12
So what do you see here? This document lists both resources currently provided by the server app. They both got a unique name which in this case in an URN. One of them (urn:sna:rel:users) has a fixed URL (/users). The other one (urn:sna:rel:user) uses an URI template ala RFC6570. It also lists all parameters that are contained in the template explicitly, together with a link to the documentation of that parameter.
This homedocument will be served at the entry point of the API, which in the case of my sample app is ‘/api’.
1 2 3 4 5 6 7 8 9
If you query this URI you’ll get the following response
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
There’s obviously a lot more in there that we’ve specified in our DSL. That’s because our DSL assumes some defaults when they’re not configured. You can find the complete implementation of it here.
That’s when I stopped working on the server part for a while and started working on a ruby based client. I’ve pointed this out in the intro of this blog series, but it’s worth repeating. Understanding the client side as well as possible and being able to provide guidance, especially in times of transition from a more tranditional HTTP API to a more Hypermedia based approach, is one of my main motivations behind this series. That’s why I’m building the client in parallel to the actual server part.
As usual the Ruby ecosystem was incredibly helpful to get a good start for the client implementation. The commandline client I’ve been building uses Thor for the CLI, Typhoeus as the HTTP library and the uri_template gem for building up URIs from templates. The rest is more or less handrolled at the moment.
1 2 3 4 5
Some words upfront, before we take a look at the implementation. I didn’t intend to build a browser-like client for the server in this iteration. My goal at this stage is merely understanding how the integration of a home document and the constraint of not building URIs themselves impacts the client. And how we can solve it that the impact on the rest of the client code isn’t that high.
So let’s drill down into the current client design. Let’s start at the API the client code (in our case the thor CLI interacts with). It looks like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
What is maybe interesting here, is that I took a design approach somewhat different to what I see in the typical client libraries out there. The API class is merely a wrapper around various invokations to a RequestDispatcher instance. The RequestDispatcher is the only place in the code that holds the base URI of the API (or in other words the endpoint where we can find our home document). All other URIs are specified relative to the entry point URI.
The dispatcher is configured with a stack of middlewares. Anyone who has done some Rails/Rack development probably already knows what it does, it’s the same deal. We’re going to take a closer look at this later in the post, because I consider it an interesting realization I had during the development.
What we also see here is that the requests to the server are explicitly modeled. So what do they do?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
A request encapsulates the logic to build up a request from the home document. It’s got two methods that’re invoked by the dispatcher during request execution. The first one is prepare, in which the request class finds the target url for its relation. Remember the only thing the client is supposed to know about the URI scheme is the relation identifier. After obtaining the request it simply requests the target URI. The other request is probably a bit more interesting.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
It looks similar except three things. On the one hand it’s parametrized with a user_id. Which is also used as some kind of identfier in the method with that exact name. Even more interesting is the interaction with the home_document in the prepare method. This is the place where the target URI is constructed from the URI template contained in the home document.
As you might have guessed the home_document we’re dealing with here isn’t a deserialized JSON response, it’s a proxy wrapped around it. This class encapsulates the logic of building up the target URIs from the home document received from the server. Its implementation looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
With this little piece falling into place, we can finally take a look at how the dispatcher is implemented who orchestrates the interaction between the home document and the requests.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
The RequestDispatcher is actually pretty forward and (at least on the surface) highly inefficient. At runtime it holds the configured base URI of the API and the reference to the actual HTTP library that is being used. When a request is dispatched, it requests the home document from the server before performing the actual request. Yes, everytime a server request is made the home document is also requested. I can already hear my colleagues from the mobile team staring at me and telling me ‘Hangover’ like: “Yeah, that’s not going to happen!”.
But it’s not bad as it looks. That’s were the middleware stack comes into play.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
It certainly doesn’t look so polished like a Rails middleware stack, but it does the same. It keeps the nitty gritty details of authentication, defaulting accept headers and most importantly caching away from the actual client code (it’s still reachable from the requests though in case it’s needed). I’ll spare you the details of the actual middlewares. You can take a look at them here. We’re only going to take a look at one of them, but before we do, we take a slight detour to the server.
Back to the server app
One part that we as developers often overlook when talking about communication patterns between different involved parties is that certain technologies are optimized for certain ways of communication. In our case that means that the HTTP infrastructure provides all the means to deal with our chatty communication. We just need to use them. You don’t need to build a highly sophisticated chaching solution yourself, HTTP has already a scheme for this. So let’s add caching instructions to our server side.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
So what did we add here? For the controller that serves the home document we added support for caching via expiration and/or via validation (Conditional GET support). A full discussion about HTTP caching is naturally out of scope for this post. The short story is that every response send by the server will now contain some HTTP caching information that can be used to make the communication more efficient, either by the client itself or some HTTP intermediaries (like caches).
For instance the ‘expires_in’ in the HomeDocumentController tells clients that it’s safe to cache the response in their local private cache for an hour. It’s probably safe to say that the home document doesn’t change frequently (if we leave custom tailored home documents for authenticated users out for the moment). The ‘stale?’ method adds Conditional GET support to a controller.
Every response will now contain an ‘ETag’ header. This is typically some kind of checksum or calculated hash value that can be used to detect a change in the HTTP response body. When supplied with a request via the ‘If-Non-Match’ header, the server can now decide whether he really needs to send the response body back to the client or simply answers with a 304 response code (ala ‘Yo dawg, nothing new for you on my side’), effectively saving bandwidth.
Those two caching techiques together can become a pretty powerful tool to make the chatty nature of a Hypermedia API less burdensome while still retaining its benefits.
Back to the client
So how can we leverage caching on the client side? It would’ve been awesome if the HTTP client library provided this out of the box, but unfortunately most of the ones (including the one I’ve been using for the sample app so far) don’t implement some sort of caching support). That’s where the middleware in the client came handy.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
That piece of code should explain why there’s something like an identifier on a request instance. It’s used as a lookup key for caching. Plugging this middleware into the stack will check whether the related resource has been accessed previously. If so it sends the ‘ETag’ with the request. The server either answers with a 304 and no content (then the content from the local cache can be used) or it answers with a new payload and the local cache needs to be updated.
Well, Hypermedia wise we haven’t implemented much yet. There’s some things I learned while implementing the current state though. Most importantly that separating out an HTTP middleware stack from the other communication related code in the client can have a huge impact on the code itself. I really like how the code lays itself out at the moment.
It’s probably a bit overengineered, but I like the direction to which this is going. HTTP provides such a rich set of caching capabilities and also a lot of great debugging tools for the communication. If that is the case, why’re we still building local (even relational) caches that neglect the nature of HTTP?
The interesting question though is why (at least in the Ruby space) does none of the HTTP client libraries implement HTTP caching? Any thoughts on this?
One interesting idea (or so it seems) I would like to pursuit until my next post is whether we can use the Rack infrastructure especially Rack::Cache on the client side. A quick googling suggested that Faraday might give me the integration between Typhoeus and Rack::Cache I’m looking for. If all goes well I should be able to get rid of my naive middleware stack implementation. We’ll see …
See you next time around!