Well-behaving RESTful Client Interactions

Question

I have what seems to be a fairly simple question about implementing a data access client that strictly adheres to REST architectural principles. To start, let's assume I have a well-behaving REST API that I want to consume using a Django application. I'll start by discovering what services are available (edited for follow-up):

GET example.com/services/ HTTP/1.1

HTTP/1.1 200 OK
<?xml version="1.0" encoding="UTF-8"?>
<services>
  <service>
    <name>Widgets</name>
    <link>http://example.com/services/widgets/</link>
    <item_link>http://example.com/services/widgets/{widget_id}/</item_link>
  </service>
  <service>
    <name>Factories</name>
    <link>http://example.com/services/factories/</link>
    <item_link>http://example.com/services/factories/{factory_id}/</item_link>
  </service>
  ...
</services>

Now, since I'm building a Django application based around consuming this API, how would I continue to keep exploring these services RESTfully? To adhere to REST principles, my application must be driven by the hypermedia received. I suppose the first step is easy enough -- interacting with a service by the name given. I set up a Django view as follows:

def get_service(request, service_name):
    doc = etree.parse(urllib.urlopen('http://example.com/services/'))
    uri = doc.xpath("service/name[.='%s']/following-sibling::*" % service_name)[0].text
    ...

From which I'll perform another request (edited for follow-up):

GET example.com/services/widgets/ HTTP/1.1

HTTP/1.1 200 OK
<?xml version="1.0" encoding="UTF-8"?>
<widgets>
  <item_link>http://example.com/services/widgets/{widget_id}/</item_link>
  <widget>
    <id>1</id>
    <name>Whizbang Foobar</name>
    <link>http://example.com/services/widgets/1</link>
  </widget>
  ...
</widgets>

Now I'll display a simple list of widgets in a rendered Django template. From here though, how do I continue to interact with this service RESTfully? Perhaps I've befuddled myself into confusion, but the only reasonable things I can come up with are implementing a numerous amount of application views or a thin Django data model to persist the service URI.

My main concern boils down to that this is trivial to do without strictly adhering to REST architectural guidelines, but I feel like I've missed the boat completely in trying to do so. I understand designing proper REST APIs and clients isn't "easy", but it seems that I'm in dire need of a similar example to work through the actual implementation.

I apologize for the length and verbosity of the question and the inevitable facepalming of wizened readers.

Follow-up:

Is the following a valid way (using URI templates) of implementing these interactions? For demonstration purposes (in lieu of a more abstract implementation), another Django view to retrieve a resource collection item:

def get_item(request, service_name, item_id):
    doc = etree.parse(urllib.urlopen('http://example.com/services/'))
    uri = doc.xpath("service/name[.='%s']/following-sibling::item_link" % service_name)[0].text
    ...

Then the subsequent request:

GET example.com/services/widgets/1 HTTP/1.1

HTTP/1.1 200 OK
<?xml version="1.0" encoding="UTF-8"?>
<widget>
  <id>1</id>
  <name>Whizbang Foobar</name>
  <tags>foo bar baz ham eggs</tags>
  <index_link>http://example.com/services/widgets/</index_link>
</widget>

Answer 1

If I understand your question correctly, you want to explore an unknown service, correct?

If so, then you could, for example continue with an OPTIONS request to the "widget" resource, to see which HTTP methods it supports (these should be listed in the Allow header of the response).
You can then do the same for all URIs found in <link rel="whatever"> elements. If a resource found this way indicates that it supports GET, then fetch it and repeat ...
This way you should be able to explore all nested resources.

This kind of exploration will of course only get you so far, because to really interact with the service, you will need to know about its media types (or representations), and what the different <link rel="whatever"> actions you found actually mean. This step can't be automated, you'll have to read the documentation for the service and build you client accordingly. I suggest reading the article "How to GET a Cup of Coffee", which I think explains this interaction very well.

Answer 2

From my experience the REST model makes much more sense if the representations and their contained links are translated directly to a client UI. In this scenario it is the user that directs the exploring of the REST interface.

Often I see people trying to use REST interfaces as a kind of HTTP based data access layer. With this mental model, hyperlinked data provides little more than structural data relationships. It becomes difficult to build application behaviour on top of that interface without violating RESTful constraints.

I like to think of a RESTful interface as delivering UI content to an application that is going to render that content with some arbitrary technology. Unfortunately, REST is frequently compared to web services which, in my opinion, fit in a different architectural layer. Web services deliver data that is to be processed by a client application. RESTful interfaces should deliver content that will be rendered to the user.

Sure you can use a REST interface to deliver data to remote applications but think of it as simplified screen scraping.

When writing a client for REST interface I find it useful to imagine that I am writing a custom web browser that only understands the media-types that are delivered by that RESTful interface and is hard-coded to start at a specific URL and there is no address bar!

Answer 3

Sun Cloud API documentation is a great example of a RESTful API, focusing on the media types.

Answer 4

My main concern boils down to that this is trivial to do without strictly adhering to REST architectural guidelines, but I feel like I've missed the boat completely in trying to do so. I understand designing proper REST APIs and clients isn't "easy", but it seems that I'm in dire need of a similar example to work through the actual implementation.

The best example I've been able to find is the Sun Cloud API. Most of the documentation describes the various media types used by the system, which seems to be the key to pulling this kind of thing off.

I find that it helps to be writing your client at the same time you're developing your API. That way you can spot what's likely to make your API a pain to code for right away and fix the problem.

It isn't easy. If you follow the HATEOAS constraint to its logical conclusion, each media type you define will be handled by one of a family of clients. To the extent that you can make all of your resources follow a similar pattern of behavior, your job of writing clients will become easier.

For example, you could define a media type 'Index' that simply lists related resources. Index defines links for pagination, getting items in the list, finding items by name, etc.

Then, you might define a base media type called 'Item'. Item has a link for displaying its parent Index, updating/deleting itself, etc. Your resource Widget could then be represented by two different media types - one Index and one based on Item.

You could begin by implementing a single class that handles the Index media type. Then you could write a base class that handles all common Item media type behavior. Finally, you could write a Widget client that handles all widget-specific behavior and which extends the Item client. These clients could expose their functionality (availability of more links and data fields) in an idiomatic way for the language they're written in.

Processing a response from your server would then be a matter of matching the mime type of the response to one of the clients you've written.

In other words, even though the client for your service as a whole would be made up of many clients of limited scope, they would each be based on common behaviors and so could be implemented in a DRY way.