I'm designing a (more or less) RESTful internal web service running on ASP.NET and IIS. I want clients to be able to pass query details to the server when accessing large collections of entries, using JSON to describe the query in a known manner. The issue is that the queries sent to the server will be complex; they may include aggregation, filtering, mapping—essentially anything that is supported by the LINQ query operators. This will result in relatively large JSON objects representing the queries.
The conflict I'm facing is that, while a query is semantically a GET in the world of REST, there's no standardized way to pass a large block of data to a web server during a GET. I've come up with a few options to get around this issue.
Option 1: Send the query object in the body of the GET request.
GET /namespace/collection/ HTTP/1.1
Content-Length: 22
{ /* query object */ }
Obviously, this is non-standard, and some software may choke on a GET request that has a body. (Or worse, simply strip the body and handle the request without it, which would cause the server to return an incorrect result set.)
Option 2: Use a non-standard HTTP verb (perhaps QUERY) instead of GET.
QUERY /namespace/collection/ HTTP/1.1
Content-Length: 22
{ /* query object */ }
While this doesn't fit exactly with the REST pattern, it seems (to me) like a safe alternative because other software (such as anything that uses WebDAV) seems to use non-standard HTTP verbs with sufficient success.
Option 3: Put the query object in a non-standard HTTP header.
GET /namespace/collection/ HTTP/1.1
ProjectName-Query: { /* query object */ }
This option keeps the request as a GET, but requires stuffing what could potentially be a very large object in an HTTP header. I understand some software places arbitrary length limits on HTTP headers, so this may cause issues if the object gets too big.
Option 4: Use the POST verb and provide an alternate endpoint for querying.
POST /namespace/collection/query HTTP/1.1
Content-Length: 22
{ /* query object */ }
Because this uses a standard verb and no standard headers, this method is guaranteed to work in all scenarios. The only issue is that it strays from RESTful architecture, which I'm trying to stay aligned with as best I can.
None of these options are quite right. What I want to know is which way makes the most sense for the service I'm writing; it's an internal web service (it will never exposed to the public) but it may be accessed through a variety of network security applications (firewalls, content filters, etc..) and I want to stick to known development styles, standards, and architecture as best I can.
I would use Option 4. It is difficult to put the query representation in json for a large search request into an url, especially against a search server. I agree, in that case it does not fit into a Restful style since the resources cannot be identified by the URI. REST is a guideline. If the scenario cannot be realized by REST then i guess do something that solves the problem. Here using POST is not restful but it seems to be the correct solution.
I would think about "RESTful querying" as having two resources: Query and QueryResult.
You POST your Query to one end-point (e.g. "POST /queries/") and receive a CREATED Status back with the URI of your specific query (/queries/123) and a nice and RESTful hypertext body telling you the URL of your query result (e.g. /result/123 ). Then you access your query result with a GET /result/123. (Bonus points if you use hypertext to link back to /queries/123 so that the consumer of the query result can check and modify the query.
To elaborate the point I'm trying to make:
If RESTful is basically reduced to "map business entities to URIs" than the obvious question arises: "how can I query a subset of my entities"? Often the solution is "adding a query string to the 'all entities of this type'-URL" - Why else would it be called "query string"?. But it starts to feel "wrong" - as stated in the OP - if you want to have a full fledged query interface.
The reason is that with this requirement the Query becomes a full business object itself and is no longer an addendum to an resource address. It's no longer secondary but primary. It becomes important enough to become a resource in its own right - with it's own address (e.g. URL) and representation.
I'm not sure how much it would look "canonical" to you, but you could have a serious look at OData (open data protocol):
OData is a standardized protocol for creating and consuming data APIs.
OData builds on core protocols like HTTP and commonly accepted
methodologies like REST. The result is a uniform way to expose
full-featured data APIs.
Even if you don't implement it as is, there are ideas that could be reused.
Specifically, OData defines batch processing. It's used for executing multiple operations sent in a single HTTP request. So, with OData, you have two choices:
- use the GET + query string operation for queries that are not too long
- use a POST + multipart body operation for bigger things.
More on maximum uri length in an OData context: OData Url Length Limitations
Also, many security devices (routers, firewall, etc.) will simply not let your option 1, 2 and 3 go through. GET + Body is unusual, GET + a big form value may get killed, and a custom HTTP verb is also very unusual.
Overall, I think the POST + body seems the best choice (whether it's strictly multipart - like in OData - or not is up to you)
After thinking more about this, I am going to give another answer.
What do you mean, in estimated number of characters, when you state the JSON representations will be "relatively large"? IE can handle URLs over 2,000 characters. Will the queries ever get bigger than that? Because I think the querystring is the way to go. Right now I am working on a system that uses JSONP so we have no other option than to pass all data as a JSON package in the querystring and it works fine. Not only will using the GET verb be semantically correct, this will also include the feature of being able to bookmark URLs to the results. The users could easily share links to the data results through email or other electronic communication systems you use internally.
I'm not sure if this helps but even for all Quickbooks APIs, queries which return large resultsets like Read All, or a LINQ extender query which returns large JSON resultsets, we use GET with the relevant content type and encoding like ASCII. The request uses compressionFormat as None and response uses a GZIP compressionFormat.
https://developer.intuit.com/apiexplorer?apiName=V3QBO
The best way would be to serialize the search JSON object and pass it as a query parameter. Are you sure it will be too long for modern browsers and servers? Modern browsers and servers can handle pretty hefty GET query parameter lengths, thousands of characters.
Perhaps an extension header like X-Custom-Query-Parameters-JSON if objects are going to be more on the order of 8k characters.
How many characters would a serialized JSON object be in your particular case?
Some related questions about character limits:
What is the limit on QueryString / GET / URL parameters
Is there a practical HTTP Header length limit?
An interesting problem. I don't have the specifics on what you are trying to do, but I wonder if it is too much to gracefully handle with one resource. You may want to break it up into several different types depending on the main characteristics of the request. If you are just trying to expose what should be a SQL query through an HTTP request, then I don't think there is any way it can can be implemented without a mess. Just pass the SQL query in the query string and stop trying to find a proper way to do it - it doesn't exist.
Use POST, and pass the queries/parameters as key-value pairs in the body as json. It also becomes easier in your asp.net code to translate the payload into a dictionary object.
Dictionary<string,object>
