Processing same concurrent request in REST

Question

Greetings SO community! I have a REST, jersey based, application. This application(due to nature of it's clients) receives same http requests (3-6 of them) at roughly the same time (~2-5s apart). Each request takes around 10s to process and brings back a ton of data (hitting DB, doing data massaging etc). In an ideal world I'd like to avoid having to process the same request multiple times and was thinking of coding some sort of request filter that would only allow unique requests go through, whist others would be blocked until such time when the allowed one returns. Blocked requests would also then return the same data to the caller (by looking up cached response on the server)

What are the pros/cons of this approach? Are there any better solutions to this other than changing client logic ;)

Answer 1

You could create a unique object to lock on for each "key." where the key is some request parameters, in this example a String. That way you just hold onto the request (because of synchronization) and once the computation is done, both clients get the results back at about the same time. This way the clients don't have to make multiple requests, instead the clients who are not first have to wait for the first client to prime the cache.

import java.util.HashMap;
import java.util.concurrent.ConcurrentHashMap;

public class Hold {
    private ConcurrentHashMap<String, String> holds =
            new ConcurrentHashMap<>(new HashMap<String, String>());

    // compose a "hash" for lack of better word, could append a timeout as well
    public String hashString (String string) {
        return string + "wackystuff";
    }

    public String doLongComputation () {
        // do crazy computation here
        return new String();
    }


    public synchronized String getResults (String key) {
        // give us a unique object for this key to lock on
        holds.putIfAbsent(key, hashString(key));

        // lock on that key
        synchronized (holds.get(key)) {
            // we have a non lock value so return it
            if (!holds.get(key).equals(hashString(key))) {
                // could do some timeout here
                return holds.get(key);
            }
            // the cache is empty so do the long computation
            holds.put(key, doLongComputation());
        }
        return holds.get(key);
    }

}

This is just one kinda sly approach, the book Java Concurrency in Practice has much more robust approach, its in section 5.19 and the code sample can be found here.

Here are the pros and cons of this approach:

• Pro: the client has to make only one request
• Pro: you are not recalculating the results
• Pro: the wait time for no individual client is increased over the serial case
• Con: you are holding up a VM thread for the duration of the calculation and this is per client. Seeing as you have view clients this shouldn't be an issue.
• Con: Thinking of a good schedule to clear the cache might be tricky

Answer 2

During developing an integration with a number of REST services to avoid unnecessary calls to services we are creating a cash in a database of received responses (JSON strings).

For each such cash record we save params that were used to call a web-service. If we have a request we compare the params with existing ones in database. We make real requests to REST only for new params. Also it is necessary for us because some requests are not free of charge.

Also we have a param that shows a validation date of every cash record in hours (or days) so that after some time we automatically make a real request if it was being made a long time ago to receive free information.

This approach was created after several years of working with REST services and it works very good in our solution.