Limiting number of requests to a servlet

问题:

We have a servlet which occupies more virtual memory on the server due to the logic it has. For this reason, we would like to limit the concurrent requests to this server say for example we would only want 10 concurrent requests processed. The other requests have to wait in the queue.

Can a custom thread pool be created and assign for this servlet to handle this scenario? We are using WebLogic server 9.2. Or is there any other better approach to do this? Appreciate any thoughts.

回答1:

Can a custom thread pool be created and assign for this servlet to handle this scenario? We are using WebLogic server 9.2. Or is there any other better approach to do this? Appreciate any thoughts.

Yes, this is possible. Instead of using the default self-tuning work manager (starting with Weblogic 9.x, execute queues are replaced by work managers for thread pools¹), you could create a work manager with specific constraints like the max-threads-constraint and possibly the capacity. You can then assign a Servlet to a specific work manager using the wl-dispatch-policy of the weblogic.xml deployment descriptor file.

---

¹ Note that it's still possible to enable WebLogic 8.1 Thread Pool Model and to use Execute Queues.

回答2:

You need something in front or the machine hosting the servlet because when the requests hit the machine, it is somewhat too late: resources are already being used. You can't control the demand side: you can only react to it and plan for it.

You need probably a load-balancer either software or hardware depending on your target requirements. The software load-balancer can be simply a "dispatcher servlet" with session control (e.g. 10 concurrent to servlet X).

There is another possibility: you "throttle" the requesters by issuing an appropriate HTTP code. Of course, this means additional logic on the requester side... and it still consumes some resources on the server side.

回答3:

You could load-balance such that there's a secondary server that processes all requests for the expensive servlet.

回答4:

You could have a static counter, and a servlet that just acts as a gateway to the expensive method call. You just need to deal with a probably race condition on this static counter.

So, you would turn your current servlet into a method call.

Then, the gateway servlet will get the request, see if the counter is low enough and then increment. If over 10, then return some error message.

This is not an ideal situation but if you put things into a queue then browsers will begin to timeout after a while, or users get impatient and click the submit button over and over, as it is taking too long.

If you could use javascript to send the request then there are some better solutions that may help you.

回答5:

Without using load-balancers etc., it seems to me that you want to decouple the request from the processing.

e.g.

1. the browser sends a request. The servlet takes it, queues it and hands back a ticket.
2. The servlet will work on this work request as resources permit (using a separate thread pool pulling the work items from the queue).
3. The browser can refresh (re-GET) using that ticket, and the servlet will return an appropriate result (e.g. not processing, processing, processed).

This is quite a common pattern. Note that the browser isn't blocked, but merely dispatches the request and then regularly performs checks to see if the work item is complete. I've used this successfully (for example) in the situation where I've had users asking for charts that take 5 mins or more to process, and that used a native library that wasn't thread-safe. In that scenario I had to restrict the processing to a single thread regardless of the number of simultaneous requests.

回答6:

I like the idea of using static counter and redirecting to show an error message when the counter has reached beyond a limit.

Could we configure a separate servlet and configure the thread pool to only allow X number of concurrent requests, all other requests would be placed in queue to use the next available servlet. Does this approach throw a timeout error? Can you please share more details around this? Thanks

http://download.oracle.com/docs/cd/E13222_01/wls/docs92/perform/appb_queues.html