Tomcat 7 Async Processing

Question

I am wondering how Tomcat 7 implements async processing. I understand that the request thread returns immediately, allowing the request thread to immediately listen for a new request and respond to it.

How is the 'async' request handled? Is there a separate thread pool that handles async requests? I assume blocking IO is handled using something like java.nio.Selector for performance. What about threads that are blocking on CPU calculations?

Answer 1

You are mixing up different concepts. You must distinguish between:

1. Asynchronous request handing as per Servlet 3.0; an API that enables you to decouple an incoming servlet request from the web container thread pool. It does not create any threads on the fly. It's on the user of the interface to implement a proper multi-threaded solution. It does not relate to non-blocking IO.
2. Thread pool; provides a mechanism to obtain and manage threads in a web container. When it comes to asynchronous request handing you have 2 options. You can define your own ExecutorService and use it to further process the request or you can create a new Runnable and submit it to the obtained AsyncContext by calling AsyncContext.start(). In case of Tomcat the latter approach uses Tomcat's thread pool defined in server.xml.
3. Non-blocking IO (NIO); Although it is asynchronous as well it is a different story. It relates to non-blocking IO operations like disk or network IO. If you want to enable NIO for HTTP request processing, have a look at Tomcat's documentation.

---

The below example outlines how it can work. It uses only one thread for worker jobs. If you run it from 2 different browsers in parallel the output looks like this (I use a custom logger):

   DATE                         THREAD_ID  LEVEL      MESSAGE
2011-09-03 11:51:22.198 +0200      26        I:     >doGet: chrome
2011-09-03 11:51:22.204 +0200      26        I:     <doGet: chrome
2011-09-03 11:51:22.204 +0200      28        I:     >run: chrome
2011-09-03 11:51:27.908 +0200      29        I:     >doGet: firefox
2011-09-03 11:51:27.908 +0200      29        I:     <doGet: firefox
2011-09-03 11:51:32.227 +0200      28        I:     <run: chrome
2011-09-03 11:51:32.228 +0200      28        I:     >run: firefox
2011-09-03 11:51:42.244 +0200      28        I:     <run: firefox

You see that the doGet methods immediately finish, whereas the worker still runs. The 2 test requests: http://localhost:8080/pc/TestServlet?name=chrome and http://localhost:8080/pc/TestServlet?name=firefox.

Simple example Servlet

@WebServlet(asyncSupported = true, value = "/TestServlet", loadOnStartup = 1)
public class TestServlet extends HttpServlet {
    private static final Logger LOG = Logger.getLogger(TestServlet.class.getName());
    private static final long serialVersionUID = 1L;
    private static final int NUM_WORKER_THREADS = 1;

    private ExecutorService executor = null;

    @Override
    public void init() throws ServletException {
        this.executor = Executors.newFixedThreadPool(NUM_WORKER_THREADS);
    }

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

        final String name = request.getParameter("name");
        LOG.info(">doGet: " + name);

        AsyncContext ac = request.startAsync(); // obtain async context
        ac.setTimeout(0); // test only, no timeout

        /* Create a worker */
        Runnable worker = new TestWorker(name, ac);

        /* use your own executor service to execute a worker thread (TestWorker) */
        this.executorService.execute(worker);

        /* OR delegate to the container */
        // ac.start(worker);

        LOG.info("<doGet: " + name);
    }
}

...and the TestWorker

public class TestWorker implements Runnable {
    private static final Logger LOG = Logger.getLogger(TestWorker.class.getName());
    private final String name;
    private final AsyncContext context;
    private final Date queued;

    public TestWorker(String name, AsyncContext context) {
        this.name = name;
        this.context = context;
        this.queued = new Date(System.currentTimeMillis());
    }

    @Override
    public void run() {

        LOG.info(">run: " + name);

        /* do some work for 10 sec */
        for (int i = 0; i < 100; i++) {
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        }

        ServletResponse response = this.context.getResponse();
        response.setContentType("text/plain");

        try {
            PrintWriter out = response.getWriter();
            out.println("Name:\t\t" + this.name);
            out.println("Queued:\t\t" + this.queued);
            out.println("End:\t\t" + new Date(System.currentTimeMillis()));
            out.println("Thread:\t\t" + Thread.currentThread().getId());
            out.flush();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        this.context.complete();

        LOG.info("<run: " + name);
    }
}