Jetty HttpClient starts up threads when started, and seems to generate a fair number of them. I was a bit surprised by this, as I thought one of the virtues of nio-based networking was multiplexing conversations over threads. It'd be easy to take control over the number of threads using HttpClient.setExecutor(...), but I'm not sure how to think about how many threads should be required. Does the library need one thread per simultaneous HTTP request, as HttpURLConnection might? Does it multiplex to some degree?
I'm defining an API client that will be long-lived within my application, and am trying to balance keeping a small footprint with enabling good concurrent performance.
Many thanks for any insight.
Jetty's HttpClient thread pool, by default, is Jetty's QueuedThreadPool and therefore it does start a few threads.
These threads are used to perform DNS lookups (that are blocking in Java, and with no possibility to make them non-blocking) and to receive responses.
Requests may be sent by application threads and by pooled threads (the latter in case a request has been queued).
Jetty's HttpClient is fully non-blocking since it's based on the Jetty NIO library which in turn is based on JDK's NIO library.
As such it does not require a thread per request and it's able to multiplex many requests/responses in few threads.
Even if HttpClient is fully non-blocking when performing network I/O, there are things such as timeouts and DNS that do require additional threads, and that is the reason you see additional threads.
If you need to balance small footprint and good concurrency, you definitely have to try out and find your best spot by tuning the executor passed to HttpClient.
Another important factor to tune is the number of selectors, configured at the HttpClientTransportOverHTTP level, see http://www.eclipse.org/jetty/documentation/current/http-client-transport.html#_http_1_1_transport.
There is no universal answer to what is the right number of threads, since it depends on many factors. As I said, you have to try it out and tune the various parameters.
For example, if destination addresses are all well known (e.g. internal network), you can replace the default asynchronous SocketAddressResolver with:
httpClient.setSocketAddressResolver(new SocketAddressResolver.Sync());
and this will get rid of an extra dispatch that performs the DNS lookup.
The number of selectors can be kept at 1 for a number of sockets typically below 1-5 thousands, depending on the hardware.
The lower the number of threads available to HttpClient, the longer the latencies when the load increases.
QueuedThreadPool is able to spawn new threads as needed and terminate them when they're not needed anymore, behaving elastically (and therefore is typically left at default configuration), but you can try out different configurations depending on your cases.
