I'm trying to understand the point in specifying separate core and maximum pool sizes for Java 5's ThreadPoolExecutor. My understanding is that the number of threads is only increased once the queue is full, which seems a bit late (at least with larger queues).
Isn't it that I'm either happy to allocate a larger number of threads to the tasks, in which case I might just increase the core pool size; or I am not really willing to do so, in which case I should rather have a larger queue? What is a scenario where separate core and maximum pool sizes are useful?
There is a discussion of this here.
The pool is intended to work under normal load at corePoolSize (to
which it ramps up unless pre-start is used). When an overload
condition occurs (defined by there being more pending/in-process
tasks than workers) we use the queue to act as a buffer - with the
expectation that the normal workload will be restored in the near
future. If we're worried about excessive overload then we can use a
bounded queue, and that says "if the queue fills up add more workers
up to maxPoolSize". If we use an unbounded queue we saying we don't
expect (or else don't care about) excessive overload.
The aim is to balance the ability to handle the expected workload,
even under transient overloads, without having excessive thread
creation and without too much thread churn (ie
create-work-die-create).
The difference is that if you are below the core pool size, each new task creates a new thread regardless of the idle threads in the pool. The number of threads is only increased once the queue is full when you already have met the core pool size but are still under the maximum.
The perfect example of this is when you have a system that you don't know exactly how much concurrent load it will have (e.g. a webserver). This function allows you to specify a core set of threads, perhaps based on the number of cores your machine has, but allow for more load than you anticipated.
This is specifically useful if you have more I/O load than you expected, and the threads in your pool spend a lot of time blocking. Your queue can easily fill up without having a large concurrent load in this scenario, and it is easily fixed by adding a couple of new threads to service some more concurrent requests.