I wonder where is the verge after which a thread pool should be used. How many new threads per second can I create without using a thread pool still avoiding noticeable performance penalty?
Are there any observable open-source thread-pool implementations?
You should always use a thread pool. Not just for performance, but for the ease of use the java.util.concurrent package gives you. With Java 5 and later, thread pooling is built in.
Instead of thinking in terms of 'threads', use the Executor interface to execute tasks you need to be performed. Creating a new thread pool is as simple as:
Executor executor = Executors.newFixedThreadPool(5);
Full documentation on the java.util.concurrent package is here:
http://java.sun.com/javase/6/docs/api/java/util/concurrent/package-frame.html
Considering the cost, the only valid reply is to test it for yourself (not-so-elegant way to tell you I have never done such a test, and will never do it, as modern Execution mechanism provides far advanced creation/destruction mechanisms).
Consdering existing implementations, Java modern versions (starting with Java 5) offers various subclasses of ThreadPoolExecutor that combines the benefits of a thread pool with the most modern concepts of java.util.concurrent : Executors.
Besides, I would never recommand enough to you to forget about Threads and to repalce them with Runnable, callable and other more advanced computation objects. This way, you can easily switch implementation of Executors.
Creating threads is always expensive, regardless of the platform. Actual time for creating a thread depends on the OS.
I would recommend using a thread pool if you create short-lived threads very often. For example, if the primary task of your application is to service such short-lived tasks, a thread pool would be very nice.
A thread pool is also nice if you create too many threads. Context switching between threads can kill performance, too. Long story short, on contemporary quad-core PC architecture you should aim to have no more than 200 threads running a the same time.