How to solve Java error “pool-1-thread-xxxx” java.

Question

I have searched the posts about this issue, but I did't see similar situation like mine.

My java console shows the error message "pool-1-thread-xxxx" java.lang.OutOfMemory as the picture bellow:

• Red Line: CPU usage
• Green line: Memory usage

I have increased the RAM from 6G to 10G, and set -Xms=8G -Xmx=8G -Xmn=3G in *.bat file before I start the program. I also keep watching performance monitor but the memory is always around 20%. I have no idea how could this happen. Any idea?

Here is my run.bat code.

@echo off 
javapro @java -Xoptimize -Xms8G -Xmx8G -Xmn3G -Xss1024k -XX:+UseConcMarkSweepGC -cp javaPro.exe;
cls
run.bat

• OS: windows server 2012 R2
• Java version : jre1.8.0_171
• RAM: 10G
• CPU: 2.5 GHz Intel Xeon(R)(Hyper Threaded)

Answer 1

The error message java.lang.OutOfMemoryError: unable to create new native thread means that

the JVM is asking a new thread from the OS and the underlying OS cannot allocate a new thread anymore.

See OOM explanations and your specific case explained.

So you won't see high memory usage in the graph, because the heap size is not close to its limit.

You have to check your application code and inspect the thread pool usages there. It's hard to say anything without the source code, but there may be few suggestions:

1. If you use CachedThreadPoolExecutor, then the submitted tasks may be not fast enough, so your app is unable to process all submitted tasks.
2. You may use FixedThreadPoolExecutor with the maximum (Integer.MAX_VALUE) capacity, in such case you have to limit the pool capacity and its queue size. So the pool eats all available threads and OOM happens.

Generally, you should always control your thread pool configuration and be sure it won't hog all system resources.

Answer 2

think that I've found a possible reason / solution approach:

one can check for the soft-limit for processes with ulimit -a:

$ cat /proc/22666/limits | grep processes
Max processes             1024                 62265                processes 

$ ulimit -a | grep processes
max user processes   

in order to change these values:

$ ulimit -Su 2000

$ ulimit -a | grep processes
max user processes (-u) 2000

$ cat /proc/22666/limits | grep processes
Max processes 2000 

one can also adjust the default configuration:

1. modify the limits.conf file with the following:

2. sudo nano /etc/security/limits.conf or look inside /etc/security/limit.d/.

3. add the following for the user who runs java.

limits.conf

#<domain>        <type>  <item>    <value>

# adjust these values as needed:
someuser          soft    nofile    4096
someuser          hard    nofile    8192

then modify the common-session file with the following:

sudo nano /etc/pam.d/common-session

add the following line:

common-session

session required pam_limits.so

and restart java.

• also see the ThreadPoolExecutor ...
• and the ThreadPoolExecutor Configuration.

for a Windows 2012 Server, this would be about the same.

• see Pushing the Limits of Windows on the Tech Net Blog.