Out Of Memory Tomcat (unable to create new native

2019-05-07 10:29发布

问题:

I am getting continuously outofmemory error for 1 of my tomcat where my application is deployed after given the error tomcat is exit(shutdow).

I took the log file and found this

SEVERE: Error allocating socket processor
java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:597)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.start(JIoEndpoint.java:513)
    at org.apache.tomcat.util.net.JIoEndpoint.newWorkerThread(JIoEndpoint.java:744)
    at org.apache.tomcat.util.net.JIoEndpoint.createWorkerThread(JIoEndpoint.java:723)
    at org.apache.tomcat.util.net.JIoEndpoint.getWorkerThread(JIoEndpoint.java:757)
    at org.apache.tomcat.util.net.JIoEndpoint.processSocket(JIoEndpoint.java:789)
    at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:355)
    at java.lang.Thread.run(Thread.java:619)
18 Feb, 2015 5:43:30 PM org.apache.tomcat.util.net.JIoEndpoint createWorkerThread
INFO: Maximum number of threads (750) created for connector with address null and port 80

I am using this connector settings in the server.xml

<Connector port="80" protocol="HTTP/1.1" 
               connectionTimeout="10000" maxThreads="750" minSpareThreads="50" redirectPort="8443" />

Can anybody suggest me what can i do?

Thread Dump

"http-80-123" daemon prio=6 tid=0x5f5e7400 nid=0xcfc runnable [0x619be000..0x619bf9e8]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at com.microsoft.sqlserver.jdbc.DBComms.receive(Unknown Source)
    at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(Unknown Source)
    at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PreparedStatementExecutionRequest.executeStatement(Unknown Source)
    at com.microsoft.sqlserver.jdbc.CancelableRequest.execute(Unknown Source)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeRequest(Unknown Source)
    - locked <0x15c69398> (a com.microsoft.sqlserver.jdbc.TDSWriter)

I have taken 4 thread dump over 20 seconds duration and found this tid alive all the time

Can we find something with this dump?

I got this from thread dump is this indicating the deadlock condition 'BLOCKED' since i am getting this several times 80- 90 with same state

"http-80-342" daemon prio=6 tid=0x5c0d7c00 nid=0x1d0c waiting for monitor entry [0x6ee0e000..0x6ee0fce8]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at java.sql.DriverManager.getCallerClass(DriverManager.java:477)
    at java.sql.DriverManager.getConnection(DriverManager.java:576)
    at java.sql.DriverManager.getConnection(DriverManager.java:185)
    at t4u.common.DBConnection.getConnectionToDB(DBConnection.java:32)
    at t4u.functions.CommonFunctions.getProcessID(CommonFunctions.java:1465)

回答1:

java.lang.OutOfMemoryError: unable to create new native thread

This means you've hit some limit on the number of threads the JVM can create. This could be an OS imposed limit or it could be due to lack of resources (like memory).

If it's an OS imposed limit, you may be able to raise it, assuming you have that level of access to the system.

If it's memory related, you need to either increase the available memory (add more physical or increase memory on your VM) or lower the memory requirement for each thread. To do the later, you can change the thread stack size with the -Xss argument to the JVM. To find the ideal value, start with it as very low, like 128K or 256K, and see if you get any StackOverflow exceptions. If you do, increase until they go away.

One other possibility here is to use a different Connector implementation. You haven't said which one you're currently using, but the BIO (blocking IO) Connector consumes one thread per request. Depending on what your application is doing, you may see better thread utilization (i.e. less threads required to handle the same amount of requests) if you use the NIO or APR connectors. There's not enough information here for me to say one way or another, but you might want to test it out with your apps and see if the number of threads required drops.

INFO: Maximum number of threads (750) created for connector with address null and port 80

This looks like you might be hitting a limit on your thread pool in Tomcat. Check your configuration in conf/server.xml. If you're using an Executor, look for maxThreads on that element. If not, look for maxThreads on the Connector element.