Java 6 Update 25 VM crash: insufficient memory

Question

For an update of this question - see below.

I experience a (reproducible, at least for me) jvm crash (not an OutOfMemoryError) (The application which crashes is eclipse 3.6.2). However, looking at the crash log makes me wonder:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 65544 bytes for Chunk::new
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.

Current thread (0x531d6000):  JavaThread "C2 CompilerThread1" daemon 
[_thread_in_native, id=7812, stack(0x53af0000,0x53bf0000)]

Stack: [0x53af0000,0x53bf0000],  sp=0x53bee860,  free space=1018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x1484aa]
V  [jvm.dll+0x1434fc]
V  [jvm.dll+0x5e6fc]
V  [jvm.dll+0x5e993]
V  [jvm.dll+0x27a571]
V  [jvm.dll+0x258672]
V  [jvm.dll+0x25ed93]
V  [jvm.dll+0x260072]
V  [jvm.dll+0x24e59a]
V  [jvm.dll+0x47edd]
V  [jvm.dll+0x48a6f]
V  [jvm.dll+0x12dcd4]
V  [jvm.dll+0x155a0c]
C  [MSVCR71.dll+0xb381]
C  [kernel32.dll+0xb729]

I am using Windows XP 32-bit SP3. I have 4GB RAM. Before starting the application I had 2 GB free according the task manager (+ 1 GB system cache which might be freed as well.). I am definitely having enough free RAM.

From the start till the crash I logged the jvm memory statistics using visualvm and jconsole. I acquired the memory consumption statistics until the last moments before the crash.

The statistics shows the following allocated memory sizes:

• HeapSize: 751 MB (used 248 MB)
• Non-HeapSize(PermGen & CodeCache): 150 MB (used 95 MB)
• Size of memory management areas (Edenspace, Old-gen etc.): 350 MB
• Thread stack sizes: 17 MB (according to oracle and due the fact that 51 threads are running)

I am running the application (jre 6 update 25, server vm) using the parameters:

-XX:PermSize=128m
-XX:MaxPermSize=192m
-XX:ReservedCodeCacheSize=96m
-Xms500m
-Xmx1124m

Question:

• Why does the jvm crash when there's obviously enough memory on the vm and OS?
  With the above settings I think that I cannot hit the 2GB 32-bit limit (1124MB+192MB+96MB+thread stacks < 2GB). In any other case (too much heap allocation), I would rather expect an OutOfMemoryError than a jvm crash

Who can help me to figure out what is going wrong here?

(Note: I upgraded recently to Eclipse 3.6.2 from Eclipse 3.4.2 and from Java 5 to Java 6. I suspect that there's a connection between the crashes and these changes, because I haven't seen these before)

UPDATE

It seems to be a jvm bug introduced in Java 6 Update 25 and has something to do with the new jit compiler. See also this blog entry. According to the blog the fix of this bug should be part of the next java 6 update. In the meanwhile I got a native stack trace during a crash. I've updated the above crash log.

The proposed workaround, using the vm argument -XX:-DoEscapeAnalysis works (at least it notably lowers the probability of a crash)

Answer 1

The JVM has its own limits that will stop it long before it hits the physical or virtual memory limits. What you need to adjust is the heap size, which is with another one of the -X flags. (I think it's something creative like -XHeapSizeLimit but I'll check in a second.)

Here we go:

-Xmsn Specify the initial size, in bytes, of the memory allocation pool. This value must be a multiple of 1024 greater than 1MB. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. The default value is 2MB. Examples:

   -Xms6291456
   -Xms6144k
   -Xms6m

-Xmxn Specify the maximum size, in bytes, of the memory allocation pool. This value must a multiple of 1024 greater than 2MB. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. The default value is 64MB. Examples:

   -Xmx83886080
   -Xmx81920k
   -Xmx80m

Answer 2

I stumbled upon a similar problem at work. We had set -Xmx65536M for our application but kept getting exactly the same kind of errors. The funny thing is that the errors happened always at a time when our application was actually doing pretty lightweight calculations, relatively speaking, and was thus nowhere near this limit.

We found a possible solution for the problem online: http://www.blogsoncloud.com/jsp/techSols/java-lang-OutOfMemoryError-unable-to-create-new-native-thread.jsp , and it seemed to solve our problem. After lowering -Xmx to 50G, we've had none of these issues.

What actually happens in the case is still somewhat unclear to us.