Limit total memory consumption of Java process (in

Question

Related to these two questions:

• How to set the maximum memory usage for JVM?
• What would cause a java process to greatly exceed the Xmx or Xss limit?

I run a Java application on Cloud Foundry and need to make sure that the allocated memory is not exceeded. Otherwise, and this is the current issue, the process is killed by Cloud Foundry monitoring mechanisms (Linux CGROUP).

The Java Buildpack automatically sets sane values for -Xmx and -Xss. By tuning the arguments and configuring the (maximum) number of expected threads, I'm pretty sure that the memory consumed by the Java process should be less than the upper limit which I assigned to my Cloud Foundry application.

However, I still experience Cloud Foundry "out of memory" errors (NOT the Java OOM error!): index: 3, reason: CRASHED, exit_description: out of memory, exit_status: 255

I experimented with the MALLOC_ARENA_MAX setting. Setting the value to 1 or 2 leads to slow startups. With MALLOC_ARENA_MAX=4 I still saw an error as described above, so this is no solution for my problem.

Currently I test with very tight memory settings so that the problem is easier to reproduce. However, even with this, I have to wait about 20-25 minutes for the error to occur.

Which arguments and/or environment variables do I have to specify to ensure that my Java process never exceeds a certain memory limit? Crashing with a Java OOM Error is acceptable if the application actually needs more memory.

Further information regarding MALLOC_ARENA_MAX:

• https://github.com/cloudfoundry/java-buildpack/pull/160
• https://www.infobright.com/index.php/malloc_arena_max/#.VmgdprgrJaQ
• https://www.ibm.com/developerworks/community/blogs/kevgrig/entry/linux_glibc_2_10_rhel_6_malloc_may_show_excessive_virtual_memory_usage?lang=en

EDIT: A possible explaination is this: http://www.evanjones.ca/java-bytebuffer-leak.html. As I currently see the OOM issue when doing lots of outgoing HTTP/REST requests, these buffers might be to blame.

Answer 1

Unfortunately, there is no way to definitively enforce a memory limit on the JVM. Most of the memory regions are configurable (-Xmx, -Xss, -XX:MaxPermSize, -XX: MaxMetaspaceSize, etc.) but the one you can't control is Native memory. Native memory contains a whole host of things from memory mapped files to native libraries to JNI code. The best you can do is profile your application, find out where the memory growth is occurring, and either solve the growth or give yourself enough breathing room to survive.

Certainly unsatisfying, but in the end not much different from other languages and runtimes that have no control over their memory footprint.