Java Heap Dump error - Metadata does not appear to

Question

I get this Stacktrace when trying to take a heap dump from a running Java process. What causes this and what do I have to do to make a proper heap dump?

Dumping heap to dump.bin ...
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at sun.tools.jmap.JMap.runTool(JMap.java:201)
    at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: java.lang.InternalError: Metadata does not appear to be polymorphic
    at sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:278)
    at sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
    at sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:68)
    at sun.jvm.hotspot.memory.DictionaryEntry.klass(DictionaryEntry.java:71)
    at sun.jvm.hotspot.memory.Dictionary.classesDo(Dictionary.java:66)
    at sun.jvm.hotspot.memory.SystemDictionary.classesDo(SystemDictionary.java:190)
    at sun.jvm.hotspot.memory.SystemDictionary.allClassesDo(SystemDictionary.java:183)
    at sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeClasses(HeapHprofBinWriter.java:942)
    at sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:427)
    at sun.jvm.hotspot.tools.HeapDumper.run(HeapDumper.java:62)
    at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
    at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
    at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
    at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:83)
    ... 6 more

Environment: CentOS 64 bit, Java OpenJDK Runtime Environment (build 1.8.0_31-b13) OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

Usign ps to see the java version that is used:

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.31-1.b13.el6_6.x86_64/jre/bin/java

My first try was:

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.31-1.b13.el6_6.x86_64/bin/jmap  -dump:format=b,file=dump.bin 14984

That got me :

14984: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

So I ran with the -F option

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.31-1.b13.el6_6.x86_64/bin/jmap -F -dump:format=b,file=dump.bin 14984

Answer 1

Ok, I found it.

I was running the jmap command as root, but I had to run as the user that started the java process.

In my case:

sudo -u robau ./jmap -dump:format=b,file=/tmp/dump.bin 14984

Seems to be related to this JDK bug: https://bugs.openjdk.java.net/browse/JDK-8075773

Answer 2

On Ubuntu, I resolved this issue by installing the openjdk-dbg package:

sudo apt-get install openjdk-8-dbg

I now suspect that this resolved my issue indirectly by upgrading the jdk to a version that corrected the bug. You might be able to resolve this issue with:

 sudo apt-get install openjdk-8

Note: After java is upgraded, you must restart the java process in order to get a proper dump. If you don't, you will get something like this:

Attaching to process ID 21957, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.tools.jmap.JMap.runTool(JMap.java:201)
    at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.debugger.UnmappedAddressException: 7f21453c9bf8
    at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208)
    at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63)
    at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:225)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:498)
    at sun.jvm.hotspot.debugger.linux.LinuxAddress.getCIntegerAt(LinuxAddress.java:69)
    at sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:604)
    at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:164)
    at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:89)
    at sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:395)
    at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:305)
    at sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:140)
    at sun.jvm.hotspot.tools.Tool.start(Tool.java:185)
    at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
    at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:83)
    ... 6 more

Answer 3

I had this issue on CentOS even when running as the user that started the process. What solved it for me was installing the debuginfo package corresponding to the package supplying the jmap utility.

To install the debuginfo package, see this answer (substituting your java package for glibc). It requires getting/using the debuginfo-install utility, and making sure the CentOS-Debuginfo.repo is set up correctly and enabled.

Answer 4

I ran into the same issue with trying to run jmap on an AWS ElasticBeanstalk instance. The command that fixed it was

sudo debuginfo-install java-1.8.0-openjdk-devel

BTW, jmap was installed on the AWS ElasticBeanstalk instance with command

sudo yum install java-1.8.0-openjdk-devel-1.8.0.91-0.b14.10.amzn1.x86_64