I have encountered a rather confusing GC case: While the Eden space is 100% full, 0% survivor space is used. When the Eden is full, garbage collection should be triggered, right?
Can there be a case in which the GC daemon is prevented from running? Like a 100% CPU?
We are using jdk-1.7.
What can be the cause? Below is the jmap output.
We also tried to capture a more detailed memory usage using jmap -histo -F, but then the CPU usage dropped to 0% and the java process became unreachable.
using thread-local object allocation.
Parallel GC with 18 thread(s)
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 12884901888 (12288.0MB)
NewSize = 1310720 (1.25MB)
MaxNewSize = 17592186044415 MB
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 85983232 (82.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 4265738240 (4068.125MB)
used = 4265738240 (4068.125MB)
free = 0 (0.0MB)
100.0% used
From Space:
capacity = 14352384 (13.6875MB)
used = 0 (0.0MB)
free = 14352384 (13.6875MB)
0.0% used
To Space:
capacity = 14680064 (14.0MB)
used = 0 (0.0MB)
free = 14680064 (14.0MB)
0.0% used
PS Old Generation
capacity = 8589934592 (8192.0MB)
used = 8589931920 (8191.997451782227MB)
free = 2672 (0.0025482177734375MB)
99.99996889382601% used
PS Perm Generation
capacity = 41353216 (39.4375MB)
used = 41079056 (39.17604064941406MB)
free = 274160 (0.2614593505859375MB)
99.33702858805468% used
When i see at your Heap Configuration i found that
MaxNewSize = 17592186044415 MB is incorrect however it must be in bytes.
What i see that almost all generations are full and same time and Collector is trying to collect both the generations so they are blocking each other.
I suggest to please tune the memory with below parameters .
-XX:NewRatio=3 - the young generation will occupy 1/4 the overall heap
-XX:NewSize - Calculated automatically if you specify -XX:NewRatio
-XX:MaxNewSize - The largest size the young generation can grow to (unlimited if this value is not specified at command line)
I also suggest to use some survivor space it will give time collector when objects being copied from eden to 'tenured' generation.
-XX:SurvivorRatio=6 - each survivor space will be 1/8 the young generation
If survivor spaces are too small copying collection overflows directly into the tenured generation.
For any clarification refer this link.
Edit:
-XX:NewRatio=3 - the young generation will occupy 1/4 the overall heap
Calculation:
y/t=1/3
y+t=h
y+3y=h
y=h/4
t=tenured
y=young
h=heap
The thread local buffers are allocated from the eden space. As soon as a buffer is allocated, the free space in the eden space is reduce in full. This can mean you have many TLAB which are nearly empty, but the eden space appears to be full. A GC is triggered when there is not enough free space in the eden space.
If you turn off TLAB -XX:-UseTLAB this will slow performance but you will get much accurate accounting of how space is used.
Just a thought but I see that the Old Gen is also full. Could it be that the GC instead of trying to cleanup Eden is busy running full GC trying to cleanup the Old Generation to avoid OOM.
SurvivorRatio Calculation:
If SurvivorRatio=6 then the ratio of (One SurvivorSpace : Eden) = (1:6).
So two Survivor will take (2 part), and Eden has 6 parts. So total 8 parts.
So if i have young gen of 40 MB, Eden is (6/8 * 40)MB and one survivor will have (1/8 * 40)MB.
If SurvivorRatio=7. Then total parts are (7 + 1 + 1 = 9).
Eden is (7/9 * 40)MB and survivor will have (1/9 * 40)MB
Theoretically
The SurvivorRatio parameter controls the size of the two survivor spaces.
For example, -XX:SurvivorRatio=6 sets the ratio between each survivor space
and eden to be 1:6, each survivor space will be one eighth of the young
generation.
Mathamatically
S - Survivor
E - Eden
Y - Young generation
if SurvivorRatio=6
then S / E = 1/6
E = 6S
2S + E = Y
2S + 6S = Y
S = Y / 8
Effect
If increasing to more SurvivorRatio value results in lesser space for Survivor will results, pushing the eden space objects directly into the old/tenured gen. Lesser opportunity for filtering out shot live objects at Survivor Space through minor GC. Also this may increase the no of full GC.
If decreasing this to more low value will results in lesser eden space will make frequent the minor GC.
Know that the time taken for minor GC will be more especially if we are using Serial GC algorithm where only single thread will be used for GC, though your machine is multiple core machine.
Also Note that -XX:+UseAdaptiveSizePolicy will determine JVM's own SurvivorRatio during run time, But it always better to mention it manually based on your application by trying multiple load run.
