Finding Memory Usage in Java

Question

Following is the scenario i need to solve. I have struck with two solutions.

I need to maintain a cache of data fetched from database to be shown on a Swing GUI. Whenever my JVM memory exceeds 70% of its allocated memory, i need to warn user regarding excessive usage. And once JVM memory usage exceeds 80%, then i have to halt all the database querying and clean up the existing cache fetched as part of the user operations and notifying the user. During cleanup process, i will manually handle deleting some data based up on some rules and instructs JVM for a GC. Whenever GC occurs, if memory cleans up and reaches 60% of the allocated memory, I need to restart all the Database handling and giving back control to the user.

For checking JVM memory statistics i found following two solutions. Could not able to decide which is best way and why.

1. Runtime.freeMemory() - Thread created to run every 10 seconds and check for the free memory and if memory exceeds the limits mentioned, necessary popups will intimate user and will call the methods to halt the operations and freeing up the memory.

2. MemoryPoolMXBean.getUsage() - Java 5 has introduced JMX to get the snapshot of the memory at runtime. In, JMX i cannot use Threshold notification since it will only notify when memory reaches/exceeds the given threshhold. Only way to use is Polling in MemoryMXBean and check the memory statistics over a period.

In case of using polling, it seems for me both the implementations are going to be same.

Please suggest the advantages of the methods and if there are any other alternatives/any corrections to the methods using.

Answer 1

It is entirely normal for a JVM to go up to 100% memory usage and them back to say 10% after a GC and do this every few second.

You shouldn't need to try managing the memory in this way. You cannot say how much memory is being retained until a full GC has been run.

I suggest you work out what you are really trying to achieve and look at the problem another way.

Answer 2

The requirements you mention are a clear contradiction with how Garbage Collection works in a JVM.

because of the behaviour of the JVM it will be very hard to warn you users in a correct way. Altogether stopping als database manipulation , cleaning stuff up and starting again really is not the way to go.

Let the JVM do what it is supposed to do, handle all memory related for you. Modern generations of the JVM are very good at it and with some finetuning of the GC parameters you will get a a much cleaner memory handling then forcing things yourself

Articles like http://www.kodewerk.com/advice_on_jvm_heap_tuning_dont_touch_that_dial.htm mention the pros and cons and offer a nice explanation of what the VM does for you

Answer 3

The usual way to handle this sort of thing is to use WeakReferences and SoftReferences. You need to use both - the weak reference means you are not holding multiple copies of things, and the soft references mean that the GC will hang onto things until it starts running out of memory.

If you need to do additional cleanup, then you can add references to queues, and override the queue notification methods to trigger the cleanup. It's all good fun, but you do need to understand what these classes do.

Answer 4

I've only used the first method for similar task and it was OK.

One thing you should note, for both methods, is to implement some kind of debouncing - i.e. once you recognize you've hit 70% of memory, wait for a minute (or any other time you find appropriate) - GC can run at that time and clean up lots of memory.

If you implement a Runtime.freeMemory() graph in your system you'll see how the memory is constantly going up and down, up and down.

Answer 5

VisualVM is a bit nicer than JConsole because it gives you a nice visual Garbage Collector view.

Answer 6

Just a side note: Runtime.freeMemory() doesn't state the amount of memory that's left of allocating, it's just the amount of memory that's free within the currently allocated memory (which is initially smaller than the maximum memory the VM is configured to use), but grows over time.

When starting a VM, the max memory (Runtime.maxMemory()) just defines the upper limit of memory that the VM may allocate (configurable using the -Xmx VM option). The total memory (Runtime.totalMemory()) is the initial size of the memory allocated for the VM process (configurable using the -Xms VM option), and will dynamically grow every time you allocate more than the currently free portion of it (Runtime.freeMemory()), until it reaches the max memory.

The metric you're interested in is the memory available for further allocation:

long usableFreeMemory= Runtime.getRuntime().maxMemory()
    -Runtime.getRuntime().totalMemory()
    +Runtime.getRuntime().freeMemory()

or:

double usedPercent=(double)(Runtime.getRuntime().totalMemory()
    -Runtime.getRuntime().freeMemory())/Runtime.getRuntime().maxMemory()