JVM heap not released

I am new to analyzing memory issues in Java. So pardon me if this question seems naive

I have application running with following JVM parameters set:

-Xms3072m -Xmx3072m 
-XX:MaxNewSize=1008m -XX:NewSize=1008m 
-XX:PermSize=224m -XX:MaxPermSize=224m -XX:SurvivorRatio=6

I am using visualVM to monitor the usage : Here is what I see

The problem is, even when the application is not receiving any data for processing, the used memory doesn't go down. When the application is started, the used space starts low (around 1GB) but grows as the application is running. and then the used memory never goes down. My question is why the used heap memory doesn't go down even when no major processing happening in application and what configurations can be set to correct it. My understanding is if application is not doing any processing then the heap used should be less and heap memory available ( or max heap) should remain the same (3GB) in this case.

This is a totally normal trend, even if you believe that it is not used there are probably threads running doing tasks that create objects that are unreferenced once the tasks are done, those objects are eligible for the next GC but as long as there is no minor/major GC they take more and more room in your heap so it goes up until a GC is triggered then you get the normal heap size and so on.

An abnormal trend will be the same thing but after a GC the heap size would be higher than the heap size just after the previous GC which is not the case here.

Your real question is more what my application is doing when is not receiving any data to process? For that a thread dump should help, you can launch jcmd to get the PID then launch jstack $pid to get the thread dump.

Here is an example of a typical trend in case of memory leak:

As you can see the starting heap size has changed between two GC, the new starting heap size is higher than the previous one which may be due to a memory leak.