Strategy or tools to find “non-leak” memory usage

Question

One old application started to consume memory a lot after server update. Memory usage seems to rise with out limit until program hangs.

According to FastMM4 and EurekaLog, there's no memory leak (except 28 bytes), so I assume all memory is freed when application is shutdown.

Are there any tools or strategies suitable for tracking this kind of memory problem?

Answer 1

Since September 2012, there is a very simple and comfortable way to find this type of "run-time only" memory leaks.

FastMM4991 introduced a new method, LogMemoryManagerStateToFile:

Added the LogMemoryManagerStateToFile call. This call logs a summary of the memory manager state to file: The total allocated memory, overhead, efficiency, and a breakdown of allocated memory by class and string type. This call may be useful to catch objects that do not necessarily leak, but do linger longer than they should.

To discover the leak at run time, you only need these steps

1. add a call to LogMemoryManagerStateToFile('memory.log', '') in a place where it will be called in intervals
2. run the application
3. open the log file with a tail program (for example BareTail), which will auto-refresh when the file content changes
4. watch the first lines of the file, they will contain the memory allocations which occupy the highest amount of memory
5. if you see a class or memory type constantly has a growing number of instances, this can be the reason of your leak

Answer 2

AQTime (a commercial tool which is quite expensive) can report your memory usage, down to the line of source code that allocated each object. In the case of very large memory usage scenarios, you might want the AQTime functionality that can show the number of objects and the size (total plus individual instance size) for each object. AQTime worked great for me, starting with Delphi 7, and all later versions, including your version (2006) and the latest versions (XE and XE2).

As the program memory usage grows, AQTime can be used to grab "snapshots" of the runtime heap, you can use to understand memory usage of your application; What is being created, and how many of each object exists. Even when no leaks exist, understanding the runtime behaviour of your application in terms of the objects it creates and manages, is very important, and AQTime is the most powerful tool I know of for Delphi users.

If you are willing to upgrade to Delphi XE/XE2, you might have an included light version of AQTime already, if so, check it out. If not, I recommend you try their demo. I am unaware of any free or open source alternatives that can provide the same functionality.

Lesser functionality could be cobbled together manually by writing lots of trace messages, or using the FastMM full-debug-mode. If you could write a complete dump of your memory usage into a very large file, you might be able to write some tools to parse, and create a summary. The problem I have with FastMM in this case, is that you will be drowned in detail information, without the ability to extract exactly the summary information that helps you understand your situation. So, you can try to write your own tool to summarize the memory usage. In one application I had that used a series of components that I knew would use a lot of memory, I wrote a dialog box into my application that showed current memory usage by these large memory-blob-of-data objects.

Answer 3

Have you ever think about the Leak that is causing the IDE... it is so huge!!!

In my case (2GB of RAM) i do the next... 1. Open the IDE 2. Leave it minimized for near six hours 3. See how Physical memory is getting used

The result: While IDE is oppened (remember i also do the test having it minimized) it is getting more and more RAM... till no more ram free. It gets all 2GB RAM + all Pagefile hard disk space (i have it configured to a mas of 4GB) In less that six hours (doing nothing on IDE) it tries to use more than 6GB.

That is called a Memory Leak casused by the IDE... i do not type any letter on IDE, do not compile anything, do not even open any project... just open IDE and minimize it... leave the computer without doing anything on it for about six hours and IDE is consuming 6GB of memory.

Of course, after that, the IDE start with annoying messages of SystemOutOfMemory... and i must kill it... then all that 6GB are freed!!!

When on the hell will this get fixed?

Please note i have all patches applied, i also tested without applying each patch/hotfix, etc...

The best i got was dissabling some options on Tools, like the one that underlines bad code, etc... so why on the hell that option has any influence... i am not typing anything on the IDE (on the tests)... and if i have it dissabled the memory leak gets reduced a lot...

Of course, if i use the IDE (write code on an opened project) without even compiling / running it... the thing goes much more worst... memory leak upto 6GB can got reached on less than an hour, sometimes occurs after 15 minutes of Copy/Paste source code.

Seems there will not be a solution in a short time!!!

So i got the next solution that works perfect: -Close the IDE an reopen it each 15 minutes or less

Ugly solution, i know... but works!!!

Answer 4

1. The growing memory consumption is an application issue. It is not a bug, which can discover FastMM4 or EurekaLog. As from they point of view - application just correctly uses the memory.
2. Using AQTime, MemProof (hard to find, D7 is last supported version (?)), SleuthQA (similar to MemProof) or similar memory profilers, you can track the memory usage outside of application in real-time.
3. Using FastMM4, GetMemoryManagerState / GetMemoryManagerUsageSummary you can track memory usage from application. Output this information into trace file and analyze it after run. Or make simple wrapping function for one of the above procedures, which will return curent memory usage. And call it from IDE Debugger Evalute / Modify, add to Watches or call OutputDebugString, and see the current memory usage.

Note, if memory is eated by some DLL then you may not see her memory usage using (3). Use (2).

Analyzing the memory usage and the tasks performed by the application, you may discover what leads to raised memory usage.