Linux c application memory usage

Question

I have C Linux application which continuously allocates and frees memory (around 200 alloc/free per sec) using malloc, calloc, realloc & free functions. Even though all allocated memory are freed (verified by wrapping *alloc and free), the VmSize, VmRSS & VmData numbers are keep on increasing and finally application is getting killed by OOM killer.

Why the VmSize, VmRSS & VmData are keep on increasing? if it is Memory management issue, any pointers to avoid this?

I saw this Problem usage memory in C, but the answers are not explaining the OOM behavior.

Answer 1

You should first use valgrind (to debug potential hard to find memory leaks or misbehavior). Don't forget to compile with gcc -Wall -g (then, when it works, use -Wall -O); of course, improve your code till no warnings are given.

You could probably (if the algorithms fit) try to (usefully) allocate memory zone of e.g. either power of two, or 3 times a power of two [perhaps minus 2 or 3 words]; at least try to avoid too many different random sizes of allocation.

You may want to try using Boehm's conservative garbage collector - i.e. replacing all your malloc with GC_MALLOC (or GC_MALLOC_ATOMIC & strdupwith GC_STRDUP), your free with GC_FREE, etc...

At least for testing purposes, use setrlimit(2) perhaps thru the bash ulimit builtin. You want RLIMIT_AS - possibly with RLIMIT_DATA (setting these limits sensibly avoids the OOM killer and makes your mmap -called by malloc- fail when memory is exhausted).

You may want to compile with GCC 4.8 which accepts -fsanitize=address.

You could also implement your own application specific garbage collector (read that wikipage, it gives you insights and terminology); a mark & compact algorithm will fight fragmentation.

See also this question about memory fragmentation. Look into the Plug tool.