If/When does the does deallocated heap memory get

问题:

I have been running overnight memory tests on an embedded Linux system. Using vmstat I have observed that the free memory steadily decreases over time. According to some smaps analysis in procfs, the heap of one process grows at roughly the same rate. I suspected a memory leak and found a few spots in the code where new and delete are regularly used. However, I did not see a new calls without matching delete calls.

I ran the memory test again and this morning cleared the memory caches with the following call

echo 3 > /proc/sys/vm/drop_caches

The free memory listed in vmstat went down to a value close to when the test was started.

Does the kernel regularly reclaim unused heap pages? If so, are there other times besides the one above that this is done? Probably when free memory gets below a certain threshold?

回答1:

As others said, it is the process's duty to return memory to the kernel.

Usually there are 2 ways to allocate memory: if you malloc()/new a memory block above a certain size, the memory gets allocated from the OS via mmap() and eturned as soon as it is free. Smaller blocks are allocated by increasing the process's data area by shifting the sbrk border upwards. This memory is only freed if a block over a certain size is free at the end of that segment.

E.g.: (pseudo code, I don't know C++ very well)

a = new char[1000];
b = new char[1000];

Memory map:

---------------+---+---+
end of program | a | b |
---------------+---+---+

If you free a now, you have a hole in the middle. It is not freed because it cannot be freed. If you free b, the process's memory may or may not be reduced; the unused remainder is returned to the system.

A test with a program as simple as

#include <stdlib.h>

int main()
{
    char * a = malloc(100000);
    char * b = malloc(100000);
    char * c = malloc(100000);
    free(c);
    free(b);
    free(a);
}

leads to a strace output like

brk(0)                                  = 0x804b000
brk(0x8084000)                          = 0x8084000
brk(0x80b5000)                          = 0x80b5000
brk(0x809c000)                          = 0x809c000
brk(0x8084000)                          = 0x8084000
brk(0x806c000)                          = 0x806c000

is shows that the brk value is first increased (for malloc()) and then decreased again (for free()).

回答2:

The kernel will reclaim cached memory pages when it needs them, i.e. when the system would otherwise run out of memory. whether memory pages from the processes's heap (free store) are ever returned to the OS is at the discretion of the process's memory manager, in this case the new/delete implementation in the C++ library. This is a completely voluntary operation with which the kernel has nothing to do.

From the fact that drop_caches did the trick, you can infer that it was the kernel cache, not the process's heap, that was filling up memory. Use the free command to find out how much memory is actually available for application use, esp. the -/+ buffers/cache line it reports.

回答3:

Calling delete in your program causes memory to return to the memory manager that is part of your program runtime. In principle this could be written so as to return freed memory to the OS, but I would be surprised if it did. Rather the recycled memory is kept aside for subsequent calls to new.

Note that this is the virtual memory of your process; how much of it is actually residing in physical memory at any time during program execution depends on overall system load and is handled by the operating system.

回答4:

User calls to malloc and free (or new and delete), to the best of my knowledge never return no-longer used pages to the O/S. Instead, they just remember what memory has been freed so that if you do a malloc/new of a size that can be satisfied by previously freed memory, then it will use that, rather than going to the O/S and using sbrk to get more memory.

Thus this code:

for (;;)
{
    struct { char data[200 * 1024 * 1024] } HugeBuffer;
    HugeBuffer *buff = new HugeBuffer;
    delete buff;
}

Will allocate 200Mb once, and then just steadily use that memory forever. It will go to the O/S once on the original allocation, and then loop fiddling around in user space.