Problem usage memory in C

问题:

Please help :) OS : Linux

Where in " sleep(1000);", at this time "top (display Linux tasks)" wrote me 7.7 %MEM use. valgrind : not found memory leak.

I understand, wrote correctly and all malloc result is NULL. But Why in this time "sleep" my program NOT decreased memory ? What missing ?

Sorry for my bad english, Thanks

~ # tmp_soft
For : Is it free??  no
Is it free??  yes
For 0 
For : Is it free??  no
Is it free??  yes
For 1 
END : Is it free??  yes
END 

~ #top
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                
23060 root      20   0  155m 153m  448 S    0  7.7   0:01.07 tmp_soft    

Full source : tmp_soft.c

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

struct cache_db_s
{
 int       table_update;
 struct    cache_db_s * p_next;
};

void free_cache_db (struct cache_db_s ** cache_db)
{
 struct cache_db_s * cache_db_t;
 while (*cache_db != NULL)
 {
  cache_db_t = *cache_db;
  *cache_db = (*cache_db)->p_next;
  free(cache_db_t);
  cache_db_t = NULL;
 }
 printf("Is it free??  %s\n",*cache_db==NULL?"yes":"no");
}

void make_cache_db (struct cache_db_s ** cache_db)
{
 struct cache_db_s * cache_db_t = NULL;
 int n = 10000000;

 for (int i=0; i = n; i++)
 {
  if ((cache_db_t=malloc(sizeof(struct cache_db_s)))==NULL) {
   printf("Error : malloc 1 -> cache_db_s (no free memory) \n");
   break;
  }
  memset(cache_db_t, 0, sizeof(struct cache_db_s));

  cache_db_t->table_update = 1; // tmp 

  cache_db_t->p_next = *cache_db;
  *cache_db = cache_db_t;
  cache_db_t = NULL;
 }
}

int main(int argc, char **argv)
{
 struct cache_db_s * cache_db = NULL;

 for (int ii=0; ii  2; ii++) {
  make_cache_db(&cache_db);
  printf("For : Is it free??  %s\n",cache_db==NULL?"yes":"no");
  free_cache_db(&cache_db);
  printf("For %d \n", ii);
 }

 printf("END : Is it free??  %s\n",cache_db==NULL?"yes":"no");
 printf("END \n");
 sleep(1000);
 return 0;
}

回答1:

If you're trying to establish whether your program has a memory leak, then top isn't the right tool for the job (valrind is).

top shows memory usage as seen by the OS. Even if you call free, there is no guarantee that the freed memory would get returned to the OS. Typically, it wouldn't. Nonetheless, the memory does become "free" in the sense that your process can use it for subsequent allocations.

edit If your libc supports it, you could try experimenting with M_TRIM_THRESHOLD. Even if you do follow this path, it's going to be tricky (a single used block sitting close to the top of the heap would prevent all free memory below it from being released to the OS).

回答2:

For good reasons, virtually no memory allocator returns blocks to the OS

---

Memory can only be removed from your program in units of pages, and even that is unlikely to be observed.

calloc(3) and malloc(3) do interact with the kernel to get memory, if necessary. But very, very few implementations of free(3) ever return memory to the kernel¹, they just add it to a free list that calloc() and malloc() will consult later in order to reuse the released blocks. There are good reasons for this design approach.

Even if a free() wanted to return memory to the system, it would need at least one contiguous memory page in order to get the kernel to actually protect the region, so releasing a small block would only lead to a protection change if it was the last small block in a page.

Theory of Operation

So malloc(3) gets memory from the kernel when it needs it, ultimately in units of discrete page multiples. These pages are divided or consolidated as the program requires. Malloc and free cooperate to maintain a directory. They coalesce adjacent free blocks when possible in order to be able to provide large blocks. The directory may or may not involve using the memory in freed blocks to form a linked list. (The alternative is a bit more shared-memory and paging-friendly, and it involves allocating memory specifically for the directory.) Malloc and free have little if any ability to enforce access to individual blocks even when special and optional debugging code is compiled into the program.

---

^{1. The fact that very few implementations of free() attempt to return memory to the system is not at all due to the implementors slacking off.}

^{Interacting with the kernel is much slower than simply executing library code, and the benefit would be small. Most programs have a steady-state or increasing memory footprint, so the time spent analyzing the heap looking for returnable memory would be completely wasted. Other reasons include the fact that internal fragmentation makes page-aligned blocks unlikely to exist, and it's likely that returning a block would fragment blocks to either side. Finally, the few programs that do return large amounts of memory are likely to bypass malloc() and simply allocate and free pages anyway.}

回答3:

When you free() memory, it is returned to the standard C library's pool of memory, and not returned to the operating system. In the vision of the operating system, as you see it through top, the process is still "using" this memory. Within the process, the C library has accounted for the memory and could return the same pointer from malloc() in the future.

I will explain it some more with a different beginning:

During your calls to malloc, the standard library implementation may determine that the process does not have enough allocated memory from the operating system. At that time, the library will make a system call to receive more memory from the operating system to the process (for example, sbrk() or VirtualAlloc() system calls on Unix or Windows, respectively).

After the library requests additional memory from the operating system, it adds this memory to its structure of memory available to return from malloc. Later calls to malloc will use this memory until it runs out. Then, the library asks the operating system for even more memory.

When you free memory, the library usually does not return the memory to the operating system. There are many reasons for this. One reason is that the library author believed you will call malloc again. If you will not call malloc again, your program will probably end soon. Either case, there is not much advantage to return the memory to the operating system.

Another reason that the library may not return the memory to the operating system is that the memory from operating system is allocated in large, contiguous ranges. It could only be returned when an entire contiguous range is no longer in use. The pattern of calling malloc and free may not clear the entire range of use.

回答4:

Generally free() doesn't give back physical memory to OS, they are still mapped in your process's virtual memory. If you allocate a big chunk of memory, libc may allocate it by mmap(); then if you free it, libc may release the memory to OS by munmap(), in this case, top will show that your memory usage comes down.

So, if you want't to release memory to OS explicitly, you can use mmap()/munmap().

回答5:

Two problems:

• In make_cache_db(), the line

  for (int i=0; i = n; i++)
  

  should probably read

  for (int i=0; i<n; i++)
  

  Otherwise, you'll only allocate a single cache_db_s node.

• The way you're assigning cache_db in make_cache_db() seems to be buggy. It seems that your intention is to return a pointer to the first element of the linked list; but because you're reassigning cache_db in every iteration of the loop, you'll end up returning a pointer to the last element of the list.

  If you later free the list using free_cache_db(), this will cause you to leak memory. At the moment, though, this problem is masked by the bug described in the previous bullet point, which causes you to allocate lists of only length 1.

Independent of these bugs, the point raised by aix is very valid: The runtime library need not return all free()d memory to the operating system.