I am running a C program on Linux 2.6.16 kernel. I do not think there are memory leaks in my program however the memory consumption for the program remains stable after certain operations and does not decrease. I use the 'ps v ' command to monitor the RSS value of my program.
The valgrind massif tool shows a large portion of heap is allocated by mmap in my process. But according to the code those allocations should have been freed after the operations are done. Is it because the freed memory is still mapped and/or still contributes to the RSS value of the process?
Any insight will be very appreciated!
Below is the snip from the valgrind massif report. Note I have turned on the –pages-as-heap option for the massif tool to measure all memories used by the program.
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
85 701,483,989,262 173,576,192 173,576,192 0 0
86 704,352,949,469 173,367,296 173,367,296 0 0
87 707,582,275,643 173,367,296 173,367,296 0 0
88 710,536,145,814 173,367,296 173,367,296 0 0
100.00% (173,367,296B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->53.40% (92,581,888B) 0x649248B: mmap (in /lib64/tls/libc.so.6)
| ->41.13% (71,303,168B) 0x6446D85: _int_malloc (in /lib64/tls/libc.so.6)
| | ->39.31% (68,157,440B) 0x6448D62: calloc (in /lib64/tls/libc.so.6)
......[my own functions are omitted]
->35.28% (61,157,376B) 0x400F51B: mmap (in /lib64/ld-2.3.3.so)
| ->28.81% (49,954,816B) 0x4004CE8: _dl_map_object_from_fd (in /lib64/ld-2.3.3.so)
| | ->28.81% (49,954,816B) 0x400636B: _dl_map_object (in /lib64/ld-2.3.3.so)
| | ->18.89% (32,755,712B) 0x400AB42: openaux (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x400AF7C: _dl_catch_error (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x4009FCF: _dl_map_object_deps (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x40021FD: dl_main (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x400E7F6: _dl_sysdep_start (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x4001477: _dl_start (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x4000CF6: ??? (in /lib64/ld-2.3.3.so)
| | | ->18.89% (32,755,712B) 0x0: ???
| | | ->18.89% (32,755,712B) 0x7FF0003D5: ???
| | | ->18.89% (32,755,712B) 0x7FF0003E4: ???
| | |
......
Best Answer
The C-library function
free()
can, but does not have to, return memory to the kernel.Some implementations of
malloc()
move the boundary between "heap" and otherwise unused address space (the "system break") via thesbrk()
system call, then dole out smaller pieces of those large allocations. Without getting every smaller piece de-allocated,free()
can't really return the memory to the OS.That same reason applies to
malloc()
implementations that don't usesbrk(2)
, but maybe usemmap("/dev/zero")
or something.. I can't find a reference, but I seem to remember that one or another of the BSD's usedmmap()
that way to get pages of memory. Nevertheless,free()
can't return a page to the operating system unless every sub-allocation is deallocated by the program.Some
malloc()
implementations do return memory to the system: ChorusOS(?) apparently did. It's not clear if it moved the system break, ormunmap()'ed
pages.Here's a paper about a memory allocator that improves performance by "aggressively giving up free pages to the virtual memory manager". Slide show for a talk about the allocator.