Performance RAM – Why System Uses More RAM After an Hour of Usage

performanceram

I am using Arch Linux (5.1.8-arch1-1-ARCH) with the XFCE DE and XFWM4 WM. Things are pretty elegant and low on RAM and CPU usage.

After the boot, and when the DE is loaded completely, I see 665 MiB of RAM usage.

But after opening applications like Atom, Code, Firefox, Chromium, or after working in GIMP, Blender etc. the RAM usage increases, which is obvious. But after closing all the applications and left with nothing but a gnome-system-monitor, I can see that the RAM usage is 1.2 – 1.4 GiB. /proc/meminfo agrees with gnome-system-monitor, but htop gives different results all the time.

The worse thing is that when I open a RAM hogging application later on, it again consumes needed memory on top of that 1.4 GiB. This is always the case. No files that could add up to megabytes are stored in the /tmp/ directory.

Also, if I look for the process that's using that much RAM (from 700 MiB at start to 1.4 GiB after closing the browser!!), I see nothing. In fact I faced the same issue even on my raspberry pi running Arch ARM.

Here's A Screenshot 1
Here's A Screenshot 2

The Ruby code:

#!/usr/bin/ruby -w
STDOUT.sync = true

loop do
    IO.readlines(File.join(%w(/ proc meminfo))).then { |x| [x[0], x[2]] }.map { |x| x.split[1].to_i }.reduce(:-)
        .tap { |x| print "\e[2K\rRAM Usage:".ljust(20), "#{x / 1024.0} MiB".ljust(24), "#{(x / 1000.0)} MB" }
    Kernel.sleep(0.1)
end

The cat /proc/meminfo command has the following output:

MemTotal:        3851796 kB
MemFree:         1135680 kB
MemAvailable:    2055708 kB
Buffers:            1048 kB
Cached:          1463960 kB
SwapCached:          284 kB
Active:          1622148 kB
Inactive:         660952 kB
Active(anon):     923580 kB
Inactive(anon):   269360 kB
Active(file):     698568 kB
Inactive(file):   391592 kB
Unevictable:      107012 kB
Mlocked:              32 kB
SwapTotal:       3978216 kB
SwapFree:        3966696 kB
Dirty:               280 kB
Writeback:             0 kB
AnonPages:        924844 kB
Mapped:           563732 kB
Shmem:            374848 kB
KReclaimable:      74972 kB
Slab:             130016 kB
SReclaimable:      74972 kB
SUnreclaim:        55044 kB
KernelStack:        8000 kB
PageTables:        14700 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     5904112 kB
Committed_AS:    3320548 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:             1456 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      226736 kB
DirectMap2M:     3778560 kB
DirectMap1G:           0 kB

Firstly you noticed htop never agrees. I don't know much about that.

And secondly you can see the xfdesktop uses 44 MiB, and some other processes uses some of the memory, the kernel uses ~150 MiB, and apart from that, why am I seeing 1.5 GiB RAM is being used? Does this really affect the performance of the system?

Best Answer

Unused RAM is wasted RAM. The Linux kernel has advanced memory management features and tries to avoid putting a burden on the bottleneck in your system, your hard drive/SSD. It tries to cache files in memory.

The memory management system works in complex ways, better performance is the goal.

You can see what it is doing by inspecting /proc/meminfo.

cat /proc/meminfo

You can reclaim this cached memory, using "drop_caches". However, note the documentation says "use outside of a testing or debugging environment is not recommended", simply because "it may cost a significant amount of I/O and CPU to recreate the dropped objects" when they are needed again :-).

Clear PageCache only:

# sync; echo 1 > /proc/sys/vm/drop_caches

Clear dentries and inodes:

# sync; echo 2 > /proc/sys/vm/drop_caches

Clear PageCache, dentries and inodes:

# sync; echo 3 > /proc/sys/vm/drop_caches

Note that sync will flush the file system buffer to ensure all data has been written.

From the kernel docs:

Page cache

The physical memory is volatile and the common case for getting data into the memory is to read it from files. Whenever a file is read, the data is put into the page cache to avoid expensive disk access on the subsequent reads. Similarly, when one writes to a file, the data is placed in the page cache and eventually gets into the backing storage device. The written pages are marked as dirty and when Linux decides to reuse them for other purposes, it makes sure to synchronize the file contents on the device with the updated data.

Reclaim

Throughout the system lifetime, a physical page can be used for storing different types of data. It can be kernel internal data structures, DMA’able buffers for device drivers use, data read from a filesystem, memory allocated by user space processes etc.

Depending on the page usage it is treated differently by the Linux memory management. The pages that can be freed at any time, either because they cache the data available elsewhere, for instance, on a hard disk, or because they can be swapped out, again, to the hard disk, are called reclaimable. The most notable categories of the reclaimable pages are page cache and anonymous memory.

In most cases, the pages holding internal kernel data and used as DMA buffers cannot be repurposed, and they remain pinned until freed by their user. Such pages are called unreclaimable. However, in certain circumstances, even pages occupied with kernel data structures can be reclaimed. For instance, in-memory caches of filesystem metadata can be re-read from the storage device and therefore it is possible to discard them from the main memory when system is under memory pressure.

The process of freeing the reclaimable physical memory pages and repurposing them is called (surprise!) reclaim. Linux can reclaim pages either asynchronously or synchronously, depending on the state of the system. When the system is not loaded, most of the memory is free and allocation requests will be satisfied immediately from the free pages supply. As the load increases, the amount of the free pages goes down and when it reaches a certain threshold (high watermark), an allocation request will awaken the kswapd daemon. It will asynchronously scan memory pages and either just free them if the data they contain is available elsewhere, or evict to the backing storage device (remember those dirty pages?). As memory usage increases even more and reaches another threshold - min watermark - an allocation will trigger direct reclaim. In this case allocation is stalled until enough memory pages are reclaimed to satisfy the request.

Memory Leaks

Now, some programs can have "memory leaks", that is, they "forget" to free up memory they no longer use. You can see this if you leave a program running for some time, its memory usage constantly increases, when you close it, the memory is never freed. Now, programmers try to avoid memory leaks, of course, but programs can have some. The way to reclaim this memory is a reboot.

Related Question