Linux – how to see the contents in the RAM when a program is executed in Linux

It appears that there is no surefire way to tell, however various approaches can get you some sort of answer. Apparently you pretty much have to try the different ones until you find one that tells you ECC is working.

In my case memtest86+ 4.20 couldn't be coaxed into realizing it was dealing with ECC RAM; even if I configured it for ECC On, it still reported ECC: Disabled on the IMC line. I haven't yet tried with a newer version. However (possibly after installing edac-utils, unfortunately I did both essentially at the same time), Linux reports in the boot logs (interspersed with some other entries):

[    4.867198] EDAC MC: Ver: 2.1.0
...
[    4.874374] MCE: In-kernel MCE decoding enabled.
[    4.875414] AMD64 EDAC driver v3.4.0
[    4.875438] EDAC amd64: DRAM ECC enabled.
...
[    4.875542] EDAC amd64: CS0: Unbuffered DDR3 RAM
[    4.875545] EDAC amd64: CS1: Unbuffered DDR3 RAM
[    4.875546] EDAC amd64: CS2: Unbuffered DDR3 RAM
[    4.875548] EDAC amd64: CS3: Unbuffered DDR3 RAM

which is a pretty good indication. Manually doing /etc/init.d/edac restart does not create similar log entries, and looking at an older log from a few reboots ago, I see:

[   13.886688] EDAC MC: Ver: 2.1.0
[   13.890389] MCE: In-kernel MCE decoding enabled.
[   13.891082] AMD64 EDAC driver v3.4.0
[   13.891107] EDAC amd64: DRAM ECC disabled.
[   13.891116] EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will not load.
[   13.891117]  Either enable ECC checking or force module loading by setting 'ecc_enable_override'.
[   13.891118]  (Note that use of the override may cause unknown side effects.)

dmidecode --type memory also gives two pretty strong indications: the physical memory array's "error correction type" property (which however for some reason showed the same on non-ECC RAM, so this may be related to the motherboard's support rather than the memory's capabilities),

Handle 0x0026, DMI type 16, 23 bytes
Physical Memory Array
    Location: System Board Or Motherboard
    Use: System Memory
    Error Correction Type: Multi-bit ECC

and each memory device's total width and data width, respectively (the additional bits being those used for the ECC):

Handle 0x0028, DMI type 17, 34 bytes
Memory Device
    Array Handle: 0x0026
    Error Information Handle: Not Provided
    Total Width: 72 bits
    Data Width: 64 bits

Unused RAM is wasted RAM. The Linux kernel has advanced memory management features and tries to avoid putting a burden on the bottleneck in your system, your hard drive/SSD. It tries to cache files in memory.

The memory management system works in complex ways, better performance is the goal.

You can see what it is doing by inspecting /proc/meminfo.

cat /proc/meminfo

You can reclaim this cached memory, using "drop_caches". However, note the documentation says "use outside of a testing or debugging environment is not recommended", simply because "it may cost a significant amount of I/O and CPU to recreate the dropped objects" when they are needed again :-).

Clear PageCache only:

# sync; echo 1 > /proc/sys/vm/drop_caches

Clear dentries and inodes:

# sync; echo 2 > /proc/sys/vm/drop_caches

Clear PageCache, dentries and inodes:

# sync; echo 3 > /proc/sys/vm/drop_caches

Note that sync will flush the file system buffer to ensure all data has been written.

From the kernel docs:

Page cache

The physical memory is volatile and the common case for getting data into the memory is to read it from files. Whenever a file is read, the data is put into the page cache to avoid expensive disk access on the subsequent reads. Similarly, when one writes to a file, the data is placed in the page cache and eventually gets into the backing storage device. The written pages are marked as dirty and when Linux decides to reuse them for other purposes, it makes sure to synchronize the file contents on the device with the updated data.

Reclaim

Throughout the system lifetime, a physical page can be used for storing different types of data. It can be kernel internal data structures, DMA’able buffers for device drivers use, data read from a filesystem, memory allocated by user space processes etc.

Depending on the page usage it is treated differently by the Linux memory management. The pages that can be freed at any time, either because they cache the data available elsewhere, for instance, on a hard disk, or because they can be swapped out, again, to the hard disk, are called reclaimable. The most notable categories of the reclaimable pages are page cache and anonymous memory.

In most cases, the pages holding internal kernel data and used as DMA buffers cannot be repurposed, and they remain pinned until freed by their user. Such pages are called unreclaimable. However, in certain circumstances, even pages occupied with kernel data structures can be reclaimed. For instance, in-memory caches of filesystem metadata can be re-read from the storage device and therefore it is possible to discard them from the main memory when system is under memory pressure.

The process of freeing the reclaimable physical memory pages and repurposing them is called (surprise!) reclaim. Linux can reclaim pages either asynchronously or synchronously, depending on the state of the system. When the system is not loaded, most of the memory is free and allocation requests will be satisfied immediately from the free pages supply. As the load increases, the amount of the free pages goes down and when it reaches a certain threshold (high watermark), an allocation request will awaken the kswapd daemon. It will asynchronously scan memory pages and either just free them if the data they contain is available elsewhere, or evict to the backing storage device (remember those dirty pages?). As memory usage increases even more and reaches another threshold - min watermark - an allocation will trigger direct reclaim. In this case allocation is stalled until enough memory pages are reclaimed to satisfy the request.

Memory Leaks

Now, some programs can have "memory leaks", that is, they "forget" to free up memory they no longer use. You can see this if you leave a program running for some time, its memory usage constantly increases, when you close it, the memory is never freed. Now, programmers try to avoid memory leaks, of course, but programs can have some. The way to reclaim this memory is a reboot.