The Mem: total figure is the total amount of RAM that can be used by applications. This is the total RAM installed on the system, minus:
- memory reserved by hardware devices (often video memory if the graphics card doesn't have its own RAM);
- memory used by the kernel itself.
That total includes:
free: memory that is currently used for any purpose;shared: a concept that no longer exists. It's left in the output for backward compatibility (there are scripts that parse the output from free). (On current systems you'll typically see nonzero values because shared has been repurposed to show memory that's explicitly shared via a shared memory mechanism. On older systems, it included files mapped by more than one process and shareable memory that remained shared after fork().)buffers: memory that is backed by files, and that can be written out to disk if needed;cache: memory that is backed by files, and that can be reclaimed at any time (the difference with buffers is that buffers must be saved to disk before they're reused, whereas cache consists of things that can be reloaded from disk);used -buffers/cache: memory used by applications (and not paged out to swap).
In a pinch, the system could run without buffers and cache, reserving RAM for applications and systematically performing disk reads and writes without any caching. The -/+ buffers/cache figures indicate the amount of RAM used directly by applications (used column) and the amount of RAM not used by applications (free column).
Although this can vary a lot, a healthy system typically has around half its RAM devoted to applications and half devoted to buffers and cache. Unless you're running a dedicated file server, your system has more RAM than it needs for what you're currently doing. If the free - buffers/cache figure was low, that would indicate a system that doesn't have enough RAM (contrary to a widespread belief, having a lot of memory devoted to buffers and cache is important for system performance, and trying to reserve more memory for applications would make 99.99% of systems slower).
The swap line is straightforward, it shows the amount of swap that's in use (either by applications or for tmpfs storage), and the amount that isn't.
Seeing another of your post I guess you are using zram. So that will be my assumption here.
I did the experience to install zram and consume lot of memory, and I got the same output of smem than you. smem does not take into account zram into its counting, it only uses /proc/meminfo to compute its value, and if you look and try to understand the code you will see that the zram RAM occupation is gets in the end counted under the noncache column of the kernel dynamic memory line.
Further investigations
Following my gut feeling that zram was behind this behavious, I setted up a VM with similar spec as your machine: 4 GB RAM and 2 GB zram swap, no swap file.
I have loaded the VM with heavy weight applications and got the following state:
huygens@ubuntu:~$ smem -wt -K ~/vmlinuz-3.2.0-38-generic.unpacked -R 4096M
Area Used Cache Noncache
firmware/hardware 130717 0 130717
kernel image 13951 0 13951
kernel dynamic memory 1063520 922172 141348
userspace memory 2534684 257136 2277548
free memory 451432 451432 0
----------------------------------------------------------
4194304 1630740 2563564
huygens@ubuntu:~$ free -m
total used free shared buffers cached
Mem: 3954 3528 426 0 79 858
-/+ buffers/cache: 2589 1365
Swap: 1977 0 1977
As you can see free reports 858 MB cache memory and that is also what smem seems to report within the cached kernel dynamic memory.
Then I further stressed the system using Chromium Browser. At the beginning, it was only have 83 MB of swap used. But then after a few more tabs opened, the swap switch quickly to almost it's maximum and I experienced OOM! zram has really a dangerous side where wrongly configured (too big sizes) it can quickly hit you back like a trebuchet-like mechanism.
At that time I had the following outputs:
huygens@ubuntu:~$ smem -wt -K ~/vmlinuz-3.2.0-38-generic.unpacked -R 4096M
Area Used Cache Noncache
firmware/hardware 130717 0 130717
kernel image 13951 0 13951
kernel dynamic memory 1355344 124072 1231272
userspace memory 961004 36456 924548
free memory 1733288 1733288 0
----------------------------------------------------------
4194304 1893816 2300488
huygens@ubuntu:~$ free -m
total used free shared buffers cached
Mem: 3954 2256 1698 0 4 132
-/+ buffers/cache: 2118 1835
Swap: 1977 1750 227
See how the kernel dynamic memory (columns cache and non-cache) look like inverted? It is because in the first case, the kernel had "cached" memory such as reported by free but then it had swap memory held by zram which smem does not know how to compute (check smem source code, zram occupation is not reported in /proc/meminfo, this it is not computed by smem which does simple "total kernel mem" - "type of memory reported by meminfo that I know are cache", what it does not know is that in the computed total kernel mem it has added the size of the swap which is in RAM!)
When I was in this state, I activated a hard disk swap and turned off the zram swap and I reset the zram devices: echo 1 > /sys/block/zram0/reset.
After that the noncache kernel memory melted like snow in summer and returned to "normal" value.
Conclusion
smem does not know about zram (yet) maybe because it is still staging and thus not part of /proc/meminfo which reports global parameters (like (in)active pages size, total memory) and then only report on a few specific parameters. smem identified a few of this specific parameters as "cache", sum them up and compare that to total memory. Because of that zram used memory gets counted in the noncache column.
Note: by the way, in modern kernel, meminfo reports also the shared memory consumed. smem does not take that yet into account, so even without zram the output of smem is to consider carefully esp. if you use application that make big use of shared memory.
References used:
Best Answer
MemAvailableis included in/proc/meminfosince version 3.14 of the kernel; it was added by commit 34e431b0a. That's the determining factor in the output variations you show. The commit message indicates how to estimate available memory withoutMemAvailable:The low watermarks are the level beneath which the system will swap. So in the absence of
MemAvailableyou can at least add up the values given forMemFree,Active(file),Inactive(file)andSReclaimable(whichever are present in/proc/meminfo), and subtract the low watermarks from/proc/zoneinfo. The latter also lists the number of free pages per zone, that might be useful as a comparison...The complete algorithm is given in the patch to
meminfo.cand seems reasonably easy to adapt:MemFree);Active(file)andInactive(file)): that's the amount of memory used by the page cache, minus either half the page cache, or the low watermark, whichever is smaller;SReclaimable), following the same algorithm.So, putting all this together, you can get the memory available for a new process with: