Seeing another of your post I guess you are using zram. So that will be my assumption here.
I did the experience to install zram and consume lot of memory, and I got the same output of smem
than you. smem
does not take into account zram
into its counting, it only uses /proc/meminfo
to compute its value, and if you look and try to understand the code you will see that the zram RAM occupation is gets in the end counted under the noncache column of the kernel dynamic memory line.
Further investigations
Following my gut feeling that zram was behind this behavious, I setted up a VM with similar spec as your machine: 4 GB RAM and 2 GB zram swap, no swap file.
I have loaded the VM with heavy weight applications and got the following state:
huygens@ubuntu:~$ smem -wt -K ~/vmlinuz-3.2.0-38-generic.unpacked -R 4096M
Area Used Cache Noncache
firmware/hardware 130717 0 130717
kernel image 13951 0 13951
kernel dynamic memory 1063520 922172 141348
userspace memory 2534684 257136 2277548
free memory 451432 451432 0
----------------------------------------------------------
4194304 1630740 2563564
huygens@ubuntu:~$ free -m
total used free shared buffers cached
Mem: 3954 3528 426 0 79 858
-/+ buffers/cache: 2589 1365
Swap: 1977 0 1977
As you can see free
reports 858 MB cache memory and that is also what smem
seems to report within the cached kernel dynamic memory.
Then I further stressed the system using Chromium Browser. At the beginning, it was only have 83 MB of swap used. But then after a few more tabs opened, the swap switch quickly to almost it's maximum and I experienced OOM! zram
has really a dangerous side where wrongly configured (too big sizes) it can quickly hit you back like a trebuchet-like mechanism.
At that time I had the following outputs:
huygens@ubuntu:~$ smem -wt -K ~/vmlinuz-3.2.0-38-generic.unpacked -R 4096M
Area Used Cache Noncache
firmware/hardware 130717 0 130717
kernel image 13951 0 13951
kernel dynamic memory 1355344 124072 1231272
userspace memory 961004 36456 924548
free memory 1733288 1733288 0
----------------------------------------------------------
4194304 1893816 2300488
huygens@ubuntu:~$ free -m
total used free shared buffers cached
Mem: 3954 2256 1698 0 4 132
-/+ buffers/cache: 2118 1835
Swap: 1977 1750 227
See how the kernel dynamic memory (columns cache and non-cache) look like inverted? It is because in the first case, the kernel had "cached" memory such as reported by free
but then it had swap memory held by zram
which smem
does not know how to compute (check smem source code, zram occupation is not reported in /proc/meminfo, this it is not computed by smem
which does simple "total kernel mem" - "type of memory reported by meminfo that I know are cache", what it does not know is that in the computed total kernel mem it has added the size of the swap which is in RAM!)
When I was in this state, I activated a hard disk swap and turned off the zram swap and I reset the zram devices: echo 1 > /sys/block/zram0/reset
.
After that the noncache kernel memory melted like snow in summer and returned to "normal" value.
Conclusion
smem
does not know about zram
(yet) maybe because it is still staging and thus not part of /proc/meminfo
which reports global parameters (like (in)active pages size, total memory) and then only report on a few specific parameters. smem
identified a few of this specific parameters as "cache", sum them up and compare that to total memory. Because of that zram
used memory gets counted in the noncache column.
Note: by the way, in modern kernel, meminfo
reports also the shared memory consumed. smem
does not take that yet into account, so even without zram
the output of smem
is to consider carefully esp. if you use application that make big use of shared memory.
References used:
Best Answer
You really ought not to ask two questions in one, but...
Question 1
Some of that memory is used for the kernel code itself, some is reserved, etc. The kernel spits it out in the system boot messages:
The "absent" line is memory that isn't actually there (this machine currently has 6GiB of RAM installed). The kernel also spits out the memory map (this is earlier in the boot messages):
The kernel then does various fixups to that map, usually reserving more memory. Especially as the drivers load.
Question 2
The kernel/user split is of virtual address space, not memory. Its pretty much irrelevant on a 64-bit box, because there is so much address space to go around.
On a 32-bit box, virtual addresses 0x00000000–0xBFFFFFFF were used for user address space. 0xC0000000–0xFFFFFFFF were used by the kernel (that's the 3:1 split, other options inlcuded a 2:2 split. Note those numbers are gigabytes, so it's 2:2 not 1:1). Virtual addresses are process-specific, too (each process can have a page at 0x00001000, and it's a different page).
But a virtual address doesn't correspond to a byte of memory. It can be backed by basically four things:
Transparent huge pages makes more cases. There are probably a few less-important ones I've forgotten, too...
Anyway, my 64-bit chip has a 48-bit virtual address size. I'm not sure what split the kernel uses, but even if its half, that's 47 bits of space, well in excess of the 36-bit physical address size. And 131,072 GiB of RAM is too expensive... (And, remember, when it gets cheaper, there are a lot of bits left in 64, future processors will probably just allow more of them).