Linux – How and where is file system meta data cached in Linux

cachefilesystemslinux

My earlier question apparently had a wrong premise. I thought file system meta data was cached in the inode and dentry cache, but apparently not.

When I do:

# time find . > /dev/null

real    10m4.435s
user    0m3.904s
sys     0m15.505s

# time find . > /dev/null

real    0m5.681s
user    0m1.400s
sys     0m4.224s

You can see the second run is a lot faster. But, it reverts back to 10 minutes when I free the page cache only:

echo 1 > /proc/sys/vm/drop_caches

Slabtop still shows a big cache after dropping those caches:

   OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
1758897 1758713  99%    0.19K  83757       21    335028K dentry
1216908 1211861  99%    0.76K  57948       21    927168K ext3_inode_cache

I don't get how that works anyway, because after dropping all caches (echo 3 > /proc/sys/vm/drop_caches), slabtop still reports a big cache size:

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
1288434 955168  74%    0.19K  61354       21    245416K dentry                 
1216908 1211861  99%    0.76K  57948       21    927168K ext3_inode_cache

So, is the data obtained by find stored in the page cache? I would like to store that data in cache permanently, to not have disk IO because of daily backups, but if it's stored in the page_cache, I can't use vfs_cache_pressure to control it.

Edit:

When I do a find and echo 2 to drop_caches (which would drop the dentry and inode cache), find is still fast, and /proc/meminfo still shows slab use:

# time find . > /dev/null    
real    8m11.918s
user    0m3.888s
sys     0m15.313s

# echo 2 > /proc/sys/vm/drop_caches

# time find . > /dev/null
real    0m8.883s
user    0m1.540s
sys     0m4.724s

And meminfo:

# cat /proc/meminfo |grep -i "^cache\|Slab"
Cached:           425224 kB
Slab:             891648 kB

echo 2 > /proc/sys/vm/drop_caches

# cat /proc/meminfo |grep -i "^cache\|Slab"
Cached:           333740 kB
Slab:             793428 kB

It's ubuntu 12.04 on ext3.

Best Answer

This is because your test is flawed. Running find . merely calls getdents() on the directory tree. A directory in this case is just a file that contains directory entries and is thus stored in page cache. Note you do nothing to actually access the files you are attempting to cache in this manner.

Your test is basically caching all the directories in the filesystem tree and nothing else.

Try your test giving find a harder task to do. Such as forcing it to call lstat for every file and you'll see a different behaviour.

In my test there are a million files in this directory.

[root@home test]# echo 3 >/proc/sys/vm/drop_caches 
[root@home test]# time find ./tmp -printf "%p %c\n" >/dev/null

real    0m16.443s
user    0m2.123s
sys 0m9.320s
[root@home test]# time find ./tmp -printf "%p %c\n" >/dev/null

real    0m2.704s
user    0m1.224s
sys 0m1.479s
[root@home test]# echo 1 >/proc/sys/vm/drop_caches 
[root@home test]# time find ./tmp -printf "%p %c\n" >/dev/null

real    0m3.791s
user    0m1.359s
sys 0m1.756s

Note in the last test it takes slightly longer, I imagine this is directly related to all the reading of the directory 'file' itself rather than because the dentry and inode cache does not exist.

Related Question