To control how Linux caches things refer to this
https://www.kernel.org/doc/Documentation/sysctl/vm.txt
In particular look at vfs_cache_pressure, you probably want a really low value or maybe even zero (1 sounds a bit safer to me though):
vfs_cache_pressure
------------------
Controls the tendency of the kernel to reclaim the memory which is used for
caching of directory and inode objects.
At the default value of vfs_cache_pressure=100 the kernel will attempt to
reclaim dentries and inodes at a "fair" rate with respect to pagecache and
swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer
to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will
never reclaim dentries and inodes due to memory pressure and this can easily
lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100
causes the kernel to prefer to reclaim dentries and inodes.
Also you may want to modify swappiness
so that you never swap data or make it so that it only happens in extreme cases.
The drop_caches
option might be handy for explicitly dropping the data you don't want cached anymore.
I'm sure there are probably other options that may help, so review the kernel documentation.
To apply them I'd put the settings you want to change in /etc/sysctl.conf
or whatever your OS has to restore them at boot.
I think I know how it works.
I connected another disk to my machine because it has a big almost empty partition ~458G . I checked its free space via e2freefrag
:
HISTOGRAM OF FREE EXTENT SIZES:
Extent Size Range : Free extents Free Blocks Percent
64M... 128M- : 6 146233 0.12%
128M... 256M- : 5 322555 0.27%
256M... 512M- : 3 263897 0.22%
512M... 1024M- : 6 1159100 0.98%
1G... 2G- : 228 116312183 98.40%
It's just a contiguous free blocks. So because the partition is almost empty, there's lots of free space and you have 228 chunks of 1-2G.
I placed a big 2,5G file inside of the partition, and the table above changed a little bit:
HISTOGRAM OF FREE EXTENT SIZES:
Extent Size Range : Free extents Free Blocks Percent
2M... 4M- : 5 5114 0.00%
64M... 128M- : 7 170777 0.14%
128M... 256M- : 1 64511 0.05%
256M... 512M- : 4 361579 0.31%
512M... 1024M- : 5 930749 0.79%
1G... 2G- : 227 116025495 98.16%
This doesn't tell anything about the allocated block extents, but it gave me some ideas. When I looked at the file in e4defrag
, there was something like this:
# e4defrag -cv file
<File>
[ext 1]: start 34816: logical 0: len 32768
[ext 2]: start 67584: logical 32768: len 30720
[ext 3]: start 100352: logical 63488: len 32768
[ext 4]: start 133120: logical 96256: len 30720
[ext 5]: start 165888: logical 126976: len 32768
[ext 6]: start 198656: logical 159744: len 30720
[ext 7]: start 231424: logical 190464: len 32768
[ext 8]: start 264192: logical 223232: len 30720
[ext 9]: start 296960: logical 253952: len 32768
[ext 10]: start 329728: logical 286720: len 32768
[ext 11]: start 362496: logical 319488: len 32768
[ext 12]: start 395264: logical 352256: len 32768
[ext 13]: start 428032: logical 385024: len 32768
[ext 14]: start 460800: logical 417792: len 32768
[ext 15]: start 493568: logical 450560: len 30720
[ext 16]: start 557056: logical 481280: len 32768
[ext 17]: start 589824: logical 514048: len 32768
[ext 18]: start 622592: logical 546816: len 32768
[ext 19]: start 655360: logical 579584: len 32768
[ext 20]: start 688128: logical 612352: len 32768
[ext 21]: start 720896: logical 645120: len 622
The number 32768
means blocks (4K), which equals to 128MiB. Some of them have fewer blocks and I don't know why because the filesystem is empty and I think all the extents should have 32768 blocks.
Anyway I checked the main partition to see its free space, and there was something like this:
HISTOGRAM OF FREE EXTENT SIZES:
Extent Size Range : Free extents Free Blocks Percent
4K... 8K- : 3955 3955 0.06%
8K... 16K- : 3495 8194 0.13%
16K... 32K- : 2601 13165 0.20%
32K... 64K- : 2622 28991 0.45%
64K... 128K- : 2565 58267 0.90%
128K... 256K- : 1576 71371 1.11%
256K... 512K- : 1331 118346 1.83%
512K... 1024K- : 1058 190532 2.95%
1M... 2M- : 1202 444210 6.89%
2M... 4M- : 1211 884489 13.71%
4M... 8M- : 1249 1803998 27.97%
8M... 16M- : 622 1643226 25.48%
16M... 32M- : 198 1024999 15.89%
32M... 64M- : 16 163082 2.53%
As you can see, there's no free contiguous blocks that could provide 128M (and more) space and that's why they've written on the wiki that you can have extents "up to" 128M.
I'm not sure why the file in question has 10 extents because there's still 16 chunks that are at least 32M.
Best Answer
e4fsck
supports-D
flag which seems to do what you want:Of course, you'll need to unmount the filesystem to use
fsck
, meaning downtime for your server.You'll want to use the
-f
option to make suree4fsck
processes the file system even if clean.Testing: