Files – Why ls -lh Reports Total Size Less Than Sum of Individual Sizes

disk-usagefilesls

Under what circumstances would ls -lh show a total that is less than the sum of the individual files? For example:

$ ls -lh /var/lib/nova/instances/_base
total 100G
-rw-rw-r-- 1 nova         nova 4.3M 2012-02-14 14:07 00000001
-rw-rw-r-- 1 nova         nova 5.7M 2012-02-14 14:07 00000002
-rw-rw-r-- 1 nova         nova  42G 2012-03-08 15:24 1574bddb75c78a6fd2251d61e2993b5146201319.part
-rw-rw-r-- 1 libvirt-qemu kvm   24M 2012-02-14 14:07 77de68daecd823babbb58edb1c8e14d7106e83bb_sm
-rw-r--r-- 1 libvirt-qemu kvm   65G 2012-03-02 12:43 bd307a3ec329e10a2cff8fb87480823da114f8f4
-rw-rw-r-- 1 libvirt-qemu kvm  160G 2012-02-24 16:06 ephemeral_0_160_None
-rw-rw-r-- 1 libvirt-qemu kvm   80G 2012-02-24 22:38 ephemeral_0_80_None
-rw-r--r-- 1 libvirt-qemu kvm   10G 2012-02-24 22:37 fe5dbbcea5ce7e2988b8c69bcfdfde8904aabc1f
-rw-r--r-- 1 libvirt-qemu kvm   10G 2012-02-24 11:09 fe5dbbcea5ce7e2988b8c69bcfdfde8904aabc1f_sm

Edit: now showing some extra flags are per request in comment: 

$ ls -aiFlh  /var/lib/nova/instances/_base/
total 143G
29884440 drwxrwxr-x 2 nova         nova 4.0K 2012-03-08 15:45 ./
29884427 drwxr-xr-x 6 nova         nova 4.0K 2012-03-08 15:05 ../
29884444 -rw-rw-r-- 1 nova         nova 4.3M 2012-02-14 14:07 00000001
29884445 -rw-rw-r-- 1 nova         nova 5.7M 2012-02-14 14:07 00000002
29884468 -rw-r--r-- 1 nova         nova  65G 2012-03-08 15:59 1574bddb75c78a6fd2251d61e2993b5146201319.converted
29884466 -rw-rw-r-- 1 nova         nova  58G 2012-03-08 15:35 1574bddb75c78a6fd2251d61e2993b5146201319.part
29884446 -rw-rw-r-- 1 libvirt-qemu kvm   24M 2012-02-14 14:07 77de68daecd823babbb58edb1c8e14d7106e83bb_sm
29884467 -rw-r--r-- 1 libvirt-qemu kvm   65G 2012-03-02 12:43 bd307a3ec329e10a2cff8fb87480823da114f8f4
29884443 -rw-rw-r-- 1 libvirt-qemu kvm  160G 2012-02-24 16:06 ephemeral_0_160_None
29884442 -rw-rw-r-- 1 libvirt-qemu kvm   80G 2012-02-24 22:38 ephemeral_0_80_None
29884447 -rw-r--r-- 1 libvirt-qemu kvm   10G 2012-02-24 22:37 fe5dbbcea5ce7e2988b8c69bcfdfde8904aabc1f
29884441 -rw-r--r-- 1 libvirt-qemu kvm   10G 2012-02-24 11:09 fe5dbbcea5ce7e2988b8c69bcfdfde8904aabc1f_sm

Best Answer

It will happen if you have sparse files:

$ mkdir test; cd test
$ truncate -s 1000000000 file-with-zeroes
$ ls -l
total 0
-rw-r--r-- 1 gim gim 1000000000 03-08 22:18 file-with-zeroes

A sparse file is a file which has not been populated with filesystem blocks (or only partially). When you read a non-populated zone of a sparse file you will obtain zeros. Such blank zones do not require actual disk space, and the 'total' reported by ls corresponds to the disk space occupied by the files (just like du).

Related Question