Linux – Are short-lived files flushed to disk

diskext4linux

My program creates many small short-lived files. They are typically deleted within a second after creation. The files are in an ext4 file system backed by a real hard disk. I know that Linux periodically flushes (pdflush) dirty pages to disk. Since my files are short-lived, most likely they are not cached by pdflush. My question is, does my program cause a lot of disk writes? My concern is my hard disk's life.

Since the files are small, let's assume the sum of their size is smaller than dirty_bytes and dirty_background_bytes.

Ext4 has default journal turned on, i.e. metadata journal. I also want to know whether the metadata or the data is written to disk.

Best Answer

A simple experiment using ext4:

Create a 100MB image...

# dd if=/dev/zero of=image bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 0.0533049 s, 2.0 GB/s

Make it a loop device...

# losetup -f --show image
/dev/loop0

Make filesystem and mount...

# mkfs.ext4 /dev/loop0
# mount /dev/loop0 /mnt/tmp

Make some kind of run with short lived files. (Change this to any method you prefer.)

for ((x=0; x<1000; x++))
do
    (echo short-lived-content-$x > /mnt/tmp/short-lived-file-$x
     sleep 1
     rm /mnt/tmp/short-lived-file-$x ) &
done

Umount, sync, unloop.

# umount /mnt/tmp
# sync
# losetup -d /dev/loop0

Check the image contents.

# strings image | grep short-lived-file | tail -n 3
short-lived-file-266
short-lived-file-895
short-lived-file-909
# strings image | grep short-lived-content | tail -n 3

In my case it listed all the file names, but none of the file contents. So only the contents were not written.

Related Question