Filesystem for archiving

backupfilesystemshard link

I have some complex read-only data in my file system. It contains thousands of snapshots of certain revisions of a svn repository, and the output of regression tests. Identical files between snapshots are already de-duplicated using hard links. This way, the storage capacity doesn't need to be large, but it still consumes a lot of inodes, and this makes fsck painfully long for my main file system.

I'd like to move these data to another file system, so that it doesn't affect the main file system too much. Do you have suggestions? Squashfs seems to be a possible choice, but I'll have to check if it can handle hard links efficiently.

Best Answer

If it's abot fsck slowness, did you try ext4? They added a few features to it that make fsck really quick by not looking at unused inodes:

Fsck is a very slow operation, especially the first step: checking all the inodes in the file system. In Ext4, at the end of each group's inode table will be stored a list of unused inodes (with a checksum, for safety), so fsck will not check those inodes. The result is that total fsck time improves from 2 to 20 times, depending on the number of used inodes (http://kerneltrap.org/Linux/Improving_fsck_Speeds_in_Ext4). It must be noticed that it's fsck, and not Ext4, who will build the list of unused inodes. This means that you must run fsck to get the list of unused inodes built, and only the next fsck run will be faster (you need to pass a fsck in order to convert a Ext3 filesystem to Ext4 anyway). There's also a feature that takes part in this fsck speed up - "flexible block groups" - that also speeds up filesystem operations.

Related Question