Convert old /home from ext3 to ext4

conversionext3ext4home

I have an old /home partition, that dates back to former linux systems, and it is still in ext3 format. Whereas the rest of my system, / and some other mounted point are devices formated in ext4.

I have grasped some sites on the net that describes how to convert an ext3 partition to an ext4.

In this UL.SE question Can I convert an ext3 partition into ext4 without formatting?, there are also warnings recommending backup of the data before convertion… if ever…

So I wonder if is generally a good idea to convert an existing ext3 partition to ext4. I know it's possible, I know there is a little risk that need a back up if ever. Are there enough benefits such that I should do it ?

Best Answer

Both ext3 and ext4 are journaling filesystems, in addition this list several differences, the most relevant are:

Maximum individual file size can be from 16 GB to 16 TB
Overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte). 1 PB = 1024 TB (terabyte).
Directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3)
Several other new features are introduced in ext4: multiblock allocation, delayed allocation, journal checksum. fast fsck, etc. All you need to know is that these new features have improved the performance and reliability of the filesystem when compared to ext3.

The interesting thing for you might be the faster fsck, the others are probably of less significance in this particular situation (unless your disk gets a growth spurt and magically can contain much larger files).

If you are not going to use that partition intensively I would not recommend converting (at least not without a backup).

Related Solutions

How are files laid out in ext2/ext3/ext4

PhotoRec scans a disk (or disk image) searching for contiguous chunks of bytes-that-look-like-known-file-formats (for example, it can recognize JFIF/EXIF (JPEG) by the segment headers). Pretty simple but limited.

The Sleuth Kit is a great tool for digging into filesystems. With a bit of care (and scripting its tools and hex-editing the disk image when it goes astray), it can be used for recovery.

For a tool that more deeply understands ext, try ext4magic? (I haven't yet had a need to use this yet.)

Documentation/filesystems/ext2.txt in the kernel sources has a high-level overview of the general structure. The Ext4 Wiki has good information, including Ext4 Disk Layout containing more details (largely applying to ext[23] as well).

But yes, a file's data is split up into blocks. In ext2, each file is represented by an inode which contains direct blocks (pointers to data blocks), indirect blocks (which contain pointers to direct blocks), double indirect blocks, and triple indirect blocks. There are no backlinks, so to find a data block's siblings, you'll have to scan all inodes and block pointers to find its owner first.

Reliable way to detect ext2 or ext3 or ext4

After looking at the code for various utilities and the kernel code for some time, it does seem that what @Hauke suggested is true - whether a filesystem is ext2/ext3/ext4 is purely defined by the options that are enabled.

From the Wikipedia page on ext4:

Backward compatibility

ext4 is backward compatible with ext3 and ext2, making it possible to mount ext3 and ext2 as ext4. This will slightly improve performance, because certain new features of ext4 can also be used with ext3 and ext2, such as the new block allocation algorithm.

ext3 is partially forward compatible with ext4. That is, ext4 can be mounted as ext3 (using "ext3" as the filesystem type when mounting). However, if the ext4 partition uses extents (a major new feature of ext4), then the ability to mount as ext3 is lost.

As most probably already know, there is similar compatibility between ext2 and ext3.

After looking at the code which blkid uses to distinguish different ext filesystems, I was able to turn an ext4 filesystem into something recognised as ext3 (and from there to ext2). You should be able to repeat this with:

truncate -s 100M testfs
mkfs.ext4 -O ^64bit,^extent,^flex_bg testfs <<<y
blkid testfs
tune2fs -O ^huge_file,^dir_nlink,^extra_isize,^mmp testfs
e2fsck testfs
tune2fs -O metadata_csum testfs
tune2fs -O ^metadata_csum testfs
blkid testfs
./e2fsprogs/misc/tune2fs -O ^has_journal testfs
blkid testfs

First blkid output is:

testfs: UUID="78f4475b-060a-445c-a5d2-0f45688cc954" SEC_TYPE="ext2" TYPE="ext4"

Second is:

testfs: UUID="78f4475b-060a-445c-a5d2-0f45688cc954" SEC_TYPE="ext2" TYPE="ext3"

And the final one:

testfs: UUID="78f4475b-060a-445c-a5d2-0f45688cc954" TYPE="ext2"

Note that I had to use a new version of e2fsprogs than was available in my distro to get the metadata_csum flag. The reason for setting, then clearing this was because I found no other way to affect the underlying EXT4_FEATURE_RO_COMPAT_GDT_CSUM flag. The underlying flag for metadata_csum (EXT4_FEATURE_RO_COMPAT_METADATA_CSUM) and EXT4_FEATURE_RO_COMPAT_GDT_CSUM are mutually exclusive. Setting metadata_csum disables EXT4_FEATURE_RO_COMPAT_GDT_CSUM, but un-setting metadata_csum does not re-enable the latter.

Conclusions

Lacking a deep knowledge of the filesystem internals, it seems either:

Journal checksumming is meant to be a defining feature of a filesystem created as ext4 that you are really not supposed to disable and that fact that I have managed this is really a bug in e2fsprogs. Or,
All ext4 features were always designed to be disabled and disabling them does make the filesystem to all intents an purposes an ext3 filesystem.

Either way a high level of compatibility between the filesystems is clearly a design goal, compare this to ReiserFS and Reiser4 where Reiser4 is a complete redesign. What really matters is whether the features present are supported by the driver that is used to mount the system. As the Wikipedia article notes the ext4 driver can be used with ext3 and ext2 as well (in fact there is a kernel option to always use the ext4 driver and ditch the others). Disabling features just means that the earlier drivers will have no problems with the filesystem and so there are no reasons to stop them from mounting the filesystem.

Recommendations

To distinguish between the different ext filesystems in a C program, libblkid seems to be the best thing to use. It is part of util-linux and this is what the mount command uses to try to determine the filesystem type. API documentation is here.

If you have to do your own implementation of the check, then testing the same flags as libblkid seems to be the right way to go. Although notably the file linked has no mention of the EXT4_FEATURE_RO_COMPAT_METADATA_CSUM flag which appears to be tested in practice.

If you really wanted to go the whole hog, then looking at for journal checksums might be a surefire way of finding if a filesystem without these flags is (or perhaps was) ext4.

Update

It is actually somewhat easier to go in the opposite direction and promote an ext2 filesystem to ext4:

truncate -s 100M test
mkfs.ext2 test
blkid test
tune2fs -O has_journal test
blkid test
tune2fs -O huge_file test
blkid test

The three blkid ouputs:

test: UUID="59dce6f5-96ed-4307-9b39-6da2ff73cb04" TYPE="ext2"

test: UUID="59dce6f5-96ed-4307-9b39-6da2ff73cb04" SEC_TYPE="ext2" TYPE="ext3"

test: UUID="59dce6f5-96ed-4307-9b39-6da2ff73cb04" SEC_TYPE="ext2" TYPE="ext4"

The fact that ext3/ext4 features can so easily by enabled on a filesystem that started out as ext2 is probably the best demonstration that the filesystem type really is defined by the features.