Linux – Root file system is mounted read-only on boot on Gentoo Linux

bootfilesystemsgentoolinuxmount

I am using Gentoo Linux and for a while now, the root file system is mounted read-only on booting. For obvious reasons, this is quite annoying as most services do not start up correctly (I do not use a separate file system for /var). After the system is up, I have to log in, remount the root file system read-write, fix /etc/mtab, mount all other file systems in from /etc/fstab and then start up all the missing daemons. I know that there are ways to make a system run properly with a read-only file system, but I would rather restore the old behaviour of a writable root file system.

The strange thing is that after running mount / -o remount,rw, the file system is mounted in writable mode without any errors. I suspected some problem with fsck, but now I have disabled automatic file system checks on the partition (tune2fs -c0 -i0).

When I run dmesg, only these lines mention the partition at all, although I am not sure if not something gets lost because /var/log is not writable:

EXT3-fs (sda5): mounted filesystem with writeback data mode</code>  
EXT3-fs (sda5): using internal journal

The line in /etc/fstab looks like this:

/dev/sda5 / ext3 noatime 0 1

I am using the kernel 2.6.34-gentoo-r6 (the same problem existed with a previous 2.6.31 kernel). I have created it using genkernel 3.4.10.906. My grub configuration looks like this:

title=Gentoo Linux (2.6.34-gentoo-r6)
root (hd0,0)
kernel /kernel-genkernel-x86_64-2.6.34-gentoo-r6 root=/dev/ram0 real_root=/dev/sda5 vga=792 CONSOLE=/dev/tty1 resume=/dev/sda6
initrd /initramfs-genkernel-x86_64-2.6.34-gentoo-r6

Apart from that, I run baselayout 2.0.0 with openrc 0.6.3, if that is any important. sysvinit 2.87-r3 is also installed, I don’t know if it’s actually used.

Here is the output of dumpe2fs:

Filesystem volume name:   hd-root
Last mounted on:          <not available>
Filesystem UUID:          387432ca-2464-4c61-ba15-11c4af1c0418
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              1528912
Block count:              6104692
Reserved block count:     0
Free blocks:              413799
Free inodes:              674036
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1022
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8176
Inode blocks per group:   511
Filesystem created:       Tue Dec  9 14:48:56 2008
Last mount time:          Mon Sep 27 00:00:15 2010
Last write time:          Sun Sep 26 23:55:12 2010
Mount count:              39
Maximum mount count:      -1
Last checked:             Sun Sep 26 23:51:51 2010
Check interval:           0 (<none>)
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Journal inode:            8
First orphan inode:       698281
Default directory hash:   tea
Directory Hash Seed:      4229715b-4ad1-4285-940b-9960db1cb4e1
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             128M
Journal length:           32768
Journal sequence:         0x003d9991
Journal start:            1

I have no idea what could cause this problem. I cannot find any error messages, and searching the internet, I only find manuals how to deliberately mount the root filesystem read-

Best Answer

The solution for the problem is easier than expected. Appearantly in baselayout 2, there is a service called root that just remounts the root file-system read-write.

The reason why I didn’t have this service in my boot runlevel is probably that I have updated to baselayout 2 rather early, and probably this service (and several others) was introduced later. The following services are in my boot runlevel now, and everything works:

bootmisc
consolefont
device-mapper
dmcrypt
fsck
hostname
hwclock
keymaps
localmount
modules
net.lo
netmount
network
procfs
root
svscan
swap
sysctl
sysfs
termencoding
urandom

Related Solutions

Linux – Ext3 drive will not mount after power failure; how to recover files

The problems you're having sound far more extensive than what I'd expect from mere loss of power (even during fairly heavy write activity) on a device. I have to wonder if you're really having more problems at the interface/driver level, or a corrupted partition table or something of that sort.

From the sounds of things you may have exacerbated the problem further with all the thrashing around you've done while trying to fix the issue.

I don't know if we can help with this case but don't give up yet.

For the future I'd suggest that you learn the following technique:

When you have trouble with a drive under Linux or UNIX you can usually use dd to make a bit-image copy of the whole device to some other location. Find a drive that's at least as large as the one in question and try a command like: dd if=$PROBLEMATIC of=$TARGET bs=4M ... be very careful about the if (input file) and of (output file) directives. Leave that run. It's a good idea to run tail -f /var/log/messages & (or possible variant as appropriate to your /etc/syslog.conf) ... either do that in the background or in another window. There are enhanced versions of dd which can handle retries and continuing past bad blocks more robustly (sdd is a name that comes to mind). But try just using the stock GNU dd command at first.

You can make such a copy of the whole device (/dev/sdd, for example) or just the partition (/dev/sdd1). If you get "short read or similar errors then it suggests that either the device has physical errors preventing reads past certain cylinders or, in the case of a partition, that the partition table is mangled in some way. You can even make two different dd images ... one of each.

Here's the trick: do all your fsck and mount attempts, and use your various other recovery tools such as TCT (The Coroner's Toolkit) on the copied image!

This minimizes the time spent running the drive (which is possibly degrading at the hardware level as you operate it) and minimizes the impact of failed and possibly misguided recovery attempts. (In some situations you make one image, then another based on that and always operate on the tertiary image ... depends on how much the data is worth).

I personally suggest that you run something like hexdump or strings to read through the image ... just let it scroll past for a long time and look for plain text that looks like it might be fragments of your data. I have used grep to recover useful (textual) data from otherwise completely mangled filesystems. In case I'm not suggesting it as data recovery heroics ... but as a sanity check. If you scroll through 10s of megabytes or a few gigabytes of data and don't see any recognizable text ... then you probably have a hopeless case or you've done something very wrong (were you really careful about those if= and of= options?).

I don't know if any of this will help you with the current effort. But learn these tricks now and they will definitely make your next foray into data recovery much less scary. (Yes, practice on a healthy system once or twice --- go use a hex editor and try adding your own creative corruption here and there --- to the COPY of course! Then try fix it).

Oh, and this is a really good time to review your backup and data recovery plans and procedures (or provide better advice to your customer/colleague/client/friend/whatever).

UNIX – mount: only root can do that

When mounting a volume as a regular user, you only need to provide the mount point name, i.e.:

mount /mnt/sdcard

The /etc/fstab file provides the device that will be used.

Best Answer

Related Solutions

Linux – Ext3 drive will not mount after power failure; how to recover files

UNIX – mount: only root can do that

Related Question