Ubuntu – Error performing sudo apt update in persistent live boot system (ubuntu 16.04 LTS)

aptdependenciesdpkgpersistenceupdates

I had the following issue while performing sudo apt-get update while trying to install opencv and python packages:

root@ubuntu:~# apt-get update
Ign:1 cdrom://Ubuntu 16.04.2 LTS _Xenial Xerus_ - Release amd64 (20170215.2) xenial InRelease
Hit:2 cdrom://Ubuntu 16.04.2 LTS _Xenial Xerus_ - Release amd64 (20170215.2) xenial Release Ign:4 http://dl.google.com/linux/chrome/deb stable InRelease Hit:5 http://dl.google.com/linux/chrome/deb stable Release Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Hit:8 http://archive.ubuntu.com/ubuntu xenial InRelease Get:9 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] Get:10 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [60.2 kB] Ign:11 http://www.openprinting.org/download/printdriver/debian lsb3.2 InRelease Get:12 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [57.6 kB] Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [49.7 kB] Get:14 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [80.0 kB] Hit:15 http://www.openprinting.org/download/printdriver/debian lsb3.2 Release Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metadata [305 kB] Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [217 kB] Get:19 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [173 kB] Get:20 http://archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [240 kB] Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 DEP-11 Metadata [5,892 B]
Fetched 1,392 kB in 1s (1,068 kB/s)
** (appstreamcli:7142): CRITICAL **: Error while moving old database out of the way.
AppStream cache update failed.
Reading package lists... Done W: http://www.openprinting.org/download/printdriver/debian/dists/lsb3.2/Release.gpg: Signature by key F8897B6F00075648E248B7EC24CBF5474CFD1E2F uses weak digest algorithm (SHA1) 

To mitigate the AppStream cache problem, I referred to this post: Unable to 'apt-get dist-upgrade' on a persistent Ubuntu 16.04 USB and ran the following:

root@ubuntu:~# sudo chmod -R a+rX,u+w /var/cache/app-info/xapian/default
root@ubuntu:~# apt-get update
Ign:1 cdrom://Ubuntu 16.04.2 LTS _Xenial Xerus_ - Release amd64 (20170215.2) xenial InRelease
Hit:2 cdrom://Ubuntu 16.04.2 LTS _Xenial Xerus_ - Release amd64 (20170215.2) xenial Release Ign:4 http://dl.google.com/linux/chrome/deb stable InRelease Hit:5 http://dl.google.com/linux/chrome/deb stable Release Hit:7 http://security.ubuntu.com/ubuntu xenial-security InRelease Hit:8 http://archive.ubuntu.com/ubuntu xenial InRelease Hit:9 http://archive.ubuntu.com/ubuntu xenial-updates InRelease Ign:10 http://www.openprinting.org/download/printdriver/debian lsb3.2 InRelease Hit:11 http://www.openprinting.org/download/printdriver/debian lsb3.2 Release
Reading package lists... Done W: http://www.openprinting.org/download/printdriver/debian/dists/lsb3.2/Release.gpg: Signature by key F8897B6F00075648E248B7EC24CBF5474CFD1E2F uses weak digest algorithm (SHA1)

I was not sure what exactly was causing this problem. Hence I also tried apt get upgrade and got the following in my command line:

root@ubuntu:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
gnome-software gnome-software-common libegl1-mesa libgbm1 libgl1-mesa-dri libinput10 libmirclient9 libwayland-egl1-mesa libxatracker2
linux-generic-hwe-16.04 linux-headers-generic-hwe-16.04 linux-image-generic-hwe-16.04 linux-signed-generic-hwe-16.04
linux-signed-image-generic-hwe-16.04 python3-software-properties python3-update-manager software-properties-common software-properties-gtk
ubuntu-software update-manager update-manager-core update-notifier update-notifier-common xserver-xorg-core-hwe-16.04
xserver-xorg-hwe-16.04 xserver-xorg-input-evdev-hwe-16.04 xserver-xorg-input-synaptics-hwe-16.04 xserver-xorg-input-wacom-hwe-16.04
xserver-xorg-video-amdgpu-hwe-16.04 xserver-xorg-video-ati-hwe-16.04 xserver-xorg-video-fbdev-hwe-16.04 xserver-xorg-video-intel-hwe-16.04
xserver-xorg-video-nouveau-hwe-16.04 xserver-xorg-video-qxl-hwe-16.04 xserver-xorg-video-radeon-hwe-16.04 xserver-xorg-video-vesa-hwe-16.04
xserver-xorg-video-vmware-hwe-16.04
0 upgraded, 0 newly installed, 0 to remove and 37 not upgraded.
3 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up cryptsetup (2:1.6.6-5ubuntu2.1) ...
update-initramfs is disabled since running on read-only media
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
insserv: Service checkroot has to be enabled to start service cryptdisks-early
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing package cryptsetup (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of ubiquity:
ubiquity depends on cryptsetup; however:
Package cryptsetup is not configured yet.
dpkg: error processing package ubiquity (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ubiquity-frontend-gtk:
ubiquity-frontend-gtk depends on ubiquity (= 2.21.63.4); however:
Package ubiquity is not configured yet.
dpkg: error processing package ubiquity-frontend-gtk (--configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
No apport report written because the error message indicates its a followup error from a previous failure.
Errors were encountered while processing:
cryptsetup
ubiquity
ubiquity-frontend-gtk
E: Sub-process /usr/bin/dpkg returned an error code (1) 

I also tried the following command (Reference: https://askubuntu.com/a/899786/484475):

root@ubuntu:~# sudo apt-get autoremove && sudo apt-get autoclean
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 37 not upgraded.
3 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up cryptsetup (2:1.6.6-5ubuntu2.1) ...
update-initramfs is disabled since running on read-only media
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
insserv: Service checkroot has to be enabled to start service cryptdisks-early
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing package cryptsetup (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of ubiquity:
ubiquity depends on cryptsetup; however:
Package cryptsetup is not configured yet.
dpkg: error processing package ubiquity (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ubiquity-frontend-gtk:
ubiquity-frontend-gtk depends on ubiquity (= 2.21.63.4); however:
Package ubiquity is not configured yet.
dpkg: error processing package ubiquity-frontend-gtk (--configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
No apport report written because the error message indicates its a followup error from a previous failure.
Errors were encountered while processing:
cryptsetup
ubiquity
ubiquity-frontend-gtk
E: Sub-process /usr/bin/dpkg returned an error code (1) 

I again referred to another link update-rc.d: error: insserv rejected the script header dpkg: erreur de traitement du paquet util-linux (–configure) : but I dont find missing init headers for cryptsetup.
Another answer says that the issue is with the boot sequencing: https://unix.stackexchange.com/questions/289667/unable-to-install-anything-using-apt-get-because-of-insserv but I am not able to find any issues with the header.
I infer that there are dependencies between packages and hence tried the following command(Reference link: cryptsetup error while doing sudo-apt upgrade):

root@ubuntu:~# sudo dpkg --configure cryptsetup
Setting up cryptsetup (2:1.6.6-5ubuntu2.1) ...
update-initramfs is disabled since running on read-only media
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
insserv: Service checkroot has to be enabled to start service cryptdisks-early
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing package cryptsetup (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
Cryptsetup 

Another attempt in fixing the issue:

root@ubuntu:~# sudo dpkg --configure --pending
Setting up cryptsetup (2:1.6.6-5ubuntu2.1) ...
update-initramfs is disabled since running on read-only media
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
insserv: Service checkroot has to be enabled to start service cryptdisks-early
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing package cryptsetup (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of ubiquity:
ubiquity depends on cryptsetup; however:
Package cryptsetup is not configured yet.
dpkg: error processing package ubiquity (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ubiquity-frontend-gtk:
ubiquity-frontend-gtk depends on ubiquity (= 2.21.63.4); however:
Package ubiquity is not configured yet.
dpkg: error processing package ubiquity-frontend-gtk (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
cryptsetup
ubiquity
ubiquity-frontend-gtk 

Anything that I try (including the clean statement) redirects me to the same issue of:

Errors were encountered while processing:
cryptsetup
ubiquity
ubiquity-frontend-gtk

As a last try, I tried searching for what was the problem and found this online https://superuser.com/questions/903142/how-to-run-update-initramfs-from-bootable-usb to check what was causing the update-initramfs is disabled since running on read-only media problem and found the solution beyond my grasp.

sudo dpkg --configure cryptsetup
Setting up cryptsetup (2:1.6.6-5ubuntu2.1) ...
update-initramfs is disabled since running on read-only media
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
insserv: Service checkroot has to be enabled to start service cryptdisks-early
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing package cryptsetup (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 cryptsetup

Since dpkg reconfigure cryptsetup depends on checkroot, I am not sure on what is to be done. I have posted the contents of checkroot.sh:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          checkroot mtab
# Required-Start:    mountdevsubfs hostname
# Required-Stop:     
# Should-Start:      keymap hwclockfirst hdparm bootlogd
# Should-stop:
# Default-Start:     S
# Default-Stop:
# X-Interactive:     true
# Short-Description: Check to root file system.
### END INIT INFO
# Include /usr/bin in path to find on_ac_power if /usr/ is on the root
# partition.
PATH=/sbin:/bin:/usr/bin
FSCK_LOGFILE=/var/log/fsck/checkroot
[ "$FSCKFIX" ] || FSCKFIX=no
[ "$SULOGIN" ] || SULOGIN=no
. /lib/init/vars.sh
. /lib/lsb/init-functions
. /lib/init/mount-functions.sh
do_start () {
    # Trap SIGINT so that we can handle user interrupt of fsck.
    trap "" INT
    #
    # Set SULOGIN in /etc/default/rcS to yes if you want a sulogin to
    # be spawned from this script *before anything else* with a timeout,
    # like sysv does.
    #
    [ "$SULOGIN" = yes ] && sulogin -t 30 $CONSOLE
    KERNEL="$(uname -s)"
    MACHINE="$(uname -m)"
    read_fstab
    #
    # Activate the swap device(s) in /etc/fstab. This needs to be done
    # before fsck, since fsck can be quite memory-hungry.
    #
    ENABLE_SWAP=no
    case "$KERNEL" in
      Linux)
        if [ "$NOSWAP" = yes ]
        then
            [ "$VERBOSE" = no ] || log_warning_msg "Not activating swap as requested via bootoption noswap."
            ENABLE_SWAP=no
        else
            if [ "$swap_on_lv" = yes ]
            then
                [ "$VERBOSE" = no ] || log_warning_msg "Not activating swap on logical volume."
            elif [ "$swap_on_file" = yes ]
            then
                [ "$VERBOSE" = no ] || log_warning_msg "Not activating swap on swapfile."
            else
                ENABLE_SWAP=yes
            fi
        fi
        ;;
      *)
        ENABLE_SWAP=yes
        ;;
    esac
    if [ "$ENABLE_SWAP" = yes ]
    then
        if [ "$VERBOSE" = no ]
        then
            log_action_begin_msg "Activating swap"
            swapon -a -e >/dev/null 2>&1
            log_action_end_msg $?
        else
            log_daemon_msg "Activating swap"
            swapon -a -v
            log_end_msg $?
        fi
    fi
    #
    # Does the root device in /etc/fstab match with the actual device ?
    # If not we try to use the /dev/root alias device, and if that
    # fails we create a temporary node in /run.
    #
    # Do this only on Linux. Neither kFreeBSD nor Hurd have
    # /dev/root and the device ids used here are specific to
    # Linux.
    KERNEL="$(uname)"
    if [ "$rootcheck" = yes ] && [ "$KERNEL" = Linux ]
    then
        ddev="$(mountpoint -qx $rootdev)"
        rdev="$(mountpoint -d /)"
        if [ "$ddev" != "$rdev" ] && [ "$ddev" != "4:0" ]
        then
            if [ "$(mountpoint -qx /dev/root)" = "4:0" ]
            then
                rootdev=/dev/root
            else
                if \
                    rm -f /run/rootdev \
                    && mknod -m 600 /run/rootdev b ${rdev%:*} ${rdev#*:} \
                    && [ -e /run/rootdev ]
                then
                    rootdev=/run/rootdev
                else
                    rootfatal=yes
                fi
            fi
        fi
    fi
    #
    # Bother, said Pooh.
    #
    if [ "$rootfatal" = yes ]
    then
        log_failure_msg "The device node $rootdev for the root filesystem is missing or incorrect 
or there is no entry for the root filesystem listed in /etc/fstab. 
The system is also unable to create a temporary node in /run. 
This means you have to fix the problem manually."
        log_warning_msg "A maintenance shell will now be started. 
CONTROL-D will terminate this shell and restart the system."
        # Start a single user shell on the console
        if ! sulogin $CONSOLE
        then
            log_failure_msg "Attempt to start maintenance shell failed. 
Will restart in 5 seconds."
            sleep 5
        fi
        [ "$VERBOSE" = no ] || log_action_msg "Will now restart"
        reboot -f
    fi
    # See if we're on AC Power.  If not, we're not gonna run our
    # check.  If on_ac_power (in /usr/) is unavailable, behave as
    # before and check all file systems needing it.
# Disabled AC power check until fsck can be told to only check the
# file system if it is corrupt when running on battery. (bug #526398)
#   if which on_ac_power >/dev/null 2>&1 && [ "$rootcheck" = yes ]
#   then
#       on_ac_power >/dev/null 2>&1
#       if [ "$?" -eq 1 ]
#       then
#           log_warning_msg "On battery power, so skipping file system check."
#           rootcheck=no
#       fi
#   fi
    #
    # See if we want to check the root file system.
    #
    FSCKCODE=0
    if [ -e /run/initramfs/fsck-root ]
    then
        rootcheck=no
    fi
    if is_fastboot_active
    then
        [ "$rootcheck" = yes ] && log_warning_msg "Fast boot enabled, so skipping root file system check."
        rootcheck=no
    fi
    if which findmnt >/dev/null 2>&1
    then
        if [ "$(findmnt -f -n -o FSTYPE /)" = "btrfs" ]
        then
            [ "$rootcheck" = yes ] && log_warning_msg "btrfs root detected, so skipping root file system check."
            rootcheck=no
        fi
    fi
    if [ "$rootcheck" = yes ]
    then
        #
        # Ensure that root is quiescent and read-only before fsck'ing.
        #
        # mount -n -o remount,ro / would be the correct syntax but
        # mount can get confused when there is a "bind" mount defined
        # in fstab that bind-mounts "/" somewhere else.
        #
        # So we use mount -n -o remount,ro $rootdev / but that can
        # fail on older kernels on sparc64/alpha architectures due
        # to a bug in sys_mount().
        #
        # As a compromise we try both.
        #
        if \
            ! mount    -n -o remount,ro              $rootdev /              \
            && ! mount -n -o remount,ro -t dummytype $rootdev /  2>/dev/null \
            && ! mount -n -o remount,ro                       /  2>/dev/null
        then
            log_failure_msg "Cannot check root file system because it is not mounted read-only."
            rootcheck=no
        fi
    fi
    #
    # The actual checking is done here.
    #
    if [ "$rootcheck" = yes ]
    then
        if [ -f /forcefsck ] || grep -q -s -w -i "forcefsck" /proc/cmdline
        then
            force="-f"
        else
            force=""
        fi
        if [ "$FSCKFIX" = yes ]
        then
            fix="-y"
        else
            fix="-a"
        fi
        spinner="-C"
        case "$TERM" in
          dumb|network|unknown|"")
            spinner="" ;;
        esac
        # This Linux/s390x special case should go away.
        if [ "${KERNEL}:${MACHINE}" = Linux:s390x ]
        then
            spinner=""
        fi
        if [ "$VERBOSE" = no ]
        then
            log_action_begin_msg "Checking root file system"
            logsave -s $FSCK_LOGFILE fsck $spinner $force $fix -t $roottype $rootdev
            FSCKCODE=$?
            if [ "$FSCKCODE" = 0 ]
            then
                log_action_end_msg 0
            else
                log_action_end_msg 1 "code $FSCKCODE"
            fi
        else
            log_daemon_msg "Will now check root file system"
            logsave -s $FSCK_LOGFILE fsck $spinner $force $fix -V -t $roottype $rootdev
            FSCKCODE=$?
            log_end_msg $FSCKCODE
        fi
    fi
    #
    # If there was a failure, drop into single-user mode.
    #
    # NOTE: "failure" is defined as exiting with a return code of
    # 4 or larger. A return code of 1 indicates that file system
    # errors were corrected but that the boot may proceed. A return
    # code of 2 or 3 indicates that the system should immediately reboot.
    #
    if [ "$FSCKCODE" -eq 32 ]
    then
        log_warning_msg "File system check was interrupted by user"
    elif [ "$FSCKCODE" -gt 3 ]
    then
        # Surprise! Re-directing from a HERE document (as in "cat << EOF")
        # does not work because the root is currently read-only.
        log_failure_msg "An automatic file system check (fsck) of the root filesystem failed. 
A manual fsck must be performed, then the system restarted. 
The fsck should be performed in maintenance mode with the 
root filesystem mounted in read-only mode."
        log_warning_msg "The root filesystem is currently mounted in read-only mode. 
A maintenance shell will now be started. 
After performing system maintenance, press CONTROL-D 
to terminate the maintenance shell and restart the system."
        # Start a single user shell on the console
        if ! sulogin $CONSOLE
        then
            log_failure_msg "Attempt to start maintenance shell failed. 
Will restart in 5 seconds."
            sleep 5
        fi
        [ "$VERBOSE" = no ] || log_action_msg "Will now restart"
        reboot -f
    elif [ "$FSCKCODE" -gt 1 ]
    then
        log_failure_msg "The file system check corrected errors on the root partition 
but requested that the system be restarted."
        log_warning_msg "The system will be restarted in 5 seconds."
        sleep 5
        [ "$VERBOSE" = no ] || log_action_msg "Will now restart"
        reboot -f
    fi
    #
    # Remount root to final mode (rw or ro).
    #
    # See the comments above at the previous "mount -o remount"
    # for an explanation why we try this twice.
    #
    if ! mount -n -o remount,$rootopts,$rootmode $fstabroot / 2>/dev/null
    then
        mount -n -o remount,$rootopts,$rootmode /
    fi
    # If possible, migrate /etc/mtab to be a symlink to
    # /proc/mounts.  Note that not all systems e.g. Hurd currently
    # support this.
    if [ "$rootmode" != "ro" ]; then
        mtab_migrate
    fi
    if selinux_enabled && [ -x /sbin/restorecon ] && [ -r /etc/mtab ]
    then
        restorecon /etc/mtab
    fi
    #
    # Remove /run/rootdev if we created it.
    #
    rm -f /run/rootdev
    # Update mount options for mounts created in early boot
    # S01mountkernfs.sh
    /etc/init.d/mountkernfs.sh reload
    # S03mountdevsubfs.sh
    /etc/init.d/mountdevsubfs.sh reload
}
do_status () {
    # If / is read-write or swap is enabled, this script have done
    # its job.
    rootrw=false
    swapon=false
    if [ -f /etc/mtab ] ; then
        if grep " / " /etc/mtab |grep -q rw ; then
        rootrw=true
        fi
    fi
    if [ -f /proc/swaps ] ; then
        if [ "$(cat /proc/swaps |grep -v ^Filename)" ] ; then
        swapon=true
        fi
    fi
    if [ true = "$rootrw" ] || [ true = "$swapon" ] ; then
        return 0
    else
        return 4
    fi
}
case "$1" in
  start|"")
    do_start
    ;;
  restart|reload|force-reload)
    echo "Error: argument '$1' not supported" >&2
    exit 3
    ;;
  stop)
    # No-op
    ;;
  status)
    do_status
    exit $?
    ;;
  *)
    echo "Usage: checkroot.sh [start|stop]" >&2
    exit 3
    ;;
esac
:

Kindly help me resolve the above issue.I am new to linux. Simple answers will be much appreciated.

Best Answer

The simple answer is you are abusing the persistence mechanism and you need to stop =)

The somewhat longer answer is persistence uses a COW (copy on write) mechanism - https://en.wikipedia.org/wiki/Copy-on-write

Sorry for the Wikipedia link =)

So the iso (well technically the root files system , squashfs) is a read only file system. This "iso" or root file system can never be changed short or remastering the iso.

Enter the idea of persistence, a mechanism to create an overlay if you will . So if you update your system or make changes, the changes are made to a file (persistence) and not the iso.

This works well for simple tasks such as making adjustments to your interface, customizing the desktop, saving some data, saving wireless passwords.

But the COW system breaks down and you make changes on changes. So you have a base RO file system, and you run sudo apt-update over and over, the COW system is not a database to tack all those changes to changes to changes and eventually fails.

The fix is to either:

  1. Use Persistence sparingly, and make as few changes as possible, perhaps storing user settings, wireless passwords, and installing, one time, some applications. But do not abuse it by treating it as an installed system and try to run sudo apt update && sudo apt upgrade , the COW can not handle multiple changes to hundreds of packages.

  2. Just do a regular install. Probably the best idea as you obviously intend to continue to use Ubuntu.

Hope that answer is not too technical and I hope I did not over simplify the situation too much .