Debian – how to do version-specific backup/restore of Debian packages

backupdebianpackage-managementrestore

summary

Note version-specific in the question title! This question is not, how to

  1. On one Debian host, capture a list of all installed packages.
  2. On another Debian host, install the latest versions of the listed packages.

This question is rather, how to

  1. On a Debian host at some point in time,
    1. capture a package map containing
      • the name of each installed package
      • the version of each package currently installed
    2. any additional data required to …
  2. On that same host later in time, restore the package map, such that subsequently
    1. only the packages in the package map are installed
    2. for each packages in the map, only the appropriate version is installed

details

motivation

Recently I did a "big update" (i.e., involving lots of packages, or major packages like kernels, or both) to one of my Debian boxes, which did not update "cleanly" (i.e., with no significant problems). In future, when I apply a bad update to a Debian box, I'd like to be able to rollback (a) the package set on that same box (b) to the previous versions. (Without previously doing an image backup, which is slow.) Unfortunately I don't know how to do that. I do know how to

  1. On one Debian box, capture a list of every package name in its installed package set.
  2. On another box, install the latest version of every package in the list.

thanks to numerous webpages discussing how to do this, including several SEs. Most (including this, this, and this) do more or less sophisticated variations on

  1. backup: dpkg --get-selections > ${PACKAGE_LIST_FILEPATH}
  2. restore: dpkg --set-selections < ${PACKAGE_LIST_FILEPATH}

Probably the most comprehensive (and among the most scripted, which I prefer) SE on this topic is this one: it's one of few to recommend also backing up repository lists (with cp -R /etc/apt/sources.list*) and repo keys (with apt-key exportall), but it also uses dpkg as above. This SE recommends using aptitude (which, FWIW, I prefer, and suspect does a better job than dpkg) for both backup and install, and this SE recommends deborphan for the backup.

But none of the above save package versions, and so none work for my usecase. (IIUC–am I missing something?)

hypothesis

Obviously one must backup (and be able to restore as needed) arbitrary files from one's devices. Similarly, one should be able to backup/restore ones packages, and integrate that into one's periodically-run backup tool. Fortunately, backup/restore of a package list is well-understood. One can backup with, e.g., bash code like

## setup

PACKAGE_LIST_FILENAME='package.list'         # see use below
REPO_KEYS_FILENAME='repo.keys'               # ditto
DATE_FORMAT="%Y%m%d_%H%M%S"
BACKUP_TIMESTAMP="$(date +${DATE_FORMAT})"   # get current timestamp, to the second

# Parent directory of that to which you will backup.
BACKUP_ROOT="/media/$(whoami)/my_backup_drive"
BACKUP_DIR="${BACKUP_ROOT}/backup_${BACKUP_TIMESTAMP}"
echo -e "Backing up to ${BACKUP_DIR}" # debugging
# TODO: check BACKUP_DIR is writable, has sufficient freespace, etc

# ASSERT: all following are subdirs of BACKUP_DIR
ETC_BACKUP_DIR="${BACKUP_DIR}/etc"           # backup /etc , typically not large
PKG_BACKUP_DIR="${BACKUP_DIR}/packages"
REPO_BACKUP_DIR="${BACKUP_DIR}/repos"   # duplicates some of /etc, but disk is cheap ...
for DIR in \
  "${ETC_BACKUP_DIR}" \
  "${PKG_BACKUP_DIR}" \
  "${REPO_BACKUP_DIR}" \
; do
  # make the backup dirs
  mkdir -p "${DIR}"  # TODO: test retval/errorcode, exit on failure
done

PACKAGE_LIST_FILEPATH="${PKG_BACKUP_DIR}/${PACKAGE_LIST_FILENAME}"
touch "${PACKAGE_LIST_FILEPATH}"             # TODO: check that it's writable, exit if not

REPO_KEYS_FILEPATH="${REPO_BACKUP_DIR}/${REPO_KEYS_FILENAME}"
touch "${REPO_KEYS_FILEPATH}"                # TODO: check that it's writable, exit if not

## backup
## TODO: for all following: test retval/errorcode, exit on failure
## TODO: give user some progress indication (e.g., echo commands)

deborphan -a --no-show-section > "${PACKAGE_LIST_FILEPATH}" # or other op you prefer
sudo cp -R /etc/apt/sources.list* "${REPO_BACKUP_DIR}/"
sudo apt-key exportall > "${REPO_KEYS_FILEPATH}"
rsync --progress /etc "${ETC_BACKUP_DIR}"

And one can restore with, e.g., bash code like

## setup
## (remember to transfer constants used above)

RESTORE_DIR="set this equal to the BACKUP_DIR from which you are restoring!"
ETC_RESTORE_DIR="${RESTORE_DIR}/etc"
PKG_RESTORE_DIR="${RESTORE_DIR}/packages"
REPO_RESTORE_DIR="${PKG_RESTORE_DIR}/repos"
PACKAGE_LIST_FILEPATH="${PKG_RESTORE_DIR}/${PACKAGE_LIST_FILENAME}"
REPO_KEYS_FILEPATH="${REPO_RESTORE_DIR}/${REPO_KEYS_FILENAME}"

## restore
## TODO: test that all following are readable, exit if not
## TODO: for all following: test retval/errorcode, exit on failure
## TODO: give user some progress indication (e.g., echo commands)

rsync --progress "${ETC_BACKUP_DIR}" /etc
# following will overwrite some of /etc restored above
sudo apt-key add "${REPO_KEYS_FILEPATH}"
sudo cp -R "${REPO_RESTORE_DIR}/sources.list*" /etc/apt/
# TODO: CHECK THIS SYNTAX!
# see my question @ https://serverfault.com/questions/56848/install-the-same-debian-packages-on-another-system/61472#comment920243_61472
sudo xargs aptitude --schedule-only install < "${PACKAGE_LIST_FILEPATH}" ; aptitude install

So IIUC, we must extend the above to handle package versions. This seems doable using a strategy like the following:

backup

  1. Write a package map file, instead of a package list file. Thanks to RobertL and cas, I now know that I can get a package map file for a Debian host in a useful format (one package per line, each line containing {package name (and architecture, but only if more than one), TAB, package version string}. One can then access
    • package names with awk '{print $1}' < "${PACKAGE_LIST_FILEPATH}"
    • package versions with awk '{print $2}' < "${PACKAGE_LIST_FILEPATH}"
  2. (extra credit) Archive the current versions of the raw/.deb packages (as indicated by cas) by feeding the package names to dpkg-repack.
  3. (extra credit) Convert the archived packages into a local APT repository. This is hopefully described in operational detail by this Debian wikipage, but I'm still trying to figure that out.

restore

glossary:
* current refers to the state of the host before the user attempts to restore.
* current PMF refers to a package map file generated for the current host, containing package tuples (key=package name, value=package version) currently installed.
* backup PMF refers to a package map file generated for the host at some prior time, containing package tuples to be restored.

  1. Create a current PMF.
  2. Restore repositories and keys by "the usual means" (see above).
  3. Read the backup PMF.
  4. For each package tuple in the current PMF but not the backup PMF: uninstall the package (e.g., sudo aptitude remove [package_name]).
  5. For each package tuple in the backup PMF: install the given version of the package (e.g., sudo aptitude install [package_name]=[package_version]). Thanks to Rui F Ribeiro and bigbenaugust for noting that aptitude and apt-get support install package=version.
    • (extra credit) If the restored repositories do not contain the desired version of a package, install it from the backup-local APT repository.

test

TODO! I'll start by doing a manual restore for a box for which I don't yet have a backup like the above (since, of course, I don't yet have a backup like the above for any box 🙂 but for which I have console spew.

Best Answer

Some useful notes to help you achieve your goal:

  1. You can get a list of installed packages and their versions by running:

dpkg-query -W

  1. You can create an archive of .deb packages of all currently installed packages by installing dpkg-repack and running something like this:

dpkg-query -W | awk '{print $1}' | xargs dpkg-repack

This will dpkg-repack all currently installed packages. This archive is the crucial missing part to your restore - without it, you may not be able to restore the exact same package set (especially if you use testing or unstable).

WARNING: dpkg-repack repacks the current, possibly modified, contents of all conffiles. If you want the pristine, original packages, you'll have to get the .deb files from /var/cache/apt/archives or from a Debian mirror.

The .deb archive can be turned into an apt-gettable repository by following the instructions at https://wiki.debian.org/HowToSetupADebianRepository or you can just install them all with dpkg -iBE *.deb (or dpkg -iRBE /path/to/deb/files/ if there are too many to fit on one command line).

  1. You'll still need to use -get-selections and --set-selections in order to retain details like de-installed packages.
Related Question