Windows – Could hardlinks be used for backup deduplication purposes

backuphardlinkwindows

I know about full backups, incremental backups and decremental backups. However, I wondered why nobody (Windows Backup, TrueImage, Paragon) seems to have implemented the following backup algorithm.

It needs a backup medium which supports links, e.g. NTFS. Ideally the backup medium is of the same format in order to support all features such as alternate data streams (ADS).

  1. First backup is a full backup. This will copy all files onto the backup medium into a subfolder of \. Let's call this folder L (for "last"). There's no special file format, just copy the files.
  2. With the next backup, a new subfolder of \ will be created, let's call it C (for "current"). Files which have changed from the full backup will be copied again from the source disk. Files which have not changed are moved from L to C and a hardlink is created to point from L to C.
  3. On repeated backups, the same procedure will be applied with C and another new folder.

Is there anything I miss in this algorithm which would not work?

While I did notice any problems yet, I can see the following advantages:

  • the last backup (C) is always a full backup. For restoring the backup, you only need this one backup. The user can delete any old backup without destroying the possibility of recovering (which is not the case in full, incremental and decremental backups).
  • Old backups will act like full backups due to the links, but take much less space on disk.
  • there's a full history of file changes if the user did not delete a file. But unlike SVN, it is possible to delete old revisions.
  • Moving files and creating links are very fast operations. Creating the backup should be accordingly performant.
  • It is possible to selectively delete changed files in old backups (e.g. only the big ones), not deleting a complete backup

Best Answer

What you describe is already in use, with rsync and its --link-dest= option, via dozens of wrapper programs, such as dirvish, among others.

Related Question