Working Syntax for Rsync Copy of a set of 2 or more Hard-linked Folder Trees while maintaining this specific archive structure

cygwin;directory-structurehard linkrsync

Working Syntax for Rsync Copy of a set of 2 or more Hard-linked Folder Trees while maintaining this specific archive structure?

PS: Would this be a better fit in SO or SU SE?

Template of Current/ Example Folder Structure: 

Base Folder (Source): 

C:\iMazing
   \Backups

    \{device A UDID} // Last backup of Device A (AL)
    \{device B UDID} // Last backup of Device B (BL)
    \iMazing.Versions 

       \Versions
          \{device A UDID} // Backup history of Device A (AL) 
          \{device B UDID} // Backup history of Device B (BH)

Base Folder (Destination): 

S:\ExtHDD\APPLE\iMazing.Backups  

    \{device A UDID} // Last backup of Device A (AL)
    \{device B UDID} // Last backup of Device B (BL)
    \iMazing.Versions 

       \Versions
          \{device A UDID} // Backup history of Device A (AH) 
          \{device B UDID} // Backup history of Device B (BH)

Folder Trees under Folders with UDID Names

  • AL : {device A UDID} // Last backup of Device A
  • AH : {device A UDID} // Backup history of Device A
  • AL & AH Form a SINGLE 2 SET with Hard Links in between.
  • We'd like to work INDIVIDUAL SET at a time

How to use Rsync (preferable via Cygwin RSync or one of mentioned Windows variations) to :

  • Copy a Hard Linked INDIVIDUAL SET e.g. {AL + AH} from C drive to S drive while
    • Preserving Hard Links
    • Maintain the Sub-tree folder structure
  • Without causing issues on Windows or its NTFS Folder & File structure

Also:

  • Can we do only an INDIVIDUAL SET at a time (one single RSync command) or multiple SPECIFIED SETs?
  • Typically we have 2 SETs, how would commandline for Rsync change if it was a 3 SET (SET with 3 HL folders) within a structure

A Real/ Actual Example with UDID Folder name:

  • XL & XH which are Hard Linked folder trees, together make up a SET. 
Base Folder (Source): 
C:\Users\UserName\AppData\Roaming\iMazing\Backups

  \f22ebcffcdd1b508d3f7564431a9db98d69208ef-20190414-170122 //..(XL)  
  \iMazing.Versions
   \Versions  
    \f22ebcffcdd1b508d3f7564431a9db98d69208ef-20190414-170122 //..(XH)

Reference list of Tools for Hard Link preserving Copying:
https://superuser.com/questions/997190/copy-to-another-ntfs-disk-and-preserve-hard-links/

  • Rsync variations under Windows
    • (Several *Nix & Windows: Cygwin variations, CW RSync, Delta Copy/ Syncrify/ Synaman, GRsync, RsyncBackup, Rsync.exe pkg, Rclone, AcroSync, YInterSync)
  • ln.exe – command line hardlinks – http://schinagl.priv.at/nt/ln/ln.html (Win)

Best Answer

I've found it rather difficult to understand your earlier iterations of this question, so I'm going to restate a simplified version of what I think you mean, along with a corresponding worked example:

Structure

C:\iMazing\Backups\
    AL\
        files...
    BL\
        files...
    iMazing.Versions\Versions\
        AH\
            files...
        BH\

S:\ExtHDD\APPLE\iMazing.Backups\
    AL\
        destination files...
    BL\
        destination files...
    iMazing.Versions\Versions\
        AH\
            destination files...
        BH\
            destination files...

Requirement is to copy AL\... and AH\... from C: to S: while maintaining hard links and relative structure.

Worked example

mkdir -p iMazing/Backups/{AL,BL,iMazing.Versions/Versions/{AH,BH}} ExtHDD/APPLE/iMazing.Backups
touch iMazing/Backups/{AL/albackup,BL/blbackup}
touch iMazing/Backups/iMazing.Versions/Versions/{AH/ahbackup,BH/bhbackup}

At this point you can ls -R or find to see what I've done. There should be files albackup, ahbackup, blbackup and bhbackup in the AL, AH, BL, and BH directories. We're going to copy the AL and AH directories to the target:

rsync --dry-run -avHPR iMazing/Backups/./AL iMazing/Backups/./iMazing.Versions/Versions/AH ExtHDD/APPLE/iMazing.Backups/

The -R (--relative) flag retains the source path(s) from the /./ marker. In the example this means that iMazing/Backups/ is stripped, and the remainder appended to the destination path.

Once the --dry-run (-n) is removed this will copy files and directories from the source AL and AH directories to the target, maintaining internal hard links and structure.

Question solution

Since this is Cygwin you need to use forward slashes and paths starting with /cygdrive/{DriveLetter}. To maintain links you need to include the -H flag (--hard-links).

rsync -aHR /cygdrive/C/iMazing/Backups/./AL /cygdrive/C/iMazing/Backups/./iMazing.Versions/Versions/AH /cygdrive/ExtHDD/APPLE/iMazing.Backups/

You can include the -v (--verbose) and/or -P (--partial --progress) flags if you want to see what's going on. In this situation rsync will not bother with its infamous "delta" algorithm or compression (these only work when there are two servers involved in an rsync client-server configuration across a network). If you could run an instance of rsync on the server hosting your S: drive that could be far more efficient.

Add --dry-run to see what would happen without it actually doing anything.

Related Question