Divide out local files in different servers with limited space with rsync

backupdavfs2rsync

I have 80 GB of files in a local directory that I want to back up, dividing them out onto different servers with a limited space quota, using rsync, like so:

LOCAL 80 Gb ====> server1: first 30 Gb
             ||
             ||=> server2: following 15 Gb
             ||=> server3: last 35 Gb

Every server is mounted on the local machine using WebDAV connection and davfs2, so I can avoid all SSH, keys, and similar ramblings. How can I split my files onto several destinations?

Best Answer

If these files are just for backup you could

  • create a (dated?) tar file of all the files you want to backup
  • then use the 'split' command to break it up into appropriately sized chunks
  • distribute one chunk to each server

As far as I now rsync doesn't have the functionality to distibute a file across servers. Another possibility is this:

  • create one directory representing each server
  • manually move/copy the files you want in each server to each of these directories, but if you copy, use hard-links so no more space is used.
  • rsync each directory to its respective server. create a script to do this if this is a regular task.
Related Question