Rsync – How to Use Only mtime Comparison

Go with what's already setup, if ssh on the work box is already an open/monitored/supported/audited service then try to do the rsync via that. Not opening up new ports/services is generally safest. Not opening up insecure protocols to the internet is even better =)

You can get ssh access to the ReadyNas (if you don't mind some hassle from Netgear in the event of a "i deleted my nas" support call). Then rsync -e ssh from the command line which leaves nothing else to setup. Auth, wire security and user/file permissions are all provided by ssh/remote shell setup on work box.

For opening up ssh on your home network:

• The dynamic IP hassles can be covered by running a free dynamic DNS service.

• Restrict access to the port to your work box's public IP.
  Some routers allow you to set a source IP in the NAT rule.
  SSH can be secured with iptables and more
  

As DarkHeart mentioned, rsync by itself over an unsecure network is not a good idea. The tunnel and vpn mentioned is a good work around. You can keep the aforementioned ssh tunnel up with autossh. You may want to depend on some rsync security if your going to leave a tunnel up depending on who else has access to either end.

Also, if you haven't already, discuss what your doing with someone. Detail what data is going in/out of the network. Think about what data might be able to go out if everything goes wrong. Document your process somewhere.

If you're not going to use the remote file system as the data source of what has been transferred then you need to externally track the files that have been successfully transferred previously, then exclude them from future transfers.

rsync can include and exclude files based on patterns in a file so you can include a specific list of files in a transfer. Then exclude that list from future transfers.

#!/usr/bin/env bash

set -e

track_dir=~/.track_xfer
inc_file="$track_dir/include_files"
exc_file="$track_dir/exclude_files"
xfer_dir=~/testrsync
xfer_dest=~/testrsync_dest

mkdir -p "$track_dir"
touch $exc_file
cd "$xfer_dir"

# find files and create rsync filter list
find . -type f -print0 | perl -e '
  $/="\0"; 
  while (<>){ 
   chomp; 
   $_ =~ s!^\.!!;    # remove leading .
   $f = quotemeta;   # quote special chars
   $f =~ s!\\/!/!g;  # fix quoted paths `/`
   print $f."\n"; 
  }' > "$inc_file"

# Run the rsync
rsync -va --delete --exclude-from "$exc_file" --include-from "$inc_file" "$xfer_dir/" "$xfer_dest"

# Add the included/transferred files to the exclusion list
cat "$inc_file" "$exc_file" > "$exc_file".tmp
sort "$exc_file".tmp | uniq > "$exc_file"

You might need some more rsync specific regex quoting but the Perl quotemeta function and replacements was the first easy solution that came to mind.

The main problem will be dealing with any special characters in files names. If you want to deal with new lines or tabs and other strange things in the names then you will have to put a bit more work into the perl (or whatever) that parses and generates the inclusion pattern list. If you can restrict the names of your transfer files to a simple character set then you don't need to worry about this step as much. The perl is a halfway solution that should get you past most common regex chars.

The reason for using the include list rather than letting rsync pull the whole directory it self is so that you have a defined/complete list of files for the subsequent exclude list. You could probably achieve the same result by parsing the rsync output or a --log-file=FILE for the files that were transferred but that looked a little harder.