What are good bi-directional, real-time synchronization utilities between Mac local folder and web server

data synchronizationfile-transfersoftware-recommendationwebserver

I have an iMac that is connected to my local network (and, subsequently, the Internet) that has a folder named DocShare.

I have a web server hosted off-site through a hosting company, with a similar folder called Online DocShare that hosts an online repository of files that would be related to the DocShare folder on my iMac. This server supports WebDAV, (S)FTP, SSH, etc. for file transfer.

I would love to find a way to have those two folders, one locally on my iMac and one on my server, synchronize so that anytime I upload a file to the local DocShare, it will appear on my Online DocShare, and vice-versa. It would be great if changes to files could be reflected, and if I deleted/modified/created a file on either side, it would reflect on the opposite folder.

So, it would be real-time syncing with a constant connection to the web server, or a scheduled sync process every time interval. So far, I have found a handful of programs for Windows, but none that do this for Mac.

If possible, I would love to see it in a GUI, with a possible command-line add-on (not necessarily command-line exclusively). Please let me know what possible options I have, or if I'm all washed up and there isn't anything that does this sort of thing.

Best Answer

Unison is my go-to tool for a bi-directional file sync. (Mac binaries are available on a contributor's site.) I tend to think of it as a more interactive, configurable, and bi-directional version of rsync. Since it's a good *nix citizen, it can be automated via your favorite method. It's open-source and has ports to major OS platforms.

Unfortunately, Unison is no longer under active development, which may or may not be an issue for you. (I think the last time I used it was under 10.6.) But, it does fit your requirements nicely:

  • It handles bi-directional sync and works great on large files and data sets.
  • It can run in a completely automated way or interactively (where your review and confirm every change).
  • It runs over SSH.
  • GUI and CLI interfaces are available. (FWIW, the Mac GUI version, while nice/convenient, was always slightly flaky for me.)
  • Its end-user documentation is excellent and thorough.
  • It has configuration options for a multitude of use cases.
  • It has a design principle of leaving things in a Good State at all times, which makes it easier to recover should the connection be lost at any time.

And here's the main selling point for using it (for me, at least):

For remote SSH targets, it logs into the remote machine, runs a separate copy of Unison there to scan for changes, and then reconciles the state with your local machine to only transmit the bits (or bytes :-)) that differ, much like rsync does. This makes it very efficient with bandwidth, and it's also able to maintain caches of its file index on each side to make subsequent scans faster. For large data sets, this is a huge deal.

There are other Mac-specific tools for file synchronization, but, IIRC, they tend to assume that the filesystem is local, and are either very inefficient with bandwidth or very presumptuous about file "changes" (metadata, actual data) on large remote files / data sets.

Unison worked the way I needed it to, was reliable, and made me confident in using it due to its great documentation. Thank you, Benjamin C. Pierce!

Related Question