Yes, rsync
is the answer.
Having the drive plugged in locally is going to be a lot faster than the network, but for the purpose of doing backups, I would actually recommend leaving the hardware in place. Do the slower thing and transfer it over the network ... remote to remote if need be, so that you don't break anything by touching it.
There are four main parts to your rsync command. First, the transport mechanism. For local to local transfers this is skipped. If any of the sides are remote, you usually specify a tranfer agent like rsync -e ssh
to use SSH transport. After that you specify a source for your files (end with a slash if giving a directory!) and then a target (again end with a slash syncing directories!). So we have rsync -e ssh machine1:/source/folder/ machine2:/source/folder/
so far. Then you can add options like what to skip using --exclude PATTERN
, over the wire compression using -z
, maybe the -a
archive option to handle recursive stuff, matching permissions and such, then -v
or --progress
if you want to see details or progress as it goes along.
Sample:
rsync -avze ssh ubuntu:/path/to/source/files/ osxhost:/path/to/backup/drive/
My question is, if I purchase a standard Windows external hard drive with a USB connection, will I be able to copy the files from the Linux cluster's files server to the external drive?
Yes, there is no technical problem to this, however:
The hardware us not a "standard windows hard drive with USB connection". Please scrap the windows part from that sentence. And external USB HDD will work equally well with or without windows as the OS.
I am assuming that the Linux cluster has a USB port, but this is something that I will need to verify.
For a large amount of data (and 1TB is a lot) connecting the drive locally is probably a lot faster. However with USB2 you are still limited to 35-ish MB/sec. That means that copying 1TB over USB2 takes about 8-9 hours.*
You can speed that up a lot if the drive is locally mounted (via plain SATA), if the cluster and your drive have eSATA, if both have USB3 or if both have firewire.
Alternatively you can connect the drive to your own desktop and copy the files. In this case the network might be the speed limit. You also risk an angry administrator asking why you are making the network so slow for other users. :-)
It looks like many standard Windows external hard drives are formatted in either NTFS or FAT32, whereas our Ubuntu Linux file server uses NFS.
uhm, no.
The hard disk does not care which filesystem is used. It may come pre-formatted with NTFS (which is a sensible choice for most people who buy them), but nothing stop you from changing the filesystem and reformatting. That should only take a few minutes.
Also, your file server does not use NFS on its hard disks. It is probably using ext2, ext4 or ZFS. Neither of which you need to worry about. As long as you can read the data you can write it in any format.
(Consider the analogy: You copy the text written in a notebook. Do not worry about the form or the colour of the original notebook. As long as you can read it and have a large enough notebook of your own you can copy the content from one notebook to another).
*: 8-9 hours estimated based on this:
35 MiB/second
100 MiB per 3 seconds.
1000 MiB per 30 seconds, which is the same as 1GiB per 30 seconds.
1GiB per 30 seconds
1000GiB per 30000 seconds
1TiB per 30000 seconds. 30000/3600=8.3 (3600 seconds per hour)
Best Answer
I found this tutorial. It's untested by me but several commenters to the article attested to it working. The article is titled: Mount a ufs2 Volume in MacOS/X 10.7 (Lion).
excerpt