My question is, if I purchase a standard Windows external hard drive with a USB connection, will I be able to copy the files from the Linux cluster's files server to the external drive?
Yes, there is no technical problem to this, however:
The hardware us not a "standard windows hard drive with USB connection". Please scrap the windows part from that sentence. And external USB HDD will work equally well with or without windows as the OS.
I am assuming that the Linux cluster has a USB port, but this is something that I will need to verify.
For a large amount of data (and 1TB is a lot) connecting the drive locally is probably a lot faster. However with USB2 you are still limited to 35-ish MB/sec. That means that copying 1TB over USB2 takes about 8-9 hours.*
You can speed that up a lot if the drive is locally mounted (via plain SATA), if the cluster and your drive have eSATA, if both have USB3 or if both have firewire.
Alternatively you can connect the drive to your own desktop and copy the files. In this case the network might be the speed limit. You also risk an angry administrator asking why you are making the network so slow for other users. :-)
It looks like many standard Windows external hard drives are formatted in either NTFS or FAT32, whereas our Ubuntu Linux file server uses NFS.
uhm, no.
The hard disk does not care which filesystem is used. It may come pre-formatted with NTFS (which is a sensible choice for most people who buy them), but nothing stop you from changing the filesystem and reformatting. That should only take a few minutes.
Also, your file server does not use NFS on its hard disks. It is probably using ext2, ext4 or ZFS. Neither of which you need to worry about. As long as you can read the data you can write it in any format.
(Consider the analogy: You copy the text written in a notebook. Do not worry about the form or the colour of the original notebook. As long as you can read it and have a large enough notebook of your own you can copy the content from one notebook to another).
*: 8-9 hours estimated based on this:
35 MiB/second
100 MiB per 3 seconds.
1000 MiB per 30 seconds, which is the same as 1GiB per 30 seconds.
1GiB per 30 seconds
1000GiB per 30000 seconds
1TiB per 30000 seconds. 30000/3600=8.3 (3600 seconds per hour)
You can use ext2
. Support for ext2
has existed in FreeBSD for a while and can probably be considered stable. Of course it is native in GNU/Linux as you know.
You could also use ext3
but without journal and extended attributes (use mount options in Linux /etc/fstab
), which would increase some limits.
This is probably much better than using a fs which is not native on any of the two systems, like NTFS and the like.
Source: https://www.freebsd.org/doc/handbook/filesystems-linux.html
Best Answer
What I do is to store tarballs on the USB drive (formatted as VFAT). I'm wary of reformatting USB drives, they are build/optimized for VFAT so to level wear, and I'm afraid it will die much sooner with other filesystems. Besides, formatting another way will make it useless for ThatOtherSystem...