I am a graduate student and a relative Linux novice. My institution has an in-house Linux cluster on which I run many scientific simulations. I have a Windows desktop computer from which I access the Linux cluster via SSH.
I have a large amount (~1 TB) of simulation results data on the Linux cluster's file server. When the project is finished, the research group probably will not have the space to save the simulation results. However, I would like to save the files (with the group's permission, of course) on an external drive that I myself will purchase.
My question is, if I purchase a standard Windows external hard drive with a USB connection, will I be able to copy the files from the Linux cluster's files server to the external drive? (I am assuming that the Linux cluster has a USB port, but this is something that I will need to verify.)
It looks like many standard Windows external hard drives are formatted in either NTFS or FAT32, whereas our Ubuntu Linux file server uses NFS. Here are some examples from Amazon:
-
Seagate Backup Plus 4 TB USB 3.0 Desktop External Hard Drive
STCA4000100 (NTFS) -
WD My Book 4TB External Hard Drive Storage USB 3.0 File Backup and
Storage (NTFS) -
BUFFALO MiniStation Plus 1 TB USB 3.0 Portable Hard Drive –
HD-PNT1.0U3BS Silver (FAT32)
Do you think any or all of the above hard drives will be able to be easily reformatted in NFS for use with the Linux cluster?
On the other hand, Amazon does have a section for "Linux platform support" external hard drives, such as:
But, even if standard Windows external hard drives are easily reformatted, the problem is that I may subsequently want to copy the files from the external hard drive to a Windows computer, which is NTFS. This part of the question may require a separate question or a question on SuperUser, but is it possible to copy NFS files from an external hard drive to a Windows NTFS computer? Thanks for your time.
Best Answer
Yes, there is no technical problem to this, however:
The hardware us not a "standard windows hard drive with USB connection". Please scrap the windows part from that sentence. And external USB HDD will work equally well with or without windows as the OS.
For a large amount of data (and 1TB is a lot) connecting the drive locally is probably a lot faster. However with USB2 you are still limited to 35-ish MB/sec. That means that copying 1TB over USB2 takes about 8-9 hours.*
You can speed that up a lot if the drive is locally mounted (via plain SATA), if the cluster and your drive have eSATA, if both have USB3 or if both have firewire.
Alternatively you can connect the drive to your own desktop and copy the files. In this case the network might be the speed limit. You also risk an angry administrator asking why you are making the network so slow for other users. :-)
uhm, no.
The hard disk does not care which filesystem is used. It may come pre-formatted with NTFS (which is a sensible choice for most people who buy them), but nothing stop you from changing the filesystem and reformatting. That should only take a few minutes.
Also, your file server does not use NFS on its hard disks. It is probably using ext2, ext4 or ZFS. Neither of which you need to worry about. As long as you can read the data you can write it in any format.
(Consider the analogy: You copy the text written in a notebook. Do not worry about the form or the colour of the original notebook. As long as you can read it and have a large enough notebook of your own you can copy the content from one notebook to another).
*: 8-9 hours estimated based on this:
35 MiB/second
100 MiB per 3 seconds.
1000 MiB per 30 seconds, which is the same as 1GiB per 30 seconds.
1GiB per 30 seconds
1000GiB per 30000 seconds
1TiB per 30000 seconds. 30000/3600=8.3 (3600 seconds per hour)