Linux – Reformatting USB to support large files in Windows, Linux and Mac

filesystemslinuxmacosusbwindows 7

I'm trying to copy a 5.3 GB file from hard disk to a USB 3.0 SanDisk 16 GB USB key.
I've tried this both from OS X as well as from Linux. In both cases the file is
partially copied and then the operation fails. On OS X it fails with a message
like "file too large". I had to use the split command to split the file into
several 100MB chunks and then copy the directory containing them to the USB
key in order to copy the file.

Then, on Linux I reassembled the files
with the cat command into a single file, sending the output to the
computer's hard drive. However, when I tried to reassemble the
split files into one file with the cat command and sent the
output to the USB key, the USB key still did not accept the
resulting file, which again, was too large, and complained
with the error message

"cat: write error: Protocol error".

When I inspected the successful output with:

$ ls -l --block-size=M MyFileName

the file size was exactly:

4096M (that is, 4 GB).

This seems like a limitation tied to the USB key's default filesystem
with which it was formatted prior to being sold.

When I examine its filesystem under Windows 7 by right-clicking on the
USB disk and inspecting its properties I can see that the filesystem
is a FAT32 filesystem.

Can I somehow, starting from Windows, format the USB key so that it
uses a filesystem which will accept my 5.3 GB file, and can at the
same time be read from and written to from all of OS X, Linux, and
Windows 7? What filesystem should I format the USB key to in order
to achieve this very purpose?

Best Answer

No perfect solution, but your top candidates are NTFS and exFAT:

  • Windows (recent versions) will read and write both, natively
  • OS X will read NTFS and read/write exFAT
  • Linux will read NTFS (kernel support)

Third-party drivers can fill in the missing (non-Windows) gaps; some Linux distros include them. If your usage is actually just reading from Windows, then NTFS will work out of the box. If you're primarily OS X and not Linux, then exFAT would be better.

Either one can be formatted right in the Windows (File) Explorer GUI on the desktop.

Related Question