How to deal with characters like “:” or “?” that make invalid filenames

fat32filenamesfilesystemsthunar

I just tried to move a directory containing music files with thunar 4.10
It complained that a file name was invalid.

It turned out that one file name (song title) contained a question mark.
I suspected that this was a problem, removed the question mark and could indeed copy the file.
Adding the "?" back in was not possible. I also tried it with rename on the command line but that didn't work either. (not sure what thunar uses under the hood, so this test might be moot)

Now if a question mark makes the file name invalid, how could this file be created in the first place? I created the files with SoundJuicer from a newly obtained CD. I was able to play the file (with "?" in the name) in various players.

What's going on here? Can I have the "?" in the name or not? Why is the file manager unable to handle such files while other applications seem to be ok with it?

Update:
Next song has a ":" in it. Same problem as with the "?".

These are not invalid characters to Unix; typically only the NUL character and the / character are invalid filenames (the / being the directory separator).

This was what my intuition told me as well, because I never had any issues with file names in Linux and could throw pretty much everything sensible at it and it worked ok. This is what motivated the question here. I never encountered invalid file names before.

Were you trying to move the files to a USB stick? If so, is that stick formatted as FAT32 or as a native Linux filesystem?

The target is indeed a USB stick that I bought today. I opened gparted and it is formatted as FAT32.

I'm not exactly sure but that's a Windows thing right? And Windows has a bunch of characters that it doesn't support, apaprently including ? and :. Am I right?

Best Answer

These characters ? and : are not valid on a FAT32 filesystem, so if that is where you need to copy your files you will need to rename them.

From the command line you can use command-line tools such as rename (sometimes known as prename) to replace these characters with _ or even to remove them:

rename 's/[?<>\\:*|\"]/_/g'    # Change invalid characters to _
rename 's/[?<>\\:*|\"]//g'     # Remove invalid characters

I am not familiar with thunar so I do not know if there is a way to perform this substitution/replacement operation directly.

I have just found Linux copy to fat32 filesystem: invalid argument which suggests adding this into the pax command (another tool to copy files), so that you can keep your full filenames on your local disk but convert the filenames during the copy to your USB device:

pax -rw -s '/[?<>\\:*|\"]/_/gp' *.mp3 /media/usb_device

If the complete filenames are really important to you, I would suggest that you reformat the USB stick to use a Linux-native filesystem such as ext4. (There are Windows drivers available for the extN family of filesystems if that's necessary.)

Related Question