Bash – Why does this command to copy files in a for loop work in bash but not in zsh

bashforzsh

I'm trying to copy a bunch of files with the same name, but in different subdirectories, to a single directory, changing the names to ones based on the paths to the original files. I use a for loop that does what I intend in bash, but behaves very oddly in zsh.

Command (linebreaks added for legibility):

for f in */Flavors/*/stuff1/filename.txt;
    do l=$(echo $f | cut -d'/' --output-delimiter '' -f1,3);
    dest=/stuff2/${l}.utf8.txt;
    echo $f $dest;
    cp -v -- $f $dest;
done

Output in zsh. I intend the output file names to be e.g. "EnglishAU.utf8.txt" but instead they are just "English", "French" and "Spanish". Note that the echo in the loop shows $dest containing the correct path, and then the cp uses the wrong one!

English/Flavors/AU/stuff1/filename.txt /stuff2/EnglishAU.utf8.txt
`English/Flavors/AU/stuff1/filename.txt' -> `/stuff2/English'
English/Flavors/UK/stuff1/filename.txt /stuff2/EnglishUK.utf8.txt
`English/Flavors/UK/stuff1/filename.txt' -> `/stuff2/English'
English/Flavors/US/stuff1/filename.txt /stuff2/EnglishUS.utf8.txt
`English/Flavors/US/stuff1/filename.txt' -> `/stuff2/English'
French/Flavors/CA/stuff1/filename.txt /stuff2/FrenchCA.utf8.txt
`French/Flavors/CA/stuff1/filename.txt' -> `/stuff2/French'
French/Flavors/FR/stuff1/filename.txt /stuff2/FrenchFR.utf8.txt
`French/Flavors/FR/stuff1/filename.txt' -> `/stuff2/French'
Spanish/Flavors/ES/stuff1/filename.txt /stuff2/SpanishES.utf8.txt
`Spanish/Flavors/ES/stuff1/filename.txt' -> `/stuff2/Spanish'
Spanish/Flavors/OT/stuff1/filename.txt /stuff2/SpanishOT.utf8.txt
`Spanish/Flavors/OT/stuff1/filename.txt' -> `/stuff2/Spanish'

As mentioned above, this works as intended in bash. What's zsh doing?

Best Answer

This is happening because cut is outputting NULL characters in the output. You can't pass a program arguments which contain a null character (see this).

In bash this works because bash can't handle NULL characters in strings, and it strips them out. Zsh is a bit more powerful, and it can handle NULL characters. However when it comes time to pass the string to the program, it still contains the null, which signals the end of the argument.

Let's look at this in detail.

$ echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3 | xxd
0000000: 456e 676c 6973 6800 4155 0a              English.AU.

Here we simulated one of your files, passing the path through cut. Notice the xxd output which has a NULL character between English and AU.

Now lets run through and simulate the rest of the script.

$ l=$(echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3)
$ dest=/stuff2/${l}.utf8.txt
$ echo "$dest" | xxd
0000000: 2f73 7475 6666 322f 456e 676c 6973 6800  /stuff2/English.
0000010: 4155 2e75 7466 382e 7478 740a            AU.utf8.txt.

Notice the NULL after the English. The echo displays it properly because echo is a shell built-in. If we use an external echo, it also exhibits the issue.

$ /bin/echo "$dest" | xxd 
0000000: 2f73 7475 6666 322f 456e 676c 6973 680a  /stuff2/English.

P.S. You really should be quoting too :-)

The solution is to not use cut, use awk instead.

$ echo 'English/Flavors/AU/stuff1/filename.txt' | awk -F/ '{ print $1$3 }' | xxd
0000000: 456e 676c 6973 6841 550a                 EnglishAU.
Related Question