Bash – can file expansion work for filenames with a newline character

bash

$ rm Think\ Python:\ How\ to\ Think\ Like\ a\ Computer
\ Scientist\ 2014.pdf 
rm: cannot remove ‘Think Python: How to Think Like a Computer’: No such file or directory
$  Scientist 2014.pdf: command not found
$ rm Think*
$

In the first rm, I use autocompletion in bash to specify a file with a newline character in its filename, and it doesn't work because of the newline character.

In the second rm, I use file expansion to avoid explicitly to specify the new line character. Why can file expansion avoid the problem? Doesn't file expansion expand to the full filename which contains the new line character too?

Best Answer

In the case of rm of the file ending in \n, rm file*, the regexp file expansion "eats" the new line as part of the rm argument, so the shell does not see it and does not parse it as a command.

As seen in a strace:

$strace rm teste.txt* 
execve("/bin/rm", ["rm", "teste.txt\n"],

As for autocompletion obeys to a set of pre-defined rules, and it does not really expect to have a '\n' there.

Related Solutions

Bash – Sum of filesize of a list of files

With GNU stat:

stat -c %s -- $(<list) | paste -d+ -s - | bc

stat displays information about the file
- -c specifies the format, %s gives the filesize in bytes
paste -d+ -s concats the output together line by line with a + as delimiter
bc piped to bc, it will be calculated together.

Add a -L option to stat, if for symlinks, you'd rather count the size of the file that the symlink eventually resolves to.

That assumes a shell like ksh, bash or zsh with the $(<file) operator to invoke split+glob on the content of a file.

Here list is expected to be a space, tab or newline (assuming the default value of $IFS) delimited list of file patterns (as in *.txt /bin/*). For a list of file paths, one per line, you'd need to disable globbing and limit $IFS to newline only, or with GNU xargs:

xargs -rd '\n' -a list stat -c %s -- | paste -sd+ - | bc

Bash – Read a line-oriented file which may not end with a newline

You'd do:

while IFS= read -r url || [ -n "$url" ]; do
  printf '%s\n' "$url"
done < url.list

(effectively, that loop adds back the missing newline on the last (non-)line).

Best Answer

Related Solutions

Bash – Sum of filesize of a list of files

Bash – Read a line-oriented file which may not end with a newline

Related Question