Shell – How to have a script work with “$@” or a default list of parameters while not breaking paths with whitespace

argumentsshell-script

I want a script that will run another utility over some default paths if no parameters are passed to it; ideally I want this safe for paths that contain spaces.

So far I have script.sh:

#!/bin/sh
base=$(dirname "$0")
exec touch "${@:-"$base/aaa" "$base/bbb"}"

If I put this into a folder called "foo bar" and run it as:

foo\ bar/script.sh

I want it to should end up doing:

touch foo\ bar/aaa foo\ bar/bbb

i.e. create files "aaa" and "bbb" under "foo bar", the directory in which the script is located.

Instead I get the error

touch: cannot touch 'foo bar/aaa foo bar/bbb': No such file or directory

(If I pass in parameters to the script it seems to work fine. Presumably removing the outer quotes in the last command would reverse my cases.)

Best Answer

It appears you can't set default parameters in an expansion of ${@:-...}, and "${@:-"$base/aaa" "$base/bbb"}" is expanded as a single string.

If you want to set default parameters you might want to do this:

base=$(dirname -- "$0")
# test explicitly for no parameters, and set them.
if [ "$#" -eq 0 ]; then
    set -- "$base/aaa" "$base/bbb"
fi

Then, the "$@" magically quoted parameter substitution can happen unabated:

touch -- "$@"

Related Solutions

Shell – POSIX compliant way to work with a list of filenames possibly with whitespace

POSIX shells have one array: the positional parameters ($1, $2, etc., collectively refered to as "$@").

set -- 'foo/target/a.jar' 'foo/target/b.jar' 'bar/target/b.jar' 'bar/target/lol whitespace.jar'
set -- "$@" '/another/one at the end.jar'
…
for jar do
  dostuffwith "$jar"
done

This is inconvenient because there's only one, and it destroys any other use of the positional parameters. Positional parameters are local to a function, which is sometimes a blessing and sometimes a curse.

If your file names are guaranteed not to contain newlines, you can use newlines as the separator. When you expand the variable, first turn off globbing with set -f and set the list of field splitting characters IFS to contain only a newline.

INPUT="foo/target/a.jar
foo/target/b.jar
bar/target/b.jar
bar/target/lol whitespace.jar"
…
set -f; IFS='
'                           # turn off variable value expansion except for splitting at newlines
for jar in $INPUT; do
  set +f; unset IFS
  dostuffwith "$jar"        # restore globbing and field splitting at all whitespace
done
set +f; unset IFS           # do it again in case $INPUT was empty

With items in your list separated by newlines, you can use many text processing commands usefully, in particular sort.

Remember to always put double quotes around variable substitutions, except when you explicitly want field splitting to happen (as well as globbing, unless you've turned that off).

Shell – Will $0 always include the path to the script

In the most common cases, $0 will contain a path, absolute or relative to the script, so

script_path=$(readlink -e -- "$0")

(assuming there's a readlink command and it supports -e) generally is a good enough way to obtain the canonical absolute path to the script.

$0 is assigned from the argument specifying the script as passed to the interpreter.

For example, in:

the-shell -shell-options the/script its args

$0 gets the/script.

When you run:

the/script its args

Your shell will do a:

exec("the/script", ["the/script", "its", "args"])

If the script contains a #! /bin/sh - she-bang for instance, the system will transform that to:

exec("/bin/sh", ["/bin/sh" or "the/script", "-", "the/script", "its", "args"])

(if it doesn't contain a she-bang, or more generally if the system returns a ENOEXEC error, then it's your shell that will do the same thing)

There's an exception for setuid/setgid scripts on some systems, where the system will open the script on some fd x and run instead:

exec("/bin/sh", ["/bin/sh" or "the/script", "-", "/dev/fd/x", "its", "args"])

to avoid race conditions (in which case $0 will contain /dev/fd/x).

Now, you may argue that /dev/fd/x is a path to that script. Note however that if you read from $0, you'll break the script as you consume the input.

Now, there's a difference if the script command name as invoked doesn't contain a slash. In:

the-script its args

Your shell will look up the-script in $PATH. $PATH may contain absolute or relative (including the empty string) paths to some directories. For instance, if $PATH contains /bin:/usr/bin: and the-script is found in the current directory, the shell will do a:

exec("the-script", ["the-script", "its", "args"])

which will become:

exec("/bin/sh", ["/bin/sh" or "the-script", "-", "the-script", "its", "args"]

Or if it's found in /usr/bin:

exec("/usr/bin/the-script", ["the-script", "its", "args"])
exec("/bin/sh", ["/bin/sh" or "the-script" or "/usr/bin/the-script",
     "-", "/usr/bin/the-script", "its", "args")

In all those cases above except the setuid corner case, $0 will contain a path (absolute or relative) to the script.

Now, a script can also be called as:

the-interpreter the-script its args

When the-script as above doesn't contain slash characters, the behaviour varies slightly from shell to shell.

Old AT&T ksh implementations were actually looking up the-script unconditionally in $PATH (which was actually a bug and a security hole for setuid scripts), so $0 actually did not contain a path to the script unless the $PATH lookup actually happened to find the-script in the current directory.

Newer AT&T ksh would try and interpret the-script in the current directory if it's readable. If not it would lookup for a readable and executable the-script in $PATH.

For bash, it checks if the-script is in the current directory (and is not a broken symlink) and if not, lookup for a readable (not necessarily executable) the-script in $PATH.

zsh in sh emulation would do like bash except that if the-script is a broken symlink in the current directory, it would not search for a the-script in $PATH and would instead report an error.

All the other Bourne-like shells don't look the-script up in $PATH.

For all those shells anyway, if you find that $0 doesn't contain a / and is not readable, then it probably has been looked up in $PATH. Then, as files in $PATH are likely to be executable, it's probably a safe approximation to use command -v -- "$0" to find its path (though that wouldn't work if $0 happens to also be the name of a shell builtin or keyword (in most shells)).

So if you really want to cover for that case, you could write it:

progname=$0
[ -r "$progname" ] || progname=$(
    IFS=:; set -f
    for i in ${PATH-$(getconf PATH)}""; do
      case $i in
        "") p=$progname;;
        */) p=$i$progname;;
        *) p=$i/$progname
      esac
      [ -r "$p" ] && exec printf '%s\n' "$p"
    done
    exit 1
  ) && progname=$(readlink -e -- "$progname") ||
  progname=unknown

(the "" appended to $PATH is to preserve a trailing empty element with shells whose $IFS acts as delimiter instead of separator).

Now, there are more esoteric ways to invoke a script. One could do:

the-shell < the-script

Or:

cat the-script | the-shell

In that case, $0 will be the first argument (argv[0]) that the interpreter received (above the-shell, but that could be anything though generally either the basename or one path to that interpreter).

Detecting that you're in that situation based on the value of $0 is not reliable. You could look at the output of ps -o args= -p "$$" to get a clue. In the pipe case, there's no real way you can get back to a path to the script.

One could also do:

the-shell -c '. the-script' blah blih

Then, except in zsh (and some old implementation of the Bourne shell), $0 would be blah. Again, hard to get to the path of the script in those shells.

Or:

the-shell -c "$(cat the-script)" blah blih

etc.

To make sure you have the right $progname, you could search for a specific string in it like:

progname=$0
[ -r "$progname" ] || progname=$(
    IFS=:; set -f
    for i in ${PATH-$(getconf PATH)}:; do
      case $i in
        "") p=$progname;;
        */) p=$i$progname;;
        *) p=$i/$progname
      esac
      [ -r "$p" ] && exec printf '%s\n' "$p"
    done
    exit 1
  ) && progname=$(readlink -e -- "$progname") ||
  progname=unknown

[ -f "$progname" ] && grep -q 7YQLVVD3UIUDTA32LSE8U9UOHH < "$progname" ||
  progname=unknown

But again I don't think it's worth the effort.

Best Answer

Related Solutions

Shell – POSIX compliant way to work with a list of filenames possibly with whitespace

Shell – Will $0 always include the path to the script

Related Question