Bash – Pipe List of Values to Script

argumentsbashxargs

I have a script. myScript.

It takes two args. myScript arg1 arg2.

Let's say I have three sets of arguments.

argA1 argB1
argA2 argB2
argA3 argB3

Is there a way to pipe each line to myScript. I am open to having the set of args in a file, maybe use xargs. Not certain what to investigate.

So I would run command like faux example below:

cat myFile | xargs > myScript

Not to have another script, just commands on command line.

Best Answer

Just:

xargs -l1 <myFile myScript

-l1 tells xargs to run myScript once for each 1 line of input, with the words on that 1 line passed as separate arguments.

For xargs, words are separated with whitespace, and you can still have whitespace (even newlines) in a words by using quoting. Note however that while xargs supports "...", '...' and \ for quoting like most shells, its parsing is different from that of Bourne-like shells. In particular, there's no backslash processing in neither "..." nor '...' and only \ can escape a newline.

For for instance, myFile could contain:

word1.1 word1.2
"word 2.1" 'word 2.2'
"word'3.1" 'word"3.2'
'word"'"4'1" word\"4\'2
'word '\
5.1 "word "\
5.2

And

xargs -l1 <myFile printf 'First word: "%s", second word: "%s"\n'

would produce:

First word: "word1.1", second word: "word1.2"
First word: "word 2.1", second word: "word 2.2"
First word: "word'3.1", second word: "word"3.2"
First word: "word"4'1", second word: "word"4'2"
First word: "word
5.1", second word: "word
5.2"

With the GNU implementation of xargs, you can improve it to:

xargs -l1 -r -a myFile myScript

Which has the benefit of leaving stdin alone and of not running myScript if myFile is empty.

Related Solutions

Bash how to remove options from parameters after processing

POSIXly, the parsing for options should stop at -- or at the first non-option (or non-option-argument) argument whichever comes first. So in

cp -R file1 -t /mybackup file2 -f

that's at file1, so cp should recursively copy all of file1, -t, /mybackup and file2 into the -f directory.

GNU getopt(3) however (that GNU cp uses to parse options (and here you're using GNU cp since you're using the GNU-specific -t option)), unless the $POSIXLY_CORRECT environment variable is set, accepts options after arguments. So it is actually equivalent to POSIX option style parsing's:

cp -R -t /mybackup -f -- file1 file2

The getopts shell built-in, even in the GNU shell (bash) only handles the POSIX style. It also doesn't support long options or options with optional arguments.

If you want to parse the options the same way as GNU cp does, you'll need to use the GNU getopt(3) API. For that, if on Linux, you can use the enhanced getopt utility from util-linux (that enhanced version of the getopt command has also been ported to some other Unices like FreeBSD).

That getopt will rearrange the options in a canonical way which allows you to parse it simply with a while/case loop.

$ getopt -n "$0" -o t:Rf -- -Rf file1 -t/mybackup file2
 -R -f -t '/mybackup' -- 'file1' 'file2'

You'd typically use it as:

parsed_options=$(
  getopt -n "$0" -o t:Rf -- "$@"
) || exit
eval "set -- $parsed_options"
while [ "$#" -gt 0 ]; do
  case $1 in
    (-[Rf]) shift;;
    (-t) shift 2;;
    (--) shift; break;;
    (*) exit 1 # should never be reached.
  esac
done
echo "Now, the arguments are $*"

Also note that that getopt will parse options the same way as GNU cp does. In particular, it supports the long options (and entering them abbreviated) and honours the $POSIXLY_CORRECT environment variables (which when set disables support for options after arguments) the same way GNU cp does.

Note that using gdb and printing the arguments that getopt_long() receives can help building the parameters to getopt(1):

(gdb) bt
#0  getopt_long (argc=2, argv=0x7fffffffdae8, options=0x4171a6 "abdfHilLnprst:uvxPRS:T", long_options=0x417c20, opt_index=0x0) at getopt1.c:64
(gdb) set print pretty on
(gdb) p *long_options@40
$10 = {{
    name = 0x4171fb "archive",
    has_arg = 0,
    flag = 0x0,
    val = 97
  }, {
    name = 0x417203 "attributes-only",
[...]

Then you can use getopt as:

getopt -n cp -o abdfHilLnprst:uvxPRS:T -l archive... -- "$@"

Remember that GNU cp's list of supported options may change from one version to the next and that getopt will not be able to check if you pass a legal value to the --sparse option for instance.

Best Answer

Related Solutions

Bash how to remove options from parameters after processing

Related Question