IO Redirection – Using xargs with stdin/stdout Redirection

io-redirectionxargs

I would like to run:

./a.out < x.dat > x.ans

for each *.dat file in the directory A.

Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):

ls A/*.dat | xargs -I file -a file ./a.out

But -a in xargs doesn't understand replace-str 'file'.

Thank you for help.

Best Answer

First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.

1. Simple way: for loop

If you want to process just the files under A/, then a simple for loop should be enough:

for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done

2.pre1 Why not   ls | xargs ?

Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:

  • first, let's create some empty files:

    $ touch A/mypreciousfile.dat\ with\ junk\ at\ the\ end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans

  • see the files and that they contain nothing:

    $ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans

$ cat A/*

  • run a magic command using xargs:

    $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"

  • the result:

    $ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans

$ cat A/mypreciousfile.dat.ans
TRICKED

So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.

2. Using  xargs : the proper way with  find 

If you'd like to insist on using xargs, use -0 (null-terminated names) : 

find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'

Notice two things:

  1. this way you'll create files with .dat.ans ending;
  2. this will break if some file name contains a quote sign (").

Both issues can be solved by different way of shell invocation:

find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'

3. All done within find ... -exec

 find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' \;

This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:

 find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} \;
Related Question