Linux – Expanding globs in xargs

bashlinuxmacosunixxargs

I have a directory like this

mkdir test
cd test
touch file{0,1}.txt otherfile{0,1}.txt stuff{0,1}.txt

I want to run some command such as ls on certain types of files in the directory and have the * (glob) expand to all possibilities for the filename.

echo 'file otherfile' | tr ' ' '\n' | xargs -I % ls %*.txt

This command does not expand the glob and tries to look for the literal 'file*.txt'

How do I write a similar command that expands the globs?
(I want to use xargs so the command can be run in parallel)

Best Answer

This is old, but i ran into this issue and found another way to fix it. Have xargs execute bash with -c and your command. For example:

echo 'file otherfile' | tr ' ' '\n' | xargs -I % bash -c "ls %*.txt"

This forces glob expansion.

Related Solutions

Subshell arguments in -exec parameter for find(1)

exec does what it says: uses exec(). It doesn't involve a shell unless it's told to (-exec bash ...), and if it did you would still need single quotes to keep your interactive shell from interpreting the variable interpolations. (The shell is not magic and does not know that you intended those to be interpolated by something else.)

For example, when you use the following command:

find . -type f -exec echo $(file={}; echo ${file:0:5}) \;

your shell first performs process substitution by executing $(file={}; echo ${file:0:5}), which simply outputs {}, then executes the final command:

find . -type f -exec echo {} \;

Linux – Spawning multiple parallel wgets and storing results in a bash array to be pretty printed when all wgets are done

Make a small script that does the right thing given a single url (based on terdon's code):

#!/bin/bash

url=$1
echo "=======================================";
echo "$url"
echo "=======================================";
echo -n "First Hit: Caching: ";
wget --server-response -q -O - $url 2>&1 | grep Caching >/dev/null
if [ $? == 0 ]; then echo HIT; else echo MISS; fi;
echo -n "Second Hit: Caching: ";      
wget --server-response -q -O - $url 2>&1 | grep Caching >/dev/null
if [ $? == 0 ]; then echo HIT; else echo MISS; fi; echo "";

Then run this script in parallel (say, 500 jobs at a time) using GNU Parallel:

cat urls.txt | parallel -j500 my_script

GNU Parallel will make sure the output from two processes are never mixed - a guarantee xargs does not give.

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

wget -O - pi.dk/3 | sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Best Answer

Related Solutions

Subshell arguments in -exec parameter for find(1)

Linux – Spawning multiple parallel wgets and storing results in a bash array to be pretty printed when all wgets are done

Related Question