Using awk with find -exec

awkfindunix

I have a directory structure with 14 directories containing a bunch of files containing data in a three-column format (separated with tabs). I intended to use find and awk to extract the second column from each of those files and output it with the same filename but under a different root folder. Here the sketch of my directory.

data/all -> AA, AB, AC, AD … (A* being folders containing the files with data stored in a 3-column format, e.g. AA100.txt, AA101.txt …)

i want to have the modified (one-column) files with the same name, but all under a new root directory data/pos (as opposed to data/all/) -> AA, AB, AC, AD … (again, each containing A*100.txt, A*101…)

My try was to use find -exec and to give it the awk command, but I'm having issues with outputting the file to the right place.

when being in data/all/

find * -type f -exec awk '{print$2}' '{}' > ../pos/'{}' \;

However {} as wildcard for the input file doesn't seem to work when outputting the file?

What am I doing wrong? (I'm on a ubuntu server btw)

Best Answer

You could try without find, if all you want is all the files. While in data/all/, run this:

for file in ./*; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done

If you want to cover the files in the whole hierarchy under /data/all, you can enable the globstar option if you are using bash (I believe this would "just work" on zsh) and then use ** to match all files:

shopt -s globstar
for file in ./**; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done
Related Question