AWK – Handling Multiple Input and Output Files

awkbashperl

I have a folder full of text files named 000.txt to 181.txt. How can I process all of them (on the command line) with the same awk script (program.awk) and send them to their respective output files (output000.txt – output181.txt)?

Best Answer

Suppose that we have a folder full of text files named 000.txt to 181.txt. Here is an example of processing all of them with the same awk script and send them to respective output files (output000.txt - output181.txt):

awk 'NR>6{print>("output" FILENAME)}' {000..181}.txt

If we are using bash, then {000..181}.txt will expand to the names of our 182 input files.

In awk (this may require GNU awk), FILENAME is the name of the input file that awk is currently working on. Thus ("output" FILENAME) is the name of our current output file.

Intended just as an example, the above simply prints all but the first six lines of the input file to the output file. More complicated programs would use the same principles.

Related Solutions

Reading output of shell command in awk script

Figured it out, but please feel free to add better answers

cmd = "dig +short -x " ;
cmd ip_address | getline hostname;
close(cmd)

Then I can use the hostname anywhere in the script.

Using awk with find -exec

You could try without find, if all you want is all the files. While in data/all/, run this:

for file in ./*; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done

If you want to cover the files in the whole hierarchy under /data/all, you can enable the globstar option if you are using bash (I believe this would "just work" on zsh) and then use ** to match all files:

shopt -s globstar
for file in ./**; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done

Best Answer

Related Solutions

Reading output of shell command in awk script

Using awk with find -exec

Related Question