Bash – Grep over multiple files redirecting to a different filename each time

bashfilenamesgrepio-redirection

I have a directory full of .tsv files and I want to run a grep command on each of them to pull out a certain group of text lines and then save it to an associated text file with a similar file name. So for example, if I was grepping just one of the files, my grep command looks like this:

grep -h 8-K 2008-QTR1.tsv > 2008Q1.txt

But I have a list of tsv files that look like:

2008-QTR1.tsv
2008-QTR2.tsv
2008-QTR3.tsv
2008-QTR4.tsv
2009-QTR1.tsv
2009-QTR2.tsv
2009-QTR3.tsv
...

And after grepping they need to be stored as:

2008Q1.txt
2008Q2.txt
2008Q3.txt
2008Q4.txt
2009Q1.txt
2009Q2.txt
2009Q3.txt

Any thoughts?

Best Answer

In ksh93/bash/zsh, with a simple for loop and parameter expansion:

for f in *-QTR*.tsv
do 
  grep 8-K < "$f" > "${f:0:4}"Q"${f:8:1}".txt
done

This runs the grep on one file at a time (where that list of files is generated from a wildcard pattern that requires "-QTR" to exist in the filename as well as a ".tsv" ending to the filename), redirecting the output to a carefully-constructed filename based on:

the first four characters of the filename -- the year
the letter Q
the 9th character of the filename -- the quarter

Related Solutions

How to extract parts of file into separate files in linux using awk or grep or sed commands

I would try

awk '{print >> $1 ".txt" ;}'

where

print print the whole line.
>> $1 ."txt" write (append) this line to the file indicated by $1, with .txt added.

edit:

in case you have comment lines, lines with dots, etc

try

awk '$1 ~ /cha/ {print >> $1 ".txt" ;}'

which would only fill file begining with 'cha'.

Printing filename along with grep results in find -exec

To get each filename printed to the output and THEN have the grep results corresponding to that file printed after, you could wrap the -exec foo {} | grep pipe in a shell:

find . -name '*.o' -type f -print -exec sh -c 'foo "$1" | grep "bar"' sh {} \;

To to make the -H argument of grep work with standard input, if your version of grep supports the --label= option you could do

find . -name '*.o' -type f -exec sh -c '
  foo "$1" | grep -H --label="$1" "bar"
' sh {} \;

or (if your find supports the + multi-argument alternative to \;):

find . -name '*.o' -type f -exec sh -c '
  for f; do foo "$f" | grep -H --label="$f" "bar"; done
' sh {} +

Best Answer

Related Solutions

How to extract parts of file into separate files in linux using awk or grep or sed commands

Printing filename along with grep results in find -exec

Related Question