Fastest way to read last line within pattern of multiple files with the same extension

command linefilesscriptingtext processing

I have in a directory /home/files/ some files that all have the same extension .txt.

For example

1.txt
2.txt
3.txt
4.txt
5.txt
test.txt

These files are all the same in data format but the data are different.
All these files have at least the following line inside

frames=[number here]

This line appears multiple times in a file so i need to get the value of frames= that appears last in the file.

And as i stated i want to get that line, from all the above files that match the extension .txt in a directory (no need for recursive).

Is there any single bash command to do that?

EDIT

Expected output:

1.txt=5000
2.txt=123
3.txt=3452
4.txt=111
5.txt=958
test.txt=1231

Best Answer

With find + grep:

find . -name '*.txt' -exec sh -c 'grep -HPo "frames=\K.*" "$0" | tail -n1' {} \;

And with shell for loop + grep in similar fashion:

for file in *.txt; do grep -HPo 'frames=\K.*' "$file" | tail -n1; done

Related Solutions

Unix Command to return all files that end with single digit and TXT extension

One of the most basic tools for locating files (or other kinds of nodes) is the find utility.

find ./ -type f -name '*[!0-9][0-9].txt'

This will search:

...recursivly from the current directory (./). You could change this to another path or even leave it off since this is the default value for most versions of find.
...for items that are files as opposed to directories, devices nodes, symbolic links, etc (-type f). You could leave this off if you wanted to find other types too.
...for items that match the name pattern given. Note that I have used single quotes to encose the pattern so that bash or your shell does not try to expland it to a glob pattern before find even gets the command. The star matches any number of characters, then the end of the file must be something other than a digit, then a digit, then your extention. (-name '*[0-9].txt')

If there are files whose name is only a digit followed by .txt, the command above will miss them, because it requires one non-digit before the digit. The following equivalent commands use boolean operators to include file names with just a digit as well (-o is “or” and ! is “not”):

find ./ -type f \( -name '*[!0-9][0-9].txt' -o -name '[0-9].txt' \)
find ./ -type f -name '*[0-9].txt' ! -name '*[0-9][0-9].txt'

Note that this will be case sensitive. If you would like an insensitive match, you can use -iname instead of -name to match things like file4.TXT as well. Also be aware that just because a file claims to be a text file with that extention does not mean it is. On linux any file can be any type no matter the name. There might also be text files with other extentions or with no extention at all.

Shell Script – How to Concatenate Multiple Files with Same Header

If you know how to do it in R, then by all means do it in R. With classical unix tools, this is most naturally done in awk.

awk '
    FNR==1 && NR!=1 { while (/^<header>/) getline; }
    1 {print}
' file*.txt >all.txt

The first line of the awk script matches the first line of a file (FNR==1) except if it's also the first line across all files (NR==1). When these conditions are met, the expression while (/^<header>/) getline; is executed, which causes awk to keep reading another line (skipping the current one) as long as the current one matches the regexp ^<header>. The second line of the awk script prints everything except for the lines that were previously skipped.

EDIT

Best Answer

Related Solutions

Unix Command to return all files that end with single digit and TXT extension

Shell Script – How to Concatenate Multiple Files with Same Header

Related Question