Linux – Merging and sorting multiple files with “sort”

linuxsorting

I have a bunch of text logfiles in the following format:

ID          (17 characters)
Timestamp   (14 characters YYYYmmddHHMMSS e.g. "20060210100040" -> 2006/02/10 10:00:40)
Random data (? characters)
end of line

The files are already sorted by timestamp.
I need to get 1 log file with all the logs from multiple logs files, sorted by timestamp. Note that the log files are really huge, around 3-4G each (and there are dozens of them)
I tried the following command:

sort -s -m -t '|' -k1n,1n +17 -o data_sort.txt *.TXT

Here is how I ended up with this command:

-s     : don't bother with tie results
-m     : merge all logs files
-t '|' : there is no | in my logs, so the whole line should be field 1
-k1n,1n: sort on the first field as a numeric value
+17    : the timestamp starts at index 17
-o     : output file

Actually… it fails miserably. The output file data_sort.txt is just the concatenation of all files, not sorted at all 🙁

I would greatly appreciate if anyone could provide any help on this problem!

Thanks

Best Answer

Your key should be -k1.17n and omit the -t and the +17.

Is there a space between the ID and the timestamp? Then the timestamp is field 2 and the key should be -k2.

Related Question