Bash – A shell script to write selected fields to a single line

bashbash-scriptcutgrepshell-script

I have a log file trace.log which prints the time stamp, thread name and Transaction method and transaction ID as below.

2014-01-23 15:50:41,724 [catalina-exec-35] INFO  TRANSACTION getConnection REQUEST, ID=1308::
2014-01-23 15:50:41,725 [catalina-exec-33] INFO  TRANSACTION getConnection REQUEST, ID=1304::
2014-01-23 15:50:41,727 [catalina-exec-10] INFO  TRANSACTION getConnection REQUEST, ID=1298::
2014-01-23 15:50:41,727 [catalina-exec-24] INFO  TRANSACTION getConnection REQUEST, ID=1307::
2014-01-23 15:50:41,727 [catalina-exec-12] INFO  TRANSACTION getConnection DONE, ID=1305::
2014-01-23 15:50:41,733 [catalina-exec-10] INFO  TRANSACTION getConnection DONE, ID=1298::
2014-01-23 15:50:41,734 [catalina-exec-26] INFO  TRANSACTION getConnection REQUEST, ID=1313::
2014-01-23 15:50:41,734 [catalina-exec-26] INFO  TRANSACTION getConnection DONE, ID=1313::
2014-01-23 15:50:41,738 [catalina-exec-39] INFO  TRANSACTION getConnection REQUEST, ID=1311::
2014-01-23 15:50:41,733 [catalina-exec-35] INFO  TRANSACTION getConnection DONE, ID=1308::
2014-01-23 15:50:41,738 [catalina-exec-27] INFO  TRANSACTION getConnection REQUEST, ID=1309::
2014-01-23 15:50:41,737 [catalina-exec-22] INFO  TRANSACTION getConnection REQUEST, ID=1310::
2014-01-23 15:50:41,743 [catalina-exec-30] INFO  TRANSACTION getConnection REQUEST, ID=1315::
2014-01-23 15:50:41,744 [catalina-exec-39] INFO  TRANSACTION getConnection DONE, ID=1311::
2014-01-23 15:50:41,747 [catalina-exec-2] INFO  TRANSACTION getConnection REQUEST, ID=1318::

I want to grep and print the time stamp of the getConnection REQUEST and getConnection DONE for a particular ID in a single line into a file.

I have a written a shell script that printing the time stamp in multiple lines as below.
Here is my shell script

for i in {1..800}
do
     echo "Welcome $i times"
     echo "ID=$i, getConnection " >> time.log
     grep ID=$i: trace.log | grep getConnection | cut -d'[' -s -f1 >> time.log
echo "      " >> time.log
done

The output is as shown below

ID=791, getConnection
2014-01-23 15:50:16,703
2014-01-23 15:50:16,706

ID=792, getConnection
2014-01-23 15:50:16,704
2014-01-23 15:50:16,704

ID=793, getConnection
2014-01-23 15:50:16,704
2014-01-23 15:50:16,709

ID=794, getConnection
2014-01-23 15:50:16,708
2014-01-23 15:50:16,712

How I can change that? I need output as shown below:

ID=792, getConnection 2014-01-23 15:50:16,703 2014-01-23 15:50:16,706

ID=792, getConnection 2014-01-23 15:50:16,704 2014-01-23 15:50:16,704

ID=793, getConnection 2014-01-23 15:50:16,704 2014-01-23 15:50:16,709

ID=794, getConnection 2014-01-23 15:50:16,708 2014-01-23 15:50:16,712

Best Answer

Don't use shell loops to process text, that's bad practice.

The shell's job is to run commands (the right ones) and make them cooperate to a task.

Here, the right command is the practical extraction and report language interpreter called once, not several commands run (in sequence!) for each line of a file.

perl -lne '
  if (/(.*?) \[.*getConnection (.*?), (ID=\d+)/) {
    if ($2 eq "REQUEST") {$r{$3}=$1}
    elsif ($2 eq "DONE") {print "$3, getConnection $r{$3} $1" if $r{$3}}
  }' < your-file
Related Question