Linux – Bash: better way to output to both console and output file than tee

bashbash-scriptingcommand linelinux

What I need to display is a log refreshing periodically. It's a block of about 10 lines of text. I'm using |tee and it works right now. However, the performance is less satisfying. It waits a while and then outputs several blocks of texts from multiple refreshes (especially when the program just starts, it takes quite a while to start displaying anything on the console and the first time I saw this, I thought the program was hanging). In addition, it breaks randomly in the middle of the last block, so it's quite ugly to present.

Is there a way to improve this? (Maybe output less each time and switch between output file and console more frequently?)

Update: Here is what's in my bash script right now:
tail -f /var/log/syslog | egrep --line-buffered "my search string" > tmp.txt & python script.py | tee result.log

Best Answer

I think your problem arises from a fundamental feature of pipes i.e. buffering.

The workarounds are messy but you should look at commands like unbuffer or script or stdbuf.

Perhaps something to stop output bufferring by tee like this:

 your_program | stdbuf -o0 tee

PS. I'm not at a console now so I cannot try this.

Related Solutions

Linux – Filter tail command through multiple grep commands to separate files

It might be easier if you use more than one line to do this. You could write a bash script:

#!/bin/bash

tail -f test.log | while read line; do
    if echo "$line" | grep -q "Error"; then
        echo "$line" >> error.log
    elif echo "$line" | grep -q "Warning"; then
        echo "$line" >> warning.log
    # The following is in case you want to print out lines that do not match 
    else
        echo "$line"
    fi
done

Bash tar and output to log file

The key issue is that

tar -czf arh.tar.gz public_html 1

doesn't create any output at all. So you need at least

tar -czvf arh.tar.gz public_html 1

to create any output. But as the output goes to STDERR instead of STDOUT you will need

tar -czvf arh.tar.gz public_html 1 2> 1.log

to catch any output in the log file

It's usually better to ask new questions instead of following-up in the same topic but here we go

for i in */ ; do
    sitefolder="$HOME/domains/${i%%/}/public_html"
    if [ -d "$sitefolder" ]; then
        echo "Backing up ${i%%/}" >> $logs/backups.log
        tar -czvf $tbackups/$i".tar.gz" public_html 2>> $logs/backups.log
        echo "Backup of ${i%%/} successfull" >> $logs/backups.log
    fi
done

Best Answer

Related Solutions

Linux – Filter tail command through multiple grep commands to separate files

Bash tar and output to log file

Related Question