Shell – Kill process when it stops to produce further output

killoutputprocessshellstdouttimeout

Is there a shell tool which can probe if a subprocess is still producing output, and otherwise kill it after a specified timeout?

I'm using offlineimap for Gmail backup, but that's very very very flaky. Processing stops regularily due to IMAP conditions or whatever. The process needs to be constantly restarted to see any progress. Hencewhy I'm currently using:

(ulimit -t 300 ; offlineimap)

in a loop. Which restarts the tool every five minutes; no matter what.

That's somewhat unsatisfactory, as it either kills the process prematurely the few times when it's actually still performing some work, or still waits 4 minutes longer when it's already blocking. I would rather have something like:

offlineimap > output.log &
stillmakingoutput? --timeout 60 output.log || kill $!

That's probably doable with a custom exec/watching script, but isn't there some generic approach for such cases?


Best Answer

So succeeded with a "little" timeout script that checks the log file size to determine if the process is still working. Not pretty, but did help:

#!/bin/sh
TIMEOUT=$1
FILE=$2
PID=$3
if [ -z "$PID" ]
then
   echo $0 timeout file pid
   echo "     e.g. 60 /tmp/log 16325"
   exit
fi
echo "stalekill: timeout=$TIMEOUT file=$FILE pid=$PID"    

sleep 1

SIZE=0
while true
do
   sleep 3

   NEWSIZE=$(stat -c%s "$FILE")
   if [ "$NEWSIZE" -eq "$SIZE" ]
   then
      echo "stalekill '$FILE' unchanged; $SIZE"

      sleep $TIMEOUT
      NEWSIZE=$(stat -c%s "$FILE")

      if [ "$NEWSIZE" -eq "$SIZE" ]
      then
         kill $PID || sleep 2 && kill -9 $PID
         exit
      fi
      echo "stalekill '$FILE' changed; $SIZE..$NEWSIZE"
   else
      echo "stalekill '$FILE' changed; $SIZE..$NEWSIZE"
   fi
   SIZE=$NEWSIZE 
done

Invoked as follows (in a loop):

./offlineimap.py >> ./log.txt 2>&1   &   stalekill 25 ./log.txt $!

The timing is not overly professional, and it would be even nicer if it also checked if the process already terminated itself; but basically did the feat.

But nevertheless, if anybody knows a different or more standardized solution to this task: accepted answer tick still available.

Related Question