Suppose I have a executable which for which I want to log STDOUT and STDERR to separate files, like so:
python write.py > out 2> err
When I use this command with GNU timeout
, my out
file is always empty. Why does this happen, and how can I fix this?
timeout 5s python write.py > out 2> err
Example write.py:
#!/bin/bash
import sys, time
i = 0
while True:
print i
print >> sys.stderr, i
time.sleep(1)
i += 1
Best Answer
python
uses buffered writes onstdout
but not onstderr
. So everything written tosys.stderr
is written immediately but stuff forsys.stdout
is kept in a buffer until the buffer is full to minimise write operations. Within 5 seconds, the buffer does not fill enough to get written even once andtimeout
terminates thepython
interpreter.Adding the
-u
option topython
's invocation, writes tostdout
will be unbuffered too.You can force the same behaviour for a Python program by setting the environment variable
PYTHONUNBUFFERED
. For other programs you can useunbuffer
from the expect package orstdbuf
from GNU coreutils (see Turn off buffering in pipe).