I have tried using all the commands mention in Byte Commander's answer to the question here but they do not work unless the program finishes.
I am running a python script like so 'python script.py' and have tried replacing 'command' in Byte Commander's answer with 'python script.py' however the terminal output is not shown in the terminal anymore with any of the commands and only gets written to the file output.txt in the case that the python script completes (actually I've discovered it comes in large chunks as it proceeds, but not line-by-line as I require). I believe it may be because the python script calls another non-python program in another shell (it calls a finite element package called gmesh). The code takes a long time (several hours) to complete and I want to be able to see the output written to the file as it proceeds so I can see it's progress and what the terminal output looks like even if the program crashes part way through. How might I be able to go about this?
I've added a video to demonstrate the issue. When the program runs without capture the information comes line by line. When I attempt to capture the output it comes in chunks, when these chunks takes hours and it crashes during this process I get no information on where it crashed.
Best Answer
You can do as in the following example.
Suppose we have a script called
myscript.py
that looks like this:Then if we run it like this:
it will hang there until it completes, so we will not see the output while it is running.
To be able to see the output while it is running, we can do this instead:
where the ampersand means that the terminal will not hang, we can give more commands while it is running. Then, we can do this to look at the contents of the log file while myscript.sh is writing to it:
(Another possibility is to open a separate terminal window and do
tail -f mylog.txt
from there.)Note that for this to work, the
flush=True
part of theprint
line is important, withoutflush=True
the file contents will only be seen after the program completes.In case the output is generated from some other program inside your python script, you can try adding
import sys
and then doingsys.stdout.flush()
in the python code after calling the other program. See https://stackoverflow.com/a/230774/6708867