Bash – How to redirect stdout to a file and read from the same file simultaneously with another task

bashfilesio-redirectionshellstdout

I want to run a task (which takes quite a long time) remotely in Ubuntu Linux Bash via nohup and redirect stdout to a file, e.g.:

nohup task > out.txt &

From time to time I would like to check the progress of the task by looking into the output file:

cat out.txt

I haven't yet figured out how to do that because the file's content is shown as empty as long as the file is opened for writing by Bash.

So my question is:
Is there any linux command that allows a 'shared' stdout redirection?

Best Answer

the file's content is shown as empty as long as the file is opened for writing by Bash.

That's not exactly what's happening. What's happening is that the task buffers its output — it is accumulated in memory for a while, then written to the file in chunks. The in-memory buffer has a fixed size of a few kilobytes. This is done for performance, as each file write has a relatively high overhead. So you won't see your program's output at the beginning, but you will see it come piece by piece. This has nothing to do with bash.

If you want to see the output immediately, you can start your program with unbuffer (from the expect software package). You trade performance for immediacy.

A second issue is that cat will show the output that has already been emitted, and exit as soon as it reaches the end of the file. If you want to keep reading output when it's appended, use tail's follow mode.

nohup unbuffer task >out.txt &
tail -n +1 -f out.txt

Press Ctrl+C to exit tail. Instead of tail, you can use less and its follow mode (press F; press Ctrl+C to stop reading).