Text Processing Tail Logs – How to Keep Only the Last N Lines of a Log File

logstailtext processing

A script I wrote does something and, at the end, appends some lines to its own logfile. I'd like to keep only the last n lines (say, 1000 lines) of the logfile. This can be done at the end of the script in this way:

tail -n 1000 myscript.log > myscript.log.tmp
mv -f myscript.log.tmp myscript.log

but is there a more clean and elegant solution? Perhaps accomplished via a single command?

Best Answer

It is possible like this, but as others have said, the safest option is the generation of a new file and then a move of that file to overwrite the original.

The below method loads the lines into BASH, so depending on the number of lines from tail, that's going to affect the memory usage of the local shell to store the content of the log lines.

The below also removes empty lines should they exist at the end of the log file (due to the behaviour of BASH evaluating "$(tail -1000 test.log)") so does not give a truly 100% accurate truncation in all scenarios, but depending on your situation, may be sufficient.

$ wc -l myscript.log
475494 myscript.log

$ echo "$(tail -1000 myscript.log)" > myscript.log

$ wc -l myscript.log
1000 myscript.log
Related Question