Ubuntu – Remove any trailing blank lines or lines with whitespaces from end of file

command linetext processing

I want to get delete all of the blank lines and lines with spaces (if any exist (ONLY from the bottom of the file)) and then to remove one more line (also ONLY from the bottom of the file).

I have this code:

while [[ "$last_line" =~ $ ]] || [[ "$last_line" =~ ^[[:space:]]+$ ]]
do
    sed -i -e '${/$/d}' "./file.txt"
done
    sed -i -e '${/$/d}' "./file.txt"

For some reason the loop doesn't stop and it deletes everything in the file. What is the matter?

Best Answer

Your script should work if fixed like so:

while
 last_line=$(tail -1 "./file.txt")
 [[ "$last_line" =~ ^$ ]] || [[ "$last_line" =~ ^[[:space:]]+$ ]]
do
 sed -i '$d' "./file.txt"
done

Your script had two main problems: (1) you never updated $last_line, so the loop's guard would always evaluate the same thing; (2) your [[ "$last_line" =~ $ ]] test matched any line, since any line has an end. (This is the reason why your script emptied your file completely.) You probably want to match against ^$ instead, which matches only empty lines. Additionally, I simplified the sed command to delete the last line in the loop's body (simply $d does the job).

However, this script is unnecessarily complicated. sed is there for just that kind of thing! This one-liner will do the same thing as the above script:

sed -i ':a;/^[ \n]*$/{$d;N;ba}' ./file.txt

Roughly,

  1. Match current line against ^[ \n]*$. (i.e, can only contain whitespaces and newlines)
  2. If it doesn't match, just print it. Read in next line and continue with step 1.
  3. If it does match,
    • If we are at the end of the file, delete it.
    • If we are not at the end of the file, append the next line to the current line, inserting a newline character between the two, and go back to step 1 with this new, longer line.

There are lots of awesome sed tutorials on the Internet. For example, I can recommend this one. Happy learning! :-)

Update: And of course, if you additionally want to remove the last (non-blank) line of the file after having truncated the trailing blank lines, you can just use another sed -i '$d' ./file.txt after either your script or the above one-liner. I intentionally did not want to include that in the sed one-liner since I thought that removing trailing blank lines is quite a reusable piece of code that may be interesting for other people; but removing the last non-blank line is really specific to your use case, and trivial anyway once you removed the trailing blank lines.

Related Question