Vi – Does Vi Silently Add a Newline (LF) at the End of File?

historytext;vi

I have trouble understanding a weird behavior: vi seems to add a newline (ASCII: LF, as it is a Unix (AIX) system) at the end of the file, when I did NOT specifically type it.

I edit the file as such in vi (taking care to not input a newline at the end):

# vi foo   ## Which I will finish on the char "9" and not input a last newline, then `:wq`
123456789
123456789
123456789
123456789
~
~
  ## When I save, the cursor is just above the last "9", and no newline was added.

I expect vi to save it "as is", so to have 39 bytes: 10 ASCII characters on each of the first three lines (numbers 1 to 9, followed by a newline (LF on my system)) and only 9 on the last line (characters 1 to 9, no terminating newline/LF).

But it appears when I save it it is 40 bytes (instead of 39), and od shows a terminating LF:

# wc foo
       4       4      40 foo  ## I expected 39 here! as I didn't add the last newline
# od -a toto
0000000    1   2   3   4   5   6   7   8   9  lf   1   2   3   4   5   6
0000020    7   8   9  lf   1   2   3   4   5   6   7   8   9  lf   1   2
0000040    3   4   5   6   7   8   9  lf
0000050
     ## An "lf" terminates the file?? Did vi add it silently?

If I create the file with a printf doing exactly what I did inside vi, it works as expected:

# ## I create a file with NO newline at the end:
# printf "123456789\n123456789\n123456789\n123456789" > foo2
# wc foo2  ## This one is as expected: 39 bytes, exactly as I was trying to do above with vi.
       3       4      39 foo  ## As expected, as I didn't add the last newline

  ## Note that for wc, there are only three lines!
  ## (So wc -l doesn't count lines; it counts the [newline] chars... Which is rather odd.)

# root@SPU0WMY1:~  ## od -a foo2
0000000    1   2   3   4   5   6   7   8   9  lf   1   2   3   4   5   6
0000020    7   8   9  lf   1   2   3   4   5   6   7   8   9  lf   1   2
0000040    3   4   5   6   7   8   9
0000047                                ## As expected, no added LF.

Both files (foo (40 characters) and foo2 (39 characters) appear exactly the same if I re-open them with vi…

And if I open foo2 (39 characters, no terminating newline) in vi and just do :wq without editing it whatsoever, it says it writes 40 chars, and the linefeed appears!

I can't have access to a more recent vi (I do this on AIX, vi (not Vim) version 3.10 I think? (no "-version" or other means of knowing it)).

# strings /usr/bin/vi | grep -i 'version.*[0-9]'
@(#) Version 3.10

Is it normal for vi (and perhaps not in more recent version? Or Vim?) to silently add a newline at the end of a file? (I thought the ~ indicated that the previous line did NOT end with a newline.)

—

Edit: some additional updates and a bit of a summary, with a big thanks to the answers below :

  • vi silently add a trailing newline at the moment it writes a file that lacked it (unless file is empty).

  • it only does so at the writing time! (ie, until you :w, you can use :e to verify that the file is still as you openened it… (ie: it still shows "filename" [Last line is not complete] N line, M character). When you save, a newline is silently added, without a specific warning (it does say how many bytes it saves, but this is in most cases not enough to know a newline was added) (thanks to @jiliagre for talking to me about the opening vi message, it helped me to find a way to know when the change really occurs)

  • This (silent correction) is POSIX behavior! (see @barefoot-io answer for references)

Best Answer

This is the expected vi behavior.

Your file has an incomplete last line so strictly speaking (i.e. according to the POSIX standard), it is not a text file but a binary file.

vi which is a text file editor, not a binary one, gracefully fixes it when you save it.

This allows other text file tools like wc, sed and the likes to provide the expected output. Note that vi isn't silent about the issue:


$ printf "one\ntwo" >file     # Create a unterminated file
$ cat file                    # Note the missing newline before the prompt
one
two$ wc -l file               # wc ignores the incomplete last line
       1 file
$ sed '' file > file1
$ cat file1                   # so does a legacy sed
one
$ PATH=$(getconf PATH) sed  '' file
one                           # while a POSIX conformant sed warns you:
sed: Missing newline at end of file file.
two
$ vi file
one
two
~
~
~                             # vi tells you too about the issue
"file" [Incomplete last line] 2 lines, 7 characters

:w

"file" 2 lines, 8 characters  # and tells it writes two lines
                              # You'll even notice it writes one more
                              # character if you are a very shrewd observer :-)
:q
$ cat file                    # the file is now valid text
one
two
$ wc -l file                  # wc reports the expected number of lines
       2 file
$ sed '' file > file1         # sed works as expected
$ cat file1
one
two

Note, to get some clues about what vi version you are running, you can use the :ve command. It shows here I'm using a legacy SVR4 one here, definitely not vim:

:ve
Version SVR4.0, Solaris 2.5.0

Apparently, yours is stating:

:ve
Version 3.10

That likely means AIX vi is based on SVR3 source code.

In any case, this behavior, and the [Incomplete last line] warning message have been in the legacy Bill Joy's vi source code since at least 1979 and AFAIK, retained in all branches created from System V source code releases, from which proprietary Unix like AIX were built.

Chronologically speaking, this behavior is then not a consequence of POSIX conformance but more a consequence of Bill Joy's original decision to be helpful with users editing bogus text files, and then, a decade later, the POSIX committee decision to keep this tolerance.

If you use ed instead of vi, you'll notice that the former is more verbose about the issue, at least if your ed is from SVR3 or newer source branch:

$ ed file
'\n' appended
8
q

Note also that an empty file is a valid text file that happens to contain zero lines. As there is then no unterminated line to fix, vi doesn't append a newline when saving the file.

Related Question