When did Unix stop storing passwords in clear text

historypasswdpasswordshadow

When did Unix move away from storing clear text passwords in passwd? Also, when was the shadow file introduced?

Best Answer

For the early history of Unix password storage, read Robert Morris and Ken Thompson's Password Security: A Case History. They explain why and how early Unix systems acquired most the features that are still seen today as the important features of password storage (but done better).

  • The first Unix systems stored passwords in plaintext. Unix Third Edition introduced the crypt function which hashes the password. It's described as “encryption” rather than “hashing” because modern cryptographic terminology wasn't established yet and it used an encryption algorithm, albeit in an unconventional way. Rather than encrypt the password with a key, which would be trivial to undo when you have the key (which would have to be stored on the system), they use the password as the key.
  • When Unix switched from an earlier cipher to the then-modern DES, it was also made slower by iterating DES multiple times. I don't know exactly when that happened: V6? V7?
  • Merely hashing the password is vulnerable to multi-target attacks: hash all the most common passwords once and for all, and look in the password table for a match. Including a salt in the hashing mechanism, where each account has a unique salt, defeats this precomputation. Unix acquired a salt in Seventh Edition in 1979.
  • Unix also acquired password complexity rules such as a minimum length in the 1970s.

Originally the password hash was in the publicly-readable file /etc/passwd. Putting the hash in a separate file /etc/shadow that only the system (and the system administrator) could access was one of the many innovations to come from Sun, dating from around SunOS 4 in the mid-1980s. It spread out gradually to other Unix variants (partly via the third party shadow suite whose descendent is still used on Linux today) and wasn't available everywhere until the mid-1990s or so.

Over the years, there have been improvements to the hashing algorithm. The biggest jump was Poul-Henning Kamp's MD5-based algorithm in 1994, which replaced the DES-based algorithm by one with a better design. It removed the limitation to 8 password characters and 2 salt characters and had increased slowness. See IEEE's Developing with open source software, Jan–Feb. 2004, p. 7–8. The SHA-2-based algorithms that are the de facto standard today are based on the same principle, but with slightly better internal design and, most importantly, a configurable slowness factor.

Related Question