SHA512 salted hash from mkpasswd doesn’t match an online version

I think this is not feasible.

As @Eir Nym says, you fairly obviously can't translate, for example, an MD5-hash of a password into a SHA-256 hash, without knowing the password.

However you would think that you'd be able to translate a SHA-256 hash (according to crypt(3)) into a SHA-256 hash (according to OpenLDAP): surely it's just a matter of identifying the salt and hash in one format, and reordering them into the format the other expects. No? What, it's not?!

In OpenLDAP, the format of the userPassword attribute, though not quite documented, is relatively straightforward: it's the fixed-length hash of the password followed by the salt (see eg contrib/slapd-modules/passwd/sha2 in the OpenLDAP sources). This is also evident in notes such as this OpenLDAP FAQ-o-matic article which shows how to generate a {SSHA} hash from password + salt.

For systems which use crypt(3), however, you'll see ‘password hashes’ of the form

$id$salt$hash

where $5$ indicates SHA-256, $6$ indicates SHA-512, and so on. The hash element of that is the base64 encoded form of some function of key, salt, and (in some variants) number of encryption rounds, but the actual function is... not documented.

• The Wikipedia page on crypt(3) mentions that ‘Over time various algorithms have been introduced.’
• It points to the ‘Password Hashing Competition’ specification (PHC), which narrows down the gross format of the ‘hash’ string, but not as much as you'd hope. It doesn't even conclusively identify what base64 variant the string uses (it appears to, but acknowledges that characters [.-] can appear in a base64 string without saying what they are).
• The passlib library acknowledges some of the mess (‘All of the above is guesswork based on examination of existing hashes and OS implementations’) and recommends the PHC format.
• A linuxquestions.org question indicates that the relevant base64 encoding is peculiar.

So we can look at the glibc sources for crypt. There, we discover (in crypt_util.c) that the base64 alphabet is indeed odd, using "./0-9A-Za-b" rather than the more usual (and PHC-specified) "A-Za-z0-9+/" (with occasional variants in the last two characters). Not helpful, but we could cope if this were the only difference.

However, the sha256-crypt.c function in that library (for example) takes the key and salt and does something very odd and long-winded with them to produce the hash value. I wouldn't presume to venture an opinion on whether what it's doing is sensible, but it's certainly not straightforward.

What that means is that although the crypt(3) $5$ and OpenLDAP ‘SHA-256’ hashes are on the face of it using the same cryptographic primitive, they are actually using it in sufficiently different ways that they amount to different hash functions so that, as with the point at the top, you can't translate one to the other without knowing the password. A similar point can be made about $6$/SHA-512 passwords.

This is very annoying.

Parenthetical note: As it happens, it is possible to convert the $1$/SMD5 ‘hashes’ between crypt(3) format and OpenLDAP:

$1$salt$hash

corresponds to

'{SMD5}' + hash + salt

where + is simple string concatenation. But we probably shouldn't be using MD5 hashes any more, so this doesn't much help.

Note also that you can port your crypt(3) hashes to OpenLDAP unchanged, using

{CRYPT}$1$salt$hash

if your libc is compatible (which it typically will be, if OpenLDAP is running on Linux), but OpenLDAP won't generate new passwords this way, so this is a one-way street.

You can (ab)use htpasswd from apache-utils package, provided you have version 2.4 or higher.

htpasswd -bnBC 10 "" password | tr -d ':\n'

-b takes the password from the second command argument
-n prints the hash to stdout instead of writing it to a file
-B instructs to use bcrypt
-C 10 sets the bcrypt cost to 10

The bare htpasswd command outputs in format <name>:<hash> followed by two newlines. Hence the empty string for name and tr stripping the colon and newlines.

The command outputs bcrypt with $2y$ prefix, which may be problem for some uses, but can easily be fixed by another sed since the OpenBSD variant using $2a$ is compatible with the fixed crypt_blowfish variant using $2y$.

htpasswd -bnBC 10 "" password | tr -d ':\n' | sed 's/$2y/$2a/'

Link to htpasswd man page: https://httpd.apache.org/docs/2.4/programs/htpasswd.html
Details about bcrypt variants: https://stackoverflow.com/a/36225192/6732096