Why does openssl print to stderr for a successful command

certificatesopenssl

When I generate a CSR using openssl, the normal output goes to stderr. I don't understand why it isn't on stdout. I'm scripting the generation of some certificate/key pairs and want to be able to detect error conditions, but this confuses the issue.

admin@ip-10-248-185-66:~> openssl req -rand $RAND -new -newkey rsa:2048 -nodes -keyout $KEYPATH -subj /C=US/ST=CA/L=LA/O=TS/OU=server/CN=primary -out $CSRPATH -config $CONFPATH
Generating a 2048 bit RSA private key
.........................+++
..................................+++
writing new private key to '$KEYPATH'
-----
admin@ip-10-248-185-66:~> echo $?
0
admin@ip-10-248-185-66:~> openssl req -rand $RAND -new -newkey rsa:2048 -nodes -keyout $KEYPATH -subj /C=US/ST=CA/L=LA/O=TS/OU=server/CN=primary -out $CSRPATH -config $CONFPATH 2>/dev/null
admin@ip-10-248-185-66:~>

Best Answer

Messages to the users go on stderr. What goes to stdout is the result of the openssl command.

By default, unless you use -in or -out, openssl takes data (keys, certificates...) in from stdin and writes data out on stdout (the result like the request pem file).

In a shell you typically use it as:

openssl cmd < in.pem > out.pem

You don't want the messages to the user to end up in out.pem which is why they are issued on stderr.

Related Solutions

RSA 2048 keypair generation: via openssl 0.5s via gpg 30s, why the difference

GnuPG consumes several bytes from /dev/random for each random byte it actually uses. You can easily check that with this command:

start cmd:> strace -e trace=open,read gpg --armor --gen-random 2 16 2>&1 | tail
open("/etc/gcrypt/rngseed", O_RDONLY)   = -1 ENOENT (No such file or directory)
open("/dev/urandom", O_RDONLY)          = 3
read(3, "\\\224F\33p\314j\235\7\200F9\306V\3108", 16) = 16
open("/dev/random", O_RDONLY)           = 4
read(4, "/\311\342\377...265\213I"..., 300) = 128
read(4, "\325\3\2161+1...302@\202"..., 172) = 128
read(4, "\5[\372l\16?\...6iY\363z"..., 44) = 44
open("/home/hl/.gnupg/random_seed", O_WRONLY|O_CREAT, 0600) = 5
cCVg2XuvdjzYiV0RE1uzGQ==
+++ exited with 0 +++

In order to output 16 bytes of high-quality entropy GnuPG reads 300 bytes from /dev/random.

This is explained here: Random-Number Subsystem Architecture

Linux stores a maximum of 4096 bytes (see cat /proc/sys/kernel/random/poolsize) of entropy. If a process needs more than available (see cat /proc/sys/kernel/random/entropy_avail) then the CPU usage becomes more or less irrelevant as the feeding speed of the kernel's entropy pool becomes the relevant factor.

Why openssl s_client verifies a cert against a mismatching CAfile

It turns out that the openssl s_client on Ubuntu 10.04 still queries a default location for system installed certificates, even if -CApath and -CAfile are specified:

8466  open("/usr/lib/ssl/certs/4e18c148.0", O_RDONLY) = 4

(strace output)

Where:

$ ls -l /usr/lib/ssl/certs/4e18c148.0
lrwxrwxrwx 1 root root 30 2014-04-11 21:50 /usr/lib/ssl/certs/4e18c148.0 ->
    Deutsche_Telekom_Root_CA_2.pem

The directory /usr/lib/ssl/certs is a symlink to /etc/ssl/certs on Ubuntu 10.04, thus the open line from the strace log is not selected when grepping for '/etc/ssl' ...

Source

Looking at the openssl-0.9.8k, the source of this issue is located in crypto/x509/by_dir.c, dir_ctrl():

dir=(char *)Getenv(X509_get_default_cert_dir_env());
if (dir)
    ret=add_cert_dir(ld,dir,X509_FILETYPE_PEM);
else
    ret=add_cert_dir(ld,X509_get_default_cert_dir(),
                     X509_FILETYPE_PEM);

Where X509_get_default_cert_dir returns /usr/lib/ssl/certs and X509_get_default_cert_dir_env returns SSL_CERT_DIR.

Workaround

Thus, one can use following workaround under Ubuntu 10.04/openssl 0.9.8k to get the expected behavior:

$ SSL_CERT_DIR="" openssl s_client -crlf -verify 9 \
    -CAfile /etc/ssl/certs/TURKTRUST_Certificate_Services_Provider_Root_1.crt \
    -starttls smtp -host mx-ha03.web.de -port 25

And with the verification fails:

Verify return code: 19 (self signed certificate in certificate chain)

Current Situation

This is a Ubuntu issue. For example, with the Fedora 20's openssl 1.0.1e or Fedora 29's openssl 1.1.1, this workaround is not necessary, because the issue cannot be reproduced. That means when specifying an option like -CAfile or -CApath, no default certificate system directory is added to the directory search list on Fedora systems.

On Ubuntu 16 with openssl 1.0.2g the issue is still present.

It's also present on CentOS 7 with openssl-1.0.2k-16 - and unfortunately, the above workaround doesn't help there and the gnutls-3.3.29-8 fails due to unknown/unexpected TLS packet types.