Shell – Output of command not in stderr nor stdout

shellstderrstdout

I've stumbled on this issue, so I'm wondering how is this possible?

Standard run of command:

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14
info from server: "Processed 0 Failed 1 Total 1 Seconds spent 0.000017"
sent: 1; skipped: 0; total: 1

OK, lets try to get the first line only:

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 | head -1
sent: 1; skipped: 0; total: 1

What about standard head?

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 | head 
sent: 1; skipped: 0; total: 1

Inverse grep? sed? tee?!?!?!!?

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 | grep -v pero
sent: 1; skipped: 0; total: 1

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 | sed 's/foo/bar/'
sent: 1; skipped: 0; total: 1

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 | tee
sent: 1; skipped: 0; total: 1

stderr to stdout?

# zabbix_sender -c zabbix_agentd.conf -k mmysql.QCInserts -o 14 2>&1 | tee
sent: 1; skipped: 0; total: 1

I'm really puzzled…

Best Answer

This can happen if the application is writing directly to the TTY instead of STDOUT or STDERR.

You can play with this behavior by comparing the 2 examples below

( echo foo ) &>/dev/null
( echo foo > $(tty) ) &>/dev/null

Notice the first doesn't show anything, but the second does. That's because we sent the output directly to the tty and bypassed the redirection to /dev/null.

You can get around stuff like this by using script

script -c '( echo foo > $(tty) ) &>/dev/null'  >/dev/null

Basically the script utility creates a fake tty and launches the command in that tty. Any output from the command is sent to STDOUT which you can then redirect as normal.

Related Solutions

Output is in console but not part of stdout or stderr

The hideous command:

script -q -f -c "pocketsphinx_continuous -samprate 48000 -nfft 2048 -hmm /usr/local/share/pocketsphinx/model/en-us/en-us -lm 9745.lm -dict 9745.dic -inmic yes -logfn /dev/null" words.txt &

works for me, it logs:

READY....
Listening...
HELLO

to the file, and it is easy to weed out the unwanted stuff.

EDIT: If anyone is doing this themselves in the future, remove the & at the end of that command if you are executing it in a console, if you are executing it via Python or another language, keep the & at the end.

Shell – How to suppress stdout / stderr from a shell script output

When you see [1] + 7766 Suspended (signal) ..., this is a message printed by the shell, telling you that one of its child processes has been suspended.

In the fff example, there are two shell processes to consider. Your initial interactive shell, and the shell script which runs as a child process.

Your script arranges to suspend itself. The "Suspended" message is printed by the interactive shell. So this is not an option that you can toggle on or off inside the script.

Additionally, there is no option you could set in the interactive shell to toggle this specific message on or off. It is not possible to "suppress" this message individually... basically because this is never what you want to do :-).

I think the fff script does not successfully resume itself anyway. I think it might be possible to modify it to do so. However, resuming itself is a terrible idea. When you suspend the script, the interactive shell shows its command prompt again. I.e. if you didn't see the "suspended" message, it looks like your script has finished. But either way, if your script managed to resume itself, it would try to steal back control of the terminal from the user, no matter what they had started to do in the mean time. This is not good!

I think you need to understand e.g. that if you launched a process from your terminal and it is a child process of your interactive shell, then it is absolutely not a daemon process.

To start a daemon from your terminal, the program must fork() itself during startup. The original process should exit, and this allows the process which continues to be re-parented e.g. (in traditional unix) to PID 1 a.k.a. the init process. And there are more steps it must include as well, to detach from the terminal. See for example here: Daemonize a process in shell?

Additionally if your system uses systemd, you need to understand that legacy /etc/init.d/ scripts are required to be started using systemctl start. The packaged init.d scripts in Debian and elsewhere include a compatibility hack that does this for you. But you should not use that feature. It is a recipe for confusion. The script you have writen does not have this feature, so you must use systemctl start fff.

I strongly recommend not trying to use the games in the fff script in combination with systemctl start.

In reality systemctl uses a different approach to start background service processes. It sends a message to PID 1 (the systemd init process), and PID 1 starts the requested program. If you define a native systemd service, the program does not need to daemonize itself. If you use a legacy init.d script, the program still needs to daemonize itself, but the only thing it achieves is to tell systemd that the init.d script has finished starting. (This makes it possible to notice some startup failures in daemons, though unfortunately many existing programs have startup failures that can happen after the daemonization step).

You do not want to be worrying about whether and how systemd will react to the init.d script stopping and resuming itself.

Best Answer

Related Solutions

Output is in console but not part of stdout or stderr

Shell – How to suppress stdout / stderr from a shell script output

Related Question