What is Meant by Connecting STDOUT and STDIN?

io-redirectionpipeshellstdinstdout

I'm reading a book, it says:

Every process has at least three communication channels available to it: “standard
input” (STDIN), “standard output” (STDOUT), and “standard error” (STDERR).

Most commands accept their input from STDIN and write their output to STDOUT.
They write error messages to STDERR. This convention lets you string
commands together like building blocks to create composite pipelines.

The shell interprets the symbols <, >, and >> as instructions to reroute a command’s input or output to or from a file.

To connect the STDOUT of one command to the STDIN of another, use the |
symbol, commonly known as a pipe.
ps -ef | grep httpd

So basically what this is saying is that standard input is a command that allows user to write to a file, while standard output is a command that has the bash shell write output to the shell, and standard error is just like output but it is only invoked when there is an error in file system. Then we get to the part of connecting STDOUT and STDIN and I'm lost.

Best Answer

Standard input and standard output are not commands.

Imagine commands as machines in a factory with an assembly line. Most machines are designed to have one conveyor belt to feed data in and one conveyor belt to feed data out; they are the standard input and the standard output respectively. The standard error is an opening on the side of the machine where it can eject rejects.

+-------+     +------------------+       +------------------+     +------+
| input |     |    machine A     |       |    machine B     |     |output|
| reser |=====|<stdin     stdout>|=======|<stdin     stdout>|=====|bucket|
| ‑voir |  →  |      stderr      |   →   |      stderr      |  →  |      |
+-------+     +------------------+       +------------------+     +------+
                      ||                          ||

The diagram above shows a conveyor belt that goes through two machines. The data comes from the input reservoir on the left, is fed to machine A, then the output is conveyed further to machine B (for which it is input), and machine B's output is deposited in the output bucket on the right.

In unix terms, this is called a pipeline. The metaphor is that of plumbing: a pipe connects machine A to machine B. The shell syntax for the pipeline above is

<input-file.txt commandA | commandB >output-file.txt

The < redirection symbol tells the shell to connect commandA's standard input to the file input-file.txt before launching commandA. (You can put the redirection before or after the command name.) The > redirection symbol tells the shell to connect commandB's standard output to output-file.txt. The pipe ("|") symbol in the middle tells the shell to connect commandA's standard output to commandB's standard input before launching them.

Commands can have more than one input and more than one output, but that's material for another day.

Related Solutions

Bash – Use a Variable to Store stderr|stdout Redirection

Another solution could be the following:

#!/bin/bash

verbose=0

exec 3>&1
exec 4>&2

if ((verbose)); then
  echo "verbose=1"
else
  echo "verbose=0"
  exec 1>/dev/null
  exec 2>/dev/null
fi

echo "this should be seen if verbose"
echo "this should always be seen" 1>&3 2>&4

Then add 1>&3 2>&4 only to commands of which you want to see the output.

Bash Scripting – Use Filename Arguments or Default to stdin/stdout

I'd make heavier use of I/O redirection:

#!/bin/bash
[[ $1 ]] && [[ ! -f $1 ]] && echo "file $1 dne" && exit 1
[[ $1 ]] && exec 3<$1 || exec 3<&0
[[ $2 ]] && exec 4>$2 || exec 4>&1
fgrep -v "stuff" <&3 >&4

Explanation

[[ $1 ]] && [[ ! -f $1 ]] && echo "file $1 dne" && exit 1

Test if an input file has been specified as a command line argument and if the file exists.
[[ $1 ]] && exec 3<$1 || exec 3<&0

If $1 is set, i.e. an input file has been specified, the specified file is opened at file descriptor 3, otherwise stdin is duplicated at file descriptor 3.
[[ $2 ]] && exec 4>$2 || exec 4>&1

Similarly if the $2 is set, i.e. an output file has been specified, the specified file is opened at file descriptor 4, otherwise stdout is duplicated at file descriptor 4.
fgrep -v "stuff" <&3 >&4

Lastly fgrep is invoked, redirecting its stdin and stdout to the previously set file descriptors 3 and 4 respectively.

Reopening standard input and output

If you'd prefer not to open intermediate file descriptors, an alternative is to replace the file descriptors corresponding to stdin and stdout directly with the specified input and output files:

#!/bin/bash
[[ $1 ]] && [[ ! -f $1 ]] && echo "file $1 dne" && exit 1
[[ $1 ]] && exec 0<$1
[[ $2 ]] && exec 1>$2
fgrep -v "stuff"

A drawback with this approach is that you loose the ability to differentiate output from the script itself from the output of the command which is the target for the redirection. In the original approach, you can direct script output to the unmodified stdin and stdout, which in turn might have been redirected by the caller of the script. The specified input and output files could still be accessed via the corresponding file descriptors, which are distinct from the script stdin and stdout.

Best Answer

Related Solutions

Bash – Use a Variable to Store stderr|stdout Redirection

Bash Scripting – Use Filename Arguments or Default to stdin/stdout

Related Question