When I run this command:
echo "1" > /dev/tty # runs successfully
but when I run this:
echo "1" | /dev/tty #permission denied
What is the difference between these two operators >
and |
and why does the pipe cause an error?
io-redirectionpipeshell
When I run this command:
echo "1" > /dev/tty # runs successfully
but when I run this:
echo "1" | /dev/tty #permission denied
What is the difference between these two operators >
and |
and why does the pipe cause an error?
Looking at the two commands separately:
utility 2>&1 >output.log
Here, since redirections are processed in a left-to-right manner, the standard error stream would first be redirected to wherever the standard output stream goes (possibly to the console), and then the standard output stream would be redirected to a file. The standard error stream would not be redirected to that file.
The visible effect of this would be that you get what's produced on standard error on the screen and what's produced on standard output in the file.
utility 2>&1 | tee output.log
Here, you redirect standard error to the same place as the standard output stream. This means that both streams will be piped to the tee
utility as a single intermingled output stream, and that this standard output data will be saved to the given file by tee
. The data would additionally be reproduced by tee
in the console (this is what tee
does, it duplicates data streams).
Which ever one of these is used depends on what you'd like to achieve.
Note that you would not be able to reproduce the effect of the second pipeline with just >
(as in utility >output.log 2>&1
, which would save both standard output and error in the file by first redirecting standard output to the output.log
file and then redirecting standard error to where standard output is now going). You would need to use tee
to get the data in the console as well as in the output file.
Additional notes:
The visible effect of the first command,
utility 2>&1 >output.log
would be the same as
utility >output.log
I.e., the standard output goes to the file and standard error goes to the console.
If a further processing step was added to the end of each of the above commands, there would be a big difference though:
utility 2>&1 >output.log | more_stuff
utility >output.log | more_stuff
In the first pipeline, more_stuff
would get what's originally the standard error stream from utility
as its standard input data, while in the second pipeline, since it's only the resulting standard output stream that is ever sent across a pipe, the more_stuff
part of the pipeline would get nothing to read on its standard input.
In
./binary < file
binary
's stdin is the file open in read-only mode. Note that bash
doesn't read the file at all, it just opens it for reading on the file descriptor 0 (stdin) of the process it executes binary
in.
In:
./binary << EOF
test
EOF
Depending on the shell, binary
's stdin will be either a deleted temporary file (AT&T ksh, zsh, bash...) that contains test\n
as put there by the shell or the reading end of a pipe (dash
, yash
; and the shell writes test\n
in parallel at the other end of the pipe). In your case, if you're using bash
, it would be a temp file.
In:
cat file | ./binary
Depending on the shell, binary
's stdin will be either the reading end of a pipe, or one end of a socket pair where the writing direction has been shut down (ksh93) and cat
is writing the content of file
at the other end.
When stdin is a regular file (temporary or not), it is seekable. binary
may go to the beginning or end, rewind, etc. It can also mmap it, do some ioctl()s
like FIEMAP/FIBMAP (if using <>
instead of <
, it could truncate/punch holes in it, etc).
pipes and socket pairs on the other hand are an inter-process communication means, there's not much binary
can do beside read
ing the data (though there are also some operations like some pipe-specific ioctl()
s that it could do on them and not on regular files).
Most of the times, it's the missing ability to seek
that causes applications to fail/complain when working with pipes, but it could be any of the other system calls that are valid on regular files but not on different types of files (like mmap()
, ftruncate()
, fallocate()
). On Linux, there's also a big difference in behaviour when you open /dev/stdin
while the fd 0 is on a pipe or on a regular file.
There are many commands out there that can only deal with seekable files, but when that's the case, that's generally not for the files open on their stdin.
$ unzip -l file.zip
Archive: file.zip
Length Date Time Name
--------- ---------- ----- ----
11 2016-12-21 14:43 file
--------- -------
11 1 file
$ unzip -l <(cat file.zip)
# more or less the same as cat file.zip | unzip -l /dev/stdin
Archive: /proc/self/fd/11
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of /proc/self/fd/11 or
/proc/self/fd/11.zip, and cannot find /proc/self/fd/11.ZIP, period.
unzip
needs to read the index stored at the end of the file, and then seek within the file to read the archive members. But here, the file (regular in the first case, pipe in the second) is given as a path argument to unzip
, and unzip
opens it itself (typically on fd other than 0) instead of inheriting a fd already opened by the caller. It doesn't read zip files from its stdin. stdin is mostly used for user interaction.
If you run that binary
of yours without redirection at the prompt of an interactive shell running in a terminal emulator, then binary
's stdin will be inherited from its caller the shell, which itself will have inherited it from its caller the terminal emulator and will be a pty device open in read+write mode (something like /dev/pts/n
).
Those devices are not seekable either. So, if binary
works OK when taking input from the terminal, possibly the issue is not about seeking.
If that 14 is meant to be an errno (an error code set by failing system calls), then on most systems, that would be EFAULT
(Bad address). The read()
system call would fail with that error if asked to read into a memory address that is not writable. That would be independent of whether the fd to read the data from points to a pipe or regular file and would generally indicate a bug1.
binary
possibly determines the type of file open on its stdin (with fstat()
) and runs into a bug when it's neither a regular file nor a tty device.
Hard to tell without knowing more about the application. Running it under strace
(or truss
/tusc
equivalent on your system) could help us see what is the system call if any that is failing here.
1 The scenario envisaged by Matthew Ife in a comment to your question sounds a lot plausible here. Quoting him:
I suspect it is seeking to the end of file to get a buffer size for reading the data, badly handling the fact that seek doesn't work and attempting to allocate a negative size (not handling a bad malloc). Passing the buffer to read which faults given the buffer is not valid.
Best Answer
Short answer:
>
must be followed by a filename or&n
(n is a number), and|
must be followed by another command invocation.Details: In shell syntax, a call to some command contains several components. Example:
Here, parameters
2>/dev/null
,>file
and3>&4
are special parameters (containing an unescaped>
¹), they are used to establish io-redirections, and can appear anywhere in the command line. Filedesciptor 2 is redirected to/dev/null
, filedescriptor1
(implicit) is redirected tofile
and filedescriptor3
is redirected to what filedescriptor 4 was linked to.Then, among remaining parameters,
A=foo
andB=bar
contain=
, so they are not considered as the command name: they give specific values to environment variables of the process to be launched.Then comes the command
cmd
and the real arguments:arg1
,arg2
,arg3
.The pipe
|
is not part of a command invocation, it links two such invocations together. Example:Output on filedescriptor 1 by the first process will be received as input on filedescriptor 0 by the second process, through a “pipe” which acts like a buffer.
—
1. In fact, the special characters like
>
are sometimes seen followed by a space. Even though this is allowed, the two (space-separated) strings must be understood as a single ‘entity’.