I have a binary (that I can't modify) and I can do:
./binary < file
I also can do:
./binary << EOF
> "line 1 of file"
> "line 2 of file"
...
> "last line of file"
> EOF
But
cat file | ./binary
gives me an error. I don't know why it doesn't work with a pipe.
In all 3 cases the content of file is given to the standard input
of binary (in different ways):
- bash reads the file and gives it to stdin of binary
- bash reads lines from stdin (until EOF) and gives it to stdin of binary
- cat reads and puts the lines of file to stdout, bash redirects them to stdin of binary
The binary shouldn't notice the difference between those 3 as far
as I understood it. Can someone explain why the 3rd case
doesn't work?
BTW: The error given by the binary is:
20170116/125624.689 – U3000011 Could not read script file '', error
code '14'.
But my main question is, how is there a difference for any program with that 3 options.
Here are some further details: I tried it again with strace
and there were in fact some errors ESPIPE (Illegal seek) from lseek
followed by EFAULT (Bad address) from read right before the
error message.
The binary I tried to control with a ruby script (without using
temporary files) is part of the callapi from Automic (UC4).
Best Answer
In
binary
's stdin is the file open in read-only mode. Note thatbash
doesn't read the file at all, it just opens it for reading on the file descriptor 0 (stdin) of the process it executesbinary
in.In:
Depending on the shell,
binary
's stdin will be either a deleted temporary file (AT&T ksh, zsh, bash...) that containstest\n
as put there by the shell or the reading end of a pipe (dash
,yash
; and the shell writestest\n
in parallel at the other end of the pipe). In your case, if you're usingbash
, it would be a temp file.In:
Depending on the shell,
binary
's stdin will be either the reading end of a pipe, or one end of a socket pair where the writing direction has been shut down (ksh93) andcat
is writing the content offile
at the other end.When stdin is a regular file (temporary or not), it is seekable.
binary
may go to the beginning or end, rewind, etc. It can also mmap it, do someioctl()s
like FIEMAP/FIBMAP (if using<>
instead of<
, it could truncate/punch holes in it, etc).pipes and socket pairs on the other hand are an inter-process communication means, there's not much
binary
can do besideread
ing the data (though there are also some operations like some pipe-specificioctl()
s that it could do on them and not on regular files).Most of the times, it's the missing ability to
seek
that causes applications to fail/complain when working with pipes, but it could be any of the other system calls that are valid on regular files but not on different types of files (likemmap()
,ftruncate()
,fallocate()
). On Linux, there's also a big difference in behaviour when you open/dev/stdin
while the fd 0 is on a pipe or on a regular file.There are many commands out there that can only deal with seekable files, but when that's the case, that's generally not for the files open on their stdin.
unzip
needs to read the index stored at the end of the file, and then seek within the file to read the archive members. But here, the file (regular in the first case, pipe in the second) is given as a path argument tounzip
, andunzip
opens it itself (typically on fd other than 0) instead of inheriting a fd already opened by the caller. It doesn't read zip files from its stdin. stdin is mostly used for user interaction.If you run that
binary
of yours without redirection at the prompt of an interactive shell running in a terminal emulator, thenbinary
's stdin will be inherited from its caller the shell, which itself will have inherited it from its caller the terminal emulator and will be a pty device open in read+write mode (something like/dev/pts/n
).Those devices are not seekable either. So, if
binary
works OK when taking input from the terminal, possibly the issue is not about seeking.If that 14 is meant to be an errno (an error code set by failing system calls), then on most systems, that would be
EFAULT
(Bad address). Theread()
system call would fail with that error if asked to read into a memory address that is not writable. That would be independent of whether the fd to read the data from points to a pipe or regular file and would generally indicate a bug1.binary
possibly determines the type of file open on its stdin (withfstat()
) and runs into a bug when it's neither a regular file nor a tty device.Hard to tell without knowing more about the application. Running it under
strace
(ortruss
/tusc
equivalent on your system) could help us see what is the system call if any that is failing here.1 The scenario envisaged by Matthew Ife in a comment to your question sounds a lot plausible here. Quoting him: