How Input Redirection Works in Ubuntu 14.04 – Command Line Guide

14.04command lineredirectscripts

On my understanding, any command that reads from standard input (i.e keyboard), that it gets its input from a file.

$echo < text_content.txt

$

But echo command does not read and display the text_content.txt on the terminal. what's wrong here?

Best Answer

stdin and Commands

On my understanding, any command that reads from standard input (i.e keyboard), that it gets its input from a file.

Effectively, yes. As I've discussed in my answer on What characterizes a file in Linux/Unix?, a file is any object on which you can execute standard operations such as read(), open(), write(), close(). stdin being represented via file descriptor 0 is effectively a file in that regard, and any command/process in Linux gets 3 standard file descriptors - stdin,stdout,stderr - when that process is started. What are the actual files behind those file descriptors ? The command doesn't care and shouldn't, as long as it can do operations on it.

But echo command does not read and display the text_content.txt on the terminal. what's wrong here?

Now, command is free to do what it will with those file descriptors1. In case of echo it only deals with stdout and doesn't perform any action on stdin at all. So there's nothing wrong with the command itself.

< redirection will open() the file text_content.txt for reading, and it still will assign file descriptor ( for example 3 ) returned from open() call to file descriptor 0, and if a command is concerned with stdin - it will read from file descriptor 0 as if nothing happened.In fact, you will see that in action, if you run strace -f -e dup2,write,openat bash -c 'echo < text_content.txt

openat(AT_FDCWD, "/etc/passwd", O_RDONLY) = 3
dup2(3, 0)                              = 0
write(1, "\n", 1
)                       = 1
dup2(10, 0)                             = 0
+++ exited with 0 +++

Notice the dup2() system call. That's how file descriptor 3 (the file) is assigned/redirected. In sort of cp original copy syntax, dup2(3,0) makes a copy of file descriptor 3 to file descriptor 0, and they point to same file.

Notice also that write() output a newline to file descriptor 1. That's the default behavior. If we do strace -f -e dup2,write,openat bash -c 'echo FOO < /etc/passwd' here's what we will see

dup2(3, 0)                              = 0
write(1, "FOO\n", 4FOO
)                    = 4
dup2(10, 0)                             = 0
+++ exited with 0 +++

So again, nothing wrong here - redirections are performed properly and echo does its job of writing stuff to stdout which is file descriptor 1.


How to actually read a file

Now, lets address something else. How can we read a file in shell ? Well, for that there exists cat command, which accepts arguments, so you can do just cat file.txt. Can you do cat < file.txt ? Sure. But that means shell will have to do that dup2() call, whereas cat file.txt doesn't - so there's less unnecessary syscalls is what I'm saying.

In complex cases, such as when you need to perform an action on each line of the file, you would do

while IFS= read -r line || [ -n "$line" ]; do
    # command to process line variable here
done < /etc/passwd

Now, for the whole loop file descriptor 0 will be a copy of whatever file descriptor is returned from opening /etc/passwd. Of course, if you can use cat or another specific command to the job of reading a file - do that. Shell is slow method and has a lot of pitfalls. See also, Why is using a shell loop to process text considered bad practice?


1. Some applications still might care about what they can do with stdin or detect if stdin is a file or pipeline. When stdin file descriptor is assigned as read-end of pipeline (which is also a file descriptor) the output is not seekable (meaning that application written in C or another language cannot use seek() syscall to quickly navigate to specific byte offset in a file). A good example of that is given in the question titled What is the difference between “cat file | ./binary” and “./binary < file”?

Sidenote: on Linux it's not exactly keyboard from where stdin gets its input. If you do

$ ls -l /proc/self/fd/0
lrwx------ 1 serg serg 64 Feb 23 16:45 /proc/self/fd/0 -> /dev/pts/0

you will see in the output that it is /dev/pts/0 terminal device to which stdin points initially. The terminal device then interfaces with keyboard, or it could as well be serial cable.

Additionally, if file is not very large, you could take advantage of bash's mapfile builtin to read lines into array:

mapfile  -t  < /etc/passwd
for i in "${MAPFILE[@]}"; do echo "$i"; done

Related Question