Linux Bash – Understanding File Descriptor Duplication Behavior

bashfile-descriptorslinux

I have this file:

1
2
3
4

When I run paste in this manner

paste - - <file

this results

1       2
3       4

so far so good, standard input(redirected from file) is passed twice to paste therefore odd- and even-numbered lines print in pairs.

However, when this is changed to

paste - /dev/fd/0 <file

the output becomes

1       1
2       2
3       3
4       4

paste appears to be opening file twice (leading to two distinct entries in the kernel's file table?), once for the - argument and once again for /dev/fd/0.

I am at a loss to explain this – if I'm interpreting this correctly, both - and /dev/fd/0 ought to be referencing a single kernel file table entry therefore paste should be yielding results identical to the previous case.

Best Answer

That's specific to Linux. While on most Unices, opening /dev/fd/n is more or less the same as dup(n) (get a file descriptor to the same open file description as on fd n), on Linux, /dev/fd/n is a symlink to the file open on file descriptor n.

So, on Linux:

paste - /dev/fd/0 < file

is the same as:

paste - file < file

(or paste file file).

The two fds (0 for - and the own obtained by opening /dev/fd/0 or file), are independent and have their own cursor within the file.

You'll also notice that on Linux, you can't use /dev/fd/n with sockets.

Generally on Linux, you only want to use /dev/fd/n with pipes. But in this case

cat file | paste - /dev/fd/0

(or switching to a non-Linux-based OS) won't really help. paste - - works because paste knows it's stdin in both cases. But not here, so it will read a whole chunk (as opposed to a single line) from - (fd 0 to the pipe), and then another chunk from /dev/fd/0 (an independent fd to the same pipe on Linux, but whether those fds point to the same open file description or not doesn't matter for a pipe). Both read()s will read separate parts of the file, but several lines at a time.

You'd need to tell paste to read one byte at a time so it doesn't read more than one line from - before reading a line from /dev/fd/0, which you can probably not do without recompiling. You might be able to get paste to read its stdin one byte at a time with stdbuf, but probably not /dev/fd/0:

$ cat file | paste - /dev/fd/0
1
2
3
4
$ cat file | stdbuf -i1 paste - /dev/fd/0
1       2
        3
        4
Related Question