Bash – the difference between &6 and /dev/fd/6

bashfile-descriptorslinuxsocket

To read from file descriptor 6 I can use <&6 or </dev/fd/6 (aka /proc/self/fd/6). Usually both work equally well. However if that file descriptor happens to be a socket, strange things happen. For example:

$ bash -c 'ls -l /dev/fd/6;cat /dev/fd/6' 6</dev/tcp/localhost/12345
lrwx------ 1 michas michas 64 Jan 10 19:50 /dev/fd/6 -> socket:[315010]
cat: /dev/fd/6: No such device or address

Here ls shows the descriptor is indeed present. But accessing the data is not possible this way. If I use cat <&6 instead everything works well again.

What is the difference between both ways of accessing the file descriptor?

Is there a good way to access a descriptor if the number if given in a variable? (</dev/fd/$fd would work, but <&$fd does not.)

(The above situation can be observed on linux, but not on OpenBSD. – Seems like that file descriptor is a regular character device there.)

Best Answer

It's so because reading from /dev/fd/ entries which represents sockets isn't implemented on Linux. You can find quite a good writeup on reasoning here. So you can call stat on the link, and that's why you see it with ls, but access is deliberately disallowed.

Now for the second part - why does bash -c 'ls -l /dev/fd/6; cat <&6' 6</dev/tcp/localhost/12345 work? That's because socket is read from using socket/file API, not /proc filesystem. This is what I've observed happening:

bash instance running in your terminal creates socket with fd 6.
Child bash runs and calls dup2(6, 0), in order to attach your socket as cat's stdin.
If dup2 call didn't fail, cat reads from stdin.

You can reproduce and observe it with:

netcat -lp 12345    # in another terminal session (GNU netcat)
strace -f -e trace=open,read,write,dup2 bash -c 'ls -l /dev/fd/6; cat <&6' \
 6</dev/tcp/localhost/12345

If you're wondering why does the bash child process have access to fd 6 - file descriptors survive fork, and if they aren't marked for closing on exec, they don't get closed there as well.

Related Solutions

Bash – the difference between “eval” and “source /dev/stdin”

What is the difference between the ways?

from bash manpage:

eval [arg ...]
              The  args  are read and concatenated together into a single com‐
              mand.  This command is then read and executed by the shell,  and
              its  exit status is returned as the value of eval.  If there are
              no args, or only null arguments, eval returns 0.

source filename [arguments]
              Read and execute commands from filename  in  the  current  shell
              environment  and return the exit status of the last command exe‐
              cuted from filename.  If filename does not contain a slash, file
              names  in  PATH  are used to find the directory containing file‐
              name.  The file searched for in PATH  need  not  be  executable.
              When  bash  is  not  in  posix  mode,  the  current directory is
              searched if no file is found in PATH.  If the sourcepath  option
              to  the  shopt  builtin  command  is turned off, the PATH is not
              searched.  If any arguments are supplied, they become the  posi‐
              tional  parameters  when  filename  is  executed.  Otherwise the
              positional parameters are unchanged.  The return status  is  the
              status  of  the  last  command exited within the script (0 if no
              commands are executed), and false if filename is  not  found  or
              cannot be read.

There are no differences between the two ways.

There is only one note: eval concatenated all of its arguments, which is then run as a single command. source reads the contents of a file and executes them. eval can only build commands from its arguments, not stdin. So you can not do like this:

printf "ls" | eval

Which is more preferred?

Your example provides the same result, but the purpose of eval and source is different. source is usually used for providing a library for other scripts, while eval is used only to evaluate commands. You should avoid using eval if possible, because there is no guarantee that the evaled string is clean; we must do some sanity checks, using subshell instead.

If we run some commands in () or {}, which is more preferred?

When you run sequences commands inside curly brace { }, all commands are run in the current shell, instead of a subshell (which is the case if you run inside parentheses (see bash reference)).

Using subshell ( ) uses more resources, but your current environment is not affected. Using { } runs all the commands in the current shell, so your environment is affected. Depending on your purpose, you can choose one of them.

Linux – a generic socket and how does it relate to a network device

Let's quickly review device files: In Linux, application programs communicate rad and write operations to the kernel through file descriptors. That works great for files, and it turned out that the same API could be used for character devices that produce and consume streams of characters, and block devices that read and write blocks of fixed size at a random access address, just by pretending that these are also files.

But a way was needed to configure those devices (set baud rates etc.), and for that, the ioctl call was invented. It just passes a data structure that's specific to the device and the kind of I/O control used to the kernel, and gets back the results in the same data structure, so it's a very generic extensible API and can be used for lots of things.

Now, how do network operations fit in? A typical network server application wants to bind to some network address, listen on a certain port (e.g. 80 for HTTP, or 22 for ssh), and if a client connects, it wants to send data to and receive data from this client. And the dual operations for the client.

It's not obvious how to fit this in with file operations (though it can be done, see Plan 9), that's why the UNIX designers invented a new API: sockets. You can find details in the section 2 man pages for socket, bind, listen, connect, send and recv. Note that while it is distinct from the file I/O API, the socket call nevertheless also returns a file descriptor. There are numerous tutorials on how to use sockets on the web, google a bit.

So far this is all pure UNIX, nobody was talking about network interfaces at the time sockets were invented. And because this API is really old, it is defined for a variety of network protocols beyond the Internet protocol (look at the AF_* constants), though only a few of those are supported in Linux.

But as computers started to get multiple network cards, some abstraction for this was needed. In Linux, that is the network interface (NI). It's not only used for a piece of hardware, but also for various tunnels, user application endpoints that server as tunnels like OpenVPN etc. As explained, the socket API isn't based on (special) files and independent of the filesystem. In the same way, network interfaces don't show up in the file system, either. However, the NIs are made available in the /proc and /sys filesystem (as well as other networking tunables).

A NI is simple a kernel abstraction of an endpoint where network packets enter and leave the kernel. Sockets, on the other hand, are used to communicate packets with applications. No socket needs to be involved with the processing of a packet. For example, when forwarding is enabled, a packet may enter on one NI and leave on another. In that sense, sockets and network interfaces are totally independent.

But there had to be a way to configure NIs, just like you needed a way to configure block and character devices. And since sockets already returned a file descriptor, it was somewhat logical to just allow an ioctl on that file descriptor. That's the netdevice interface you linked.

There are quite a few other abuses of system calls in a similar way, for example for packet filtering, packet capture etc.

All of this has grown piece after piece, and is not particularly logical in many places. If it had be designed all at once, one could probably have made a more orthogonal API.

Best Answer

Related Solutions

Bash – the difference between “eval” and “source /dev/stdin”

Linux – a generic socket and how does it relate to a network device

Related Question