Pipe – Advantages of Using Named Pipe Over Unnamed Pipe

pipe

I was reviewing a set of interview questions that are asked from a unix admin; I found a topic called "named pipe".

I googled the topic; to some extent I have been able to understand it :- named pipes || FIFO

But still I feel that I am lacking the knowledge of when to use this particular type of pipe. Are there any special situations where unnamed pipes would fail to work ?

Best Answer

Named pipes (fifo) have ~~four~~ three advantages I can think of:

you don't have to start the reading/writing processes at the same time
you can have multiple readers/writers which do not need common ancestry
as a file you can control ownership and permissions
~~they are bi-directional, unnamed pipes may be unidirectional~~ *

^{*) Think of a standard shell | pipeline which is unidirectional, several shells (ksh, zsh, and bash) also offer coprocesses which allow bi-directional communication. POSIX treats pipes as half-duplex (i.e. each side can only read or write), the pipe() system call returns two file handles and you may be required to treat one as read-only and the other as write-only. Some (BSD) systems support read and write simultaneously (not forbidden by POSIX), on others you would need two pipes, one for each direction. Check your pipe(), popen() and possibly popen2() man pages. The undirectionality may not be dependent on whether the pipe is named or not, though on Linux 2.6 it is dependent.}

(Updated, thanks to feedback from Stephane Chazelas)

So one immediately obvious task you cannot achieve with an unnamed pipe is a conventional client/server application.

The last (stricken) point above about unidirectional pipes is relevant on Linux, POSIX (see popen()) says that a pipe need only be readable or writeable, on Linux they are unidirectional. See Understanding The Linux Kernel (3rd Ed. O'Reilly) for Linux-specific details (p787). Other OS's offer bidirectional (unnamed) pipes.

As an example, Nagios uses a fifo for its command file. Various external processes (CGI scripts, external checks, NRPE etc) write commands/updates to this fifo and these are processed by the persistent Nagios process.

Named pipes have features not unlike TCP connections, but there are important differences. Because a fifo has a persistent filesystem name you can write to it even when there is no reader, admittedly the writes will block (without async or non-blocking I/O), though you won't loose data if the receiver isn't started (or is being restarted).

For reference, see also Unix domain sockets, and the answer to this Stackoverflow question which summarises the main IPC methods, and this one which talks about popen()

Related Solutions

Linux – Guarantees for Concurrent Writes into a Named Pipe

It depends on how much each process is writing (assuming your OS is POSIX-compliant in this regard). From write():

Write requests to a pipe or FIFO shall be handled in the same way as a regular file with the following exceptions:
[...]

Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.

Also in the Rationale section regarding pipes and FIFOs:

Atomic/non-atomic: A write is atomic if the whole amount written in one operation is not interleaved with data from any other process. This is useful when there are multiple writers sending data to a single reader. Applications need to know how large a write request can be expected to be performed atomically. This maximum is called {PIPE_BUF}. This volume of POSIX.1-2008 does not say whether write requests for more than {PIPE_BUF} bytes are atomic, but requires that writes of {PIPE_BUF} or fewer bytes shall be atomic.

The value if PIPE_BUF is defined by each implementation, but the minimum is 512 bytes (see limits.h). On Linux, it's 4096 bytes (see pipe(7)).

Shell – broken pipe error with popen and JS ffi

Normally, tr shouldn't be able to write that error message because it should have been killed by a SIGPIPE signal when trying to write something after the other end of the pipe has been closed upon termination of head.

You get that error message because somehow, the process running tr has been configured to ignore SIGPIPEs. I suspect that might be done by the popen() implementation in your language there.

You can reproduce it by doing:

sh -c 'trap "" PIPE; tr -dc "[:alpha:]" < /dev/urandom | head -c 8'

You can confirm that's what is happening by doing:

strace -fe signal sh your-program

(or the equivalent on your system if not using Linux). You'll then see something like:

rt_sigaction(SIGPIPE, {SIG_IGN, ~[RTMIN RT_1], SA_RESTORER, 0x37cfc324f0}, NULL, 8) = 0

signal(SIGPIPE, SIG_IGN)

done in one process before that same process or one of its descendants executes the /bin/sh that interprets that command line and starts tr and head.

If you do a strace -fe write, you'll see something like:

write(1, "AJiYTlFFjjVIzkhCAhccuZddwcydwIIw"..., 4096) = -1 EPIPE (Broken pipe)

The write system call fails with an EPIPE error instead of triggering a SIGPIPE.

In any case tr will exit. When ignoring SIGPIPE, because of that error (but that also triggers an error message). When not, it exits upon receiving the SIGPIPE. You do want it to exit, since you don't want it carrying on reading /dev/urandom after those 8 bytes have been read by head.

To avoid that error message, you can restore the default handler for SIGPIPE with:

trap - PIPE

Prior to calling tr:

popen("trap - PIPE; { tr ... | head -c 8; } 2>&1", ...)

Best Answer

Related Solutions

Linux – Guarantees for Concurrent Writes into a Named Pipe

Shell – broken pipe error with popen and JS ffi

Related Question