Linux – Writing on a FIFO with multiple processes

fifolinux

I need to make a named pipe with mkfifo. For instance:

mkfifo mypipe

I'm going to have a systemd service (a simple shell script) that read from the pipe and I will write on the same pipe calling another script (e.g. write-to-pipe.sh).

What if I have multiple instances of write-to-pipe.sh writing simultaneously on the pipe?
I mean, many users can execute write-to-pipe.sh. If two users try to write on the pipe at the same time, is it going to create trouble (like when two users try to write on a regular file (not pipe) and one overwrite the other) or not?

I hope this is not a duplicate, but I couldn't find an answer to this question…

Thanks

Best Answer

Here's a quick test I ran:

mkfifo foo
yes "Process 1 reporting" > foo & yes "This is process 2" > foo &
awk '!a[$0]++' < foo

And the output I got from awk:

Process 1 reporting
Process 1 reportThis is process 2
This is process 2
This is pring
Process 1 reportocess 2
ing
ocess 2
This is prProcess 1 reporting

Related Solutions

Why is a named pipe as slow as writing to a file

You're not seeing any performance benefit because you're not actually hitting disk when using a file - the data's on its way to the disk, but your execution thread doesn't need to wait for it to land there, so you're not actually seeing the speed penalty of hitting the disk.

If you want to wait for the disk operation to complete to see how much slower that gets, call a sync() (how to varies on your python version, see here) - you'll be looking at tens of thousands of microseconds just for your disk to seek a couple times to get the file written out (assuming it doesn't have some kind of fast write cache like in a RAID controller).

Opening named pipe blocks forever, if pipe is deleted without being connected

As suggested by Julie Pelletier, I'm making this answer about the workaround we found in our discussion.

You cannot easily identify deadlocked situations as described in my question, but you can pre-emptively vent the named pipe as a workaround before anyone deletes it (if you really cannot avoid such a deletion, as in my case). This venting should allow any writer currently blocked trying to open the pipe, to succeed with the open operation but fail during the actual write operation (-> broken pipe). Failing might be better than a deadlock. In order to not immediately run into the next deadlock, you should move the pipe before venting it, and delete it afterwards.

# rename the pipe, i.e. move it out of the way
mv -f /tmp/test.pipe /tmp/test.pipe~ 2>/dev/null

# vent the pipe, i.e. shortly open it for reading but don't read from it.
# call the subshell dd and empty echo calls in the background to avoid 
# deadlocking on redundant venting.
(dd if=/tmp/test.pipe~ count=0 2>/dev/null & echo -n "" >/tmp/test.pipe~ &)

# delete the old pipe
rm -f /tmp/test.pipe~

If you know which program executes the faulty deletion, you might want to wrap that program into a small script, that does the venting and forgoes the deletion.

Best Answer

Related Solutions

Why is a named pipe as slow as writing to a file

Opening named pipe blocks forever, if pipe is deleted without being connected

Related Question