Shell – broken pipe error with popen and JS ffi

headpipeshelltr

I am using a ffi for nodejs, which for the most part has nothing to do with this question, which is really about understanding pipes better, but does offer some context.

function exec(cmd) {
  var buffer = new Buffer(32);
  var result = '';
  var fp = libc.popen('( ' + cmd + ') 2>&1', 'r');
  var code;

  if (!fp) throw new Error('execSync error: '+cmd);

  while( !libc.feof(fp) ){
    libc.fgets(buffer, 32, fp)
    result += buffer.readCString();
  }
  code = libc.pclose(fp) >> 8;

  return {
    stdout: result,
    code: code
  };
}

which brings me to this bit of code that, when I run using this exec function 

 tr -dc "[:alpha:]" < /dev/urandom | head -c ${1-8}

I get the error:

write error: Broken pipe
tr: write error

but I do get the output I expect: 8 random numbers. This confused the hell out of me but then in some wild googling I found this stack answer which perfectly fit my situation.

I am left with some questions, though.

Why does:

tr -dc "[:alpha:]" < /dev/urandom | head -c ${1-8}

throw a broken pipe error when called with my exec command but not when called from the shell? I don`t understand why when I call:

tr -dc "[:alpha:]" < /dev/urandom

it reads endlessly, but when I pipe it to:

head -c ${1-8}

It works without throwing a broken pipe error. It seems that head would take what it needs and tr would just read forever. At least it should throw broken pipe; head would consume the first 8 bytes and then tr would still be putting out output and broken pipe would be thrown by tr because head has stopped running.

Both situations make sense to me, but it seems that they are some what exclusive to each other. I don't understand what is different between calling: 

exec(tr -dc "[:alpha:]" < /dev/urandom | head -c ${1-8})

and 

tr -dc "[:alpha:]" < /dev/urandom | head -c ${1-8}

directly from the command line, and specifically why < an endless file into something and then | it to something makes it not run endlessly. I've been doing this for years and never questioned why it works this way.

Lastly, is it OK to ignore this broken pipe error? Is there a way to fix it? Am I doing something wrong in my C++ ish javascript code? Am I missing some kind of popen basics?

—— EDIT

messing around some more the code

exec('head -10 /dev/urandom | tr -dc "[:alpha:]" | head -c 8')

throws no pipe error!

Best Answer

Normally, tr shouldn't be able to write that error message because it should have been killed by a SIGPIPE signal when trying to write something after the other end of the pipe has been closed upon termination of head.

You get that error message because somehow, the process running tr has been configured to ignore SIGPIPEs. I suspect that might be done by the popen() implementation in your language there.

You can reproduce it by doing:

sh -c 'trap "" PIPE; tr -dc "[:alpha:]" < /dev/urandom | head -c 8'

You can confirm that's what is happening by doing:

strace -fe signal sh your-program

(or the equivalent on your system if not using Linux). You'll then see something like:

rt_sigaction(SIGPIPE, {SIG_IGN, ~[RTMIN RT_1], SA_RESTORER, 0x37cfc324f0}, NULL, 8) = 0

or

signal(SIGPIPE, SIG_IGN)

done in one process before that same process or one of its descendants executes the /bin/sh that interprets that command line and starts tr and head.

If you do a strace -fe write, you'll see something like:

write(1, "AJiYTlFFjjVIzkhCAhccuZddwcydwIIw"..., 4096) = -1 EPIPE (Broken pipe)

The write system call fails with an EPIPE error instead of triggering a SIGPIPE.

In any case tr will exit. When ignoring SIGPIPE, because of that error (but that also triggers an error message). When not, it exits upon receiving the SIGPIPE. You do want it to exit, since you don't want it carrying on reading /dev/urandom after those 8 bytes have been read by head.

To avoid that error message, you can restore the default handler for SIGPIPE with:

trap - PIPE

Prior to calling tr:

popen("trap - PIPE; { tr ... | head -c 8; } 2>&1", ...)
Related Question