Shell – How to use namedpipe as temporary file

pipeshell-script

One vim plugin that I use uses this script to pass some input to linters that don't support reading from stdin.

set -eu

# All of the following arguments are read as command to run.
file_extension="$1"
shift

temp_dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'ale_linter')
temp_file="$temp_dir/file$file_extension"
trap 'rm -r "$temp_dir"' EXIT

while read -r; do
  echo "$REPLY" >> "$temp_file"
done

"$@" "$temp_file"

At first I was a bit confused about why they haven't just used something like that

some input | some_program /dev/stdin

But after trying ghc as linter I've found it that it's complaining about /dev/stdin saying something about that it isn't real file (which it isn't)

So I wonder can I use namedpipe instead of temporary file.
The reason I'm not quite satisfied with writing to temporary files is SSD health and if there is a better way to do it why not do it, right?

Best Answer

No, the programs that reject those files usually reject them on the ground that the file is not seekable (they need to access the content at arbitrary offsets, or several times after rewinding etc.). Or they would want to open the file several times. They may also want to rewrite (part of) the file or truncate it.

Unnamed pipe (like with | and /dev/stdin) or named ones make no difference in any of those cases.

Actually, on Linux, /dev/stdin when stdin is pipe (named or not) behaves exactly like a named pipe, the program would not be able to differentiate that /dev/stdin from a real named pipe.

On other systems, it's not exactly the same, but in effect, opening /dev/stdin or a named pipe will get you a file descriptor to a pipe, something that is not seekable either way.

So, you'll need to create the temporary file. Note that some shells make it easier. With zsh, it's just:

#! /bin/zsh -
"$@" =(cat)

On Linux and With shells that use a deleted temporary files for here documents (like bash, zsh and some implementations of ksh), you can do:

#! /bin/bash -
"$@" /dev/fd/3 3<< EOF
$(cat)
EOF

However, that may mangle the contents of the file if it contains NUL characters or ends in empty lines.

Note that since version 5, bash makes the here doc temporary file read-only, so if the application needs to make modifications to that file, you'll to restore the write permissions with:

#! /bin/bash -
{
  chmod u+w /dev/fd/3 && # only needed in bash 5+
    "$@" /dev/fd/3
} 3<< EOF
$(cat)
EOF

A note about that while read loop since you asked.

First read -r without a variable name is not valid sh syntax. The sh syntax is specified by POSIX (ISO 9945, also IEEE Std 1003.1) like the C syntax is specified by ISO 9899.

In that specification, you'll notice that read requires a variable name argument. The behaviour when you omit it is unspecified and in practice vary with the sh interpreter implementation.

bash is the GNU sh interpreter, like gcc is the GNU C compiler. Both bash and gcc have extensions over what those standards specify.

In the case of read, bash treats read -r as if it was IFS= read -r REPLY. In the POSIX spec, IFS= read -r REPLY reads stdin until either a \n character or the end of input is reached and stores the read characters into the $REPLY variable and returns with a success exit status if a newline character was read (a full line) or failure otherwise (like EOF before the newline) and leaves the behaviour undefined if the read data contains NUL characters or sequences of bytes that don't form valid characters.

In the case of bash, it will store the bytes read even if they don't form valid characters and removes the NUL characters.

read -r is like read -r REPLY in ksh or zsh and reports an error in yash or ash-based POSIX-like shells.

The behaviour of echo is unspecified unless its arguments don't contain backslash characters and the first one is not -n.

So, to sum up, unless you know the particular sh implementation (and version) you're dealing with, you can't tell what

while read -r; do
  echo "$REPLY" >> "$temp_file"
done

will do. In the case of bash specifically, it will store stdin into the temp_file only as long as the data doesn't contain NUL characters, ends in a newline character and none of the lines matches the ^-[neE]+$ extended regular expression (and/or depending on the environment or how bash was compiled like the sh of OS/X, doesn't contain backslash characters).

It's also very inefficient and not the way you process text in shells.

Here, you want:

cat > "$temp_file"

cat is a standard command, which when not given any argument just dumps its stdin onto its stdout as-is.

Related Question