I suspect your script and your shell are different. Perhaps you have #!/bin/sh
at the top of your script as the interpreter but you are using bash
as your personal shell. You can find out what shell you run in a terminal by running echo $SHELL
.
An easier way to do this which should work across most shells would be to use a pipe redirect instead of the file read operator you give. The symbol '-' is a standard nomenclature for reading STDIN and can frequently be used as a replacement for a file name in an argument list:
cat file2 | sort | diff file1 -
Or to avoid a useless use of cat:
sort < file2 | diff file1 -
Gilles' answer explains the race condition. I'm just going to answer this part:
Is there any way I can force this script to output always 0 lines (so the I/O redirection to tmp is always prepared first and so the data is always destroyed)? To be clear, I mean changing the system settings
IDK if a tool for this already exists, but I have an idea for how one could be implemented. (But note this wouldn't be always 0 lines, just a useful tester that catches simple races like this easily, and some more complicated races. See @Gilles' comment.) It wouldn't guarantee that a script was safe, but might be a useful tool in testing, similar to testing a multi-threaded program on different CPUs, including weakly-ordered non-x86 CPUs like ARM.
You'd run it as racechecker bash foo.sh
Use the same system-call tracing / intercepting facilities that strace -f
and ltrace -f
use to attach to every child process. (On Linux, this is the same ptrace
system call used by GDB and other debuggers to set breakpoints, single step, and modify memory / registers of another process.)
Instrument the open
and openat
system calls: when any process running under this tool makes a an open(2)
system call (or openat
) with O_RDONLY
, sleep for maybe 1/2 or 1 second. Let other open
system calls (especially ones including O_TRUNC
) execute without delay.
This should allow the writer to win the race in nearly every race condition, unless system load was also high, or it was a complicated race condition where the truncation didn't happen until after some other read. So random variation of which open()
s (and maybe read()
s or writes) are delayed would increase the detection power of this tool, but of course without testing for an infinite amount of time with a delay simulator that will eventually cover all possible situations you can encounter in the real world, you can't be sure your scripts are free from races unless you read them carefully and prove they're not.
You would probably need it to whitelist (not delay open
) for files in /usr/bin
and /usr/lib
so process-startup doesn't take forever. (Runtime dynamic linking has to open()
multiple files (look at strace -eopen /bin/true
or /bin/ls
sometime), although if the parent shell itself is doing the truncation, that will be ok. But it will still be good for this tool to not make scripts unreasonably slow).
Or maybe whitelist every file the calling process doesn't have permission to truncate in the first place. i.e. the tracing process can make an access(2)
system call before actually suspending the process that wanted to open()
a file.
racechecker
itself would have to be written in C, not in shell, but could maybe use strace
's code as a starting point and might not take much work to implement.
You could maybe get the same functionality with a FUSE filesystem. There's probably a FUSE example of a pure passthrough filesystem, so you could add checks to the open()
function in that which make it sleep for read-only opens but let truncation happen right away.
Best Answer
Yes, reading and writing from the same file in parallel could result in a race condition. An input and an output redirection for the same file on the same command would truncate the file before starting to read it.
But no, this isn't what's happening here. It's a false positive in Shellcheck. Here the redirection is inside an arithmetic expression. All substitutions (arithmetic, variable, command, as well as splitting and globbing) are performed before redirections are executed. So at the time
> "$1"
opens the file, the reading bit is finished.