Rm -rf failing if deleting in parallel

concurrencyfilesrm

rm -rf will fail if something tries to delete the same file tree (I think because rm enumerates the files first, then deletes).

A simple test:

# Terminal 1
for i in `seq 1 1000`; do mkdir -p /tmp/dirtest/$i; done

# Now at the same time in terminal 1 and 2
rm -rf /tmp/dirtest

There will be some output into stderr, e.g.:

...
rm: cannot remove directory `/tmp/dirtest/294': No such file or directory
rm: cannot remove directory `/tmp/dirtest/297': No such file or directory
rm: cannot remove directory `/tmp/dirtest/304': No such file or directory

I can ignore all the stderr output by redirecting it to /dev/null, but removing of /tmp/dirtest actually fails! After both commands are finished, /tmp/dirtest is still there.

How can I make rm delete the directory tree properly and really ignore all the errors?

Best Answer

Nasty. But in a sense, you're looking for trouble when two concurrent processes are manipulating a directory tree. Unix provides primitives for atomic manipulation of a single file, but not for whole directory trees.

A simple workaround would be for your script to rename the directory before removing it. Since your use case has cooperating scripts, it's ok for the new name to be predictable.

mv /build/tree /build/tree.rm.$$
mkdir /build/tree
rm -rf /build/tree.rm.$$

Maybe you can even do the rm in the background later, while your build performs some CPU-bound tasks.

Related Question