Cannot Create Regular File ‘filename’: File Exists Error

concurrencycp

I've been getting this strange error message in one of my build scripts – cp fails, returning the error "File exists". I've even tried using cp -f, which should overwrite the file if it exists, but the error still shows up. Running cp to overwrite existing files works perfectly when I do it manually. What could cause this error?

Best Answer

This turned out to be caused by a race condition. cp checks if the destination file already exists, and if not - overwrites it. The problem was happening because this cp command was being run twice in parallel, which caused the file in question to sometimes appear after checking whether it exists, but before the attempt to create the file. The strace output looks like this:

# Command was "cp a b"
stat("b", 0x7fff89510620)               = -1 ENOENT (No such file or directory)
stat("a", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("b", 0x7fff895103a0)               = -1 ENOENT (No such file or directory)
# File b will be created at this point in time
open("a", O_RDONLY)                     = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("b", O_WRONLY|O_CREAT|O_EXCL, 0644) = -1 EEXIST (File exists)

Here's some bash code used to catch this:

#!/bin/bash

touch a

f() {
  while true; do
    rm -f b
    strace -o /tmp/cp${BASHPID}.trace cp a b || break
  done
}

cleanup() {
  kill -9 %1 %2
}

f &
f &

trap cleanup exit

wait

This same error can occur with mkdir -p or any other action that attempts to overwrite a file. Using flock can help avoid race conditions in cases like this.

Related Question