Way to tell if a file is done copying

scp

The scenario is this: Machine A has files I want to copy to Machine C. Machine A can't access C directly, but can access Machine B that can access Machine C. I am using scp to copy from Machine A to B, and then from B to C.

Machine B has limited storage space, so as files come in, I need to copy them to C and delete them from B. The second copy is much faster, so this is no problem with bandwidth.

I could do this by hand, but I am lazy. What I would like is to run a script on B or C that will copy each file to C as each one finishes. The scp job is running from A.

So what I need is a way to ask (preferably from a bash script) if file X.avi is "done" copying. Each of these files is a different size, and I can't really predict size or time of completion.

Edit: by the way, the file transfer times are something about 1 hour from A to B and about 10 minutes from B to C, if time scale matters at all.

Best Answer

A common way of doing this is to first copy it to a temporary file name, preferably a hidden file. When the copy finishes, the script that is doing the copy then renames it to the non-hidden filename.

The script on machine B could then watch for non-hidden files.

The script on machine A would look something like this:

for file in `ls *` ; do
    scp $file user@host:~/.${file}.tmp
    ssh user@host "mv ~/.${file}.tmp $file"
done

Although this does not satisfy OP's desire to use the one-line

scp * user@host:~/

it accomplishes the same thing and also allows machine B to transfer each file as it finishes without waiting for the next file.

Related Question