Debian SCP – Transfer Files Between Two Remote Servers

debianscp

I have one big file on server one and I want to copy it to server two using scp. I have the keys setup properly and I can ssh/scp to both servers from my desktop.

The file I need to copy is larger then the free space on my workstation's hdd, so I wanted to do:

scp one:/opt/bigfile.tar.gz two:/opt/bigfile.tar.gz

but I got:

ssh: Could not resolve hostname one: Name or service not known

We do not have DNS here (don't ask me why), so I have this in my ~/.ssh/config:

Host one
    Hostname        <IP address of server one>
    User            jspurny

Host two
    Hostname        <IP address of server two>
    User            jspurny

If I try with a smaller file and transfer it from one to my workstation and then to two, it works fine:

scp one:/opt/smallerfile.tar.gz .
scp smallerfile.tar.gz two:/opt/

When using IP addresses directly as suggested in comment, I got:

$ scp jspurny@<one's IP>:bigfile.tar.gz jspurny@<two's ip>:bigfile.tar.gz
Host key verification failed.
lost connection

Not an issue:

The size is not an issue here – it was only a "trigger" to this problem as there was no way to store bigfile.tar.gz on my workstation.
The problem occurs regardless of the file size.

Question:

Why does the command:

scp oneremote:file secondremote:file

throws an error regardless if using .ssh/config aliases or directly using ip addresses?

Solved – sort of – still looking for explanation – I've split the bigfile into smaller files and transfered them one by one through my workstation. I'm still wondering why it didn't work. So I would still appreciate some explanation of what was wrong ..

Found a reason why it fails: It seems I was being foolish. I thought that the command 

scp one:file two:file

was creating two connections to each server and then receive data from one and immediately send them to two and thus acting like a relay.

This is clearly not the case, because a simple -v option revealed that it in fact just connects to one and from one it tries to connect to two. Which obviously isn't possible because server one is not supposed to connect to two.

Best Answer

Simple pipe

Try this:

ssh one 'cat file' | ssh two 'cat > file'

The first one should send the file content to your machine, while the second should send that on to second machine. I would compute a checksum at both ends after the transfer, to ensure that nothing got lost or garbled along the way.

Elaborate tunnels

For more elaborate applications, you could use ssh tunnels. For example, you could try something like this:

ssh -R 5001:127.0.0.1:5002 one
ssh -L 5002:127.0.0.1:22 two

Then you can open a connection on one to machine localhost port 5001 and it will be forwarded twice and end up as a connection on two to localhost port 22. This is the ssh port, so you can use this for yet another scp, or for rsync, or whatever. You could also start an rsync server on two, and forward port 873 instead of 22. Or you could use nc on both sides to transfer raw data, using an arbitrary port number.

The main benefit between the above approach is that you have a two-directional tcp connection between the two machines, instead of a one-directional pipe only. That way, the two parties can exchange information, which is particularly important in the rsync case.

Related Question