Linux – How to make a local tar file of a remote directory

linuxscpsshtar

This link describes how to copy tarred files to minimise the amount of date sent over the network. I am trying to do something slightly different.

I have a number of remote files on different subdirectory levels:

remote:/directory/subdir1/file1.ext
remote:/directory/subdir1/subsubdir11/file11.ext
remote:/directory/subdir2/subsubdir21/file21.ext

And I have a file that lists all of them:

remote:/directory/allfiles.txt

To copy them most efficiently, on the remote site I could just do

tar zcvf allfiles.tgz `cat /directory/allfiles.txt`

but there is not enough space to do that.

I do have enough storage space on my local disc. Is there a way to tar an incoming stream from a remote server (using scp or ssh for the transfer)?

Something like /localdir$ tar zc - | ssh remotecat /directory/allfiles.txt` I would guess – but that would only list the remote files on the local host.

Best Answer

You got it almost right, just run the tar on the remote host instead of locally. The command should look something like the following:

ssh remote_host tar cvfz - -T /directory/allfiles.txt > remote_files.tar.gz

Related Solutions

SSH File Transfer – How to Transfer Files Over Two-Hop SSH Session

Use the SSH ProxyCommand configuration option to in the configuration on your client. This allows you to make a direct connection to the destination.

Host remotebox
    ProxyCommand /usr/bin/ssh username@bridge.work.com "/bin/netcat -w 1 10.10.10.130 22"
    User username

scp remotebox:file local

Command Line – Fix ‘No Such File or Directory’ Error When Using SCP

scp does not require that you SSH to the remote computer in order to make the copy (and this is where you're currently running into trouble with your command).

scp essentially works in either a push or pull fashion. You can push files/folders from your local PC to a remote. The command syntax for this method is as follows:

scp /folderpath/tofile/file.txt user@remotehost:/folderpath/tocopyfileto/

Which will prompt you for the password of user on remotehost.

You can also pull file/folders from a remote PC to your local PC. The command syntax for this method is as follows:

scp user@remotehost:/folderpath/tofile/file.txt /folderpath/tolocalfolder/

Which will also prompt you for the password to user on remotehost.

The problem you were running into with your command above is that you were using the data pull syntax of the scp command in order to grab a file from a remotehost, but you were also SSH'd into that remotehost while running it.

The correct way to run this command is to run it from your local machine

scp -P 2222 user@remotehost.com:/home4/user/public_html/website.com/folder_1/folder_2/file_to_copy.zip /Users/username/Desktop/

**Note that I removed the superfluous -r from your original command. It's not something that will throw off an error, but it's just that it's not necessary in your case. The -r option of scp is to be used when recursively copying a folder and all its contents. In your case you were just copying a file so it wasn't necessary.

**I also added the -P 2222 since subsequent comments from you indicated that you needed to use port 2222 instead of standard port 22.

Best Answer

Related Solutions

SSH File Transfer – How to Transfer Files Over Two-Hop SSH Session

Command Line – Fix ‘No Such File or Directory’ Error When Using SCP

Related Question