How to speed up the ftp upload process

ftpperformanceupload

So why are FTP uploads soo slow? I am using filezila as a client.

I have like 10 mb in 1000 files and I can upload each individual file with 300-500kb/s yet the whole upload is incredibily slow due to the queueing process that occurs as files are uploaded. For every singe file the client performs all kind of commands and connection operations before actually uploading.

Is there no way to skip over these commands? I am new to ftp clients/uploads/websites etc Is this standard practice? Is this just the way ftp uploads work? Don't you get bored waiting like 20 minutes for 8-10 mb?

How can I efficiently upload 100 mb or more?

Best Answer

Sadly, this is the way that FTP functions. To efficiently transfer lots of small files, either archive them locally, transmit the entire archive via ftp, and then unarchive the files on the remote machine, or turn on simultaneous uploads, where the client is uploading 10 files at once. This will help to fully saturate your upload link.

Related Solutions

Upload from URL to FTP server

There is no cross specification of HTTP and FTP which would allow the two to interact. Someone, somewhere, would have to download, and then upload it.

If you have shell access to the web server, the easiest way would be to upload it directly from the webserver to the ftp server with the ftp command (Assuming it's a *nix server.)

If you have shell access to the ftp server, then you could also use the wget command to download the file directly to the ftp server, again assuming it's a *nix server.

Linux – How to upload local changed files to an FTP server

Install lftp if you don't have it yet. Then create a bash script containing the following:

#!/bin/bash
TARGET='/remote/folder'
SOURCE='/your/local/folder'  
lftp -f "
open <hostname>
user <username> <password>
lcd $SOURCE
mirror --reverse --delete --verbose $SOURCE $TARGET
bye
"

Replace <hostname>, <username> and <password> with your actual data.

This will effectively sync your local files to the remote folder, deleting everything that doesn't exist locally anymore (this is the delete option). --reverse specifies that you want to actually update your remote folder instead of your local one.

Have a look at lftp's man page for more details. And always have a backup ready so you don't delete anything in case a command goes wrong.

_{This is adapted from this script here.}

Best Answer

Related Solutions

Upload from URL to FTP server

Linux – How to upload local changed files to an FTP server

Related Question