Shell – Uploading multiple files via FTP using curl

curlftpshell-script

I'm trying to upload all the text files within the current folder via FTP to a server location using curl. I tried the following line:

 curl -T "{file1.txt, file2.txt}" ftp://XXX --user YYY

where XXX is the server's IP address and YYY is the username and password.

I'm able to transfer file1.txt to the server successfully, but it complains about the second file saying 'Can't open 'file_name'!'

I swapped the file names and it worked for file2.txt and not file1.txt. Seems like I've got the syntax wrong, but this is what the manual says?

Also, ideally I would be able to do something like this:

 curl -T *.txt ftp://XXX --user YYY

because I won't always know the names of the txt files in the current folder or the number of files to be transferred.

I'm of the opinion I may have to write a bash script that collects the output of ls *.txt into an array and put it into the multiple-files-format required by curl.

I've not done bash scripting before – is this the simplest way to achieve this?

Best Answer

Your first command should work without whitespaces:

curl -T "{file1.txt,file2.txt}" ftp://XXX/ -user YYY

Also note the trailing "/" in the URLs above.

This is curl's manual entry about option "-T":

-T, --upload-file

This transfers the specified local file to the remote URL. If there is no file part in the specified URL, Curl will append the local file name. NOTE that you must use a trailing / on the last directory to really prove to Curl that there is no file name or curl will think that your last directory name is the remote file name to use. That will most likely cause the upload operation to fail. If this is used on an HTTP(S) server, the PUT command will be used.

Use the file name "-" (a single dash) to use stdin instead of a given file. Alternately, the file name "." (a single period) may be specified instead of "-" to use stdin in non-blocking mode to allow reading server output while stdin is being uploaded.

You can specify one -T for each URL on the command line. Each -T + URL pair specifies what to upload and to where. curl also supports "globbing" of the -T argument, meaning that you can upload multiple files to a single URL by using the same URL globbing style supported in the URL, like this:

curl -T "{file1,file2}" http://www.uploadtothissite.com

or even

curl -T "img[1-1000].png" ftp://ftp.picturemania.com/upload/

"*.txt" expansion does not work because curl supports only the same syntax as for URLs:

You can specify multiple URLs or parts of URLs by writing part sets within braces as in:

http://site.{one,two,three}.com

or you can get sequences of alphanumeric series by using [] as in:

ftp://ftp.numericals.com/file[1-100].txt

ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)

ftp://ftp.letters.com/file[a-z].txt

[...]

When using [] or {} sequences when invoked from a command line prompt, you probably have to put the full URL within double quotes to avoid the shell from interfering with it. This also goes for other characters treated special, like for example '&', '?' and '*'.

But you could use the "normal" shell globbing like this:

curl -T "{$(echo *.txt | tr ' ' ',')}" ftp://XXX/ -user YYY

(The last example may not work in all shells or with any kind of exotic file names.)

Related Question