Shell – posting data using cURL in a script

curlescape-charactersfile-transferprintfshell-script

I'm trying to write a simple alternative script for uploading files to the transfer.sh service. One of the examples on the website mentions a way of uploading multiple files in a single "session":

$ curl -i -F filedata=@/tmp/hello.txt \
  -F filedata=@/tmp/hello2.txt https://transfer.sh/

I'm trying to make a function that would take any number of arguments (files) and pass them to cURL in such fashion. The function is as follows:

transfer() {
    build() {
        for file in $@
        do
            printf "-F filedata=@%q " $file
        done
    }

    curl --progress-bar -i \
        $(build $@) https://transfer.sh | grep https
}

The function works as expected as long as no spaces are in the filenames.
The output of printf "-f filedata=@%q " "hello 1.txt" is -F filedata=@test\ 1.txt, so I expected the special characters to be escaped correctly. However, when the function is called with a filename that includes spaces:

$ transfer hello\ 1.txt

cURL does not seem to interpret the escapes and reports an error:

curl: (26) couldn't open file "test\"

I also tried quoting parts of the command, e.g. printf "-F filedata=@\"%s\" " "test 1.txt", which produces -F filedata=@"test 1.txt".
In such case cURL returns the same error: curl: (26) couldn't open file ""test". It seems that it does not care about quotes at all.

I'm not really sure what causes such behaviour. How could I fix the function to also work with filenames that include whitespace?

Edit/Solution

It was possible to solve the issue by using an array, as explained by @meuh. A solution that works in both bash and zsh is:

transfer () {
    if [[ "$#" -eq 0 ]]; then
        echo "No arguments specified."
        return 1
    fi

    local -a args
    args=()
    for file in "$@"
    do
        args+=("-F filedata=@$file")
    done

    curl --progress-bar -i "${args[@]}" https://transfer.sh | grep https
}

The output in both zsh and bash is:

$ ls
test space.txt    test'special.txt
$ transfer test\ space.txt test\'special.txt
######################################################################## 100.0%
https://transfer.sh/z5R7y/test-space.txt
https://transfer.sh/z5R7y/testspecial.txt
$ transfer *
######################################################################## 100.0%
https://transfer.sh/4GDQC/test-space.txt
https://transfer.sh/4GDQC/testspecial.txt

It might be a good idea to pipe the output of the function to the clipboard with xsel --clipboard or xclip on Linux and pbcopy on OS X.

The solution provided by @Jay also works perfectly well:

transfer() {
  printf -- "-F filedata=@%s\0" "$@" \
    | xargs -0 sh -c \ 
    'curl --progress-bar -i "$@" https://transfer.sh | grep -i https' zerop
}

Best Answer

One way to avoid having bash word-splitting is to use an array to carry each argument without any need for escaping:

push(){ args[${#args[*]}]="$1"; }
build() {
    args=()
    for file
    do  push "-F"
        push "filedata=@$file"
    done
}
build "$@"
curl --progress-bar -i "${args[@]}" https://transfer.sh | grep https

The build function creates an array args and the push function adds a new value to the end of the array. The curl simply uses the array.

The first part can be simplified, as push can also be written simply as args+=("$1"), so we can remove it and change build to

build() {
    args=()
    for file
    do  args+=("-F" "filedata=@$file")
    done
}
Related Question