Windows – curl capturing http status and timing the request

command linecurlperformancewindows

I have a loop in wich I check urls from file one-by-one and want to get the status header and the execution time of every request as result.
I have two curl commands:

This one outputs the header communication (not wanted) and at the end the http status code and time 200 - 0,016

curl -w "%{http_code} - %{time_connect}" -I -s http://superuser.com >> test_result.txt

This one gets the line with the http status code from the header and prints it to the file HTTP/1.1 200 OK

curl -v -I -s http://superuser.com | findstr /c:"HTTP" >> test_result.txt

How can I combine these two commands to get output from one request as this

HTTP/1.1 200 OK - 0,016

so extracting the line with http header and appending the execution time without all other headers in file

Best Answer

You can use a batch file

@echo off
    setlocal enableextensions disabledelayedexpansion

    rem Just to test - Generate a file with URLs
    > ".\urls.txt" (
        echo http://superuser.com
        echo http://www.google.com
    )

    > ".\testResults.txt" (
        for /f "useback delims=" %%u in (".\urls.txt") do (
            set "statusCode="
            echo([%%u]
            for /f "tokens=1,2 delims=#" %%a in ('
                curl -w "##%%{time_connect}##." -I -s --url "%%~u"
                ^| findstr /l /b /c:"HTTP/" /c:"##"
            ') do if "%%b"=="." (
                setlocal enabledelayedexpansion
                echo(    !statusCode! - %%a
                endlocal
            ) else (
                set "statusCode=%%a"
            )
        )
    )

Just two nested loops. The first iterates over the URLs file and the second executes and processes the output of the curl command.

The output of curl command is filtered with findstr to retrieve only the lines with the status code and the time to connect (the output string in -w has been modified to locate this data in the output)

Related Solutions

HTTP responses curl and wget different results

You are including invalid HTTP headers in your request:

$header[ ] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[ ] = "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";

On the first line, the list ends with a , – that is, an empty content type – which is the cause of 406 Not acceptable errors. The second line is not even a HTTP header.

If you were looking at Firefox HTTP conversations with a packet sniffer, you probably saw something like this:

Accept: text/xml,application/xml,application/xhtml+xml,
    text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Since the second line starts with whitespace, they are treated as a single header by the server. They must also be passed as one header to curl:

$header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";

You can use http://echo.opera.com to compare the requests being sent.

How to split the HTTP error code from the contents in cURL

There is no need to use a temporary file. The following bash script snippet send a single request and prints the exit code of curl and the HTTP status code, or the HTTP status code and response, as appropriate.

# get output, append HTTP status code in separate line, discard error message
OUT=$( curl -qSfsw '\n%{http_code}' http://superuser.com ) 2>/dev/null

# get exit code
RET=$?

if [[ $RET -ne 0 ]] ; then
    # if error exit code, print exit code
    echo "Error $RET"

    # print HTTP error
    echo "HTTP Error: $(echo "$OUT" | tail -n1 )"
else
    # otherwise print last line of output, i.e. HTTP status code
    echo "Success, HTTP status is:"
    echo "$OUT" | tail -n1

    # and print all but the last line, i.e. the regular response
    echo "Response is:"
    echo "$OUT" | head -n-1
fi

head -n-1 (print all but the last line) requires GNU, doesn't work on BSD/OS X.

Best Answer

Related Solutions

HTTP responses curl and wget different results

How to split the HTTP error code from the contents in cURL

Related Question