Jq filtering curl- ‘Cannot index number with string’

curljson

I am trying to filter a curl request with jq to get specific fields and put them in variables. Initially, I was using a basic curl request such as this:

myResponse=$(curl -u myUsername:myPassword -XGET "https://myWebsite.com/_count?pretty&q=TEST")

I am providing my username and password, and getting a count for how many times the term 'TEST' is present, which is returned in pretty json. This response is put into a variable called myResponse.

I used jq to filter this to get a specific count value from the json:

count=`echo $myResponse | jq -r '.count'`

This worked, and provided me with a number.

So now I wanted to experiment a bit. As well as getting the response from the curl request I also wanted to retrieve the response code (200, 404 etc). So I adapted my curl request to the following:

myResponse=$(curl --write-out %{http_code} --silent -u myUsername:myPassword -XGET "https://myWebsite.com/_count?pretty&q=DEBUG")

This successfully returned both the JSON response and then after this it returned the http response code as a number.

However, once this is passed into jq, I get the following error:

jq: error: Cannot index number with string

How do I resolve this error? When it was just json jq handled it fine, but as soon as this responsecode was added onto the end it can't parse it.

Thanks

Best Answer

There was both JSON and HTTP in the variable I was piping into jq. You first must split the variable into JSON and HTTP using tail and head. Once I piped pure JSON into jq, it worked.

Related Solutions

Curl http_code of 000

The response 000 indicates that cURL failed to execute for some reason. In such a case, you should test for cURL exit code rather than making assumptions. See the "Exit Codes" section of the curl manpage for a full list of exit codes and their meanings.

Failed DNS resolution (6)

$ curl -w "%{http_code}\n" http://example.invalid/ ; echo "Exit code: $?"
000
curl: (6) Could not resolve host: example.invalid
Exit code: 6

(As answered by ILIV)

Connection refused (7)

$ curl -w "%{http_code}\n" http://localhost:81/ ; echo "Exit code: $?"
000
curl: (7) Failed to connect to localhost port 81: Connection refused
Exit code: 7

Connection timed out (28)

$ curl -w "%{http_code}\n" -m 5 http://10.255.255.1/ ; echo "Exit code: $?"
000
curl: (28) Connection timed out after 5001 milliseconds
Exit code: 28

(As answered by Arun)

Server actually returns 000 for some reason (0)

Start a fake server:

$ nc -l -p 65535 & <<EOF
> HTTP/1.1 000 Fake Status Code
> Content-Length: 0
> Connection: close
>
> EOF

Client request:

$ curl -w "%{http_code}\n" http://localhost:65535/ ; echo "Exit code: $?"
000
Exit code: 0

No idea why this would happen in the real world, but hey. If cURL doesn't get a valid status code at all, it assumes 200.

How to split the HTTP error code from the contents in cURL

There is no need to use a temporary file. The following bash script snippet send a single request and prints the exit code of curl and the HTTP status code, or the HTTP status code and response, as appropriate.

# get output, append HTTP status code in separate line, discard error message
OUT=$( curl -qSfsw '\n%{http_code}' http://superuser.com ) 2>/dev/null

# get exit code
RET=$?

if [[ $RET -ne 0 ]] ; then
    # if error exit code, print exit code
    echo "Error $RET"

    # print HTTP error
    echo "HTTP Error: $(echo "$OUT" | tail -n1 )"
else
    # otherwise print last line of output, i.e. HTTP status code
    echo "Success, HTTP status is:"
    echo "$OUT" | tail -n1

    # and print all but the last line, i.e. the regular response
    echo "Response is:"
    echo "$OUT" | head -n-1
fi

head -n-1 (print all but the last line) requires GNU, doesn't work on BSD/OS X.

Related Question

How to display curl’s individual Exit Status from multiple requests