Convert JSON Array to Bash Variables Using jq – Tutorial

bashjq

I've got a JSON array like so:

{
  "SITE_DATA": {
    "URL": "example.com",
    "AUTHOR": "John Doe",
    "CREATED": "10/22/2017"
  }
}

I'm looking to iterate over this array using jq so I can set the key of each item as the variable name and the value as it's value.

Example:

URL="example.com"
AUTHOR="John Doe"
CREATED="10/22/2017"

What I've got so far iterates over the array but creates a string:

constants=$(cat ${1} | jq '.SITE_DATA' | jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]")

Which outputs:

URL=example.com
AUTHOR=John Doe
CREATED=10/22/2017

I am looking to use these variables further down in the script:

echo ${URL}

But this echos an empty output at the moment. I'm guessing I need an eval or something in there but can't seem to put my finger on it.

Best Answer

Your original version isn't going to be evalable because the author name has spaces in it - it would be interpreted as running a command Doe with the environment variable AUTHOR set to John. There's also virtually never a need to pipe jq to itself - the internal piping & dataflow can connect different filters together.

You can make a much simpler version of the jq program:

jq -r '.SITE_DATA | to_entries | .[] | .key + "=\"" + .value + "\""'

which outputs:

URL="example.com"
AUTHOR="John Doe"
CREATED="10/22/2017"

There's no need for a map: .[] deals with taking each object in the array through the rest of the pipeline as a separate item, so everything after the last | is applied to each one separately. At the end, we just assemble a valid shell assignment string with ordinary + concatenation, including quotes around the value.

All the pipes matter here - without them you get fairly unhelpful error messages, where parts of the program are evaluated in subtly different contexts.

This string is evalable as long as the characters `, $, newline and null don't appear in the data:

eval "$(jq -r '.SITE_DATA | to_entries | .[] | .key + "=\"" + .value + "\""' < data.json)"
echo "$AUTHOR"

As ever when using eval, be careful that you trust the data you're getting, since if it's malicious or just in an unexpected format things could go very wrong.

Example

$ array[0]=0
$ array[1]=1
$ array[2]=2

You can then recall them individually:

$ echo ${array[0]}
0

$ echo ${array[1]}
1

$ echo ${array[2]}
2

Or as a list:

$ echo "${array[@]}"
0 1 2

You cannot just call the array as another variable:

$ echo $array
0

I believe this may have been what was misleading you into thinking it wasn't working correctly. Also setting an array element with another variable's value also works.

Example

$ newval=3
$ array[2]=$newval

$ echo "${array[@]}"
0 1 3

You could also use printf here, doing so will show you why you want to quote the results of ${array[@]}. But first let's add a string to the array:

$ array[0]="a string"

Without the quotes:

$ printf "%s\n" ${array[@]}
a
string
1
3

With quotes:

$ printf "%s\n" "${array[@]}"
a string
1
3

Passing things around

Well you have 2 tactics that you can make use of within Bash. You can simply set variables so that they're globally scoped. This is often the technique that most people employ.

The other method involves passing by value, values to functions within Bash. I'll show a basic function where I pass it 2 values.

Example

$ function hello() {
 printf "I got arg1: %d and arg2: %d\n" $1 $2
}

Now when I execute it with different arguments:

$ hello 1 2
I got arg1: 1 and arg2: 2

$ hello 3 4
I got arg1: 3 and arg2: 4

$ hello 3 blah
bash: printf: blah: invalid number
I got arg1: 3 and arg2: 0

You can read more about arrays and functions within Bash here to get a better understanding of the technologies.

Best Answer

Related Solutions

Bash – How to reassign new values to array elements

Example

Example

Passing things around

Example

References

Related Question