How to parse an xml file in a shell script

text processingxml

I would like to know how can I split my data from the following format:

<datas>
 <data>
  <name>Name1</name>
 </data>
 <data>
  <name>Name2</name>
 </data>
</datas>

to the following format:

<data><name>Name1</name></data>
<data><name>Name2</name></data>

The parsed data would be sent to a Python script as follows:

 python script.py <data><name>Name1<name></data>
 python script.py <data><name>Name2<name></data>

I have tried commands like:

echo 'cat /datas/data' | xmllint --shell file.xml

but how can I pass the output in the desired format to the Python script?

Best Answer

I would preprocess the data with XMLStarlet:

$ xml sel -t -c '/datas/data' -nl data.xml
<data>
  <name>Name1</name>
 </data><data>
  <name>Name2</name>
 </data>

Then it depends on how you Python script wants to read this data. Hopefully, it's from a file or from standard input...

Related Solutions

Take two columns in a tab delimited file and merge into one

Similar to cut , you can also do it with awk:

$ awk '{print $1,$2}' aa.txt && awk '{print $1,$3}' aa.txt
A red
B yellow
C blue
A green
B orange
C purple
# OR to send the output in a new file:
$ (awk '{print $1,$2}' aa.txt && awk '{print $1,$3}' aa.txt) >aaa.txt

The difference is that awk handles better the white space than cut. This is useful if fields in each line are separated with more than one space.

For example if the file line is A red = one space separated, then cut solution as advised can do it also successfully, but if the line is A red = 3 spaces , then cut will fail, while awk will succeed to get fields 1 and 2 or fields 1 and 3.

Update:
As advised in comments (thanks don_crissti) this can also be done in pure awk:

awk 'BEGIN{FS=OFS=" "}{z[NR]=$1FS$3; print $1,$2}END{for (i=1; i<=NR; i++){print z[i]}}' a.txt

Explanation:

FS           : Input Field Separator
OFS          : Output Field Separator
FS=OFS=" "   : input & output field separator is set to "space"
z[NR]        : Creating an array with name 'z' and index the record number: 
             z[1] for first line, z[2] for second line , z[3] for third line
z[NR]=$1FS$3 : to each array element assign field1-FieldSeparator FS=space)-field2
So for first line the fields1=A and Fields 3=green will be stored in z[1] => equals to z[1]="A green"

print $1,$2  : Justs prints on screen 1stfield (A) and 2ndfield (red) of the current line, printed separated by OFS

When the file is finished (END) then with a for loop we print out the whole z array entries => print z[i]
For i=1 => print z[1] => prints "A green"
For i=2 => print z[2] => prints "B orange"
For i=3 => print z[3] => prints "C purple"

PS: If fields are not separated by space but by tab , then Begin section of this awk one-liner must be changed to `awk 'BEGIN {FS=OFS="\t"}....`

Parse logs for certain values

You asked for a shell script, but hopefully awk will do:

find_thrust_torque.awk:

/^Total thrust =/ {thrust = $4}
/^Total torque =/ {torque = $4}
/^Time =/ {if (found) exit; if ($3 == time) found=1}
END {print torque " " thrust}

Test It:

$ awk -v time=25 -f find_thrust_torque.awk file1
1.03723 -8.49093

$ awk -v time=26 -f find_thrust_torque.awk file1
1.03953 -8.51463