How to replace multiple lines with “sed” and convert to JSON

jqjsonreplacesed

{
  "auth": 'log',
  "appid": 21,
  "custid": "599c1f910f53ada8468b4567",
  "hwid": "59e719ba0f53adfd6a8b4597"
}

24/10/2017 12:44:24
--------------------------------------------------------------------------------
{
  "auth": 'log',
  "appid": 21,
  "custid": "599c1f910f53ada8468b4567",
  "hwid": "59e719ba0f53adfd6a8b4597"
}

I need to transform a list of jsons delimited by some log lines from a file into a single json . so far i've tried this:

tac tst.txt | sed '/---------/I,+2 d' | tac > out.json

and managed to delete the log lines, but I need to replace them with a comma and add them into a list. How can I do this ? or there is any alternative for this using python ?

Best Answer

sed + jq solution:

sed -E "/^(---|[0-9][0-9])/d; s/'([^']+)'/\"\1\"/" tst.txt | jq -s '' out.json

  • /^(---|[0-9][0-9])/d - delete the unnecessary lines which start either with --- or 2 digits

  • s/'([^']+)'/\"\1\"/" - replace a value in single quotes to be a valid JSON value (enclosed in double quotes)

  • jq -s '' - instead of running the filter for each JSON object in the input, read the entire input stream into a large array

The out.json contents:

[
  {
    "auth": "log",
    "appid": 21,
    "custid": "599c1f910f53ada8468b4567",
    "hwid": "59e719ba0f53adfd6a8b4597"
  },
  {
    "auth": "log",
    "appid": 21,
    "custid": "599c1f910f53ada8468b4567",
    "hwid": "59e719ba0f53adfd6a8b4597"
  }
]
Related Question