I'm currently working on a class that has us submit our code to an auto-grader and then returns our results. The format it returns is kind of hard to visually parse, so I would like to write a script that I can use in a pipe to make it easier to read.
Here is the output of the auto-grader:
Problem,Correct?,Correct Answer,Agent's Answer
"Challenge Problem B-04",0,4,-1
"Basic Problem B-12",0,1,-1
"Challenge Problem B-05",0,6,-1
"Challenge Problem B-07",0,6,-1
"Challenge Problem B-06",0,3,-1
"Basic Problem B-11",0,1,-1
"Basic Problem B-10",0,3,-1
"Challenge Problem B-03",0,3,-1
"Challenge Problem B-02",0,1,-1
"Challenge Problem B-01",0,6,-1
"Challenge Problem B-09",0,4,-1
"Challenge Problem B-08",0,4,-1
"Basic Problem B-08",0,6,-1
"Basic Problem B-09",0,5,-1
"Basic Problem B-04",0,3,-1
"Basic Problem B-05",0,4,-1
"Basic Problem B-06",0,5,-1
"Basic Problem B-07",0,6,-1
"Basic Problem B-01",0,2,-1
"Basic Problem B-02",0,5,-1
"Basic Problem B-03",0,1,-1
"Challenge Problem B-10",0,4,-1
"Challenge Problem B-11",0,5,-1
"Challenge Problem B-12",0,1,-1
{
"Basic Problems B": {
"Incorrect": "0",
"Skipped": "12",
"Correct": "0",
"Set": "Basic Problems B"
},
"Challenge Problems B": {
"Incorrect": "0",
"Skipped": "12",
"Correct": "0",
"Set": "Challenge Problems B"
}
}
It's a mix of comma separated values and JSON. It would be nice to place this all in a pretty table that I can read.
Currently, I have something like
python submit.py --provider gt --assignment error-check | column -t -s, | less -S
Which outputs:
{
"Basic Problems B": {
"Incorrect": "0",
"Skipped": "12",
"Correct": "0",
"Set": "Basic Problems B"
},
"Challenge Problems B": {
"Incorrect": "0",
"Skipped": "12",
"Correct": "0",
"Set": "Challenge Problems B"
}
}
Problem Correct? Correct Answer Agent's Answer
"Challenge Problem B-04" 0 4 -1
"Basic Problem B-12" 0 1 -1
"Challenge Problem B-05" 0 6 -1
"Challenge Problem B-07" 0 6 -1
"Challenge Problem B-06" 0 3 -1
"Basic Problem B-11" 0 1 -1
"Basic Problem B-10" 0 3 -1
"Challenge Problem B-03" 0 3 -1
"Challenge Problem B-02" 0 1 -1
"Challenge Problem B-01" 0 6 -1
"Challenge Problem B-09" 0 4 -1
"Challenge Problem B-08" 0 4 -1
"Basic Problem B-08" 0 6 -1
"Basic Problem B-09" 0 5 -1
"Basic Problem B-04" 0 3 -1
"Basic Problem B-05" 0 4 -1
"Basic Problem B-06" 0 5 -1
"Basic Problem B-07" 0 6 -1
"Basic Problem B-01" 0 2 -1
"Basic Problem B-02" 0 5 -1
"Basic Problem B-03" 0 1 -1
"Challenge Problem B-10" 0 4 -1
"Challenge Problem B-11" 0 5 -1
"Challenge Problem B-12" 0 1 -1
This gets me most of the way there. Now I'm wondering if there is a way I can handle the JSON?
I can't rely on splitting the output at a certain line-number, but am thinking I can segment the output when it first finds a {
.
I would like to do this as minimal as possible so I can share it with classmates. So the less dependencies the better.
I've seen other JSON parsing posts suggesting to use external codes.
An ideal output would look something like:
Problem Correct? Correct Answer Agent's Answer
"Challenge Problem B-04" 0 4 -1
"Basic Problem B-12" 0 1 -1
"Challenge Problem B-05" 0 6 -1
"Challenge Problem B-07" 0 6 -1
"Challenge Problem B-06" 0 3 -1
"Basic Problem B-11" 0 1 -1
"Basic Problem B-10" 0 3 -1
"Challenge Problem B-03" 0 3 -1
"Challenge Problem B-02" 0 1 -1
"Challenge Problem B-01" 0 6 -1
"Challenge Problem B-09" 0 4 -1
"Challenge Problem B-08" 0 4 -1
"Basic Problem B-08" 0 6 -1
"Basic Problem B-09" 0 5 -1
"Basic Problem B-04" 0 3 -1
"Basic Problem B-05" 0 4 -1
"Basic Problem B-06" 0 5 -1
"Basic Problem B-07" 0 6 -1
"Basic Problem B-01" 0 2 -1
"Basic Problem B-02" 0 5 -1
"Basic Problem B-03" 0 1 -1
"Challenge Problem B-10" 0 4 -1
"Challenge Problem B-11" 0 5 -1
"Challenge Problem B-12" 0 1 -1
Set Incorrect Skipped Correct
Basic Problems B 0 12 0
Challenge Problems B 0 12 0
Best Answer
Separating the JSON from the rest is quite easy. This will give you the non JSON only:
And this, only the JSON:
To illustrate, I have saved your example input as
file
and:And
Now, you already deal with the non-JSON perfectly well, so I won't change that. Ideally, the JSON data should be parsed using a JSON parser, like
jq
. Sadly, I don't know enoughjq
to do this properly, so the best I could come up with is this, rather inelegant, solution. At least it does do what you want (replacecat file
with yourpython submit.py --provider gt --assignment error-check
command:Putting all this together in a shell script gives:
Which produces the following output:
It feels hacky though, and I hope someone can come up with a cleaner solution with dedicated JSON parsers.
Steeldriver was nice enough to give a proper
jq
solution in a comment, so if we incorporate that, we get the far simpler (and safer):