bashcommand line
I have a file that is built like this:
(MATH[NUMBER1=(50, unknown, unknown), NUMBER2=(unknown, 4, unknown), OPERATOR='times']
(NUM[SEM=(50, unknown, unknown)] (DIZAINE[SEM=50] cinquante))
(OPERATEUR[SEM='times'] multiplie)
(NUM[SEM=(unknown, 4, unknown)] (UNITE[SEM=4] quatre)))
How can I extract the values 50, 'times' & 4?
I've tried with awk but there are parentheses balancing issues
Bash doesn't support that kind of syntax directly. You could use 'eval', however with modern versions of 'bash' the cleanest way imho is via an array
input=( "/tmp/dir1" "/tmp/dir2" "/tmp/dir3" )
for i in {0,1,2}; do echo "${input[i]}"; done
(note that bash arrays are zero-indexed); or to list all elements you can use
for i in "${input[@]}"; do echo "$i"; done
You could use the command sort with the option --unique:
sort -u input-file
If you want to write result to FILE instead of standard output, use the option --output=FILE:
sort -u input-file -o output-file
The command uniq also could be applied. In this case the identical lines must be consequential, so the input must be sorted preliminary - thanks to @RonJohn for this note:
sort input-file | uniq > output-file
I like the sort command for similar cases, because of its simplicity, but if you work with large arrays the awk approach from John1024's answer could be more powerful. Here is a time comparison between the mentioned approaches, applied on a file (based on the above example) with almost 5 million lines:
$ cat input-file | wc -l
20000000
$ TIMEFORMAT=%R
$ time sort -u input-file | wc -l
64
7.495
$ time sort input-file | uniq | wc -l
64
7.703
$ time awk '!a[$0]++' input-file | wc -l # from John1024's answer
64
1.271
$ time datamash rmdup 1 < input-file | wc -l # from αғsнιη's answer
64
0.770
Other significant difference is that mentioned by @Ruslan:
sort -uwill only print the result once the input has ended, while thisawkcommand will do print each new result line on the fly (this may be more important for piped input than file).
Here is an illustration:
In the above example, the loop (shown below) generates 500 random combinations, each with a length of three characters, of the letters A-D. These combinations are piped to awk or sort.
for i in {1..500}; do cat /dev/urandom | tr -dc A-D | head -c 3; echo; done
Best Answer
If you want to extract the non parenthesized values of the
SEMattribute then you can do so usinggrepin PCRE mode:or
perlitselfBoth approaches use regular expression lookarounds.