Delete lines with value below (or above) threshold

awkdeletesed

My file looks like this:

AA  110 B   10   ..  BB  15   ... BBB 20   D   F   25   D   D
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F
AA  112 C    2   ..  BB   3   ... BBB  0   D   F    0   D   F
AA  120 D    2   ..  FF   3   ... FFF  3   D   F    3   D   D

I would like to delete lines containing a value =< 10 in any of the columns. I am aware of the use of sed and awk '$3 !=< 10' but this would only delete lines at the third field. Is there a way to tell qwk to consider all columns?

Best Answer

perl to the rescue

$ cat ip.txt 
AA  110 B   10   ..  BB  15   ... BBB 20   D   F   25   D   D
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F
AA  112 C    2   ..  BB   3   ... BBB  0   D   F    0   D   F
AA  120 D    2   ..  FF   3   ... FFF  3   D   F    3   D   D

$ perl -ae 'print if !(grep { $_ <= 10 && /^\d+$/ } @F)' ip.txt 
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F
  • -a split input line on space and save to @F array
  • grep { $_ <= 10 && /^\d+$/ } @F get all elements of @F array which are made of only digits and whose value is <= 10
  • then print lines if grep returns 0. The () around grep means it will return count of matches rather than elements themselves

Let's test another condition:

$ perl -ae 'print if !(grep { $_ < 10 && /^\d+$/ } @F)' ip.txt 
AA  110 B   10   ..  BB  15   ... BBB 20   D   F   25   D   D
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F


Certain conditions, like in this question, can be solved with grep too (which probably is faster than perl solution)

$ grep -vw '[0-9]\|10' ip.txt 
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F

$ grep -vw '[0-9]' ip.txt 
AA  110 B   10   ..  BB  15   ... BBB 20   D   F   25   D   D
AA  111 B   50   ..  BB  55   ... BBB 30   F   F   45   F   F
  • -v lines other than matching pattern
  • -w match whole words only
Related Question