Ubuntu – How to prevent grep from printing the same string multiple times

bashcommand linegrep

If I grep a file containing the following:

These are words
These are words
These are words
These are words

…for the word These, it will print the string These are words four times.

How can I prevent grep from printing recurring strings more than once? Otherwise, how can I manipulate the output of grep to remove duplicate lines?

Best Answer

The Unix philosophy is to have tools that do one thing and do them well. In this case, grep is the tool that selects text from a file. To find out if there are duplicates, one sorts the text. To remove the duplicates, one uses the -u option to sort. Thus:

grep These filename | sort -u

sort has many options: see man sort. If you want to count duplicates or have a more complicated scheme for determining what is or is not a duplicate, then pipe the sort output to uniq: grep These filename | sort | uniq and see manuniq` for options.

Related Solutions

Ubuntu – How to grep in the content of a string variable

If you are just looking for a word you can use a for loop.

STRING="upgrade this if you can"
for x in $STRING; do
   echo $x
   if [ "$x" = 'upgrade' ]; then
       echo found
       y=$x
       break
   fi
done 
echo $y

If upgrade is always in the same position you could try array assignment.

declare -a z
z=($STRING)
echo ${z[0]}

Ubuntu – command to remove specific string from multiple files

You can achieve this rather easily with sed which can happily look into multiple files

sed '/D PRINT/d' dash7/*

/D PRINT/ find a line with D PRINT
d delete the line
dash7/* look in all the files in the directory dash7 (add the path to it, for example ~/dash7 if required)

To actually change the files rather than print the edited text in the terminal, you need to add the -i flag to modify in place

sed -i '/D PRINT/d' dash7/*

Best Answer

Related Solutions

Ubuntu – How to grep in the content of a string variable

Ubuntu – command to remove specific string from multiple files

Related Question