Ubuntu – command to remove specific string from multiple files

grep approach

To create a copy of the file without lines matching "cat" or "rat", one can use grep in reverse (-v) and with the whole-word option (-w).

grep -vwE "(cat|rat)" sourcefile > destinationfile

The whole-word option makes sure it won't match cats or grateful for example. Output redirection of your shell is used (>) to write it to a new file. We need the -E option to enable the extended regular expressions for the (one|other) syntax.

sed approach

Alternatively, to remove the lines in-place one can use sed -i:

sed -i "/\b\(cat\|rat\)\b/d" filename

The \b sets word boundaries and the d operation deletes the line matching the expression between the forward slashes. cat and rat are both being matched by the (one|other) syntax we apparently need to escape with backslashes.

Tip: use sed without the -i operator to test the output of the command before overwriting the file.

(Based on Sed - Delete a line containing a specific string)

Here are a few approaches. I am using brace expansion (file{1..4}.txt) which means file1.txt file2.txt file3.txt file4.txt

1. Perl

   perl -i -pe 's/.*/ Good Morning / if $.==5' file{1..4}.txt
   

   Explanation:

   • -i: causes perl to edit the files in place, changing the original file.

     If -i is followed with a file extension suffix, then a backup is created for every file that is modified. Ex: -i.bak creates a file1.txt.bak if file1.txt is modified during the execution.

   • -p: means read the input file line by line, apply the script and print it.

   • -e: allows you to pass a script from the command line.

   • s/.*/ Good Morning /: That will replace the text in the current line (.*) with Good Morning.

   • $. is a special Perl variable that holds the current line number of the input file. So, s/foo/bar/ if $.==5, means replace foo with bar only on the 5th line.

2. sed

   sed -i '5s/.*/ Good Morning /' file{1..4}.txt
   

   Explanation:

   • -i: Like for perl, edit file in place.

   By default, sed prints each line of the input file. The 5s/pattern/replacement/ means substitute pattern with replacement on the 5th line.

3. Awk

   for f in file{1..4}.txt; do 
       awk 'NR==5{$0=" Good Morning "}1;' "$f" > foobar && mv foobar "$f"; 
   done
   

   Explanation:

   awk has no equivalent to the -i option¹ which means that we need to create a temporary file (foobar) which is then renamed to overwrite the original. The bash loop for f in file{1..4}.txt; do ... ; done simply goes through each of file{1..4}.txt, saving the current file name as $f. In awk, NR is the current line number and $0 is the content of the current line. So, the script will replace the line ($0) with " Good Morning " only on the 5th line. 1; is awk for "print the line".

   ¹_{Newer versions do as devnull showed in his answer.}

4. coreutils

   for f in file{1..4}.txt; do 
       (head -4 "$f"; echo " Good Morning "; tail -n +6 "$f") > foobar && 
       mv foobar "$f"; 
   done 
   

   Explanation:

   The loop is explained in the previous section.

   • head -4: print the first 4 lines

   • echo " Good Morning ": print " Good Morning "

   • tail -n +6: print everything from the 6th line to the end of the file

   The parentheses ( ) around those three commands allow you to capture the output of all three (so, 1st 4 lines, then " Good morning ", then the rest of the lines) and redirect them to a file.