Linux – How to do multiline search and replace text (delete) recursively

linuxreplacesearchtext processing

I want to go recursively through a directory and search for specific content in all files. The content should then be replaced if found. With a single line I would make something like this:

find . -type f -exec sed -i -e 's@code2replace@@g' {} \;

It sometimes worked and sometimes not (not sufficient hard drive space …). Now I would need something for a multine replacement. The orignal file should be overwritten and the multiline pattern could be rather long. So perhaps storing in an external file would be easier I think.

The code which should be replaced looks like the following:

#c3284d#
echo(gzinflate(base64_decode("sylOLsos....")));
#/c3284d#

Also which characters do I have to escape?

Best Answer

Sed can do a lot of things beside simple search-and-replace, and that includes multiline support.

Here's a blog post on how someone did this (he wrote a sedml script for "sed multiline": http://austinmatzko.com/2008/04/26/sed-multi-line-search-and-replace/

The basic idea is to copy the whole file to sed's "hold buffer", run the regex on that and then write out the changed file. However, this approach may be rather slow if your files are large as you're loading a whole file at a time into memory.

Related Solutions

Replace a marker in a text file with arbitrary new text

If you have the GNU version of sed, you should be able to use the r command to read and insert new content from a file, then delete the marker line e.g.

sed '/%SUBSTITUTE%/{
r path/to/newcontent
d
}' file

If you want to retain the %SUBSTITUTE% marker after the insertion, that's tricky because whatever you do, the GNU r extension queues the file contents until the end of the current pattern cycle (retaining it before would just be a matter of removing the d command). Probably the simplest way is to append it to the newcontent file: you could do that on the fly like

sed '/%SUBSTITUTE%/{
r /dev/stdin
d
}' file < <(sed '$a %SUBSTITUTE%' path/to/newcontent)

Taking a completely different approach, you could split the first file on %SUBSTITUTE% and then cat in the new content

csplit -s file '/%SUBSTITUTE%/'
cat xx00 newcontent xx01

You could also do a bash read loop over the lines of the first file, and cat the new content file when you match the marker string - however I've had my knuckles rapped for suggesting read loops for text processing on this forum. Unfortunately neither provides an in-place solution.

Text Processing – Search and Replace Using SED

This is basic task for sed command:

sed 's/AA\(someArbitraryString\)BB/CC\1DD/g'

Eventually if you want to do this for all "arbitrary strings":

sed 's/AA\(.*\)BB/CC\1DD/g'

Best Answer

Related Solutions

Replace a marker in a text file with arbitrary new text

Text Processing – Search and Replace Using SED

Related Question