Replace a marker in a text file with arbitrary new text

awksedtext processing

I have a file which contains some text and – on a single line – a marker which indicates where new content should be added

foo
bar
%SUBSTITUTE%
foo

The line line substitute should be replaced by a new multiline string text="some text" (note that I do not know the string it may for instance be the result of reading a file text=cat "file"“). The result of the replacement should read

foo
bar
some text
%SUBSTITUTE%
foo

I had a working version based on perl which stopped working (apparently due to perl version change). Now I am trying standard utilities like tr and sed to replace the line.
I am having some trouble with it, because the string to be pasted may contain arbitrary characters including backslashes etc. I do not want to escape these before pasting.

Is there a safe way to do it which works with standard tools? The other questions I find for the problem are particular solutions where the text to be pasted is known.

Best Answer

If you have the GNU version of sed, you should be able to use the r command to read and insert new content from a file, then delete the marker line e.g.

sed '/%SUBSTITUTE%/{
r path/to/newcontent
d
}' file

If you want to retain the %SUBSTITUTE% marker after the insertion, that's tricky because whatever you do, the GNU r extension queues the file contents until the end of the current pattern cycle (retaining it before would just be a matter of removing the d command). Probably the simplest way is to append it to the newcontent file: you could do that on the fly like

sed '/%SUBSTITUTE%/{
r /dev/stdin
d
}' file < <(sed '$a %SUBSTITUTE%' path/to/newcontent)

Taking a completely different approach, you could split the first file on %SUBSTITUTE% and then cat in the new content

csplit -s file '/%SUBSTITUTE%/'
cat xx00 newcontent xx01

You could also do a bash read loop over the lines of the first file, and cat the new content file when you match the marker string - however I've had my knuckles rapped for suggesting read loops for text processing on this forum. Unfortunately neither provides an in-place solution.

Related Solutions

Sed / awk / perl / etc: Move lines backward in file

awk:

awk '
    /__DELETE_THIS_LINE_BEGIN__/ {keep=1; next} 
    /__DELETE_THIS_LINE_END__/   {keep=0; move=4; next}
    keep {saved[++s]=$0; next} 
    move-- == 0 {for (i=1; i<=s; i++) print saved[i]; delete saved; s=0}
    1
    END {for (i=1; i<=s; i++) print saved[i]}
' file

01 text text text
02 text text text
09 four
10 interesting
11 lines
12 follow
04 text text text
05 text text text
06 text text text
07 text text text
13 text text text
14 text text text
15 text text text
16 text text text
23 even
24 more
25 interesting
26 lines
18 text text text
19 text text text
20 text text text
21 text text text

Also, with awk, you can redefine the record separator:

awk -v RS='\n[0-9]+ __DELETE_THIS_LINE_(BEGIN|END)__\n' '
    NR%2 == 0 {saved=$0; next} 
    {
        n=split($0, lines, "\n")
        for (i=1; i<=4 && i<=n; i++) print lines[i]
        if (saved) print saved
        for (i=5; i<=n; i++) print lines[i]
    }
' file

produces the same result.

Replace “o” with “0” in large wordlist and save the original word

You can use sed:

sed -i -e 'p' -e 's/o/0/g' file

Explanation:

-i: activates in-place editing of the file
-e 'p': just pastes the line
-e 's/o/0/g': replaces o with 0 and pastes the altered line

And if you want an awk solution:

awk '1;gsub("o", "0")' file >new_file

Best Answer

Related Solutions

Sed / awk / perl / etc: Move lines backward in file

Replace “o” with “0” in large wordlist and save the original word

Related Question