Sed Multi-line Replace – How to Perform Multi-line Replace

sed

I spent ages trying to figure this out. As it happens, I don't need to do it any more; I found another way. But for the sake of my sanity, I'd like to know the answer anyway.

It's quite easy to use sed to replace a marker with some piece of text. What I can't figure out is how to make sed insert a huge block of text, with complex formatting such as multiple lines, tab characters, etc. How do you do that? Is sed the best thing to use or what?

Best Answer

If you have the text to be inserted at the marker(s) in a file named /tmp/insert.txt you can accomplish this task as follow:

sed '/MARKER/ r /tmp/insert.txt' < inputfile

The above sed command will read "inputfile" and look for MARKER. When it finds it, it inserts the contents of /tmp/insert.txt into the output stream.

If you want the marker itself to be deleted, try this:

sed '/MARKER/ {s/MARKER//; r /tmp/insert.txt
}' <inputfile

Note the "}" closing brace must be preceded by a new line.

As in the first command, the sed will operate on lines with the MARKER. It will substitute MARKER with nothing, and then it will read in /tmp/insert.txt

Related Solutions

Shell – sed: multi-line replace of config block

If you want to use sed, you can read from a named pipe. Beware that this code doesn't try to cope with errors. The script will block if the dynamic block header is present more than once.

CONTENT_URL="http://$SERVER/get_config.php"
tmp=$(mktemp -d)
(
  cd "$tmp"
  mkfifo dynamic_seen dynamic_content
  : >dynamic_seen & seen_pid=$!
  wget -O dynamic_content "$CONTENT_URL" & wget_pid=$!
  sed -e '/^# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY$/ p' \
      -e '/^# END DYNAMIC BLOCK$/ {'
          -e p -e 'r dynamic_seen' -e 'r dynamic_content' -e '}' \
      -e '/^# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY$/, /^# END DYNAMIC BLOCK$/ d'
  if ! kill $dynamic_seen 2>/dev/null; then
    # The pipe hasn't been read, so there was no dynamic block. Add one.
    echo "# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY"
    cat dynamic_pipe
    echo "# END DYNAMIC BLOCK - DO NOT EDIT MANUALLY"
  fi
)
rm -rf "$tmp"

But I'd go for awk.

export CONTENT_URL="http://$SERVER/get_config.php"
awk '
    $0 == "# END DYNAMIC BLOCK - DO NOT EDIT MANUALLY" {skip=0; system("wget \"$CONTENT_URL\""); substituted=1}
    !skip {print}
    $0 == "# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY" {skip=1}
    END {
         if (!substituted) {
            print "# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY";
            system("wget \"$CONTENT_URL\"");
            print "# END DYNAMIC BLOCK - DO NOT EDIT MANUALLY";
        }
    }
'

Bash – Sed? Stripping all formatting, i.e line breaks and whitespaces, from a report text file, while masking out certain pieces

I am not quite sure what you're trying to do. Using your first input file, I create this output:

LUNName=mysql05-dbdat02|CollectionStartTime=2012-09-20T15:43:03-04:00|CollectionEndTime=2012-09-20T15:43:34-04:00|Capacity|CurrentCapacity=512|IOOperations|Reads=100|Writes=0|ReadsPerSecond=0.000000|WritesPerSecond=0.000000|ReadMBPerSecond=0.000|WriteMBPerSecond=0.000|TotalMBPerSecond=0.000|NonOptimizedIOPerSecond=0.000000|CacheHitPercentage=0.000|PerformanceMetrics|TotalIOsPerSecond=0.000|ReadIOsPerSecond=0.000|WriteIOsPerSecond=0.000|TotalMBPerSecond=0.000|ReadMBPerSecond=0.000|WriteMBPerSecond=0.000|Performance|

With this perl one liner:

perl -pe 's/\n/|/;s/\s*//g; s/:/=/; END{print "\n"}' file

You could also do it with this:

sed -r 's/\s*//g; s/:/=/;' file | tr '\n' '|'

Best Answer

Related Solutions

Shell – sed: multi-line replace of config block

Bash – Sed? Stripping all formatting, i.e line breaks and whitespaces, from a report text file, while masking out certain pieces

Related Question