sed Command – Why Does sed s/// Remove a Particular String?

sed

I have two files: junk.txt and lines.txt

$>cat junk.txt
This is a junk text
$>cat lines.txt
Line1
Line2
Line3
Line4

When I fire the following sed script

 sed -i -e  "/Line3/r junk.txt" -e "s///" lines.txt

I get this output:

Line1
Line2

This is a junk text
Line4

As you can see, Line3 is no more there. Ok, the /Line3/r junk.txt part finds the text in the lines.txt matching Line3 and appends the contents of the junk.txt. But why does the s/// command remove exactly the matched line?

To be honest, that's the behavior I need but I want to know why it behaves like that.

Best Answer

When the editing command s is invoked with an empty regular expression, it reuses the expression most recently used. In this case, it reuses Line3 from the matching of it earlier. This applies to // (empty regular expressions) in both the s command and in address ranges.

Your command could also be written, without this feature, as

sed -i -e '/Line3/r junk.txt' -e 's/Line3//' lines.txt

The Line3 text is removed because it is replaced by nothing.

You would sometimes see this in constructs like

/expression/ {
    s//replacement/
    # other code here
}

The POSIX specification for sed formulates this as

If an RE is empty (that is, no pattern is specified) sed shall behave as if the last RE used in the last command applied (either as an address or as part of a substitute command) was specified.

Related Solutions

Replace or append a line with string within a file

You can do this with sed — it's Turing-complete. But it isn't the best tool for the job. Sed doesn't have a convenient way of remembering that it's already made a replacement.

What you can relatively easily do with sed is to blank all the lines starting /swapfile, and add a new one at the end:

sed -i '$! s/^\/swapfile[\t ]//; $s/\(^\/swapfile.*\)\?$/\n\/swapfile none swap sw/' /etc/fstab

but beyond that we're quickly getting into territory where I wouldn't leave such sed code for another sysadmin to maintain, especially when a simple, readable combination of shell commands would do a better job:

{ </etc/fstab grep -v '/swapfile[\t ]'; echo '/swapfile none swap sw'; } >/etc/fstab.new && mv /etc/fstab.new /etc/fstab

If you want to preserve the existing position of the /swapfile line if it's there and only modify the file if it needs modifying, a combination of shell logic and awk is a better tool. I've used multiple lines here for clarity but you can put all the code on the same line if you like. As a bonus, if the file already contained the intended line (with exact spacing), it won't be modified.

awk '
  /\/swapfile[\t ]/ {if (replaced) {next}
                     else {replaced=1; print "/swapfile none swap sw"}}
  1 {print}
  END {if (!replaced) print "/swapfile none swap sw"}
' /etc/fstab >/etc/fstab.new &&
if cmp -s /etc/fstab.new /etc/fstab.new; then
  rm /etc/fstab.new;
else
  mv /etc/fstab.new /etc/fstab;
fi

sed – Extra Backslash Required in First Appended Newline

It's not exactly documented behaviour, but then the entire thing is an extension. All three commands that were extended similarly (a, i and c) behave the same way: leading whitespace is ignored, and if the text begins with \, then it is discarded. Essentially, it doesn't matter to sed which of these forms are used:

i foo
i \foo
i \
foo

They're all identical. (I know I used i here, but check the the descriptions of a, i and c - it's surreal.)

And the code shows as much:

1111         case 'a':
1112         case 'i':
1113         case 'c':
1114           ch = in_nonblank ();
1115 
1116         read_text_to_slash:
1117           if (ch == EOF)
1118             bad_prog (_(EXPECTED_SLASH));
1119 
1120           if (ch == '\\')
1121             ch = inchar ();

Best Answer

Related Solutions

Replace or append a line with string within a file

sed – Extra Backslash Required in First Appended Newline

Related Question