This uses the extended regex syntax -r
, which clears up a lot of the clutter. Also, because you already know some of the field values, you don't actually need to back-reference them, again reducing clutter (and overhead).
&
is a special replacement value: it hold the entire matched pattern. Using the &
, again reduces clutter. As it is not a back-reference, it has significantly less overhead.
I've used ( +)
vs. ( *)
. The +
assumes that there is at least one space between input fields. Just change it to the *
it that is not the case.
EXPL=
dom=oracle
typ=hard
itm=nproc
val=666
echo "oracle hard nproc 131072" |
sed -r "s/^$dom( +)$typ( +)$itm( +).*/$EXPL#&\n$dom\1$typ\2$itm\3$val/"
output
#oracle hard nproc 131072
oracle hard nproc 666
That was raised on the Austin group mailing list in March 2012. Here's the final message on that (by Geoff Clare of the Austin Group (the body that maintains POSIX), who is also the one who raised the issue in the first place). Here copied from the gmane NNTP interface:
Date: Fri, 16 Mar 2012 17:09:42 +0000
From: Geoff Clare <gwc-7882/jkIBncuagvECLh61g@public.gmane.org>
To: austin-group-l-7882/jkIBncuagvECLh61g@public.gmane.org
Newsgroups: gmane.comp.standards.posix.austin.general
Subject: Re: Strange addressing issue in sed
Stephane Chazelas <stephane_chazelas-Qt13gs6zZMY@public.gmane.org> wrote, on 16 Mar 2012:
>
> 2012-03-16 15:44:35 +0000, Geoff Clare:
> > I've been alerted to an odd behaviour of sed on certified UNIX
> > systems that doesn't seem to match the requirements of the
> > standard. It concerns an interaction between the 'n' command
> > and address matching.
> >
> > According to the standard, this command:
> >
> > printf 'A\nB\nC\nD\n' | sed '1,3s/A/B/;1,3n;1,3s/B/C/'
> >
> > should produce the output:
> >
> > B
> > C
> > C
> > D
> >
> > GNU sed does produce this, but certified UNIX systems produce this:
> >
> > B
> > B
> > C
> > D
> >
> > However, if I change the 1,3s/B/C/ to 2,3s/B/C/ then they produce
> > the expected output (tested on Solaris and HP-UX).
> >
> > Is this just an obscure bug from common ancestor code, or is there
> > some legitimate reason why this address change alters the behaviour?
> [...]
>
> I suppose the idea is that for the second 1,3cmd, line "1" has
> not been seen, so the 1,3 range is not entered.
Ah yes, now it makes sense, and it looks like the standard does
require this slightly strange behaviour, given how the processing
of the "two addresses" case is specified:
An editing command with two addresses shall select the inclusive
range from the first pattern space that matches the first address
through the next pattern space that matches the second. (If the
second address is a number less than or equal to the line number
first selected, only one line shall be selected.) Starting at the
first line following the selected range, sed shall look again for
the first address. Thereafter, the process shall be repeated.
It's specified this way because the addresses can be BREs, but if
the same matching process is applied to the line numbers (even though
they can only match at most once), then the 1,3 range on that last
command is never entered.
--
Geoff Clare <g.clare-7882/jkIBncuagvECLh61g@public.gmane.org>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
And here's the relevant part of the rest of the message (by me) that Geoff was quoting:
I suppose the idea is that for the second 1,3cmd, line "1" has
not been seen, so the 1,3 range is not entered.
Same idea as in
printf '%s\n' A B C | sed -n '1d;1,2p'
whose behavior differ in traditional (heirloom toolchest at
least) and GNU.
It's unclear to me whether POSIX wants one behavior or the
other.
So, (according to Geoff) POSIX is clear that the GNU behaviour is non-compliant.
And it's true it's less consistent (compare seq 10 | sed -n '1d;1,2p'
with seq 10 | sed -n '1d;/^1$/,2p'
) even if potentially less surprising to people who don't realise how ranges are processed (even Geoff initially found the conforming behaviour "strange").
Nobody bothered reporting it as a bug to the GNU folks. I'm not sure I'd qualify it as a bug. Probably the best option would be for the POSIX specification to be updated to allow both behaviours to make it clear that one cannot rely on either.
Edit. I've now had a look at the original sed
implementation in Unix V7 from the late 70s, and it looks pretty much like that behaviour for numeric addresses was not intended or at least not thought through completely there.
With Geoff's reading of the spec (and my original interpretation of why it happens), conversely, in:
seq 5 | sed -n '3d;1,3p'
lines 1, 2, 4 and 5 should be output, because this time, it's the end address that is never encountered by the 1,3p
ranged command, like in seq 5 | sed -n '3d;/1/,/3/p'
Yet, that doesn't happen in the original implementation, nor any other implementation I tried (busybox sed
returns lines 1, 2 and 4 which looks more like a bug).
If you look at the UNIX v7 code, it does check for the case where the current line number is greater than the (numerical) end address, and gets out of the range then. The fact that it doesn't do it for the start address looks more like an oversight then than an intentional design.
What that means is that there's no implementation that is actually compliant to that interpretation of the POSIX spec in that regard at the moment.
Another confusing behaviour with the GNU implementation is:
$ seq 5 | sed -n '2d;2,/3/p'
3
4
5
Since line 2 was skipped, the 2,/3/
is entered upon line 3 (the first line whose number is >= 2). But as it's the line that made us enter the range, it's not checked for the end address. It gets worse with busybox sed
in:
$ seq 10 | busybox sed -n '2,7d; 2,3p'
8
Since lines 2 to 7 were deleted, line 8 is the first one that is >= 2 so the 2,3 range is entered then!
Best Answer
You can use branches:
(note that you can also add some
20,25b1
line ranges, or/re/b1
to include lines that match there
).Or you could use
awk
:Or using a hash:
(or
BEGIN{lines[1]lines[7]lines[14]lines[16]}
if there aren't too many)