How to instruct BSD sed to interpret escape sequences like \n and \t

bsdsed

I have a sed replacement command that I would like to be compatible with BSD sed as well as GNU sed. Extended regular expressions are not an issue as I do not need them in this case. My primary problem is difference in the way that the two seds interpret character escape sequences in the replacement strings. My replacement string contains tabs and newlines and I'd like them to be visible in the command strings for ease of maintenance, however, BSD sed doesn't interpret the escape sequences and GNU sed does. What is the appropriate way to instruct sed to interpret these escape sequences on BSD? The following two snippets epitomize my problem:

GNU sed

echo ABC | sed 's/B/\n\tB\n'

yeilds

A
    B
C

BSD sed

echo ABC | sed 's/B\n\tB\n'

yields

AntBnC

Clearly, \n and \t aren't interpreted as escape sequences by BSD sed

Now, to my question. According the BSD sed manpage:

To specify a newline character in the replacement string, precede it
with a backslash.

Does this imply that I'd need to precede a literal newline by a backslash? What is the appropriate way to instruct sed to interpret escape sequences like \n in the replacement text?

Best Answer

If you need to write portable scripts, you should stick to features in the POSIX standard (a.k.a. Single Unix a.k.a Open Group Base Specification). Issue 7 a.k.a. POSIX-1.2008 is the latest, but many systems haven't finished adopting it yet. Issue 6 a.k.a POSIX-1.2001 is by and large provided by all modern unices.

In sed, the meaning of escape sequences like \t and \n is not portable, except that in a regex, \n stands for a newline. In the replacement text for an s command, \n is not portable, but you can use the sequence backslash-newline to stand for a newline.

A portable way to generate a tab character (or any other character expressed in octal) is with tr. Store the character in a shell variable and substitute this variable in the sed snippet.

tab=$(echo | tr '\n' '\t')
escape=$(echo | tr '\n' '\033')
embolden () {
  sed -e 's/^/'"$escape"'[1m/' -e 's/$/'"$escape"'[0m/'
}

Note again that newlines need to be expressed differently in regexes and in s replacement texts.

You might want to use awk instead. It allows backslash escapes, including octal escapes \ooo, in every string literal.

Related Solutions

BSD sed vs. GNU sed, and -i

Macs use the BSD version of utilities such as sed and date, which have their own idiosyncrasies.

In this specific case, the BSD build of sed mandates the extension for the backup file with -i, rather than it being optional, as in GNU sed.

As such:

sed -i .bak 's/needle/pin/g' haystack

The shown command will replace all instances of needle with pin in the file haystack, and the original file will be preserved in haystack.bak.

From the manual for the implementation of sed on a Mac:

-i extension
         Edit files in-place, saving backups with the specified extension.  If a zero-length extension is given, no backup will be saved.
         It is not recommended to give a zero-length extension when in-place editing files, as you risk corruption or partial content in
         situations where disk space is exhausted, etc.

As opposed to on a Linux host:

  -i[SUFFIX], --in-place[=SUFFIX]

          edit files in place (makes backup if SUFFIX supplied)

Note that "a zero-length extension" is distinct from "no extension". You can eschew the backup entirely, then, with:

sed -i '' 's/needle/pin/g' haystack

sed – Extra Backslash Required in First Appended Newline

It's not exactly documented behaviour, but then the entire thing is an extension. All three commands that were extended similarly (a, i and c) behave the same way: leading whitespace is ignored, and if the text begins with \, then it is discarded. Essentially, it doesn't matter to sed which of these forms are used:

i foo
i \foo
i \
foo

They're all identical. (I know I used i here, but check the the descriptions of a, i and c - it's surreal.)

And the code shows as much:

1111         case 'a':
1112         case 'i':
1113         case 'c':
1114           ch = in_nonblank ();
1115 
1116         read_text_to_slash:
1117           if (ch == EOF)
1118             bad_prog (_(EXPECTED_SLASH));
1119 
1120           if (ch == '\\')
1121             ch = inchar ();

Best Answer

Related Solutions

BSD sed vs. GNU sed, and -i

sed – Extra Backslash Required in First Appended Newline

Related Question