sed – Nexenta Bash Script Uses /usr/sun/bin/sed Instead of /usr/bin/sed

nexentasedsolaris

I'm using a script for Linux and for Solaris (Nexenta).

This line works on Linux, but not on solaris (but when run from shell it works):

cat "pg_hba.conf" | sed "0,/^local/{s/md5/trust/}"

The error message is:

sed: command garbled: 0,/^local/{s/md5/trust/}

After some research, I found out that the sed that bash uses in the script is different.

from shell: /usr/bin/sed

from script: /usr/sun/bin/sed

I want to make the script use /usr/bin/sed.

What I tried to do:

call sed with full path. Same results. It seems that it is still uses the other sed…
tried to call it via bash -l. Same results.
tried to declare a different command: S=/usr/lib/sed and use $S instead. Same results.
checked PATH – both cmd and script have /usr/bin in it.
Tried replacing the double quotes to single. Same results.
tried running the sed with -r flag. Output is :

# /usr/xpg4/bin/sed -r /usr/xpg4/bin/sed: illegal option -- r Usage: sed [-n] script [file...] sed [-n] [-e script]...[-f script_file]...[file...]

HELP??

What I need to do is to replace the first match of "md5" with "trust" on the first line that starts with "local"). I know I can do it otherwise but I the sed issue is itching me too much!

EDIT:

I hope this makes a little order…

PATH from login shell = /usr/local/ctera/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/ctera/apache-ant-1.8.2/bin
PATH from script =
/usr/sbin:/usr/bin:/usr/local/ctera/apache-ant-1.8.2/bin:/usr/local/ctera/apache-ant-1.8.2/bin
PATH from script when PATH=$(command -p getconf PATH):$PATH is called =
/usr/xpg4/bin:/usr/ccs/bin:/usr/bin:/opt/SUNWspro/bin:/usr/xpg4/bin:/usr/ccs/bin:/usr/bin:/opt/SUNWspro/bin:/usr/sbin:/usr/bin:/usr/local/ctera/apache-ant-1.8.2/bin:/usr/local/ctera/apache-ant-1.8.2/bin
truss -f sed from login shell calls /usr/bin/sed
truss -f sed from script calls /usr/sun/bin/sed
truss -f /usr/bin/sed from script calls /usr/sun/bin/sed !!!
After setting PATH=$(command -p getconf PATH):$PATH:
7.1 truss -f sed from script calls /usr/xpg4/bin/sed
7.2 truss -f /usr/bin/sed from script calls /usr/sun/bin/sed !!!

MORE INFO:

Commands output: (run both from shell prompt and from within the script)

truss -ft execve /usr/bin/sed q
as shell command:
8604: execve("/usr/bin/sed", 0x08047D20, 0x08047D2C) argc = 2
from script:
8545: execve("/usr/sun/bin/sed", 0x08047768, 0x08047774) argc = 2
file /usr/bin/sed
as shell command:
/usr/bin/sed: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
from script:
file: /usr/bin/sed zero size or zero entry ELF section - ELF capabilities ignored file: /usr/bin/sed: can't read ELF header /usr/bin/sed: data
ls -l /usr/bin/sed
as shell command:
-rwxr-xr-x 1 root root 96440 May 31 2008 /usr/bin/sed
from script:
-rwxr-xr-x 1 root root 96440 May 31 2008 /usr/bin/sed
ls -ld $(type -pa sed)
as shell command:
-rwxr-xr-x 1 root root 96440 May 31 2008 /bin/sed -rwxr-xr-x 1 root root 96440 May 31 2008 /usr/bin/sed
from script:
-rwxr-xr-x 1 root root 96440 May 31 2008 /usr/bin/sed

md5sum $(type -pa sed)
as shell command:
385361c5111226c8eac8e25b53fed29c /bin/sed 385361c5111226c8eac8e25b53fed29c /usr/bin/sed
from script:
385361c5111226c8eac8e25b53fed29c /usr/bin/sed

The script is invoced by JAVA code.
uname -a
SunOS cteraportal 5.11 NexentaOS_134f i86pc i386 i86pc Solaris

This might add info about the sed version on my machine

~# ll `find / -name sed`  
-rwxr-xr-x 1 root root 96440 May 31  2008 /usr/bin/sed  
-r-xr-xr-x 1 root bin  35656 Sep  7  2010 /usr/sun/bin/sed  
-r-xr-xr-x 1 root bin  32104 Sep  7  2010 /usr/ucb/sed  
-r-xr-xr-x 1 root bin  35636 Sep  7  2010 /usr/xpg4/bin/sed  

/usr/share/doc/sed:  
total 113  
-rw-r--r-- 1 root root   168 Jun 21  2005 AUTHORS.gz  
-rw-r--r-- 1 root root  2507 Jun 21  2005 BUGS.gz  
-rw-r--r-- 1 root root  6584 Feb  3  2006 NEWS.gz  
-rw-r--r-- 1 root root   285 Jun 21  2005 README.gz  
-rw-r--r-- 1 root root  1071 Jan 12  2006 THANKS.gz  
-rw-r--r-- 1 root root  4806 May 31  2008 changelog.Debian.gz  
-rw-r--r-- 1 root root 32312 Feb  3  2006 changelog.gz  
-rw-r--r-- 1 root root   796 May 31  2008 copyright  
drwxr-xr-x 2 root root     3 May 30  2011 examples  
drwxr-xr-x 2 root root     3 May 30  2011 sed-4.1.5  
-rw-r--r-- 1 root root 56538 May 31  2008 sedfaq.txt.gz

Best Answer

OK, found it. That now makes sense.

the behaviour is Nexenta-specific and explained at http://lwn.net/Articles/334756/

GNU and not GNU

The default behavior of Nexenta is to prefer GNU utilities, which are installed in /usr/bin and /usr/sbin and so on. The Sun versions of these utilities are installed in /usr/sun/bin and /usr/sun/sbin. Nexenta uses a trick to be able to switch between a GNU and a SUN personality: if the environment variable SUN_PERSONALITY is set to one, the search paths /usr/sun/bin and /usr/sun/sbin take preference, even if the user executes the commands explicitly by their absolute path, e.g. /usr/bin/sed. This ensures that Solaris-based scripts work in Nexenta without modifications. Nexenta also uses this functionality in its SVR4 package commands. They can be used to install native Solaris packages in SVR4 format, calling alien to convert the package on-the-fly to a Debian package.

That's done somewhere in the libc.

So,

$ sed --version
GNU sed version 4.1.5
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
$ SUN_PERSONALITY=1 sed --version
sed: illegal option -- version

So, your script started from Java, must have the SUN_PERSONALITY set.

You can unset that in your script if you want the GNU tools.

Related Solutions

The way to use `/usr/bin/env sed -f ` in shebang

You can't, portably, put more than one argument on a #! line. That means only a full path and one argument (e.g. #!/bin/sed -f or #!/usr/bin/sed -f), or #!/usr/bin/env and no argument to the interpreter.

A workaround to get a portable script is to use #!/bin/sh and a shell wrapper, passing the sed script as a command-line argument. Note that this is not sanctioned by POSIX (multi-instruction scripts must be written with a separate -e argument for each instruction for portability), but it works with many implementations.

#!/bin/sh
exec sed '
s/a/b/
' "$@"

For a long script, it may be more convenient to use a heredoc. An advantage of a heredoc is that you don't need to quote the single quotes inside, if any. A major downside is that the script is fed to sed on its standard input, with two annoying consequences. Some versions of sed require -f /dev/stdin instead of -f -, which is a problem for portability. Worse, the script can't act as a filter, because the standard input is the script and can't be the data.

#!/bin/sh
exec sed -f - -- "$@" <<'EOF'
s/a/b/
EOF

The downside of the heredoc can be remedied by a useful use of cat. Since this puts the whole script on the command line again, it's not POSIX-compliant, but largely portable in practice.

#!/bin/sh
exec sed "$(cat <<'EOF')" -- "$@"
s/a/b/
EOF

Another workaround is to write a script that can be parsed both by sh and by sed. This is portable, reasonably efficient, just a little ugly.

#! /bin/sh
b ()
{
x
}
i\
f true; then exec sed -f "$0" "$@"; fi
: ()
# sed script starts here
s/a/b/

Explanations:

Under sh: define a function called b; the contents don't matter as long as the function is syntactically well-formed (in particular, you can't have an empty function). Then if true (i.e. always), execute sed on the script.
Under sed: branch to the () label, then some well-formed input. Then an i command, which has no effect because it's always skipped. Finally the () label followed by the useful part of the script.
Tested under GNU sed, BusyBox and OpenBSD. (You can get away with something simpler on GNU sed, but OpenBSD sed is picky about the parts it skips.)

Bash Script – Why ‘myscript.sh’ Runs in Bash with ‘#!/usr/bin/sh’

The first line of the script is called the #! line (sometimes called the "shebang" line).

It is only used when a script is executed directly, as in

./test.sh

Explanation

When you run an executable, the operating system looks at the first two bytes to tell what type of program it is.

When you run

./test.sh

the operating system sees that the first two bytes of ./test.sh are #!, so it knows the program is a script.

It reads the rest of the line to see which shell¹ it should use to run the script. In this case, the rest of the line is /usr/bin/sh, so it runs /usr/bin/sh test.sh².

When you run

bash test.sh

it finds bash in /usr/bin/bash³. It sees that the first two bytes of /usr/bin/bash are not #!, so it runs /usr/bin/bash test.sh without even looking at the first line of test.sh.

In both cases, sh and bash actually ignore the shebang line because any line beginning with # is a comment. The # only has this magic effect because of the operating system, not the shell.

More info:

Footnotes:

In fact, it can be any program, it doesn't have to be a shell.
Actually more like execl("./test.sh", "/usr/bin/sh", "./test.sh", NULL).
/bin/sh on most Linux systems, but the question was about Solaris.