While:
sed "0,\~$var~s~$var~replacement~"
Can be used to change the regex delimiter, embedding variable expansions inside sed
(or any other interpreter) code is a very unwise thing to do in the general case.
First, here, the delimiter is not the only character that needs to be escaped. All the regular expression operators need to as well.
But more importantly, and especially with GNU sed
, that's a command injection vulnerability. If the content of $var
is not under your control, it's just as bad as passing arbitrary data to eval
.
Try for instance:
$ var='^~s/.*/uname/e;#'
$ echo | sed "0,\~$var~s~$var~replacement~"
Linux
The uname
command was run, thankfully a harmless one... this time.
Non-GNU sed
implementations can't run arbitrary commands, but can overwrite any file (with the w
command), which is virtually as bad.
A more correct way is to escape the problematic characters in $var
first:
NL='
'
case $var in
(*"$NL"*)
echo >&2 "Sorry, can't handle variables with newline characters"
exit 1
esac
escaped_var=$(printf '%s\n' "$var" | sed 's:[][\/.^$*]:\\&:g')
# and then:
sed "0,/$escaped_var/s/$escaped_var/replacement/" < file
Another approach is to use perl
:
var=$var perl -pe 's/\Q$ENV{var}\E/replacement/g && $n++ unless $n' < file
Note that we're not expanding the content of $var
inside the code passed to perl
(which would be another command injection vulnerability), but are letting perl
expand its content as part of its regexp processing (and within \Q...\E
which means regexp operators are not treated specially).
If $var
contains newline characters, that may only match if there's only one at the end. Alternatively, one may pass the -0777
option so the input be processed as a single record instead of line-by-line.
sed -r 'H;$!d;x;s:\n::g;:l;s:(\\x..)(.*)\1:\2:;tl' allHexChars.txt allowedChars.txt > missingChars.txt
The above GNU sed script assumes two things as I understood them from the question:
- inside the files no hex character is listed more than one time
- the first file contains all the hex characters from the second file
To visualize the differences, use:
diff -y <(fold -4 allHexChars.txt) <(fold -4 allowedChars.txt)
Best Answer
First thing: don't write to a file you're reading from, you're likely to end up erasing it. Use the inplace-edit feature (
-i
) instead.Next, inside hard quotes (
'
), variable substitution doesn't operate, so you'll need to do something else to get the expanded$PATH
substituted in there.Lastly,
$PATH
is very likely to contain/
characters, which would break the substitution syntax forsed
, so you'll need to use another separator.Assuming your paths never contain
;
s, try (after having backed-up your current file of course):The match part of the substitution means: match a string that starts with (
^
) the stringPATH=
and followed by any character (.
) any number of times (*
). I.e. it will match lines that start withPATH=
completely, but not lines that havePATH=
somewhere in the middle. The.*
is important because you want to replace the whole line (try without it to see what happens).The quoting to get
$PATH
substituted inside the replacement is necessary to account for cases where$PATH
would contain whitespace characters.Demo:
Env. vars are not substituted inside hard quotes. So you get a literal
$foo
there. Now let the shell expand it by bringing it out of the hard quotes:Ok! But this isn't safe:
sed
received two arguments after-e
above:s/hello/bar
andbaz/
, separately. Not good. To fix that, you need a bit more quotes:Putting an environment variable inside
"
quotes is something you should pretty much always do when you're passing it as an argument to something else, otherwise (assuming$IFS
hasn't been changed) it will get split on whitespace.