Sed – Match Special Color Characters

regular expressionsedshell

I have a file containing special color encoding characters:

$ cat zz
aaa.gpg
bbb.gpg
ccc.gpg

$ cat -A zz
^[[38;5;216maaa.gpg^[[00m$
^[[38;5;216mbbb.gpg^[[00m$
^[[38;5;216mccc.gpg^[[00m$

I need to use sed command, to match the ending .gpg and remove it. So, if there were no special characters, I would use:

cat zz | sed 's/\.gpg$//'

So how can I match the .gpg^[[00m$ pattern with sed ?

I tried all possible permutations, but still does not work. For example:

cat zz | sed 's/\.gpg\^\[\[00m$//'

Best Answer

What you see on the terminal as ^[ is the escape character. The second [ is a [.

You need to include the code for escape.

replace the ^[ with an escape character.

esc="$(echo '\033')"
sed 's/\.gpg'"${esc}"'\[00m$//'

esc='\x1b`
...

Related Solutions

Using sed to get rid of characters < > ,

With sed:

sed 's|[<>,]||g'

With tr:

tr -d '<>,'

Bash – How does storing the regular expression in a shell variable avoid problems with quoting characters that are special to the shell

[[ ... ]] tokenisation clashes with regular expressions (more on that in my answer to your follow-up question) and \ is overloaded as a shell quoting operator and a regexp operator (with some interference between the two in bash), and even when there's no apparent reason for a clash, the behaviour can be surprising. Rules can be confusing.

Who can tell what these will do without trying it (on all possible input) with any given version of bash?

[[ $a = a|b ]]
[[ $a =~ a|b ]]
[[ $a =~ a&b ]]
[[ $a =~ (a|b) ]]
[[ $a =~ ([)}]*) ]]
[[ $a =~ [/\(] ]]
[[ $a =~ \s+ ]]
[[ $a =~ ( ) ]]
[[ $a =~ [ ] ]]
[[ $a =~ ([ ]) ]]

You can't quote the regexps, because if you do, since bash 3.2 and if bash 3.1 compatibility has not been enabled, quoting the regexps removes the special meaning of RE operator. For instance,

[[ $a =~ 'a|b' ]]

Matches if $a contains a litteral a|b only.

Storing the regexp in a variable avoids all those problems and also makes the code compatible to ksh93 and zsh (provided you limit yourself to POSIX EREs):

regexp='a|b'
[[ $a =~ $regexp ]] # $regexp should *not* be quoted.

There's no ambiguity in the parsing/tokenising of that shell command, and the regexp that is used is the one stored in the variable without any transformation.

Best Answer

Related Solutions

Using sed to get rid of characters < > ,

Bash – How does storing the regular expression in a shell variable avoid problems with quoting characters that are special to the shell

Related Question