Matching special characters with Regular Expression

regular expression

Say it's very easy if I want to find something containing lower-case letters and numbers with

produce_text | grep -E '[0-9a-z]'

Brackets are useful to match a set of characters, but what about those that are somewhat special?
If I want to, using brackets, match any character but one of these: a closing bracket ], a dash (or hyphen) "-", both slashes / and \, a caret ^, a colon :.
Will it look like this (I know this doesn't work)?

[^]-/\^:]

Best Answer

To match a literal ] and a literal - in a Bracket Expression you'll have to use them like this:

[^]/\^:-]

or, even better, since some tools require the backslash to be escaped:

[^]/\\^:-]

that is
The right-square-bracket ( ']' ) shall lose its special meaning and represent itself in a bracket expression if it occurs first in the list (after an initial '^', if any)
and
The hyphen-minus character shall be treated as itself if it occurs first (after an initial '^', if any) or last in the list
hence
If a bracket expression specifies both '-' and ']', the ']' shall be placed first (after the '^', if any) and the '-' last within the bracket expression.
The rules for bracket expressions are the same for ERE and BRE.

Related Solutions

Bash – How does storing the regular expression in a shell variable avoid problems with quoting characters that are special to the shell

[[ ... ]] tokenisation clashes with regular expressions (more on that in my answer to your follow-up question) and \ is overloaded as a shell quoting operator and a regexp operator (with some interference between the two in bash), and even when there's no apparent reason for a clash, the behaviour can be surprising. Rules can be confusing.

Who can tell what these will do without trying it (on all possible input) with any given version of bash?

[[ $a = a|b ]]
[[ $a =~ a|b ]]
[[ $a =~ a&b ]]
[[ $a =~ (a|b) ]]
[[ $a =~ ([)}]*) ]]
[[ $a =~ [/\(] ]]
[[ $a =~ \s+ ]]
[[ $a =~ ( ) ]]
[[ $a =~ [ ] ]]
[[ $a =~ ([ ]) ]]

You can't quote the regexps, because if you do, since bash 3.2 and if bash 3.1 compatibility has not been enabled, quoting the regexps removes the special meaning of RE operator. For instance,

[[ $a =~ 'a|b' ]]

Matches if $a contains a litteral a|b only.

Storing the regexp in a variable avoids all those problems and also makes the code compatible to ksh93 and zsh (provided you limit yourself to POSIX EREs):

regexp='a|b'
[[ $a =~ $regexp ]] # $regexp should *not* be quoted.

There's no ambiguity in the parsing/tokenising of that shell command, and the regexp that is used is the one stored in the variable without any transformation.

Best Answer

Related Solutions

Bash – How does storing the regular expression in a shell variable avoid problems with quoting characters that are special to the shell

Related Question