Shell – Strange behaviour of tr using ranges

command lineshelltrwildcards

I have one particular server that is exhibiting strange behaviour when using tr. Here is an example from a working server:

-bash-3.2$ echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-z]
1234567890
-bash-3.2$

That makes perfect sense to me.

This, however, is from the 'special' server:

[root@host~]# echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-z]
abcdefghijklmnpqrstuvwxyz1234567890

As you can see, deleting all lower case characters fails. BUT, it has deleted the letter 'o'

The interesting part is the following two examples, which make no sense to me whatsoever:

[root@host~]# echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-n]
opqrstuvwxyz1234567890
[root@host~]# echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-o]
abcdefghijklmnpqrstuvwxyz1234567890
[root@host~]#

(again, the 'o' is deleted in the last example)

Does anyone have any idea what is going on here? I can't reproduce on any other linux box that I am using.

Best Answer

you have a file named o in current directory

foo> ls
foo> echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-z]
1234567890
foo> touch o
foo> echo "abcdefghijklmnopqrstuvwxyz1234567890"|tr -d [a-z]
abcdefghijklmnpqrstuvwxyz1234567890

shell will expand [a-z] string if a match is found.

This is called pathname expansion, according to man bash

Pathname Expansion
After word splitting, unless the -f option has been set, bash scans each word for the characters *, ?, and [. ... (...)

bash will perform expansion.

[...] Matches any one of the enclosed characters.

Related Solutions

Bash – Why Prefer Character Classes Over Character Ranges

According to the bash manpage, the LC_COLLATE environment variable affects character ranges, exactly as per Hauke Laging's answer:

LC_COLLATE This variable determines the collation order used when sorting the results of pathname expansion, and determines the behavior of range expressions, equivalence classes, and collating sequences within pathname expansion and pattern matching.

On the other hand, LC_CTYPE affects character classes:

LC_CTYPE This variable determines the interpretation of characters and the behavior of character classes within pathname expansion and pattern matching.

What this means is that both cases are potentially problematic if you're thinking in a English, left-to-right, Latin alphabet, Arabic-digit context.

If you're really proper, and/or are scripting for a multi-locale environment, it's probably best to make sure you know what your locale variables are when you're matching files, or to be sure that you're coding in a completely generic way.

It's very difficult to foresee some situations though, unless you've studied linguistics.

However, I don't know of a Latin-using locale that changes the order of letters, so [a-z] would work. There are extensions to the Latin alphabet that collate ligatures and diacriticals differently. However, here's a little experiment:

mkdir /tmp/test
cd /tmp/test
export LC_CTYPE=de_DE.UTF-8
export LC_COLLATE=de_DE.UTF-8
touch Grüßen
ls G* # This says ‘Grüßen’
ls *[a-z]en # This says nothing!
ls *[a-zß]en # This says ‘Grüßen’
ls Gr[a-z]*en # This says nothing!

This is interesting: at least for German, neither diacriticals like ü nor ligatures like ß are folded into latin characters. (either that, or I messed up the locale change!)

This may be bad for you, of course, if you're trying to find filenames that start with a letter, use [a-z]* and apply it to a file that starts with ‘Ä’.

linux cron – Email Only Occasionally Sent on Output and Errors

Upon further testing, I suspect the & is messing with your results. As you point out, &>/dev/null is bash syntax, not sh syntax. As a result, sh is creating a subshell and backgrounding it. Sure, the subshell's echo creates stderr, but my theory is that:

cron is not catching the subshell's stderr, and
the backgrounding of the subshell always completes successfully, thus bypassing your || echo ....

... causing the cron job to have no output and thus no mail. Based on my reading of the vixie-cron source, it would seem that the job's stderr and stdout would be captured by cron, but it must be getting lost by the subshell.

Test it yourself in a /bin/sh environment (assuming you do not have a file named 'bar' here):

(grep foo bar) &
echo $?

Best Answer

Related Solutions

Bash – Why Prefer Character Classes Over Character Ranges

linux cron – Email Only Occasionally Sent on Output and Errors

Related Question