Default Order of Linux Sort Command

The LC_COLLATE locale category controls the sorting order. LC_ALL sets all categories.

With LC_COLLATE=C, strings are sorted byte by byte. The bytes don't have to be ASCII characters (only byte values between 0 and 127 are ASCII). On a unix system, Unicode is almost always encoded as UTF-8. UTF-8 has the property that the encoding of characters as byte sequences preserves their ordering, and so sorting UTF-8 strings in byte lexicographic order is equivalent to sorting them in character lexicographic order. Therefore LC_COLLATE=C is suitable for sorting Unicode encoded in UTF-8 lexicographically according to the character values.

Note that Java does not actually sort according to the Unicode character values but according to their UTF-16 encoding. This makes a difference with surrogate pairs, i.e. if you have code points above 65535.

Neither UTF-8 byte representation sorting nor Java sorting nor the sort utility in a UTF-8 locale on GNU/Linux take combining characters into account, e.g. á (U+0061 LATIN SMALL LETTER A followed by U+0301 COMBINING ACUTE ACCENT) is sorted differently from á (U+00E1 LATIN SMALL LETTER A WITH ACUTE) (in a UTF-8 locale, both end up equivalent to a in the first pass but the second pass sorts by code point).

A solution in perl:

$ perl -anle '
    push @h, [$F[-1],$_];
    END {
        print for map  { $_->[0] }
                  sort { $a->[1] <=> $b->[1] }
                  map  { [$_->[1],hex($_->[0])] } @h;
    }
' file
4 jjk 7
5 hhf 25
2 ukr 9f
3 ezh ae
1 hdh d12

Explanation

• While processing file, we create an array of array @h, each of its element is an array reference [$F[-1],$_], with first element is the hex value to compare, and the second element is the whole line.

• In END block, we use Schwartzian transform:

  • With each element of @h, create an anonymous array, contains the whole line ( $_->[1] the second element of each array ref in @h ) and the hex value to compare hex($_->[0])]

  • Sort above array base on the hex value $a->[1] <=> $b->[1]

  • Get the first element of each array ref in sorted array map { $_->[0] } then print the result.

Update

With @Joseph R's suggestion, without using Schwartzian Transform:

$ perl -anle '
    push @h, [hex($F[-1]),$_];
    END {
        print $_->[1] for
            sort { $a->[0] <=> $b->[0] } @h;
    }
' file

Update 2

After reading stefan's comment, I think this can call direct:

$ perl -e '
    print sort {hex((split(/\s+/,$a))[-1]) <=> hex((split(/\s+/,$b))[-1])} <>;
' file
4 jjk 7
5 hhf 25
2 ukr 9f
3 ezh ae
1 hdh d12