Shell – Bash – Convert escape character colored text to BBCode

escape-charactersreplacesedshell-scripttext processing

I want to convert a terminal escape-character colored text file to BBCode colored text.
For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color escape characters. The output looks like this:

[0m[38;5;231mV/Zygote  ( 4666): Switching descriptor 55 to /dev/null
[0m[38;5;231mV/Zygote  ( 4666): Switching descriptor 9 to /dev/null
[0m[38;5;40mI/ggheart ( 1111): onStop
[0m[38;5;40mI/Test    ( 1111): onStop
[0m[38;5;75mD/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
[0m[38;5;75mD/ActivityThread( 4666): setTargetHeapUtilization:0.75
[0m[38;5;75mD/ActivityThread( 4666): setTargetHeapMinFree:2097152

To convert the color codes to BBCode i've written this sed script:

#!/bin/bash
#Use 'logcat -Cd > /sdcard/logcat.txt' as input file

sed '/\x1b/ {
    s/\x1b\[0m\x1b\[38;5;40m/\[COLOR="Green"\]/
    s/\x1b\[0m\x1b\[38;5;196m/\[COLOR="Red"\]/
    s/\x1b\[0m\x1b\[38;5;75m/\[COLOR="Blue"\]/
    s/\x1b\[0m\x1b\[38;5;166m/\[COLOR="Sienna"\]/
    s/\x1b\[0m\x1b\[38;5;231m/\[COLOR="DarkSlateGray"\]/
    s/\x1b\[38;5;40m/\[COLOR="Green"\]/
    s/\x1b\[38;5;196m/\[COLOR="Red"\]/
    s/\x1b\[38;5;75m/\[COLOR="Blue"\]/
    s/\x1b\[38;5;166m/\[COLOR="Sienna"\]/
    s/\x1b\[38;5;231m/\[COLOR="DarkSlateGray"\]/
    s/\x1b\[0m/\[COLOR="Black"\]/
    s/$/\[\/COLOR\]/
    }' <logcat.txt >logcat2.txt

The processed output text looks like this:

[COLOR="DarkSlateGray"]V/Zygote  ( 4666): Switching descriptor 55 to /dev/null[/COLOR]
[COLOR="DarkSlateGray"]V/Zygote  ( 4666): Switching descriptor 9 to /dev/null[/COLOR]
[COLOR="Green"]I/ggheart ( 1111): onStop[/COLOR]
[COLOR="Green"]I/Test    ( 1111): onStop[/COLOR]
[COLOR="Blue"]D/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad[/COLOR]
[COLOR="Blue"]D/ActivityThread( 4666): setTargetHeapUtilization:0.75[/COLOR]
[COLOR="Blue"]D/ActivityThread( 4666): setTargetHeapMinFree:2097152[/COLOR]

This is syntactically correct and works properly in corresponding forum boards but its not optimized and wastes too many characters (that are limited in most boards) due to COLOR tags not spanning over multiple lines.

It should rather look like this where same colored lines do not close/reopen the same COLOR tag:

[COLOR="DarkSlateGray"]V/Zygote  ( 4666): Switching descriptor 55 to /dev/null
V/Zygote  ( 4666): Switching descriptor 9 to /dev/null[/COLOR]
[COLOR="Green"]I/ggheart ( 1111): onStop
I/Test    ( 1111): onStop[/COLOR]
[COLOR="Blue"]D/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
D/ActivityThread( 4666): setTargetHeapUtilization:0.75
D/ActivityThread( 4666): setTargetHeapMinFree:2097152[/COLOR]

Any idea how to achieve this? Is it even possible with line based text processor? Its not mandatory as long as it runs on shell/bash/cygwin.

Best Answer

Noting a comment about "Easy using awk", that would be relative to sed.

OP was describing a not-very-well documented feature of logcat (see commit) which tells logcat to assign hardcoded-colors to each line based on the log entry's priority.

There are other choices, such as logcat-color, PID cat and coloredlogcat which let you color different fields, e.g., raising the possibility of multiple colors per line.

Here's an awk script which recognizes the hardcoded logcat colors, but allows multiple colors per line:

#!/usr/bin/awk -f
BEGIN {
        colors["0"]        = "Black";
        colors["38;5;40"]  = "Green";
        colors["38;5;196"] = "Red";
        colors["38;5;75"]  = "Blue";
        colors["38;5;166"] = "Sienna";
        colors["38;5;231"] = "DarkSlateGray";
        color = "";
        last = "";
        this = "";
        save = "";
}
/\033/ {
        done = "";
        while ( $0 ~ /\033\[[;0-9]*m/ ) {
                mark = match($0, /\033\[[;0-9]*m/ );
                if ( mark > 1 ) { done = done substr($0, 1, mark - 1); }
                item = substr($0, RSTART + 2, RLENGTH - 3);
                $0 = substr($0, RSTART + RLENGTH);
                if ( match($0, /^\033\[[;0-9]*m/ ) > 0 ) continue;
                color = colors[item];
                if ( done == "" ) this = color;
                if ( item == "0" ) color = "";
                if ( color == "" ) {
                        if ( $0 != "" ) last = color;
                        $0 = "[/COLOR]" $0;
                } else if (color != last) {
                        $0 = "[COLOR=\"" color "\"]" $0;
                        last = color;
                }
        }
        $0 = done $0;
        if ( last != "" ) $0 = $0 "[/COLOR]";
}
{
        if ( NR > 1 ) {
                if ( this == last) sub("\[/COLOR\]$", "", save);
                print save;
        }
        save = $0;
}
END {
        if ( NR > 0 ) print save;
}

With the original example:

^[[0m^[[38;5;231mV/Zygote  ( 4666): Switching descriptor 55 to /dev/null
^[[0m^[[38;5;231mV/Zygote  ( 4666): Switching descriptor 9 to /dev/null
^[[0m^[[38;5;40mI/ggheart ( 1111): onStop
^[[0m^[[38;5;40mI/Test    ( 1111): onStop
^[[0m^[[38;5;75mD/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
^[[0m^[[38;5;75mD/ActivityThread( 4666): setTargetHeapUtilization:0.75
^[[0m^[[38;5;75mD/ActivityThread( 4666): setTargetHeapMinFree:2097152

you would get the requested output:

[COLOR="DarkSlateGray"]V/Zygote  ( 4666): Switching descriptor 55 to /dev/null
V/Zygote  ( 4666): Switching descriptor 9 to /dev/null
[COLOR="Green"]I/ggheart ( 1111): onStop
I/Test    ( 1111): onStop
[COLOR="Blue"]D/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
D/ActivityThread( 4666): setTargetHeapUtilization:0.75
D/ActivityThread( 4666): setTargetHeapMinFree:2097152[/COLOR]

but changing the input to

^[[0m^[[38;5;231mV/Zygote  ( 4666): Switching descriptor 55 to /dev/null
^[[0m^[[38;5;231mV/Zygote  ( 4666): Switching descriptor 9 to /dev/null
^[[0m^[[38;5;40mI/ggheart ( ^[[38;5;75mI/ggheart 1111^[[0m): onStop
^[[0m^[[38;5;40mI/Test    ( 1111): onStop
^[[0m^[[38;5;75mD/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
^[[0m^[[38;5;75mD/ActivityThread( 4666): setTargetHeapUtilization:0.75
^[[0m^[[38;5;75mD/ActivityThread( 4666): setTargetHeapMinFree:2097152

gives this result:

[COLOR="DarkSlateGray"]V/Zygote  ( 4666): Switching descriptor 55 to /dev/null
V/Zygote  ( 4666): Switching descriptor 9 to /dev/null[/COLOR]
[COLOR="Green"]I/ggheart ( [COLOR="Blue"]I/ggheart 1111[/COLOR]): onStop
[COLOR="Green"]I/Test    ( 1111): onStop
[COLOR="Blue"]D/ActivityThread( 4666): handleBindApplication:com.mxtech.videoplayer.ad
D/ActivityThread( 4666): setTargetHeapUtilization:0.75
D/ActivityThread( 4666): setTargetHeapMinFree:2097152[/COLOR]
Related Question