Bash scripting problem – compare two lists and create a third

Change this line:

if [[ "$TEST" == "ON Master" ]];

To this:

if [ "$TEST" == "ON Master" ];

Details

The issue is the use of [[ .. ]]. The output is showing you the difference. Your value that you're getting for $STATUS is not simply "On Master". It most likely contains other characters, that are most likely not printable so are not being seen.

[[ .. ]]

++ echo On Master
+ TEST='On Master'
+ [[ On Master == \O\N\ \M\a\s\t\e\r ]]

[ .. ]

++ echo On Master
+ TEST='On Master'
+ '[' 'On Master' == 'ON Master' ']'

The use of double square brackets ([[ .. ]]) is discussed here on the TLDP Advanced Bash Scripting pages.

echo $STATUS

This line seems a little suspicious to me as well. I'd protect the contents of $STATUS by wrapping it in double quotes as well.

TEST=`echo "$STATUS"`

Also I'd drop the back ticks ( `...` ) and use the $( ... ) notation instead for executing this command. This change is just a best practice and isn't part of your issue though.

TEST=$(echo "$STATUS")

Control characters in output (^[[92mON^[[0m Master)

Given you're seeing these control characters in your output (^[[92m & ^[[0m) I'm suspicious that the grep command is introducing these into your output in the pipe. It may be that grep is aliases to always include the --color switch, I'd temporarily try calling the executable directly, and by pass any aliases that may be there. Just change the grep to this, /bin/grep.

The presence of these is what we suspected and is why the text was wrapping when you echo the variable $STATUS. These characters are unprintable, and change the color of the terminal to highlight matches that grep has found.

The presence of these also explains why the =~ operator didn't match too. You were trying to match 'On Master' with '^[[92mON^[[0m Master'.

Lastly the colored output with the control characters may be coming from another tool before the grep. I would need to see the actual output from /root/setup_ha to confirm this, but I would be suspicious of that tool as well in producing these characters in the pipe stream.

Stripping the control characters

I found this U&L Q&A titled: Program that passes STDIN to STDOUT with color codes stripped?.

Use either of these methods to get rid of the control characters.

Perl

$ cmd-with-colored-output | perl -pe 's/\e\[?.*?[\@-~]//g'

Sed

$ cmd-with-colored-output | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"

You can do that with a while loop:

while read l; do
  [ ${#l} -gt 65 ] && \
    echo "$l" | langid --line | grep -q "is" && \
    echo "$l"
done <file

• read l read the input line by line and store the current line in the variable $l.
• [ ${#l} -gt 65 ] if the line contains more than 65 characters.
  • echo "$l" | langid --line | grep -q "is" process the line and grep for the language, notice with -q, grep will be silent. We just want to check if the string is there, no output.
  • echo "$l" If the string is there, print the original line.
• <file use the contents of file as input.

Edit: The above runs the langid command on each line, this is very slow. If you want it to run in one transit (faster) use this:

awk 'FNR==NR{a[NR]=$0}
  FNR!=NR&&$1~"is"{print a[FNR]}' \
<(sed -n '/^.\{65\}/p' file) \
<(sed -n '/^.\{65\}/p' file | langid --line)

• awk processes two "files":
  • The output of sed -n '/^.\{65\}/p' file: All sentences with 65 or more characters.
  • The output of sed -n '/^.\{65\}/p' file | langid --line which processes all lines with 65 or more characters in one transit.
• Inside awk:
  • FNR==NR applies in the first "file"
  • a[NR]=$0 Fill an array with the line number as index
  • FNR!=NR&&$1~"is" applies to the second "file" and checks if the line contains the string is
  • print a[FNR] if thats true, print the corresponding line in the prevously created array a which contains the original sentence.