Bash HISTCONTROL=ignorespace – Purpose and Usage

The ! character invokes bash's history substitution (enabled by default in interactive shells). When followed by a string (as in your failing example) it tries to expand to the last history event that began with that string. Just like $var gets expanded to the value of that string, !echo would expand to the last echo command in your history.

Space is a breaking character in such expansions. First note how this would work with variables:

# var="like"
# echo "$var"
like
# echo "$"
$
# echo "Do you $var frogs?"
Do you like frogs?       <- as expected, variable name broken at space
# echo "Do you $varfrogs?"
Do you?                  <- $varfrogs not defined, replaced with blank
# echo "Do you $ var frogs?"
Do you $ var frogs?      <- $ not a valid variable name, ignored

The same thing will happen for history expansion. The bang character (!) starts off a history replacement sequence, but only if followed by a string. Following it with a space make it literal bang instead of part of a replace sequence.

You can avoid this kind of replacement for both variable and history expantion by using single quotes. Your first examples used single quotes and so ran fine. Your last examples are in double quotes and thus bash scanned them for expantion sequences before it did anything else. The only reason the first one didn't trip is that that the space is a break character as shown above.

As explained in the Bash manual, history lines prefixed with a * have been modified. This happens when you navigate to a command (e.g. by using the Up key), edit it and then navigate away from it without hitting Enter. So a history line like this:

1095* 

is usually the result of navigating to a command in history, backspacing and navigating away from it. Knowing this, you can easily prevent such lines appearing in your Bash history.

BTW, you can revert modified commands to their unedited state by navigating to them and hitting Ctrl + _ repeatedly.