SQL AlwaysOn – Can Auto Failover Cause Split-Brain?

After a forced failover testing with async mode, do I need to rebuild my old primary database?

I'm not sure exactly what you mean by "rebuild" the database, but provided the databases are still in a working condition then you shouldn't need to take any actions like that.

What you're seeing, by performing a forced failover, is by design. If you do a forced failover, you could potentially have failed over to a replica that isn't completely caught up or at the same point-in-time as the primary replica. Because of that, data movement is suspended to the secondary replica(s) from the now primary replica so there is a way to have manual intervention if you are now on a database that is "behind". That behavior that you are seeing is a good thing.

This BOL reference explains it all:

After a forced failover, all secondary databases are suspended. This includes the former primary databases, after the former primary replica comes back online and discovers that it is now a secondary replica. You must manually resume each suspended database individually on each secondary replica.

When a secondary database is resumed, it initiates data synchronization with the corresponding primary database. The secondary database rolls back any log records that were never committed on the new primary database. Therefore, if you are concerned about possible data loss on the post-failover primary databases, you should attempt to create a database snapshot on the suspended databases on one of the synchronous-commit secondary databases.

Please see this BOL reference on how to resume an AG database.

The T-SQL for this would be:

alter database YourDatabaseName
set hadr resume;

NOTE/WARNING/DISCLAIMER: You really need to do some leg work to ensure that you are not causing data loss by resuming data movement. See above, it could be a huge problem. The point of suspending data movement is for this very reason: To manually make sure you can recover as much data as possible. When you resume data movement, that could be irreversible. When you resume data movement after a forced failover, you have to have the words "potential data loss" in the forefront of your mind always.

You can have AGs within an FCI but I don't see the point (I mean when you do this you usually do away with the FCIs unless there was some specific requirement).

In an AG the listener acts similar to a load balancer - kind of. If applications support using application intent read only in the connection string it can offload reads to the "passive" server. It's not load balancing though as it only goes to one server. In 2016 those reads are round robin and so it works better.

If your apps don't support application intent and splitting reads and writes (very few do) though then you're largely out of luck. And you're not going to be able to load balance writes.

In your comment to me you mentioned you'd want to keep the FCI in order to balance load. That isn't really going to work that way with an AG and it would be wasteful of resources - which are one of the best parts of having an AG.

Let's say you have two nodes in an AG and ten databases. You can (if queries don't cross the databases, or only groups of them), create ten groups with a different name each. Then to do balancing you can failover just groups within the node to other nodes until you're happy with it. (Note though that you'll need to do scripting to do this; AGs don't have a notion of preferred nodes to go to automatically unlike FCIs).

As the nodes are "as" available as an FCI there's not many reasons to leave the FCI with its shared storage in place. Those resources are kind of wasted.