After a forced failover testing with async mode, do I need to rebuild my old primary database?
I'm not sure exactly what you mean by "rebuild" the database, but provided the databases are still in a working condition then you shouldn't need to take any actions like that.
What you're seeing, by performing a forced failover, is by design. If you do a forced failover, you could potentially have failed over to a replica that isn't completely caught up or at the same point-in-time as the primary replica. Because of that, data movement is suspended to the secondary replica(s) from the now primary replica so there is a way to have manual intervention if you are now on a database that is "behind". That behavior that you are seeing is a good thing.
This BOL reference explains it all:
After a forced failover, all secondary databases are suspended. This includes the former primary databases, after the former primary replica comes back online and discovers that it is now a secondary replica. You must manually resume each suspended database individually on each secondary replica.
When a secondary database is resumed, it initiates data synchronization with the corresponding primary database. The secondary database rolls back any log records that were never committed on the new primary database. Therefore, if you are concerned about possible data loss on the post-failover primary databases, you should attempt to create a database snapshot on the suspended databases on one of the synchronous-commit secondary databases.
Please see this BOL reference on how to resume an AG database.
The T-SQL for this would be:
alter database YourDatabaseName
set hadr resume;
NOTE/WARNING/DISCLAIMER: You really need to do some leg work to ensure that you are not causing data loss by resuming data movement. See above, it could be a huge problem. The point of suspending data movement is for this very reason: To manually make sure you can recover as much data as possible. When you resume data movement, that could be irreversible. When you resume data movement after a forced failover, you have to have the words "potential data loss" in the forefront of your mind always.
Yes, you need to populate databases on secondary nodes in an availability group yourself, they do not appear "automatically". The full documentation is found on MSDN, with multiple ways to execute. The short version is:
- Your primary database needs to be in FULL recovery mode.
- You must restore the database in NORECOVERY mode on the secondary node. Logs must be applied so that the secondary database is as close to the current transaction state of the primary as possible.
- Once you have done this, you need to join the secondary database to the availability group:
ALTER DATABASE [foo] SET HADR AVAILABILITY GROUP = [bar];
Best Answer
Listen to your adviser. By restoring a backup, you are essentially replacing the database schema and data. You will need to turn synchronization off, remove the DB from HA and perform the restore on the primary and replica, leaving the replica version in a restoring state by using WITH NORECOVERY. Once your backup is in place, put the DB back into HA and start synchronization again.
HA is very similar to mirroring and uses similar technology, just not nearly as finicky. You will want to treat your HA DBs similarly as well.
Code would be similar to the following:
--on primary
--on primary
--on secondary
--on primary
--on secondary