What happens if one DC goes down in Cassandra multi DC setup

cassandra

We have a Cassandra multi DC setup with 3 nodes in each DC, enabled with NetworkTopologyStrategy and RF 3.

The java client interacts with Cassandra cluster using read and write consistency as LOCAL_QUORUM.

We have our service in each of these DC (same service, just config changes), so that LOCAL_QUORUM helps pick the co-ordinator in the same DC and responds back without waiting for other DC.

There is no interaction between the services running in these 2 DCs.

What would happen if one of the DC goes down completely and is not able to recover soon? Obviously, the service running in the same DC would be affected, but what would happen to the other DC which is still up? I guess the service won't be affected as it talks to local DC only, and think that the read/writes won't be affected even if they are not relayed to the other DC.

Best Answer

All writes are always happens to all DCs (see the diagram in DataStax Architecture guide).

But if the one DC is down, the nodes in the other DC will detect this, and will start to collect hints during the configured hints time window (3 hours by default, and cross-DC hints aren't disabled) - if servers in first DC comes back during this window, hints will be replayed