I'm not sure what documentation you've read so apologies if I'm repeating anything here.
To distribute reads to secondary nodes, most drivers allow you to set a readPreference value for the current session. Clients set read preference on a per-connection basis. With slaveOk, the driver should will always send queries to the secondaries, if they're available.
Distributing reads to secondaries requires the use of
ReplicaSetConnection with ReadPreference.SECONDARY.
See “rs.slaveOk()” for more information and this link.
In the mongo shell, to enable secondary reads, issue the following command :
rs.slaveOk()
The PHP documentation for it is here but I'm guessing that may be the documentation you're referring to.
As a FYI, here's an old discussion about it on the MongoDB Google Group.
If you're still having issues, I'd recommend using the MongoDB Google Group and providing some further information such as the version of MongoDB you're using, the version of the PHP driver, your log files, rs.conf() and rs.status().
As a FYI, you have to be careful with read scaling as sending too many reads to the secondaries can often result in the secondaries lagging the primary and becoming stale, thus requiring a full resync.
Its not a good idea to run the backup against the Primary. The extra load you adding on the Primary might cause heartbeat failures and trigger an election. Now in your case, overloading the primary might caused an election, you can see that info on your logs, but the other secondary didn't become new primary which expected to. I see some cases why didn't. 1) Replication lag was big 2) you run the replica set on the same server or storage so all nodes get affected but here logs are your best friend on what really happened.
Best Answer
Steps: