mongodb
Suppose a replication set R1 contains one primary, 2 secondary and one arbiter node. Voting will happen if the primery node goes down. Based on what criteria secondary members will vote to select the next primary node? Is the voting random process ? Whether the members will check the health / efficiency of the other members before voting?
Consider all the nodes have same priority and no salve delay
Some time ago my one of my data replicas which was secondary at a time crushed due to hard drive failure. After I fixed that problem and restarted secondary it went into “Recovering” state. But my arbiter is now “Secondary”
A MongoDB arbiter cannot automatically become a secondary or a primary node, as it does not have a copy of the data set.
If you try to manually reconfigure the arbiter as a regular node via rs.reconfig() you should get an exception similar to:
{
"errmsg" : "exception: arbiterOnly may not change for members",
"code" : 13510,
"ok" : 0
}
Furthermore, the data directory for this Arbiter shows bunch of files of total size >10GB. They indeed look to me like data files. Are they? What is going to happen to these files when Recovery completes?
Assuming this node is definitely an arbiter, I would expect those files are unused (check the timestamps?) and are either:
local database created if this node was incorrectly initialized with an oplogYou can always log into the arbiter mongod directly to see what data it appears to have.
From the MMS Dashboard; what is the MMS Agent Log saying?
Have you looked in the mms-monitoring agent log locally on the server?
/var/log/mongodb-mms/monitoring-agent.log
Update
Please add a screen shot of how you are configuring the host
Update 14:02
Can you ping S143, S144 and mongo-client2 from the server where the mms-agent is installed?
Can you telnet to those servers from the mms-agent machine?
ex
telnet S143 37007
telnet S144 37007
telnet mongo-client2 37007
If you can't, then perhaps there is a network problem between the server with the MMS agent and the three mongods you listed above.
Best Answer
Replica Set Elections
The MongoDB manual describes Replica Set Elections and the associated election mechanics such as Vetoes in detail.
Broadly speaking, the criteria for electing a node as primary includes:
Is that node currently in
PRIMARYorSECONDARYstate?Is that node eligible to become primary (
priority> 0)?Do a majority of the nodes of the replica set see that memory as available (by way of replica set heartbeat)?
Does that node have the least replication lag in relation to the current primary?
Is that node the highest priority eligible node?
Would the node veto itself if elected primary? For example, if you step down a primary it will veto itself for election for 60 seconds by default.
The first node to receive a majority of votes subject to the election mechanics will become primary.
There are some further subtleties to the process depending what triggered the election (i.e. a unplanned failover versus stepping down the current primary).
In order to ensure consensus can be reached by a strict majority of voting nodes, the general recommendation is to have an odd number of voting nodes. With your example configuration an arbiter node is not required, and you should instead only have the primary and two secondaries. An arbiter node does not have any special role in the election process aside from being available to cast a vote.
With all things being equal (no replication lag, default priorities, no network issues, etc) your two secondaries in this scenario are equally likely to be elected primary in the event the current primary unexpectedly fails.