Mongodb – How verify if the MongoDb Sharded Replica Set works well

mongodb

I set up a MongoDb Sharded Replica Set with 2 clusters as described in this tutorial but I don't know how to test if everything works well.

I have no error messages on the any of the hosts, which suggest that my setup is ok. All hosts can ping each other and are using the default port 27017.

Here are the hostnames.

Shard 1:

  • mongo-rs1-1
  • mongo-rs1-2
  • mongo-rs1-3

Shard 2:

  • mongo-rs2-1
  • mongo-rs2-2
  • mongo-rs2-3

3 config servers:

  • cfg-1
  • cfg-2
  • cfg-3

A router (Mongos):

  • mongos-1

Here is what I have when running sh.status():

mongos> sh.status()  
                    
--- Sharding Status ---              
  sharding version: {     
        "_id" : 1,  
        "version" : 4,          
        "minCompatibleVersion" : 4, 
        "currentVersion" : 5,       
        "clusterId" : ObjectId("562491c645300851d95d85bb")                      
}     
  shards:  
        {  "_id" : "rs1",  "host" : "rs1/mongo-rs1-1:27017,mongo-rs1-2:27017,mon
go-rs1-3:27017" }  
        {  "_id" : "rs2",  "host" : "rs2/mongo-rs2-1:27017,mongo-rs2-2:27017,mon
go-rs2-3:27017" } 
  databases:          
        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" } 
        {  "_id" : "testshard",  "partitioned" : true,  "primary" : "rs1" } 
        {  "_id" : "testrs2",  "partitioned" : false,  "primary" : "rs2" }
                                                              

I run my mongos with commands:

mongos --configdb cfg-1:27017,cfg-2:27017,cfg-3:27017

I run my config servers with commands:

mongod --noprealloc --smallfiles --configsvr --dbpath /data/db --port 27017

I run my replicat sets (rs1 and rs2) with commands:

mongod --replSet rs1 --noprealloc --smallfiles

The replica sets are set like this:

rs1:PRIMARY> rs.status()                                                        
{                                                                               
        "set" : "rs1",                                                          
        "date" : ISODate("2015-10-19T15:45:31Z"),                               
        "myState" : 1,                                                          
        "members" : [                                                           
                {                                                               
                        "_id" : 0,                                              
                        "name" : "mongo-rs1-1:27017",                           
                        "health" : 1,                                           
                        "state" : 1,                                            
                        "stateStr" : "PRIMARY",                                 
                        "uptime" : 28658,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "electionTime" : Timestamp(1445240879, 1),              
                        "electionDate" : ISODate("2015-10-19T07:47:59Z"),       
                        "self" : true                                           
                },    
                {                                                               
                        "_id" : 1,                                              
                        "name" : "mongo-rs1-2:27017",                           
                        "health" : 1,                                           
                        "state" : 2,                                            
                        "stateStr" : "SECONDARY",                               
                        "uptime" : 28654,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "lastHeartbeat" : ISODate("2015-10-19T15:45:30Z"),      
                        "lastHeartbeatRecv" : ISODate("2015-10-19T15:45:30Z"),  
                        "pingMs" : 0,                                           
                        "syncingTo" : "mongo-rs1-1:27017"                       
                },  
                {                                                               
                        "_id" : 2,                                              
                        "name" : "mongo-rs1-3:27017",                           
                        "health" : 1,                                           
                        "state" : 2,                                            
                        "stateStr" : "SECONDARY",                               
                        "uptime" : 28654,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "lastHeartbeat" : ISODate("2015-10-19T15:45:29Z"),      
                        "lastHeartbeatRecv" : ISODate("2015-10-19T15:45:30Z"),  
                        "pingMs" : 1,                                           
                        "syncingTo" : "mongo-rs1-1:27017"                       
                }                                                               
        ],                                                                      
        "ok" : 1                                                                
}                          

Tests I made:

  1. I connected with my MongoDB client to the Mongos URL.
  2. I created a DB: "testshard"
  3. I created a document in "testshard"
  4. I logged in the 3 replica: mongo-rs1-* and mongo-rs2-* to check where the data was saved

Result

I noticed that the DB was created on RS1 (shard 1) and was replicated on the 3 replicas: mongo-rs1-1 , mongo-rs1-2, mongo-rs1-3. The DB was not created on the shard 2; I think this is normal as the sharding feature save on one shard but not both.

From here, it looks like the replica works. But what about the shard 2? How to check if data can be saved to shard 2?

Test #2

  1. I logged in the Mongos and I created the database rs2 which I force to rs2 in the database "config" of the Mongos.
  2. I create a document in testrs2.

I was expecting to see the new documents created in rs2, but rs2 is still an empty DB.

Questions

Q1: How can I test the "sharding" feature? I would expect to save a document and see it in the right replica set (rs1 or rs2).

Q2: How does Mongo know where to save the data between rs1 or rs2 ?

Q3: Is there a way to automatize the sharding? I don't want to tell which database should go where, this should be automatic. I was expecting the data to be saved randomly between the 2 shards.

Q4: What are the best backup strategies in my case? I guess I just need to do a mongodump of rs1 and rs2 ?

Q5: Am I right if I say that the only access point to the MongoDB is through the Mongos? Nobody should query the replica set.

Best Answer

I will try to be brief. Most of the answers are documented.

Q1: There are many ways. Shard a collection using _id:hashed and set initial number of chunks 2 and then start inserting documents (100 is a good number of docs). You will see documents on both shards.

Q2: It uses range partitioning. Each shard holds ranges of the shard key.

Q3: Sharding is an automated process, all you need to do is to determine a shard key.

Q4: Stop the balancer, backup shard rs1, rs2 and the config server, start the balancer.

Q5: On a sharded cluster the access point should be the mongos