Mongodb – Moving MongoDB shards from one server to another

mongodbmongodb-3.0sharding

I have a new server on which i have to copy the old version of the MongoDB already configured. On the old server(3.0.4), there are two shards: one shard has 350GB, and the other has 300GB of data.

I followed the following steps:

  1. Installed MongoDB 3.0.15 on new server
  2. Copied the data from two shards to the new server

  3. then ran the following commands

    mongod    --configsvr --dbpath C:\mongoConfigData/configdb1 --port 27019
mongod    --configsvr --dbpath C:\mongoConfigData/configdb2 --port 27020
mongod    --configsvr --dbpath C:\mongoConfigData/configdb3 --port 27021

mongos    --configdb 127.0.0.1:27019,127.0.0.1:27020,127.0.0.1:27021 --port 27017

mongod    --dbpath F:\data --storageEngine wiredTiger --wiredTigerJournalCompressor zlib --wiredTigerCollectionBlockCompressor zlib --port 27010

mongod    --dbpath D:\data --storageEngine wiredTiger --wiredTigerJournalCompressor zlib --wiredTigerCollectionBlockCompressor zlib --port 27011

  4. Added the shard and the shard collection details

    sh.addShard("localhost:27010") 
sh.addShard("localhost:27011") 
sh.shardCollection("log", {"id" : 1, "eventId": 1})

Shard distribution shows both the shards but the total docs and the collection size does not match the one in the initial server.
Further the content in the mongodb is not the complete data
Why is it so? What am i doing wrong?

Best Answer

You are not copying the contents of the config servers and effectively creating a new sharded deployment rather than restoring a backup of an existing one. Config servers include essential information about where data lives in a sharded deployment, so you must backup & restore the config server data along with the shards.

Note: there have been changes to sharded cluster configuration in successive releases, so make sure you are using documentation for the correct MongoDB release series. The links below are specific to MongoDB 3.0.

For supported backup approaches see the MongoDB 3.0 Backup & Restore a Sharded Cluster tutorials.

sh.addShard("localhost:27010")

sh.addShard("localhost:27011")

If you've restored a sharded cluster from a backup, the shards will already be configured. If host names have changed from the original deployment, you will need to update the config.shards information via a mongos rather than adding new shards (and then restart the mongod instances in your sharded cluster to ensure the new names are used). See the MongoDB 3.0 Restore a Sharded Cluster tutorial for more specific details.

sh.shardCollection("log", {"id" : 1, "eventId": 1})

If a sharded cluster is already configured, sharding an existing sharded collection will return an error. Since you ran this command starting from fresh config server data, the sh.shardCollection() command will be unaware of how existing data is distributed on the shards and assume that the primary shard for a database with an unsharded collection has the full range of data.

Related Question