Mongodb – Where to insert data after sharding

mongodbsharding

Actual database in localhost:30000 ————- I used this as Shard1

another mongod in localhost:30001 ————- Shard2

another mongod in localhost:30002 ————- Shard3

Added 1 config server and Router pointing the configdb

Now in Router,

1. mongo admin
2. sh.addShard("localhost:30000")..30002           //Added 3 shards successfully
3. sh.enableSharding("myDatabasename in shard1")  //Enabled successfully
4. sh.shardCollection(...)                       //Shard Collection successfully

I checked each shards and I am able to see that it is split-ed in ascending order like

shard key starting with A-D in shard2 as chunk 1
shard key starting with E-H in shard3 as chunk 1
shard key starting with I-O in shard1 as chunk 1
shard key starting with P in shard1 as chunk 2               //had upto P only

Now I need to insert a new record means in which mongo instance I have to insert?

I tried in shard1 and inserted a new record with sharding key starting with E, which is getting inserted in shard1 itself whereas it has to be inserted in shard3.

Best Answer

With a sharded cluster, you need to insert data via a mongos process. A mongos process acts as a query router: it directs queries/updates to the appropriate shard(s) based on the current cluster configuration and metadata. Typically you deploy one mongos per application server.

From your driver point of view, you can update your connect string and replace the address of your standalone server or replica set seed lists with one or more mongos servers to use.

For more information, you're best starting with the Sharding Introduction in the MongoDB manual.

Related Solutions

MongoDB sharding

In order for data to be distributed across multiple shards, you need to have multiple chunks for a given sharded collection. A chunk is a contiguous range of shard key values representing approximately 64 megabytes of data by default.

Chunk ranges are normally created automatically based on observation of the shard key values and size of documents being inserted through mongos. When documents are added to a chunk range which is approaching the maximum chunk size, mongos will trigger a chunk split which will try to create smaller logical chunks for balancing.

Currently all data in your sharded testDB.testdata collection is covered by a single chunk on shard0000 (aka your Shard1 server):

    testDB.testData
        shard key: { "name" : 1 }
        chunks:
            shard0000   1
        { "name" : { "$minKey" : 1 } } -->> { "name" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)

Using Object.bsonsize() in the mongo shell to determine document size, your sample document in the question is about 43 bytes in BSON. With only 10,000 documents you would not be close to reaching the 64 megabytes of data required to trigger any chunk splits. Something closer to 1.5 million documents of that size would be required.

You will also need to ensure the values for your chosen shard key have reasonable uniqueness. A poor shard key (i.e. with many common values) can lead to indivisible jumbo chunks that cannot be further split.

If you just want to test how sharding works you could:

insert larger (or more) test documents
lower the chunk size from the default of 64MB
pre-split an empty collection before inserting data

MongoDB with Sharding has slow performance

First of all I heavily doubt that you config servers build a replica set. By default, config servers are three single instances, which is important to find metadata deviations. If you set up a replica set manually, this would be a single config server. And tbh, I am not even sure wether a mongos would actually switch to a secondary in a failover situation if given a replica set.

Now for the performance gain. All you mongods (at least your shard) run on localhost. And you only have a single shard. Sharding as a load distribution tool only works when the load is distributed across multiple machines. In your setup, your various mongods (6, if I am not mistaken) will battle for resources. Depending on your write concern, for example, the two secondaries may battle for disk IO when syncing data from the primary. If you have WT as your storage engine, the situation gets even more severe, as WT alone will eat up to 50% of your RAM...

I am not sure what you are trying to achieve, but if it is for testing a sharded setup, your test plan is very improvable.

Best Answer

Related Solutions

MongoDB sharding

MongoDB with Sharding has slow performance

Related Question