Mongodb – Multiple MongoDB databases (1 per client) and sharding – are they compatible

mongodbmulti-tenantsharding

Currently I'm working with one MongoDB database per client, on a single replica set. We would like to keep one database per client because of security concerns.

Each client has the same 15 collections and we are growing our customer base so we need to scale this into different machines with groups of customers.

I'm reading the MongoDB docs about sharding but the approach is by collection and the shard key so I'm concerned about how it would be when you have many databases.

Is it possible to use sharding having multiple databases? and if it is possible, what would be a good approach?

Best Answer

Yes, they are compatible: if you have multiple shards, then different databases will be automatically allocated to different shards in a balanced way.

The sharding system does distribute large collections across shards by using a shard key, but it also distributes separate databases across different shards. The shards documentation explains:

Each database in a sharded cluster has a primary shard that holds all the un-sharded collections for that database. Each database has its own primary shard. The primary shard has no relation to the primary in a replica set.
The mongos selects the primary shard when creating a new database by picking the shard in the cluster that has the least amount of data.

You can use that to let each client's database be allocated to different shards, as follows:

  1. Set up multiple shards (perhaps 3 for now)
  2. When you add a new database for a new client, mongos will allocate the database to whichever shard has least data
  3. This has the effect of 'balancing' the database among the available shards

A few warnings about this approach:

  1. When you first add new shards, mongos does not automatically move databases from one shard to another to balance the system. If you need to do that, you will have to do so deliberately using the movePrimary command
  2. If your databases grow, again you might need databases moved from one shard to another to give them enough space; again, you will have to do this yourself using the movePrimary command
  3. If a single collection in one of your databases is getting too large for a single shard, you will have to manually enable sharding for that collection; then mongos will do the allocation of chunks to different shards, automatically.