MongoDB in micro-services structure

dockermongodb

I'm building a game that have micro-services structure using docker.

The way i started to build my services is having a mongoDB instance to each service, so having OneToOne service to mongoDB instance, 

Resources ---> resourcesDB (mongoDB instance 1)
Chat      ---> chatDB      (mongoDB instance 2)
Buildings ---> buildingsDB (mongoDB instance 3)
Battle    ---> battleDB    (mongoDB instance 4)
...

This really made sense at the beginning as those services for example have nothing in common and don't need have a connection to the same mongo instance, and when they do have something in common (building <-> resources) they just send a call to the needed service and that handles the rest (update his DB).

But i starting to think that this is not the right approach because of couple of things:

1 – resourcesDB for example have only 2 collections,

    Logs      ---> logging every resources change (building built, war lost etc.) 
    Resources ---> 1 document per user holding how much he have.

meaning that this DB will not hold any huge data (logging will be deleted after some time), so setting an instance just for those 2 collections is looking like a huge over kill, even if this instance will not have its own server and will share server with the service.

2 – maintaining and doing backups will require some extra work that I'm not sure that its needed in my case.

Each of the service will update his DB quite often, resource is doing calculation every second to see how much each user have, so it will update the db 'users amount' per second, aside with the rest of the requests like build a building (reduce resources) and other updates,
Chat service will update the DB every private message or chat message, Battle service will update troops place and stats and more stuff every second for each battle that is going on.

I'm thinking to chance that structure to one of the next:

  1. maybe have only 1 instance of database with separated DB, so each service will connect to a separated database, but i read that it will still be a hassle to maintain backups of the DB's and scaling.

  2. have 1 database, and have all the collections in it, every collection will be accessed by a different service and will never be accessed by 2, so i have nothing to worry about 2 service changing same collection.

  3. leave it as it is, and go with many instances of mongoDB

few questions about the methods above:

  1. what should i choose? having 1 database sounds good as it will save me a lot of resources and will make my life easier, but separating the services to many database sound logic to me as one service don't need to see collections or be able to access a collection that have nothing to do with his job.

  2. I'm not sure how mongoDB handle many connections to the same instance, or to same database, as you can see it might get a lot of calls in a second even if i have pretty low amount of users.

  3. I read about mongo Sharding, from what i understood it happens per database and not on the instance level, that's why i'm thinking about the 1 database option, as it quite useful in my case, am i right?

Best Answer

It's sensible to think about how to scale your application in future, but you can start with a simpler deployment that can grow with your requirements.

what should i choose? having 1 database sounds good as it will save me a lot of resources and will make my life easier, but separating the services to many database sound logic to me as one service don't need to see collections or be able to access a collection that have nothing to do with his job.

My recommendation would be to set up a single MongoDB deployment (ideally starting with a replica set or hosted service with high availability & data redundancy):

  • A single deployment of MongoDB can have multiple databases; a MongoDB deployment per service adds unnecessary admin complexity.

  • For your use case it sounds reasonable to have a database per service and setup a user per service so actions are limited to direct interaction with the expected database.

  • A database per service also limits potential impact of administrative actions (eg. a foreground index build) that may create database-level contention.

Later you can scale/tune the deployment starting with options like directoryPerDB (eg. faster storage for some databases) or partitioning your data using sharding.

I'm not sure how mongoDB handle many connections to the same instance, or to same database, as you can see it might get a lot of calls in a second even if i have pretty low amount of users.

This is a more nuanced question as it depends on server resources as well as what those connections are doing and how/if your driver implements connection pooling. Ideally I would load test your actual application and determine how resource usage scales with increased user activity. If you are administering your own MongoDB deployment you should review the production notes and in particular make sure you have your ulimit settings increased appropriately to ensure you don't run out of file descriptors.

If you're concerned about ease of scaling you could also consider a hosted database-as-a-service solution (for example, MongoDB Atlas).

I read about mongo Sharding, from what i understood it happens per database and not on the instance level, that's why i'm thinking about the 1 database option, as it quite useful in my case, am i right?

Sharding is initially enabled at the database level but the partitioning happens at the collection level based on a shard key. You can have a mix of sharded/unsharded databases and sharded/unsharded collections in the same MongoDB deployment.

Related Question