MongoDB Scalability – Scaling Out on the Same Storage

mongodbscalability

We are planning to have some tests to predict how mongo db operates and functions with 2 terabyte of inforamtion on a single collection.

So the plan is to have 15-20 sharded machines (on the same environment / cabinet).
All are considering to have shared EMC storage

I wanted to ask before we start the test :

IS there any different between:
* Having physical machines writing to independent disks (on the machine).
* Having physical machines writing to the same storage.
* Having virtual machines writing to same storage.
About memory consumption, What are the hardware considerations about querying this amount of data? If I understand correctly, the advantage when you have massive writes is to do sharding with small-level servers, but is it going to change when you need to have some reads?

Thanks

Best Answer

If you have multiple databases, writing to independent disks (with the directory per db) option will be your best bet for performance. Even if you just have one big huge sharded database, you can still get some performance boost with multiple disks by putting your journal and local database (database where replication info goes) on separate disks, but your main database data files would all have to be on a single disk.
Not sure what you're after here. The recommendation is that your working set fit in memory for best performance and that still applies for sharding, its just split across multiple servers and thus your memory requirements are also split up across those servers. That will of course also depend on how well you pick your sharding key, hopefully the load is distributed evenly across all the nodes. If you plan on returning lots of data from your queries then you might need extra memory on your mongo-s servers as they have to do some sorting and aggregation of what you're bringing back.

Related Solutions

Mongodb – How to automate sharding in the presence of continuous reads

Some thoughts on each of the options:

I would say that writing this code in any case is probably a good idea - if something does go wrong on the performance side of things you already have the code written to deal with it in some way, rather than just sending exceptions or timeout messages back to the users.
This sounds like pre-splitting and pre-allocation, which again, is probably a good idea to do anyway - why wait until just before use to create a database? In fact, this would mirror how mongodb does data file allocation, it always has one empty data file waiting to be filled before it is needed. The pre-allocation it does is for adding data to a DB though, it can't psychically figure out the need for a new DB, hence that initial allocation may be slowing you down. Also, make sure you are on ext4/XFS and not using an old kernel to ensure that the allocation is as quick as possible.
I believe that doing pre-allocation is a better option here - I have not heard of anyone using copyDatabase in this manner, so it would require a lot of testing. Pre-allocating databases in advance though is quite common.
This does sound a bit painful, I think there are better options described above

Overall, if you know you are going to need a new database every hour, create multiple databases in advance of their need. I know this would mean extra disk usage, but it should be quite predictable (look into the --smallfiles option if your data set is quite small).

Mongodb – Improving concurrency in MongoDB via sharding

You are correct, with MongoDB the way to engineer around a write contention issue is to shard.

Your environment sounds like it's fairly bursty, in that you're not continually ingesting data and instead ingesting it in fairly discrete chunks. With this in mind, you could go with a collector/distributor model such as this:

     ------>{ workers }<------
     |                       |
[Shard01]               [Shard02]
    |                      |
    ---->[Persistant]<------

The workers would upsert/push their results into the sharded collection/database, and once the job was completed a batch-process then uses something like db.copyDatabases() to push the result set at the monolithic (and cheaper to run) single instance Mongo. As the copy database process can push all of its updates in one run, it should experience much fewer write-lock problems.

Best Answer

Related Solutions

Mongodb – How to automate sharding in the presence of continuous reads

Mongodb – Improving concurrency in MongoDB via sharding

Related Question