Once you add a user to the individual shards, which you indicate you have done for MMS, you must then have valid credentials to connect for any purpose, including mongodump. Up until you added that user for MMS, the shards were running with authentication enabled but with no users populated (this only happens if all your users are in the admin database ans using delegated auth for other databases, otherwise with 2.4 and below you would have at least one shard with users for each database - 2.6+ changes this behavior) and so you were able to connect without credentials.
Essentially this is a loophole left open so that you don't accidentally lock yourself out of your instances when you turn on auth with no users (and one that would probably have stopped working at some point as default security is tightened anyway).
The bottom line is that you will need to add a user for use with mongodump, and it's a good idea to do so anyway rather than allowing non-authenticated users free access to your instances. If you are running 2.6 or later, then the built in backup role exists precisely for this purpose, if you are on 2.4 or earlier, then the description for that role gives you a great outline of what is needed to backup successfully (and in particular if you want to backup the users themselves).
So even if you have 1M users, only 10 articles must be kept? I have a strong feeling that your data model has this or that flaw.
Does sharding a collection with 10 documents (give and take) make sense?
If you only need 10 documents in a collection, you don't need to shard it, since it is not going to be balanced anyway, unless the documents are exceeding around 4.8Mb in size. This size computes like this:
- maxChunkSize/number of your documents is the size pre document when you chunk gets split latest, equalling 6.4Mb per document
- In order to cross the migration threshold, which is only 2 for collections with less than 20 documents, we need to have your chunk split at least once to trigger the migration
- Since the chunk split may be already triggered when the chunk reaches half of it's max size resulting in a split when all documents are slightly over 3.2Mb, we add the median of the difference between half of the chunks max size and and it's max size (silently assuming that a chunk is guaranteed to be split at it's max size)
But sharding this collection would not make sense the first place, as , assuming the 10 documents are a hard limit, the collection's max size can only be 160Mb as per MongoDB's BSON size limit
Do I need to have my data balanced?
Let us find out wether it is a good idea to have a bad shard key and a disabled balancer. First, if you disable the cluster balancer, this affects all sharded collections. Let us take the user collection as an example:
- We have 1M users
- Their user id is stored in
_id, starts with 1 and each new user's _id is a simple increment
- We have two shards
- We disable the balancer.
Now, you shard the collection from the start. What happens internally is that there are two chunks created. In the first chunk, created on the first shard, the _id from -∞ to _id < 0 are stored. In the second chunk, created on the second shard, the _ids from 0 to +∞ are stored. Now here comes the thing: since our _id increments from 1 for each user, there is never a single user stored in the first chunk and subsequently not on the first shard. No disk space of the first chunks utilized and – with more immediate importance – no RAM, too. Since indices are (tried to be) kept in RAM along with the recently used data (called working set) amongst other things to speed operations up, sooner or later we will have the situation that the RAM on the first shard is rather empty, while the second shard will start to evict data set items or even indices out of RAM. Bad idea, huh? Now things get worse: Since we have disabled the balancer, the cluster can do nothing to mitigate the situation.
Now let's assume we were slightly smarter and have pre-split our chunks so that the _id for our users collection are distributed in a way where the _ids ranging from -∞ to 500,000 are stored on the first shard and the rest on the second. It is obvious that this is only a temporary solution, since when we exceed 1M users, the whole problem starts again. And without the balancer running, the cluster still can not mitigate this situation.
Taking this a step further: We have found out that we can use a hash sum of our _id as our shard key. Jay! Problem solved! Except it isn't.
In theory, the hash algorithm should cause our users to be evenly distributed among our shards. But there is a little thing called variance.
It can be easily demonstrated like this (heavily simplified for the sake of shortness): When tossing a coin 20 times, in theory you should get an equal number of heads and tails, since the probability of either of them is 50% right? Try it. Now. The odds are very low that you will have an equal amount of heads and tails.
How does this translate to our problem? Well, chances are high that either the first or the second shard get's slightly more documents than the other over time. And this sooner or later will add up to a point where it becomes a problem – RAM and disk space is underutilized on one shard and over utilized on the other. Again. And again, the balancer can not help us to mitigate the problem.
So this should have made it crytsal clear that you should have your data balanced and that you should never have the balancer disabled by default (there are some administrative tasks during which you should or must have the balancer disabled).
Conclusion
For some 10 documents, no matter how big they are, you don't even need to shard the collection.
Disabling the balancer might cause severe problems with your cluster. Unless you absolutely have to or you are absolutely positively sure that you can live with the consequences, do not disable it.
Please note that I left out some more complicated topics, like hard drive IO bandwidth bottlenecks, network bandwidth distribution and alike for the sake of readability and – as funny as this might sound – shortness.
Best Answer
The problem explained
As per your comment, your shard key is the
_idfield of the document. This field is monotonically increasing, basically like an incremented integer.Put simply, sharding works this way: documents are stored in chunks. Those chunks are spread over the cluster based on ranges of the shard key. Let's look at a simple example:
And here is where the problem occurs: there is one shard were all new documents go to. What happens then is that new chunks are created all the time (as chunks have fixed size). So not only are all writes focussed on one server, this server also has to do some overhead work. After a certain threshold is met, the cluster will start to move chunks to the other shards, which only will balance out disk space, though. In out example, it might look like this:
Obviously, all write operations will still go to s3.
What went wrong?
You chose the wrong shard key. Monotonically increasing shard keys lead to the problem explained above. Altough
ObjectIds look like hash sums, but they aren't. They are monotonically increasing.What can be done?
You need a better shard key. Since a shard key can not be changed after a collection is sharded, migrating to a new shard key is not a simple thing to do. However, there is a quite detailed explanation in mongoDB's ticket system. Basically, it works like this:
mongosinstance