Mongodb – Will MongoDB exceed the 64000 ulimit threshold

limitsmongodbmongodb-3.2

We are using MongoDB as a newbie and today we have exceeded the ulimit of the system causing chaos and panic all around.

However I have fixed it by increasing the hard and soft limits like below.

mongouser soft nofile 65535
mongouser hard nofile 65535

From what I understand, these are the limits for opened files.

My question is, will mongodb ever exceed this limit? If so how do I prevent it?
Does MongoDB have a mechanism to open/close files or will they always stay open and the journal files or db files always increase in size or do I have to implement some mechanism for it.

P.S. We are also using MMS to monitor the health of the DB.

Best Answer

Without knowing more about your database and its usage, I can't say for sure, but in general, the high number of open files found in most busy MongoDB instances is caused by connections (one open file per connection) rather than the data files themselves.

What does MMS have to say about your concurrent connection count?

You need to have huge number of databases/collections, and they need to be quite large before you start to hit this purely from data files themselves.

Hence, the usual method to limit how many open files MongoDB will create is to limit the inbound connections. The most common way to do that is to limit/control the number of client connections from the drivers (connection pool size) and in a sharded environment the number of mongos processes is also important. You can usually work out your theoretical maximum for connections to a MongoDB instance if you know the max number of clients, their max connection pool sizes, how many mongos processes you have (if sharded). There are a couple of other minor factors (replication, chunk migrations etc.) but that calculation will get you in the right ballpark.

If fixing the client side turns out to be an issue, then you can look at limiting incoming connections on the mongod too with the net.maxIncomingConnections option. However, that will just mean that rather than ulimit issues you will be rejecting client connections due to a configuration limit.

Related Question