MongoDB Maxing CPU Usage

mongodbmongodb-3.4wiredtiger

I have a weird one here. We have set up our production MongoDB which was populated by our test one so we had something to work off of. As soon as we started the service we hit instant 100% CPU load with nothing running, so we removed the server from it's replica set and got it down to 2-3% CPU.

When we had all of our users active we saw it spike back up to 100% and we said we would leave it over night and monitor it for any signs of improvement.

The next morning we came in and the CPU was down to around 40% and it seemed very stable. Now about a week later we have come across the same issue as above except this has been happening for almost 4 days where we are maxed on CPU.

I used the db.currentOP() command and found one from "WT RecordStoreThread: local.oplog.rs" that has been running for 355254 seconds which equated to around 98 hours which matches the CPU being maxed.

The machine has 8 cores @ 2.4GHz and 16GB RAM so I wouldn't think that CPU would be this massive an issue at such an early stage.

Our config file looks like the following with the IP and Port stared out:

systemLog:
   destination: file
   path: E:\data\log\mongod.log
storage:
   dbPath: E:\data\db
net:
   port: xxxxx
   bindIp: xxx.xxx.xxx.xxx
#replication:
#   replSetName: "rs1"
security:
   authorization: enabled
#   keyFile: E:\data\mongodb.key

I have restarted the service today hoping to maybe release some resources but it instantly maxed out again.

I have looked at other issues on SO and none seem to have a real answer to resolve this problem. Thanks for taking the time to read this and if anybody has come across the same issue or might have any idea on how to solve it that would be great to get your feedback!

Best Answer

Strange indeed.

That local.oplog.rs is journaling what is enable (default) at 64 bit systems (32bit default value is false). So, running time for that thread should be same than your mongod uptime.

But journaling should not rise CPU usage. When you have high CPU usage, what mongotop and mongostat shows? What is happening in the system?

If you look result of top-command, "who" is eating most CPU power (one or more mongod processes?) and what is highest %-value us(er), sy(stem), ni(ce), id(le), wa(it)?

Related Question