MongoDB – Thread Over Memory Limit

linuxmemorymongodb

What does it mean?

2017-04-28T04:37:53.126+0000 D – [conn1571] thread over memory limit, cleaning up, current: 67k
2017-04-28T04:37:53.911+0000 D – [conn1585] thread over memory limit, cleaning up, current: 79k

Should I make something to solve it? If I increas server memory will it help?

Best Answer

This is from source code:

// If many clients are used, the per-thread caches become smaller and chances of
// rebalancing of free space during critical sections increases. In such situations,
// it is better to release memory when it is likely the thread will be blocked for
// a long time.

And when numOpenSessions is over 40 and threadCacheSizeBytes (GetThreadCacheSize()) goes under 0x10000, system frees memory.

So, to "prevent" this, you should have less (concurrent) open sessions OR more (cache) memory. You can define (wiredTiger) bigger (than default) cache size, with current memory. But, yes, it's better give more memory than define bigger cache size, because when mongodb have more memory, cache size is automatically bigger.

Related Solutions

MySQL Memory Grows Over Time

The formulas that tuning scripts like mysqltuner use to estimate potential memory usage are bunk. These posts explain why and give some examples of things that could be causing the memory usage pattern you're seeing:

How much memory can MySQL use in the worst case?

MySQL Server Memory Usage

This post talks about isolating the problem by running tests:

Troubleshooting MySQL Memory Usage

The simplest explanation is that there is a query with large memory overhead, which should be identifiable by running the query in isolation and observing its impact on memory usage.

As the post says, memory usage issues are one of the more annoying problems to have in MySQL, and the lack of instrumentation features that other RDBMS have really shows here and makes this type of problem difficult to diagnose.

Oracle Memory Configuration – How to Limit Oracle’s Memory Use

The problem was that when the database was installed, conflicting other parameters were set. So I needed to export those from the "spfile" to a "pfile", back them up, edit them, test them and import from the pfile to the spfile:

SQL> create pfile='some/file/path' from spfile;

File created.

Then I set the conflicting parameters to zero:

...
--*.pga_aggregate_target=3270m
*.pga_aggregate_target=0m
...
--*.sga_target=9811m
*.sga_target=0m
...

I then try starting up again:

SQL> startup pfile='some/file/path';

ORACLE instance started.
...

And check the memory_target and memory_max_target parameters as above. Once happy with things, to keep the settings I did:

SQL> create spfile from pfile='some/file/path';

File created.

Best Answer

Related Solutions

MySQL Memory Grows Over Time

Oracle Memory Configuration – How to Limit Oracle’s Memory Use

Related Question