Memory – How to Solve Memory Issues Gracefully

cgroupslimitmemoryulimit

I have a standard Linux (Debian testing) laptop, with a swap partition.

I do a lot of experiments with it. Some of them are really memory hungry and the way Linux behaves by default is an issue for me… Let's give a stupid example:

  1. Sit in front of the laptop
  2. Open a terminal
  3. Type python, then a = [0]*100000000

Now chances are high that you won't have enough RAM to handle that big list. Linux will fill the RAM, then the swap and, a couple of minutes later, the OOM killer will be triggered off and kill (almost) random services and hopefully, if you hit Ctrl+C at the good time, python, and if the terminal still had focus, the computer will become responsive again.

I'd like to enforce some memory limits to avoid that unwanted swapping and to refuse to a process the right to allocate more memory than I have (in RAM). If the memory demand is below a certain limit or asked by root, then just kill the most memory hungry process of any user except root.

ulimit -Sv [mem] I hear in the back!

Ho Ho! "Use cgroups via cgexec!" someone says at the first row!

Yes, you are right: these are indeed very good solutions. But:

  • They do not apply system-wide
  • The limits are set per-process
  • The limits are static, disregarding the real amount a free RAM (AFAIK)
  • Here and there, they say these are not really a good solution to enforce hard limits.

What I'd like is that the kernel say: "You belongs to user foo (not root), you use a lot of memory and we're gonna run out of memory. Sorry dude… die now!"

Or: "What the hell are you doing? You need x MB and there is only y MB available. Yes, SWAP is empty, but you don't intend to use the SWAP to do your dirty work, do you? No, I said no! No memory for you! If you insist, you're gonna die!"

Best Answer

Someone suggested in your hear cgroups. Well, try to seek that direction as it can provide you with:

  • applied to a group of task you choose (thus not system wide but neither per process)
  • the limits are set for the group
  • the limits are static
  • they can enforce hard limit on memory and/or memory+swap

Something like that could bring you closer to your goals:

group limited {
  memory {
    memory.limit_in_bytes = 50M;
    memory.memsw.limit_in_bytes = 50M;
  }
}

This tells that the tasks under this cgroup can use at maximum 50M of memory only and 50M of memory+swap, so when the memory is full, it won't swap, but if the memory is not full and some data could be mapped in swap, this could be allowed.

Here is an excerpt from the cgroup's memory documentation:

By using memsw limit, you can avoid system OOM which can be caused by swap shortage.

Related Question