Ubuntu – Memory limiting solutions for greedy applications that can crash OS

memory usageramresource-limiting

I use my computer for scientific programming. It has a healthy 8GB of RAM and 12GB of swap space. Often, as my problems have gotten larger, I exceed all of the available RAM. Rather than crashing (which would be preferred), it seems Ubuntu starts loading everything into swap, including Unity and any open terminals. If I don't catch a run-away program in time, there is nothing I can do but wait – it takes 4-5 minutes to switch to a command prompt eg. Ctrl-Alt-F2 where I can kill the offending process.

Since my own stupidity is out of scope of this forum, how can I prevent Ubuntu from crashing via thrashing when I use up all of the available memory from a single offending program?

At-home experiment*!

Open a terminal, launch python and if you have numpy installed try this:

>>> import numpy
>>> [numpy.zeros((10**4, 10**4)) for _ in xrange(50)]

* Warning: may have adverse effects, monitor the process via iotop or top to kill it in time. If not, I'll see you after your reboot.

Best Answer

The shell built-in ulimit allows you to restrict resources. For your case, to limit memory use in the shell (and its children), use ulimit -v.

Demonstration setting a memory limit of 100 MB (100000 KB):

$ ulimit -v
unlimited
$ python -c '[ "x" * 100000000 ]'
$ ulimit -v 100000
$ python -c '[ "x" * 100000000 ]'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError

It's observed using ps uww -C script-name-here that python requires at least 29MB of memory (VSZ column). The RSS limit grows as your python script needs more memory so adapt that column.

Related Question