I've been running a process (gdb) that can easily use up all my ram and swap space. Every time that happens the whole system becomes very non-responsive and Xorg will barely update, I can't swith to another tty I can't type commands in the console etc.
I've tried to setting aside memory with:
sudo sysctl -w vm.min_free_kbytes=407580
sudo sysctl -w vm.swappiness=5
Some of the virtual memory was spared but the swap space was used up and then the system still became non-responsive.
Does anyone have any foolproof ways to prevent the system from freezing up completely? So that I can kill memory hogging processes?
Update:
I found a way to limit memory usage per process launched from a particular shell with:
ulimit -v 2037900
This caps each process to 2037900 KiB (Which is my total memory) of virtual memory. Which is more than any one process should use. So far this seems to prevent the system from slowing down and freezing.
I would still like to know if anyone knows of any good system resource management practices.
Best Answer
Look into the Linux Out Of Memory Killer. Doesn't it kick in automatically on your server? It has on mine.
http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html