I use KVM to manage my virtual machines. I am trying to limit the resources provided for VMs. I am able to limit the CPU and memory allocated for VMs using libvirt/cgroups. Now I want to control the disk time allotted for each VM by applying some weights. I looked at blkio control from cgroups. Since VM is just a Linux process, I will be able to use cgroups but I am not sure whether it will work for asynchronous IO too. If not, can someone suggest a good alternative solution?
Linux – Weight-based block IO control for VMs
block-devicecgroupsiokvmlinux
Related Solutions
As far as I'm concerned, I think cgroups would be overkill here. However, I tend to use ulimit whenever I run something witk a fork system call in it (bad experiences made it a habit...) :
$ ulimit -u 2500
$ ./mypotentiallydeadlyprogram
This way, I put a 2500 processes limit on my current shell. Thanks to this, my fork calls will end up failing if they get too numerous, hence preventing the system from going down, and allowing me to furiously hit Ctrl + C.
On my machine, I find 2500 to be a good limit, but you might want to increase/decrease this value according to what your machine can take, and how far you want your fork bomb to go. Also remember that your machine needs to spawn things to survive, don't suffocate it. I have seen people writing this in their ~/.bashrc, therefore restricting even their session's main bash. While this was very funny to the sysadmin, the user was very unhappy to freeze after login.
While ulimit can be used to set up a temporary limit, you can set something more permanent if you have root access (and want to enforce the limit on specific users). This can be done through /etc/security/limits.conf:
# <domain> <type> <item> <value>
youruser soft nproc 2500
youruser hard nproc 2750
In the above setup, youruser has got a soft limit of 2500 processes (max. 2750). This file allows you to set up various kinds of limits, for various entities on your system (users, groups, ...). Have a look at its documentation if you need more information. Note however that this is system-wide configuration, which means that this limit isn't applied per-shell for youruser.
By the way, /proc/sys/kernel/pid_max will contain the maximum PID which can be granted by your kernel. Since PIDs are reusable, you may consider this really close from your maximum number of processes.
Yes, you can use cgroups and SELinux/AppArmor exclusively to monitor and control the arbitrary code that you will execute.
With cgroups, you can do the following:
- Limit CPU core usage to 1 CPU with the
cpusetsubsystem - Set memory usage limits with the
memorysubsystem, tracking even the forks. See https://github.com/gsauthof/cgmemtime for an example. - Prevent network access to anything that isn't on
lowithnet_priosubsystem.
And with SELinux/AppArmor, you can limit the process's read/write access.
Note: I am unfamiliar with AppArmor, but it is a Mandatory Access Control (MAC) system, meaning that guarding writing and reading is it's job.
Using these systems is a matter of the writing the proper configurations. Of course, all this is much easier said than done. So here are a few reference links to get you started:
Good Luck!
Best Answer
Blkio in cgroup terminology stands for access to I/O on block devices. It does not seem to be about regulating all the different ways software developers have at hand for I/O-related purposes.
It seems to be targeted mainly to I/O on devices, not on the way software has access to devices. It can limit the number of iops, the bandwidth or a weight with other processes, in other things. It seems that buffered write is not supported by blockio at the moment. It's in the official documentation:
If you take a look at this presentation from Linda Wang and Bob Kozdemba of Red Hat, on page 20+, you'll see that the graph is about the device bandwidth per VM, not about random vs blocking vs asynchronous I/O.
It seems there has been recent work by Red Hat to implement it directly into virsh. It has been released last week in libvirt 0.9.9. In a few months, you'll be able to do something like this in your favorite distribution: