Re: [RFC, PATCH 0/5] Going forward with Resource Management - A cpu controller

[Kirill Korotaev <dev@sw.ru>]

>>I think the risk is that OpenVZ has all the controls and resource
>>managers we need, while CKRM is still more research-ish. I find the
>>OpenVZ code much clearer, cleaner and complete at the moment, although
>>also much more conservative in its approach to solving problems.
> 
> 
> I think it would be nice to compare first the features provided by ckrm and 
> openvz at some point and agree upon the minimum common features we need to have 
> as we go forward. For instance I think Openvz assumes that tasks do
> not need to move between containers (task-groups), whereas ckrm provides this
> flexibility for workload management. This may have some effect on the 
> controller/interface design, no?

BTW, to help to compare (as you noted above) here is the list of features provided by OpenVZ:

Memory and some other resources related to mem
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- kernel memory. vmas, LDT, page tables, poll, select, ipc undos and many other kernel
  structures which can be created on user requests.
  without it's accounting/limiting a system is DoS'able.

user memory (private memory, shared memory, tmpfs, swap):
- locked pages
- shmpages
- physpages. accounting only. Correctly accounts fractions of memory
  shared between containers. Can't be limited in a user friendly manner,
  since memory denials from page faults are not handled from user space :/
- private memory pages. These are private pages which has are not backed up
  in the file or swap and which are pure user pages. These are anonymous
  private mappings and cow-able mappings (e.g. glibc .data) which result in private memory.
  Accounted correctly taking into acount sharing between containers (i.e. page
  fraction is accounted).
  This resource is limited on mmap() call.

others:
- 2-level OOM killer. The most fat container should be selected to kill first.
  We introduce some guarantee against OOM, so that if the container
  consumes less memory than it is guaranteed to, then it won't be killed.
- memory pinned by dcache (there is a simple DoS which can be done
  by any Linux user to consume the whole normal zone)
- number of iptables entries (with virtualized networking
  containers can allocate memory for iptable rules)
- other socket buffers (unix, netlinks)
- TCP rcv/snd buffers
- UDP rcv buffers
- number of TCP sockets
- number of unix/netlink/other sockets
- number of flocks
- number of ptys
- number of siginfo's
- number of files
- number of tasks

CPU management
~~~~~~~~~~~~~~
1. 2 level fair CPU scheduler with known theoretical fairness and latency bounds:
- 1st level selects a container to run based on the container weight
- 2nd level selects a runqueue in the container and a task in the runqueue

2. cpu limits. Limitation of the container to some CPU rate even if CPUs are idle.


2 level disk quota
~~~~~~~~~~~~~~~~~~
allows to limit directory subtree to some amount of disk space.
inside this quota std linux per-user quotas are available.

Thanks,
Kirill