Re: [ckrm-tech] [RFC 3/5] sched: Add CPU rate hard caps

[Chandra Seetharaman <sekharan@us.ibm.com>]

On Thu, 2006-06-01 at 14:04 +0530, Balbir Singh wrote:
> Hi, Kirill,
> 
> Kirill Korotaev wrote:
> >> Do you have any documented requirements for container resource 
> >> management?
> >> Is there a minimum list of features and nice to have features for 
> >> containers
> >> as far as resource management is concerned?
> > 
> > Sure! You can check OpenVZ project (http://openvz.org) for example of 
> > required resource management. BTW, I must agree with other people here 
> > who noticed that per-process resource management is really useless and 
> > hard to use :(
> 

I totally agree.
> I'll take a look at the references. I agree with you that it will be useful
> to have resource management for a group of tasks.
> 
> > 
> > Briefly about required resource management:
> > 1) CPU:
> > - fairness (i.e. prioritization of containers). For this we use SFQ like 
> > fair cpu scheduler with virtual cpus (runqueues). Linux-vserver uses 
> > tocken bucket algorithm. I can provide more details on this if you are 
> > interested.
> 
> Yes, any information or pointers to them will be very useful.
> 
> > - cpu limits (soft, hard). OpenVZ provides only hard cpu limits. For 
> > this we account the time in cycles. And after some credit is used do 
> > delay of container execution. We use cycles as our experiments show that 
> > statistical algorithms work poorly on some patterns :(
> > - cpu guarantees. I'm not sure any of solutions provide this yet.
> 
> ckrm has a solution to provide cpu guarantees. 
> 
> I think as far as CPU resource management is concerned (limits or guarantees),
> there are common problems to be solved, for example
> 
> 1. Tracking when a limit or a gaurantee is not met
> 2. Taking a decision to cap the group
> 3. Selecting the next task to execute (keeping O(1) in mind)
> 
> For the existing resource controller in OpenVZ I would be
> interested in the information on the kinds of patterns it does not
> perform well on and the patterns it performs well on.
> 
> > 
> > 2) disk:
> > - overall disk quota for container
> > - per-user/group quotas inside container
> > 
> > in OpenVZ we wrote a 2level disk quota which works on disk subtrees. 
> > vserver imho uses 1 partition per container approach.
> > 
> > - disk I/O bandwidth:
> > we started to use CFQv2, but it is quite poor in this regard. First, it 
> > doesn't prioritizes writes and async disk operations :( And even for 
> > sync reads we found some problems we work on now...

CKRM (on e-series) had an implementation based on a modified CFQ
scheduler. Shailabh is currently working on porting that controller to
f-series.

> > 
> > 3) memory and other resources.
> > - memory
> > - files
> > - signals and so on and so on.
> > For example, in OpenVZ we have user resource beancounters (original 
> > author is Alan Cox), which account the following set of parameters:
> > kernel memory (vmas, page tables, different structures etc.), dcache 

i started looking at UBC. They provide only max limits, not min
guarantees, right ?
 
> > pinned size, different user pages (locked, physical, private, shared), 
> > number of files, sockets, ptys, signals, network buffers, netfilter 
> > rules etc.
> > 
<snip>
> 
-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan@us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------