Re: [ckrm-tech] [RFC 3/5] sched: Add CPU rate hard caps

[Peter Williams <peterw@aurema.com>]

Chandra Seetharaman wrote:
> On Fri, 2006-06-02 at 13:21 +1000, Peter Williams wrote:
>> Chandra Seetharaman wrote:
>>> On Fri, 2006-06-02 at 09:26 +1000, Peter Williams wrote:
>>>> Chandra Seetharaman wrote:
>>>>> On Thu, 2006-06-01 at 14:04 +0530, Balbir Singh wrote:
>>>>>> Hi, Kirill,
>>>>>>
>>>>>> Kirill Korotaev wrote:
>>>>>>>> Do you have any documented requirements for container resource 
>>>>>>>> management?
>>>>>>>> Is there a minimum list of features and nice to have features for 
>>>>>>>> containers
>>>>>>>> as far as resource management is concerned?
>>>>>>> Sure! You can check OpenVZ project (http://openvz.org) for example of 
>>>>>>> required resource management. BTW, I must agree with other people here 
>>>>>>> who noticed that per-process resource management is really useless and 
>>>>>>> hard to use :(
>>>>> I totally agree.
>>>>>> I'll take a look at the references. I agree with you that it will be useful
>>>>>> to have resource management for a group of tasks.
>>>> But you don't need something as complex as CKRM either.  This capping
>>> All CKRM^W Resource Groups does is to group unrelated/related tasks to a
>>> group and apply resource limits. 
>>>
>>>>  
>>>> functionality coupled with (the lamented) PAGG patches (should have been 
>>>> called TAGG for "task aggregation" instead of PAGG for "process 
>>>> aggregation") would allow you to implement a kernel module that could 
>>>> apply caps to arbitrary groups of tasks.
>>> I do not follow how PAGG + this cap feature can be used to put cap of
>>> related/unrelated tasks. Can you provide little more explanation,
>>> please.
>> I would have thought it was fairly obvious.  PAGG supplies the task 
>> aggregation mechanism, these patches provide per task caps and all 
>> that's needed is the code that marries the two.
> 
> May be obvious from your usage point of view. It wasn't for what i was
> thinking as resource management.

I was thinking of it from the resource management POV.

> 
> I thought there is some way the user can associate some amount of
> resources (limits and guarantees) to a PAGG group and move the
> corresponding tasks to that PAGG and that is all needed from user
> space. 

No.  PAGG just provides the infrastructure for grouping tasks together 
and adding any extra per task data that you may require.  Of course, you 
can provide your own per group data as well.  It's then up to the author 
of the module utilizing PAGG to implement whatever group specific 
functionality that they want.

> 
> In other words i thought there is some clever way to manage resources at
> the PAGG level (without needing to tinker with the per task caps), which
> wasn't obvious for me, and it is now clear that is not the case, and one
> still have to keep tweaking the "per task" caps to get the result they
> want. 
> 
>>From your explanation, complex stuff need to happen in the user space to
> manage resource for a group of tasks.
> 
> Knobs that are available to the user are
>  - per task nice values
>  - per task cap limits and
>  - per task statistics, if and when they become available.
> 
> user level application has to constantly monitor the stats of _all_ the
> tasks and constantly keep changing the knobs if they want to keep the
> "group of tasks" within their guarantees and limits. As others pointed
> already, this may still _not_ yield what one wants, if you have tasks
> with disparate need for a resource.

I think that it can especially now that load balancing takes "nice" into 
account.  Without the smpnice patches it would have been difficult due 
to the effects of "soft" affinity tending to undermine "nice"'s 
functionality.

> 
> I certainly do not see it as the result of a simple marriage between
> PAGG and "per task caps".

It's simple but there's a non trivial amount of work required.

I know from previous discussion with CKRM folks that you find the idea 
of providing a number of small independent capabilities that enable more 
complex capabilities to be built on top of them difficult to come to 
terms with.  But that doesn't mean it won't work.  It has the advantage 
of being less intrusive and causing less angst to those users who don't 
want sophisticated resource control.

Peter
-- 
Dr Peter Williams, Chief Scientist         <peterw@aurema.com>
Aurema Pty Limited
Level 2, 130 Elizabeth St, Sydney, NSW 2000, Australia
Tel:+61 2 9698 2322  Fax:+61 2 9699 9174 http://www.aurema.com