Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

[Shailabh Nagar <nagar@watson.ibm.com>]

Paul Jackson wrote:

> Martin wrote:
> 
>>but they're still close enough, that especially when programming
>>them in combination, it seems silly to have 2 separate interfaces. 
> 
> 
> The specific attributes needed of CKRM classes are not the same as those
> needed for cpusets.
> 
> The semantics of the two are distinct -- each has different rules that
> have little relevance to the other.


> 
> The typical uses of the two have little overlap.  More often than not,
> the applications that customers want to run in isolation in cpusets are
> not the same as those which customers want to run while sharing compute
> resources with a managed balance.

If you want to emphasize the differences, this might help:  cpusets 
allows apps to be confined to a set for gaining benefits like cache 
affinity and reduced memory latency. CKRM doesn't and cannot and in this 
use case, the two are orthogonal.

But when apps are being confined to a set of cpus *only* for purposes of 
  getting a certain fraction of the total compute power, cpusets are not 
orthogonal in intent, not implementation, from a CKRM CPU class 
implementing hard limits. More capable of achieving those limits, yes, 
but orthogonal, no.

Note that this does not suggest the joint use of the two mechanisms - 
merely that there exists a usage scenario where both are relevant and 
for users of which, a common interface might be handy.


> No, it is not silly to have 2 separate interfaces.  What's silly is to
> presume that everything that seems similar at the 10,000 foot level
> should be combined.
> 
> The details matter.  Show me the synergy.

What's your opinion on the commonalities between the two interfaces 
pointed out in my previous mail ?

Also, if CKRM were to move to the "each controller exports its own 
interface" model, how would this affect the discussion ?


> It is fitting and proper for kernels to provide independent mechanisms,
> and let user space connect them as it will.  


> Look at the actual hooks
> in the kernel code to implement these two facilities....  
> Perhaps the proper place to resolve this discussion in is a detailed
> examination of the kernel hooks required for CKRM and cpusets, the
> hooks in the scheduler, allocator and such.

No one is questioning that the internals differ. There is very little in 
common between a CKRM I/O controller and its CPU controller too. But 
that doesn't prevent them from sharing the same interface.

I repeat - the question isn't one of the internals - its about the 
interface. Do you think there's *any* merit to cpusets sharing the rcfs 
interface *if* the latter were to make the changes mentioned in earlier 
mail ?

If not (and others agree), lets end this discussion and move on - both 
projects have enough to do. If there is some commonality, lets see what 
we can do to enhance the eventual user's experience.

-- Shailabh

> You have both patches available to you.  Examine them.  Especially
> examine the hooks in the scheduler and allocator code.  These are not
> the same hooks.  I defy you to make them the same and propose such with
> a straight face.  If you do so successfully, I will sit up and take
> notice.