Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

[Peter Williams <pwil3058@bigpond.net.au>]

Simon Derr wrote:
> On Tue, 5 Oct 2004, Paul Jackson wrote:
> 
> 
>>Matthew wrote:
>>
>>>By adding locking and reference counting, and simplifying the way in which
>>>sched_domains are created, linked, unlinked and eventually destroyed we
>>>can use sched_domains as the implementation of cpusets.
>>
>>I'd be inclined to turn this sideways from what you say.
>>
>>Rather, add another couple of properties to cpusets:
>>
>> 1) An isolated flag, that guarantees whatever isolation properties
>>    we agree that schedulers, allocators and resource allocators
>>    require between domains, and
>>
>> 2) For those cpusets which are so isolated, the option to add
>>    links of some form, between that cpuset, and distinct scheduler,
>>    allocator and/or resource domains.
>>
> 
> 
> Just to make sure we speak the same language:
> 
> That would lead to three kinds of cpusets:
> 
> 1-'isolated' cpusets, with maybe a distinct scheduler, allocator and/or 
> resource domains.
> 
> 2-'exclusive' cpusets (maybe with a better name?), that just don't overlap 
> with other cpusets who have the same parent.
> 
> 3-'non-exclusive, non isolated' cpusets, with no restriction of any kind.
> 
> I suppose it would still be possible to create cpusets of type 2 or 3 
> inside a type-1 cpuset. They would be managed by the scheduler of the 
> parent 'isolated' cpuset.
> 
> I was thinking that the top cpuset is a particular case of type-1, but 
> actually no.
> 
> 'isolated' cpusets should probably be at the same level as the top cpuset 
> (who should lose this name, then).
> 
> How should 'isolated' cpusets be created ? Should the top_cpuset be shrunk 
> to free some CPUs so we have room to create a new 'isolated' cpuset ?
> 
> Or should 'isolated' cpusets stay inside the top cpuset, that whould have 
> to schedule its processes outside the 'isolated' cpusets ? Should it then 
> be forbidden to cover the whole system with 'isolated' cpusets ?
> 
> That's a lot of question marks...
> 

I think that this is becoming overly complicated.  I think that you need 
(at most) two types of cpuset: 1. the top level non overlapping type and 
2. possibly overlapping sets within the top level ones. I think that the 
term cpuset should be reserved for the top level ones and some other 
term be coined for the others.  The type 2 ones are really just the 
equivalent of the current affinity mask but with the added constraint 
that it be a (non empty) proper subset of the containing cpuset.

The three types that you've described are then just examples of 
configurations that could be achieved using this model.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce