Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

Peter Williams
2004-10-06T21:58:45Z

Simon Derr wrote:
> On Tue, 5 Oct 2004, Paul Jackson wrote:
>
>
>>Matthew wrote:
>>
>>>By adding locking and reference counting, and simplifying the way in which
>>>sched_domains are created, linked, unlinked and eventually destroyed we
>>>can use sched_domains as the implementation of cpusets.
>>
>>I'd be inclined to turn this sideways from what you say.
>>
>>Rather, add another couple of properties to cpusets:
>>
>> 1) An isolated flag, that guarantees whatever isolation properties
>> we agree that schedulers, allocators and resource allocators
>> require between domains, and
>>
>> 2) For those cpusets which are so isolated, the option to add
>> links of some form, between that cpuset, and distinct scheduler,
>> allocator and/or resource domains.
>>
>
>
> Just to make sure we speak the same language:
>
> That would lead to three kinds of cpusets:
>
> 1-'isolated' cpusets, with maybe a distinct scheduler, allocator and/or
> resource domains.
>
> 2-'exclusive' cpusets (maybe with a better name?), that just don't overlap
> with other cpusets who have the same parent.
>
> 3-'non-exclusive, non isolated' cpusets, with no restriction of any kind.
>
> I suppose it would still be possible to create cpusets of type 2 or 3
> inside a type-1 cpuset. They would be managed by the scheduler of the
> parent 'isolated' cpuset.
>
> I was thinking that the top cpuset is a particular case of type-1, but
> actually no.
>
> 'isolated' cpusets should probably be at the same level as the top cpuset
> (who should lose this name, then).
>
> How should 'isolated' cpusets be created ? Should the top_cpuset be shrunk
> to free some CPUs so we have room to create a new 'isolated' cpuset ?
>
> Or should 'isolated' cpusets stay inside the top cpuset, that whould have
> to schedule its processes outside the 'isolated' cpusets ? Should it then
> be forbidden to cover the whole system with 'isolated' cpusets ?
>
> That's a lot of question marks...
>

I think that this is becoming overly complicated. I think that you need
(at most) two types of cpuset: 1. the top level non overlapping type and
2. possibly overlapping sets within the top level ones. I think that the
term cpuset should be reserved for the top level ones and some other
term be coined for the others. The type 2 ones are really just the
equivalent of the current affinity mask but with the added constraint
that it be a (non empty) proper subset of the containing cpuset.

The three types that you've described are then just examples of
configurations that could be achieved using this model.

Peter
--
Peter Williams pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
-- Ambrose Bierce