Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

[Nick Piggin <nickpiggin@yahoo.com.au>]

Matthew Dobson wrote:
> On Thu, 2004-10-07 at 01:51, Paul Jackson wrote:
> 
>>>I don't see what non-exclusive cpusets buys us.
>>
>>One can nest them, overlap them, and duplicate them ;)
> 
> 
> <snip example>
> 
>>We now have several nested cpusets, each overlapping its ancestors,
>>with tasks in each cpuset.
>>
>>But only the top hpcarena cpuset has the exclusive ownership
>>with no form of overlap of everything in its subtree that
>>something like a distinct scheduler domain wants.
>>
>>Hopefully the above is not what you meant by "little more than a
>>convenient way to group tasks."
> 
> 
> I think this example is easily achievable with the sched_domains
> modifications I am proposing.  You can still create your 128 CPU
> exclusive domain, called big_domain (due to my lack of naming
> creativity), and further divide big_domain into smaller, non-exclusive
> sched_domains.  We do this all the time, albeit statically at boot time,
> with the current sched_domains code.  When we create a 4-node domain on
> IA64, and underneath it we create 4 1-node domains.  We've now
> partitioned the system into 4 sched_domains, each containing 4 cpus. 
> Balancing between these 4 node-level sched_domains is allowed, but can
> be disallowed by not setting the SD_LOAD_BALANCE flag.  Your example
> does show that it can be more than just a convenient way to group tasks,
> but your example can be done with what I'm proposing.
> 

You wouldn't be able to do this just with sched domains, because
it doesn't know anything about individual tasks. As soon as you
have some overlap, all your tasks can escape out of your domain.

I don't think there is a really nice way to do overlapping sets.
Those that want them need to just use cpu affinity for now.