Re: RT sched: cpupri_vec lock contention with def_root_domain and no load balance

[Gregory Haskins <ghaskins@novell.com>]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

Max Krasnyansky wrote:
> Gregory Haskins wrote:
>   
>> If you tried creating different cpusets and it still had them all end up
>> in the def_root_domain, something is very broken indeed.  I will take a
>> look.
>>     
>
> I beleive that's the intended behaviour.
Heh...well, as the guy that wrote root-domans, I can definitively say
that is not the behavior that I personally intended ;)



>  We always put cpus that are not
> balanced into null sched domains. This was done since day one (ie when
> cpuisol= option was introduced) and cpusets just followed the same convention.
>   

It sounds like the problem with my code is that "null sched domain"
translates into "default root-domain" which is understandably unexpected
by Dimitri (and myself).  Really I intended root-domains to become
associated with each exclusive/disjoint cpuset that is created.  In a
way, non-balanced/isolated cpus could be modeled as an exclusive cpuset
with one member, but that is somewhat beyond the scope of the
root-domain code as it stands today.  My primary concern was that
Dimitri reports that even creating a disjoint cpuset per cpu does not
yield an isolated root-domain per cpu.  Rather they all end up in the
default root-domain, and this is not what I intended at all.

However, as a secondary goal it would be nice to somehow directly
support the "no-load-balance" option without requiring explicit
exclusive per-cpu cpusets to do it.  The proper mechanism (IMHO) to
scope the scheduler to a subset of cpus (including only "self") is
root-domains so I would prefer to see the solution based on that. 
However, today there is a rather tight coupling of root-domains and
cpusets, so this coupling would likely have to be relaxed a little bit
to get there.

There are certainly other ways to solve the problem as well.  But seeing
as how I intended root-domains to represent the effective partition
scope of the scheduler, this seems like a natural fit in my mind until
its proven to me otherwise.

Regards,
-Greg



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]