Re: RT task scheduling

[Peter Williams <pwil3058@bigpond.net.au>]

Darren Hart wrote:
> On Wednesday 05 April 2006 21:19, Peter Williams wrote:
>> Darren Hart wrote:
>>> My last mail specifically addresses preempt-rt, but I'd like to know
>>> people's thoughts regarding this issue in the mainline kernel.  Please
>>> see my previous post "realtime-preempt scheduling - rt_overload behavior"
>>> for a testcase that produces unpredictable scheduling results.
>>>
>>> Part of the issue here is to define what we consider "correct behavior"
>>> for SCHED_FIFO realtime tasks.  Do we (A) need to strive for "strict
>>> realtime priority scheduling" where the NR_CPUS highest priority runnable
>>> SCHED_FIFO tasks are _always_ running?  Or do we (B) take the best effort
>>> approach with an upper limit RT priority imbalances, where an imbalance
>>> may occur (say at wakeup or exit) but will be remedied within 1 tick. 
>>> The smpnice patches improve load balancing, but don't provide (A).
>>>
>>> More details in the previous mail...
>> I'm currently researching some ideas to improve smpnice that may help in
>> this situation.  The basic idea is that as well as trying to equally
>> distribute the weighted load among the groups/queues we should also try
>> to achieve equal "average load per task" for each group/queue.  (As well
>> as helping with problems such as yours, this will help to restore the
>> "equal distribution of nr_running" amongst groups/queues aim that is
>> implicit without smpnice due to the fact that load is just a smoothed
>> version of nr_running.)
> 
> Can you elaborate on what you mean by "average load per task" ?  

It's the total weighted load on a run group/queue divided by the 
nr_running for that group/queue.  If this is equal for all groups/queues 
and the total weighted load for them are also equal then the 
distribution of priorities in each group/queue should be similar and 
this will give a high probability that (for an N CPU system) the N 
highest priority tasks will be on different CPUs and hence the highest 
priority task on their CPU.  But these are just tendencies not 
guarantees as it's a statistical process not a deterministic one.

> 
> Also, since smpnice is (correct me if I am wrong) load_balancing, I don't 
> think it will prevent the problem from happening, but rather fix it when it 
> does.  If we want to prevent it from happening, I think we need to do 
> something like the rt_overload code from the RT patchset.

I agree.  Changes to smpnice (as described above) would help with this 
problem (i.e. they'll make the distribution of tasks TEND towards the 
desired state) but would not provide the necessary determinism.  I think 
special measures (such as rt_overload) are required if you want determinism.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce