Re: [PATCH Resend v6] sched: fix wrong rq's runnable_avg update with rt tasks

[Mike Galbraith <efault@gmx.de>]

On Fri, 2013-04-19 at 11:21 +0200, Mike Galbraith wrote: 
> On Fri, 2013-04-19 at 10:50 +0200, Vincent Guittot wrote: 
> > On 19 April 2013 10:14, Mike Galbraith <efault@gmx.de> wrote:
> > > On Fri, 2013-04-19 at 09:49 +0200, Vincent Guittot wrote:
> > >> On 19 April 2013 06:30, Mike Galbraith <efault@gmx.de> wrote:
> > >> > On Thu, 2013-04-18 at 18:34 +0200, Vincent Guittot wrote:
> > >> >> The current update of the rq's load can be erroneous when RT tasks are
> > >> >> involved
> > >> >>
> > >> >> The update of the load of a rq that becomes idle, is done only if the avg_idle
> > >> >> is less than sysctl_sched_migration_cost. If RT tasks and short idle duration
> > >> >> alternate, the runnable_avg will not be updated correctly and the time will be
> > >> >> accounted as idle time when a CFS task wakes up.
> > >> >>
> > >> >> A new idle_enter function is called when the next task is the idle function
> > >> >> so the elapsed time will be accounted as run time in the load of the rq,
> > >> >> whatever the average idle time is. The function update_rq_runnable_avg is
> > >> >> removed from idle_balance.
> > >> >>
> > >> >> When a RT task is scheduled on an idle CPU, the update of the rq's load is
> > >> >> not done when the rq exit idle state because CFS's functions are not
> > >> >> called. Then, the idle_balance, which is called just before entering the
> > >> >> idle function, updates the rq's load and makes the assumption that the
> > >> >> elapsed time since the last update, was only running time.
> > >> >>
> > >> >> As a consequence, the rq's load of a CPU that only runs a periodic RT task,
> > >> >> is close to LOAD_AVG_MAX whatever the running duration of the RT task is.
> > >> >
> > >> > Why do we care what rq's load says, if the only thing running is a
> > >> > periodic RT task?  I _think_ I recall that stuff being put under the
> > >>
> > >> cfs scheduler will use a wrong rq load the next time it wants to schedule a task
> > >>
> > >> > throttle specifically to not waste cycles doing that on every
> > >> > microscopic idle.
> > >>
> > >> yes but this lead to the wrong computation of runnable_avg_sum. To be
> > >> more precise, we only need to call __update_entity_runnable_avg,
> > >> __update_tg_runnable_avg is not mandatory in this step.
> > >
> > > If it only scares fair class tasks away from the periodic rt load, that
> > > seems like a benefit to me, not a liability.  If we really really need
> > 
> > I'm not sure that such behavior that is only based on erroneous value,
> > is good one.
> > 
> > > perfect load numbers, fine, we have to eat some cycles, but when I look
> > > at it, it looks like one of those "Perfect is the enemy of good" things.
> > 
> > The target is not perfect number but good enough to be usable. The
> > systctl_migration_cost threshold is good for idle balancing but can
> > generates wrong load value
> 
> But again, why do we care?  To be able to mix rt and fair loads and
> still make pretty mixed load utilization numbers?  Paying a general case
> fast path price to make strange (to me) load utilization numbers pretty
> is not very attractive.

So I'm not convinced this is a good thing to do, but it's not my call,
that's Peter and Ingos job, so having expressed my opinion, I'll shut up
and let them do their thing ;-)

-Mike