Re: [tip:sched/urgent] sched: Fix cross-cpu clock sync on remote wakeups

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Fri, 2011-06-03 at 04:57 -0500, Milton Miller wrote:

> [me looks closely at patch and finds early return]

Yeah, in case there's nothing to do, all the old conditions hold and
irq_enter isn't strictly required.

> > 
> > We could of course add it in sched.c since the logic recurses just
> > fine.. its not pretty though.. :/
> > 
> > Thoughts?
> 
> 
> Many architectures already have an irq_enter becuase they have a single
> interrupt to the cpu for all external causes including software; they
> do the irq_enter before reading from the irq controller to know the
> reason for the interrupt.  A quick glance at irq_enter and irq_exit
> shows they will do several things twice when nested, even if that
> is safe.

Agreed, and its a worry I had. The flip side is that doing it in the
arch code means I have to audit all the archs again (not that I mind too
much, but it takes a wee bit longer), also I'll have to look at all the
code using this IPI for the old purpose.

> Are there really that many calls with the empty list that it makes
> sense to avoid and optimize this on x86 while penalizing the several
> architectures with a nested irq_enter and exit?

I _think_ the now predominant case is this remote wakeup, so adding
irq_enter() to all arch paths isn't too big of a problem, but I need to
make sure.

>   When it also duplicates
> sched_ttwu_pending (because it can't be common with the additional tests)?

yeah, sad that.

> We said the perf mon callback (now irq_work) had to be under irq_enter.

Correct, anything that actually does something in the handler needs
irq_enter, the problem with the resched ipi was that it never actually
did anything and the idle loop exit took care of the no_hz funnies.

> Can we get some numbers for how often the two cases occur on some
> various workloads?

Sure, let me stick some counters in.