Re: [RFC PATCH] kernel/sched/core: busy wait before going idle

[Nicholas Piggin <npiggin@gmail.com>]

On Fri, 20 Apr 2018 09:44:56 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> On Sun, Apr 15, 2018 at 11:31:49PM +1000, Nicholas Piggin wrote:
> > This is a quick hack for comments, but I've always wondered --
> > if we have a short term polling idle states in cpuidle for performance
> > -- why not skip the context switch and entry into all the idle states,
> > and just wait for a bit to see if something wakes up again.  
> 
> Is that context switch so expensive?

I guess relatively much more than taking one branch mispredict on the
loop exit when the task wakes. 10s of cycles vs 1000s?

> And what kernel did you test on? We recently merged a bunch of patches
> from Rafael that avoided disabling the tick for short idle predictions.
> This also has a performance improvements for such workloads.  Did your
> kernel include those?

Yes that actually improved profiles quite a lot, but these numbers were
with those changes. I'll try to find some fast disks or network and get
some more more interesting numbers.

> > It's not uncommon to see various going-to-idle work in kernel profiles.
> > This might be a way to reduce that (and just the cost of switching
> > registers and kernel stack to idle thread). This can be an important
> > path for single thread request-response throughput.  
> 
> So I feel that _if_ we do a spin here, it should only be long enough to
> amortize the schedule switch context.
> 
> However, doing busy waits here has the downside that the 'idle' time is
> not in fact fed into the cpuidle predictor.

That's why I cc'ed Rafael :)

Yes the latency in my hack is probably too long, but I think if we
did this, the cpuile predictor could become involved here. There is
no fundamental reason it has to wait for the idle task to be context
switched for that... it's already become involved in core scheduler
code.

Thanks,
Nick