[PATCH 00/14 v2] idle performance improvements

[PATCH 00/14 v2] idle performance improvements

[Nicholas Piggin]

I rebased this on the powerpc next tree.

A couple of things are changed since last post:

- Patch 1 now properly accounts for the fact the powernv idle
  wakeups do not re-enable interrupts until the cpuidle driver
  enables them. This was not quite right in the previous patch
  (and prep_irq_for_idle() is not quite right for that case so
  a new primitive has to be introduced).

- Patch to replace interrupts from system reset wakeup changed
  rather than replaying directly, it just marks the IRQ in the
  lazy pending bit and it will get replayed at the right time
  when irqs are re-enabled.

Thanks,
Nick

Nicholas Piggin (14):
  powerpc/64s: idle move soft interrupt mask logic into C code
  powerpc/64s: idle hotplug lazy-irq simplification
  powerpc/64s: idle provide a default idle for POWER9
  powerpc/64s: idle process interrupts from system reset wakeup
  powerpc/64s: msgclr when handling doorbell exceptions
  powerpc/64s: interrupt replay balance the return branch predictor
  powerpc/64s: idle branch to handler with virtual mode offset
  powerpc/64s: idle avoid SRR usage in idle sleep/wake paths
  powerpc/64s: idle hmi wakeup is unlikely
  powerpc/64s: cpuidle set polling before enabling irqs
  powerpc/64s: cpuidle read mostly for common globals
  powerpc/64s: cpuidle no memory barrier after break from idle
  powerpc/64: runlatch CTRL[RUN] set optimisation
  powerpc/64s: idle runlatch switch is done with MSR[EE]=0

 arch/powerpc/include/asm/dbell.h         |  13 +++
 arch/powerpc/include/asm/exception-64s.h |  17 +++-
 arch/powerpc/include/asm/hw_irq.h        |   5 ++
 arch/powerpc/include/asm/machdep.h       |   1 +
 arch/powerpc/include/asm/ppc-opcode.h    |   3 +
 arch/powerpc/include/asm/processor.h     |  10 +--
 arch/powerpc/kernel/asm-offsets.c        |   1 +
 arch/powerpc/kernel/exceptions-64s.S     |  33 ++++++--
 arch/powerpc/kernel/idle_book3s.S        | 135 +++++++++----------------------
 arch/powerpc/kernel/irq.c                |  58 ++++++++++++-
 arch/powerpc/kernel/process.c            |  12 +--
 arch/powerpc/kvm/book3s_hv_rmhandlers.S  |   8 +-
 arch/powerpc/platforms/powernv/idle.c    |  90 +++++++++++++++++++--
 arch/powerpc/platforms/powernv/smp.c     |  31 ++++---
 arch/powerpc/platforms/powernv/subcore.c |   3 +-
 drivers/cpuidle/cpuidle-powernv.c        |  37 +++++----
 drivers/cpuidle/cpuidle-pseries.c        |  22 +++--
 17 files changed, 309 insertions(+), 170 deletions(-)

-- 
2.11.0

[PATCH 00/14 v2] idle performance improvements

[Nicholas Piggin]

I rebased this on the powerpc next tree.

A couple of things are changed since last post:

- Patch 1 now properly accounts for the fact the powernv idle
  wakeups do not re-enable interrupts until the cpuidle driver
  enables them. This was not quite right in the previous patch
  (and prep_irq_for_idle() is not quite right for that case so
  a new primitive has to be introduced).

- Patch to replace interrupts from system reset wakeup changed
  rather than replaying directly, it just marks the IRQ in the
  lazy pending bit and it will get replayed at the right time
  when irqs are re-enabled.

Thanks,
Nick

Nicholas Piggin (14):
  powerpc/64s: idle move soft interrupt mask logic into C code
  powerpc/64s: idle hotplug lazy-irq simplification
  powerpc/64s: idle provide a default idle for POWER9
  powerpc/64s: idle process interrupts from system reset wakeup
  powerpc/64s: msgclr when handling doorbell exceptions
  powerpc/64s: interrupt replay balance the return branch predictor
  powerpc/64s: idle branch to handler with virtual mode offset
  powerpc/64s: idle avoid SRR usage in idle sleep/wake paths
  powerpc/64s: idle hmi wakeup is unlikely
  powerpc/64s: cpuidle set polling before enabling irqs
  powerpc/64s: cpuidle read mostly for common globals
  powerpc/64s: cpuidle no memory barrier after break from idle
  powerpc/64: runlatch CTRL[RUN] set optimisation
  powerpc/64s: idle runlatch switch is done with MSR[EE]=0

 arch/powerpc/include/asm/dbell.h         |  13 +++
 arch/powerpc/include/asm/exception-64s.h |  17 +++-
 arch/powerpc/include/asm/hw_irq.h        |   5 ++
 arch/powerpc/include/asm/machdep.h       |   1 +
 arch/powerpc/include/asm/ppc-opcode.h    |   3 +
 arch/powerpc/include/asm/processor.h     |  10 +--
 arch/powerpc/kernel/asm-offsets.c        |   1 +
 arch/powerpc/kernel/exceptions-64s.S     |  33 ++++++--
 arch/powerpc/kernel/idle_book3s.S        | 135 +++++++++----------------------
 arch/powerpc/kernel/irq.c                |  58 ++++++++++++-
 arch/powerpc/kernel/process.c            |  12 +--
 arch/powerpc/kvm/book3s_hv_rmhandlers.S  |   8 +-
 arch/powerpc/platforms/powernv/idle.c    |  90 +++++++++++++++++++--
 arch/powerpc/platforms/powernv/smp.c     |  31 ++++---
 arch/powerpc/platforms/powernv/subcore.c |   3 +-
 drivers/cpuidle/cpuidle-powernv.c        |  37 +++++----
 drivers/cpuidle/cpuidle-pseries.c        |  22 +++--
 17 files changed, 309 insertions(+), 170 deletions(-)

-- 
2.11.0

Re: [PATCH 00/14 v2] idle performance improvements

[Benjamin Herrenschmidt]

On Sun, 2017-06-11 at 19:30 +1000, Nicholas Piggin wrote:
> I rebased this on the powerpc next tree.
> 
> A couple of things are changed since last post:
> 
> - Patch 1 now properly accounts for the fact the powernv idle
>   wakeups do not re-enable interrupts until the cpuidle driver
>   enables them. This was not quite right in the previous patch
>   (and prep_irq_for_idle() is not quite right for that case so
>   a new primitive has to be introduced).

What do you mean ? We shouldn't be going to sleep with the CPU thinking
it's interrupts are off, otherwise we end up effectively "taking an
interrupt while off" which is not right and it will cause accounting to
think we are off for too long.

Is this a generic cpuidle problem or a powerpc issue ?

I'd rather we don't have to of those "prep_for_idle...". If necessary
sync the other one.

> - Patch to replace interrupts from system reset wakeup changed
>   rather than replaying directly, it just marks the IRQ in the
>   lazy pending bit and it will get replayed at the right time
>   when irqs are re-enabled.
> 
> Thanks,
> Nick
> 
> Nicholas Piggin (14):
>   powerpc/64s: idle move soft interrupt mask logic into C code
>   powerpc/64s: idle hotplug lazy-irq simplification
>   powerpc/64s: idle provide a default idle for POWER9
>   powerpc/64s: idle process interrupts from system reset wakeup
>   powerpc/64s: msgclr when handling doorbell exceptions
>   powerpc/64s: interrupt replay balance the return branch predictor
>   powerpc/64s: idle branch to handler with virtual mode offset
>   powerpc/64s: idle avoid SRR usage in idle sleep/wake paths
>   powerpc/64s: idle hmi wakeup is unlikely
>   powerpc/64s: cpuidle set polling before enabling irqs
>   powerpc/64s: cpuidle read mostly for common globals
>   powerpc/64s: cpuidle no memory barrier after break from idle
>   powerpc/64: runlatch CTRL[RUN] set optimisation
>   powerpc/64s: idle runlatch switch is done with MSR[EE]=0
> 
>  arch/powerpc/include/asm/dbell.h         |  13 +++
>  arch/powerpc/include/asm/exception-64s.h |  17 +++-
>  arch/powerpc/include/asm/hw_irq.h        |   5 ++
>  arch/powerpc/include/asm/machdep.h       |   1 +
>  arch/powerpc/include/asm/ppc-opcode.h    |   3 +
>  arch/powerpc/include/asm/processor.h     |  10 +--
>  arch/powerpc/kernel/asm-offsets.c        |   1 +
>  arch/powerpc/kernel/exceptions-64s.S     |  33 ++++++--
>  arch/powerpc/kernel/idle_book3s.S        | 135 +++++++++----------------------
>  arch/powerpc/kernel/irq.c                |  58 ++++++++++++-
>  arch/powerpc/kernel/process.c            |  12 +--
>  arch/powerpc/kvm/book3s_hv_rmhandlers.S  |   8 +-
>  arch/powerpc/platforms/powernv/idle.c    |  90 +++++++++++++++++++--
>  arch/powerpc/platforms/powernv/smp.c     |  31 ++++---
>  arch/powerpc/platforms/powernv/subcore.c |   3 +-
>  drivers/cpuidle/cpuidle-powernv.c        |  37 +++++----
>  drivers/cpuidle/cpuidle-pseries.c        |  22 +++--
>  17 files changed, 309 insertions(+), 170 deletions(-)
> 

Re: [PATCH 00/14 v2] idle performance improvements

[Nicholas Piggin]

On Mon, 12 Jun 2017 14:25:24 +1000
Benjamin Herrenschmidt <benh@au1.ibm.com> wrote:

> On Sun, 2017-06-11 at 19:30 +1000, Nicholas Piggin wrote:
> > I rebased this on the powerpc next tree.
> > 
> > A couple of things are changed since last post:
> > 
> > - Patch 1 now properly accounts for the fact the powernv idle
> >   wakeups do not re-enable interrupts until the cpuidle driver
> >   enables them. This was not quite right in the previous patch
> >   (and prep_irq_for_idle() is not quite right for that case so
> >   a new primitive has to be introduced).  
> 
> What do you mean ? We shouldn't be going to sleep with the CPU thinking
> it's interrupts are off, otherwise we end up effectively "taking an
> interrupt while off" which is not right and it will cause accounting to
> think we are off for too long.
> 
> Is this a generic cpuidle problem or a powerpc issue ?

cpuidle drivers enter their idle state with local_irq_disable().
powernv cpuidle driver currently does not call trace_hardirqs_on()
before going to sleep (e.g., it does not use prep_irq_for_idle()).

I did a previous patch that uses prep_irq_for_idle directly, but
that assumes when we return from idle that local irqs should be
on. The generic cpuidle does not want that, I haven't dug into
exactly why not. But it seems to work better just to put the SRR1
wakeup reason into the irq_pending bit and let the lazy irq logic
take care of the rest.
 
> I'd rather we don't have to of those "prep_for_idle...". If necessary
> sync the other one.

Okay one can call the other rather than implementing twice.

Thanks,
Nick