Re: [PATCH 3/3] powerpc/64s/idle: POWER9 ESL=0 stop avoid save/restore overhead

[Paul Mackerras <paulus@ozlabs.org>]

On Thu, Mar 01, 2018 at 09:57:34PM +1000, Nicholas Piggin wrote:
> On Thu, 1 Mar 2018 00:04:39 +0530
> Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com> wrote:
> 
> > * Nicholas Piggin <npiggin@gmail.com> [2017-11-18 00:08:07]:
[snip]
> > > diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> > > index a921d5428d76..610b1637c16f 100644
> > > --- a/arch/powerpc/platforms/powernv/idle.c
> > > +++ b/arch/powerpc/platforms/powernv/idle.c
> > > @@ -621,7 +621,12 @@ static int __init pnv_power9_idle_init(struct device_node *np, u32 *flags,
> > >  			continue;
> > >  		}
> > > 
> > > -		if (max_residency_ns < residency_ns[i]) {
> > > +		/*
> > > +		 * Deepest stop for unplug must be PSSCR[EC]=1 (wakeup at
> > > +		 * 0x100.
> > > +		 */
> > > +		if ((max_residency_ns < residency_ns[i])&&
> > > +				(psscr_val[i] & PSSCR_EC)) {
> > >  			max_residency_ns = residency_ns[i];
> > >  			pnv_deepest_stop_psscr_val = psscr_val[i];
> > >  			pnv_deepest_stop_psscr_mask = psscr_mask[i];  
> > 
> > If firmware did not provide any ESL=EC=1 state, we can still leave
> > threads in stop ESL=0 state.  This is just a corner case or random
> > test scenario.  Why do we want to enforce that offline cpus really use
> > a ESL=0 state or just spin? 
> 
> It's because power9_offline_stop only has cases for EC=ESL=1
> states now.
> 
> It actually looks like EC=ESL=0 unplug today is broken KVM, because
> the wakeup side does not check HWTHREAD_REQ, and yet they do set
> HWTHREAD_IN_IDLE. That would probably hang in KVM if we run with
> dependent threads, wouldn't it?

Right.  KVM with indep_threads_mode=N is broken at the moment if you
run with powersave=off or if firmware provides no stop states with
EC=ESL=1.  I'm not sure what's the best way to fix that.

> I think banning it for now should be okay.

Banning what exactly?

Paul.