Re: Fix 80d20d35af1e ("nohz: Fix local_timer_softirq_pending()") may have revealed another problem

[Frederic Weisbecker <frederic@kernel.org>]

On Fri, Aug 24, 2018 at 10:01:35AM +0200, Thomas Gleixner wrote:
> On Fri, 24 Aug 2018, Heiner Kallweit wrote:
> > On 24.08.2018 06:12, Frederic Weisbecker wrote:
> > > On Thu, Aug 16, 2018 at 08:13:03AM +0200, Heiner Kallweit wrote:
> > >> Recently I started to get warning "NOHZ: local_softirq_pending 202" and
> > >> I think it's related to mentioned commit (didn't bisect it yet).
> > >> See log from suspending.
> > >>
> > >> I have no reason to think the fix is wrong, it may just have revealed
> > >> another issue which existed before and was hidden by the bug.
> > >>
> > >> Rgds, Heiner
> > >>
> > >> [   75.073353] random: crng init done
> > >> [   75.073402] random: 7 urandom warning(s) missed due to ratelimiting
> > >> [   78.619564] PM: suspend entry (deep)
> > >> [   78.619675] PM: Syncing filesystems ... done.
> > >> [   78.653684] Freezing user space processes ... (elapsed 0.002 seconds) done.
> > >> [   78.656094] OOM killer disabled.
> > >> [   78.656113] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> > >> [   78.658177] Suspending console(s) (use no_console_suspend to debug)
> > >> [   78.663066] nuvoton-cir 00:07: disabled
> > >> [   78.671817] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> > >> [   78.672210] sd 0:0:0:0: [sda] Stopping disk
> > >> [   78.786651] ACPI: Preparing to enter system sleep state S3
> > >> [   78.789613] PM: Saving platform NVS memory
> > >> [   78.789759] Disabling non-boot CPUs ...
> > >> [   78.805154] NOHZ: local_softirq_pending 202
> > >> [   78.805182] NOHZ: local_softirq_pending 202
> > >> [   78.807102] smpboot: CPU 1 is now offline
> > > 
> > > I've tried to reproduce with suspend on disk but got unsuccessful.
> > > 
> > > A small question as I see someone is having a similar issue with a stable
> > > release only. On which kernel did you trigger that: upstream or stable?
> > > 
> > > I'll continue investigating.
> > > 
> > > Thanks.
> > > 
> > Affected is recent linux-next, after the commit mentioned in the subject.
> > I can work around the warning (not sure whether it's a proper fix),
> > see here:
> > https://lkml.org/lkml/2018/8/18/272
> 
> Can you try the one I posted in this thread:
> 
>  https://lkml.kernel.org/r/alpine.DEB.2.21.1808240851420.1668@nanos.tec.linutronix.de
> 
> Also below for reference.
> 
> Thanks,
> 
> 	tglx
> 
> 8<----------------
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index 5b33e2f5c0ed..6aab9d54a331 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -888,7 +888,7 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)
>  	if (unlikely(local_softirq_pending() && cpu_online(cpu))) {
>  		static int ratelimit;
>  
> -		if (ratelimit < 10 &&
> +		if (ratelimit < 10 && !in_softirq() &&
>  		    (local_softirq_pending() & SOFTIRQ_STOP_IDLE_MASK)) {
>  			pr_warn("NOHZ: local_softirq_pending %02x\n",
>  				(unsigned int) local_softirq_pending());

I fear it may not work in his case because it happens in -next and we don't stop
the idle tick from IRQ tail anymore. So we shouldn't be interrupting a softirq
in this path. Still it's worth trying, I may well be missing something.

Thanks.