Re: [PATCH] arm64/smp: Move rcu_cpu_starting() earlier

From: Will Deacon <will@kernel.org>
To: Qian Cai <cai@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	linux-arm-kernel@lists.infradead.org,
	"Paul E. McKenney" <paulmck@kernel.org>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] arm64/smp: Move rcu_cpu_starting() earlier
Date: Fri, 30 Oct 2020 08:15:05 +0000	[thread overview]
Message-ID: <20201030081505.GA32066@willie-the-truck> (raw)
In-Reply-To: <4375b3c87d91af36509291ec18e98ed41420ec41.camel@redhat.com>

On Thu, Oct 29, 2020 at 09:17:35AM -0400, Qian Cai wrote:
> On Thu, 2020-10-29 at 09:10 +0000, Will Deacon wrote:
> > On Wed, Oct 28, 2020 at 02:26:14PM -0400, Qian Cai wrote:
> > > The call to rcu_cpu_starting() in secondary_start_kernel() is not early
> > > enough in the CPU-hotplug onlining process, which results in lockdep
> > > splats as follows:
> > > 
> > >  WARNING: suspicious RCU usage
> > >  -----------------------------
> > >  kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!!
> > > 
> > >  other info that might help us debug this:
> > > 
> > >  RCU used illegally from offline CPU!
> > >  rcu_scheduler_active = 1, debug_locks = 1
> > >  no locks held by swapper/1/0.
> > > 
> > >  Call trace:
> > >   dump_backtrace+0x0/0x3c8
> > >   show_stack+0x14/0x60
> > >   dump_stack+0x14c/0x1c4
> > >   lockdep_rcu_suspicious+0x134/0x14c
> > >   __lock_acquire+0x1c30/0x2600
> > >   lock_acquire+0x274/0xc48
> > >   _raw_spin_lock+0xc8/0x140
> > >   vprintk_emit+0x90/0x3d0
> > >   vprintk_default+0x34/0x40
> > >   vprintk_func+0x378/0x590
> > >   printk+0xa8/0xd4
> > >   __cpuinfo_store_cpu+0x71c/0x868
> > >   cpuinfo_store_cpu+0x2c/0xc8
> > >   secondary_start_kernel+0x244/0x318
> > > 
> > > This is avoided by moving the call to rcu_cpu_starting up near the
> > > beginning of the secondary_start_kernel() function.
> > 
> > Hmm, it's not really a move though -- we'll end up calling this thing twice
> > afaict. It would be better to make sure we've called notify_cpu_starting()
> > early enough. Can we do that instead?
> 
> Paul mentioned that it is fine to call rcu_cpu_starting() multiple times, and
> Peter mentioned that CPU bringup is complicated. Thus, I thought about doing
> something safe here.
> 
> I tested a bit of patch below which seems fine, but I can't tell for sure if it
> is safe. Any suggestion?

No, you're right -- this does look dodgy as I think we'll end up kicking the
CPU notifiers before things like CPU errata have been figured out. So I'll
pick up your original patch with Paul's ack. Thanks!

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel