From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3604C3063F for ; Mon, 3 Jul 2023 17:34:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230258AbjGCRet (ORCPT ); Mon, 3 Jul 2023 13:34:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229932AbjGCRer (ORCPT ); Mon, 3 Jul 2023 13:34:47 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F802E5D for ; Mon, 3 Jul 2023 10:34:46 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1B80760FE8 for ; Mon, 3 Jul 2023 17:34:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7AA0BC433C8; Mon, 3 Jul 2023 17:34:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1688405685; bh=1zve+hWveRnQsj6AkpxQ15rXfh1FCEn24VOyA8/pC34=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=pVqLWF52Tg9xTT87v0lmtarJI1WS/rF52kgqYjB3kncmX0SKRvVHRIeP/idcR+adJ qOUc7IfGwBwcHOdiJWLBfdQGejr5fA/DYAjjBfwWNuBLeHgj9CHORM4hZ5gWvtZcVi PnOWcLfgF337e+otuu0x1U1z+6l6ofm8KXFxwtXcGrJ20CSgAbSpiPpt244FZ58vhU kGoqkihd8GLiVXJ5h/2GPsGD2y+igsuTnCCaNYFZ/RSVANdsu56UXAc31RaeFRrgrD 3oYuobQlhbRO36tvrpOmMTxp4ZI0fFR6Y02QVtW+G7DHBElu+/jYqqyshWHpX0vFB/ xAUJTxSEP9XHA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qGNRj-00AE4U-3T; Mon, 03 Jul 2023 18:34:43 +0100 Date: Mon, 03 Jul 2023 18:34:42 +0100 Message-ID: <861qhoyk3x.wl-maz@kernel.org> From: Marc Zyngier To: Conor Dooley Cc: Linus Torvalds , Anup Patel , Palmer Dabbelt , lkp , "Liam R. Howlett" , LKML , lkp@lists.01.org, lkp@intel.com Subject: Re: [mm] 408579cd62: WARNING:suspicious_RCU_usage In-Reply-To: <20230703-regalia-preachy-136bf396e320@spud> References: <20230703-dupe-frying-79ae2ccf94eb@spud> <20230703-regalia-preachy-136bf396e320@spud> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: conor@kernel.org, torvalds@linux-foundation.org, apatel@ventanamicro.com, palmer@rivosinc.com, oliver.sang@intel.com, Liam.Howlett@oracle.com, linux-kernel@vger.kernel.org, lkp@lists.01.org, lkp@intel.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 03 Jul 2023 18:20:52 +0100, Conor Dooley wrote: > > [1 ] > On Mon, Jul 03, 2023 at 10:07:28AM -0700, Linus Torvalds wrote: > > On Mon, 3 Jul 2023 at 10:00, Conor Dooley wrote: > > > > > > I'm not entirely sure if it is related, as stuff in the guts of mm like > > > this is beyond me, but I've been seeing similar warnings on RISC-V. > > > > No, that RISC-V warning is also about bad RCU usage, but that's a > > different thing. > > > > > RCU used illegally from offline CPU! > > > rcu_scheduler_active = 1, debug_locks = 1 > > > 1 lock held by swapper/1/0: > > > #0: ffffffff8169ceb0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x32 > > > > > > stack backtrace: > > > CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.4.0-10173-ga901a3568fd2 #1 > > > Hardware name: riscv-virtio,qemu (DT) > > > Call Trace: > > > [] show_stack+0x2c/0x38 > > > [] dump_stack_lvl+0x5e/0x80 > > > [] dump_stack+0x14/0x1c > > > [] lockdep_rcu_suspicious+0x19e/0x232 > > > [] mtree_load+0x18a/0x3b6 > > > [] __irq_get_desc_lock+0x2c/0x82 > > > [] enable_percpu_irq+0x36/0x9e > > > [] riscv_ipi_enable+0x32/0x4e > > > [] smp_callin+0x24/0x66 > > > > This is also triggering on the maple tree sanity checks, but it' sa > > different maple tree, and a different code sequence. > > > > And a different case of suspicious RCU usage - not a lack of locking, > > but simply using RCU before marking the CPU online. > > Ah, I probably should've known from the > RCU used illegally from offline CPU! > that it was different. > > > I suspect the riscv_ipi_enable() in the RISC-V version of smp_callin() > > needs to be moved down to below the > > > > set_cpu_online(curr_cpuid, 1); > > > > or was there some reason why it needed to be done quite _that_ early > > in commit 832f15f42646 ("RISC-V: Treat IPIs as normal Linux IRQs")? > > > > Added guilty parties to the cc. > > Taking the rationale & potential problems out of the equation, that > code movement does suppress the complaints from rcu/maple tree, > thanks. Comparing with what we do on arm64, a less radical change would be to move the IPI init after notify_cpu_starting(), which explicitly enables RCU usage. Something like: diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index bb0b76e1a6d4..f4d6acb38dd0 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -238,10 +238,11 @@ asmlinkage __visible void smp_callin(void) mmgrab(mm); current->active_mm = mm; - riscv_ipi_enable(); - store_cpu_topology(curr_cpuid); notify_cpu_starting(curr_cpuid); + + riscv_ipi_enable(); + numa_add_cpu(curr_cpuid); set_cpu_online(curr_cpuid, 1); probe_vendor_features(curr_cpuid); which I obviously haven't tested at all. Thanks, M. -- Without deviation from the norm, progress is not possible.