From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DF58C19F2A for ; Thu, 11 Aug 2022 15:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236089AbiHKPgV (ORCPT ); Thu, 11 Aug 2022 11:36:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236088AbiHKPf3 (ORCPT ); Thu, 11 Aug 2022 11:35:29 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E2409925A; Thu, 11 Aug 2022 08:32:56 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C7406B82168; Thu, 11 Aug 2022 15:32:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15744C433D6; Thu, 11 Aug 2022 15:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660231973; bh=owvKDXizFCEdAgXs7T2Sko6/6Jb9ujIiikjKrAcPITk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UTDIJ9fqU2pnDfVJZQFB4/+NPee1+rG36m4S6LXEstKtnaIhxrY7M2nKQbZhsyIxS 2Tn2gZqCRwHIu0ViWhEEAI3dqdmB9fNFHlIUHWvxMoqSs1QF+zxai+bjaR61W6vI/8 72YMxzDsemJVA2njIOl/M0mNgK8nvw86SsuIupQc29WdyAcQUS3RjtLuhqQQyfh9Uh 1a63NZ81xB3OHxO1j/mOn6f4TsR3TpKVoj8kkYnKVPslsxtlYM9AQmmO3lEkidMmZt OIq/Tt4ZTl2ulVRZWs+lAeubV09XgWSJ93S99frA2G7ANY7zOyprTl5wmRdGb6DROo P2kUiTjLgi4Mg== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: "Paul E. McKenney" , Jiri Olsa , Alexei Starovoitov , Andrii Nakryiko , Yonghong Song , Sasha Levin , frederic@kernel.org, quic_neeraju@quicinc.com, josh@joshtriplett.org, rcu@vger.kernel.org Subject: [PATCH AUTOSEL 5.19 035/105] rcu: Apply noinstr to rcu_idle_enter() and rcu_idle_exit() Date: Thu, 11 Aug 2022 11:27:19 -0400 Message-Id: <20220811152851.1520029-35-sashal@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220811152851.1520029-1-sashal@kernel.org> References: <20220811152851.1520029-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: "Paul E. McKenney" [ Upstream commit ed4ae5eff4b38797607cbdd80da394149110fb37 ] This commit applies the "noinstr" tag to the rcu_idle_enter() and rcu_idle_exit() functions, which are invoked from portions of the idle loop that cannot be instrumented. These tags require reworking the rcu_eqs_enter() and rcu_eqs_exit() functions that these two functions invoke in order to cause them to use normal assertions rather than lockdep. In addition, within rcu_idle_exit(), the raw versions of local_irq_save() and local_irq_restore() are used, again to avoid issues with lockdep in uninstrumented code. This patch is based in part on an earlier patch by Jiri Olsa, discussions with Peter Zijlstra and Frederic Weisbecker, earlier changes by Thomas Gleixner, and off-list discussions with Yonghong Song. Link: https://lore.kernel.org/lkml/20220515203653.4039075-1-jolsa@kernel.org/ Reported-by: Jiri Olsa Reported-by: Alexei Starovoitov Reported-by: Andrii Nakryiko Signed-off-by: Paul E. McKenney Reviewed-by: Yonghong Song Signed-off-by: Sasha Levin --- kernel/rcu/tree.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index c25ba442044a..9a5edab5558c 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -631,8 +631,8 @@ static noinstr void rcu_eqs_enter(bool user) return; } - lockdep_assert_irqs_disabled(); instrumentation_begin(); + lockdep_assert_irqs_disabled(); trace_rcu_dyntick(TPS("Start"), rdp->dynticks_nesting, 0, atomic_read(&rdp->dynticks)); WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current)); rcu_preempt_deferred_qs(current); @@ -659,9 +659,9 @@ static noinstr void rcu_eqs_enter(bool user) * If you add or remove a call to rcu_idle_enter(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_enter(void) +void noinstr rcu_idle_enter(void) { - lockdep_assert_irqs_disabled(); + WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled()); rcu_eqs_enter(false); } EXPORT_SYMBOL_GPL(rcu_idle_enter); @@ -861,7 +861,7 @@ static void noinstr rcu_eqs_exit(bool user) struct rcu_data *rdp; long oldval; - lockdep_assert_irqs_disabled(); + WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled()); rdp = this_cpu_ptr(&rcu_data); oldval = rdp->dynticks_nesting; WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0); @@ -896,13 +896,13 @@ static void noinstr rcu_eqs_exit(bool user) * If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit(void) +void noinstr rcu_idle_exit(void) { unsigned long flags; - local_irq_save(flags); + raw_local_irq_save(flags); rcu_eqs_exit(false); - local_irq_restore(flags); + raw_local_irq_restore(flags); } EXPORT_SYMBOL_GPL(rcu_idle_exit); -- 2.35.1