From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B1CDC25B06 for ; Thu, 11 Aug 2022 15:57:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236996AbiHKP52 (ORCPT ); Thu, 11 Aug 2022 11:57:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236567AbiHKPzJ (ORCPT ); Thu, 11 Aug 2022 11:55:09 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FC4F8C46E; Thu, 11 Aug 2022 08:46:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1A15C61633; Thu, 11 Aug 2022 15:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0D98C433D6; Thu, 11 Aug 2022 15:46:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660232784; bh=DCDfHfUjK28zcHTqoWDOHHNlJEDMzVX9PjsSbjAHoH8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ppn6xhnrknWKvBXPHQfGQKk+pizBHy1Ur52eDjAabf1shMqXXZOWgvqnSWPNJwntH MOtvDBwQCJS0wON/3NExX2eqscEAo3tqcv3HiUR9Z9oUEZjUFg1L1YmFNkT0zuDtSm y5GQXbZesd4NGqszGHT6i/pDevVKrNWJ4y0bxfOUzSi+4wQ6Y/9FgkYUwCerGSkcnt xkOd88v00pZbdLvNw8OViYpb4lPAsyVpUoth2uigCCGvDK6ll9ScDqDzI94rjYWBk/ G5E/dsnJQJahcYNLsLBL/iD/ym8W73xWE5xNUjsT5xkewhqSGaegeDBs0bv32glQf1 3HE5Q48evdZUQ== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: "Paul E. McKenney" , Jiri Olsa , Alexei Starovoitov , Andrii Nakryiko , Yonghong Song , Sasha Levin , frederic@kernel.org, quic_neeraju@quicinc.com, josh@joshtriplett.org, rcu@vger.kernel.org Subject: [PATCH AUTOSEL 5.18 34/93] rcu: Apply noinstr to rcu_idle_enter() and rcu_idle_exit() Date: Thu, 11 Aug 2022 11:41:28 -0400 Message-Id: <20220811154237.1531313-34-sashal@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220811154237.1531313-1-sashal@kernel.org> References: <20220811154237.1531313-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: "Paul E. McKenney" [ Upstream commit ed4ae5eff4b38797607cbdd80da394149110fb37 ] This commit applies the "noinstr" tag to the rcu_idle_enter() and rcu_idle_exit() functions, which are invoked from portions of the idle loop that cannot be instrumented. These tags require reworking the rcu_eqs_enter() and rcu_eqs_exit() functions that these two functions invoke in order to cause them to use normal assertions rather than lockdep. In addition, within rcu_idle_exit(), the raw versions of local_irq_save() and local_irq_restore() are used, again to avoid issues with lockdep in uninstrumented code. This patch is based in part on an earlier patch by Jiri Olsa, discussions with Peter Zijlstra and Frederic Weisbecker, earlier changes by Thomas Gleixner, and off-list discussions with Yonghong Song. Link: https://lore.kernel.org/lkml/20220515203653.4039075-1-jolsa@kernel.org/ Reported-by: Jiri Olsa Reported-by: Alexei Starovoitov Reported-by: Andrii Nakryiko Signed-off-by: Paul E. McKenney Reviewed-by: Yonghong Song Signed-off-by: Sasha Levin --- kernel/rcu/tree.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a4b8189455d5..eb22b43e9ef9 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -631,8 +631,8 @@ static noinstr void rcu_eqs_enter(bool user) return; } - lockdep_assert_irqs_disabled(); instrumentation_begin(); + lockdep_assert_irqs_disabled(); trace_rcu_dyntick(TPS("Start"), rdp->dynticks_nesting, 0, atomic_read(&rdp->dynticks)); WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current)); rcu_preempt_deferred_qs(current); @@ -659,9 +659,9 @@ static noinstr void rcu_eqs_enter(bool user) * If you add or remove a call to rcu_idle_enter(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_enter(void) +void noinstr rcu_idle_enter(void) { - lockdep_assert_irqs_disabled(); + WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled()); rcu_eqs_enter(false); } EXPORT_SYMBOL_GPL(rcu_idle_enter); @@ -861,7 +861,7 @@ static void noinstr rcu_eqs_exit(bool user) struct rcu_data *rdp; long oldval; - lockdep_assert_irqs_disabled(); + WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled()); rdp = this_cpu_ptr(&rcu_data); oldval = rdp->dynticks_nesting; WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0); @@ -896,13 +896,13 @@ static void noinstr rcu_eqs_exit(bool user) * If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit(void) +void noinstr rcu_idle_exit(void) { unsigned long flags; - local_irq_save(flags); + raw_local_irq_save(flags); rcu_eqs_exit(false); - local_irq_restore(flags); + raw_local_irq_restore(flags); } EXPORT_SYMBOL_GPL(rcu_idle_exit); -- 2.35.1