From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8ABB606A31D for ; Wed, 28 Oct 2026 03:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=e57vDR37; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=nmgNis7f DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=Jtc1fVPemAMqEoN4axaTKj+FevgUODBLdLJA6bSk8XM=; b=e57vDR37SlhiW0 2cNoMgir0JGghRUdz8ABN2UuCTJ2h9mGgSAGiRMMZixaO3i5PLV1Sq2UcAp0BTGHNwUZJBD1kI7o+ xOtZGqWvsPofydsX2BAdJVaJ+8Lv0k90TGK3WLFdTOXrZbDzI/ZZRELsBb6eV/e/kKwTY05MGFp8D O6I1RHWlPXkmnj2nL0n0QNrF+EedwUdDLXmIRS7LB59FgEjf+SMiNLppV2SAraPXmqpy4LVHfpcuT i2CHrLX12NWHAH2tw67yEyfghqGemTe7FUp2vzLOsY/RWJNH4ArBKcaz85RvI2bBkqPkYbRIi+0eY 6sBz7rS3PVwTjKLSqkOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xLuOq-00000006DNh-3g2B; Wed, 28 Oct 2026 03:32:28 +0000 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by bombadil.infradead.org with esmtps (Exim 4.100.1 #2 (Red Hat Linux)) id 1xLuOT-00000006D2x-1Tzb for linux-arm-kernel@lists.infradead.org; Wed, 28 Oct 2026 03:32:06 +0000 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-4a3517554f7so4134315e9.2 for ; Tue, 27 Oct 2026 20:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1793158322; x=1793763122; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Jtc1fVPemAMqEoN4axaTKj+FevgUODBLdLJA6bSk8XM=; b=nmgNis7f23b3x1K98wqEitSfCmF4wFF1CMHLdVvA/kZQ9cdCBQV+a5tU7xT9vKMC8T HduotuniGDOv0QB6cL4tGKRbTAESTsKYdpTzi1XTnM7QkA0lSmdX9QOaOp3tKTfuysz2 jbleLXWwug7VuGIktynAP7eMpsYBpnd7vxAlvCP6I68/2r/c624vwH36dWG8UlmvV9B+ WxfKjZK4DYhTaXkaTRp9dVXJqBiIk11IBjrUOZF/AtMWp7Erg3hB6kLLOB7yXtUxNsqY sRAM3bO62qvJt4LlblEmr8y+Vd5oK8hkOERS/UKhdYxn2m5Gbk+Ko8MrSafesiXvaL1z xt1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793158322; x=1793763122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Jtc1fVPemAMqEoN4axaTKj+FevgUODBLdLJA6bSk8XM=; b=CVBqy0ZY6Oy6rlP7uF1cMRL/cplhNyeFqDpHfB7m4M3tqaVdHXju937nD3i8q6aDKL jg0MtSGyZRHy/I1+0Z16tYiNIFoMS4wCFaA+1c0CItNhqlcb2AYBmdifBEkoMzqWpgzt RBK0ewHcBbKRipgF/KdDsRZ5vVqJ/zqzrlxxehqbajN+PuP9mvjDtOacUDwPciYGxe/2 oOtbDJ+3CfFA5offKfyXb6OyfI5X/pTwZCEemM4Sl5MtlW4sY//896Gdp3epRbo5fw9R tnFkXwUVy6pR4tBkA39spijELeCVVxM7+3kwt+UJRq76g/Y7511VioNX1qumYXS1mULW Zf6g== X-Forwarded-Encrypted: i=1; AKwUvBzg+fhEC3jXqomhXPhYwkOgbrw0QR5tp6fDsQLKKHx3Gf/+GbEqNz1LKB5UW5hdRNbCMDXPxXN1X8+TOTkt8fg1@lists.infradead.org X-Gm-Message-State: AFq9FYIzfk6PKphPyIx1ybP05Z8pNdyT9RWFi3ibTSKjMklaEwEEUIMf IvDT0/sNjUb6S14bxjdBvHwzNoQtDn0+RWRaHbKdjQkjr408RgbZ9CwR X-Gm-Gg: AYBFou0bEEyUIIte97b+wSL9Qgt+FzO3cD80KCVZG/HGlmjJ6LFvh3wF7TEk6djp7UC SHbJkReTxuBehW0UNGmTSxYRJQyIZ7z0bt3fPVWJFDyD0I0JdCO9UDtDxz5f5V60b0niSRgyEPo d3wdCfOB5GyYcIR0eEUIFIRrYj/mRapMdBEm1cQi0jL5URHD9Cz277gHcTFDUdURlk5MrBPOvxG SPjYagsEMGc349zW8BoVFQ0QfEzohAPwlOQvzJfwaR2OSAUG3/G9G4TFFoV1UjZtGwSnzYle0tg VKpJJkOdPWCTu5joontozT1nzP8JpFO5ePiMWu0a+ZD6vtfwpwTi+uXIw7d0VKw8WWBLk5KYmNl 38qEycc94kQIfMxDYvJC6x/JOVvmTtvnRlqZAAAfmiiAkOD5u7sBAalffQ7zP24zxaECXCjkyby sYHu4hDmldEhkfQlCc6Q+oaSfVdZdAcef8OfOLaAFH3S+5PxXoWHQs1ZCGGMgCmpY95Hr1rG/18 7iaDA7jlrB8YEb+sJGFVM6heCKAVF8c1z4hH0WZDaNdIXaxQHBkxmGbWUTrcVz8zCMmfxw5F1lm RLmReXka27m2YC1g/rTyPDi7imTFE0rIE27TR3rLwrFXCsWB8WK3nJgfETT2pXO39Seplmggk41 zWavj2CMeWr5kfLjDYOVXXL+cuaSXJMbC85N3kr45MSp2KSsSGQDd2nb/vdlV9VbtRdUlLWXxhN s7IPfaM00W8XIvVn2o9iLjCwkwahsacXvr X-Received: by 2002:a05:600c:a314:b0:4a3:5d64:df91 with SMTP id 5b1f17b1804b1-4a96b29d208mr13460435e9.18.1793158322397; Tue, 27 Oct 2026 20:32:02 -0700 (PDT) Received: from localhost.localdomain (dynamic-2a02-3100-a45b-7d01-5035-b393-698f-53f0.310.pool.telefonica.de. [2a02:3100:a45b:7d01:5035:b393:698f:53f0]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-48e3ecdfd8dsm24290569f8f.56.2026.10.27.20.31.59 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 27 Oct 2026 20:32:01 -0700 (PDT) From: Karl Mehltretter To: Steven Rostedt , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior Cc: Karl Mehltretter , Catalin Marinas , Will Deacon , Mark Rutland , Ingo Molnar , Andy Lutomirski , Frederic Weisbecker , "Paul E. McKenney" , Boqun Feng , Waiman Long , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Hongyan Xia , Jonathan Corbet , Shuah Khan , Randy Dunlap , Andrew Morton , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , linux-arm-kernel@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linux-rt-devel@lists.linux.dev, linux-doc@vger.kernel.org, llvm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC PATCH v2 07/13] context_tracking: Split IRQ-exit preparation from the EQS transition Date: Wed, 28 Oct 2026 04:31:21 +0100 Message-Id: <20261028033127.82193-8-kmehltretter@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20261028033127.82193-1-kmehltretter@gmail.com> References: <20261028033127.82193-1-kmehltretter@gmail.com> X-Archived-By: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20261027_203205_692687_7D7898FF X-CRM114-Status: GOOD ( 21.58 ) X-Spam-Score: -2.1 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on aws-us-west-2-korg-listserv-alma10-1 ct_irq_exit() combines traceable nesting and tracepoint work with the final transition to an RCU extended quiescent state. A pre-counted ordinary hardirq return needs to run that preparation before its synthetic hardirq classification is removed, but it must enter the quiescent state only after the raw count and lockdep teardown. Split IRQ exit into prepare and finish phases. The prepare phase validates and updates interrupt nesting, then emits the tracepoint while RCU is watching. The finish phase performs the raw quiescent-state transition and RCU Tasks bookkeeping. Keep ct_irq_exit() and ct_nmi_exit() as adjacent prepare and finish wrappers. Existing IRQ and NMI callers retain their ordering and behavior. Assisted-by: LLM --- include/linux/context_tracking_irq.h | 32 +++++++++++- kernel/context_tracking.c | 78 ++++++++++++++++++++++------ 2 files changed, 93 insertions(+), 17 deletions(-) diff --git a/include/linux/context_tracking_irq.h b/include/linux/context_tracking_irq.h index 197916ee91a4b..b038cd1249148 100644 --- a/include/linux/context_tracking_irq.h +++ b/include/linux/context_tracking_irq.h @@ -2,8 +2,23 @@ #ifndef _LINUX_CONTEXT_TRACKING_IRQ_H #define _LINUX_CONTEXT_TRACKING_IRQ_H +#include + +/** + * struct ct_irq_exit_state - Opaque context-tracking IRQ-exit state + * @exit_rcu: The interrupted context was in an RCU extended quiescent state + * + * This object is filled in by ct_irq_exit_prepare() and must be passed to the + * matching ct_irq_exit_finish(). Callers must consider its members private. + */ +struct ct_irq_exit_state { + bool exit_rcu; +}; + #ifdef CONFIG_CONTEXT_TRACKING_IDLE void ct_irq_enter(void); +struct ct_irq_exit_state ct_irq_exit_prepare(void); +void ct_irq_exit_finish(struct ct_irq_exit_state state); void ct_irq_exit(void); void ct_irq_enter_irqson(void); void ct_irq_exit_irqson(void); @@ -11,7 +26,22 @@ void ct_nmi_enter(void); void ct_nmi_exit(void); #else static __always_inline void ct_irq_enter(void) { } -static __always_inline void ct_irq_exit(void) { } +static __always_inline struct ct_irq_exit_state ct_irq_exit_prepare(void) +{ + return (struct ct_irq_exit_state) { }; +} + +static __always_inline void ct_irq_exit_finish(struct ct_irq_exit_state state) +{ + (void)state; +} + +static __always_inline void ct_irq_exit(void) +{ + struct ct_irq_exit_state state = ct_irq_exit_prepare(); + + ct_irq_exit_finish(state); +} static inline void ct_irq_enter_irqson(void) { } static inline void ct_irq_exit_irqson(void) { } static __always_inline void ct_nmi_enter(void) { } diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c index a743e7ffa6c00..80ed8ec606c6a 100644 --- a/kernel/context_tracking.c +++ b/kernel/context_tracking.c @@ -169,20 +169,10 @@ static void noinstr ct_kernel_enter(bool user, int offset) instrumentation_end(); } -/** - * ct_nmi_exit - inform RCU of exit from NMI context - * - * If we are returning from the outermost NMI handler that interrupted an - * RCU-idle period, update ct->state and ct->nmi_nesting - * to let the RCU grace-period handling know that the CPU is back to - * being RCU-idle. - * - * If you add or remove a call to ct_nmi_exit(), be sure to test - * with CONFIG_RCU_EQS_DEBUG=y. - */ -void noinstr ct_nmi_exit(void) +static struct ct_irq_exit_state noinstr __ct_nmi_exit_prepare(void) { struct context_tracking *ct = this_cpu_ptr(&context_tracking); + struct ct_irq_exit_state state = { }; instrumentation_begin(); /* @@ -203,7 +193,7 @@ void noinstr ct_nmi_exit(void) WRITE_ONCE(ct->nmi_nesting, /* No store tearing. */ ct_nmi_nesting() - 2); instrumentation_end(); - return; + return state; } /* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */ @@ -213,6 +203,15 @@ void noinstr ct_nmi_exit(void) // instrumentation for the noinstr ct_kernel_exit_state() instrument_atomic_write(&ct->state, sizeof(ct->state)); instrumentation_end(); + state.exit_rcu = true; + + return state; +} + +static void noinstr __ct_nmi_exit_finish(struct ct_irq_exit_state state) +{ + if (!state.exit_rcu) + return; // RCU is watching here ... ct_kernel_exit_state(CT_RCU_WATCHING); @@ -222,6 +221,24 @@ void noinstr ct_nmi_exit(void) rcu_task_exit(); } +/** + * ct_nmi_exit - inform RCU of exit from NMI context + * + * If we are returning from the outermost NMI handler that interrupted an + * RCU-idle period, update ct->state and ct->nmi_nesting + * to let the RCU grace-period handling know that the CPU is back to + * being RCU-idle. + * + * If you add or remove a call to ct_nmi_exit(), be sure to test + * with CONFIG_RCU_EQS_DEBUG=y. + */ +void noinstr ct_nmi_exit(void) +{ + struct ct_irq_exit_state state = __ct_nmi_exit_prepare(); + + __ct_nmi_exit_finish(state); +} + /** * ct_nmi_enter - inform RCU of entry to NMI context * @@ -348,7 +365,35 @@ noinstr void ct_irq_enter(void) } /** - * ct_irq_exit - inform RCU that current CPU is exiting irq towards idle + * ct_irq_exit_prepare - prepare to exit an IRQ towards idle + * + * Perform the traceable part of an IRQ exit and return an opaque object for + * ct_irq_exit_finish(). RCU remains watching until the matching finish call. + * The caller must keep interrupts disabled, must not schedule or migrate, and + * must invoke ct_irq_exit_finish() exactly once. + * + * Return: An opaque object to pass to ct_irq_exit_finish(). + */ +struct ct_irq_exit_state noinstr ct_irq_exit_prepare(void) +{ + lockdep_assert_irqs_disabled(); + return __ct_nmi_exit_prepare(); +} + +/** + * ct_irq_exit_finish - finish exiting an IRQ towards idle + * @state: Return value from the matching ct_irq_exit_prepare() + * + * Complete a pending transition to an RCU extended quiescent state. Nothing + * traceable may run between this function and the low-level return path. + */ +void noinstr ct_irq_exit_finish(struct ct_irq_exit_state state) +{ + __ct_nmi_exit_finish(state); +} + +/** + * ct_irq_exit - inform RCU that current CPU is exiting IRQ towards idle * * Exit from an interrupt handler, which might possibly result in entering * idle mode, in other words, leaving the mode in which read-side critical @@ -368,8 +413,9 @@ noinstr void ct_irq_enter(void) */ noinstr void ct_irq_exit(void) { - lockdep_assert_irqs_disabled(); - ct_nmi_exit(); + struct ct_irq_exit_state state = ct_irq_exit_prepare(); + + ct_irq_exit_finish(state); } /* -- 2.39.5 (Apple Git-154)