From: Karl Mehltretter <kmehltretter@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>,
Peter Zijlstra <peterz@infradead.org>,
Thomas Gleixner <tglx@kernel.org>
Cc: Karl Mehltretter <kmehltretter@gmail.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Ingo Molnar <mingo@redhat.com>, Andy Lutomirski <luto@kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
"Paul E . McKenney" <paulmck@kernel.org>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Boqun Feng <boqun@kernel.org>, Waiman Long <longman@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
Hongyan Xia <hongyan.xia@transsion.com>,
Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Nathan Chancellor <nathan@kernel.org>,
Nick Desaulniers <ndesaulniers@google.com>,
Bill Wendling <morbo@google.com>,
Justin Stitt <justinstitt@google.com>,
linux-arm-kernel@lists.infradead.org,
linux-trace-kernel@vger.kernel.org,
linux-rt-devel@lists.linux.dev, linux-doc@vger.kernel.org,
llvm@lists.linux.dev, linux-kernel@vger.kernel.org
Subject: [RFC PATCH 10/12] docs: entry: Describe the pre-counted hardirq lifecycle
Date: Thu, 22 Oct 2026 07:16:52 +0200 [thread overview]
Message-ID: <20261022051654.79224-11-kmehltretter@gmail.com> (raw)
In-Reply-To: <20261022051654.79224-1-kmehltretter@gmail.com>
The entry documentation covers only the traditional irq_enter_rcu()
path. It does not explain the early-counted entry contract or the retained
and synthetic hardirq exit phases.
Document the raw count and lockdep entry, delayed traceable bookkeeping,
hardirq accounting, hardirq-to-softirq count swap, retained real IRQ exit,
possible preemption and migration, and the classification-only return tail.
Also cover late worker closure, raw teardown and context-tracking order.
Record that arm64 excludes pseudo-NMIs from this lifecycle and identify its
external noinstr inlining prerequisites.
Assisted-by: LLM
---
Documentation/core-api/entry.rst | 75 +++++++++++++++++++++++++++-----
1 file changed, 65 insertions(+), 10 deletions(-)
diff --git a/Documentation/core-api/entry.rst b/Documentation/core-api/entry.rst
index 09942135aad18..e02c463dcb08f 100644
--- a/Documentation/core-api/entry.rst
+++ b/Documentation/core-api/entry.rst
@@ -193,21 +193,14 @@ Note that the invocation of the actual interrupt handler is within a
irq_enter_rcu() and irq_exit_rcu() pair.
irq_enter_rcu() updates the preemption count which makes in_hardirq()
-return true, handles NOHZ tick state and interrupt time accounting. This
-means that up to the point where irq_enter_rcu() is invoked in_hardirq()
-returns false.
+return true, handles NOHZ tick state and interrupt time accounting. On this
+entry path, in_hardirq() therefore returns false until irq_enter_rcu() is
+invoked.
irq_exit_rcu() handles interrupt time accounting, handles soft interrupts if
possible, handles the NOHZ tick state and finally undoes the preemption count
update.
-In theory, the preemption count could be updated in irqentry_enter(). In
-practice, deferring this update to irq_enter_rcu() allows the preemption-count
-code to be traced, while also maintaining symmetry with irq_exit_rcu() and
-irqentry_exit(), which are described in the next paragraph. The only downside
-is that the early entry code up to irq_enter_rcu() must be aware that the
-preemption count has not yet been updated with the HARDIRQ_OFFSET state.
-
Note that soft interrupt handlers must run in BH context rather than in hard
interrupt context. irq_exit_rcu() therefore replaces HARDIRQ_OFFSET with
SOFTIRQ_OFFSET in the preemption count while it handles soft interrupts and
@@ -216,6 +209,68 @@ is still attributed to the interrupt. HARDIRQ_OFFSET is removed before
irq_exit_rcu() returns because irqentry_exit() might schedule, which requires
that HARDIRQ_OFFSET has been removed from the preemption count.
+Pre-counted hardirq lifecycle
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Architectures which need hardirq state before early context-tracking or
+tracing callbacks can use the pre-counted lifecycle. It applies only to an
+ordinary hard interrupt after the architecture has distinguished it from an
+NMI-like event:
+
+ * irqentry_hardirq_enter() adds HARDIRQ_OFFSET with a raw update and
+ establishes IRQ-disabled and lockdep hardirq state before instrumentable
+ entry work. It then runs the generic entry and context-tracking path,
+ including any required transition out of an RCU extended quiescent state.
+
+ * irq_enter_rcu_preempt_counted() later performs the traceable debug and
+ latency bookkeeping for that raw update. It also performs the deferred
+ timer repair, NOHZ tick handling and interrupt accounting from
+ irq_enter_rcu().
+
+ * irq_exit_rcu_preempt_counted_prepare() performs interrupt accounting,
+ softirq handling and NOHZ tick handling while retaining the hardirq count
+ and lockdep nesting through every traceable exit callback. Softirq
+ handling uses the same temporary HARDIRQ_OFFSET to SOFTIRQ_OFFSET and back
+ transition described above. The architecture immediately follows the
+ preparation with irqentry_hardirq_exit_finish(), which applies the raw
+ count and lockdep teardown.
+
+The real interrupt is complete after that teardown. Return-to-kernel
+preemption can schedule and migrate the task before the architecture-specific
+tail runs, so the current preemption count cannot identify the original entry
+class. irqentry_state_t retains that provenance instead.
+
+For an ordinary hardirq returning to kernel mode,
+irqentry_hardirq_tail_enter() restores only HARDIRQ_OFFSET and lockdep hardirq
+nesting after the preemption decision. This synthetic classification covers
+traceable architecture tail work, deferred timer rearming, IRQ-on preparation
+and context-tracking preparation. It does not repeat interrupt accounting,
+RCU or tick entry. IRQ-on preparation accepts exactly this one synthetic
+lockdep nesting level.
+
+irqentry_hardirq_tail_exit() performs the final traceable teardown preparation
+and then removes the raw synthetic state. Both retained exit points recheck
+regular softirq work and forced-thread timer work raised by late callbacks and
+wake the corresponding workers before dropping hardirq state. If the
+interrupted context was in an RCU extended quiescent state,
+ct_irq_exit_prepare() runs before the raw tail teardown and
+ct_irq_exit_finish() completes the transition afterwards.
+
+On arm64 the ordinary lifecycle is selected only after the pseudo-NMI
+decision. Pseudo-NMIs continue to use irqentry_nmi_enter() and
+irqentry_nmi_exit(). All raw transitions are in noinstr paths, so
+preempt_count(), the raw preempt-count update helpers and
+__kern_my_cpu_offset() must remain inline. The arm64 conversion includes the
+first requirement and depends on the `arm64 raw preempt-count inlining
+prerequisite`_ and `arm64 per-CPU offset inlining prerequisite`_ for the
+others.
+
+.. _`arm64 raw preempt-count inlining prerequisite`:
+ https://lore.kernel.org/all/c3a90368dd6c096af9d918e036a27728696fb8a4.1786603168.git.hongyan.xia@transsion.com/
+
+.. _`arm64 per-CPU offset inlining prerequisite`:
+ https://lore.kernel.org/r/20260918161407.2300-12-will@kernel.org
+
Even though interrupt handlers are expected to run with local interrupts
disabled, interrupt nesting is common from an entry/exit perspective. For
example, softirq handling happens within an irqentry_{enter,exit}() block with
--
2.39.5 (Apple Git-154)
next prev parent reply other threads:[~2026-10-22 5:17 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-22 5:16 [RFC PATCH 00/12] entry, tracing: Classify early hardirq entry and return tails Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 01/12] sched: Split preempt-count bookkeeping from raw updates Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 02/12] lockdep: Use raw access for pinned hardirq state Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 03/12] entry: Add the pre-counted ordinary-hardirq entry/exit contract Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 04/12] entry: Skip inactive IRQ-return reschedule calls Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 05/12] context_tracking: Split IRQ-exit preparation from the EQS transition Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 06/12] lockdep: Prepare IRQ-on state from a hardirq return tail Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 07/12] entry: Carry ordinary-hardirq provenance through kernel return Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 08/12] entry: Classify ordinary hardirq return-tail work as hardirq context Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 09/12] arm64: Use the pre-counted hardirq lifecycle Karl Mehltretter
2026-10-22 5:16 ` Karl Mehltretter [this message]
2026-10-22 5:16 ` [RFC PATCH 11/12] locking/selftest: Exercise hardirq return-tail preparation Karl Mehltretter
2026-10-22 5:16 ` [RFC PATCH 12/12] tracing: Pin recursion state for its complete lifetime Karl Mehltretter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261022051654.79224-11-kmehltretter@gmail.com \
--to=kmehltretter@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=bigeasy@linutronix.de \
--cc=boqun@kernel.org \
--cc=bsegall@google.com \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=dietmar.eggemann@arm.com \
--cc=frederic@kernel.org \
--cc=hongyan.xia@transsion.com \
--cc=juri.lelli@redhat.com \
--cc=justinstitt@google.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=longman@redhat.com \
--cc=luto@kernel.org \
--cc=mark.rutland@arm.com \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=morbo@google.com \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rostedt@goodmis.org \
--cc=skhan@linuxfoundation.org \
--cc=tglx@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox