From: "Alexandre Courbot" <acourbot@nvidia.com>
To: "John Hubbard" <jhubbard@nvidia.com>
Cc: "Danilo Krummrich" <dakr@kernel.org>,
"Timur Tabi" <ttabi@nvidia.com>,
"Alistair Popple" <apopple@nvidia.com>,
"Eliot Courtney" <ecourtney@nvidia.com>,
"Zhi Wang" <zhiw@nvidia.com>, "David Airlie" <airlied@gmail.com>,
"Simona Vetter" <simona@ffwll.ch>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Miguel Ojeda" <ojeda@kernel.org>,
"Alex Gaynor" <alex.gaynor@gmail.com>,
"Boqun Feng" <boqun.feng@gmail.com>,
"Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <lossin@kernel.org>,
"Andreas Hindborg" <a.hindborg@kernel.org>,
"Alice Ryhl" <aliceryhl@google.com>,
"Trevor Gross" <tmgross@umich.edu>,
nova-gpu@lists.linux.dev, LKML <linux-kernel@vger.kernel.org>,
"Will Pierce" <wpierce@nvidia.com>
Subject: Re: [PATCH v2 13/15] gpu: nova-core: retrigger the GSP falcon and clear every latched cause
Date: Thu, 03 Sep 2026 00:00:29 +0900 [thread overview]
Message-ID: <DL4X5B86OROD.22YISVN1ADX1E@nvidia.com> (raw)
In-Reply-To: <20260829013324.499542-18-jhubbard@nvidia.com>
On Sat Aug 29, 2026 at 10:33 AM JST, John Hubbard wrote:
<...>
> diff --git a/drivers/gpu/nova-core/irq/gsp.rs b/drivers/gpu/nova-core/irq/gsp.rs
> index 6366380eef98..a1030d66cc70 100644
> --- a/drivers/gpu/nova-core/irq/gsp.rs
> +++ b/drivers/gpu/nova-core/irq/gsp.rs
> @@ -50,18 +50,17 @@
>
> /// Clears the interrupt state that GSP boot left behind.
> ///
> -/// Disables every vector in every implemented leaf, clears the falcon's SWGEN0 latch, clears the
> -/// tree's pending bits, and rearms PCI interrupt delivery. On return no vector is enabled, so the
> -/// tree delivers nothing.
> +/// Disables every vector in every implemented leaf, clears the tree's pending bits, clears the
> +/// falcon's SWGEN0 latch, and rearms PCI interrupt delivery. On return no vector is enabled, so
> +/// the tree delivers nothing.
> pub(crate) fn quiesce(bar: Bar0<'_>, chipset: Chipset, irq_type: pci::IrqType) {
> let tree = Tree::new(bar, chipset, irq_type, GSP_SUBTREE.into());
> tree.disable_all_leaves();
> - // GSP boot consumes its notifications by polling the queue, which leaves SWGEN0 latched.
> - // Clear it before the tree drain below, so the drain clears the tree state the clear sets.
> - // Messages already posted raise no interrupt of their own, and the caller's queue drain
> - // covers them.
> - GspFalcon::clear_swgen0_intr(bar);
> tree.drain();
> + // GSP boot consumes its notifications by polling the queue, which leaves SWGEN0 latched, and
> + // the GSP drives no new signal while it is set. Clear it after the tree drain, which erases
> + // every leaf bit and would erase the one a message posted since the clear had set.
> + GspFalcon::clear_swgen0_intr(bar);
This code was introduced in the previous patch and is now immediately
moved around, and the comments for the two versions contradict each
other. This patch's order seems to be the correct one, so let's use it
from the get-go.
I guess this stems from the general ordering problem with this patch: if
its new falcon registers were introduced before patch 12, then patch 12
could be written in its final form instead of having its code amended
right away. It would be nice if you could reorder things this way for
v3.
> // The `TOP_EN` cycle in `drain` is the rearm for the two enable-cycle methods, but pre-Hopper
> // MSI rearms through a configuration-space write instead. An interrupt delivered before probe
> // leaves delivery un-armed on that path, with no handler to have rearmed it.
> @@ -80,6 +79,8 @@ pub(crate) struct GspInterrupt<'a> {
> cmdq: Arc<Cmdq>,
> /// The GIN interrupt tree for this chipset.
> tree: Tree<'a>,
> + /// Chipset, for the falcon retrigger, which Turing does not implement.
> + chipset: Chipset,
> /// Device, for logging from interrupt context without taking the command-queue lock.
> dev: ARef<device::Device>,
> }
> @@ -98,15 +99,18 @@ pub(crate) fn new(
> bar,
> cmdq,
> tree: Tree::new(bar, chipset, irq_type, GSP_SUBTREE.into()),
> + chipset,
> dev,
> }? Error)
> }
> }
>
> impl irq::ThreadedHandler for GspInterrupt<'_> {
> - /// Top half: clears the GIN leaf, takes the falcon SWGEN0 latch, and rearms PCI interrupt
> - /// delivery.
> + /// Top half: clears the GIN leaf, takes every cause the falcon reports, and rearms PCI
> + /// interrupt delivery.
> fn handle(&self) -> irq::ThreadedIrqReturn {
> + let bar = self.bar;
> +
> // Only service our own vector: require the GSP bit in the leaf and clear just that bit, so
> // a co-pending vector in the same leaf stays pending for whoever services it. The subtree
> // stays enabled, so there is no whole-tree disable and enable.
> @@ -119,27 +123,40 @@ fn handle(&self) -> irq::ThreadedIrqReturn {
> }
> leaf.clear_vectors(GSP_INTR_0_VECTOR.leaf_mask());
>
> - // SWGEN0 is the message-queue notification, so wake the IRQ thread to drain it.
> - let status = GspFalcon::take_swgen0_intr(self.bar);
> - let ret = if status.swgen0() {
> - irq::ThreadedIrqReturn::WakeThread
> - } else {
> - // The tree routes every falcon cause to this vector, so something other than a posted
> - // message fired it, for example a HALT from a GSP crash. There is no recovery path for
> - // those causes, so report the status rather than discarding it.
> + let status = GspFalcon::take_swgen0_intr(bar);
> +
> + // Every cause the falcon reports leaves the falcon's enabled set on this invocation. A
> + // cause left latched holds that set non-empty, and the falcon signals the tree only on a
> + // transition of the set, so no later SWGEN0 would signal at all.
> + let unserviceable = status.with_swgen0(false);
> + if unserviceable.into_raw() != 0 {
> + // The tree routes every falcon cause to this vector, so a cause other than a posted
> + // message also arrives here, for example a HALT from a GSP crash. nova-core has no
> + // recovery path for those, so report the status rather than discarding it, then mask
> + // the cause.
> dev_err!(
> &self.dev,
> - "GSP interrupt with no SWGEN0, falcon IRQSTAT {:#x}\n",
> + "unserviceable GSP falcon interrupt, IRQSTAT {:#x}\n",
> status.into_raw()
> );
> - irq::ThreadedIrqReturn::Handled
> - };
> + GspFalcon::mask_and_clear_intr(bar, unserviceable);
Is this correct? My Clanker told me (and looking closer it seems to be
correct) that `IRQSTAT` also reports causes that are masked, or routed
to the RISC-V core itself rather than to the host (i.e. GSP-RM's own
interrupts), and that OpenRM filters these out: `kgspService_TU102()`
gets its status from `kflcnGetPendingHostInterrupts()`, which for a
RISC-V falcon is `FALCON_IRQSTAT & PRISCV_RISCV_IRQMASK &
PRISCV_RISCV_IRQDEST`. Without that filter, any GSP-internal cause that
happens to be latched when SWGEN0 fires gets reported here, then cleared
and masked out from under the firmware.
Also in RISC-V mode the mask gating host routing appears to be
`PRISCV_RISCV_IRQMASK`, not `FALCON_IRQMASK` - OpenRM never writes the
latter for the GSP. So the comment about the tree routing every falcon
cause here also seems inaccurate.
next prev parent reply other threads:[~2026-09-02 15:00 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-29 1:22 [PATCH v2 00/15] nova-core: GPU interrupt support and GSP event delivery John Hubbard
2026-08-29 1:22 ` [PATCH v2 01/15] rust: pci: declare IrqType and IrqTypes with impl_flags John Hubbard
2026-08-31 1:10 ` Alexandre Courbot
2026-08-29 1:22 ` [PATCH v2 02/15] rust: sync: completion: add wait_for_completion_timeout() John Hubbard
2026-08-31 1:10 ` Alexandre Courbot
2026-08-29 1:22 ` [PATCH v2 03/15] gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers John Hubbard
2026-08-29 1:22 ` [PATCH v2 04/15] gpu: nova-core: add the GIN vector and subtree newtypes John Hubbard
2026-08-31 14:24 ` Alexandre Courbot
2026-09-01 13:16 ` Alexandre Courbot
2026-08-29 1:22 ` [PATCH v2 05/15] gpu: nova-core: add the per-architecture GIN CPU interrupt HAL John Hubbard
2026-09-01 1:15 ` Alexandre Courbot
2026-08-29 1:25 ` [PATCH v2 00/15] nova-core: GPU interrupt support and GSP event delivery John Hubbard
2026-08-29 1:35 ` John Hubbard
2026-08-29 1:33 ` [PATCH v2 06/15] gpu: nova-core: add the GIN interrupt tree and allocate its vectors John Hubbard
2026-09-01 7:03 ` Alexandre Courbot
2026-08-29 1:33 ` [PATCH v2 07/15] gpu: nova-core: add an interrupt delivery self-test John Hubbard
2026-09-01 12:52 ` Alexandre Courbot
2026-08-29 1:33 ` [PATCH v2 08/15] gpu: nova-core: dispatch GSP events instead of discarding them John Hubbard
2026-08-31 5:06 ` Alexandre Courbot
2026-08-29 1:33 ` [PATCH v2 09/15] gpu: nova-core: match GSP RPC replies by sequence, not just function John Hubbard
2026-08-31 1:09 ` Alexandre Courbot
2026-08-31 4:33 ` John Hubbard
2026-08-31 22:18 ` John Hubbard
2026-08-31 22:46 ` John Hubbard
2026-08-29 1:33 ` [PATCH v2 10/15] gpu: nova-core: recover the GSP receive path from corrupt framing John Hubbard
2026-08-31 5:35 ` Alexandre Courbot
2026-08-29 1:33 ` [PATCH v2 11/15] gpu: nova-core: bound a GSP wait by a single deadline John Hubbard
2026-08-31 6:04 ` Alexandre Courbot
2026-08-29 1:33 ` [PATCH v2 12/15] gpu: nova-core: drive GSP events with the SWGEN0 interrupt John Hubbard
2026-09-01 14:54 ` Alexandre Courbot
2026-09-01 15:08 ` Danilo Krummrich
2026-09-02 14:33 ` Alexandre Courbot
2026-09-03 3:06 ` John Hubbard
2026-08-29 1:33 ` [PATCH v2 13/15] gpu: nova-core: retrigger the GSP falcon and clear every latched cause John Hubbard
2026-09-02 15:00 ` Alexandre Courbot [this message]
2026-08-29 1:33 ` [PATCH v2 14/15] gpu: nova-core: add KUnit tests for the interrupt tree and HALs John Hubbard
2026-08-29 1:33 ` [PATCH v2 15/15] gpu: nova-core: document the GIN interrupt controller and GSP events John Hubbard
2026-09-02 15:07 ` Alexandre Courbot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DL4X5B86OROD.22YISVN1ADX1E@nvidia.com \
--to=acourbot@nvidia.com \
--cc=a.hindborg@kernel.org \
--cc=airlied@gmail.com \
--cc=alex.gaynor@gmail.com \
--cc=aliceryhl@google.com \
--cc=apopple@nvidia.com \
--cc=bhelgaas@google.com \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun.feng@gmail.com \
--cc=dakr@kernel.org \
--cc=ecourtney@nvidia.com \
--cc=gary@garyguo.net \
--cc=jhubbard@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lossin@kernel.org \
--cc=nova-gpu@lists.linux.dev \
--cc=ojeda@kernel.org \
--cc=simona@ffwll.ch \
--cc=tmgross@umich.edu \
--cc=ttabi@nvidia.com \
--cc=wpierce@nvidia.com \
--cc=zhiw@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox