[PATCH v3 00/14] nova-core: GPU interrupt support and GSP event delivery

[John Hubbard <jhubbard@nvidia.com>]

I'm posting a v3 because Alexandre's and Danilo's reviews prompted a
number of significant changes, and it's time for a refresh.

This series adds support for GIN, the GPU Interrupt and Notification
unit, which is the GPU's interrupt controller, so that GSP events reach
the driver as interrupts instead of only when the driver polls.

The design uses a threaded IRQ handler. The top half touches only GPU
registers, while the threaded bottom half drains the message queue.

Fine-grained locking is left for a follow-up patchset. I'm working on
that next. For now, there is just a big ugly lock around anything that
even gets close to the GSP message queue. :)

This is based on drm-rust-next, which now includes Danilo Krummrich's
PCI interrupt-vector series. v3 carries no prerequisite commits.

There is a git branch with the patches as applied to drm-rust-next:

    https://github.com/johnhubbard/linux/tree/nova-core-gin-interrupt-tree-v3/

Changes in v3:

* Dropped "match GSP RPC replies by sequence, not just function" from
  this series, and moved it to the r000 firmware series, where the GSP
  does match on sequence. The GSP matches sequence numbers only on r580
  and later. On r570, the UnloadingGuestDriver reply arrives with
  sequence 0, so matching on it made module unload time out. Alexandre
  hit this while unloading the module, and I reproduced it after adding
  an unload step to my test suite. (Alexandre)

* Split "retrigger the GSP falcon and clear every latched cause". The
  new falcon registers are now in a patch before the SWGEN0 patch, and
  the handler changes are in the SWGEN0 patch, so nothing is added and
  then rewritten one patch later. (Alexandre)

* Moved the vector and subtree newtypes before the register
  definitions, so the registers use those types from the start. Moved
  the interrupt-tree changes out of the self-test patch and into the
  tree patch, so the self-test patch adds only the test, its Kconfig
  option, and the line that gates a dead-code annotation on that
  option. (Alexandre)

* GinVector now contains a Bounded<u32, 9>, so every GinVector value is
  below 512 by construction. leaf_index() shifts and casts the value,
  with no from_expr() or build_assert!(). The trigger register write
  needs no fallible field setter. LEAF_INDEX_MASK is gone. (Alexandre)

* GIN registers declare their fields as LeafMask and SubtreeSet, with
  conversions to and from Bounded<u32, 32>. Every tree write uses
  zeroed().with_...(), so the field types prevent a SubtreeSet from
  being written where a LeafMask belongs. Leaf accesses use Bounded's
  Deref and the build-time-checked at(), so the try_at() and
  unwrap_or(0) pairs are gone. GinVector::validate is const again, and
  the LeafCount casts have CAST comments. (Alexandre)

* nova-core now defines its own MsiType, with Msi and MsiX only, rather
  than using PCI's IrqType, which also includes INTx. SubtreeVectors
  stores the MsiType, the HAL's rearm method is no longer an Option,
  and the INTx match arms are gone. (Alexandre)

* Tree::new returns EINVAL for a subtree the architecture does not
  implement. drain() leaves TOP disabled for its caller. One iterator
  walks every implemented leaf, replacing the nested subtree and leaf
  loops. request_for computes its entry index inline. (Alexandre)

* The self-test disables every leaf before draining, and uses one Tree,
  which it reaches through the registered handler. That gives
  disable_all_leaves a caller, so its dead-code annotation is gone.
  (Alexandre)

* Renamed dispatch_event to classify_event. A non-matching message
  reaches it through the existing else branch, so the second check of
  the same condition is gone, and send_command's documentation no
  longer names a private type. A poison() helper logs the reason and
  returns EIO. (Alexandre)

* await_msg takes the queue lock once for the entire wait, so a
  concurrent send_command cannot consume the awaited event. The outer
  Cmdq::receive_msg wrapper is gone, and the errors it propagated are
  documented. (Alexandre)

* The GPU owns its interrupt now. After GSP boot, Gpu quiesces the
  tree, registers the handler, and drains the queue that the GSP filled
  during boot. Gpu allocates the vectors as well. The handler borrows
  the command queue and the device instead of taking an Arc and an
  ARef. The chipset accessor, the command-queue accessor, and the
  unsafe BAR lifetime conversion are gone. (Alexandre, Danilo)

* The tree reset is now a SubtreeVectors method, which resets the tree
  that its own vector allocation covers. The vestigial SWGEN0 clear
  during falcon construction is gone, along with the Falcon method that
  wrapped it. GspInterrupt no longer uses pin-data, and its constructor
  returns Self. (Alexandre)

* The top half intersects IRQSTAT with PRISCV_RISCV_IRQMASK and
  PRISCV_RISCV_IRQDEST, as Open RM does. It sees only the causes routed
  to the host, leaves the firmware's own causes alone, and clears the
  latch of a host cause it cannot service. It masks nothing, because
  PRISCV_RISCV_IRQMASK is read-only to the host and FALCON_IRQMASK does
  not control host routing on a RISC-V falcon. (Alexandre)

* I found one more problem while reworking the series, rather than in
  review. The subtree has to stay enabled at TOP while the handler is
  registered. The startup tree walk leaves TOP disabled, and the
  pre-Hopper MSI rearm is a configuration-space write that does not
  enable it, so nothing enabled it and the GSP vector never reached the
  CPU on that path.

* The interrupt document's glossary defines leaf, subtree and tree, and
  the entry for a disabled vector now says that its pending bit does
  not set the subtree's TOP bit. Register imports use a module glob,
  commit-message bullets start at column 1, and Alexandre's Reviewed-by
  is on the two rust patches. (Alexandre)

* The document dropped its description of matching a reply by sequence
  number, along with the patch that added it, and it no longer says
  that the startup tree walk restores the TOP enables. The rest is an
  editing pass: the register list was written out twice, so it is one
  section now, and I rewrote the sentences that needed two readings.

Tested on Turing (TU117), Ampere (GA104) and Blackwell (GB202): probe
and chipset identification, the interrupt self-test, a GSP name query
over the command queue, driver unload/reload, and the KUnit suites.


Joel Fernandes (2):
  rust: sync: completion: add wait_for_completion_timeout()
  gpu: nova-core: add the GIN interrupt tree and allocate its vectors

John Hubbard (12):
  rust: pci: declare IrqType and IrqTypes with impl_flags
  gpu: nova-core: add the GIN vector and subtree newtypes
  gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers
  gpu: nova-core: add the per-architecture GIN CPU interrupt HAL
  gpu: nova-core: add an interrupt delivery self-test
  gpu: nova-core: log GSP events instead of discarding them
  gpu: nova-core: recover the GSP receive path from corrupt framing
  gpu: nova-core: bound a GSP wait by a single deadline
  gpu: nova-core: add the falcon interrupt status and routing registers
  gpu: nova-core: drive GSP events with the SWGEN0 interrupt
  gpu: nova-core: add KUnit tests for the interrupt tree and HALs
  gpu: nova-core: document the GIN interrupt controller and GSP events

 Documentation/gpu/nova/core/interrupts.rst  | 716 ++++++++++++++++++++
 Documentation/gpu/nova/index.rst            |   1 +
 drivers/gpu/nova-core/Kconfig               |  15 +
 drivers/gpu/nova-core/falcon/gsp.rs         |  71 +-
 drivers/gpu/nova-core/falcon/hal.rs         |  92 ++-
 drivers/gpu/nova-core/gpu.rs                |  68 +-
 drivers/gpu/nova-core/gsp.rs                |   2 +-
 drivers/gpu/nova-core/gsp/cmdq.rs           | 241 +++++--
 drivers/gpu/nova-core/gsp/commands.rs       |   8 +-
 drivers/gpu/nova-core/gsp/sequencer.rs      |   8 +-
 drivers/gpu/nova-core/irq.rs                | 163 +++++
 drivers/gpu/nova-core/irq/doorbell_test.rs  | 301 ++++++++
 drivers/gpu/nova-core/irq/gsp.rs            | 239 +++++++
 drivers/gpu/nova-core/irq/hal.rs            | 170 +++++
 drivers/gpu/nova-core/irq/hal/gh100.rs      |  29 +
 drivers/gpu/nova-core/irq/hal/tu102.rs      |  28 +
 drivers/gpu/nova-core/irq/interrupt_tree.rs | 664 ++++++++++++++++++
 drivers/gpu/nova-core/irq/regs.rs           |  91 +++
 drivers/gpu/nova-core/nova_core.rs          |   1 +
 drivers/gpu/nova-core/regs.rs               |  66 ++
 rust/kernel/pci/irq.rs                      |  68 +-
 rust/kernel/sync/completion.rs              |  23 +-
 22 files changed, 2942 insertions(+), 123 deletions(-)
 create mode 100644 Documentation/gpu/nova/core/interrupts.rst
 create mode 100644 drivers/gpu/nova-core/irq.rs
 create mode 100644 drivers/gpu/nova-core/irq/doorbell_test.rs
 create mode 100644 drivers/gpu/nova-core/irq/gsp.rs
 create mode 100644 drivers/gpu/nova-core/irq/hal.rs
 create mode 100644 drivers/gpu/nova-core/irq/hal/gh100.rs
 create mode 100644 drivers/gpu/nova-core/irq/hal/tu102.rs
 create mode 100644 drivers/gpu/nova-core/irq/interrupt_tree.rs
 create mode 100644 drivers/gpu/nova-core/irq/regs.rs


base-commit: 6cb331644c441ff4101a4f8726283a6ed0d5947e
-- 
2.55.0