[PATCH v5 00/10] iommu/arm-smmu-v3: Implement Runtime/System Sleep ops

[Pranjal Shrivastava <praan@google.com>]

As arm-smmu-v3 rapidly finds its way into SoCs designed for hand-held
devices, power management capabilities, similar to its predecessors, are
crucial for these applications. This series introduces power management
support for the arm-smmu-v3 driver.

Design
=======
The arm-smmu-v3 primarily operates with in-memory data structures
through HW registers pointing to these data structures. The proposed design
makes use of this fact for implementing suspend and resume ops, centered
around a driver-specific biased usage counter.

1. Biased Usage Counter
To safely manage runtime PM state, this series introduces an atomic
`nr_cmdq_users` counter. This counter is "biased" by being initialized to
1, representing an "idle but active" state.

The suspend callback waits for this counter to be exactly 1 and then
atomically transitions it to 0, signifying a "suspended" state. Any
other value indicates that the command queue is in use, and the suspend
operation is aborted.

Code paths that submit commands to the hardware (the "fast path") use
`atomic_inc_not_zero()` to acquire a reference. This operation only
succeeds if the counter is not zero (i.e., not suspended), effectively
gating hardware access. The reference is dropped using
`atomic_dec_return_release()` after the hardware interaction is
complete.

2. Suspend / Resume Flow
The "suspend" op now polls for a short period (100us) for the usage
counter to become 1. Once successful, it configures the GBPA register
to abort new transactions and disables the SMMU, EVTQ, and PRIQ, leaving
only the CMDQ enabled to drain any in-flight commands.

The "resume" operation uses the `arm_smmu_device_reset` function, which
re-initializes the HW using the SW-copies maintained by the driver
(e.g., prod/cons pointers, queue base addresses) and clears all cached
configurations.

3. Guarding Hardware Access
Instead of eliding operations, the driver now ensures the SMMU is active
before any hardware access. This is managed by `arm_smmu_rpm_get()` and
`arm_smmu_rpm_put()` helpers, which wrap the standard runtime PM calls. 
These wrappers are now used throughout the driver in interrupt handlers,
TLB invalidation paths, and any other function that touches hardware
registers. This ensures that operations are implicitly queued until the
device resumes.

4. Interrupt Re-config
a. Wired irqs: The series refactors the `arm_smmu_setup_irqs` to be
able to enable/disable irqs and install their handlers separately to
help with the re-initialization of the interrupts correctly.

b. MSIs: The series relies on caching the `msi_msg` and retrieving it
through a newly introduced helper `arm_smmu_resume_msis()` which
re-configures the *_IRQ_CFGn registers by writing back the cached msi_msg.

Call for review
Any insights/comments on the proposed changes are appreciated,
especially in areas related to locking, atomic contexts, and potential
optimizations.

Note: The series isn't tested with MSIs and is weakly tested for PCIe
clients. The same holds true for tegra241_cmdv changes. Any help in
reviewing and testing these parts is much appreciated.

[v5]
 - Refactored GERROR handling into a helper function and invoked it during
   runtime suspend after disabling the SMMU to capture any late-breaking
   gerrors as suggested by Jason.
 - Updated `arm_smmu_page_response` to be power-state aware and drop
   page faults received while suspended.
 - Included a patch from Ashish to correctly restore PROD and CONS
   indices for tegra241-cmdqv after a hardware reset.
 - Collected Reviewed-bys from Mostafa and Nicolin.

[v4]
 - https://lore.kernel.org/all/20251117191433.3360130-1-praan@google.com/
 - Dropped the `pm_runtime_get_if_not_suspended()` API in favor of a
   simpler, driver-specific biased counter (`nr_cmdq_users`) to manage
   runtime PM state.
 - Reworked the suspend callback to poll on the biased counter before
   disabling the SMMU.
 - Addressed comments for the MSI refactor.

[v3]
 - https://lore.kernel.org/all/20250616203149.2649118-1-praan@google.com/
 - Introduced `pm_runtime_get_if_not_suspended` API to avoid races due
   to bouncing RPM states while eliding TLBIs as pointed out by Daniel.
 - Addressed Nicolin's comments regarding msi_resume and CMDQV flush
 - Addressed Daniel's comments about CMDQ locking and draining
 - Addressed issues related to draining the evtq and priq
 - Dropped the code to identify and track user-space attachments

[v2]
 - https://lore.kernel.org/all/20250418233409.3926715-1-praan@google.com/
 - Introduced `arm_smmu_rpm_get_if_active` for eliding TLBIs & CFGIs
 - Updated the rpm helper invocation strategy.
 - Drained all queues in suspend callback (including tegra241-cmdv)
 - Cache and restore msi_msg instead of free-ing realloc-ing on resume
 - Added support to identify and track user-space attachments
 - Fixed the setup_irqs as per Nicolin & Mostafa's suggestions
 - Used force_runtime_suspend/resume instead as per Mostafa's suggestion.
 - Added "Reviewed-by" line from Mostafa on an unchanged patch

[v1]
 - https://lore.kernel.org/all/20250319004254.2547950-1-praan@google.com/

Ashish Mhetre (1):
  iommu/tegra241-cmdqv: Restore PROD and CONS after resume

Pranjal Shrivastava (9):
  iommu/arm-smmu-v3: Refactor arm_smmu_setup_irqs
  iommu/arm-smmu-v3: Add a helper to drain cmd queues
  iommu/tegra241-cmdqv: Add a helper to drain VCMDQs
  iommu/arm-smmu-v3: Cache and restore MSI config
  iommu/arm-smmu-v3: Add a usage counter for cmdq
  iommu/arm-smmu-v3: Implement pm_runtime & system sleep ops
  iommu/arm-smmu-v3: Handle gerror during suspend
  iommu/arm-smmu-v3: Enable pm_runtime and setup devlinks
  iommu/arm-smmu-v3: Invoke pm_runtime before hw access

 .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c     |  10 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 429 ++++++++++++++++--
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |  13 +
 .../iommu/arm/arm-smmu-v3/tegra241-cmdqv.c    |  29 ++
 4 files changed, 445 insertions(+), 36 deletions(-)

-- 
2.52.0.457.g6b5491de43-goog