Re: [RFC v5 00/28] hw/arm/smmuv3: Support Secure state for SMMUv3

[Mostafa Saleh <smostafa@google.com>]

On Fri, Aug 14, 2026 at 12:15:15AM +0800, Tao Tang wrote:
> Hi all,
> 
> This is v5 of the Secure SMMUv3 series.
> 
> Many thanks to everyone who reviewed the earlier versions. This version contains
> 28 commits in total. Only one patch touches hw/pci (patch #23); the remaining
> patches are confined to the Arm SMMUv3 model. Also some patches in v4 have been
> merged into the mainline QEMU tree, so the v5 series is now smaller than v4.
> 
> Secure Stage 2 is not implemented. Secure STEs which enable Stage 2 are
> rejected with C_BAD_STE. Realm and Root programming interfaces are outside the
> scope of this series.
> 
> Commits layout:
> ---------------
> The commits are roughly grouped as follows:
> 1. (#23, hw/pci only) Add a PCIDevice "sec-sid" property so boards can mark a
>   device's SEC_SID (system integration knob).
> 2. (#1-2, #24-27) Core data-structure and plumbing refactors.
> 3. (#3-8) Introduce SEC_SID through caches, notifiers, and helpers.
> 4. (#9-12) Page table walk related updates (e.g. NSCFG/PTE helpers).
> 5. (#13-15) EVTQ/CMDQ handling and invalidation paths.
> 6. (#16-22) MMIO/register access checks and architectural corner cases.
> 7. (#28) A temporary [NOT-MERGE] commit used with Hafnium to exercise
>   secure-bank register access.
> 
> 
> Main updates:
> -------------
> - (#23, hw/pci only) Add a "sec-sid" property to PCIDevice and consume it from
>   the SMMU in #24 to select the security context and validate capabilities.
> 
>   @Michael: could you please review the hw/pci change
>   ("hw/pci: Add sec-sid property to PCIDevice") and confirm this kind of
>   system-integration property is acceptable for the PCI subtree?
> 
>   For context, I previously posted this as a standalone patch [1]:
>   [1] https://lore.kernel.org/qemu-devel/20251211102729.227376-1-tangtao1634@phytium.com.cn/
> 
>   The commit in this v5 series is an updated version based on Pierrick's
>   feedback (notably switching sec-sid to a string property). We also discussed
>   the need for this knob as a prerequisite for accurately modelling SMMU secure
>   state here [2]:
> 
>   [2] https://lore.kernel.org/qemu-devel/4870b7df-4cb3-457e-9a18-87f3558adf09@linaro.org/
> 
>   We do need a stable way for boards to statically mark devices that may issue
>   Secure transactions.
> 
> other updates:
> - Drop the CR0 and CFGI_CD fixes which are already upstream.
> - Select MemTxAttrs and AddressSpace on demand from SEC_SID and keep the
>   programming-interface namespace in the configuration and IOTLB keys.
> - Route CMDQ, EventQ, IRQ and GERROR handling through the originating
>   programming-interface bank.
> - Tighten MMIO RAZ/WI, writability and reserved-bit handling.
> - Reject unsupported non-NS IOMMU notifier registration.
> - Make secure-impl an on/off/auto property, resolve auto from the Secure
>   AddressSpace, and require Stage 1 support when enabled.
> 
> 
> Testing notes:
> --------------
> - Testing MMIO accesses
>   The branch currently contains one commit explicitly marked [NOT-MERGE]. It is
>   only used to cooperate with Hafnium to exercise and validate secure-bank
>   register read/write paths in a small, reproducible setup as described in [3]:
>   [3] https://hnusdr.github.io/2025/08/09/Test-Secure-SMMU-with-Hafnium-ENG/
> 
>   This commit is not intended for upstream and will be dropped before posting
>   the final mergeable series.
> 
>   For reference, the Hafnium test logs show the SMMU being initialized
>   successfully:
> 
>   INFO: Loading VM id 0x8001: op-tee.
>   INFO: Loaded with 4 vCPUs, entry at 0xe300000.
>   INFO: Hafnium initialisation completed
> 
>   With all smmu* traces enabled in QEMU, we can also observe secure-bank MMIO
>   accesses and secure CMDQ consumption (sec_sid=1), including CFGI invalidations
>   with ssec=1.
> 
> - Testing DMA translation
>   DMA translation/invalidation is tested with qtest and iommu-testdev.
>   The required secure/space extensions are still under RFC at [4].
>   Branch [5] adds the integration needed to test this series.
> 
>   [4] https://gitlab.com/TaoTang/qemu/tree/qtest-secure-v5
>   [5] https://gitlab.com/TaoTang/qemu/tree/integration/secure-smmu-v5-qtest
> 
>   This covers Secure S1 translation and the unsupported SEL2 negative case:
> 
>     QTEST_QEMU_BINARY=./build/qemu-system-aarch64 \
>       ./build/tests/qtest/iommu-smmuv3-test --tap -k \
>       -p /aarch64/iommu-testdev/translation/secure-s1-only
> 
>     QTEST_QEMU_BINARY=./build/qemu-system-aarch64 \
>       ./build/tests/qtest/iommu-smmuv3-test --tap -k \
>       -p /aarch64/iommu-testdev/translation/secure-s2-only-bad-ste
> 
> 
> The patches are available in:
> https://gitlab.com/TaoTang/qemu/tree/series/secure-smmu-v5
> 
> CI is all green:
> https://gitlab.com/TaoTang/qemu/-/pipelines/2757878178
> 
> Best regards,
> Tao
> 
> 
> Tao Tang (28):
>   hw/arm/smmuv3: Introduce secure registers
>   hw/arm/smmuv3: Introduce banked registers for SMMUv3 state
>   hw/arm/smmuv3: Thread SEC_SID through helper APIs
>   hw/arm/smmuv3: Track SEC_SID in configs and events
>   hw/arm/smmu-common: Add security-aware address space selector
>   hw/arm/smmuv3: Plumb transaction attributes into config helpers
>   hw/arm/smmuv3: Reject secure STEs with stage-2 enabled
>   hw/arm/smmu-common: Key configuration cache on SMMUDevice and SEC_SID
>   hw/arm/smmu: Add PTE NS/NSTable helpers
>   hw/arm/smmuv3: Store CD NSCFG in TT info
>   hw/arm/smmu-common: Implement secure state handling in ptw
>   hw/arm/smmuv3: Tag IOTLB cache keys with SEC_SID
>   hw/arm/smmuv3: Pass sec_sid into cmdq consume path
>   hw/arm/smmuv3: Make evtq producer use SEC_SID
>   hw/arm/smmu: Make CMDQ invalidation security-state aware
>   hw/arm/smmuv3: Add access checks for GERROR_IRQ_CFG registers
>   hw/arm/smmuv3: Add access checks for STRTAB_BASE and CR2 registers
>   hw/arm/smmuv3: Add access checks for CMDQ and EVENTQ registers
>   hw/arm/smmuv3: Determine register bank from MMIO offset
>   hw/arm/smmuv3: Route IRQ and GERROR handling by SEC_SID
>   hw/arm/smmuv3: Implement SMMU_S_INIT register
>   hw/arm/smmuv3: Harden security checks in MMIO handlers
>   hw/pci: Add sec-sid property to PCIDevice
>   hw/arm/smmuv3: Select sec-sid from PCI property and validate
>     SECURE_IMPL
>   hw/arm/smmuv3: Reject IOMMU notifiers for non-NS devices
>   hw/arm/smmuv3: Initialize the secure register bank
>   hw/arm/smmuv3: Add secure bank migration and secure-impl property
>   [NOT-MERGE] hw/arm/smmuv3: temporarily enable SEL2 bit and some other
>     features
> 
>  hw/arm/smmu-common.c           |  406 +++++++--
>  hw/arm/smmu-internal.h         |   13 +
>  hw/arm/smmuv3-accel-stubs.c    |    6 +-
>  hw/arm/smmuv3-accel.c          |  104 ++-
>  hw/arm/smmuv3-accel.h          |    6 +-
>  hw/arm/smmuv3-internal.h       |   39 +-
>  hw/arm/smmuv3.c                | 1426 +++++++++++++++++++++++++-------
>  hw/arm/tegra241-cmdqv.c        |    6 +-
>  hw/arm/trace-events            |   37 +-
>  hw/pci/pci.c                   |    7 +
>  include/hw/arm/smmu-common.h   |   74 +-
>  include/hw/arm/smmuv3-common.h |   77 +-
>  include/hw/arm/smmuv3.h        |   33 +-
>  include/hw/pci/pci_device.h    |    3 +
>  14 files changed, 1780 insertions(+), 457 deletions(-)

I verified this series without the last patch does not break my
nested (NS) SMMUv3 setup.

Also, I see the series is getting in a good shape and have many tags,
so it might be a good idea to drop the RFC.

Thanks,
Mostafa

> 
> --
> 2.34.1
>