Re: [PATCH v3 0/9] Initial support for SMMUv3 nested translation

Jerry Snitselaar
2024-10-26T20:26:10Z

On Wed, Oct 09, 2024 at 01:23:06PM -0300, Jason Gunthorpe wrote:
> This brings support for the IOMMFD ioctls:
>
> - IOMMU_GET_HW_INFO
> - IOMMU_HWPT_ALLOC_NEST_PARENT
> - IOMMU_DOMAIN_NESTED
> - ops->enforce_cache_coherency()
>
> This is quite straightforward as the nested STE can just be built in the
> special NESTED domain op and fed through the generic update machinery.
>
> The design allows the user provided STE fragment to control several
> aspects of the translation, including putting the STE into a "virtual
> bypass" or a aborting state. This duplicates functionality available by
> other means, but it allows trivially preserving the VMID in the STE as we
> eventually move towards the vIOMMU owning the VMID.
>
> Nesting support requires the system to either support S2FWB or the
> stronger CANWBS ACPI flag. This is to ensure the VM cannot bypass the
> cache and view incoherent data, currently VFIO lacks any cache flushing
> that would make this safe.
>
> Yan has a series to add some of the needed infrastructure for VFIO cache
> flushing here:
>
> https://lore.kernel.org/linux-iommu/20240507061802.20184-1-yan.y.zhao@intel.com/
>
> Which may someday allow relaxing this further.
>
> Remove VFIO_TYPE1_NESTING_IOMMU since it was never used and superseded by
> this.
>
> This is the first series in what will be several to complete nesting
> support. At least:
> - IOMMU_RESV_SW_MSI related fixups
> https://lore.kernel.org/linux-iommu/cover.1722644866.git.nicolinc@nvidia.com/
> - vIOMMU object support to allow ATS and CD invalidations
> https://lore.kernel.org/linux-iommu/cover.1723061377.git.nicolinc@nvidia.com/
> - vCMDQ hypervisor support for direct invalidation queue assignment
> https://lore.kernel.org/linux-iommu/cover.1712978212.git.nicolinc@nvidia.com/
> - KVM pinned VMID using vIOMMU for vBTM
> https://lore.kernel.org/linux-iommu/20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com/
> - Cross instance S2 sharing
> - Virtual Machine Structure using vIOMMU (for vMPAM?)
> - Fault forwarding support through IOMMUFD's fault fd for vSVA
>
> The vIOMMU series is essential to allow the invalidations to be processed
> for the CD as well.
>
> It is enough to allow qemu work to progress.
>
> This is on github: https://github.com/jgunthorpe/linux/commits/smmuv3_nesting
>
> v3:
> - Rebase on v6.12-rc2
> - Revise commit messages
> - Consolidate CANWB checks into arm_smmu_master_canwbs()
> - Add CONFIG_ARM_SMMU_V3_IOMMUFD to compile out iommufd only features
> like nesting
> - Shift code into arm-smmu-v3-iommufd.c
> - Add missed IS_ERR check
> - Add S2FWB to arm_smmu_get_ste_used()
> - Fixup quirks checks
> - Drop ARM_SMMU_FEAT_COHERENCY checks for S2FWB
> - Limit S2FWB to S2 Nesting Parent domains "just in case"
> v2: https://patch.msgid.link/r/0-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com
> - Revise commit messages
> - Guard S2FWB support with ARM_SMMU_FEAT_COHERENCY, since it doesn't make
> sense to use S2FWB to enforce coherency on inherently non-coherent hardware.
> - Add missing IO_PGTABLE_QUIRK_ARM_S2FWB validation
> - Include formal ACPIA commit for IORT built using
> generate/linux/gen-patch.sh
> - Use FEAT_NESTING to block creating a NESTING_PARENT
> - Use an abort STE instead of non-valid if the user requests a non-valid
> vSTE
> - Consistently use 'nest_parent' for naming variables
> - Use the right domain for arm_smmu_remove_master_domain() when it
> removes the master
> - Join bitfields together
> - Drop arm_smmu_cache_invalidate_user patch, invalidation will
> exclusively go via viommu
> v1: https://patch.msgid.link/r/0-v1-54e734311a7f+14f72-smmuv3_nesting_jgg@nvidia.com
>
> Jason Gunthorpe (6):
> vfio: Remove VFIO_TYPE1_NESTING_IOMMU
> iommu/arm-smmu-v3: Report IOMMU_CAP_ENFORCE_CACHE_COHERENCY for CANWBS
> iommu/arm-smmu-v3: Implement IOMMU_HWPT_ALLOC_NEST_PARENT
> iommu/arm-smmu-v3: Expose the arm_smmu_attach interface
> iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED
> iommu/arm-smmu-v3: Use S2FWB for NESTED domains
>
> Nicolin Chen (3):
> ACPICA: IORT: Update for revision E.f
> ACPI/IORT: Support CANWBS memory access flag
> iommu/arm-smmu-v3: Support IOMMU_GET_HW_INFO via struct
> arm_smmu_hw_info
>
> drivers/acpi/arm64/iort.c | 13 ++
> drivers/iommu/Kconfig | 9 +
> drivers/iommu/arm/arm-smmu-v3/Makefile | 1 +
> .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 204 ++++++++++++++++++
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 114 ++++++----
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 83 ++++++-
> drivers/iommu/arm/arm-smmu/arm-smmu.c | 16 --
> drivers/iommu/io-pgtable-arm.c | 27 ++-
> drivers/iommu/iommu.c | 10 -
> drivers/iommu/iommufd/vfio_compat.c | 7 +-
> drivers/vfio/vfio_iommu_type1.c | 12 +-
> include/acpi/actbl2.h | 3 +-
> include/linux/io-pgtable.h | 2 +
> include/linux/iommu.h | 5 +-
> include/uapi/linux/iommufd.h | 55 +++++
> include/uapi/linux/vfio.h | 2 +-
> 16 files changed, 465 insertions(+), 98 deletions(-)
> create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
>
>
> base-commit: 8cf0b93919e13d1e8d4466eb4080a4c4d9d66d7b
> --
> 2.46.2
>

Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com>