[PATCH RFCv1 00/14] Add Tegra241 (Grace) CMDQV Support (part 2/2)

Nicolin Chen
2024-04-13T03:47:51Z

This is an experimental RFC series for VIOMMU infrastructure, using NVIDIA
Tegra241 (Grace) CMDQV as a test instance.

VIOMMU obj is used to represent a virtual interface (iommu) backed by an
underlying IOMMU's HW-accelerated feature for virtualizaion: for example,
NVIDIA's VINTF (v-interface for CMDQV) and AMD"s vIOMMU.

VQUEUE obj is used to represent a virtual command queue (buffer) backed by
an underlying IOMMU command queue to passthrough for VMs to use directly:
for example, NVIDIA's Virtual Command Queue and AMD's Command Buffer.

NVIDIA's CMDQV requires a pair of physical and virtual device Stream IDs
to process ATC invalidation commands by ARM SMMU. So, set/unset_dev_id ops
and ioctls are introduced to VIOMMU.

Also, a passthrough queue has a pair of start and tail pointers/indexes in
the real HW registers, which should be mmaped to user space for hypervisor
to map to VM's mmio region directly. Thus, iommufd needs an mmap op too.

Some todos/opens:
1. Add selftest coverages for new ioctls
2. The mmap needs a way to get viommu_id. Currently it's getting from
vma->vm_pgoff, which might not be ideal.
3. This series is only verified with a single passthrough device that's
hehind a physical ARM SMMU. So, devices behind two+ IOMMUs might need
some additional support (and verifications).
4. Requires for comments from AMD folks to support AMD's vIOMMU feature.

This series is on Github (for review and reference only):
https://github.com/nicolinc/iommufd/commits/vcmdq_user_space-rfc-v1

Real HW tests wre conducted with this QEMU branch:
https://github.com/nicolinc/qemu/commits/wip/iommufd_vcmdq/

Thanks

Nicolin Chen (14):
iommufd: Move iommufd_object to public iommufd header
iommufd: Swap _iommufd_object_alloc and __iommufd_object_alloc
iommufd: Prepare for viommu structures and functions
iommufd: Add struct iommufd_viommu and iommufd_viommu_ops
iommufd: Add IOMMUFD_OBJ_VIOMMU and IOMMUFD_CMD_VIOMMU_ALLOC
iommufd/selftest: Add IOMMU_VIOMMU_ALLOC test coverage
iommufd: Add viommu set/unset_dev_id ops
iommufd: Add IOMMU_VIOMMU_SET_DEV_ID ioctl
iommufd/selftest: Add IOMMU_VIOMMU_SET_DEV_ID test coverage
iommufd/selftest: Add IOMMU_TEST_OP_MV_CHECK_DEV_ID
iommufd: Add struct iommufd_vqueue and its related viommu ops
iommufd: Add IOMMUFD_OBJ_VQUEUE and IOMMUFD_CMD_VQUEUE_ALLOC
iommufd: Add mmap infrastructure
iommu/tegra241-cmdqv: Add user-space use support

drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 19 ++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 19 ++
.../iommu/arm/arm-smmu-v3/tegra241-cmdqv.c | 284 +++++++++++++++++-
drivers/iommu/iommufd/Makefile | 3 +-
drivers/iommu/iommufd/device.c | 11 +
drivers/iommu/iommufd/hw_pagetable.c | 4 +-
drivers/iommu/iommufd/iommufd_private.h | 71 +++--
drivers/iommu/iommufd/iommufd_test.h | 5 +
drivers/iommu/iommufd/main.c | 69 ++++-
drivers/iommu/iommufd/selftest.c | 100 ++++++
drivers/iommu/iommufd/viommu.c | 235 +++++++++++++++
include/linux/iommu.h | 16 +
include/linux/iommufd.h | 100 ++++++
include/uapi/linux/iommufd.h | 98 ++++++
tools/testing/selftests/iommu/iommufd.c | 44 +++
tools/testing/selftests/iommu/iommufd_utils.h | 71 +++++
16 files changed, 1103 insertions(+), 46 deletions(-)
create mode 100644 drivers/iommu/iommufd/viommu.c

--
2.43.0