RE: [PATCH v5 00/10] Enhance vfio PCI hot reset for vfio cdev device

Xu, Terrence
2023-05-18T05:52:16Z

> -----Original Message-----
> From: Liu, Yi L <yi.l.liu@intel.com>
> Subject: [PATCH v5 00/10] Enhance vfio PCI hot reset for vfio cdev device
>
> VFIO_DEVICE_PCI_HOT_RESET requires user to pass an array of group fds to
> prove that it owns all devices affected by resetting the calling device. While
> for cdev devices, user can use an iommufd-based ownership checking model
> and invoke VFIO_DEVICE_PCI_HOT_RESET with a zero-length fd array.
>
> This series first creates iommufd_access for noiommu devices to fill the gap
> for adding iommufd-based ownership checking model, then extends
> VFIO_DEVICE_GET_PCI_HOT_RESET_INFO to check ownership and return
> the check result and the devid of affected devices to user. In the end,
> extends the VFIO_DEVICE_PCI_HOT_RESET to accept zero-length fd array for
> hot-reset with cdev devices.
>
> The new hot reset method and updated _INFO ioctl are tested with the
> below qemu:
>
> https://github.com/yiliu1765/qemu/tree/iommufd_rfcv4.mig.reset.v4_var3
> (requires to test with the cdev kernel)
>
> Change log:
>
> v5:
> - Drop patch 01 of v4 (Alex)
> - Create noiommu_access for noiommu devices (Jason)
> - Reserve all negative iommufd IDs, hence VFIO can encode negative
> values (Jason)
> - Make vfio_iommufd_physical_devid() return -EINVAL if it's not called
> with a physical device or a noiommu device.
> - Add vfio_find_device_in_devset() in vfio_main.c (Alex)
> - Add iommufd_ctx_has_group() to replace
> vfio_devset_iommufd_has_group().
> Reason: vfio_devset_iommufd_has_group() only loops the devices within
> the given devset to check the iommufd an iommu_group, but an
> iommu_group
> can span into multiple devsets. So if failed to find the group in a
> devset doesn't mean the group is not owned by the iommufd. So here
> either
> needs to search all the devsets or add an iommufd API to check it. It
> appears an iommufd API makes more sense.
> - Adopt suggestions from Alex on patch 08 and 09 of v4, refine the hot-reset
> uapi description and minor tweaks
> - Use bitfields for bool members (Alex)
>
> v4: https://lore.kernel.org/kvm/20230426145419.450922-1-yi.l.liu@intel.com/
> - Rename the patch series subject
> - Patch 01 is moved from the cdev series
> - Patch 02, 06 are new per review comments in v3
> - Patch 03/04/05/07/08/09 are from v3 with updates
>
> v3: https://lore.kernel.org/kvm/20230401144429.88673-1-yi.l.liu@intel.com/
> - Remove the new _INFO ioctl of v2, extend the existing _INFO ioctl to
> report devid (Alex)
> - Add r-b from Jason
> - Add t-b from Terrence Xu and Yanting Jiang (mainly regression test)
>
> v2: https://lore.kernel.org/kvm/20230327093458.44939-1-yi.l.liu@intel.com/
> - Split the patch 03 of v1 to be 03, 04 and 05 of v2 (Jaon)
> - Add r-b from Kevin and Jason
> - Add patch 10 to introduce a new _INFO ioctl for the usage of device
> fd passing usage in cdev path (Jason, Alex)
>
> v1: https://lore.kernel.org/kvm/20230316124156.12064-1-yi.l.liu@intel.com/
>
> Regards,
> Yi Liu
>
> Yi Liu (10):
> vfio-iommufd: Create iommufd_access for noiommu devices
> vfio/pci: Update comment around group_fd get in
> vfio_pci_ioctl_pci_hot_reset()
> vfio/pci: Move the existing hot reset logic to be a helper
> vfio: Mark cdev usage in vfio_device
> iommufd: Reserve all negative IDs in the iommufd xarray
> vfio-iommufd: Add helper to retrieve iommufd_ctx and devid for
> vfio_device
> vfio: Add helper to search vfio_device in a dev_set
> iommufd: Add iommufd_ctx_has_group()
> vfio/pci: Extend VFIO_DEVICE_GET_PCI_HOT_RESET_INFO for vfio device
> cdev
> vfio/pci: Allow passing zero-length fd array in
> VFIO_DEVICE_PCI_HOT_RESET
>
> drivers/iommu/iommufd/device.c | 53 +++++++++
> drivers/iommu/iommufd/main.c | 2 +-
> drivers/vfio/iommufd.c | 63 ++++++++++-
> drivers/vfio/pci/vfio_pci_core.c | 184 ++++++++++++++++++++++++-------
> drivers/vfio/vfio_main.c | 15 +++
> include/linux/iommufd.h | 14 +++
> include/linux/vfio.h | 23 ++++
> include/uapi/linux/vfio.h | 60 +++++++++-
> 8 files changed, 368 insertions(+), 46 deletions(-)
>
> --
> 2.34.1

The new uapi works fine.
Tested GVT-g / GVT-d VFIO legacy mode / compat mode / cdev mode, including negative tests. No regression be introduced.

Tested-by: Terrence Xu <terrence.xu@intel.com>