Re: [PATCH v7 00/17] Add a host IOMMU device abstraction to check with vIOMMU

["Cédric Le Goater" <clg@redhat.com>]

On 6/24/24 12:26 PM, Michael S. Tsirkin wrote:
> On Wed, Jun 05, 2024 at 04:30:26PM +0800, Zhenzhong Duan wrote:
>> Hi,
>>
>> This series introduce a HostIOMMUDevice abstraction and sub-classes.
>> Also HostIOMMUDeviceCaps structure in HostIOMMUDevice and a new interface
>> between vIOMMU and HostIOMMUDevice.
> 
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> 
> Who is merging this? Me? Or Alex?

I will and I will include this series also :

   [v4] VIRTIO-IOMMU/VFIO: Fix host iommu geometry
   https://lore.kernel.org/all/20240614095402.904691-1-eric.auger@redhat.com

Thanks,

C.


> 
> 
> 
>> A HostIOMMUDevice is an abstraction for an assigned device that is protected
>> by a physical IOMMU (aka host IOMMU). The userspace interaction with this
>> physical IOMMU can be done either through the VFIO IOMMU type 1 legacy
>> backend or the new iommufd backend. The assigned device can be a VFIO device
>> or a VDPA device. The HostIOMMUDevice is needed to interact with the host
>> IOMMU that protects the assigned device. It is especially useful when the
>> device is also protected by a virtual IOMMU as this latter use the translation
>> services of the physical IOMMU and is constrained by it. In that context the
>> HostIOMMUDevice can be passed to the virtual IOMMU to collect physical IOMMU
>> capabilities such as the supported address width. In the future, the virtual
>> IOMMU will use the HostIOMMUDevice to program the guest page tables in the
>> first translation stage of the physical IOMMU.
>>
>> HostIOMMUDeviceClass::realize() is introduced to initialize
>> HostIOMMUDeviceCaps and other fields of HostIOMMUDevice variants.
>>
>> HostIOMMUDeviceClass::get_cap() is introduced to query host IOMMU
>> device capabilities.
>>
>> The class tree is as below:
>>
>>                                HostIOMMUDevice
>>                                       | .caps
>>                                       | .realize()
>>                                       | .get_cap()
>>                                       |
>>              .-----------------------------------------------.
>>              |                        |                      |
>> HostIOMMUDeviceLegacyVFIO  {HostIOMMUDeviceLegacyVDPA}  HostIOMMUDeviceIOMMUFD
>>              |                        |                      | [.iommufd]
>>                                                              | [.devid]
>>                                                              | [.ioas_id]
>>                                                              | [.attach_hwpt()]
>>                                                              | [.detach_hwpt()]
>>                                                              |
>>                                              .----------------------.
>>                                              |                      |
>>                           HostIOMMUDeviceIOMMUFDVFIO  {HostIOMMUDeviceIOMMUFDVDPA}
>>                                            | [.vdev]                | {.vdev}
>>
>> * The attributes in [] will be implemented in nesting series.
>> * The classes in {} will be implemented in future.
>> * .vdev in different class points to different agent device,
>> * i.e., VFIODevice or VDPADevice.
>>
>> PATCH1-4: Introduce HostIOMMUDevice and its sub classes
>> PATCH5-10: Implement .realize() and .get_cap() handler
>> PATCH11-14: Create HostIOMMUDevice instance and pass to vIOMMU
>> PATCH15-17: Implement compatibility check between host IOMMU and vIOMMU(intel_iommu)
>>
>> Test done:
>> make check
>> vfio device hotplug/unplug with different backend on linux
>> reboot, kexec
>> build test on linux and windows11
>>
>> Qemu code can be found at:
>> https://github.com/yiliu1765/qemu/tree/zhenzhong/iommufd_nesting_preq_v7
>>
>> Besides the compatibility check in this series, in nesting series, this
>> host IOMMU device is extended for much wider usage. For anyone interested
>> on the nesting series, here is the link:
>> https://github.com/yiliu1765/qemu/tree/zhenzhong/iommufd_nesting_rfcv2
>>
>> Thanks
>> Zhenzhong
>>
>> Changelog:
>> v7:
>> - drop config CONFIG_HOST_IOMMU_DEVICE (Cédric)
>> - introduce HOST_IOMMU_DEVICE_CAP_AW_BITS_MAX (Eric)
>> - use iova_ranges method in iommufd.realize() (Eric)
>> - introduce HostIOMMUDevice::name to facilitate tracing (Eric)
>> - implement a custom destroy hash function (Cédric)
>> - drop VTDHostIOMMUDevice and save HostIOMMUDevice in hash table (Eric)
>> - move patch5 after patch1 (Eric)
>> - squash patch3 and 4, squash patch12 and 13 (Eric)
>> - refine comments (Eric)
>> - collect Eric's R-B
>>
>> v6:
>> - open coded host_iommu_device_get_cap() to avoid #ifdef in intel_iommu.c (Cédric)
>>
>> v5:
>> - pci_device_set_iommu_device return true (Cédric)
>> - fix build failure on windows (thanks Cédric found that issue)
>>
>> v4:
>> - move properties vdev, iommufd and devid to nesting series where need it (Cédric)
>> - fix 32bit build with clz64 (Cédric)
>> - change check_cap naming to get_cap (Cédric)
>> - return bool if error is passed through errp (Cédric)
>> - drop HostIOMMUDevice[LegacyVFIO|IOMMUFD|IOMMUFDVFIO] declaration (Cédric)
>> - drop HOST_IOMMU_DEVICE_CAP_IOMMUFD (Cédric)
>> - replace include directive with forward declaration (Cédric)
>>
>> v3:
>> - refine declaration and doc for HostIOMMUDevice (Cédric, Philippe)
>> - introduce HostIOMMUDeviceCaps, .realize() and .check_cap() (Cédric)
>> - introduce helper range_get_last_bit() for range operation (Cédric)
>> - separate pci_device_get_iommu_bus_devfn() in a prereq patch (Cédric)
>> - replace HIOD_ abbreviation with HOST_IOMMU_DEVICE_ (Cédric)
>> - add header in include/sysemu/iommufd.h (Cédric)
>>
>> v2:
>> - use QOM to abstract host IOMMU device and its sub-classes (Cédric)
>> - move host IOMMU device creation in attach_device() (Cédric)
>> - refine pci_device_set/unset_iommu_device doc further (Eric)
>> - define host IOMMU info format of different backend
>> - implement get_host_iommu_info() for different backend (Cédric)
>> - drop cap/ecap update logic (MST)
>> - check aw-bits from get_host_iommu_info() in legacy mode
>>
>> v1:
>> - use HostIOMMUDevice handle instead of union in VFIODevice (Eric)
>> - change host_iommu_device_init to host_iommu_device_create
>> - allocate HostIOMMUDevice in host_iommu_device_create callback
>>    and set the VFIODevice base_hdev handle (Eric)
>> - refine pci_device_set/unset_iommu_device doc (Eric)
>> - use HostIOMMUDevice handle instead of union in VTDHostIOMMUDevice (Eric)
>> - convert HostIOMMUDevice to sub object pointer in vtd_check_hdev
>>
>> rfcv2:
>> - introduce common abstract HostIOMMUDevice and sub struct for different BEs (Eric, Cédric)
>> - remove iommufd_device.[ch] (Cédric)
>> - remove duplicate iommufd/devid define from VFIODevice (Eric)
>> - drop the p in aliased_pbus and aliased_pdevfn (Eric)
>> - assert devfn and iommu_bus in pci_device_get_iommu_bus_devfn (Cédric, Eric)
>> - use errp in iommufd_device_get_info (Eric)
>> - split and simplify cap/ecap check/sync code in intel_iommu.c (Cédric)
>> - move VTDHostIOMMUDevice declaration to intel_iommu_internal.h (Cédric)
>> - make '(vtd->cap_reg >> 16) & 0x3fULL' a MACRO and add missed '+1' (Cédric)
>> - block migration if vIOMMU cap/ecap updated based on host IOMMU cap/ecap
>> - add R-B
>>
>> Yi Liu (2):
>>    hw/pci: Introduce pci_device_[set|unset]_iommu_device()
>>    intel_iommu: Implement [set|unset]_iommu_device() callbacks
>>
>> Zhenzhong Duan (15):
>>    backends: Introduce HostIOMMUDevice abstract
>>    backends/host_iommu_device: Introduce HostIOMMUDeviceCaps
>>    vfio/container: Introduce TYPE_HOST_IOMMU_DEVICE_LEGACY_VFIO device
>>    backends/iommufd: Introduce TYPE_HOST_IOMMU_DEVICE_IOMMUFD[_VFIO]
>>      devices
>>    range: Introduce range_get_last_bit()
>>    vfio/container: Implement HostIOMMUDeviceClass::realize() handler
>>    backends/iommufd: Introduce helper function
>>      iommufd_backend_get_device_info()
>>    vfio/iommufd: Implement HostIOMMUDeviceClass::realize() handler
>>    vfio/container: Implement HostIOMMUDeviceClass::get_cap() handler
>>    backends/iommufd: Implement HostIOMMUDeviceClass::get_cap() handler
>>    vfio: Create host IOMMU device instance
>>    hw/pci: Introduce helper function pci_device_get_iommu_bus_devfn()
>>    vfio/pci: Pass HostIOMMUDevice to vIOMMU
>>    intel_iommu: Extract out vtd_cap_init() to initialize cap/ecap
>>    intel_iommu: Check compatibility with host IOMMU capabilities
>>
>>   MAINTAINERS                           |   2 +
>>   include/hw/i386/intel_iommu.h         |   2 +
>>   include/hw/pci/pci.h                  |  38 ++++-
>>   include/hw/vfio/vfio-common.h         |   8 +
>>   include/hw/vfio/vfio-container-base.h |   3 +
>>   include/qemu/range.h                  |  11 ++
>>   include/sysemu/host_iommu_device.h    |  91 ++++++++++++
>>   include/sysemu/iommufd.h              |  19 +++
>>   backends/host_iommu_device.c          |  33 +++++
>>   backends/iommufd.c                    |  76 ++++++++--
>>   hw/i386/intel_iommu.c                 | 203 ++++++++++++++++++++------
>>   hw/pci/pci.c                          |  75 +++++++++-
>>   hw/vfio/common.c                      |  16 +-
>>   hw/vfio/container.c                   |  41 +++++-
>>   hw/vfio/helpers.c                     |  17 +++
>>   hw/vfio/iommufd.c                     |  37 ++++-
>>   hw/vfio/pci.c                         |  19 ++-
>>   backends/meson.build                  |   1 +
>>   18 files changed, 623 insertions(+), 69 deletions(-)
>>   create mode 100644 include/sysemu/host_iommu_device.h
>>   create mode 100644 backends/host_iommu_device.c
>>
>> -- 
>> 2.34.1
>