Re: [PATCH v4 00/17] Add iommufd physical device operations for replace and alloc hwpt

[Nicolin Chen <nicolinc@nvidia.com>]

On Mon, Mar 27, 2023 at 03:37:54PM -0300, Jason Gunthorpe wrote:
> This is the basic functionality for iommufd to support
> iommufd_device_replace() and IOMMU_HWPT_ALLOC for physical devices.
> 
> iommufd_device_replace() allows changing the HWPT associated with the
> device to a new IOAS or HWPT. Replace does this in way that failure leaves
> things unchanged, and utilizes the iommu iommu_group_replace_domain() API
> to allow the iommu driver to perform an optional non-disruptive change.
> 
> IOMMU_HWPT_ALLOC allows HWPTs to be explicitly allocated by the user and
> used by attach or replace. At this point it isn't very useful since the
> HWPT is the same as the automatically managed HWPT from the IOAS. However
> a following series will allow userspace to customize the created HWPT.
> 
> The implementation is complicated because we have to introduce some
> per-iommu_group memory in iommufd and redo how we think about multi-device
> groups to be more explicit. This solves all the locking problems in the
> prior attempts.
> 
> This series is infrastructure work for the following series which:
>  - Add replace for attach
>  - Expose replace through VFIO APIs
>  - Implement driver parameters for HWPT creation (nesting)
> 
> Once review of this is complete I will keep it on a side branch and
> accumulate the following series when they are ready so we can have a
> stable base and make more incremental progress. When we have all the parts
> together to get a full implementation it can go to Linus.
> 
> This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_hwpt
> 
> v4:
>  - Refine comments and commit messages
>  - Move the group lock into iommufd_hw_pagetable_attach()
>  - Fix error unwind in iommufd_device_do_replace()

This runs well with iommufd selftest on x86 and QEMU sanity on
ARM64, applying nesting series on top of this and cdev series:
https://github.com/nicolinc/iommufd/commits/wip/iommufd_nesting-03272023

Tested-by: Nicolin Chen <nicolinc@nvidia.com>