Re: [RFC PATCH 13/15] iommufd: Persist iommu domains for live update

[Jason Gunthorpe <jgg@ziepe.ca>]

On Wed, Oct 01, 2025 at 06:00:58PM -0700, Samiullah Khawaja wrote:
> > No, finish should never do anything on the restore path, IMHO. User
> > should directly attach the newly created HWPT when it is ready.
> 
> Makes sense. But if the user never replaces the restored iommu_domain
> with a new HWPT, we will have to discard the old (restored) domain on
> finish since it doesn't have any associated HWPT. I see you already
> hinted at this below. This needs to be handled carefully considering
> the vfio cdev FD state also. Discussed further below.

I think the simplest thing is the domain exists forever until
userspace attaches an iommufd, takes ownership of it and frees it.
Nothing to do with finish.

While the domain is attached iommu_device_use_default_domain() will
fail.

> This is the part that I was concerned about since I was looking into
> the auto_domain. Users that attach to ioas directly and use
> auto_domain would not be able to restore the mappings before attaching
> to the device.

IMHO luo users need to be sophisticated enough to avoid auto_domain.

> That's a good point. But it might be tricky since the ownership of the
> device is with the vfio cdev FD. So if vfio cdev FD is never
> restored/reclaimed the device can be FLR'd. iommufd will follow along
> and discard the domain.

Honestly, I keep wanting things to be kept as simple as possible with
as few exception flows as necessary.

If we make it so that iommu_device_claim_dma_owner() is aware of luo
and the only way vfio can get ownership is if it is also restoring the
luo session then that sounds perfect.

Attaching a non-luo VFIO would be blocked by the kernel so we never
get these inconsistencies.

> The more interesting case might be where cdev is restored and bound to
> iommufd but the user never recreates and hotswaps a new HWPT. In this
> case we can discard the restored iommu_domain and replace it with the
> blocking domain as it should have been if the device was not
> preserved.

Maybe the HWPT has to be auto-created inside the iommufd as soon as it
is attached. The "restore" ioctl would just return back the ID of this
already created HWPT.

Again, this seems to avoid special cases as once we exit the special
luo mode of iommu_device_claim_dma_owner() iommufd is always
responsible for the iommu_domain.

Jason