Re: [RFC PATCH 13/15] iommufd: Persist iommu domains for live update

[Jason Gunthorpe <jgg@ziepe.ca>]

On Tue, Sep 30, 2025 at 01:02:31PM -0700, Samiullah Khawaja wrote:
> > There are HWPTs outside the IOAS so it is inconsisent.
> 
> This makes sense. But if I understand correctly a HWPT should be
> associated one way or another to a preserved device or IOAS. Also the
> nested ones will have parent HWPT. Can we not look at the dependencies
> here and find the HWPTs that need to preserved.

Maybe in some capacity, but I would say more of don't allow preserving
things that depend on things not already preserved somehow.

> > Finally we expect to discard the preserved HWPTs and replace them we
> > rebuilt ones at least as a first step. Userspace needs to sequence all
> > of this..
> 
> But if we discard the old HWPTs and replace them with the new ones, we
> shouldn't need labeling of the old HWPTs? We would definitely need to
> sequence the replacement and discard of the old ones, but that can
> also be inferred through the dependencies between the new HWPTs?

It depends how this ends up being designed and who is responsible to
free the restored iommu_domain.

The iommu core code should be restoring the iommu_domain as soon as
the attached device is plugged in and attaching the preserved domain
instead of something else during the device probe sequence

This logic should not be in drivers.

From there you either put the hwpt back into iommufd and have it free
the iommu_domain when it destroys the hwpt

Or you have the iommu core code free the iommu_domain at some point
after iommufd has replaced the attachment with a new iommu_domain?

I'm not sure which is a better option..

Also there is an interesting behavior to note that if the iommu driver
restores a domain then it will also prevent a non-vfio driver from
binding to that device.

Jason