Re: [RFC PATCH v3 06/15] hw/arm/smmuv3-accel: Restrict accelerated SMMUv3 to vfio-pci endpoints with iommufd

[Nicolin Chen <nicolinc@nvidia.com>]

On Wed, Jul 16, 2025 at 10:51:23AM -0700, Nicolin Chen wrote:
> On Wed, Jul 16, 2025 at 09:34:04AM +0000, Shameerali Kolothum Thodi wrote:
> > > >> Seems aggressive for a hotplug, could we fail hotplug instead of kill
> > > QEMU?
> > > >
> > > >Hotplug will unlikely be supported well, as it would introduce
> > > >too much complication.
> > > >
> > > >With iommufd, a vIOMMU object is allocated per device (vfio). If
> > > >the device fd (cdev) is not yet given to the QEMU. It isn't able
> > > >to allocate a vIOMMU object when creating a VM.
> > > >
> > > >While a vIOMMU object can be allocated at a later stage once the
> > > >device is hotplugged. But things like IORT mappings aren't able
> > > >to get refreshed since the OS is likely already booted. Even an
> > > >IOMMU capability sync via the hw_info ioctl will be difficult to
> > > >do at the runtime post the guest iommu driver's initialization.
> > > >
> > > >I am not 100% sure. But I think Intel model could have a similar
> > > >problem if the guest boots with zero cold-plugged device and then
> > > >hot-plugs a PASID-capable device at the runtime, when the guest-
> > > >level IOMMU driver is already inited?
> > > 
> > > For vtd we define a property for each capability we care about.
> > > When hotplug a device, we get hw_info through ioctl and compare
> > > host's capability with virtual vtd's property setting, if incompatible,
> > > we fail the hotplug.
> > > 
> > > In old implementation we sync host iommu caps into virtual vtd's cap,
> > > but that's Naked by maintainer. The suggested way is to define property
> > > for each capability we care and do compatibility check.
> > > 
> > > There is a "pasid" property in virtual vtd, only when it's true, the PASID-
> > > capable
> > > device can work with pasid.
> > 
> > Thanks for this information. I think probably we need to take a look at this as
> > this doesn't have a dependency on cold-plug device to be present for SMMUv3.
> > Will go through intel vtd implementation.
> 
> I see. A compatibility test sounds promising.
> 
> It still feels tricky when dealing with multi vSMMU instances, if
> some instances don't have a cold-plug device to poll hw_info. We
> would need to pre-define all the feature bits. Then, run the test
> on every hotplug device attached later to the vSMMU instance.
> 
> Maybe we could do something wise:
> The sysfs node provides all the IOMMU nodes. So, we could compare
> the node names to see if they are likely symmetric or not. Nodes
> sharing the same naming pattern are more likely created by the
> same IOMMU driver. So, as a speculation, a vSMMU instance with no
> coldplug device could borrow the bits from a vSMMU instance with
> a device?
> 
> Sure, individual IOMMU instances could differ in specific fields
> despite using the same node name. This would unfortunately lead
> to hotplug failure upon the compatibility check.

Hmm, forget what I said here. Each vSMMU instance should be pre
defined with a list of parameters. So, we will need to run the
compatibility test not only for hotplug devices, but coldplug
ones too.

Thanks
Nicolin