Re: [RFC PATCH v1 00/38] ARM CCA Device Assignment support

Jason Gunthorpe
2025-08-05T17:27:43Z

On Tue, Aug 05, 2025 at 10:37:01AM +0530, Aneesh Kumar K.V wrote:
> > To me it is an unfortunate PCI specification wrinkle that writing to the
> > command register drops the device from RUN to ERROR. So you can LOCK
> > without setting BME, but then no DMA.
>
> This is only w.r.t clearing BME isn't ?
>
> According to section 11.2.6 DSM Tracking and Handling of Locked TDI Configurations
>
> Clearing any of the following bits causes the TDI hosted
> by the Function to transition to ERROR:
>
> • Memory Space Enable
> • Bus Master Enable

Oh that's nice, yeah!

> Which implies the flow described in the cover-letter where driver enable the BME works?
> However clearing BME may be problematic? I did have a FIXME!!/comment in [1]
>
> vfio_pci_core_close_device():
>
> #if 0
> /*
> * destroy vdevice which involves tsm unbind before we disable pci disable
> * A MSE/BME clear will transition the device to error state.
> */
> if (core_vdev->iommufd_device)
> iommufd_device_tombstone_vdevice(core_vdev->iommufd_device);
> #endif
>
> vfio_pci_core_disable(vdev);

Here is where I feel the VMM should be trapping this and NOPing it, or
failing that the guest PCI Core should NOP it.

With the ideal version being the TSM and VMM would be able to block
the iommu as a functional stand in for BME.

> Currently, we destroy (TSM unbind) the vdevice after calling
> vfio_pci_core_disable(), which means BME is cleared before unbinding,
> and the TDI transitions to the ERROR state.

I don't think this ordering is deliberate, we can destroy the vdevice
much earlier??

Jason