Re: [PATCH 2/3] vmd: disable MSI remapping bypass under Xen

["Roger Pau Monné" <roger.pau@citrix.com>]

On Fri, Jan 10, 2025 at 04:25:25PM -0600, Bjorn Helgaas wrote:
> Match historical subject line style for prefix and capitalization:
> 
>   PCI: vmd: Set devices to D0 before enabling PM L1 Substates
>   PCI: vmd: Add DID 8086:B06F and 8086:B60B for Intel client SKUs
>   PCI: vmd: Fix indentation issue in vmd_shutdown()
> 
> On Fri, Jan 10, 2025 at 03:01:49PM +0100, Roger Pau Monne wrote:
> > MSI remapping bypass (directly configuring MSI entries for devices on the VMD
> > bus) won't work under Xen, as Xen is not aware of devices in such bus, and
> > hence cannot configure the entries using the pIRQ interface in the PV case, and
> > in the PVH case traps won't be setup for MSI entries for such devices.
> > 
> > Until Xen is aware of devices in the VMD bus prevent the
> > VMD_FEAT_CAN_BYPASS_MSI_REMAP capability from being used when running as any
> > kind of Xen guest.
> 
> Wrap to fit in 75 columns.

Hm, OK, but isn't the limit 80 columns according to the kernel coding
style (Documentation/process/coding-style.rst)?

I don't mind adjusting, but if you are going to ask every submitter to
limit to 75 columns then the coding style document should be updated
to reflect that.

> Can you include a hint about *why* Xen is not aware of devices below
> VMD?  That will help to know whether it's a permanent unfixable
> situation or something that could be done eventually.

Xen would need to be made aware of the devices exposed behind the VMD
bridge, so it can manage them.  For example Xen is the entity that
controls the local APICs, and hence interrupts must be configured by
Xen.  Xen needs knowledge about the devices behind the VMD bridge,
and how to access those devices PCI config space to at least configure
MSI or MSI-X capabilities.  It could possibly be exposed similarly to
how Xen currently deals with ECAM areas.

None of this is present at the moment, could always be added later and
Linux be made aware that the limitation no longer applies.  That would
require changes in both Xen and Linux to propagate the VMD information
into Xen.

> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  drivers/pci/controller/vmd.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> > 
> > diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> > index 264a180403a0..d9b7510ace29 100644
> > --- a/drivers/pci/controller/vmd.c
> > +++ b/drivers/pci/controller/vmd.c
> > @@ -965,6 +965,15 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
> >  	struct vmd_dev *vmd;
> >  	int err;
> >  
> > +	if (xen_domain())
> > +		/*
> > +		 * Xen doesn't have knowledge about devices in the VMD bus.
> 
> Also here.

Would you be OK with something like:

"Xen doesn't have knowledge about devices in the VMD bus because the
config space of devices behind the VMD bridge is not known to Xen, and
hence Xen cannot discover or configure them in any way.

Bypass of MSI remapping won't work in that case as direct write by
Linux to the MSI entries won't result in functional interrupts, as
it's Xen the entity that manages the local APIC and must configure
interrupts."

Thanks, Roger.