Re: [PATCH v3 06/11] iommu: Compute iommu_groups properly for PCIe MFDs

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Sep 09, 2025 at 04:24:57PM -0500, Bjorn Helgaas wrote:
> > 
> >                       -- MFD 00:1f.0 ACS != REQ_ACS_FLAGS
> >       Root 00:00.00 --|- MFD 00:1f.2 ACS != REQ_ACS_FLAGS
> >                       |- MFD 00:1f.6 ACS = REQ_ACS_FLAGS
> 
> REQ_ACS_FLAGS was renamed in an earlier patch.
> 
> I don't quite understand the "Root 00:00.00" notation.  I guess it
> must refer to the root bus 00?  It looks sort of like a bridge, and
> the ".00" makes it look sort of like a bus/device/function address,
> but I don't think it is.

Call it the host bridge or whatever is creating the bus segment, it
doesn't actually matter for the examples.

> > However, the PCI spec does not really support this:
> > 
> >    PCI Section 6.12.1.2 ACS Functions in SR-IOV, SIOV, and Multi-Function
> >    Devices
> > 
> >     ACS P2P Request Redirect: must be implemented by Functions that
> >     support peer-to-peer traffic with other Functions.
> 
> I would include the PCIe r7.0 spec revision, even though the PCI SIG
> seems to try to preserve section numbers across revisions.
> 
> It seems pretty clear that Multi-Function Devices that have an ACS
> Capability and support peer-to-peer traffic with other Functions are
> required to implement ACS P2P Request Redirect.
> 
> > Meaning from a spec perspective the absence of ACS indicates the absence
> > of internal loopback. Granted I think we are aware of older real devices
> > that ignore this, but it seems to be the only way forward.
> 
> It's not as clear to me that Multi-Function Devices that support
> peer-to-peer traffic are required to have an ACS Capability at all.

How do you read it that way?

6.12.1.1 is reasonably clear that "This section applies to Root Ports
and Switch Downstream Ports that implement an ACS Extended Capability
structure."

While 6.12.2.2 is less so "This section applies to Multi-Function
Device ACS Functions"

I don't know what the author's intent was, but I have a hard time
reading the "must be implemented" as optional..

Frankly PCI SIG has made a mess here :(

> Alex might remember more, but I kind of suspect the current system of
> quirks is there because of devices that do internal loopback but have
> no ACS Capability.

This is correct, there are a few cases where it was confirmed that
internal loopback exists with no ACS

But mostly we have haphazardly added ACS quirks on demand whenever
someone was annoyed by what the current algorithm did. Most of the
investigations seem to have determined there is no actual loopback
suggesting people are reading the spec as above.

So, I don't see how to make it workable to default that most compliant
systems require quirks. Effectively this is a proposal to invert that
and only quirk those we know have internal loopback without ACS..

I will fix the other remarks

Thanks,
Jason