Re: [PATCH 03/11] iommu: Compute iommu_groups properly for PCIe switches

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Sep 22, 2025 at 07:10:29PM -0600, Alex Williamson wrote:
> On Mon, 22 Sep 2025 20:15:41 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> > On Mon, Sep 22, 2025 at 04:32:00PM -0600, Alex Williamson wrote:
> > > The ACS capability was only introduced in PCIe 2.0 and vendors have
> > > only become more diligent about implementing it as it's become
> > > important for device isolation and assignment.    
> > 
> > IDK about this, I have very new systems and they still not have ACS
> > flags according to this interpretation.
> 
> But how can we assume that lack of a non-required capability means
> anything at all??
>  
> > > IMO, we can't assume anything at all about a multifunction device
> > > that does not implement ACS.  
> > 
> > Yeah this is all true. 
> > 
> > But we are already assuming. Today we assume MFDs without caps must
> > have internal loopback in some cases, and then in other cases we
> > assume they don't.
> 
> Where?  Is this in reference to our handling of multi-function
> endpoints vs whether downstream switch ports are represented as
> multi-function vs multi-slot?

If you have a MFD Linux with no ACS it will group the whole MFD if any
of it lacks ACS caps because it assumes there is an internal loopback
between functions.

If the MFD has a single function with ACS then only that function is
removed from the group. The only way we can understand this as correct
by our grouping definition is to require the MFD have no internal
loopback. ACS is an egress control, not an ingress control.

If a MFD function is a bridge/port then the group doesn't propogate
the group downstream of the bridge - again this requires assuming
there is no internal loopback between functions.

It is taking the undefined behavior in the spec and selectively making
both interpretations at once.

> > Assuming the MFD does not have internal loopback, while not entirely
> > satisfactory, is the one that gives the least practical breakage.
> 
> Seems like it's fixing one gap and opening another.  I don't see that we
> can implement ingress and egress isolation without breakage.  

Yeah, either we risk more insecurities or we risk large group sizes.

> We may need an opt-in to continue egress only isolation.

It isn't "egress only isolation" - the thing is I can't really
articulate what the current rules even fully are..

I'm not keen on an opt in. I'd rather find some rules we can live
with.

How about we answer the question "does this MFD have internal
loopback" as:
 - NO if any function has an appropriate ACS cap or quirk.
 - NO if any function is bridge/port
 - YES otherwise - all functions are end functions and no ACS declared

As above this is quite a bit closer to what Linux is doing now. It is
a practical estimation of the undefined spec behavior based on the
historical security posture of Linux.

> And hardware vendors are going to volunteer that they lack p2p
> isolation and we need to add a quirk to reduce the isolation...
> dynamics are not in our favor.  Hardware vendors have no incentive to
> do the right thing. 

They do, otherwise they have major security holes in
virtualization. In an enterprise setting I have no doubt it is already
being done right, and has been for a decade.

I think the above rules will broadly be pessimistic toward add in
cards and optimistic toward the root complex.

Jason