Re: [PATCH v2 11/16] iommu: Compute iommu_groups properly for PCIe MFDs

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Jul 28, 2025 at 11:47:44AM +0200, Cédric Le Goater wrote:
> > +	for_each_set_bit(devfn, devfns.devfns, NR_DEVFNS) {
> > +		struct iommu_group *group;
> > +		struct pci_dev *pdev_slot;
> > +
> > +		pdev_slot = pci_get_slot(pdev->bus, devfn);
> > +		group = iommu_group_get(&pdev_slot->dev);
> > +		pci_dev_put(pdev_slot);
> > +		if (group) {
> > +			if (WARN_ON(!(group->bus_data &
> > +				      BUS_DATA_PCI_NON_ISOLATED)))
> > +				group->bus_data |= BUS_DATA_PCI_NON_ISOLATED;
> > +			return group;
> > +		}
> > +	}
> > +	return pci_group_alloc_non_isolated();
> >   }
> >   static struct iommu_group *pci_hierarchy_group(struct pci_dev *pdev)
> 
> 
> I am seeing this WARN_ON when creating VFs of a CX-7 adapter :
> 
> [   31.436294] pci 0000:b1:00.2: enabling Extended Tags
> [   31.448767] ------------[ cut here ]------------
> [   31.453392] WARNING: CPU: 47 PID: 1673 at drivers/iommu/iommu.c:1533 pci_device_group+0x307/0x3b0
> ....

Ah, yeah, it thinks the SRIOV VFs are part of the MFD because they
match the slot. I guess the old code had this same issue but it was
more harmless.

>  *IOMMU Group 11 :
> 	b1:00.0 Ethernet controller: Mellanox Technologies MT2910 Family [ConnectX-7]
> 	b1:00.2 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
> 	b1:00.3 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
> 	b1:00.4 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
> 	b1:00.5 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
> 	b1:00.6 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
> 	b1:00.7 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function

This are all on the MFD's slot slot..

>  *IOMMU Group 12 :
> 	b1:00.1 Ethernet controller: Mellanox Technologies MT2910 Family [ConnectX-7]

The one is basically what triggered the WARN_ON, it was done before
SRIOV "changed" the slot.

>  *IOMMU Group 184 :
> 	b1:01.0 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
>  *IOMMU Group 185 :
> 	b1:01.1 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function

These two are on another slot so they no longer matched

It should be fixed by ignoring VFs when doing MFD matching.

> Other differences are on the onboard graphic card:
> 
>  *IOMMU Group 26 :
> 	02:00.0 PCI bridge: PLDA PCI Express Bridge (rev 02)
> 	03:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. Integrated Matrox G200eW3 Graphics Controller (rev 04)
> 
> becomes :
> 
>  *IOMMU Group 26 :
> 	02:00.0 PCI bridge: PLDA PCI Express Bridge (rev 02)
>  *IOMMU Group 27 :
> 	03:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. Integrated Matrox G200eW3 Graphics Controller (rev 04)

Yes, this is a deliberate improvement. PCIe to PCI bridges like this
do keep the bridge isolated from the VGA. Only the downstream PCI
devices alias with each other and are non-isolated.

Thanks,
Jason