[RFC] virtio-iommu: Take into account possible aliasing in virtio_iommu_mr()

[Eric Auger <eric.auger@redhat.com>]

When devices are plugged downstream to a pcie to pci bridge,
the virtio-iommu driver stills sends virtio-iommu commands using
the rid without aliasing (ie. devfn != 0). However only an IOMMU
MR matching <bus>:00.0 was registered so virtio_iommu_mr fails
to identify the memory region matching the ep_id. This results
into the probe command failing and the devices beeing not
attached. As a result they are not associated to any iommu group.

Recognize the case where the device is plugged onto a pcie-to-pci
bridge and apply the mask to retrieve the right IOMMU MR.

This trick allows to have the correct iommu group topology on guest
with virtio-pci devices. However this is not sufficient to fix
the iommu group topology issue with basic emulated NIC or VFIO
devices. Even with that patch, once we attempt to plug such
devices downstream to the pcie-to-pci bridge, those devices are
put in a singleton group. The pcie-to-pci bridge disappears from
the radar (not attached to any group), and the pcie root port the
pcie-to-pci bridge is plugged onto is put in a separate singleton
group. So the topology is wrong and definitively different from the
one observed with the intel iommu.

I suspect some other issue in the viot/pci probing on guest kernel
side. I would appreciate on that kernel part.

qemu command excerpt:
for x86_64:

-device ioh3420,bus=pcie.0,chassis=1,id=pcie.1,addr=0x2.0x0
-device pcie-pci-bridge,id=pcie_pci_bridge1,bus=pcie.1
-netdev tap,id=mynet0,ifname=tap0,script=qemu-ifup,downscript=qemu-ifdown
-device e1000,netdev=mynet0,bus=pcie_pci_bridge1

guest pci topology:

-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Device 1234:1111
           +-02.0-[01-02]----00.0-[02]----00.0  Intel Corporation 82540EM Gigabit Ethernet Controller

wrong guest topology:
/sys/kernel/iommu_groups/2/devices/0000:00:02.0 (pcie root port)
/sys/kernel/iommu_groups/1/devices/0000:02:00.0 (E1000)
no group for 0000:01:00:0 (the pcie-to-pci bridge)

with intel iommu we get the following topology:
/sys/kernel/iommu_groups/2/devices/0000:02:00.0
/sys/kernel/iommu_groups/2/devices/0000:01:00.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0

on aarch64 we get the same using those devices:
-device pcie-root-port,bus=pcie.0,chassis=1,id=pcie.1,addr=0x2.0x0
-device pcie-pci-bridge,id=pcie_pci_bridge1,bus=pcie.1
-netdev tap,id=mynet0,ifname=tap0,script=qemu-ifup,downscript=qemu-ifdown
-device e1000,netdev=mynet0,bus=pcie_pci_bridge1

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 hw/virtio/virtio-iommu.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 23c470977e..58c367b744 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -178,6 +178,21 @@ static IOMMUMemoryRegion *virtio_iommu_mr(VirtIOIOMMU *s, uint32_t sid)
         dev = iommu_pci_bus->pbdev[devfn];
         if (dev) {
             return &dev->iommu_mr;
+        } else { /* check possible aliasing */
+            PCIBus *pbus = iommu_pci_bus->bus;
+
+            if (!pci_bus_is_express(pbus)) {
+                PCIDevice *parent = pbus->parent_dev;
+
+                if (pci_is_express(parent) &&
+                    pcie_cap_get_type(parent) == PCI_EXP_TYPE_PCI_BRIDGE) {
+                    devfn = PCI_DEVFN(0, 0);
+                    dev = iommu_pci_bus->pbdev[devfn];
+                    if (dev) {
+                        return &dev->iommu_mr;
+                    }
+                }
+            }
         }
     }
     return NULL;
-- 
2.31.1