Re: [Qemu-devel] vfio with iommu groups

[Alex Williamson <alex.williamson@redhat.com>]

On Sat, 2012-05-12 at 17:31 +1000, Alexey Kardashevskiy wrote:
> Hi!
> 
> I pulled new VFIO from github, ported to POWER and got some issues/thoughts which I post as patches.
> However PCI bridges handling is an open question to discuss.
> 
> My test setup includes PCIe card Intel E1000E which looks in the host like this
> (cut device names in the tree below as they were too long to look nice):
> aik@vpl2:~$ lspci -vt
>  +-[0003:00]---00.0-[01-ff]----00.0-[02-04]--+-02.0-[03]--+-00.0  Intel Gigabit Ethernet
>  |                                           |            \-00.1  Intel Gigabit Ethernet
>  |                                           \-04.0-[04]--+-00.0  Intel Gigabit Ethernet
>  |                                                        \-00.1  Intel Gigabit Ethernet
> 
> aik@vpl2:~$ lspci
> ...
> 0003:00:00.0 PCI bridge: IBM Device 02ea (rev 02)
> 0003:01:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:03:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:03:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 
> 
> At the moment I bind all devices from domain 0003 to the vfio_pci driver and give as many
> devices from it to QEMU as I want. vfio-pci owns devices, the guest sees what he needs to see -
> ethernet adapters, it works.

Yay!

> However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
> For example, INTx lines should be swizzled when the guest parses a device tree and
> tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
> functions.
> 
> Is there any idea what to do with bridges?

I don't really have a problem with the idea of exposing bridges, but it
get's pretty complicated.  What happens if the guest reprograms the
subordinate bus numbers?  Is there any advantage vs exposing an emulated
bridge in it's place?  Do you have a proposal for it?  Thanks,

Alex