Re: [Qemu-devel] [PATCH v2 0/2] intel_iommu: Add support for translation for devices behind bridges

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Wed, 2015-09-02 at 10:12 -0600, Alex Williamson wrote:

> There are very specific rules for translating requester IDs across
> bridges.  Bus numbers can change during enumeration, devfn cannot.
> devfn can however be masked by topology changes from PCIe to PCI.  If 
> we pretend that the IOMMU can distinguish requester IDs where it 
> can't on real hardware, we're going to break the guest.  Thanks,

Note that whether a PCI / PCI-X bridge will mask devfn, bus# or both or
even mask it partially (number of bits) or replace some transfers with
its own RID ... depends on a given bridge implementation.

Another thing is while I agree that the bus number is problematic,
since it changes, it is still what the HW actually uses to match the
requester in practice, at least on PHB and I would think on Intel.

The problem is more fundamental. qemu is trying to bind devices to
address spaces in a fixed way at device creation time, while this is
lazily resolved in HW at the point of the DMA occurring.

One way to fix it is to effectively have an address space per device,
and have the iommu translate function figure out the binding
dynamically and flush things if it detects a change. But that is tricky
for vfio and it means invalidations will have to iterate all address
spaces.

The other option is to create Address Spaces on the fly as we lookup
domains, and bind them to devices lazily, but again, we need to deal
with changes/invalidations and that can be nasty with VFIO.

Sadly, I can't think of a silver bullet.

Cheers,
Ben.

> Alex
> 
> > > I would suggest to address them so it will be easier to continue
> > > the 
> > > review process.
> > 
> > Knut
> > 
> > > Thank you,
> > > Marcel
> > > 
> > > > 
> > > > This is the thread following the initial patch set:
> > > > 
> > > >    http://thread.gmane.org/gmane.comp.emulators.qemu/302246
> > > > 
> > > > The patch set was also discussed in this thread:
> > > > 
> > > >    http://thread.gmane.org/gmane.comp.emulators.qemu/316949
> > > > 
> > > > Changes from v1:
> > > >    - Rebased to current master
> > > >    - Fixed minor syntax issues
> > > > 
> > > > Knut Omang (2):
> > > >    iommu: Replace bus+devfn arguments with PCIDevice* in 
> > > > PCIIOMMUFunc
> > > >    intel_iommu: Add support for translation for devices behind 
> > > > bridges.
> > > > 
> > > >   hw/alpha/typhoon.c            |  2 +-
> > > >   hw/i386/intel_iommu.c         | 56 +++++++++++++++++++-------
> > > > ----
> > > > -------------
> > > >   hw/pci-host/apb.c             |  2 +-
> > > >   hw/pci-host/prep.c            |  3 +--
> > > >   hw/pci-host/q35.c             | 42 +++++++++++++-------------
> > > > ----
> > > > --
> > > >   hw/pci/pci.c                  |  7 +++---
> > > >   hw/pci/pci_bridge.c           |  6 +++++
> > > >   hw/ppc/spapr_pci.c            |  2 +-
> > > >   include/hw/i386/intel_iommu.h |  6 +++--
> > > >   include/hw/pci/pci.h          |  5 +++-
> > > >   10 files changed, 62 insertions(+), 69 deletions(-)
> > > > 
> > > > --
> > > > 2.4.3
> > > > 
> > > 
> 
>