[Qemu-devel] Debugging PCI Enhanced Allocation - IOMMU of a bridge?

[Qemu-devel] Debugging PCI Enhanced Allocation - IOMMU of a bridge?

[Alexander von Gluck IV]

Good morning!

I'm working through adding support (or at least ignoring) PCI Enhance Allocation devices under Haiku.
We run into one of these devices, and crash at boot due to attempting to read an invalid address.

Our testing showed that with a USB C dongle plugged in, we don't crash. Without it plugged in we get
invalid addresses from PCI

(for a long-drawn out debug ticket, see https://dev.haiku-os.org/ticket/12885 )
So, i'm trying to see if I can emulate a PCI bridge with EA under QEMU, I found this patch, but it seems  it wasn't accepted:   https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02073.html (https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02073.html)

My other thought is since I have access to two laptops implementing this, I could attach the bridge to
qemu via iommu, but the new vfio stuff doesn't seem to work with bridges. :-|   Anyone know of a way to:  * emulate a PCI EA in qemu.  * attach the local PCI EA bridge to qemu to troubleshoot using real hardware.  For context, here is the suspect EA device:
(no USB C dongle)
$ lspci -tvnn
-[0000:00]-+-00.0 Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [8086:1910]
 +-1d.6-[06-3e]--

vs

(USB C dongle)
$ lspci -tvnn
-[0000:00]-+-00.0 Intel Corporation Skylake Host Bridge/DRAM Registers [8086:1904]
 +-1d.6-[06-3e]----00.0-[07-3e]--+-00.0-[08]--
 | +-01.0-[09-3d]--
 | -02.0-[3e]----00.0 Intel Corporation DSL6340 USB 3.1 Controller [Alpine Ridge] [8086:15b5]
The bride is at 1d.6 in it's own IOMMU group:

IOMMU group 12
 00:1d.6 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #15 [8086:a11e] (rev f1)

 -- Alex

Re: [Qemu-devel] Debugging PCI Enhanced Allocation - IOMMU of a bridge?

[Michael S. Tsirkin]

On Wed, Sep 12, 2018 at 05:30:29PM +0000, Alexander von Gluck IV wrote:
> Good morning!
> 
> I'm working through adding support (or at least ignoring) PCI Enhance
> Allocation devices under Haiku.
> We run into one of these devices, and crash at boot due to attempting to read
> an invalid address.
> 
> Our testing showed that with a USB C dongle plugged in, we don't crash. Without
> it plugged in we get
> invalid addresses from PCI
> 
> (for a long-drawn out debug ticket, see https://dev.haiku-os.org/ticket/12885 )
> 
> 
> So, i'm trying to see if I can emulate a PCI bridge with EA under QEMU, I found
> this patch, but it seems
> it wasn't accepted:
> https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02073.html

v2 never surfaced I think.

> My other thought is since I have access to two laptops implementing this, I
> could attach the bridge to
> qemu via iommu, but the new vfio stuff doesn't seem to work with bridges. :-|

It's fixable, AFAIK no one bothered though. BTW old style kvm dev
assignment is still around but probably doesn't work with bridges
either.


> Anyone know of a way to:
> * emulate a PCI EA in qemu.

Start with above patch, add bios support?

> * attach the local PCI EA bridge to qemu to troubleshoot using real hardware.
> For context, here is the suspect EA device:
> 
> 
> (no USB C dongle)
> $ lspci -tvnn
> -[0000:00]-+-00.0 Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core
> Processor Host Bridge/DRAM Registers [8086:1910]
> +-1d.6-[06-3e]--
> 
> vs
> 
> (USB C dongle)
> $ lspci -tvnn
> -[0000:00]-+-00.0 Intel Corporation Skylake Host Bridge/DRAM Registers
> [8086:1904]
> +-1d.6-[06-3e]----00.0-[07-3e]--+-00.0-[08]--
> | +-01.0-[09-3d]--
> | \-02.0-[3e]----00.0 Intel Corporation DSL6340 USB 3.1 Controller [Alpine
> Ridge] [8086:15b5]
> 
> 
> The bride is at 1d.6 in it's own IOMMU group:
> 
> IOMMU group 12
> 00:1d.6 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset
> Family PCI Express Root Port #15 [8086:a11e] (rev f1)
> 
> -- Alex

Re: [Qemu-devel] Debugging PCI Enhanced Allocation - IOMMU of a bridge?

[Alex Williamson]

On Wed, 12 Sep 2018 13:38:59 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Wed, Sep 12, 2018 at 05:30:29PM +0000, Alexander von Gluck IV wrote:
> > Good morning!
> > 
> > I'm working through adding support (or at least ignoring) PCI Enhance
> > Allocation devices under Haiku.
> > We run into one of these devices, and crash at boot due to attempting to read
> > an invalid address.
> > 
> > Our testing showed that with a USB C dongle plugged in, we don't crash. Without
> > it plugged in we get
> > invalid addresses from PCI
> > 
> > (for a long-drawn out debug ticket, see https://dev.haiku-os.org/ticket/12885 )
> > 
> > 
> > So, i'm trying to see if I can emulate a PCI bridge with EA under QEMU, I found
> > this patch, but it seems
> > it wasn't accepted:
> > https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02073.html  
> 
> v2 never surfaced I think.
> 
> > My other thought is since I have access to two laptops implementing this, I
> > could attach the bridge to
> > qemu via iommu, but the new vfio stuff doesn't seem to work with bridges. :-|  
> 
> It's fixable, AFAIK no one bothered though. BTW old style kvm dev
> assignment is still around but probably doesn't work with bridges
> either.

Bridge assignment doesn't make sense.  The main component of a bridge
is the resource aperture windows, which is exactly the part that we'd
virtualize if we assigned it.  Therefore it makes far more sense to
simply emulate the bridge.  The host resource settings are never
manipulated by the user.  Thanks,

Alex

> > Anyone know of a way to:
> > * emulate a PCI EA in qemu.  
> 
> Start with above patch, add bios support?
> 
> > * attach the local PCI EA bridge to qemu to troubleshoot using real hardware.
> > For context, here is the suspect EA device:
> > 
> > 
> > (no USB C dongle)
> > $ lspci -tvnn
> > -[0000:00]-+-00.0 Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core
> > Processor Host Bridge/DRAM Registers [8086:1910]
> > +-1d.6-[06-3e]--
> > 
> > vs
> > 
> > (USB C dongle)
> > $ lspci -tvnn
> > -[0000:00]-+-00.0 Intel Corporation Skylake Host Bridge/DRAM Registers
> > [8086:1904]
> > +-1d.6-[06-3e]----00.0-[07-3e]--+-00.0-[08]--
> > | +-01.0-[09-3d]--
> > | \-02.0-[3e]----00.0 Intel Corporation DSL6340 USB 3.1 Controller [Alpine
> > Ridge] [8086:15b5]
> > 
> > 
> > The bride is at 1d.6 in it's own IOMMU group:
> > 
> > IOMMU group 12
> > 00:1d.6 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset
> > Family PCI Express Root Port #15 [8086:a11e] (rev f1)
> > 
> > -- Alex