Re: [BUG] PCIe passthrough of MCS9990 fails

[Timothy Pearson <tpearson@raptorengineering.com>]

I've also seen the following failure in the guest when passing through more than one PCIe function:

[    0.710454] pci 0000:01:00.1: EHCI: unrecognized capability 00
[    0.710546] EEH: Recovering PHB#0-PE#10001
[    0.710598] EEH: PE location: N/A, PHB location: N/A
[    0.710669] EEH: Frozen PHB#0-PE#10001 detected
[    0.710745] EEH: Call Trace:
[    0.710832] EEH: [000000007d519995] __eeh_send_failure_event+0x7c/0x160
[    0.710944] EEH: [00000000419515d5] eeh_dev_check_failure+0x2e0/0x6d0
[    0.711011] EEH: [000000008950df7e] quirk_usb_early_handoff+0xca4/0xfd0
[    0.711076] EEH: [000000002528ae55] pci_fixup_device+0x140/0x2d0
[    0.711137] EEH: [000000002e85a626] pci_apply_final_quirks+0xb4/0x1b8
[    0.711199] EEH: [00000000e0915ba4] do_one_initcall+0x80/0x3b0
[    0.711255] EEH: [0000000014cb0304] kernel_init_freeable+0x340/0x3e4
[    0.711311] EEH: [00000000b18e64ad] kernel_init+0x30/0x1a0
[    0.711357] EEH: [0000000047d4e5e3] ret_from_kernel_user_thread+0x14/0x1c
[    0.711413] EEH: This PCI device has failed 1 times in the last hour and will be permanently disabled after 5 failures.
[    0.711493] EEH: Notify device drivers to shutdown
[    0.711538] EEH: Beginning: 'error_detected(IO frozen)'
[    0.711584] PCI 0000:01:00.1#10001: EEH: no driver
[    0.711589] PCI 0000:01:00.3#10001: EEH: no driver
[    0.711644] PCI 0000:01:00.5#10001: EEH: no driver
[    0.711689] PCI 0000:01:00.7#10001: EEH: no driver
[    0.711737] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'none'
[    0.711860] EEH: Collect temporary log
[    0.712602] EEH: of node=0000:01:00.1
[    0.712654] EEH: PCI device/vendor: 99909710
[    0.712720] EEH: PCI cmd/status register: 00100146
[    0.712761] EEH: PCI-E capabilities and status follow:
[    0.712876] EEH: PCI-E 00: 00010010 00008fc1 0000283f 01078011
[    0.712989] EEH: PCI-E 10: 00110008 00000000 000003c0 00000000
[    0.713044] EEH: PCI-E 20: 00000000
[    0.713082] EEH: of node=0000:01:00.3
[    0.713129] EEH: PCI device/vendor: 99909710
[    0.713187] EEH: PCI cmd/status register: 00100146
[    0.713232] EEH: PCI-E capabilities and status follow:
[    0.713345] EEH: PCI-E 00: 00010010 00008fc1 0000283f 01078011
[    0.713460] EEH: PCI-E 10: 00110008 00000000 000003c0 00000000
[    0.713513] EEH: PCI-E 20: 00000000
[    0.713547] EEH: of node=0000:01:00.5
[    0.713595] EEH: PCI device/vendor: 99909710
[    0.713655] EEH: PCI cmd/status register: 00100146
[    0.713700] EEH: PCI-E capabilities and status follow:
[    0.713812] EEH: PCI-E 00: 00010010 00008fc1 0000283f 01078011
[    0.713938] EEH: PCI-E 10: 00110008 00000000 000003c0 00000000
[    0.714007] EEH: PCI-E 20: 00000000
[    0.714052] EEH: of node=0000:01:00.7
[    0.714108] EEH: PCI device/vendor: 99909710
[    0.714182] EEH: PCI cmd/status register: 00100146
[    0.714239] EEH: PCI-E capabilities and status follow:
[    0.714371] EEH: PCI-E 00: 00010010 00008fc1 0000283f 01078011
[    0.714501] EEH: PCI-E 10: 00110008 00000000 000003c0 00000000
[    0.714571] EEH: PCI-E 20: 00000000
[    0.714623] EEH: Reset with hotplug activity
[    0.714942] pci 0000:01:00.1: quirk_usb_early_handoff+0x0/0xfd0 took 12340 usecs
[    0.716273] PCI: CLS 0 bytes, default 128
[    0.716424] Trying to unpack rootfs image as initramfs...
[    0.725599] Initialise system trusted keyrings
[    0.725679] Key type blacklist registered
[    0.725797] workingset: timestamp_bits=38 max_order=19 bucket_order=0
[    0.725891] zbud: loaded

Passing through one single EHCI function seems to at least allow the guest to boot, but only *once* after each power cycle.

All functions are in the same IOMMU group:

[  617.749136] pci 0031:01:00.0: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.749797] pci 0031:01:00.1: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.750412] pci 0031:01:00.2: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.751035] pci 0031:01:00.3: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.751668] pci 0031:01:00.4: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.752270] pci 0031:01:00.5: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.752871] pci 0031:01:00.6: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.753467] pci 0031:01:00.7: BAR 0 [mem size 0x00001000]: can't claim; no address assigned
[  617.754039] pci 0031:00:00.0: disabling bridge window [io  0x0000-0xffffffffffffffff] to [bus 01] (unused)
[  617.754642] pci 0031:00:00.0: disabling bridge window [mem 0x00000000-0xffffffffffffffff 64bit pref] to [bus 01] (unused)
[  617.755257] pci 0031:01:00.0: BAR 0 [mem 0x620c080000000-0x620c080000fff]: assigned
[  617.755813] pci 0031:01:00.1: BAR 0 [mem 0x620c080010000-0x620c080010fff]: assigned
[  617.756195] pci 0031:01:00.2: BAR 0 [mem 0x620c080020000-0x620c080020fff]: assigned
[  617.756559] pci 0031:01:00.3: BAR 0 [mem 0x620c080030000-0x620c080030fff]: assigned
[  617.756923] pci 0031:01:00.4: BAR 0 [mem 0x620c080040000-0x620c080040fff]: assigned
[  617.757281] pci 0031:01:00.5: BAR 0 [mem 0x620c080050000-0x620c080050fff]: assigned
[  617.757635] pci 0031:01:00.6: BAR 0 [mem 0x620c080060000-0x620c080060fff]: assigned
[  617.757980] pci 0031:01:00.7: BAR 0 [mem 0x620c080070000-0x620c080070fff]: assigned
[  617.758332] pci 0031:00:00.0: PCI bridge to [bus 01]
[  617.758662] pci 0031:00:00.0:   bridge window [mem 0x620c080000000-0x620c0ffefffff]
[  617.759016] pci_bus 0031:01: Configuring PE for bus
[  617.759357] pci 0031:01     : [PE# fd] Secondary bus 0x0000000000000001 associated with PE#fd
[  617.759875] pci 0031:01:00.0: Configured PE#fd
[  617.760213] pci 0031:01     : [PE# fd] Setting up 32-bit TCE table at 0..80000000
[  617.762565] pci 0031:01     : [PE# fd] Setting up window#0 0..7ffffffffff pg=10000
[  617.763014] pci 0031:01     : [PE# fd] Enabling 64-bit DMA bypass
[  617.763631] pci 0031:01:00.0: Adding to iommu group 10
[  617.764218] pci 0031:01:00.0: enabling device (0140 -> 0142)
[  617.764735] pci 0031:01:00.1: Added to existing PE#fd
[  617.765064] pci 0031:01:00.1: Adding to iommu group 10
[  617.765483] pci 0031:01:00.1: enabling device (0140 -> 0142)
[  617.766003] pci 0031:01:00.2: Added to existing PE#fd
[  617.766331] pci 0031:01:00.2: Adding to iommu group 10
[  617.766764] pci 0031:01:00.2: enabling device (0140 -> 0142)
[  617.767179] pci 0031:01:00.3: Added to existing PE#fd
[  617.767517] pci 0031:01:00.3: Adding to iommu group 10
[  617.767932] pci 0031:01:00.3: enabling device (0140 -> 0142)
[  617.768339] pci 0031:01:00.4: Added to existing PE#fd
[  617.768655] pci 0031:01:00.4: Adding to iommu group 10
[  617.769068] pci 0031:01:00.4: enabling device (0140 -> 0142)
[  617.769468] pci 0031:01:00.5: Added to existing PE#fd
[  617.769768] pci 0031:01:00.5: Adding to iommu group 10
[  617.770176] pci 0031:01:00.5: enabling device (0140 -> 0142)
[  617.770623] pci 0031:01:00.6: Added to existing PE#fd
[  617.770921] pci 0031:01:00.6: Adding to iommu group 10
[  617.771321] pci 0031:01:00.6: enabling device (0140 -> 0142)
[  617.771729] pci 0031:01:00.7: Added to existing PE#fd
[  617.772014] pci 0031:01:00.7: Adding to iommu group 10
[  617.772451] pci 0031:01:00.7: enabling device (0140 -> 0142)


----- Original Message -----
> From: "Timothy Pearson" <tpearson@raptorengineeringinc.com>
> To: "linuxppc-dev" <linuxppc-dev@lists.ozlabs.org>
> Cc: "Ritesh Harjani" <ritesh.list@gmail.com>
> Sent: Friday, September 18, 2026 1:32:51 PM
> Subject: [BUG] PCIe passthrough of MCS9990 fails

> When passing through all eight PCIe functions of a MCS9990 PCIe USB
> multifunction 1.0/2.0 controller on a PowerNV POWER9 host using qemu, the guest
> fails to launch:
> 
> qemu-system-ppc64: -device vfio-pci,host=0031:01:00.1,bus=bridge-1,addr=00.1:
> vfio 0031:01:00.1: Failed to set up TRIGGER eventfd signaling for interrupt
> INTX-0: VFIO_DEVICE_SET_IRQS failure: Device or resource busy
> 
> If I instead pass through only one function, or only pass through odd functions
> (the EHCI controller set), I get the following errors in the guest:
> 
> [    0.886625] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002,
> bcdDevice= 6.18
> [    0.886710] usb usb1: New USB device strings: Mfr=3, Product=2,
> SerialNumber=1
> [    0.886791] usb usb1: Product: EHCI Host Controller
> [    0.886845] usb usb1: Manufacturer: Linux 6.18.38 ehci_hcd
> [    0.886902] usb usb1: SerialNumber: 0000:01:00.0
> [    1.142469] usb 1-1: new high-speed USB device number 2 using ehci-pci
> [    6.530490] usb 1-1: device descriptor read/64, error -110
> [   22.147501] usb 1-1: device descriptor read/64, error -110
> [   22.371536] usb 1-1: new high-speed USB device number 3 using ehci-pci
> [   27.526896] usb 1-1: device descriptor read/64, error -110
> [   43.141934] usb 1-1: device descriptor read/64, error -110
> [   43.245985] usb usb1-port1: attempt power cycle
> [   43.638173] usb 1-1: new high-speed USB device number 4 using ehci-pci
> [   54.491254] usb 1-1: device not accepting address 4, error -110
> [   54.611274] usb 1-1: new high-speed USB device number 5 using ehci-pci
> 
> If I reboot the guest, SLOF fails as follows:
> 
> Scanning USB
>  EHCI: Initializing
> usb-ehci: reset failed
> usb-ehci: control transfer timed out_
> usb-ehci: unable to setup device on port 0
> 
> Note that this card works perfectly on the host kernel.