Re: genirq: Flags mismatch irq 17. 00000000 (vfio-intx(0000:07:04.0)) vs. 00000000 (vfio-intx(0000:01:00.1))

[Alex Williamson <alex.williamson@redhat.com>]

On Fri, 2014-01-10 at 12:18 -0800, Dana Goyette wrote:
> On 01/10/2014 09:33 AM, Alex Williamson wrote:
> > On Fri, 2014-01-10 at 08:23 -0800, Dana Goyette wrote:
> >> I'm using the SuperMicro X10SAT, kernel 3.13-rc5, with ACS override on
> >> Intel root ports.
> >>
> >> I'm trying to pass several devices to the same guest:
> >>
> >> 01:00.0 -- [1002:68be] Radeon HD 5750
> >> 01:00.1 -- [1002:aa58] HDMI Audio (not really needed)
> >> 07:04.0 -- [13f6:8788] Xonar D1/DX sound card, behind PEX8112
> >> 09:00.0 -- [1912:0014] Renesas uPD720201 (USB 3.0)
> >>
> >> When trying to start qemu with various combinations of those devices:
> >> vfio: Error: Failed to setup INTx fd: Device or resource busy
> >>
> >> Sound card conflicts with HDMI audio:
> >> genirq: Flags mismatch irq 17.
> >> 00000000 (vfio-intx(0000:07:04.0)) vs.
> >> 00000000 (vfio-intx(0000:01:00.1))
> >>
> >> USB controller conflicts with video card:
> >> genirq: Flags mismatch irq 16.
> >> 00000000 (vfio-intx(0000:09:00.0)) vs.
> >> 00000000 (vfio-intx(0000:01:00.0))
> >>
> >> On the ArchLinux forums, I was told that this means each device wants
> >> the interrupt line to itself.
> >>
> >> Oddly, the host locks up if I launch Xorg with 'radeon' and
> >> 'snd-virtuoso' both loaded.  (Primary video is the Intel graphics.)
> >>
> >> These devices all work fine together using pci-assign, but pci-assign
> >> requires ejecting the Radeon before VM shutdown.
> >>
> >> VFIO allows me to start the VM if I forward only the sound card and the
> >> video card, but I really need the USB controller, as well.
> >>
> >> What can I do to forward those three devices via VFIO?
> >
> > Change slots until the interrupts don't conflict, maybe there's an
> > IO-APIC option you can enable in the BIOS, or get devices that properly
> > support PCI2.3 INTx disable.  pci-assign tries to work around this by
> > using MSI to emulate INTx, but I'm not convinced that this doesn't
> > create more problems than it solves or I'd try to do the same with vfio.
> > The Xorg problem might also suggest there's an existing interrupt
> > routing problem on the system.  It is surprising that you have so many
> > devices that doesn't support INTx disable, can you try running this
> > script on them:
> >
> > #!/bin/sh
> > # Usage $0 <PCI device>, ex: 9:00.0
> >
> > INTX=$(( 0x400 ))
> > ORIG=$(( 0x$(setpci -s $1 4.w) ))
> >
> > if [ $(( $INTX & $ORIG )) -ne 0 ]; then
> > 	echo "INTx disable supported and enabled on $1"
> > 	exit 0
> > fi
> >
> > NEW=$(printf %04x $(( $INTX | $ORIG )))
> > setpci -s $1 4.w=$NEW
> > NEW=$(( 0x$(setpci -s $1 4.w) ))
> >
> > if [ $(( $INTX & $NEW )) -ne 0 ]; then
> > 	echo "INTx disable support available on $1"
> > else
> > 	echo "INTx disable support NOT available on $1"
> > fi
> >
> > NEW=$(printf %04x $ORIG)
> > setpci -s $1 4.w=$NEW
> >
> After cold boot, without any attempts to run VM:
> 
> Video card:
> INTx disable support available on 01:00.0
> 
> HDMI Audio:
> INTx disable supported and enabled on 01:00.1
> 
> USB:
> INTx disable supported and enabled on 09:00.0
> 
> PEX8112 (sound card sits behind this):
> INTx disable support available on 06:00.0)
> 
> Sound card:
> INTx disable support NOT available on 07:04.0
> 
> If the sound card is the only one lacking this feature, then it seems 
> odd that the USB controller is the one I have to remove for the VM to boot.

Please report the output of:

$ cat /sys/module/vfio_pci/parameters/nointxmask

I don't understand why your flags are 0 for all the devices, only the
sound card should have that.

> The lspci -nnvv listing for my current sound card is quite sparse:
> 
> 07:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc 
> CMI8788 [Oxygen HD Audio] [13f6:8788]
> 	Subsystem: ASUSTeK Computer Inc. Virtuoso 100 (Xonar D1) [1043:834f]
> 	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- 
> Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
> <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Interrupt: pin A routed to IRQ 10
> 	Region 0: I/O ports at c000 [disabled] [size=256]
> 	Capabilities: [c0] Power Management version 2
> 		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> 
> I'm willing to purchase a different sound card if that will fix the 
> problem.  Does lspci (-v or -vv) tell you if INTx Disable is available?

No, lspci only reports the state of the DisINTx bit, not whether it's
supported.