Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

Chris Wright
2011-02-25T23:32:06Z

* James Neave (roboj1m@gmail.com) wrote:
> On Fri, Feb 25, 2011 at 11:02 PM, James Neave <roboj1m@gmail.com> wrote:
> > On Fri, Feb 25, 2011 at 10:47 PM, James Neave <roboj1m@gmail.com> wrote:
> >> On Fri, Feb 25, 2011 at 12:06 AM, Chris Wright <chrisw@sous-sol.org> wrote:
> >>> * James Neave (roboj1m@gmail.com) wrote:
> >>>> OK, here's my latest dmesg with amd_iommu_dump and debug with no quiet
> >>>> http://pastebin.com/JxEwvqRA
> >>>
> >>> Yeah, that's what I expected:
> >>>
> >>> [    0.724403] AMD-Vi:   DEV_ALIAS_RANGE                 devid: 08:00.0 flags: 00 devid_to: 00:14.4
> >>> [    0.724439] AMD-Vi:   DEV_RANGE_END           devid: 08:1f.7
> >>>
> >>> That basically says 08:00.0 - 08:1f.7 will show up as 00:14.4 (and
> >>> should all go into same iommu domain).
> >>>
> >>>> I've just figured out a sequence of "echo DEV > PATH" commands to call
> >>>> for 14.4 gets me past the "claimed by pci-stub" error and gets me to
> >>>> the "failed to assign IRQ" error.
> >>>> I'm going to narrow down the required sequence and then post it.
> >>>
> >>> Kind of afraid to ask, but does it include:
> >>>
> >>> (assuming 1002 4384 is the pci to pci bridge)
> >>> echo 1002 4384 > /sys/bus/pci/drivers/pci-stub/new_id
> >>> echo 0000:00:14.4 > /sys/bus/pci/drivers/pci-stub/unbind
> >>>
> >>> (this has the side effect of detaching the bridge from its domain)
> >>>
> >>> thanks,
> >>> -chris
> >>>
> >>
> >> Exact sequence is:
> >>
> >> echo "1002 4384" > /sys/bus/pci/drivers/pci-stub/new_id
> >> echo "0000:00:14.4" > /sys/bus/pci/devices/0000:00:14.4/driver/unbind
> >>
> >> I take it this is a bad thing then?
> >>
> >>> I assume this means that 00:14.4 is still left claimed by pci-stub?
> >>
> >> Yes
> >>
> >>> How are you determining this?  The lspci paste above has pci-stub for all
> >>> of them.  The easiest thing might be to start with manually disabling
> >>> host driver and reassigning pci-stub to: 00:14.4, 08: 06.2,3 and 0e.0
> >>> Then giving the guest only 08:06.1.
> >>
> >> I determined it by being half asleep and not reading it properly... >.<
> >> You're right, all 5 devices were using pci-stub
> >>
> >>>> libvirtError: this function is not supported by the connection driver:
> >>>> Unable to reset PCI device 0000:00:14.4: no FLR, PM reset or bus reset
> >>>> available
> >>
> >>> Right, libvirt is more restrictive than qemu-kvm (forgot you were using
> >>> libvirt here).
> >>
> >> What does that libvirt error mean? I can't find a definition.
> >> Am I limiting myself by using libvirt? Would not using it help and how
> >> would I go about not using it?
> >>
> >>> Trouble now is that
> >>> with shared IRQ we don't have a good way to handle that right now.
> >>
> >> Game over then?
> >> I've tried assigning the USB devices before, I couldn't do it because
> >> qemu doesn't support USB2 devices.
> >> I don't really understand where this IRQ conflict is, the firewire and
> >> the USB2 device share IRQ22 but I'm assigning them both to the VM?
> >> Is that still a problem?
> >> I don't suppose there's any way to change which IRQ they use in the
> >> BIOS or with a command is there?
> >>
> >> I don't know if it means anything but this page:
> >>
> >> http://linuxtv.org/wiki/index.php/Hauppauge_WinTV-HVR-2200
> >>
> >> Has the lspci output for the HVR-2200 which mentions MSI and IRQ255.
> >> My knowledge it very limited on this subject so I don't know if that's
> >> meaningless looking at the output from another person's lspci.
> >>
> >> Anything left to try?
> >>
> >> Regardless, many thanks for your help,
> >>
> >> James.
> >>
> >
> > On the off chance I tried disabling the firewire in the BIOS, which
> > leaves only my tuner card using IRQ 20, 21 and 22.
> > No difference, still complains about IRQs:
> >
> > Using raw in/out ioport access (sysfs - Input/output error)
> > Failed to assign irq for "hostdev0": Operation not permitted
> > Perhaps you are assigning a device that shares an IRQ with another device?
> >
> > It does say "Operation not permitted" and that only "perhaps" I am
> > assigning a device that shares an IRQ.
> > Perhaps IRQ conflict it not the problem? They really are sitting on
> > their own. Another permissions problem perhaps?
> >
> > Regards,
> >
> > James.
> >
>
> I'm reading something about this error message being related to
> libvirt and CAP_SYS_RAWIO?

Depending on how new your libvirt is, you can force it to stop dropping
capabilities. Look for the config item "clear_emulator_capabilities"
in /etc/libvirt/qemu.conf. Setting this to 0 would verify that's the
problem (and not a real shared irq...i thought i saw sharing on
/proc/interrupts though).

>
> http://www.mail-archive.com/kvm@vger.kernel.org/msg34338.html
> http://www.google.co.uk/#hl=en&xhr=t&q=libvirt+CAP_SYS_RAWIO&cp=21&pf=p&sclient=psy&aq=f&aqi=&aql=&oq=libvirt+CAP_SYS_RAWIO&pbx=1&fp=2d8e3f69fec095f4
>
> "When I patch libvirt to not drop the capabilities, everything works
> as expected."

Well, that's a good point. We fixed that a while ago, but I'm not sure
your kernel has that fix.

2.6.35.10-dmar (btw, random nitpick, dmar == intel dma remapping engine,
aka vt-d not amd iommu ;)

This was fixed in 2.6.36, commit:

48bb09e KVM: remove CAP_SYS_RAWIO requirement from kvm_vm_ioctl_assign_irq

The last 2.6.35 stable release is 2.6.35.9 and does not have that fix.
So unless your .10-dmar has it, you could patch it in.

thanks,
-chris