From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Neave Subject: Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2 Date: Mon, 28 Feb 2011 13:42:40 +0000 Message-ID: References: <20110222015119.GY9869@sequoia.sous-sol.org> <20110223001103.GZ9869@sequoia.sous-sol.org> <20110225000622.GB4988@sequoia.sous-sol.org> <20110225233148.GF4988@sequoia.sous-sol.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Alex Williamson , kvm@vger.kernel.org, "Roedel, Joerg" To: Chris Wright Return-path: Received: from mail-qw0-f46.google.com ([209.85.216.46]:61463 "EHLO mail-qw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753860Ab1B1NnB convert rfc822-to-8bit (ORCPT ); Mon, 28 Feb 2011 08:43:01 -0500 Received: by qwd7 with SMTP id 7so2808252qwd.19 for ; Mon, 28 Feb 2011 05:43:00 -0800 (PST) In-Reply-To: <20110225233148.GF4988@sequoia.sous-sol.org> Sender: kvm-owner@vger.kernel.org List-ID: On Fri, Feb 25, 2011 at 11:31 PM, Chris Wright wr= ote: > * James Neave (roboj1m@gmail.com) wrote: >> On Fri, Feb 25, 2011 at 11:02 PM, James Neave wr= ote: >> > On Fri, Feb 25, 2011 at 10:47 PM, James Neave = wrote: >> >> On Fri, Feb 25, 2011 at 12:06 AM, Chris Wright wrote: >> >>> * James Neave (roboj1m@gmail.com) wrote: >> >>>> OK, here's my latest dmesg with amd_iommu_dump and debug with n= o quiet >> >>>> http://pastebin.com/JxEwvqRA >> >>> >> >>> Yeah, that's what I expected: >> >>> >> >>> [ =C2=A0 =C2=A00.724403] AMD-Vi: =C2=A0 DEV_ALIAS_RANGE =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 devid: 08:00.0 flags: 00 = devid_to: 00:14.4 >> >>> [ =C2=A0 =C2=A00.724439] AMD-Vi: =C2=A0 DEV_RANGE_END =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 devid: 08:1f.7 >> >>> >> >>> That basically says 08:00.0 - 08:1f.7 will show up as 00:14.4 (a= nd >> >>> should all go into same iommu domain). >> >>> >> >>>> I've just figured out a sequence of "echo DEV > PATH" commands = to call >> >>>> for 14.4 gets me past the "claimed by pci-stub" error and gets = me to >> >>>> the "failed to assign IRQ" error. >> >>>> I'm going to narrow down the required sequence and then post it= =2E >> >>> >> >>> Kind of afraid to ask, but does it include: >> >>> >> >>> (assuming 1002 4384 is the pci to pci bridge) >> >>> echo 1002 4384 > /sys/bus/pci/drivers/pci-stub/new_id >> >>> echo 0000:00:14.4 > /sys/bus/pci/drivers/pci-stub/unbind >> >>> >> >>> (this has the side effect of detaching the bridge from its domai= n) >> >>> >> >>> thanks, >> >>> -chris >> >>> >> >> >> >> Exact sequence is: >> >> >> >> echo "1002 4384" > /sys/bus/pci/drivers/pci-stub/new_id >> >> echo "0000:00:14.4" > /sys/bus/pci/devices/0000:00:14.4/driver/un= bind >> >> >> >> I take it this is a bad thing then? >> >> >> >>> I assume this means that 00:14.4 is still left claimed by pci-st= ub? >> >> >> >> Yes >> >> >> >>> How are you determining this? =C2=A0The lspci paste above has pc= i-stub for all >> >>> of them. =C2=A0The easiest thing might be to start with manually= disabling >> >>> host driver and reassigning pci-stub to: 00:14.4, 08: 06.2,3 and= 0e.0 >> >>> Then giving the guest only 08:06.1. >> >> >> >> I determined it by being half asleep and not reading it properly.= =2E. >.< >> >> You're right, all 5 devices were using pci-stub >> >> >> >>>> libvirtError: this function is not supported by the connection = driver: >> >>>> Unable to reset PCI device 0000:00:14.4: no FLR, PM reset or bu= s reset >> >>>> available >> >> >> >>> Right, libvirt is more restrictive than qemu-kvm (forgot you wer= e using >> >>> libvirt here). >> >> >> >> What does that libvirt error mean? I can't find a definition. >> >> Am I limiting myself by using libvirt? Would not using it help an= d how >> >> would I go about not using it? >> >> >> >>> Trouble now is that >> >>> with shared IRQ we don't have a good way to handle that right no= w. >> >> >> >> Game over then? >> >> I've tried assigning the USB devices before, I couldn't do it bec= ause >> >> qemu doesn't support USB2 devices. >> >> I don't really understand where this IRQ conflict is, the firewir= e and >> >> the USB2 device share IRQ22 but I'm assigning them both to the VM= ? >> >> Is that still a problem? >> >> I don't suppose there's any way to change which IRQ they use in t= he >> >> BIOS or with a command is there? >> >> >> >> I don't know if it means anything but this page: >> >> >> >> http://linuxtv.org/wiki/index.php/Hauppauge_WinTV-HVR-2200 >> >> >> >> Has the lspci output for the HVR-2200 which mentions MSI and IRQ2= 55. >> >> My knowledge it very limited on this subject so I don't know if t= hat's >> >> meaningless looking at the output from another person's lspci. >> >> >> >> Anything left to try? >> >> >> >> Regardless, many thanks for your help, >> >> >> >> James. >> >> >> > >> > On the off chance I tried disabling the firewire in the BIOS, whic= h >> > leaves only my tuner card using IRQ 20, 21 and 22. >> > No difference, still complains about IRQs: >> > >> > Using raw in/out ioport access (sysfs - Input/output error) >> > Failed to assign irq for "hostdev0": Operation not permitted >> > Perhaps you are assigning a device that shares an IRQ with another= device? >> > >> > It does say "Operation not permitted" and that only "perhaps" I am >> > assigning a device that shares an IRQ. >> > Perhaps IRQ conflict it not the problem? They really are sitting o= n >> > their own. Another permissions problem perhaps? >> > >> > Regards, >> > >> > James. >> > >> >> I'm reading something about this error message being related to >> libvirt and CAP_SYS_RAWIO? > > Depending on how new your libvirt is, you can force it to stop droppi= ng > capabilities. =C2=A0Look for the config item "clear_emulator_capabili= ties" > in /etc/libvirt/qemu.conf. =C2=A0Setting this to 0 would verify that'= s the > problem (and not a real shared irq...i thought i saw sharing on > /proc/interrupts though). > >> >> http://www.mail-archive.com/kvm@vger.kernel.org/msg34338.html >> http://www.google.co.uk/#hl=3Den&xhr=3Dt&q=3Dlibvirt+CAP_SYS_RAWIO&c= p=3D21&pf=3Dp&sclient=3Dpsy&aq=3Df&aqi=3D&aql=3D&oq=3Dlibvirt+CAP_SYS_R= AWIO&pbx=3D1&fp=3D2d8e3f69fec095f4 >> >> "When I patch libvirt to not drop the capabilities, everything works >> as expected." > > Well, that's a good point. =C2=A0We fixed that a while ago, but I'm n= ot sure > your kernel has that fix. > > 2.6.35.10-dmar (btw, random nitpick, dmar =3D=3D intel dma remapping = engine, > aka vt-d not amd iommu ;) > > This was fixed in 2.6.36, commit: > > 48bb09e KVM: remove CAP_SYS_RAWIO requirement from kvm_vm_ioctl_assig= n_irq > > The last 2.6.35 stable release is 2.6.35.9 and does not have that fix= =2E > So unless your .10-dmar has it, you could patch it in. > > thanks, > -chris > HOLY CRAP IT WORKS!!!! 8@ =2E..almost... OK, "clear_emulator_capabilities=3D0" solved the IRQ problem (which was= , as it turns out, the rawio problem) My VM came up, both the tuners were there and after the firmware install I was able to tune and watch the slowest TV in the world over VNC. Thank god for that, i was really starting to believe that slashing out a lot of cash on my 890FX board and the fancy DDR3 ram it needed was a collosal waste of money. "Sigh of relief" Well, thank you all so much for helping me to get to this point! And yes, I did say "almost works" Looks like I've run straight into Chris' ref counting problem when shutting the guest down. Some sort of critical error barf was on the servers' screen when I shut down the guest, appeared to be very similar to Chris' example, in amd_iommu.c I'd post it but the server locks up after it's been shown and needed resetting. No idea how I would post that bit of dmesg as it gets reset after each boot. Is there a solution for this at the moment or will I have to wait for it to be patched? Many Thanks, James.