From mboxrd@z Thu Jan 1 00:00:00 1970 From: Erik Brakkee Subject: Re: USB Passthrough 1.1 performance problem... Date: Thu, 16 Dec 2010 22:46:38 +0100 Message-ID: <4D0A88BE.8000707@brakkee.org> References: <4D04B645.3010100@brakkee.org> <4D0537A8.8000607@brakkee.org> <4D0549AA.2020007@web.de> <4D054D37.7040107@brakkee.org> <3047113345.976756218@brakkee.org> <5085473063.976781602@brakkee.org> <20101214120519.GH32153@redhat.com> <4D07E5F7.1000400@brakkee.org> <65300.194.237.142.7.1292516619.squirrel@brakkee.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kenni Lund , kvm To: erik@brakkee.org Return-path: Received: from cpsmtp-fia04.kpnxchange.com ([195.121.247.8]:1193 "EHLO cpsmtp-fia04.kpnxchange.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754045Ab0LPVqh (ORCPT ); Thu, 16 Dec 2010 16:46:37 -0500 In-Reply-To: <65300.194.237.142.7.1292516619.squirrel@brakkee.org> Sender: kvm-owner@vger.kernel.org List-ID: Erik Brakkee wrote: > > >> 2010/12/14 Erik Brakkee: >> >>> Daniel P. Berrange wrote: >>> >>>> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote: >>>> >>>> >>>>> 2010/12/14 Erik Brakkee: >>>>> >>>>> >>>>>>> From: Kenni Lund >>>>>>> 2010/12/14 Erik Brakkee: >>>>>>> >>>>>>> >>>>>>>>> From: Kenni Lund >>>>>>>>> >>>>>>>>> >>>>>>>>>> Does this mean I have a chance now that PCI passthrough of my >>>>>>>>>> WinTV >>>>>>>>>> PVR-500 >>>>>>>>>> might work now? >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Passthrough of a PVR-500 has been working for a long time. I've >>>>>>>>> been >>>>>>>>> running with passthrough of a PVR-500 in my HTPC, since >>>>>>>>> November/December 2009...so it should work with any recent kernel >>>>>>>>> and >>>>>>>>> any recent version of qemu-kvm you can find today - No patching >>>>>>>>> needed. The only issue I had with the PVR-500 card, was when *I* >>>>>>>>> didn't free up the shared interrupts...once I fixed that, it "just >>>>>>>>> worked". >>>>>>>>> >>>>>>>>> >>>>>>>> How did you free up those shared interrupts then? I tried different >>>>>>>> slots >>>>>>>> but always get conflicts with the USB irqs. >>>>>>>> >>>>>>>> >>>>>>> I did an unbind of the conflicting device (eg. disabled it). I moved >>>>>>> the PVR-500 card around in the different slots and once I got a >>>>>>> conflict with the integrated sound card, I left the PVR-500 card in >>>>>>> that slot (it's a headless machine, so no need for sound) and >>>>>>> configured unbind of the sound card at boot time. On my old system I >>>>>>> think it was conflicting with one of the USB controllers as well, >>>>>>> but >>>>>>> it didn't really matter, as I only lost a few of the ports on the >>>>>>> back >>>>>>> of the computer for that particular USB controller - I still had >>>>>>> plenty of USB ports left and if I really needed more ports, I could >>>>>>> just plug in an extra USB PCI card. >>>>>>> >>>>>>> My /etc/rc.local boot script looks like the following today: >>>>>>> -- >>>>>>> #Remove HDA conflicting with ivtv1 >>>>>>> echo "0000:00:1b.0"> /sys/bus/pci/drivers/HDA\ Intel/unbind >>>>>>> >>>>>>> # ivtv0 >>>>>>> echo "4444 0016"> /sys/bus/pci/drivers/pci-stub/new_id >>>>>>> echo "0000:04:08.0"> /sys/bus/pci/drivers/ivtv/unbind >>>>>>> echo "0000:04:08.0"> /sys/bus/pci/drivers/pci-stub/bind >>>>>>> echo "4444 0016"> /sys/bus/pci/drivers/pci-stub/remove_id >>>>>>> >>>>>>> # ivtv1 >>>>>>> echo "4444 0016"> /sys/bus/pci/drivers/pci-stub/new_id >>>>>>> echo "0000:04:09.0"> /sys/bus/pci/drivers/ivtv/unbind >>>>>>> echo "0000:04:09.0"> /sys/bus/pci/drivers/pci-stub/bind >>>>>>> echo "4444 0016"> /sys/bus/pci/drivers/pci-stub/remove_id >>>>>>> >>>>>>> >>>>>> I did not try unbinding the usb device so I can also try that. >>>>>> >>>>>> I don'.t understand what is happening with the 4444 0016. I >>>>>> configured >>>>>> the >>>>>> pci card in kvm and I believe kvm does the binding to pci-stub in >>>>>> recent >>>>>> versions. Where is the 4444 0016%oming from? >>>>>> >>>>>> >>>>> Okay, qemu-kvm might do it today, I don't know - I haven't changed >>>>> that script for the past year. But are you sure that it's not >>>>> libvirt/virsh/virt-manager which does that for you? >>>>> >>>>> >>>> If you use the managed="yes" attribute on the in libvirt >>>> XML, then libvirt will automatically do the pcistub bind/unbind, >>>> followed by a device reset at guest startup& the reverse at shutdown. >>>> If you have conflicting devices on the bus though, libvirt won't >>>> attempt to unbind them, unless you had also explicitly assigned all >>>> those conflicting devices to the same guest. >>>> >>>> Daniel >>>> >>>> >>> I definitely have to try again (right now having some stability problems >>> on >>> the server that I am debugging). >>> >>> The shared IRQs are as follows: >>> >>> 16: 0 0 0 0 0 0 >>> 0 0 IO-APIC-fasteoi uhci_hcd:usb3 >>> 18: 252995 0 0 0 0 0 >>> 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb8, ivtv0 >>> 19: 58281 0 0 0 0 0 >>> 0 0 IO-APIC-fasteoi ata_piix, ata_piix, uhci_hcd:usb5, >>> uhci_hcd:usb7, ivtv1 >>> 21: 0 0 0 0 0 0 >>> 0 0 IO-APIC-fasteoi uhci_hcd:usb4 >>> 23: 713 6906 0 76919 0 0 >>> 0 0 IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb6 >>> >>> So I have IRQ sharing with usb1, usb8, usb5, usb7. >>> >> Uff....and your ata HDD controller. I guess i was much luckier than >> you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted >> with USB. >> >> >>> I have also read that >>> ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how >>> would I now identify the USB PCI devices that I would need to unbind to >>> get >>> rid of the sharing with the USB ports? >>> >> Play around with: >> lspci -v >> lspci -n >> lsusb -v >> lsusb -t >> >> You can also just start by unbinding the first one and take note when >> you hit the right ones...once you unbind one, it will disappear from >> cat /proc/interrupts. When you're down to having only ivtv0 on one >> interrupt and only ivtv1 on another interrupt, then you're ready to >> bind with pci-stub and boot your guest. >> >> >>> It also doesn't really matter in >>> which slot I put the PVR-500 card because both cards share IRQs with USB >>> in >>> all cases. >>> >> You also have your conflicting ata controller to take into >> consideration. I don't remember if "ata_piix" and "ata_piix" is IDE >> only, if it is, you might not even use it. Otherwise it might be >> easier for you to run qemu-kvm git with the new patches for shared >> interrupts...it really depends on your setup. >> >> >>> I have also used an add on USB PCI card but still got these conflicts. I >>> was >>> considering to get a PCIe USB card instead to try out in the hope that >>> that >>> would use different IRQs. Is that a realistic expectation? That way, I >>> could >>> disable all on-board USB (in the BIOS even) and use the add-on USB only. >>> >> Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got >> its own unique IRQs which wasn't shared with anything. >> >> > I think I am a bit out of luck here. The two IRQs that are interesting are > 18 and 19. One of them has the SMBUS and the other one has SATA drives. I > think the best for me is to wait until shared IRQ support in KVM is > available. My only option is to try it out every once in a while. > > > > Meanwhile, I haven't given up and experimented with determining the effect of unbinding the various USB PCI devices. I haven't tried virtualization yet though. The problem is I managed to free up one tuner (ivtv0) from shared IRQs, all USB ports are still operational, but then how would I determine which USB ports still support USB 2.0? I have now at least found a way to make ivtv0 available without any shared IRQ by unbinding two USB PCI devices: one UHCI and one EHCI device. The list of USB PCI devices is: 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 I unbound 00:1a.7 and 00:1d.2. To my great surprise, all USB ports on the server are still operational. I tried this by inserting a USB memory stick and trying to mount it. The motherboard has a total of 8 USB connections with 6 USB in use, I don't really understand this. Was I simply in luck? The output of lsucb after doing the unbinding is: Bus 005 Device 003: ID 046b:ff10 American Megatrends, Inc. Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 006 Device 002: ID 045e:001e Microsoft Corp. IntelliMouse Explorer Bus 006 Device 003: ID 051d:0002 American Power Conversion Uninterruptible Power Supply Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Alternatively, I am also considering to by a new PCIe card. In particular, the ASUS U3S6 looks interesting because it has 2 PCIe ports and 2 USB 3.0 ports. This means that I could disable 2 more USB devices (00:1a.2 and 00:1d.1) and get some extra USB devices. The ata_piix drivers are being used for an internal disk and CDROM. The RAID array is managed through arcmsr (Areca) so I don't have a problem there. Of course, if I managed to get completely free of shared IRQs with this add-on card, then the issue of determining which of the remaining ports is still USB 2.0 is not that important anymore. So, enough ideas to try out.