From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sebastian Hetze Subject: Re: pci passthrough with KVM Date: Thu, 30 Sep 2010 17:53:19 +0200 Message-ID: <20100930155320.0152398032@mail.linux-ag.de> References: <948322C1-8652-425D-B749-44AA2EEC0419@ibex.es> <20100929085611.36098409000B@mail.linux-ag.de> <1ECA8273-9C25-4A9A-8F4C-81ABF73E73F2@ibex.es> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Sebastian Hetze , kvm@vger.kernel.org To: Inigo Losada Return-path: Received: from ironport.linux-ag.com ([62.245.157.240]:44073 "EHLO ironport.linux-ag.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752925Ab0I3QCx (ORCPT ); Thu, 30 Sep 2010 12:02:53 -0400 Received: from localhost (mail.linux-ag.de [62.245.157.206]) by mail.linux-ag.de (Postfix) with ESMTP id 0152398032 for ; Thu, 30 Sep 2010 17:53:20 +0200 (CEST) Content-Disposition: inline In-Reply-To: <1ECA8273-9C25-4A9A-8F4C-81ABF73E73F2@ibex.es> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Sep 30, 2010 at 11:38:21AM +0200, Inigo Losada wrote: >=20 > El 29/09/2010, a las 10:56, Sebastian Hetze escribi=F3: >=20 > > On Tue, Sep 28, 2010 at 10:27:18AM +0200, Inigo Losada wrote: > >> We are using pci passthrough with an SCSI Adapter card. The system= is: > >>=20 > >> - O.S: Ubuntu 10.04.1 LTS > >> - KVM Packages:=20 > >>=20 > >> kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2 > >> kvm-pxe 5.4.4-1ubuntu1 > >> qemu-kvm 0.12.3+noroms-0ubuntu9.2=20 > >> libvirt-bin 0.7.5-5ubuntu27.2 > >> python-libvirt 0.7.5-5ubuntu27.2 > >> libvirt0 0.7.5-5ubuntu27.2 > >>=20 > >> - Kernel 2.6.32.15+drm33.5.iommu recompiled with following opt= ions : > >>=20 > >> CONFIG_DMAR=3Dy > >> CONFIG_INTR_REMAP=3Dy > >>=20 > >> - Apparmor is stopped > >>=20 > >> When we started the virtual machine we obtain the following error:= =20 > >>=20 > >> char device redirected to /dev/pts/3=20 > >> device: 04:04.0: driver=3D"pci-assign" host=3D"04:04.0"=20 > >> Failed to assign irq for "04:04.0": Operation not permitted=20 > >> Perhaps you are assigning a device that shares an IRQ with another= device?=20 > >=20 > > So do you? > >=20 > > The notes in > > http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KV= M > > give you this information: > >=20 > > * VT-d spec specifies that all conventional PCI devices behind = a > > * PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only = be > > * collectively assigned to the same guest. PCIe devices do not = have > > * this restriction. > > * If the device doesn't support MSI, and it shares IRQ with oth= er > > * devices, then it cannot be assigned due to host irq sharing f= or > > * assigned devices is not supported. You will get warning messa= ge > > * when you assign it. Notice this also apply to the devices whi= ch > > * only support MSI-X.=20 > >=20 > > What do you find in /proc/interrupts ? > >=20 >=20 > cat /proc/interrupts=20 > CPU0 CPU1 CPU2 CPU3 CPU4 CP= U5 CPU6 CPU7 =20 > 0: 1000831147 0 0 0 0 = 0 0 0 IR-IO-APIC-edge timer > 1: 90 0 0 0 0 = 0 0 0 IR-IO-APIC-edge i8042 > 3: 2 0 0 0 0 = 0 0 0 IR-IO-APIC-edge =20 > 4: 2 0 0 0 0 = 0 0 0 IR-IO-APIC-edge =20 > 8: 1 0 0 0 0 = 0 0 0 IR-IO-APIC-edge rtc0 > 9: 0 0 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi acpi > 12: 135 300 0 0 0 = 0 0 0 IR-IO-APIC-edge i8042 > 16: 0 0 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb3 > 18: 2 0 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:= usb8 > 19: 0 0 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi ata_piix, ata_piix, uhci= _hcd:usb5, uhci_hcd:usb7 > 21: 31 0 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb4 > 23: 93 0 0 21758350 0 = 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:= usb6 > 26: 157 0 317 0 0 = 0 0 0 IR-IO-APIC-fasteoi=20 > 30: 4437 6102626 0 0 0 = 0 0 0 IR-IO-APIC-fasteoi 3w-9xxx > 48: 0 0 0 0 0 = 0 0 0 DMAR_MSI-edge dmar0 > 55: 0 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge eth0 > 56: 28 0 4148662 0 12 = 22 70 0 IR-PCI-MSI-edge eth0-TxRx-0 > 57: 29 0 0 2495606 0 = 0 0 0 IR-PCI-MSI-edge eth0-TxRx-1 > 58: 21 2704034 0 0 13 = 10 0 0 IR-PCI-MSI-edge eth0-TxRx-2 > 59: 26 0 0 0 0 = 0 0 1838420 IR-PCI-MSI-edge eth0-TxRx-3 > 60: 51 0 0 0 0 = 0 8249009 0 IR-PCI-MSI-edge eth0-TxRx-4 > 61: 1531450 0 0 0 10 = 0 0 0 IR-PCI-MSI-edge eth0-TxRx-5 > 62: 44 0 0 0 0 21766= 63 0 0 IR-PCI-MSI-edge eth0-TxRx-6 > 63: 24 0 0 0 1359843 = 0 0 0 IR-PCI-MSI-edge eth0-TxRx-7 > 91: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 92: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 93: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 94: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 95: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 96: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 97: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > 98: 3 0 0 0 0 = 0 0 0 IR-PCI-MSI-edge ioat-msix > NMI: 0 0 0 0 0 = 0 0 0 Non-maskable interrupts > LOC: 550286961 678566029 748548019 733777570 517827192 4303313= 67 268714238 527520322 Local timer interrupts > SPU: 0 0 0 0 0 = 0 0 0 Spurious interrupts > PMI: 0 0 0 0 0 = 0 0 0 Performance monitoring interrupts > PND: 0 0 0 0 0 = 0 0 0 Performance pending work > RES: 73223541 59311048 49506524 42871500 37975243 321065= 28 31106285 27747552 Rescheduling interrupts > CAL: 2206175 1509174 2056330 1638734 1690520 17394= 39 1196679 985504 Function call interrupts > TLB: 3277203 2666047 3158856 2614034 2243049 22154= 14 1477134 1538333 TLB shootdowns > TRM: 0 0 0 0 0 = 0 0 0 Thermal event interrupts > THR: 0 0 0 0 0 = 0 0 0 Threshold APIC interrupts > MCE: 0 0 0 0 0 = 0 0 0 Machine check exceptions > MCP: 2591 2591 2591 2591 2591 25= 91 2591 2591 Machine check polls > ERR: 7 > MIS: 0 >=20 > But I do not see the Adaptec card (yes with lspci): >=20 > 04:04.0 SCSI storage controller: Adaptec ASC-29320ALP U320 (rev ff) We are coming closer... With 'lspci -v -s04:04.0' you will see the IRQ your Adaptec controller is assigned to. http://www.linux.org/docs/ldp/howto/Plug-and-Play-HOWTO-7.html gives more information about interrupts in general. You may check your BIOS for means to assign interrupts to certain slots. Moving either your SCSI controller or the other conflicting card to another slot might help.=20