* pci passthrough with KVM
@ 2010-09-28 8:27 Inigo Losada
2010-09-28 18:26 ` Alex Williamson
2010-09-29 8:56 ` Sebastian Hetze
0 siblings, 2 replies; 5+ messages in thread
From: Inigo Losada @ 2010-09-28 8:27 UTC (permalink / raw)
To: kvm
We are using pci passthrough with an SCSI Adapter card. The system is:
- O.S: Ubuntu 10.04.1 LTS
- KVM Packages:
kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2
kvm-pxe 5.4.4-1ubuntu1
qemu-kvm 0.12.3+noroms-0ubuntu9.2
libvirt-bin 0.7.5-5ubuntu27.2
python-libvirt 0.7.5-5ubuntu27.2
libvirt0 0.7.5-5ubuntu27.2
- Kernel 2.6.32.15+drm33.5.iommu recompiled with following options :
CONFIG_DMAR=y
CONFIG_INTR_REMAP=y
- Apparmor is stopped
When we started the virtual machine we obtain the following error:
char device redirected to /dev/pts/3
device: 04:04.0: driver="pci-assign" host="04:04.0"
Failed to assign irq for "04:04.0": Operation not permitted
Perhaps you are assigning a device that shares an IRQ with another device?
Thanks in advance,
Inigo
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pci passthrough with KVM
2010-09-28 8:27 pci passthrough with KVM Inigo Losada
@ 2010-09-28 18:26 ` Alex Williamson
2010-09-29 8:56 ` Sebastian Hetze
1 sibling, 0 replies; 5+ messages in thread
From: Alex Williamson @ 2010-09-28 18:26 UTC (permalink / raw)
To: Inigo Losada; +Cc: kvm
On Tue, Sep 28, 2010 at 2:27 AM, Inigo Losada <ilosada@ibex.es> wrote:
>
> We are using pci passthrough with an SCSI Adapter card. The system is:
>
> - O.S: Ubuntu 10.04.1 LTS
> - KVM Packages:
>
> kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2
> kvm-pxe 5.4.4-1ubuntu1
> qemu-kvm 0.12.3+noroms-0ubuntu9.2
> libvirt-bin 0.7.5-5ubuntu27.2
> python-libvirt 0.7.5-5ubuntu27.2
> libvirt0 0.7.5-5ubuntu27.2
>
> - Kernel 2.6.32.15+drm33.5.iommu recompiled with following options :
>
> CONFIG_DMAR=y
> CONFIG_INTR_REMAP=y
>
> - Apparmor is stopped
>
> When we started the virtual machine we obtain the following error:
>
> char device redirected to /dev/pts/3
> device: 04:04.0: driver="pci-assign" host="04:04.0"
> Failed to assign irq for "04:04.0": Operation not permitted
> Perhaps you are assigning a device that shares an IRQ with another device?
Does the device share an IRQ with another device? If not, does it
work if you launch qemu manually with sudo? It might be that the
kernel and tool set aren't new enough to fully support running a
de-privileged guest with device assignment. Thanks,
Alex
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pci passthrough with KVM
2010-09-28 8:27 pci passthrough with KVM Inigo Losada
2010-09-28 18:26 ` Alex Williamson
@ 2010-09-29 8:56 ` Sebastian Hetze
2010-09-30 9:38 ` Inigo Losada
1 sibling, 1 reply; 5+ messages in thread
From: Sebastian Hetze @ 2010-09-29 8:56 UTC (permalink / raw)
To: Inigo Losada; +Cc: kvm
On Tue, Sep 28, 2010 at 10:27:18AM +0200, Inigo Losada wrote:
> We are using pci passthrough with an SCSI Adapter card. The system is:
>
> - O.S: Ubuntu 10.04.1 LTS
> - KVM Packages:
>
> kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2
> kvm-pxe 5.4.4-1ubuntu1
> qemu-kvm 0.12.3+noroms-0ubuntu9.2
> libvirt-bin 0.7.5-5ubuntu27.2
> python-libvirt 0.7.5-5ubuntu27.2
> libvirt0 0.7.5-5ubuntu27.2
>
> - Kernel 2.6.32.15+drm33.5.iommu recompiled with following options :
>
> CONFIG_DMAR=y
> CONFIG_INTR_REMAP=y
>
> - Apparmor is stopped
>
> When we started the virtual machine we obtain the following error:
>
> char device redirected to /dev/pts/3
> device: 04:04.0: driver="pci-assign" host="04:04.0"
> Failed to assign irq for "04:04.0": Operation not permitted
> Perhaps you are assigning a device that shares an IRQ with another device?
So do you?
The notes in
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
give you this information:
* VT-d spec specifies that all conventional PCI devices behind a
* PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only be
* collectively assigned to the same guest. PCIe devices do not have
* this restriction.
* If the device doesn't support MSI, and it shares IRQ with other
* devices, then it cannot be assigned due to host irq sharing for
* assigned devices is not supported. You will get warning message
* when you assign it. Notice this also apply to the devices which
* only support MSI-X.
What do you find in /proc/interrupts ?
Best regards,
Sebastian
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pci passthrough with KVM
2010-09-29 8:56 ` Sebastian Hetze
@ 2010-09-30 9:38 ` Inigo Losada
2010-09-30 15:53 ` Sebastian Hetze
0 siblings, 1 reply; 5+ messages in thread
From: Inigo Losada @ 2010-09-30 9:38 UTC (permalink / raw)
To: Sebastian Hetze; +Cc: kvm
El 29/09/2010, a las 10:56, Sebastian Hetze escribió:
> On Tue, Sep 28, 2010 at 10:27:18AM +0200, Inigo Losada wrote:
>> We are using pci passthrough with an SCSI Adapter card. The system is:
>>
>> - O.S: Ubuntu 10.04.1 LTS
>> - KVM Packages:
>>
>> kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2
>> kvm-pxe 5.4.4-1ubuntu1
>> qemu-kvm 0.12.3+noroms-0ubuntu9.2
>> libvirt-bin 0.7.5-5ubuntu27.2
>> python-libvirt 0.7.5-5ubuntu27.2
>> libvirt0 0.7.5-5ubuntu27.2
>>
>> - Kernel 2.6.32.15+drm33.5.iommu recompiled with following options :
>>
>> CONFIG_DMAR=y
>> CONFIG_INTR_REMAP=y
>>
>> - Apparmor is stopped
>>
>> When we started the virtual machine we obtain the following error:
>>
>> char device redirected to /dev/pts/3
>> device: 04:04.0: driver="pci-assign" host="04:04.0"
>> Failed to assign irq for "04:04.0": Operation not permitted
>> Perhaps you are assigning a device that shares an IRQ with another device?
>
> So do you?
>
> The notes in
> http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
> give you this information:
>
> * VT-d spec specifies that all conventional PCI devices behind a
> * PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only be
> * collectively assigned to the same guest. PCIe devices do not have
> * this restriction.
> * If the device doesn't support MSI, and it shares IRQ with other
> * devices, then it cannot be assigned due to host irq sharing for
> * assigned devices is not supported. You will get warning message
> * when you assign it. Notice this also apply to the devices which
> * only support MSI-X.
>
> What do you find in /proc/interrupts ?
>
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 1000831147 0 0 0 0 0 0 0 IR-IO-APIC-edge timer
1: 90 0 0 0 0 0 0 0 IR-IO-APIC-edge i8042
3: 2 0 0 0 0 0 0 0 IR-IO-APIC-edge
4: 2 0 0 0 0 0 0 0 IR-IO-APIC-edge
8: 1 0 0 0 0 0 0 0 IR-IO-APIC-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi acpi
12: 135 300 0 0 0 0 0 0 IR-IO-APIC-edge i8042
16: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb3
18: 2 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb8
19: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi ata_piix, ata_piix, uhci_hcd:usb5, uhci_hcd:usb7
21: 31 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb4
23: 93 0 0 21758350 0 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb6
26: 157 0 317 0 0 0 0 0 IR-IO-APIC-fasteoi
30: 4437 6102626 0 0 0 0 0 0 IR-IO-APIC-fasteoi 3w-9xxx
48: 0 0 0 0 0 0 0 0 DMAR_MSI-edge dmar0
55: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0
56: 28 0 4148662 0 12 22 70 0 IR-PCI-MSI-edge eth0-TxRx-0
57: 29 0 0 2495606 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-1
58: 21 2704034 0 0 13 10 0 0 IR-PCI-MSI-edge eth0-TxRx-2
59: 26 0 0 0 0 0 0 1838420 IR-PCI-MSI-edge eth0-TxRx-3
60: 51 0 0 0 0 0 8249009 0 IR-PCI-MSI-edge eth0-TxRx-4
61: 1531450 0 0 0 10 0 0 0 IR-PCI-MSI-edge eth0-TxRx-5
62: 44 0 0 0 0 2176663 0 0 IR-PCI-MSI-edge eth0-TxRx-6
63: 24 0 0 0 1359843 0 0 0 IR-PCI-MSI-edge eth0-TxRx-7
91: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
92: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
93: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
94: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
95: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
96: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
97: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
98: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
NMI: 0 0 0 0 0 0 0 0 Non-maskable interrupts
LOC: 550286961 678566029 748548019 733777570 517827192 430331367 268714238 527520322 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 0 0 0 0 Performance monitoring interrupts
PND: 0 0 0 0 0 0 0 0 Performance pending work
RES: 73223541 59311048 49506524 42871500 37975243 32106528 31106285 27747552 Rescheduling interrupts
CAL: 2206175 1509174 2056330 1638734 1690520 1739439 1196679 985504 Function call interrupts
TLB: 3277203 2666047 3158856 2614034 2243049 2215414 1477134 1538333 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 2591 2591 2591 2591 2591 2591 2591 2591 Machine check polls
ERR: 7
MIS: 0
But I do not see the Adaptec card (yes with lspci):
04:04.0 SCSI storage controller: Adaptec ASC-29320ALP U320 (rev ff)
Regards,
Inigo
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pci passthrough with KVM
2010-09-30 9:38 ` Inigo Losada
@ 2010-09-30 15:53 ` Sebastian Hetze
0 siblings, 0 replies; 5+ messages in thread
From: Sebastian Hetze @ 2010-09-30 15:53 UTC (permalink / raw)
To: Inigo Losada; +Cc: Sebastian Hetze, kvm
On Thu, Sep 30, 2010 at 11:38:21AM +0200, Inigo Losada wrote:
>
> El 29/09/2010, a las 10:56, Sebastian Hetze escribió:
>
> > On Tue, Sep 28, 2010 at 10:27:18AM +0200, Inigo Losada wrote:
> >> We are using pci passthrough with an SCSI Adapter card. The system is:
> >>
> >> - O.S: Ubuntu 10.04.1 LTS
> >> - KVM Packages:
> >>
> >> kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9.2
> >> kvm-pxe 5.4.4-1ubuntu1
> >> qemu-kvm 0.12.3+noroms-0ubuntu9.2
> >> libvirt-bin 0.7.5-5ubuntu27.2
> >> python-libvirt 0.7.5-5ubuntu27.2
> >> libvirt0 0.7.5-5ubuntu27.2
> >>
> >> - Kernel 2.6.32.15+drm33.5.iommu recompiled with following options :
> >>
> >> CONFIG_DMAR=y
> >> CONFIG_INTR_REMAP=y
> >>
> >> - Apparmor is stopped
> >>
> >> When we started the virtual machine we obtain the following error:
> >>
> >> char device redirected to /dev/pts/3
> >> device: 04:04.0: driver="pci-assign" host="04:04.0"
> >> Failed to assign irq for "04:04.0": Operation not permitted
> >> Perhaps you are assigning a device that shares an IRQ with another device?
> >
> > So do you?
> >
> > The notes in
> > http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
> > give you this information:
> >
> > * VT-d spec specifies that all conventional PCI devices behind a
> > * PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only be
> > * collectively assigned to the same guest. PCIe devices do not have
> > * this restriction.
> > * If the device doesn't support MSI, and it shares IRQ with other
> > * devices, then it cannot be assigned due to host irq sharing for
> > * assigned devices is not supported. You will get warning message
> > * when you assign it. Notice this also apply to the devices which
> > * only support MSI-X.
> >
> > What do you find in /proc/interrupts ?
> >
>
> cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
> 0: 1000831147 0 0 0 0 0 0 0 IR-IO-APIC-edge timer
> 1: 90 0 0 0 0 0 0 0 IR-IO-APIC-edge i8042
> 3: 2 0 0 0 0 0 0 0 IR-IO-APIC-edge
> 4: 2 0 0 0 0 0 0 0 IR-IO-APIC-edge
> 8: 1 0 0 0 0 0 0 0 IR-IO-APIC-edge rtc0
> 9: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi acpi
> 12: 135 300 0 0 0 0 0 0 IR-IO-APIC-edge i8042
> 16: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb3
> 18: 2 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb8
> 19: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi ata_piix, ata_piix, uhci_hcd:usb5, uhci_hcd:usb7
> 21: 31 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb4
> 23: 93 0 0 21758350 0 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb6
> 26: 157 0 317 0 0 0 0 0 IR-IO-APIC-fasteoi
> 30: 4437 6102626 0 0 0 0 0 0 IR-IO-APIC-fasteoi 3w-9xxx
> 48: 0 0 0 0 0 0 0 0 DMAR_MSI-edge dmar0
> 55: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0
> 56: 28 0 4148662 0 12 22 70 0 IR-PCI-MSI-edge eth0-TxRx-0
> 57: 29 0 0 2495606 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-1
> 58: 21 2704034 0 0 13 10 0 0 IR-PCI-MSI-edge eth0-TxRx-2
> 59: 26 0 0 0 0 0 0 1838420 IR-PCI-MSI-edge eth0-TxRx-3
> 60: 51 0 0 0 0 0 8249009 0 IR-PCI-MSI-edge eth0-TxRx-4
> 61: 1531450 0 0 0 10 0 0 0 IR-PCI-MSI-edge eth0-TxRx-5
> 62: 44 0 0 0 0 2176663 0 0 IR-PCI-MSI-edge eth0-TxRx-6
> 63: 24 0 0 0 1359843 0 0 0 IR-PCI-MSI-edge eth0-TxRx-7
> 91: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 92: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 93: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 94: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 95: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 96: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 97: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> 98: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge ioat-msix
> NMI: 0 0 0 0 0 0 0 0 Non-maskable interrupts
> LOC: 550286961 678566029 748548019 733777570 517827192 430331367 268714238 527520322 Local timer interrupts
> SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
> PMI: 0 0 0 0 0 0 0 0 Performance monitoring interrupts
> PND: 0 0 0 0 0 0 0 0 Performance pending work
> RES: 73223541 59311048 49506524 42871500 37975243 32106528 31106285 27747552 Rescheduling interrupts
> CAL: 2206175 1509174 2056330 1638734 1690520 1739439 1196679 985504 Function call interrupts
> TLB: 3277203 2666047 3158856 2614034 2243049 2215414 1477134 1538333 TLB shootdowns
> TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
> THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
> MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
> MCP: 2591 2591 2591 2591 2591 2591 2591 2591 Machine check polls
> ERR: 7
> MIS: 0
>
> But I do not see the Adaptec card (yes with lspci):
>
> 04:04.0 SCSI storage controller: Adaptec ASC-29320ALP U320 (rev ff)
We are coming closer...
With 'lspci -v -s04:04.0' you will see the IRQ your Adaptec controller
is assigned to.
http://www.linux.org/docs/ldp/howto/Plug-and-Play-HOWTO-7.html gives
more information about interrupts in general.
You may check your BIOS for means to assign interrupts to certain
slots. Moving either your SCSI controller or the other conflicting
card to another slot might help.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-09-30 16:02 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-09-28 8:27 pci passthrough with KVM Inigo Losada
2010-09-28 18:26 ` Alex Williamson
2010-09-29 8:56 ` Sebastian Hetze
2010-09-30 9:38 ` Inigo Losada
2010-09-30 15:53 ` Sebastian Hetze
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox