* PCI passthrough = NMIs @ 2008-10-13 6:09 Zwane Mwaikambo 2008-10-13 6:35 ` Muli Ben-Yehuda 2008-10-14 7:33 ` Han, Weidong 0 siblings, 2 replies; 9+ messages in thread From: Zwane Mwaikambo @ 2008-10-13 6:09 UTC (permalink / raw) To: kvm Hi, I've been trying to get a Linux guest working with PCI passthrough of an ethernet card using the vtd branches. The device detection works and the guest reports a link, however as soon as i try and ping the guest it receives an NMI (i'm guessing this is PCI DMA related). Interrupt delivery to the guest looks fine (count increases at a low rate) and isn't shared with anything else on the host. Thanks for any hints. Zwane ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-13 6:09 PCI passthrough = NMIs Zwane Mwaikambo @ 2008-10-13 6:35 ` Muli Ben-Yehuda 2008-10-13 15:04 ` Zwane Mwaikambo 2008-10-14 7:33 ` Han, Weidong 1 sibling, 1 reply; 9+ messages in thread From: Muli Ben-Yehuda @ 2008-10-13 6:35 UTC (permalink / raw) To: Zwane Mwaikambo; +Cc: kvm On Sun, Oct 12, 2008 at 11:09:14PM -0700, Zwane Mwaikambo wrote: > Hi, > I've been trying to get a Linux guest working with PCI passthrough > of an ethernet card using the vtd branches. The device detection works and > the guest reports a link, however as soon as i try and ping the guest it > receives an NMI (i'm guessing this is PCI DMA related). Interrupt delivery > to the guest looks fine (count increases at a low rate) and isn't shared > with anything else on the host. > > Thanks for any hints. Anything in the host dmesg? Are you using VT-d? Cheers, Muli -- The First Workshop on I/O Virtualization (WIOV '08) Dec 2008, San Diego, CA, http://www.usenix.org/wiov08/ <-> SYSTOR 2009---The Israeli Experimental Systems Conference http://www.haifa.il.ibm.com/conferences/systor2009/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-13 6:35 ` Muli Ben-Yehuda @ 2008-10-13 15:04 ` Zwane Mwaikambo 2008-10-14 3:08 ` Sheng Yang 0 siblings, 1 reply; 9+ messages in thread From: Zwane Mwaikambo @ 2008-10-13 15:04 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: kvm On Mon, 13 Oct 2008, Muli Ben-Yehuda wrote: > On Sun, Oct 12, 2008 at 11:09:14PM -0700, Zwane Mwaikambo wrote: > > Hi, > > I've been trying to get a Linux guest working with PCI passthrough > > of an ethernet card using the vtd branches. The device detection works and > > the guest reports a link, however as soon as i try and ping the guest it > > receives an NMI (i'm guessing this is PCI DMA related). Interrupt delivery > > to the guest looks fine (count increases at a low rate) and isn't shared > > with anything else on the host. > > > > Thanks for any hints. > > Anything in the host dmesg? Are you using VT-d? Yep using VT-d on an intel Q35, i get the following in order; pci 0000:11:0a.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 DMAR:[DMA Read] Request device [00:1e.0] fault addr 7841000 DMAR:[fault reason 06] PTE Read access is not set Uhhuh. NMI received for unknown reason a1. You have some hardware problem, likely on the PCI bus. Dazed and confused, but trying to continue DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 DMAR:[fault reason 05] PTE Write access is not set DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 DMAR:[fault reason 05] PTE Write access is not set The DMAR write fault printouts continue as long as you ping the IP address of the network card so it appears to be receiving packets. 00:1e.0 is the PCI bridge; 00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02) 00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02) 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) 00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02) 00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02) 00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02) 00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02) 00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02) 00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02) 00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 5 (rev 02) 00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) 00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) 00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) 00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92) 00:1f.0 ISA bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 02) 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) 00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem (rev 02) 0d:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller 11:04.0 IDE interface: Integrated Technology Express, Inc. Unknown device 8213 11:09.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 10) 11:0a.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-13 15:04 ` Zwane Mwaikambo @ 2008-10-14 3:08 ` Sheng Yang 2008-10-14 4:39 ` Zwane Mwaikambo 0 siblings, 1 reply; 9+ messages in thread From: Sheng Yang @ 2008-10-14 3:08 UTC (permalink / raw) To: kvm; +Cc: Zwane Mwaikambo, Muli Ben-Yehuda On Monday 13 October 2008 23:04:34 Zwane Mwaikambo wrote: > On Mon, 13 Oct 2008, Muli Ben-Yehuda wrote: > > On Sun, Oct 12, 2008 at 11:09:14PM -0700, Zwane Mwaikambo wrote: > > > Hi, > > > I've been trying to get a Linux guest working with PCI passthrough > > > of an ethernet card using the vtd branches. The device detection works > > > and the guest reports a link, however as soon as i try and ping the > > > guest it receives an NMI (i'm guessing this is PCI DMA related). > > > Interrupt delivery to the guest looks fine (count increases at a low > > > rate) and isn't shared with anything else on the host. > > > > > > Thanks for any hints. > > > > Anything in the host dmesg? Are you using VT-d? > > Yep using VT-d on an intel Q35, i get the following in order; > > pci 0000:11:0a.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 > DMAR:[DMA Read] Request device [00:1e.0] fault addr 7841000 > DMAR:[fault reason 06] PTE Read access is not set > Uhhuh. NMI received for unknown reason a1. > You have some hardware problem, likely on the PCI bus. > Dazed and confused, but trying to continue > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > DMAR:[fault reason 05] PTE Write access is not set > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > DMAR:[fault reason 05] PTE Write access is not set Do you have VT-d enabled? Also reinstalled host kernel with KVM upstream(we modified host kernel to support VT-d in KVM)? Look like VT-d table is not set up... You can check what you got with "dmesg|grep DMAR". -- regards Yang, Sheng > > The DMAR write fault printouts continue as long as you ping the IP address > of the network card so it appears to be receiving packets. > > 00:1e.0 is the PCI bridge; > > 00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev > 02) 00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express > Integrated Graphics Controller (rev 02) 00:19.0 Ethernet controller: Intel > Corporation 82566DM-2 Gigabit Network Connection (rev 02) 00:1a.0 USB > Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 > (rev 02) 00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB > UHCI Controller #5 (rev 02) 00:1a.2 USB Controller: Intel Corporation > 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02) 00:1a.7 USB > Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 > (rev 02) 00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD > Audio Controller (rev 02) 00:1c.0 PCI bridge: Intel Corporation 82801I > (ICH9 Family) PCI Express Port 1 (rev 02) 00:1c.4 PCI bridge: Intel > Corporation 82801I (ICH9 Family) PCI Express Port 5 (rev 02) 00:1d.0 USB > Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 > (rev 02) 00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB > UHCI Controller #2 (rev 02) 00:1d.2 USB Controller: Intel Corporation > 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) 00:1d.7 USB > Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 > (rev 02) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92) > 00:1f.0 ISA bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface > Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation > 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 02) 00:1f.3 > SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) > 00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 > Family) Thermal Subsystem (rev 02) 0d:00.0 Ethernet controller: Intel > Corporation 82573L Gigabit Ethernet Controller 11:04.0 IDE interface: > Integrated Technology Express, Inc. Unknown device 8213 11:09.0 Ethernet > controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 10) > 11:0a.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet > Controller (rev 05) -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-14 3:08 ` Sheng Yang @ 2008-10-14 4:39 ` Zwane Mwaikambo 2008-10-14 7:16 ` Sheng Yang 0 siblings, 1 reply; 9+ messages in thread From: Zwane Mwaikambo @ 2008-10-14 4:39 UTC (permalink / raw) To: Sheng Yang; +Cc: kvm, Muli Ben-Yehuda On Tue, 14 Oct 2008, Sheng Yang wrote: > On Monday 13 October 2008 23:04:34 Zwane Mwaikambo wrote: > > On Mon, 13 Oct 2008, Muli Ben-Yehuda wrote: > > > On Sun, Oct 12, 2008 at 11:09:14PM -0700, Zwane Mwaikambo wrote: > > > > Hi, > > > > I've been trying to get a Linux guest working with PCI passthrough > > > > of an ethernet card using the vtd branches. The device detection works > > > > and the guest reports a link, however as soon as i try and ping the > > > > guest it receives an NMI (i'm guessing this is PCI DMA related). > > > > Interrupt delivery to the guest looks fine (count increases at a low > > > > rate) and isn't shared with anything else on the host. > > > > > > > > Thanks for any hints. > > > > > > Anything in the host dmesg? Are you using VT-d? > > > > Yep using VT-d on an intel Q35, i get the following in order; > > > > pci 0000:11:0a.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 > > DMAR:[DMA Read] Request device [00:1e.0] fault addr 7841000 > > DMAR:[fault reason 06] PTE Read access is not set > > Uhhuh. NMI received for unknown reason a1. > > You have some hardware problem, likely on the PCI bus. > > Dazed and confused, but trying to continue > > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > > DMAR:[fault reason 05] PTE Write access is not set > > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > > DMAR:[fault reason 05] PTE Write access is not set > > Do you have VT-d enabled? Also reinstalled host kernel with KVM upstream(we > modified host kernel to support VT-d in KVM)? Look like VT-d table is not set > up... You can check what you got with "dmesg|grep DMAR". Yep, i have VT-d enabled in the BIOS, i also had to disable legacy USB in order to get Linux to boot. This is what i get on the host kernel; ACPI: DMAR 7F487CA8, 00C8 (r1 Intel OEMDMAR 6040000 LOHR 1) DMAR:Host address width 36 DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000 DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000 DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000 DMAR:RMRR base: 0x000000007f4e4000 end: 0x000000007f4f0000 Although legacy USB not working would suggest that there are table entries already missing in the BIOS? Thanks, Zwane ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-14 4:39 ` Zwane Mwaikambo @ 2008-10-14 7:16 ` Sheng Yang 0 siblings, 0 replies; 9+ messages in thread From: Sheng Yang @ 2008-10-14 7:16 UTC (permalink / raw) To: Zwane Mwaikambo; +Cc: kvm, Muli Ben-Yehuda On Tuesday 14 October 2008 12:39:52 Zwane Mwaikambo wrote: > On Tue, 14 Oct 2008, Sheng Yang wrote: > > On Monday 13 October 2008 23:04:34 Zwane Mwaikambo wrote: > > > On Mon, 13 Oct 2008, Muli Ben-Yehuda wrote: > > > > On Sun, Oct 12, 2008 at 11:09:14PM -0700, Zwane Mwaikambo wrote: > > > > > Hi, > > > > > I've been trying to get a Linux guest working with PCI passthrough > > > > > of an ethernet card using the vtd branches. The device detection > > > > > works and the guest reports a link, however as soon as i try and > > > > > ping the guest it receives an NMI (i'm guessing this is PCI DMA > > > > > related). Interrupt delivery to the guest looks fine (count > > > > > increases at a low rate) and isn't shared with anything else on the > > > > > host. > > > > > > > > > > Thanks for any hints. > > > > > > > > Anything in the host dmesg? Are you using VT-d? > > > > > > Yep using VT-d on an intel Q35, i get the following in order; > > > > > > pci 0000:11:0a.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 > > > DMAR:[DMA Read] Request device [00:1e.0] fault addr 7841000 > > > DMAR:[fault reason 06] PTE Read access is not set > > > Uhhuh. NMI received for unknown reason a1. > > > You have some hardware problem, likely on the PCI bus. > > > Dazed and confused, but trying to continue > > > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > > > DMAR:[fault reason 05] PTE Write access is not set > > > DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 > > > DMAR:[fault reason 05] PTE Write access is not set > > > > Do you have VT-d enabled? Also reinstalled host kernel with KVM > > upstream(we modified host kernel to support VT-d in KVM)? Look like VT-d > > table is not set up... You can check what you got with "dmesg|grep DMAR". > > Yep, i have VT-d enabled in the BIOS, i also had to disable legacy USB in > order to get Linux to boot. This is what i get on the host kernel; > > ACPI: DMAR 7F487CA8, 00C8 (r1 Intel OEMDMAR 6040000 LOHR 1) > DMAR:Host address width 36 > DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000 > DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000 > DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000 > DMAR:RMRR base: 0x000000007f4e4000 end: 0x000000007f4f0000 > > Although legacy USB not working would suggest that there are table entries > already missing in the BIOS? The VT-d table should be set up by KVM, but it's very strange that if you enable VT-d for native, USB can't work. If you sure you are using kernel compiled from KVM upstream, I think it maybe a BIOS issue... -- regards Yang, Sheng ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: PCI passthrough = NMIs 2008-10-13 6:09 PCI passthrough = NMIs Zwane Mwaikambo 2008-10-13 6:35 ` Muli Ben-Yehuda @ 2008-10-14 7:33 ` Han, Weidong 2008-10-14 17:16 ` Zwane Mwaikambo 1 sibling, 1 reply; 9+ messages in thread From: Han, Weidong @ 2008-10-14 7:33 UTC (permalink / raw) To: Zwane Mwaikambo, kvm Zwane Mwaikambo wrote: > Hi, > I've been trying to get a Linux guest working with PCI passthrough > of an ethernet card using the vtd branches. The device detection > works and the guest reports a link, however as soon as i try and ping > the guest it receives an NMI (i'm guessing this is PCI DMA related). > Interrupt delivery to the guest looks fine (count increases at a low > rate) and isn't shared with anything else on the host. > > Thanks for any hints. > > Zwane What do you mean "vtd branches"? Is it the vtd branch of Amit's tree? VT-d patches are in kvm.git, as long as you apply amit's userspace patch (you can find it on mailing list), you should can assign device to guest. Make sure you enable VT-d in BIOS and set CONFIG_DMAR in config, and remove device driver before you assign device. NIC and USB controller assignment works fine for me. Regards, Weidong ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: PCI passthrough = NMIs 2008-10-14 7:33 ` Han, Weidong @ 2008-10-14 17:16 ` Zwane Mwaikambo 2008-10-15 9:50 ` Amit Shah 0 siblings, 1 reply; 9+ messages in thread From: Zwane Mwaikambo @ 2008-10-14 17:16 UTC (permalink / raw) To: Han, Weidong; +Cc: kvm On Tue, 14 Oct 2008, Han, Weidong wrote: > Zwane Mwaikambo wrote: > > Hi, > > I've been trying to get a Linux guest working with PCI > passthrough > > of an ethernet card using the vtd branches. The device detection > > works and the guest reports a link, however as soon as i try and ping > > the guest it receives an NMI (i'm guessing this is PCI DMA related). > > Interrupt delivery to the guest looks fine (count increases at a low > > rate) and isn't shared with anything else on the host. > > > > Thanks for any hints. > > > > Zwane > > What do you mean "vtd branches"? Is it the vtd branch of Amit's tree? > > VT-d patches are in kvm.git, as long as you apply amit's userspace patch > (you can find it on mailing list), you should can assign device to > guest. Make sure you enable VT-d in BIOS and set CONFIG_DMAR in config, > and remove device driver before you assign device. NIC and USB > controller assignment works fine for me. I'm using the following repositories without any additional patches; git://git.kernel.org/pub/scm/linux/kernel/git/amit/kvm.git vtd git://git.kernel.org/pub/scm/linux/kernel/git/amit/kvm-userspace.git vtd The host kernel only has the e100 network driver enabled, the guest kernel only has the e1000 network driver enabled. I have enabled VT-d in the BIOS and enabled CONFIG_DMAR in the host kernel config. I'm really beginning to suspect the BIOS unfortunately :( Thanks, Zwane ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PCI passthrough = NMIs 2008-10-14 17:16 ` Zwane Mwaikambo @ 2008-10-15 9:50 ` Amit Shah 0 siblings, 0 replies; 9+ messages in thread From: Amit Shah @ 2008-10-15 9:50 UTC (permalink / raw) To: Zwane Mwaikambo; +Cc: Han, Weidong, kvm * On Tuesday 14 Oct 2008 22:46:37 Zwane Mwaikambo wrote: > On Tue, 14 Oct 2008, Han, Weidong wrote: > > Zwane Mwaikambo wrote: > > > Hi, > > > I've been trying to get a Linux guest working with PCI > > > > passthrough > > > > > of an ethernet card using the vtd branches. The device detection > > > works and the guest reports a link, however as soon as i try and ping > > > the guest it receives an NMI (i'm guessing this is PCI DMA related). > > > Interrupt delivery to the guest looks fine (count increases at a low > > > rate) and isn't shared with anything else on the host. > > > > > > Thanks for any hints. > > > > > > Zwane > > > > What do you mean "vtd branches"? Is it the vtd branch of Amit's tree? > > > > VT-d patches are in kvm.git, as long as you apply amit's userspace patch > > (you can find it on mailing list), you should can assign device to > > guest. Make sure you enable VT-d in BIOS and set CONFIG_DMAR in config, > > and remove device driver before you assign device. NIC and USB > > controller assignment works fine for me. > > I'm using the following repositories without any additional patches; > > git://git.kernel.org/pub/scm/linux/kernel/git/amit/kvm.git vtd > git://git.kernel.org/pub/scm/linux/kernel/git/amit/kvm-userspace.git vtd Can you try using Avi's kvm.git tree and the userspace tree same as above (it's been updated now and you'll have to use +vtd as the branch)? That's the newest code; though I'm not sure if it will solve your problem. > The host kernel only has the e100 network driver enabled, the guest kernel > only has the e1000 network driver enabled. I have enabled VT-d in the BIOS > and enabled CONFIG_DMAR in the host kernel config. I'm really beginning to > suspect the BIOS unfortunately :( Amiit. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2008-10-15 9:51 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-10-13 6:09 PCI passthrough = NMIs Zwane Mwaikambo 2008-10-13 6:35 ` Muli Ben-Yehuda 2008-10-13 15:04 ` Zwane Mwaikambo 2008-10-14 3:08 ` Sheng Yang 2008-10-14 4:39 ` Zwane Mwaikambo 2008-10-14 7:16 ` Sheng Yang 2008-10-14 7:33 ` Han, Weidong 2008-10-14 17:16 ` Zwane Mwaikambo 2008-10-15 9:50 ` Amit Shah
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.