* RE: PCI Passthrough Problems/Questions
@ 2010-10-28 14:01 Nick Couchman
2010-10-29 2:54 ` Weidong Han
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-28 14:01 UTC (permalink / raw)
To: weidong.han; +Cc: xen-devel, konrad.wilk
[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]
>>> "Han, Weidong" 10/27/10 10:38 PM >>>
> I don't want xend.log. I want xen log (xm dmesg or captured by serial port). And pls also post qemu log (/var/log/xen/dm-qemu-xxx.log).
Sorry about that, attached are the output from xm dmesg and the qemu-dm-XP.log file.
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
[-- Attachment #2: xm-dmesg.log --]
[-- Type: application/octet-stream, Size: 16384 bytes --]
: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 4 CPUs
(XEN) Created cpupool 0 with scheduler SMP Credit Scheduler (credit)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8af000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000112000000->0000000114000000 (951968 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80002000->ffffffff808af000
(XEN) Init. ramdisk: ffffffff808af000->ffffffff81e7a600
(XEN) Phys-Mach map: ffffea0000000000->ffffea0000753500
(XEN) Start info: ffffffff81e7b000->ffffffff81e7b4b4
(XEN) Page tables: ffffffff81e7c000->ffffffff81e8f000
(XEN) Boot stack: ffffffff81e8f000->ffffffff81e90000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80002000
(XEN) Dom0 has maximum 4 VCPUs
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 8<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 16[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:2.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 17[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:2.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 24[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 26[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 27[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.3
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 200[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:19.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 208[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 209[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 210[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 215[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 216[VT-D]iommu.c:1358: d0:PCIe: map bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 224<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 225<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 232[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 233[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 234[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 239[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 240<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 248[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 250[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 251[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 1, devfn: 0<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff55000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff57000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff54000
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 180kB init memory.
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d1:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=1
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d1:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=1 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d2:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d2:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d2:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=2
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d2: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d2:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d2:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d2:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d2: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d2:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d2:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=2 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d3:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d3:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d3:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=3
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d3: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d3:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d3:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d3:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d3: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d3:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d3:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=3 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d4:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d4:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d4:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=4
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d4: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d4:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d4:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d4:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d4: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d4:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d4:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=4 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d5:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d5:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d5:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=5
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d5: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d5:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d5:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d5:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d5: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d5:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d5:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=5 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
[-- Attachment #3: qemu-dm-XP.log --]
[-- Type: application/octet-stream, Size: 2817 bytes --]
domid: 5
config qemu network with xen bridge for tap5.0 br0
Using file /dev/local/XP in read-write mode
Watching /local/domain/0/device-model/5/logdirty/cmd
Watching /local/domain/0/device-model/5/command
char device redirected to /dev/pts/1
qemu_map_cache_init nr_buckets = 10000 size 4194304
/usr/src/packages/BUILD/xen-4.0.1-testing/tools/ioemu-dir/hw/xen_blktap.c:715: Init blktap pipes
shared page at pfn feffd
buffered io page at pfn feffb
Guest uuid = 8f6bf60c-f931-d1a5-efda-cbbe66724cfa
Time offset set 0
populating video RAM at ff000000
do_xenmem_op_retry: cmd 6 count 400
mapping video RAM from ff000000
Register xen platform.
Done register platform.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
xs_read(/local/domain/0/device-model/5/xen_extended_power_mgmt): read error
xs_read(): vncpasswd get error. /vm/8f6bf60c-f931-d1a5-efda-cbbe66724cfa/vncpasswd.
medium change watch on `hdc' (index: 1):
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
Log-dirty: no command yet.
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
xs_read(/local/domain/5/log-throttling): read error
qemu: ignoring not-understood drive `/local/domain/5/log-throttling'
medium change watch on `/local/domain/5/log-throttling' - unknown device, ignored
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 02:00.0 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Disable power management
pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x2:0x0.0x0
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dc81)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fc000)
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dca1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fd000)
pci_intx: intx=1
register_real_device: Real physical device 02:00.0 registered successfuly!
IRQ type = INTx
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 02:00.1 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Disable power management
pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x2:0x0.0x1
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dcc1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fe000)
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dce1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79ff000)
pci_intx: intx=1
register_real_device: Real physical device 02:00.1 registered successfuly!
IRQ type = INTx
[-- Attachment #4: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-28 14:01 PCI Passthrough Problems/Questions Nick Couchman
@ 2010-10-29 2:54 ` Weidong Han
2010-10-29 18:52 ` Nick Couchman
0 siblings, 1 reply; 19+ messages in thread
From: Weidong Han @ 2010-10-29 2:54 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel@lists.xensource.com, konrad.wilk@Oracle.Com
Nick Couchman wrote:
>>>> "Han, Weidong" 10/27/10 10:38 PM >>>
>>>>
>> I don't want xend.log. I want xen log (xm dmesg or captured by serial port). And pls also post qemu log (/var/log/xen/dm-qemu-xxx.log).
>>
>
> Sorry about that, attached are the output from xm dmesg and the qemu-dm-XP.log file.
>
>
qemu-dm-XP.log looks fine. Per "xm dmesg" log, unmap[ing and mapping
00:02.0 and 00:02.1 are no problem. But why are there many unmap and map
(d1 -> d5)? did you create guest 5 times? Could you describe the crash
you encountered? BTW, the "xm dmesg" log looks not complete, it should
include following logs when you create hvm guest:
(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.0.0-rc3-pre
(XEN) HVM1: CPU speed is 2533 MHz
...
and pls add "iommu=verbose" in grub to get more VT-d info.
Regards,
Weidong
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-29 2:54 ` Weidong Han
@ 2010-10-29 18:52 ` Nick Couchman
2010-11-02 3:30 ` Weidong Han
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-29 18:52 UTC (permalink / raw)
To: weidong.han; +Cc: xen-devel, konrad.wilk
[-- Attachment #1: Type: text/plain, Size: 2153 bytes --]
> qemu-dm-XP.log looks fine. Per "xm dmesg" log, unmap[ing and mapping
> 00:02.0 and 00:02.1 are no problem. But why are there many unmap and map
> (d1 -> d5)? did you create guest 5 times? Could you describe the crash
> you encountered? BTW, the "xm dmesg" log looks not complete, it should
> include following logs when you create hvm guest:
> (XEN) HVM1: HVM Loader
> (XEN) HVM1: Detected Xen v4.0.0-rc3-pre
> (XEN) HVM1: CPU speed is 2533 MHz
> ...
>
> and pls add "iommu=verbose" in grub to get more VT-d info.
>
> Regards,
> Weidong
Weidong,
First, I have the following already specified on the xen.gz kernel line
in grub:
loglevel=all guest_lvl=all iommu=verbose
Yes, I had started the guest five times...I've rebooted the system and
started it once, and have attached the log files. As far as the domU
crashing - there's not much to describe. There is no longer any error
after issuing XM start, but the domU doesn't ever get to the point of
displaying anything on the screen - it crashes almost as soon as the xm
start command is issued (right after doing "xm start" the "xm list"
command shows the domU not running). So, the log file is going to be
the only form of output I can get you - there's nothing to report from
the screen. I don't see the output you mentioned anywhere in xm dmesg.
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
[-- Attachment #2: qemu-dm-XP.log --]
[-- Type: text/plain, Size: 2868 bytes --]
domid: 1
config qemu network with xen bridge for tap1.0 br0
Using file /dev/local/XP in read-write mode
Watching /local/domain/0/device-model/1/logdirty/cmd
Watching /local/domain/0/device-model/1/command
char device redirected to /dev/pts/1
qemu_map_cache_init nr_buckets = 10000 size 4194304
/usr/src/packages/BUILD/xen-4.0.1-testing/tools/ioemu-dir/hw/xen_blktap.c:715: Init blktap pipes
shared page at pfn feffd
buffered io page at pfn feffb
Guest uuid = 8f6bf60c-f931-d1a5-efda-cbbe66724cfa
Time offset set 0
populating video RAM at ff000000
do_xenmem_op_retry: cmd 6 count 400
mapping video RAM from ff000000
Register xen platform.
Done register platform.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
xs_read(/local/domain/0/device-model/1/xen_extended_power_mgmt): read error
xs_read(): vncpasswd get error. /vm/8f6bf60c-f931-d1a5-efda-cbbe66724cfa/vncpasswd.
medium change watch on `hdc' (index: 1):
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
Log-dirty: no command yet.
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
xs_read(/local/domain/1/log-throttling): read error
qemu: ignoring not-understood drive `/local/domain/1/log-throttling'
medium change watch on `/local/domain/1/log-throttling' - unknown device, ignored
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 02:00.0 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Disable power management
pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x2:0x0.0x0
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dc81)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fc000)
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dca1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fd000)
pci_intx: intx=1
register_real_device: Real physical device 02:00.0 registered successfuly!
IRQ type = INTx
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 02:00.1 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Disable power management
pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x2:0x0.0x1
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dcc1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79fe000)
pt_register_regions: IO region registered (size=0x00000020 base_addr=0x0000dce1)
pt_register_regions: IO region registered (size=0x00001000 base_addr=0xf79ff000)
pci_intx: intx=1
register_real_device: Real physical device 02:00.1 registered successfuly!
IRQ type = INTx
[-- Attachment #3: xm-dmesg.log --]
[-- Type: text/plain, Size: 13877 bytes --]
__ __ _ _ ___ _
\ \/ /___ _ __ | || | / _ \ / |
\ // _ \ '_ \ | || |_| | | || |
/ \ __/ | | | |__ _| |_| || |
/_/\_\___|_| |_| |_|(_)___(_)_|
(XEN) Xen version 4.0.1 (root@it.seakr.com) (gcc version 4.5.0 20100604 [gcc-4_5-branch revision 160292] (SUSE Linux) ) Wed Oct 27 08:49:08 MDT 2010
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com2 com2=115200,8n1,0xecb8,0 loglevel=all guest_lvl=all iommu=verbose
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009a800 (usable)
(XEN) 00000000000f0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000dd9ffc00 (usable)
(XEN) 00000000dd9ffc00 - 00000000dda53c00 (ACPI NVS)
(XEN) 00000000dda53c00 - 00000000dda55c00 (ACPI data)
(XEN) 00000000dda55c00 - 00000000e0000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fed00400 (reserved)
(XEN) 00000000fed20000 - 00000000feda0000 (reserved)
(XEN) 00000000fee00000 - 00000000fef00000 (reserved)
(XEN) 00000000ffb00000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000000118000000 (usable)
(XEN) ACPI: RSDP 000FEC00, 0024 (r2 DELL )
(XEN) ACPI: XSDT 000FC7E9, 0074 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: FACP 000FC919, 00F4 (r3 DELL B10K 15 ASL 61)
(XEN) ACPI: DSDT FFE9FB93, 5035 (r1 DELL dt_ex 1000 INTL 20050624)
(XEN) ACPI: FACS DD9FFC00, 0040
(XEN) ACPI: SSDT FFEA4CE9, 00AC (r1 DELL st_ex 1000 INTL 20050624)
(XEN) ACPI: APIC 000FCA0D, 0092 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: BOOT 000FCA9F, 0028 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: ASF! 000FCAC7, 0096 (r32 DELL B10K 15 ASL 61)
(XEN) ACPI: MCFG 000FCB5D, 003C (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: HPET 000FCB99, 0038 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: TCPA 000FCDF5, 0032 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: DMAR 000FCE27, 0120 (r1 DELL B10K 15 ASL 61)
(XEN) ACPI: SLIC 000FCBD1, 0176 (r1 DELL B10K 15 ASL 61)
(XEN) System RAM: 3929MB (4023908kB)
(XEN) Domain heap initialised
(XEN) Processor #0 7:7 APIC version 20
(XEN) Processor #1 7:7 APIC version 20
(XEN) Processor #2 7:7 APIC version 20
(XEN) Processor #3 7:7 APIC version 20
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) [VT-D]dmar.c:702: Host address width 36
(XEN) [VT-D]dmar.c:717: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:413: dmaru->address = fedc1000
(XEN) [VT-D]iommu.c:1105: drhd->address = fedc1000 iommu->reg = ffff82c3fff57000
(XEN) [VT-D]iommu.c:1107: cap = c9008020e30272 ecap = 1000
(XEN) [VT-D]dmar.c:341: endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:717: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:413: dmaru->address = fedc2000
(XEN) [VT-D]iommu.c:1105: drhd->address = fedc2000 iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:1107: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]dmar.c:341: endpoint: 0:2.0
(XEN) [VT-D]dmar.c:341: endpoint: 0:2.1
(XEN) [VT-D]dmar.c:717: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:413: dmaru->address = fedc3000
(XEN) [VT-D]iommu.c:1105: drhd->address = fedc3000 iommu->reg = ffff82c3fff55000
(XEN) [VT-D]iommu.c:1107: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]dmar.c:341: endpoint: 0:3.0
(XEN) [VT-D]dmar.c:341: endpoint: 0:3.2
(XEN) [VT-D]dmar.c:341: endpoint: 0:3.3
(XEN) [VT-D]dmar.c:717: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:413: dmaru->address = fedc4000
(XEN) [VT-D]iommu.c:1105: drhd->address = fedc4000 iommu->reg = ffff82c3fff54000
(XEN) [VT-D]iommu.c:1107: cap = c9008020630272 ecap = 1000
(XEN) [VT-D]dmar.c:427: flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:722: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:341: endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:341: endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:341: endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:341: endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:341: endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:341: endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:341: endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:341: endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:594: RMRR region: base_addr dda58000 end_address dda6ffff
(XEN) [VT-D]dmar.c:722: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:341: endpoint: 0:2.0
(XEN) [VT-D]dmar.c:341: endpoint: 0:2.1
(XEN) [VT-D]dmar.c:594: RMRR region: base_addr ddc00000 end_address dfffffff
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2660.082 MHz processor.
(XEN) Initing memory sharing.
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 4 CPUs
(XEN) Created cpupool 0 with scheduler SMP Credit Scheduler (credit)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8af000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000112000000->0000000114000000 (951968 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80002000->ffffffff808af000
(XEN) Init. ramdisk: ffffffff808af000->ffffffff81e7a600
(XEN) Phys-Mach map: ffffea0000000000->ffffea0000753500
(XEN) Start info: ffffffff81e7b000->ffffffff81e7b4b4
(XEN) Page tables: ffffffff81e7c000->ffffffff81e8f000
(XEN) Boot stack: ffffffff81e8f000->ffffffff81e90000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80002000
(XEN) Dom0 has maximum 4 VCPUs
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 8<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 16[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:2.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 17[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:2.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 24[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 26[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 27[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:3.3
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 200[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:19.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 208[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 209[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 210[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 215[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 216[VT-D]iommu.c:1358: d0:PCIe: map bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 224<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 225<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 232[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 233[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 234[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 239[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 240<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 248[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 250[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 0, devfn: 251[VT-D]iommu.c:1368: d0:PCI: map bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 1, devfn: 0<0>[VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff55000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff57000
(XEN) [VT-D]iommu.c:722: iommu_enable_translation: iommu->reg = ffff82c3fff54000
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 180kB init memory.
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d1:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=1
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d1:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI: map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=1 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
[-- Attachment #4: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-10-29 18:52 ` Nick Couchman
@ 2010-11-02 3:30 ` Weidong Han
2010-11-02 19:07 ` Nick Couchman
0 siblings, 1 reply; 19+ messages in thread
From: Weidong Han @ 2010-11-02 3:30 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel@lists.xensource.com, konrad.wilk@Oracle.Com
Nick Couchman wrote:
>> qemu-dm-XP.log looks fine. Per "xm dmesg" log, unmap[ing and mapping
>> 00:02.0 and 00:02.1 are no problem. But why are there many unmap and map
>> (d1 -> d5)? did you create guest 5 times? Could you describe the crash
>> you encountered? BTW, the "xm dmesg" log looks not complete, it should
>> include following logs when you create hvm guest:
>> (XEN) HVM1: HVM Loader
>> (XEN) HVM1: Detected Xen v4.0.0-rc3-pre
>> (XEN) HVM1: CPU speed is 2533 MHz
>> ...
>>
>> and pls add "iommu=verbose" in grub to get more VT-d info.
>>
>> Regards,
>> Weidong
>>
>
> Weidong,
> First, I have the following already specified on the xen.gz kernel line
> in grub:
>
> loglevel=all guest_lvl=all iommu=verbose
>
> Yes, I had started the guest five times...I've rebooted the system and
> started it once, and have attached the log files. As far as the domU
> crashing - there's not much to describe. There is no longer any error
> after issuing XM start, but the domU doesn't ever get to the point of
> displaying anything on the screen - it crashes almost as soon as the xm
> start command is issued (right after doing "xm start" the "xm list"
> command shows the domU not running). So, the log file is going to be
> the only form of output I can get you - there's nothing to report from
> the screen. I don't see the output you mentioned anywhere in xm dmesg.
>
> -Nick
>
>
>
The logs look no problem. I'm wondering if the device implies PCI
standard. At last, I think you can try: only assign 02:00.0
(pci=[02:00.0]), or assign them as multiple devices in guest
(pci=[02:00.0-1]).
Regards,
Weidong
> --------
>
> This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-11-02 3:30 ` Weidong Han
@ 2010-11-02 19:07 ` Nick Couchman
2010-11-03 1:01 ` Weidong Han
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-11-02 19:07 UTC (permalink / raw)
To: weidong.han; +Cc: xen-devel, konrad.wilk
> >
> >
> >
> The logs look no problem. I'm wondering if the device implies PCI
> standard. At last, I think you can try: only assign 02:00.0
> (pci=[02:00.0]), or assign them as multiple devices in guest
> (pci=[02:00.0-1]).
>
> Regards,
> Weidong
Should have tried that on my own. Forwarding just 02:00.0 through
worked just fine!
So, the patch you had me apply to the Xen 4 source - is that a final
version of the patch, or is there still some work to do to come up with
a final fix for this?
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-11-02 19:07 ` Nick Couchman
@ 2010-11-03 1:01 ` Weidong Han
2010-11-03 14:54 ` Nick Couchman
0 siblings, 1 reply; 19+ messages in thread
From: Weidong Han @ 2010-11-03 1:01 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel@lists.xensource.com, konrad.wilk@Oracle.Com
[-- Attachment #1: Type: text/plain, Size: 751 bytes --]
Nick Couchman wrote:
>>>
>>>
>> The logs look no problem. I'm wondering if the device implies PCI
>> standard. At last, I think you can try: only assign 02:00.0
>> (pci=[02:00.0]), or assign them as multiple devices in guest
>> (pci=[02:00.0-1]).
>>
>> Regards,
>> Weidong
>>
>
> Should have tried that on my own. Forwarding just 02:00.0 through
> worked just fine!
>
> So, the patch you had me apply to the Xen 4 source - is that a final
> version of the patch, or is there still some work to do to come up with
> a final fix for this?
>
>
The patch is just for your testing. I think Jan's patch should fix the
issue. Jan sent it out in xen-devel mailing list yesterday, I attached
it here, pls try with it.
Regards,
Weidong
[-- Attachment #2: vtd-check-secbus-devfn.patch --]
[-- Type: text/plain, Size: 1750 bytes --]
If the device at <secbus>:00.0 is the device the mapping operation was
initiated for, trying to map it a second time will fail, and hence
this second mapping attempt must be prevented (as was done prior to
said c/s).
Once at it, simplify the code a little, too.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1371,23 +1371,16 @@ static int domain_context_mapping(struct
if ( find_upstream_bridge(&bus, &devfn, &secbus) < 1 )
break;
- /* PCIe to PCI/PCIx bridge */
- if ( pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE )
- {
- ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
- if ( ret )
- return ret;
+ ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
- /*
- * Devices behind PCIe-to-PCI/PCIx bridge may generate
- * different requester-id. It may originate from devfn=0
- * on the secondary bus behind the bridge. Map that id
- * as well.
- */
+ /*
+ * Devices behind PCIe-to-PCI/PCIx bridge may generate different
+ * requester-id. It may originate from devfn=0 on the secondary bus
+ * behind the bridge. Map that id as well if we didn't already.
+ */
+ if ( !ret && pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE &&
+ (secbus != pdev->bus || pdev->devfn != 0) )
ret = domain_context_mapping_one(domain, drhd->iommu, secbus, 0);
- }
- else /* Legacy PCI bridge */
- ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
break;
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-11-03 1:01 ` Weidong Han
@ 2010-11-03 14:54 ` Nick Couchman
0 siblings, 0 replies; 19+ messages in thread
From: Nick Couchman @ 2010-11-03 14:54 UTC (permalink / raw)
To: weidong.han; +Cc: xen-devel, konrad.wilk
I can confirm that Jan's patch fixes the problem!
Thanks for the help!
-Nick
On Wed, 2010-11-03 at 09:01 +0800, Weidong Han wrote:
> Nick Couchman wrote:
> >>>
> >>>
> >> The logs look no problem. I'm wondering if the device implies PCI
> >> standard. At last, I think you can try: only assign 02:00.0
> >> (pci=[02:00.0]), or assign them as multiple devices in guest
> >> (pci=[02:00.0-1]).
> >>
> >> Regards,
> >> Weidong
> >>
> >
> > Should have tried that on my own. Forwarding just 02:00.0 through
> > worked just fine!
> >
> > So, the patch you had me apply to the Xen 4 source - is that a final
> > version of the patch, or is there still some work to do to come up with
> > a final fix for this?
> >
> >
> The patch is just for your testing. I think Jan's patch should fix the
> issue. Jan sent it out in xen-devel mailing list yesterday, I attached
> it here, pls try with it.
>
> Regards,
> Weidong
> plain text document attachment (vtd-check-secbus-devfn.patch)
> If the device at <secbus>:00.0 is the device the mapping operation was
> initiated for, trying to map it a second time will fail, and hence
> this second mapping attempt must be prevented (as was done prior to
> said c/s).
>
> Once at it, simplify the code a little, too.
>
> Signed-off-by: Jan Beulich <jbeulich@novell.com>
>
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -1371,23 +1371,16 @@ static int domain_context_mapping(struct
> if ( find_upstream_bridge(&bus, &devfn, &secbus) < 1 )
> break;
>
> - /* PCIe to PCI/PCIx bridge */
> - if ( pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE )
> - {
> - ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
> - if ( ret )
> - return ret;
> + ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
>
> - /*
> - * Devices behind PCIe-to-PCI/PCIx bridge may generate
> - * different requester-id. It may originate from devfn=0
> - * on the secondary bus behind the bridge. Map that id
> - * as well.
> - */
> + /*
> + * Devices behind PCIe-to-PCI/PCIx bridge may generate different
> + * requester-id. It may originate from devfn=0 on the secondary bus
> + * behind the bridge. Map that id as well if we didn't already.
> + */
> + if ( !ret && pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE &&
> + (secbus != pdev->bus || pdev->devfn != 0) )
> ret = domain_context_mapping_one(domain, drhd->iommu, secbus, 0);
> - }
> - else /* Legacy PCI bridge */
> - ret = domain_context_mapping_one(domain, drhd->iommu, bus, devfn);
>
> break;
>
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <4CC8A62602000099000BAA42@collaborate.seakr.com>]
* RE: PCI Passthrough Problems/Questions
[not found] <4CC8A62602000099000BAA42@collaborate.seakr.com>
@ 2010-10-28 4:37 ` Han, Weidong
0 siblings, 0 replies; 19+ messages in thread
From: Han, Weidong @ 2010-10-28 4:37 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel@lists.xensource.com, konrad.wilk@Oracle.Com
I don't want xend.log. I want xen log (xm dmesg or captured by serial port). And pls also post qemu log (/var/log/xen/dm-qemu-xxx.log).
Regards,
Weidong
-----Original Message-----
From: Nick Couchman [mailto:Nick.Couchman@seakr.com]
Sent: Thursday, October 28, 2010 12:23 PM
To: Han, Weidong
Cc: xen-devel@lists.xensource.com; konrad.wilk@Oracle.Com
Subject: Re: [Xen-devel] PCI Passthrough Problems/Questions
> the log of xm dmesg looks no problem. you'd better post complete log
> and lspci -v.
Sorry about that, xend.log is attached, and here's the output of lspci -v:
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
Subsystem: Dell Device 0420
Flags: bus master, fast devsel, latency 0
Capabilities: [e0] Vendor Specific Information: Len=0c
Kernel driver in use: agpgart-intel
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=01, subordinate=02, sec-latency=0
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: f7900000-f79fffff
Prefetchable memory behind bridge: 00000000f0000000-00000000f00fffff
Capabilities: [88] Subsystem: Dell Device 0420
Capabilities: [80] Power Management version 3
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [a0] Express Root Port (Slot+), MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [140] Root Complex Link
Kernel driver in use: pcieport
00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03) (prog-if 00 [VGA controller])
Subsystem: Dell Device 0420
Flags: bus master, fast devsel, latency 0, IRQ 435
Memory at f7c00000 (64-bit, non-prefetchable) [size=4M]
Memory at e0000000 (64-bit, prefetchable) [size=256M]
I/O ports at ecb0 [size=8]
Expansion ROM at [disabled]
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [d0] Power Management version 2
Kernel driver in use: i915
00:02.1 Display controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
Subsystem: Dell Device 0420
Flags: bus master, fast devsel, latency 0
Memory at f7b00000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [d0] Power Management version 2
00:03.0 Communication controller: Intel Corporation 4 Series Chipset HECI Controller (rev 03)
Subsystem: Dell Device 0420
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at feda6000 (64-bit, non-prefetchable) [size=16]
Capabilities: [50] Power Management version 3
Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+
00:03.2 IDE interface: Intel Corporation 4 Series Chipset PT IDER Controller (rev 03) (prog-if 85 [Master SecO PriO])
Subsystem: Dell Device 0420
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 9
I/O ports at fe80 [size=8]
I/O ports at fe90 [size=4]
I/O ports at fea0 [size=8]
I/O ports at feb0 [size=4]
I/O ports at fef0 [size=16]
Capabilities: [c8] Power Management version 3
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
00:03.3 Serial controller: Intel Corporation 4 Series Chipset Serial KT Controller (rev 03) (prog-if 02 [16550])
Subsystem: Dell Device 0420
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 17
I/O ports at ecb8 [size=8]
Memory at f7ad8000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [c8] Power Management version 3
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Kernel driver in use: serial
00:19.0 Ethernet controller: Intel Corporation 82567LM-3 Gigabit Network Connection (rev 02)
Subsystem: Dell Device 0276
Flags: bus master, fast devsel, latency 0, IRQ 434
Memory at f7ae0000 (32-bit, non-prefetchable) [size=128K]
Memory at f7ad9000 (32-bit, non-prefetchable) [size=4K]
I/O ports at ece0 [size=32]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [e0] PCI Advanced Features
Kernel driver in use: e1000e
00:1a.0 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #4 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 16
I/O ports at ff20 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1a.1 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #5 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 17
I/O ports at ff00 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1a.2 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #6 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 22
I/O ports at fc00 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1a.7 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB2 EHCI Controller #2 (rev 02) (prog-if 20 [EHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 22
Memory at f7ada000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
Kernel driver in use: ehci_hcd
00:1b.0 Audio device: Intel Corporation 82801JD/DO (ICH10 Family) HD Audio Controller (rev 02)
Subsystem: Dell Device 0420
Flags: bus master, fast devsel, latency 0, IRQ 433
Memory at f7adc000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [130] Root Complex Link
Kernel driver in use: HDA Intel
00:1c.0 PCI bridge: Intel Corporation 82801JD/DO (ICH10 Family) PCI Express Port 1 (rev 02) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
I/O behind bridge: 00001000-00001fff
Memory behind bridge: f7800000-f78fffff
Prefetchable memory behind bridge: 00000000fc000000-00000000fc1fffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Dell Device 0420
Capabilities: [a0] Power Management version 2
Capabilities: [100] Virtual Channel
Capabilities: [180] Root Complex Link
Kernel driver in use: pcieport
00:1c.1 PCI bridge: Intel Corporation 82801JD/DO (ICH10 Family) PCI Express Port 2 (rev 02) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
I/O behind bridge: 00002000-00002fff
Memory behind bridge: f7700000-f77fffff
Prefetchable memory behind bridge: 00000000fc200000-00000000fc3fffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Dell Device 0420
Capabilities: [a0] Power Management version 2
Capabilities: [100] Virtual Channel
Capabilities: [180] Root Complex Link
Kernel driver in use: pcieport
00:1d.0 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #1 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 23
I/O ports at ff80 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1d.1 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #2 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 17
I/O ports at ff60 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1d.2 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #3 (rev 02) (prog-if 00 [UHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 18
I/O ports at ff40 [size=32]
Capabilities: [50] PCI Advanced Features
Kernel driver in use: uhci_hcd
00:1d.7 USB Controller: Intel Corporation 82801JD/DO (ICH10 Family) USB2 EHCI Controller #1 (rev 02) (prog-if 20 [EHCI])
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0, IRQ 23
Memory at ff980000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
Kernel driver in use: ehci_hcd
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a2) (prog-if 01 [Subtractive decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=05, subordinate=05, sec-latency=32
Capabilities: [50] Subsystem: Dell Device 0420
00:1f.0 ISA bridge: Intel Corporation 82801JDO (ICH10DO) LPC Interface Controller (rev 02)
Subsystem: Dell Device 0420
Flags: bus master, medium devsel, latency 0
Capabilities: [e0] Vendor Specific Information: Len=0c
00:1f.2 RAID bus controller: Intel Corporation 82801 SATA RAID Controller (rev 02)
Subsystem: Dell Device 0420
Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 436
I/O ports at fe00 [size=8]
I/O ports at fe10 [size=4]
I/O ports at fe20 [size=8]
I/O ports at fe30 [size=4]
I/O ports at fec0 [size=32]
Memory at ff970000 (32-bit, non-prefetchable) [size=2K]
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit-
Capabilities: [70] Power Management version 3
Capabilities: [a8] SATA HBA v1.0
Capabilities: [b0] PCI Advanced Features
Kernel driver in use: ahci
00:1f.3 SMBus: Intel Corporation 82801JD/DO (ICH10 Family) SMBus Controller (rev 02)
Subsystem: Dell Device 0420
Flags: medium devsel, IRQ 18
Memory at f7adb000 (64-bit, non-prefetchable) [size=256]
I/O ports at 0980 [size=32]
Kernel driver in use: i801_smbus
01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at f0000000 (64-bit, prefetchable) [size=64K]
Bus: primary=01, secondary=02, subordinate=02, sec-latency=64
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: f7900000-f79fffff
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express PCI/PCI-X Bridge, MSI 00
Capabilities: [100] Power Budgeting
Kernel driver in use: pciback
02:00.0 Serial controller: Perle Systems Ltd Device 0331 (prog-if 06 [16950])
Subsystem: Oxford Semiconductor Ltd Device 9501
Flags: medium devsel, IRQ 16
I/O ports at dc80 [disabled] [size=32]
Memory at f79fc000 (32-bit, non-prefetchable) [disabled] [size=4K]
I/O ports at dca0 [disabled] [size=32]
Memory at f79fd000 (32-bit, non-prefetchable) [disabled] [size=4K]
Capabilities: [40] Power Management version 1
Kernel driver in use: pciback
02:00.1 Bridge: Perle Systems Ltd Device 0331
Subsystem: Oxford Semiconductor Ltd Device 9511
Flags: medium devsel, IRQ 16
I/O ports at dcc0 [disabled] [size=32]
Memory at f79fe000 (32-bit, non-prefetchable) [disabled] [size=4K]
I/O ports at dce0 [disabled] [size=32]
Memory at f79ff000 (32-bit, non-prefetchable) [disabled] [size=4K]
Capabilities: [40] Power Management version 1
Kernel driver in use: pciback
and lspci -vt:
-[0000:00]-+-00.0 Intel Corporation 4 Series Chipset DRAM Controller
+-01.0-[01-02]----00.0-[02]--+-00.0 Perle Systems Ltd Device 0331
| \-00.1 Perle Systems Ltd Device 0331
+-02.0 Intel Corporation 4 Series Chipset Integrated Graphics Controller
+-02.1 Intel Corporation 4 Series Chipset Integrated Graphics Controller
+-03.0 Intel Corporation 4 Series Chipset HECI Controller
+-03.2 Intel Corporation 4 Series Chipset PT IDER Controller
+-03.3 Intel Corporation 4 Series Chipset Serial KT Controller
+-19.0 Intel Corporation 82567LM-3 Gigabit Network Connection
+-1a.0 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #4
+-1a.1 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #5
+-1a.2 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #6
+-1a.7 Intel Corporation 82801JD/DO (ICH10 Family) USB2 EHCI Controller #2
+-1b.0 Intel Corporation 82801JD/DO (ICH10 Family) HD Audio Controller
+-1c.0-[03]--
+-1c.1-[04]--
+-1d.0 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #1
+-1d.1 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #2
+-1d.2 Intel Corporation 82801JD/DO (ICH10 Family) USB UHCI Controller #3
+-1d.7 Intel Corporation 82801JD/DO (ICH10 Family) USB2 EHCI Controller #1
+-1e.0-[05]--
+-1f.0 Intel Corporation 82801JDO (ICH10DO) LPC Interface Controller
+-1f.2 Intel Corporation 82801 SATA RAID Controller
-Nick
-------- This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* PCI Passthrough Problems/Questions
@ 2010-10-25 16:35 Nick Couchman
2010-10-25 17:40 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-25 16:35 UTC (permalink / raw)
To: xen-devel
This is a message I posted to xen-users without getting any good leads...I'm hoping someone on the devel list has some ideas as to what might be going on here.
I've asked a couple of these questions, already, but haven't really had
any response, so I thought I'd try, again, and try to provide a little
more detail. I need to pass through a PCIe 4-port Serial card. I've
done this successfully multiple times with different serial cards, but
the current one, a PCIe Perle quad-port RJ45, is causing some problems.
I believe the problem stems from the fact that the card is a PCI card
and that Perle just built a PCIe-PCI bridge so that they could reuse an
existing PCI design. The following three PCI devices are all on the
same card:
# lspci
...
01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
Express-to-PCI Bridge (rev aa)
02:00.0 Serial controller: Perle Systems Ltd Device 0331
02:00.1 Bridge: Perle Systems Ltd Device 0331
I've tried various combinations of hiding 01:00.0, 02:00.0, and/or
02:00.1 from dom0 and passing those through. First, if I try all three,
I put the following in my dom0 kernel arguments:
pciback.hide=(01:00.0)(02:00.0)(02:00.1)
Before someone suggests that maybe I have my kernel arguments wrong,
this is a SuSE kernel, so it is not pvops, and that is the correct
syntax - I can add other PCI(e) devices on the same exact host (like a
graphics card) and pass those through without any problem at all. So
syntax is correct. After I boot with these arguments, I check the dmesg
output for pciback messages:
# dmesg|grep pciback
[ 0.000000] Command line: root=/dev/local/dom0 resume=/dev/local/swap
pciback.hide=(01:00.0)(02:00.0)(02:00.1) splash=silent quiet showopts
[ 0.000000] Kernel command line: root=/dev/local/dom0
resume=/dev/local/swap pciback.hide=(01:00.0)(02:00.0)(02:00.1)
splash=silent quiet showopts
[ 3.646529] pciback 0000:01:00.0: seizing device
[ 3.646536] pciback 0000:02:00.0: seizing device
[ 3.646540] pciback 0000:02:00.1: seizing device
[ 3.646594] pciback 0000:02:00.1: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
[ 3.646599] pciback 0000:02:00.1: PCI INT A disabled
[ 3.646637] pciback 0000:02:00.0: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
[ 3.646642] pciback 0000:02:00.0: PCI INT A disabled
So, everything points to the devices having been removed from dom0's
visibility and placed under the control of pciback. A check with "xm"
confirms this:
# xm pci-list-assignable-devices
0000:02:00.0
0000:02:00.1
0000:01:00.0
Next, I set up the pci pass through in my HVM domU configuration. By
the way, VTd is enabled and working fine - xm dmesg shows it as enabled,
and, as already mentioned, I can forward through other PCI(e) devices
without problem. I set the following in my domU configuration:
pci=['01:00.0','02:00.0','02:00.1']
and then I try to start the domU:
# xm start XP
Error: pci: PCI Backend and pci-stub don't own device 0000:01:00.0
Usage: xm start <DomainName>
??? Ummm, actually, the device is owned by pciback, so why is Xen
telling me it's not?? I can confirm via lspci -v:
# lspci -v
...
01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at d0000000 (64-bit, prefetchable) [size=64K]
Bus: primary=01, secondary=02, subordinate=02, sec-latency=64
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: fe500000-fe5fffff
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express PCI/PCI-X Bridge, MSI 00
Capabilities: [100] Power Budgeting <?>
Kernel driver in use: pciback
Any hints on this particular part of the problem would be appreciated.
My next step was to leave this one out and just try forwarding through
02:00.0 and 02:00.1. However, when I modify the domU configuration and
remove the 01:00.0 device and try to start the domU:
# xm start XP
Error: Failed to assign device to IOMMU
(0000:02:00.0@100,msitranslate=1,power_mgmt=0)
Usage: xm start <DomainName>
I've been able to find very, very little useful information on this
particular error - most of the issues are with trying to forward through
one or two of the devices on a card to one domU and the others to
another or not at all - splitting the devices on a card between domains.
I'm not really trying to do this, but I suspect that there's some sort
of tie between the PCIe-PCI bridge device (01:00.0) and the actual
serial card (02:00.0) that is causing Xen to need to have both those
devices present in the same domain. Of course, because of the problems
I'm having forwarding the bridge through, that's not really working for
me.
Environment:
Xen: 4.0.0 (4.0.0_21091_06-0.1)
Kernel: 2.6.34.7 (2.6.34.7-0.4-xen, not pvops)
Hardware: Dell Optiplex 780
Any help, hints, information, would be greatly appreciated!
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-25 16:35 Nick Couchman
@ 2010-10-25 17:40 ` Konrad Rzeszutek Wilk
2010-10-25 18:33 ` Nick Couchman
0 siblings, 1 reply; 19+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-10-25 17:40 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel
On Mon, Oct 25, 2010 at 10:35:08AM -0600, Nick Couchman wrote:
> This is a message I posted to xen-users without getting any good leads...I'm hoping someone on the devel list has some ideas as to what might be going on here.
>
> I've asked a couple of these questions, already, but haven't really had
> any response, so I thought I'd try, again, and try to provide a little
> more detail. I need to pass through a PCIe 4-port Serial card. I've
> done this successfully multiple times with different serial cards, but
> the current one, a PCIe Perle quad-port RJ45, is causing some problems.
> I believe the problem stems from the fact that the card is a PCI card
> and that Perle just built a PCIe-PCI bridge so that they could reuse an
> existing PCI design. The following three PCI devices are all on the
> same card:
>
> # lspci
> ...
> 01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> Express-to-PCI Bridge (rev aa)
> 02:00.0 Serial controller: Perle Systems Ltd Device 0331
> 02:00.1 Bridge: Perle Systems Ltd Device 0331
>
> I've tried various combinations of hiding 01:00.0, 02:00.0, and/or
> 02:00.1 from dom0 and passing those through. First, if I try all three,
> I put the following in my dom0 kernel arguments:
>
> pciback.hide=(01:00.0)(02:00.0)(02:00.1)
>
> Before someone suggests that maybe I have my kernel arguments wrong,
> this is a SuSE kernel, so it is not pvops, and that is the correct
> syntax - I can add other PCI(e) devices on the same exact host (like a
> graphics card) and pass those through without any problem at all. So
> syntax is correct. After I boot with these arguments, I check the dmesg
> output for pciback messages:
>
> # dmesg|grep pciback
> [ 0.000000] Command line: root=/dev/local/dom0 resume=/dev/local/swap
> pciback.hide=(01:00.0)(02:00.0)(02:00.1) splash=silent quiet showopts
> [ 0.000000] Kernel command line: root=/dev/local/dom0
> resume=/dev/local/swap pciback.hide=(01:00.0)(02:00.0)(02:00.1)
> splash=silent quiet showopts
> [ 3.646529] pciback 0000:01:00.0: seizing device
> [ 3.646536] pciback 0000:02:00.0: seizing device
> [ 3.646540] pciback 0000:02:00.1: seizing device
> [ 3.646594] pciback 0000:02:00.1: PCI INT A -> GSI 16 (level, low) ->
> IRQ 16
> [ 3.646599] pciback 0000:02:00.1: PCI INT A disabled
> [ 3.646637] pciback 0000:02:00.0: PCI INT A -> GSI 16 (level, low) ->
> IRQ 16
> [ 3.646642] pciback 0000:02:00.0: PCI INT A disabled
>
> So, everything points to the devices having been removed from dom0's
> visibility and placed under the control of pciback. A check with "xm"
> confirms this:
>
> # xm pci-list-assignable-devices
> 0000:02:00.0
> 0000:02:00.1
> 0000:01:00.0
>
> Next, I set up the pci pass through in my HVM domU configuration. By
> the way, VTd is enabled and working fine - xm dmesg shows it as enabled,
> and, as already mentioned, I can forward through other PCI(e) devices
> without problem. I set the following in my domU configuration:
>
> pci=['01:00.0','02:00.0','02:00.1']
>
> and then I try to start the domU:
>
> # xm start XP
> Error: pci: PCI Backend and pci-stub don't own device 0000:01:00.0
> Usage: xm start <DomainName>
>
> ??? Ummm, actually, the device is owned by pciback, so why is Xen
> telling me it's not?? I can confirm via lspci -v:
>
> # lspci -v
> ...
> 01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> Flags: bus master, fast devsel, latency 0
> Memory at d0000000 (64-bit, prefetchable) [size=64K]
> Bus: primary=01, secondary=02, subordinate=02, sec-latency=64
> I/O behind bridge: 0000d000-0000dfff
> Memory behind bridge: fe500000-fe5fffff
> Capabilities: [40] Power Management version 2
> Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> Capabilities: [60] Express PCI/PCI-X Bridge, MSI 00
> Capabilities: [100] Power Budgeting <?>
> Kernel driver in use: pciback
>
> Any hints on this particular part of the problem would be appreciated.
> My next step was to leave this one out and just try forwarding through
> 02:00.0 and 02:00.1. However, when I modify the domU configuration and
> remove the 01:00.0 device and try to start the domU:
>
> # xm start XP
> Error: Failed to assign device to IOMMU
> (0000:02:00.0@100,msitranslate=1,power_mgmt=0)
> Usage: xm start <DomainName>
>
> I've been able to find very, very little useful information on this
> particular error - most of the issues are with trying to forward through
> one or two of the devices on a card to one domU and the others to
> another or not at all - splitting the devices on a card between domains.
> I'm not really trying to do this, but I suspect that there's some sort
> of tie between the PCIe-PCI bridge device (01:00.0) and the actual
> serial card (02:00.0) that is causing Xen to need to have both those
> devices present in the same domain. Of course, because of the problems
> I'm having forwarding the bridge through, that's not really working for
> me.
What do you see on your Xen serial output? I presume you cranked up logging:
loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
Is there anything that shows up when you get the 'Failed to assign.." ?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-25 17:40 ` Konrad Rzeszutek Wilk
@ 2010-10-25 18:33 ` Nick Couchman
2010-10-25 18:48 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-25 18:33 UTC (permalink / raw)
To: konrad.wilk; +Cc: xen-devel
On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
>
> What do you see on your Xen serial output? I presume you cranked up logging:
> loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
>
> Is there anything that shows up when you get the 'Failed to assign.." ?
>
The only messages I get on the serial console after setting those
parameters on the xen.gz kernel line in grub (and rebooting, of course)
are the following:
(XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
(XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
failed
(XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-25 18:33 ` Nick Couchman
@ 2010-10-25 18:48 ` Konrad Rzeszutek Wilk
2010-10-25 18:50 ` Nick Couchman
0 siblings, 1 reply; 19+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-10-25 18:48 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel
On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
> On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
>
> >
> > What do you see on your Xen serial output? I presume you cranked up logging:
> > loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
> >
> > Is there anything that shows up when you get the 'Failed to assign.." ?
> >
>
> The only messages I get on the serial console after setting those
> parameters on the xen.gz kernel line in grub (and rebooting, of course)
> are the following:
>
> (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
> (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
> (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
> failed
> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
So, -EINVAL. How comfortable are you sticking a bunch of
dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
you need to figure which of the functions that are past line 1364
are being called and return -EINVAL.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-25 18:48 ` Konrad Rzeszutek Wilk
@ 2010-10-25 18:50 ` Nick Couchman
2010-10-25 19:07 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-25 18:50 UTC (permalink / raw)
To: konrad.wilk; +Cc: xen-devel
On Mon, 2010-10-25 at 14:48 -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
> > On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
> >
> > >
> > > What do you see on your Xen serial output? I presume you cranked up logging:
> > > loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
> > >
> > > Is there anything that shows up when you get the 'Failed to assign.." ?
> > >
> >
> > The only messages I get on the serial console after setting those
> > parameters on the xen.gz kernel line in grub (and rebooting, of course)
> > are the following:
> >
> > (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
> > (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
> > (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
> > failed
> > (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
>
> So, -EINVAL. How comfortable are you sticking a bunch of
> dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
> you need to figure which of the functions that are past line 1364
> are being called and return -EINVAL.
I'm happy to give it a shot...it'll take a while to get the devel
environment configured, as I'm using packages right now and I don't even
think I have a compiler on this system. I'll report back once I get
that done and give that a try.
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-25 18:50 ` Nick Couchman
@ 2010-10-25 19:07 ` Konrad Rzeszutek Wilk
2010-10-25 19:45 ` Nick Couchman
` (2 more replies)
0 siblings, 3 replies; 19+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-10-25 19:07 UTC (permalink / raw)
To: Nick Couchman, weidong.han; +Cc: xen-devel
On Mon, Oct 25, 2010 at 12:50:41PM -0600, Nick Couchman wrote:
> On Mon, 2010-10-25 at 14:48 -0400, Konrad Rzeszutek Wilk wrote:
> > On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
> > > On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
> > >
> > > >
> > > > What do you see on your Xen serial output? I presume you cranked up logging:
> > > > loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
> > > >
> > > > Is there anything that shows up when you get the 'Failed to assign.." ?
> > > >
> > >
> > > The only messages I get on the serial console after setting those
> > > parameters on the xen.gz kernel line in grub (and rebooting, of course)
> > > are the following:
> > >
> > > (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
> > > (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
> > > (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
> > > failed
> > > (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
> >
> > So, -EINVAL. How comfortable are you sticking a bunch of
> > dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
> > you need to figure which of the functions that are past line 1364
> > are being called and return -EINVAL.
>
> I'm happy to give it a shot...it'll take a while to get the devel
> environment configured, as I'm using packages right now and I don't even
> think I have a compiler on this system. I'll report back once I get
> that done and give that a try.
Excellent. You might also want to CC Weidong (weidong.han@intel.com) in the future
who is right now on travel and he might have better suggestions. CC-ing him on this e-mail.
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-10-25 19:07 ` Konrad Rzeszutek Wilk
@ 2010-10-25 19:45 ` Nick Couchman
2010-10-25 20:11 ` Nick Couchman
2010-10-27 6:40 ` Weidong Han
2 siblings, 0 replies; 19+ messages in thread
From: Nick Couchman @ 2010-10-25 19:45 UTC (permalink / raw)
To: konrad.wilk; +Cc: xen-devel, weidong.han
On Mon, 2010-10-25 at 15:07 -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 25, 2010 at 12:50:41PM -0600, Nick Couchman wrote:
> > On Mon, 2010-10-25 at 14:48 -0400, Konrad Rzeszutek Wilk wrote:
> > > On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
> > > > On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
> > > >
> > > > >
> > > > > What do you see on your Xen serial output? I presume you cranked up logging:
> > > > > loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
> > > > >
> > > > > Is there anything that shows up when you get the 'Failed to assign.." ?
> > > > >
> > > >
> > > > The only messages I get on the serial console after setting those
> > > > parameters on the xen.gz kernel line in grub (and rebooting, of course)
> > > > are the following:
> > > >
> > > > (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
> > > > (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
> > > > (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
> > > > failed
> > > > (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
> > >
> > > So, -EINVAL. How comfortable are you sticking a bunch of
> > > dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
> > > you need to figure which of the functions that are past line 1364
> > > are being called and return -EINVAL.
> >
> > I'm happy to give it a shot...it'll take a while to get the devel
> > environment configured, as I'm using packages right now and I don't even
> > think I have a compiler on this system. I'll report back once I get
> > that done and give that a try.
>
> Excellent. You might also want to CC Weidong (weidong.han@intel.com) in the future
> who is right now on travel and he might have better suggestions. CC-ing him on this e-mail.
Okay, I've traced it down to this section of code in iommu.c:
/*
* Devices behind PCIe-to-PCI/PCIx bridge may generate
* different requester-id. It may originate from devfn=0
* on the secondary bus behind the bridge. Map that id
* as well.
*/
ret = domain_context_mapping_one(domain, drhd->iommu,
secbus, 0);
dprintk(XENLOG_ERR VTDPREFIX, "domain_conext_mapping_one
ret: %d\n", ret);
I guess that shouldn't surprise me, given this device is behind a
PCIe-to-PCI bridge. On to the domain_context_mapping_one function.
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-10-25 19:07 ` Konrad Rzeszutek Wilk
2010-10-25 19:45 ` Nick Couchman
@ 2010-10-25 20:11 ` Nick Couchman
2010-10-27 6:40 ` Weidong Han
2 siblings, 0 replies; 19+ messages in thread
From: Nick Couchman @ 2010-10-25 20:11 UTC (permalink / raw)
To: konrad.wilk; +Cc: xen-devel, weidong.han
On Mon, 2010-10-25 at 15:07 -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 25, 2010 at 12:50:41PM -0600, Nick Couchman wrote:
> > On Mon, 2010-10-25 at 14:48 -0400, Konrad Rzeszutek Wilk wrote:
> > > On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
> > > > On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
> > > >
> > > > >
> > > > > What do you see on your Xen serial output? I presume you cranked up logging:
> > > > > loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
> > > > >
> > > > > Is there anything that shows up when you get the 'Failed to assign.." ?
> > > > >
> > > >
> > > > The only messages I get on the serial console after setting those
> > > > parameters on the xen.gz kernel line in grub (and rebooting, of course)
> > > > are the following:
> > > >
> > > > (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
> > > > (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
> > > > (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
> > > > failed
> > > > (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
> > >
> > > So, -EINVAL. How comfortable are you sticking a bunch of
> > > dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
> > > you need to figure which of the functions that are past line 1364
> > > are being called and return -EINVAL.
> >
> > I'm happy to give it a shot...it'll take a while to get the devel
> > environment configured, as I'm using packages right now and I don't even
> > think I have a compiler on this system. I'll report back once I get
> > that done and give that a try.
>
> Excellent. You might also want to CC Weidong (weidong.han@intel.com) in the future
> who is right now on travel and he might have better suggestions. CC-ing him on this e-mail.
Okay, I'm fairly certain that the root cause is actually the issue Xen
has trying to forward through the actual PCIe-to-PCI bridge. In my
original e-mail, I stated that if I try to add that bridge to the domU,
I get an error about it not being owned by PCI Backend or pci-stub. In
the function domain_context_mapping, bus is the bus of the device you're
actually trying to forward through to the new domain, and secbus is set
with the function find_upstream_bridge, which appears to find the bus to
which your device is attached. Here's the section of code in question
from iommu.c:
if ( pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE )
{
ret = domain_context_mapping_one(domain, drhd->iommu, bus,
devfn);
dprintk(XENLOG_ERR VTDPREFIX, "domain_conext_mapping_one
ret: %d\n", ret);
if ( ret )
return ret;
/*
* Devices behind PCIe-to-PCI/PCIx bridge may generate
* different requester-id. It may originate from devfn=0
* on the secondary bus behind the bridge. Map that id
* as well.
*/
ret = domain_context_mapping_one(domain, drhd->iommu,
secbus, 0);
dprintk(XENLOG_ERR VTDPREFIX, "domain_conext_mapping_one
ret: %d\n", ret);
}
else /* Legacy PCI bridge */ {
ret = domain_context_mapping_one(domain, drhd->iommu, bus,
devfn);
dprintk(XENLOG_ERR VTDPREFIX, "domain_conext_mapping_one
ret: %d\n", ret);
}
If I understand this code, if the device is a PCIe2PCI bridge, the first
thing that's done is that the device itself is mapped, then the upstream
bridge device is mapped (secbus with devfn=0). This is where the
command is failing for the device that I'm trying to forward. So, the
question is, why can't Xen seem to forward through the bridge itself?
It doesn't seem to matter whether I have that particular bridge listed
on the kernel line in the pciback.hide section - Xen still thinks that
dom0 has a hold of that device and won't forward it on.
Where do I go from here?
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-10-25 19:07 ` Konrad Rzeszutek Wilk
2010-10-25 19:45 ` Nick Couchman
2010-10-25 20:11 ` Nick Couchman
@ 2010-10-27 6:40 ` Weidong Han
2010-10-27 15:07 ` Nick Couchman
2 siblings, 1 reply; 19+ messages in thread
From: Weidong Han @ 2010-10-27 6:40 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel@lists.xensource.com, Nick Couchman
Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 25, 2010 at 12:50:41PM -0600, Nick Couchman wrote:
>
>> On Mon, 2010-10-25 at 14:48 -0400, Konrad Rzeszutek Wilk wrote:
>>
>>> On Mon, Oct 25, 2010 at 12:33:09PM -0600, Nick Couchman wrote:
>>>
>>>> On Mon, 2010-10-25 at 13:40 -0400, Konrad Rzeszutek Wilk wrote:
>>>>
>>>>
>>>>> What do you see on your Xen serial output? I presume you cranked up logging:
>>>>> loglevel=all guest_lvl=all iommu=verbose on your Xen command line.
>>>>>
>>>>> Is there anything that shows up when you get the 'Failed to assign.." ?
>>>>>
>>>>>
>>>> The only messages I get on the serial console after setting those
>>>> parameters on the xen.gz kernel line in grub (and rebooting, of course)
>>>> are the following:
>>>>
>>>> (XEN) [VT-D]iommu.c:1496: d0:PCI: unmap bdf = 2:0.0
>>>> (XEN) [VT-D]iommu.c:1364: d1:PCI: map bdf = 2:0.0
>>>> (XEN) domctl.c:848:d0 XEN_DOMCTL_assign_device: assign device (2:0.0)
>>>> failed
>>>> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
>>>>
Nick,
I think the issue is 02:00.0 was mapped twice. Could you try with below
patch? Then post the xen log. Pls post all output of 'lspci -v' on your
system.
diff -r eff592364826 xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Wed Sep 01 11:23:49 2010 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Wed Oct 27 14:33:23 2010 +0800
@@ -1225,7 +1225,11 @@ static int domain_context_mapping_one(
if (!pdev)
res = -ENODEV;
else if (pdev->domain != domain)
- res = -EINVAL;
+ {
+ dprintk(VTDPREFIX, "context_present: %x:%x.%x:
pdev->domain=%d domain=%d\n",
+ bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
pdev->domain, domain->domain_id);
+ // res = -EINVAL;
+ }
unmap_vtd_domain_page(context_entries);
spin_unlock(&iommu->lock);
return res;
@@ -1352,6 +1356,8 @@ static int domain_context_mapping(struct
/* PCIe to PCI/PCIx bridge */
if ( pdev_type(bus, devfn) == DEV_TYPE_PCIe2PCI_BRIDGE )
{
+ dprintk(VTDPREFIX, "d%d:PCI: map PCIe2PCI bdf = %x:%x.%x\n",
+ domain->domain_id, bus, PCI_SLOT(devfn),
PCI_FUNC(devfn));
ret = domain_context_mapping_one(domain, drhd->iommu, bus,
devfn);
if ( ret )
return ret;
@@ -1362,6 +1368,8 @@ static int domain_context_mapping(struct
* on the secondary bus behind the bridge. Map that id
* as well.
*/
+ dprintk(VTDPREFIX, "d%d:PCI: map secbus (%d) with devfn 0\n",
+ domain->domain_id, secbus);
ret = domain_context_mapping_one(domain, drhd->iommu,
secbus, 0);
}
else /* Legacy PCI bridge */
>>> So, -EINVAL. How comfortable are you sticking a bunch of
>>> dprintk(VTDPREFIX, " in the drivers/passthrough/vtd/iommu.c file? Basically
>>> you need to figure which of the functions that are past line 1364
>>> are being called and return -EINVAL.
>>>
>> I'm happy to give it a shot...it'll take a while to get the devel
>> environment configured, as I'm using packages right now and I don't even
>> think I have a compiler on this system. I'll report back once I get
>> that done and give that a try.
>>
>
> Excellent. You might also want to CC Weidong (weidong.han@intel.com) in the future
> who is right now on travel and he might have better suggestions. CC-ing him on this e-mail.
>
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: PCI Passthrough Problems/Questions
2010-10-27 6:40 ` Weidong Han
@ 2010-10-27 15:07 ` Nick Couchman
2010-10-28 2:31 ` Weidong Han
0 siblings, 1 reply; 19+ messages in thread
From: Nick Couchman @ 2010-10-27 15:07 UTC (permalink / raw)
To: weidong.han; +Cc: xen-devel, konrad.wilk
> Nick,
>
> I think the issue is 02:00.0 was mapped twice. Could you try with below
> patch? Then post the xen log. Pls post all output of 'lspci -v' on your
> system.
>
I applied the patch, with one minor change. The line:
dprintk(XENLOG_ERR VTDPREFIX, "context_present: %x:%x.%x:pdev->domain=%d
domain=%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn), pdev->domain,
domain->domain_id);
should be:
dprintk(XENLOG_ERR VTDPREFIX, "context_present: %x:%x.%x:pdev->domain=%d
domain=%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
pdev->domain->domain_id, domain->domain_id);
(notice the pdev->domain->domain_id instead of pdev->domain).
The domU no longer generates the error about failing to assign device to
IOMMU, but now it just silently crashes. xm dmesg output:
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d1:PCI:
map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=1
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=36 device=5 intx=0
(XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d1:PCI:
map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=40 device=6 intx=0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.1
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI:
map bdf = 2:0.1
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1415: Return value: 0
(XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.0
(XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI:
map bdf = 2:0.0
(XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
(XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
(XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
(XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
(XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=1 domain=0
(XEN) [VT-D]iommu.c:1415: Return value: 0
And from xend.log I've pasted into this pastebin:
http://pastebin.com/b4bwdBPq
I have some extra dprintk calls that I threw in there, so there may be a
little more output than with the patch you sent.
-Nick
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: PCI Passthrough Problems/Questions
2010-10-27 15:07 ` Nick Couchman
@ 2010-10-28 2:31 ` Weidong Han
0 siblings, 0 replies; 19+ messages in thread
From: Weidong Han @ 2010-10-28 2:31 UTC (permalink / raw)
To: Nick Couchman; +Cc: xen-devel@lists.xensource.com, konrad.wilk@Oracle.Com
Nick Couchman wrote:
>> Nick,
>>
>> I think the issue is 02:00.0 was mapped twice. Could you try with below
>> patch? Then post the xen log. Pls post all output of 'lspci -v' on your
>> system.
>>
>>
>
> I applied the patch, with one minor change. The line:
>
> dprintk(XENLOG_ERR VTDPREFIX, "context_present: %x:%x.%x:pdev->domain=%d
> domain=%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn), pdev->domain,
> domain->domain_id);
>
> should be:
>
> dprintk(XENLOG_ERR VTDPREFIX, "context_present: %x:%x.%x:pdev->domain=%d
> domain=%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> pdev->domain->domain_id, domain->domain_id);
>
> (notice the pdev->domain->domain_id instead of pdev->domain).
>
> The domU no longer generates the error about failing to assign device to
> IOMMU, but now it just silently crashes. xm dmesg output:
>
> (XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.0
> (XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d1:PCI:
> map bdf = 2:0.0
> (XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
> (XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
> (XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
> (XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
> (XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=0 domain=1
> (XEN) [VT-D]iommu.c:1415: Return value: 0
> (XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=36 device=5 intx=0
> (XEN) [VT-D]iommu.c:1511: d0:PCI: unmap bdf = 2:0.1
> (XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d1:PCI:
> map bdf = 2:0.1
> (XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
> (XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
> (XEN) [VT-D]iommu.c:1383: d1:PCI: map PCIe2PCI bdf = 1:0.0
> (XEN) [VT-D]iommu.c:1394: d1:PCI: map secbus (2) with devfn 0
> (XEN) [VT-D]iommu.c:1415: Return value: 0
> (XEN) [VT-D]io.c:300: d1: bind: m_gsi=16 g_gsi=40 device=6 intx=0
> (XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.1
> (XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 1[VT-D]iommu.c:1368: d0:PCI:
> map bdf = 2:0.1
> (XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
> (XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
> (XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
> (XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
> (XEN) [VT-D]iommu.c:1415: Return value: 0
> (XEN) [VT-D]iommu.c:1511: d1:PCI: unmap bdf = 2:0.0
> (XEN) [VT-D]iommu.c:1340: bus: 2, devfn: 0[VT-D]iommu.c:1368: d0:PCI:
> map bdf = 2:0.0
> (XEN) [VT-D]iommu.c:1371: domain_conext_mapping_one ret: 0
> (XEN) [VT-D]iommu.c:1378: Upstream bridge for 1:0 is 2.
> (XEN) [VT-D]iommu.c:1383: d0:PCI: map PCIe2PCI bdf = 1:0.0
> (XEN) [VT-D]iommu.c:1394: d0:PCI: map secbus (2) with devfn 0
> (XEN) [VT-D]iommu.c:1249: context_present: 2:0.0:pdev->domain=1 domain=0
> (XEN) [VT-D]iommu.c:1415: Return value: 0
>
> And from xend.log I've pasted into this pastebin:
>
> http://pastebin.com/b4bwdBPq
>
> I have some extra dprintk calls that I threw in there, so there may be a
> little more output than with the patch you sent.
>
>
>
the log of xm dmesg looks no problem. you'd better post complete log and
lspci -v.
Regards,
Weidong
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2010-11-03 14:54 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-10-28 14:01 PCI Passthrough Problems/Questions Nick Couchman
2010-10-29 2:54 ` Weidong Han
2010-10-29 18:52 ` Nick Couchman
2010-11-02 3:30 ` Weidong Han
2010-11-02 19:07 ` Nick Couchman
2010-11-03 1:01 ` Weidong Han
2010-11-03 14:54 ` Nick Couchman
[not found] <4CC8A62602000099000BAA42@collaborate.seakr.com>
2010-10-28 4:37 ` Han, Weidong
-- strict thread matches above, loose matches on Subject: below --
2010-10-25 16:35 Nick Couchman
2010-10-25 17:40 ` Konrad Rzeszutek Wilk
2010-10-25 18:33 ` Nick Couchman
2010-10-25 18:48 ` Konrad Rzeszutek Wilk
2010-10-25 18:50 ` Nick Couchman
2010-10-25 19:07 ` Konrad Rzeszutek Wilk
2010-10-25 19:45 ` Nick Couchman
2010-10-25 20:11 ` Nick Couchman
2010-10-27 6:40 ` Weidong Han
2010-10-27 15:07 ` Nick Couchman
2010-10-28 2:31 ` Weidong Han
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.