From mboxrd@z Thu Jan 1 00:00:00 1970 From: Weidong Han Subject: Re: RE: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb controllor to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from 00000a07:00000000 to 00000000:00000000. Date: Mon, 22 Mar 2010 17:32:22 +0800 Message-ID: <4BA73926.40503@intel.com> References: <1017192770.20100321221949@eikelenboom.it> <1A42CE6F5F474C41B63392A5F80372B21D641D6D@shsmsx501.ccr.corp.intel.com> <20100322090450.GA1878@reaktio.net> <4BA734BD.5030305@intel.com> <20100322091917.GB1878@reaktio.net> <4BA737A3.4090708@intel.com> <20100322092952.GD1878@reaktio.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20100322092952.GD1878@reaktio.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: =?ISO-8859-1?Q?Pasi_K=E4rkk=E4inen?= Cc: Sander Eikelenboom , "xen-devel@lists.xensource.com" , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org Pasi K=E4rkk=E4inen wrote: > On Mon, Mar 22, 2010 at 05:25:55PM +0800, Weidong Han wrote: > =20 >> Pasi K=E4rkk=E4inen wrote: >> =20 >>> On Mon, Mar 22, 2010 at 05:13:33PM +0800, Weidong Han wrote: >>> =20 >>> =20 >>>> Pasi K=E4rkk=E4inen wrote: >>>> =20 >>>> =20 >>>>> On Mon, Mar 22, 2010 at 04:04:09PM +0800, Han, Weidong wrote: >>>>> =20 >>>>> =20 >>>>>> The faults were caused by that the DMA address was not mapped in V= T-d page table. >>>>>> >>>>>> Could you have following two tries: >>>>>> 1) assign it to pv domU without VT-d >>>>>> =20 >>>>>> =20 >>>>> Btw are there other methods of disabling VT-d PCI passhtru than=20 >>>>> having iommu=3Doff for xen.gz in grub.conf? ie. can you somehow=20 >>>>> select "use normal PV passthru for this guest" and "but still use=20 >>>>> VT-d passthru for this guest" ? >>>>> >>>>> =20 >>>>> =20 >>>> I'm not quite understand your point. do you mean use normal PV=20 >>>> passthru for pv guest, but still can passthru device to hvm guest?=20 >>>> if so, current xen VT-d already does like this. >>>> >>>> =20 >>>> =20 >>> Yeah, that's what I meant. Ok.=20 >>> >>> I thought it was also possible to use VT-d for a PV guest? no? =20 >>> =20 >> yes. you can use VT-d to assign device to pv guest. Sander was doing =20 >> that and encountered below issue. >> >> =20 > > Ok, so iommu=3Dpv is the flag that enabled VT-d passthru for PV guests.= =20 > As a default Xen uses normal PCI passhthru for PV guests. > > Thanks. > > -- Pasi > > =20 exactly. Regards, Weidong >> Regards, >> Weidong >> =20 >>> -- Pasi >>> >>> =20 >>> =20 >>>> Regards, >>>> Weidong >>>> =20 >>>> =20 >>>>> ps. I'm just writing a xen wiki page for PCI passthru and adding th= ese things and the common failure scenarios there. >>>>> >>>>> -- Pasi >>>>> >>>>> =20 >>>>> =20 >>>>>> 2) assign it to a hvm guest >>>>>> >>>>>> Regards, >>>>>> Weidong >>>>>> >>>>>> -----Original Message----- >>>>>> From: Sander Eikelenboom [mailto:linux@eikelenboom.it] Sent:=20 >>>>>> Monday, March 22, 2010 5:20 AM >>>>>> To: Konrad Rzeszutek Wilk >>>>>> Cc: Han, Weidong; xen-devel@lists.xensource.com >>>>>> Subject: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb= controllor to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 0000= 00000000008b from 00000a07:00000000 to 00000000:00000000. >>>>>> >>>>>> Hi Han/Konrad, >>>>>> >>>>>> In my setup i'm trying to passthrough an USB 3.0 pci-e controller = to a PV domU. >>>>>> - xen: 4.0.0-rc6 >>>>>> - dom0: kernel xen/next >>>>>> - domU: kernel 2.6.33 from git://git.kernel.org/pub/scm/linux/kern= el/git/konrad/xen.git >>>>>> ( to have pci-front together with most recent usb3.0 xhci = drivers. >>>>>> >>>>>> >>>>>> - USB 3.0 xhci drivers work fine on the baremetal with the 2.6.33 = kernel. >>>>>> >>>>>> This is on a intel Q45 chipset with IOMMU. >>>>>> >>>>>> This is my boot config: >>>>>> title xen-4.0.0-rc6.gz / Debian GNU/Linux, kernel 2.6.3= 2 >>>>>> root (hd0,0) >>>>>> kernel /boot/xen-4.0.0-rc6.gz dom0_mem=3D768M loglvl=3Dal= l loglvl_guest=3Dall iommu=3Dpv iommu_inclusive_mapping=3D1 >>>>>> module /boot/vmlinuz-2.6.32 root=3D/dev/sda1 ro earlyprin= tk=3Dxen max_loop=3D255 xen-pciback.hide=3D(03:00.0) >>>>>> module /boot/initrd.img-2.6.32 >>>>>> >>>>>> When booting the domU xm dmesg gets filled with the following when= the usb controller tries to initialize/: >>>>>> >>>>>> (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from= 00000a07:00000000 to 00000000:00000000. >>>>>> (XEN) [VT-D]iommu.c:821: iommu_fault_status: Primary Pending Fault >>>>>> (XEN) [VT-D]iommu.c:796: DMAR:[DMA Read] Request device [03:00.0] = fault addr 1ff94000, iommu reg =3D ffff82c3fff54000 >>>>>> (XEN) DMAR:[fault reason 06h] PTE Read access is not set >>>>>> (XEN) print_vtd_entries: iommu =3D ffff83007c866970 bdf =3D 3:0.0 = gmfn =3D 1ff94 >>>>>> (XEN) root_entry =3D ffff83007c872000 >>>>>> (XEN) root_entry[3] =3D 78f56001 >>>>>> (XEN) context =3D ffff830078f56000 >>>>>> (XEN) context[0] =3D 101_2f0e1001 >>>>>> (XEN) l3 =3D ffff83002f0e1000 >>>>>> (XEN) l3_index =3D 0 >>>>>> (XEN) l3[0] =3D 2f0e0003 >>>>>> (XEN) l2 =3D ffff83002f0e0000 >>>>>> (XEN) l2_index =3D ff >>>>>> (XEN) l2[ff] =3D 0 >>>>>> (XEN) l2[ff] not present >>>>>> >>>>>> >>>>>> >>>>>> Anyone any tips on what i could try ?, is this something caused by= xen, or something by the usb driver not adhering to kernel DMA-api ? >>>>>> >>>>>> Attached: >>>>>> >>>>>> - xm-info.txt >>>>>> - xm-dmesg.txt >>>>>> - xend.log >>>>>> >>>>>> - dom0-dmesg.txt >>>>>> - dom0-lspci-tree.txt >>>>>> - dom0-lspci.txt >>>>>> >>>>>> - domU-lspci.txt >>>>>> - domU-dmesg.txt >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Xen-devel mailing list >>>>>> Xen-devel@lists.xensource.com >>>>>> http://lists.xensource.com/xen-devel >>>>>> =20 >>>>>> =20