From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= Subject: Re: RE: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb controllor to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from 00000a07:00000000 to 00000000:00000000. Date: Mon, 22 Mar 2010 11:29:52 +0200 Message-ID: <20100322092952.GD1878@reaktio.net> References: <1017192770.20100321221949@eikelenboom.it> <1A42CE6F5F474C41B63392A5F80372B21D641D6D@shsmsx501.ccr.corp.intel.com> <20100322090450.GA1878@reaktio.net> <4BA734BD.5030305@intel.com> <20100322091917.GB1878@reaktio.net> <4BA737A3.4090708@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <4BA737A3.4090708@intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Weidong Han Cc: Sander Eikelenboom , "xen-devel@lists.xensource.com" , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org On Mon, Mar 22, 2010 at 05:25:55PM +0800, Weidong Han wrote: > Pasi K=E4rkk=E4inen wrote: >> On Mon, Mar 22, 2010 at 05:13:33PM +0800, Weidong Han wrote: >> =20 >>> Pasi K=E4rkk=E4inen wrote: >>> =20 >>>> On Mon, Mar 22, 2010 at 04:04:09PM +0800, Han, Weidong wrote: >>>> =20 >>>>> The faults were caused by that the DMA address was not mapped in VT= -d page table. >>>>> >>>>> Could you have following two tries: >>>>> 1) assign it to pv domU without VT-d >>>>> =20 >>>> Btw are there other methods of disabling VT-d PCI passhtru than=20 >>>> having iommu=3Doff for xen.gz in grub.conf? ie. can you somehow=20 >>>> select "use normal PV passthru for this guest" and "but still use=20 >>>> VT-d passthru for this guest" ? >>>> >>>> =20 >>> I'm not quite understand your point. do you mean use normal PV=20 >>> passthru for pv guest, but still can passthru device to hvm guest?=20 >>> if so, current xen VT-d already does like this. >>> >>> =20 >> >> Yeah, that's what I meant. Ok.=20 >> >> I thought it was also possible to use VT-d for a PV guest? no? =20 > yes. you can use VT-d to assign device to pv guest. Sander was doing =20 > that and encountered below issue. > Ok, so iommu=3Dpv is the flag that enabled VT-d passthru for PV guests.=20 As a default Xen uses normal PCI passhthru for PV guests. Thanks. -- Pasi > Regards, > Weidong >> -- Pasi >> >> =20 >>> Regards, >>> Weidong >>> =20 >>>> ps. I'm just writing a xen wiki page for PCI passthru and adding the= se things and the common failure scenarios there. >>>> >>>> -- Pasi >>>> >>>> =20 >>>>> 2) assign it to a hvm guest >>>>> >>>>> Regards, >>>>> Weidong >>>>> >>>>> -----Original Message----- >>>>> From: Sander Eikelenboom [mailto:linux@eikelenboom.it] Sent:=20 >>>>> Monday, March 22, 2010 5:20 AM >>>>> To: Konrad Rzeszutek Wilk >>>>> Cc: Han, Weidong; xen-devel@lists.xensource.com >>>>> Subject: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb = controllor to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 00000= 0000000008b from 00000a07:00000000 to 00000000:00000000. >>>>> >>>>> Hi Han/Konrad, >>>>> >>>>> In my setup i'm trying to passthrough an USB 3.0 pci-e controller = to a PV domU. >>>>> - xen: 4.0.0-rc6 >>>>> - dom0: kernel xen/next >>>>> - domU: kernel 2.6.33 from git://git.kernel.org/pub/scm/linux/kerne= l/git/konrad/xen.git >>>>> ( to have pci-front together with most recent usb3.0 xhci d= rivers. >>>>> >>>>> >>>>> - USB 3.0 xhci drivers work fine on the baremetal with the 2.6.33 k= ernel. >>>>> >>>>> This is on a intel Q45 chipset with IOMMU. >>>>> >>>>> This is my boot config: >>>>> title xen-4.0.0-rc6.gz / Debian GNU/Linux, kernel 2.6.32 >>>>> root (hd0,0) >>>>> kernel /boot/xen-4.0.0-rc6.gz dom0_mem=3D768M loglvl=3Dall= loglvl_guest=3Dall iommu=3Dpv iommu_inclusive_mapping=3D1 >>>>> module /boot/vmlinuz-2.6.32 root=3D/dev/sda1 ro earlyprint= k=3Dxen max_loop=3D255 xen-pciback.hide=3D(03:00.0) >>>>> module /boot/initrd.img-2.6.32 >>>>> >>>>> When booting the domU xm dmesg gets filled with the following when = the usb controller tries to initialize/: >>>>> >>>>> (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from = 00000a07:00000000 to 00000000:00000000. >>>>> (XEN) [VT-D]iommu.c:821: iommu_fault_status: Primary Pending Fault >>>>> (XEN) [VT-D]iommu.c:796: DMAR:[DMA Read] Request device [03:00.0] f= ault addr 1ff94000, iommu reg =3D ffff82c3fff54000 >>>>> (XEN) DMAR:[fault reason 06h] PTE Read access is not set >>>>> (XEN) print_vtd_entries: iommu =3D ffff83007c866970 bdf =3D 3:0.0 g= mfn =3D 1ff94 >>>>> (XEN) root_entry =3D ffff83007c872000 >>>>> (XEN) root_entry[3] =3D 78f56001 >>>>> (XEN) context =3D ffff830078f56000 >>>>> (XEN) context[0] =3D 101_2f0e1001 >>>>> (XEN) l3 =3D ffff83002f0e1000 >>>>> (XEN) l3_index =3D 0 >>>>> (XEN) l3[0] =3D 2f0e0003 >>>>> (XEN) l2 =3D ffff83002f0e0000 >>>>> (XEN) l2_index =3D ff >>>>> (XEN) l2[ff] =3D 0 >>>>> (XEN) l2[ff] not present >>>>> >>>>> >>>>> >>>>> Anyone any tips on what i could try ?, is this something caused by = xen, or something by the usb driver not adhering to kernel DMA-api ? >>>>> >>>>> Attached: >>>>> >>>>> - xm-info.txt >>>>> - xm-dmesg.txt >>>>> - xend.log >>>>> >>>>> - dom0-dmesg.txt >>>>> - dom0-lspci-tree.txt >>>>> - dom0-lspci.txt >>>>> >>>>> - domU-lspci.txt >>>>> - domU-dmesg.txt >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Xen-devel mailing list >>>>> Xen-devel@lists.xensource.com >>>>> http://lists.xensource.com/xen-devel >>>>> =20 >