From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: PCI Passthrough and PV-OPS Date: Thu, 3 Mar 2011 10:03:12 -0500 Message-ID: <20110303150312.GA8024@dumpdata.com> References: <7D07DB03D5AECA4CB23F2CA801AA720BF9694916@irsmsx502.ger.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <7D07DB03D5AECA4CB23F2CA801AA720BF9694916@irsmsx502.ger.corp.intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Kinsella, Ray" Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org On Thu, Mar 03, 2011 at 02:25:47PM +0000, Kinsella, Ray wrote: > Hi all, > > I am having a problem with Xen 4.0.1, PCI Passthrough with VT-d and Linux PV Guests, and I was wondering if anyone else had seen it. > In both Dom 0 and Dom U's I am using 64bit PV-OPS 2.6.32.26 Kernels from Jeremy Fitzhartridge's Git repo. Do you have 'iommu=soft' in your guest configuration? > > I am using dual port SR-IOV NIC's to test, passing through physical functions only. > Passthrough appears to work, I can passthrough the NIC and it will appear in the guest with lspci. > The guest detects the new device and loads the driver to service the NIC, as you would expect. > > The guest is complaining about the NIC being hung, the message "Detected Tx Unit Hang..." is appearing in the system log on the guest. > In the Xen log, VT-d is producing errors similar to this one; > > (XEN) [VT-D]iommu.c:824: iommu_fault_status: Primary Pending Fault > (XEN) [VT-D]iommu.c:799: DMAR:[DMA Read] Request device [03:00.0] fault addr 7c584000, iommu reg = ffff82c3fff57000 > (XEN) DMAR:[fault reason 06h] PTE Read access is not set > (XEN) print_vtd_entries: iommu = ffff83016fffa5f0 bdf = 3:0.0 gmfn = 7c584 > (XEN) root_entry = ffff83016ff38000 > (XEN) root_entry[3] = 14d90a001 > (XEN) context = ffff83014d90a000 > (XEN) context[0] = 102_152209005 > (XEN) l4 = ffff830152209000 > (XEN) l4_index = 0 > (XEN) l4[0] = 152208003 > (XEN) l3 = ffff830152208000 > (XEN) l3_index = 1 > (XEN) l3[1] = 0 > (XEN) l3[1] not present > > My understanding of what is happening here is that page table mappings are missing from VT-d's page table, mapping machine physical addresses to my PV guest's physical addresses. I have had a look at iommu_populate_page_table and I dumped the mappings as they are being setup but couldn't see a mapping for the GPA 0x7c584000 above? > > Has anyone else encountered this issue? Yes. And it was fixed by running a PV guest with 'iommu=soft' as Linux kernel command line argument.