Re: PCI Passthrough and PV-OPS

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Mar 03, 2011 at 02:25:47PM +0000, Kinsella, Ray wrote:
> Hi all,
> 
> I am having a problem with Xen 4.0.1, PCI Passthrough with VT-d and Linux PV Guests, and I was wondering if anyone else had seen it.
> In both Dom 0 and Dom U's I am using 64bit PV-OPS 2.6.32.26 Kernels from Jeremy Fitzhartridge's Git repo.

Do you have 'iommu=soft' in your guest configuration?

> 
> I am using dual port SR-IOV NIC's to test, passing through physical functions only.
> Passthrough appears to work, I can passthrough the NIC and it will appear in the guest with lspci.
> The guest detects the new device and loads the driver to service the NIC, as you would expect.
> 
> The guest is complaining about the NIC being hung, the message "Detected Tx Unit Hang..." is appearing in the system log on the guest.
> In the Xen log, VT-d is producing errors similar to this one;
> 
> (XEN) [VT-D]iommu.c:824: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:799: DMAR:[DMA Read] Request device [03:00.0] fault addr 7c584000, iommu reg = ffff82c3fff57000
> (XEN) DMAR:[fault reason 06h] PTE Read access is not set
> (XEN) print_vtd_entries: iommu = ffff83016fffa5f0 bdf = 3:0.0 gmfn = 7c584
> (XEN)     root_entry = ffff83016ff38000
> (XEN)     root_entry[3] = 14d90a001
> (XEN)     context = ffff83014d90a000
> (XEN)     context[0] = 102_152209005
> (XEN)     l4 = ffff830152209000
> (XEN)     l4_index = 0
> (XEN)     l4[0] = 152208003
> (XEN)     l3 = ffff830152208000
> (XEN)     l3_index = 1
> (XEN)     l3[1] = 0
> (XEN)     l3[1] not present
> 
> My understanding of what is happening here is that page table mappings are missing from VT-d's page table, mapping machine physical addresses to my PV guest's physical addresses. I have had a look at iommu_populate_page_table and I dumped the mappings as they are being setup but couldn't see a mapping for the GPA 0x7c584000 above?
> 
> Has anyone else encountered this issue?

Yes. And it was fixed by running a PV guest with 'iommu=soft' as Linux kernel command line argument.