Re: Struggling with PCI-Passthrough

[Gordan Bobic <gordan@bobich.net>]

On 2014-03-24 19:18, Alexander Brychcy wrote:

> As ever there are only problems with domU #2. The first one, that i
> could not passthrough an USB controller resolved. No matter at which
> port you plug an USB device on the mainboard, it is always assigned to
> the XHCI controller.

I found that on Intel ICH USB controllers the PCI USB device that the
USB device is attached to changes dependant on what is connected. There
is some odd routing going on. This is especially the case when a USB
hub is connected to a port. If you just want to pass through, say, a
keyboard, it is stable, but IIRC the device has to be connected to the
port before you re-assign the PCI device that is the port to 
xen-pcistub.

My motherboard also comes with a NEC USB 3.0 adapter on-board which
doesn't suffer from this problem, so for VMs that require multiple
USB devices I have a USB hub on that controller and I pass the
whole controller to the VM.

> My workaround is to add another PCIe USB controller
> for the vm, so i'm still able to have usb connection in dom0 for backup
> or other purposes.

Sounds similar to what I do. Note that you can also pass specific USB
devices to the domU, rather than using PCI passthrough for the USB
controllers. Unfortunately this comes with ~5% CPU idling overhead
in domU due to USB polling.

> Tried to passthrough a NIC to verify if it is a bios related bug. This
> works without any problems and nothing freezes at all.

I suspect you are just seeing the Intel ICH USB weirdness I described
above.

> So last but not least the VGA Passthrough. Issue is still present. HVM
> boots and GPU is assigned and working. But after some time respectively
> an application tries to access some kind of function the whole system
> freezes.

How much RAM are you passing to the VM? It is possible your motherboard
suffers from the same problem as mine and IOMEM DMA is bypassing the
root PCIe hub. Do the following:

lspci -v | grep "Memory at" | sed -e 's/.*Memory at //' -e 's/ .*//' | 
sort

Convert the first address that comes back from hex, and set the amount
of memory you are passing to your VM to a few MB below that.

Then see if the crash still happens after a while. If it doesn't, then
there is a good chance that what is happening is that your VM's
memory is trampling over the IOMEM region of the host because it
isn't being translated by the IOMMU, probably because DMA is
bypassing it. In my case this is caused by a bug/feature of NF200
PCIe switches, but it could also be caused by a buggy motherboard
chipset or firmware.

> Nearly forgot the RAM problem. The maximum amount of RAM i can assign 
> to
> the HVM when i passthrough the GPU is 3GB. Haven't tried it yet but may
> this patch still be usable with 4.4.0?
> http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram

I think this is for a different bug, but probably worth a try
depending on what results you get with domU memory size reduced to
ensure it cannot trample IOMEM regions.

Gordan