Re: Status of FLR in Xen 4.4

[Gordan Bobic <gordan@bobich.net>]

On 09/26/2013 07:41 PM, Matthias wrote:
> Hi,
>
> thanks for your answers, the cards are a AMD HD 5750 and a HD 5400, both
> with dual functions (due to audio capabilities), both co-assigned to
> their respective domU and both not capable of FLR from lspci -vvv output.
>
> also, @Ross, I'm running a 3.8.2 Kernel, so this should be fine, but I
> assume that the 'official' command where xl asks the dom0 about the
> reset do not work (if I have understand david correctly) since it's dual
> function so no dual bus reset is actually executed causing the
> misbehaviour, and on the other side xm doing a bus reset so it works in
> this specific case.
>
> I'm currently recompiling the kernel to see if your patch works David.
>
> Also, just to understand it better, is the secondary bus reset the thing
> which you can manually invoke via /sys/bus/pci/devices/.../reset ?
>
> So as a workaround, would the following work in principle?
>
> xl pci-assignable-remove 0X:00.0
> xl pci-assignable-remove 0X:00.1
> echo "1" > /sys/bus/pci/devices/0X:00.0/reset
> echo "1" > /sys/bus/pci/devices/0X:00.1/reset

This bit is up to the driver to implement. Since pciback is a 
placeholder rather than a driver that knows about the hardware the reset 
node won't be there.

You could try to do something with setpci to force the registers between 
D0 and D3 power states in a vague hope that might do something, but I 
doubt it.

The reason nvidia cards work OK is because the domU driver knows how to 
reinitialize the hardware and acts accordingly. If the manufacturer 
won't implement a standard function to reset the hardware, then it is up 
to their drivers to handle the situation.

As a workaround, if (on Windows domUs) ejecting the card before 
shutdown/reboot of domU works, you could probably write some powershell 
magic that does that on shutdown/reboot as a reasonable workaround.

Gordan