PCI-passthrough - issues/questions/ideas

PCI-passthrough - issues/questions/ideas

[Patrick Ringl]

Hi,

I just wanted to introduce a problem I currently face including some 
questions regarding my temporary fix.
Anyway, I have a PCI-device that I want to passthrough to a hvm guest. 
Now there are several problems that add up:

a) the PCI-device is bound to a PCI-to-PCI bridge (which in turn is 
directly attached to the rootbus) (mainboard has a AMD 970/SB950 chipset).
[since pciIsParent shows that the secondary bus equals the device's bus]:

     bridge:
 > lspci -s00:14.4 -vvv | grep Bus:
 > Bus: primary=00, secondary=07, subordinate=07, sec-latency=64

     PCI-device
 > 07:06.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)

b) neither the bridge nor the PCI device itself have the currently 
implemented reset functionality that you trigger in pciResetDevice

c) the PCI-device is mapped through the PCI bridge (IOMMU-wise):

     ACPI IOMMU dump:
 > [    1.121239] AMD-Vi:   DEV_SELECT             devid: 00:14.4 flags: 00
 > [    1.121274] AMD-Vi:   DEV_ALIAS_RANGE         devid: 07:00.0 
flags: 00 devid_to: 00:14.4
 > [    1.121311] AMD-Vi:   DEV_RANGE_END         devid: 07:1f.7


What I did to get (temporarily and in a rather hackish (maybe even 
wrong) manner) rid of the problem, is to ignore the error thrown in 
pciResetDevice when no reset had been possible at all.

     if (ret < 0) {
/*
     -- I know what you did last summer!

         virErrorPtr err = virGetLastError();
         pciReportError(VIR_ERR_INTERNAL_ERROR,
                        _("Unable to reset PCI device %s: %s"),
                        dev->name,
                        err ? err->message : _("no FLR, PM reset or bus 
reset available"));
*/
         ret = 0;
     }


Concludingly I'd ask the following questions:

a) Why is a secondary_bus_reset a bad idea if the "device in question"'s 
_primary_ bus is the root bus?
b) Why would it be a bad idea adding some sort of 'override attribute' 
to the guest's config, so libvirt may intentionally skip the reset? What 
are the possible consequences if no reset takes place at all?
c) What options do I have besides implementing c) or just going the 
dirty way and ignore the case when no reset is possible (like I 
described above)?

PS : Since I am not subscribed, please CC to me
PPS: Please cut me slack ;)


regards,
Patrick

Re: PCI-passthrough - issues/questions/ideas

[Chris Wright]

* Patrick Ringl (patrick_@freenet.de) wrote:
> Hi,
> 
> I just wanted to introduce a problem I currently face including some
> questions regarding my temporary fix.
> Anyway, I have a PCI-device that I want to passthrough to a hvm
> guest. Now there are several problems that add up:
> 
> a) the PCI-device is bound to a PCI-to-PCI bridge (which in turn is
> directly attached to the rootbus) (mainboard has a AMD 970/SB950
> chipset).
> [since pciIsParent shows that the secondary bus equals the device's bus]:
> 
>     bridge:
> > lspci -s00:14.4 -vvv | grep Bus:
> > Bus: primary=00, secondary=07, subordinate=07, sec-latency=64
> 
>     PCI-device
> > 07:06.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
> 
> b) neither the bridge nor the PCI device itself have the currently
> implemented reset functionality that you trigger in pciResetDevice
> 
> c) the PCI-device is mapped through the PCI bridge (IOMMU-wise):
> 
>     ACPI IOMMU dump:
> > [    1.121239] AMD-Vi:   DEV_SELECT             devid: 00:14.4 flags: 00
> > [    1.121274] AMD-Vi:   DEV_ALIAS_RANGE         devid: 07:00.0
> flags: 00 devid_to: 00:14.4
> > [    1.121311] AMD-Vi:   DEV_RANGE_END         devid: 07:1f.7
> 
> 
> What I did to get (temporarily and in a rather hackish (maybe even
> wrong) manner) rid of the problem, is to ignore the error thrown in
> pciResetDevice when no reset had been possible at all.
> 
>     if (ret < 0) {
> /*
>     -- I know what you did last summer!
> 
>         virErrorPtr err = virGetLastError();
>         pciReportError(VIR_ERR_INTERNAL_ERROR,
>                        _("Unable to reset PCI device %s: %s"),
>                        dev->name,
>                        err ? err->message : _("no FLR, PM reset or
> bus reset available"));
> */
>         ret = 0;
>     }
> 
> 
> Concludingly I'd ask the following questions:
> 
> a) Why is a secondary_bus_reset a bad idea if the "device in
> question"'s _primary_ bus is the root bus?

That's not the issue the code is guarding against.  It is guarding
against issuing a secondary bus reset on the root bus.  Your dev->bus
should be 7, not 0.  However, the part that should be failing is
pciTrySecondaryBusReset().  And this will fail if there are other devices
on that bus (07) that are not assigned to your guest, because a secondary
bus resest will reset _all_ devices on the secondary bus.

> b) Why would it be a bad idea adding some sort of 'override
> attribute' to the guest's config, so libvirt may intentionally skip
> the reset? What are the possible consequences if no reset takes
> place at all?

The problem with skipping the reset is primarily a security concern.
Device state will leak between users of the device.  An override is
possible, you can discuss that with libvirt developers to see if they'd
support an insecure flag like that.

> c) What options do I have besides implementing c) or just going the
> dirty way and ignore the case when no reset is possible (like I
> described above)?

You can try assiging all devices on bus 7 to the guest.  This should
allow a sbus reset to be issued.

thanks,
-chris