Re: Dealing with non-existent BDF devices in VT-d and in the hardware.

[Gordan Bobic <gordan@bobich.net>]

On 03/20/2014 12:48 AM, Zhang, Yang Z wrote:
> Jan Beulich wrote on 2014-03-19:
>>>>> On 19.03.14 at 13:57, Konrad Rzeszutek Wilk
>>>>> <konrad.wilk@oracle.com>
>> wrote:
>>> On Wed, Mar 19, 2014 at 12:32:31AM +0000, Zhang, Yang Z wrote:
>>>> Konrad Rzeszutek Wilk wrote on 2014-03-18:
>>>>> On Mon, Mar 17, 2014 at 01:03:00AM +0000, Zhang, Yang Z wrote:
>>>>> I think there are two issues here:
>>>>>
>>>>> a) Missing device assigments via groups. That should be done irregardless
>>>>>     if the device / hardware is buggy.
>>>>
>>>> Yes, this is missing.
>>>>
>>>>> b) Buggy devices like the IDT bridge that I see. That is a
>>>>> seperate issue -
>>> and
>>>>>     we just discussion if we want to inject that in the VT-d (or
>>>>> AMD-VI)
>> what
>>>>>     would be the mechanism to do that.
>>>>
>>>> The question is that device 08:00.0 doesn't exist in your platform,
>>>> you only
>>> saw the BDF in the DMA transaction. How can you add a non-exist
>>> device to a group?
>>>
>>> Why do I need to add it to a group? The patch I posted (see first
>>> email in this thread) just made a fake PCI device in the Xen
>>> hypervisor. But I don't see libxl nor QEMU doing any group
>>> operations
>>> - so why are they required? If I just bundle all of the PCI devices
>>> underneath that bridge to the guest it should be OK, shouldn't it?
>>
>> It should. You're in trouble if (by mistake) you don't pass them all,
>> and to avoid that is what the grouping seems to have been intended
>> for. The fact that only xend used it (and even then only for checking
>> rather to enforce the grouping) doesn't help it of course. But that
>> grouping issue is orthogonal to your issue, it's just that the group
>> assignment (if it were there) could take care of the assignment part
>> of your issue - the create-a-fake-device part would remain.
>
> fake a device is a solution. But I am thinking (maybe I am wrong) why
> not setup all VT-d entries under a bridge if passing a PCI device under
> a bridge. Because when passing a PCI device under a bridge, all devices
> under bridge should be assigned to the guest too. What current Xen dose
> is only set the entry which has device, so why not extend it to setup
> all entries? In this case, there is no user input is required.

I'm not sure if I'm reading this right, but you wouldn't necessarily be 
passing all devices under a particular bridge to guests (and certainly 
not necessarily to the same guest). You could have multiple levels of 
bridges to provide extra PCIe links. One obvious example is dual GPU 
cards where both GPUs are under a bridge, but you wouldn't necessarily 
be passing both devices to a guest (one might be the primary GPU for the 
host).

Gordan