Re: [edk2] OVMF broken under Xen (in PCI initialisation)

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/04/16 10:12, Gary Lin wrote:
> On Thu, Apr 28, 2016 at 05:08:50AM +0000, Ni, Ruiyu wrote:
>>
>> Regards,
>> Ray
>>
>>> -----Original Message-----
>>> From: Laszlo Ersek [mailto:lersek@redhat.com]
>>> Sent: Wednesday, April 27, 2016 6:44 PM
>>> To: Ni, Ruiyu <ruiyu.ni@intel.com>; Gary Lin <glin@suse.com>
>>> Cc: edk2-devel@lists.01.org <edk2-devel@ml01.01.org>; Xen Devel <xen-devel@lists.xen.org>; Kinney, Michael D
>>> <michael.d.kinney@intel.com>
>>> Subject: Re: [edk2] OVMF broken under Xen (in PCI initialisation)
>>>
>>> On 04/27/16 11:50, Ni, Ruiyu wrote:
>>>> Copying Mike.
>>>>
>>>> Regards,
>>>> Ray
>>>>
>>>>> -----Original Message-----
>>>>> From: Ni, Ruiyu
>>>>> Sent: Wednesday, April 27, 2016 5:49 PM
>>>>> To: 'Gary Lin' <glin@suse.com>
>>>>> Cc: edk2-devel@lists.01.org; Xen Devel <xen-devel@lists.xen.org>; Laszlo Ersek <lersek@redhat.com>
>>>>> Subject: RE: [edk2] OVMF broken under Xen (in PCI initialisation)
>>>>>
>>>>> Gary,
>>>>> Thanks for the try.
>>>>>
>>>>> I now understand why the issue happens.
>>>>> 1. PciEnumeratorLight() depends on the MinBus returned from
>>>>> PciRootBridgeIo.Configuration().
>>>>> 2. PciRootBridgeIo.Configuration() returns the resource descriptors
>>>>> initialized by PciHostBridge.NotifyPhase(EfiPciHostBridgeAllocateResources).
>>>>> 3. PciHostBridge.NotifyPhase(EfiPciHostBridgeAllocateResources)
>>>>> is not called when PcdPciDisableBusEnumeration is TRUE
>>>>>
>>>>> I am thinking about let PciRootBridgeIo.Configuration() returns
>>>>> the correct resource consumption information even when
>>>>> PcdPciDisableBusEnumeration is TRUE.
>>>>>
>>>>> I can add a new field to PCI_ROOT_BRIDGE structure defined in
>>>>> MdeModulePkg/Include/Library/PciHostBridgeLib.h to indicate
>>>>> that the resources filled in PCI_ROOT_BRIDGE.Bus/Io/Mem/
>>>>> MemAbove4G/PMem/PMemAbove4G are already assigned
>>>>> to PCI bus. So that PciRootBridgeIo.Configuration() can return
>>>>> these value even when
>>>>> PciHostBridge.NotifyPhase(EfiPciHostBridgeAllocateResources)
>>>>> is not called due to PcdPciDisableBusEnumeration is TRUE.
>>> Can you use PcdPciDisableBusEnumeration directly for this?
>>>
>>> (If you don't like the idea, we could certainly set the new field in
>>> PCI_ROOT_BRIDGE from the PCD too, in OvmfPkg/Library/PciHostBridgeLib.)
>>>
>>>>> Do you know whether Xen passes the PCI device resource
>>>>> information to firmware?
>>> I don't think so, no.
>>>
>>> But, given that the previous PciHostBridgeDxe driver was working on Xen,
>>> can we perhaps emulate that behavior in
>>> "OvmfPkg/Library/PciHostBridgeLib" somehow?
>> Let me explain the reason why previous PciHostBridgeDxe driver was
>> working on Xen:
>> The PciRootBridgeIo.Configuration() in new driver only create resource
>> descriptor when the resource status is allocated:
>>
>> if (ResAllocNode->Status != ResAllocated) {
>>   continue;
>> }
>>
>> The same function in old driver doesn't check the Status field so it
>> unconditionally returns BUS descriptor with AddrRangeMin and
>> AddrRangeMax both equal 0. So that PciEnumeratorLight()
>> in PciBus driver can still search the devices from starting bus 0.
>> It just happened to work.
>>
>> I think the new driver's Configuration() implementation is correct
>> while the old driver's one is wrong. So I don't want to change to 
>> wrong implementation to fix this issue.
>>
>> The issue can be resolved if we have a way to tell PciBus
>> PciEnumeratorLight() which bus number to start searching.
>>
>> It's almost true that starting bus number for root bridge #0
>> is 0. But it might not be true for the rest root bridges.
>>
>> OVMF's PciHostBridgeLib currently returns multiple root bridges
>> and for root bridge #1, the starting bus number is obviously
>> not 0 unless "etc/extra-pci-roots" doesn't exist or is 0 so there is
>> only one root bridge.
>>
>> My question is in OVMF over Xen, does "etc/extra-pci-roots" exist?
>> If it exists, device behind root bridge #1, #2... can *not* be found
>> with the current implementation.
>>
> From my OVMF/Xen debug log:
> [...]
> QemuFwCfg interface not supported.
>
> I guess "etc/extra-pci-roots" won't exist in Xen though I can't
> guarantee it due to my limited experience with Xen.

Information layout like this in Xen is currently in a dire state.

Buses other than bus 0 don't function, there is no MMCONFIG support and
BARs only work because access to unpopulated ranges in the physical
memory get forwarded to Qemu.

There are several related projects planned which will go a long way to
untangling this mess, but there is no quick fix.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel