Re: RFC: [PATCH 1/3] Enhance platform support for PCI

[Manish Jaggi <mjaggi@caviumnetworks.com>]

On 20/02/15 8:31 pm, Ian Campbell wrote:
> On Fri, 2015-02-20 at 14:39 +0000, Jan Beulich wrote:
>>>>> On 20.02.15 at 15:26, <ian.campbell@citrix.com> wrote:
>>> On Fri, 2015-02-20 at 14:11 +0000, Jan Beulich wrote:
>>>> Otherwise,
>>>> just sequentially assign segment numbers as you discover them or
>>>> get them reported by Dom0. You could even have Dom0 tell you
>>>> the segment numbers (just like we do on x86),
>>> Aha, how does this work on x86 then? I've been looking for a hypercall
>>> which dom0 uses to tell Xen about PCI segments to no avail (I just find
>>> ones for registering actual devices).
>> But that's the one,
> that == physdev_pci_device_add?
>
> AFAICT that tells Xen about a given device existing on a particular
> segment, but doesn't tell Xen any of the properties of that segment.
>
>>   plus the MMCFG reporting one (PHYSDEVOP_pci_mmcfg_reserved).
> This looks promising, but rather under-documented.
>
>          #define PHYSDEVOP_pci_mmcfg_reserved    24
>          struct physdev_pci_mmcfg_reserved {
>              uint64_t address;
>              uint16_t segment;
>              uint8_t start_bus;
>              uint8_t end_bus;
>              uint32_t flags;
>          };
>          
> I suppose the first 4 fields correspond to entries in the MMCFG table?
> Which x86 Xen can parse and so can dom0, so dom0 can then make this
> hypercall, passing (address,segment,start_bus,end_bus) to set the flags?
>
> What is address the address of? The CFG space I think?
>
> On ARM with DT I think we only get given address, and something has to
> make up segment, start/end_bus I'm not sure where we would get them
> from.
>
> So although I think we could perhaps bend this interface to ARMs needs
> it would have rather different semantics to x86, i.e. instead of the
> "key" being (address,segment,start_bus,end_bus) and the "value" being
> flags it would be something like "key" = (address) and "value" =
> (segment,start_bus,end_bus,flags).
>
> I don't think reusing like that would be wise.
>
>>   Without ACPI, how do you
>> know on ARM how to access config space for a particular
>> segment?
> That's the issue we are trying to resolve, with device tree there is no
> explicit segment ID, so we have an essentially unindexed set of PCI
> buses in both Xen and dom0.
>
> So something somewhere needs to make up a segment ID for each PCI bus
> and Xen and dom0 need to somehow agree on what the mapping is e.g. by
> the one which made up the segment ID telling the other or some other TBD
> means.
>
> On x86 you solve this because both Xen and dom0 can parse the same table
> and reach the same answer, sadly DT doesn't have everything needed in
> it.
In fact xen and dom0 use the same device tree nodes and in the same 
order. xen creates the device tree for dom0.
I think xen can enforce the ABI while creating device tree
>
> Ian.
>