Re: RFC: [PATCH 1/3] Enhance platform support for PCI

[Ian Campbell <ian.campbell@citrix.com>]

On Fri, 2015-02-20 at 14:39 +0000, Jan Beulich wrote:
> >>> On 20.02.15 at 15:26, <ian.campbell@citrix.com> wrote:
> > On Fri, 2015-02-20 at 14:11 +0000, Jan Beulich wrote:
> >> Otherwise,
> >> just sequentially assign segment numbers as you discover them or
> >> get them reported by Dom0. You could even have Dom0 tell you
> >> the segment numbers (just like we do on x86),
> > 
> > Aha, how does this work on x86 then? I've been looking for a hypercall
> > which dom0 uses to tell Xen about PCI segments to no avail (I just find
> > ones for registering actual devices).
> 
> But that's the one,

that == physdev_pci_device_add?

AFAICT that tells Xen about a given device existing on a particular
segment, but doesn't tell Xen any of the properties of that segment.

>  plus the MMCFG reporting one (PHYSDEVOP_pci_mmcfg_reserved).

This looks promising, but rather under-documented.

        #define PHYSDEVOP_pci_mmcfg_reserved    24
        struct physdev_pci_mmcfg_reserved {
            uint64_t address;
            uint16_t segment;
            uint8_t start_bus;
            uint8_t end_bus;
            uint32_t flags;
        };
        
I suppose the first 4 fields correspond to entries in the MMCFG table?
Which x86 Xen can parse and so can dom0, so dom0 can then make this
hypercall, passing (address,segment,start_bus,end_bus) to set the flags?

What is address the address of? The CFG space I think?

On ARM with DT I think we only get given address, and something has to
make up segment, start/end_bus I'm not sure where we would get them
from.

So although I think we could perhaps bend this interface to ARMs needs
it would have rather different semantics to x86, i.e. instead of the
"key" being (address,segment,start_bus,end_bus) and the "value" being
flags it would be something like "key" = (address) and "value" =
(segment,start_bus,end_bus,flags).

I don't think reusing like that would be wise.

>  Without ACPI, how do you
> know on ARM how to access config space for a particular
> segment?

That's the issue we are trying to resolve, with device tree there is no
explicit segment ID, so we have an essentially unindexed set of PCI
buses in both Xen and dom0.

So something somewhere needs to make up a segment ID for each PCI bus
and Xen and dom0 need to somehow agree on what the mapping is e.g. by
the one which made up the segment ID telling the other or some other TBD
means.

On x86 you solve this because both Xen and dom0 can parse the same table
and reach the same answer, sadly DT doesn't have everything needed in
it.

Ian.