From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: RFC: [PATCH 1/3] Enhance platform support for PCI Date: Mon, 23 Feb 2015 15:02:41 +0000 Message-ID: <1424703761.27930.140.camel@citrix.com> References: <54E71BDE.5020106@caviumnetworks.com> <54E7229C.7000301@linaro.org> <54E72452.3090801@caviumnetworks.com> <54E72688.9010005@linaro.org> <54E729F1.6000804@caviumnetworks.com> <54E73010.2050902@caviumnetworks.com> <1424439941.30924.243.camel@citrix.com> <54E74EA802000078000620B7@mail.emea.novell.com> <1424442385.30924.265.camel@citrix.com> <54E7552F020000780006215A@mail.emea.novell.com> <1424444504.30924.282.camel@citrix.com> <54E75D8702000078000621DD@mail.emea.novell.com> <1424453619.30924.381.camel@citrix.com> <54EAF6390200007800062593@mail.emea.novell.com> <1424695518.27930.74.camel@citrix.com> <54EB42320200007800062871@mail.emea.novell.com> <1424702013.27930.118.camel@citrix.com> <54EB4AFE02000078000628B3@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <54EB4AFE02000078000628B3@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: Prasun.kapoor@cavium.com, Manish Jaggi , Vijaya Kumar , Julien Grall , xen-devel@lists.xen.org, "StefanoStabellini(Stefano.Stabellini@citrix.com)" List-Id: xen-devel@lists.xenproject.org On Mon, 2015-02-23 at 14:45 +0000, Jan Beulich wrote: > >>> On 23.02.15 at 15:33, wrote: > > On Mon, 2015-02-23 at 14:07 +0000, Jan Beulich wrote: > >> >>> On 23.02.15 at 13:45, wrote: > >> > In which case might we be at liberty to specify that on ARM+Device Tree > >> > systems (i.e. those where the f/w tables don't give an enumeration) > >> > there is a 1:1 mapping from segments to host bridges? > >> > >> This again can only be answered knowing how bus number > >> assignment gets done on ARM (see also below). If all bus numbers > >> are distinct, there's no need for using segments numbers other > >> then zero. In the end, if you used segment numbers now, you may > >> end up closing the path to using them for much larger future setups. > >> But if that's not seen as a problem then yes, I think you could go > >> that route. > > > > Ultimately we just need to be able to go from the set of input > > parameters to e.g. PHYSDEVOP_pci_device_add to the associate host > > bridge. > > > > It seems like the appropriate pair is (segment,bus), which uniquely > > corresponds to a single host bridge (and many such pairs may do so). > > > > So I think the original question just goes from having to determine a > > way to map a segment to a host bridge to how to map a (segment,bus) > > tuple to a host bridge. > > Right. Avoiding (at this point in time) non-zero segments if at all > possible. I think it sounds like we are going to leave that up to the dom0 OS and whatever it does gets registered with Xen. So non-zero segments is no longer (directly) up to the Xen code. I think that's fine. > >> >> What I don't get from this is where the BDF is being represented. > >> > > >> > It isn't, since this is representing the host controller not any given > >> > PCI devices which it contains. > >> > > >> > I thought in general BDFs were probed (or even configured) by the > >> > firmware and/or OS by walking over the CFG space and so aren't > >> > necessarily described anywhere in the firmware tables. > >> > >> They're effectively getting assigned by firmware, yes. This mainly > >> affects bridges, which get stored (in their config space) the > >> secondary and subordinate bus numbers (bounding the bus range > >> they're responsible for when it comes to routing requests). If on > >> ARM firmware doesn't assign bus numbers, is bridge setup then a > >> job of the OS? > > > > I'm not completely sure I think it depends on the particular firmware > > (u-boot, EFI etc) but AIUI it can be the case that the OS does the > > enumeration on at least some ARM platforms (quite how/when it knows to > > do so I'm not sure). > > In which case the Dom0 OS doing so would need to communicate > its decisions to the hypervisor, as you suggest further down. So more concretely something like: #define PHYSDEVOP_pci_host_bridge_add struct physdev_pci_host_bridge_add { /* IN */ uint16_t seg; uint8_t bus; uint64_t address; }; typedef struct physdev_pci_host_bridge_add physdev_pci_host_bridge_add_t; DEFINE_XEN_GUEST_HANDLE(physdev_pci_host_bridge_add_t); Where seg+bus are enumerated/assigned by dom0 and address is some unique property of the host bridge -- most likely its pci cfg space base address (which is what physdev_pci_mmcfg_reserved also takes, I think?) Do you think we would need start_bus + end_bus here? Xen could enumerate this itself I think, and perhaps should even if dom0 tells us something? > This > basically replaces the bus scan (on segment 0) that Xen does on > x86 (which topology information gets derived from). Is the reason for the scan being of segment 0 only is that it is the one which lives at the legacy PCI CFG addresses (or those magic I/O ports)? What about other host bridges in segment 0 which aren't at that address? You could do the others based on MMCFG tables if you wanted, right? Ian.