Re: [early RFC] ARM PCI Passthrough design document

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Mar 09, 2017 at 01:26:45PM +0000, Julien Grall wrote:
> Hi Konrad,
> 
> On 09/03/17 11:17, Konrad Rzeszutek Wilk wrote:
> > On Thu, Mar 09, 2017 at 11:59:51AM +0900, Roger Pau Monné wrote:
> > > On Wed, Mar 08, 2017 at 02:12:09PM -0500, Konrad Rzeszutek Wilk wrote:
> > > > On Wed, Mar 08, 2017 at 07:06:23PM +0000, Julien Grall wrote:
> > > > > Hi,
> > > > > 
> > > > > On 02/02/17 23:06, Stefano Stabellini wrote:
> > > > > > On Thu, 2 Feb 2017, Julien Grall wrote:
> > > > > > > On 01/02/17 10:55, Roger Pau Monné wrote:
> > > > > > > > On Wed, Jan 25, 2017 at 06:53:20PM +0000, Julien Grall wrote:
> > > > > > > > > On 24/01/17 20:07, Stefano Stabellini wrote:
> > > > > > > > > > On Tue, 24 Jan 2017, Julien Grall wrote:
> > > > > > > For DT, I would have a fallback on mapping the root complex to DOM0 if we
> > > > > > > don't support it. So DOM0 could still use PCI.
> > > > > > > 
> > > > > > > For ACPI, I am expecting all the platform ECAM compliant or require few
> > > > > > > quirks. So I would mandate the support of the root complex in Xen in order to
> > > > > > > get PCI supported.
> > > > > > 
> > > > > > Sound good. Ack.
> > > > > 
> > > > > I am currently rewriting the design document to take into account all the
> > > > > comments and follow the path to have the host bridge in Xen and DOM0 will
> > > > > get an emulated one.
> > > > > 
> > > > > I began to look at scanning and configuring PCI devices in Xen. Looking at
> > > > > the PCI firmware specification, the firmware is not required to configure
> > > > > the BAR register other than for boot and console devices. This means an
> > > > > Operating System (or the hypervisor in our case) may have to configure some
> > > > > devices.
> > > > > 
> > > > > In order to configure the BAR register, Xen would need to know where are the
> > > > > PCI resources. On ACPI they can be found in ASL, however Xen is not able to
> > > > > parse it. In the case of Device Tree with can retrieve the PCI resources
> > > > > using the property "ranges".
> > > > > 
> > > > > I can see a couple of solutions:
> > > > > 	1# Rely on DOM0 to do the PCI configuration. This means that DOM0 should
> > > > > see all the PCI devices and therefore will not be possible to hide from DOM0
> > > > > if we know at boot a device will be used by a guest (i.e something similar
> > > > > to pciback.hide but directly handled in Xen).
> > > > 
> > > > .. this as for SR-IOV devices you need the drivers to kick the hardware
> > > > to generate the new bus addresses. And those (along with the BAR regions) are
> > > > not visible in ACPI (they are constructued dynamically).
> > > 
> > > There's already code in Xen [0] to find out the size of the BARs of SR-IOV
> > > devices, but I'm not sure what's the intended usage of that, does it need to
> > > happen _after_ the driver in Dom0 has done whatever magic for this to work?
> > 
> > Yes. This is called via the PHYSDEVOP_pci_device_add hypercall when
> > the device driver in dom0 has finished "creating" the VF. See drivers/xen/pci.c
> 
> We are thinking to not use PHYSDEVOP_pci_device_add hypercall for ARM and do
> the PCI scanning in Xen.
> 
> If I understand correctly what you said, only the PCI driver will be able to
> kick SR-IOV device and Xen would not be able to detect the device until it
> has been fully configured. So it would mean that we have to keep
> PHYSDEVOP_pci_device_add around to know when Xen can use the device.
> 
> Am I correct?

Yes. Unless the PCI drivers come up with some other way to tell the
OS that oh, hey, there is this new PCI device with this BDF.

Or the underlaying bus on ARM can send some 'new device' information?

> 
> Cheers,
> 
> -- 
> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel