Re: [RFC] hw/arm/virt: Provide DT binding generation for PCI eXpander Bridges

[Jonathan Cameron via <qemu-devel@nongnu.org>]

On Tue, 25 Apr 2023 09:28:44 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:

> On Mon, 24 Apr 2023 at 22:56, Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
> >
> > On Mon, 24 Apr 2023 16:45:48 +0100
> > Peter Maydell <peter.maydell@linaro.org> wrote:  
> > > On the other hand, having QEMU enumerate PCI devices is *also* a
> > > very different model from today, where we assume that the guest
> > > code is the one that is going to deal with enumerating PCI devices.
> > > To my mind one of the major advantages of PCI is exactly that it
> > > is entirely probeable and discoverable, so that there is no need
> > > for the dtb to include a lot of information that the kernel can
> > > find out for itself by directly asking the hardware...  
> >
> > I absolutely agree that QEMU enumerating PCI seem silly level of complexity
> > to introduce. So easy route is to just use the bus numbers to partition
> > resources. We have those available without any complexity. It's not the
> > same as how it's done with ACPI, but then the alternatives are either
> > (though maybe they are closer).  Note current proposed algorithm may be
> > too simplistic (perhaps some alignment forcing should adjust the division
> > of the resources to start on round number addresses)  
> 
> I think we definitely need to talk about this later this week,
> but my initial view is that if:
>  (1) the guest kernel can get the information it needs to do this
>      by probing the hardware

Not currently (from a quick look) - see below. But we could potentially
make it visible by augmenting the config space of the PCIe devices
that are discoverable.  Or we could in theory ignore the bus numbers
that we do provide as QEMU parameters, but that would be rather
surprising for users.

>  (2) doing it in QEMU gives you "this isn't a great allocation"
>      "we don't really have the info we need to do it optimally"
>      "this is more of a policy decision" effects
>      (which is what it's sounding like to me)
> 
> this is a really strong argument for "guest software should be
> doing this". DTB-booting kernels has always meant the kernel
> doing a lot of work that under ACPI/UEFI/x86-PC is typically
> done by firmware, and this seems similar to me.

We could explore only solving the problem for pxb-cxl for now.
However, we would still be talking infrastructure in kernel only
to support emulated CXL devices and I can see that being
controversial. A normal CXL host bridge is not something
we can enumerate.

I guess this may call for a PoC on the kernel side of things to
see how bad it looks. I suspect very ugly indeed :( but I could
be wrong.

I think we'll also have to augment the PXB PCI devices
that appear on the main bus to provide discoverability they don't
currently have (such as bus numbers)  Probably a DVSEC entry
in PCI extended space.

Current dump of what is there:

root@debian:~# lspci -s 05.0  -xxxx -vvv
00:05.0 Host bridge: Red Hat, Inc. QEMU PCIe Expander bridge
        Subsystem: Red Hat, Inc. QEMU PCIe Expander bridge
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
00: 36 1b 0b 00 00 00 a0 00 00 00 00 06 00 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
30: 00 00 00 00 00 00 00 00 00 00 00 00 ff 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Which is pretty light on info.

Jonathan


> 
> thanks
> -- PMM