Re: [PATCH] PCI/P2PDMA: start with a whitelist for root complexes

[Bjorn Helgaas <helgaas@kernel.org>]

On Fri, Apr 19, 2019 at 02:24:18PM +0000, Koenig, Christian wrote:
> Am 18.04.19 um 19:24 schrieb Bjorn Helgaas:
> > On Thu, Apr 18, 2019 at 10:58:55AM -0600, Logan Gunthorpe wrote:
> >> On 2019-04-18 10:33 a.m., Bjorn Helgaas wrote:
> >>> On Thu, Apr 18, 2019 at 01:58:59PM +0200, Christian König wrote:
> >>>> A lot of root complexes can still do P2P even when PCI devices
> >>>> don't share a common upstream bridge.
> >>>>
> >>>> Start adding a whitelist and allow P2P if both participants are
> >>>> attached to known good root complex.
> >>> Is there a plan for addressing this in a generic way that doesn't
> >>> require an OS modification for every new "known good root complex",
> >>> e.g., some PCIe or ACPI spec update that allows the OS to discover
> >>> this?
> >> I'm aware of work going on in the PCI SIG to address this [1].
> >>
> >> But I expect it's going to be a long time before actual hardware advertises
> >> this capability to indicate support. So in the interim we either need to not
> >> use p2pdma on root complexes or create a white list. I'm in favour of the
> >> white list approach.
> > I agree we need a whitelist; I just want to make sure we also make
> > progress on some way to limit the amount of time we need to update it.
> 
> Well as far as I know at least for AMD there is also a vendor specific 
> way to figure out if a chipset does P2P routing or not. So you don't 
> need every possible combination listed.
> 
> It's perfectly possible that Intel, ARM and PowerPC have something 
> similar and with all those on board you have pretty much every 
> interesting case covered.
> 
> I will try to dig something up into that direction.
> 
> Another problem is also that some root complexes have very very strange 
> limitations on the routing which are hard to handle. For example some 
> old chipsets can do P2P, but only for writes and not reads!
> 
> Anyway at least for my current use case a simple whitelist with 
> vendor/device should be sufficient, so I'm fine with maintaining that 
> for now. But on the other hand I also understand that in 10+ years we 
> don't want a list with 200 entries here....

Right.  And it's not just *you*; maintaining a list like that is also
a burden for me and every distro maintainer who has to backport
updates.

And the fact that it's not covered by any spec also means we're open
to all kinds of quirks about how it interacts with things that *are*
in the spec: ACS, ATS, the read/write thing you mentioned, ...

> >> [1] https://lore.kernel.org/linux-pci/20181210115653.0000615a@huawei.com/