Re: [PATCH] cxl/port: Fix port to pci device assumptions in read_cdat_data()

[Jonathan Cameron <Jonathan.Cameron@huawei.com>]

On Sat, 22 Apr 2023 13:56:20 -0700
Dan Williams <dan.j.williams@intel.com> wrote:

> Lukas Wunner wrote:
> > On Fri, Apr 21, 2023 at 07:51:47PM -0700, Dan Williams wrote:  
> > > Not all CXL ports are associated with PCI devices. Host-bridge and
> > > cxl_test ports are hosted by platform devices. Teach read_cdat_data() to
> > > be careful about non-pci hosted cxl_memdev instances. Otherwise,
> > > cxl_test crashes with this signature:
> > > 
> > >  RIP: 0010:xas_start+0x6d/0x290
> > >  [..]
> > >  Call Trace:
> > >   <TASK>
> > >   xas_load+0xa/0x50
> > >   xas_find+0x25b/0x2f0
> > >   xa_find+0x118/0x1d0
> > >   pci_find_doe_mailbox+0x51/0xc0
> > >   read_cdat_data+0x45/0x190 [cxl_core]
> > >   cxl_port_probe+0x10a/0x1e0 [cxl_port]
> > >   cxl_bus_probe+0x17/0x50 [cxl_core]
> > > 
> > > Some other cleanups are included like removing the single-use @uport
> > > variable, and removing the indirection through 'struct cxl_dev_state' to
> > > lookup the device that registered the memdev and may be a pci device.
> > > 
> > > Fixes: af0a6c3587dc ("cxl/pci: Use CDAT DOE mailbox created by PCI core")
> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com>  
> > 
> > Reviewed-by: Lukas Wunner <lukas@wunner.de>
> > 
> > Take my Reviewed-by with a grain of salt as I'm absolutely not an
> > expert on the cxl struct hierarchy, nevertheless this looks sane to me.  
> 
> Yeah, I think we need a data structure relationship diagram to explain
> endpoint devices, memory devices, endpoint ports, and switch ports all
> interconnect.

Agreed. I keep forgetting how this all works as well and end up scattering
prints everywhere to find out.

A diagram to refer to would be very useful.

Jonathan

> 
> > I note however that before af0a6c3587dc, xa_for_each() was run on an
> > xarray which was not initialized with xa_init() on non-pci cxl ports.
> > (xa_init() was run from cxl_pci_probe() -> devm_cxl_pci_create_doe()
> > but xa_for_each() was run from read_cdat_data() -> find_cdat_doe()
> > for non-pci cxl ports as well.)
> > 
> > Hence can't this crash prior to af0a6c3587dc as well?  If it can,
> > the Fixes tag would rather have to point to c97006046c79 ("cxl/port:
> > Read CDAT table"), though this patch wouldn't apply cleanly to
> > pre-6.4 kernels.  c97006046c79 went into v6.0 and there's one stable
> > kernel between it and v6.4 (for which af0a6c3587dc is queued).
> > So if the missing xa_init() can indeed cause crashes, you may want to
> > base your fix on v6.3 and fold it in at the front of cxl/next,
> > then rebase af0a6c3587dc on top of it.  Let me know if you need help
> > or want me to look into it.  
> 
> As I replied on the other note, I think older kernels are ok.
> 
> > Thanks for fixing this and sorry for not spotting it myself!  
> 
> No worries, I had missed that the unit test were not run yet.