Re: [PATCH 0/2] cxl: DVSEC Range emulation fixups

[Gregory Price <gregory.price@memverge.com>]

On Tue, Feb 21, 2023 at 05:51:13PM -0800, Dan Williams wrote:
> Jonathan points out that the kernel is too agressive in assuming that
> DVSEC range registers are in use, reliably skip emulation if
> 'mem_enabled' is not set. The helper devm_cxl_setup_emulated_hdm() is
> needlessly redoing an allocation, clean that up.
> 
> ---
> 
> Dan Williams (2):
>       cxl/hdm: Fix double allocation of @cxlhdm
>       cxl/hdm: Skip emulation when driver manages mem_enable
> 
> 
>  drivers/cxl/core/hdm.c |   65 ++++++++++++++++++------------------------------
>  drivers/cxl/cxl.h      |    4 ++-
>  drivers/cxl/port.c     |    2 +
>  3 files changed, 28 insertions(+), 43 deletions(-)
> 
> base-commit: 23c198e3dfaabbc891681aecb0855b9e0ac791e1


not *quite* sure what to make of this yet, but i get stack trace on boot
on real hardware with this patch.  I'm debugging other issues with this
hardware, so i'm not sure if it's related or not, but prior to this patch
I did not have a stack trace.


I think there's two issues here:

1) The system I'm on fails to register a CFMW/root port decoder.  I'm
   not entirely sure why, other than during cxl_decoder_add(), the
   target map contains "[0,]" as the target id's, and the only
   registered ports/decoders are the endpoints.

   I don't know whether this is because the hardware just doesn't have a
   root decoder, or what.  But it makes the volatile region patches
   non-functional, and i have to revert back to static configuration to
   use the real cxl device (i.e. don't mark it EFI_MEMORY_SP).

2) Per the second bit - there's no component registers being registered
   for this cxl device (plus some spurious DOE error).


The no root decoder thing has been throwing me for a loop, if you can
help me shed some light on this i'd greatly appreciate it.  If a socket
has no decoders, should we expect memory expanders to be managable via
the volatile region system in the driver?


relevant dmesg info

[   21.928436] cxl root0: Failed to populate active decoder targets
[   21.929077] cxl_acpi ACPI0017:00: Failed to add decode range [0x1050000000 - 0x304fffffff]
[   21.933150]  pci0000:3f: host supports CXL (restricted)
[... snip ...]
[   21.965126] cxl_pci 0000:3f:00.0: No component registers (-19)
[   22.001597] cxl_pci 0000:3f:00.0: DOE: [d80] failed to cache protocols : -5
[   22.002351] cxl_pci 0000:3f:00.0: Failed to create MB object for MB @ d80
[   22.003265] cxl_pci 0000:3f:00.0: Failed to request region 0x0000000000001fff-0x000000000010201e
[... snip ...]
[   22.339973] BUG: unable to handle page fault for address: 0000000000001000
[   22.340584] #PF: supervisor read access in kernel mode
[   22.346801] #PF: error_code(0x0000) - not-present page
[   22.349059] PGD 1339ec067 P4D 0
[   22.350877] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   22.354558] CPU: 45 PID: 1351 Comm: systemd-udevd Not tainted 6.2.0+ #7
[   22.358357] RIP: 0010:cxl_probe_component_regs+0x23/0x180 [cxl_core]
[   22.361571] Code: 90 90 90 90 90 90 90 0f 1f 44 00 00 41 57 31 c0 b9 06 00 00 00 41 56 41 55 41 54 49 89 fc 48 89 d7 55 53 48 83 ec 10 f3 48 ab <8b> 86 00 10 00 00 66 83 f8 01 0f 85 30 01 00 00 c1 e8 18 0f 84 93
[   22.367000] RSP: 0018:ff4fc33844ce7830 EFLAGS: 00010286
[   22.369016] RAX: 0000000000000000 RBX: ff4f28d002dba000 RCX: 0000000000000000
[   22.372736] RDX: ff4fc33844ce7898 RSI: 0000000000000000 RDI: ff4fc33844ce78c8
[   22.372739] RBP: ff4f28d0baaf1268 R08: 0000000000000000 R09: 0000000000000001
[   22.375842] R10: 0000000000000001 R11: 00000000e0690b8e R12: ff4f28d002dba000
[   22.375844] R13: ff4fc33844ce7920 R14: ff4f28d000f9fc28 R15: ff4f28d028b45810
[   22.375845] FS:  00007fbb75a07580(0000) GS:ff4f28df09c00000(0000) knlGS:0000000000000000
[   22.375847] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   22.375848] CR2: 0000000000001000 CR3: 0000000133ab0003 CR4: 0000000000771ee0
[   22.375849] PKRU: 55555554
[   22.375850] Call Trace:
[   22.375854]  <TASK>
[   22.375858]  map_hdm_decoder_regs+0x46/0x90 [cxl_core]
[   22.398038]  devm_cxl_setup_hdm+0x95/0x120 [cxl_core]
[   22.398111]  cxl_port_probe+0xbe/0x190 [cxl_port]
[   22.398118]  cxl_bus_probe+0x14/0x50 [cxl_core]


~Gregory