[PATCH v6] cxl/pci: Skip reset detection for DVSEC emulated decoders

[PATCH v6] cxl/pci: Skip reset detection for DVSEC emulated decoders

[Guixin Liu]

HDM decoders are emulated from the DVSEC range registers in two cases:
(a) the component registers expose no HDM decoder capability, or (b) the
capability is present but the DVSEC ranges were the ones in use at driver
load.

After an FLR or SBR, __cxl_endpoint_decoder_reset_detected() reads the HDM
decoder Committed bit for every decoder marked enabled, emulated ones
included. In case (a) regs.hdm_decoder is NULL and the read oopses. In case
(b) the Committed bit was never set, so a reset gets reported that never
happened.

Use the absence of cxld->commit to elide the check for emulated decoders.
Warn when doing so: the DVSEC range registers are non-sticky, so a reset
may have cleared the decode and, with no Committed bit to read, the loss
cannot be detected here.

Case (b) tested under QEMU: a reset on an endpoint driven down the DVSEC
emulation path no longer reports a reset or strips the decoder flags.

Fixes: 934edcd436dc ("cxl: Add post-reset warning if reset results in loss of previously committed HDM decoders")
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
---
This was patch 4/8 of the "cxl: Assorted fixes" series [1], resent
individually per review feedback.

The two synchronisation concerns the review bot raised on v2 - the missing
exclusion against a cxl_port unbind freeing the devm allocated cxl_hdm, and
the plain read-modify-write of cxld->flags in
cxl_endpoint_decoder_clear_reset_flags() while the region paths update the
same word under cxl_rwsem.region - are still untouched here. Both are
about the reset handler's locking rather than about which registers it
reads, and neither fix is local; happy to follow up with separate patches.

Testing

Reaching case (b) needs an endpoint whose DVSEC ranges are live while the
global HDM decoder enable is clear. Unbinding cxl_pci runs the
disable_hdm() and clear_mem_enable() devm actions, so both bits start
clean. setpci then programs DVSEC Range 1 Base to 0x890000000, which sits
inside a RAM capable CFMWS, and sets Mem_Enable. Rebinding takes the
emulation path:

  cxl_port endpoint3: Fallback map 1 range register
  cxl_pci 0000:35:00.0: DVSEC Range0 allowed by platform

The endpoint ends up with a single decoder, enabled and locked, with no
Committed bit behind it:

  decoder3.0: locked=1 start=0x890000000 size=0x100000000 mode=ram

The device's only available reset method is cxl_bus, which unmasks SBR
through the port DVSEC. Before this change:

  cxl_pci 0000:35:00.0: resetting
  cxl_pci 0000:35:00.0: SBR happened without memory regions removal.
  cxl_pci 0000:35:00.0: System may be unstable if regions hosted system memory.

/proc/sys/kernel/tainted picked up TAINT_USER (8192 -> 8256) and locked
dropped from 1 to 0. After this change the same reset logs only
"resetting", locked stays 1, and no taint is added. The decoder is
decoder6.0 in the second run because reloading cxl_core renumbered the
memdevs.

QEMU does not clear the DVSEC range registers on SBR, so the emulated
decode is still fully programmed by the time the reset completes. The old
report was spurious here too.

Not covered: case (a) needs an endpoint with no component register block,
which QEMU's type3 always provides, and the FLR variant needs FLR support
that QEMU's type3 does not advertise (FLReset-). I also did not run a
positive control confirming that a normally committed HDM decoder is still
detected after this change. The QEMU run above predates the v6 warning:
the warning fires on the skip path that run exercises, but it has not been
observed directly.

v1->v2:
- rebase onto cxl/next
- rewrite the commit message to describe the behaviour rather than narrate
  the code change (Alison Schofield)

v2->v3:
- test cxld->commit instead of cxlhdm->regs.hdm_decoder, so that decoders
  emulated from the DVSEC ranges are also skipped when the HDM decoder
  registers exist but are globally disabled (Richard Cheng)
- update the subject and the commit message for the widened scope

v3->v4:
- shorten the commit message (Dave Jiang)

v4->v5:
- condense the commit message further, wording taken from Jonathan
  Cameron's suggestion on v4
- note the case (b) test in the commit message

v5->v6:
- rebase onto master, per Dave's request to send patches against master
  rather than cxl/next
- warn when the check is skipped, as the DVSEC range registers are
  non-sticky and a reset may have cleared the emulated decode (Dave Jiang)

[1] https://lore.kernel.org/linux-cxl/20260811113608.2815625-1-kanie@linux.alibaba.com/

 drivers/cxl/core/pci.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c
index 9d807c1a002c..03dae6b701a4 100644
--- a/drivers/cxl/core/pci.c
+++ b/drivers/cxl/core/pci.c
@@ -683,6 +683,16 @@ static int __cxl_endpoint_decoder_reset_detected(struct device *dev, void *data)
 	if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)
 		return 0;
 
+	/*
+	 * Decoders emulated from the DVSEC range registers have no commit
+	 * callback and no HDM decoder registers to consult, so a reset
+	 * that cleared the non-sticky range registers cannot be detected.
+	 */
+	if (!cxld->commit) {
+		dev_warn(dev, "DVSEC emulated decode may have been cleared by reset\n");
+		return 0;
+	}
+
 	cxlhdm = dev_get_drvdata(&port->dev);
 	hdm = cxlhdm->regs.hdm_decoder;
 	ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id));

base-commit: 50d05c7c76c96b90462f24debacca971d2e86713
-- 
2.43.7

Re: [PATCH v6] cxl/pci: Skip reset detection for DVSEC emulated decoders

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Non-atomic read-modify-write of cxld->flags in cxl_endpoint_decoder_clear_reset_flags() can race with concurrent updates.
- [High] Missing exclusion against cxl_port unbind in the reset handler can lead to a use-after-free or NULL pointer dereference of the devm-allocated cxlhdm structure.
--

commit d4b494faf63903a18d56d255741d1947191718ae
Author: Guixin Liu <kanie@linux.alibaba.com>

cxl/pci: Skip reset detection for DVSEC emulated decoders

This commit skips reset detection for HDM decoders that are emulated from
DVSEC range registers, as they lack component registers or Committed bits.
It warns when doing so since these non-sticky registers might be cleared by
a reset and the loss cannot be detected.

> diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c
> index 9d807c1a002ca..03dae6b701a43 100644
> --- a/drivers/cxl/core/pci.c
> +++ b/drivers/cxl/core/pci.c

[ ... ]

> @@ -683,6 +683,16 @@ static int __cxl_endpoint_decoder_reset_detected(struct device *dev, void *data)

[ ... ]

>  	if (!cxld->commit) {
>  		dev_warn(dev, "DVSEC emulated decode may have been cleared by reset\n");
>  		return 0;
>  	}
>  
>  	cxlhdm = dev_get_drvdata(&port->dev);
>  	hdm = cxlhdm->regs.hdm_decoder;

[Severity: High]
This isn't a bug introduced by this patch, but does the lack of exclusion
against cxl_port unbind risk a use-after-free or NULL pointer dereference
here?

If an administrator unbinds the cxl_port driver via sysfs concurrently with
an asynchronous CXL AER reset triggering cxl_reset_done(), the devm-allocated
cxlhdm structure bound to port->dev could be freed or NULLed before it is
accessed, since there appears to be no mutual exclusion.

>  	ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id));
>  
>  	return !FIELD_GET(CXL_HDM_DECODER0_CTRL_COMMITTED, ctrl);
>  }

Additionally, looking at the wider reset handling path in drivers/cxl/pci.c:

[Severity: High]
This is a pre-existing issue, but does the non-atomic modification of
cxld->flags in cxl_endpoint_decoder_clear_reset_flags() risk torn writes
when racing against concurrent updates?

drivers/cxl/pci.c:cxl_endpoint_decoder_clear_reset_flags() {
    ...
    cxled->cxld.flags &= ~CXL_DECODER_F_RESET_MASK;
    ...
}

If the asynchronous CXL AER reset handler executes concurrently with region
setup paths (driven via sysfs or driver core), both could attempt to modify
cxld->flags. If cxl_rwsem.region is not held in the reset handler, could
this result in decoder state corruption, such as losing the
CXL_DECODER_F_ENABLE or CXL_DECODER_F_LOCK flags?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260910082141.3908416-1-kanie@linux.alibaba.com?part=1