Linux CXL
 help / color / mirror / Atom feed
From: <alucerop@amd.com>
To: <linux-cxl@vger.kernel.org>, <dave.jiang@intel.com>,
	<dave@stgolabs.net>, <jic23@kernel.org>,
	<alison.schofield@intel.com>, <vishal.l.verma@intel.com>
Cc: Alejandro Lucero <alucerop@amd.com>
Subject: [RFC 2/3] cxl: make explicit removal of endpoint and decoder for Type2
Date: Wed, 18 Nov 2026 19:14:30 +0000	[thread overview]
Message-ID: <20261118191431.21781-3-alucerop@amd.com> (raw)
In-Reply-To: <20261118191431.21781-1-alucerop@amd.com>

From: Alejandro Lucero <alucerop@amd.com>

Current design relies on devm for CXL devices unwinding but this is not
working well for Type2.

No unwinding should happen while the Type2 driver is still using the HPA
range, and although the driver is signalled about the unwinding, devm
locking interdependencies force that signalling to be asynchronous
leading to the driver still using the HPA while those CXL objects
building the path have already disappeared.

This is currently safe because Type2 does only support autodiscovery
regions implying the physical decoders building up the path are not
reset, but this is semantically wrong and it will create issues with the
coming support of adhoc region creation by Type2.

Remove devm actions for endpoint and decoder for a Type2 device. Invoke
removal of them explicitly for a Type2 when it is safe to do so.

Signed-off-by: Alejandro Lucero <alucerop@amd.com>
---
 drivers/cxl/core/memdev.c | 16 ++++++++++++++--
 drivers/cxl/core/port.c   | 23 +++++++++++++++++++++--
 drivers/cxl/core/region.c | 28 +++++++++++++++++++++++++---
 drivers/cxl/cxl.h         |  3 +++
 drivers/cxl/cxlmem.h      | 11 +++++++++++
 drivers/cxl/port.c        |  5 -----
 6 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
index 7333fe7594bd..c699c5561037 100644
--- a/drivers/cxl/core/memdev.c
+++ b/drivers/cxl/core/memdev.c
@@ -758,6 +758,14 @@ static void cxl_memdev_unregister(void *_cxlmd)
 	struct device *dev = &cxlmd->dev;
 
 	trace_cxl_memdev_unregister(cxlmd, "start");
+
+	/*
+	 * Type2 requires to explicitly remove endpoint while Type3 relies on
+	 * devm mechanisms.
+	 */
+	if (cxlmd->attach)
+		delete_endpoint(cxlmd);
+
 	cdev_device_del(&cxlmd->cdev, dev);
 	cxl_memdev_shutdown(dev);
 	put_device(dev);
@@ -776,10 +784,14 @@ static void detach_memdev(struct work_struct *work)
 	 * is required. In that case, @cxlmd detach escalates to parent device
 	 * detach.
 	 */
-	if (cxlmd->attach)
+	if (cxlmd->attach) {
 		device_release_driver(cxlmd->dev.parent);
-	else
+
+		/* Type2 attach region did a get. */
+		put_device(&cxlmd->endpoint->dev);
+	} else {
 		device_release_driver(&cxlmd->dev);
+	}
 
 	put_device(&cxlmd->dev);
 	trace_cxl_memdev_detach(cxlmd, "done");
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 9ccb0d185904..de109fb60038 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -1480,14 +1480,24 @@ static struct device *grandparent(struct device *dev)
 	return NULL;
 }
 
-static void delete_endpoint(void *data)
+static void cxld_unregister(void *dev);
+
+void delete_endpoint(void *data)
 {
 	struct cxl_memdev *cxlmd = data;
 	struct cxl_port *endpoint = cxlmd->endpoint;
 	struct device *host = port_to_host(endpoint);
+	struct cxl_attach_region *attach =
+		container_of(cxlmd->attach, typeof(*attach), attach);
 
 	trace_cxl_delete_endpoint(&endpoint->dev, "start");
 
+	/* Type2 requires to explicitly remove region and decoder */
+	if (attach && attach->cxlr) {
+		endpoint_unregister_region(attach->cxlr);
+		cxld_unregister(&attach->cxled->cxld.dev);
+	}
+
 	scoped_guard(device, host) {
 		if (host->driver && !endpoint->dead) {
 			devm_release_action(host, cxl_unlink_parent_dport, endpoint);
@@ -1509,7 +1519,10 @@ int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint)
 	get_device(host);
 	get_device(&endpoint->dev);
 	cxlmd->depth = endpoint->depth;
-	return devm_add_action_or_reset(dev, delete_endpoint, cxlmd);
+	/* Only Type3 relies on devm for decoder unwinding. */
+	if (cxlmd->cxlds->type == CXL_DEVTYPE_CLASSMEM)
+		return devm_add_action_or_reset(dev, delete_endpoint, cxlmd);
+	return 0;
 }
 EXPORT_SYMBOL_NS_GPL(cxl_endpoint_autoremove, "CXL");
 
@@ -2413,6 +2426,12 @@ EXPORT_SYMBOL_NS_GPL(cxl_bus_drain, "CXL");
 bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd)
 {
 	trace_schedule_cxl_memdev_detach(cxlmd);
+
+	if (cxlmd->detach_scheduled)
+		return 0;
+
+	cxlmd->detach_scheduled = true;
+
 	return queue_work(cxl_bus_wq, &cxlmd->detach_work);
 }
 EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, "CXL");
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index efbd53a37405..6a188041791c 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -2593,7 +2593,7 @@ static void unregister_region(struct cxl_region *cxlr)
 	trace_cxl_unregister_region(&cxlr->dev, "done");
 }
 
-static void endpoint_unregister_region(void *_cxlr)
+void endpoint_unregister_region(void *_cxlr)
 {
 	struct cxl_region *cxlr = _cxlr;
 	struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent);
@@ -4177,11 +4177,33 @@ int cxl_memdev_attach_region(struct cxl_memdev *cxlmd)
 	/* Only teardown regions that pass validation, ignore the rest */
 	get_device(&cxlr->dev);
 
-	rc = devm_add_action_or_reset(&endpoint->dev,
-				      endpoint_unregister_region, cxlr);
+	/*
+	 * While region in use by the Type2 device driver, no unbinding can
+	 * happen. Keeping a reference to the endpoint device avoids release.
+	 * The endpoint can be released after unwinding from the memdev forced
+	 * with the below actions.
+	 */
+	get_device(&endpoint->dev);
+
+	/*
+	 * All these next devm actions are necessary for dealing with unbind
+	 * triggered through sysfs with the CXL objects linked to a Type2.
+	 */
+	rc = devm_add_action_or_reset(&cxlr->dev, schedule_detach, cxlmd);
+	if (rc)
+		return rc;
+
+	rc = devm_add_action_or_reset(&endpoint->dev, schedule_detach, cxlmd);
+
+	if (rc)
+		return rc;
+
+	rc = devm_add_action_or_reset(&cxlmd->dev, schedule_detach, cxlmd);
 	if (rc)
 		return rc;
 
+	attach->cxlr = cxlr;
+	attach->cxled = cxled;
 	attach->hpa_range = (struct range) {
 		.start = cxlr->params.res->start,
 		.end = cxlr->params.res->end,
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index baa6df2d362a..eaa663f25699 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -942,6 +942,9 @@ void cxl_coordinates_combine(struct access_coordinate *out,
 			     struct access_coordinate *c1,
 			     struct access_coordinate *c2);
 
+void endpoint_unregister_region(void *_cxlr);
+void delete_endpoint(void *data);
+
 bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port);
 struct cxl_dport *devm_cxl_add_dport_by_dev(struct cxl_port *port,
 					    struct device *dport_dev);
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 35268d1c387c..8e9f6ff14d98 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -51,6 +51,7 @@
  * @err_rec_array: List of xarrarys to store the memdev error records to
  *		   check attributes for a memory repair operation are from
  *		   current boot.
+ * @detach_scheduled: shows if work does exist already for detaching the memdev.
  */
 struct cxl_memdev {
 	struct device dev;
@@ -66,8 +67,14 @@ struct cxl_memdev {
 	u8 scrub_cycle;
 	int scrub_region_id;
 	struct cxl_mem_err_rec *err_rec_array;
+	bool detach_scheduled;
 };
 
+static inline void schedule_detach(void *cxlmd)
+{
+	schedule_cxl_memdev_detach(cxlmd);
+}
+
 static inline struct cxl_memdev *to_cxl_memdev(struct device *dev)
 {
 	return container_of(dev, struct cxl_memdev, dev);
@@ -104,6 +111,8 @@ struct cxl_memdev_attach {
 /**
  * struct cxl_attach_region - coordinate mapping a region at memdev registration
  * @attach: common core attachment descriptor
+ * @cxlr: region attached.
+ * @cxled: endpoint decoder linked to the region.
  * @hpa_range: physical address range of the region
  *
  * For the common simple case of a CXL device with private (non-general purpose
@@ -112,6 +121,8 @@ struct cxl_memdev_attach {
  */
 struct cxl_attach_region {
 	struct cxl_memdev_attach attach;
+	struct cxl_region *cxlr;
+	struct cxl_endpoint_decoder *cxled;
 	struct range hpa_range;
 };
 
diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c
index c12fd0b89883..1b93853b0779 100644
--- a/drivers/cxl/port.c
+++ b/drivers/cxl/port.c
@@ -26,11 +26,6 @@
  * PCIe topology.
  */
 
-static void schedule_detach(void *cxlmd)
-{
-	schedule_cxl_memdev_detach(cxlmd);
-}
-
 static int discover_region(struct device *dev, void *unused)
 {
 	struct cxl_endpoint_decoder *cxled;
-- 
2.34.1


  parent reply	other threads:[~2026-11-18 17:43 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 19:14 [RFC 0/3] Type2 unwinding ordering change alucerop
2026-11-18 19:14 ` [RFC 1/3] cxl: add tracepoints to the unwinding code path alucerop
2026-11-18 18:00   ` sashiko-bot
2026-11-18 19:14 ` alucerop [this message]
2026-11-18 18:09   ` [RFC 2/3] cxl: make explicit removal of endpoint and decoder for Type2 sashiko-bot
2026-11-18 19:14 ` [RFC 3/3] Documentation/cxl: add document about Type2 binding and unbinding alucerop
2026-11-18 18:04   ` sashiko-bot
2026-11-18 23:39 ` [RFC 0/3] Type2 unwinding ordering change Dave Jiang
2026-11-19  8:13   ` Lucero Palau, Alejandro
2026-11-19 10:00   ` Lucero Palau, Alejandro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118191431.21781-3-alucerop@amd.com \
    --to=alucerop@amd.com \
    --cc=alison.schofield@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=dave@stgolabs.net \
    --cc=jic23@kernel.org \
    --cc=linux-cxl@vger.kernel.org \
    --cc=vishal.l.verma@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox