Linux CXL
 help / color / mirror / Atom feed
From: <alucerop@amd.com>
To: <linux-cxl@vger.kernel.org>, <dave.jiang@intel.com>,
	<dave@stgolabs.net>, <jic23@kernel.org>,
	<alison.schofield@intel.com>, <vishal.l.verma@intel.com>
Cc: Alejandro Lucero <alucerop@amd.com>
Subject: [RFC 3/3] Documentation/cxl: add document about Type2 binding and unbinding
Date: Wed, 18 Nov 2026 19:14:31 +0000	[thread overview]
Message-ID: <20261118191431.21781-4-alucerop@amd.com> (raw)
In-Reply-To: <20261118191431.21781-1-alucerop@amd.com>

From: Alejandro Lucero <alucerop@amd.com>

CXL subsystem relies on kernel device model and dev managed
infrastructure for helping with device unwinding. This is not trivial
as different objects are created for each CXL Type2 or Type3 device and
with Type2 relying on an external driver to the CXL core.

Explain here:

 1) the device chain creation when a Type2 device binds to its driver and
    Type2 CXL API is used for CXL initialization.

 2) the device chain unwinding and the potential starting points for it.

Signed-off-by: Alejandro Lucero <alucerop@amd.com>
---
 .../cxl/type2/cxl_type2_objects_unwinding.rst | 102 ++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 Documentation/driver-api/cxl/type2/cxl_type2_objects_unwinding.rst

diff --git a/Documentation/driver-api/cxl/type2/cxl_type2_objects_unwinding.rst b/Documentation/driver-api/cxl/type2/cxl_type2_objects_unwinding.rst
new file mode 100644
index 000000000000..873f95d9c202
--- /dev/null
+++ b/Documentation/driver-api/cxl/type2/cxl_type2_objects_unwinding.rst
@@ -0,0 +1,102 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================================
+Compute Express Link: Type2 unwinding
+=====================================
+
+The CXL subsystem relies on the linux kernel device abstraction for representing
+the different componentes building up the subsystem itself. The core or root
+devices of the subsystem are created from data coming through ACPI and are not
+discussed in this document, with the target being Type2 devices and the binding
+and unbinding of those physical devices into the CXL subsystem.
+
+Along with logical devices come CXL drivers which allow to perform this binding
+and unbinding through usual kernel probe mechanisms. Moreover, current CXL
+subsystem design relies on devres (managed resources) kernel functionality
+linking actions to device unbinding from drivers helping with all the unwinding
+required.
+
+When a device is unbound the kernel will first invoke those devm actions linked
+to the device and then, if there is no further references to the device kobject,
+it will be released or freed. This implies actions linked to the unbinding
+happen at the unbinding time but the final device release can happen at a later
+time. Furthermore, those actions can lead to other related objects to be removed
+as well as their existence depends on the current one being removed. For a Type2
+device doing CXL initialization, these are the objects created with order from
+left to right and with dependencies in the opposite direction:
+
+
+  cxl_memdev --> endpoint (cxl_port) --> cxl_decoder --> cxl_region
+
+
+A CXL region defines the Host physical range where the device memory (HDM) is
+mapped and someone, a client, will likely be using it somehow. For a Type2 this
+client is the Type2 "generic" driver, like a network driver using CXL for some
+kind of acceleration. For a Type3 the client could be the kernel memory
+management or some user space functionality using such a physical range through
+a dax device, with the driver taking over the device being the CXL core mem
+driver. Note this last driver is also used in Type2 for the cxl_memdev created,
+as next paragraph does explain.
+
+The above object chain is created when a new memdev device is added to the
+kernel device core and such a device binding to the cxl_mem driver. This will
+trigger port(s) creation, at least one, a cxl_port device which will bind to
+the cxl_port driver when added to the kernel device core. For a decoder only a
+device is created (or as many as decoders advertises the Type2 device) but no
+binding as there is no decoder driver to bind to. Finally, the region device
+will be automatically created during initialization for current Type2 devices
+expectation (this is called autodiscovery and relies on the system BIOS doing
+the proper physical HDM decoders programming, but work for supporting adhoc
+region creation is in the way) and it will bind to the cxl_region driver. Once
+there exists a driver bound to a device, it can be unbound through the sysfs
+files created by default by the kernel device core.
+
+
+Unwinding expectation
+=====================
+
+One could think a Type2 driver creating all those objects should control the
+unwinding of all of them, not necessarily because some driver action but maybe
+because the kernel signals the driver about that necessity. However, with the
+current design those objects can end up being unbound through different paths:
+
+ 1) The pci device owning the Type2 CXL "device" unbinds from its driver.
+ 2) The type2 driver is removed triggering the pci device unbinding.
+ 3) The cxl memdev is unbound from the cxl mem driver through sysfs.
+ 4) The cxl endpoint port is unbound from the cxl port driver through systfs.
+ 5) The cxl_region is unbound from the cxl region driver through sysfs.
+ 8) Root port linked to the device is unbound frmo the cxl port through sysfs.
+ 9) cxl acpi kernel module is removed.
+
+The first two cases go through the driver remove code path allowing a Type2
+aware driver to stop using the CXL region memory range while all the CXL
+objects are still there. Devm actions will start the CXL unwinding just after
+that code path is executed with the cxl_memdev object being the first one
+getting the unwinding signal, but the order here is not relevant as there is
+no client using the CXL region after the driver exit code path execution.
+
+The next cases rely on devm actions linked to those logical CXL devices created
+and depending on each of them, the order changes ... for a Type3 device. A Type2
+needs the first step in any unwinding chain to be the Type2 driver stopping
+the consumption of the HPA, so devm action are different for obtaining just
+this deterministic order in any case. This implies the code linked to some
+devm actions are explicitly invoked by the unwinding code at the right point.
+In other words, the unwinding relies on both, devm and explicit calls.
+
+
+Visualizing the different unwinding cases
+=========================================
+
+For helping with the debugging and/or understanding of current Type2, or even
+Type3 devices, unwinding some tracepoints are available for enabling them all:
+
+
+   # echo 1 > /sys/kernel/debug/tracing/events/cxl_unwinding/enable
+
+or individually as for example:
+
+   # echo 1 > /sys/kernel/debug/tracing/events/cxl_unwinding/cxl_memdev_release/enable
+
+	or
+
+   # echo "cxl_unwinding:cxl_memdev_release" > /sys/kernel/debug/tracing/set_event
-- 
2.34.1


  parent reply	other threads:[~2026-11-18 17:43 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 19:14 [RFC 0/3] Type2 unwinding ordering change alucerop
2026-11-18 19:14 ` [RFC 1/3] cxl: add tracepoints to the unwinding code path alucerop
2026-11-18 18:00   ` sashiko-bot
2026-11-18 19:14 ` [RFC 2/3] cxl: make explicit removal of endpoint and decoder for Type2 alucerop
2026-11-18 18:09   ` sashiko-bot
2026-11-18 19:14 ` alucerop [this message]
2026-11-18 18:04   ` [RFC 3/3] Documentation/cxl: add document about Type2 binding and unbinding sashiko-bot
2026-11-18 23:39 ` [RFC 0/3] Type2 unwinding ordering change Dave Jiang
2026-11-19  8:13   ` Lucero Palau, Alejandro
2026-11-19 10:00   ` Lucero Palau, Alejandro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118191431.21781-4-alucerop@amd.com \
    --to=alucerop@amd.com \
    --cc=alison.schofield@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=dave@stgolabs.net \
    --cc=jic23@kernel.org \
    --cc=linux-cxl@vger.kernel.org \
    --cc=vishal.l.verma@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox