From: Gregory Price <gourry@gourry.net>
To: linux-cxl@vger.kernel.org
Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
kernel-team@meta.com, dave@stgolabs.net,
jonathan.cameron@huawei.com, dave.jiang@intel.com,
alison.schofield@intel.com, vishal.l.verma@intel.com,
ira.weiny@intel.com, dan.j.williams@intel.com, corbet@lwn.net
Subject: [RFC PATCH 06/17] cxl: docs/linux - overview
Date: Tue, 29 Apr 2025 20:12:13 -0400 [thread overview]
Message-ID: <20250430001224.1028656-7-gourry@gourry.net> (raw)
In-Reply-To: <20250430001224.1028656-1-gourry@gourry.net>
Add type-3 device configuration overview that explains the probe
process for a type-3 device from early-boot through memory-hotplug.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
Documentation/driver-api/cxl/index.rst | 3 +-
.../driver-api/cxl/linux/overview.rst | 104 ++++++++++++++++++
2 files changed, 106 insertions(+), 1 deletion(-)
create mode 100644 Documentation/driver-api/cxl/linux/overview.rst
diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
index afc66759eed2..01c0284fc273 100644
--- a/Documentation/driver-api/cxl/index.rst
+++ b/Documentation/driver-api/cxl/index.rst
@@ -32,9 +32,10 @@ that have impacts on each other. The docs here break up configurations steps.
platform/example-configs
.. toctree::
- :maxdepth: 1
+ :maxdepth: 2
:caption: Linux Kernel Configuration
+ linux/overview
linux/access-coordinates
diff --git a/Documentation/driver-api/cxl/linux/overview.rst b/Documentation/driver-api/cxl/linux/overview.rst
new file mode 100644
index 000000000000..33017ccb84f1
--- /dev/null
+++ b/Documentation/driver-api/cxl/linux/overview.rst
@@ -0,0 +1,104 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Overview
+########
+
+This section presents the configuration process of a CXL Type-3 memory device,
+and how it is ultimately exposed to users as either a :code:`DAX` device or
+normal memory pages via the kernel's page allocator.
+
+Portions marked with a bullet are points at which certain kernel objects
+are generated.
+
+1) Early Boot
+
+ a) BIOS, Build, and Boot Parameters
+
+ i) EFI_MEMORY_SP
+ ii) CONFIG_EFI_SOFT_RESERVE
+ iii) CONFIG_MHP_DEFAULT_ONLINE_TYPE
+ iv) nosoftreserve
+
+ b) Memory Map Creation
+
+ i) EFI Memory Map / E820 Consulted for Soft-Reserved
+
+ * CXL Memory is set aside to be handled by the CXL driver
+
+ * IO Resources are created for CFMWS entry
+
+ c) NUMA Node Creation
+
+ * ACPI CEDT and SRAT table are used to create Nodes from Proximity domains (PXM)
+
+ d) Memory Tier Creation
+
+ * A default memory_tier is created with all nodes.
+
+ e) Contiguous Memory Allocation
+
+ * Any requested CMA is allocated from Online nodes
+
+ f) Init Finishes, Drivers start probing
+
+2) ACPI and PCI Drivers
+
+ a) Detect CXL device, marking it for probe by CXL driver
+
+ b) This portion will not be covered specifically.
+
+3) CXL Driver Operation
+
+ a) Base device creation
+
+ * root, port, and memdev devices created
+ * CEDT CFMWS IO Resource creation
+
+ b) Decoder creation
+
+ * root, switch, and endpoint decoders created
+
+ c) Logical device creation
+
+ * memory_region and endpoint devices created
+
+ d) Devices are associated with each other
+
+ * If auto-decoder (BIOS-programmed decoders), driver validates
+ configurations, builds associations, and locks configs at probe time.
+
+ * If user-configured, validation and associations are built at
+ decoder-commit time.
+
+ e) Regions surfaced as DAX region
+
+ * dax_region created
+
+ * DAX device created via DAX driver
+
+4) DAX Driver Operation
+
+ a) DAX driver surfaces DAX region as one of two dax device modes
+
+ * kmem - dax device is converted to hotplug memory blocks
+
+ * DAX kmem IO resource creation
+
+ * hmem - dax device is left as daxdev to be accessed as a file.
+
+ * If hmem, journey ends here.
+
+ b) DAX kmem surfaces memory region to Memory Hotplug to add to page
+ allocator as "driver managed memory"
+
+5) Memory Hotplug
+
+ a) mhp component surfaces a dax device memory region as multiple memory
+ blocks to the page allocator
+
+ * blocks appear in :code:`/sys/bus/memory/devices` and linked to a NUMA node
+
+ b) blocks are onlined into the requested zone (NORMAL or MOVABLE)
+
+ * Memory is marked "Driver Managed" to avoid kexec from using it as region
+ for kernel updates
--
2.49.0
next prev parent reply other threads:[~2025-04-30 0:12 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-30 0:12 [RFC PATCH 00/17] CXL Boot to Bash Documentation Gregory Price
2025-04-30 0:12 ` [RFC PATCH 01/17] cxl: update documentation structure in prep for new docs Gregory Price
2025-04-30 0:12 ` [RFC PATCH 02/17] cxl: docs/devices - device reference and uefi placeholder Gregory Price
2025-04-30 12:32 ` Jonathan Corbet
2025-04-30 14:08 ` Gregory Price
2025-04-30 0:12 ` [RFC PATCH 03/17] cxl: docs/platform/bios-and-efi documentation Gregory Price
2025-04-30 0:12 ` [RFC PATCH 04/17] cxl: docs/platform/acpi reference documentation Gregory Price
2025-04-30 0:12 ` [RFC PATCH 05/17] cxl: docs/platform/example-configs documentation Gregory Price
2025-04-30 0:12 ` Gregory Price [this message]
2025-04-30 0:12 ` [RFC PATCH 07/17] cxl: docs/linux - early boot configuration Gregory Price
2025-04-30 0:12 ` [RFC PATCH 08/17] cxl: docs/linux - add cxl-driver theory of operation Gregory Price
2025-04-30 0:12 ` [RFC PATCH 09/17] cxl: docs/linux/cxl-driver - add example configurations Gregory Price
2025-04-30 0:12 ` [RFC PATCH 10/17] cxl: docs/linux/dax-driver documentation Gregory Price
2025-04-30 0:12 ` [RFC PATCH 11/17] cxl: docs/linux/memory-hotplug Gregory Price
2025-04-30 0:12 ` [RFC PATCH 12/17] cxl: docs/allocation/dax Gregory Price
2025-04-30 0:12 ` [RFC PATCH 13/17] cxl: docs/allocation/page-allocator Gregory Price
2025-04-30 0:12 ` [RFC PATCH 14/17] cxl: docs/allocation/reclaim Gregory Price
2025-04-30 0:12 ` [RFC PATCH 15/17] cxl: docs/allocation/hugepages Gregory Price
2025-04-30 0:12 ` [RFC PATCH 16/17] cxl: docs/allocation/tiering Gregory Price
2025-04-30 0:12 ` [RFC PATCH 17/17] cxl: docs/use-cases Gregory Price
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250430001224.1028656-7-gourry@gourry.net \
--to=gourry@gourry.net \
--cc=alison.schofield@intel.com \
--cc=corbet@lwn.net \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=ira.weiny@intel.com \
--cc=jonathan.cameron@huawei.com \
--cc=kernel-team@meta.com \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox