public inbox for linux-doc@vger.kernel.org
 help / color / mirror / Atom feed
From: Gregory Price <gourry@gourry.net>
To: linux-cxl@vger.kernel.org
Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	kernel-team@meta.com, dave@stgolabs.net,
	jonathan.cameron@huawei.com, dave.jiang@intel.com,
	alison.schofield@intel.com, vishal.l.verma@intel.com,
	ira.weiny@intel.com, dan.j.williams@intel.com, corbet@lwn.net
Subject: [RFC PATCH 06/17] cxl: docs/linux - overview
Date: Tue, 29 Apr 2025 20:12:13 -0400	[thread overview]
Message-ID: <20250430001224.1028656-7-gourry@gourry.net> (raw)
In-Reply-To: <20250430001224.1028656-1-gourry@gourry.net>

Add type-3 device configuration overview that explains the probe
process for a type-3 device from early-boot through memory-hotplug.

Signed-off-by: Gregory Price <gourry@gourry.net>
---
 Documentation/driver-api/cxl/index.rst        |   3 +-
 .../driver-api/cxl/linux/overview.rst         | 104 ++++++++++++++++++
 2 files changed, 106 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/driver-api/cxl/linux/overview.rst

diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
index afc66759eed2..01c0284fc273 100644
--- a/Documentation/driver-api/cxl/index.rst
+++ b/Documentation/driver-api/cxl/index.rst
@@ -32,9 +32,10 @@ that have impacts on each other.  The docs here break up configurations steps.
    platform/example-configs
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
    :caption: Linux Kernel Configuration
 
+   linux/overview
    linux/access-coordinates
 
 
diff --git a/Documentation/driver-api/cxl/linux/overview.rst b/Documentation/driver-api/cxl/linux/overview.rst
new file mode 100644
index 000000000000..33017ccb84f1
--- /dev/null
+++ b/Documentation/driver-api/cxl/linux/overview.rst
@@ -0,0 +1,104 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Overview
+########
+
+This section presents the configuration process of a CXL Type-3 memory device,
+and how it is ultimately exposed to users as either a :code:`DAX` device or
+normal memory pages via the kernel's page allocator.
+
+Portions marked with a bullet are points at which certain kernel objects
+are generated.
+
+1) Early Boot
+
+  a) BIOS, Build, and Boot Parameters
+
+    i) EFI_MEMORY_SP
+    ii) CONFIG_EFI_SOFT_RESERVE
+    iii) CONFIG_MHP_DEFAULT_ONLINE_TYPE
+    iv) nosoftreserve
+
+  b) Memory Map Creation
+
+    i) EFI Memory Map / E820 Consulted for Soft-Reserved
+
+      * CXL Memory is set aside to be handled by the CXL driver
+
+      * IO Resources are created for CFMWS entry
+
+  c) NUMA Node Creation
+
+    * ACPI CEDT and SRAT table are used to create Nodes from Proximity domains (PXM)
+
+  d) Memory Tier Creation
+
+    * A default memory_tier is created with all nodes.
+
+  e) Contiguous Memory Allocation
+
+    * Any requested CMA is allocated from Online nodes
+
+  f) Init Finishes, Drivers start probing
+
+2) ACPI and PCI Drivers
+
+  a) Detect CXL device, marking it for probe by CXL driver
+
+  b) This portion will not be covered specifically.
+
+3) CXL Driver Operation
+
+  a) Base device creation
+
+    * root, port, and memdev devices created
+    * CEDT CFMWS IO Resource creation
+
+  b) Decoder creation
+
+    * root, switch, and endpoint decoders created
+
+  c) Logical device creation
+
+    * memory_region and endpoint devices created
+
+  d) Devices are associated with each other
+
+    * If auto-decoder (BIOS-programmed decoders), driver validates
+      configurations, builds associations, and locks configs at probe time.
+
+    * If user-configured, validation and associations are built at
+      decoder-commit time.
+
+  e) Regions surfaced as DAX region
+
+    * dax_region created
+
+    * DAX device created via DAX driver
+
+4) DAX Driver Operation
+
+  a) DAX driver surfaces DAX region as one of two dax device modes
+
+    * kmem - dax device is converted to hotplug memory blocks
+
+      * DAX kmem IO resource creation
+
+    * hmem - dax device is left as daxdev to be accessed as a file.
+
+      * If hmem, journey ends here.
+
+  b) DAX kmem surfaces memory region to Memory Hotplug to add to page
+     allocator as "driver managed memory"
+
+5) Memory Hotplug
+
+  a) mhp component surfaces a dax device memory region as multiple memory
+     blocks to the page allocator
+
+    * blocks appear in :code:`/sys/bus/memory/devices` and linked to a NUMA node
+
+  b) blocks are onlined into the requested zone (NORMAL or MOVABLE)
+
+    * Memory is marked "Driver Managed" to avoid kexec from using it as region
+      for kernel updates
-- 
2.49.0


  parent reply	other threads:[~2025-04-30  0:12 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-04-30  0:12 [RFC PATCH 00/17] CXL Boot to Bash Documentation Gregory Price
2025-04-30  0:12 ` [RFC PATCH 01/17] cxl: update documentation structure in prep for new docs Gregory Price
2025-04-30  0:12 ` [RFC PATCH 02/17] cxl: docs/devices - device reference and uefi placeholder Gregory Price
2025-04-30 12:32   ` Jonathan Corbet
2025-04-30 14:08     ` Gregory Price
2025-04-30  0:12 ` [RFC PATCH 03/17] cxl: docs/platform/bios-and-efi documentation Gregory Price
2025-04-30  0:12 ` [RFC PATCH 04/17] cxl: docs/platform/acpi reference documentation Gregory Price
2025-04-30  0:12 ` [RFC PATCH 05/17] cxl: docs/platform/example-configs documentation Gregory Price
2025-04-30  0:12 ` Gregory Price [this message]
2025-04-30  0:12 ` [RFC PATCH 07/17] cxl: docs/linux - early boot configuration Gregory Price
2025-04-30  0:12 ` [RFC PATCH 08/17] cxl: docs/linux - add cxl-driver theory of operation Gregory Price
2025-04-30  0:12 ` [RFC PATCH 09/17] cxl: docs/linux/cxl-driver - add example configurations Gregory Price
2025-04-30  0:12 ` [RFC PATCH 10/17] cxl: docs/linux/dax-driver documentation Gregory Price
2025-04-30  0:12 ` [RFC PATCH 11/17] cxl: docs/linux/memory-hotplug Gregory Price
2025-04-30  0:12 ` [RFC PATCH 12/17] cxl: docs/allocation/dax Gregory Price
2025-04-30  0:12 ` [RFC PATCH 13/17] cxl: docs/allocation/page-allocator Gregory Price
2025-04-30  0:12 ` [RFC PATCH 14/17] cxl: docs/allocation/reclaim Gregory Price
2025-04-30  0:12 ` [RFC PATCH 15/17] cxl: docs/allocation/hugepages Gregory Price
2025-04-30  0:12 ` [RFC PATCH 16/17] cxl: docs/allocation/tiering Gregory Price
2025-04-30  0:12 ` [RFC PATCH 17/17] cxl: docs/use-cases Gregory Price

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250430001224.1028656-7-gourry@gourry.net \
    --to=gourry@gourry.net \
    --cc=alison.schofield@intel.com \
    --cc=corbet@lwn.net \
    --cc=dan.j.williams@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=dave@stgolabs.net \
    --cc=ira.weiny@intel.com \
    --cc=jonathan.cameron@huawei.com \
    --cc=kernel-team@meta.com \
    --cc=linux-cxl@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=vishal.l.verma@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox