From: Ben Cheatham <Benjamin.Cheatham@amd.com>
To: <linux-cxl@vger.kernel.org>, <dave@stgolabs.net>,
<jic23@kernel.org>, <dave.jiang@intel.com>,
<alison.schofield@intel.com>
Cc: <linux-iommu@vger.kernel.org>, <benjamin.cheatham@amd.com>,
<terry.bowman@amd.com>, <robert.richter@amd.com>
Subject: [PATCH 02/15] cxl/cache: Add cxl_cache driver
Date: Wed, 23 Sep 2026 12:33:29 -0500 [thread overview]
Message-ID: <20260923173342.5584-3-Benjamin.Cheatham@amd.com> (raw)
In-Reply-To: <20260923173342.5584-1-Benjamin.Cheatham@amd.com>
Add the cxl_cache driver. This driver will provide management functions
for common portions of CXL.cache capable endpoints, manage struct
cxl_cachedev devices, and validate the system's CXL.cache
configuration.
Add a function for getting the CXL.cache information from the CXL
device DVSEC and store it in the new struct cxl_cache_state member of
cxl_dev_state for use by endpoint drivers.
Add another set of functions to enable/disable CXL.cache on a device.
The cxl_cache driver disables CXL.cache on a device during probe until
the device's configuration can be validated. Validation of the
configuration will come in later commits, so the driver just disables
CXL.cache for now.
Signed-off-by: Ben Cheatham <Benjamin.Cheatham@amd.com>
---
drivers/cxl/Kconfig | 14 +++++++
drivers/cxl/Makefile | 6 ++-
drivers/cxl/cache.c | 61 +++++++++++++++++++++++++++
drivers/cxl/core/cachedev.c | 2 +-
drivers/cxl/core/pci.c | 78 ++++++++++++++++++++++++++++++++---
drivers/cxl/cxlcache.h | 3 ++
include/cxl/cxl.h | 27 +++++++++++-
include/uapi/linux/pci_regs.h | 4 ++
8 files changed, 186 insertions(+), 9 deletions(-)
create mode 100644 drivers/cxl/cache.c
diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
index 80aeb0d556bd..9c3d3cb6e391 100644
--- a/drivers/cxl/Kconfig
+++ b/drivers/cxl/Kconfig
@@ -243,4 +243,18 @@ config CXL_ATL
depends on CXL_REGION
depends on ACPI_PRMT && AMD_NB
+config CXL_CACHE
+ tristate "CXL: Cache Management Support"
+ depends on CXL_BUS
+ help
+ Enables a driver that manages the CXL.cache capabilities of a CXL.cache
+ capable device. This driver validates and provides support for
+ programming the CXL cache device topology. This driver is required for
+ using multiple CXL.cache devices (Type 1 or 2) below a given CXL 3.0+
+ capable PCIe Root Port. This driver only provides CXL cache management and
+ reporting capabilities, a vendor-specific device driver is expected to
+ use the device's CXL cache and enable any other capabilities.
+
+ If unsure, say 'm'.
+
endif
diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
index 2caa90fa4bf2..6539fe19c08b 100644
--- a/drivers/cxl/Makefile
+++ b/drivers/cxl/Makefile
@@ -4,18 +4,20 @@
# - 'core' first for fundamental init
# - 'port' before platform root drivers like 'acpi' so that CXL-root ports
# are immediately enabled
-# - 'mem' and 'pmem' before endpoint drivers so that memdevs are
-# immediately enabled
+# - 'mem', 'pmem', and 'cache' before endpoint drivers so that memdevs and
+# cachedevs are immediately enabled
# - 'pci' last, also mirrors the hardware enumeration hierarchy
obj-y += core/
obj-$(CONFIG_CXL_PORT) += cxl_port.o
obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
obj-$(CONFIG_CXL_PMEM) += cxl_pmem.o
obj-$(CONFIG_CXL_MEM) += cxl_mem.o
+obj-$(CONFIG_CXL_CACHE) += cxl_cache.o
obj-$(CONFIG_CXL_PCI) += cxl_pci.o
cxl_port-y := port.o
cxl_acpi-y := acpi.o
cxl_pmem-y := pmem.o security.o
cxl_mem-y := mem.o
+cxl_cache-y := cache.o
cxl_pci-y := pci.o
diff --git a/drivers/cxl/cache.c b/drivers/cxl/cache.c
new file mode 100644
index 000000000000..1815ae3a37d5
--- /dev/null
+++ b/drivers/cxl/cache.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2026 Advanced Micro Devices, Inc. */
+
+#include <cxl/cxl.h>
+
+#include "cxlcache.h"
+
+/**
+ * DOC: cxl cache
+ *
+ * The cxl_cache driver is responsible for validating the CXL.cache system
+ * configuration and providing a common management interface for the CXL cache
+ * of CXL.cache enabled devices. This driver does not discover devices; a
+ * device-specific driver is required for discovery and portions of set up.
+ */
+
+/**
+ * devm_cxl_add_cachedev - Add a CXL cache device
+ * @cxlds: CXL device state to associate with the cachedev
+ */
+struct cxl_cachedev *devm_cxl_add_cachedev(struct cxl_dev_state *cxlds)
+{
+ return __devm_cxl_add_cachedev(cxlds, NULL);
+}
+EXPORT_SYMBOL_NS_GPL(devm_cxl_add_cachedev, "CXL");
+
+static int cxl_cache_probe(struct device *dev)
+{
+ struct cxl_cachedev *cxlcd = to_cxl_cachedev(dev);
+ struct cxl_dev_state *cxlds = cxlcd->cxlds;
+ int rc;
+
+ /* Disable CXL.cache until we can validate the device configuration */
+ cxl_clear_cache_enable(cxlds);
+
+ rc = cxl_accel_read_cache_info(cxlds);
+ if (rc)
+ return rc;
+
+ return 0;
+}
+
+static struct cxl_driver cxl_cache_driver = {
+ .name = "cxl_cache",
+ .probe = cxl_cache_probe,
+ .drv = {
+ /*
+ * Needed to guarantee probe and set up order for
+ * endpoint drivers.
+ */
+ .probe_type = PROBE_FORCE_SYNCHRONOUS,
+ },
+ .id = CXL_DEVICE_ACCELERATOR,
+};
+
+module_cxl_driver(cxl_cache_driver);
+
+MODULE_DESCRIPTION("CXL: Cache Management");
+MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS("CXL");
+MODULE_ALIAS_CXL(CXL_DEVICE_ACCELERATOR);
diff --git a/drivers/cxl/core/cachedev.c b/drivers/cxl/core/cachedev.c
index 3a7a60165f62..25a24c3bd817 100644
--- a/drivers/cxl/core/cachedev.c
+++ b/drivers/cxl/core/cachedev.c
@@ -116,4 +116,4 @@ struct cxl_cachedev *__devm_cxl_add_cachedev(struct cxl_dev_state *cxlds,
return cxl_cachedev_autoremove(no_free_ptr(cxlcd));
}
-EXPORT_SYMBOL_NS_GPL(__devm_cxl_add_cachedev, "CXL");
+EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_cachedev, "cxl_cache");
diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c
index 9d807c1a002c..796c4582d2f3 100644
--- a/drivers/cxl/core/pci.c
+++ b/drivers/cxl/core/pci.c
@@ -8,6 +8,7 @@
#include <linux/pci-doe.h>
#include <cxl/pci.h>
#include <linux/aer.h>
+#include <cxlcache.h>
#include <cxlpci.h>
#include <cxlmem.h>
#include <cxl.h>
@@ -180,7 +181,8 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds)
}
EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, "CXL");
-static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val)
+static int cxl_set_protocol_enable(struct cxl_dev_state *cxlds, u16 val,
+ u16 enable_bit)
{
struct pci_dev *pdev = to_pci_dev(cxlds->dev);
int d = cxlds->cxl_dvsec;
@@ -191,9 +193,9 @@ static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val)
if (rc)
return pcibios_err_to_errno(rc);
- if ((ctrl & PCI_DVSEC_CXL_MEM_ENABLE) == val)
+ if ((ctrl & enable_bit) == val)
return 1;
- ctrl &= ~PCI_DVSEC_CXL_MEM_ENABLE;
+ ctrl &= ~enable_bit;
ctrl |= val;
rc = pci_write_config_word(pdev, d + PCI_DVSEC_CXL_CTRL, ctrl);
@@ -205,14 +207,15 @@ static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val)
static void clear_mem_enable(void *cxlds)
{
- cxl_set_mem_enable(cxlds, 0);
+ cxl_set_protocol_enable(cxlds, 0, PCI_DVSEC_CXL_MEM_ENABLE);
}
static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds)
{
int rc;
- rc = cxl_set_mem_enable(cxlds, PCI_DVSEC_CXL_MEM_ENABLE);
+ rc = cxl_set_protocol_enable(cxlds, PCI_DVSEC_CXL_MEM_ENABLE,
+ PCI_DVSEC_CXL_MEM_ENABLE);
if (rc < 0)
return rc;
if (rc > 0)
@@ -220,6 +223,31 @@ static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds)
return devm_add_action_or_reset(host, clear_mem_enable, cxlds);
}
+static void __clear_cache_enable(void *cxlds)
+{
+ cxl_set_protocol_enable(cxlds, 0, PCI_DVSEC_CXL_CACHE_ENABLE);
+}
+
+void cxl_clear_cache_enable(struct cxl_dev_state *cxlds)
+{
+ __clear_cache_enable(cxlds);
+}
+EXPORT_SYMBOL_FOR_MODULES(cxl_clear_cache_enable, "cxl_cache");
+
+int devm_cxl_enable_cache(struct device *host, struct cxl_dev_state *cxlds)
+{
+ int rc;
+
+ rc = cxl_set_protocol_enable(cxlds, PCI_DVSEC_CXL_CACHE_ENABLE,
+ PCI_DVSEC_CXL_CACHE_ENABLE);
+ if (rc < 0)
+ return rc;
+ if (rc > 0)
+ return 0;
+ return devm_add_action_or_reset(host, __clear_cache_enable, cxlds);
+}
+EXPORT_SYMBOL_FOR_MODULES(devm_cxl_enable_cache, "cxl_cache");
+
/* require dvsec ranges to be covered by a locked platform window */
static int dvsec_range_allowed(struct device *dev, const void *arg)
{
@@ -927,3 +955,43 @@ int cxl_port_get_possible_dports(struct cxl_port *port)
return ctx.count;
}
+
+int cxl_accel_read_cache_info(struct cxl_dev_state *cxlds)
+{
+ struct cxl_cache_state *cstate = &cxlds->cstate;
+ struct pci_dev *pdev = to_pci_dev(cxlds->dev);
+ int dvsec = cxlds->cxl_dvsec;
+ u16 cap, cap2;
+ u32 unit;
+ int rc;
+
+ if (!dev_is_pci(cxlds->dev))
+ return -EINVAL;
+
+ rc = pci_read_config_word(pdev, dvsec + PCI_DVSEC_CXL_CAP, &cap);
+ if (rc)
+ return pcibios_err_to_errno(rc);
+
+ if (!FIELD_GET(PCI_DVSEC_CXL_CACHE_CAPABLE, cap))
+ return -ENXIO;
+
+ rc = pci_read_config_word(pdev, dvsec + PCI_DVSEC_CXL_CAP2, &cap2);
+ if (rc)
+ return pcibios_err_to_errno(rc);
+
+ switch (FIELD_GET(PCI_DVSEC_CXL_CACHE_UNIT, cap2)) {
+ case 1:
+ unit = SZ_64K;
+ break;
+ case 2:
+ unit = SZ_1M;
+ break;
+ default:
+ return -ENXIO;
+ }
+
+ cstate->size = FIELD_GET(PCI_DVSEC_CXL_CACHE_SIZE, cap2) * unit;
+ cstate->unit = unit;
+ return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_accel_read_cache_info, "CXL");
diff --git a/drivers/cxl/cxlcache.h b/drivers/cxl/cxlcache.h
index 6ca887e078a1..be6dac93b91e 100644
--- a/drivers/cxl/cxlcache.h
+++ b/drivers/cxl/cxlcache.h
@@ -34,4 +34,7 @@ static inline struct cxl_cachedev *to_cxl_cachedev(struct device *dev)
struct cxl_cachedev *__devm_cxl_add_cachedev(struct cxl_dev_state *cxlds,
void *attach);
+int cxl_accel_read_cache_info(struct cxl_dev_state *cxlds);
+void cxl_clear_cache_enable(struct cxl_dev_state *cxlds);
+int devm_cxl_enable_cache(struct device *host, struct cxl_dev_state *cxlds);
#endif
diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
index f0077ffa0d91..29c0687e4ac2 100644
--- a/include/cxl/cxl.h
+++ b/include/cxl/cxl.h
@@ -149,6 +149,22 @@ struct cxl_dpa_partition {
#define CXL_NR_PARTITIONS_MAX 2
+/**
+ * struct cxl_cache_state - CXL cache device state for use by external drivers
+ * @size: Size of device's cache
+ * @unit: Unit of device's cache in bytes
+ */
+struct cxl_cache_state {
+ /* Public for endpoint drivers */
+
+ /*
+ * Populated by cxl_cache driver, will be overwritten on
+ * cxl_cache::probe()
+ */
+ u64 size;
+ u64 unit;
+};
+
/**
* struct cxl_dev_state - The driver device state
*
@@ -159,6 +175,7 @@ struct cxl_dpa_partition {
* @dev: The device associated with this CXL state
* @cxlmd: The device representing the CXL.mem capabilities of @dev
* @cxlcd: The device representing the CXL.cache capabilities of @dev
+ * @cstate: CXL.cache information for @dev
* @reg_map: component and ras register mapping parameters
* @regs: Parsed register blocks
* @cxl_dvsec: Offset to the PCIe device DVSEC
@@ -177,6 +194,7 @@ struct cxl_dev_state {
struct device *dev;
struct cxl_memdev *cxlmd;
struct cxl_cachedev *cxlcd;
+ struct cxl_cache_state cstate;
/* private for Type2 drivers */
struct cxl_register_map reg_map;
@@ -228,6 +246,13 @@ struct cxl_dev_state *_devm_cxl_dev_state_create(struct device *dev,
struct cxl_memdev *devm_cxl_probe_mem(struct cxl_dev_state *cxlds,
struct range *range);
-
int cxl_set_capacity(struct cxl_dev_state *cxlds, u64 capacity);
+
+#if IS_ENABLED(CONFIG_CXL_CACHE)
+struct cxl_cachedev *devm_cxl_add_cachedev(struct cxl_dev_state *cxlds);
+#else
+static inline struct cxl_cachedev *
+devm_cxl_add_cachedev(struct cxl_dev_state *cxlds)
+{ return ERR_PTR(-ENXIO); }
+#endif /* CONFIG_CXL_CACHE */
#endif /* __CXL_CXL_H__ */
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index facaa324bd86..5383b54ede96 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1353,7 +1353,11 @@
#define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2)
#define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4)
#define PCI_DVSEC_CXL_CTRL 0xC
+#define PCI_DVSEC_CXL_CACHE_ENABLE _BITUL(0)
#define PCI_DVSEC_CXL_MEM_ENABLE _BITUL(2)
+#define PCI_DVSEC_CXL_CAP2 0x16
+#define PCI_DVSEC_CXL_CACHE_UNIT __GENMASK(3, 0)
+#define PCI_DVSEC_CXL_CACHE_SIZE __GENMASK(15, 8)
#define PCI_DVSEC_CXL_RANGE_SIZE_HIGH(i) (0x18 + (i * 0x10))
#define PCI_DVSEC_CXL_RANGE_SIZE_LOW(i) (0x1C + (i * 0x10))
#define PCI_DVSEC_CXL_MEM_INFO_VALID _BITUL(0)
--
2.53.0
next prev parent reply other threads:[~2026-09-23 17:33 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-23 17:33 [PATCH 00/15] Add initial CXL.cache support Ben Cheatham
2026-09-23 17:33 ` [PATCH 01/15] cxl/core: Add CXL.cache device struct Ben Cheatham
2026-09-23 17:41 ` sashiko-bot
2026-09-23 17:33 ` Ben Cheatham [this message]
2026-09-23 17:49 ` [PATCH 02/15] cxl/cache: Add cxl_cache driver sashiko-bot
2026-09-23 17:33 ` [PATCH 03/15] cxl/core: Change cxl_ep_load() to use device pointer parameter Ben Cheatham
2026-09-23 17:33 ` [PATCH 04/15] cxl/core: Update devm_cxl_enumerate_ports() for cxl_cachedevs Ben Cheatham
2026-09-23 17:33 ` [PATCH 05/15] cxl/port: Split endpoint port probe on device type Ben Cheatham
2026-09-23 17:46 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 06/15] cxl/core: Update devm_cxl_add_endpoint() for cxl_cachedevs Ben Cheatham
2026-09-23 17:51 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 07/15] cxl/cache: Verify port hierarchy has CXL.cache enabled Ben Cheatham
2026-09-23 17:33 ` [PATCH 08/15] cxl/core, cache: Add Cache ID register probing and init Ben Cheatham
2026-09-23 17:46 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 09/15] cxl/core: Add Cache ID verification Ben Cheatham
2026-09-23 17:49 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 10/15] cxl/core: Add Cache ID allocation Ben Cheatham
2026-09-23 17:49 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 11/15] cxl/core: Add support for HDM-D cache id programming Ben Cheatham
2026-09-23 17:51 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 12/15] cxl/cache: Add snoop filter creation and set up Ben Cheatham
2026-09-23 17:50 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 13/15] cxl/cache: Add snoop filter allocation Ben Cheatham
2026-09-23 17:58 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 14/15] iommu, cxl: Configure IOMMU for CXL.cache Ben Cheatham
2026-09-23 17:57 ` sashiko-bot
2026-09-23 17:33 ` [PATCH 15/15] cxl/cache: Enable CXL.cache on successful probe Ben Cheatham
2026-09-23 17:35 ` [PATCH 00/15] Add initial CXL.cache support Cheatham, Benjamin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260923173342.5584-3-Benjamin.Cheatham@amd.com \
--to=benjamin.cheatham@amd.com \
--cc=alison.schofield@intel.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=jic23@kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-iommu@vger.kernel.org \
--cc=robert.richter@amd.com \
--cc=terry.bowman@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox