[PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Neeraj Kumar]

Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
Moved pmem_region related code from core/region.c to core/pmem_region.c
For region label update, need to create device attribute, which calls
nvdimm exported function thus making pmem_region dependent on libnvdimm.
Because of this dependency of pmem region on libnvdimm, segregated pmem
region related code from core/region.c

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/Kconfig            |  14 +++
 drivers/cxl/core/Makefile      |   1 +
 drivers/cxl/core/core.h        |   8 +-
 drivers/cxl/core/pmem_region.c | 203 +++++++++++++++++++++++++++++++++
 drivers/cxl/core/port.c        |   2 +-
 drivers/cxl/core/region.c      | 191 +------------------------------
 drivers/cxl/cxl.h              |  30 +++--
 tools/testing/cxl/Kbuild       |   1 +
 8 files changed, 250 insertions(+), 200 deletions(-)
 create mode 100644 drivers/cxl/core/pmem_region.c

diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
index 48b7314afdb8..532eaa1bbdd6 100644
--- a/drivers/cxl/Kconfig
+++ b/drivers/cxl/Kconfig
@@ -211,6 +211,20 @@ config CXL_REGION
 
 	  If unsure say 'y'
 
+config CXL_PMEM_REGION
+	bool "CXL: Pmem Region Support"
+	default CXL_BUS
+	depends on CXL_REGION
+	depends on PHYS_ADDR_T_64BIT
+	depends on BLK_DEV
+	select LIBNVDIMM
+	help
+	  Enable the CXL core to enumerate and provision CXL pmem regions.
+	  A CXL pmem region need to update region label into LSA. For LSA
+	  updation/deletion libnvdimm is required.
+
+	  If unsure say 'y'
+
 config CXL_REGION_INVALIDATION_TEST
 	bool "CXL: Region Cache Management Bypass (TEST)"
 	depends on CXL_REGION
diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile
index 5ad8fef210b5..399157beb917 100644
--- a/drivers/cxl/core/Makefile
+++ b/drivers/cxl/core/Makefile
@@ -17,6 +17,7 @@ cxl_core-y += cdat.o
 cxl_core-y += ras.o
 cxl_core-$(CONFIG_TRACING) += trace.o
 cxl_core-$(CONFIG_CXL_REGION) += region.o
+cxl_core-$(CONFIG_CXL_PMEM_REGION) += pmem_region.o
 cxl_core-$(CONFIG_CXL_MCE) += mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 5707cd60a8eb..536636a752dc 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -34,7 +34,6 @@ int cxl_decoder_detach(struct cxl_region *cxlr,
 #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr)
 #define CXL_REGION_TYPE(x) (&cxl_region_type)
 #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr),
-#define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type)
 #define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type)
 int cxl_region_init(void);
 void cxl_region_exit(void);
@@ -74,10 +73,15 @@ static inline void cxl_region_exit(void)
 #define CXL_REGION_ATTR(x) NULL
 #define CXL_REGION_TYPE(x) NULL
 #define SET_CXL_REGION_ATTR(x)
-#define CXL_PMEM_REGION_TYPE(x) NULL
 #define CXL_DAX_REGION_TYPE(x) NULL
 #endif
 
+#ifdef CONFIG_CXL_PMEM_REGION
+#define CXL_PMEM_REGION_TYPE (&cxl_pmem_region_type)
+#else
+#define CXL_PMEM_REGION_TYPE NULL
+#endif
+
 struct cxl_send_command;
 struct cxl_mem_query_commands;
 int cxl_query_cmd(struct cxl_mailbox *cxl_mbox,
diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
new file mode 100644
index 000000000000..55b80d587403
--- /dev/null
+++ b/drivers/cxl/core/pmem_region.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2020 Intel Corporation. */
+#include <linux/device.h>
+#include <linux/memregion.h>
+#include <cxlmem.h>
+#include <cxl.h>
+#include "core.h"
+
+/**
+ * DOC: cxl pmem region
+ *
+ * The core CXL PMEM region infrastructure supports persistent memory
+ * region creation using LIBNVDIMM subsystem. It has dependency on
+ * LIBNVDIMM, pmem region need updation of cxl region information into
+ * LSA. LIBNVDIMM dependency is only for pmem region, it is therefore
+ * need this separate file.
+ */
+
+bool is_cxl_pmem_region(struct device *dev)
+{
+	return dev->type == &cxl_pmem_region_type;
+}
+EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
+
+struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
+{
+	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
+			  "not a cxl_pmem_region device\n"))
+		return NULL;
+	return container_of(dev, struct cxl_pmem_region, dev);
+}
+EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
+
+static void cxl_pmem_region_release(struct device *dev)
+{
+	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
+	int i;
+
+	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
+		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
+
+		put_device(&cxlmd->dev);
+	}
+
+	kfree(cxlr_pmem);
+}
+
+static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
+	&cxl_base_attribute_group,
+	NULL,
+};
+
+const struct device_type cxl_pmem_region_type = {
+	.name = "cxl_pmem_region",
+	.release = cxl_pmem_region_release,
+	.groups = cxl_pmem_region_attribute_groups,
+};
+
+static struct lock_class_key cxl_pmem_region_key;
+
+static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
+{
+	struct cxl_region_params *p = &cxlr->params;
+	struct cxl_nvdimm_bridge *cxl_nvb;
+	struct device *dev;
+	int i;
+
+	guard(rwsem_read)(&cxl_rwsem.region);
+	if (p->state != CXL_CONFIG_COMMIT)
+		return -ENXIO;
+
+	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
+		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets),
+			GFP_KERNEL);
+	if (!cxlr_pmem)
+		return -ENOMEM;
+
+	cxlr_pmem->hpa_range.start = p->res->start;
+	cxlr_pmem->hpa_range.end = p->res->end;
+
+	/* Snapshot the region configuration underneath the cxl_region_rwsem */
+	cxlr_pmem->nr_mappings = p->nr_targets;
+	for (i = 0; i < p->nr_targets; i++) {
+		struct cxl_endpoint_decoder *cxled = p->targets[i];
+		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
+		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
+
+		/*
+		 * Regions never span CXL root devices, so by definition the
+		 * bridge for one device is the same for all.
+		 */
+		if (i == 0) {
+			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
+			if (!cxl_nvb)
+				return -ENODEV;
+			cxlr->cxl_nvb = cxl_nvb;
+		}
+		m->cxlmd = cxlmd;
+		get_device(&cxlmd->dev);
+		m->start = cxled->dpa_res->start;
+		m->size = resource_size(cxled->dpa_res);
+		m->position = i;
+	}
+
+	dev = &cxlr_pmem->dev;
+	device_initialize(dev);
+	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
+	device_set_pm_not_required(dev);
+	dev->parent = &cxlr->dev;
+	dev->bus = &cxl_bus_type;
+	dev->type = &cxl_pmem_region_type;
+	cxlr_pmem->cxlr = cxlr;
+	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
+
+	return 0;
+}
+
+static void cxlr_pmem_unregister(void *_cxlr_pmem)
+{
+	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
+	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
+
+	/*
+	 * Either the bridge is in ->remove() context under the device_lock(),
+	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
+	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
+	 * lock).
+	 */
+	device_lock_assert(&cxl_nvb->dev);
+	cxlr->cxlr_pmem = NULL;
+	cxlr_pmem->cxlr = NULL;
+	device_unregister(&cxlr_pmem->dev);
+}
+
+static void cxlr_release_nvdimm(void *_cxlr)
+{
+	struct cxl_region *cxlr = _cxlr;
+	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
+
+	scoped_guard(device, &cxl_nvb->dev) {
+		if (cxlr->cxlr_pmem)
+			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
+					    cxlr->cxlr_pmem);
+	}
+	cxlr->cxl_nvb = NULL;
+	put_device(&cxl_nvb->dev);
+}
+
+/**
+ * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
+ * @cxlr: parent CXL region for this pmem region bridge device
+ *
+ * Return: 0 on success negative error code on failure.
+ */
+int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
+{
+	struct cxl_pmem_region *cxlr_pmem;
+	struct cxl_nvdimm_bridge *cxl_nvb;
+	struct device *dev;
+	int rc;
+
+	rc = cxl_pmem_region_alloc(cxlr);
+	if (rc)
+		return rc;
+	cxlr_pmem = cxlr->cxlr_pmem;
+	cxl_nvb = cxlr->cxl_nvb;
+
+	dev = &cxlr_pmem->dev;
+	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
+	if (rc)
+		goto err;
+
+	rc = device_add(dev);
+	if (rc)
+		goto err;
+
+	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
+		dev_name(dev));
+
+	scoped_guard(device, &cxl_nvb->dev) {
+		if (cxl_nvb->dev.driver)
+			rc = devm_add_action_or_reset(&cxl_nvb->dev,
+						      cxlr_pmem_unregister,
+						      cxlr_pmem);
+		else
+			rc = -ENXIO;
+	}
+
+	if (rc)
+		goto err_bridge;
+
+	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
+	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
+
+err:
+	put_device(dev);
+err_bridge:
+	put_device(&cxl_nvb->dev);
+	cxlr->cxl_nvb = NULL;
+	return rc;
+}
+EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 647d9ce32b64..717de1d3f70e 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
 		return CXL_DEVICE_NVDIMM_BRIDGE;
 	if (dev->type == &cxl_nvdimm_type)
 		return CXL_DEVICE_NVDIMM;
-	if (dev->type == CXL_PMEM_REGION_TYPE())
+	if (dev->type == CXL_PMEM_REGION_TYPE)
 		return CXL_DEVICE_PMEM_REGION;
 	if (dev->type == CXL_DAX_REGION_TYPE())
 		return CXL_DEVICE_DAX_REGION;
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index d5c227ce7b09..d2ef7fcc19fe 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -38,8 +38,6 @@
  */
 static nodemask_t nodemask_region_seen = NODE_MASK_NONE;
 
-static struct cxl_region *to_cxl_region(struct device *dev);
-
 #define __ACCESS_ATTR_RO(_level, _name) {				\
 	.attr	= { .name = __stringify(_name), .mode = 0444 },		\
 	.show	= _name##_access##_level##_show,			\
@@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
 }
 EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
 
-static struct cxl_region *to_cxl_region(struct device *dev)
+struct cxl_region *to_cxl_region(struct device *dev)
 {
 	if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
 			  "not a cxl_region device\n"))
@@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
 
 	return container_of(dev, struct cxl_region, dev);
 }
+EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");
 
 static void unregister_region(void *_cxlr)
 {
@@ -2814,46 +2813,6 @@ static ssize_t delete_region_store(struct device *dev,
 }
 DEVICE_ATTR_WO(delete_region);
 
-static void cxl_pmem_region_release(struct device *dev)
-{
-	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
-	int i;
-
-	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
-		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
-
-		put_device(&cxlmd->dev);
-	}
-
-	kfree(cxlr_pmem);
-}
-
-static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
-	&cxl_base_attribute_group,
-	NULL,
-};
-
-const struct device_type cxl_pmem_region_type = {
-	.name = "cxl_pmem_region",
-	.release = cxl_pmem_region_release,
-	.groups = cxl_pmem_region_attribute_groups,
-};
-
-bool is_cxl_pmem_region(struct device *dev)
-{
-	return dev->type == &cxl_pmem_region_type;
-}
-EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
-
-struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
-{
-	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
-			  "not a cxl_pmem_region device\n"))
-		return NULL;
-	return container_of(dev, struct cxl_pmem_region, dev);
-}
-EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
-
 struct cxl_poison_context {
 	struct cxl_port *port;
 	int part;
@@ -3213,64 +3172,6 @@ static int region_offset_to_dpa_result(struct cxl_region *cxlr, u64 offset,
 	return -ENXIO;
 }
 
-static struct lock_class_key cxl_pmem_region_key;
-
-static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
-{
-	struct cxl_region_params *p = &cxlr->params;
-	struct cxl_nvdimm_bridge *cxl_nvb;
-	struct device *dev;
-	int i;
-
-	guard(rwsem_read)(&cxl_rwsem.region);
-	if (p->state != CXL_CONFIG_COMMIT)
-		return -ENXIO;
-
-	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
-		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets), GFP_KERNEL);
-	if (!cxlr_pmem)
-		return -ENOMEM;
-
-	cxlr_pmem->hpa_range.start = p->res->start;
-	cxlr_pmem->hpa_range.end = p->res->end;
-
-	/* Snapshot the region configuration underneath the cxl_rwsem.region */
-	cxlr_pmem->nr_mappings = p->nr_targets;
-	for (i = 0; i < p->nr_targets; i++) {
-		struct cxl_endpoint_decoder *cxled = p->targets[i];
-		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
-		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
-
-		/*
-		 * Regions never span CXL root devices, so by definition the
-		 * bridge for one device is the same for all.
-		 */
-		if (i == 0) {
-			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
-			if (!cxl_nvb)
-				return -ENODEV;
-			cxlr->cxl_nvb = cxl_nvb;
-		}
-		m->cxlmd = cxlmd;
-		get_device(&cxlmd->dev);
-		m->start = cxled->dpa_res->start;
-		m->size = resource_size(cxled->dpa_res);
-		m->position = i;
-	}
-
-	dev = &cxlr_pmem->dev;
-	device_initialize(dev);
-	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
-	device_set_pm_not_required(dev);
-	dev->parent = &cxlr->dev;
-	dev->bus = &cxl_bus_type;
-	dev->type = &cxl_pmem_region_type;
-	cxlr_pmem->cxlr = cxlr;
-	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
-
-	return 0;
-}
-
 static void cxl_dax_region_release(struct device *dev)
 {
 	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
@@ -3334,92 +3235,6 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	return cxlr_dax;
 }
 
-static void cxlr_pmem_unregister(void *_cxlr_pmem)
-{
-	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
-	struct cxl_region *cxlr = cxlr_pmem->cxlr;
-	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
-
-	/*
-	 * Either the bridge is in ->remove() context under the device_lock(),
-	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
-	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
-	 * lock).
-	 */
-	device_lock_assert(&cxl_nvb->dev);
-	cxlr->cxlr_pmem = NULL;
-	cxlr_pmem->cxlr = NULL;
-	device_unregister(&cxlr_pmem->dev);
-}
-
-static void cxlr_release_nvdimm(void *_cxlr)
-{
-	struct cxl_region *cxlr = _cxlr;
-	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
-
-	scoped_guard(device, &cxl_nvb->dev) {
-		if (cxlr->cxlr_pmem)
-			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
-					    cxlr->cxlr_pmem);
-	}
-	cxlr->cxl_nvb = NULL;
-	put_device(&cxl_nvb->dev);
-}
-
-/**
- * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
- * @cxlr: parent CXL region for this pmem region bridge device
- *
- * Return: 0 on success negative error code on failure.
- */
-static int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
-{
-	struct cxl_pmem_region *cxlr_pmem;
-	struct cxl_nvdimm_bridge *cxl_nvb;
-	struct device *dev;
-	int rc;
-
-	rc = cxl_pmem_region_alloc(cxlr);
-	if (rc)
-		return rc;
-	cxlr_pmem = cxlr->cxlr_pmem;
-	cxl_nvb = cxlr->cxl_nvb;
-
-	dev = &cxlr_pmem->dev;
-	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
-	if (rc)
-		goto err;
-
-	rc = device_add(dev);
-	if (rc)
-		goto err;
-
-	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
-		dev_name(dev));
-
-	scoped_guard(device, &cxl_nvb->dev) {
-		if (cxl_nvb->dev.driver)
-			rc = devm_add_action_or_reset(&cxl_nvb->dev,
-						      cxlr_pmem_unregister,
-						      cxlr_pmem);
-		else
-			rc = -ENXIO;
-	}
-
-	if (rc)
-		goto err_bridge;
-
-	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
-	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
-
-err:
-	put_device(dev);
-err_bridge:
-	put_device(&cxl_nvb->dev);
-	cxlr->cxl_nvb = NULL;
-	return rc;
-}
-
 static void cxlr_dax_unregister(void *_cxlr_dax)
 {
 	struct cxl_dax_region *cxlr_dax = _cxlr_dax;
@@ -3985,6 +3800,8 @@ static int cxl_region_probe(struct device *dev)
 			dev_dbg(&cxlr->dev, "CXL EDAC registration for region_id=%d failed\n",
 				cxlr->id);
 
+		if (!IS_ENABLED(CONFIG_CXL_PMEM_REGION))
+			return -EINVAL;
 		return devm_cxl_add_pmem_region(cxlr);
 	case CXL_PARTMODE_RAM:
 		rc = devm_cxl_region_edac_register(cxlr);
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 1eb1aca7c69f..0d576b359de6 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -825,6 +825,7 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,
 			struct cxl_endpoint_dvsec_info *info);
 
 bool is_cxl_region(struct device *dev);
+struct cxl_region *to_cxl_region(struct device *dev);
 
 extern const struct bus_type cxl_bus_type;
 
@@ -869,8 +870,6 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port);
 struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port);
 
 #ifdef CONFIG_CXL_REGION
-bool is_cxl_pmem_region(struct device *dev);
-struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
 int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
 struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
 u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
@@ -880,14 +879,6 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
 				     struct cxl_pmem_region_params *params,
 				     struct cxl_decoder *cxld);
 #else
-static inline bool is_cxl_pmem_region(struct device *dev)
-{
-	return false;
-}
-static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
-{
-	return NULL;
-}
 static inline int cxl_add_to_region(struct cxl_endpoint_decoder *cxled)
 {
 	return 0;
@@ -914,6 +905,25 @@ cxl_create_region(struct cxl_root_decoder *cxlrd,
 }
 #endif
 
+#ifdef CONFIG_CXL_PMEM_REGION
+bool is_cxl_pmem_region(struct device *dev);
+struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
+int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
+#else
+static inline bool is_cxl_pmem_region(struct device *dev)
+{
+	return false;
+}
+static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
+{
+	return NULL;
+}
+static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
+{
+	return 0;
+}
+#endif
+
 void cxl_endpoint_parse_cdat(struct cxl_port *port);
 void cxl_switch_parse_cdat(struct cxl_port *port);
 
diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild
index d07f14cb7aa4..cf58ada337b7 100644
--- a/tools/testing/cxl/Kbuild
+++ b/tools/testing/cxl/Kbuild
@@ -64,6 +64,7 @@ cxl_core-y += $(CXL_CORE_SRC)/cdat.o
 cxl_core-y += $(CXL_CORE_SRC)/ras.o
 cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
 cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o
+cxl_core-$(CONFIG_CXL_PMEM_REGION) += $(CXL_CORE_SRC)/pmem_region.o
 cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o
-- 
2.34.1

[PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Neeraj Kumar]

Introduction:
=============
CXL Persistent Memory (Pmem) devices region, namespace and content must be
persistent across system reboot. In order to achieve this persistency, it
uses Label Storage Area (LSA) to store respective metadata. During system
reboot, stored metadata in LSA is used to bring back the region, namespace
and content of CXL device in its previous state.
CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox
commands to access the LSA area. nvdimm driver is using same commands to
get/set LSA data.

There are three types of LSA format and these are part of following
specifications: 
 - v1.1: https://pmem.io/documents/NVDIMM_Namespace_Spec.pdf
 - v1.2: https://uefi.org/sites/default/files/resources/UEFI_Spec_2_7.pdf
 - v2.1: https://computeexpresslink.org/wp-content/uploads/2024/02/CXL-2.0-Specification.pdf

Basic differences between these LSA formats:
 - v1.1: Support Namespace persistency. Size of Namespace Label is 128 bytes
 - v1.2: Support Namespace persistency. Size of Namespace Label is 256 bytes
 - v2.1: Support Namespace and Region persistency. Size of Namespace and
   Region Label is 256 bytes.

Linux nvdimm driver supports only v1.1 and v1.2 LSA format. CXL pmem device
require support of LSA v2.1 format for region and namespace persistency.
Initial support of LSA 2.1 was add in [1].

This patchset adds support of LSA 2.1 in nvdimm and cxl pmem driver.

Patch 1:     Introduce NDD_REGION_LABELING flag and Update cxl label index as per v2.1
Patch 2-4:   Update namespace label as per v2.1 along with code optimization
Patch 5-12:  Introduce cxl region labels and modify existing change accordingly
Patch 13:    Refactor cxl pmem region auto assembly
Patch 14-18: Save cxl region info in LSA and region recreation during reboot
Patch 19:    Segregate out cxl pmem region code from region.c to pmem_region.c
Patch 20:    Introduce cxl region addition/deletion attributes


Testing:
========
In order to test this patchset, I also added the support of LSA v2.1 format
in ndctl. ndctl changes are available at [2]. After review, I’ll push in
ndctl repo for community review.

1. Used Qemu using following CXL topology
   M2="-object memory-backend-file,id=cxl-mem1,share=on,mem-path=$TMP_DIR/cxltest.raw,size=512M \
       -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=$TMP_DIR/lsa.raw,size=1M \
       -object memory-backend-file,id=cxl-mem2,share=on,mem-path=$TMP_DIR/cxltest2.raw,size=512M \
       -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=$TMP_DIR/lsa2.raw,size=1M \
       -device pxb-cxl,bus_nr=10,bus=pcie.0,id=cxl.1 \
       -device cxl-rp,port=1,bus=cxl.1,id=root_port11,chassis=0,slot=1 \
       -device cxl-type3,bus=root_port11,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,sn=1 \
       -device cxl-rp,port=2,bus=cxl.1,id=root_port12,chassis=0,slot=2 \
       -device cxl-type3,bus=root_port12,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,sn=2 \
       -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=8k"

2. Create cxl region on both devices
	cxl create-region -d decoder0.0 -m mem0
	cxl create-region -d decoder0.0 -m mem1

	root@QEMUCXL6060pmem:~# cxl list
	[
	  {
	    "memdevs":[
	      {
	        "memdev":"mem0",
	        "pmem_size":536870912,
	        "serial":2,
	        "host":"0000:0c:00.0",
	        "firmware_version":"BWFW VERSION 00"
	      },
	      {
	        "memdev":"mem1",
	        "pmem_size":536870912,
	        "serial":1,
	        "host":"0000:0b:00.0",
	        "firmware_version":"BWFW VERSION 00"
	      }
	    ]
	  },
	  {
	    "regions":[
	      {
	        "region":"region0",
	        "resource":45365592064,
	        "size":536870912,
	        "type":"pmem",
	        "interleave_ways":1,
	        "interleave_granularity":256,
	        "decode_state":"commit",
	        "qos_class_mismatch":true
	      },
	      {
	        "region":"region1",
	        "resource":45902462976,
	        "size":536870912,
	        "type":"pmem",
	        "interleave_ways":1,
	        "interleave_granularity":256,
	        "decode_state":"commit",
	        "qos_class_mismatch":true
	      }
	    ]
	  }
	]

3. Re-Start Qemu and we could see cxl region persistency using "cxl list"

4. Create namespace for both regions
	root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=128M
	{
	  "dev":"namespace0.0",
	  "mode":"fsdax",
	  "map":"dev",
	  "size":"124.00 MiB (130.02 MB)",
	  "uuid":"8a125dcb-f992-406d-b3ad-be82fc518f05",
	  "sector_size":512,
	  "align":2097152,
	  "blockdev":"pmem0"
	}
	
	real    0m31.866s
	user    0m0.183s
	sys     0m14.855s

	root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=256M
	{
	  "dev":"namespace1.0",
	  "mode":"fsdax",
	  "map":"dev",
	  "size":"250.00 MiB (262.14 MB)",
	  "uuid":"8e16d950-c11d-4253-94a0-5b2928926433",
	  "sector_size":512,
	  "align":2097152,
	  "blockdev":"pmem1"
	}
	
	real    0m44.359s
	user    0m0.196s
	sys     0m26.768s

	root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=256M
	{
	  "dev":"namespace0.1",
	  "mode":"fsdax",
	  "map":"dev",
	  "size":"250.00 MiB (262.14 MB)",
	  "uuid":"f3170bfe-548a-4ce4-ae00-145a24e70426",
	  "sector_size":512,
	  "align":2097152,
	  "blockdev":"pmem0.1"
	}
	
	real    0m44.004s
	user    0m0.220s
	sys     0m25.711s

	root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=128M
	{
	  "dev":"namespace1.1",
	  "mode":"fsdax",
	  "map":"dev",
	  "size":"124.00 MiB (130.02 MB)",
	  "uuid":"6318d2d9-bc78-4896-84f9-6c18c3a8f58c",
	  "sector_size":512,
	  "align":2097152,
	  "blockdev":"pmem1.1"
	}
	
	real    0m36.612s
	user    0m0.225s
	sys     0m16.457s

	root@QEMUCXL6060pmem:~# ndctl list
	[
	  {
	    "dev":"namespace1.0",
	    "mode":"fsdax",
	    "map":"dev",
	    "size":262144000,
	    "uuid":"e07564eb-6653-4d67-ab07-434c22474001",
	    "sector_size":512,
	    "align":2097152,
	    "blockdev":"pmem1"
	  },
	  {
	    "dev":"namespace1.1",
	    "mode":"fsdax",
	    "map":"dev",
	    "size":130023424,
	    "uuid":"27dfd65a-c428-426a-8316-f340a59e0671",
	    "sector_size":512,
	    "align":2097152,
	    "blockdev":"pmem1.1"
	  },
	  {
	    "dev":"namespace0.1",
	    "mode":"fsdax",
	    "map":"dev",
	    "size":130023424,
	    "uuid":"689b4135-668d-4885-b138-b86f27d7602f",
	    "sector_size":512,
	    "align":2097152,
	    "blockdev":"pmem0.1"
	  },
	  {
	    "dev":"namespace0.0",
	    "mode":"fsdax",
	    "map":"dev",
	    "size":262144000,
	    "uuid":"ed15c67a-842d-4d5d-8996-be3aa24985e4",
	    "sector_size":512,
	    "align":2097152,
	    "blockdev":"pmem0"
	  }
	]

5. Re-Start Qemu and we could see
	- Region persistency using "cxl list"
	- Namespace persistency using "ndctl list" and cat /proc/iomem

	root@QEMUCXL6060pmem:~# cat /proc/iomem
	
	a90000000-b8fffffff : CXL Window 0
	  a90000000-aafffffff : Persistent Memory
	    a90000000-aafffffff : region0
	      a90000000-a97ffffff : namespace0.0
	      a98000000-aa7ffffff : namespace0.1
	  ab0000000-acfffffff : Persistent Memory
	    ab0000000-acfffffff : region1
	      ab0000000-abfffffff : namespace1.0
	      ac0000000-ac7ffffff : namespace1.1
	

	- NOTE: We can see some lag in restart, Its WIP

6. Also verify LSA version using "ndctl read-labels -j nmem0"
	root@QEMUCXL6060pmem:~# ndctl read-labels -j nmem0
	{
	  "dev":"nmem0",
	  "index":[
	    {
	      "signature":"NAMESPACE_INDEX",
	      "major":2,
	      "minor":1,
	      "labelsize":256,
	      "seq":2,
	      "nslot":4090
	    },
	    {
	      "signature":"NAMESPACE_INDEX",
	      "major":2,
	      "minor":1,
	      "labelsize":256,
	      "seq":1,
	      "nslot":4090
	    }
	  ],
	  "label":[
	    {
	      "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c",
	      "uuid":"cbb3fe1e-9345-4ae7-95ca-2638db27ee7d",
	      "name":"",
	      "flags":8,
	      "nrange":1,
	      "position":0,
	      "dpa":134217728,
	      "rawsize":268435456,
	      "slot":0,
	      "align":0,
	      "region_uuid":"50d806c8-fd11-49eb-b19e-77fc67f6364b",
	      "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225",
	      "lbasize":512
	    },
	    {
	      "type":"529d7c61-da07-47c4-a93f-ecdf2c06f444",
	      "uuid":"50d806c8-fd11-49eb-b19e-77fc67f6364b",
	      "flags":0,
	      "nlabel":1,
	      "position":0,
	      "dpa":0,
	      "rawsize":536870912,
	      "hpa":45365592064,
	      "slot":1,
	      "interleave granularity":256,
	      "align":0
	    },
	    {
	      "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c",
	      "uuid":"d9ef8d35-ef84-4111-b302-bce53c94a2ad",
	      "name":"",
	      "flags":0,
	      "nrange":1,
	      "position":0,
	      "dpa":0,
	      "rawsize":134217728,
	      "slot":2,
	      "align":0,
	      "region_uuid":"50d806c8-fd11-49eb-b19e-77fc67f6364b",
	      "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225",
	      "lbasize":512
	    },
	    {
	      "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c",
	      "uuid":"cbb3fe1e-9345-4ae7-95ca-2638db27ee7d",
	      "name":"",
	      "flags":0,
	      "nrange":1,
	      "position":0,
	      "dpa":134217728,
	      "rawsize":268435456,
	      "slot":3,
	      "align":0,
	      "region_uuid":"50d806c8-fd11-49eb-b19e-77fc67f6364b",
	      "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225",
	      "lbasize":512
	    }
	  ]
	}
	read 1 nmem

	- NOTE: We have following UUID types as per CXL Spec
		"type":"529d7c61-da07-47c4-a93f-ecdf2c06f444" is region label
		"type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c" is namespace label


Limitation (TODO):
==================
Current changes only support interleave way == 1


Observation:
============
First time namespace creation using ndctl takes around 10 to 20 second time
while executing "devm_memremap_pages" at [3]

As using this patchset, after auto region creation, namespace creation is
happening in boot sequence (if nvdimm and cxl drivers are static), It is
therefore boot sequence is increased by around 10 to 20 sec.

Changes
=======
Changes from v2->v3
-------------------
- Find v2 link at [6]
[MISC Changes]
- Drop v2 prep patch 02/20 to avoid renaming noise [Jonathan/Dave/Ira]
- Introduce v3 patch 03/20 to fix nd_label_base() signature [Dave]
- Introduce v3 patch 04/20 to update mutex_lock() with guard(mutex)() [Ira]
- Re-arrange v2 patch 04/20 to v3 patch 02/20
- Re-arrange v2 patch 03/20 to v3 patch 05/20
- Re-arrange v2 patch 12/20 to v3 patch 10/20
[PATCH 01/20]
- Rename NDD_CXL_LABEL to NDD_REGION_LABELING [Jonathan/Dave]
[PATCH 02/20]
- Elaborate comment message [Dave]
- Add Jonathan RB tag
- Add Ira AB tag
[PATCH 06/20]
- Region label update/delete in LSA without v2 patch 01/20 [Jonathan/Dave/Ira]
- Use "union nd_lsa_label" inplace of "struct nd_lsa_label" [Jonathan/Dave]
- Rename rgl/rg* to region_label* [Dave]
- Use uuid_equal() without import_uuid() [Dave]
- Refactor __pmem_label_update() to accomodate region label [Fabio/Ira]
- Merge v2 patch 07/20 here [Ira]
- Fix hardcoded value while using init_labels() [Ira]
[PATCH 07/20]
- Fix comment message [Johathan]
- Replace mutex_lock() with guard(mutex)() [Jonathan]
[PATCH 08/20]
- Fix commit message [Johathan]
- Add Jonathan RB tag
[PATCH 09/20]
- Fix comment message [Johathan]
[PATCH 10/20]
- Re-arranged with v2 patch 12/20
- Fix commit message [Johathan]
[PATCH 11/20]
- Replace REVISIT with TODO [Johathan]
[PATCH 12/20]
- Fix commit message [Johathan]
[PATCH 14/20]
- Rename resize_or_free_dpa(() with alloc_region_dpa() [Dave]
- Rename resize_or_free_region_hpa() to alloc_region_hpa() [Dave]
- Elaborate comment message [Dave]
- Use  "__free(put_cxl_region)" for cxlr allocation [Dave]


Changes from v1 -> v2
---------------------
- v1 patch-set was broken. Find the v1 links at [4] and [5]
[PATCH 01/20]
- Simplify return in nvdimm_check_cxl_label_format() [Jonathan]
- Add spec reference while updating LSA major/minor [Jonathan, Dave]
[PATCH 02/20]
- Elaborate commit message with more information [Jonathan, Ira]
- Minimize extra re-naming to avoid churn & complexity [Jonathan]
[PATCH 05/20]
- Use guard(mutex)(&nd_mapping->lock) [Jonathan]
[PATCH 06/20]
- Use switch in place if condition check [Jonathan]
- Fix wrong style for multiline comments in this file [Jonathan]
- Fix wrong condition check in del_labels()
- Bail out extra condition check using extra variable [Jonathan]
[PATCH 07/20]
- Modify init_labels function to init_labels() [Jonathan]
[PATCH 08/20]
- Simplify return in is_region_label() and flip if condition [Jonathan]
[PATCH 12/20]
- Fix comment syntax [Jonathan]
[PATCH 13/20]
- Elaborate commit message and remove history statement from comment [Jonathan]
- Move cxl_region_discovery() from core/port.c to core/region.c [Dave]
[PATCH 14/20]
- Spell check fix in commit message [Jonathan]
- Use ACQUIRE() instead of down_write_killable() [Jonathan]
- Fix extra return check [Jonathan]
- Rename port to root_port to avoid long indirection [Jonathan]
- Remove cxl_wq_flush() as its not required [Jonathan, Dave]
- Add comment of attaching just one target [Jonathan]
- Rename update_region_size() to resize_or_free_region_hpa() [Dave]
- Rename val to size in resize_or_free_region_hpa() [Dave]
- Share common code using helper function in size_store() [Dave]
- Rename update_region_dpa_size() to resize_or_free_dpa() [Dave]
- Rename u64 in place of unsigned long long [Dave]
- Share common code using helper function in dpa_size_store() [Dave]
- Rename CXL_DECODER_PMEM with CXL_PARTMODE_PMEM [Dave]
- Share common code using helper function in commit_store() [Dave]
- Update verbose information about devm_cxl_pmem_add_region() [Dave]
- Renamed and refactored __create_region() to cxl_create_region() [Dave]
[PATCH 15/20]
- Avoid blank line [Jonathan]
- Rename root-cxl-port to CXL port [Dave]
- Add comment to release device ref taken via device_find_child() [Dave]
- Rename cxl_find_root_decoder() to cxl_find_root_decoder_by_port() [Dave]
[PATCH 18/20]
- Avoid extra nvdimm validity check [Jonathan]
- Flip logic to check for unhandled case first [Jonathan]
- Simplify return in match_ep_decoder() [Dave]
- Use lockdep_assert_held() in create_pmem_region() [Dave]
- To use lock moved create_pmem_region() from cxl/pmem.c to core/region.c
[PATCH 19/20]
- Spell check fix in commit message [Jonathan]
- Fix LIBNVDIMM selection condition check in cxl/Kconfig [Jonathan]
[PATCH 20/20]
- Add Documentation/ABI/testing/sysfs-bus-cxl [Jonathan]
- Use ACQUIRE() instead of down_write_killable() [Jonathan]
- Drop trailing comma on terminating entries [Jonathan]


[1]: https://lore.kernel.org/all/163116432405.2460985.5547867384570123403.stgit@dwillia2-desk3.amr.corp.intel.com/
[2]: https://github.com/neerajoss/ndctl/tree/linux-cxl/CXL_LSA_2.1_Support
[3]: https://elixir.bootlin.com/linux/v6.13.7/source/drivers/nvdimm/pmem.c#L520
[4] v1 Cover Letter: https://lore.kernel.org/linux-cxl/1931444790.41750165203442.JavaMail.epsvc@epcpadp1new/
[5] v1 Rest Thread: https://lore.kernel.org/linux-cxl/158453976.61750165203630.JavaMail.epsvc@epcpadp1new/
[6] v2: https://lore.kernel.org/linux-cxl/20250730121209.303202-1-s.neeraj@samsung.com/ 


Neeraj Kumar (20):
  nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label
  nvdimm/label: CXL labels skip the need for 'interleave-set cookie'
  nvdimm/label: Modify nd_label_base() signature
  nvdimm/label: Update mutex_lock() with guard(mutex)()
  nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1
  nvdimm/region_label: Add region label update support
  nvdimm/region_label: Add region label delete support
  nvdimm/label: Include region label in slot validation
  nvdimm/namespace_label: Skip region label during ns label DPA reservation
  nvdimm/namespace_label: Skip region label during namespace creation
  nvdimm/region_label: Preserve cxl region information from region label
  nvdimm/region_label: Export routine to fetch region information
  cxl/mem: Refactor cxl pmem region auto-assembling
  cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation
  cxl: Add a routine to find cxl root decoder on cxl bus using cxl port
  cxl/mem: Preserve cxl root decoder during mem probe
  cxl/pmem: Preserve region information into nd_set
  cxl/pmem_region: Prep patch to accommodate pmem_region attributes
  cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion
  cxl/pmem: Add CXL LSA 2.1 support in cxl pmem

 Documentation/ABI/testing/sysfs-bus-cxl |  22 +
 drivers/cxl/Kconfig                     |  14 +
 drivers/cxl/core/Makefile               |   1 +
 drivers/cxl/core/core.h                 |   8 +-
 drivers/cxl/core/pmem_region.c          | 346 ++++++++++++++++
 drivers/cxl/core/port.c                 |  29 +-
 drivers/cxl/core/region.c               | 351 ++++++++--------
 drivers/cxl/cxl.h                       |  50 ++-
 drivers/cxl/cxlmem.h                    |   1 +
 drivers/cxl/mem.c                       |  24 +-
 drivers/cxl/pmem.c                      |  15 +-
 drivers/cxl/port.c                      |  39 +-
 drivers/nvdimm/dimm.c                   |   5 +
 drivers/nvdimm/dimm_devs.c              |  25 ++
 drivers/nvdimm/label.c                  | 509 +++++++++++++++++++-----
 drivers/nvdimm/label.h                  |  16 +
 drivers/nvdimm/namespace_devs.c         |  84 +++-
 drivers/nvdimm/nd-core.h                |   2 +
 drivers/nvdimm/nd.h                     |  74 +++-
 drivers/nvdimm/region_devs.c            |  10 +
 include/linux/libnvdimm.h               |  28 ++
 tools/testing/cxl/Kbuild                |   1 +
 22 files changed, 1301 insertions(+), 353 deletions(-)
 create mode 100644 drivers/cxl/core/pmem_region.c


base-commit: c5dca38633daa1e240144bac453cf9065604a413
-- 
2.34.1

[PATCH V3 01/20] nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label

[Neeraj Kumar]

Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
introduced in CXL 2.0 Spec, which contain region label along with
namespace label.

NDD_LABELING flag is used for namespace. Introduced NDD_REGION_LABELING
flag for region label. Based on these flags nvdimm driver performs
operation on namespace label or region label.

NDD_REGION_LABELING will be utilized by cxl driver to enable LSA 2.1
region label support

Accordingly updated label index version

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/dimm.c      |  1 +
 drivers/nvdimm/dimm_devs.c |  7 +++++++
 drivers/nvdimm/label.c     | 21 +++++++++++++++++----
 drivers/nvdimm/nd.h        |  1 +
 include/linux/libnvdimm.h  |  3 +++
 5 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
index 91d9163ee303..bda22cb94e5b 100644
--- a/drivers/nvdimm/dimm.c
+++ b/drivers/nvdimm/dimm.c
@@ -62,6 +62,7 @@ static int nvdimm_probe(struct device *dev)
 	if (rc < 0)
 		dev_dbg(dev, "failed to unlock dimm: %d\n", rc);
 
+	ndd->cxl = nvdimm_check_region_label_format(ndd->dev);
 
 	/*
 	 * EACCES failures reading the namespace label-area-properties
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 21498d461fde..918c3db93195 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -18,6 +18,13 @@
 
 static DEFINE_IDA(dimm_ida);
 
+bool nvdimm_check_region_label_format(struct device *dev)
+{
+	struct nvdimm *nvdimm = to_nvdimm(dev);
+
+	return test_bit(NDD_REGION_LABELING, &nvdimm->flags);
+}
+
 /*
  * Retrieve bus and dimm handle and return if this bus supports
  * get_config_data commands
diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 04f4a049599a..0a9b6c5cb2c3 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -688,11 +688,24 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
 		- (unsigned long) to_namespace_index(ndd, 0);
 	nsindex->labeloff = __cpu_to_le64(offset);
 	nsindex->nslot = __cpu_to_le32(nslot);
-	nsindex->major = __cpu_to_le16(1);
-	if (sizeof_namespace_label(ndd) < 256)
+
+	/* Set LSA Label Index Version */
+	if (ndd->cxl) {
+		/* CXL r3.2: Table 9-9 Label Index Block Layout */
+		nsindex->major = __cpu_to_le16(2);
 		nsindex->minor = __cpu_to_le16(1);
-	else
-		nsindex->minor = __cpu_to_le16(2);
+	} else {
+		nsindex->major = __cpu_to_le16(1);
+		/*
+		 * NVDIMM Namespace Specification
+		 * Table 2: Namespace Label Index Block Fields
+		 */
+		if (sizeof_namespace_label(ndd) < 256)
+			nsindex->minor = __cpu_to_le16(1);
+		else /* UEFI 2.7: Label Index Block Definitions */
+			nsindex->minor = __cpu_to_le16(2);
+	}
+
 	nsindex->checksum = __cpu_to_le64(0);
 	if (flags & ND_NSINDEX_INIT) {
 		unsigned long *free = (unsigned long *) nsindex->free;
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index cc5c8f3f81e8..158809c2be9e 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -522,6 +522,7 @@ void nvdimm_set_labeling(struct device *dev);
 void nvdimm_set_locked(struct device *dev);
 void nvdimm_clear_locked(struct device *dev);
 int nvdimm_security_setup_events(struct device *dev);
+bool nvdimm_check_region_label_format(struct device *dev);
 #if IS_ENABLED(CONFIG_NVDIMM_KEYS)
 int nvdimm_security_unlock(struct device *dev);
 #else
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 28f086c4a187..5696715c33bb 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -44,6 +44,9 @@ enum {
 	/* dimm provider wants synchronous registration by __nvdimm_create() */
 	NDD_REGISTER_SYNC = 8,
 
+	/* dimm supports region labels (LSA Format 2.1) */
+	NDD_REGION_LABELING = 9,
+
 	/* need to set a limit somewhere, but yes, this is likely overkill */
 	ND_IOCTL_MAX_BUFLEN = SZ_4M,
 	ND_CMD_MAX_ELEM = 5,
-- 
2.34.1

[PATCH V3 02/20] nvdimm/label: CXL labels skip the need for 'interleave-set cookie'

[Neeraj Kumar]

CXL LSA v2.1 utilizes the region labels stored in the LSA for interleave
set configuration instead of interleave-set cookie used in previous LSA
versions. As interleave-set cookie is not required for CXL LSA v2.1 format
so skip its usage for CXL LSA 2.1 format

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Acked-by: Ira Weiny <ira.weiny@intel.com>
---
 drivers/nvdimm/namespace_devs.c |  8 +++++++-
 drivers/nvdimm/region_devs.c    | 10 ++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 55cfbf1e0a95..3271b1c8569a 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -1684,7 +1684,13 @@ static struct device *create_namespace_pmem(struct nd_region *nd_region,
 	int rc = 0;
 	u16 i;
 
-	if (cookie == 0) {
+	/*
+	 * CXL LSA v2.1 utilizes the region label stored in the LSA for
+	 * interleave set configuration. Whereas EFI LSA v1.1 & v1.2
+	 * utilizes interleave-set cookie. i.e, CXL labels skip the
+	 * need for 'interleave-set cookie'
+	 */
+	if (!ndd->cxl && cookie == 0) {
 		dev_dbg(&nd_region->dev, "invalid interleave-set-cookie\n");
 		return ERR_PTR(-ENXIO);
 	}
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index de1ee5ebc851..88275f352240 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -858,6 +858,16 @@ u64 nd_region_interleave_set_cookie(struct nd_region *nd_region,
 	if (!nd_set)
 		return 0;
 
+	/*
+	 * CXL LSA v2.1 utilizes the region label stored in the LSA for
+	 * interleave set configuration. Whereas EFI LSA v1.1 & v1.2
+	 * utilizes interleave-set cookie. i.e, CXL labels skip the
+	 * need for 'interleave-set cookie'
+	 */
+	if (nsindex && __le16_to_cpu(nsindex->major) == 2
+			&& __le16_to_cpu(nsindex->minor) == 1)
+		return 0;
+
 	if (nsindex && __le16_to_cpu(nsindex->major) == 1
 			&& __le16_to_cpu(nsindex->minor) == 1)
 		return nd_set->cookie1;
-- 
2.34.1

[PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Neeraj Kumar]

nd_label_base() was being used after typecasting with 'unsigned long'. Thus
modified nd_label_base() to return 'unsigned long' instead of 'struct
nd_namespace_label *'

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 0a9b6c5cb2c3..668e1e146229 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -271,11 +271,11 @@ static void nd_label_copy(struct nvdimm_drvdata *ndd,
 	memcpy(dst, src, sizeof_namespace_index(ndd));
 }
 
-static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd)
+static unsigned long nd_label_base(struct nvdimm_drvdata *ndd)
 {
 	void *base = to_namespace_index(ndd, 0);
 
-	return base + 2 * sizeof_namespace_index(ndd);
+	return (unsigned long) (base + 2 * sizeof_namespace_index(ndd));
 }
 
 static int to_slot(struct nvdimm_drvdata *ndd,
@@ -284,7 +284,7 @@ static int to_slot(struct nvdimm_drvdata *ndd,
 	unsigned long label, base;
 
 	label = (unsigned long) nd_label;
-	base = (unsigned long) nd_label_base(ndd);
+	base = nd_label_base(ndd);
 
 	return (label - base) / sizeof_namespace_label(ndd);
 }
@@ -293,7 +293,7 @@ static struct nd_namespace_label *to_label(struct nvdimm_drvdata *ndd, int slot)
 {
 	unsigned long label, base;
 
-	base = (unsigned long) nd_label_base(ndd);
+	base = nd_label_base(ndd);
 	label = base + sizeof_namespace_label(ndd) * slot;
 
 	return (struct nd_namespace_label *) label;
@@ -684,7 +684,7 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
 			nd_label_next_nsindex(index))
 		- (unsigned long) to_namespace_index(ndd, 0);
 	nsindex->otheroff = __cpu_to_le64(offset);
-	offset = (unsigned long) nd_label_base(ndd)
+	offset = nd_label_base(ndd)
 		- (unsigned long) to_namespace_index(ndd, 0);
 	nsindex->labeloff = __cpu_to_le64(offset);
 	nsindex->nslot = __cpu_to_le32(nslot);
-- 
2.34.1

[PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Neeraj Kumar]

Updated mutex_lock() with guard(mutex)()

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c | 36 +++++++++++++++++-------------------
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 668e1e146229..3235562d0e1c 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -948,7 +948,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
 		return rc;
 
 	/* Garbage collect the previous label */
-	mutex_lock(&nd_mapping->lock);
+	guard(mutex)(&nd_mapping->lock);
 	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
 		if (!label_ent->label)
 			continue;
@@ -960,20 +960,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
 	/* update index */
 	rc = nd_label_write_index(ndd, ndd->ns_next,
 			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
-	if (rc == 0) {
-		list_for_each_entry(label_ent, &nd_mapping->labels, list)
-			if (!label_ent->label) {
-				label_ent->label = nd_label;
-				nd_label = NULL;
-				break;
-			}
-		dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
-				"failed to track label: %d\n",
-				to_slot(ndd, nd_label));
-		if (nd_label)
-			rc = -ENXIO;
-	}
-	mutex_unlock(&nd_mapping->lock);
+	if (rc)
+		return rc;
+
+	list_for_each_entry(label_ent, &nd_mapping->labels, list)
+		if (!label_ent->label) {
+			label_ent->label = nd_label;
+			nd_label = NULL;
+			break;
+		}
+	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
+			"failed to track label: %d\n",
+			to_slot(ndd, nd_label));
+	if (nd_label)
+		rc = -ENXIO;
 
 	return rc;
 }
@@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
 		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
 		if (!label_ent)
 			return -ENOMEM;
-		mutex_lock(&nd_mapping->lock);
+		guard(mutex)(&nd_mapping->lock);
 		list_add_tail(&label_ent->list, &nd_mapping->labels);
-		mutex_unlock(&nd_mapping->lock);
 	}
 
 	if (ndd->ns_current == -1 || ndd->ns_next == -1)
@@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
 	if (!preamble_next(ndd, &nsindex, &free, &nslot))
 		return 0;
 
-	mutex_lock(&nd_mapping->lock);
+	guard(mutex)(&nd_mapping->lock);
 	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
 		struct nd_namespace_label *nd_label = label_ent->label;
 
@@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
 		nd_mapping_free_labels(nd_mapping);
 		dev_dbg(ndd->dev, "no more active labels\n");
 	}
-	mutex_unlock(&nd_mapping->lock);
 
 	return nd_label_write_index(ndd, ndd->ns_next,
 			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
-- 
2.34.1

[PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Neeraj Kumar]

CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
Modified __pmem_label_update function using setter functions to update
namespace label as per CXL LSA 2.1

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c |  3 +++
 drivers/nvdimm/nd.h    | 23 +++++++++++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 3235562d0e1c..182f8c9a01bf 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -924,6 +924,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
 
 	nd_label = to_label(ndd, slot);
 	memset(nd_label, 0, sizeof_namespace_label(ndd));
+	nsl_set_type(ndd, nd_label);
 	nsl_set_uuid(ndd, nd_label, nspm->uuid);
 	nsl_set_name(ndd, nd_label, nspm->alt_name);
 	nsl_set_flags(ndd, nd_label, flags);
@@ -935,7 +936,9 @@ static int __pmem_label_update(struct nd_region *nd_region,
 	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
 	nsl_set_dpa(ndd, nd_label, res->start);
 	nsl_set_slot(ndd, nd_label, slot);
+	nsl_set_alignment(ndd, nd_label, 0);
 	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
+	nsl_set_region_uuid(ndd, nd_label, NULL);
 	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
 	nsl_calculate_checksum(ndd, nd_label);
 	nd_dbg_dpa(nd_region, ndd, res, "\n");
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 158809c2be9e..e362611d82cc 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -295,6 +295,29 @@ static inline const u8 *nsl_uuid_raw(struct nvdimm_drvdata *ndd,
 	return nd_label->efi.uuid;
 }
 
+static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
+				struct nd_namespace_label *ns_label)
+{
+	if (ndd->cxl && ns_label)
+		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);
+}
+
+static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd,
+				     struct nd_namespace_label *ns_label,
+				     u32 align)
+{
+	if (ndd->cxl)
+		ns_label->cxl.align = __cpu_to_le32(align);
+}
+
+static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
+				       struct nd_namespace_label *ns_label,
+				       const uuid_t *uuid)
+{
+	if (ndd->cxl && uuid)
+		export_uuid(ns_label->cxl.region_uuid, uuid);
+}
+
 bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
 			    struct nd_namespace_label *nd_label, guid_t *guid);
 enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
-- 
2.34.1

[PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Neeraj Kumar]

Modified __pmem_label_update() to update region labels into LSA

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c          | 269 ++++++++++++++++++++++++++------
 drivers/nvdimm/label.h          |  15 ++
 drivers/nvdimm/namespace_devs.c |  12 ++
 drivers/nvdimm/nd.h             |  38 ++++-
 include/linux/libnvdimm.h       |   8 +
 5 files changed, 289 insertions(+), 53 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 182f8c9a01bf..209c73f6b7e7 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -381,6 +381,16 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
 	nsl_set_checksum(ndd, nd_label, sum);
 }
 
+static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
+				   struct cxl_region_label *region_label)
+{
+	u64 sum;
+
+	region_label_set_checksum(region_label, 0);
+	sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1);
+	region_label_set_checksum(region_label, sum);
+}
+
 static bool slot_valid(struct nvdimm_drvdata *ndd,
 		struct nd_namespace_label *nd_label, u32 slot)
 {
@@ -884,26 +894,20 @@ enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
 	return guid_to_nvdimm_cclass(&nd_label->efi.abstraction_guid);
 }
 
-static int __pmem_label_update(struct nd_region *nd_region,
-		struct nd_mapping *nd_mapping, struct nd_namespace_pmem *nspm,
-		int pos, unsigned long flags)
+static int namespace_label_update(struct nd_region *nd_region,
+				  struct nd_mapping *nd_mapping,
+				  struct nd_namespace_pmem *nspm,
+				  int pos, u64 flags,
+				  struct nd_namespace_label *ns_label,
+				  struct nd_namespace_index *nsindex,
+				  u32 slot)
 {
 	struct nd_namespace_common *ndns = &nspm->nsio.common;
 	struct nd_interleave_set *nd_set = nd_region->nd_set;
 	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
-	struct nd_namespace_label *nd_label;
-	struct nd_namespace_index *nsindex;
-	struct nd_label_ent *label_ent;
 	struct nd_label_id label_id;
 	struct resource *res;
-	unsigned long *free;
-	u32 nslot, slot;
-	size_t offset;
 	u64 cookie;
-	int rc;
-
-	if (!preamble_next(ndd, &nsindex, &free, &nslot))
-		return -ENXIO;
 
 	cookie = nd_region_interleave_set_cookie(nd_region, nsindex);
 	nd_label_gen_id(&label_id, nspm->uuid, 0);
@@ -916,36 +920,131 @@ static int __pmem_label_update(struct nd_region *nd_region,
 		return -ENXIO;
 	}
 
+	nsl_set_type(ndd, ns_label);
+	nsl_set_uuid(ndd, ns_label, nspm->uuid);
+	nsl_set_name(ndd, ns_label, nspm->alt_name);
+	nsl_set_flags(ndd, ns_label, flags);
+	nsl_set_nlabel(ndd, ns_label, nd_region->ndr_mappings);
+	nsl_set_nrange(ndd, ns_label, 1);
+	nsl_set_position(ndd, ns_label, pos);
+	nsl_set_isetcookie(ndd, ns_label, cookie);
+	nsl_set_rawsize(ndd, ns_label, resource_size(res));
+	nsl_set_lbasize(ndd, ns_label, nspm->lbasize);
+	nsl_set_dpa(ndd, ns_label, res->start);
+	nsl_set_slot(ndd, ns_label, slot);
+	nsl_set_alignment(ndd, ns_label, 0);
+	nsl_set_type_guid(ndd, ns_label, &nd_set->type_guid);
+	nsl_set_region_uuid(ndd, ns_label, &nd_set->uuid);
+	nsl_set_claim_class(ndd, ns_label, ndns->claim_class);
+	nsl_calculate_checksum(ndd, ns_label);
+	nd_dbg_dpa(nd_region, ndd, res, "\n");
+
+	return 0;
+}
+
+static void region_label_update(struct nd_region *nd_region,
+				struct cxl_region_label *region_label,
+				struct nd_mapping *nd_mapping,
+				int pos, u64 flags, u32 slot)
+{
+	struct nd_interleave_set *nd_set = nd_region->nd_set;
+	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+
+	/* Set Region Label Format identification UUID */
+	uuid_parse(CXL_REGION_UUID, (uuid_t *) region_label->type);
+
+	/* Set Current Region Label UUID */
+	export_uuid(region_label->uuid, &nd_set->uuid);
+
+	region_label->flags = __cpu_to_le32(flags);
+	region_label->nlabel = __cpu_to_le16(nd_region->ndr_mappings);
+	region_label->position = __cpu_to_le16(pos);
+	region_label->dpa = __cpu_to_le64(nd_mapping->start);
+	region_label->rawsize = __cpu_to_le64(nd_mapping->size);
+	region_label->hpa = __cpu_to_le64(nd_set->res->start);
+	region_label->slot = __cpu_to_le32(slot);
+	region_label->ig = __cpu_to_le32(nd_set->interleave_granularity);
+	region_label->align = __cpu_to_le32(0);
+
+	/* Update fletcher64 Checksum */
+	region_label_calculate_checksum(ndd, region_label);
+}
+
+static bool is_label_reapable(struct nd_interleave_set *nd_set,
+			       struct nd_namespace_pmem *nspm,
+			       struct nvdimm_drvdata *ndd,
+			       union nd_lsa_label *label,
+			       enum label_type ltype,
+			       unsigned long *flags)
+{
+	switch (ltype) {
+	case NS_LABEL_TYPE:
+		if (test_and_clear_bit(ND_LABEL_REAP, flags) ||
+		    nsl_uuid_equal(ndd, &label->ns_label, nspm->uuid))
+			return true;
+
+		break;
+	case RG_LABEL_TYPE:
+		if (region_label_uuid_equal(&label->region_label,
+		    &nd_set->uuid))
+			return true;
+
+		break;
+	}
+
+	return false;
+}
+
+static int __pmem_label_update(struct nd_region *nd_region,
+			       struct nd_mapping *nd_mapping,
+			       struct nd_namespace_pmem *nspm,
+			       int pos, unsigned long flags,
+			       enum label_type ltype)
+{
+	struct nd_interleave_set *nd_set = nd_region->nd_set;
+	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+	struct nd_namespace_index *nsindex;
+	struct nd_label_ent *label_ent;
+	union nd_lsa_label *lsa_label;
+	unsigned long *free;
+	struct device *dev;
+	u32 nslot, slot;
+	size_t offset;
+	int rc;
+
+	if (!preamble_next(ndd, &nsindex, &free, &nslot))
+		return -ENXIO;
+
 	/* allocate and write the label to the staging (next) index */
 	slot = nd_label_alloc_slot(ndd);
 	if (slot == UINT_MAX)
 		return -ENXIO;
 	dev_dbg(ndd->dev, "allocated: %d\n", slot);
 
-	nd_label = to_label(ndd, slot);
-	memset(nd_label, 0, sizeof_namespace_label(ndd));
-	nsl_set_type(ndd, nd_label);
-	nsl_set_uuid(ndd, nd_label, nspm->uuid);
-	nsl_set_name(ndd, nd_label, nspm->alt_name);
-	nsl_set_flags(ndd, nd_label, flags);
-	nsl_set_nlabel(ndd, nd_label, nd_region->ndr_mappings);
-	nsl_set_nrange(ndd, nd_label, 1);
-	nsl_set_position(ndd, nd_label, pos);
-	nsl_set_isetcookie(ndd, nd_label, cookie);
-	nsl_set_rawsize(ndd, nd_label, resource_size(res));
-	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
-	nsl_set_dpa(ndd, nd_label, res->start);
-	nsl_set_slot(ndd, nd_label, slot);
-	nsl_set_alignment(ndd, nd_label, 0);
-	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
-	nsl_set_region_uuid(ndd, nd_label, NULL);
-	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
-	nsl_calculate_checksum(ndd, nd_label);
-	nd_dbg_dpa(nd_region, ndd, res, "\n");
+	lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
+	memset(lsa_label, 0, sizeof_namespace_label(ndd));
+
+	switch (ltype) {
+	case NS_LABEL_TYPE:
+		dev = &nspm->nsio.common.dev;
+		rc = namespace_label_update(nd_region, nd_mapping,
+				nspm, pos, flags, &lsa_label->ns_label,
+				nsindex, slot);
+		if (rc)
+			return rc;
+
+		break;
+	case RG_LABEL_TYPE:
+		dev = &nd_region->dev;
+		region_label_update(nd_region, &lsa_label->region_label,
+				    nd_mapping, pos, flags, slot);
+
+		break;
+	}
 
 	/* update label */
-	offset = nd_label_offset(ndd, nd_label);
-	rc = nvdimm_set_config_data(ndd, offset, nd_label,
+	offset = nd_label_offset(ndd, &lsa_label->ns_label);
+	rc = nvdimm_set_config_data(ndd, offset, lsa_label,
 			sizeof_namespace_label(ndd));
 	if (rc < 0)
 		return rc;
@@ -955,8 +1054,10 @@ static int __pmem_label_update(struct nd_region *nd_region,
 	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
 		if (!label_ent->label)
 			continue;
-		if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) ||
-		    nsl_uuid_equal(ndd, label_ent->label, nspm->uuid))
+
+		if (is_label_reapable(nd_set, nspm, ndd,
+				      (union nd_lsa_label *) label_ent->label,
+				      ltype, &label_ent->flags))
 			reap_victim(nd_mapping, label_ent);
 	}
 
@@ -966,19 +1067,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
 	if (rc)
 		return rc;
 
-	list_for_each_entry(label_ent, &nd_mapping->labels, list)
-		if (!label_ent->label) {
-			label_ent->label = nd_label;
-			nd_label = NULL;
-			break;
-		}
-	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
-			"failed to track label: %d\n",
-			to_slot(ndd, nd_label));
-	if (nd_label)
-		rc = -ENXIO;
+	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
+		if (label_ent->label)
+			continue;
 
-	return rc;
+		label_ent->label = &lsa_label->ns_label;
+		lsa_label = NULL;
+		break;
+	}
+	dev_WARN_ONCE(dev, lsa_label, "failed to track label: %d\n",
+		      to_slot(ndd, &lsa_label->ns_label));
+	if (lsa_label)
+		return -ENXIO;
+
+	return 0;
 }
 
 static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
@@ -1068,6 +1170,21 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
 			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
 }
 
+static int find_region_label_count(struct nvdimm_drvdata *ndd,
+				   struct nd_mapping *nd_mapping)
+{
+	struct nd_label_ent *label_ent;
+	int region_label_cnt = 0;
+
+	guard(mutex)(&nd_mapping->lock);
+	list_for_each_entry(label_ent, &nd_mapping->labels, list)
+		if (is_region_label(ndd,
+		    (union nd_lsa_label *) label_ent->label))
+			region_label_cnt++;
+
+	return region_label_cnt;
+}
+
 int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 		struct nd_namespace_pmem *nspm, resource_size_t size)
 {
@@ -1076,6 +1193,7 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 	for (i = 0; i < nd_region->ndr_mappings; i++) {
 		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
 		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+		int region_label_cnt = 0;
 		struct resource *res;
 		int count = 0;
 
@@ -1091,12 +1209,19 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 				count++;
 		WARN_ON_ONCE(!count);
 
-		rc = init_labels(nd_mapping, count);
+		region_label_cnt = find_region_label_count(ndd, nd_mapping);
+		/*
+		 * init_labels() scan labels and allocate new label based
+		 * on its second parameter (num_labels). Therefore to
+		 * allocate new namespace label also include previously
+		 * added region label
+		 */
+		rc = init_labels(nd_mapping, count + region_label_cnt);
 		if (rc < 0)
 			return rc;
 
 		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i,
-				NSLABEL_FLAG_UPDATING);
+				NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE);
 		if (rc)
 			return rc;
 	}
@@ -1108,7 +1233,47 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 	for (i = 0; i < nd_region->ndr_mappings; i++) {
 		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
 
-		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0);
+		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0,
+				NS_LABEL_TYPE);
+		if (rc)
+			return rc;
+	}
+
+	return 0;
+}
+
+int nd_pmem_region_label_update(struct nd_region *nd_region)
+{
+	int i, rc;
+
+	for (i = 0; i < nd_region->ndr_mappings; i++) {
+		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+		int region_label_cnt = 0;
+
+		/* No need to update region label for non cxl format */
+		if (!ndd->cxl)
+			return 0;
+
+		region_label_cnt = find_region_label_count(ndd, nd_mapping);
+		rc = init_labels(nd_mapping, region_label_cnt + 1);
+		if (rc < 0)
+			return rc;
+
+		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i,
+				NSLABEL_FLAG_UPDATING, RG_LABEL_TYPE);
+		if (rc)
+			return rc;
+	}
+
+	/* Clear the UPDATING flag per UEFI 2.7 expectations */
+	for (i = 0; i < nd_region->ndr_mappings; i++) {
+		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+
+		WARN_ON_ONCE(!ndd->cxl);
+		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, 0,
+				RG_LABEL_TYPE);
 		if (rc)
 			return rc;
 	}
diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
index 0650fb4b9821..284e2a763b49 100644
--- a/drivers/nvdimm/label.h
+++ b/drivers/nvdimm/label.h
@@ -30,6 +30,11 @@ enum {
 	ND_NSINDEX_INIT = 0x1,
 };
 
+enum label_type {
+	RG_LABEL_TYPE,
+	NS_LABEL_TYPE,
+};
+
 /**
  * struct nd_namespace_index - label set superblock
  * @sig: NAMESPACE_INDEX\0
@@ -183,6 +188,15 @@ struct nd_namespace_label {
 	};
 };
 
+/*
+ * LSA 2.1 format introduces region label, which can also reside
+ * into LSA along with only namespace label as per v1.1 and v1.2
+ */
+union nd_lsa_label {
+	struct nd_namespace_label ns_label;
+	struct cxl_region_label region_label;
+};
+
 #define NVDIMM_BTT_GUID "8aed63a2-29a2-4c66-8b12-f05d15d3922a"
 #define NVDIMM_BTT2_GUID "18633bfc-1735-4217-8ac9-17239282d3f8"
 #define NVDIMM_PFN_GUID "266400ba-fb9f-4677-bcb0-968f11d0d225"
@@ -223,4 +237,5 @@ struct nd_region;
 struct nd_namespace_pmem;
 int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 		struct nd_namespace_pmem *nspm, resource_size_t size);
+int nd_pmem_region_label_update(struct nd_region *nd_region);
 #endif /* __LABEL_H__ */
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 3271b1c8569a..559f822ef24f 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -232,6 +232,18 @@ static ssize_t __alt_name_store(struct device *dev, const char *buf,
 	return rc;
 }
 
+int nd_region_label_update(struct nd_region *nd_region)
+{
+	int rc;
+
+	nvdimm_bus_lock(&nd_region->dev);
+	rc = nd_pmem_region_label_update(nd_region);
+	nvdimm_bus_unlock(&nd_region->dev);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(nd_region_label_update);
+
 static int nd_namespace_label_update(struct nd_region *nd_region,
 		struct device *dev)
 {
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index e362611d82cc..f04c042dcfa9 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -318,6 +318,39 @@ static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
 		export_uuid(ns_label->cxl.region_uuid, uuid);
 }
 
+static inline bool is_region_label(struct nvdimm_drvdata *ndd,
+				   union nd_lsa_label *nd_label)
+{
+	uuid_t region_type, *ns_type;
+
+	if (!ndd->cxl || !nd_label)
+		return false;
+
+	uuid_parse(CXL_REGION_UUID, &region_type);
+	ns_type = (uuid_t *) nd_label->ns_label.cxl.type;
+	return uuid_equal(&region_type, ns_type);
+}
+
+static inline bool
+region_label_uuid_equal(struct cxl_region_label *region_label,
+			const uuid_t *uuid)
+{
+	return uuid_equal((uuid_t *) region_label->uuid, uuid);
+}
+
+static inline u64
+region_label_get_checksum(struct cxl_region_label *region_label)
+{
+	return __le64_to_cpu(region_label->checksum);
+}
+
+static inline void
+region_label_set_checksum(struct cxl_region_label *region_label,
+			  u64 checksum)
+{
+	region_label->checksum = __cpu_to_le64(checksum);
+}
+
 bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
 			    struct nd_namespace_label *nd_label, guid_t *guid);
 enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
@@ -399,7 +432,10 @@ enum nd_label_flags {
 struct nd_label_ent {
 	struct list_head list;
 	unsigned long flags;
-	struct nd_namespace_label *label;
+	union {
+		struct nd_namespace_label *label;
+		struct cxl_region_label *region_label;
+	};
 };
 
 enum nd_mapping_lock_class {
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 5696715c33bb..2c213b9dac66 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -117,6 +117,13 @@ struct nd_interleave_set {
 	u64 altcookie;
 
 	guid_t type_guid;
+
+	/* v2.1 region label info */
+	uuid_t uuid;
+	int interleave_ways;
+	int interleave_granularity;
+	struct resource *res;
+	int nr_targets;
 };
 
 struct nd_mapping_desc {
@@ -307,6 +314,7 @@ int nvdimm_has_flush(struct nd_region *nd_region);
 int nvdimm_has_cache(struct nd_region *nd_region);
 int nvdimm_in_overwrite(struct nvdimm *nvdimm);
 bool is_nvdimm_sync(struct nd_region *nd_region);
+int nd_region_label_update(struct nd_region *nd_region);
 
 static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
 		unsigned int buf_len, int *cmd_rc)
-- 
2.34.1

[PATCH V3 07/20] nvdimm/region_label: Add region label delete support

[Neeraj Kumar]

Added LSA v2.1 format region label deletion routine. This function is
used to delete region label from LSA

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c          | 79 ++++++++++++++++++++++++++++++---
 drivers/nvdimm/label.h          |  1 +
 drivers/nvdimm/namespace_devs.c | 12 +++++
 drivers/nvdimm/nd.h             |  6 +++
 include/linux/libnvdimm.h       |  1 +
 5 files changed, 92 insertions(+), 7 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 209c73f6b7e7..d33db96ba8ba 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -1126,11 +1126,13 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
 	return max(num_labels, old_num_labels);
 }
 
-static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
+static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid,
+		      enum label_type ltype)
 {
 	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
 	struct nd_label_ent *label_ent, *e;
 	struct nd_namespace_index *nsindex;
+	union nd_lsa_label *nd_label;
 	unsigned long *free;
 	LIST_HEAD(list);
 	u32 nslot, slot;
@@ -1145,15 +1147,28 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
 
 	guard(mutex)(&nd_mapping->lock);
 	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
-		struct nd_namespace_label *nd_label = label_ent->label;
+		nd_label = (union nd_lsa_label *) label_ent->label;
 
 		if (!nd_label)
 			continue;
 		active++;
-		if (!nsl_uuid_equal(ndd, nd_label, uuid))
-			continue;
+
+		switch (ltype) {
+		case NS_LABEL_TYPE:
+			if (!nsl_uuid_equal(ndd, &nd_label->ns_label, uuid))
+				continue;
+
+			break;
+		case RG_LABEL_TYPE:
+			if (!region_label_uuid_equal(&nd_label->region_label,
+			    uuid))
+				continue;
+
+			break;
+		}
+
 		active--;
-		slot = to_slot(ndd, nd_label);
+		slot = to_slot(ndd, &nd_label->ns_label);
 		nd_label_free_slot(ndd, slot);
 		dev_dbg(ndd->dev, "free: %d\n", slot);
 		list_move_tail(&label_ent->list, &list);
@@ -1161,7 +1176,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
 	}
 	list_splice_tail_init(&list, &nd_mapping->labels);
 
-	if (active == 0) {
+	if ((ltype == NS_LABEL_TYPE) && (active == 0)) {
 		nd_mapping_free_labels(nd_mapping);
 		dev_dbg(ndd->dev, "no more active labels\n");
 	}
@@ -1198,7 +1213,8 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 		int count = 0;
 
 		if (size == 0) {
-			rc = del_labels(nd_mapping, nspm->uuid);
+			rc = del_labels(nd_mapping, nspm->uuid,
+					NS_LABEL_TYPE);
 			if (rc)
 				return rc;
 			continue;
@@ -1281,6 +1297,55 @@ int nd_pmem_region_label_update(struct nd_region *nd_region)
 	return 0;
 }
 
+int nd_pmem_region_label_delete(struct nd_region *nd_region)
+{
+	struct nd_interleave_set *nd_set = nd_region->nd_set;
+	struct nd_label_ent *label_ent;
+	int ns_region_cnt = 0;
+	int i, rc;
+
+	for (i = 0; i < nd_region->ndr_mappings; i++) {
+		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+
+		/* Find non cxl format supported ndr_mappings */
+		if (!ndd->cxl) {
+			dev_info(&nd_region->dev, "Unsupported region label\n");
+			return -EINVAL;
+		}
+
+		/* Find if any NS label using this region */
+		guard(mutex)(&nd_mapping->lock);
+		list_for_each_entry(label_ent, &nd_mapping->labels, list) {
+			if (!label_ent->label)
+				continue;
+
+			/*
+			 * Check if any available NS labels has same
+			 * region_uuid in LSA
+			 */
+			if (nsl_region_uuid_equal(label_ent->label,
+						  &nd_set->uuid))
+				ns_region_cnt++;
+		}
+	}
+
+	if (ns_region_cnt) {
+		dev_dbg(&nd_region->dev, "Region/Namespace label in use\n");
+		return -EBUSY;
+	}
+
+	for (i = 0; i < nd_region->ndr_mappings; i++) {
+		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+
+		rc = del_labels(nd_mapping, &nd_set->uuid, RG_LABEL_TYPE);
+		if (rc)
+			return rc;
+	}
+
+	return 0;
+}
+
 int __init nd_label_init(void)
 {
 	WARN_ON(guid_parse(NVDIMM_BTT_GUID, &nvdimm_btt_guid));
diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
index 284e2a763b49..276dd822e142 100644
--- a/drivers/nvdimm/label.h
+++ b/drivers/nvdimm/label.h
@@ -238,4 +238,5 @@ struct nd_namespace_pmem;
 int nd_pmem_namespace_label_update(struct nd_region *nd_region,
 		struct nd_namespace_pmem *nspm, resource_size_t size);
 int nd_pmem_region_label_update(struct nd_region *nd_region);
+int nd_pmem_region_label_delete(struct nd_region *nd_region);
 #endif /* __LABEL_H__ */
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 559f822ef24f..564a73b1da41 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -244,6 +244,18 @@ int nd_region_label_update(struct nd_region *nd_region)
 }
 EXPORT_SYMBOL_GPL(nd_region_label_update);
 
+int nd_region_label_delete(struct nd_region *nd_region)
+{
+	int rc;
+
+	nvdimm_bus_lock(&nd_region->dev);
+	rc = nd_pmem_region_label_delete(nd_region);
+	nvdimm_bus_unlock(&nd_region->dev);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(nd_region_label_delete);
+
 static int nd_namespace_label_update(struct nd_region *nd_region,
 		struct device *dev)
 {
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index f04c042dcfa9..046063ea08b6 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -331,6 +331,12 @@ static inline bool is_region_label(struct nvdimm_drvdata *ndd,
 	return uuid_equal(&region_type, ns_type);
 }
 
+static inline bool nsl_region_uuid_equal(struct nd_namespace_label *ns_label,
+					 const uuid_t *uuid)
+{
+	return uuid_equal((uuid_t *) ns_label->cxl.region_uuid, uuid);
+}
+
 static inline bool
 region_label_uuid_equal(struct cxl_region_label *region_label,
 			const uuid_t *uuid)
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 2c213b9dac66..bbf14a260c93 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -315,6 +315,7 @@ int nvdimm_has_cache(struct nd_region *nd_region);
 int nvdimm_in_overwrite(struct nvdimm *nvdimm);
 bool is_nvdimm_sync(struct nd_region *nd_region);
 int nd_region_label_update(struct nd_region *nd_region);
+int nd_region_label_delete(struct nd_region *nd_region);
 
 static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
 		unsigned int buf_len, int *cmd_rc)
-- 
2.34.1

[PATCH V3 08/20] nvdimm/label: Include region label in slot validation

[Neeraj Kumar]

Slot validation routine validates label slot by calculating label
checksum. It was only validating namespace label. This changeset also
validates region label if present.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
---
 drivers/nvdimm/label.c | 72 ++++++++++++++++++++++++++++++++----------
 drivers/nvdimm/nd.h    |  5 +++
 2 files changed, 60 insertions(+), 17 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index d33db96ba8ba..5e476154cf81 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -359,7 +359,7 @@ static bool nsl_validate_checksum(struct nvdimm_drvdata *ndd,
 {
 	u64 sum, sum_save;
 
-	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
+	if (!efi_namespace_label_has(ndd, checksum))
 		return true;
 
 	sum_save = nsl_get_checksum(ndd, nd_label);
@@ -374,13 +374,25 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
 {
 	u64 sum;
 
-	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
+	if (!efi_namespace_label_has(ndd, checksum))
 		return;
 	nsl_set_checksum(ndd, nd_label, 0);
 	sum = nd_fletcher64(nd_label, sizeof_namespace_label(ndd), 1);
 	nsl_set_checksum(ndd, nd_label, sum);
 }
 
+static bool region_label_validate_checksum(struct nvdimm_drvdata *ndd,
+				struct cxl_region_label *region_label)
+{
+	u64 sum, sum_save;
+
+	sum_save = region_label_get_checksum(region_label);
+	region_label_set_checksum(region_label, 0);
+	sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1);
+	region_label_set_checksum(region_label, sum_save);
+	return sum == sum_save;
+}
+
 static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
 				   struct cxl_region_label *region_label)
 {
@@ -392,16 +404,30 @@ static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
 }
 
 static bool slot_valid(struct nvdimm_drvdata *ndd,
-		struct nd_namespace_label *nd_label, u32 slot)
+		       union nd_lsa_label *lsa_label, u32 slot)
 {
+	struct cxl_region_label *region_label = &lsa_label->region_label;
+	struct nd_namespace_label *nd_label = &lsa_label->ns_label;
+	char *label_name;
 	bool valid;
 
 	/* check that we are written where we expect to be written */
-	if (slot != nsl_get_slot(ndd, nd_label))
-		return false;
-	valid = nsl_validate_checksum(ndd, nd_label);
+	if (is_region_label(ndd, lsa_label)) {
+		label_name = "rg";
+		if (slot != region_label_get_slot(region_label))
+			return false;
+		valid = region_label_validate_checksum(ndd, region_label);
+	} else {
+		label_name = "ns";
+		if (slot != nsl_get_slot(ndd, nd_label))
+			return false;
+		valid = nsl_validate_checksum(ndd, nd_label);
+	}
+
 	if (!valid)
-		dev_dbg(ndd->dev, "fail checksum. slot: %d\n", slot);
+		dev_dbg(ndd->dev, "%s label checksum fail. slot: %d\n",
+			label_name, slot);
+
 	return valid;
 }
 
@@ -424,7 +450,7 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
 
 		nd_label = to_label(ndd, slot);
 
-		if (!slot_valid(ndd, nd_label, slot))
+		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
 			continue;
 
 		nsl_get_uuid(ndd, nd_label, &label_uuid);
@@ -575,18 +601,30 @@ int nd_label_active_count(struct nvdimm_drvdata *ndd)
 		return 0;
 
 	for_each_clear_bit_le(slot, free, nslot) {
+		struct cxl_region_label *region_label;
 		struct nd_namespace_label *nd_label;
-
-		nd_label = to_label(ndd, slot);
-
-		if (!slot_valid(ndd, nd_label, slot)) {
-			u32 label_slot = nsl_get_slot(ndd, nd_label);
-			u64 size = nsl_get_rawsize(ndd, nd_label);
-			u64 dpa = nsl_get_dpa(ndd, nd_label);
+		union nd_lsa_label *lsa_label;
+		u32 lslot;
+		u64 size, dpa;
+
+		lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
+		nd_label = &lsa_label->ns_label;
+		region_label = &lsa_label->region_label;
+
+		if (!slot_valid(ndd, lsa_label, slot)) {
+			if (is_region_label(ndd, lsa_label)) {
+				lslot = __le32_to_cpu(region_label->slot);
+				size = __le64_to_cpu(region_label->rawsize);
+				dpa = __le64_to_cpu(region_label->dpa);
+			} else {
+				lslot = nsl_get_slot(ndd, nd_label);
+				size = nsl_get_rawsize(ndd, nd_label);
+				dpa = nsl_get_dpa(ndd, nd_label);
+			}
 
 			dev_dbg(ndd->dev,
 				"slot%d invalid slot: %d dpa: %llx size: %llx\n",
-					slot, label_slot, dpa, size);
+					slot, lslot, dpa, size);
 			continue;
 		}
 		count++;
@@ -607,7 +645,7 @@ struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n)
 		struct nd_namespace_label *nd_label;
 
 		nd_label = to_label(ndd, slot);
-		if (!slot_valid(ndd, nd_label, slot))
+		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
 			continue;
 
 		if (n-- == 0)
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 046063ea08b6..c985f91728dd 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -344,6 +344,11 @@ region_label_uuid_equal(struct cxl_region_label *region_label,
 	return uuid_equal((uuid_t *) region_label->uuid, uuid);
 }
 
+static inline u32 region_label_get_slot(struct cxl_region_label *region_label)
+{
+	return __le32_to_cpu(region_label->slot);
+}
+
 static inline u64
 region_label_get_checksum(struct cxl_region_label *region_label)
 {
-- 
2.34.1

[PATCH V3 09/20] nvdimm/namespace_label: Skip region label during ns label DPA reservation

[Neeraj Kumar]

If Namespace label is present in LSA during nvdimm_probe() then DPA
reservation is required. But this reservation is not required by region
label. Therefore if LSA scanning finds any region label, skip it.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/label.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 5e476154cf81..935a0df5b47e 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -441,6 +441,7 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
 		return 0; /* no label, nothing to reserve */
 
 	for_each_clear_bit_le(slot, free, nslot) {
+		union nd_lsa_label *lsa_label;
 		struct nd_namespace_label *nd_label;
 		struct nd_region *nd_region = NULL;
 		struct nd_label_id label_id;
@@ -448,9 +449,14 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
 		uuid_t label_uuid;
 		u32 flags;
 
-		nd_label = to_label(ndd, slot);
+		lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
+		nd_label = &lsa_label->ns_label;
 
-		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
+		/* Skip region label. DPA reservation is for NS label only */
+		if (is_region_label(ndd, lsa_label))
+			continue;
+
+		if (!slot_valid(ndd, lsa_label, slot))
 			continue;
 
 		nsl_get_uuid(ndd, nd_label, &label_uuid);
-- 
2.34.1

[PATCH V3 10/20] nvdimm/namespace_label: Skip region label during namespace creation

[Neeraj Kumar]

During namespace creation, skip any region labels found. And Preserve
region label into labels list if present.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/namespace_devs.c | 52 +++++++++++++++++++++++++++++----
 1 file changed, 47 insertions(+), 5 deletions(-)

diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 564a73b1da41..735310e6fc11 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -1979,6 +1979,10 @@ static struct device **scan_labels(struct nd_region *nd_region)
 		if (!nd_label)
 			continue;
 
+		/* Skip region labels if present */
+		if (is_region_label(ndd, (union nd_lsa_label *) nd_label))
+			continue;
+
 		/* skip labels that describe extents outside of the region */
 		if (nsl_get_dpa(ndd, nd_label) < nd_mapping->start ||
 		    nsl_get_dpa(ndd, nd_label) > map_end)
@@ -2017,9 +2021,31 @@ static struct device **scan_labels(struct nd_region *nd_region)
 
 	if (count == 0) {
 		struct nd_namespace_pmem *nspm;
+		for (i = 0; i < nd_region->ndr_mappings; i++) {
+			union nd_lsa_label *nd_label;
+			struct nd_label_ent *le, *e;
+			LIST_HEAD(list);
 
-		/* Publish a zero-sized namespace for userspace to configure. */
-		nd_mapping_free_labels(nd_mapping);
+			nd_mapping = &nd_region->mapping[i];
+			if (list_empty(&nd_mapping->labels))
+				continue;
+
+			list_for_each_entry_safe(le, e, &nd_mapping->labels,
+						 list) {
+				nd_label = (union nd_lsa_label *) le->label;
+
+				/* Preserve region labels if present */
+				if (is_region_label(ndd, nd_label))
+					list_move_tail(&le->list, &list);
+			}
+
+			/*
+			 * Publish a zero-sized namespace for userspace
+			 * to configure.
+			 */
+			nd_mapping_free_labels(nd_mapping);
+			list_splice_init(&list, &nd_mapping->labels);
+		}
 		nspm = kzalloc(sizeof(*nspm), GFP_KERNEL);
 		if (!nspm)
 			goto err;
@@ -2031,7 +2057,8 @@ static struct device **scan_labels(struct nd_region *nd_region)
 	} else if (is_memory(&nd_region->dev)) {
 		/* clean unselected labels */
 		for (i = 0; i < nd_region->ndr_mappings; i++) {
-			struct list_head *l, *e;
+			union nd_lsa_label *nd_label;
+			struct nd_label_ent *le, *e;
 			LIST_HEAD(list);
 			int j;
 
@@ -2042,10 +2069,25 @@ static struct device **scan_labels(struct nd_region *nd_region)
 			}
 
 			j = count;
-			list_for_each_safe(l, e, &nd_mapping->labels) {
+			list_for_each_entry_safe(le, e, &nd_mapping->labels,
+						 list) {
+				nd_label = (union nd_lsa_label *) le->label;
+
+				/* Preserve region labels */
+				if (is_region_label(ndd, nd_label)) {
+					list_move_tail(&le->list, &list);
+					continue;
+				}
+
+				/*
+				 * Once preserving selected ns label done
+				 * break out of loop
+				 */
 				if (!j--)
 					break;
-				list_move_tail(l, &list);
+
+				/* Preserve selected ns label */
+				list_move_tail(&le->list, &list);
 			}
 			nd_mapping_free_labels(nd_mapping);
 			list_splice_init(&list, &nd_mapping->labels);
-- 
2.34.1

[PATCH V3 11/20] nvdimm/region_label: Preserve cxl region information from region label

[Neeraj Kumar]

Preserve region information from region label during nvdimm_probe. This
preserved region information is used for creating cxl region to achieve
region persistency across reboot.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/dimm.c     |  4 ++++
 drivers/nvdimm/label.c    | 41 +++++++++++++++++++++++++++++++++++++++
 drivers/nvdimm/nd-core.h  |  2 ++
 drivers/nvdimm/nd.h       |  1 +
 include/linux/libnvdimm.h | 14 +++++++++++++
 5 files changed, 62 insertions(+)

diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
index bda22cb94e5b..30fc90591093 100644
--- a/drivers/nvdimm/dimm.c
+++ b/drivers/nvdimm/dimm.c
@@ -107,6 +107,10 @@ static int nvdimm_probe(struct device *dev)
 	if (rc)
 		goto err;
 
+	/* Preserve cxl region info if available */
+	if (ndd->cxl)
+		nvdimm_cxl_region_preserve(ndd);
+
 	return 0;
 
  err:
diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 935a0df5b47e..3250e3ecd973 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -473,6 +473,47 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
 	return 0;
 }
 
+int nvdimm_cxl_region_preserve(struct nvdimm_drvdata *ndd)
+{
+	struct nvdimm *nvdimm = to_nvdimm(ndd->dev);
+	struct cxl_pmem_region_params *p = &nvdimm->cxl_region_params;
+	struct nd_namespace_index *nsindex;
+	unsigned long *free;
+	u32 nslot, slot;
+
+	if (!preamble_current(ndd, &nsindex, &free, &nslot))
+		return 0; /* no label, nothing to preserve */
+
+	for_each_clear_bit_le(slot, free, nslot) {
+		union nd_lsa_label *nd_label;
+		struct cxl_region_label *region_label;
+		uuid_t rg_type, region_type;
+
+		nd_label = (union nd_lsa_label *) to_label(ndd, slot);
+		region_label = &nd_label->region_label;
+		uuid_parse(CXL_REGION_UUID, &region_type);
+		import_uuid(&rg_type, nd_label->region_label.type);
+
+		/* TODO: Currently preserving only one region */
+		if (uuid_equal(&region_type, &rg_type)) {
+			nvdimm->is_region_label = true;
+			import_uuid(&p->uuid, region_label->uuid);
+			p->flags = __le32_to_cpu(region_label->flags);
+			p->nlabel = __le16_to_cpu(region_label->nlabel);
+			p->position = __le16_to_cpu(region_label->position);
+			p->dpa = __le64_to_cpu(region_label->dpa);
+			p->rawsize = __le64_to_cpu(region_label->rawsize);
+			p->hpa = __le64_to_cpu(region_label->hpa);
+			p->slot = __le32_to_cpu(region_label->slot);
+			p->ig = __le32_to_cpu(region_label->ig);
+			p->align = __le32_to_cpu(region_label->align);
+			break;
+		}
+	}
+
+	return 0;
+}
+
 int nd_label_data_init(struct nvdimm_drvdata *ndd)
 {
 	size_t config_size, read_size, max_xfer, offset;
diff --git a/drivers/nvdimm/nd-core.h b/drivers/nvdimm/nd-core.h
index bfc6bfeb6e24..a73fac81531e 100644
--- a/drivers/nvdimm/nd-core.h
+++ b/drivers/nvdimm/nd-core.h
@@ -46,6 +46,8 @@ struct nvdimm {
 	} sec;
 	struct delayed_work dwork;
 	const struct nvdimm_fw_ops *fw_ops;
+	bool is_region_label;
+	struct cxl_pmem_region_params cxl_region_params;
 };
 
 static inline unsigned long nvdimm_security_flags(
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index c985f91728dd..2d0f6dd64c52 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -593,6 +593,7 @@ void nvdimm_set_locked(struct device *dev);
 void nvdimm_clear_locked(struct device *dev);
 int nvdimm_security_setup_events(struct device *dev);
 bool nvdimm_check_region_label_format(struct device *dev);
+int nvdimm_cxl_region_preserve(struct nvdimm_drvdata *ndd);
 #if IS_ENABLED(CONFIG_NVDIMM_KEYS)
 int nvdimm_security_unlock(struct device *dev);
 #else
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index bbf14a260c93..07ea2e3f821a 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -108,6 +108,20 @@ struct nd_cmd_desc {
 	int out_sizes[ND_CMD_MAX_ELEM];
 };
 
+struct cxl_pmem_region_params {
+	uuid_t uuid;
+	u32 flags;
+	u16 nlabel;
+	u16 position;
+	u64 dpa;
+	u64 rawsize;
+	u64 hpa;
+	u32 slot;
+	u32 ig;
+	u32 align;
+	int nr_targets;
+};
+
 struct nd_interleave_set {
 	/* v1.1 definition of the interleave-set-cookie algorithm */
 	u64 cookie1;
-- 
2.34.1

[PATCH V3 12/20] nvdimm/region_label: Export routine to fetch region information

[Neeraj Kumar]

CXL region information preserved from the LSA needs to be exported for
use by the CXL driver for CXL region re-creation.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/nvdimm/dimm_devs.c | 18 ++++++++++++++++++
 include/linux/libnvdimm.h  |  2 ++
 2 files changed, 20 insertions(+)

diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 918c3db93195..619c8ce56dce 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -280,6 +280,24 @@ void *nvdimm_provider_data(struct nvdimm *nvdimm)
 }
 EXPORT_SYMBOL_GPL(nvdimm_provider_data);
 
+bool nvdimm_has_cxl_region(struct nvdimm *nvdimm)
+{
+	if (nvdimm)
+		return nvdimm->is_region_label;
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(nvdimm_has_cxl_region);
+
+void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm)
+{
+	if (nvdimm)
+		return &nvdimm->cxl_region_params;
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(nvdimm_get_cxl_region_param);
+
 static ssize_t commands_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 07ea2e3f821a..3ffd50ab6ac4 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -330,6 +330,8 @@ int nvdimm_in_overwrite(struct nvdimm *nvdimm);
 bool is_nvdimm_sync(struct nd_region *nd_region);
 int nd_region_label_update(struct nd_region *nd_region);
 int nd_region_label_delete(struct nd_region *nd_region);
+bool nvdimm_has_cxl_region(struct nvdimm *nvdimm);
+void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm);
 
 static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
 		unsigned int buf_len, int *cmd_rc)
-- 
2.34.1

[PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Neeraj Kumar]

In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
used to get called at last in cxl_endpoint_port_probe(), which requires
cxl_nvd presence.

For cxl region persistency, region creation happens during nvdimm_probe
which need the completion of endpoint probe.

In order to accommodate both cxl pmem region auto-assembly and cxl region
persistency, refactored following

1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
   will be called only after successful completion of endpoint probe.

2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
   cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
   completion of endpoint probe and cxl_nvd presence before its call.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/core/region.c | 33 +++++++++++++++++++++++++++++++++
 drivers/cxl/cxl.h         |  4 ++++
 drivers/cxl/mem.c         | 24 +++++++++++++++---------
 drivers/cxl/port.c        | 39 +--------------------------------------
 4 files changed, 53 insertions(+), 47 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 7a0cead24490..c325aa827992 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -3606,6 +3606,39 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled)
 }
 EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL");
 
+static int discover_region(struct device *dev, void *unused)
+{
+	struct cxl_endpoint_decoder *cxled;
+	int rc;
+
+	if (!is_endpoint_decoder(dev))
+		return 0;
+
+	cxled = to_cxl_endpoint_decoder(dev);
+	if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0)
+		return 0;
+
+	if (cxled->state != CXL_DECODER_STATE_AUTO)
+		return 0;
+
+	/*
+	 * Region enumeration is opportunistic, if this add-event fails,
+	 * continue to the next endpoint decoder.
+	 */
+	rc = cxl_add_to_region(cxled);
+	if (rc)
+		dev_dbg(dev, "failed to add to region: %#llx-%#llx\n",
+			cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end);
+
+	return 0;
+}
+
+void cxl_region_discovery(struct cxl_port *port)
+{
+	device_for_each_child(&port->dev, NULL, discover_region);
+}
+EXPORT_SYMBOL_NS_GPL(cxl_region_discovery, "CXL");
+
 u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa)
 {
 	struct cxl_region_ref *iter;
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 4fe3df06f57a..b57597e55f7e 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -873,6 +873,7 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
 int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
 struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
 u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
+void cxl_region_discovery(struct cxl_port *port);
 #else
 static inline bool is_cxl_pmem_region(struct device *dev)
 {
@@ -895,6 +896,9 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint,
 {
 	return 0;
 }
+static inline void cxl_region_discovery(struct cxl_port *port)
+{
+}
 #endif
 
 void cxl_endpoint_parse_cdat(struct cxl_port *port);
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 6e6777b7bafb..54501616ff09 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -152,15 +152,6 @@ static int cxl_mem_probe(struct device *dev)
 		return -ENXIO;
 	}
 
-	if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) {
-		rc = devm_cxl_add_nvdimm(parent_port, cxlmd);
-		if (rc) {
-			if (rc == -ENODEV)
-				dev_info(dev, "PMEM disabled by platform\n");
-			return rc;
-		}
-	}
-
 	if (dport->rch)
 		endpoint_parent = parent_port->uport_dev;
 	else
@@ -184,6 +175,21 @@ static int cxl_mem_probe(struct device *dev)
 	if (rc)
 		dev_dbg(dev, "CXL memdev EDAC registration failed rc=%d\n", rc);
 
+	if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) {
+		rc = devm_cxl_add_nvdimm(parent_port, cxlmd);
+		if (rc) {
+			if (rc == -ENODEV)
+				dev_info(dev, "PMEM disabled by platform\n");
+			return rc;
+		}
+	}
+
+	/*
+	 * Now that all endpoint decoders are successfully enumerated, try to
+	 * assemble region autodiscovery from committed decoders.
+	 */
+	cxl_region_discovery(cxlmd->endpoint);
+
 	/*
 	 * The kernel may be operating out of CXL memory on this device,
 	 * there is no spec defined way to determine whether this device
diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c
index cf32dc50b7a6..07bb909b7d2e 100644
--- a/drivers/cxl/port.c
+++ b/drivers/cxl/port.c
@@ -30,33 +30,6 @@ static void schedule_detach(void *cxlmd)
 	schedule_cxl_memdev_detach(cxlmd);
 }
 
-static int discover_region(struct device *dev, void *unused)
-{
-	struct cxl_endpoint_decoder *cxled;
-	int rc;
-
-	if (!is_endpoint_decoder(dev))
-		return 0;
-
-	cxled = to_cxl_endpoint_decoder(dev);
-	if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0)
-		return 0;
-
-	if (cxled->state != CXL_DECODER_STATE_AUTO)
-		return 0;
-
-	/*
-	 * Region enumeration is opportunistic, if this add-event fails,
-	 * continue to the next endpoint decoder.
-	 */
-	rc = cxl_add_to_region(cxled);
-	if (rc)
-		dev_dbg(dev, "failed to add to region: %#llx-%#llx\n",
-			cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end);
-
-	return 0;
-}
-
 static int cxl_switch_port_probe(struct cxl_port *port)
 {
 	struct cxl_hdm *cxlhdm;
@@ -121,17 +94,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)
 	if (rc)
 		return rc;
 
-	rc = devm_cxl_enumerate_decoders(cxlhdm, &info);
-	if (rc)
-		return rc;
-
-	/*
-	 * Now that all endpoint decoders are successfully enumerated, try to
-	 * assemble regions from committed decoders
-	 */
-	device_for_each_child(&port->dev, NULL, discover_region);
-
-	return 0;
+	return devm_cxl_enumerate_decoders(cxlhdm, &info);
 }
 
 static int cxl_port_probe(struct device *dev)
-- 
2.34.1

[PATCH V3 14/20] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation

[Neeraj Kumar]

devm_cxl_pmem_add_region() is used to create cxl region based on region
information scanned from LSA.

devm_cxl_add_region() is used to just allocate cxlr and its fields are
filled later by userspace tool using device attributes (*_store()).

Inspiration for devm_cxl_pmem_add_region() is taken from these device
attributes (_store*) calls. It allocates cxlr and fills information
parsed from LSA and calls device_add(&cxlr->dev) to initiate further
region creation porbes

Renamed __create_region() to cxl_create_region() and make it an exported
routine. This will be used in later patch to create cxl region after
fetching region information from LSA.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/core/region.c | 127 ++++++++++++++++++++++++++++++++++++--
 drivers/cxl/cxl.h         |  12 ++++
 2 files changed, 134 insertions(+), 5 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index c325aa827992..d5c227ce7b09 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -2573,6 +2573,116 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd,
 	return ERR_PTR(rc);
 }
 
+static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size)
+{
+	int rc;
+
+	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
+	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
+	if (rc)
+		return rc;
+
+	if (!size)
+		return -EINVAL;
+
+	return alloc_hpa(cxlr, size);
+}
+
+static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size)
+{
+	int rc;
+
+	if (!size)
+		return -EINVAL;
+
+	if (!IS_ALIGNED(size, SZ_256M))
+		return -EINVAL;
+
+	rc = cxl_dpa_free(cxled);
+	if (rc)
+		return rc;
+
+	return cxl_dpa_alloc(cxled, size);
+}
+
+static struct cxl_region *
+devm_cxl_pmem_add_region(struct cxl_root_decoder *cxlrd, int id,
+			 enum cxl_partition_mode mode,
+			 enum cxl_decoder_type type,
+			 struct cxl_pmem_region_params *params,
+			 struct cxl_decoder *cxld)
+{
+	struct cxl_endpoint_decoder *cxled;
+	struct cxl_region_params *p;
+	struct cxl_port *root_port;
+	struct device *dev;
+	int rc;
+
+	struct cxl_region *cxlr __free(put_cxl_region) =
+		cxl_region_alloc(cxlrd, id);
+	if (IS_ERR(cxlr))
+		return cxlr;
+
+	cxlr->mode = mode;
+	cxlr->type = type;
+
+	dev = &cxlr->dev;
+	rc = dev_set_name(dev, "region%d", id);
+	if (rc)
+		return ERR_PTR(rc);
+
+	p = &cxlr->params;
+	p->uuid = params->uuid;
+	p->interleave_ways = params->nlabel;
+	p->interleave_granularity = params->ig;
+
+	rc = alloc_region_hpa(cxlr, params->rawsize);
+	if (rc)
+		return ERR_PTR(rc);
+
+	cxled = to_cxl_endpoint_decoder(&cxld->dev);
+
+	rc = cxl_dpa_set_part(cxled, CXL_PARTMODE_PMEM);
+	if (rc)
+		return ERR_PTR(rc);
+
+	rc = alloc_region_dpa(cxled, params->rawsize);
+	if (rc)
+		return ERR_PTR(rc);
+
+	/*
+	 * TODO: Currently we have support of interleave_way == 1, where
+	 * we can only have one region per mem device. It means mem device
+	 * position (params->position) will always be 0. It is therefore
+	 * attaching only one target at params->position
+	 */
+	if (params->position)
+		return ERR_PTR(-EINVAL);
+
+	rc = attach_target(cxlr, cxled, params->position, TASK_INTERRUPTIBLE);
+	if (rc)
+		return ERR_PTR(rc);
+
+	rc = __commit(cxlr);
+	if (rc)
+		return ERR_PTR(rc);
+
+	rc = device_add(dev);
+	if (rc)
+		return ERR_PTR(rc);
+
+	root_port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent);
+	rc = devm_add_action_or_reset(root_port->uport_dev,
+			unregister_region, cxlr);
+	if (rc)
+		return ERR_PTR(rc);
+
+	dev_dbg(root_port->uport_dev, "%s: created %s\n",
+		dev_name(&cxlrd->cxlsd.cxld.dev), dev_name(dev));
+
+	return no_free_ptr(cxlr);
+}
+
 static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf)
 {
 	return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id));
@@ -2590,8 +2700,10 @@ static ssize_t create_ram_region_show(struct device *dev,
 	return __create_region_show(to_cxl_root_decoder(dev), buf);
 }
 
-static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
-					  enum cxl_partition_mode mode, int id)
+struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
+				     enum cxl_partition_mode mode, int id,
+				     struct cxl_pmem_region_params *pmem_params,
+				     struct cxl_decoder *cxld)
 {
 	int rc;
 
@@ -2613,8 +2725,12 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
 		return ERR_PTR(-EBUSY);
 	}
 
+	if (pmem_params)
+		return devm_cxl_pmem_add_region(cxlrd, id, mode,
+				CXL_DECODER_HOSTONLYMEM, pmem_params, cxld);
 	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
 }
+EXPORT_SYMBOL_NS_GPL(cxl_create_region, "CXL");
 
 static ssize_t create_region_store(struct device *dev, const char *buf,
 				   size_t len, enum cxl_partition_mode mode)
@@ -2627,7 +2743,7 @@ static ssize_t create_region_store(struct device *dev, const char *buf,
 	if (rc != 1)
 		return -EINVAL;
 
-	cxlr = __create_region(cxlrd, mode, id);
+	cxlr = cxl_create_region(cxlrd, mode, id, NULL, NULL);
 	if (IS_ERR(cxlr))
 		return PTR_ERR(cxlr);
 
@@ -3523,8 +3639,9 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd,
 	struct cxl_region *cxlr;
 
 	do {
-		cxlr = __create_region(cxlrd, cxlds->part[part].mode,
-				       atomic_read(&cxlrd->region_id));
+		cxlr = cxl_create_region(cxlrd, cxlds->part[part].mode,
+					 atomic_read(&cxlrd->region_id),
+					 NULL, NULL);
 	} while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY);
 
 	if (IS_ERR(cxlr)) {
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index b57597e55f7e..3abadc3dc82e 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -874,6 +874,10 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
 struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
 u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
 void cxl_region_discovery(struct cxl_port *port);
+struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
+				     enum cxl_partition_mode mode, int id,
+				     struct cxl_pmem_region_params *params,
+				     struct cxl_decoder *cxld);
 #else
 static inline bool is_cxl_pmem_region(struct device *dev)
 {
@@ -899,6 +903,14 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint,
 static inline void cxl_region_discovery(struct cxl_port *port)
 {
 }
+static inline struct cxl_region *
+cxl_create_region(struct cxl_root_decoder *cxlrd,
+		  enum cxl_partition_mode mode, int id,
+		  struct cxl_pmem_region_params *params,
+		  struct cxl_decoder *cxld)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
 #endif
 
 void cxl_endpoint_parse_cdat(struct cxl_port *port);
-- 
2.34.1

[PATCH V3 15/20] cxl: Add a routine to find cxl root decoder on cxl bus using cxl port

[Neeraj Kumar]

Add cxl_find_root_decoder_by_port() to find root decoder on cxl bus.
It is used to find root decoder using cxl port.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/core/port.c | 27 +++++++++++++++++++++++++++
 drivers/cxl/cxl.h       |  1 +
 2 files changed, 28 insertions(+)

diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 8f36ff413f5d..647d9ce32b64 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -518,6 +518,33 @@ struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev)
 }
 EXPORT_SYMBOL_NS_GPL(to_cxl_switch_decoder, "CXL");
 
+static int match_root_decoder(struct device *dev, const void *data)
+{
+	return is_root_decoder(dev);
+}
+
+/**
+ * cxl_find_root_decoder_by_port() - find a cxl root decoder on cxl bus
+ * @port: any descendant port in CXL port topology
+ */
+struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port)
+{
+	struct cxl_root *cxl_root __free(put_cxl_root) = find_cxl_root(port);
+	struct device *dev;
+
+	if (!cxl_root)
+		return NULL;
+
+	dev = device_find_child(&cxl_root->port.dev, NULL, match_root_decoder);
+	if (!dev)
+		return NULL;
+
+	/* Release device ref taken via device_find_child() */
+	put_device(dev);
+	return to_cxl_root_decoder(dev);
+}
+EXPORT_SYMBOL_NS_GPL(cxl_find_root_decoder_by_port, "CXL");
+
 static void cxl_ep_release(struct cxl_ep *ep)
 {
 	put_device(ep->ep);
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 3abadc3dc82e..1eb1aca7c69f 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -866,6 +866,7 @@ struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev);
 bool is_cxl_nvdimm(struct device *dev);
 int devm_cxl_add_nvdimm(struct cxl_port *parent_port, struct cxl_memdev *cxlmd);
 struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port);
+struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port);
 
 #ifdef CONFIG_CXL_REGION
 bool is_cxl_pmem_region(struct device *dev);
-- 
2.34.1

[PATCH V3 16/20] cxl/mem: Preserve cxl root decoder during mem probe

[Neeraj Kumar]

Saved root decoder info is required for cxl region persistency

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/cxlmem.h | 1 +
 drivers/cxl/mem.c    | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 434031a0c1f7..25cb115b72bd 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -59,6 +59,7 @@ struct cxl_memdev {
 	struct cxl_nvdimm_bridge *cxl_nvb;
 	struct cxl_nvdimm *cxl_nvd;
 	struct cxl_port *endpoint;
+	struct cxl_root_decoder *cxlrd;
 	int id;
 	int depth;
 	u8 scrub_cycle;
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 54501616ff09..1a0da7253a24 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -152,6 +152,8 @@ static int cxl_mem_probe(struct device *dev)
 		return -ENXIO;
 	}
 
+	cxlmd->cxlrd = cxl_find_root_decoder_by_port(parent_port);
+
 	if (dport->rch)
 		endpoint_parent = parent_port->uport_dev;
 	else
-- 
2.34.1

[PATCH V3 17/20] cxl/pmem: Preserve region information into nd_set

[Neeraj Kumar]

Save region information stored in cxlr to nd_set during
cxl_pmem_region_probe in nd_set. This saved region information is being
stored into LSA, which will be used for cxl region persistence

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/pmem.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c
index e197883690ef..38a5bcdc68ce 100644
--- a/drivers/cxl/pmem.c
+++ b/drivers/cxl/pmem.c
@@ -377,6 +377,7 @@ static int cxl_pmem_region_probe(struct device *dev)
 	struct nd_mapping_desc mappings[CXL_DECODER_MAX_INTERLEAVE];
 	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
 	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	struct cxl_region_params *p = &cxlr->params;
 	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
 	struct cxl_pmem_region_info *info = NULL;
 	struct nd_interleave_set *nd_set;
@@ -465,12 +466,12 @@ static int cxl_pmem_region_probe(struct device *dev)
 	ndr_desc.num_mappings = cxlr_pmem->nr_mappings;
 	ndr_desc.mapping = mappings;
 
-	/*
-	 * TODO enable CXL labels which skip the need for 'interleave-set cookie'
-	 */
-	nd_set->cookie1 =
-		nd_fletcher64(info, sizeof(*info) * cxlr_pmem->nr_mappings, 0);
-	nd_set->cookie2 = nd_set->cookie1;
+	nd_set->uuid = p->uuid;
+	nd_set->interleave_ways = p->interleave_ways;
+	nd_set->interleave_granularity = p->interleave_granularity;
+	nd_set->res = p->res;
+	nd_set->nr_targets =  p->nr_targets;
+
 	ndr_desc.nd_set = nd_set;
 
 	cxlr_pmem->nd_region =
-- 
2.34.1

[PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Neeraj Kumar]

Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
Moved pmem_region related code from core/region.c to core/pmem_region.c
For region label update, need to create device attribute, which calls
nvdimm exported function thus making pmem_region dependent on libnvdimm.
Because of this dependency of pmem region on libnvdimm, segregated pmem
region related code from core/region.c

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/Kconfig            |  14 +++
 drivers/cxl/core/Makefile      |   1 +
 drivers/cxl/core/core.h        |   8 +-
 drivers/cxl/core/pmem_region.c | 203 +++++++++++++++++++++++++++++++++
 drivers/cxl/core/port.c        |   2 +-
 drivers/cxl/core/region.c      | 191 +------------------------------
 drivers/cxl/cxl.h              |  30 +++--
 tools/testing/cxl/Kbuild       |   1 +
 8 files changed, 250 insertions(+), 200 deletions(-)
 create mode 100644 drivers/cxl/core/pmem_region.c

diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
index 48b7314afdb8..532eaa1bbdd6 100644
--- a/drivers/cxl/Kconfig
+++ b/drivers/cxl/Kconfig
@@ -211,6 +211,20 @@ config CXL_REGION
 
 	  If unsure say 'y'
 
+config CXL_PMEM_REGION
+	bool "CXL: Pmem Region Support"
+	default CXL_BUS
+	depends on CXL_REGION
+	depends on PHYS_ADDR_T_64BIT
+	depends on BLK_DEV
+	select LIBNVDIMM
+	help
+	  Enable the CXL core to enumerate and provision CXL pmem regions.
+	  A CXL pmem region need to update region label into LSA. For LSA
+	  updation/deletion libnvdimm is required.
+
+	  If unsure say 'y'
+
 config CXL_REGION_INVALIDATION_TEST
 	bool "CXL: Region Cache Management Bypass (TEST)"
 	depends on CXL_REGION
diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile
index 5ad8fef210b5..399157beb917 100644
--- a/drivers/cxl/core/Makefile
+++ b/drivers/cxl/core/Makefile
@@ -17,6 +17,7 @@ cxl_core-y += cdat.o
 cxl_core-y += ras.o
 cxl_core-$(CONFIG_TRACING) += trace.o
 cxl_core-$(CONFIG_CXL_REGION) += region.o
+cxl_core-$(CONFIG_CXL_PMEM_REGION) += pmem_region.o
 cxl_core-$(CONFIG_CXL_MCE) += mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 5707cd60a8eb..536636a752dc 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -34,7 +34,6 @@ int cxl_decoder_detach(struct cxl_region *cxlr,
 #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr)
 #define CXL_REGION_TYPE(x) (&cxl_region_type)
 #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr),
-#define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type)
 #define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type)
 int cxl_region_init(void);
 void cxl_region_exit(void);
@@ -74,10 +73,15 @@ static inline void cxl_region_exit(void)
 #define CXL_REGION_ATTR(x) NULL
 #define CXL_REGION_TYPE(x) NULL
 #define SET_CXL_REGION_ATTR(x)
-#define CXL_PMEM_REGION_TYPE(x) NULL
 #define CXL_DAX_REGION_TYPE(x) NULL
 #endif
 
+#ifdef CONFIG_CXL_PMEM_REGION
+#define CXL_PMEM_REGION_TYPE (&cxl_pmem_region_type)
+#else
+#define CXL_PMEM_REGION_TYPE NULL
+#endif
+
 struct cxl_send_command;
 struct cxl_mem_query_commands;
 int cxl_query_cmd(struct cxl_mailbox *cxl_mbox,
diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
new file mode 100644
index 000000000000..55b80d587403
--- /dev/null
+++ b/drivers/cxl/core/pmem_region.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2020 Intel Corporation. */
+#include <linux/device.h>
+#include <linux/memregion.h>
+#include <cxlmem.h>
+#include <cxl.h>
+#include "core.h"
+
+/**
+ * DOC: cxl pmem region
+ *
+ * The core CXL PMEM region infrastructure supports persistent memory
+ * region creation using LIBNVDIMM subsystem. It has dependency on
+ * LIBNVDIMM, pmem region need updation of cxl region information into
+ * LSA. LIBNVDIMM dependency is only for pmem region, it is therefore
+ * need this separate file.
+ */
+
+bool is_cxl_pmem_region(struct device *dev)
+{
+	return dev->type == &cxl_pmem_region_type;
+}
+EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
+
+struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
+{
+	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
+			  "not a cxl_pmem_region device\n"))
+		return NULL;
+	return container_of(dev, struct cxl_pmem_region, dev);
+}
+EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
+
+static void cxl_pmem_region_release(struct device *dev)
+{
+	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
+	int i;
+
+	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
+		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
+
+		put_device(&cxlmd->dev);
+	}
+
+	kfree(cxlr_pmem);
+}
+
+static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
+	&cxl_base_attribute_group,
+	NULL,
+};
+
+const struct device_type cxl_pmem_region_type = {
+	.name = "cxl_pmem_region",
+	.release = cxl_pmem_region_release,
+	.groups = cxl_pmem_region_attribute_groups,
+};
+
+static struct lock_class_key cxl_pmem_region_key;
+
+static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
+{
+	struct cxl_region_params *p = &cxlr->params;
+	struct cxl_nvdimm_bridge *cxl_nvb;
+	struct device *dev;
+	int i;
+
+	guard(rwsem_read)(&cxl_rwsem.region);
+	if (p->state != CXL_CONFIG_COMMIT)
+		return -ENXIO;
+
+	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
+		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets),
+			GFP_KERNEL);
+	if (!cxlr_pmem)
+		return -ENOMEM;
+
+	cxlr_pmem->hpa_range.start = p->res->start;
+	cxlr_pmem->hpa_range.end = p->res->end;
+
+	/* Snapshot the region configuration underneath the cxl_region_rwsem */
+	cxlr_pmem->nr_mappings = p->nr_targets;
+	for (i = 0; i < p->nr_targets; i++) {
+		struct cxl_endpoint_decoder *cxled = p->targets[i];
+		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
+		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
+
+		/*
+		 * Regions never span CXL root devices, so by definition the
+		 * bridge for one device is the same for all.
+		 */
+		if (i == 0) {
+			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
+			if (!cxl_nvb)
+				return -ENODEV;
+			cxlr->cxl_nvb = cxl_nvb;
+		}
+		m->cxlmd = cxlmd;
+		get_device(&cxlmd->dev);
+		m->start = cxled->dpa_res->start;
+		m->size = resource_size(cxled->dpa_res);
+		m->position = i;
+	}
+
+	dev = &cxlr_pmem->dev;
+	device_initialize(dev);
+	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
+	device_set_pm_not_required(dev);
+	dev->parent = &cxlr->dev;
+	dev->bus = &cxl_bus_type;
+	dev->type = &cxl_pmem_region_type;
+	cxlr_pmem->cxlr = cxlr;
+	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
+
+	return 0;
+}
+
+static void cxlr_pmem_unregister(void *_cxlr_pmem)
+{
+	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
+	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
+
+	/*
+	 * Either the bridge is in ->remove() context under the device_lock(),
+	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
+	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
+	 * lock).
+	 */
+	device_lock_assert(&cxl_nvb->dev);
+	cxlr->cxlr_pmem = NULL;
+	cxlr_pmem->cxlr = NULL;
+	device_unregister(&cxlr_pmem->dev);
+}
+
+static void cxlr_release_nvdimm(void *_cxlr)
+{
+	struct cxl_region *cxlr = _cxlr;
+	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
+
+	scoped_guard(device, &cxl_nvb->dev) {
+		if (cxlr->cxlr_pmem)
+			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
+					    cxlr->cxlr_pmem);
+	}
+	cxlr->cxl_nvb = NULL;
+	put_device(&cxl_nvb->dev);
+}
+
+/**
+ * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
+ * @cxlr: parent CXL region for this pmem region bridge device
+ *
+ * Return: 0 on success negative error code on failure.
+ */
+int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
+{
+	struct cxl_pmem_region *cxlr_pmem;
+	struct cxl_nvdimm_bridge *cxl_nvb;
+	struct device *dev;
+	int rc;
+
+	rc = cxl_pmem_region_alloc(cxlr);
+	if (rc)
+		return rc;
+	cxlr_pmem = cxlr->cxlr_pmem;
+	cxl_nvb = cxlr->cxl_nvb;
+
+	dev = &cxlr_pmem->dev;
+	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
+	if (rc)
+		goto err;
+
+	rc = device_add(dev);
+	if (rc)
+		goto err;
+
+	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
+		dev_name(dev));
+
+	scoped_guard(device, &cxl_nvb->dev) {
+		if (cxl_nvb->dev.driver)
+			rc = devm_add_action_or_reset(&cxl_nvb->dev,
+						      cxlr_pmem_unregister,
+						      cxlr_pmem);
+		else
+			rc = -ENXIO;
+	}
+
+	if (rc)
+		goto err_bridge;
+
+	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
+	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
+
+err:
+	put_device(dev);
+err_bridge:
+	put_device(&cxl_nvb->dev);
+	cxlr->cxl_nvb = NULL;
+	return rc;
+}
+EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 647d9ce32b64..717de1d3f70e 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
 		return CXL_DEVICE_NVDIMM_BRIDGE;
 	if (dev->type == &cxl_nvdimm_type)
 		return CXL_DEVICE_NVDIMM;
-	if (dev->type == CXL_PMEM_REGION_TYPE())
+	if (dev->type == CXL_PMEM_REGION_TYPE)
 		return CXL_DEVICE_PMEM_REGION;
 	if (dev->type == CXL_DAX_REGION_TYPE())
 		return CXL_DEVICE_DAX_REGION;
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index d5c227ce7b09..d2ef7fcc19fe 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -38,8 +38,6 @@
  */
 static nodemask_t nodemask_region_seen = NODE_MASK_NONE;
 
-static struct cxl_region *to_cxl_region(struct device *dev);
-
 #define __ACCESS_ATTR_RO(_level, _name) {				\
 	.attr	= { .name = __stringify(_name), .mode = 0444 },		\
 	.show	= _name##_access##_level##_show,			\
@@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
 }
 EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
 
-static struct cxl_region *to_cxl_region(struct device *dev)
+struct cxl_region *to_cxl_region(struct device *dev)
 {
 	if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
 			  "not a cxl_region device\n"))
@@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
 
 	return container_of(dev, struct cxl_region, dev);
 }
+EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");
 
 static void unregister_region(void *_cxlr)
 {
@@ -2814,46 +2813,6 @@ static ssize_t delete_region_store(struct device *dev,
 }
 DEVICE_ATTR_WO(delete_region);
 
-static void cxl_pmem_region_release(struct device *dev)
-{
-	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
-	int i;
-
-	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
-		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
-
-		put_device(&cxlmd->dev);
-	}
-
-	kfree(cxlr_pmem);
-}
-
-static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
-	&cxl_base_attribute_group,
-	NULL,
-};
-
-const struct device_type cxl_pmem_region_type = {
-	.name = "cxl_pmem_region",
-	.release = cxl_pmem_region_release,
-	.groups = cxl_pmem_region_attribute_groups,
-};
-
-bool is_cxl_pmem_region(struct device *dev)
-{
-	return dev->type == &cxl_pmem_region_type;
-}
-EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
-
-struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
-{
-	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
-			  "not a cxl_pmem_region device\n"))
-		return NULL;
-	return container_of(dev, struct cxl_pmem_region, dev);
-}
-EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
-
 struct cxl_poison_context {
 	struct cxl_port *port;
 	int part;
@@ -3213,64 +3172,6 @@ static int region_offset_to_dpa_result(struct cxl_region *cxlr, u64 offset,
 	return -ENXIO;
 }
 
-static struct lock_class_key cxl_pmem_region_key;
-
-static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
-{
-	struct cxl_region_params *p = &cxlr->params;
-	struct cxl_nvdimm_bridge *cxl_nvb;
-	struct device *dev;
-	int i;
-
-	guard(rwsem_read)(&cxl_rwsem.region);
-	if (p->state != CXL_CONFIG_COMMIT)
-		return -ENXIO;
-
-	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
-		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets), GFP_KERNEL);
-	if (!cxlr_pmem)
-		return -ENOMEM;
-
-	cxlr_pmem->hpa_range.start = p->res->start;
-	cxlr_pmem->hpa_range.end = p->res->end;
-
-	/* Snapshot the region configuration underneath the cxl_rwsem.region */
-	cxlr_pmem->nr_mappings = p->nr_targets;
-	for (i = 0; i < p->nr_targets; i++) {
-		struct cxl_endpoint_decoder *cxled = p->targets[i];
-		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
-		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
-
-		/*
-		 * Regions never span CXL root devices, so by definition the
-		 * bridge for one device is the same for all.
-		 */
-		if (i == 0) {
-			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
-			if (!cxl_nvb)
-				return -ENODEV;
-			cxlr->cxl_nvb = cxl_nvb;
-		}
-		m->cxlmd = cxlmd;
-		get_device(&cxlmd->dev);
-		m->start = cxled->dpa_res->start;
-		m->size = resource_size(cxled->dpa_res);
-		m->position = i;
-	}
-
-	dev = &cxlr_pmem->dev;
-	device_initialize(dev);
-	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
-	device_set_pm_not_required(dev);
-	dev->parent = &cxlr->dev;
-	dev->bus = &cxl_bus_type;
-	dev->type = &cxl_pmem_region_type;
-	cxlr_pmem->cxlr = cxlr;
-	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
-
-	return 0;
-}
-
 static void cxl_dax_region_release(struct device *dev)
 {
 	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
@@ -3334,92 +3235,6 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	return cxlr_dax;
 }
 
-static void cxlr_pmem_unregister(void *_cxlr_pmem)
-{
-	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
-	struct cxl_region *cxlr = cxlr_pmem->cxlr;
-	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
-
-	/*
-	 * Either the bridge is in ->remove() context under the device_lock(),
-	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
-	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
-	 * lock).
-	 */
-	device_lock_assert(&cxl_nvb->dev);
-	cxlr->cxlr_pmem = NULL;
-	cxlr_pmem->cxlr = NULL;
-	device_unregister(&cxlr_pmem->dev);
-}
-
-static void cxlr_release_nvdimm(void *_cxlr)
-{
-	struct cxl_region *cxlr = _cxlr;
-	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
-
-	scoped_guard(device, &cxl_nvb->dev) {
-		if (cxlr->cxlr_pmem)
-			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
-					    cxlr->cxlr_pmem);
-	}
-	cxlr->cxl_nvb = NULL;
-	put_device(&cxl_nvb->dev);
-}
-
-/**
- * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
- * @cxlr: parent CXL region for this pmem region bridge device
- *
- * Return: 0 on success negative error code on failure.
- */
-static int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
-{
-	struct cxl_pmem_region *cxlr_pmem;
-	struct cxl_nvdimm_bridge *cxl_nvb;
-	struct device *dev;
-	int rc;
-
-	rc = cxl_pmem_region_alloc(cxlr);
-	if (rc)
-		return rc;
-	cxlr_pmem = cxlr->cxlr_pmem;
-	cxl_nvb = cxlr->cxl_nvb;
-
-	dev = &cxlr_pmem->dev;
-	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
-	if (rc)
-		goto err;
-
-	rc = device_add(dev);
-	if (rc)
-		goto err;
-
-	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
-		dev_name(dev));
-
-	scoped_guard(device, &cxl_nvb->dev) {
-		if (cxl_nvb->dev.driver)
-			rc = devm_add_action_or_reset(&cxl_nvb->dev,
-						      cxlr_pmem_unregister,
-						      cxlr_pmem);
-		else
-			rc = -ENXIO;
-	}
-
-	if (rc)
-		goto err_bridge;
-
-	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
-	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
-
-err:
-	put_device(dev);
-err_bridge:
-	put_device(&cxl_nvb->dev);
-	cxlr->cxl_nvb = NULL;
-	return rc;
-}
-
 static void cxlr_dax_unregister(void *_cxlr_dax)
 {
 	struct cxl_dax_region *cxlr_dax = _cxlr_dax;
@@ -3985,6 +3800,8 @@ static int cxl_region_probe(struct device *dev)
 			dev_dbg(&cxlr->dev, "CXL EDAC registration for region_id=%d failed\n",
 				cxlr->id);
 
+		if (!IS_ENABLED(CONFIG_CXL_PMEM_REGION))
+			return -EINVAL;
 		return devm_cxl_add_pmem_region(cxlr);
 	case CXL_PARTMODE_RAM:
 		rc = devm_cxl_region_edac_register(cxlr);
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 1eb1aca7c69f..0d576b359de6 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -825,6 +825,7 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,
 			struct cxl_endpoint_dvsec_info *info);
 
 bool is_cxl_region(struct device *dev);
+struct cxl_region *to_cxl_region(struct device *dev);
 
 extern const struct bus_type cxl_bus_type;
 
@@ -869,8 +870,6 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port);
 struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port);
 
 #ifdef CONFIG_CXL_REGION
-bool is_cxl_pmem_region(struct device *dev);
-struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
 int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
 struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
 u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
@@ -880,14 +879,6 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
 				     struct cxl_pmem_region_params *params,
 				     struct cxl_decoder *cxld);
 #else
-static inline bool is_cxl_pmem_region(struct device *dev)
-{
-	return false;
-}
-static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
-{
-	return NULL;
-}
 static inline int cxl_add_to_region(struct cxl_endpoint_decoder *cxled)
 {
 	return 0;
@@ -914,6 +905,25 @@ cxl_create_region(struct cxl_root_decoder *cxlrd,
 }
 #endif
 
+#ifdef CONFIG_CXL_PMEM_REGION
+bool is_cxl_pmem_region(struct device *dev);
+struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
+int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
+#else
+static inline bool is_cxl_pmem_region(struct device *dev)
+{
+	return false;
+}
+static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
+{
+	return NULL;
+}
+static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
+{
+	return 0;
+}
+#endif
+
 void cxl_endpoint_parse_cdat(struct cxl_port *port);
 void cxl_switch_parse_cdat(struct cxl_port *port);
 
diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild
index d07f14cb7aa4..cf58ada337b7 100644
--- a/tools/testing/cxl/Kbuild
+++ b/tools/testing/cxl/Kbuild
@@ -64,6 +64,7 @@ cxl_core-y += $(CXL_CORE_SRC)/cdat.o
 cxl_core-y += $(CXL_CORE_SRC)/ras.o
 cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
 cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o
+cxl_core-$(CONFIG_CXL_PMEM_REGION) += $(CXL_CORE_SRC)/pmem_region.o
 cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o
-- 
2.34.1

[PATCH V3 19/20] cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion

[Neeraj Kumar]

Using these attributes region label is added/deleted into LSA. These
attributes are called from userspace (ndctl) after region creation.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 Documentation/ABI/testing/sysfs-bus-cxl | 22 ++++++
 drivers/cxl/core/pmem_region.c          | 91 ++++++++++++++++++++++++-
 drivers/cxl/cxl.h                       |  1 +
 3 files changed, 113 insertions(+), 1 deletion(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
index 6b4e8c7a963d..d6080fcf843a 100644
--- a/Documentation/ABI/testing/sysfs-bus-cxl
+++ b/Documentation/ABI/testing/sysfs-bus-cxl
@@ -615,3 +615,25 @@ Description:
 		The count is persistent across power loss and wraps back to 0
 		upon overflow. If this file is not present, the device does not
 		have the necessary support for dirty tracking.
+
+
+What:		/sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_update
+Date:		Sept, 2025
+KernelVersion:	v6.17
+Contact:	linux-cxl@vger.kernel.org
+Description:
+		(RW) Write a boolean 'true' string value to this attribute to
+		update cxl region information into LSA as region label. It
+		uses nvdimm nd_region_label_update() to update cxl region
+		information saved during cxl region creation into LSA. This
+		attribute must be called at last during cxl region creation.
+
+
+What:		/sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_delete
+Date:		Sept, 2025
+KernelVersion:	v6.17
+Contact:	linux-cxl@vger.kernel.org
+Description:
+		(WO) When a boolean 'true' is written to this attribute then
+		pmem_region driver deletes cxl region label from LSA using
+		nvdimm nd_region_label_delete()
diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
index 55b80d587403..665b603c907b 100644
--- a/drivers/cxl/core/pmem_region.c
+++ b/drivers/cxl/core/pmem_region.c
@@ -45,9 +45,98 @@ static void cxl_pmem_region_release(struct device *dev)
 	kfree(cxlr_pmem);
 }
 
+static ssize_t region_label_update_store(struct device *dev,
+					 struct device_attribute *attr,
+					 const char *buf, size_t len)
+{
+	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
+	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	ssize_t rc;
+	bool update;
+
+	rc = kstrtobool(buf, &update);
+	if (rc)
+		return rc;
+
+	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
+	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
+	if (rc)
+		return rc;
+
+	/* Region not yet committed */
+	if (update && cxlr && cxlr->params.state != CXL_CONFIG_COMMIT) {
+		dev_dbg(dev, "region not committed, can't update into LSA\n");
+		return -ENXIO;
+	}
+
+	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
+		rc = nd_region_label_update(cxlr->cxlr_pmem->nd_region);
+		if (!rc)
+			cxlr->params.region_label_state = 1;
+	}
+
+	if (rc)
+		return rc;
+
+	return len;
+}
+
+static ssize_t region_label_update_show(struct device *dev,
+					struct device_attribute *attr,
+					char *buf)
+{
+	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
+	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	struct cxl_region_params *p = &cxlr->params;
+	ssize_t rc;
+
+	ACQUIRE(rwsem_read_intr, rwsem)(&cxl_rwsem.region);
+	rc = ACQUIRE_ERR(rwsem_read_intr, &rwsem);
+	if (rc)
+		return rc;
+
+	return sysfs_emit(buf, "%d\n", p->region_label_state);
+}
+static DEVICE_ATTR_RW(region_label_update);
+
+static ssize_t region_label_delete_store(struct device *dev,
+					 struct device_attribute *attr,
+					 const char *buf, size_t len)
+{
+	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
+	struct cxl_region *cxlr = cxlr_pmem->cxlr;
+	ssize_t rc;
+
+	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
+	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
+	if (rc)
+		return rc;
+
+	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
+		rc = nd_region_label_delete(cxlr->cxlr_pmem->nd_region);
+		if (rc)
+			return rc;
+		cxlr->params.region_label_state = 0;
+	}
+
+	return len;
+}
+static DEVICE_ATTR_WO(region_label_delete);
+
+static struct attribute *cxl_pmem_region_attrs[] = {
+	&dev_attr_region_label_update.attr,
+	&dev_attr_region_label_delete.attr,
+	NULL
+};
+
+static struct attribute_group cxl_pmem_region_group = {
+	.attrs = cxl_pmem_region_attrs,
+};
+
 static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
 	&cxl_base_attribute_group,
-	NULL,
+	&cxl_pmem_region_group,
+	NULL
 };
 
 const struct device_type cxl_pmem_region_type = {
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 0d576b359de6..f01f8c942fdf 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -484,6 +484,7 @@ enum cxl_config_state {
  */
 struct cxl_region_params {
 	enum cxl_config_state state;
+	int region_label_state;
 	uuid_t uuid;
 	int interleave_ways;
 	int interleave_granularity;
-- 
2.34.1

[PATCH V3 20/20] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem

[Neeraj Kumar]

Add support of CXL LSA 2.1 using NDD_REGION_LABELING flag. It creates
cxl region based on region information parsed from LSA.

Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
---
 drivers/cxl/core/pmem_region.c | 53 ++++++++++++++++++++++++++++++++++
 drivers/cxl/cxl.h              |  4 +++
 drivers/cxl/pmem.c             |  2 ++
 3 files changed, 59 insertions(+)

diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
index 665b603c907b..3ef9c7d15041 100644
--- a/drivers/cxl/core/pmem_region.c
+++ b/drivers/cxl/core/pmem_region.c
@@ -290,3 +290,56 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
 	return rc;
 }
 EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
+
+static int match_free_ep_decoder(struct device *dev, const void *data)
+{
+	struct cxl_decoder *cxld = to_cxl_decoder(dev);
+
+	return !cxld->region;
+}
+
+static struct cxl_decoder *cxl_find_free_ep_decoder(struct cxl_port *port)
+{
+	struct device *dev;
+
+	dev = device_find_child(&port->dev, NULL, match_free_ep_decoder);
+	if (!dev)
+		return NULL;
+
+	/* Release device ref taken via device_find_child() */
+	put_device(dev);
+	return to_cxl_decoder(dev);
+}
+
+void create_pmem_region(struct nvdimm *nvdimm)
+{
+	struct cxl_nvdimm *cxl_nvd;
+	struct cxl_memdev *cxlmd;
+	struct cxl_pmem_region_params *params;
+	struct cxl_root_decoder *cxlrd;
+	struct cxl_decoder *cxld;
+	struct cxl_region *cxlr;
+
+	if (!nvdimm_has_cxl_region(nvdimm))
+		return;
+
+	lockdep_assert_held(&cxl_rwsem.region);
+	cxl_nvd = nvdimm_provider_data(nvdimm);
+	params = nvdimm_get_cxl_region_param(nvdimm);
+	cxlmd = cxl_nvd->cxlmd;
+	cxlrd = cxlmd->cxlrd;
+
+	 /* TODO: Region creation support only for interleave way == 1 */
+	if (!(params->nlabel == 1))
+		dev_info(&cxlmd->dev,
+			 "Region Creation is not supported with iw > 1\n");
+	else {
+		cxld = cxl_find_free_ep_decoder(cxlmd->endpoint);
+		cxlr = cxl_create_region(cxlrd, CXL_PARTMODE_PMEM,
+					 atomic_read(&cxlrd->region_id),
+					 params, cxld);
+		if (IS_ERR(cxlr))
+			dev_info(&cxlmd->dev, "Region Creation failed\n");
+	}
+}
+EXPORT_SYMBOL_NS_GPL(create_pmem_region, "CXL");
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index f01f8c942fdf..0a87ea79742a 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -910,6 +910,7 @@ cxl_create_region(struct cxl_root_decoder *cxlrd,
 bool is_cxl_pmem_region(struct device *dev);
 struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
 int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
+void create_pmem_region(struct nvdimm *nvdimm);
 #else
 static inline bool is_cxl_pmem_region(struct device *dev)
 {
@@ -923,6 +924,9 @@ static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
 {
 	return 0;
 }
+static inline void create_pmem_region(struct nvdimm *nvdimm)
+{
+}
 #endif
 
 void cxl_endpoint_parse_cdat(struct cxl_port *port);
diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c
index 38a5bcdc68ce..0cdef01dbc68 100644
--- a/drivers/cxl/pmem.c
+++ b/drivers/cxl/pmem.c
@@ -135,6 +135,7 @@ static int cxl_nvdimm_probe(struct device *dev)
 		return rc;
 
 	set_bit(NDD_LABELING, &flags);
+	set_bit(NDD_REGION_LABELING, &flags);
 	set_bit(NDD_REGISTER_SYNC, &flags);
 	set_bit(ND_CMD_GET_CONFIG_SIZE, &cmd_mask);
 	set_bit(ND_CMD_GET_CONFIG_DATA, &cmd_mask);
@@ -155,6 +156,7 @@ static int cxl_nvdimm_probe(struct device *dev)
 		return -ENOMEM;
 
 	dev_set_drvdata(dev, nvdimm);
+	create_pmem_region(nvdimm);
 	return devm_add_action_or_reset(dev, unregister_nvdimm, nvdimm);
 }
 
-- 
2.34.1

Re: [PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Jonathan Cameron]

On Wed, 17 Sep 2025 19:10:56 +0530
Neeraj Kumar <s.neeraj@samsung.com> wrote:


Hi,
Not sure what difference between the two versions I'm seeing is.
Patch 02 is missing in both of them.

Jonathan

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Jonathan Cameron]

On Wed, 17 Sep 2025 19:11:01 +0530
Neeraj Kumar <s.neeraj@samsung.com> wrote:

> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
> Modified __pmem_label_update function using setter functions to update
> namespace label as per CXL LSA 2.1
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>

Note I'm not particularly familiar with the nvdimm code so this will ideally
want review from someone who is!

Jonathan

Re: [PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Jonathan Cameron]

On Wed, 17 Sep 2025 19:11:02 +0530
Neeraj Kumar <s.neeraj@samsung.com> wrote:

> Modified __pmem_label_update() to update region labels into LSA
> 
I'm struggling to follow the use of the union for the two label types
in much of this code.  To me if feels like that should only be a thing
at the init_labels() point on the basis I think it's only there that
we need to handle both in the same storage.

For the rest I'd just pay the small price of duplication that will
occur if you just split he functions up.

> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c          | 269 ++++++++++++++++++++++++++------
>  drivers/nvdimm/label.h          |  15 ++
>  drivers/nvdimm/namespace_devs.c |  12 ++
>  drivers/nvdimm/nd.h             |  38 ++++-
>  include/linux/libnvdimm.h       |   8 +
>  5 files changed, 289 insertions(+), 53 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 182f8c9a01bf..209c73f6b7e7 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -381,6 +381,16 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
>  	nsl_set_checksum(ndd, nd_label, sum);
>  }

> +static bool is_label_reapable(struct nd_interleave_set *nd_set,
> +			       struct nd_namespace_pmem *nspm,
> +			       struct nvdimm_drvdata *ndd,
> +			       union nd_lsa_label *label,
> +			       enum label_type ltype,
> +			       unsigned long *flags)
> +{
> +	switch (ltype) {
> +	case NS_LABEL_TYPE:
> +		if (test_and_clear_bit(ND_LABEL_REAP, flags) ||
> +		    nsl_uuid_equal(ndd, &label->ns_label, nspm->uuid))
> +			return true;
> +
> +		break;
> +	case RG_LABEL_TYPE:
> +		if (region_label_uuid_equal(&label->region_label,
> +		    &nd_set->uuid))
> +			return true;
> +
> +		break;
> +	}
> +
> +	return false;
> +}
> +
> +static int __pmem_label_update(struct nd_region *nd_region,
> +			       struct nd_mapping *nd_mapping,
> +			       struct nd_namespace_pmem *nspm,
> +			       int pos, unsigned long flags,
> +			       enum label_type ltype)
> +{
> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
> +	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +	struct nd_namespace_index *nsindex;
> +	struct nd_label_ent *label_ent;
> +	union nd_lsa_label *lsa_label;
> +	unsigned long *free;
> +	struct device *dev;
> +	u32 nslot, slot;
> +	size_t offset;
> +	int rc;
> +
> +	if (!preamble_next(ndd, &nsindex, &free, &nslot))
> +		return -ENXIO;
> +
>  	/* allocate and write the label to the staging (next) index */
>  	slot = nd_label_alloc_slot(ndd);
>  	if (slot == UINT_MAX)
>  		return -ENXIO;
>  	dev_dbg(ndd->dev, "allocated: %d\n", slot);
>  
> -	nd_label = to_label(ndd, slot);
> -	memset(nd_label, 0, sizeof_namespace_label(ndd));
> -	nsl_set_type(ndd, nd_label);
> -	nsl_set_uuid(ndd, nd_label, nspm->uuid);
> -	nsl_set_name(ndd, nd_label, nspm->alt_name);
> -	nsl_set_flags(ndd, nd_label, flags);
> -	nsl_set_nlabel(ndd, nd_label, nd_region->ndr_mappings);
> -	nsl_set_nrange(ndd, nd_label, 1);
> -	nsl_set_position(ndd, nd_label, pos);
> -	nsl_set_isetcookie(ndd, nd_label, cookie);
> -	nsl_set_rawsize(ndd, nd_label, resource_size(res));
> -	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
> -	nsl_set_dpa(ndd, nd_label, res->start);
> -	nsl_set_slot(ndd, nd_label, slot);
> -	nsl_set_alignment(ndd, nd_label, 0);
> -	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
> -	nsl_set_region_uuid(ndd, nd_label, NULL);
> -	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
> -	nsl_calculate_checksum(ndd, nd_label);
> -	nd_dbg_dpa(nd_region, ndd, res, "\n");
> +	lsa_label = (union nd_lsa_label *) to_label(ndd, slot);

This cast feels rather dubious.

If the union makes sense in general, then this should be changed
to return the union.

> +	memset(lsa_label, 0, sizeof_namespace_label(ndd));
> +
> +	switch (ltype) {
> +	case NS_LABEL_TYPE:
> +		dev = &nspm->nsio.common.dev;
> +		rc = namespace_label_update(nd_region, nd_mapping,
> +				nspm, pos, flags, &lsa_label->ns_label,
> +				nsindex, slot);
> +		if (rc)
> +			return rc;
> +
> +		break;
> +	case RG_LABEL_TYPE:
> +		dev = &nd_region->dev;
> +		region_label_update(nd_region, &lsa_label->region_label,
> +				    nd_mapping, pos, flags, slot);
> +
> +		break;
> +	}
>  
>  	/* update label */
> -	offset = nd_label_offset(ndd, nd_label);
> -	rc = nvdimm_set_config_data(ndd, offset, nd_label,
> +	offset = nd_label_offset(ndd, &lsa_label->ns_label);

This doesn't make sense as the type might be either an ns_label or a region_label.
If there is a generic header (I'm guessing there is) then define that as part of the
union with just the shared parts and use that to avoid any implication of what the type
is in calls like this.  Also make nd_label_offset() take the union not the specific bit.

> +	rc = nvdimm_set_config_data(ndd, offset, lsa_label,
>  			sizeof_namespace_label(ndd));
>  	if (rc < 0)
>  		return rc;
> @@ -955,8 +1054,10 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>  		if (!label_ent->label)
>  			continue;
> -		if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) ||
> -		    nsl_uuid_equal(ndd, label_ent->label, nspm->uuid))
> +
> +		if (is_label_reapable(nd_set, nspm, ndd,
> +				      (union nd_lsa_label *) label_ent->label,

If we are going with the union that label_ent->label should be a union that
we don't need to cast.

> +				      ltype, &label_ent->flags))
>  			reap_victim(nd_mapping, label_ent);
>  	}
>  
> @@ -966,19 +1067,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	if (rc)
>  		return rc;
>  
> -	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> -		if (!label_ent->label) {
> -			label_ent->label = nd_label;
> -			nd_label = NULL;
> -			break;
> -		}
> -	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> -			"failed to track label: %d\n",
> -			to_slot(ndd, nd_label));
> -	if (nd_label)
> -		rc = -ENXIO;
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
> +		if (label_ent->label)
> +			continue;

This flow change is unrelated to the rest here. I'd push it back
to the simpler patch that change the locking. Make sure to call it out there
though.  Or just don't do it and keep this patch a little more readable!

>  
> -	return rc;
> +		label_ent->label = &lsa_label->ns_label;
> +		lsa_label = NULL;
> +		break;
> +	}
> +	dev_WARN_ONCE(dev, lsa_label, "failed to track label: %d\n",
> +		      to_slot(ndd, &lsa_label->ns_label));
> +	if (lsa_label)
> +		return -ENXIO;
> +
> +	return 0;
>  }
>  
>  static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
> @@ -1068,6 +1170,21 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
>  }
>  
> +static int find_region_label_count(struct nvdimm_drvdata *ndd,
> +				   struct nd_mapping *nd_mapping)
> +{
> +	struct nd_label_ent *label_ent;
> +	int region_label_cnt = 0;
> +
> +	guard(mutex)(&nd_mapping->lock);
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> +		if (is_region_label(ndd,
> +		    (union nd_lsa_label *) label_ent->label))

As above. If it's a union make label_ent->label that union type.

> +			region_label_cnt++;
> +
> +	return region_label_cnt;
> +}
> +
>  int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  		struct nd_namespace_pmem *nspm, resource_size_t size)
>  {
> @@ -1076,6 +1193,7 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>  		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +		int region_label_cnt = 0;
>  		struct resource *res;
>  		int count = 0;
>  
> @@ -1091,12 +1209,19 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  				count++;
>  		WARN_ON_ONCE(!count);
>  
> -		rc = init_labels(nd_mapping, count);
> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
> +		/*
> +		 * init_labels() scan labels and allocate new label based
> +		 * on its second parameter (num_labels). Therefore to
> +		 * allocate new namespace label also include previously
> +		 * added region label
> +		 */
> +		rc = init_labels(nd_mapping, count + region_label_cnt);
>  		if (rc < 0)
>  			return rc;
>  
>  		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i,
> -				NSLABEL_FLAG_UPDATING);
> +				NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE);

The amount of shared code in __pmem_label_update() across the two types in
the union isn't that high.  I'd be tempted to just split it for simplicity.

>  		if (rc)
>  			return rc;
>  	}
> @@ -1108,7 +1233,47 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>  
> -		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0);
> +		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0,
> +				NS_LABEL_TYPE);
> +		if (rc)
> +			return rc;
> +	}
> +
> +	return 0;
> +}
> +
> +int nd_pmem_region_label_update(struct nd_region *nd_region)
> +{
> +	int i, rc;
> +
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +		int region_label_cnt = 0;
> +
> +		/* No need to update region label for non cxl format */
> +		if (!ndd->cxl)
> +			return 0;
> +
> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
> +		rc = init_labels(nd_mapping, region_label_cnt + 1);
> +		if (rc < 0)
> +			return rc;
> +
> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i,
> +				NSLABEL_FLAG_UPDATING, RG_LABEL_TYPE);
> +		if (rc)
> +			return rc;
> +	}
> +
> +	/* Clear the UPDATING flag per UEFI 2.7 expectations */
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +
> +		WARN_ON_ONCE(!ndd->cxl);
> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, 0,
> +				RG_LABEL_TYPE);
>  		if (rc)
>  			return rc;
>  	}
> diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
> index 0650fb4b9821..284e2a763b49 100644
> --- a/drivers/nvdimm/label.h
> +++ b/drivers/nvdimm/label.h
> @@ -30,6 +30,11 @@ enum {
>  	ND_NSINDEX_INIT = 0x1,
>  };
>  
> +enum label_type {
> +	RG_LABEL_TYPE,
> +	NS_LABEL_TYPE,
> +};
> +
>  /**
>   * struct nd_namespace_index - label set superblock
>   * @sig: NAMESPACE_INDEX\0
> @@ -183,6 +188,15 @@ struct nd_namespace_label {
>  	};
>  };
>  
> +/*
> + * LSA 2.1 format introduces region label, which can also reside
> + * into LSA along with only namespace label as per v1.1 and v1.2
> + */
> +union nd_lsa_label {
> +	struct nd_namespace_label ns_label;
> +	struct cxl_region_label region_label;
> +};

> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 3271b1c8569a..559f822ef24f 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -232,6 +232,18 @@ static ssize_t __alt_name_store(struct device *dev, const char *buf,
>  	return rc;
>  }
>  
> +int nd_region_label_update(struct nd_region *nd_region)
> +{
> +	int rc;
> +
> +	nvdimm_bus_lock(&nd_region->dev);
> +	rc = nd_pmem_region_label_update(nd_region);
> +	nvdimm_bus_unlock(&nd_region->dev);
Looks like it would be worth introducing a
DEFINE_GUARD() for this lock.

Not necessarily in this series however.
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL_GPL(nd_region_label_update);
> +
>  static int nd_namespace_label_update(struct nd_region *nd_region,
>  		struct device *dev)
>  {
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index e362611d82cc..f04c042dcfa9 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h

>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
> @@ -399,7 +432,10 @@ enum nd_label_flags {
>  struct nd_label_ent {
>  	struct list_head list;
>  	unsigned long flags;
> -	struct nd_namespace_label *label;
> +	union {
> +		struct nd_namespace_label *label;
> +		struct cxl_region_label *region_label;

It is a bit odd to have a union above of the two types in
here but then union the pointers here.

I'm also failing to find where region_label is used in this patch.


> +	};
>  };
>  
>  enum nd_mapping_lock_class {
_rc)

Re: [PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Dave Jiang]

On 9/17/25 7:50 AM, Jonathan Cameron wrote:
> On Wed, 17 Sep 2025 19:10:56 +0530
> Neeraj Kumar <s.neeraj@samsung.com> wrote:
> 
> 
> Hi,
> Not sure what difference between the two versions I'm seeing is.
> Patch 02 is missing in both of them.
> 
> Jonathan
> 

Neeraj,
If you resend the series again with patch 2 included, please add the prefix RESEND in the patch subjects. Thanks!

Re: [PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Ira Weiny]

Neeraj Kumar wrote:
> nd_label_base() was being used after typecasting with 'unsigned long'. Thus
> modified nd_label_base() to return 'unsigned long' instead of 'struct
> nd_namespace_label *'
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>

Acked-by: Ira Weiny <ira.weiny@intel.com>

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Ira Weiny]

Neeraj Kumar wrote:
> Updated mutex_lock() with guard(mutex)()

You are missing the 'why' justification here.

The detail is that __pmem_label_update() is getting more complex and this
change helps to reduce the complexity later.

However...

[snip]

> @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
>  		if (!label_ent)
>  			return -ENOMEM;
> -		mutex_lock(&nd_mapping->lock);
> +		guard(mutex)(&nd_mapping->lock);
>  		list_add_tail(&label_ent->list, &nd_mapping->labels);
> -		mutex_unlock(&nd_mapping->lock);

... this change is of little value.  And...

>  	}
>  
>  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
> @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>  		return 0;
>  
> -	mutex_lock(&nd_mapping->lock);
> +	guard(mutex)(&nd_mapping->lock);
>  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
>  		struct nd_namespace_label *nd_label = label_ent->label;
>  
> @@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  		nd_mapping_free_labels(nd_mapping);
>  		dev_dbg(ndd->dev, "no more active labels\n");
>  	}
> -	mutex_unlock(&nd_mapping->lock);

... this technically changes the scope of the lock to include writing the
index under the lock.

It does not affect anything AFAICS but really these last two changes
should be dropped from this patch.

Ira

>  
>  	return nd_label_write_index(ndd, ndd->ns_next,
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
> -- 
> 2.34.1
> 
> 

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Ira Weiny]

Neeraj Kumar wrote:
> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
> Modified __pmem_label_update function using setter functions to update
> namespace label as per CXL LSA 2.1

Again I'm curious as to why?

Is it to be able to use the setter's later?  I see a call to
nsl_set_type() added later in the series but then deleted in an even later
patch.  (??)

I don't have time ATM to really follow this through but giving a why in
the commit message may have made this a simple patch to review.  Now I'm
not clear if it is ok or not.

Ira

> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c |  3 +++
>  drivers/nvdimm/nd.h    | 23 +++++++++++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 3235562d0e1c..182f8c9a01bf 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -924,6 +924,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  
>  	nd_label = to_label(ndd, slot);
>  	memset(nd_label, 0, sizeof_namespace_label(ndd));
> +	nsl_set_type(ndd, nd_label);
>  	nsl_set_uuid(ndd, nd_label, nspm->uuid);
>  	nsl_set_name(ndd, nd_label, nspm->alt_name);
>  	nsl_set_flags(ndd, nd_label, flags);
> @@ -935,7 +936,9 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
>  	nsl_set_dpa(ndd, nd_label, res->start);
>  	nsl_set_slot(ndd, nd_label, slot);
> +	nsl_set_alignment(ndd, nd_label, 0);
>  	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
> +	nsl_set_region_uuid(ndd, nd_label, NULL);
>  	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
>  	nsl_calculate_checksum(ndd, nd_label);
>  	nd_dbg_dpa(nd_region, ndd, res, "\n");
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index 158809c2be9e..e362611d82cc 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -295,6 +295,29 @@ static inline const u8 *nsl_uuid_raw(struct nvdimm_drvdata *ndd,
>  	return nd_label->efi.uuid;
>  }
>  
> +static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
> +				struct nd_namespace_label *ns_label)
> +{
> +	if (ndd->cxl && ns_label)
> +		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);
> +}
> +
> +static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd,
> +				     struct nd_namespace_label *ns_label,
> +				     u32 align)
> +{
> +	if (ndd->cxl)
> +		ns_label->cxl.align = __cpu_to_le32(align);
> +}
> +
> +static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
> +				       struct nd_namespace_label *ns_label,
> +				       const uuid_t *uuid)
> +{
> +	if (ndd->cxl && uuid)
> +		export_uuid(ns_label->cxl.region_uuid, uuid);
> +}
> +
>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
> -- 
> 2.34.1
> 
> 

Re: [PATCH V3 01/20] nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label

[Dave Jiang]

On 9/17/25 6:40 AM, Neeraj Kumar wrote:
> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
> introduced in CXL 2.0 Spec, which contain region label along with
> namespace label.
> 
> NDD_LABELING flag is used for namespace. Introduced NDD_REGION_LABELING
> flag for region label. Based on these flags nvdimm driver performs
> operation on namespace label or region label.
> 
> NDD_REGION_LABELING will be utilized by cxl driver to enable LSA 2.1
> region label support
> 
> Accordingly updated label index version
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>

With the change noted below,
Reviewed-by: Dave Jiang <dave.jiang@intel.com>

> ---
>  drivers/nvdimm/dimm.c      |  1 +
>  drivers/nvdimm/dimm_devs.c |  7 +++++++
>  drivers/nvdimm/label.c     | 21 +++++++++++++++++----
>  drivers/nvdimm/nd.h        |  1 +
>  include/linux/libnvdimm.h  |  3 +++
>  5 files changed, 29 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
> index 91d9163ee303..bda22cb94e5b 100644
> --- a/drivers/nvdimm/dimm.c
> +++ b/drivers/nvdimm/dimm.c
> @@ -62,6 +62,7 @@ static int nvdimm_probe(struct device *dev)
>  	if (rc < 0)
>  		dev_dbg(dev, "failed to unlock dimm: %d\n", rc);
>  
> +	ndd->cxl = nvdimm_check_region_label_format(ndd->dev);
>  
>  	/*
>  	 * EACCES failures reading the namespace label-area-properties
> diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
> index 21498d461fde..918c3db93195 100644
> --- a/drivers/nvdimm/dimm_devs.c
> +++ b/drivers/nvdimm/dimm_devs.c
> @@ -18,6 +18,13 @@
>  
>  static DEFINE_IDA(dimm_ida);
>  
> +bool nvdimm_check_region_label_format(struct device *dev)

Should be called nvdimm_region_label_supported() since a bool return is expected instead of resolving the checking in the function.

DJ

> +{
> +	struct nvdimm *nvdimm = to_nvdimm(dev);
> +
> +	return test_bit(NDD_REGION_LABELING, &nvdimm->flags);
> +}
> +
>  /*
>   * Retrieve bus and dimm handle and return if this bus supports
>   * get_config_data commands
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 04f4a049599a..0a9b6c5cb2c3 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -688,11 +688,24 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>  		- (unsigned long) to_namespace_index(ndd, 0);
>  	nsindex->labeloff = __cpu_to_le64(offset);
>  	nsindex->nslot = __cpu_to_le32(nslot);
> -	nsindex->major = __cpu_to_le16(1);
> -	if (sizeof_namespace_label(ndd) < 256)
> +
> +	/* Set LSA Label Index Version */
> +	if (ndd->cxl) {
> +		/* CXL r3.2: Table 9-9 Label Index Block Layout */
> +		nsindex->major = __cpu_to_le16(2);
>  		nsindex->minor = __cpu_to_le16(1);
> -	else
> -		nsindex->minor = __cpu_to_le16(2);
> +	} else {
> +		nsindex->major = __cpu_to_le16(1);
> +		/*
> +		 * NVDIMM Namespace Specification
> +		 * Table 2: Namespace Label Index Block Fields
> +		 */
> +		if (sizeof_namespace_label(ndd) < 256)
> +			nsindex->minor = __cpu_to_le16(1);
> +		else /* UEFI 2.7: Label Index Block Definitions */
> +			nsindex->minor = __cpu_to_le16(2);
> +	}
> +
>  	nsindex->checksum = __cpu_to_le64(0);
>  	if (flags & ND_NSINDEX_INIT) {
>  		unsigned long *free = (unsigned long *) nsindex->free;
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index cc5c8f3f81e8..158809c2be9e 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -522,6 +522,7 @@ void nvdimm_set_labeling(struct device *dev);
>  void nvdimm_set_locked(struct device *dev);
>  void nvdimm_clear_locked(struct device *dev);
>  int nvdimm_security_setup_events(struct device *dev);
> +bool nvdimm_check_region_label_format(struct device *dev);
>  #if IS_ENABLED(CONFIG_NVDIMM_KEYS)
>  int nvdimm_security_unlock(struct device *dev);
>  #else
> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index 28f086c4a187..5696715c33bb 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -44,6 +44,9 @@ enum {
>  	/* dimm provider wants synchronous registration by __nvdimm_create() */
>  	NDD_REGISTER_SYNC = 8,
>  
> +	/* dimm supports region labels (LSA Format 2.1) */
> +	NDD_REGION_LABELING = 9,
> +
>  	/* need to set a limit somewhere, but yes, this is likely overkill */
>  	ND_IOCTL_MAX_BUFLEN = SZ_4M,
>  	ND_CMD_MAX_ELEM = 5,

Re: [PATCH V3 02/20] nvdimm/label: CXL labels skip the need for 'interleave-set cookie'

[Dave Jiang]

On 9/17/25 6:40 AM, Neeraj Kumar wrote:
> CXL LSA v2.1 utilizes the region labels stored in the LSA for interleave
> set configuration instead of interleave-set cookie used in previous LSA
> versions. As interleave-set cookie is not required for CXL LSA v2.1 format
> so skip its usage for CXL LSA 2.1 format
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Acked-by: Ira Weiny <ira.weiny@intel.com>

Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> ---
>  drivers/nvdimm/namespace_devs.c |  8 +++++++-
>  drivers/nvdimm/region_devs.c    | 10 ++++++++++
>  2 files changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 55cfbf1e0a95..3271b1c8569a 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -1684,7 +1684,13 @@ static struct device *create_namespace_pmem(struct nd_region *nd_region,
>  	int rc = 0;
>  	u16 i;
>  
> -	if (cookie == 0) {
> +	/*
> +	 * CXL LSA v2.1 utilizes the region label stored in the LSA for
> +	 * interleave set configuration. Whereas EFI LSA v1.1 & v1.2
> +	 * utilizes interleave-set cookie. i.e, CXL labels skip the
> +	 * need for 'interleave-set cookie'
> +	 */
> +	if (!ndd->cxl && cookie == 0) {
>  		dev_dbg(&nd_region->dev, "invalid interleave-set-cookie\n");
>  		return ERR_PTR(-ENXIO);
>  	}
> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
> index de1ee5ebc851..88275f352240 100644
> --- a/drivers/nvdimm/region_devs.c
> +++ b/drivers/nvdimm/region_devs.c
> @@ -858,6 +858,16 @@ u64 nd_region_interleave_set_cookie(struct nd_region *nd_region,
>  	if (!nd_set)
>  		return 0;
>  
> +	/*
> +	 * CXL LSA v2.1 utilizes the region label stored in the LSA for
> +	 * interleave set configuration. Whereas EFI LSA v1.1 & v1.2
> +	 * utilizes interleave-set cookie. i.e, CXL labels skip the
> +	 * need for 'interleave-set cookie'
> +	 */
> +	if (nsindex && __le16_to_cpu(nsindex->major) == 2
> +			&& __le16_to_cpu(nsindex->minor) == 1)
> +		return 0;
> +
>  	if (nsindex && __le16_to_cpu(nsindex->major) == 1
>  			&& __le16_to_cpu(nsindex->minor) == 1)
>  		return nd_set->cookie1;

Re: [PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Dave Jiang]

On 9/17/25 6:40 AM, Neeraj Kumar wrote:
> nd_label_base() was being used after typecasting with 'unsigned long'. Thus
> modified nd_label_base() to return 'unsigned long' instead of 'struct
> nd_namespace_label *'
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>

Just a nit below:


> ---
>  drivers/nvdimm/label.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 0a9b6c5cb2c3..668e1e146229 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -271,11 +271,11 @@ static void nd_label_copy(struct nvdimm_drvdata *ndd,
>  	memcpy(dst, src, sizeof_namespace_index(ndd));
>  }
>  
> -static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd)
> +static unsigned long nd_label_base(struct nvdimm_drvdata *ndd)
>  {
>  	void *base = to_namespace_index(ndd, 0);
>  
> -	return base + 2 * sizeof_namespace_index(ndd);
> +	return (unsigned long) (base + 2 * sizeof_namespace_index(ndd));

Space is not needed between casting and the var. Also applies to other instances in this commit.

DJ

>  }
>  
>  static int to_slot(struct nvdimm_drvdata *ndd,
> @@ -284,7 +284,7 @@ static int to_slot(struct nvdimm_drvdata *ndd,
>  	unsigned long label, base;
>  
>  	label = (unsigned long) nd_label;
> -	base = (unsigned long) nd_label_base(ndd);
> +	base = nd_label_base(ndd);
>  
>  	return (label - base) / sizeof_namespace_label(ndd);
>  }
> @@ -293,7 +293,7 @@ static struct nd_namespace_label *to_label(struct nvdimm_drvdata *ndd, int slot)
>  {
>  	unsigned long label, base;
>  
> -	base = (unsigned long) nd_label_base(ndd);
> +	base = nd_label_base(ndd);
>  	label = base + sizeof_namespace_label(ndd) * slot;
>  
>  	return (struct nd_namespace_label *) label;
> @@ -684,7 +684,7 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq,
>  			nd_label_next_nsindex(index))
>  		- (unsigned long) to_namespace_index(ndd, 0);
>  	nsindex->otheroff = __cpu_to_le64(offset);
> -	offset = (unsigned long) nd_label_base(ndd)
> +	offset = nd_label_base(ndd)
>  		- (unsigned long) to_namespace_index(ndd, 0);
>  	nsindex->labeloff = __cpu_to_le64(offset);
>  	nsindex->nslot = __cpu_to_le32(nslot);

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Updated mutex_lock() with guard(mutex)()

Need a bit more in the commit log on why the change so whomever reads the commit later on has an idea what is happening. 
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c | 36 +++++++++++++++++-------------------
>  1 file changed, 17 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 668e1e146229..3235562d0e1c 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -948,7 +948,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  		return rc;
>  
>  	/* Garbage collect the previous label */
> -	mutex_lock(&nd_mapping->lock);
> +	guard(mutex)(&nd_mapping->lock);
>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>  		if (!label_ent->label)
>  			continue;
> @@ -960,20 +960,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	/* update index */
>  	rc = nd_label_write_index(ndd, ndd->ns_next,
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
> -	if (rc == 0) {
> -		list_for_each_entry(label_ent, &nd_mapping->labels, list)
> -			if (!label_ent->label) {
> -				label_ent->label = nd_label;
> -				nd_label = NULL;
> -				break;
> -			}
> -		dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> -				"failed to track label: %d\n",
> -				to_slot(ndd, nd_label));
> -		if (nd_label)
> -			rc = -ENXIO;
> -	}
> -	mutex_unlock(&nd_mapping->lock);
> +	if (rc)
> +		return rc;
> +
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> +		if (!label_ent->label) {
> +			label_ent->label = nd_label;
> +			nd_label = NULL;
> +			break;
> +		}
> +	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> +			"failed to track label: %d\n",
> +			to_slot(ndd, nd_label));
> +	if (nd_label)
> +		rc = -ENXIO;
>  
>  	return rc;
>  }
> @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
>  		if (!label_ent)
>  			return -ENOMEM;
> -		mutex_lock(&nd_mapping->lock);
> +		guard(mutex)(&nd_mapping->lock);
>  		list_add_tail(&label_ent->list, &nd_mapping->labels);
> -		mutex_unlock(&nd_mapping->lock);

I would not mix and match old and new locking flow in a function. If you are going to convert, then do the whole function. I think earlier in this function you may need a scoped_guard() call.

>  	}
>  
>  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
> @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>  		return 0;
>  
> -	mutex_lock(&nd_mapping->lock);
> +	guard(mutex)(&nd_mapping->lock);

So this change now includes nd_label_write_index() in the lock context as well compare to the old code. So either you should use a scoped_guard() or create a helper function and move the block of code being locked to the helper function with guard() to avoid changing the original code flow.

DJ

>  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
>  		struct nd_namespace_label *nd_label = label_ent->label;
>  
> @@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  		nd_mapping_free_labels(nd_mapping);
>  		dev_dbg(ndd->dev, "no more active labels\n");
>  	}
> -	mutex_unlock(&nd_mapping->lock);
>  
>  	return nd_label_write_index(ndd, ndd->ns_next,
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
> Modified __pmem_label_update function using setter functions to update
> namespace label as per CXL LSA 2.1
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c |  3 +++
>  drivers/nvdimm/nd.h    | 23 +++++++++++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 3235562d0e1c..182f8c9a01bf 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -924,6 +924,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  
>  	nd_label = to_label(ndd, slot);
>  	memset(nd_label, 0, sizeof_namespace_label(ndd));
> +	nsl_set_type(ndd, nd_label);
>  	nsl_set_uuid(ndd, nd_label, nspm->uuid);
>  	nsl_set_name(ndd, nd_label, nspm->alt_name);
>  	nsl_set_flags(ndd, nd_label, flags);
> @@ -935,7 +936,9 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
>  	nsl_set_dpa(ndd, nd_label, res->start);
>  	nsl_set_slot(ndd, nd_label, slot);
> +	nsl_set_alignment(ndd, nd_label, 0);
>  	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
> +	nsl_set_region_uuid(ndd, nd_label, NULL);
>  	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
>  	nsl_calculate_checksum(ndd, nd_label);
>  	nd_dbg_dpa(nd_region, ndd, res, "\n");
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index 158809c2be9e..e362611d82cc 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -295,6 +295,29 @@ static inline const u8 *nsl_uuid_raw(struct nvdimm_drvdata *ndd,
>  	return nd_label->efi.uuid;
>  }
>  
> +static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
> +				struct nd_namespace_label *ns_label)
> +{
> +	if (ndd->cxl && ns_label)
> +		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);

Personally would prefer something like:

if (!(ndd->cxl && ns_label))
	return;
uuid_parse(....);

Also, no space needed after casting.

> +}
> +
> +static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd,
> +				     struct nd_namespace_label *ns_label,
> +				     u32 align)
> +{
> +	if (ndd->cxl)
> +		ns_label->cxl.align = __cpu_to_le32(align);

same comment as above
> +}
> +
> +static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
> +				       struct nd_namespace_label *ns_label,
> +				       const uuid_t *uuid)
> +{
> +	if (ndd->cxl && uuid)
> +		export_uuid(ns_label->cxl.region_uuid, uuid);

same comment as above

> +}
> +
>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Ira Weiny]

Dave Jiang wrote:

[snip]

> > @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
> >  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
> >  		if (!label_ent)
> >  			return -ENOMEM;
> > -		mutex_lock(&nd_mapping->lock);
> > +		guard(mutex)(&nd_mapping->lock);
> >  		list_add_tail(&label_ent->list, &nd_mapping->labels);
> > -		mutex_unlock(&nd_mapping->lock);
> 
> I would not mix and match old and new locking flow in a function. If you are going to convert, then do the whole function. I think earlier in this function you may need a scoped_guard() call.
> 

FWIW I would limit the changes to __pmem_label_update() because that is
the function which benefits from these changes.

> >  	}
> >  
> >  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
> > @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
> >  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
> >  		return 0;
> >  
> > -	mutex_lock(&nd_mapping->lock);
> > +	guard(mutex)(&nd_mapping->lock);
> 
> So this change now includes nd_label_write_index() in the lock context as well compare to the old code. So either you should use a scoped_guard() or create a helper function and move the block of code being locked to the helper function with guard() to avoid changing the original code flow.
> 

Sure you could do this but again I don't think these updates are worth
this amount of work right now.

Ira

> DJ
> 
> >  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
> >  		struct nd_namespace_label *nd_label = label_ent->label;
> >  
> > @@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
> >  		nd_mapping_free_labels(nd_mapping);
> >  		dev_dbg(ndd->dev, "no more active labels\n");
> >  	}
> > -	mutex_unlock(&nd_mapping->lock);
> >  
> >  	return nd_label_write_index(ndd, ndd->ns_next,
> >  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
> 
> 

Re: [PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 398 bytes --]

On 17/09/25 03:50PM, Jonathan Cameron wrote:
>On Wed, 17 Sep 2025 19:10:56 +0530
>Neeraj Kumar <s.neeraj@samsung.com> wrote:
>
>
>Hi,
>Not sure what difference between the two versions I'm seeing is.
>Patch 02 is missing in both of them.
>
>Jonathan

Hi Jonathan,

Because of some compliance check, Patch 02 got delayed. Now its visible
in lore. Can you please take a look at it.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 01/20] nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]

On 19/09/25 04:10PM, Dave Jiang wrote:
>
>
>On 9/17/25 6:40 AM, Neeraj Kumar wrote:
>> Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1
>> introduced in CXL 2.0 Spec, which contain region label along with
>> namespace label.
>>
>> NDD_LABELING flag is used for namespace. Introduced NDD_REGION_LABELING
>> flag for region label. Based on these flags nvdimm driver performs
>> operation on namespace label or region label.
>>
>> NDD_REGION_LABELING will be utilized by cxl driver to enable LSA 2.1
>> region label support
>>
>> Accordingly updated label index version
>>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>
>With the change noted below,
>Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>


<Snip>
>>  	/*
>>  	 * EACCES failures reading the namespace label-area-properties
>> diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
>> index 21498d461fde..918c3db93195 100644
>> --- a/drivers/nvdimm/dimm_devs.c
>> +++ b/drivers/nvdimm/dimm_devs.c
>> @@ -18,6 +18,13 @@
>>
>>  static DEFINE_IDA(dimm_ida);
>>
>> +bool nvdimm_check_region_label_format(struct device *dev)
>
>Should be called nvdimm_region_label_supported() since a bool return is expected instead of resolving the checking in the function.
>
>DJ

Thanks Jonathan and Dave for RB tag. Sure, I will rename it accordingly


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On 19/09/25 04:34PM, Dave Jiang wrote:
>
>
>On 9/17/25 6:40 AM, Neeraj Kumar wrote:
>> nd_label_base() was being used after typecasting with 'unsigned long'. Thus
>> modified nd_label_base() to return 'unsigned long' instead of 'struct
>> nd_namespace_label *'
>>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>
>Just a nit below:
>
>
>> ---
>>  drivers/nvdimm/label.c | 10 +++++-----
>>  1 file changed, 5 insertions(+), 5 deletions(-)
>>
>> -static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd)
>> +static unsigned long nd_label_base(struct nvdimm_drvdata *ndd)
>>  {
>>  	void *base = to_namespace_index(ndd, 0);
>>
>> -	return base + 2 * sizeof_namespace_index(ndd);
>> +	return (unsigned long) (base + 2 * sizeof_namespace_index(ndd));
>
>Space is not needed between casting and the var. Also applies to other instances in this commit.
>
>DJ

Thanks Jonathan, Ira and Dave for RB tag. Sure, I will fix this in next
patch-set.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]

On 19/09/25 04:55PM, Ira Weiny wrote:
>Neeraj Kumar wrote:
>> Updated mutex_lock() with guard(mutex)()
>
>You are missing the 'why' justification here.
>
>The detail is that __pmem_label_update() is getting more complex and this
>change helps to reduce the complexity later.
>
>However...
>
>[snip]
>
>> @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>>  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
>>  		if (!label_ent)
>>  			return -ENOMEM;
>> -		mutex_lock(&nd_mapping->lock);
>> +		guard(mutex)(&nd_mapping->lock);
>>  		list_add_tail(&label_ent->list, &nd_mapping->labels);
>> -		mutex_unlock(&nd_mapping->lock);
>
>... this change is of little value.  And...
>
>>  	}
>>
>>  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
>> @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>>  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>>  		return 0;
>>
>> -	mutex_lock(&nd_mapping->lock);
>> +	guard(mutex)(&nd_mapping->lock);
>>  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
>>  		struct nd_namespace_label *nd_label = label_ent->label;
>>
>> @@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>>  		nd_mapping_free_labels(nd_mapping);
>>  		dev_dbg(ndd->dev, "no more active labels\n");
>>  	}
>> -	mutex_unlock(&nd_mapping->lock);
>
>... this technically changes the scope of the lock to include writing the
>index under the lock.
>
>It does not affect anything AFAICS but really these last two changes
>should be dropped from this patch.
>
>Ira

Sure Ira, I will drop the last two hunks and elaborate commit message.

Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

On 20/09/25 12:44PM, Ira Weiny wrote:
>Dave Jiang wrote:
>
>[snip]
>
>> > @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>> >  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
>> >  		if (!label_ent)
>> >  			return -ENOMEM;
>> > -		mutex_lock(&nd_mapping->lock);
>> > +		guard(mutex)(&nd_mapping->lock);
>> >  		list_add_tail(&label_ent->list, &nd_mapping->labels);
>> > -		mutex_unlock(&nd_mapping->lock);
>>
>> I would not mix and match old and new locking flow in a function. If you are going to convert, then do the whole function. I think earlier in this function you may need a scoped_guard() call.
>>
>
>FWIW I would limit the changes to __pmem_label_update() because that is
>the function which benefits from these changes.
>
>> >  	}
>> >
>> >  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
>> > @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>> >  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>> >  		return 0;
>> >
>> > -	mutex_lock(&nd_mapping->lock);
>> > +	guard(mutex)(&nd_mapping->lock);
>>
>> So this change now includes nd_label_write_index() in the lock context as well compare to the old code. So either you should use a scoped_guard() or create a helper function and move the block of code being locked to the helper function with guard() to avoid changing the original code flow.
>>
>
>Sure you could do this but again I don't think these updates are worth
>this amount of work right now.
>

Yes Ira,

Adding change as per Dave's suggestion would require some extra code change which may not be required here.
I will fix the locking in __pmem_label_update() only.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

On 19/09/25 04:59PM, Dave Jiang wrote:
>
>
>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
>> Modified __pmem_label_update function using setter functions to update
>> namespace label as per CXL LSA 2.1
>>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>> ---
>>  drivers/nvdimm/label.c |  3 +++
>>  drivers/nvdimm/nd.h    | 23 +++++++++++++++++++++++
>>  2 files changed, 26 insertions(+)
>>
>> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
>> index 3235562d0e1c..182f8c9a01bf 100644
>> --- a/drivers/nvdimm/label.c
>> +++ b/drivers/nvdimm/label.c
>> +static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
>> +				struct nd_namespace_label *ns_label)
>> +{
>> +	if (ndd->cxl && ns_label)
>> +		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);
>
>Personally would prefer something like:
>
>if (!(ndd->cxl && ns_label))
>	return;
>uuid_parse(....);
>
>Also, no space needed after casting.
>
>> +}
>> +
>> +static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd,
>> +				     struct nd_namespace_label *ns_label,
>> +				     u32 align)
>> +{
>> +	if (ndd->cxl)
>> +		ns_label->cxl.align = __cpu_to_le32(align);
>
>same comment as above
>> +}
>> +
>> +static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
>> +				       struct nd_namespace_label *ns_label,
>> +				       const uuid_t *uuid)
>> +{
>> +	if (ndd->cxl && uuid)
>> +		export_uuid(ns_label->cxl.region_uuid, uuid);
>
>same comment as above

Sure Dave, I will fix it in next patch-set


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]

On 19/09/25 05:00PM, Ira Weiny wrote:
>Neeraj Kumar wrote:
>> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
>> Modified __pmem_label_update function using setter functions to update
>> namespace label as per CXL LSA 2.1
>
>Again I'm curious as to why?
>
>Is it to be able to use the setter's later?  I see a call to
>nsl_set_type() added later in the series but then deleted in an even later
>patch.  (??)
>
>I don't have time ATM to really follow this through but giving a why in
>the commit message may have made this a simple patch to review.  Now I'm
>not clear if it is ok or not.

Hi Ira,

Yes these setter functions are required because of namespace
modifications as per LSA 2.1.

Actually it looks deleted due to refactoring of __pmem_label_update() in
later patch.

I got your point, I will re-arrange it after __pmem_label_update() refactoring.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 7036 bytes --]

On 17/09/25 04:36PM, Jonathan Cameron wrote:
>On Wed, 17 Sep 2025 19:11:02 +0530
>Neeraj Kumar <s.neeraj@samsung.com> wrote:
>
>> Modified __pmem_label_update() to update region labels into LSA
>>
>I'm struggling to follow the use of the union for the two label types
>in much of this code.  To me if feels like that should only be a thing
>at the init_labels() point on the basis I think it's only there that
>we need to handle both in the same storage.
>
>For the rest I'd just pay the small price of duplication that will
>occur if you just split he functions up.

I got your point Jonathan, Let me revisit it again.

>
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>> ---
>>  drivers/nvdimm/label.c          | 269 ++++++++++++++++++++++++++------
>>  drivers/nvdimm/label.h          |  15 ++
>>  drivers/nvdimm/namespace_devs.c |  12 ++
>>  drivers/nvdimm/nd.h             |  38 ++++-
>>  include/linux/libnvdimm.h       |   8 +
>>  5 files changed, 289 insertions(+), 53 deletions(-)
>>

[snip]

>> -	nsl_set_position(ndd, nd_label, pos);
>> -	nsl_set_isetcookie(ndd, nd_label, cookie);
>> -	nsl_set_rawsize(ndd, nd_label, resource_size(res));
>> -	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
>> -	nsl_set_dpa(ndd, nd_label, res->start);
>> -	nsl_set_slot(ndd, nd_label, slot);
>> -	nsl_set_alignment(ndd, nd_label, 0);
>> -	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
>> -	nsl_set_region_uuid(ndd, nd_label, NULL);
>> -	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
>> -	nsl_calculate_checksum(ndd, nd_label);
>> -	nd_dbg_dpa(nd_region, ndd, res, "\n");
>> +	lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
>
>This cast feels rather dubious.
>
>If the union makes sense in general, then this should be changed
>to return the union.

Sure, I will fix it in next patch-set

>
>> +	memset(lsa_label, 0, sizeof_namespace_label(ndd));
>> +
>> +	switch (ltype) {
>> +	case NS_LABEL_TYPE:
>> +		dev = &nspm->nsio.common.dev;
>> +		rc = namespace_label_update(nd_region, nd_mapping,
>> +				nspm, pos, flags, &lsa_label->ns_label,
>> +				nsindex, slot);
>> +		if (rc)
>> +			return rc;
>> +
>> +		break;
>> +	case RG_LABEL_TYPE:
>> +		dev = &nd_region->dev;
>> +		region_label_update(nd_region, &lsa_label->region_label,
>> +				    nd_mapping, pos, flags, slot);
>> +
>> +		break;
>> +	}
>>
>>  	/* update label */
>> -	offset = nd_label_offset(ndd, nd_label);
>> -	rc = nvdimm_set_config_data(ndd, offset, nd_label,
>> +	offset = nd_label_offset(ndd, &lsa_label->ns_label);
>
>This doesn't make sense as the type might be either an ns_label or a region_label.
>If there is a generic header (I'm guessing there is) then define that as part of the
>union with just the shared parts and use that to avoid any implication of what the type
>is in calls like this.  Also make nd_label_offset() take the union not the specific bit.

Okay, I will update the signature of nd_label_offset() to use union and not the specific bit

>
>> +	rc = nvdimm_set_config_data(ndd, offset, lsa_label,
>>  			sizeof_namespace_label(ndd));
>>  	if (rc < 0)
>>  		return rc;
>> @@ -955,8 +1054,10 @@ static int __pmem_label_update(struct nd_region *nd_region,
>>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>>  		if (!label_ent->label)
>>  			continue;
>> -		if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) ||
>> -		    nsl_uuid_equal(ndd, label_ent->label, nspm->uuid))
>> +
>> +		if (is_label_reapable(nd_set, nspm, ndd,
>> +				      (union nd_lsa_label *) label_ent->label,
>
>If we are going with the union that label_ent->label should be a union that
>we don't need to cast.

Sure, I will fix this in next patch-set

>
>> +				      ltype, &label_ent->flags))
>>  			reap_victim(nd_mapping, label_ent);
>>  	}
>>
>> @@ -966,19 +1067,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>>  	if (rc)
>>  		return rc;
>>
>> -	list_for_each_entry(label_ent, &nd_mapping->labels, list)
>> -		if (!label_ent->label) {
>> -			label_ent->label = nd_label;
>> -			nd_label = NULL;
>> -			break;
>> -		}
>> -	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
>> -			"failed to track label: %d\n",
>> -			to_slot(ndd, nd_label));
>> -	if (nd_label)
>> -		rc = -ENXIO;
>> +	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>> +		if (label_ent->label)
>> +			continue;
>
>This flow change is unrelated to the rest here. I'd push it back
>to the simpler patch that change the locking. Make sure to call it out there
>though.  Or just don't do it and keep this patch a little more readable!

Yes, I will fix this in previous patch.

>> @@ -1091,12 +1209,19 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>>  				count++;
>>  		WARN_ON_ONCE(!count);
>>
>> -		rc = init_labels(nd_mapping, count);
>> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
>> +		/*
>> +		 * init_labels() scan labels and allocate new label based
>> +		 * on its second parameter (num_labels). Therefore to
>> +		 * allocate new namespace label also include previously
>> +		 * added region label
>> +		 */
>> +		rc = init_labels(nd_mapping, count + region_label_cnt);
>>  		if (rc < 0)
>>  			return rc;
>>
>>  		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i,
>> -				NSLABEL_FLAG_UPDATING);
>> +				NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE);
>
>The amount of shared code in __pmem_label_update() across the two types in
>the union isn't that high.  I'd be tempted to just split it for simplicity.

Sure, I will split it out in next patch-set.

>
>>  		if (rc)
>>  			return rc;
>>  	}
>>
[snip]
>> +int nd_region_label_update(struct nd_region *nd_region)
>> +{
>> +	int rc;
>> +
>> +	nvdimm_bus_lock(&nd_region->dev);
>> +	rc = nd_pmem_region_label_update(nd_region);
>> +	nvdimm_bus_unlock(&nd_region->dev);
>Looks like it would be worth introducing a
>DEFINE_GUARD() for this lock.
>
>Not necessarily in this series however.

Yes, Fixing it here will add some extra change.

>> +
>> +	return rc;
>> +}
>> +EXPORT_SYMBOL_GPL(nd_region_label_update);
>> +
>>  static int nd_namespace_label_update(struct nd_region *nd_region,
>>  		struct device *dev)
>>  {
>> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
>> index e362611d82cc..f04c042dcfa9 100644
>> --- a/drivers/nvdimm/nd.h
>> +++ b/drivers/nvdimm/nd.h
>
>>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
>> @@ -399,7 +432,10 @@ enum nd_label_flags {
>>  struct nd_label_ent {
>>  	struct list_head list;
>>  	unsigned long flags;
>> -	struct nd_namespace_label *label;
>> +	union {
>> +		struct nd_namespace_label *label;
>> +		struct cxl_region_label *region_label;
>
>It is a bit odd to have a union above of the two types in
>here but then union the pointers here.

Yes Jonathan, I will replace this unnamed union with "union nd_lsa_label". I will
also help in avoid extra typecasting in is_region_label() and is_label_reapable()


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 07/20] nvdimm/region_label: Add region label delete support

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Added LSA v2.1 format region label deletion routine. This function is
> used to delete region label from LSA
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c          | 79 ++++++++++++++++++++++++++++++---
>  drivers/nvdimm/label.h          |  1 +
>  drivers/nvdimm/namespace_devs.c | 12 +++++
>  drivers/nvdimm/nd.h             |  6 +++
>  include/linux/libnvdimm.h       |  1 +
>  5 files changed, 92 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 209c73f6b7e7..d33db96ba8ba 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -1126,11 +1126,13 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>  	return max(num_labels, old_num_labels);
>  }
>  
> -static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
> +static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid,
> +		      enum label_type ltype)
>  {
>  	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>  	struct nd_label_ent *label_ent, *e;
>  	struct nd_namespace_index *nsindex;
> +	union nd_lsa_label *nd_label;
>  	unsigned long *free;
>  	LIST_HEAD(list);
>  	u32 nslot, slot;
> @@ -1145,15 +1147,28 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  
>  	guard(mutex)(&nd_mapping->lock);
>  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
> -		struct nd_namespace_label *nd_label = label_ent->label;
> +		nd_label = (union nd_lsa_label *) label_ent->label;
>  
>  		if (!nd_label)
>  			continue;
>  		active++;
> -		if (!nsl_uuid_equal(ndd, nd_label, uuid))
> -			continue;
> +
> +		switch (ltype) {
> +		case NS_LABEL_TYPE:
> +			if (!nsl_uuid_equal(ndd, &nd_label->ns_label, uuid))
> +				continue;
> +
> +			break;
> +		case RG_LABEL_TYPE:
> +			if (!region_label_uuid_equal(&nd_label->region_label,
> +			    uuid))
> +				continue;
> +
> +			break;
> +		}
> +
>  		active--;
> -		slot = to_slot(ndd, nd_label);
> +		slot = to_slot(ndd, &nd_label->ns_label);
>  		nd_label_free_slot(ndd, slot);
>  		dev_dbg(ndd->dev, "free: %d\n", slot);
>  		list_move_tail(&label_ent->list, &list);
> @@ -1161,7 +1176,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  	}
>  	list_splice_tail_init(&list, &nd_mapping->labels);
>  
> -	if (active == 0) {
> +	if ((ltype == NS_LABEL_TYPE) && (active == 0)) {
>  		nd_mapping_free_labels(nd_mapping);
>  		dev_dbg(ndd->dev, "no more active labels\n");
>  	}
> @@ -1198,7 +1213,8 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  		int count = 0;
>  
>  		if (size == 0) {
> -			rc = del_labels(nd_mapping, nspm->uuid);
> +			rc = del_labels(nd_mapping, nspm->uuid,
> +					NS_LABEL_TYPE);
>  			if (rc)
>  				return rc;
>  			continue;
> @@ -1281,6 +1297,55 @@ int nd_pmem_region_label_update(struct nd_region *nd_region)
>  	return 0;
>  }
>  
> +int nd_pmem_region_label_delete(struct nd_region *nd_region)
> +{
> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
> +	struct nd_label_ent *label_ent;
> +	int ns_region_cnt = 0;
> +	int i, rc;
> +
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +
> +		/* Find non cxl format supported ndr_mappings */
> +		if (!ndd->cxl) {
> +			dev_info(&nd_region->dev, "Unsupported region label\n");
> +			return -EINVAL;
> +		}
> +
> +		/* Find if any NS label using this region */
> +		guard(mutex)(&nd_mapping->lock);
> +		list_for_each_entry(label_ent, &nd_mapping->labels, list) {
> +			if (!label_ent->label)
> +				continue;
> +
> +			/*
> +			 * Check if any available NS labels has same
> +			 * region_uuid in LSA
> +			 */
> +			if (nsl_region_uuid_equal(label_ent->label,
> +						  &nd_set->uuid))
> +				ns_region_cnt++;

Why not just return -EBUSY here immediately? It seems the code returns -EBUSY as long as there's 1 or more below.

> +		}
> +	}
> +
> +	if (ns_region_cnt) {
> +		dev_dbg(&nd_region->dev, "Region/Namespace label in use\n");
> +		return -EBUSY;
> +	}
> +
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +
> +		rc = del_labels(nd_mapping, &nd_set->uuid, RG_LABEL_TYPE);
> +		if (rc)
> +			return rc;
> +	}

Can this be folded into the for loop above or does it a full pass to check before starting the label deletion process?

DJ
> +
> +	return 0;
> +}
> +
>  int __init nd_label_init(void)
>  {
>  	WARN_ON(guid_parse(NVDIMM_BTT_GUID, &nvdimm_btt_guid));
> diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
> index 284e2a763b49..276dd822e142 100644
> --- a/drivers/nvdimm/label.h
> +++ b/drivers/nvdimm/label.h
> @@ -238,4 +238,5 @@ struct nd_namespace_pmem;
>  int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  		struct nd_namespace_pmem *nspm, resource_size_t size);
>  int nd_pmem_region_label_update(struct nd_region *nd_region);
> +int nd_pmem_region_label_delete(struct nd_region *nd_region);
>  #endif /* __LABEL_H__ */
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 559f822ef24f..564a73b1da41 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -244,6 +244,18 @@ int nd_region_label_update(struct nd_region *nd_region)
>  }
>  EXPORT_SYMBOL_GPL(nd_region_label_update);
>  
> +int nd_region_label_delete(struct nd_region *nd_region)
> +{
> +	int rc;
> +
> +	nvdimm_bus_lock(&nd_region->dev);
> +	rc = nd_pmem_region_label_delete(nd_region);
> +	nvdimm_bus_unlock(&nd_region->dev);
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL_GPL(nd_region_label_delete);
> +
>  static int nd_namespace_label_update(struct nd_region *nd_region,
>  		struct device *dev)
>  {
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index f04c042dcfa9..046063ea08b6 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -331,6 +331,12 @@ static inline bool is_region_label(struct nvdimm_drvdata *ndd,
>  	return uuid_equal(&region_type, ns_type);
>  }
>  
> +static inline bool nsl_region_uuid_equal(struct nd_namespace_label *ns_label,
> +					 const uuid_t *uuid)
> +{
> +	return uuid_equal((uuid_t *) ns_label->cxl.region_uuid, uuid);
> +}
> +
>  static inline bool
>  region_label_uuid_equal(struct cxl_region_label *region_label,
>  			const uuid_t *uuid)
> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index 2c213b9dac66..bbf14a260c93 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -315,6 +315,7 @@ int nvdimm_has_cache(struct nd_region *nd_region);
>  int nvdimm_in_overwrite(struct nvdimm *nvdimm);
>  bool is_nvdimm_sync(struct nd_region *nd_region);
>  int nd_region_label_update(struct nd_region *nd_region);
> +int nd_region_label_delete(struct nd_region *nd_region);
>  
>  static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
>  		unsigned int buf_len, int *cmd_rc)

Re: [PATCH V3 08/20] nvdimm/label: Include region label in slot validation

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Slot validation routine validates label slot by calculating label
> checksum. It was only validating namespace label. This changeset also
> validates region label if present.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> ---
>  drivers/nvdimm/label.c | 72 ++++++++++++++++++++++++++++++++----------
>  drivers/nvdimm/nd.h    |  5 +++
>  2 files changed, 60 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index d33db96ba8ba..5e476154cf81 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -359,7 +359,7 @@ static bool nsl_validate_checksum(struct nvdimm_drvdata *ndd,
>  {
>  	u64 sum, sum_save;
>  
> -	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
> +	if (!efi_namespace_label_has(ndd, checksum))
>  		return true;
>  
>  	sum_save = nsl_get_checksum(ndd, nd_label);
> @@ -374,13 +374,25 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
>  {
>  	u64 sum;
>  
> -	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
> +	if (!efi_namespace_label_has(ndd, checksum))
>  		return;
>  	nsl_set_checksum(ndd, nd_label, 0);
>  	sum = nd_fletcher64(nd_label, sizeof_namespace_label(ndd), 1);
>  	nsl_set_checksum(ndd, nd_label, sum);
>  }
>  
> +static bool region_label_validate_checksum(struct nvdimm_drvdata *ndd,
> +				struct cxl_region_label *region_label)
> +{
> +	u64 sum, sum_save;
> +
> +	sum_save = region_label_get_checksum(region_label);
> +	region_label_set_checksum(region_label, 0);
> +	sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1);
> +	region_label_set_checksum(region_label, sum_save);
> +	return sum == sum_save;
> +}
> +
>  static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>  				   struct cxl_region_label *region_label)
>  {
> @@ -392,16 +404,30 @@ static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>  }
>  
>  static bool slot_valid(struct nvdimm_drvdata *ndd,
> -		struct nd_namespace_label *nd_label, u32 slot)
> +		       union nd_lsa_label *lsa_label, u32 slot)
>  {
> +	struct cxl_region_label *region_label = &lsa_label->region_label;
> +	struct nd_namespace_label *nd_label = &lsa_label->ns_label;
> +	char *label_name;
>  	bool valid;
>  
>  	/* check that we are written where we expect to be written */
> -	if (slot != nsl_get_slot(ndd, nd_label))
> -		return false;
> -	valid = nsl_validate_checksum(ndd, nd_label);
> +	if (is_region_label(ndd, lsa_label)) {
> +		label_name = "rg";

I suggest create a static string table enumerated by 'enum label_type'. That way you can directly address it by 'label_name[ltype]' when being used instead of assigning at run time. And maybe just use "region" and "namespace" since it's for debug output and we don't need to shorten it. 

DJ

> +		if (slot != region_label_get_slot(region_label))
> +			return false;
> +		valid = region_label_validate_checksum(ndd, region_label);
> +	} else {
> +		label_name = "ns";
> +		if (slot != nsl_get_slot(ndd, nd_label))
> +			return false;
> +		valid = nsl_validate_checksum(ndd, nd_label);
> +	}
> +
>  	if (!valid)
> -		dev_dbg(ndd->dev, "fail checksum. slot: %d\n", slot);
> +		dev_dbg(ndd->dev, "%s label checksum fail. slot: %d\n",
> +			label_name, slot);
> +
>  	return valid;
>  }
>  
> @@ -424,7 +450,7 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
>  
>  		nd_label = to_label(ndd, slot);
>  
> -		if (!slot_valid(ndd, nd_label, slot))
> +		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
>  			continue;
>  
>  		nsl_get_uuid(ndd, nd_label, &label_uuid);
> @@ -575,18 +601,30 @@ int nd_label_active_count(struct nvdimm_drvdata *ndd)
>  		return 0;
>  
>  	for_each_clear_bit_le(slot, free, nslot) {
> +		struct cxl_region_label *region_label;
>  		struct nd_namespace_label *nd_label;
> -
> -		nd_label = to_label(ndd, slot);
> -
> -		if (!slot_valid(ndd, nd_label, slot)) {
> -			u32 label_slot = nsl_get_slot(ndd, nd_label);
> -			u64 size = nsl_get_rawsize(ndd, nd_label);
> -			u64 dpa = nsl_get_dpa(ndd, nd_label);
> +		union nd_lsa_label *lsa_label;
> +		u32 lslot;
> +		u64 size, dpa;
> +
> +		lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
> +		nd_label = &lsa_label->ns_label;
> +		region_label = &lsa_label->region_label;
> +
> +		if (!slot_valid(ndd, lsa_label, slot)) {
> +			if (is_region_label(ndd, lsa_label)) {
> +				lslot = __le32_to_cpu(region_label->slot);
> +				size = __le64_to_cpu(region_label->rawsize);
> +				dpa = __le64_to_cpu(region_label->dpa);
> +			} else {
> +				lslot = nsl_get_slot(ndd, nd_label);
> +				size = nsl_get_rawsize(ndd, nd_label);
> +				dpa = nsl_get_dpa(ndd, nd_label);
> +			}
>  
>  			dev_dbg(ndd->dev,
>  				"slot%d invalid slot: %d dpa: %llx size: %llx\n",
> -					slot, label_slot, dpa, size);
> +					slot, lslot, dpa, size);
>  			continue;
>  		}
>  		count++;
> @@ -607,7 +645,7 @@ struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n)
>  		struct nd_namespace_label *nd_label;
>  
>  		nd_label = to_label(ndd, slot);
> -		if (!slot_valid(ndd, nd_label, slot))
> +		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
>  			continue;
>  
>  		if (n-- == 0)
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index 046063ea08b6..c985f91728dd 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -344,6 +344,11 @@ region_label_uuid_equal(struct cxl_region_label *region_label,
>  	return uuid_equal((uuid_t *) region_label->uuid, uuid);
>  }
>  
> +static inline u32 region_label_get_slot(struct cxl_region_label *region_label)
> +{
> +	return __le32_to_cpu(region_label->slot);
> +}
> +
>  static inline u64
>  region_label_get_checksum(struct cxl_region_label *region_label)
>  {

Re: [PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Modified __pmem_label_update() to update region labels into LSA
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c          | 269 ++++++++++++++++++++++++++------
>  drivers/nvdimm/label.h          |  15 ++
>  drivers/nvdimm/namespace_devs.c |  12 ++
>  drivers/nvdimm/nd.h             |  38 ++++-
>  include/linux/libnvdimm.h       |   8 +
>  5 files changed, 289 insertions(+), 53 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 182f8c9a01bf..209c73f6b7e7 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -381,6 +381,16 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
>  	nsl_set_checksum(ndd, nd_label, sum);
>  }
>  
> +static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
> +				   struct cxl_region_label *region_label)
> +{
> +	u64 sum;
> +
> +	region_label_set_checksum(region_label, 0);
> +	sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1);
> +	region_label_set_checksum(region_label, sum);
> +}
> +
>  static bool slot_valid(struct nvdimm_drvdata *ndd,
>  		struct nd_namespace_label *nd_label, u32 slot)
>  {
> @@ -884,26 +894,20 @@ enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
>  	return guid_to_nvdimm_cclass(&nd_label->efi.abstraction_guid);
>  }
>  
> -static int __pmem_label_update(struct nd_region *nd_region,
> -		struct nd_mapping *nd_mapping, struct nd_namespace_pmem *nspm,
> -		int pos, unsigned long flags)
> +static int namespace_label_update(struct nd_region *nd_region,
> +				  struct nd_mapping *nd_mapping,
> +				  struct nd_namespace_pmem *nspm,
> +				  int pos, u64 flags,
> +				  struct nd_namespace_label *ns_label,
> +				  struct nd_namespace_index *nsindex,
> +				  u32 slot)
>  {
>  	struct nd_namespace_common *ndns = &nspm->nsio.common;
>  	struct nd_interleave_set *nd_set = nd_region->nd_set;
>  	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> -	struct nd_namespace_label *nd_label;
> -	struct nd_namespace_index *nsindex;
> -	struct nd_label_ent *label_ent;
>  	struct nd_label_id label_id;
>  	struct resource *res;
> -	unsigned long *free;
> -	u32 nslot, slot;
> -	size_t offset;
>  	u64 cookie;
> -	int rc;
> -
> -	if (!preamble_next(ndd, &nsindex, &free, &nslot))
> -		return -ENXIO;
>  
>  	cookie = nd_region_interleave_set_cookie(nd_region, nsindex);
>  	nd_label_gen_id(&label_id, nspm->uuid, 0);
> @@ -916,36 +920,131 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  		return -ENXIO;
>  	}
>  
> +	nsl_set_type(ndd, ns_label);
> +	nsl_set_uuid(ndd, ns_label, nspm->uuid);
> +	nsl_set_name(ndd, ns_label, nspm->alt_name);
> +	nsl_set_flags(ndd, ns_label, flags);
> +	nsl_set_nlabel(ndd, ns_label, nd_region->ndr_mappings);
> +	nsl_set_nrange(ndd, ns_label, 1);
> +	nsl_set_position(ndd, ns_label, pos);
> +	nsl_set_isetcookie(ndd, ns_label, cookie);
> +	nsl_set_rawsize(ndd, ns_label, resource_size(res));
> +	nsl_set_lbasize(ndd, ns_label, nspm->lbasize);
> +	nsl_set_dpa(ndd, ns_label, res->start);
> +	nsl_set_slot(ndd, ns_label, slot);
> +	nsl_set_alignment(ndd, ns_label, 0);
> +	nsl_set_type_guid(ndd, ns_label, &nd_set->type_guid);
> +	nsl_set_region_uuid(ndd, ns_label, &nd_set->uuid);
> +	nsl_set_claim_class(ndd, ns_label, ndns->claim_class);
> +	nsl_calculate_checksum(ndd, ns_label);
> +	nd_dbg_dpa(nd_region, ndd, res, "\n");
> +
> +	return 0;
> +}
> +
> +static void region_label_update(struct nd_region *nd_region,
> +				struct cxl_region_label *region_label,
> +				struct nd_mapping *nd_mapping,
> +				int pos, u64 flags, u32 slot)
> +{
> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
> +	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +
> +	/* Set Region Label Format identification UUID */
> +	uuid_parse(CXL_REGION_UUID, (uuid_t *) region_label->type);
> +
> +	/* Set Current Region Label UUID */
> +	export_uuid(region_label->uuid, &nd_set->uuid);
> +
> +	region_label->flags = __cpu_to_le32(flags);
> +	region_label->nlabel = __cpu_to_le16(nd_region->ndr_mappings);
> +	region_label->position = __cpu_to_le16(pos);
> +	region_label->dpa = __cpu_to_le64(nd_mapping->start);
> +	region_label->rawsize = __cpu_to_le64(nd_mapping->size);
> +	region_label->hpa = __cpu_to_le64(nd_set->res->start);
> +	region_label->slot = __cpu_to_le32(slot);
> +	region_label->ig = __cpu_to_le32(nd_set->interleave_granularity);
> +	region_label->align = __cpu_to_le32(0);
> +
> +	/* Update fletcher64 Checksum */
> +	region_label_calculate_checksum(ndd, region_label);
> +}
> +
> +static bool is_label_reapable(struct nd_interleave_set *nd_set,
> +			       struct nd_namespace_pmem *nspm,
> +			       struct nvdimm_drvdata *ndd,
> +			       union nd_lsa_label *label,
> +			       enum label_type ltype,
> +			       unsigned long *flags)
> +{
> +	switch (ltype) {
> +	case NS_LABEL_TYPE:
> +		if (test_and_clear_bit(ND_LABEL_REAP, flags) ||
> +		    nsl_uuid_equal(ndd, &label->ns_label, nspm->uuid))
> +			return true;
> +
> +		break;
> +	case RG_LABEL_TYPE:
> +		if (region_label_uuid_equal(&label->region_label,
> +		    &nd_set->uuid))
> +			return true;
> +
> +		break;
> +	}
> +
> +	return false;
> +}
> +
> +static int __pmem_label_update(struct nd_region *nd_region,
> +			       struct nd_mapping *nd_mapping,
> +			       struct nd_namespace_pmem *nspm,
> +			       int pos, unsigned long flags,
> +			       enum label_type ltype)
> +{
> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
> +	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +	struct nd_namespace_index *nsindex;
> +	struct nd_label_ent *label_ent;
> +	union nd_lsa_label *lsa_label;
> +	unsigned long *free;
> +	struct device *dev;
> +	u32 nslot, slot;
> +	size_t offset;
> +	int rc;
> +
> +	if (!preamble_next(ndd, &nsindex, &free, &nslot))
> +		return -ENXIO;
> +
>  	/* allocate and write the label to the staging (next) index */
>  	slot = nd_label_alloc_slot(ndd);
>  	if (slot == UINT_MAX)
>  		return -ENXIO;
>  	dev_dbg(ndd->dev, "allocated: %d\n", slot);
>  
> -	nd_label = to_label(ndd, slot);
> -	memset(nd_label, 0, sizeof_namespace_label(ndd));
> -	nsl_set_type(ndd, nd_label);
> -	nsl_set_uuid(ndd, nd_label, nspm->uuid);
> -	nsl_set_name(ndd, nd_label, nspm->alt_name);
> -	nsl_set_flags(ndd, nd_label, flags);
> -	nsl_set_nlabel(ndd, nd_label, nd_region->ndr_mappings);
> -	nsl_set_nrange(ndd, nd_label, 1);
> -	nsl_set_position(ndd, nd_label, pos);
> -	nsl_set_isetcookie(ndd, nd_label, cookie);
> -	nsl_set_rawsize(ndd, nd_label, resource_size(res));
> -	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
> -	nsl_set_dpa(ndd, nd_label, res->start);
> -	nsl_set_slot(ndd, nd_label, slot);
> -	nsl_set_alignment(ndd, nd_label, 0);
> -	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
> -	nsl_set_region_uuid(ndd, nd_label, NULL);
> -	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
> -	nsl_calculate_checksum(ndd, nd_label);
> -	nd_dbg_dpa(nd_region, ndd, res, "\n");
> +	lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
> +	memset(lsa_label, 0, sizeof_namespace_label(ndd));
> +
> +	switch (ltype) {
> +	case NS_LABEL_TYPE:
> +		dev = &nspm->nsio.common.dev;
> +		rc = namespace_label_update(nd_region, nd_mapping,
> +				nspm, pos, flags, &lsa_label->ns_label,
> +				nsindex, slot);
> +		if (rc)
> +			return rc;
> +
> +		break;
> +	case RG_LABEL_TYPE:
> +		dev = &nd_region->dev;
> +		region_label_update(nd_region, &lsa_label->region_label,
> +				    nd_mapping, pos, flags, slot);
> +
> +		break;
> +	}
>  
>  	/* update label */
> -	offset = nd_label_offset(ndd, nd_label);
> -	rc = nvdimm_set_config_data(ndd, offset, nd_label,
> +	offset = nd_label_offset(ndd, &lsa_label->ns_label);
> +	rc = nvdimm_set_config_data(ndd, offset, lsa_label,
>  			sizeof_namespace_label(ndd));
>  	if (rc < 0)
>  		return rc;
> @@ -955,8 +1054,10 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>  		if (!label_ent->label)
>  			continue;
> -		if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) ||
> -		    nsl_uuid_equal(ndd, label_ent->label, nspm->uuid))
> +
> +		if (is_label_reapable(nd_set, nspm, ndd,
> +				      (union nd_lsa_label *) label_ent->label,
> +				      ltype, &label_ent->flags))
>  			reap_victim(nd_mapping, label_ent);
>  	}
>  
> @@ -966,19 +1067,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	if (rc)
>  		return rc;
>  
> -	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> -		if (!label_ent->label) {
> -			label_ent->label = nd_label;
> -			nd_label = NULL;
> -			break;
> -		}
> -	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> -			"failed to track label: %d\n",
> -			to_slot(ndd, nd_label));
> -	if (nd_label)
> -		rc = -ENXIO;
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
> +		if (label_ent->label)
> +			continue;
>  
> -	return rc;
> +		label_ent->label = &lsa_label->ns_label;
> +		lsa_label = NULL;
> +		break;
> +	}
> +	dev_WARN_ONCE(dev, lsa_label, "failed to track label: %d\n",
> +		      to_slot(ndd, &lsa_label->ns_label));
> +	if (lsa_label)
> +		return -ENXIO;
> +
> +	return 0;
>  }
>  
>  static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
> @@ -1068,6 +1170,21 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
>  }
>  
> +static int find_region_label_count(struct nvdimm_drvdata *ndd,
> +				   struct nd_mapping *nd_mapping)
> +{
> +	struct nd_label_ent *label_ent;
> +	int region_label_cnt = 0;
> +
> +	guard(mutex)(&nd_mapping->lock);
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> +		if (is_region_label(ndd,
> +		    (union nd_lsa_label *) label_ent->label))
> +			region_label_cnt++;
> +
> +	return region_label_cnt;
> +}
> +
>  int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  		struct nd_namespace_pmem *nspm, resource_size_t size)
>  {
> @@ -1076,6 +1193,7 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>  		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +		int region_label_cnt = 0;
>  		struct resource *res;
>  		int count = 0;
>  
> @@ -1091,12 +1209,19 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  				count++;
>  		WARN_ON_ONCE(!count);
>  
> -		rc = init_labels(nd_mapping, count);
> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
> +		/*
> +		 * init_labels() scan labels and allocate new label based
> +		 * on its second parameter (num_labels). Therefore to
> +		 * allocate new namespace label also include previously
> +		 * added region label
> +		 */
> +		rc = init_labels(nd_mapping, count + region_label_cnt);
>  		if (rc < 0)
>  			return rc;
>  
>  		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i,
> -				NSLABEL_FLAG_UPDATING);
> +				NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE);
>  		if (rc)
>  			return rc;
>  	}
> @@ -1108,7 +1233,47 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>  
> -		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0);
> +		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0,
> +				NS_LABEL_TYPE);
> +		if (rc)
> +			return rc;
> +	}
> +
> +	return 0;
> +}
> +
> +int nd_pmem_region_label_update(struct nd_region *nd_region)
> +{
> +	int i, rc;
> +
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +		int region_label_cnt = 0;
> +
> +		/* No need to update region label for non cxl format */
> +		if (!ndd->cxl)
> +			return 0;
> +
> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
> +		rc = init_labels(nd_mapping, region_label_cnt + 1);
> +		if (rc < 0)
> +			return rc;
> +
> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i,
> +				NSLABEL_FLAG_UPDATING, RG_LABEL_TYPE);
> +		if (rc)
> +			return rc;
> +	}
> +
> +	/* Clear the UPDATING flag per UEFI 2.7 expectations */
> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> +
> +		WARN_ON_ONCE(!ndd->cxl);
> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, 0,
> +				RG_LABEL_TYPE);
>  		if (rc)
>  			return rc;
>  	}
> diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
> index 0650fb4b9821..284e2a763b49 100644
> --- a/drivers/nvdimm/label.h
> +++ b/drivers/nvdimm/label.h
> @@ -30,6 +30,11 @@ enum {
>  	ND_NSINDEX_INIT = 0x1,
>  };
>  
> +enum label_type {
> +	RG_LABEL_TYPE,
> +	NS_LABEL_TYPE,
> +};
> +
>  /**
>   * struct nd_namespace_index - label set superblock
>   * @sig: NAMESPACE_INDEX\0
> @@ -183,6 +188,15 @@ struct nd_namespace_label {
>  	};
>  };
>  
> +/*
> + * LSA 2.1 format introduces region label, which can also reside
> + * into LSA along with only namespace label as per v1.1 and v1.2
> + */
> +union nd_lsa_label {
> +	struct nd_namespace_label ns_label;
> +	struct cxl_region_label region_label;
> +};
> +
>  #define NVDIMM_BTT_GUID "8aed63a2-29a2-4c66-8b12-f05d15d3922a"
>  #define NVDIMM_BTT2_GUID "18633bfc-1735-4217-8ac9-17239282d3f8"
>  #define NVDIMM_PFN_GUID "266400ba-fb9f-4677-bcb0-968f11d0d225"
> @@ -223,4 +237,5 @@ struct nd_region;
>  struct nd_namespace_pmem;
>  int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>  		struct nd_namespace_pmem *nspm, resource_size_t size);
> +int nd_pmem_region_label_update(struct nd_region *nd_region);
>  #endif /* __LABEL_H__ */
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 3271b1c8569a..559f822ef24f 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -232,6 +232,18 @@ static ssize_t __alt_name_store(struct device *dev, const char *buf,
>  	return rc;
>  }
>  
> +int nd_region_label_update(struct nd_region *nd_region)
> +{
> +	int rc;
> +
> +	nvdimm_bus_lock(&nd_region->dev);
> +	rc = nd_pmem_region_label_update(nd_region);
> +	nvdimm_bus_unlock(&nd_region->dev);
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL_GPL(nd_region_label_update);
> +
>  static int nd_namespace_label_update(struct nd_region *nd_region,
>  		struct device *dev)
>  {
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index e362611d82cc..f04c042dcfa9 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -318,6 +318,39 @@ static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
>  		export_uuid(ns_label->cxl.region_uuid, uuid);
>  }
>  
> +static inline bool is_region_label(struct nvdimm_drvdata *ndd,
> +				   union nd_lsa_label *nd_label)
> +{
> +	uuid_t region_type, *ns_type;
> +
> +	if (!ndd->cxl || !nd_label)
> +		return false;
> +
> +	uuid_parse(CXL_REGION_UUID, &region_type);
> +	ns_type = (uuid_t *) nd_label->ns_label.cxl.type;

So in addition to Jonathan's comments, I think we should consider utilizing the common field (UUID) of all the labels. i.e. if you are to use a union, you can also include 'uuid_t label_type' as a member. Perhaps this function can just pass in a UUID rather than a label struct. And we can probably skip passing in 'ndd' and not bother with ndd->cxl check. This function can just rely on checking the UUID and see if it's the expected region label UUID. Just thinking of the places we can maybe avoid doing casting if possible. 

DJ 

> +	return uuid_equal(&region_type, ns_type);
> +}
> +
> +static inline bool
> +region_label_uuid_equal(struct cxl_region_label *region_label,
> +			const uuid_t *uuid)
> +{
> +	return uuid_equal((uuid_t *) region_label->uuid, uuid);
> +}
> +
> +static inline u64
> +region_label_get_checksum(struct cxl_region_label *region_label)
> +{
> +	return __le64_to_cpu(region_label->checksum);
> +}
> +
> +static inline void
> +region_label_set_checksum(struct cxl_region_label *region_label,
> +			  u64 checksum)
> +{
> +	region_label->checksum = __cpu_to_le64(checksum);
> +}
> +
>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
> @@ -399,7 +432,10 @@ enum nd_label_flags {
>  struct nd_label_ent {
>  	struct list_head list;
>  	unsigned long flags;
> -	struct nd_namespace_label *label;
> +	union {
> +		struct nd_namespace_label *label;
> +		struct cxl_region_label *region_label;
> +	};
>  };
>  
>  enum nd_mapping_lock_class {
> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index 5696715c33bb..2c213b9dac66 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -117,6 +117,13 @@ struct nd_interleave_set {
>  	u64 altcookie;
>  
>  	guid_t type_guid;
> +
> +	/* v2.1 region label info */
> +	uuid_t uuid;
> +	int interleave_ways;
> +	int interleave_granularity;
> +	struct resource *res;
> +	int nr_targets;
>  };
>  
>  struct nd_mapping_desc {
> @@ -307,6 +314,7 @@ int nvdimm_has_flush(struct nd_region *nd_region);
>  int nvdimm_has_cache(struct nd_region *nd_region);
>  int nvdimm_in_overwrite(struct nvdimm *nvdimm);
>  bool is_nvdimm_sync(struct nd_region *nd_region);
> +int nd_region_label_update(struct nd_region *nd_region);
>  
>  static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
>  		unsigned int buf_len, int *cmd_rc)

Re: [PATCH V3 12/20] nvdimm/region_label: Export routine to fetch region information

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> CXL region information preserved from the LSA needs to be exported for
> use by the CXL driver for CXL region re-creation.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/dimm_devs.c | 18 ++++++++++++++++++
>  include/linux/libnvdimm.h  |  2 ++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
> index 918c3db93195..619c8ce56dce 100644
> --- a/drivers/nvdimm/dimm_devs.c
> +++ b/drivers/nvdimm/dimm_devs.c
> @@ -280,6 +280,24 @@ void *nvdimm_provider_data(struct nvdimm *nvdimm)
>  }
>  EXPORT_SYMBOL_GPL(nvdimm_provider_data);
>  
> +bool nvdimm_has_cxl_region(struct nvdimm *nvdimm)
> +{
> +	if (nvdimm)
> +		return nvdimm->is_region_label;
> +
> +	return false;

Just a nit. Would prefer return error early and return the success case last.

> +}
> +EXPORT_SYMBOL_GPL(nvdimm_has_cxl_region);
> +
> +void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm)
> +{
> +	if (nvdimm)
> +		return &nvdimm->cxl_region_params;
> +
> +	return NULL;

same comment

> +}
> +EXPORT_SYMBOL_GPL(nvdimm_get_cxl_region_param);
> +
>  static ssize_t commands_show(struct device *dev,
>  		struct device_attribute *attr, char *buf)
>  {
> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index 07ea2e3f821a..3ffd50ab6ac4 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -330,6 +330,8 @@ int nvdimm_in_overwrite(struct nvdimm *nvdimm);
>  bool is_nvdimm_sync(struct nd_region *nd_region);
>  int nd_region_label_update(struct nd_region *nd_region);
>  int nd_region_label_delete(struct nd_region *nd_region);
> +bool nvdimm_has_cxl_region(struct nvdimm *nvdimm);
> +void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm);
>  
>  static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
>  		unsigned int buf_len, int *cmd_rc)

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5
> Modified __pmem_label_update function using setter functions to update
> namespace label as per CXL LSA 2.1
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c |  3 +++
>  drivers/nvdimm/nd.h    | 23 +++++++++++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 3235562d0e1c..182f8c9a01bf 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -924,6 +924,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  
>  	nd_label = to_label(ndd, slot);
>  	memset(nd_label, 0, sizeof_namespace_label(ndd));
> +	nsl_set_type(ndd, nd_label);
>  	nsl_set_uuid(ndd, nd_label, nspm->uuid);
>  	nsl_set_name(ndd, nd_label, nspm->alt_name);
>  	nsl_set_flags(ndd, nd_label, flags);
> @@ -935,7 +936,9 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
>  	nsl_set_dpa(ndd, nd_label, res->start);
>  	nsl_set_slot(ndd, nd_label, slot);
> +	nsl_set_alignment(ndd, nd_label, 0);
>  	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
> +	nsl_set_region_uuid(ndd, nd_label, NULL);
>  	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
>  	nsl_calculate_checksum(ndd, nd_label);
>  	nd_dbg_dpa(nd_region, ndd, res, "\n");
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index 158809c2be9e..e362611d82cc 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -295,6 +295,29 @@ static inline const u8 *nsl_uuid_raw(struct nvdimm_drvdata *ndd,
>  	return nd_label->efi.uuid;
>  }
>  
> +static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
> +				struct nd_namespace_label *ns_label)
> +{
> +	if (ndd->cxl && ns_label)
> +		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);

You can do:
uuid_copy(ns_label->cxl.type, cxl_namespace_uuid);

All the constant UUIDs are preparsed by nd_label_init(). For the entire series you can use those pre-parsed uuid_t and just use uuid_copy() instead of having to do string to uuid_t conversions.

DJ

> +}
> +
> +static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd,
> +				     struct nd_namespace_label *ns_label,
> +				     u32 align)
> +{
> +	if (ndd->cxl)
> +		ns_label->cxl.align = __cpu_to_le32(align);
> +}
> +
> +static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd,
> +				       struct nd_namespace_label *ns_label,
> +				       const uuid_t *uuid)
> +{
> +	if (ndd->cxl && uuid)
> +		export_uuid(ns_label->cxl.region_uuid, uuid);
> +}
> +
>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
> used to get called at last in cxl_endpoint_port_probe(), which requires
> cxl_nvd presence.
> 
> For cxl region persistency, region creation happens during nvdimm_probe
> which need the completion of endpoint probe.
> 
> In order to accommodate both cxl pmem region auto-assembly and cxl region
> persistency, refactored following
> 
> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
>    will be called only after successful completion of endpoint probe.
> 
> 2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
>    cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
>    completion of endpoint probe and cxl_nvd presence before its call.

Given that we are going forward with this implementation [1] from Dan and drivers like the type2 enabling are going to be using it as well, can you please consider converting this change to Dan's mechanism instead of creating a whole new one?

I think the region discovery can be done via the ops->probe() callback. Thanks.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6

DJ

> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/core/region.c | 33 +++++++++++++++++++++++++++++++++
>  drivers/cxl/cxl.h         |  4 ++++
>  drivers/cxl/mem.c         | 24 +++++++++++++++---------
>  drivers/cxl/port.c        | 39 +--------------------------------------
>  4 files changed, 53 insertions(+), 47 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 7a0cead24490..c325aa827992 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -3606,6 +3606,39 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled)
>  }
>  EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL");
>  
> +static int discover_region(struct device *dev, void *unused)
> +{
> +	struct cxl_endpoint_decoder *cxled;
> +	int rc;
> +
> +	if (!is_endpoint_decoder(dev))
> +		return 0;
> +
> +	cxled = to_cxl_endpoint_decoder(dev);
> +	if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0)
> +		return 0;
> +
> +	if (cxled->state != CXL_DECODER_STATE_AUTO)
> +		return 0;
> +
> +	/*
> +	 * Region enumeration is opportunistic, if this add-event fails,
> +	 * continue to the next endpoint decoder.
> +	 */
> +	rc = cxl_add_to_region(cxled);
> +	if (rc)
> +		dev_dbg(dev, "failed to add to region: %#llx-%#llx\n",
> +			cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end);
> +
> +	return 0;
> +}
> +
> +void cxl_region_discovery(struct cxl_port *port)
> +{
> +	device_for_each_child(&port->dev, NULL, discover_region);
> +}
> +EXPORT_SYMBOL_NS_GPL(cxl_region_discovery, "CXL");
> +
>  u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa)
>  {
>  	struct cxl_region_ref *iter;
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index 4fe3df06f57a..b57597e55f7e 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -873,6 +873,7 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
>  int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
>  struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
>  u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
> +void cxl_region_discovery(struct cxl_port *port);
>  #else
>  static inline bool is_cxl_pmem_region(struct device *dev)
>  {
> @@ -895,6 +896,9 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint,
>  {
>  	return 0;
>  }
> +static inline void cxl_region_discovery(struct cxl_port *port)
> +{
> +}
>  #endif
>  
>  void cxl_endpoint_parse_cdat(struct cxl_port *port);
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 6e6777b7bafb..54501616ff09 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -152,15 +152,6 @@ static int cxl_mem_probe(struct device *dev)
>  		return -ENXIO;
>  	}
>  
> -	if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) {
> -		rc = devm_cxl_add_nvdimm(parent_port, cxlmd);
> -		if (rc) {
> -			if (rc == -ENODEV)
> -				dev_info(dev, "PMEM disabled by platform\n");
> -			return rc;
> -		}
> -	}
> -
>  	if (dport->rch)
>  		endpoint_parent = parent_port->uport_dev;
>  	else
> @@ -184,6 +175,21 @@ static int cxl_mem_probe(struct device *dev)
>  	if (rc)
>  		dev_dbg(dev, "CXL memdev EDAC registration failed rc=%d\n", rc);
>  
> +	if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) {
> +		rc = devm_cxl_add_nvdimm(parent_port, cxlmd);
> +		if (rc) {
> +			if (rc == -ENODEV)
> +				dev_info(dev, "PMEM disabled by platform\n");
> +			return rc;
> +		}
> +	}
> +
> +	/*
> +	 * Now that all endpoint decoders are successfully enumerated, try to
> +	 * assemble region autodiscovery from committed decoders.
> +	 */
> +	cxl_region_discovery(cxlmd->endpoint);
> +
>  	/*
>  	 * The kernel may be operating out of CXL memory on this device,
>  	 * there is no spec defined way to determine whether this device
> diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c
> index cf32dc50b7a6..07bb909b7d2e 100644
> --- a/drivers/cxl/port.c
> +++ b/drivers/cxl/port.c
> @@ -30,33 +30,6 @@ static void schedule_detach(void *cxlmd)
>  	schedule_cxl_memdev_detach(cxlmd);
>  }
>  
> -static int discover_region(struct device *dev, void *unused)
> -{
> -	struct cxl_endpoint_decoder *cxled;
> -	int rc;
> -
> -	if (!is_endpoint_decoder(dev))
> -		return 0;
> -
> -	cxled = to_cxl_endpoint_decoder(dev);
> -	if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0)
> -		return 0;
> -
> -	if (cxled->state != CXL_DECODER_STATE_AUTO)
> -		return 0;
> -
> -	/*
> -	 * Region enumeration is opportunistic, if this add-event fails,
> -	 * continue to the next endpoint decoder.
> -	 */
> -	rc = cxl_add_to_region(cxled);
> -	if (rc)
> -		dev_dbg(dev, "failed to add to region: %#llx-%#llx\n",
> -			cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end);
> -
> -	return 0;
> -}
> -
>  static int cxl_switch_port_probe(struct cxl_port *port)
>  {
>  	struct cxl_hdm *cxlhdm;
> @@ -121,17 +94,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)
>  	if (rc)
>  		return rc;
>  
> -	rc = devm_cxl_enumerate_decoders(cxlhdm, &info);
> -	if (rc)
> -		return rc;
> -
> -	/*
> -	 * Now that all endpoint decoders are successfully enumerated, try to
> -	 * assemble regions from committed decoders
> -	 */
> -	device_for_each_child(&port->dev, NULL, discover_region);
> -
> -	return 0;
> +	return devm_cxl_enumerate_decoders(cxlhdm, &info);
>  }
>  
>  static int cxl_port_probe(struct device *dev)

Re: [PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Alison Schofield]

On Wed, Sep 17, 2025 at 07:10:56PM +0530, Neeraj Kumar wrote:
> Introduction:
> =============
> CXL Persistent Memory (Pmem) devices region, namespace and content must be
> persistent across system reboot. In order to achieve this persistency, it
> uses Label Storage Area (LSA) to store respective metadata. During system
> reboot, stored metadata in LSA is used to bring back the region, namespace
> and content of CXL device in its previous state.
> CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox
> commands to access the LSA area. nvdimm driver is using same commands to
> get/set LSA data.

big snip...

> Limitation (TODO):
> ==================
> Current changes only support interleave way == 1

I see your test setup with the one way interleave and read this
limitation. What are your plans regarding this?

ie. Do you see this limitation as something you could merge with
or do you plan to extend this patchset with support for namespaces
build upon region ways > 1.

-- Alison

Re: [PATCH V3 14/20] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> devm_cxl_pmem_add_region() is used to create cxl region based on region
> information scanned from LSA.
> 
> devm_cxl_add_region() is used to just allocate cxlr and its fields are
> filled later by userspace tool using device attributes (*_store()).
> 
> Inspiration for devm_cxl_pmem_add_region() is taken from these device
> attributes (_store*) calls. It allocates cxlr and fills information
> parsed from LSA and calls device_add(&cxlr->dev) to initiate further
> region creation porbes
> 
> Renamed __create_region() to cxl_create_region() and make it an exported
> routine. This will be used in later patch to create cxl region after
> fetching region information from LSA.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/core/region.c | 127 ++++++++++++++++++++++++++++++++++++--
>  drivers/cxl/cxl.h         |  12 ++++
>  2 files changed, 134 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index c325aa827992..d5c227ce7b09 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -2573,6 +2573,116 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd,
>  	return ERR_PTR(rc);
>  }
>  
> +static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size)
> +{
> +	int rc;
> +
> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
> +	if (rc)
> +		return rc;

Just a nit. Please conform to existing style in the subsystem for this new usage.

  +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
  +	if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))))
  +		return rc;

> +
> +	if (!size)
> +		return -EINVAL;
> +
> +	return alloc_hpa(cxlr, size);
> +}

I think you can create another helper free_region_hpa() and call them in size_store() function to remove the duplicate code.
> +
> +static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size)
> +{
> +	int rc;
> +
> +	if (!size)
> +		return -EINVAL;
> +
> +	if (!IS_ALIGNED(size, SZ_256M))
> +		return -EINVAL;
> +
> +	rc = cxl_dpa_free(cxled);
> +	if (rc)
> +		return rc;
> +
> +	return cxl_dpa_alloc(cxled, size);
> +}
> +
> +static struct cxl_region *
> +devm_cxl_pmem_add_region(struct cxl_root_decoder *cxlrd, int id,
> +			 enum cxl_partition_mode mode,

Wouldn't this not needed since it would be CXL_PARTMODE_PMEM always? I also wonder if we need to rename devm_cxl_add_region() to devm_cxl_add_ram_region() to be explicit.

> +			 enum cxl_decoder_type type,
> +			 struct cxl_pmem_region_params *params,
> +			 struct cxl_decoder *cxld)
> +{
> +	struct cxl_endpoint_decoder *cxled;
> +	struct cxl_region_params *p;
> +	struct cxl_port *root_port;
> +	struct device *dev;
> +	int rc;
> +
> +	struct cxl_region *cxlr __free(put_cxl_region) =
> +		cxl_region_alloc(cxlrd, id);
> +	if (IS_ERR(cxlr))
> +		return cxlr;
> +
> +	cxlr->mode = mode;
> +	cxlr->type = type;
> +
> +	dev = &cxlr->dev;
> +	rc = dev_set_name(dev, "region%d", id);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	p = &cxlr->params;
> +	p->uuid = params->uuid;
> +	p->interleave_ways = params->nlabel;
> +	p->interleave_granularity = params->ig;
> +
> +	rc = alloc_region_hpa(cxlr, params->rawsize);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	cxled = to_cxl_endpoint_decoder(&cxld->dev);
> +
> +	rc = cxl_dpa_set_part(cxled, CXL_PARTMODE_PMEM);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	rc = alloc_region_dpa(cxled, params->rawsize);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	/*
> +	 * TODO: Currently we have support of interleave_way == 1, where
> +	 * we can only have one region per mem device. It means mem device
> +	 * position (params->position) will always be 0. It is therefore
> +	 * attaching only one target at params->position
> +	 */
> +	if (params->position)
> +		return ERR_PTR(-EINVAL);

EOPNOTSUPP?

Speaking of which, are there plans to support interleave in the near future?

DJ

> +
> +	rc = attach_target(cxlr, cxled, params->position, TASK_INTERRUPTIBLE);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	rc = __commit(cxlr);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	rc = device_add(dev);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	root_port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent);
> +	rc = devm_add_action_or_reset(root_port->uport_dev,
> +			unregister_region, cxlr);
> +	if (rc)
> +		return ERR_PTR(rc);
> +
> +	dev_dbg(root_port->uport_dev, "%s: created %s\n",
> +		dev_name(&cxlrd->cxlsd.cxld.dev), dev_name(dev));
> +
> +	return no_free_ptr(cxlr);
> +}
> +
>  static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf)
>  {
>  	return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id));
> @@ -2590,8 +2700,10 @@ static ssize_t create_ram_region_show(struct device *dev,
>  	return __create_region_show(to_cxl_root_decoder(dev), buf);
>  }
>  
> -static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
> -					  enum cxl_partition_mode mode, int id)
> +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
> +				     enum cxl_partition_mode mode, int id,
> +				     struct cxl_pmem_region_params *pmem_params,
> +				     struct cxl_decoder *cxld)
>  {
>  	int rc;
>  
> @@ -2613,8 +2725,12 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
>  		return ERR_PTR(-EBUSY);
>  	}
>  
> +	if (pmem_params)
> +		return devm_cxl_pmem_add_region(cxlrd, id, mode,
> +				CXL_DECODER_HOSTONLYMEM, pmem_params, cxld);
>  	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
>  }
> +EXPORT_SYMBOL_NS_GPL(cxl_create_region, "CXL");
>  
>  static ssize_t create_region_store(struct device *dev, const char *buf,
>  				   size_t len, enum cxl_partition_mode mode)
> @@ -2627,7 +2743,7 @@ static ssize_t create_region_store(struct device *dev, const char *buf,
>  	if (rc != 1)
>  		return -EINVAL;
>  
> -	cxlr = __create_region(cxlrd, mode, id);
> +	cxlr = cxl_create_region(cxlrd, mode, id, NULL, NULL);
>  	if (IS_ERR(cxlr))
>  		return PTR_ERR(cxlr);
>  
> @@ -3523,8 +3639,9 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd,
>  	struct cxl_region *cxlr;
>  
>  	do {
> -		cxlr = __create_region(cxlrd, cxlds->part[part].mode,
> -				       atomic_read(&cxlrd->region_id));
> +		cxlr = cxl_create_region(cxlrd, cxlds->part[part].mode,
> +					 atomic_read(&cxlrd->region_id),
> +					 NULL, NULL);
>  	} while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY);
>  
>  	if (IS_ERR(cxlr)) {
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index b57597e55f7e..3abadc3dc82e 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -874,6 +874,10 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
>  struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
>  u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
>  void cxl_region_discovery(struct cxl_port *port);
> +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
> +				     enum cxl_partition_mode mode, int id,
> +				     struct cxl_pmem_region_params *params,
> +				     struct cxl_decoder *cxld);
>  #else
>  static inline bool is_cxl_pmem_region(struct device *dev)
>  {
> @@ -899,6 +903,14 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint,
>  static inline void cxl_region_discovery(struct cxl_port *port)
>  {
>  }
> +static inline struct cxl_region *
> +cxl_create_region(struct cxl_root_decoder *cxlrd,
> +		  enum cxl_partition_mode mode, int id,
> +		  struct cxl_pmem_region_params *params,
> +		  struct cxl_decoder *cxld)
> +{
> +	return ERR_PTR(-EOPNOTSUPP);
> +}
>  #endif
>  
>  void cxl_endpoint_parse_cdat(struct cxl_port *port);

Re: [PATCH V3 15/20] cxl: Add a routine to find cxl root decoder on cxl bus using cxl port

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Add cxl_find_root_decoder_by_port() to find root decoder on cxl bus.
> It is used to find root decoder using cxl port.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/core/port.c | 27 +++++++++++++++++++++++++++
>  drivers/cxl/cxl.h       |  1 +
>  2 files changed, 28 insertions(+)
> 
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index 8f36ff413f5d..647d9ce32b64 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -518,6 +518,33 @@ struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev)
>  }
>  EXPORT_SYMBOL_NS_GPL(to_cxl_switch_decoder, "CXL");
>  
> +static int match_root_decoder(struct device *dev, const void *data)
> +{
> +	return is_root_decoder(dev);
> +}
> +
> +/**
> + * cxl_find_root_decoder_by_port() - find a cxl root decoder on cxl bus
> + * @port: any descendant port in CXL port topology
> + */
> +struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port)
> +{
> +	struct cxl_root *cxl_root __free(put_cxl_root) = find_cxl_root(port);
> +	struct device *dev;
> +
> +	if (!cxl_root)
> +		return NULL;
> +
> +	dev = device_find_child(&cxl_root->port.dev, NULL, match_root_decoder);
> +	if (!dev)
> +		return NULL;
> +
> +	/* Release device ref taken via device_find_child() */
> +	put_device(dev);

Should the caller release the device reference instead?

DJ

> +	return to_cxl_root_decoder(dev);
> +}
> +EXPORT_SYMBOL_NS_GPL(cxl_find_root_decoder_by_port, "CXL");
> +
>  static void cxl_ep_release(struct cxl_ep *ep)
>  {
>  	put_device(ep->ep);
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index 3abadc3dc82e..1eb1aca7c69f 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -866,6 +866,7 @@ struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev);
>  bool is_cxl_nvdimm(struct device *dev);
>  int devm_cxl_add_nvdimm(struct cxl_port *parent_port, struct cxl_memdev *cxlmd);
>  struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port);
> +struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port);
>  
>  #ifdef CONFIG_CXL_REGION
>  bool is_cxl_pmem_region(struct device *dev);

Re: [PATCH V3 16/20] cxl/mem: Preserve cxl root decoder during mem probe

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Saved root decoder info is required for cxl region persistency

Should squash this patch into the previous patch. It's small enough that the usage and the implementation can be in the same patch.

> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/cxlmem.h | 1 +
>  drivers/cxl/mem.c    | 2 ++
>  2 files changed, 3 insertions(+)
> 
> diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> index 434031a0c1f7..25cb115b72bd 100644
> --- a/drivers/cxl/cxlmem.h
> +++ b/drivers/cxl/cxlmem.h
> @@ -59,6 +59,7 @@ struct cxl_memdev {
>  	struct cxl_nvdimm_bridge *cxl_nvb;
>  	struct cxl_nvdimm *cxl_nvd;
>  	struct cxl_port *endpoint;
> +	struct cxl_root_decoder *cxlrd;
>  	int id;
>  	int depth;
>  	u8 scrub_cycle;
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 54501616ff09..1a0da7253a24 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -152,6 +152,8 @@ static int cxl_mem_probe(struct device *dev)
>  		return -ENXIO;
>  	}
>  
> +	cxlmd->cxlrd = cxl_find_root_decoder_by_port(parent_port);
> +
>  	if (dport->rch)
>  		endpoint_parent = parent_port->uport_dev;
>  	else

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
> Moved pmem_region related code from core/region.c to core/pmem_region.c
> For region label update, need to create device attribute, which calls
> nvdimm exported function thus making pmem_region dependent on libnvdimm.
> Because of this dependency of pmem region on libnvdimm, segregated pmem
> region related code from core/region.c

We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.


> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/Kconfig            |  14 +++
>  drivers/cxl/core/Makefile      |   1 +
>  drivers/cxl/core/core.h        |   8 +-
>  drivers/cxl/core/pmem_region.c | 203 +++++++++++++++++++++++++++++++++
>  drivers/cxl/core/port.c        |   2 +-
>  drivers/cxl/core/region.c      | 191 +------------------------------
>  drivers/cxl/cxl.h              |  30 +++--
>  tools/testing/cxl/Kbuild       |   1 +
>  8 files changed, 250 insertions(+), 200 deletions(-)
>  create mode 100644 drivers/cxl/core/pmem_region.c
> 
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> index 48b7314afdb8..532eaa1bbdd6 100644
> --- a/drivers/cxl/Kconfig
> +++ b/drivers/cxl/Kconfig
> @@ -211,6 +211,20 @@ config CXL_REGION
>  
>  	  If unsure say 'y'
>  
> +config CXL_PMEM_REGION
> +	bool "CXL: Pmem Region Support"
> +	default CXL_BUS
> +	depends on CXL_REGION

> +	depends on PHYS_ADDR_T_64BIT
> +	depends on BLK_DEV
These 2 deps are odd. What are the actual dependencies?


> +	select LIBNVDIMM
> +	help
> +	  Enable the CXL core to enumerate and provision CXL pmem regions.
> +	  A CXL pmem region need to update region label into LSA. For LSA
> +	  updation/deletion libnvdimm is required.

s/updation/update/

> +
> +	  If unsure say 'y'
> +
>  config CXL_REGION_INVALIDATION_TEST
>  	bool "CXL: Region Cache Management Bypass (TEST)"
>  	depends on CXL_REGION
> diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile
> index 5ad8fef210b5..399157beb917 100644
> --- a/drivers/cxl/core/Makefile
> +++ b/drivers/cxl/core/Makefile
> @@ -17,6 +17,7 @@ cxl_core-y += cdat.o
>  cxl_core-y += ras.o
>  cxl_core-$(CONFIG_TRACING) += trace.o
>  cxl_core-$(CONFIG_CXL_REGION) += region.o
> +cxl_core-$(CONFIG_CXL_PMEM_REGION) += pmem_region.o
>  cxl_core-$(CONFIG_CXL_MCE) += mce.o
>  cxl_core-$(CONFIG_CXL_FEATURES) += features.o
>  cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o
> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
> index 5707cd60a8eb..536636a752dc 100644
> --- a/drivers/cxl/core/core.h
> +++ b/drivers/cxl/core/core.h
> @@ -34,7 +34,6 @@ int cxl_decoder_detach(struct cxl_region *cxlr,
>  #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr)
>  #define CXL_REGION_TYPE(x) (&cxl_region_type)
>  #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr),
> -#define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type)
>  #define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type)
>  int cxl_region_init(void);
>  void cxl_region_exit(void);
> @@ -74,10 +73,15 @@ static inline void cxl_region_exit(void)
>  #define CXL_REGION_ATTR(x) NULL
>  #define CXL_REGION_TYPE(x) NULL
>  #define SET_CXL_REGION_ATTR(x)
> -#define CXL_PMEM_REGION_TYPE(x) NULL
>  #define CXL_DAX_REGION_TYPE(x) NULL
>  #endif
>  
> +#ifdef CONFIG_CXL_PMEM_REGION
> +#define CXL_PMEM_REGION_TYPE (&cxl_pmem_region_type)
> +#else
> +#define CXL_PMEM_REGION_TYPE NULL
> +#endif
> +
>  struct cxl_send_command;
>  struct cxl_mem_query_commands;
>  int cxl_query_cmd(struct cxl_mailbox *cxl_mbox,
> diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
> new file mode 100644
> index 000000000000..55b80d587403
> --- /dev/null
> +++ b/drivers/cxl/core/pmem_region.c
> @@ -0,0 +1,203 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright(c) 2020 Intel Corporation. */
> +#include <linux/device.h>
> +#include <linux/memregion.h>
> +#include <cxlmem.h>
> +#include <cxl.h>
> +#include "core.h"
> +
> +/**
> + * DOC: cxl pmem region
> + *
> + * The core CXL PMEM region infrastructure supports persistent memory
> + * region creation using LIBNVDIMM subsystem. It has dependency on
> + * LIBNVDIMM, pmem region need updation of cxl region information into
> + * LSA. LIBNVDIMM dependency is only for pmem region, it is therefore
> + * need this separate file.
> + */
> +
> +bool is_cxl_pmem_region(struct device *dev)
> +{
> +	return dev->type == &cxl_pmem_region_type;
> +}
> +EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
> +
> +struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
> +{
> +	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
> +			  "not a cxl_pmem_region device\n"))
> +		return NULL;
> +	return container_of(dev, struct cxl_pmem_region, dev);
> +}
> +EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
> +
> +static void cxl_pmem_region_release(struct device *dev)
> +{
> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
> +	int i;
> +
> +	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
> +		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
> +
> +		put_device(&cxlmd->dev);
> +	}
> +
> +	kfree(cxlr_pmem);
> +}
> +
> +static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
> +	&cxl_base_attribute_group,
> +	NULL,
> +};
> +
> +const struct device_type cxl_pmem_region_type = {
> +	.name = "cxl_pmem_region",
> +	.release = cxl_pmem_region_release,
> +	.groups = cxl_pmem_region_attribute_groups,
> +};
> +
> +static struct lock_class_key cxl_pmem_region_key;
> +
> +static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
> +{
> +	struct cxl_region_params *p = &cxlr->params;
> +	struct cxl_nvdimm_bridge *cxl_nvb;
> +	struct device *dev;
> +	int i;
> +
> +	guard(rwsem_read)(&cxl_rwsem.region);
> +	if (p->state != CXL_CONFIG_COMMIT)
> +		return -ENXIO;
> +
> +	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
> +		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets),
> +			GFP_KERNEL);
> +	if (!cxlr_pmem)
> +		return -ENOMEM;
> +
> +	cxlr_pmem->hpa_range.start = p->res->start;
> +	cxlr_pmem->hpa_range.end = p->res->end;
> +
> +	/* Snapshot the region configuration underneath the cxl_region_rwsem */
> +	cxlr_pmem->nr_mappings = p->nr_targets;
> +	for (i = 0; i < p->nr_targets; i++) {
> +		struct cxl_endpoint_decoder *cxled = p->targets[i];
> +		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> +		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
> +
> +		/*
> +		 * Regions never span CXL root devices, so by definition the
> +		 * bridge for one device is the same for all.
> +		 */
> +		if (i == 0) {
> +			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
> +			if (!cxl_nvb)
> +				return -ENODEV;
> +			cxlr->cxl_nvb = cxl_nvb;
> +		}
> +		m->cxlmd = cxlmd;
> +		get_device(&cxlmd->dev);
> +		m->start = cxled->dpa_res->start;
> +		m->size = resource_size(cxled->dpa_res);
> +		m->position = i;
> +	}
> +
> +	dev = &cxlr_pmem->dev;
> +	device_initialize(dev);
> +	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
> +	device_set_pm_not_required(dev);
> +	dev->parent = &cxlr->dev;
> +	dev->bus = &cxl_bus_type;
> +	dev->type = &cxl_pmem_region_type;
> +	cxlr_pmem->cxlr = cxlr;
> +	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
> +
> +	return 0;
> +}
> +
> +static void cxlr_pmem_unregister(void *_cxlr_pmem)
> +{
> +	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
> +	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
> +
> +	/*
> +	 * Either the bridge is in ->remove() context under the device_lock(),
> +	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
> +	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
> +	 * lock).
> +	 */
> +	device_lock_assert(&cxl_nvb->dev);
> +	cxlr->cxlr_pmem = NULL;
> +	cxlr_pmem->cxlr = NULL;
> +	device_unregister(&cxlr_pmem->dev);
> +}
> +
> +static void cxlr_release_nvdimm(void *_cxlr)
> +{
> +	struct cxl_region *cxlr = _cxlr;
> +	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
> +
> +	scoped_guard(device, &cxl_nvb->dev) {
> +		if (cxlr->cxlr_pmem)
> +			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
> +					    cxlr->cxlr_pmem);
> +	}
> +	cxlr->cxl_nvb = NULL;
> +	put_device(&cxl_nvb->dev);
> +}
> +
> +/**
> + * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
> + * @cxlr: parent CXL region for this pmem region bridge device
> + *
> + * Return: 0 on success negative error code on failure.
> + */
> +int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
> +{
> +	struct cxl_pmem_region *cxlr_pmem;
> +	struct cxl_nvdimm_bridge *cxl_nvb;
> +	struct device *dev;
> +	int rc;
> +
> +	rc = cxl_pmem_region_alloc(cxlr);
> +	if (rc)
> +		return rc;
> +	cxlr_pmem = cxlr->cxlr_pmem;
> +	cxl_nvb = cxlr->cxl_nvb;
> +
> +	dev = &cxlr_pmem->dev;
> +	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
> +	if (rc)
> +		goto err;
> +
> +	rc = device_add(dev);
> +	if (rc)
> +		goto err;
> +
> +	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
> +		dev_name(dev));
> +
> +	scoped_guard(device, &cxl_nvb->dev) {
> +		if (cxl_nvb->dev.driver)
> +			rc = devm_add_action_or_reset(&cxl_nvb->dev,
> +						      cxlr_pmem_unregister,
> +						      cxlr_pmem);
> +		else
> +			rc = -ENXIO;
> +	}
> +
> +	if (rc)
> +		goto err_bridge;
> +
> +	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
> +	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
> +
> +err:
> +	put_device(dev);
> +err_bridge:
> +	put_device(&cxl_nvb->dev);
> +	cxlr->cxl_nvb = NULL;
> +	return rc;
> +}
> +EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index 647d9ce32b64..717de1d3f70e 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
>  		return CXL_DEVICE_NVDIMM_BRIDGE;
>  	if (dev->type == &cxl_nvdimm_type)
>  		return CXL_DEVICE_NVDIMM;
> -	if (dev->type == CXL_PMEM_REGION_TYPE())
> +	if (dev->type == CXL_PMEM_REGION_TYPE)

Stray edit? I don't think anything changed in the declaration.

>  		return CXL_DEVICE_PMEM_REGION;
>  	if (dev->type == CXL_DAX_REGION_TYPE())
>  		return CXL_DEVICE_DAX_REGION;
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index d5c227ce7b09..d2ef7fcc19fe 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -38,8 +38,6 @@
>   */
>  static nodemask_t nodemask_region_seen = NODE_MASK_NONE;
>  
> -static struct cxl_region *to_cxl_region(struct device *dev);
> -
>  #define __ACCESS_ATTR_RO(_level, _name) {				\
>  	.attr	= { .name = __stringify(_name), .mode = 0444 },		\
>  	.show	= _name##_access##_level##_show,			\
> @@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
>  }
>  EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
>  
> -static struct cxl_region *to_cxl_region(struct device *dev)
> +struct cxl_region *to_cxl_region(struct device *dev)
>  {
>  	if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
>  			  "not a cxl_region device\n"))
> @@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
>  
>  	return container_of(dev, struct cxl_region, dev);
>  }
> +EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");

Maybe just move this into the header file instead.

DJ

>  
>  static void unregister_region(void *_cxlr)
>  {
> @@ -2814,46 +2813,6 @@ static ssize_t delete_region_store(struct device *dev,
>  }
>  DEVICE_ATTR_WO(delete_region);
>  
> -static void cxl_pmem_region_release(struct device *dev)
> -{
> -	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
> -	int i;
> -
> -	for (i = 0; i < cxlr_pmem->nr_mappings; i++) {
> -		struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd;
> -
> -		put_device(&cxlmd->dev);
> -	}
> -
> -	kfree(cxlr_pmem);
> -}
> -
> -static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
> -	&cxl_base_attribute_group,
> -	NULL,
> -};
> -
> -const struct device_type cxl_pmem_region_type = {
> -	.name = "cxl_pmem_region",
> -	.release = cxl_pmem_region_release,
> -	.groups = cxl_pmem_region_attribute_groups,
> -};
> -
> -bool is_cxl_pmem_region(struct device *dev)
> -{
> -	return dev->type == &cxl_pmem_region_type;
> -}
> -EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL");
> -
> -struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
> -{
> -	if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev),
> -			  "not a cxl_pmem_region device\n"))
> -		return NULL;
> -	return container_of(dev, struct cxl_pmem_region, dev);
> -}
> -EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL");
> -
>  struct cxl_poison_context {
>  	struct cxl_port *port;
>  	int part;
> @@ -3213,64 +3172,6 @@ static int region_offset_to_dpa_result(struct cxl_region *cxlr, u64 offset,
>  	return -ENXIO;
>  }
>  
> -static struct lock_class_key cxl_pmem_region_key;
> -
> -static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
> -{
> -	struct cxl_region_params *p = &cxlr->params;
> -	struct cxl_nvdimm_bridge *cxl_nvb;
> -	struct device *dev;
> -	int i;
> -
> -	guard(rwsem_read)(&cxl_rwsem.region);
> -	if (p->state != CXL_CONFIG_COMMIT)
> -		return -ENXIO;
> -
> -	struct cxl_pmem_region *cxlr_pmem __free(kfree) =
> -		kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets), GFP_KERNEL);
> -	if (!cxlr_pmem)
> -		return -ENOMEM;
> -
> -	cxlr_pmem->hpa_range.start = p->res->start;
> -	cxlr_pmem->hpa_range.end = p->res->end;
> -
> -	/* Snapshot the region configuration underneath the cxl_rwsem.region */
> -	cxlr_pmem->nr_mappings = p->nr_targets;
> -	for (i = 0; i < p->nr_targets; i++) {
> -		struct cxl_endpoint_decoder *cxled = p->targets[i];
> -		struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> -		struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i];
> -
> -		/*
> -		 * Regions never span CXL root devices, so by definition the
> -		 * bridge for one device is the same for all.
> -		 */
> -		if (i == 0) {
> -			cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint);
> -			if (!cxl_nvb)
> -				return -ENODEV;
> -			cxlr->cxl_nvb = cxl_nvb;
> -		}
> -		m->cxlmd = cxlmd;
> -		get_device(&cxlmd->dev);
> -		m->start = cxled->dpa_res->start;
> -		m->size = resource_size(cxled->dpa_res);
> -		m->position = i;
> -	}
> -
> -	dev = &cxlr_pmem->dev;
> -	device_initialize(dev);
> -	lockdep_set_class(&dev->mutex, &cxl_pmem_region_key);
> -	device_set_pm_not_required(dev);
> -	dev->parent = &cxlr->dev;
> -	dev->bus = &cxl_bus_type;
> -	dev->type = &cxl_pmem_region_type;
> -	cxlr_pmem->cxlr = cxlr;
> -	cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem);
> -
> -	return 0;
> -}
> -
>  static void cxl_dax_region_release(struct device *dev)
>  {
>  	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
> @@ -3334,92 +3235,6 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
>  	return cxlr_dax;
>  }
>  
> -static void cxlr_pmem_unregister(void *_cxlr_pmem)
> -{
> -	struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem;
> -	struct cxl_region *cxlr = cxlr_pmem->cxlr;
> -	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
> -
> -	/*
> -	 * Either the bridge is in ->remove() context under the device_lock(),
> -	 * or cxlr_release_nvdimm() is cancelling the bridge's release action
> -	 * for @cxlr_pmem and doing it itself (while manually holding the bridge
> -	 * lock).
> -	 */
> -	device_lock_assert(&cxl_nvb->dev);
> -	cxlr->cxlr_pmem = NULL;
> -	cxlr_pmem->cxlr = NULL;
> -	device_unregister(&cxlr_pmem->dev);
> -}
> -
> -static void cxlr_release_nvdimm(void *_cxlr)
> -{
> -	struct cxl_region *cxlr = _cxlr;
> -	struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb;
> -
> -	scoped_guard(device, &cxl_nvb->dev) {
> -		if (cxlr->cxlr_pmem)
> -			devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister,
> -					    cxlr->cxlr_pmem);
> -	}
> -	cxlr->cxl_nvb = NULL;
> -	put_device(&cxl_nvb->dev);
> -}
> -
> -/**
> - * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge
> - * @cxlr: parent CXL region for this pmem region bridge device
> - *
> - * Return: 0 on success negative error code on failure.
> - */
> -static int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
> -{
> -	struct cxl_pmem_region *cxlr_pmem;
> -	struct cxl_nvdimm_bridge *cxl_nvb;
> -	struct device *dev;
> -	int rc;
> -
> -	rc = cxl_pmem_region_alloc(cxlr);
> -	if (rc)
> -		return rc;
> -	cxlr_pmem = cxlr->cxlr_pmem;
> -	cxl_nvb = cxlr->cxl_nvb;
> -
> -	dev = &cxlr_pmem->dev;
> -	rc = dev_set_name(dev, "pmem_region%d", cxlr->id);
> -	if (rc)
> -		goto err;
> -
> -	rc = device_add(dev);
> -	if (rc)
> -		goto err;
> -
> -	dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent),
> -		dev_name(dev));
> -
> -	scoped_guard(device, &cxl_nvb->dev) {
> -		if (cxl_nvb->dev.driver)
> -			rc = devm_add_action_or_reset(&cxl_nvb->dev,
> -						      cxlr_pmem_unregister,
> -						      cxlr_pmem);
> -		else
> -			rc = -ENXIO;
> -	}
> -
> -	if (rc)
> -		goto err_bridge;
> -
> -	/* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */
> -	return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr);
> -
> -err:
> -	put_device(dev);
> -err_bridge:
> -	put_device(&cxl_nvb->dev);
> -	cxlr->cxl_nvb = NULL;
> -	return rc;
> -}
> -
>  static void cxlr_dax_unregister(void *_cxlr_dax)
>  {
>  	struct cxl_dax_region *cxlr_dax = _cxlr_dax;
> @@ -3985,6 +3800,8 @@ static int cxl_region_probe(struct device *dev)
>  			dev_dbg(&cxlr->dev, "CXL EDAC registration for region_id=%d failed\n",
>  				cxlr->id);
>  
> +		if (!IS_ENABLED(CONFIG_CXL_PMEM_REGION))
> +			return -EINVAL;
>  		return devm_cxl_add_pmem_region(cxlr);
>  	case CXL_PARTMODE_RAM:
>  		rc = devm_cxl_region_edac_register(cxlr);
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index 1eb1aca7c69f..0d576b359de6 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -825,6 +825,7 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,
>  			struct cxl_endpoint_dvsec_info *info);
>  
>  bool is_cxl_region(struct device *dev);
> +struct cxl_region *to_cxl_region(struct device *dev);
>  
>  extern const struct bus_type cxl_bus_type;
>  
> @@ -869,8 +870,6 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port);
>  struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port);
>  
>  #ifdef CONFIG_CXL_REGION
> -bool is_cxl_pmem_region(struct device *dev);
> -struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
>  int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
>  struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
>  u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa);
> @@ -880,14 +879,6 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
>  				     struct cxl_pmem_region_params *params,
>  				     struct cxl_decoder *cxld);
>  #else
> -static inline bool is_cxl_pmem_region(struct device *dev)
> -{
> -	return false;
> -}
> -static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
> -{
> -	return NULL;
> -}
>  static inline int cxl_add_to_region(struct cxl_endpoint_decoder *cxled)
>  {
>  	return 0;
> @@ -914,6 +905,25 @@ cxl_create_region(struct cxl_root_decoder *cxlrd,
>  }
>  #endif
>  
> +#ifdef CONFIG_CXL_PMEM_REGION
> +bool is_cxl_pmem_region(struct device *dev);
> +struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
> +int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
> +#else
> +static inline bool is_cxl_pmem_region(struct device *dev)
> +{
> +	return false;
> +}
> +static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
> +{
> +	return NULL;
> +}
> +static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
> +{
> +	return 0;
> +}
> +#endif
> +
>  void cxl_endpoint_parse_cdat(struct cxl_port *port);
>  void cxl_switch_parse_cdat(struct cxl_port *port);
>  
> diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild
> index d07f14cb7aa4..cf58ada337b7 100644
> --- a/tools/testing/cxl/Kbuild
> +++ b/tools/testing/cxl/Kbuild
> @@ -64,6 +64,7 @@ cxl_core-y += $(CXL_CORE_SRC)/cdat.o
>  cxl_core-y += $(CXL_CORE_SRC)/ras.o
>  cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
>  cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o
> +cxl_core-$(CONFIG_CXL_PMEM_REGION) += $(CXL_CORE_SRC)/pmem_region.o
>  cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o
>  cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o
>  cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o

Re: [PATCH V3 19/20] cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Using these attributes region label is added/deleted into LSA. These
> attributes are called from userspace (ndctl) after region creation.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  Documentation/ABI/testing/sysfs-bus-cxl | 22 ++++++
>  drivers/cxl/core/pmem_region.c          | 91 ++++++++++++++++++++++++-
>  drivers/cxl/cxl.h                       |  1 +
>  3 files changed, 113 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
> index 6b4e8c7a963d..d6080fcf843a 100644
> --- a/Documentation/ABI/testing/sysfs-bus-cxl
> +++ b/Documentation/ABI/testing/sysfs-bus-cxl
> @@ -615,3 +615,25 @@ Description:
>  		The count is persistent across power loss and wraps back to 0
>  		upon overflow. If this file is not present, the device does not
>  		have the necessary support for dirty tracking.
> +
> +
> +What:		/sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_update
> +Date:		Sept, 2025
> +KernelVersion:	v6.17
> +Contact:	linux-cxl@vger.kernel.org
> +Description:
> +		(RW) Write a boolean 'true' string value to this attribute to
> +		update cxl region information into LSA as region label. It
> +		uses nvdimm nd_region_label_update() to update cxl region
> +		information saved during cxl region creation into LSA. This
> +		attribute must be called at last during cxl region creation.
> +
> +
> +What:		/sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_delete
> +Date:		Sept, 2025
> +KernelVersion:	v6.17
> +Contact:	linux-cxl@vger.kernel.org
> +Description:
> +		(WO) When a boolean 'true' is written to this attribute then
> +		pmem_region driver deletes cxl region label from LSA using
> +		nvdimm nd_region_label_delete()
> diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
> index 55b80d587403..665b603c907b 100644
> --- a/drivers/cxl/core/pmem_region.c
> +++ b/drivers/cxl/core/pmem_region.c
> @@ -45,9 +45,98 @@ static void cxl_pmem_region_release(struct device *dev)
>  	kfree(cxlr_pmem);
>  }
>  
> +static ssize_t region_label_update_store(struct device *dev,
> +					 struct device_attribute *attr,
> +					 const char *buf, size_t len)
> +{
> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
> +	ssize_t rc;
> +	bool update;
> +
> +	rc = kstrtobool(buf, &update);
> +	if (rc)
> +		return rc;
> +
> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
> +	if (rc)
> +		return rc;
> +
> +	/* Region not yet committed */
> +	if (update && cxlr && cxlr->params.state != CXL_CONFIG_COMMIT) {
> +		dev_dbg(dev, "region not committed, can't update into LSA\n");
> +		return -ENXIO;
> +	}
> +
> +	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
> +		rc = nd_region_label_update(cxlr->cxlr_pmem->nd_region);
> +		if (!rc)
> +			cxlr->params.region_label_state = 1;
> +	}
> +
> +	if (rc)
> +		return rc;

Feels like this segment should look like

if (!cxlr || !cxlr->cxlr_pmem || ! cxlr->cxlr_pmem->nd_region)
	return 0;

rc = nd_region_label_update(..);
if (rc)
	return rc;

cxlr->params.region_label_state = 1;

> +
> +	return len;
> +}
> +
> +static ssize_t region_label_update_show(struct device *dev,
> +					struct device_attribute *attr,
> +					char *buf)
> +{
> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
> +	struct cxl_region_params *p = &cxlr->params;
> +	ssize_t rc;
> +
> +	ACQUIRE(rwsem_read_intr, rwsem)(&cxl_rwsem.region);
> +	rc = ACQUIRE_ERR(rwsem_read_intr, &rwsem);
> +	if (rc)
> +		return rc;
> +
> +	return sysfs_emit(buf, "%d\n", p->region_label_state);
> +}
> +static DEVICE_ATTR_RW(region_label_update);
> +
> +static ssize_t region_label_delete_store(struct device *dev,
> +					 struct device_attribute *attr,
> +					 const char *buf, size_t len)
> +{
> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
> +	ssize_t rc;
> +
> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
> +	if (rc)
> +		return rc;
> +
> +	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
> +		rc = nd_region_label_delete(cxlr->cxlr_pmem->nd_region);
> +		if (rc)
> +			return rc;
> +		cxlr->params.region_label_state = 0;
> +	}

Similar to above. You can exit early and not have to indent.

> +
> +	return len;
> +}
> +static DEVICE_ATTR_WO(region_label_delete);
> +
> +static struct attribute *cxl_pmem_region_attrs[] = {
> +	&dev_attr_region_label_update.attr,
> +	&dev_attr_region_label_delete.attr,
> +	NULL
> +};
> +
> +static struct attribute_group cxl_pmem_region_group = {
> +	.attrs = cxl_pmem_region_attrs,
> +};
> +
>  static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
>  	&cxl_base_attribute_group,
> -	NULL,
> +	&cxl_pmem_region_group,
> +	NULL
>  };
>  
>  const struct device_type cxl_pmem_region_type = {
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index 0d576b359de6..f01f8c942fdf 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -484,6 +484,7 @@ enum cxl_config_state {
>   */
>  struct cxl_region_params {
>  	enum cxl_config_state state;
> +	int region_label_state;

Maybe a enum?

>  	uuid_t uuid;
>  	int interleave_ways;
>  	int interleave_granularity;

Re: [PATCH V3 20/20] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem

[Dave Jiang]

On 9/17/25 6:41 AM, Neeraj Kumar wrote:
> Add support of CXL LSA 2.1 using NDD_REGION_LABELING flag. It creates
> cxl region based on region information parsed from LSA.
> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/core/pmem_region.c | 53 ++++++++++++++++++++++++++++++++++
>  drivers/cxl/cxl.h              |  4 +++
>  drivers/cxl/pmem.c             |  2 ++
>  3 files changed, 59 insertions(+)
> 
> diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c
> index 665b603c907b..3ef9c7d15041 100644
> --- a/drivers/cxl/core/pmem_region.c
> +++ b/drivers/cxl/core/pmem_region.c
> @@ -290,3 +290,56 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
>  	return rc;
>  }
>  EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
> +
> +static int match_free_ep_decoder(struct device *dev, const void *data)
> +{
> +	struct cxl_decoder *cxld = to_cxl_decoder(dev);

I think this is needed if the function is match_free_ep_decoder().

if (!is_endpoint_decoder(dev))
	return 0;

> +
> +	return !cxld->region;
> +}

May want to borrow some code from match_free_decoder() in core/region.c. I think the decoder commit order matters?

> +
> +static struct cxl_decoder *cxl_find_free_ep_decoder(struct cxl_port *port)
> +{
> +	struct device *dev;
> +
> +	dev = device_find_child(&port->dev, NULL, match_free_ep_decoder);
> +	if (!dev)
> +		return NULL;
> +
> +	/* Release device ref taken via device_find_child() */
> +	put_device(dev);

Should have the caller put the device.

> +	return to_cxl_decoder(dev);
> +}
> +
> +void create_pmem_region(struct nvdimm *nvdimm)
> +{
> +	struct cxl_nvdimm *cxl_nvd;
> +	struct cxl_memdev *cxlmd;
> +	struct cxl_pmem_region_params *params;
> +	struct cxl_root_decoder *cxlrd;
> +	struct cxl_decoder *cxld;
> +	struct cxl_region *cxlr;
> +
> +	if (!nvdimm_has_cxl_region(nvdimm))
> +		return;
> +
> +	lockdep_assert_held(&cxl_rwsem.region);
> +	cxl_nvd = nvdimm_provider_data(nvdimm);
> +	params = nvdimm_get_cxl_region_param(nvdimm);
> +	cxlmd = cxl_nvd->cxlmd;
> +	cxlrd = cxlmd->cxlrd;
> +
> +	 /* TODO: Region creation support only for interleave way == 1 */
> +	if (!(params->nlabel == 1))
> +		dev_info(&cxlmd->dev,
> +			 "Region Creation is not supported with iw > 1\n");

Why not just exit here. Then the else is not necessary.

Also maybe deb_dbg().

> +	else {
> +		cxld = cxl_find_free_ep_decoder(cxlmd->endpoint);
> +		cxlr = cxl_create_region(cxlrd, CXL_PARTMODE_PMEM,
> +					 atomic_read(&cxlrd->region_id),
> +					 params, cxld);
> +		if (IS_ERR(cxlr))
> +			dev_info(&cxlmd->dev, "Region Creation failed\n");

dev_warn()

> +	}
> +}
> +EXPORT_SYMBOL_NS_GPL(create_pmem_region, "CXL");
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index f01f8c942fdf..0a87ea79742a 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -910,6 +910,7 @@ cxl_create_region(struct cxl_root_decoder *cxlrd,
>  bool is_cxl_pmem_region(struct device *dev);
>  struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
>  int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
> +void create_pmem_region(struct nvdimm *nvdimm);
>  #else
>  static inline bool is_cxl_pmem_region(struct device *dev)
>  {
> @@ -923,6 +924,9 @@ static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
>  {
>  	return 0;
>  }
> +static inline void create_pmem_region(struct nvdimm *nvdimm)
> +{
> +}
>  #endif
>  
>  void cxl_endpoint_parse_cdat(struct cxl_port *port);
> diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c
> index 38a5bcdc68ce..0cdef01dbc68 100644
> --- a/drivers/cxl/pmem.c
> +++ b/drivers/cxl/pmem.c
> @@ -135,6 +135,7 @@ static int cxl_nvdimm_probe(struct device *dev)
>  		return rc;
>  
>  	set_bit(NDD_LABELING, &flags);
> +	set_bit(NDD_REGION_LABELING, &flags);
>  	set_bit(NDD_REGISTER_SYNC, &flags);
>  	set_bit(ND_CMD_GET_CONFIG_SIZE, &cmd_mask);
>  	set_bit(ND_CMD_GET_CONFIG_DATA, &cmd_mask);
> @@ -155,6 +156,7 @@ static int cxl_nvdimm_probe(struct device *dev)
>  		return -ENOMEM;
>  
>  	dev_set_drvdata(dev, nvdimm);
> +	create_pmem_region(nvdimm);
>  	return devm_add_action_or_reset(dev, unregister_nvdimm, nvdimm);
>  }
>  

Re: [PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Alison Schofield]

On Mon, Sep 22, 2025 at 06:14:40PM +0530, Neeraj Kumar wrote:
> On 19/09/25 04:34PM, Dave Jiang wrote:
> > 
> > 
> > On 9/17/25 6:40 AM, Neeraj Kumar wrote:
> > > nd_label_base() was being used after typecasting with 'unsigned long'. Thus
> > > modified nd_label_base() to return 'unsigned long' instead of 'struct
> > > nd_namespace_label *'
> > > 
> > > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> > Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> > 
> > Just a nit below:
> > 
> > 
> > > ---
> > >  drivers/nvdimm/label.c | 10 +++++-----
> > >  1 file changed, 5 insertions(+), 5 deletions(-)
> > > 
> > > -static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd)
> > > +static unsigned long nd_label_base(struct nvdimm_drvdata *ndd)
> > >  {
> > >  	void *base = to_namespace_index(ndd, 0);
> > > 
> > > -	return base + 2 * sizeof_namespace_index(ndd);
> > > +	return (unsigned long) (base + 2 * sizeof_namespace_index(ndd));
> > 
> > Space is not needed between casting and the var. Also applies to other instances in this commit.
> > 
> > DJ
> 
> Thanks Jonathan, Ira and Dave for RB tag. Sure, I will fix this in next
> patch-set.

This is independent of the patchset, right?
How about just sending a one off patch for this, and shortening this
set by a tiny bit :)


> 
> 
> Regards,
> Neeraj

Re: [PATCH V3 08/20] nvdimm/label: Include region label in slot validation

[Alison Schofield]

On Wed, Sep 17, 2025 at 07:11:04PM +0530, Neeraj Kumar wrote:
> Slot validation routine validates label slot by calculating label
> checksum. It was only validating namespace label. This changeset also
> validates region label if present.

Neeraj,

Was it previously wrong 'only validating namespace label'
- or -
Is validating region label something new available with cxl
namespaces that we now must add?

I'm looking for that pattern in these commit messages, ie
this is how it use to be, this is why it doesn't work now,
so here is the change.

-- Alison


> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> ---
>  drivers/nvdimm/label.c | 72 ++++++++++++++++++++++++++++++++----------
>  drivers/nvdimm/nd.h    |  5 +++
>  2 files changed, 60 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index d33db96ba8ba..5e476154cf81 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -359,7 +359,7 @@ static bool nsl_validate_checksum(struct nvdimm_drvdata *ndd,
>  {
>  	u64 sum, sum_save;
>  
> -	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
> +	if (!efi_namespace_label_has(ndd, checksum))
>  		return true;
>  
>  	sum_save = nsl_get_checksum(ndd, nd_label);
> @@ -374,13 +374,25 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
>  {
>  	u64 sum;
>  
> -	if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum))
> +	if (!efi_namespace_label_has(ndd, checksum))
>  		return;
>  	nsl_set_checksum(ndd, nd_label, 0);
>  	sum = nd_fletcher64(nd_label, sizeof_namespace_label(ndd), 1);
>  	nsl_set_checksum(ndd, nd_label, sum);
>  }
>  
> +static bool region_label_validate_checksum(struct nvdimm_drvdata *ndd,
> +				struct cxl_region_label *region_label)
> +{
> +	u64 sum, sum_save;
> +
> +	sum_save = region_label_get_checksum(region_label);
> +	region_label_set_checksum(region_label, 0);
> +	sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1);
> +	region_label_set_checksum(region_label, sum_save);
> +	return sum == sum_save;
> +}
> +
>  static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>  				   struct cxl_region_label *region_label)
>  {
> @@ -392,16 +404,30 @@ static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>  }
>  
>  static bool slot_valid(struct nvdimm_drvdata *ndd,
> -		struct nd_namespace_label *nd_label, u32 slot)
> +		       union nd_lsa_label *lsa_label, u32 slot)
>  {
> +	struct cxl_region_label *region_label = &lsa_label->region_label;
> +	struct nd_namespace_label *nd_label = &lsa_label->ns_label;
> +	char *label_name;
>  	bool valid;
>  
>  	/* check that we are written where we expect to be written */
> -	if (slot != nsl_get_slot(ndd, nd_label))
> -		return false;
> -	valid = nsl_validate_checksum(ndd, nd_label);
> +	if (is_region_label(ndd, lsa_label)) {
> +		label_name = "rg";
> +		if (slot != region_label_get_slot(region_label))
> +			return false;
> +		valid = region_label_validate_checksum(ndd, region_label);
> +	} else {
> +		label_name = "ns";
> +		if (slot != nsl_get_slot(ndd, nd_label))
> +			return false;
> +		valid = nsl_validate_checksum(ndd, nd_label);
> +	}
> +
>  	if (!valid)
> -		dev_dbg(ndd->dev, "fail checksum. slot: %d\n", slot);
> +		dev_dbg(ndd->dev, "%s label checksum fail. slot: %d\n",
> +			label_name, slot);
> +
>  	return valid;
>  }
>  
> @@ -424,7 +450,7 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd)
>  
>  		nd_label = to_label(ndd, slot);
>  
> -		if (!slot_valid(ndd, nd_label, slot))
> +		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
>  			continue;
>  
>  		nsl_get_uuid(ndd, nd_label, &label_uuid);
> @@ -575,18 +601,30 @@ int nd_label_active_count(struct nvdimm_drvdata *ndd)
>  		return 0;
>  
>  	for_each_clear_bit_le(slot, free, nslot) {
> +		struct cxl_region_label *region_label;
>  		struct nd_namespace_label *nd_label;
> -
> -		nd_label = to_label(ndd, slot);
> -
> -		if (!slot_valid(ndd, nd_label, slot)) {
> -			u32 label_slot = nsl_get_slot(ndd, nd_label);
> -			u64 size = nsl_get_rawsize(ndd, nd_label);
> -			u64 dpa = nsl_get_dpa(ndd, nd_label);
> +		union nd_lsa_label *lsa_label;
> +		u32 lslot;
> +		u64 size, dpa;
> +
> +		lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
> +		nd_label = &lsa_label->ns_label;
> +		region_label = &lsa_label->region_label;
> +
> +		if (!slot_valid(ndd, lsa_label, slot)) {
> +			if (is_region_label(ndd, lsa_label)) {
> +				lslot = __le32_to_cpu(region_label->slot);
> +				size = __le64_to_cpu(region_label->rawsize);
> +				dpa = __le64_to_cpu(region_label->dpa);
> +			} else {
> +				lslot = nsl_get_slot(ndd, nd_label);
> +				size = nsl_get_rawsize(ndd, nd_label);
> +				dpa = nsl_get_dpa(ndd, nd_label);
> +			}
>  
>  			dev_dbg(ndd->dev,
>  				"slot%d invalid slot: %d dpa: %llx size: %llx\n",
> -					slot, label_slot, dpa, size);
> +					slot, lslot, dpa, size);
>  			continue;
>  		}
>  		count++;
> @@ -607,7 +645,7 @@ struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n)
>  		struct nd_namespace_label *nd_label;
>  
>  		nd_label = to_label(ndd, slot);
> -		if (!slot_valid(ndd, nd_label, slot))
> +		if (!slot_valid(ndd, (union nd_lsa_label *) nd_label, slot))
>  			continue;
>  
>  		if (n-- == 0)
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index 046063ea08b6..c985f91728dd 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -344,6 +344,11 @@ region_label_uuid_equal(struct cxl_region_label *region_label,
>  	return uuid_equal((uuid_t *) region_label->uuid, uuid);
>  }
>  
> +static inline u32 region_label_get_slot(struct cxl_region_label *region_label)
> +{
> +	return __le32_to_cpu(region_label->slot);
> +}
> +
>  static inline u64
>  region_label_get_checksum(struct cxl_region_label *region_label)
>  {
> -- 
> 2.34.1
> 
> 

Re: [PATCH V3 16/20] cxl/mem: Preserve cxl root decoder during mem probe

[Alison Schofield]

On Wed, Sep 17, 2025 at 07:11:12PM +0530, Neeraj Kumar wrote:
> Saved root decoder info is required for cxl region persistency

It seem there must be a more detailed story here.
Saving the root decoder in struct cxl_memdev does not sound
persistent. Please add more detail on how this step fits
into the grander scheme.



> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/cxl/cxlmem.h | 1 +
>  drivers/cxl/mem.c    | 2 ++
>  2 files changed, 3 insertions(+)
> 
> diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> index 434031a0c1f7..25cb115b72bd 100644
> --- a/drivers/cxl/cxlmem.h
> +++ b/drivers/cxl/cxlmem.h
> @@ -59,6 +59,7 @@ struct cxl_memdev {
>  	struct cxl_nvdimm_bridge *cxl_nvb;
>  	struct cxl_nvdimm *cxl_nvd;
>  	struct cxl_port *endpoint;
> +	struct cxl_root_decoder *cxlrd;
>  	int id;
>  	int depth;
>  	u8 scrub_cycle;
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 54501616ff09..1a0da7253a24 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -152,6 +152,8 @@ static int cxl_mem_probe(struct device *dev)
>  		return -ENXIO;
>  	}
>  
> +	cxlmd->cxlrd = cxl_find_root_decoder_by_port(parent_port);
> +
>  	if (dport->rch)
>  		endpoint_parent = parent_port->uport_dev;
>  	else
> -- 
> 2.34.1
> 
> 

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Alison Schofield]

On Wed, Sep 17, 2025 at 07:11:00PM +0530, Neeraj Kumar wrote:
> Updated mutex_lock() with guard(mutex)()

Along with the other reviewer comments to limit the application
of guard() to where it is most useful, can this be spun off
as a single patch that would be done irregardless on the
new label support?

I'm asking the question. I don't know the answer ;)


> 
> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
> ---
>  drivers/nvdimm/label.c | 36 +++++++++++++++++-------------------
>  1 file changed, 17 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
> index 668e1e146229..3235562d0e1c 100644
> --- a/drivers/nvdimm/label.c
> +++ b/drivers/nvdimm/label.c
> @@ -948,7 +948,7 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  		return rc;
>  
>  	/* Garbage collect the previous label */
> -	mutex_lock(&nd_mapping->lock);
> +	guard(mutex)(&nd_mapping->lock);
>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>  		if (!label_ent->label)
>  			continue;
> @@ -960,20 +960,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>  	/* update index */
>  	rc = nd_label_write_index(ndd, ndd->ns_next,
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
> -	if (rc == 0) {
> -		list_for_each_entry(label_ent, &nd_mapping->labels, list)
> -			if (!label_ent->label) {
> -				label_ent->label = nd_label;
> -				nd_label = NULL;
> -				break;
> -			}
> -		dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> -				"failed to track label: %d\n",
> -				to_slot(ndd, nd_label));
> -		if (nd_label)
> -			rc = -ENXIO;
> -	}
> -	mutex_unlock(&nd_mapping->lock);
> +	if (rc)
> +		return rc;
> +
> +	list_for_each_entry(label_ent, &nd_mapping->labels, list)
> +		if (!label_ent->label) {
> +			label_ent->label = nd_label;
> +			nd_label = NULL;
> +			break;
> +		}
> +	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
> +			"failed to track label: %d\n",
> +			to_slot(ndd, nd_label));
> +	if (nd_label)
> +		rc = -ENXIO;
>  
>  	return rc;
>  }
> @@ -998,9 +998,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>  		label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL);
>  		if (!label_ent)
>  			return -ENOMEM;
> -		mutex_lock(&nd_mapping->lock);
> +		guard(mutex)(&nd_mapping->lock);
>  		list_add_tail(&label_ent->list, &nd_mapping->labels);
> -		mutex_unlock(&nd_mapping->lock);
>  	}
>  
>  	if (ndd->ns_current == -1 || ndd->ns_next == -1)
> @@ -1039,7 +1038,7 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>  		return 0;
>  
> -	mutex_lock(&nd_mapping->lock);
> +	guard(mutex)(&nd_mapping->lock);
>  	list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) {
>  		struct nd_namespace_label *nd_label = label_ent->label;
>  
> @@ -1061,7 +1060,6 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>  		nd_mapping_free_labels(nd_mapping);
>  		dev_dbg(ndd->dev, "no more active labels\n");
>  	}
> -	mutex_unlock(&nd_mapping->lock);
>  
>  	return nd_label_write_index(ndd, ndd->ns_next,
>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
> -- 
> 2.34.1
> 
> 

Re: [PATCH V3 07/20] nvdimm/region_label: Add region label delete support

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1776 bytes --]

On 22/09/25 02:37PM, Dave Jiang wrote:
>
>> +int nd_pmem_region_label_delete(struct nd_region *nd_region)
>> +{
>> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
>> +	struct nd_label_ent *label_ent;
>> +	int ns_region_cnt = 0;
>> +	int i, rc;
>> +
>> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
>> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>> +
>> +		/* Find non cxl format supported ndr_mappings */
>> +		if (!ndd->cxl) {
>> +			dev_info(&nd_region->dev, "Unsupported region label\n");
>> +			return -EINVAL;
>> +		}
>> +
>> +		/* Find if any NS label using this region */
>> +		guard(mutex)(&nd_mapping->lock);
>> +		list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>> +			if (!label_ent->label)
>> +				continue;
>> +
>> +			/*
>> +			 * Check if any available NS labels has same
>> +			 * region_uuid in LSA
>> +			 */
>> +			if (nsl_region_uuid_equal(label_ent->label,
>> +						  &nd_set->uuid))
>> +				ns_region_cnt++;
>
>Why not just return -EBUSY here immediately? It seems the code returns -EBUSY as long as there's 1 or more below.
>
>> +		}
>> +	}
>> +
>> +	if (ns_region_cnt) {
>> +		dev_dbg(&nd_region->dev, "Region/Namespace label in use\n");
>> +		return -EBUSY;
>> +	}
>> +
>> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
>> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>> +
>> +		rc = del_labels(nd_mapping, &nd_set->uuid, RG_LABEL_TYPE);
>> +		if (rc)
>> +			return rc;
>> +	}
>
>Can this be folded into the for loop above or does it a full pass to check before starting the label deletion process?
>
>DJ

Yes, if we use "return -EBUSY" in first loop itself then we can call del_labels()
I will fix this in next patch-set

Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 08/20] nvdimm/label: Include region label in slot validation

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]

On 22/09/25 03:17PM, Dave Jiang wrote:
>>  static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>>  				   struct cxl_region_label *region_label)
>>  {
>> @@ -392,16 +404,30 @@ static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd,
>>  }
>>
>>  static bool slot_valid(struct nvdimm_drvdata *ndd,
>> -		struct nd_namespace_label *nd_label, u32 slot)
>> +		       union nd_lsa_label *lsa_label, u32 slot)
>>  {
>> +	struct cxl_region_label *region_label = &lsa_label->region_label;
>> +	struct nd_namespace_label *nd_label = &lsa_label->ns_label;
>> +	char *label_name;
>>  	bool valid;
>>
>>  	/* check that we are written where we expect to be written */
>> -	if (slot != nsl_get_slot(ndd, nd_label))
>> -		return false;
>> -	valid = nsl_validate_checksum(ndd, nd_label);
>> +	if (is_region_label(ndd, lsa_label)) {
>> +		label_name = "rg";
>
>I suggest create a static string table enumerated by 'enum label_type'. That way you can directly address it by 'label_name[ltype]' when being used instead of assigning at run time. And maybe just use "region" and "namespace" since it's for debug output and we don't need to shorten it.
>
>DJ

Sure Dave, Thanks for your suggestion. I will fix it accordingly

Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 985 bytes --]

On 22/09/25 04:11PM, Dave Jiang wrote:
>> +static inline bool is_region_label(struct nvdimm_drvdata *ndd,
>> +				   union nd_lsa_label *nd_label)
>> +{
>> +	uuid_t region_type, *ns_type;
>> +
>> +	if (!ndd->cxl || !nd_label)
>> +		return false;
>> +
>> +	uuid_parse(CXL_REGION_UUID, &region_type);
>> +	ns_type = (uuid_t *) nd_label->ns_label.cxl.type;
>
>So in addition to Jonathan's comments, I think we should consider utilizing the common field (UUID) of all the labels. i.e. if you are to use a union, you can also include 'uuid_t label_type' as a member. Perhaps this function can just pass in a UUID rather than a label struct. And we can probably skip passing in 'ndd' and not bother with ndd->cxl check. This function can just rely on checking the UUID and see if it's the expected region label UUID. Just thinking of the places we can maybe avoid doing casting if possible.
>
>DJ

Thanks Dave for detailed suggestion. I will handle this in next
patch-set


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 12/20] nvdimm/region_label: Export routine to fetch region information

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

On 23/09/25 01:23PM, Dave Jiang wrote:
>
>
>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> +bool nvdimm_has_cxl_region(struct nvdimm *nvdimm)
>> +{
>> +	if (nvdimm)
>> +		return nvdimm->is_region_label;
>> +
>> +	return false;
>
>Just a nit. Would prefer return error early and return the success case last.
>
>> +}
>> +EXPORT_SYMBOL_GPL(nvdimm_has_cxl_region);
>> +
>> +void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm)
>> +{
>> +	if (nvdimm)
>> +		return &nvdimm->cxl_region_params;
>> +
>> +	return NULL;
>
>same comment
>

Sure Dave, Will fix it accordingly


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 05/20] nvdimm/namespace_label: Add namespace label changes as per CXL LSA v2.1

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 650 bytes --]

On 23/09/25 02:48PM, Dave Jiang wrote:
>> +static inline void nsl_set_type(struct nvdimm_drvdata *ndd,
>> +				struct nd_namespace_label *ns_label)
>> +{
>> +	if (ndd->cxl && ns_label)
>> +		uuid_parse(CXL_NAMESPACE_UUID, (uuid_t *) ns_label->cxl.type);
>
>You can do:
>uuid_copy(ns_label->cxl.type, cxl_namespace_uuid);
>
>All the constant UUIDs are preparsed by nd_label_init(). For the entire series you can use those pre-parsed uuid_t and just use uuid_copy() instead of having to do string to uuid_t conversions.
>
>DJ
>

Yes using preparsed UUID's will help to avoid extra uuid_parse() usage.
Sure, I will fix it accordingly


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]

On 23/09/25 03:37PM, Dave Jiang wrote:
>
>
>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
>> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
>> used to get called at last in cxl_endpoint_port_probe(), which requires
>> cxl_nvd presence.
>>
>> For cxl region persistency, region creation happens during nvdimm_probe
>> which need the completion of endpoint probe.
>>
>> In order to accommodate both cxl pmem region auto-assembly and cxl region
>> persistency, refactored following
>>
>> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
>>    will be called only after successful completion of endpoint probe.
>>
>> 2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
>>    cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
>>    completion of endpoint probe and cxl_nvd presence before its call.
>
>Given that we are going forward with this implementation [1] from Dan and drivers like the type2 enabling are going to be using it as well, can you please consider converting this change to Dan's mechanism instead of creating a whole new one?
>
>I think the region discovery can be done via the ops->probe() callback. Thanks.
>
>[1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6
>
>DJ
>

Sure, Let me revisit this.
It seems [1] is there in seperate branch "for-6.18/cxl-probe-order", and not yet merged into next, right?


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 00/20] Add CXL LSA 2.1 format support in nvdimm and cxl pmem

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]

On 23/09/25 04:04PM, Alison Schofield wrote:
>On Wed, Sep 17, 2025 at 07:10:56PM +0530, Neeraj Kumar wrote:
>> Introduction:
>> =============
>> CXL Persistent Memory (Pmem) devices region, namespace and content must be
>> persistent across system reboot. In order to achieve this persistency, it
>> uses Label Storage Area (LSA) to store respective metadata. During system
>> reboot, stored metadata in LSA is used to bring back the region, namespace
>> and content of CXL device in its previous state.
>> CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox
>> commands to access the LSA area. nvdimm driver is using same commands to
>> get/set LSA data.
>
>big snip...
>
>> Limitation (TODO):
>> ==================
>> Current changes only support interleave way == 1
>
>I see your test setup with the one way interleave and read this
>limitation. What are your plans regarding this?
>
>ie. Do you see this limitation as something you could merge with
>or do you plan to extend this patchset with support for namespaces
>build upon region ways > 1.
>
>-- Alison

Hi Alison,

My current focus is to get current patch-set upstreamed as its in good
shape wrt interleave way == 1.

Multi-interleave support will require some more efforts on top of this
change. So will take it in another series.


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 14/20] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 2991 bytes --]

On 23/09/25 04:50PM, Dave Jiang wrote:
>
>> +static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size)
>> +{
>> +	int rc;
>> +
>> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
>> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
>> +	if (rc)
>> +		return rc;
>
>Just a nit. Please conform to existing style in the subsystem for this new usage.
>
>  +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
>  +	if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))))
>  +		return rc;
>

Actually because of checkpatch.pl error, it is different from existing
style. But recent fix by Alison at [1] will allow to fix it as per others.
Sure, I will fix it in next patch-set
[1]: https://lore.kernel.org/linux-cxl/20250815010645.2980846-1-alison.schofield@intel.com/

>> +
>> +	if (!size)
>> +		return -EINVAL;
>> +
>> +	return alloc_hpa(cxlr, size);
>> +}
>
>I think you can create another helper free_region_hpa() and call them in size_store() function to remove the duplicate code.

Sure Dave, I will fix it in next patch-set

>> +
>> +static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size)
>> +{
>> +	int rc;
>> +
>> +	if (!size)
>> +		return -EINVAL;
>> +
>> +	if (!IS_ALIGNED(size, SZ_256M))
>> +		return -EINVAL;
>> +
>> +	rc = cxl_dpa_free(cxled);
>> +	if (rc)
>> +		return rc;
>> +
>> +	return cxl_dpa_alloc(cxled, size);
>> +}
>> +
>> +static struct cxl_region *
>> +devm_cxl_pmem_add_region(struct cxl_root_decoder *cxlrd, int id,
>> +			 enum cxl_partition_mode mode,
>
>Wouldn't this not needed since it would be CXL_PARTMODE_PMEM always? I also wonder if we need to rename devm_cxl_add_region() to devm_cxl_add_ram_region() to be explicit.
>

Yes devm_cxl_pmem_add_region() always need CXL_PARTMODE_PMEM, So I will
modify it accordingly. Also I will rename devm_cxl_add_region() to
devm_cxl_add_ram_region().

>> +			 enum cxl_decoder_type type,
>> +			 struct cxl_pmem_region_params *params,
>> +			 struct cxl_decoder *cxld)
>> +{
>> +	struct cxl_endpoint_decoder *cxled;
>> +	struct cxl_region_params *p;
>> +	struct cxl_port *root_port;
>> +	struct device *dev;
>> +	int rc;
>> +

<snip>

>> +	rc = alloc_region_dpa(cxled, params->rawsize);
>> +	if (rc)
>> +		return ERR_PTR(rc);
>> +
>> +	/*
>> +	 * TODO: Currently we have support of interleave_way == 1, where
>> +	 * we can only have one region per mem device. It means mem device
>> +	 * position (params->position) will always be 0. It is therefore
>> +	 * attaching only one target at params->position
>> +	 */
>> +	if (params->position)
>> +		return ERR_PTR(-EINVAL);
>
>EOPNOTSUPP?

Yes, EOPNOTSUPP would be more appropriate than EINVAL. I will fix it in
next patch-set

>
>Speaking of which, are there plans to support interleave in the near future?
>
>DJ

My current focus is to get this upstreamed and after that will focus on
multi-interleave support. Multi-interleave support will require some more
efforts on top of this change. So will take it in another series.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 15/20] cxl: Add a routine to find cxl root decoder on cxl bus using cxl port

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

On 24/09/25 11:11AM, Dave Jiang wrote:
>
>> +struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port)
>> +{
>> +	struct cxl_root *cxl_root __free(put_cxl_root) = find_cxl_root(port);
>> +	struct device *dev;
>> +
>> +	if (!cxl_root)
>> +		return NULL;
>> +
>> +	dev = device_find_child(&cxl_root->port.dev, NULL, match_root_decoder);
>> +	if (!dev)
>> +		return NULL;
>> +
>> +	/* Release device ref taken via device_find_child() */
>> +	put_device(dev);
>
>Should the caller release the device reference instead?
>
>DJ

Actually caller of this function wants to find root decoder information from cxl_port.
So in order to find root decoder we have used device_find_child() which internally
takes device ref. Therefore, just after finding the appropriate dev, I am releasing
device ref.
Its like taking device ref temporarly and releasing it then and there after finding
proper root decoder.
I believe, Releasing device ref from caller would make it look little out of context.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 16/20] cxl/mem: Preserve cxl root decoder during mem probe

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

On 24/09/25 11:23AM, Dave Jiang wrote:
>
>
>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> Saved root decoder info is required for cxl region persistency
>
>Should squash this patch into the previous patch. It's small enough that the usage and the implementation can be in the same patch.
>

Yes its small change. I will squash this in previous patch.

Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 4014 bytes --]

On 24/09/25 11:53AM, Dave Jiang wrote:
>
>
>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
>> Moved pmem_region related code from core/region.c to core/pmem_region.c
>> For region label update, need to create device attribute, which calls
>> nvdimm exported function thus making pmem_region dependent on libnvdimm.
>> Because of this dependency of pmem region on libnvdimm, segregated pmem
>> region related code from core/region.c
>
>We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.

Hi Dave,

As per LSA 2.1, during region creation we need to intract with nvdimmm
driver to write region label into LSA.
This dependency of libnvdimm is only for PMEM region, therefore I have
created a seperate file core/pmem_region.c and copied pmem related functions
present in core/region.c into core/pmem_region.c.
Because of this movemement of code we have churn introduced in this patch.
Can you please suggest optimized way to handle dependency on libnvdimm
with minimum code changes.

>
>> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
>> index 48b7314afdb8..532eaa1bbdd6 100644
>> --- a/drivers/cxl/Kconfig
>> +++ b/drivers/cxl/Kconfig
>> @@ -211,6 +211,20 @@ config CXL_REGION
>>
>>  	  If unsure say 'y'
>>
>> +config CXL_PMEM_REGION
>> +	bool "CXL: Pmem Region Support"
>> +	default CXL_BUS
>> +	depends on CXL_REGION
>
>> +	depends on PHYS_ADDR_T_64BIT
>> +	depends on BLK_DEV
>These 2 deps are odd. What are the actual dependencies?
>

We need to add these 2 deps to fix v2 0Day issue [1]
I have taken reference from bdf97013ced5f [2]
Seems, I also have to add depends on ARCH_HAS_PMEM_API. I will update it
in V3.

[1] https://lore.kernel.org/linux-cxl/202507311017.7ApKmtQc-lkp@intel.com/
[2] https://elixir.bootlin.com/linux/v6.13.7/source/drivers/acpi/nfit/Kconfig#L4

>
>> +	select LIBNVDIMM
>> +	help
>> +	  Enable the CXL core to enumerate and provision CXL pmem regions.
>> +	  A CXL pmem region need to update region label into LSA. For LSA
>> +	  updation/deletion libnvdimm is required.
>
>s/updation/update/
>

Sure, Will fix it

>> +
>> +	  If unsure say 'y'
>> +
>>  config CXL_REGION_INVALIDATION_TEST
>>  	bool "CXL: Region Cache Management Bypass (TEST)"
>>  	depends on CXL_REGION

<snip>

>> --- a/drivers/cxl/core/port.c
>> +++ b/drivers/cxl/core/port.c
>> @@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
>>  		return CXL_DEVICE_NVDIMM_BRIDGE;
>>  	if (dev->type == &cxl_nvdimm_type)
>>  		return CXL_DEVICE_NVDIMM;
>> -	if (dev->type == CXL_PMEM_REGION_TYPE())
>> +	if (dev->type == CXL_PMEM_REGION_TYPE)
>
>Stray edit? I don't think anything changed in the declaration.
>

Sure, Will fix it

>>  		return CXL_DEVICE_PMEM_REGION;
>>  	if (dev->type == CXL_DAX_REGION_TYPE())
>>  		return CXL_DEVICE_DAX_REGION;

<snip>

>> @@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
>>  }
>>  EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
>>
>> -static struct cxl_region *to_cxl_region(struct device *dev)
>> +struct cxl_region *to_cxl_region(struct device *dev)
>>  {
>>  	if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
>>  			  "not a cxl_region device\n"))
>> @@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
>>
>>  	return container_of(dev, struct cxl_region, dev);
>>  }
>> +EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");
>
>Maybe just move this into the header file instead.
>
>DJ

Actually to_cxl_region() is internal to cxl/core and especially to core/region.c
So, Its better to compeletly remove EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL")

Even EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL") is internal to cxl/core/region.c
Should I also remove it?

Even we can remove declaration of is_cxl_region() and to_cxl_region()
from drivers/cxl/cxl.h as these functions are internal to cxl/core/region.c


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 19/20] cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 2500 bytes --]

On 24/09/25 01:25PM, Dave Jiang wrote:
>
>

<snip>

>On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>> Using these attributes region label is added/deleted into LSA. These
>> attributes are called from userspace (ndctl) after region creation.
>> +static ssize_t region_label_update_store(struct device *dev,
>> +					 struct device_attribute *attr,
>> +					 const char *buf, size_t len)
>> +{
>> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
>> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
>> +	ssize_t rc;
>> +	bool update;
>> +
>> +	rc = kstrtobool(buf, &update);
>> +	if (rc)
>> +		return rc;
>> +
>> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
>> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
>> +	if (rc)
>> +		return rc;
>> +
>> +	/* Region not yet committed */
>> +	if (update && cxlr && cxlr->params.state != CXL_CONFIG_COMMIT) {
>> +		dev_dbg(dev, "region not committed, can't update into LSA\n");
>> +		return -ENXIO;
>> +	}
>> +
>> +	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
>> +		rc = nd_region_label_update(cxlr->cxlr_pmem->nd_region);
>> +		if (!rc)
>> +			cxlr->params.region_label_state = 1;
>> +	}
>> +
>> +	if (rc)
>> +		return rc;
>
>Feels like this segment should look like
>
>if (!cxlr || !cxlr->cxlr_pmem || ! cxlr->cxlr_pmem->nd_region)
>	return 0;
>
>rc = nd_region_label_update(..);
>if (rc)
>	return rc;
>
>cxlr->params.region_label_state = 1;
>
>> +
>> +	return len;
>> +}
>> +

<snip>

>> +static ssize_t region_label_delete_store(struct device *dev,
>> +					 struct device_attribute *attr,
>> +					 const char *buf, size_t len)
>> +{
>> +	struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev);
>> +	struct cxl_region *cxlr = cxlr_pmem->cxlr;
>> +	ssize_t rc;
>> +
>> +	ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region);
>> +	rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem);
>> +	if (rc)
>> +		return rc;
>> +
>> +	if (cxlr && cxlr->cxlr_pmem && cxlr->cxlr_pmem->nd_region) {
>> +		rc = nd_region_label_delete(cxlr->cxlr_pmem->nd_region);
>> +		if (rc)
>> +			return rc;
>> +		cxlr->params.region_label_state = 0;
>> +	}
>
>Similar to above. You can exit early and not have to indent.
>
>> +
>> +	return len;
>> +}

<snip>

>> --- a/drivers/cxl/cxl.h
>> +++ b/drivers/cxl/cxl.h
>> @@ -484,6 +484,7 @@ enum cxl_config_state {
>>   */
>>  struct cxl_region_params {
>>  	enum cxl_config_state state;
>> +	int region_label_state;
>
>Maybe a enum?
>

Thanks for you suggestion Dave, I will fix it accrodingly.


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 20/20] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

On 24/09/25 01:47PM, Dave Jiang wrote:
>
>> +++ b/drivers/cxl/core/pmem_region.c
>> @@ -290,3 +290,56 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
>>  	return rc;
>>  }
>>  EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
>> +
>> +static int match_free_ep_decoder(struct device *dev, const void *data)
>> +{
>> +	struct cxl_decoder *cxld = to_cxl_decoder(dev);
>
>I think this is needed if the function is match_free_ep_decoder().
>
>if (!is_endpoint_decoder(dev))
>	return 0;
>

Yes this check is required, I will add this.

>> +
>> +	return !cxld->region;
>> +}
>
>May want to borrow some code from match_free_decoder() in core/region.c. I think the decoder commit order matters?
>

Yes Dave, Looking at [1], seems commit order matters. Sure I will look
at match_free_decoder() in core/region.c
[1] https://lore.kernel.org/all/172964783668.81806.14962699553881333486.stgit@dwillia2-xfh.jf.intel.com/


>> +
>> +static struct cxl_decoder *cxl_find_free_ep_decoder(struct cxl_port *port)
>> +{
>> +	struct device *dev;
>> +
>> +	dev = device_find_child(&port->dev, NULL, match_free_ep_decoder);
>> +	if (!dev)
>> +		return NULL;
>> +
>> +	/* Release device ref taken via device_find_child() */
>> +	put_device(dev);
>
>Should have the caller put the device.

Its like taking device ref temporarly and releasing it then and there
after finding proper root decoder. I believe, releasing device ref
from caller would make it look little out of context.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 03/20] nvdimm/label: Modify nd_label_base() signature

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1519 bytes --]

On 24/09/25 02:02PM, Alison Schofield wrote:
>On Mon, Sep 22, 2025 at 06:14:40PM +0530, Neeraj Kumar wrote:
>> On 19/09/25 04:34PM, Dave Jiang wrote:
>> >
>> >
>> > On 9/17/25 6:40 AM, Neeraj Kumar wrote:
>> > > nd_label_base() was being used after typecasting with 'unsigned long'. Thus
>> > > modified nd_label_base() to return 'unsigned long' instead of 'struct
>> > > nd_namespace_label *'
>> > >
>> > > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>> > Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>> >
>> > Just a nit below:
>> >
>> >
>> > > ---
>> > >  drivers/nvdimm/label.c | 10 +++++-----
>> > >  1 file changed, 5 insertions(+), 5 deletions(-)
>> > >
>> > > -static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd)
>> > > +static unsigned long nd_label_base(struct nvdimm_drvdata *ndd)
>> > >  {
>> > >  	void *base = to_namespace_index(ndd, 0);
>> > >
>> > > -	return base + 2 * sizeof_namespace_index(ndd);
>> > > +	return (unsigned long) (base + 2 * sizeof_namespace_index(ndd));
>> >
>> > Space is not needed between casting and the var. Also applies to other instances in this commit.
>> >
>> > DJ
>>
>> Thanks Jonathan, Ira and Dave for RB tag. Sure, I will fix this in next
>> patch-set.
>
>This is independent of the patchset, right?
>How about just sending a one off patch for this, and shortening this
>set by a tiny bit :)
>

Yes Alison, This patch is independent and not directly related with LSA
2.1 support series. Sure, Will push it seperately.


Regards,
Neeraj



[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 08/20] nvdimm/label: Include region label in slot validation

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

On 24/09/25 02:30PM, Alison Schofield wrote:
>On Wed, Sep 17, 2025 at 07:11:04PM +0530, Neeraj Kumar wrote:
>> Slot validation routine validates label slot by calculating label
>> checksum. It was only validating namespace label. This changeset also
>> validates region label if present.
>
>Neeraj,
>
>Was it previously wrong 'only validating namespace label'
>- or -
>Is validating region label something new available with cxl
>namespaces that we now must add?
>
>I'm looking for that pattern in these commit messages, ie
>this is how it use to be, this is why it doesn't work now,
>so here is the change.
>
>-- Alison
>

Hi Alison,

Earlier labels were meant only as namespace lables. LSA 2.1 introduces
new region label because of this slots for both namespace and region
labels should be validated.

May be I will elaborate the commit message to properly communicate
this information


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 16/20] cxl/mem: Preserve cxl root decoder during mem probe

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On 24/09/25 02:38PM, Alison Schofield wrote:
>On Wed, Sep 17, 2025 at 07:11:12PM +0530, Neeraj Kumar wrote:
>> Saved root decoder info is required for cxl region persistency
>
>It seem there must be a more detailed story here.
>Saving the root decoder in struct cxl_memdev does not sound
>persistent. Please add more detail on how this step fits
>into the grander scheme.
>

Yes it has story. For region creation using cxl_create_region()
we need to know the root decoder. 

In current case cxl root decocer is provided from ndctl using device
attribute create_region_store(). And saving root decoder (cxlrd) in
cxl_memdev(cxlmd) helps to get root decoder during region creation.

Actually, cxl_memdev instance is saved in struct cxl_nvdimm(cxl_nvd). So
after parsing region information from nvdimm driver we recreate region
and thus cxlmd is available from cxl_nvd along with saved cxlrd

I will elaborate commit message with above information


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 04/20] nvdimm/label: Update mutex_lock() with guard(mutex)()

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

On 24/09/25 02:42PM, Alison Schofield wrote:
>On Wed, Sep 17, 2025 at 07:11:00PM +0530, Neeraj Kumar wrote:
>> Updated mutex_lock() with guard(mutex)()
>
>Along with the other reviewer comments to limit the application
>of guard() to where it is most useful, can this be spun off
>as a single patch that would be done irregardless on the
>new label support?
>
>I'm asking the question. I don't know the answer ;)
>

This patch is independent and not related with this series.
Its good to send this seperately.


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Dave Jiang]

On 9/29/25 6:30 AM, Neeraj Kumar wrote:
> On 23/09/25 03:37PM, Dave Jiang wrote:
>>
>>
>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
>>> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
>>> used to get called at last in cxl_endpoint_port_probe(), which requires
>>> cxl_nvd presence.
>>>
>>> For cxl region persistency, region creation happens during nvdimm_probe
>>> which need the completion of endpoint probe.
>>>
>>> In order to accommodate both cxl pmem region auto-assembly and cxl region
>>> persistency, refactored following
>>>
>>> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
>>>    will be called only after successful completion of endpoint probe.
>>>
>>> 2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
>>>    cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
>>>    completion of endpoint probe and cxl_nvd presence before its call.
>>
>> Given that we are going forward with this implementation [1] from Dan and drivers like the type2 enabling are going to be using it as well, can you please consider converting this change to Dan's mechanism instead of creating a whole new one?
>>
>> I think the region discovery can be done via the ops->probe() callback. Thanks.
>>
>> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6
>>
>> DJ
>>
> 
> Sure, Let me revisit this.
> It seems [1] is there in seperate branch "for-6.18/cxl-probe-order", and not yet merged into next, right?

Right. I believe Smita and Alejandro are using that as well. Depending on who gets there first. We can setup an immutable branch at some point.

[1]: https://lore.kernel.org/linux-cxl/20250822034202.26896-1-Smita.KoralahalliChannabasappa@amd.com/T/#t

DJ

> 
> 
> Regards,
> Neeraj
> 

Re: [PATCH V3 15/20] cxl: Add a routine to find cxl root decoder on cxl bus using cxl port

[Dave Jiang]

On 9/29/25 6:40 AM, Neeraj Kumar wrote:
> On 24/09/25 11:11AM, Dave Jiang wrote:
>>
>>> +struct cxl_root_decoder *cxl_find_root_decoder_by_port(struct cxl_port *port)
>>> +{
>>> +    struct cxl_root *cxl_root __free(put_cxl_root) = find_cxl_root(port);
>>> +    struct device *dev;
>>> +
>>> +    if (!cxl_root)
>>> +        return NULL;
>>> +
>>> +    dev = device_find_child(&cxl_root->port.dev, NULL, match_root_decoder);
>>> +    if (!dev)
>>> +        return NULL;
>>> +
>>> +    /* Release device ref taken via device_find_child() */
>>> +    put_device(dev);
>>
>> Should the caller release the device reference instead?
>>
>> DJ
> 
> Actually caller of this function wants to find root decoder information from cxl_port.
> So in order to find root decoder we have used device_find_child() which internally
> takes device ref. Therefore, just after finding the appropriate dev, I am releasing
> device ref.
> Its like taking device ref temporarly and releasing it then and there after finding
> proper root decoder.
> I believe, Releasing device ref from caller would make it look little out of context.

With the caller acquiring the root decoder, it should hold a reference until it is done with it. See usages of cxl_find_root_decoder(). i.e. core/region.c:cxl_add_to_region(). cxl_find_root_decoder_by_port() should do something similar. 

DJ
 
> 
> 
> Regards,
> Neeraj
> 

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Dave Jiang]

On 9/29/25 6:57 AM, Neeraj Kumar wrote:
> On 24/09/25 11:53AM, Dave Jiang wrote:
>>
>>
>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
>>> Moved pmem_region related code from core/region.c to core/pmem_region.c
>>> For region label update, need to create device attribute, which calls
>>> nvdimm exported function thus making pmem_region dependent on libnvdimm.
>>> Because of this dependency of pmem region on libnvdimm, segregated pmem
>>> region related code from core/region.c
>>
>> We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.
> 
> Hi Dave,
> 
> As per LSA 2.1, during region creation we need to intract with nvdimmm
> driver to write region label into LSA.
> This dependency of libnvdimm is only for PMEM region, therefore I have
> created a seperate file core/pmem_region.c and copied pmem related functions
> present in core/region.c into core/pmem_region.c.
> Because of this movemement of code we have churn introduced in this patch.
> Can you please suggest optimized way to handle dependency on libnvdimm
> with minimum code changes.

Hmm....maybe relegate the introduction of core/pmem_region.c new file and only the moving of the existing bits into the new file to a patch. And then your patch will be rid of the delete/add bits of the old code? Would that work?

DJ
 
> 
>>
>>> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
>>> index 48b7314afdb8..532eaa1bbdd6 100644
>>> --- a/drivers/cxl/Kconfig
>>> +++ b/drivers/cxl/Kconfig
>>> @@ -211,6 +211,20 @@ config CXL_REGION
>>>
>>>        If unsure say 'y'
>>>
>>> +config CXL_PMEM_REGION
>>> +    bool "CXL: Pmem Region Support"
>>> +    default CXL_BUS
>>> +    depends on CXL_REGION
>>
>>> +    depends on PHYS_ADDR_T_64BIT
>>> +    depends on BLK_DEV
>> These 2 deps are odd. What are the actual dependencies?
>>
> 
> We need to add these 2 deps to fix v2 0Day issue [1]
> I have taken reference from bdf97013ced5f [2]
> Seems, I also have to add depends on ARCH_HAS_PMEM_API. I will update it
> in V3.
> 
> [1] https://lore.kernel.org/linux-cxl/202507311017.7ApKmtQc-lkp@intel.com/
> [2] https://elixir.bootlin.com/linux/v6.13.7/source/drivers/acpi/nfit/Kconfig#L4
> 
>>
>>> +    select LIBNVDIMM
>>> +    help
>>> +      Enable the CXL core to enumerate and provision CXL pmem regions.
>>> +      A CXL pmem region need to update region label into LSA. For LSA
>>> +      updation/deletion libnvdimm is required.
>>
>> s/updation/update/
>>
> 
> Sure, Will fix it
> 
>>> +
>>> +      If unsure say 'y'
>>> +
>>>  config CXL_REGION_INVALIDATION_TEST
>>>      bool "CXL: Region Cache Management Bypass (TEST)"
>>>      depends on CXL_REGION
> 
> <snip>
> 
>>> --- a/drivers/cxl/core/port.c
>>> +++ b/drivers/cxl/core/port.c
>>> @@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
>>>          return CXL_DEVICE_NVDIMM_BRIDGE;
>>>      if (dev->type == &cxl_nvdimm_type)
>>>          return CXL_DEVICE_NVDIMM;
>>> -    if (dev->type == CXL_PMEM_REGION_TYPE())
>>> +    if (dev->type == CXL_PMEM_REGION_TYPE)
>>
>> Stray edit? I don't think anything changed in the declaration.
>>
> 
> Sure, Will fix it
> 
>>>          return CXL_DEVICE_PMEM_REGION;
>>>      if (dev->type == CXL_DAX_REGION_TYPE())
>>>          return CXL_DEVICE_DAX_REGION;
> 
> <snip>
> 
>>> @@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
>>>  }
>>>  EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
>>>
>>> -static struct cxl_region *to_cxl_region(struct device *dev)
>>> +struct cxl_region *to_cxl_region(struct device *dev)
>>>  {
>>>      if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
>>>                "not a cxl_region device\n"))
>>> @@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
>>>
>>>      return container_of(dev, struct cxl_region, dev);
>>>  }
>>> +EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");
>>
>> Maybe just move this into the header file instead.
>>
>> DJ
> 
> Actually to_cxl_region() is internal to cxl/core and especially to core/region.c
> So, Its better to compeletly remove EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL")
> 
> Even EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL") is internal to cxl/core/region.c
> Should I also remove it?
> 
> Even we can remove declaration of is_cxl_region() and to_cxl_region()
> from drivers/cxl/cxl.h as these functions are internal to cxl/core/region.c
> 
> 
> Regards,
> Neeraj
> 
> 

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Dave Jiang]

On 9/29/25 6:57 AM, Neeraj Kumar wrote:
> On 24/09/25 11:53AM, Dave Jiang wrote:
>>
>>
>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
>>> Moved pmem_region related code from core/region.c to core/pmem_region.c
>>> For region label update, need to create device attribute, which calls
>>> nvdimm exported function thus making pmem_region dependent on libnvdimm.
>>> Because of this dependency of pmem region on libnvdimm, segregated pmem
>>> region related code from core/region.c
>>
>> We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.
> 
> Hi Dave,
> 
> As per LSA 2.1, during region creation we need to intract with nvdimmm
> driver to write region label into LSA.
> This dependency of libnvdimm is only for PMEM region, therefore I have
> created a seperate file core/pmem_region.c and copied pmem related functions
> present in core/region.c into core/pmem_region.c.
> Because of this movemement of code we have churn introduced in this patch.
> Can you please suggest optimized way to handle dependency on libnvdimm
> with minimum code changes.
> 
>>
>>> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
>>> index 48b7314afdb8..532eaa1bbdd6 100644
>>> --- a/drivers/cxl/Kconfig
>>> +++ b/drivers/cxl/Kconfig
>>> @@ -211,6 +211,20 @@ config CXL_REGION
>>>
>>>        If unsure say 'y'
>>>
>>> +config CXL_PMEM_REGION
>>> +    bool "CXL: Pmem Region Support"
>>> +    default CXL_BUS
>>> +    depends on CXL_REGION
>>
>>> +    depends on PHYS_ADDR_T_64BIT
>>> +    depends on BLK_DEV
>> These 2 deps are odd. What are the actual dependencies?
>>
> 
> We need to add these 2 deps to fix v2 0Day issue [1]
> I have taken reference from bdf97013ced5f [2]
> Seems, I also have to add depends on ARCH_HAS_PMEM_API. I will update it
> in V3.
> 
> [1] https://lore.kernel.org/linux-cxl/202507311017.7ApKmtQc-lkp@intel.com/
> [2] https://elixir.bootlin.com/linux/v6.13.7/source/drivers/acpi/nfit/Kconfig#L4
> 
>>
>>> +    select LIBNVDIMM
>>> +    help
>>> +      Enable the CXL core to enumerate and provision CXL pmem regions.
>>> +      A CXL pmem region need to update region label into LSA. For LSA
>>> +      updation/deletion libnvdimm is required.
>>
>> s/updation/update/
>>
> 
> Sure, Will fix it
> 
>>> +
>>> +      If unsure say 'y'
>>> +
>>>  config CXL_REGION_INVALIDATION_TEST
>>>      bool "CXL: Region Cache Management Bypass (TEST)"
>>>      depends on CXL_REGION
> 
> <snip>
> 
>>> --- a/drivers/cxl/core/port.c
>>> +++ b/drivers/cxl/core/port.c
>>> @@ -53,7 +53,7 @@ static int cxl_device_id(const struct device *dev)
>>>          return CXL_DEVICE_NVDIMM_BRIDGE;
>>>      if (dev->type == &cxl_nvdimm_type)
>>>          return CXL_DEVICE_NVDIMM;
>>> -    if (dev->type == CXL_PMEM_REGION_TYPE())
>>> +    if (dev->type == CXL_PMEM_REGION_TYPE)
>>
>> Stray edit? I don't think anything changed in the declaration.
>>
> 
> Sure, Will fix it
> 
>>>          return CXL_DEVICE_PMEM_REGION;
>>>      if (dev->type == CXL_DAX_REGION_TYPE())
>>>          return CXL_DEVICE_DAX_REGION;
> 
> <snip>
> 
>>> @@ -2382,7 +2380,7 @@ bool is_cxl_region(struct device *dev)
>>>  }
>>>  EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL");
>>>
>>> -static struct cxl_region *to_cxl_region(struct device *dev)
>>> +struct cxl_region *to_cxl_region(struct device *dev)
>>>  {
>>>      if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type,
>>>                "not a cxl_region device\n"))
>>> @@ -2390,6 +2388,7 @@ static struct cxl_region *to_cxl_region(struct device *dev)
>>>
>>>      return container_of(dev, struct cxl_region, dev);
>>>  }
>>> +EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL");
>>
>> Maybe just move this into the header file instead.
>>
>> DJ
> 
> Actually to_cxl_region() is internal to cxl/core and especially to core/region.c
> So, Its better to compeletly remove EXPORT_SYMBOL_NS_GPL(to_cxl_region, "CXL")
> 
> Even EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL") is internal to cxl/core/region.c
> Should I also remove it?
> 
> Even we can remove declaration of is_cxl_region() and to_cxl_region()
> from drivers/cxl/cxl.h as these functions are internal to cxl/core/region.c

Yes we should probably clean that up if it's not being used externally. I don't think it should break cxl_test, but you should compile verify after you make the changes.

make M=tools/testing/cxl

> 
> 
> Regards,
> Neeraj
> 
> 

Re: [PATCH V3 20/20] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem

[Dave Jiang]

On 9/29/25 7:02 AM, Neeraj Kumar wrote:
> On 24/09/25 01:47PM, Dave Jiang wrote:
>>
>>> +++ b/drivers/cxl/core/pmem_region.c
>>> @@ -290,3 +290,56 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr)
>>>      return rc;
>>>  }
>>>  EXPORT_SYMBOL_NS_GPL(devm_cxl_add_pmem_region, "CXL");
>>> +
>>> +static int match_free_ep_decoder(struct device *dev, const void *data)
>>> +{
>>> +    struct cxl_decoder *cxld = to_cxl_decoder(dev);
>>
>> I think this is needed if the function is match_free_ep_decoder().
>>
>> if (!is_endpoint_decoder(dev))
>>     return 0;
>>
> 
> Yes this check is required, I will add this.
> 
>>> +
>>> +    return !cxld->region;
>>> +}
>>
>> May want to borrow some code from match_free_decoder() in core/region.c. I think the decoder commit order matters?
>>
> 
> Yes Dave, Looking at [1], seems commit order matters. Sure I will look
> at match_free_decoder() in core/region.c
> [1] https://lore.kernel.org/all/172964783668.81806.14962699553881333486.stgit@dwillia2-xfh.jf.intel.com/
> 
> 
>>> +
>>> +static struct cxl_decoder *cxl_find_free_ep_decoder(struct cxl_port *port)
>>> +{
>>> +    struct device *dev;
>>> +
>>> +    dev = device_find_child(&port->dev, NULL, match_free_ep_decoder);
>>> +    if (!dev)
>>> +        return NULL;
>>> +
>>> +    /* Release device ref taken via device_find_child() */
>>> +    put_device(dev);
>>
>> Should have the caller put the device.
> 
> Its like taking device ref temporarly and releasing it then and there
> after finding proper root decoder. I believe, releasing device ref
> from caller would make it look little out of context.

As mentioned in my response to 15/20, the caller should be releasing the device reference since the caller is using the endpoint decoder. I would also add a comment to the new functions acquiring the decoders that the caller is expected to put_device() on the decoder dev when done.

DJ

> 
> 
> Regards,
> Neeraj
> 

Re: [PATCH V3 06/20] nvdimm/region_label: Add region label update support

[Dave Jiang]

On 9/17/25 8:36 AM, Jonathan Cameron wrote:
> On Wed, 17 Sep 2025 19:11:02 +0530
> Neeraj Kumar <s.neeraj@samsung.com> wrote:
> 
>> Modified __pmem_label_update() to update region labels into LSA
>>
> I'm struggling to follow the use of the union for the two label types
> in much of this code.  To me if feels like that should only be a thing
> at the init_labels() point on the basis I think it's only there that
> we need to handle both in the same storage.
> 
> For the rest I'd just pay the small price of duplication that will
> occur if you just split he functions up.
> 
>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com>
>> ---
>>  drivers/nvdimm/label.c          | 269 ++++++++++++++++++++++++++------
>>  drivers/nvdimm/label.h          |  15 ++
>>  drivers/nvdimm/namespace_devs.c |  12 ++
>>  drivers/nvdimm/nd.h             |  38 ++++-
>>  include/linux/libnvdimm.h       |   8 +
>>  5 files changed, 289 insertions(+), 53 deletions(-)
>>
>> diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
>> index 182f8c9a01bf..209c73f6b7e7 100644
>> --- a/drivers/nvdimm/label.c
>> +++ b/drivers/nvdimm/label.c
>> @@ -381,6 +381,16 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd,
>>  	nsl_set_checksum(ndd, nd_label, sum);
>>  }
> 
>> +static bool is_label_reapable(struct nd_interleave_set *nd_set,
>> +			       struct nd_namespace_pmem *nspm,
>> +			       struct nvdimm_drvdata *ndd,
>> +			       union nd_lsa_label *label,
>> +			       enum label_type ltype,
>> +			       unsigned long *flags)
>> +{
>> +	switch (ltype) {
>> +	case NS_LABEL_TYPE:
>> +		if (test_and_clear_bit(ND_LABEL_REAP, flags) ||
>> +		    nsl_uuid_equal(ndd, &label->ns_label, nspm->uuid))
>> +			return true;
>> +
>> +		break;
>> +	case RG_LABEL_TYPE:
>> +		if (region_label_uuid_equal(&label->region_label,
>> +		    &nd_set->uuid))
>> +			return true;
>> +
>> +		break;
>> +	}
>> +
>> +	return false;
>> +}
>> +
>> +static int __pmem_label_update(struct nd_region *nd_region,
>> +			       struct nd_mapping *nd_mapping,
>> +			       struct nd_namespace_pmem *nspm,
>> +			       int pos, unsigned long flags,
>> +			       enum label_type ltype)
>> +{
>> +	struct nd_interleave_set *nd_set = nd_region->nd_set;
>> +	struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>> +	struct nd_namespace_index *nsindex;
>> +	struct nd_label_ent *label_ent;
>> +	union nd_lsa_label *lsa_label;
>> +	unsigned long *free;
>> +	struct device *dev;
>> +	u32 nslot, slot;
>> +	size_t offset;
>> +	int rc;
>> +
>> +	if (!preamble_next(ndd, &nsindex, &free, &nslot))
>> +		return -ENXIO;
>> +
>>  	/* allocate and write the label to the staging (next) index */
>>  	slot = nd_label_alloc_slot(ndd);
>>  	if (slot == UINT_MAX)
>>  		return -ENXIO;
>>  	dev_dbg(ndd->dev, "allocated: %d\n", slot);
>>  
>> -	nd_label = to_label(ndd, slot);
>> -	memset(nd_label, 0, sizeof_namespace_label(ndd));
>> -	nsl_set_type(ndd, nd_label);
>> -	nsl_set_uuid(ndd, nd_label, nspm->uuid);
>> -	nsl_set_name(ndd, nd_label, nspm->alt_name);
>> -	nsl_set_flags(ndd, nd_label, flags);
>> -	nsl_set_nlabel(ndd, nd_label, nd_region->ndr_mappings);
>> -	nsl_set_nrange(ndd, nd_label, 1);
>> -	nsl_set_position(ndd, nd_label, pos);
>> -	nsl_set_isetcookie(ndd, nd_label, cookie);
>> -	nsl_set_rawsize(ndd, nd_label, resource_size(res));
>> -	nsl_set_lbasize(ndd, nd_label, nspm->lbasize);
>> -	nsl_set_dpa(ndd, nd_label, res->start);
>> -	nsl_set_slot(ndd, nd_label, slot);
>> -	nsl_set_alignment(ndd, nd_label, 0);
>> -	nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid);
>> -	nsl_set_region_uuid(ndd, nd_label, NULL);
>> -	nsl_set_claim_class(ndd, nd_label, ndns->claim_class);
>> -	nsl_calculate_checksum(ndd, nd_label);
>> -	nd_dbg_dpa(nd_region, ndd, res, "\n");
>> +	lsa_label = (union nd_lsa_label *) to_label(ndd, slot);
> 
> This cast feels rather dubious.
> 
> If the union makes sense in general, then this should be changed
> to return the union.
> 
>> +	memset(lsa_label, 0, sizeof_namespace_label(ndd));
>> +
>> +	switch (ltype) {
>> +	case NS_LABEL_TYPE:
>> +		dev = &nspm->nsio.common.dev;
>> +		rc = namespace_label_update(nd_region, nd_mapping,
>> +				nspm, pos, flags, &lsa_label->ns_label,
>> +				nsindex, slot);
>> +		if (rc)
>> +			return rc;
>> +
>> +		break;
>> +	case RG_LABEL_TYPE:
>> +		dev = &nd_region->dev;
>> +		region_label_update(nd_region, &lsa_label->region_label,
>> +				    nd_mapping, pos, flags, slot);
>> +
>> +		break;
>> +	}
>>  
>>  	/* update label */
>> -	offset = nd_label_offset(ndd, nd_label);
>> -	rc = nvdimm_set_config_data(ndd, offset, nd_label,
>> +	offset = nd_label_offset(ndd, &lsa_label->ns_label);
> 
> This doesn't make sense as the type might be either an ns_label or a region_label.
> If there is a generic header (I'm guessing there is) then define that as part of the
> union with just the shared parts and use that to avoid any implication of what the type
> is in calls like this.  Also make nd_label_offset() take the union not the specific bit.
> 
>> +	rc = nvdimm_set_config_data(ndd, offset, lsa_label,
>>  			sizeof_namespace_label(ndd));
>>  	if (rc < 0)
>>  		return rc;
>> @@ -955,8 +1054,10 @@ static int __pmem_label_update(struct nd_region *nd_region,
>>  	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>>  		if (!label_ent->label)
>>  			continue;
>> -		if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) ||
>> -		    nsl_uuid_equal(ndd, label_ent->label, nspm->uuid))
>> +
>> +		if (is_label_reapable(nd_set, nspm, ndd,
>> +				      (union nd_lsa_label *) label_ent->label,
> 
> If we are going with the union that label_ent->label should be a union that
> we don't need to cast.
> 
>> +				      ltype, &label_ent->flags))
>>  			reap_victim(nd_mapping, label_ent);
>>  	}
>>  
>> @@ -966,19 +1067,20 @@ static int __pmem_label_update(struct nd_region *nd_region,
>>  	if (rc)
>>  		return rc;
>>  
>> -	list_for_each_entry(label_ent, &nd_mapping->labels, list)
>> -		if (!label_ent->label) {
>> -			label_ent->label = nd_label;
>> -			nd_label = NULL;
>> -			break;
>> -		}
>> -	dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label,
>> -			"failed to track label: %d\n",
>> -			to_slot(ndd, nd_label));
>> -	if (nd_label)
>> -		rc = -ENXIO;
>> +	list_for_each_entry(label_ent, &nd_mapping->labels, list) {
>> +		if (label_ent->label)
>> +			continue;
> 
> This flow change is unrelated to the rest here. I'd push it back
> to the simpler patch that change the locking. Make sure to call it out there
> though.  Or just don't do it and keep this patch a little more readable!
> 
>>  
>> -	return rc;
>> +		label_ent->label = &lsa_label->ns_label;
>> +		lsa_label = NULL;
>> +		break;
>> +	}
>> +	dev_WARN_ONCE(dev, lsa_label, "failed to track label: %d\n",
>> +		      to_slot(ndd, &lsa_label->ns_label));
>> +	if (lsa_label)
>> +		return -ENXIO;
>> +
>> +	return 0;
>>  }
>>  
>>  static int init_labels(struct nd_mapping *nd_mapping, int num_labels)
>> @@ -1068,6 +1170,21 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid)
>>  			nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0);
>>  }
>>  
>> +static int find_region_label_count(struct nvdimm_drvdata *ndd,
>> +				   struct nd_mapping *nd_mapping)
>> +{
>> +	struct nd_label_ent *label_ent;
>> +	int region_label_cnt = 0;
>> +
>> +	guard(mutex)(&nd_mapping->lock);
>> +	list_for_each_entry(label_ent, &nd_mapping->labels, list)
>> +		if (is_region_label(ndd,
>> +		    (union nd_lsa_label *) label_ent->label))
> 
> As above. If it's a union make label_ent->label that union type.
> 
>> +			region_label_cnt++;
>> +
>> +	return region_label_cnt;
>> +}
>> +
>>  int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>>  		struct nd_namespace_pmem *nspm, resource_size_t size)
>>  {
>> @@ -1076,6 +1193,7 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>>  		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>> +		int region_label_cnt = 0;
>>  		struct resource *res;
>>  		int count = 0;
>>  
>> @@ -1091,12 +1209,19 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>>  				count++;
>>  		WARN_ON_ONCE(!count);
>>  
>> -		rc = init_labels(nd_mapping, count);
>> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
>> +		/*
>> +		 * init_labels() scan labels and allocate new label based
>> +		 * on its second parameter (num_labels). Therefore to
>> +		 * allocate new namespace label also include previously
>> +		 * added region label
>> +		 */
>> +		rc = init_labels(nd_mapping, count + region_label_cnt);
>>  		if (rc < 0)
>>  			return rc;
>>  
>>  		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i,
>> -				NSLABEL_FLAG_UPDATING);
>> +				NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE);
> 
> The amount of shared code in __pmem_label_update() across the two types in
> the union isn't that high.  I'd be tempted to just split it for simplicity.
> 
>>  		if (rc)
>>  			return rc;
>>  	}
>> @@ -1108,7 +1233,47 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region,
>>  	for (i = 0; i < nd_region->ndr_mappings; i++) {
>>  		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>>  
>> -		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0);
>> +		rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0,
>> +				NS_LABEL_TYPE);
>> +		if (rc)
>> +			return rc;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +int nd_pmem_region_label_update(struct nd_region *nd_region)
>> +{
>> +	int i, rc;
>> +
>> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
>> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>> +		int region_label_cnt = 0;
>> +
>> +		/* No need to update region label for non cxl format */
>> +		if (!ndd->cxl)
>> +			return 0;
>> +
>> +		region_label_cnt = find_region_label_count(ndd, nd_mapping);
>> +		rc = init_labels(nd_mapping, region_label_cnt + 1);
>> +		if (rc < 0)
>> +			return rc;
>> +
>> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i,
>> +				NSLABEL_FLAG_UPDATING, RG_LABEL_TYPE);
>> +		if (rc)
>> +			return rc;
>> +	}
>> +
>> +	/* Clear the UPDATING flag per UEFI 2.7 expectations */
>> +	for (i = 0; i < nd_region->ndr_mappings; i++) {
>> +		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
>> +		struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
>> +
>> +		WARN_ON_ONCE(!ndd->cxl);
>> +		rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, 0,
>> +				RG_LABEL_TYPE);
>>  		if (rc)
>>  			return rc;
>>  	}
>> diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h
>> index 0650fb4b9821..284e2a763b49 100644
>> --- a/drivers/nvdimm/label.h
>> +++ b/drivers/nvdimm/label.h
>> @@ -30,6 +30,11 @@ enum {
>>  	ND_NSINDEX_INIT = 0x1,
>>  };
>>  
>> +enum label_type {
>> +	RG_LABEL_TYPE,
>> +	NS_LABEL_TYPE,
>> +};
>> +
>>  /**
>>   * struct nd_namespace_index - label set superblock
>>   * @sig: NAMESPACE_INDEX\0
>> @@ -183,6 +188,15 @@ struct nd_namespace_label {
>>  	};
>>  };
>>  
>> +/*
>> + * LSA 2.1 format introduces region label, which can also reside
>> + * into LSA along with only namespace label as per v1.1 and v1.2
>> + */
>> +union nd_lsa_label {
>> +	struct nd_namespace_label ns_label;
>> +	struct cxl_region_label region_label;
>> +};
> 
>> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
>> index 3271b1c8569a..559f822ef24f 100644
>> --- a/drivers/nvdimm/namespace_devs.c
>> +++ b/drivers/nvdimm/namespace_devs.c
>> @@ -232,6 +232,18 @@ static ssize_t __alt_name_store(struct device *dev, const char *buf,
>>  	return rc;
>>  }
>>  
>> +int nd_region_label_update(struct nd_region *nd_region)
>> +{
>> +	int rc;
>> +
>> +	nvdimm_bus_lock(&nd_region->dev);
>> +	rc = nd_pmem_region_label_update(nd_region);
>> +	nvdimm_bus_unlock(&nd_region->dev);
> Looks like it would be worth introducing a
> DEFINE_GUARD() for this lock.
> 
> Not necessarily in this series however.

https://lore.kernel.org/nvdimm/20250923174013.3319780-1-dave.jiang@intel.com/T/#t
Merged in nvdimm/next for 6.18.

DJ

>> +
>> +	return rc;
>> +}
>> +EXPORT_SYMBOL_GPL(nd_region_label_update);
>> +
>>  static int nd_namespace_label_update(struct nd_region *nd_region,
>>  		struct device *dev)
>>  {
>> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
>> index e362611d82cc..f04c042dcfa9 100644
>> --- a/drivers/nvdimm/nd.h
>> +++ b/drivers/nvdimm/nd.h
> 
>>  bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd,
>>  			    struct nd_namespace_label *nd_label, guid_t *guid);
>>  enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd,
>> @@ -399,7 +432,10 @@ enum nd_label_flags {
>>  struct nd_label_ent {
>>  	struct list_head list;
>>  	unsigned long flags;
>> -	struct nd_namespace_label *label;
>> +	union {
>> +		struct nd_namespace_label *label;
>> +		struct cxl_region_label *region_label;
> 
> It is a bit odd to have a union above of the two types in
> here but then union the pointers here.
> 
> I'm also failing to find where region_label is used in this patch.
> 
> 
>> +	};
>>  };
>>  
>>  enum nd_mapping_lock_class {
> _rc)
> 
> 

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 4587 bytes --]

On 06/10/25 08:55AM, Dave Jiang wrote:
>
>
>On 9/29/25 6:30 AM, Neeraj Kumar wrote:
>> On 23/09/25 03:37PM, Dave Jiang wrote:
>>>
>>>
>>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>>> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
>>>> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
>>>> used to get called at last in cxl_endpoint_port_probe(), which requires
>>>> cxl_nvd presence.
>>>>
>>>> For cxl region persistency, region creation happens during nvdimm_probe
>>>> which need the completion of endpoint probe.
>>>>
>>>> In order to accommodate both cxl pmem region auto-assembly and cxl region
>>>> persistency, refactored following
>>>>
>>>> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
>>>>    will be called only after successful completion of endpoint probe.
>>>>
>>>> 2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
>>>>    cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
>>>>    completion of endpoint probe and cxl_nvd presence before its call.
>>>
>>> Given that we are going forward with this implementation [1] from Dan and drivers like the type2 enabling are going to be using it as well, can you please consider converting this change to Dan's mechanism instead of creating a whole new one?
>>>
>>> I think the region discovery can be done via the ops->probe() callback. Thanks.
>>>
>>> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6
>>>
>>> DJ
>>>
>>
>> Sure, Let me revisit this.
>> It seems [1] is there in seperate branch "for-6.18/cxl-probe-order", and not yet merged into next, right?
>
>Right. I believe Smita and Alejandro are using that as well. Depending on who gets there first. We can setup an immutable branch at some point.
>
>[1]: https://lore.kernel.org/linux-cxl/20250822034202.26896-1-Smita.KoralahalliChannabasappa@amd.com/T/#t
>
>DJ

Hi Dave,

As per Dan's [1] newly introduced infra, Following is my understanding.

Currently cxl_pci does not care about the attach state of the cxl_memdev
because all generic memory expansion functionality can be handled by the
cxl_core. For accelerators, the driver needs to know and perform driver
specific initialization if CXL is available, or exectute a fallback to PCIe
only operation.

Dan's new infra is needed for CXL accelerator device drivers that need to
make decisions about enabling CXL dependent functionality in the device, or
falling back to PCIe-only operation.

During cxl_pci_probe() we call devm_cxl_add_memdev(struct cxl_memdev_ops *ops)
where function pointer as ops gets registered which gets called in cxl_mem_probe()
using cxlmd->ops->probe()

The probe callback runs after the port topology is successfully attached for
the given memdev.

So to use this infra we have to pass cxl_region_discovery() as ops parameter
of devm_cxl_add_memdev() getting called from cxl_pci_probe().
  
In this patch-set cxl_region_discovery() signature is different from cxlmd->ops->probe()

    {{{
	void cxl_region_discovery(struct cxl_port *port)
	{
         	device_for_each_child(&port->dev, NULL, discover_region);
	}

	struct cxl_memdev_ops {
	        int (*probe)(struct cxl_memdev *cxlmd);
	};
    }}}

Even after changing the signature of cxl_region_discovery() as per cxlmd->ops->probe()
may create problem as when the ops->probe() fails, then it will halts the probe sequence
of cxl_pci_probe()

It is because discover_region() may fail if two memdevs are participating into one region

Also, region auto assembly is mandatory functionality which creates region
if (cxled->state == CXL_DECODER_STATE_AUTO) gets satisfied.

Currently region auto assembly (added by a32320b71f085) happens after successfull
enumeration of endpoint decoders at cxl_endpoint_port_probe(), which I have moved at
cxl_mem_probe() after devm_cxl_add_nvdimm() which prepares cxl_nvd infra required by it.

As discussed in [1], this patch-set does the movement of auto region assembly from
cxl_endpoint_port_probe() to cxl_mem_probe() and resolved the conflicting dependency
of cxl_nvd infra required by both region creation using LSA and auto region assembly.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6 
[2]: https://lore.kernel.org/linux-cxl/1931444790.41752909482841.JavaMail.epsvc@epcpadp2new/

Please let me know if my understanding is correct or I am missing something?


Regards,
Neeraj


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 4385 bytes --]

On 06/10/25 09:06AM, Dave Jiang wrote:
>
>
>On 9/29/25 6:57 AM, Neeraj Kumar wrote:
>> On 24/09/25 11:53AM, Dave Jiang wrote:
>>>
>>>
>>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>>> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
>>>> Moved pmem_region related code from core/region.c to core/pmem_region.c
>>>> For region label update, need to create device attribute, which calls
>>>> nvdimm exported function thus making pmem_region dependent on libnvdimm.
>>>> Because of this dependency of pmem region on libnvdimm, segregated pmem
>>>> region related code from core/region.c
>>>
>>> We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.
>>
>> Hi Dave,
>>
>> As per LSA 2.1, during region creation we need to intract with nvdimmm
>> driver to write region label into LSA.
>> This dependency of libnvdimm is only for PMEM region, therefore I have
>> created a seperate file core/pmem_region.c and copied pmem related functions
>> present in core/region.c into core/pmem_region.c.
>> Because of this movemement of code we have churn introduced in this patch.
>> Can you please suggest optimized way to handle dependency on libnvdimm
>> with minimum code changes.
>
>Hmm....maybe relegate the introduction of core/pmem_region.c new file and only the moving of the existing bits into the new file to a patch. And then your patch will be rid of the delete/add bits of the old code? Would that work?
>
>DJ

Hi Dave,

As per LSA 2.1, during region creation we need to intract with nvdimmm
driver to write region label into LSA.

This dependency of libnvdimm is only for PMEM region, therefore I have
created a seperate file core/pmem_region.c and copied pmem related functions
present in core/region.c into core/pmem_region.c

I have moved following 7 pmem related functions from core/region.c to core/pmem_region.c
  - cxl_pmem_region_release()
  - cxl_pmem_region_alloc()
  - cxlr_release_nvdimm()
  - cxlr_pmem_unregister()
  - devm_cxl_add_pmem_region()
  - is_cxl_pmem_region()
  - to_cxl_pmem_region()

I have created region_label_update_show/store() and region_label_delete_store() which
internally calls following libnvdimm exported function
  - region_label_update_show/store()
  - region_label_delete_store()

I have added above attributes as following
    {{{
	static struct attribute *cxl_pmem_region_attrs[] = {
         	&dev_attr_region_label_update.attr,
         	&dev_attr_region_label_delete.attr,
         	NULL
	};
	static struct attribute_group cxl_pmem_region_group = {
	        .attrs = cxl_pmem_region_attrs,
	};
	static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
	        &cxl_base_attribute_group,
	        &cxl_pmem_region_group,      ------> New addition in this patch
	        NULL
	};
	const struct device_type cxl_pmem_region_type = {
	        .name = "cxl_pmem_region",
	        .release = cxl_pmem_region_release,
	        .groups = cxl_pmem_region_attribute_groups,
	};

	static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
	{
	        <snip>
	        dev = &cxlr_pmem->dev;
	        dev->parent = &cxlr->dev;
	        dev->bus = &cxl_bus_type;
	        dev->type = &cxl_pmem_region_type;
		<snip>
	}
    }}}

So I mean to say all above mentioned functions are inter-connected and dependent on libnvdimm
Keeping any of them in core/region.c to avoid churn, throws following linking error
    {{{
	ERROR: modpost: "nd_region_label_delete" [drivers/cxl/core/cxl_core.ko] undefined!
	ERROR: modpost: "nd_region_label_update" [drivers/cxl/core/cxl_core.ko] undefined!
	make[2]: *** [scripts/Makefile.modpost:147: Module.symvers] Error 1
    }}}

Keeping these functions in core/region.c (CONFIG_REGION)) and manually enabling CONFIG_LIBNVDIMM=y
will make it pass.

Even if we can put these functions in core/region.c and forcefully make
libnvdimm (CONFIG_LIBNVDIMM) dependent using Kconfig. But I find it little improper as
this new dependency is not for all type of cxl devices (vmem and pmem) but only for cxl pmem
device region.

Therefore I have seperated it out in core/pmem_region.c under Kconfig control
making libnvdimm forcefully enable if CONFIG_CXL_PMEM_REGION == y

So, I believe this prep patch is required for this LSA 2.1 support.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Dave Jiang]

On 11/7/25 5:49 AM, Neeraj Kumar wrote:
> On 06/10/25 09:06AM, Dave Jiang wrote:
>>
>>
>> On 9/29/25 6:57 AM, Neeraj Kumar wrote:
>>> On 24/09/25 11:53AM, Dave Jiang wrote:
>>>>
>>>>
>>>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>>>> Created a separate file core/pmem_region.c along with CONFIG_PMEM_REGION
>>>>> Moved pmem_region related code from core/region.c to core/pmem_region.c
>>>>> For region label update, need to create device attribute, which calls
>>>>> nvdimm exported function thus making pmem_region dependent on libnvdimm.
>>>>> Because of this dependency of pmem region on libnvdimm, segregated pmem
>>>>> region related code from core/region.c
>>>>
>>>> We can minimize the churn in this patch by introduce the new core/pmem_region.c and related bits in the beginning instead of introduce new functions and then move them over from region.c.
>>>
>>> Hi Dave,
>>>
>>> As per LSA 2.1, during region creation we need to intract with nvdimmm
>>> driver to write region label into LSA.
>>> This dependency of libnvdimm is only for PMEM region, therefore I have
>>> created a seperate file core/pmem_region.c and copied pmem related functions
>>> present in core/region.c into core/pmem_region.c.
>>> Because of this movemement of code we have churn introduced in this patch.
>>> Can you please suggest optimized way to handle dependency on libnvdimm
>>> with minimum code changes.
>>
>> Hmm....maybe relegate the introduction of core/pmem_region.c new file and only the moving of the existing bits into the new file to a patch. And then your patch will be rid of the delete/add bits of the old code? Would that work?
>>
>> DJ
> 
> Hi Dave,
> 
> As per LSA 2.1, during region creation we need to intract with nvdimmm
> driver to write region label into LSA.
> 
> This dependency of libnvdimm is only for PMEM region, therefore I have
> created a seperate file core/pmem_region.c and copied pmem related functions
> present in core/region.c into core/pmem_region.c
> 
> I have moved following 7 pmem related functions from core/region.c to core/pmem_region.c
>  - cxl_pmem_region_release()
>  - cxl_pmem_region_alloc()
>  - cxlr_release_nvdimm()
>  - cxlr_pmem_unregister()
>  - devm_cxl_add_pmem_region()
>  - is_cxl_pmem_region()
>  - to_cxl_pmem_region()
> 
> I have created region_label_update_show/store() and region_label_delete_store() which
> internally calls following libnvdimm exported function
>  - region_label_update_show/store()
>  - region_label_delete_store()
> 
> I have added above attributes as following
>    {{{
>     static struct attribute *cxl_pmem_region_attrs[] = {
>             &dev_attr_region_label_update.attr,
>             &dev_attr_region_label_delete.attr,
>             NULL
>     };
>     static struct attribute_group cxl_pmem_region_group = {
>             .attrs = cxl_pmem_region_attrs,
>     };
>     static const struct attribute_group *cxl_pmem_region_attribute_groups[] = {
>             &cxl_base_attribute_group,
>             &cxl_pmem_region_group,      ------> New addition in this patch
>             NULL
>     };
>     const struct device_type cxl_pmem_region_type = {
>             .name = "cxl_pmem_region",
>             .release = cxl_pmem_region_release,
>             .groups = cxl_pmem_region_attribute_groups,
>     };
> 
>     static int cxl_pmem_region_alloc(struct cxl_region *cxlr)
>     {
>             <snip>
>             dev = &cxlr_pmem->dev;
>             dev->parent = &cxlr->dev;
>             dev->bus = &cxl_bus_type;
>             dev->type = &cxl_pmem_region_type;
>         <snip>
>     }
>    }}}
> 
> So I mean to say all above mentioned functions are inter-connected and dependent on libnvdimm
> Keeping any of them in core/region.c to avoid churn, throws following linking error
>    {{{
>     ERROR: modpost: "nd_region_label_delete" [drivers/cxl/core/cxl_core.ko] undefined!
>     ERROR: modpost: "nd_region_label_update" [drivers/cxl/core/cxl_core.ko] undefined!
>     make[2]: *** [scripts/Makefile.modpost:147: Module.symvers] Error 1
>    }}}
> 
> Keeping these functions in core/region.c (CONFIG_REGION)) and manually enabling CONFIG_LIBNVDIMM=y
> will make it pass.
> 
> Even if we can put these functions in core/region.c and forcefully make
> libnvdimm (CONFIG_LIBNVDIMM) dependent using Kconfig. But I find it little improper as
> this new dependency is not for all type of cxl devices (vmem and pmem) but only for cxl pmem
> device region.
> 
> Therefore I have seperated it out in core/pmem_region.c under Kconfig control
> making libnvdimm forcefully enable if CONFIG_CXL_PMEM_REGION == y
> 
> So, I believe this prep patch is required for this LSA 2.1 support.

I think you misunderstood what I said. What I was trying to say is if possible to move all the diff changes of moving the existing code to a different place to a different patch. That way this patch is not full of those diff changes and make it easier to review.

DJ


> 
> 
> Regards,
> Neeraj
> 

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Dave Jiang]

On 11/7/25 5:39 AM, Neeraj Kumar wrote:
> On 06/10/25 08:55AM, Dave Jiang wrote:
>>
>>
>> On 9/29/25 6:30 AM, Neeraj Kumar wrote:
>>> On 23/09/25 03:37PM, Dave Jiang wrote:
>>>>
>>>>
>>>> On 9/17/25 6:41 AM, Neeraj Kumar wrote:
>>>>> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called
>>>>> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly
>>>>> used to get called at last in cxl_endpoint_port_probe(), which requires
>>>>> cxl_nvd presence.
>>>>>
>>>>> For cxl region persistency, region creation happens during nvdimm_probe
>>>>> which need the completion of endpoint probe.
>>>>>
>>>>> In order to accommodate both cxl pmem region auto-assembly and cxl region
>>>>> persistency, refactored following
>>>>>
>>>>> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This
>>>>>    will be called only after successful completion of endpoint probe.
>>>>>
>>>>> 2. Moved cxl pmem region auto-assembly from cxl_endpoint_port_probe() to
>>>>>    cxl_mem_probe() after devm_cxl_add_nvdimm(). It gurantees both the
>>>>>    completion of endpoint probe and cxl_nvd presence before its call.
>>>>
>>>> Given that we are going forward with this implementation [1] from Dan and drivers like the type2 enabling are going to be using it as well, can you please consider converting this change to Dan's mechanism instead of creating a whole new one?
>>>>
>>>> I think the region discovery can be done via the ops->probe() callback. Thanks.
>>>>
>>>> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6
>>>>
>>>> DJ
>>>>
>>>
>>> Sure, Let me revisit this.
>>> It seems [1] is there in seperate branch "for-6.18/cxl-probe-order", and not yet merged into next, right?
>>
>> Right. I believe Smita and Alejandro are using that as well. Depending on who gets there first. We can setup an immutable branch at some point.
>>
>> [1]: https://lore.kernel.org/linux-cxl/20250822034202.26896-1-Smita.KoralahalliChannabasappa@amd.com/T/#t
>>
>> DJ
> 
> Hi Dave,
> 
> As per Dan's [1] newly introduced infra, Following is my understanding.
> 
> Currently cxl_pci does not care about the attach state of the cxl_memdev
> because all generic memory expansion functionality can be handled by the
> cxl_core. For accelerators, the driver needs to know and perform driver
> specific initialization if CXL is available, or exectute a fallback to PCIe
> only operation.
> 
> Dan's new infra is needed for CXL accelerator device drivers that need to
> make decisions about enabling CXL dependent functionality in the device, or
> falling back to PCIe-only operation.
> 
> During cxl_pci_probe() we call devm_cxl_add_memdev(struct cxl_memdev_ops *ops)
> where function pointer as ops gets registered which gets called in cxl_mem_probe()
> using cxlmd->ops->probe()
> 
> The probe callback runs after the port topology is successfully attached for
> the given memdev.
> 
> So to use this infra we have to pass cxl_region_discovery() as ops parameter
> of devm_cxl_add_memdev() getting called from cxl_pci_probe().
>  
> In this patch-set cxl_region_discovery() signature is different from cxlmd->ops->probe()
> 
>    {{{
>     void cxl_region_discovery(struct cxl_port *port)
>     {
>             device_for_each_child(&port->dev, NULL, discover_region);
>     }
> 
>     struct cxl_memdev_ops {
>             int (*probe)(struct cxl_memdev *cxlmd);
>     };
>    }}}
> 
> Even after changing the signature of cxl_region_discovery() as per cxlmd->ops->probe()
> may create problem as when the ops->probe() fails, then it will halts the probe sequence
> of cxl_pci_probe()
> 
> It is because discover_region() may fail if two memdevs are participating into one region

While discover_region() may fail, the return value is ignored. The current code disregards failures from device_for_each_child(). And also above, cxl_region_discovery() returns void. So I don't follow how ops->probe() would fail if we ignore errors from discover_region().

DJ

> 
> Also, region auto assembly is mandatory functionality which creates region
> if (cxled->state == CXL_DECODER_STATE_AUTO) gets satisfied.
> 
> Currently region auto assembly (added by a32320b71f085) happens after successfull
> enumeration of endpoint decoders at cxl_endpoint_port_probe(), which I have moved at
> cxl_mem_probe() after devm_cxl_add_nvdimm() which prepares cxl_nvd infra required by it.
> 
> As discussed in [1], this patch-set does the movement of auto region assembly from
> cxl_endpoint_port_probe() to cxl_mem_probe() and resolved the conflicting dependency
> of cxl_nvd infra required by both region creation using LSA and auto region assembly.
> 
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?h=for-6.18/cxl-probe-order&id=88aec5ea7a24da00dc92c7778df4851fe4fd3ec6 [2]: https://lore.kernel.org/linux-cxl/1931444790.41752909482841.JavaMail.epsvc@epcpadp2new/
> 
> Please let me know if my understanding is correct or I am missing something?
> 
> 
> Regards,
> Neeraj
> 
>

Re: [PATCH V3 13/20] cxl/mem: Refactor cxl pmem region auto-assembling

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

On 12/11/25 08:55AM, Dave Jiang wrote:
>
>

<snip>

>> During cxl_pci_probe() we call devm_cxl_add_memdev(struct cxl_memdev_ops *ops)
>> where function pointer as ops gets registered which gets called in cxl_mem_probe()
>> using cxlmd->ops->probe()
>>
>> The probe callback runs after the port topology is successfully attached for
>> the given memdev.
>>
>> So to use this infra we have to pass cxl_region_discovery() as ops parameter
>> of devm_cxl_add_memdev() getting called from cxl_pci_probe().
>>  
>> In this patch-set cxl_region_discovery() signature is different from cxlmd->ops->probe()
>>
>>    {{{
>>     void cxl_region_discovery(struct cxl_port *port)
>>     {
>>             device_for_each_child(&port->dev, NULL, discover_region);
>>     }
>>
>>     struct cxl_memdev_ops {
>>             int (*probe)(struct cxl_memdev *cxlmd);
>>     };
>>    }}}
>>
>> Even after changing the signature of cxl_region_discovery() as per cxlmd->ops->probe()
>> may create problem as when the ops->probe() fails, then it will halts the probe sequence
>> of cxl_pci_probe()
>>
>> It is because discover_region() may fail if two memdevs are participating into one region
>
>While discover_region() may fail, the return value is ignored. The current code disregards failures from device_for_each_child(). And also above, cxl_region_discovery() returns void. So I don't follow how ops->probe() would fail if we ignore errors from discover_region().
>
>DJ

Hi Dave,

Yes, you are correct. We can just change signature of cxl_region_discovery() as per
cxlmd->ops->probe(), anyway we are ignoring errors from discover_region().
With this change we can directly register cxl_region_discovery() with
devm_cxl_add_memdev(struct cxl_memdev_ops *ops) during pci_probe() using Dan's Infra.

I will use this new infra for region auto-assembling and share the v4 series shortly.


Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH V3 18/20] cxl/pmem_region: Prep patch to accommodate pmem_region attributes

[Neeraj Kumar]

[-- Attachment #1: Type: text/plain, Size: 1343 bytes --]

On 12/11/25 08:40AM, Dave Jiang wrote:
>

<snip>

>> Keeping these functions in core/region.c (CONFIG_REGION)) and manually enabling CONFIG_LIBNVDIMM=y
>> will make it pass.
>>
>> Even if we can put these functions in core/region.c and forcefully make
>> libnvdimm (CONFIG_LIBNVDIMM) dependent using Kconfig. But I find it little improper as
>> this new dependency is not for all type of cxl devices (vmem and pmem) but only for cxl pmem
>> device region.
>>
>> Therefore I have seperated it out in core/pmem_region.c under Kconfig control
>> making libnvdimm forcefully enable if CONFIG_CXL_PMEM_REGION == y
>>
>> So, I believe this prep patch is required for this LSA 2.1 support.
>
>I think you misunderstood what I said. What I was trying to say is if possible to move all the diff changes of moving the existing code to a different place to a different patch. That way this patch is not full of those diff changes and make it easier to review.
>
>DJ

Hi Dave, Thanks for the clarification. Yes now I am able to get your point clear.

I should split this patch into two patches
1. First prep patch with no functionality change (Just code movement from core/region.c to newly core/pmem_region.c)
2. Second patch which calls the libnvdimm exported routines
	- region_label_update_show/store()
	- region_label_delete_store()

Regards,
Neeraj

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]