* [ndctl PATCH v4 1/9] ndctl/cxl-events: Don't fail test until event counts are reported
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 2/9] ndctl/cxl/region: Report max size for region creation Ira Weiny
` (8 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
In testing DCD event modifications a failed cxl-event test lacked
details on the event counts. This was because the greps were failing
the test rather than the check against the counts.
Suppress the grep failure and rely on event count checks for pass/fail
of the test.
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
test/cxl-events.sh | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/test/cxl-events.sh b/test/cxl-events.sh
index ff4f3fdff1d8f6fd80f093126a27bf14b52d167f..c216d6aa9148c938a649cb22656127b3df440039 100644
--- a/test/cxl-events.sh
+++ b/test/cxl-events.sh
@@ -71,10 +71,10 @@ echo 0 > /sys/kernel/tracing/tracing_on
echo "TEST: Events seen"
trace_out=$(cat /sys/kernel/tracing/trace)
-num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}")
-num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}")
-num_failure=$(grep -c "log=Failure" <<< "${trace_out}")
-num_info=$(grep -c "log=Informational" <<< "${trace_out}")
+num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}" || true)
+num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}" || true)
+num_failure=$(grep -c "log=Failure" <<< "${trace_out}" || true)
+num_info=$(grep -c "log=Informational" <<< "${trace_out}" || true)
echo " LOG (Expected) : (Found)"
echo " overflow ($num_overflow_expected) : $num_overflow"
echo " Fatal ($num_fatal_expected) : $num_fatal"
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [ndctl PATCH v4 2/9] ndctl/cxl/region: Report max size for region creation
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 1/9] ndctl/cxl-events: Don't fail test until event counts are reported Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 3/9] libcxl: Separate region mode from decoder mode Ira Weiny
` (7 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
When creating a region if the size exceeds the max an error is printed.
However, the max available space is not reported which makes it harder
to determine what is wrong.
Add the max size available to the output error.
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fan Ni <fan.ni@samsung.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
cxl/region.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/cxl/region.c b/cxl/region.c
index 96aa5931d2281c7577679b7f6165218964fa0425..207cf2d003148992255c715f286bc0f38de2ca84 100644
--- a/cxl/region.c
+++ b/cxl/region.c
@@ -677,8 +677,8 @@ static int create_region(struct cxl_ctx *ctx, int *count,
}
if (!default_size && size > max_extent) {
log_err(&rl,
- "%s: region size %#lx exceeds max available space\n",
- cxl_decoder_get_devname(p->root_decoder), size);
+ "%s: region size %#lx exceeds max available space (%#lx)\n",
+ cxl_decoder_get_devname(p->root_decoder), size, max_extent);
return -ENOSPC;
}
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [ndctl PATCH v4 3/9] libcxl: Separate region mode from decoder mode
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 1/9] ndctl/cxl-events: Don't fail test until event counts are reported Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 2/9] ndctl/cxl/region: Report max size for region creation Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 4/9] cxl/region: Use new region mode in cxl-cli Ira Weiny
` (6 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
With the introduction of DCD, region mode and decoder mode no longer
remain a 1:1 relation. An interleaved region may be made up of Dynamic
Capacity partitions with different indexes on each of the target
devices.
Introduce a new region mode enumeration and access function to libcxl.
To maintain compatibility with existing software the region mode values
are defined the same as the current decoder mode. In addition
cxl_region_get_mode() is retained.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Documentation/cxl/lib/libcxl.txt | 20 ++++++++++++++++++--
cxl/lib/libcxl.c | 25 +++++++++++++++----------
cxl/lib/libcxl.sym | 5 +++++
cxl/lib/private.h | 2 +-
cxl/libcxl.h | 35 +++++++++++++++++++++++++++++++++++
5 files changed, 74 insertions(+), 13 deletions(-)
diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt
index 40598a08b9f4840f79e3ab43f62f412d8b2136ed..d5c3558aacecb08d7f5754fdcc77d6e743560601 100644
--- a/Documentation/cxl/lib/libcxl.txt
+++ b/Documentation/cxl/lib/libcxl.txt
@@ -553,11 +553,20 @@ struct cxl_region *cxl_region_get_next(struct cxl_region *region);
===== REGION: Attributes
----
+enum cxl_region_mode {
+ CXL_REGION_MODE_NONE = CXL_DECODER_MODE_NONE,
+ CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED,
+ CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM,
+ CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM,
+};
+const char *cxl_region_mode_name(enum cxl_region_mode mode);
+enum cxl_region_mode cxl_region_mode_from_ident(const char *ident);
+enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region);
+
int cxl_region_get_id(struct cxl_region *region);
const char *cxl_region_get_devname(struct cxl_region *region);
void cxl_region_get_uuid(struct cxl_region *region, uuid_t uu);
unsigned long long cxl_region_get_size(struct cxl_region *region);
-enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region);
unsigned long long cxl_region_get_resource(struct cxl_region *region);
unsigned int cxl_region_get_interleave_ways(struct cxl_region *region);
unsigned int cxl_region_get_interleave_granularity(struct cxl_region *region);
@@ -576,8 +585,12 @@ int cxl_region_clear_all_targets(struct cxl_region *region);
int cxl_region_decode_commit(struct cxl_region *region);
int cxl_region_decode_reset(struct cxl_region *region);
struct daxctl_region *cxl_region_get_daxctl_region(struct cxl_region *region);
-----
+DEPRECATED:
+
+enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region);
+
+----
A region's resource attribute is the Host Physical Address at which the region's
address space starts. The region's address space is a subset of the parent root
decoder's address space.
@@ -601,6 +614,9 @@ where its properties can be interrogated by daxctl. The helper
cxl_region_get_daxctl_region() returns an 'struct daxctl_region *' that
can be used with other libdaxctl APIs.
+Regions now have a mode distinct from decoders. cxl_region_get_mode() is
+deprecated in favor of cxl_region_get_region_mode().
+
include::../../copyright.txt[]
SEE ALSO
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 63aa4ef3acdc2fb3c4ec6c13be5feb802e817d0d..35a40091e8f5813c1b3ef2ffb931c9ec584b02ad 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -431,10 +431,10 @@ CXL_EXPORT bool cxl_region_qos_class_mismatch(struct cxl_region *region)
if (!memdev)
continue;
- if (region->mode == CXL_DECODER_MODE_RAM) {
+ if (region->mode == CXL_REGION_MODE_RAM) {
if (root_decoder->qos_class != memdev->ram_qos_class)
return true;
- } else if (region->mode == CXL_DECODER_MODE_PMEM) {
+ } else if (region->mode == CXL_REGION_MODE_PMEM) {
if (root_decoder->qos_class != memdev->pmem_qos_class)
return true;
}
@@ -619,9 +619,9 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base)
sprintf(path, "%s/mode", cxlregion_base);
if (sysfs_read_attr(ctx, path, buf) < 0)
- region->mode = CXL_DECODER_MODE_NONE;
+ region->mode = CXL_REGION_MODE_NONE;
else
- region->mode = cxl_decoder_mode_from_ident(buf);
+ region->mode = cxl_region_mode_from_ident(buf);
sprintf(path, "%s/modalias", cxlregion_base);
if (sysfs_read_attr(ctx, path, buf) == 0)
@@ -748,11 +748,16 @@ CXL_EXPORT unsigned long long cxl_region_get_resource(struct cxl_region *region)
return region->start;
}
-CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region)
+CXL_EXPORT enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region)
{
return region->mode;
}
+CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region)
+{
+ return (enum cxl_decoder_mode)cxl_region_get_region_mode(region);
+}
+
CXL_EXPORT unsigned int
cxl_region_get_interleave_ways(struct cxl_region *region)
{
@@ -2700,7 +2705,7 @@ cxl_decoder_get_region(struct cxl_decoder *decoder)
}
static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder,
- enum cxl_decoder_mode mode)
+ enum cxl_region_mode mode)
{
struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder);
char *path = decoder->dev_buf;
@@ -2708,9 +2713,9 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder,
struct cxl_region *region;
int rc;
- if (mode == CXL_DECODER_MODE_PMEM)
+ if (mode == CXL_REGION_MODE_PMEM)
sprintf(path, "%s/create_pmem_region", decoder->dev_path);
- else if (mode == CXL_DECODER_MODE_RAM)
+ else if (mode == CXL_REGION_MODE_RAM)
sprintf(path, "%s/create_ram_region", decoder->dev_path);
rc = sysfs_read_attr(ctx, path, buf);
@@ -2754,13 +2759,13 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder,
CXL_EXPORT struct cxl_region *
cxl_decoder_create_pmem_region(struct cxl_decoder *decoder)
{
- return cxl_decoder_create_region(decoder, CXL_DECODER_MODE_PMEM);
+ return cxl_decoder_create_region(decoder, CXL_REGION_MODE_PMEM);
}
CXL_EXPORT struct cxl_region *
cxl_decoder_create_ram_region(struct cxl_decoder *decoder)
{
- return cxl_decoder_create_region(decoder, CXL_DECODER_MODE_RAM);
+ return cxl_decoder_create_region(decoder, CXL_REGION_MODE_RAM);
}
CXL_EXPORT int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder)
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 0c155a40ad4765106f0eab1745281d462af782fe..17a660f508ad1e053af2992824535ccf7ce877b2 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -288,3 +288,8 @@ global:
cxl_memdev_trigger_poison_list;
cxl_region_trigger_poison_list;
} LIBCXL_7;
+
+LIBCXL_9 {
+global:
+ cxl_region_get_region_mode;
+} LIBECXL_8;
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index b6cd910e93359b53cac34427acfe84c7abcb78b0..0f45be89b6a00477d13fb6d7f1906213a3073c48 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -171,7 +171,7 @@ struct cxl_region {
unsigned int interleave_ways;
unsigned int interleave_granularity;
enum cxl_decode_state decode_state;
- enum cxl_decoder_mode mode;
+ enum cxl_region_mode mode;
struct daxctl_region *dax_region;
struct kmod_module *module;
struct list_head mappings;
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 0a5fd0e13cc24e0032d4a83d780278fbe0038d32..06b87a0924faafec6c80eca83ea7551d4e117256 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -303,6 +303,39 @@ int cxl_memdev_is_enabled(struct cxl_memdev *memdev);
for (endpoint = cxl_endpoint_get_first(port); endpoint != NULL; \
endpoint = cxl_endpoint_get_next(endpoint))
+enum cxl_region_mode {
+ CXL_REGION_MODE_NONE = CXL_DECODER_MODE_NONE,
+ CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED,
+ CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM,
+ CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM,
+};
+
+static inline const char *cxl_region_mode_name(enum cxl_region_mode mode)
+{
+ static const char *names[] = {
+ [CXL_REGION_MODE_NONE] = "none",
+ [CXL_REGION_MODE_MIXED] = "mixed",
+ [CXL_REGION_MODE_PMEM] = "pmem",
+ [CXL_REGION_MODE_RAM] = "ram",
+ };
+
+ if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM)
+ mode = CXL_REGION_MODE_NONE;
+ return names[mode];
+}
+
+static inline enum cxl_region_mode
+cxl_region_mode_from_ident(const char *ident)
+{
+ if (strcmp(ident, "ram") == 0)
+ return CXL_REGION_MODE_RAM;
+ else if (strcmp(ident, "volatile") == 0)
+ return CXL_REGION_MODE_RAM;
+ else if (strcmp(ident, "pmem") == 0)
+ return CXL_REGION_MODE_PMEM;
+ return CXL_REGION_MODE_NONE;
+}
+
struct cxl_region;
struct cxl_region *cxl_region_get_first(struct cxl_decoder *decoder);
struct cxl_region *cxl_region_get_next(struct cxl_region *region);
@@ -318,6 +351,8 @@ const char *cxl_region_get_devname(struct cxl_region *region);
void cxl_region_get_uuid(struct cxl_region *region, uuid_t uu);
unsigned long long cxl_region_get_size(struct cxl_region *region);
unsigned long long cxl_region_get_resource(struct cxl_region *region);
+enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region);
+/* Deprecated: use cxl_region_get_region_mode() */
enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region);
unsigned int cxl_region_get_interleave_ways(struct cxl_region *region);
unsigned int cxl_region_get_interleave_granularity(struct cxl_region *region);
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [ndctl PATCH v4 4/9] cxl/region: Use new region mode in cxl-cli
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (2 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 3/9] libcxl: Separate region mode from decoder mode Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support Ira Weiny
` (5 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
With the introduction of DCD, region mode and decoder mode no longer
remain a 1:1 relation. An interleaved region may be made up of Dynamic
Capacity partitions with different indexes on each of the target
devices.
Modify cxl-cli to use the new region mode interface from libcxl. Modify
parameter processing and variable name changes for clarity in the
future.
Functionality remains the same.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
cxl/json.c | 6 +++---
cxl/region.c | 50 ++++++++++++++++++++++++++++++++------------------
2 files changed, 35 insertions(+), 21 deletions(-)
diff --git a/cxl/json.c b/cxl/json.c
index 5066d3bed13f8fcc36ab8f0ea127685c246d94d7..dcd3cc28393faf7e8adf299a857531ecdeaac50a 100644
--- a/cxl/json.c
+++ b/cxl/json.c
@@ -1147,7 +1147,7 @@ void util_cxl_mappings_append_json(struct json_object *jregion,
struct json_object *util_cxl_region_to_json(struct cxl_region *region,
unsigned long flags)
{
- enum cxl_decoder_mode mode = cxl_region_get_mode(region);
+ enum cxl_region_mode mode = cxl_region_get_region_mode(region);
const char *devname = cxl_region_get_devname(region);
struct json_object *jregion, *jobj;
u64 val;
@@ -1174,8 +1174,8 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region,
json_object_object_add(jregion, "size", jobj);
}
- if (mode != CXL_DECODER_MODE_NONE) {
- jobj = json_object_new_string(cxl_decoder_mode_name(mode));
+ if (mode != CXL_REGION_MODE_NONE) {
+ jobj = json_object_new_string(cxl_region_mode_name(mode));
if (jobj)
json_object_object_add(jregion, "type", jobj);
}
diff --git a/cxl/region.c b/cxl/region.c
index 207cf2d003148992255c715f286bc0f38de2ca84..527bd6708b162815068a95ddb360fce3914347de 100644
--- a/cxl/region.c
+++ b/cxl/region.c
@@ -49,7 +49,8 @@ struct parsed_params {
int argc;
const char **argv;
struct cxl_decoder *root_decoder;
- enum cxl_decoder_mode mode;
+ enum cxl_decoder_mode decoder_mode;
+ enum cxl_region_mode region_mode;
bool enforce_qos;
};
@@ -301,19 +302,28 @@ static int parse_create_options(struct cxl_ctx *ctx, int count,
return -ENXIO;
p->num_memdevs = json_object_array_length(p->memdevs);
+ p->region_mode = CXL_REGION_MODE_NONE;
if (param.type) {
- p->mode = cxl_decoder_mode_from_ident(param.type);
- if (p->mode == CXL_DECODER_MODE_RAM && param.uuid) {
+ p->region_mode = cxl_region_mode_from_ident(param.type);
+ if (p->region_mode == CXL_REGION_MODE_RAM && param.uuid) {
log_err(&rl,
"can't set UUID for ram / volatile regions");
goto err;
}
- if (p->mode == CXL_DECODER_MODE_NONE) {
+ if (p->region_mode == CXL_REGION_MODE_NONE) {
log_err(&rl, "unsupported type: %s\n", param.type);
goto err;
}
- } else {
- p->mode = CXL_DECODER_MODE_PMEM;
+ }
+
+ switch (p->region_mode) {
+ case CXL_REGION_MODE_RAM:
+ p->decoder_mode = CXL_DECODER_MODE_RAM;
+ break;
+ case CXL_REGION_MODE_PMEM:
+ default:
+ p->decoder_mode = CXL_DECODER_MODE_PMEM;
+ break;
}
if (param.size) {
@@ -410,7 +420,7 @@ static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p)
struct cxl_memdev *memdev = json_object_get_userdata(jobj);
u64 size = 0;
- switch(p->mode) {
+ switch(p->decoder_mode) {
case CXL_DECODER_MODE_RAM:
size = cxl_memdev_get_ram_size(memdev);
break;
@@ -446,7 +456,7 @@ static int create_region_validate_qos_class(struct parsed_params *p)
json_object_array_get_idx(p->memdevs, i);
struct cxl_memdev *memdev = json_object_get_userdata(jobj);
- if (p->mode == CXL_DECODER_MODE_RAM)
+ if (p->decoder_mode == CXL_DECODER_MODE_RAM)
qos_class = cxl_memdev_get_ram_qos_class(memdev);
else
qos_class = cxl_memdev_get_pmem_qos_class(memdev);
@@ -475,7 +485,7 @@ static int validate_decoder(struct cxl_decoder *decoder,
const char *devname = cxl_decoder_get_devname(decoder);
int rc;
- switch(p->mode) {
+ switch(p->decoder_mode) {
case CXL_DECODER_MODE_RAM:
if (!cxl_decoder_is_volatile_capable(decoder)) {
log_err(&rl, "%s is not volatile capable\n", devname);
@@ -512,10 +522,14 @@ static void set_type_from_decoder(struct cxl_ctx *ctx, struct parsed_params *p)
* default to pmem if both types are set, otherwise the single
* capability dominates.
*/
- if (cxl_decoder_is_volatile_capable(p->root_decoder))
- p->mode = CXL_DECODER_MODE_RAM;
- if (cxl_decoder_is_pmem_capable(p->root_decoder))
- p->mode = CXL_DECODER_MODE_PMEM;
+ if (cxl_decoder_is_volatile_capable(p->root_decoder)) {
+ p->decoder_mode = CXL_DECODER_MODE_RAM;
+ p->region_mode = CXL_REGION_MODE_RAM;
+ }
+ if (cxl_decoder_is_pmem_capable(p->root_decoder)) {
+ p->decoder_mode = CXL_DECODER_MODE_PMEM;
+ p->region_mode = CXL_REGION_MODE_PMEM;
+ }
}
static int create_region_validate_config(struct cxl_ctx *ctx,
@@ -685,14 +699,14 @@ static int create_region(struct cxl_ctx *ctx, int *count,
if (size > max_extent)
size = ALIGN_DOWN(max_extent, SZ_256M * p->ways);
- if (p->mode == CXL_DECODER_MODE_PMEM) {
+ if (p->region_mode == CXL_REGION_MODE_PMEM) {
region = cxl_decoder_create_pmem_region(p->root_decoder);
if (!region) {
log_err(&rl, "failed to create region under %s\n",
param.root_decoder);
return -ENXIO;
}
- } else if (p->mode == CXL_DECODER_MODE_RAM) {
+ } else if (p->region_mode == CXL_REGION_MODE_RAM) {
region = cxl_decoder_create_ram_region(p->root_decoder);
if (!region) {
log_err(&rl, "failed to create region under %s\n",
@@ -714,7 +728,7 @@ static int create_region(struct cxl_ctx *ctx, int *count,
try(cxl_region, set_interleave_granularity, region, granularity);
try(cxl_region, set_interleave_ways, region, p->ways);
- if (p->mode == CXL_DECODER_MODE_PMEM) {
+ if (p->region_mode == CXL_REGION_MODE_PMEM) {
if (!param.uuid)
uuid_generate(p->uuid);
try(cxl_region, set_uuid, region, p->uuid);
@@ -732,14 +746,14 @@ static int create_region(struct cxl_ctx *ctx, int *count,
rc = -ENXIO;
goto out;
}
- if (cxl_decoder_get_mode(ep_decoder) != p->mode) {
+ if (cxl_decoder_get_mode(ep_decoder) != p->decoder_mode) {
/*
* The cxl_memdev_find_decoder() helper returns a free
* decoder whose size has been checked for 0.
* Thus it is safe to change the mode here if needed.
*/
try(cxl_decoder, set_dpa_size, ep_decoder, 0);
- try(cxl_decoder, set_mode, ep_decoder, p->mode);
+ try(cxl_decoder, set_mode, ep_decoder, p->decoder_mode);
}
try(cxl_decoder, set_dpa_size, ep_decoder, size/p->ways);
rc = cxl_region_set_target(region, i, ep_decoder);
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (3 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 4/9] cxl/region: Use new region mode in cxl-cli Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2025-02-11 3:12 ` Alison Schofield
2024-12-15 2:58 ` [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions Ira Weiny
` (4 subsequent siblings)
9 siblings, 1 reply; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity
with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be
sparse and defined as dynamic capacity (dc).
Add support for DCD devices and regions to libcxl. Add documentation
for the new interfaces.
Based on an original patch from Navneet Singh.
Signed-off-by: Sushant1 Kumar <sushant1.kumar@intel.com>
Co-developed-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Documentation/cxl/lib/libcxl.txt | 17 ++++++-
cxl/lib/libcxl.c | 98 ++++++++++++++++++++++++++++++++++++++++
cxl/lib/libcxl.sym | 3 ++
cxl/lib/private.h | 4 ++
cxl/libcxl.h | 52 ++++++++++++++++++++-
5 files changed, 171 insertions(+), 3 deletions(-)
diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt
index d5c3558aacecb08d7f5754fdcc77d6e743560601..abca08fc81e6e84d176facafad6decae2f875880 100644
--- a/Documentation/cxl/lib/libcxl.txt
+++ b/Documentation/cxl/lib/libcxl.txt
@@ -68,6 +68,7 @@ int cxl_memdev_get_major(struct cxl_memdev *memdev);
int cxl_memdev_get_minor(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, enum cxl_decoder_mode mode);
const char *cxl_memdev_get_firmware_version(struct cxl_memdev *memdev);
size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev);
int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev);
@@ -422,11 +423,23 @@ enum cxl_decoder_mode {
CXL_DECODER_MODE_MIXED,
CXL_DECODER_MODE_PMEM,
CXL_DECODER_MODE_RAM,
+ CXL_DECODER_MODE_DC0,
+ CXL_DECODER_MODE_DC1,
+ CXL_DECODER_MODE_DC2,
+ CXL_DECODER_MODE_DC3,
+ CXL_DECODER_MODE_DC4,
+ CXL_DECODER_MODE_DC5,
+ CXL_DECODER_MODE_DC6,
+ CXL_DECODER_MODE_DC7,
};
enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder);
+const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode);
+enum cxl_decoder_mode cxl_decoder_mode_from_ident(const char *ident);
+bool cxl_decoder_mode_is_dc(enum cxl_decoder_mode mode);
int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode);
bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder);
+bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, enum cxl_decoder_mode mode);
bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder);
bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder);
bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder);
@@ -558,6 +571,7 @@ enum cxl_region_mode {
CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED,
CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM,
CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM,
+ CXL_REGION_MODE_DC,
};
const char *cxl_region_mode_name(enum cxl_region_mode mode);
enum cxl_region_mode cxl_region_mode_from_ident(const char *ident);
@@ -615,7 +629,8 @@ cxl_region_get_daxctl_region() returns an 'struct daxctl_region *' that
can be used with other libdaxctl APIs.
Regions now have a mode distinct from decoders. cxl_region_get_mode() is
-deprecated in favor of cxl_region_get_region_mode().
+deprecated in favor of cxl_region_get_region_mode(). Dynamic capacity regions
+require the use of cxl_region_get_region_mode().
include::../../copyright.txt[]
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index 35a40091e8f5813c1b3ef2ffb931c9ec584b02ad..df250db9dbacb2f0f34e8a592ce194159584fe4f 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -1304,6 +1304,19 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base)
else
memdev->ram_qos_class = atoi(buf);
+ memdev->dc_partition_count = 0;
+ for (int partition = 0; partition < MAX_NUM_DC_REGIONS; partition++) {
+ sprintf(path, "%s/dc%d/size", cxlmem_base, partition);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ continue;
+ memdev->dc_size[partition] = strtoull(buf, NULL, 0);
+ memdev->dc_partition_count++;
+ sprintf(path, "%s/dc%d/qos_class", cxlmem_base, partition);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ continue;
+ memdev->dc_qos_class[partition] = strtoull(buf, NULL, 0);
+ }
+
sprintf(path, "%s/payload_max", cxlmem_base);
if (sysfs_read_attr(ctx, path, buf) == 0) {
memdev->payload_max = strtoull(buf, NULL, 0);
@@ -1540,6 +1553,23 @@ CXL_EXPORT int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev)
return memdev->ram_qos_class;
}
+static int cxl_decoder_dc_mode_to_index(enum cxl_decoder_mode mode)
+{
+ if (mode < CXL_DECODER_MODE_DC0 || mode > CXL_DECODER_MODE_DC7)
+ return -EINVAL;
+ return mode - CXL_DECODER_MODE_DC0;
+}
+
+CXL_EXPORT unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev,
+ enum cxl_decoder_mode mode)
+{
+ int index = cxl_decoder_dc_mode_to_index(mode);
+
+ if (index < 0)
+ return 0;
+ return memdev->dc_size[index];
+}
+
CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev)
{
return memdev->firmware_version;
@@ -2275,6 +2305,22 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base)
decoder->mode = CXL_DECODER_MODE_RAM;
else if (strcmp(buf, "pmem") == 0)
decoder->mode = CXL_DECODER_MODE_PMEM;
+ else if (strcmp(buf, "dc0") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC0;
+ else if (strcmp(buf, "dc1") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC1;
+ else if (strcmp(buf, "dc2") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC2;
+ else if (strcmp(buf, "dc3") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC3;
+ else if (strcmp(buf, "dc4") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC4;
+ else if (strcmp(buf, "dc5") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC5;
+ else if (strcmp(buf, "dc6") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC6;
+ else if (strcmp(buf, "dc7") == 0)
+ decoder->mode = CXL_DECODER_MODE_DC7;
else if (strcmp(buf, "mixed") == 0)
decoder->mode = CXL_DECODER_MODE_MIXED;
else if (strcmp(buf, "none") == 0)
@@ -2318,6 +2364,8 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base)
case CXL_PORT_SWITCH:
decoder->pmem_capable = true;
decoder->volatile_capable = true;
+ for (unsigned index = 0; index < MAX_NUM_DC_REGIONS; index++)
+ decoder->dc_capable[index] = true;
decoder->mem_capable = true;
decoder->accelmem_capable = true;
sprintf(path, "%s/locked", cxldecoder_base);
@@ -2341,6 +2389,14 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base)
{ "cap_type2", &decoder->accelmem_capable },
{ "cap_type3", &decoder->mem_capable },
{ "cap_ram", &decoder->volatile_capable },
+ { "cap_ram", &decoder->dc_capable[0] },
+ { "cap_ram", &decoder->dc_capable[1] },
+ { "cap_ram", &decoder->dc_capable[2] },
+ { "cap_ram", &decoder->dc_capable[3] },
+ { "cap_ram", &decoder->dc_capable[4] },
+ { "cap_ram", &decoder->dc_capable[5] },
+ { "cap_ram", &decoder->dc_capable[6] },
+ { "cap_ram", &decoder->dc_capable[7] },
{ "cap_pmem", &decoder->pmem_capable },
{ "locked", &decoder->locked },
};
@@ -2592,6 +2648,30 @@ CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder,
case CXL_DECODER_MODE_RAM:
sprintf(buf, "ram");
break;
+ case CXL_DECODER_MODE_DC0:
+ sprintf(buf, "dc0");
+ break;
+ case CXL_DECODER_MODE_DC1:
+ sprintf(buf, "dc1");
+ break;
+ case CXL_DECODER_MODE_DC2:
+ sprintf(buf, "dc2");
+ break;
+ case CXL_DECODER_MODE_DC3:
+ sprintf(buf, "dc3");
+ break;
+ case CXL_DECODER_MODE_DC4:
+ sprintf(buf, "dc4");
+ break;
+ case CXL_DECODER_MODE_DC5:
+ sprintf(buf, "dc5");
+ break;
+ case CXL_DECODER_MODE_DC6:
+ sprintf(buf, "dc6");
+ break;
+ case CXL_DECODER_MODE_DC7:
+ sprintf(buf, "dc7");
+ break;
default:
err(ctx, "%s: unsupported mode: %d\n",
cxl_decoder_get_devname(decoder), mode);
@@ -2648,6 +2728,16 @@ CXL_EXPORT bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder)
return decoder->mem_capable;
}
+CXL_EXPORT bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder,
+ enum cxl_decoder_mode mode)
+{
+ int index = cxl_decoder_dc_mode_to_index(mode);
+
+ if (index < 0)
+ return false;
+ return decoder->dc_capable[index];
+}
+
CXL_EXPORT bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder)
{
return decoder->accelmem_capable;
@@ -2717,6 +2807,8 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder,
sprintf(path, "%s/create_pmem_region", decoder->dev_path);
else if (mode == CXL_REGION_MODE_RAM)
sprintf(path, "%s/create_ram_region", decoder->dev_path);
+ else if (mode == CXL_REGION_MODE_DC)
+ sprintf(path, "%s/create_dc_region", decoder->dev_path);
rc = sysfs_read_attr(ctx, path, buf);
if (rc < 0) {
@@ -2768,6 +2860,12 @@ cxl_decoder_create_ram_region(struct cxl_decoder *decoder)
return cxl_decoder_create_region(decoder, CXL_REGION_MODE_RAM);
}
+CXL_EXPORT struct cxl_region *
+cxl_decoder_create_dc_region(struct cxl_decoder *decoder)
+{
+ return cxl_decoder_create_region(decoder, CXL_REGION_MODE_DC);
+}
+
CXL_EXPORT int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder)
{
return decoder->nr_targets;
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index 17a660f508ad1e053af2992824535ccf7ce877b2..fdb227789985443a13c72751bbd42ab383db5f97 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -292,4 +292,7 @@ global:
LIBCXL_9 {
global:
cxl_region_get_region_mode;
+ cxl_memdev_get_dc_size;
+ cxl_decoder_is_dc_capable;
+ cxl_decoder_create_dc_region;
} LIBECXL_8;
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 0f45be89b6a00477d13fb6d7f1906213a3073c48..3efa230bfb632e6c6048aadd18f799b07d4bdfd3 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -47,6 +47,9 @@ struct cxl_memdev {
struct list_node list;
unsigned long long pmem_size;
unsigned long long ram_size;
+ unsigned long long dc_size[MAX_NUM_DC_REGIONS];
+ unsigned long long dc_qos_class[MAX_NUM_DC_REGIONS];
+ int dc_partition_count;
int ram_qos_class;
int pmem_qos_class;
int payload_max;
@@ -140,6 +143,7 @@ struct cxl_decoder {
bool pmem_capable;
bool volatile_capable;
bool mem_capable;
+ bool dc_capable[MAX_NUM_DC_REGIONS];
bool accelmem_capable;
bool locked;
enum cxl_decoder_target_type target_type;
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index 06b87a0924faafec6c80eca83ea7551d4e117256..d7f8a37816f236acd71fc834eae70a7a17a2721a 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -72,6 +72,9 @@ int cxl_memdev_get_minor(struct cxl_memdev *memdev);
struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev);
unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev);
+enum cxl_decoder_mode;
+unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev,
+ enum cxl_decoder_mode mode);
int cxl_memdev_get_pmem_qos_class(struct cxl_memdev *memdev);
int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev);
const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev);
@@ -191,11 +194,20 @@ unsigned long long
cxl_decoder_get_max_available_extent(struct cxl_decoder *decoder);
int cxl_root_decoder_get_qos_class(struct cxl_decoder *decoder);
+#define MAX_NUM_DC_REGIONS 8
enum cxl_decoder_mode {
CXL_DECODER_MODE_NONE,
CXL_DECODER_MODE_MIXED,
CXL_DECODER_MODE_PMEM,
CXL_DECODER_MODE_RAM,
+ CXL_DECODER_MODE_DC0,
+ CXL_DECODER_MODE_DC1,
+ CXL_DECODER_MODE_DC2,
+ CXL_DECODER_MODE_DC3,
+ CXL_DECODER_MODE_DC4,
+ CXL_DECODER_MODE_DC5,
+ CXL_DECODER_MODE_DC6,
+ CXL_DECODER_MODE_DC7,
};
static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode)
@@ -205,9 +217,17 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode)
[CXL_DECODER_MODE_MIXED] = "mixed",
[CXL_DECODER_MODE_PMEM] = "pmem",
[CXL_DECODER_MODE_RAM] = "ram",
+ [CXL_DECODER_MODE_DC0] = "dc0",
+ [CXL_DECODER_MODE_DC1] = "dc1",
+ [CXL_DECODER_MODE_DC2] = "dc2",
+ [CXL_DECODER_MODE_DC3] = "dc3",
+ [CXL_DECODER_MODE_DC4] = "dc4",
+ [CXL_DECODER_MODE_DC5] = "dc5",
+ [CXL_DECODER_MODE_DC6] = "dc6",
+ [CXL_DECODER_MODE_DC7] = "dc7",
};
- if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_RAM)
+ if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_DC7)
mode = CXL_DECODER_MODE_NONE;
return names[mode];
}
@@ -221,9 +241,30 @@ cxl_decoder_mode_from_ident(const char *ident)
return CXL_DECODER_MODE_RAM;
else if (strcmp(ident, "pmem") == 0)
return CXL_DECODER_MODE_PMEM;
+ else if (strcmp(ident, "dc0") == 0)
+ return CXL_DECODER_MODE_DC0;
+ else if (strcmp(ident, "dc1") == 0)
+ return CXL_DECODER_MODE_DC1;
+ else if (strcmp(ident, "dc2") == 0)
+ return CXL_DECODER_MODE_DC2;
+ else if (strcmp(ident, "dc3") == 0)
+ return CXL_DECODER_MODE_DC3;
+ else if (strcmp(ident, "dc4") == 0)
+ return CXL_DECODER_MODE_DC4;
+ else if (strcmp(ident, "dc5") == 0)
+ return CXL_DECODER_MODE_DC5;
+ else if (strcmp(ident, "dc6") == 0)
+ return CXL_DECODER_MODE_DC6;
+ else if (strcmp(ident, "dc7") == 0)
+ return CXL_DECODER_MODE_DC7;
return CXL_DECODER_MODE_NONE;
}
+static inline bool cxl_decoder_mode_is_dc(enum cxl_decoder_mode mode)
+{
+ return (mode >= CXL_DECODER_MODE_DC0 && mode <= CXL_DECODER_MODE_DC7);
+}
+
enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder);
int cxl_decoder_set_mode(struct cxl_decoder *decoder,
enum cxl_decoder_mode mode);
@@ -248,6 +289,8 @@ enum cxl_decoder_target_type {
enum cxl_decoder_target_type
cxl_decoder_get_target_type(struct cxl_decoder *decoder);
bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder);
+bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder,
+ enum cxl_decoder_mode mode);
bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder);
bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder);
bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder);
@@ -258,6 +301,7 @@ unsigned int cxl_decoder_get_interleave_ways(struct cxl_decoder *decoder);
struct cxl_region *cxl_decoder_get_region(struct cxl_decoder *decoder);
struct cxl_region *cxl_decoder_create_pmem_region(struct cxl_decoder *decoder);
struct cxl_region *cxl_decoder_create_ram_region(struct cxl_decoder *decoder);
+struct cxl_region *cxl_decoder_create_dc_region(struct cxl_decoder *decoder);
struct cxl_decoder *cxl_decoder_get_by_name(struct cxl_ctx *ctx,
const char *ident);
struct cxl_memdev *cxl_decoder_get_memdev(struct cxl_decoder *decoder);
@@ -308,6 +352,7 @@ enum cxl_region_mode {
CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED,
CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM,
CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM,
+ CXL_REGION_MODE_DC,
};
static inline const char *cxl_region_mode_name(enum cxl_region_mode mode)
@@ -317,9 +362,10 @@ static inline const char *cxl_region_mode_name(enum cxl_region_mode mode)
[CXL_REGION_MODE_MIXED] = "mixed",
[CXL_REGION_MODE_PMEM] = "pmem",
[CXL_REGION_MODE_RAM] = "ram",
+ [CXL_REGION_MODE_DC] = "dc",
};
- if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM)
+ if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_DC)
mode = CXL_REGION_MODE_NONE;
return names[mode];
}
@@ -333,6 +379,8 @@ cxl_region_mode_from_ident(const char *ident)
return CXL_REGION_MODE_RAM;
else if (strcmp(ident, "pmem") == 0)
return CXL_REGION_MODE_PMEM;
+ else if (strcmp(ident, "dc") == 0)
+ return CXL_REGION_MODE_DC;
return CXL_REGION_MODE_NONE;
}
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support
2024-12-15 2:58 ` [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support Ira Weiny
@ 2025-02-11 3:12 ` Alison Schofield
0 siblings, 0 replies; 16+ messages in thread
From: Alison Schofield @ 2025-02-11 3:12 UTC (permalink / raw)
To: Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
On Sat, Dec 14, 2024 at 08:58:32PM -0600, Ira Weiny wrote:
> CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity
> with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be
> sparse and defined as dynamic capacity (dc).
>
> Add support for DCD devices and regions to libcxl. Add documentation
> for the new interfaces.
It seems helpers (defined or modified) in this set can be used more.
I gave one example below. Again, I'll catch it on the next rev.
>
> Based on an original patch from Navneet Singh.
>
> Signed-off-by: Sushant1 Kumar <sushant1.kumar@intel.com>
> Co-developed-by: Ira Weiny <ira.weiny@intel.com>
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> ---
> Documentation/cxl/lib/libcxl.txt | 17 ++++++-
> cxl/lib/libcxl.c | 98 ++++++++++++++++++++++++++++++++++++++++
> cxl/lib/libcxl.sym | 3 ++
> cxl/lib/private.h | 4 ++
> cxl/libcxl.h | 52 ++++++++++++++++++++-
> 5 files changed, 171 insertions(+), 3 deletions(-)
snip
> @@ -2275,6 +2305,22 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base)
> decoder->mode = CXL_DECODER_MODE_RAM;
> else if (strcmp(buf, "pmem") == 0)
> decoder->mode = CXL_DECODER_MODE_PMEM;
> + else if (strcmp(buf, "dc0") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC0;
> + else if (strcmp(buf, "dc1") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC1;
> + else if (strcmp(buf, "dc2") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC2;
> + else if (strcmp(buf, "dc3") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC3;
> + else if (strcmp(buf, "dc4") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC4;
> + else if (strcmp(buf, "dc5") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC5;
> + else if (strcmp(buf, "dc6") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC6;
> + else if (strcmp(buf, "dc7") == 0)
> + decoder->mode = CXL_DECODER_MODE_DC7;
> else if (strcmp(buf, "mixed") == 0)
> decoder->mode = CXL_DECODER_MODE_MIXED;
> else if (strcmp(buf, "none") == 0)
If the modes survive, above can be simplified, probably as a separate
function using cxl_region_mode_name(), cxl_decoder_mode_is_dc() to get
to the mode without all those strcmps. Check all the predefined modes
first, then all the dc modes.
snip
> @@ -2592,6 +2648,30 @@ CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder,
> case CXL_DECODER_MODE_RAM:
> sprintf(buf, "ram");
> break;
> + case CXL_DECODER_MODE_DC0:
> + sprintf(buf, "dc0");
> + break;
> + case CXL_DECODER_MODE_DC1:
> + sprintf(buf, "dc1");
> + break;
> + case CXL_DECODER_MODE_DC2:
> + sprintf(buf, "dc2");
> + break;
> + case CXL_DECODER_MODE_DC3:
> + sprintf(buf, "dc3");
> + break;
> + case CXL_DECODER_MODE_DC4:
> + sprintf(buf, "dc4");
> + break;
> + case CXL_DECODER_MODE_DC5:
> + sprintf(buf, "dc5");
> + break;
> + case CXL_DECODER_MODE_DC6:
> + sprintf(buf, "dc6");
> + break;
> + case CXL_DECODER_MODE_DC7:
> + sprintf(buf, "dc7");
> + break;
> default:
> err(ctx, "%s: unsupported mode: %d\n",
> cxl_decoder_get_devname(decoder), mode);
Again, if all these modes survive, this can tidy up like:
@@ -2593,9 +2646,14 @@ CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder,
sprintf(buf, "ram");
break;
default:
- err(ctx, "%s: unsupported mode: %d\n",
- cxl_decoder_get_devname(decoder), mode);
- return -EINVAL;
+ if (cxl_decoder_mode_is_dc(mode)) {
+ sprintf(buf, "dc%d", mode - CXL_DECODER_MODE_DC0);
+ } else {
+ err(ctx, "%s: unsupported mode: %d\n",
+ cxl_decoder_get_devname(decoder), mode);
+ return -EINVAL;
+ }
+ break;
}
snip to end.
^ permalink raw reply [flat|nested] 16+ messages in thread
* [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (4 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2025-02-11 3:21 ` Alison Schofield
2024-12-15 2:58 ` [ndctl PATCH v4 7/9] libcxl: Add extent functionality to DC regions Ira Weiny
` (3 subsequent siblings)
9 siblings, 1 reply; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity
with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be
sparse and defined as dynamic capacity (dc).
DCD region creation requires a specific partition, or decoder mode, to
be supplied. Introduce a required option for dc regions to specify the
decoder mode.
Add support for dynamic capacity region creation.
Based on an original patch from Navneet Singh.
Signed-off-by: Sushant1 Kumar <sushant1.kumar@intel.com>
Co-developed-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Documentation/cxl/cxl-create-region.txt | 11 ++++++++--
cxl/json.c | 27 ++++++++++++++++++++++-
cxl/memdev.c | 4 +++-
cxl/region.c | 39 ++++++++++++++++++++++++++++++++-
4 files changed, 76 insertions(+), 5 deletions(-)
diff --git a/Documentation/cxl/cxl-create-region.txt b/Documentation/cxl/cxl-create-region.txt
index b244af60b8a63281ed63d0d6f4027ea729ad51b0..a12cc8d3f19fa582376599ecc8512640f15ce42c 100644
--- a/Documentation/cxl/cxl-create-region.txt
+++ b/Documentation/cxl/cxl-create-region.txt
@@ -75,8 +75,9 @@ include::bus-option.txt[]
-t::
--type=::
- Specify the region type - 'pmem' or 'ram'. Default to root decoder
- capability, and if that is ambiguous, default to 'pmem'.
+ Specify the region type - 'pmem', 'ram', or 'dc'. Default to root
+ decoder capability including the first of any DC partition found. If
+ the decoder capability is ambiguous, default to 'pmem'.
-U::
--uuid=::
@@ -105,6 +106,12 @@ include::bus-option.txt[]
supplied, the first cross-host bridge (if available), decoder that
supports the largest interleave will be chosen.
+-M::
+--decoder-mode=::
+ For a 'dc' region type, specify the dynamic capacity partition to be
+ used on each device, 'dcX' [where X is 0-7]. Ignored for region type
+ of 'ram' or 'pmem'.
+
-Q::
--enforce-qos::
Parameter to enforce qos_class mismatch failure. Region create operation
diff --git a/cxl/json.c b/cxl/json.c
index dcd3cc28393faf7e8adf299a857531ecdeaac50a..c5391be84fba51da57fc15ece7c1f94cce139276 100644
--- a/cxl/json.c
+++ b/cxl/json.c
@@ -754,12 +754,15 @@ err_free:
return jpoison;
}
+#define DC_SIZE_NAME_LEN 64
struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
unsigned long flags)
{
const char *devname = cxl_memdev_get_devname(memdev);
+ char size_name[DC_SIZE_NAME_LEN];
struct json_object *jdev, *jobj;
unsigned long long serial, size;
+ enum cxl_decoder_mode mode;
const char *fw_version;
int numa_node;
int qos_class;
@@ -800,6 +803,16 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev,
}
}
+ for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) {
+ size = cxl_memdev_get_dc_size(memdev, mode);
+ if (!size)
+ continue;
+ jobj = util_json_object_size(size, flags);
+ if (!jobj)
+ continue;
+ sprintf(size_name, "%s_size", cxl_decoder_mode_name(mode));
+ json_object_object_add(jdev, size_name, jobj);
+ }
if (flags & UTIL_JSON_HEALTH) {
jobj = util_cxl_memdev_health_to_json(memdev, flags);
if (jobj)
@@ -948,12 +961,15 @@ struct json_object *util_cxl_bus_to_json(struct cxl_bus *bus,
return jbus;
}
+#define DC_CAPABILITY_NAME_LEN 16
struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder,
unsigned long flags)
{
const char *devname = cxl_decoder_get_devname(decoder);
struct cxl_port *port = cxl_decoder_get_port(decoder);
+ char dc_capable_name[DC_CAPABILITY_NAME_LEN];
struct json_object *jdecoder, *jobj;
+ enum cxl_decoder_mode mode;
struct cxl_region *region;
u64 val, size;
@@ -1013,7 +1029,7 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder,
}
if (cxl_port_is_endpoint(port)) {
- enum cxl_decoder_mode mode = cxl_decoder_get_mode(decoder);
+ mode = cxl_decoder_get_mode(decoder);
size = cxl_decoder_get_dpa_size(decoder);
val = cxl_decoder_get_dpa_resource(decoder);
@@ -1059,6 +1075,15 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder,
json_object_object_add(
jdecoder, "volatile_capable", jobj);
}
+ for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) {
+ if (!cxl_decoder_is_dc_capable(decoder, mode))
+ continue;
+ jobj = json_object_new_boolean(true);
+ if (!jobj)
+ continue;
+ sprintf(dc_capable_name, "%s_capable", cxl_decoder_mode_name(mode));
+ json_object_object_add(jdecoder, dc_capable_name, jobj);
+ }
}
if (cxl_port_is_root(port) &&
diff --git a/cxl/memdev.c b/cxl/memdev.c
index 6e44d1578d03b6af998502e54714635b8f31b556..b132527d7c21e99c9da75ea7cbf1497fd924a142 100644
--- a/cxl/memdev.c
+++ b/cxl/memdev.c
@@ -269,8 +269,10 @@ static int __reserve_dpa(struct cxl_memdev *memdev,
if (mode == CXL_DECODER_MODE_RAM)
avail_dpa = cxl_memdev_get_ram_size(memdev);
- else
+ else if (mode == CXL_DECODER_MODE_PMEM)
avail_dpa = cxl_memdev_get_pmem_size(memdev);
+ else if (cxl_decoder_mode_is_dc(mode))
+ avail_dpa = cxl_memdev_get_dc_size(memdev, mode);
cxl_decoder_foreach(port, decoder) {
size = cxl_decoder_get_dpa_size(decoder);
diff --git a/cxl/region.c b/cxl/region.c
index 527bd6708b162815068a95ddb360fce3914347de..79f434b0c99545523f3b8209f90ff2f02111881c 100644
--- a/cxl/region.c
+++ b/cxl/region.c
@@ -26,6 +26,7 @@ static struct region_params {
const char *uuid;
const char *root_decoder;
const char *region;
+ const char *decoder_mode;
int ways;
int granularity;
bool memdevs;
@@ -79,9 +80,11 @@ OPT_INTEGER('w', "ways", ¶m.ways, \
OPT_INTEGER('g', "granularity", ¶m.granularity, \
"granularity of the interleave set"), \
OPT_STRING('t', "type", ¶m.type, \
- "region type", "region type - 'pmem' or 'ram'"), \
+ "region type", "region type - 'pmem', 'ram', or 'dc'"), \
OPT_STRING('U', "uuid", ¶m.uuid, \
"region uuid", "uuid for the new region (default: autogenerate)"), \
+OPT_STRING('M', "decoder-mode", ¶m.decoder_mode, "decoder mode", \
+ "decoder mode for dc regions - 'dcX' [where X is 0-7]"), \
OPT_BOOLEAN('m', "memdevs", ¶m.memdevs, \
"non-option arguments are memdevs"), \
OPT_BOOLEAN('u', "human", ¶m.human, "use human friendly number formats"), \
@@ -314,12 +317,19 @@ static int parse_create_options(struct cxl_ctx *ctx, int count,
log_err(&rl, "unsupported type: %s\n", param.type);
goto err;
}
+ if (p->region_mode == CXL_REGION_MODE_DC && !param.decoder_mode) {
+ log_err(&rl, "dc type requires a decoder mode\n");
+ goto err;
+ }
}
switch (p->region_mode) {
case CXL_REGION_MODE_RAM:
p->decoder_mode = CXL_DECODER_MODE_RAM;
break;
+ case CXL_REGION_MODE_DC:
+ p->decoder_mode = cxl_decoder_mode_from_ident(param.decoder_mode);
+ break;
case CXL_REGION_MODE_PMEM:
default:
p->decoder_mode = CXL_DECODER_MODE_PMEM;
@@ -427,6 +437,9 @@ static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p)
case CXL_DECODER_MODE_PMEM:
size = cxl_memdev_get_pmem_size(memdev);
break;
+ case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7:
+ size = cxl_memdev_get_dc_size(memdev, p->decoder_mode);
+ break;
default:
/* Shouldn't ever get here */ ;
}
@@ -498,6 +511,13 @@ static int validate_decoder(struct cxl_decoder *decoder,
return -EINVAL;
}
break;
+ case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7:
+ if (!cxl_decoder_is_dc_capable(decoder, p->decoder_mode)) {
+ log_err(&rl, "%s is not %s capable\n", devname,
+ cxl_decoder_mode_name(p->decoder_mode));
+ return -EINVAL;
+ }
+ break;
default:
log_err(&rl, "unknown type: %s\n", param.type);
return -EINVAL;
@@ -514,10 +534,20 @@ static int validate_decoder(struct cxl_decoder *decoder,
static void set_type_from_decoder(struct cxl_ctx *ctx, struct parsed_params *p)
{
+ enum cxl_decoder_mode mode;
+
/* if param.type was explicitly specified, nothing to do here */
if (param.type)
return;
+ for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) {
+ if (cxl_decoder_is_dc_capable(p->root_decoder, mode)) {
+ p->decoder_mode = mode;
+ p->region_mode = CXL_REGION_MODE_DC;
+ break;
+ }
+ }
+
/*
* default to pmem if both types are set, otherwise the single
* capability dominates.
@@ -713,6 +743,13 @@ static int create_region(struct cxl_ctx *ctx, int *count,
param.root_decoder);
return -ENXIO;
}
+ } else if (p->region_mode == CXL_REGION_MODE_DC) {
+ region = cxl_decoder_create_dc_region(p->root_decoder);
+ if (!region) {
+ log_err(&rl, "failed to create region under %s\n",
+ param.root_decoder);
+ return -ENXIO;
+ }
} else {
log_err(&rl, "region type '%s' is not supported\n",
param.type);
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions
2024-12-15 2:58 ` [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions Ira Weiny
@ 2025-02-11 3:21 ` Alison Schofield
0 siblings, 0 replies; 16+ messages in thread
From: Alison Schofield @ 2025-02-11 3:21 UTC (permalink / raw)
To: Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
On Sat, Dec 14, 2024 at 08:58:33PM -0600, Ira Weiny wrote:
> CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity
> with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be
> sparse and defined as dynamic capacity (dc).
>
> DCD region creation requires a specific partition, or decoder mode, to
> be supplied. Introduce a required option for dc regions to specify the
> decoder mode.
>
> Add support for dynamic capacity region creation.
>
> Based on an original patch from Navneet Singh.
>
> Signed-off-by: Sushant1 Kumar <sushant1.kumar@intel.com>
> Co-developed-by: Ira Weiny <ira.weiny@intel.com>
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> ---
> Documentation/cxl/cxl-create-region.txt | 11 ++++++++--
> cxl/json.c | 27 ++++++++++++++++++++++-
> cxl/memdev.c | 4 +++-
> cxl/region.c | 39 ++++++++++++++++++++++++++++++++-
> 4 files changed, 76 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/cxl/cxl-create-region.txt b/Documentation/cxl/cxl-create-region.txt
> index b244af60b8a63281ed63d0d6f4027ea729ad51b0..a12cc8d3f19fa582376599ecc8512640f15ce42c 100644
> --- a/Documentation/cxl/cxl-create-region.txt
> +++ b/Documentation/cxl/cxl-create-region.txt
> @@ -75,8 +75,9 @@ include::bus-option.txt[]
>
> -t::
> --type=::
> - Specify the region type - 'pmem' or 'ram'. Default to root decoder
> - capability, and if that is ambiguous, default to 'pmem'.
> + Specify the region type - 'pmem', 'ram', or 'dc'. Default to root
> + decoder capability including the first of any DC partition found. If
> + the decoder capability is ambiguous, default to 'pmem'.
When or why is a root decoders capability ambiguous?
Is type 'dc' really analagous to 'pmem' or 'ram'? As a newbie to DCD,
I'd have guessed it would say 'dyn' or 'dynamic' because dc is more
analagous to 'pc' and 'rc' and we don't use those.
-- snip to end
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [ndctl PATCH v4 7/9] libcxl: Add extent functionality to DC regions
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (5 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2024-12-15 2:58 ` [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query Ira Weiny
` (2 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
DCD regions have 0 or more extents. The ability to list those and their
properties is useful to end users.
Add extent scanning and reporting functionality to libcxl.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Documentation/cxl/lib/libcxl.txt | 27 ++++++++
cxl/lib/libcxl.c | 138 +++++++++++++++++++++++++++++++++++++++
cxl/lib/libcxl.sym | 5 ++
cxl/lib/private.h | 11 ++++
cxl/libcxl.h | 11 ++++
5 files changed, 192 insertions(+)
diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt
index abca08fc81e6e84d176facafad6decae2f875880..0b53cf9a3a09a3e8c9059f796823b52d22d1077f 100644
--- a/Documentation/cxl/lib/libcxl.txt
+++ b/Documentation/cxl/lib/libcxl.txt
@@ -632,6 +632,33 @@ Regions now have a mode distinct from decoders. cxl_region_get_mode() is
deprecated in favor of cxl_region_get_region_mode(). Dynamic capacity regions
require the use of cxl_region_get_region_mode().
+EXTENTS
+-------
+
+=== EXTENT: Enumeration
+----
+struct cxl_region_extent;
+struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region);
+struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent);
+#define cxl_extent_foreach(region, extent) \
+ for (extent = cxl_extent_get_first(region); \
+ extent != NULL; \
+ extent = cxl_extent_get_next(extent))
+
+----
+
+=== EXTENT: Attributes
+----
+unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent);
+unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent);
+void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag);
+----
+
+Extents represent available memory within a dynamic capacity region. Extent
+objects are available for informational purposes to aid in allocation of
+memory.
+
+
include::../../copyright.txt[]
SEE ALSO
diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c
index df250db9dbacb2f0f34e8a592ce194159584fe4f..a029b14dcccf038b02b28d05df6f0dc71557df5e 100644
--- a/cxl/lib/libcxl.c
+++ b/cxl/lib/libcxl.c
@@ -568,6 +568,7 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base)
region->ctx = ctx;
region->decoder = decoder;
list_head_init(®ion->mappings);
+ list_head_init(®ion->extents);
region->dev_path = strdup(cxlregion_base);
if (!region->dev_path)
@@ -1178,6 +1179,143 @@ cxl_mapping_get_next(struct cxl_memdev_mapping *mapping)
return list_next(®ion->mappings, mapping, list);
}
+static void cxl_extents_init(struct cxl_region *region)
+{
+ const char *devname = cxl_region_get_devname(region);
+ struct cxl_ctx *ctx = cxl_region_get_ctx(region);
+ char *extent_path, *dax_region_path;
+ struct dirent *de;
+ DIR *dir = NULL;
+
+ if (region->extents_init)
+ return;
+ region->extents_init = 1;
+
+ dax_region_path = calloc(1, strlen(region->dev_path) + 64);
+ if (!dax_region_path) {
+ err(ctx, "%s: allocation failure\n", devname);
+ return;
+ }
+
+ extent_path = calloc(1, strlen(region->dev_path) + 100);
+ if (!extent_path) {
+ err(ctx, "%s: allocation failure\n", devname);
+ free(dax_region_path);
+ return;
+ }
+
+ sprintf(dax_region_path, "%s/dax_region%d",
+ region->dev_path, region->id);
+ dir = opendir(dax_region_path);
+ if (!dir) {
+ err(ctx, "no extents found (%s): %s\n",
+ strerror(errno), dax_region_path);
+ free(extent_path);
+ free(dax_region_path);
+ return;
+ }
+
+ while ((de = readdir(dir)) != NULL) {
+ struct cxl_region_extent *extent;
+ char buf[SYSFS_ATTR_SIZE];
+ u64 offset, length;
+ int id, region_id;
+
+ if (sscanf(de->d_name, "extent%d.%d", ®ion_id, &id) != 2)
+ continue;
+
+ sprintf(extent_path, "%s/extent%d.%d/offset",
+ dax_region_path, region_id, id);
+ if (sysfs_read_attr(ctx, extent_path, buf) < 0) {
+ err(ctx, "%s: failed to read extent%d.%d/offset\n",
+ devname, region_id, id);
+ continue;
+ }
+
+ offset = strtoull(buf, NULL, 0);
+ if (offset == ULLONG_MAX) {
+ err(ctx, "%s extent%d.%d: failed to read offset\n",
+ devname, region_id, id);
+ continue;
+ }
+
+ sprintf(extent_path, "%s/extent%d.%d/length",
+ dax_region_path, region_id, id);
+ if (sysfs_read_attr(ctx, extent_path, buf) < 0) {
+ err(ctx, "%s: failed to read extent%d.%d/length\n",
+ devname, region_id, id);
+ continue;
+ }
+
+ length = strtoull(buf, NULL, 0);
+ if (length == ULLONG_MAX) {
+ err(ctx, "%s extent%d.%d: failed to read length\n",
+ devname, region_id, id);
+ continue;
+ }
+
+ sprintf(extent_path, "%s/extent%d.%d/tag",
+ dax_region_path, region_id, id);
+ buf[0] = '\0';
+ if (sysfs_read_attr(ctx, extent_path, buf) != 0)
+ dbg(ctx, "%s extent%d.%d: failed to read tag\n",
+ devname, region_id, id);
+
+ extent = calloc(1, sizeof(*extent));
+ if (!extent) {
+ err(ctx, "%s extent%d.%d: allocation failure\n",
+ devname, region_id, id);
+ continue;
+ }
+ if (strlen(buf) && uuid_parse(buf, extent->tag) < 0)
+ err(ctx, "%s:%s\n", extent_path, buf);
+ extent->region = region;
+ extent->offset = offset;
+ extent->length = length;
+
+ list_node_init(&extent->list);
+ list_add(®ion->extents, &extent->list);
+ dbg(ctx, "%s added extent%d.%d\n", devname, region_id, id);
+ }
+ free(dax_region_path);
+ free(extent_path);
+ closedir(dir);
+}
+
+CXL_EXPORT struct cxl_region_extent *
+cxl_extent_get_first(struct cxl_region *region)
+{
+ cxl_extents_init(region);
+
+ return list_top(®ion->extents, struct cxl_region_extent, list);
+}
+
+CXL_EXPORT struct cxl_region_extent *
+cxl_extent_get_next(struct cxl_region_extent *extent)
+{
+ struct cxl_region *region = extent->region;
+
+ return list_next(®ion->extents, extent, list);
+}
+
+CXL_EXPORT unsigned long long
+cxl_extent_get_offset(struct cxl_region_extent *extent)
+{
+ return extent->offset;
+}
+
+CXL_EXPORT unsigned long long
+cxl_extent_get_length(struct cxl_region_extent *extent)
+{
+ return extent->length;
+}
+
+CXL_EXPORT void
+cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag)
+{
+ memcpy(tag, extent->tag, sizeof(uuid_t));
+}
+
CXL_EXPORT struct cxl_decoder *
cxl_mapping_get_decoder(struct cxl_memdev_mapping *mapping)
{
diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym
index fdb227789985443a13c72751bbd42ab383db5f97..d8e8dbc7e091792fe48faa4657ab7cf1d795efdd 100644
--- a/cxl/lib/libcxl.sym
+++ b/cxl/lib/libcxl.sym
@@ -295,4 +295,9 @@ global:
cxl_memdev_get_dc_size;
cxl_decoder_is_dc_capable;
cxl_decoder_create_dc_region;
+ cxl_extent_get_first;
+ cxl_extent_get_next;
+ cxl_extent_get_offset;
+ cxl_extent_get_length;
+ cxl_extent_get_tag;
} LIBECXL_8;
diff --git a/cxl/lib/private.h b/cxl/lib/private.h
index 3efa230bfb632e6c6048aadd18f799b07d4bdfd3..62278ec79963c198dcca490015e4c3f7621109b2 100644
--- a/cxl/lib/private.h
+++ b/cxl/lib/private.h
@@ -164,6 +164,7 @@ struct cxl_region {
struct cxl_decoder *decoder;
struct list_node list;
int mappings_init;
+ int extents_init;
struct cxl_ctx *ctx;
void *dev_buf;
size_t buf_len;
@@ -179,6 +180,7 @@ struct cxl_region {
struct daxctl_region *dax_region;
struct kmod_module *module;
struct list_head mappings;
+ struct list_head extents;
};
struct cxl_memdev_mapping {
@@ -188,6 +190,15 @@ struct cxl_memdev_mapping {
struct list_node list;
};
+#define CXL_REGION_EXTENT_TAG 0x10
+struct cxl_region_extent {
+ struct cxl_region *region;
+ u64 offset;
+ u64 length;
+ uuid_t tag;
+ struct list_node list;
+};
+
enum cxl_cmd_query_status {
CXL_CMD_QUERY_NOT_RUN = 0,
CXL_CMD_QUERY_OK,
diff --git a/cxl/libcxl.h b/cxl/libcxl.h
index d7f8a37816f236acd71fc834eae70a7a17a2721a..1d294ac0278d798214acb2f62e98aaaccaf60ea5 100644
--- a/cxl/libcxl.h
+++ b/cxl/libcxl.h
@@ -445,6 +445,17 @@ unsigned int cxl_mapping_get_position(struct cxl_memdev_mapping *mapping);
mapping != NULL; \
mapping = cxl_mapping_get_next(mapping))
+struct cxl_region_extent;
+struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region);
+struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent);
+#define cxl_extent_foreach(region, extent) \
+ for (extent = cxl_extent_get_first(region); \
+ extent != NULL; \
+ extent = cxl_extent_get_next(extent))
+unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent);
+unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent);
+void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag);
+
struct cxl_cmd;
const char *cxl_cmd_get_devname(struct cxl_cmd *cmd);
struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode);
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (6 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 7/9] libcxl: Add extent functionality to DC regions Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2025-02-11 3:24 ` Alison Schofield
2024-12-15 2:58 ` [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests Ira Weiny
2025-02-11 2:47 ` [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Alison Schofield
9 siblings, 1 reply; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
DCD regions have 0 or more extents. The ability to list those and their
properties is useful to end users.
Add an option for extent output to region queries. An example of this
is:
$ ./build/cxl/cxl list -r 8 -Nu
{
"region":"region8",
...
"type":"dc",
...
"extents":[
{
"offset":"0x10000000",
"length":"64.00 MiB (67.11 MB)",
"tag":"00000000-0000-0000-0000-000000000000"
},
{
"offset":"0x8000000",
"length":"64.00 MiB (67.11 MB)",
"tag":"00000000-0000-0000-0000-000000000000"
}
]
}
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Documentation/cxl/cxl-list.txt | 29 ++++++++++++++++++++++++++
cxl/filter.h | 3 +++
cxl/json.c | 47 ++++++++++++++++++++++++++++++++++++++++++
cxl/json.h | 3 +++
cxl/list.c | 3 +++
util/json.h | 1 +
6 files changed, 86 insertions(+)
diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt
index 9a9911e7dd9bba561c6202784017db1bb4b9f4bd..43453cc72245f586070f8c4f31b3ee475e3c6cd2 100644
--- a/Documentation/cxl/cxl-list.txt
+++ b/Documentation/cxl/cxl-list.txt
@@ -411,6 +411,35 @@ OPTIONS
}
----
+-N::
+--extents::
+ Append Dynamic Capacity extent information.
+----
+13:34:28 > ./build/cxl/cxl list -r 8 -Nu
+{
+ "region":"region8",
+ "resource":"0xf030000000",
+ "size":"512.00 MiB (536.87 MB)",
+ "type":"dc",
+ "interleave_ways":1,
+ "interleave_granularity":256,
+ "decode_state":"commit",
+ "extents":[
+ {
+ "offset":"0x10000000",
+ "length":"64.00 MiB (67.11 MB)",
+ "tag":"00000000-0000-0000-0000-000000000000"
+ },
+ {
+ "offset":"0x8000000",
+ "length":"64.00 MiB (67.11 MB)",
+ "tag":"00000000-0000-0000-0000-000000000000"
+ }
+ ]
+}
+----
+
+
-r::
--region::
Specify CXL region device name(s), or device id(s), to filter the listing.
diff --git a/cxl/filter.h b/cxl/filter.h
index 956a46e0c7a9f05abf696cce97a365164e95e50d..a31b80c87ccac407bd4ff98b302a23b33cbe413c 100644
--- a/cxl/filter.h
+++ b/cxl/filter.h
@@ -31,6 +31,7 @@ struct cxl_filter_params {
bool alert_config;
bool dax;
bool media_errors;
+ bool extents;
int verbose;
struct log_ctx ctx;
};
@@ -91,6 +92,8 @@ static inline unsigned long cxl_filter_to_flags(struct cxl_filter_params *param)
flags |= UTIL_JSON_DAX | UTIL_JSON_DAX_DEVS;
if (param->media_errors)
flags |= UTIL_JSON_MEDIA_ERRORS;
+ if (param->extents)
+ flags |= UTIL_JSON_EXTENTS;
return flags;
}
diff --git a/cxl/json.c b/cxl/json.c
index c5391be84fba51da57fc15ece7c1f94cce139276..450e62243ecdfec1aa011241ff7257ac3b37196f 100644
--- a/cxl/json.c
+++ b/cxl/json.c
@@ -1169,6 +1169,50 @@ void util_cxl_mappings_append_json(struct json_object *jregion,
json_object_object_add(jregion, "mappings", jmappings);
}
+void util_cxl_extents_append_json(struct json_object *jregion,
+ struct cxl_region *region,
+ unsigned long flags)
+{
+ struct json_object *jextents;
+ struct cxl_region_extent *extent;
+
+ jextents = json_object_new_array();
+ if (!jextents)
+ return;
+
+ cxl_extent_foreach(region, extent) {
+ struct json_object *jextent, *jobj;
+ unsigned long long val;
+ char tag_str[40];
+ uuid_t tag;
+
+ jextent = json_object_new_object();
+ if (!jextent)
+ continue;
+
+ val = cxl_extent_get_offset(extent);
+ jobj = util_json_object_hex(val, flags);
+ if (jobj)
+ json_object_object_add(jextent, "offset", jobj);
+
+ val = cxl_extent_get_length(extent);
+ jobj = util_json_object_size(val, flags);
+ if (jobj)
+ json_object_object_add(jextent, "length", jobj);
+
+ cxl_extent_get_tag(extent, tag);
+ uuid_unparse(tag, tag_str);
+ jobj = json_object_new_string(tag_str);
+ if (jobj)
+ json_object_object_add(jextent, "tag", jobj);
+
+ json_object_array_add(jextents, jextent);
+ json_object_set_userdata(jextent, extent, NULL);
+ }
+
+ json_object_object_add(jregion, "extents", jextents);
+}
+
struct json_object *util_cxl_region_to_json(struct cxl_region *region,
unsigned long flags)
{
@@ -1255,6 +1299,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region,
}
}
+ if (flags & UTIL_JSON_EXTENTS)
+ util_cxl_extents_append_json(jregion, region, flags);
+
if (cxl_region_qos_class_mismatch(region)) {
jobj = json_object_new_boolean(true);
if (jobj)
diff --git a/cxl/json.h b/cxl/json.h
index eb7572be4106baf0469ba9243a9a767d07df8882..f9c07ab41a337838b75ffee4486f6c48ddc99863 100644
--- a/cxl/json.h
+++ b/cxl/json.h
@@ -20,6 +20,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region,
void util_cxl_mappings_append_json(struct json_object *jregion,
struct cxl_region *region,
unsigned long flags);
+void util_cxl_extents_append_json(struct json_object *jregion,
+ struct cxl_region *region,
+ unsigned long flags);
void util_cxl_targets_append_json(struct json_object *jdecoder,
struct cxl_decoder *decoder,
const char *ident, const char *serial,
diff --git a/cxl/list.c b/cxl/list.c
index 0b25d78248d5f4f529fd2c2e073e43895c722568..47d135166212b87449f960e94ee75657f7040ca9 100644
--- a/cxl/list.c
+++ b/cxl/list.c
@@ -59,6 +59,8 @@ static const struct option options[] = {
"include alert configuration information"),
OPT_BOOLEAN('L', "media-errors", ¶m.media_errors,
"include media-error information "),
+ OPT_BOOLEAN('N', "extents", ¶m.extents,
+ "include extent information (Dynamic Capacity regions only)"),
OPT_INCR('v', "verbose", ¶m.verbose, "increase output detail"),
#ifdef ENABLE_DEBUG
OPT_BOOLEAN(0, "debug", &debug, "debug list walk"),
@@ -135,6 +137,7 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx)
param.decoders = true;
param.targets = true;
param.regions = true;
+ param.extents = true;
/*fallthrough*/
case 0:
break;
diff --git a/util/json.h b/util/json.h
index 560f845c6753ee176f7c64b4310fe1f9b1ce6d39..79ae3240e7ce151be75f6666fcaba0ba90aba7fc 100644
--- a/util/json.h
+++ b/util/json.h
@@ -21,6 +21,7 @@ enum util_json_flags {
UTIL_JSON_TARGETS = (1 << 11),
UTIL_JSON_PARTITION = (1 << 12),
UTIL_JSON_ALERT_CONFIG = (1 << 13),
+ UTIL_JSON_EXTENTS = (1 << 14),
};
void util_display_json_array(FILE *f_out, struct json_object *jarray,
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query
2024-12-15 2:58 ` [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query Ira Weiny
@ 2025-02-11 3:24 ` Alison Schofield
2025-02-12 22:03 ` Ira Weiny
0 siblings, 1 reply; 16+ messages in thread
From: Alison Schofield @ 2025-02-11 3:24 UTC (permalink / raw)
To: Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
On Sat, Dec 14, 2024 at 08:58:35PM -0600, Ira Weiny wrote:
> DCD regions have 0 or more extents. The ability to list those and their
> properties is useful to end users.
>
> Add an option for extent output to region queries. An example of this
> is:
>
> $ ./build/cxl/cxl list -r 8 -Nu
> {
> "region":"region8",
> ...
> "type":"dc",
> ...
> "extents":[
> {
> "offset":"0x10000000",
> "length":"64.00 MiB (67.11 MB)",
> "tag":"00000000-0000-0000-0000-000000000000"
Why do we call these 'tag' and not uid?
We use the uid helpers on them so I know they share the format.
-- snip to end
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query
2025-02-11 3:24 ` Alison Schofield
@ 2025-02-12 22:03 ` Ira Weiny
0 siblings, 0 replies; 16+ messages in thread
From: Ira Weiny @ 2025-02-12 22:03 UTC (permalink / raw)
To: Alison Schofield, Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
Alison Schofield wrote:
> On Sat, Dec 14, 2024 at 08:58:35PM -0600, Ira Weiny wrote:
> > DCD regions have 0 or more extents. The ability to list those and their
> > properties is useful to end users.
> >
> > Add an option for extent output to region queries. An example of this
> > is:
> >
> > $ ./build/cxl/cxl list -r 8 -Nu
> > {
> > "region":"region8",
> > ...
> > "type":"dc",
> > ...
> > "extents":[
> > {
> > "offset":"0x10000000",
> > "length":"64.00 MiB (67.11 MB)",
> > "tag":"00000000-0000-0000-0000-000000000000"
>
> Why do we call these 'tag' and not uid?
Because that is what they were called in the spec. But Linux has decided
they will all be uuids so. yea this should change.
> We use the uid helpers on them so I know they share the format.
Yep. I just omitted changing the label on the output.
Beyond this I think if we are going this way it is best to purge the
kernel of such language as well. So I went and did that too.
Ira
[snip]
^ permalink raw reply [flat|nested] 16+ messages in thread
* [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (7 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query Ira Weiny
@ 2024-12-15 2:58 ` Ira Weiny
2025-02-11 3:33 ` Alison Schofield
2025-02-11 2:47 ` [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Alison Schofield
9 siblings, 1 reply; 16+ messages in thread
From: Ira Weiny @ 2024-12-15 2:58 UTC (permalink / raw)
To: Alison Schofield
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm, Ira Weiny
cxl_test provides a good way to ensure quick smoke and regression
testing. The complexity of DCD and the new sparse DAX regions required
to use them benefits greatly with a series of smoke tests.
The only part of the kernel stack which must be bypassed is the actual
irq of DCD events. However, the event processing itself can be tested
via cxl_test calling directly into the event processing.
In this way the rest of the stack; management of sparse regions, the
extent device lifetimes, and the dax device operations can be tested.
Add Dynamic Capacity Device tests for kernels which have DCD support.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes:
[iweiny: Fix checks for dax devices]
[iweiny: Fix documentation on tests]
---
test/cxl-dcd.sh | 879 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
test/meson.build | 2 +
2 files changed, 881 insertions(+)
diff --git a/test/cxl-dcd.sh b/test/cxl-dcd.sh
new file mode 100644
index 0000000000000000000000000000000000000000..193b71923b2f44142ab258e1d74b88361f6b7a5e
--- /dev/null
+++ b/test/cxl-dcd.sh
@@ -0,0 +1,879 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 Intel Corporation. All rights reserved.
+
+. "$(dirname "$0")/common"
+
+rc=77
+set -ex
+
+trap 'err $LINENO' ERR
+
+check_prereq "jq"
+
+rc=1
+
+dev_path="/sys/bus/platform/devices"
+cxl_path="/sys/bus/cxl/devices"
+
+# a test extent tag
+test_tag=dc-test-tag
+
+#
+# The test devices have 2G of non DC capacity. A single DC reagion of 1G is
+# added beyond that.
+#
+# The testing centers around 3 extents. Two are "pre-existing" on test load
+# called pre-ext and pre2-ext. The other is created within this script alone
+# called base.
+
+#
+# | 2G non- | DC region (1G) |
+# | DC cap | |
+# | ... |-------------------------------------------------------|
+# | |--------| |----------| |----------| |
+# | | (base) | |(pre-ext) | |(pre2-ext)| |
+
+dc0_size=""
+dc1_size=""
+
+base_dpa=0x80000000
+
+# base extent at dpa 2G - 64M long
+base_ext_offset=0x0
+base_ext_dpa=$(($base_dpa + $base_ext_offset))
+base_ext_length=0x4000000
+
+# pre existing extent base + 128M, 64M length
+# 0x00000088000000-0x0000008bffffff
+pre_ext_offset=0x8000000
+pre_ext_dpa=$(($base_dpa + $pre_ext_offset))
+pre_ext_length=0x4000000
+
+# pre2 existing extent base + 256M, 64M length
+# 0x00000090000000-0x00000093ffffff
+pre2_ext_offset=0x10000000
+pre2_ext_dpa=$(($base_dpa + $pre2_ext_offset))
+pre2_ext_length=0x4000000
+
+mem=""
+bus=""
+device=""
+decoder=""
+
+# ========================================================================
+# Support functions
+# ========================================================================
+
+create_dcd_region()
+{
+ mem="$1"
+ decoder="$2"
+ reg_size_string=""
+ if [ "$3" != "" ]; then
+ reg_size_string="-s $3"
+ fi
+
+ dcd_partition="dc0"
+ if [ "$4" != "" ]; then
+ dcd_partition="$4"
+ fi
+
+ # create region
+ rc=$($CXL create-region -t dc -M ${dcd_partition} -d "$decoder" -m "$mem" ${reg_size_string} | jq -r ".region")
+
+ if [[ ! $rc ]]; then
+ echo "create-region failed for $decoder / $mem"
+ err "$LINENO"
+ fi
+
+ echo ${rc}
+}
+
+check_region()
+{
+ search=$1
+ region_size=$2
+
+ result=$($CXL list -r "$search" | jq -r ".[].region")
+ if [ "$result" != "$search" ]; then
+ echo "check region failed to find $search"
+ err "$LINENO"
+ fi
+
+ result=$($CXL list -r "$search" | jq -r ".[].size")
+ if [ "$result" != "$region_size" ]; then
+ echo "check region failed invalid size $result != $region_size"
+ err "$LINENO"
+ fi
+}
+
+check_not_region()
+{
+ search=$1
+
+ result=$($CXL list -r "$search" | jq -r ".[].region")
+ if [ "$result" == "$search" ]; then
+ echo "check not region failed; $search found"
+ err "$LINENO"
+ fi
+}
+
+destroy_region()
+{
+ local region=$1
+ $CXL disable-region $region
+ $CXL destroy-region $region
+}
+
+inject_extent()
+{
+ device="$1"
+ dpa="$2"
+ length="$3"
+ tag="$4"
+
+ more="0"
+ if [ "$5" != "" ]; then
+ more="1"
+ fi
+
+ echo ${dpa}:${length}:${tag}:${more} > "${dev_path}/${device}/dc_inject_extent"
+}
+
+remove_extent()
+{
+ device="$1"
+ dpa="$2"
+ length="$3"
+
+ echo ${dpa}:${length} > "${dev_path}/${device}/dc_del_extent"
+}
+
+create_dax_dev()
+{
+ reg="$1"
+
+ dax_dev=$($DAXCTL create-device -r $reg | jq -er '.[].chardev')
+
+ echo ${dax_dev}
+}
+
+fail_create_dax_dev()
+{
+ reg="$1"
+
+ set +e
+ result=$($DAXCTL create-device -r $reg)
+ set -e
+ if [ "$result" == "0" ]; then
+ echo "FAIL device created"
+ err "$LINENO"
+ fi
+}
+
+shrink_dax_dev()
+{
+ dev="$1"
+ new_size="$2"
+
+ $DAXCTL disable-device $dev
+ $DAXCTL reconfigure-device $dev -s $new_size
+ $DAXCTL enable-device $dev
+}
+
+destroy_dax_dev()
+{
+ dev="$1"
+
+ $DAXCTL disable-device $dev
+ $DAXCTL destroy-device $dev
+}
+
+check_dax_dev()
+{
+ search="$1"
+ size=$(($2))
+
+ result=$($DAXCTL list -d $search | jq -er '.[].chardev')
+ if [ "$result" != "$search" ]; then
+ echo "check dax device failed to find $search"
+ err "$LINENO"
+ fi
+ result=$($DAXCTL list -d $search | jq -er '.[].size')
+ if [ "$result" -ne "$size" ]; then
+ echo "check dax device failed incorrect size $result; exp $size"
+ err "$LINENO"
+ fi
+}
+
+# check that the dax device is not there.
+check_not_dax_dev()
+{
+ reg="$1"
+ search="$2"
+ result=$($DAXCTL list -r $reg -D | jq -r '.[].chardev')
+ if [ "$result" == "$search" ]; then
+ echo "FAIL found $search"
+ err "$LINENO"
+ fi
+}
+
+check_extent()
+{
+ region=$1
+ offset=$(($2))
+ length=$(($3))
+
+ result=$($CXL list -r "$region" -N | jq -r ".[].extents[] | select(.offset == ${offset}) | .length")
+ if [[ $result != $length ]]; then
+ echo "FAIL region $1 could not find extent @ $offset ($length)"
+ err "$LINENO"
+ fi
+}
+
+check_extent_cnt()
+{
+ region=$1
+ count=$(($2))
+
+ result=$($CXL list -r $region -N | jq -r '.[].extents[].offset' | wc -l)
+ if [[ $result != $count ]]; then
+ echo "FAIL region $1: found wrong number of extents $result; expect $count"
+ err "$LINENO"
+ fi
+}
+
+
+# ========================================================================
+# Tests
+# ========================================================================
+
+# testing pre existing extents must be called first as the extents were created
+# by cxl-test being loaded
+test_pre_existing_extents()
+{
+ echo ""
+ echo "Test: pre-existing extent"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |----------| |----------| |
+ # | | |(pre-ext) | |(pre2-ext)| |
+ check_region ${region} ${dc0_size}
+ # should contain pre-created extents
+ check_extent ${region} ${pre_ext_offset} ${pre_ext_length}
+ check_extent ${region} ${pre2_ext_offset} ${pre2_ext_length}
+
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ # | | |----------| |
+ # | | |(pre2-ext)| |
+ remove_extent ${device} $pre2_ext_dpa $pre2_ext_length
+ # | | |
+ # | | |
+ check_extent_cnt ${region} 0
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_remove_extent_under_dax_device()
+{
+ # Remove the pre-created test extent out from under dax device
+ # stack should hold ref until dax device deleted
+ echo ""
+ echo "Test: Remove extent from under DAX dev"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |
+ # | | |
+
+
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ # | | |----------| |
+ # | | |(pre-ext) | |
+ check_extent ${region} ${pre_ext_offset} ${pre_ext_length}
+
+ dax_dev=$(create_dax_dev ${region})
+ # | | |----------| |
+ # | | |(pre-ext) | |
+ # | | | daxX.1 | |
+ check_extent_cnt ${region} 1
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ # In-use extents are not released.
+ check_dax_dev ${dax_dev} $pre_ext_length
+
+ check_extent_cnt ${region} 1
+ destroy_dax_dev ${dax_dev}
+ # | | |----------| |
+ # | | |(pre-ext) | |
+ check_not_dax_dev ${region} ${dax_dev}
+
+ check_extent_cnt ${region} 1
+ # Remove after use
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ # | | |
+ # | | |
+ check_extent_cnt ${region} 0
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_remove_extents_in_use()
+{
+ echo ""
+ echo "Test: Remove extents under sparse dax device"
+ echo ""
+ remove_extent ${device} $base_ext_dpa $base_ext_length
+ check_extent_cnt ${region} 2
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ check_extent_cnt ${region} 2
+}
+
+test_create_dax_dev_spanning_two_extents()
+{
+ echo ""
+ echo "Test: Create dax device spanning 2 extents"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ check_extent ${region} ${pre_ext_offset} ${pre_ext_length}
+ # | | |----------| |
+ # | | |(pre-ext) | |
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ check_extent ${region} ${base_ext_offset} ${base_ext_length}
+ # | |--------| |----------| |
+ # | | (base) | |(pre-ext) | |
+
+ check_extent_cnt ${region} 2
+ dax_dev=$(create_dax_dev ${region})
+ # | |--------| |----------| |
+ # | | (base) | |(pre-ext) | |
+ # | | daxX.1 | | daxX.1 | |
+
+ echo "Checking if dev dax is spanning sparse extents"
+ ext_sum_length="$(($base_ext_length + $pre_ext_length))"
+ check_dax_dev ${dax_dev} $ext_sum_length
+
+ test_remove_extents_in_use
+
+ destroy_dax_dev ${dax_dev}
+ check_not_dax_dev ${region} ${dax_dev}
+
+ # In-use extents were not released. Check they can be removed after the
+ # dax device is removed.
+ check_extent_cnt ${region} 2
+ remove_extent ${device} $base_ext_dpa $base_ext_length
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ check_extent_cnt ${region} 0
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_inject_tag_support()
+{
+ echo ""
+ echo "Test: inject without/with tag"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ check_extent ${region} ${pre_ext_offset} ${pre_ext_length}
+ inject_extent ${device} $base_ext_dpa $base_ext_length "ira"
+
+ # extent with tag should be rejected
+ check_extent_cnt ${region} 1
+ remove_extent ${device} $pre_ext_dpa $pre_ext_length
+ check_extent_cnt ${region} 0
+
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_partial_extent_remove ()
+{
+ echo ""
+ echo "Test: partial extent remove"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ # | ... |-------------------------------------------------------|
+ # | |--------| |
+ # | | (base) | |
+
+ dax_dev=$(create_dax_dev ${region})
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |
+ # | | (base) | |
+ # | | daxX.1 | |
+
+ partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))"
+ partial_ext_length="$(($base_ext_length / 2))"
+ echo "Removing Partial : $partial_ext_dpa $partial_ext_length"
+
+ # | | |---| |
+ # Partial
+
+ remove_extent ${device} $partial_ext_dpa $partial_ext_length
+ # In-use extents are not released.
+ check_extent_cnt ${region} 1
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |
+ # | | (base) | |
+ # | | daxX.1 | |
+
+ destroy_dax_dev ${dax_dev}
+ check_not_dax_dev ${region} ${dax_dev}
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |
+ # | | (base) | |
+
+ # Partial results in whole extent removal
+ check_extent_cnt ${region} 1
+ remove_extent ${device} $partial_ext_dpa $partial_ext_length
+ # | | |---| |
+ # Partial
+ check_extent_cnt ${region} 0
+
+ # | ... |-------------------------------------------------------|
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_multiple_extent_remove ()
+{
+ # Test multiple extent remove
+ echo ""
+ echo "Test: multiple extent remove with single extent remove command"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ # | ... |-------------------------------------------------------|
+ # | |--------| |-------------------| |
+ # | | (base) | | (pre)-existing | |
+
+ check_extent_cnt ${region} 2
+ dax_dev=$(create_dax_dev ${region})
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |-------------------| |
+ # | | (base) | | (pre)-existing | |
+ # | | daxX.1 | | daxX.1 | |
+
+ partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))"
+ partial_ext_length="$(($pre_ext_dpa - $base_ext_dpa))"
+ echo "Removing multiple in span : $partial_ext_dpa $partial_ext_length"
+ # |------------------|
+ # Partial
+ remove_extent ${device} $partial_ext_dpa $partial_ext_length
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |-------------------| |
+ # | | (base) | | (pre)-existing | |
+ # | | daxX.1 | | daxX.1 | |
+
+ # In-use extents are not released.
+ check_extent_cnt ${region} 2
+
+ destroy_dax_dev ${dax_dev}
+ check_not_dax_dev ${region} ${dax_dev}
+
+ # | ... |-------------------------------------------------------|
+ # | |--------| |-------------------| |
+ # | | (base) | | (pre)-existing | |
+
+ # Remove both extents
+ check_extent_cnt ${region} 2
+ # |------------------|
+ # Partial
+ remove_extent ${device} $partial_ext_dpa $partial_ext_length
+ # | ... |-------------------------------------------------------|
+ check_extent_cnt ${region} 0
+
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_destroy_region_without_extent_removal ()
+{
+ echo ""
+ echo "Test: Destroy region without extent removal"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ check_extent_cnt ${region} 2
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_destroy_with_extent_and_dax ()
+{
+ echo ""
+ echo "Test: Destroy region with extents and dax devices"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ check_extent_cnt ${region} 0
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |----------| |
+ # | | |(pre-ext) | |
+
+ check_extent_cnt ${region} 1
+ dax_dev=$(create_dax_dev ${region})
+ # | | |<dax_dev> | |
+ check_dax_dev ${dax_dev} ${pre_ext_length}
+ destroy_region ${region}
+ check_not_region ${region}
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ check_extent_cnt ${region} 0
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_dax_device_ops ()
+{
+ echo ""
+ echo "Test: Fail sparse dax dev creation without space"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |-------------------| |
+ # | | | (pre)-existing | |
+
+ check_extent_cnt ${region} 1
+
+ # | | | daxX.1 | |
+
+ dax_dev=$(create_dax_dev ${region})
+ check_dax_dev ${dax_dev} $pre_ext_length
+ fail_create_dax_dev ${region}
+
+ echo ""
+ echo "Test: Resize sparse dax device"
+ echo ""
+
+ # Shrink
+ # | | | daxX.1 | |
+ resize_ext_length=$(($pre_ext_length / 2))
+ shrink_dax_dev ${dax_dev} $resize_ext_length
+ check_dax_dev ${dax_dev} $resize_ext_length
+
+ # Fill
+ # | | | daxX.1 | daxX.2 | |
+ dax_dev=$(create_dax_dev ${region})
+ check_dax_dev ${dax_dev} $resize_ext_length
+ destroy_region ${region}
+ check_not_region ${region}
+
+
+ # 2 extent
+ # create dax dev
+ # resize into 1st extent
+ # create dev on rest of 1st and all of second
+ # Ensure both devices are correct
+
+ echo ""
+ echo "Test: Resize sparse dax device across extents"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | |--------| |-------------------| |
+ # | | (base) | | (pre)-existing | |
+
+ check_extent_cnt ${region} 2
+ dax_dev=$(create_dax_dev ${region})
+ ext_sum_length="$(($base_ext_length + $pre_ext_length))"
+
+ # | | daxX.1 | | daxX.1 | |
+
+ check_dax_dev ${dax_dev} $ext_sum_length
+ resize_ext_length=33554432 # 32MB
+
+ # | | D1 | |
+
+ shrink_dax_dev ${dax_dev} $resize_ext_length
+ check_dax_dev ${dax_dev} $resize_ext_length
+
+ # | | D1 | D2| | daxX.2 | |
+
+ dax_dev=$(create_dax_dev ${region})
+ remainder_length=$((ext_sum_length - $resize_ext_length))
+ check_dax_dev ${dax_dev} $remainder_length
+
+ # | | D1 | D2| | daxX.2 | |
+
+ remainder_length=$((remainder_length / 2))
+ shrink_dax_dev ${dax_dev} $remainder_length
+ check_dax_dev ${dax_dev} $remainder_length
+
+ # | | D1 | D2| | daxX.2 | daxX.3 | |
+
+ dax_dev=$(create_dax_dev ${region})
+ check_dax_dev ${dax_dev} $remainder_length
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_reject_overlapping ()
+{
+ echo ""
+ echo "Test: Rejecting overlapping extents"
+ echo ""
+
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+
+ # | 2G non- | DC region |
+ # | DC cap | |
+ # | ... |-------------------------------------------------------|
+ # | | |-------------------| |
+ # | | | (pre)-existing | |
+
+ check_extent_cnt ${region} 1
+
+ # Attempt overlapping extent
+ #
+ # | | |-----------------| |
+ # | | | overlapping | |
+
+ partial_ext_dpa="$(($base_ext_dpa + ($pre_ext_dpa / 2)))"
+ partial_ext_length=$pre_ext_length
+ inject_extent ${device} $partial_ext_dpa $partial_ext_length ""
+
+ # Should only see the original ext
+ check_extent_cnt ${region} 1
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_two_regions()
+{
+ echo ""
+ echo "Test: create 2 regions in the same DC partition"
+ echo ""
+ region_size=$(($dc1_size / 2))
+ region=$(create_dcd_region ${mem} ${decoder_dc1} ${region_size} dc1)
+ check_region ${region} ${region_size}
+
+ region_two=$(create_dcd_region ${mem} ${decoder_dc1} ${region_size} dc1)
+ check_region ${region_two} ${region_size}
+
+ destroy_region ${region_two}
+ check_not_region ${region_two}
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_more_bit()
+{
+ echo ""
+ echo "Test: More bit"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length "" 1
+ # More bit should hold off surfacing extent until the more bit is 0
+ check_extent_cnt ${region} 0
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ check_extent_cnt ${region} 2
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_driver_tear_down()
+{
+ echo ""
+ echo "Test: driver remove tear down"
+ echo ""
+ region=$(create_dcd_region ${mem} ${decoder})
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ check_extent ${region} ${pre_ext_offset} ${pre_ext_length}
+ dax_dev=$(create_dax_dev ${region})
+ # remove driver releases extents
+ modprobe -r dax_cxl
+ check_extent_cnt ${region} 0
+}
+
+test_driver_bring_up()
+{
+ # leave region up, driver removed.
+ echo ""
+ echo "Test: no driver inject ok"
+ echo ""
+ check_region ${region} ${dc0_size}
+ inject_extent ${device} $pre_ext_dpa $pre_ext_length ""
+ check_extent_cnt ${region} 1
+
+ modprobe dax_cxl
+ check_extent_cnt ${region} 1
+
+ destroy_region ${region}
+ check_not_region ${region}
+}
+
+test_driver_reload()
+{
+ test_driver_tear_down
+ test_driver_bring_up
+}
+
+test_event_reporting()
+{
+ # Test event reporting
+ # results expected
+ num_dcd_events_expected=2
+
+ echo "Test: Prep event trace"
+ echo "" > /sys/kernel/tracing/trace
+ echo 1 > /sys/kernel/tracing/events/cxl/enable
+ echo 1 > /sys/kernel/tracing/tracing_on
+
+ inject_extent ${device} $base_ext_dpa $base_ext_length ""
+ remove_extent ${device} $base_ext_dpa $base_ext_length
+
+ echo 0 > /sys/kernel/tracing/tracing_on
+
+ echo "Test: Events seen"
+ trace_out=$(cat /sys/kernel/tracing/trace)
+
+ # Look for DCD events
+ num_dcd_events=$(grep -c "cxl_dynamic_capacity" <<< "${trace_out}")
+ echo " LOG (Expected) : (Found)"
+ echo " DCD events ($num_dcd_events_expected) : $num_dcd_events"
+
+ if [ "$num_dcd_events" -ne $num_dcd_events_expected ]; then
+ err "$LINENO"
+ fi
+}
+
+
+# ========================================================================
+# main()
+# ========================================================================
+
+modprobe -r cxl_test
+modprobe cxl_test
+
+readarray -t memdevs < <("$CXL" list -b cxl_test -Mi | jq -r '.[].memdev')
+
+for mem in ${memdevs[@]}; do
+ dc0_size=$($CXL list -m $mem | jq -r '.[].dc0_size')
+ if [ "$dc0_size" == "null" ]; then
+ continue
+ fi
+ dc1_size=$($CXL list -m $mem | jq -r '.[].dc1_size')
+ if [ "$dc1_size" == "null" ]; then
+ continue
+ fi
+ decoder=$($CXL list -b cxl_test -D -d root -m "$mem" |
+ jq -r ".[] |
+ select(.dc0_capable == true) |
+ select(.nr_targets == 1) |
+ select(.max_available_extent >= ${dc0_size}) |
+ .decoder")
+ decoder_dc1=$($CXL list -b cxl_test -D -d root -m "$mem" |
+ jq -r ".[] |
+ select(.dc1_capable == true) |
+ select(.nr_targets == 1) |
+ select(.max_available_extent >= ${dc1_size}) |
+ .decoder")
+ if [[ $decoder ]]; then
+ bus=`"$CXL" list -b cxl_test -m ${mem} | jq -r '.[].bus'`
+ device=$($CXL list -m $mem | jq -r '.[].host')
+ break
+ fi
+done
+
+echo "TEST: DCD test device bus:${bus} decoder:${decoder} mem:${mem} device:${device} size:${dc0_size}"
+
+if [ "$decoder" == "" ] || [ "$device" == "" ] || [ "$dc0_size" == "" ]; then
+ echo "No mem device/decoder found with DCD support"
+ exit 77
+fi
+
+if [ "$decoder_dc1" == "" ]; then
+ echo "insufficient DC capability for ${mem}/${device}"
+ exit 77
+fi
+
+# testing pre existing extents must be called first as the extents were created
+# by cxl-test being loaded
+test_pre_existing_extents
+test_remove_extent_under_dax_device
+test_create_dax_dev_spanning_two_extents
+test_inject_tag_support
+test_partial_extent_remove
+test_multiple_extent_remove
+test_destroy_region_without_extent_removal
+test_destroy_with_extent_and_dax
+test_dax_device_ops
+test_reject_overlapping
+test_two_regions
+test_more_bit
+test_driver_reload
+test_event_reporting
+
+modprobe -r cxl_test
+
+check_dmesg "$LINENO"
+
+exit 0
diff --git a/test/meson.build b/test/meson.build
index d871e28e17ce512cd1e7b43f3ec081729fe5e03a..1cfcb60d16e05272893ae1c67820aa8614281505 100644
--- a/test/meson.build
+++ b/test/meson.build
@@ -161,6 +161,7 @@ cxl_sanitize = find_program('cxl-sanitize.sh')
cxl_destroy_region = find_program('cxl-destroy-region.sh')
cxl_qos_class = find_program('cxl-qos-class.sh')
cxl_poison = find_program('cxl-poison.sh')
+cxl_dcd = find_program('cxl-dcd.sh')
tests = [
[ 'libndctl', libndctl, 'ndctl' ],
@@ -194,6 +195,7 @@ tests = [
[ 'cxl-destroy-region.sh', cxl_destroy_region, 'cxl' ],
[ 'cxl-qos-class.sh', cxl_qos_class, 'cxl' ],
[ 'cxl-poison.sh', cxl_poison, 'cxl' ],
+ [ 'cxl-dcd.sh', cxl_dcd, 'cxl' ],
]
if get_option('destructive').enabled()
--
2.47.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests
2024-12-15 2:58 ` [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests Ira Weiny
@ 2025-02-11 3:33 ` Alison Schofield
0 siblings, 0 replies; 16+ messages in thread
From: Alison Schofield @ 2025-02-11 3:33 UTC (permalink / raw)
To: Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
On Sat, Dec 14, 2024 at 08:58:36PM -0600, Ira Weiny wrote:
> cxl_test provides a good way to ensure quick smoke and regression
> testing. The complexity of DCD and the new sparse DAX regions required
> to use them benefits greatly with a series of smoke tests.
>
> The only part of the kernel stack which must be bypassed is the actual
> irq of DCD events. However, the event processing itself can be tested
> via cxl_test calling directly into the event processing.
>
> In this way the rest of the stack; management of sparse regions, the
> extent device lifetimes, and the dax device operations can be tested.
>
> Add Dynamic Capacity Device tests for kernels which have DCD support.
>
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Ira, This looks awesome but I'm going to wait for all the things updated
to try it out. Shellcheck has some complaints for you to consider before
the next rev.
--snip to end
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli
2024-12-15 2:58 [ndctl PATCH v4 0/9] ndctl: Dynamic Capacity additions for cxl-cli Ira Weiny
` (8 preceding siblings ...)
2024-12-15 2:58 ` [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests Ira Weiny
@ 2025-02-11 2:47 ` Alison Schofield
9 siblings, 0 replies; 16+ messages in thread
From: Alison Schofield @ 2025-02-11 2:47 UTC (permalink / raw)
To: Ira Weiny
Cc: Vishal Verma, Jonathan Cameron, Fan Ni, Sushant1 Kumar,
Dan Williams, Dave Jiang, linux-cxl, nvdimm
On Sat, Dec 14, 2024 at 08:58:27PM -0600, Ira Weiny wrote:
> Further testing showed some bugs in the 'jq' command use in cxl-test.
> Fix those bugs and adjust test to work around false positive lockdep
> splats.
>
> This series can be found here:
>
> https://github.com/weiny2/ndctl/tree/dcd-region2-2024-12-10
>
> CXL Dynamic Capacity Device (DCD) support is close to landing in the
> upstream kernel. cxl-cli requires modifications to interact with those
> devices. This includes creating and operating on DCD regions.
> cxl-testing allows for quick regression testing as well as helping to
> design the cxl-cli interfaces.
>
> Add preliminary patches with some fixes. Update libcxl, cxl-cli and
> cxl-test with DCD support.
>
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> ---
> Changes in v4:
> - iweiny: Fix dax device checks in cxl-test
> - iweiny: Update some documentation
> - Link to v3: https://patch.msgid.link/20241115-dcd-region2-v3-0-585d480ccdab@intel.com
>
> ---
Hi Ira,
Thanks for the separating out the libcxl and other updates.
I got a bit hung up on MODEs and then you made me aware that
MODEs were simplifying so I backed off trying to put together
a kernel build and run this. I eyeballed it, compiled, and
poked at a few things. See the individual patches.
> Ira Weiny (9):
> ndctl/cxl-events: Don't fail test until event counts are reported
> ndctl/cxl/region: Report max size for region creation
Above 2 applied to ndctl/pending. Sorry about bad email. I do think
I got the v4 of these.
> libcxl: Separate region mode from decoder mode
> cxl/region: Use new region mode in cxl-cli
> libcxl: Add Dynamic Capacity region support
> cxl/region: Add cxl-cli support for DCD regions
> libcxl: Add extent functionality to DC regions
> cxl/region: Add extent output to region query
> cxl/test: Add Dynamic Capacity tests
^ permalink raw reply [flat|nested] 16+ messages in thread