Linux Documentation
 help / color / mirror / Atom feed
* [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD)
@ 2024-08-16 13:59 Ira Weiny
  2024-08-16 13:59 ` [PATCH v2 02/25] printk: Add print format (%par) for struct range Ira Weiny
  2024-08-16 14:17 ` [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny
  0 siblings, 2 replies; 3+ messages in thread
From: Ira Weiny @ 2024-08-16 13:59 UTC (permalink / raw)
  To: Dave Jiang, Fan Ni, Jonathan Cameron, Navneet Singh
  Cc: Dan Williams, Davidlohr Bueso, Alison Schofield, Vishal Verma,
	Ira Weiny, linux-btrfs, linux-cxl, linux-kernel, Chris Mason,
	Josef Bacik, David Sterba, Johannes Thumshirn, Petr Mladek,
	Steven Rostedt, Jonathan Corbet, open list:DOCUMENTATION,
	Li, Ming, Jonathan Cameron

A git tree of this series can be found here:

	https://github.com/weiny2/linux-kernel/tree/dcd-v4-2024-08-15

This series requires the CXL memory notifier lock change:

	https://lore.kernel.org/all/20240814-fix-notifiers-v2-1-6bab38192c7c@intel.com/

Background
==========

A Dynamic Capacity Device (DCD) (CXL 3.1 sec 9.13.3) is a CXL memory
device that allows memory capacity within a region to change
dynamically without the need for resetting the device, reconfiguring
HDM decoders, or reconfiguring software DAX regions.

One of the biggest use cases for Dynamic Capacity is to allow hosts to
share memory dynamically within a data center without increasing the
per-host attached memory.

The general flow for the addition or removal of memory is to have an
orchestrator coordinate the use of the memory.  Generally there are 5
actors in such a system, the Orchestrator, Fabric Manager, the Logical
device, the Host Kernel, and a Host User.

Typical work flows are shown below.

Orchestrator      FM         Device       Host Kernel    Host User

    |             |           |            |              |
    |-------------- Create region ----------------------->|
    |             |           |            |              |
    |             |           |            |<-- Create ---|
    |             |           |            |    Region    |
    |<------------- Signal done --------------------------|
    |             |           |            |              |
    |-- Add ----->|-- Add --->|--- Add --->|              |
    |  Capacity   |  Extent   |   Extent   |              |
    |             |           |            |              |
    |             |<- Accept -|<- Accept  -|              |
    |             |   Extent  |   Extent   |              |
    |             |           |            |<- Create --->|
    |             |           |            |   DAX dev    |-- Use memory
    |             |           |            |              |   |
    |             |           |            |              |   |
    |             |           |            |<- Release ---| <-+
    |             |           |            |   DAX dev    |
    |             |           |            |              |
    |<------------- Signal done --------------------------|
    |             |           |            |              |
    |-- Remove -->|- Release->|- Release ->|              |
    |  Capacity   |  Extent   |   Extent   |              |
    |             |           |            |              |
    |             |<- Release-|<- Release -|              |
    |             |   Extent  |   Extent   |              |
    |             |           |            |              |
    |-- Add ----->|-- Add --->|--- Add --->|              |
    |  Capacity   |  Extent   |   Extent   |              |
    |             |           |            |              |
    |             |<- Accept -|<- Accept  -|              |
    |             |   Extent  |   Extent   |              |
    |             |           |            |<- Create ----|
    |             |           |            |   DAX dev    |-- Use memory
    |             |           |            |              |   |
    |             |           |            |<- Release ---| <-+
    |             |           |            |   DAX dev    |
    |<------------- Signal done --------------------------|
    |             |           |            |              |
    |-- Remove -->|- Release->|- Release ->|              |
    |  Capacity   |  Extent   |   Extent   |              |
    |             |           |            |              |
    |             |<- Release-|<- Release -|              |
    |             |   Extent  |   Extent   |              |
    |             |           |            |              |
    |-- Add ----->|-- Add --->|--- Add --->|              |
    |  Capacity   |  Extent   |   Extent   |              |
    |             |           |            |<- Create ----|
    |             |           |            |   DAX dev    |-- Use memory
    |             |           |            |              |   |
    |-- Remove -->|- Release->|- Release ->|              |   |
    |  Capacity   |  Extent   |   Extent   |              |   |
    |             |           |            |              |   |
    |             |           |     (Release Ignored)     |   |
    |             |           |            |              |   |
    |             |           |            |<- Release ---| <-+
    |             |           |            |   DAX dev    |
    |<------------- Signal done --------------------------|
    |             |           |            |              |
    |             |- Release->|- Release ->|              |
    |             |  Extent   |   Extent   |              |
    |             |           |            |              |
    |             |<- Release-|<- Release -|              |
    |             |   Extent  |   Extent   |              |
    |             |           |            |<- Destroy ---|
    |             |           |            |   Region     |
    |             |           |            |              |

Previous versions of this series[0] resulted in architectural comments
as well as confusion on the architecture based on the organization of
patch series itself.

This version has reordered the patches to clarify the architecture.
It also streamlines extent handling more.

The series still requires the creation of regions and DAX devices to be
synchronized with the Orchestrator and Fabric Manager.  The host kernel
will reject an add extent event if the region is not created yet.  It
will also ignore a release if the DAX device is created and referencing
an extent.

These synchronizations are not anticipated to be an issue with real
applications.

In order to allow for capacity to be added and removed a new concept of
a sparse DAX region is introduced.  A sparse DAX region may have 0 or
more bytes of available space.  The total space depends on the number
and size of the extents which have been added.

Initially it is anticipated that users of the memory will carefully
coordinate the surfacing of additional capacity with the creation of DAX
devices which use that capacity.  Therefore, the allocation of the
memory to DAX devices does not allow for specific associations between
DAX device and extent.  This keeps allocations very similar to existing
DAX region behavior.

Great care was taken to keep the extent tracking simple.  Some xarray's
needed to be added but extra software objects were kept to a minimum.

Region extents continue to be tracked as sub-devices of the DAX region.
This ensures that region destruction cleans up all extent allocations
properly.

Due to these major changes all reviews were removed from the larger
patches.  A few of the straight forward patches have kept the tags.

In summary the major functionality of this series includes:

- Getting the dynamic capacity (DC) configuration information from cxl
  devices

- Configuring the DC partitions reported by hardware

- Enhancing the CXL and DAX regions for dynamic capacity support
	a. Maintain a logical separation between hardware extents and
	   software managed region extents.  This provides an
	   abstraction between the layers and should allow for
	   interleaving in the future

- Get hardware extent lists for endpoint decoders upon
  region creation.

- Adjust extent/region memory available on the following events.
        a. Add capacity Events
	b. Release capacity events

- Host response for add capacity
	a. do not accept the extent if:
		If the region does not exist
		or an error occurs realizing the extent
	b. If the region does exist
		realize a DAX region extent with 1:1 mapping (no
		interleave yet)
	c. Support the more bit by processing a list of extents marked
	   with the more bit together before setting up a response.

- Host response for remove capacity
	a. If no DAX device references the extent; release the extent
	b. If a reference does exist, ignore the request.
	   (Require FM to issue release again.)

- Modify DAX device creation/resize to account for extents within a
  sparse DAX region

- Trace Dynamic Capacity events for debugging

- Add cxl-test infrastructure to allow for faster unit testing
  (See new ndctl branch for cxl-dcd.sh test[1])

Fan Ni's upstream of Qemu DCD was used for testing.

Remaining work:

	1) Integrate the QoS work from Dave Jiang
	2) Interleave support

Possible additional work depending on requirements:

	1) Allow mapping to specific extents (perhaps based on
	   label/tag)
	2) Release extents when DAX devices are released if a release
	   was previously seen from the device
	3) Accept a new extent which extends (but overlaps) an existing
	   extent(s)
	4) Rework DAX device interfaces, memfd has been explored a bit

[0] v1: https://lore.kernel.org/all/20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com/
[1] https://github.com/weiny2/ndctl/tree/dcd-region2-2024-08-15

---
Major changes:
- Jonathan: support the more bit
- djbw: Allow more than 1 region per DC partition
- All: Address the many comments on the series.
- iweiny: rebase
- iweiny: Rework the series to make it easier to review and understand
          the flow
- Link to v1: https://lore.kernel.org/r/20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com

---
Ira Weiny (13):
      range: Add range_overlaps()
      printk: Add print format (%par) for struct range
      dax: Document dax dev range tuple
      cxl/pci: Delay event buffer allocation
      cxl/region: Refactor common create region code
      cxl/events: Split event msgnum configuration from irq setup
      cxl/pci: Factor out interrupt policy check
      cxl/core: Return endpoint decoder information from region search
      dax/bus: Factor out dev dax resize logic
      dax/region: Create resources on sparse DAX regions
      cxl/region: Read existing extents on region creation
      tools/testing/cxl: Make event logs dynamic
      tools/testing/cxl: Add DC Regions to mock mem data

Navneet Singh (12):
      cxl/mbox: Flag support for Dynamic Capacity Devices (DCD)
      cxl/mem: Read dynamic capacity configuration from the device
      cxl/core: Separate region mode from decoder mode
      cxl/region: Add dynamic capacity decoder and region modes
      cxl/hdm: Add dynamic capacity size support to endpoint decoders
      cxl/port: Add endpoint decoder DC mode support to sysfs
      cxl/mem: Expose DCD partition capabilities in sysfs
      cxl/region: Add sparse DAX region support
      cxl/mem: Configure dynamic capacity interrupts
      cxl/extent: Process DCD events and realize region extents
      cxl/region/extent: Expose region extent information in sysfs
      cxl/mem: Trace Dynamic capacity Event Record

 Documentation/ABI/testing/sysfs-bus-cxl   |  68 ++-
 Documentation/core-api/printk-formats.rst |  14 +
 drivers/cxl/core/Makefile                 |   2 +-
 drivers/cxl/core/core.h                   |  33 +-
 drivers/cxl/core/extent.c                 | 467 ++++++++++++++
 drivers/cxl/core/hdm.c                    | 206 ++++++-
 drivers/cxl/core/mbox.c                   | 578 +++++++++++++++++-
 drivers/cxl/core/memdev.c                 | 101 ++-
 drivers/cxl/core/port.c                   |  13 +-
 drivers/cxl/core/region.c                 | 173 ++++--
 drivers/cxl/core/trace.h                  |  65 ++
 drivers/cxl/cxl.h                         | 122 +++-
 drivers/cxl/cxlmem.h                      | 128 +++-
 drivers/cxl/pci.c                         | 123 +++-
 drivers/dax/bus.c                         | 352 +++++++++--
 drivers/dax/bus.h                         |   4 +-
 drivers/dax/cxl.c                         |  73 ++-
 drivers/dax/dax-private.h                 |  39 +-
 drivers/dax/hmem/hmem.c                   |   2 +-
 drivers/dax/pmem.c                        |   2 +-
 fs/btrfs/ordered-data.c                   |  10 +-
 include/linux/cxl-event.h                 |  32 +
 include/linux/range.h                     |   7 +
 lib/vsprintf.c                            |  37 ++
 tools/testing/cxl/Kbuild                  |   3 +-
 tools/testing/cxl/test/mem.c              | 981 ++++++++++++++++++++++++++----
 26 files changed, 3327 insertions(+), 308 deletions(-)
---
base-commit: 3cef9316df4cda21b5bf25e4230221b02050dfa1
change-id: 20230604-dcd-type2-upstream-0cd15f6216fd

Best regards,
-- 
Ira Weiny <ira.weiny@intel.com>


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH v2 02/25] printk: Add print format (%par) for struct range
  2024-08-16 13:59 [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny
@ 2024-08-16 13:59 ` Ira Weiny
  2024-08-16 14:17 ` [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny
  1 sibling, 0 replies; 3+ messages in thread
From: Ira Weiny @ 2024-08-16 13:59 UTC (permalink / raw)
  To: Dave Jiang, Fan Ni, Jonathan Cameron, Navneet Singh
  Cc: Dan Williams, Davidlohr Bueso, Alison Schofield, Vishal Verma,
	Ira Weiny, linux-btrfs, linux-cxl, linux-kernel, Petr Mladek,
	Steven Rostedt, Jonathan Corbet, open list:DOCUMENTATION

The use of struct range in the CXL subsystem is growing.  In particular,
the addition of Dynamic Capacity devices uses struct range in a number
of places which are reported in debug and error messages.

To wit requiring the printing of the start/end fields in each print
became cumbersome.  Dan Williams mentions in [1] that it might be time
to have a print specifier for struct range similar to struct resource

A few alternatives were considered including '%pn' for 'print raNge' but
%par follows that struct range is most often used to store a range of
physical addresses.  So use '%par' for 'print address range'.

To: Petr Mladek <pmladek@suse.com> (maintainer:VSPRINTF)
To: Steven Rostedt <rostedt@goodmis.org> (maintainer:VSPRINTF)
To: Jonathan Corbet <corbet@lwn.net> (maintainer:DOCUMENTATION)
Cc: linux-doc@vger.kernel.org (open list:DOCUMENTATION)
Cc: linux-kernel@vger.kernel.org (open list)
Link: https://lore.kernel.org/all/663922b475e50_d54d72945b@dwillia2-xfh.jf.intel.com.notmuch/ [1]
Suggested-by: "Dan Williams" <dan.j.williams@intel.com>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
 Documentation/core-api/printk-formats.rst | 14 ++++++++++++
 lib/vsprintf.c                            | 37 +++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+)

diff --git a/Documentation/core-api/printk-formats.rst b/Documentation/core-api/printk-formats.rst
index 4451ef501936..a02ef899b2a6 100644
--- a/Documentation/core-api/printk-formats.rst
+++ b/Documentation/core-api/printk-formats.rst
@@ -231,6 +231,20 @@ width of the CPU data path.
 
 Passed by reference.
 
+Struct Range
+------------
+
+::
+
+	%par	[range 0x60000000-0x6fffffff] or
+		[range 0x0000000060000000-0x000000006fffffff]
+
+For printing struct range.  A variation of printing a physical address is to
+print the value of struct range which are often used to hold a physical address
+range.
+
+Passed by reference.
+
 DMA address types dma_addr_t
 ----------------------------
 
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 2d71b1115916..c132178fac07 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -1140,6 +1140,39 @@ char *resource_string(char *buf, char *end, struct resource *res,
 	return string_nocheck(buf, end, sym, spec);
 }
 
+static noinline_for_stack
+char *range_string(char *buf, char *end, const struct range *range,
+		      struct printf_spec spec, const char *fmt)
+{
+#define RANGE_PRINTK_SIZE		16
+#define RANGE_DECODED_BUF_SIZE		((2 * sizeof(struct range)) + 4)
+#define RANGE_PRINT_BUF_SIZE		sizeof("[range - ]")
+	char sym[RANGE_DECODED_BUF_SIZE + RANGE_PRINT_BUF_SIZE];
+	char *p = sym, *pend = sym + sizeof(sym);
+
+	static const struct printf_spec str_spec = {
+		.field_width = -1,
+		.precision = 10,
+		.flags = LEFT,
+	};
+	static const struct printf_spec range_spec = {
+		.base = 16,
+		.field_width = RANGE_PRINTK_SIZE,
+		.precision = -1,
+		.flags = SPECIAL | SMALL | ZEROPAD,
+	};
+
+	*p++ = '[';
+	p = string_nocheck(p, pend, "range ", str_spec);
+	p = number(p, pend, range->start, range_spec);
+	*p++ = '-';
+	p = number(p, pend, range->end, range_spec);
+	*p++ = ']';
+	*p = '\0';
+
+	return string_nocheck(buf, end, sym, spec);
+}
+
 static noinline_for_stack
 char *hex_string(char *buf, char *end, u8 *addr, struct printf_spec spec,
 		 const char *fmt)
@@ -1802,6 +1835,8 @@ char *address_val(char *buf, char *end, const void *addr,
 		return buf;
 
 	switch (fmt[1]) {
+	case 'r':
+		return range_string(buf, end, addr, spec, fmt);
 	case 'd':
 		num = *(const dma_addr_t *)addr;
 		size = sizeof(dma_addr_t);
@@ -2364,6 +2399,8 @@ char *rust_fmt_argument(char *buf, char *end, void *ptr);
  *            to use print_hex_dump() for the larger input.
  * - 'a[pd]' For address types [p] phys_addr_t, [d] dma_addr_t and derivatives
  *           (default assumed to be phys_addr_t, passed by reference)
+ * - 'ar' For decoded struct ranges (a variation of physical address which are
+ *        most often stored in struct ranges.
  * - 'd[234]' For a dentry name (optionally 2-4 last components)
  * - 'D[234]' Same as 'd' but for a struct file
  * - 'g' For block_device name (gendisk + partition number)

-- 
2.45.2


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD)
  2024-08-16 13:59 [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny
  2024-08-16 13:59 ` [PATCH v2 02/25] printk: Add print format (%par) for struct range Ira Weiny
@ 2024-08-16 14:17 ` Ira Weiny
  1 sibling, 0 replies; 3+ messages in thread
From: Ira Weiny @ 2024-08-16 14:17 UTC (permalink / raw)
  To: Ira Weiny, Dave Jiang, Fan Ni, Jonathan Cameron, Navneet Singh
  Cc: Dan Williams, Davidlohr Bueso, Alison Schofield, Vishal Verma,
	Ira Weiny, linux-btrfs, linux-cxl, linux-kernel, Chris Mason,
	Josef Bacik, David Sterba, Johannes Thumshirn, Petr Mladek,
	Steven Rostedt, Jonathan Corbet, open list:DOCUMENTATION,
	Li, Ming, Jonathan Cameron

Please ignore this series __and__ the RESEND.

The series did not get sent properly.  Something went wrong with my smtp
server in the middle.

  [PATCH v2 22/25] cxl/region: Read existing extents on region creation
CRITICAL: Error running /usr/bin/msmtp -i: msmtp: cannot locate host smtpauth.intel.com: No address associated with hostname
msmtp: could not send mail (account default from /home/iweiny/.msmtprc)

Then I used b4 --resend v2.  But glossed over the fact that it was going
to do something very bad and send a very old version.

https://lore.kernel.org/all/20240816-dcd-type2-upstream-v2-0-b4044aadf2bd@intel.com/

So please ignore that too.  :-(

At this point I'm going to send v3.


<fingers crossed>
Ira



Ira Weiny wrote:
> A git tree of this series can be found here:
> 
> 	https://github.com/weiny2/linux-kernel/tree/dcd-v4-2024-08-15
> 
> This series requires the CXL memory notifier lock change:
> 
> 	https://lore.kernel.org/all/20240814-fix-notifiers-v2-1-6bab38192c7c@intel.com/
> 
> Background
> ==========
> 
> A Dynamic Capacity Device (DCD) (CXL 3.1 sec 9.13.3) is a CXL memory
> device that allows memory capacity within a region to change
> dynamically without the need for resetting the device, reconfiguring
> HDM decoders, or reconfiguring software DAX regions.
> 
> One of the biggest use cases for Dynamic Capacity is to allow hosts to
> share memory dynamically within a data center without increasing the
> per-host attached memory.
> 
> The general flow for the addition or removal of memory is to have an
> orchestrator coordinate the use of the memory.  Generally there are 5
> actors in such a system, the Orchestrator, Fabric Manager, the Logical
> device, the Host Kernel, and a Host User.
> 
> Typical work flows are shown below.
> 
> Orchestrator      FM         Device       Host Kernel    Host User
> 
>     |             |           |            |              |
>     |-------------- Create region ----------------------->|
>     |             |           |            |              |
>     |             |           |            |<-- Create ---|
>     |             |           |            |    Region    |
>     |<------------- Signal done --------------------------|
>     |             |           |            |              |
>     |-- Add ----->|-- Add --->|--- Add --->|              |
>     |  Capacity   |  Extent   |   Extent   |              |
>     |             |           |            |              |
>     |             |<- Accept -|<- Accept  -|              |
>     |             |   Extent  |   Extent   |              |
>     |             |           |            |<- Create --->|
>     |             |           |            |   DAX dev    |-- Use memory
>     |             |           |            |              |   |
>     |             |           |            |              |   |
>     |             |           |            |<- Release ---| <-+
>     |             |           |            |   DAX dev    |
>     |             |           |            |              |
>     |<------------- Signal done --------------------------|
>     |             |           |            |              |
>     |-- Remove -->|- Release->|- Release ->|              |
>     |  Capacity   |  Extent   |   Extent   |              |
>     |             |           |            |              |
>     |             |<- Release-|<- Release -|              |
>     |             |   Extent  |   Extent   |              |
>     |             |           |            |              |
>     |-- Add ----->|-- Add --->|--- Add --->|              |
>     |  Capacity   |  Extent   |   Extent   |              |
>     |             |           |            |              |
>     |             |<- Accept -|<- Accept  -|              |
>     |             |   Extent  |   Extent   |              |
>     |             |           |            |<- Create ----|
>     |             |           |            |   DAX dev    |-- Use memory
>     |             |           |            |              |   |
>     |             |           |            |<- Release ---| <-+
>     |             |           |            |   DAX dev    |
>     |<------------- Signal done --------------------------|
>     |             |           |            |              |
>     |-- Remove -->|- Release->|- Release ->|              |
>     |  Capacity   |  Extent   |   Extent   |              |
>     |             |           |            |              |
>     |             |<- Release-|<- Release -|              |
>     |             |   Extent  |   Extent   |              |
>     |             |           |            |              |
>     |-- Add ----->|-- Add --->|--- Add --->|              |
>     |  Capacity   |  Extent   |   Extent   |              |
>     |             |           |            |<- Create ----|
>     |             |           |            |   DAX dev    |-- Use memory
>     |             |           |            |              |   |
>     |-- Remove -->|- Release->|- Release ->|              |   |
>     |  Capacity   |  Extent   |   Extent   |              |   |
>     |             |           |            |              |   |
>     |             |           |     (Release Ignored)     |   |
>     |             |           |            |              |   |
>     |             |           |            |<- Release ---| <-+
>     |             |           |            |   DAX dev    |
>     |<------------- Signal done --------------------------|
>     |             |           |            |              |
>     |             |- Release->|- Release ->|              |
>     |             |  Extent   |   Extent   |              |
>     |             |           |            |              |
>     |             |<- Release-|<- Release -|              |
>     |             |   Extent  |   Extent   |              |
>     |             |           |            |<- Destroy ---|
>     |             |           |            |   Region     |
>     |             |           |            |              |
> 
> Previous versions of this series[0] resulted in architectural comments
> as well as confusion on the architecture based on the organization of
> patch series itself.
> 
> This version has reordered the patches to clarify the architecture.
> It also streamlines extent handling more.
> 
> The series still requires the creation of regions and DAX devices to be
> synchronized with the Orchestrator and Fabric Manager.  The host kernel
> will reject an add extent event if the region is not created yet.  It
> will also ignore a release if the DAX device is created and referencing
> an extent.
> 
> These synchronizations are not anticipated to be an issue with real
> applications.
> 
> In order to allow for capacity to be added and removed a new concept of
> a sparse DAX region is introduced.  A sparse DAX region may have 0 or
> more bytes of available space.  The total space depends on the number
> and size of the extents which have been added.
> 
> Initially it is anticipated that users of the memory will carefully
> coordinate the surfacing of additional capacity with the creation of DAX
> devices which use that capacity.  Therefore, the allocation of the
> memory to DAX devices does not allow for specific associations between
> DAX device and extent.  This keeps allocations very similar to existing
> DAX region behavior.
> 
> Great care was taken to keep the extent tracking simple.  Some xarray's
> needed to be added but extra software objects were kept to a minimum.
> 
> Region extents continue to be tracked as sub-devices of the DAX region.
> This ensures that region destruction cleans up all extent allocations
> properly.
> 
> Due to these major changes all reviews were removed from the larger
> patches.  A few of the straight forward patches have kept the tags.
> 
> In summary the major functionality of this series includes:
> 
> - Getting the dynamic capacity (DC) configuration information from cxl
>   devices
> 
> - Configuring the DC partitions reported by hardware
> 
> - Enhancing the CXL and DAX regions for dynamic capacity support
> 	a. Maintain a logical separation between hardware extents and
> 	   software managed region extents.  This provides an
> 	   abstraction between the layers and should allow for
> 	   interleaving in the future
> 
> - Get hardware extent lists for endpoint decoders upon
>   region creation.
> 
> - Adjust extent/region memory available on the following events.
>         a. Add capacity Events
> 	b. Release capacity events
> 
> - Host response for add capacity
> 	a. do not accept the extent if:
> 		If the region does not exist
> 		or an error occurs realizing the extent
> 	b. If the region does exist
> 		realize a DAX region extent with 1:1 mapping (no
> 		interleave yet)
> 	c. Support the more bit by processing a list of extents marked
> 	   with the more bit together before setting up a response.
> 
> - Host response for remove capacity
> 	a. If no DAX device references the extent; release the extent
> 	b. If a reference does exist, ignore the request.
> 	   (Require FM to issue release again.)
> 
> - Modify DAX device creation/resize to account for extents within a
>   sparse DAX region
> 
> - Trace Dynamic Capacity events for debugging
> 
> - Add cxl-test infrastructure to allow for faster unit testing
>   (See new ndctl branch for cxl-dcd.sh test[1])
> 
> Fan Ni's upstream of Qemu DCD was used for testing.
> 
> Remaining work:
> 
> 	1) Integrate the QoS work from Dave Jiang
> 	2) Interleave support
> 
> Possible additional work depending on requirements:
> 
> 	1) Allow mapping to specific extents (perhaps based on
> 	   label/tag)
> 	2) Release extents when DAX devices are released if a release
> 	   was previously seen from the device
> 	3) Accept a new extent which extends (but overlaps) an existing
> 	   extent(s)
> 	4) Rework DAX device interfaces, memfd has been explored a bit
> 
> [0] v1: https://lore.kernel.org/all/20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com/
> [1] https://github.com/weiny2/ndctl/tree/dcd-region2-2024-08-15
> 
> ---
> Major changes:
> - Jonathan: support the more bit
> - djbw: Allow more than 1 region per DC partition
> - All: Address the many comments on the series.
> - iweiny: rebase
> - iweiny: Rework the series to make it easier to review and understand
>           the flow
> - Link to v1: https://lore.kernel.org/r/20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com
> 
> ---
> Ira Weiny (13):
>       range: Add range_overlaps()
>       printk: Add print format (%par) for struct range
>       dax: Document dax dev range tuple
>       cxl/pci: Delay event buffer allocation
>       cxl/region: Refactor common create region code
>       cxl/events: Split event msgnum configuration from irq setup
>       cxl/pci: Factor out interrupt policy check
>       cxl/core: Return endpoint decoder information from region search
>       dax/bus: Factor out dev dax resize logic
>       dax/region: Create resources on sparse DAX regions
>       cxl/region: Read existing extents on region creation
>       tools/testing/cxl: Make event logs dynamic
>       tools/testing/cxl: Add DC Regions to mock mem data
> 
> Navneet Singh (12):
>       cxl/mbox: Flag support for Dynamic Capacity Devices (DCD)
>       cxl/mem: Read dynamic capacity configuration from the device
>       cxl/core: Separate region mode from decoder mode
>       cxl/region: Add dynamic capacity decoder and region modes
>       cxl/hdm: Add dynamic capacity size support to endpoint decoders
>       cxl/port: Add endpoint decoder DC mode support to sysfs
>       cxl/mem: Expose DCD partition capabilities in sysfs
>       cxl/region: Add sparse DAX region support
>       cxl/mem: Configure dynamic capacity interrupts
>       cxl/extent: Process DCD events and realize region extents
>       cxl/region/extent: Expose region extent information in sysfs
>       cxl/mem: Trace Dynamic capacity Event Record
> 
>  Documentation/ABI/testing/sysfs-bus-cxl   |  68 ++-
>  Documentation/core-api/printk-formats.rst |  14 +
>  drivers/cxl/core/Makefile                 |   2 +-
>  drivers/cxl/core/core.h                   |  33 +-
>  drivers/cxl/core/extent.c                 | 467 ++++++++++++++
>  drivers/cxl/core/hdm.c                    | 206 ++++++-
>  drivers/cxl/core/mbox.c                   | 578 +++++++++++++++++-
>  drivers/cxl/core/memdev.c                 | 101 ++-
>  drivers/cxl/core/port.c                   |  13 +-
>  drivers/cxl/core/region.c                 | 173 ++++--
>  drivers/cxl/core/trace.h                  |  65 ++
>  drivers/cxl/cxl.h                         | 122 +++-
>  drivers/cxl/cxlmem.h                      | 128 +++-
>  drivers/cxl/pci.c                         | 123 +++-
>  drivers/dax/bus.c                         | 352 +++++++++--
>  drivers/dax/bus.h                         |   4 +-
>  drivers/dax/cxl.c                         |  73 ++-
>  drivers/dax/dax-private.h                 |  39 +-
>  drivers/dax/hmem/hmem.c                   |   2 +-
>  drivers/dax/pmem.c                        |   2 +-
>  fs/btrfs/ordered-data.c                   |  10 +-
>  include/linux/cxl-event.h                 |  32 +
>  include/linux/range.h                     |   7 +
>  lib/vsprintf.c                            |  37 ++
>  tools/testing/cxl/Kbuild                  |   3 +-
>  tools/testing/cxl/test/mem.c              | 981 ++++++++++++++++++++++++++----
>  26 files changed, 3327 insertions(+), 308 deletions(-)
> ---
> base-commit: 3cef9316df4cda21b5bf25e4230221b02050dfa1
> change-id: 20230604-dcd-type2-upstream-0cd15f6216fd
> 
> Best regards,
> -- 
> Ira Weiny <ira.weiny@intel.com>
> 



^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2024-08-16 14:17 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-16 13:59 [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny
2024-08-16 13:59 ` [PATCH v2 02/25] printk: Add print format (%par) for struct range Ira Weiny
2024-08-16 14:17 ` [PATCH v2 00/25] DCD: Add support for Dynamic Capacity Devices (DCD) Ira Weiny

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox