Re: [PATCH 0/6] daxctl: use dax kmem atomic whole-device hotplug

[Alison Schofield <alison.schofield@intel.com>]

On Fri, Oct 16, 2026 at 05:42:42PM -0400, Gregory Price (Meta) wrote:
> Kernels with the dax_kmem daxX.Y/state attribute [1] can unplug or
> plug-and-online a whole device atomically. Use it in daxctl
> online-memory and offline-memory so other agents (udev, auto-online)
> can't race daxctl block by block, notably the offline-then-unbind race
> on teardown.
> 
> Kernels without the attribute keep the per-block path.
> 
> Behavior change: offline-memory now always unplugs the device (state
> 'unplugged'), removing its memory blocks, even if they were all
> offline already. Previously that was a no-op.
> 
> 'daxctl list' (and 'ndctl/cxl list -X') gains 'hotplug_state',
> reporting daxX.Y/state. The kernel only updates that when written, so
> it goes stale if blocks are toggled individually.
> 
> If there is a mismatch between 'dax/state' and the blocks, then
> 'hotplug_state' reports 'inconsistent' (online state / zones disagree).
> That is intentional: it flags agents fighting over the device.
> It only appears when the attribute exists.
> 
> Compatibility notes:
> - Where the unplug fails (e.g. no memory hotremove support),
>   online-memory falls back to the per-block path and offline-memory
>   of an already offline device remains a no-op.
> - Older daxctl/ndctl/cxl binaries don't expect a device with no memory
>   blocks.  Basically multiple daxctl versions on the same box can cause
>   older versions to fail when in the unplugged state.  Not a huge issue.
> - If re-adding fails after an unplug (e.g. region can't be reserved)
>   the device is left unplugged.
> - For multi-range devices the kernel succeeds if any range is added.
>   daxctl can't tell that a range is missing.
> 
> [1] https://lore.kernel.org/all/20260712154505.3564379-1-gourry@gourry.net/

Hi Greg,

Next version, please add the ndctl label, ie [ndctl PATCH v2 0/6]
(https://github.com/pmem/ndctl?tab=contributing-ov-file)

I see this goes with the kernel functionality that arrived in kernel 7.3.
That took me a bit to realized because the linked kernel series shows
crickets at v7.

I'd like to get this into the pending ndctl release, so let's see if we
can close on a few things.

I tested against 7.3, regression and new tests and all is good there.
I tested against 7.2 because daxctl command functionality itself needs to
be backward compatible, and it is!

Some per patch feedback and tags follow.

-- Alison

> 
> Gregory Price (6):
>   libdaxctl: add dax kmem atomic hotplug state accessors
>   daxctl: online whole-device memory atomically when available
>   daxctl: offline whole-device memory via atomic unplug when available
>   daxctl: report whole-device hotplug state in 'daxctl list'
>   Documentation: describe atomic whole-device hotplug in daxctl man
>     pages
>   test/daxctl-devices.sh: cycle offline/online to cover unplug and
>     replug
> 
>  Documentation/daxctl/daxctl-list.txt          |   5 +
>  .../daxctl/daxctl-offline-memory.txt          |   5 +
>  Documentation/daxctl/daxctl-online-memory.txt |   5 +
>  .../daxctl/daxctl-reconfigure-device.txt      |   5 +-
>  daxctl/device.c                               | 156 +++++++++++++++++-
>  daxctl/json.c                                 |  53 +++++-
>  daxctl/lib/libdaxctl-private.h                |   9 +
>  daxctl/lib/libdaxctl.c                        |  71 +++++++-
>  daxctl/lib/libdaxctl.sym                      |   6 +
>  daxctl/libdaxctl.h                            |  18 ++
>  test/daxctl-devices.sh                        |  53 ++++++
>  11 files changed, 379 insertions(+), 7 deletions(-)
> 
> -- 
> 2.53.0-Meta
> 
>