Re: [PATCH V5 2/2] Add test/daxctl-famfs.sh to test famfs mode transitions:

[Alison Schofield <alison.schofield@intel.com>]

On Thu, Apr 30, 2026 at 03:34:18PM +0000, John Groves wrote:
> From: John Groves <John@Groves.net>

Hi John,

Thanks for the famfs mode-transition test. As promised, I took a look
at how we can make it work as a unit test in an NDCTL test environment.

The main blocker is the device-selection assumption:
    region_id="$("$DAXCTL" list -R |
        jq -r ".[] | select(.path | contains(\"cxl_acpi.0\")) | .id")"

and then the test grabs whatever happens to be the first dax device on
that region. That works in your QEMU setup but not in the ndctl unit-test
model. NDCTL tests here are always tied to one of the emulated backends
(nfit_test or cxl_test) and build their own test device from a known
starting state. They don't scan for "an existing dax device" because there
is no such thing in a fresh `meson test` run. Whatever is there may be
leftover state from something else.

So the test as written will either skip (no dax device found) or get
tangled up with whatever prior test left the system in. Traditionally
DAX has used nfit for its emulated backing, but CXL is a reasonable
choice too. I've prepared both examples and appended to the end of
this message.

test/daxctl-famfs-nfit.sh
    ACPI.NFIT-backed testdev with full devdax/famfs/system-ram coverage

test/daxctl-famfs-cxltest.sh
    cxl_test-backed testdev with devdax/famfs only, plus the 
    system-ram -> famfs rejection via -N

The cxl_test version cannot exercise kmem onlining because the memory
has no real DRAM backing.

My preference is the nfit version since it exercises the full transition
matrix end-to-end, but if FAMFS is primarily a CXL story the cxl_test
version is the equivalent. And of course, there is no lock-down here,
ie. we can start with nfit and then when DCD support comes around and
that is in cxl-test, we can switch completely to CXL or use both nfit
and CXL. But - you do need to pick a lane to start in.

Beyond selecting a backend, both versions were also aligned with the
existing test style:
- Drop per-step narration as set -x logs commands
- Only emit failure path messages
- Replaced printf with echo 
- Suppress reconfigure-device JSON output with >/dev/null.
- Remove restore-to-original-mode cleanup logic; rely on fixture
  teardown instead.
- Use err helpers for traps and cleanup
- Extend ensure_devdax_mode to handle system-ram start state
- Add check_dmesg "$LINENO" at test completion.

Both versions have been tested, but please verify that the exact
coverage and transition matrix still match what you want after my
editing.

Please post the selected version as part of the next revision of
this series

Patches appended below...

-- Alison

======

test/daxctl-famfs-nfit.sh: test famfs mode transitions on nfit_test

Exercise devdax <-> famfs <-> system-ram transitions and JSON mode
reporting on a dax device backed by the emulated ACPI.NFIT bus.

nfit_test was chosen because its pmem ranges have real DRAM backing, so
kmem onlining of the system-ram mode works normally and the full matrix
of transitions can be tested end-to-end.
---
 test/daxctl-famfs-nfit.sh | 215 ++++++++++++++++++++++++++++++++++++++
 test/meson.build          |   4 +-
 2 files changed, 218 insertions(+), 1 deletion(-)
 create mode 100755 test/daxctl-famfs-nfit.sh

diff --git a/test/daxctl-famfs-nfit.sh b/test/daxctl-famfs-nfit.sh
new file mode 100755
index 000000000000..57302795c89f
--- /dev/null
+++ b/test/daxctl-famfs-nfit.sh
@@ -0,0 +1,215 @@
+#!/bin/bash -Ex
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2025 Micron Technology, Inc. All rights reserved.
+#
+# Test daxctl famfs mode transitions and mode detection, targeting a
+# nfit_test-backed dax device.
+#
+# nfit_test-backed dax devices have real DRAM backing, so kmem onlining
+# works normally. This test exercises the full matrix of transitions
+# between devdax, famfs, and system-ram.
+
+rc=77
+. $(dirname $0)/common
+
+trap 'cleanup $LINENO' ERR
+
+testbus=""
+testdev=""
+daxdev=""
+
+cleanup()
+{
+	# Best-effort return to devdax so destroy-namespace can succeed.
+	if [[ -n $daxdev ]]; then
+		"$DAXCTL" reconfigure-device -f -m devdax "$daxdev" 2>/dev/null || true
+	fi
+	[[ -n $testdev ]] && reset_dev
+	err "$1"
+}
+
+check_fsdev_dax()
+{
+	modinfo fsdev_dax &>/dev/null && return 0
+	grep -qF "fsdev_dax" "/lib/modules/$(uname -r)/modules.builtin" 2>/dev/null && return 0
+	do_skip "fsdev_dax module not available"
+}
+
+check_kmem()
+{
+	modinfo kmem &>/dev/null && return 0
+	grep -qF "kmem" "/lib/modules/$(uname -r)/modules.builtin" 2>/dev/null && return 0
+	do_skip "kmem module not available"
+}
+
+find_testdev()
+{
+	testbus="$ACPI_BUS"
+
+	# Ensure the bus has labels, like align.sh / daxctl-devices.sh rely on.
+	"$NDCTL" disable-region -b "$testbus" all
+	"$NDCTL" init-labels -f -b "$testbus" all
+	"$NDCTL" enable-region -b "$testbus" all
+
+	testdev=$("$NDCTL" list -b "$testbus" -Ni | jq -er '.[0].dev | .//""')
+	[[ $testdev ]] || do_skip "no victim device on $testbus"
+}
+
+setup_dev()
+{
+	test -n "$testbus"
+	test -n "$testdev"
+
+	"$NDCTL" destroy-namespace -f -b "$testbus" "$testdev"
+	# x86_64 memory hotplug can require up to a 2GiB-aligned chunk of
+	# memory. Create a 4GiB namespace, so enough space is left after
+	# alignment for kmem + online.
+	testdev=$("$NDCTL" create-namespace -b "$testbus" -m devdax -fe "$testdev" -s 4G | \
+		jq -er '.dev')
+	test -n "$testdev"
+
+	daxdev=$("$NDCTL" list -n "$testdev" -X | jq -er '.[].daxregion.devices[0].chardev')
+	test -n "$daxdev"
+}
+
+reset_dev()
+{
+	"$NDCTL" destroy-namespace -f -b "$testbus" "$testdev"
+}
+
+daxctl_get_mode()
+{
+	"$DAXCTL" list -d "$1" | jq -er '.[].mode'
+}
+
+save_online_policy()
+{
+	saved_policy="$(cat /sys/devices/system/memory/auto_online_blocks)"
+}
+
+restore_online_policy()
+{
+	echo "$saved_policy" > /sys/devices/system/memory/auto_online_blocks
+}
+
+unset_online_policy()
+{
+	echo "offline" > /sys/devices/system/memory/auto_online_blocks
+}
+
+ensure_devdax_mode()
+{
+	local mode
+	mode=$(daxctl_get_mode "$daxdev")
+
+	case "$mode" in
+	devdax)      return 0 ;;
+	famfs)       "$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null ;;
+	system-ram)  "$DAXCTL" reconfigure-device -f -m devdax "$daxdev" >/dev/null ;;
+	*)
+		echo "unexpected starting mode: $mode"
+		return 1
+		;;
+	esac
+
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+}
+
+test_famfs_mode_transitions()
+{
+	ensure_devdax_mode
+
+	# devdax -> famfs
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "famfs" ]]
+
+	# famfs -> famfs (re-enable in same mode)
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "famfs" ]]
+
+	# famfs -> devdax
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+
+	# devdax -> devdax (re-enable in same mode)
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+}
+
+test_json_output()
+{
+	ensure_devdax_mode
+	[[ $("$DAXCTL" list -d "$daxdev" | jq -er '.[].mode') == "devdax" ]]
+
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $("$DAXCTL" list -d "$daxdev" | jq -er '.[].mode') == "famfs" ]]
+
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+}
+
+test_error_handling()
+{
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+
+	# Invalid mode must be rejected
+	if "$DAXCTL" reconfigure-device -m invalidmode "$daxdev" &>/dev/null; then
+		echo "FAIL: invalid mode should be rejected"
+		return 1
+	fi
+
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+}
+
+# Full system-ram transitions (real backing, so online_pages() works).
+# Turns auto-online off so daxctl drives onlining explicitly.
+test_system_ram_transitions()
+{
+	save_online_policy
+	unset_online_policy
+
+	ensure_devdax_mode
+
+	# devdax -> system-ram (no-online)
+	"$DAXCTL" reconfigure-device -N -m system-ram "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "system-ram" ]]
+
+	# system-ram -> famfs must be rejected
+	if "$DAXCTL" reconfigure-device -m famfs "$daxdev" &>/dev/null; then
+		echo "FAIL: system-ram -> famfs should be rejected"
+		restore_online_policy
+		return 1
+	fi
+
+	# system-ram -> devdax -> famfs
+	"$DAXCTL" reconfigure-device -f -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "famfs" ]]
+
+	# Full online cycle: devdax -> system-ram (with online) -> devdax.
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+	"$DAXCTL" reconfigure-device -m system-ram "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "system-ram" ]]
+	"$DAXCTL" reconfigure-device -f -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+
+	restore_online_policy
+}
+
+check_fsdev_dax
+check_kmem
+
+rc=1
+
+find_testdev
+setup_dev
+
+test_famfs_mode_transitions
+test_json_output
+test_error_handling
+test_system_ram_transitions
+
+ensure_devdax_mode
+reset_dev
+
+check_dmesg "$LINENO"
diff --git a/test/meson.build b/test/meson.build
index 56aed9cc3c9d..d765fd99a4b1 100644
--- a/test/meson.build
+++ b/test/meson.build
@@ -218,6 +218,7 @@ if get_option('destructive').enabled()
   daxctl_devices = find_program('daxctl-devices.sh')
   daxctl_create = find_program('daxctl-create.sh')
   daxctl_famfs = find_program('daxctl-famfs.sh')
+  daxctl_famfs_nfit = find_program('daxctl-famfs-nfit.sh')
   dm = find_program('dm.sh')
   mmap_test = find_program('mmap.sh')
 
@@ -235,7 +236,8 @@ if get_option('destructive').enabled()
     [ 'device-dax-fio.sh', device_dax_fio, 'dax'   ],
     [ 'daxctl-devices.sh', daxctl_devices, 'dax'   ],
     [ 'daxctl-create.sh',  daxctl_create,  'dax'   ],
-    [ 'daxctl-famfs.sh',   daxctl_famfs,   'dax'   ],
+    [ 'daxctl-famfs.sh',        daxctl_famfs,       'dax' ],
+    [ 'daxctl-famfs-nfit.sh',   daxctl_famfs_nfit,  'dax' ],
     [ 'dm.sh',             dm,		   'dax'   ],
     [ 'mmap.sh',           mmap_test,	   'dax'   ],
   ]
-- 
2.37.3

=========

test/daxctl-famfs-cxltest.sh: test famfs mode transitions on cxl_test

Exercise devdax <-> famfs transitions and JSON mode reporting on a dax
device backed by the emulated cxl_test bus.

cxl_test was chosen as an alternative to nfit_test for callers who want
the famfs unit test to run against a CXL-backed dax device. Note that
cxl_test emulated memory has no real DRAM backing, so this test avoids
code paths that online memory through kmem; only the userspace-side
rejection of system-ram -> famfs is exercised (via -N, which skips
online_pages()).
---
 test/daxctl-famfs-cxltest.sh | 139 +++++++++++++++++++++++++++++++++++
 test/meson.build             |   4 +-
 2 files changed, 142 insertions(+), 1 deletion(-)
 create mode 100755 test/daxctl-famfs-cxltest.sh

diff --git a/test/daxctl-famfs-cxltest.sh b/test/daxctl-famfs-cxltest.sh
new file mode 100755
index 000000000000..aa85946c42ee
--- /dev/null
+++ b/test/daxctl-famfs-cxltest.sh
@@ -0,0 +1,139 @@
+#!/bin/bash -Ex
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2025 Micron Technology, Inc. All rights reserved.
+#
+# Test daxctl famfs mode transitions and mode detection, targeting a
+# cxl_test-backed dax device.
+#
+# NOTE: cxl_test backs its "memory" with a gen_pool region carved from high
+# iomem space above iomem_resource.end. There is no real DRAM behind those
+# pfns. Therefore this test does NOT exercise any code path that calls
+# online_pages() on cxl_test memory. In particular there is no "-m system-ram"
+# that runs without -N, and no explicit 'daxctl online-memory'.
+#
+# The 'system-ram -> famfs rejection' case below uses -N so that add_memory()
+# runs but online_pages() does not. The rejection itself is a userspace-only
+# check in reconfig_mode_famfs() (refuses when daxctl_memory is present), so
+# it does not depend on the memory being onlined and is safe here.
+
+rc=77
+. $(dirname $0)/common
+
+trap 'err $LINENO' ERR
+
+modprobe -r cxl_test
+modprobe cxl_test
+
+rc=1
+
+daxdev=""
+
+check_fsdev_dax()
+{
+	modinfo fsdev_dax &>/dev/null && return 0
+	grep -qF "fsdev_dax" "/lib/modules/$(uname -r)/modules.builtin" 2>/dev/null && return 0
+	do_skip "fsdev_dax module not available"
+}
+
+find_daxdev()
+{
+	region_id="$("$DAXCTL" list -R |
+		jq -r ".[] | select(.path | contains(\"cxl_acpi.0\")) | .id")"
+	[[ $region_id ]] || do_skip "Unable to find a CXL region"
+
+	daxdev=$("$DAXCTL" list -D -r "$region_id" |
+		jq -er '.[0].chardev | .//""')
+	[[ $daxdev ]] || do_skip "Unable to find a DAX device"
+}
+
+daxctl_get_mode()
+{
+	"$DAXCTL" list -d "$1" | jq -er '.[].mode'
+}
+
+ensure_devdax_mode()
+{
+	local mode
+	mode=$(daxctl_get_mode "$daxdev")
+
+	case "$mode" in
+	devdax)      return 0 ;;
+	famfs)       "$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null ;;
+	system-ram)  "$DAXCTL" reconfigure-device -f -m devdax "$daxdev" >/dev/null ;;
+	*)           err "$LINENO" ;;
+	esac
+
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+}
+
+test_famfs_mode_transitions()
+{
+	ensure_devdax_mode
+
+	# devdax -> famfs
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "famfs" ]]
+
+	# famfs -> famfs (re-enable in same mode)
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "famfs" ]]
+
+	# famfs -> devdax
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+
+	# devdax -> devdax (re-enable in same mode)
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+}
+
+test_json_output()
+{
+	ensure_devdax_mode
+	[[ $("$DAXCTL" list -d "$daxdev" | jq -er '.[].mode') == "devdax" ]]
+
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+	[[ $("$DAXCTL" list -d "$daxdev" | jq -er '.[].mode') == "famfs" ]]
+
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+}
+
+test_error_handling()
+{
+	"$DAXCTL" reconfigure-device -m famfs "$daxdev" >/dev/null
+
+	# Invalid mode must be rejected
+	if "$DAXCTL" reconfigure-device -m invalidmode "$daxdev" &>/dev/null; then
+		err "$LINENO"
+	fi
+
+	"$DAXCTL" reconfigure-device -m devdax "$daxdev" >/dev/null
+}
+
+test_system_ram_rejects_famfs()
+{
+	ensure_devdax_mode
+
+	# -N: bind kmem without onlining memory
+	"$DAXCTL" reconfigure-device -N -m system-ram "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "system-ram" ]]
+
+	if "$DAXCTL" reconfigure-device -m famfs "$daxdev" &>/dev/null; then
+		err "$LINENO"
+	fi
+
+	"$DAXCTL" reconfigure-device -f -m devdax "$daxdev" >/dev/null
+	[[ $(daxctl_get_mode "$daxdev") == "devdax" ]]
+}
+
+check_fsdev_dax
+find_daxdev
+
+test_famfs_mode_transitions
+test_json_output
+test_error_handling
+test_system_ram_rejects_famfs
+
+check_dmesg "$LINENO"
+
+modprobe -r cxl_test
diff --git a/test/meson.build b/test/meson.build
index d765fd99a4b1..ae5477408617 100644
--- a/test/meson.build
+++ b/test/meson.build
@@ -219,6 +219,7 @@ if get_option('destructive').enabled()
   daxctl_create = find_program('daxctl-create.sh')
   daxctl_famfs = find_program('daxctl-famfs.sh')
   daxctl_famfs_nfit = find_program('daxctl-famfs-nfit.sh')
+  daxctl_famfs_cxltest = find_program('daxctl-famfs-cxltest.sh')
   dm = find_program('dm.sh')
   mmap_test = find_program('mmap.sh')
 
@@ -237,7 +238,8 @@ if get_option('destructive').enabled()
     [ 'daxctl-devices.sh', daxctl_devices, 'dax'   ],
     [ 'daxctl-create.sh',  daxctl_create,  'dax'   ],
     [ 'daxctl-famfs.sh',        daxctl_famfs,       'dax' ],
-    [ 'daxctl-famfs-nfit.sh',   daxctl_famfs_nfit,  'dax' ],
+    [ 'daxctl-famfs-nfit.sh',     daxctl_famfs_nfit,     'dax' ],
+    [ 'daxctl-famfs-cxltest.sh',  daxctl_famfs_cxltest,  'dax' ],
     [ 'dm.sh',             dm,		   'dax'   ],
     [ 'mmap.sh',           mmap_test,	   'dax'   ],
   ]
-- 
2.37.3