[PATCH blktests v3 0/4] blktests: add target test cases

[PATCH blktests v3 0/4] blktests: add target test cases

[Daniel Wagner]

As discussed in v2, the tests are only valid for tcp, rdma and fc
transport. Thus I updated the requires section accordingly. Also
addressed the typo and missing quotes.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
Changes in v3:
- Fixed typo in requires section
- limit tests to tcp, rdma and fc transport
- added missing quotes
- Link to v2: https://patch.msgid.link/20250408-test-target-v2-0-e9e2512586f8@kernel.org

Changes in v2:
- fixes shellcheck warnings
- removed dead code
- reworded comments
- updated requires section
- moved nvmf_wait_for_state to common code
- collected tags
- Link to v1: https://lore.kernel.org/r/20250318-test-target-v1-0-01e01142cf2b@kernel.org

---
Daniel Wagner (4):
      common/nvme: add debug nvmet path variable
      common/nvme: move nvmf_wait_for_state to common code
      nvme/060: add test nvme fabrics target resetting during I/O
      nvme/061: add test teardown and setup fabrics target during I/O

 common/nvme        | 29 ++++++++++++++++++++++++
 tests/nvme/048     | 31 ++-----------------------
 tests/nvme/060     | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/060.out |  2 ++
 tests/nvme/061     | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/061.out | 21 +++++++++++++++++
 6 files changed, 182 insertions(+), 29 deletions(-)
---
base-commit: 236edfd5d892f0abb0747f2668d1b9734349e2f6
change-id: 20250318-test-target-0d63599d94b0

Best regards,
-- 
Daniel Wagner <wagi@kernel.org>

[PATCH blktests v3 1/4] common/nvme: add debug nvmet path variable

[Daniel Wagner]

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 common/nvme | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/common/nvme b/common/nvme
index 3761329d39e3763136f60a4751ad15de347f6e9b..04b49c2c1f9edc6de516398b537502fc30a92969 100644
--- a/common/nvme
+++ b/common/nvme
@@ -26,6 +26,8 @@ nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
 NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
 nvme_target_control="${NVME_TARGET_CONTROL:-}"
 NVMET_CFS="/sys/kernel/config/nvmet/"
+# shellcheck disable=SC2034
+NVMET_DFS="/sys/kernel/debug/nvmet/"
 nvme_trtype=${nvme_trtype:-}
 nvme_adrfam=${nvme_adrfam:-}
 

-- 
2.49.0

[PATCH blktests v3 2/4] common/nvme: move nvmf_wait_for_state to common code

[Daniel Wagner]

In preperation to add another tests which needs the nvmf_wait_for_state
function, move it to the common code base.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 common/nvme    | 27 +++++++++++++++++++++++++++
 tests/nvme/048 | 31 ++-----------------------------
 2 files changed, 29 insertions(+), 29 deletions(-)

diff --git a/common/nvme b/common/nvme
index 04b49c2c1f9edc6de516398b537502fc30a92969..68720ddc20cf3ed5cfe2841f9921321d9899ce0d 100644
--- a/common/nvme
+++ b/common/nvme
@@ -573,6 +573,33 @@ _nvmf_wait_for_ns() {
 	return 0
 }
 
+_nvmf_wait_for_state() {
+	local def_state_timeout=5
+	local subsys_name="$1"
+	local state="$2"
+	local timeout="${3:-$def_state_timeout}"
+	local nvmedev
+	local state_file
+	local start_time
+	local end_time
+
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
+	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
+
+	start_time=$(date +%s)
+	while ! grep -q "${state}" "${state_file}"; do
+		sleep 1
+		end_time=$(date +%s)
+		if (( end_time - start_time > timeout )); then
+			echo "expected state \"${state}\" not " \
+				"reached within ${timeout} seconds"
+			return 1
+		fi
+	done
+
+	return 0
+}
+
 _create_nvmet_ns() {
 	local subsysnqn="${def_subsysnqn}"
 	local nsid="${def_nsid}"
diff --git a/tests/nvme/048 b/tests/nvme/048
index bd41faef4e5431957f63184408c147b8ada1a8dd..afd9272c1a31b5a3d2df2e1ce9fe3268de768420 100755
--- a/tests/nvme/048
+++ b/tests/nvme/048
@@ -19,33 +19,6 @@ set_conditions() {
 	_set_nvme_trtype "$@"
 }
 
-nvmf_wait_for_state() {
-	local def_state_timeout=5
-	local subsys_name="$1"
-	local state="$2"
-	local timeout="${3:-$def_state_timeout}"
-	local nvmedev
-	local state_file
-	local start_time
-	local end_time
-
-	nvmedev=$(_find_nvme_dev "${subsys_name}")
-	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
-
-	start_time=$(date +%s)
-	while ! grep -q "${state}" "${state_file}"; do
-		sleep 1
-		end_time=$(date +%s)
-		if (( end_time - start_time > timeout )); then
-			echo "expected state \"${state}\" not " \
-				"reached within ${timeout} seconds"
-			return 1
-		fi
-	done
-
-	return 0
-}
-
 nvmf_check_queue_count() {
 	local subsys_name="$1"
 	local queue_count="$2"
@@ -87,7 +60,7 @@ set_qid_max() {
 
 	set_nvmet_attr_qid_max "${subsys_name}" "${qid_max}"
 	nvmf_check_queue_count "${subsys_name}" "${qid_max}" || return 1
-	nvmf_wait_for_state "${subsys_name}" "live" || return 1
+	_nvmf_wait_for_state "${subsys_name}" "live" || return 1
 
 	return 0
 }
@@ -106,7 +79,7 @@ test() {
 		_nvme_connect_subsys --keep-alive-tmo 1 \
 				     --reconnect-delay 2
 
-		if ! nvmf_wait_for_state "${def_subsysnqn}" "live" ; then
+		if ! _nvmf_wait_for_state "${def_subsysnqn}" "live" ; then
 			echo FAIL
 		else
 			set_qid_max "${def_subsysnqn}" 1 || echo FAIL

-- 
2.49.0

[PATCH blktests v3 3/4] nvme/060: add test nvme fabrics target resetting during I/O

[Daniel Wagner]

Newer kernel support to reset the target via the debugfs. Add a new test
case which exercises this interface.

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 tests/nvme/060     | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/060.out |  2 ++
 2 files changed, 64 insertions(+)

diff --git a/tests/nvme/060 b/tests/nvme/060
new file mode 100755
index 0000000000000000000000000000000000000000..aa048198708cc82e684aaf94c145ae88c4af3099
--- /dev/null
+++ b/tests/nvme/060
@@ -0,0 +1,62 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2025 Daniel Wagner, SUSE Labs
+#
+# Test nvme fabrics controller reset/disconnect/reconnect.
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme fabrics target reset"
+
+requires() {
+	_nvme_requires
+	_have_loop
+	_require_nvme_trtype tcp rdma fc
+	_have_kernel_option NVME_TARGET_DEBUGFS
+}
+
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
+nvmet_debug_trigger_reset() {
+	local nvmet_subsystem="$1"
+	local dfs_path="${NVMET_DFS}/${nvmet_subsystem}"
+
+	find "${dfs_path}" -maxdepth 1 -type d -name 'ctrl*' -exec sh -c 'echo "fatal" > "$1/state"' _ {} \;
+}
+
+nvmet_reset_loop() {
+	while true; do
+		nvmet_debug_trigger_reset "${def_subsysnqn}"
+		sleep 2
+	done
+}
+
+test() {
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	_nvmet_target_setup
+
+	nvmet_reset_loop &
+	reset_loop_pid=$!
+
+	# Reset the host in different states e.g when the host is in the
+	# connected or connecting state.
+	#
+	# The target reset is triggered with an even number timeout, while the
+	# host reconnects with an odd number timeout.
+	for ((i = 0; i <= 5; i++)); do
+		_nvme_connect_subsys --keep-alive-tmo 1 --reconnect-delay 1
+		sleep 3
+		_nvme_disconnect_subsys >> "$FULL" 2>&1
+	done
+
+	{ kill "${reset_loop_pid}"; wait; } &> /dev/null
+
+	_nvmet_target_cleanup
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/060.out b/tests/nvme/060.out
new file mode 100644
index 0000000000000000000000000000000000000000..517ff2dfcfd41c4088991e669af9fef52bde570b
--- /dev/null
+++ b/tests/nvme/060.out
@@ -0,0 +1,2 @@
+Running nvme/060
+Test complete

-- 
2.49.0

[PATCH blktests v3 4/4] nvme/061: add test teardown and setup fabrics target during I/O

[Daniel Wagner]

Add a new test case which forcefully removes the target and setup it
again.

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 tests/nvme/061     | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/061.out | 21 +++++++++++++++++
 2 files changed, 87 insertions(+)

diff --git a/tests/nvme/061 b/tests/nvme/061
new file mode 100755
index 0000000000000000000000000000000000000000..c22046a6f547ea3325e2f5bdd6fed807f445391d
--- /dev/null
+++ b/tests/nvme/061
@@ -0,0 +1,66 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2025 Daniel Wagner, SUSE Labs
+#
+# Test if the host keeps running IO when the target is forcefully removed and
+# added back.
+
+. tests/nvme/rc
+
+DESCRIPTION="test fabric target teardown and setup during I/O"
+TIMED=1
+
+requires() {
+	_nvme_requires
+	_have_loop
+	_have_fio
+	_require_nvme_trtype tcp rdma fc
+}
+
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
+test() {
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	local ns
+
+	_nvmet_target_setup
+
+	_nvme_connect_subsys --keep-alive-tmo 1 --reconnect-delay 1
+
+	ns=$(_find_nvme_ns "${def_subsys_uuid}")
+
+	_run_fio_rand_io --filename="/dev/${ns}" \
+		--group_reporting \
+		--time_based --runtime=1d &> /dev/null &
+	fio_pid=$!
+	sleep 1
+
+	nvmedev=$(_find_nvme_dev "${def_subsysnqn}")
+	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
+	for ((i = 0; i <= 5; i++)); do
+		echo "iteration $i"
+
+		_nvmet_target_cleanup
+
+		_nvmf_wait_for_state "${def_subsysnqn}" "connecting" || return 1
+		echo "state: $(cat "${state_file}")"
+
+		_nvmet_target_setup
+
+		_nvmf_wait_for_state "${def_subsysnqn}" "live" || return 1
+		echo "state: $(cat "${state_file}")"
+	done
+
+	{ kill "${fio_pid}"; wait; } &> /dev/null
+
+	_nvme_disconnect_subsys
+
+	_nvmet_target_cleanup
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/061.out b/tests/nvme/061.out
new file mode 100644
index 0000000000000000000000000000000000000000..75516abdac005854c2be165005c076ef8891c518
--- /dev/null
+++ b/tests/nvme/061.out
@@ -0,0 +1,21 @@
+Running nvme/061
+iteration 0
+state: connecting
+state: live
+iteration 1
+state: connecting
+state: live
+iteration 2
+state: connecting
+state: live
+iteration 3
+state: connecting
+state: live
+iteration 4
+state: connecting
+state: live
+iteration 5
+state: connecting
+state: live
+disconnected 1 controller(s)
+Test complete

-- 
2.49.0

Re: [PATCH blktests v3 0/4] blktests: add target test cases

[Shinichiro Kawasaki]

On Apr 14, 2025 / 16:05, Daniel Wagner wrote:
> As discussed in v2, the tests are only valid for tcp, rdma and fc
> transport. Thus I updated the requires section accordingly. Also
> addressed the typo and missing quotes.

Thank you for this v3 series! I applied it.

P.S. I noticed that some local variables are missing declarations, then I took
     the liberty to add them.

Re: [PATCH blktests v3 0/4] blktests: add target test cases

[Daniel Wagner]

On Tue, Apr 15, 2025 at 07:37:40AM +0000, Shinichiro Kawasaki wrote:
> P.S. I noticed that some local variables are missing declarations, then I took
>      the liberty to add them.

Thanks!