[PATCH blktests 0/3] blktests: add target test cases

[PATCH blktests 0/3] blktests: add target test cases

[Daniel Wagner]

For testing the nvmet-fcloop series I used these two test cases to
exercise the target setup/teardown code paths.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
Daniel Wagner (3):
      common/nvme: add debug nvmet path variable
      nvme/060: add test nvme fabrics target resetting during I/O
      nvme/061: add test teardown and setup fabrics target during I/O

 common/nvme        |  1 +
 tests/nvme/060     | 88 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/060.out |  2 ++
 tests/nvme/061     | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/061.out | 21 ++++++++++++
 5 files changed, 211 insertions(+)
---
base-commit: 236edfd5d892f0abb0747f2668d1b9734349e2f6
change-id: 20250318-test-target-0d63599d94b0

Best regards,
-- 
Daniel Wagner <wagi@kernel.org>

[PATCH blktests 1/3] common/nvme: add debug nvmet path variable

[Daniel Wagner]

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 common/nvme | 1 +
 1 file changed, 1 insertion(+)

diff --git a/common/nvme b/common/nvme
index 3761329d39e3763136f60a4751ad15de347f6e9b..a3176ecff2e5fbc0fbe09460c21e9f50c68d3bce 100644
--- a/common/nvme
+++ b/common/nvme
@@ -26,6 +26,7 @@ nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
 NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
 nvme_target_control="${NVME_TARGET_CONTROL:-}"
 NVMET_CFS="/sys/kernel/config/nvmet/"
+NVMET_DFS="/sys/kernel/debug/nvmet/"
 nvme_trtype=${nvme_trtype:-}
 nvme_adrfam=${nvme_adrfam:-}
 

-- 
2.48.1

[PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Daniel Wagner]

Newer kernel support to reset the target via the debugfs. Add a new test
case which exercises this interface.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 tests/nvme/060     | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/060.out |  2 ++
 2 files changed, 90 insertions(+)

diff --git a/tests/nvme/060 b/tests/nvme/060
new file mode 100755
index 0000000000000000000000000000000000000000..16a3c0d20274428ae175693109c694f42004a619
--- /dev/null
+++ b/tests/nvme/060
@@ -0,0 +1,88 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2025 Daniel Wagner, SUSE Labs
+#
+# Test nvme fabrics controller reset/disconnect/reconnect operation during I/O
+# This test is somewhat similar to test 032 but for fabrics controllers.
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme fabrics target resetting during I/O"
+
+requires() {
+	_nvme_requires
+	_have_loop
+	_have_fio
+	_require_nvme_trtype_is_fabrics
+}
+
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
+nvmet_debug_trigger_reset() {
+	local nvmet_subsystem="$1"
+	local dfs_path="${NVMET_DFS}/${nvmet_subsystem}"
+
+	find "${dfs_path}" -maxdepth 1 -type d -name 'ctrl*' -exec sh -c 'echo "fatal" > "$1/state"' _ {} \;
+}
+
+nvmf_wait_for_state() {
+	local def_state_timeout=5
+	local subsys_name="$1"
+	local state="$2"
+	local timeout="${3:-$def_state_timeout}"
+	local nvmedev
+	local state_file
+	local start_time
+	local end_time
+
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
+	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
+
+	start_time=$(date +%s)
+	while ! grep -q "${state}" "${state_file}"; do
+		sleep 1
+		end_time=$(date +%s)
+		if (( end_time - start_time > timeout )); then
+			echo "expected state \"${state}\" not " \
+				"reached within ${timeout} seconds"
+			return 1
+		fi
+	done
+
+	return 0
+}
+
+nvmet_reset_loop() {
+	while true; do
+		nvmet_debug_trigger_reset "${def_subsysnqn}"
+		sleep 2
+	done
+}
+
+test() {
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	_nvmet_target_setup
+
+	nvmet_reset_loop &
+	fio_pid=$!
+
+	# Try to trigger resets when the host is in different states connected,
+	# or connecting by calling target reset with an even number and the
+	# reconnect attempt with an uneven timeout.
+	for ((i = 0; i <= 5; i++)); do
+		_nvme_connect_subsys --keep-alive-tmo 1 --reconnect-delay 1
+		sleep 3
+		_nvme_disconnect_subsys >> "$FULL" 2>&1
+	done
+
+	{ kill "${fio_pid}"; wait; } &> /dev/null
+
+	_nvmet_target_cleanup
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/060.out b/tests/nvme/060.out
new file mode 100644
index 0000000000000000000000000000000000000000..517ff2dfcfd41c4088991e669af9fef52bde570b
--- /dev/null
+++ b/tests/nvme/060.out
@@ -0,0 +1,2 @@
+Running nvme/060
+Test complete

-- 
2.48.1

[PATCH blktests 3/3] nvme/061: add test teardown and setup fabrics target during I/O

[Daniel Wagner]

Add a new test case which forcefully removes the target and setup it
again.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 tests/nvme/061     | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/061.out | 21 ++++++++++++
 2 files changed, 120 insertions(+)

diff --git a/tests/nvme/061 b/tests/nvme/061
new file mode 100755
index 0000000000000000000000000000000000000000..b7a9ca216e6b71db209e80dc69dfd934618b54c2
--- /dev/null
+++ b/tests/nvme/061
@@ -0,0 +1,99 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2025 Daniel Wagner, SUSE Labs
+#
+# Test if the host keeps running IO when the target is forcefully removed and
+# recreated.
+
+. tests/nvme/rc
+
+DESCRIPTION="test teardown and setup fabrics target during I/O"
+TIMED=1
+
+requires() {
+	_nvme_requires
+	_have_loop
+	_have_fio
+	_require_nvme_trtype_is_fabrics
+}
+
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
+nvmf_wait_for_state() {
+	local def_state_timeout=5
+	local subsys_name="$1"
+	local state="$2"
+	local timeout="${3:-$def_state_timeout}"
+	local nvmedev
+	local state_file
+	local start_time
+	local end_time
+
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
+	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
+
+	start_time=$(date +%s)
+	while ! grep -q "${state}" "${state_file}"; do
+		sleep 1
+		end_time=$(date +%s)
+		if (( end_time - start_time > timeout )); then
+			echo "expected state \"${state}\" not " \
+				"reached within ${timeout} seconds"
+			return 1
+		fi
+	done
+
+	return 0
+}
+
+test() {
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	local ns
+
+	_nvmet_target_setup
+
+	local connect_args=""
+
+	if [[ "${nvme_trtype}" == "fc" ]]; then
+		connect_args="--ctrl-dev-loss 0"
+	fi
+
+	_nvme_connect_subsys --keep-alive-tmo 1 --reconnect-delay 1
+
+	ns=$(_find_nvme_ns "${def_subsys_uuid}")
+
+	_run_fio_rand_io --filename="/dev/${ns}" \
+		--group_reporting \
+		--time_based --runtime=1d &> /dev/null &
+	fio_pid=$!
+	sleep 1
+
+	nvmedev=$(_find_nvme_dev "${def_subsysnqn}")
+	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
+	for ((i = 0; i <= 5; i++)); do
+		echo "iteration $i"
+
+		_nvmet_target_cleanup
+
+		nvmf_wait_for_state "${def_subsysnqn}" "connecting" || return 1
+		echo "state: $(cat $state_file)"
+
+		_nvmet_target_setup
+
+		nvmf_wait_for_state "${def_subsysnqn}" "live" || return 1
+		echo "state: $(cat $state_file)"
+	done
+
+	{ kill "${fio_pid}"; wait; } &> /dev/null
+
+	_nvme_disconnect_subsys
+
+	_nvmet_target_cleanup
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/061.out b/tests/nvme/061.out
new file mode 100644
index 0000000000000000000000000000000000000000..75516abdac005854c2be165005c076ef8891c518
--- /dev/null
+++ b/tests/nvme/061.out
@@ -0,0 +1,21 @@
+Running nvme/061
+iteration 0
+state: connecting
+state: live
+iteration 1
+state: connecting
+state: live
+iteration 2
+state: connecting
+state: live
+iteration 3
+state: connecting
+state: live
+iteration 4
+state: connecting
+state: live
+iteration 5
+state: connecting
+state: live
+disconnected 1 controller(s)
+Test complete

-- 
2.48.1

Re: [PATCH blktests 1/3] common/nvme: add debug nvmet path variable

[Chaitanya Kulkarni]

On 3/18/25 03:38, Daniel Wagner wrote:
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>   common/nvme | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/common/nvme b/common/nvme
> index 3761329d39e3763136f60a4751ad15de347f6e9b..a3176ecff2e5fbc0fbe09460c21e9f50c68d3bce 100644
> --- a/common/nvme
> +++ b/common/nvme
> @@ -26,6 +26,7 @@ nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
>   NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
>   nvme_target_control="${NVME_TARGET_CONTROL:-}"
>   NVMET_CFS="/sys/kernel/config/nvmet/"
> +NVMET_DFS="/sys/kernel/debug/nvmet/"
>   nvme_trtype=${nvme_trtype:-}
>   nvme_adrfam=${nvme_adrfam:-}
>   
>

Looks good.

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>

-ck

Re: [PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Chaitanya Kulkarni]

On 3/18/25 03:39, Daniel Wagner wrote:
> Newer kernel support to reset the target via the debugfs. Add a new test
> case which exercises this interface.
>
> Signed-off-by: Daniel Wagner<wagi@kernel.org>

Looks useful to me given that its a different code path in the target.

do we have any testcaes similar for non-debugfs code path ?
(don't remember at this point)

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>

-ck

Re: [PATCH blktests 3/3] nvme/061: add test teardown and setup fabrics target during I/O

[Chaitanya Kulkarni]

On 3/18/25 03:39, Daniel Wagner wrote:
> Add a new test case which forcefully removes the target and setup it
> again.
>
> Signed-off-by: Daniel Wagner<wagi@kernel.org>

Looks good.

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>

-ck

Re: [PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Daniel Wagner]

On Tue, Mar 18, 2025 at 08:40:39PM +0000, Chaitanya Kulkarni wrote:
> On 3/18/25 03:39, Daniel Wagner wrote:
> > Newer kernel support to reset the target via the debugfs. Add a new test
> > case which exercises this interface.
> >
> > Signed-off-by: Daniel Wagner<wagi@kernel.org>
> 
> Looks useful to me given that its a different code path in the target.

One thing I forgot to add a check if the feature is available. I think
the only way is to setup a target and see if the relevant file shows
up...

> do we have any testcaes similar for non-debugfs code path ?
> (don't remember at this point)

There is not direct way to trigger a target reset via a nvme command
yet. The upcoming TP8028 brings cross controller reset.

We have an indirect way to trigger a reset, via changing the number of
queues the target supports (this test case is already there).

Re: [PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Chaitanya Kulkarni]

On 3/19/25 02:26, Daniel Wagner wrote:
> On Tue, Mar 18, 2025 at 08:40:39PM +0000, Chaitanya Kulkarni wrote:
>> On 3/18/25 03:39, Daniel Wagner wrote:
>>> Newer kernel support to reset the target via the debugfs. Add a new test
>>> case which exercises this interface.
>>>
>>> Signed-off-by: Daniel Wagner<wagi@kernel.org>
>> Looks useful to me given that its a different code path in the target.
> One thing I forgot to add a check if the feature is available. I think
> the only way is to setup a target and see if the relevant file shows
> up...

perhaps create a controller and in absence of NVMET_DFS skipping the 
test with right reason is a way to go ?

It will also be helpful if you just create a helper that will create
a controller that way any future tests that needs to check for
specific feature via configfs files can use it. That helper will also
get called from requires() in order to set the right SKIP_REASON.

>> do we have any testcaes similar for non-debugfs code path ?
>> (don't remember at this point)
> There is not direct way to trigger a target reset via a nvme command
> yet. The upcoming TP8028 brings cross controller reset.

It will be nice.

> We have an indirect way to trigger a reset, via changing the number of
> queues the target supports (this test case is already there).

ahh thanks..

-ck

Re: [PATCH blktests 1/3] common/nvme: add debug nvmet path variable

[Shinichiro Kawasaki]

On Mar 18, 2025 / 11:38, Daniel Wagner wrote:
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>  common/nvme | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/common/nvme b/common/nvme
> index 3761329d39e3763136f60a4751ad15de347f6e9b..a3176ecff2e5fbc0fbe09460c21e9f50c68d3bce 100644
> --- a/common/nvme
> +++ b/common/nvme
> @@ -26,6 +26,7 @@ nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
>  NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
>  nvme_target_control="${NVME_TARGET_CONTROL:-}"
>  NVMET_CFS="/sys/kernel/config/nvmet/"
> +NVMET_DFS="/sys/kernel/debug/nvmet/"

This causes the shellcheck warning SC2034.

  common/nvme:29:1: warning: NVMET_DFS appears unused. Verify use (or export if used externally). [SC2034]

I suggest to annotate "# shellcheck disable=SC2034" above the added line.

Re: [PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Shinichiro Kawasaki]

On Mar 18, 2025 / 11:39, Daniel Wagner wrote:
> Newer kernel support to reset the target via the debugfs. Add a new test
> case which exercises this interface.

I find the kernel commit 649fd41420a8 ("nvmet: add debugfs support") enables
this test. Looking at the kernel commit, I fount this test case depends on the
kernel config NVME_TARGET_DEBUGFS. So let's add,

 _have_kernel_option NVME_TARGET_DEBUGFS

in requires().

> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>  tests/nvme/060     | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/nvme/060.out |  2 ++
>  2 files changed, 90 insertions(+)
> 
> diff --git a/tests/nvme/060 b/tests/nvme/060
> new file mode 100755
> index 0000000000000000000000000000000000000000..16a3c0d20274428ae175693109c694f42004a619
> --- /dev/null
> +++ b/tests/nvme/060
> @@ -0,0 +1,88 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-3.0+
> +# Copyright (C) 2025 Daniel Wagner, SUSE Labs
> +#
> +# Test nvme fabrics controller reset/disconnect/reconnect operation during I/O
> +# This test is somewhat similar to test 032 but for fabrics controllers.
> +
> +. tests/nvme/rc
> +
> +DESCRIPTION="test nvme fabrics target resetting during I/O"
> +
> +requires() {
> +	_nvme_requires
> +	_have_loop
> +	_have_fio

I don't find fio command in this test case. Do I miss it?

> +	_require_nvme_trtype_is_fabrics
> +}
> +
> +set_conditions() {
> +	_set_nvme_trtype "$@"
> +}
> +
> +nvmet_debug_trigger_reset() {
> +	local nvmet_subsystem="$1"
> +	local dfs_path="${NVMET_DFS}/${nvmet_subsystem}"
> +
> +	find "${dfs_path}" -maxdepth 1 -type d -name 'ctrl*' -exec sh -c 'echo "fatal" > "$1/state"' _ {} \;
> +}
> +
> +nvmf_wait_for_state() {
> +	local def_state_timeout=5
> +	local subsys_name="$1"
> +	local state="$2"
> +	local timeout="${3:-$def_state_timeout}"
> +	local nvmedev
> +	local state_file
> +	local start_time
> +	local end_time
> +
> +	nvmedev=$(_find_nvme_dev "${subsys_name}")
> +	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
> +
> +	start_time=$(date +%s)
> +	while ! grep -q "${state}" "${state_file}"; do
> +		sleep 1
> +		end_time=$(date +%s)
> +		if (( end_time - start_time > timeout )); then
> +			echo "expected state \"${state}\" not " \
> +				"reached within ${timeout} seconds"
> +			return 1
> +		fi
> +	done
> +
> +	return 0
> +}

Is this function used in this test case?

Re: [PATCH blktests 3/3] nvme/061: add test teardown and setup fabrics target during I/O

[Shinichiro Kawasaki]

On Mar 18, 2025 / 11:39, Daniel Wagner wrote:
> Add a new test case which forcefully removes the target and setup it
> again.

This test case has a few shellcheck warnings.

tests/nvme/061:63:3: warning: connect_args appears unused. Verify use (or export if used externally). [SC2034]
tests/nvme/061:84:22: note: Double quote to prevent globbing and word splitting. [SC2086]
tests/nvme/061:89:22: note: Double quote to prevent globbing and word splitting. [SC2086]

When I ran the test case on my test system, I observed it failed. Is it
expected?

$ sudo ./check nvme/061
nvme/061 (tr=loop) (test teardown and setup fabrics target during I/O) [failed]
    runtime  8.567s  ...  8.896s
    --- tests/nvme/061.out      2025-04-03 18:21:40.829107999 +0900
    +++ /home/shin/Blktests/blktests/results/nodev_tr_loop/nvme/061.out.bad     2025-04-04 17:17:45.246929857 +0900
    @@ -1,21 +1,9 @@
     Running nvme/061
     iteration 0
    -state: connecting
    -state: live
    -iteration 1
    -state: connecting
    -state: live
    ...
    (Run 'diff -u tests/nvme/061.out /home/shin/Blktests/blktests/results/nodev_tr_loop/nvme/061.out.bad' to see the entire diff)
$ cat results/nodev_tr_loop/nvme/061.out.bad
Running nvme/061
iteration 0
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
grep: /sys/class/nvme-fabrics/ctl//state: No such file or directory
expected state "connecting" not  reached within 5 seconds

Re: [PATCH blktests 1/3] common/nvme: add debug nvmet path variable

[Daniel Wagner]

On Fri, Apr 04, 2025 at 08:03:33AM +0000, Shinichiro Kawasaki wrote:
> On Mar 18, 2025 / 11:38, Daniel Wagner wrote:
> > Signed-off-by: Daniel Wagner <wagi@kernel.org>
> > ---
> >  common/nvme | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/common/nvme b/common/nvme
> > index 3761329d39e3763136f60a4751ad15de347f6e9b..a3176ecff2e5fbc0fbe09460c21e9f50c68d3bce 100644
> > --- a/common/nvme
> > +++ b/common/nvme
> > @@ -26,6 +26,7 @@ nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
> >  NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
> >  nvme_target_control="${NVME_TARGET_CONTROL:-}"
> >  NVMET_CFS="/sys/kernel/config/nvmet/"
> > +NVMET_DFS="/sys/kernel/debug/nvmet/"
> 
> This causes the shellcheck warning SC2034.
> 
>   common/nvme:29:1: warning: NVMET_DFS appears unused. Verify use (or export if used externally). [SC2034]
> 
> I suggest to annotate "# shellcheck disable=SC2034" above the added line.

Will do. I keep forgetting to run shellcheck, sorry about that.

Re: [PATCH blktests 2/3] nvme/060: add test nvme fabrics target resetting during I/O

[Daniel Wagner]

On Fri, Apr 04, 2025 at 08:08:31AM +0000, Shinichiro Kawasaki wrote:
> On Mar 18, 2025 / 11:39, Daniel Wagner wrote:
> > Newer kernel support to reset the target via the debugfs. Add a new test
> > case which exercises this interface.
> 
> I find the kernel commit 649fd41420a8 ("nvmet: add debugfs support") enables
> this test. Looking at the kernel commit, I fount this test case depends on the
> kernel config NVME_TARGET_DEBUGFS. So let's add,
> 
>  _have_kernel_option NVME_TARGET_DEBUGFS
> 
> in requires().

Ah yes, this works for this test case!

> > +requires() {
> > +	_nvme_requires
> > +	_have_loop
> > +	_have_fio
> 
> I don't find fio command in this test case. Do I miss it?

No, it's a left over from the development phase.

> > +nvmf_wait_for_state() {
> > +	local def_state_timeout=5
> > +	local subsys_name="$1"
> > +	local state="$2"
> > +	local timeout="${3:-$def_state_timeout}"
> > +	local nvmedev
> > +	local state_file
> > +	local start_time
> > +	local end_time
> > +
> > +	nvmedev=$(_find_nvme_dev "${subsys_name}")
> > +	state_file="/sys/class/nvme-fabrics/ctl/${nvmedev}/state"
> > +
> > +	start_time=$(date +%s)
> > +	while ! grep -q "${state}" "${state_file}"; do
> > +		sleep 1
> > +		end_time=$(date +%s)
> > +		if (( end_time - start_time > timeout )); then
> > +			echo "expected state \"${state}\" not " \
> > +				"reached within ${timeout} seconds"
> > +			return 1
> > +		fi
> > +	done
> > +
> > +	return 0
> > +}
> 
> Is this function used in this test case?

Oops, another leftover.

Re: [PATCH blktests 3/3] nvme/061: add test teardown and setup fabrics target during I/O

[Daniel Wagner]

On Fri, Apr 04, 2025 at 08:22:47AM +0000, Shinichiro Kawasaki wrote:
> On Mar 18, 2025 / 11:39, Daniel Wagner wrote:
> > Add a new test case which forcefully removes the target and setup it
> > again.
> 
> This test case has a few shellcheck warnings.
> 
> tests/nvme/061:63:3: warning: connect_args appears unused. Verify use (or export if used externally). [SC2034]
> tests/nvme/061:84:22: note: Double quote to prevent globbing and word splitting. [SC2086]
> tests/nvme/061:89:22: note: Double quote to prevent globbing and word splitting. [SC2086]
> 
> When I ran the test case on my test system, I observed it failed. Is it
> expected?

Yes, this should be fixed with my fcloop work eventually.

Thanks for the review!