[PATCH 0/9] selftests: ublk: test infrastructure improvements

public inbox for linux-block@vger.kernel.org
 help / color / mirror / Atom feed

* [PATCH 0/9] selftests: ublk: test infrastructure improvements
@ 2026-01-31 16:23 Ming Lei
  2026-01-31 16:23 ` [PATCH 1/9] selftests: ublk: simplify UBLK_TEST_DIR handling Ming Lei
                   ` (9 more replies)
  0 siblings, 10 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Hi,

This series improves the ublk selftests infrastructure with better
organization and parallel test execution support.

Key improvements:

1. Infrastructure enhancements:
   - Add helper functions for device cleanup and parallel execution
   - Implement group-based test targets for running test subsets
   - Track created devices for proper per-test cleanup
   - Increase timeouts to accommodate parallel test execution
   - Test time can be reduced to 1/3 with parallel execution by passing JOBS=8

2. Test organization:
   - Reorganize tests into logical groups (integrity, recover)
   - Refactor large tests into modular functions for maintainability

3. I/O ordering verification:
   - Remove test_generic_01.sh prone to false positives from block layer reordering
   - Improve test_generic_02.sh with bpftrace-based verification
   - Use block_io_start/block_rq_complete tracepoints for accurate tracking

These changes make the test suite more maintainable, easier to run in parallel,
and more reliable in detecting actual ublk driver issues vs. false positives.


Ming Lei (9):
  selftests: ublk: simplify UBLK_TEST_DIR handling
  selftests: ublk: refactor test_loop_08 into separate functions
  selftests: ublk: add _ublk_del_dev helper function
  selftests: ublk: track created devices for per-test cleanup
  selftests: ublk: add group-based test targets
  selftests: ublk: add _ublk_sleep helper for parallel execution
  selftests: ublk: increase timeouts for parallel test execution
  selftests: ublk: reorganize tests into integrity and recover groups
  selftests: ublk: improve I/O ordering test with bpftrace

 tools/testing/selftests/ublk/Makefile         |  54 ++++++-
 tools/testing/selftests/ublk/test_common.sh   |  51 +++++--
 .../testing/selftests/ublk/test_generic_01.sh |  47 ------
 .../testing/selftests/ublk/test_generic_02.sh |  22 ++-
 .../testing/selftests/ublk/test_generic_16.sh |   4 +-
 .../{test_null_04.sh => test_integrity_01.sh} |   8 +-
 .../selftests/ublk/test_integrity_02.sh       | 141 ++++++++++++++++++
 tools/testing/selftests/ublk/test_loop_08.sh  | 110 --------------
 tools/testing/selftests/ublk/test_part_02.sh  |   6 +-
 ...{test_generic_04.sh => test_recover_01.sh} |   0
 ...{test_generic_05.sh => test_recover_02.sh} |   0
 ...{test_generic_11.sh => test_recover_03.sh} |   0
 ...{test_generic_14.sh => test_recover_04.sh} |   0
 tools/testing/selftests/ublk/trace/seq_io.bt  |  47 ++++--
 14 files changed, 284 insertions(+), 206 deletions(-)
 delete mode 100755 tools/testing/selftests/ublk/test_generic_01.sh
 rename tools/testing/selftests/ublk/{test_null_04.sh => test_integrity_01.sh} (96%)
 create mode 100755 tools/testing/selftests/ublk/test_integrity_02.sh
 delete mode 100755 tools/testing/selftests/ublk/test_loop_08.sh
 rename tools/testing/selftests/ublk/{test_generic_04.sh => test_recover_01.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_05.sh => test_recover_02.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_11.sh => test_recover_03.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_14.sh => test_recover_04.sh} (100%)

-- 
2.47.0


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/9] selftests: ublk: simplify UBLK_TEST_DIR handling
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 2/9] selftests: ublk: refactor test_loop_08 into separate functions Ming Lei
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Remove intermediate TDIR variable and set UBLK_TEST_DIR directly
in _prep_test(). Remove default initialization since the directory
is created dynamically when tests run.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/test_common.sh | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index 21ba51fcc7d7..8d298a7ee7b1 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -124,8 +124,7 @@ _prep_test() {
 	local type=$1
 	shift 1
 	modprobe ublk_drv > /dev/null 2>&1
-	TDIR=$(mktemp -d ${TMPDIR:-.}/ublktest-dir.XXXXXX)
-	export UBLK_TEST_DIR=${TDIR}
+	UBLK_TEST_DIR=$(mktemp -d ${TMPDIR:-.}/ublktest-dir.XXXXXX)
 	UBLK_TMP=$(mktemp ${UBLK_TEST_DIR}/ublk_test_XXXXX)
 	[ "$UBLK_TEST_QUIET" -eq 0 ] && echo "ublk $type: $*"
 	echo "ublk selftest: $TID starting at $(date '+%F %T')" | tee /dev/kmsg
@@ -408,8 +407,6 @@ UBLK_PROG=$(_ublk_test_top_dir)/kublk
 UBLK_TEST_QUIET=1
 UBLK_TEST_SHOW_RESULT=1
 UBLK_BACKFILES=()
-UBLK_TEST_DIR=${TMPDIR:-.}
 export UBLK_PROG
 export UBLK_TEST_QUIET
 export UBLK_TEST_SHOW_RESULT
-export UBLK_TEST_DIR
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/9] selftests: ublk: refactor test_loop_08 into separate functions
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
  2026-01-31 16:23 ` [PATCH 1/9] selftests: ublk: simplify UBLK_TEST_DIR handling Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 3/9] selftests: ublk: add _ublk_del_dev helper function Ming Lei
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Encapsulate each test case in its own function for better organization
and maintainability:

- _setup_device(): device and backfile initialization
- _test_fill_and_verify(): initial data population
- _test_corrupted_reftag(): reftag corruption detection test
- _test_corrupted_data(): data corruption detection test
- _test_bad_apptag(): apptag mismatch detection test

Also fix temp file creation to use ${UBLK_TEST_DIR}/fio_err_XXXXX instead of
creating in current directory.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/test_loop_08.sh | 199 +++++++++++--------
 1 file changed, 115 insertions(+), 84 deletions(-)

diff --git a/tools/testing/selftests/ublk/test_loop_08.sh b/tools/testing/selftests/ublk/test_loop_08.sh
index 2caa7ba748fb..aaf1f52da559 100755
--- a/tools/testing/selftests/ublk/test_loop_08.sh
+++ b/tools/testing/selftests/ublk/test_loop_08.sh
@@ -13,98 +13,129 @@ if [[ "$fio_version" =~ fio-[0-9]+\.[0-9]+$ ]]; then
 	exit $UBLK_SKIP_CODE
 fi
 
+ERR_CODE=0
 
-_prep_test "loop" "end-to-end integrity"
+# Global variables set during device setup
+dev_id=""
+fio_args=""
+fio_err=""
 
-_create_backfile 0 256M
-_create_backfile 1 32M # 256M * (64 integrity bytes / 512 data bytes)
-integrity_params="--integrity_capable --integrity_reftag
-                  --metadata_size 64 --pi_offset 56 --csum_type t10dif"
-dev_id=$(_add_ublk_dev -t loop -u $integrity_params "${UBLK_BACKFILES[@]}")
-_check_add_dev $TID $?
-
-# 1M * (64 integrity bytes / 512 data bytes) = 128K
-fio_args="--ioengine io_uring --direct 1 --bsrange 512-1M --iodepth 32
-          --md_per_io_size 128K --pi_act 0 --pi_chk GUARD,REFTAG,APPTAG
-          --filename /dev/ublkb$dev_id"
-fio --name fill --rw randwrite $fio_args > /dev/null
-err=$?
-if [ $err != 0 ]; then
-	echo "fio fill failed"
-	_show_result $TID $err
-fi
+_setup_device() {
+	_create_backfile 0 256M
+	_create_backfile 1 32M # 256M * (64 integrity bytes / 512 data bytes)
 
-fio --name verify --rw randread $fio_args > /dev/null
-err=$?
-if [ $err != 0 ]; then
-	echo "fio verify failed"
-	_show_result $TID $err
-fi
+	local integrity_params="--integrity_capable --integrity_reftag
+		--metadata_size 64 --pi_offset 56 --csum_type t10dif"
+	dev_id=$(_add_ublk_dev -t loop -u $integrity_params "${UBLK_BACKFILES[@]}")
+	_check_add_dev "$TID" $?
 
-fio_err=$(mktemp fio_err_XXXXX)
+	# 1M * (64 integrity bytes / 512 data bytes) = 128K
+	fio_args="--ioengine io_uring --direct 1 --bsrange 512-1M --iodepth 32
+		--md_per_io_size 128K --pi_act 0 --pi_chk GUARD,REFTAG,APPTAG
+		--filename /dev/ublkb$dev_id"
 
-# Overwrite 4-byte reftag at offset 56 + 4 = 60
-dd_reftag_args="bs=1 seek=60 count=4 oflag=dsync conv=notrunc status=none"
-dd if=/dev/urandom "of=${UBLK_BACKFILES[1]}" $dd_reftag_args
-err=$?
-if [ $err != 0 ]; then
-	echo "dd corrupted_reftag failed"
-	rm -f "$fio_err"
-	_show_result $TID $err
-fi
-if fio --name corrupted_reftag --rw randread $fio_args > /dev/null 2> "$fio_err"; then
-	echo "fio corrupted_reftag unexpectedly succeeded"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
-expected_err="REFTAG compare error: LBA: 0 Expected=0, Actual="
-if ! grep -q "$expected_err" "$fio_err"; then
-	echo "fio corrupted_reftag message not found: $expected_err"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
-# Reset to 0
-dd if=/dev/zero "of=${UBLK_BACKFILES[1]}" $dd_reftag_args
-err=$?
-if [ $err != 0 ]; then
-	echo "dd restore corrupted_reftag failed"
-	rm -f "$fio_err"
-	_show_result $TID $err
-fi
+	fio_err=$(mktemp "${UBLK_TEST_DIR}"/fio_err_XXXXX)
+}
 
-dd_data_args="bs=512 count=1 oflag=direct,dsync conv=notrunc status=none"
-dd if=/dev/zero "of=${UBLK_BACKFILES[0]}" $dd_data_args
-err=$?
-if [ $err != 0 ]; then
-	echo "dd corrupted_data failed"
-	rm -f "$fio_err"
-	_show_result $TID $err
-fi
-if fio --name corrupted_data --rw randread $fio_args > /dev/null 2> "$fio_err"; then
-	echo "fio corrupted_data unexpectedly succeeded"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
-expected_err="Guard compare error: LBA: 0 Expected=0, Actual="
-if ! grep -q "$expected_err" "$fio_err"; then
-	echo "fio corrupted_data message not found: $expected_err"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
+_test_fill_and_verify() {
+	fio --name fill --rw randwrite $fio_args > /dev/null
+	if [ $? != 0 ]; then
+		echo "fio fill failed"
+		ERR_CODE=255
+		return 1
+	fi
 
-if fio --name bad_apptag --rw randread $fio_args --apptag 0x4321 > /dev/null 2> "$fio_err"; then
-	echo "fio bad_apptag unexpectedly succeeded"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
-expected_err="APPTAG compare error: LBA: [0-9]* Expected=4321, Actual=1234"
-if ! grep -q "$expected_err" "$fio_err"; then
-	echo "fio bad_apptag message not found: $expected_err"
-	rm -f "$fio_err"
-	_show_result $TID 255
-fi
+	fio --name verify --rw randread $fio_args > /dev/null
+	if [ $? != 0 ]; then
+		echo "fio verify failed"
+		ERR_CODE=255
+		return 1
+	fi
+}
+
+_test_corrupted_reftag() {
+	local dd_reftag_args="bs=1 seek=60 count=4 oflag=dsync conv=notrunc status=none"
+	local expected_err="REFTAG compare error: LBA: 0 Expected=0, Actual="
+
+	# Overwrite 4-byte reftag at offset 56 + 4 = 60
+	dd if=/dev/urandom "of=${UBLK_BACKFILES[1]}" $dd_reftag_args
+	if [ $? != 0 ]; then
+		echo "dd corrupted_reftag failed"
+		ERR_CODE=255
+		return 1
+	fi
+
+	if fio --name corrupted_reftag --rw randread $fio_args > /dev/null 2> "$fio_err"; then
+		echo "fio corrupted_reftag unexpectedly succeeded"
+		ERR_CODE=255
+		return 1
+	fi
+
+	if ! grep -q "$expected_err" "$fio_err"; then
+		echo "fio corrupted_reftag message not found: $expected_err"
+		ERR_CODE=255
+		return 1
+	fi
+
+	# Reset to 0
+	dd if=/dev/zero "of=${UBLK_BACKFILES[1]}" $dd_reftag_args
+	if [ $? != 0 ]; then
+		echo "dd restore corrupted_reftag failed"
+		ERR_CODE=255
+		return 1
+	fi
+}
+
+_test_corrupted_data() {
+	local dd_data_args="bs=512 count=1 oflag=direct,dsync conv=notrunc status=none"
+	local expected_err="Guard compare error: LBA: 0 Expected=0, Actual="
+
+	dd if=/dev/zero "of=${UBLK_BACKFILES[0]}" $dd_data_args
+	if [ $? != 0 ]; then
+		echo "dd corrupted_data failed"
+		ERR_CODE=255
+		return 1
+	fi
+
+	if fio --name corrupted_data --rw randread $fio_args > /dev/null 2> "$fio_err"; then
+		echo "fio corrupted_data unexpectedly succeeded"
+		ERR_CODE=255
+		return 1
+	fi
+
+	if ! grep -q "$expected_err" "$fio_err"; then
+		echo "fio corrupted_data message not found: $expected_err"
+		ERR_CODE=255
+		return 1
+	fi
+}
+
+_test_bad_apptag() {
+	local expected_err="APPTAG compare error: LBA: [0-9]* Expected=4321, Actual=1234"
+
+	if fio --name bad_apptag --rw randread $fio_args --apptag 0x4321 > /dev/null 2> "$fio_err"; then
+		echo "fio bad_apptag unexpectedly succeeded"
+		ERR_CODE=255
+		return 1
+	fi
+
+	if ! grep -q "$expected_err" "$fio_err"; then
+		echo "fio bad_apptag message not found: $expected_err"
+		ERR_CODE=255
+		return 1
+	fi
+}
+
+_prep_test "loop" "end-to-end integrity"
+
+_setup_device
+
+_test_fill_and_verify && \
+_test_corrupted_reftag && \
+_test_corrupted_data && \
+_test_bad_apptag
 
 rm -f "$fio_err"
 
 _cleanup_test
-_show_result $TID 0
+_show_result "$TID" $ERR_CODE
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/9] selftests: ublk: add _ublk_del_dev helper function
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
  2026-01-31 16:23 ` [PATCH 1/9] selftests: ublk: simplify UBLK_TEST_DIR handling Ming Lei
  2026-01-31 16:23 ` [PATCH 2/9] selftests: ublk: refactor test_loop_08 into separate functions Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 4/9] selftests: ublk: track created devices for per-test cleanup Ming Lei
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Add _ublk_del_dev() to delete a specific ublk device by ID and
use it in all test scripts instead of calling UBLK_PROG directly.

Also remove unused _remove_ublk_devices() function.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/test_common.sh     | 13 +++++++------
 tools/testing/selftests/ublk/test_generic_16.sh |  4 ++--
 tools/testing/selftests/ublk/test_null_04.sh    |  8 ++++----
 tools/testing/selftests/ublk/test_part_02.sh    |  4 ++--
 4 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index 8d298a7ee7b1..0f1fdb0892b4 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -106,11 +106,6 @@ _check_root() {
 	fi
 }
 
-_remove_ublk_devices() {
-	${UBLK_PROG} del -a
-	modprobe -r ublk_drv > /dev/null 2>&1
-}
-
 _get_ublk_dev_state() {
 	${UBLK_PROG} list -n "$1" | grep "state" | awk '{print $11}'
 }
@@ -277,10 +272,16 @@ __ublk_kill_daemon()
 	echo "$state"
 }
 
-__remove_ublk_dev_return() {
+_ublk_del_dev() {
 	local dev_id=$1
 
 	${UBLK_PROG} del -n "${dev_id}"
+}
+
+__remove_ublk_dev_return() {
+	local dev_id=$1
+
+	_ublk_del_dev "${dev_id}"
 	local res=$?
 	udevadm settle
 	return ${res}
diff --git a/tools/testing/selftests/ublk/test_generic_16.sh b/tools/testing/selftests/ublk/test_generic_16.sh
index 42e8d2e16ec9..3ef367836ac5 100755
--- a/tools/testing/selftests/ublk/test_generic_16.sh
+++ b/tools/testing/selftests/ublk/test_generic_16.sh
@@ -24,7 +24,7 @@ if ! ${UBLK_PROG} stop -n "${dev_id}" --safe; then
 fi
 
 # Clean up device
-${UBLK_PROG} del -n "${dev_id}" > /dev/null 2>&1
+_ublk_del_dev "${dev_id}" > /dev/null 2>&1
 udevadm settle
 
 # Test 2: stop --safe on device with active opener should fail
@@ -49,7 +49,7 @@ kill $dd_pid 2>/dev/null
 wait $dd_pid 2>/dev/null
 
 # Now device should be idle, regular delete should work
-${UBLK_PROG} del -n "${dev_id}"
+_ublk_del_dev "${dev_id}"
 udevadm settle
 
 _cleanup_test "null"
diff --git a/tools/testing/selftests/ublk/test_null_04.sh b/tools/testing/selftests/ublk/test_null_04.sh
index a5599d38583a..6713b280a6ff 100755
--- a/tools/testing/selftests/ublk/test_null_04.sh
+++ b/tools/testing/selftests/ublk/test_null_04.sh
@@ -34,7 +34,7 @@ _test_metadata_only() {
 		"$(cat "/sys/block/ublkb$dev_id/integrity/protection_interval_bytes")" 512 &&
 	_check_value "tag_size" "$(cat "/sys/block/ublkb$dev_id/integrity/tag_size")" 0
 
-	${UBLK_PROG} del -n "${dev_id}"
+	_ublk_del_dev "${dev_id}"
 }
 
 _test_integrity_capable_ip() {
@@ -53,7 +53,7 @@ _test_integrity_capable_ip() {
 		"$(cat "/sys/block/ublkb$dev_id/integrity/protection_interval_bytes")" 512 &&
 	_check_value "tag_size" "$(cat "/sys/block/ublkb$dev_id/integrity/tag_size")" 0
 
-	${UBLK_PROG} del -n "${dev_id}"
+	_ublk_del_dev "${dev_id}"
 }
 
 _test_integrity_reftag_t10dif() {
@@ -72,7 +72,7 @@ _test_integrity_reftag_t10dif() {
 		"$(cat "/sys/block/ublkb$dev_id/integrity/protection_interval_bytes")" 512 &&
 	_check_value "tag_size" "$(cat "/sys/block/ublkb$dev_id/integrity/tag_size")" 0
 
-	${UBLK_PROG} del -n "${dev_id}"
+	_ublk_del_dev "${dev_id}"
 }
 
 _test_nvme_csum() {
@@ -91,7 +91,7 @@ _test_nvme_csum() {
 		"$(cat "/sys/block/ublkb$dev_id/integrity/protection_interval_bytes")" 512 &&
 	_check_value "tag_size" "$(cat "/sys/block/ublkb$dev_id/integrity/tag_size")" 8
 
-	${UBLK_PROG} del -n "${dev_id}"
+	_ublk_del_dev "${dev_id}"
 }
 
 _prep_test "null" "integrity params"
diff --git a/tools/testing/selftests/ublk/test_part_02.sh b/tools/testing/selftests/ublk/test_part_02.sh
index 727d0f4610d6..acd098deda3a 100755
--- a/tools/testing/selftests/ublk/test_part_02.sh
+++ b/tools/testing/selftests/ublk/test_part_02.sh
@@ -46,13 +46,13 @@ _test_partition_scan_no_hang()
 	if [ "$state" != "${expected_state}" ]; then
 		echo "FAIL: Device state is $state, expected ${expected_state}"
 		ERR_CODE=255
-		${UBLK_PROG} del -n "${dev_id}" > /dev/null 2>&1
+		_ublk_del_dev "${dev_id}" > /dev/null 2>&1
 		return
 	fi
 	echo "PASS: Device transitioned to ${expected_state} in ${elapsed}s without hanging"
 
 	# Clean up the device
-	${UBLK_PROG} del -n "${dev_id}" > /dev/null 2>&1
+	_ublk_del_dev "${dev_id}" > /dev/null 2>&1
 }
 
 _prep_test "partition_scan" "verify async partition scan prevents IO hang"
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 4/9] selftests: ublk: track created devices for per-test cleanup
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (2 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 3/9] selftests: ublk: add _ublk_del_dev helper function Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 5/9] selftests: ublk: add group-based test targets Ming Lei
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Track device IDs in UBLK_DEVS array when created. Update
_cleanup_test() to only delete devices created by this test
instead of using 'del -a' which removes all devices.

This prepares for running tests concurrently where each test
should only clean up its own devices.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/test_common.sh | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index 0f1fdb0892b4..422882c32490 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -164,7 +164,12 @@ _check_add_dev()
 }
 
 _cleanup_test() {
-	"${UBLK_PROG}" del -a
+	if [ -f "${UBLK_TEST_DIR}/.ublk_devs" ]; then
+		while read -r dev_id; do
+			${UBLK_PROG} del -n "${dev_id}"
+		done < "${UBLK_TEST_DIR}/.ublk_devs"
+		rm -f "${UBLK_TEST_DIR}/.ublk_devs"
+	fi
 
 	_remove_files
 	rmdir ${UBLK_TEST_DIR}
@@ -205,6 +210,7 @@ _create_ublk_dev() {
 	fi
 
 	if [[ "$dev_id" =~ ^[0-9]+$ ]]; then
+		echo "$dev_id" >> "${UBLK_TEST_DIR}/.ublk_devs"
 		echo "${dev_id}"
 	else
 		return 255
@@ -276,6 +282,11 @@ _ublk_del_dev() {
 	local dev_id=$1
 
 	${UBLK_PROG} del -n "${dev_id}"
+
+	# Remove from tracking file
+	if [ -f "${UBLK_TEST_DIR}/.ublk_devs" ]; then
+		sed -i "/^${dev_id}$/d" "${UBLK_TEST_DIR}/.ublk_devs"
+	fi
 }
 
 __remove_ublk_dev_return() {
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 5/9] selftests: ublk: add group-based test targets
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (3 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 4/9] selftests: ublk: track created devices for per-test cleanup Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 6/9] selftests: ublk: add _ublk_sleep helper for parallel execution Ming Lei
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Add convenient Makefile targets for running specific test groups:
- run_generic, run_batch, run_null, run_loop, run_stripe, run_stress, etc.
- run_all for running all tests

Test groups are auto-detected from TEST_PROGS using pattern matching
(test_<group>_<num>.sh -> group), and targets are generated dynamically
using define/eval templates.

Supports parallel execution via JOBS variable:
- JOBS=1 (default): sequential with kselftest TAP output
- JOBS>1: parallel execution with xargs -P

Usage examples:
  make run_null           # Sequential execution
  make run_stress JOBS=4  # Parallel with 4 jobs
  make run_all JOBS=8     # Run all tests with 8 parallel jobs

With JOBS=8, running time of `make run_all` is reduced to 2m2s from 6m5s
in my test VM.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile | 36 +++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index ca8588ed962c..37e012d3a8a7 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -72,3 +72,39 @@ $(OUTPUT)/kublk: $(filter-out $(STANDALONE_UTILS),$(wildcard *.c))
 
 check:
 	shellcheck -x -f gcc *.sh
+
+# Test groups for running subsets of tests
+# JOBS=1 (default): sequential with kselftest TAP output
+# JOBS>1: parallel execution with xargs -P
+# Usage: make run_null JOBS=4
+JOBS ?= 1
+
+# Auto-detect test groups from TEST_PROGS (test_<group>_<num>.sh -> group)
+TEST_GROUPS := $(shell echo "$(TEST_PROGS)" | tr ' ' '\n' | \
+	sed 's/test_\([^_]*\)_.*/\1/' | sort -u)
+
+# Template for group test targets
+# $(1) = group name (e.g., null, generic, stress)
+define RUN_GROUP
+run_$(1): all
+	@if [ $$(JOBS) -gt 1 ]; then \
+		echo $$(filter test_$(1)_%.sh,$$(TEST_PROGS)) | tr ' ' '\n' | \
+			xargs -P $$(JOBS) -n1 sh -c './"$$$$0"' || true; \
+	else \
+		$$(call RUN_TESTS, $$(filter test_$(1)_%.sh,$$(TEST_PROGS))); \
+	fi
+.PHONY: run_$(1)
+endef
+
+# Generate targets for each discovered test group
+$(foreach group,$(TEST_GROUPS),$(eval $(call RUN_GROUP,$(group))))
+
+# Run all tests (parallel when JOBS>1)
+run_all: all
+	@if [ $(JOBS) -gt 1 ]; then \
+		echo $(TEST_PROGS) | tr ' ' '\n' | \
+			xargs -P $(JOBS) -n1 sh -c './"$$0"' || true; \
+	else \
+		$(call RUN_TESTS, $(TEST_PROGS)); \
+	fi
+.PHONY: run_all
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 6/9] selftests: ublk: add _ublk_sleep helper for parallel execution
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (4 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 5/9] selftests: ublk: add group-based test targets Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 7/9] selftests: ublk: increase timeouts for parallel test execution Ming Lei
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Add _ublk_sleep() helper function that uses different sleep times
depending on whether tests run in parallel or sequential mode.

Usage: _ublk_sleep <normal_secs> <parallel_secs>

Export JOBS variable from Makefile so test scripts can detect parallel
execution, and use _ublk_sleep in test_part_02.sh to handle the
partition scan delay (1s normal, 5s parallel).

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile        |  1 +
 tools/testing/selftests/ublk/test_common.sh  | 10 ++++++++++
 tools/testing/selftests/ublk/test_part_02.sh |  2 +-
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 37e012d3a8a7..1ceae611acb7 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -78,6 +78,7 @@ check:
 # JOBS>1: parallel execution with xargs -P
 # Usage: make run_null JOBS=4
 JOBS ?= 1
+export JOBS
 
 # Auto-detect test groups from TEST_PROGS (test_<group>_<num>.sh -> group)
 TEST_GROUPS := $(shell echo "$(TEST_PROGS)" | tr ' ' '\n' | \
diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index 422882c32490..bd27a6875c1a 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -15,6 +15,16 @@ _have_program() {
 	return 1
 }
 
+# Sleep with awareness of parallel execution.
+# Usage: _ublk_sleep <normal_secs> <parallel_secs>
+_ublk_sleep() {
+	if [ "${JOBS:-1}" -gt 1 ]; then
+		sleep "$2"
+	else
+		sleep "$1"
+	fi
+}
+
 _get_disk_dev_t() {
 	local dev_id=$1
 	local dev
diff --git a/tools/testing/selftests/ublk/test_part_02.sh b/tools/testing/selftests/ublk/test_part_02.sh
index acd098deda3a..7d42ab4d6e83 100755
--- a/tools/testing/selftests/ublk/test_part_02.sh
+++ b/tools/testing/selftests/ublk/test_part_02.sh
@@ -33,7 +33,7 @@ _test_partition_scan_no_hang()
 	# The add command should return quickly because partition scan is async.
 	# Now sleep briefly to let the async partition scan work start and hit
 	# the delay in the fault_inject handler.
-	sleep 1
+	_ublk_sleep 1 5
 
 	# Kill the ublk daemon while partition scan is potentially blocked
 	# And check state transitions properly
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 7/9] selftests: ublk: increase timeouts for parallel test execution
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (5 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 6/9] selftests: ublk: add _ublk_sleep helper for parallel execution Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 8/9] selftests: ublk: reorganize tests into integrity and recover groups Ming Lei
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

When running tests in parallel with high JOBS count (e.g., JOBS=64),
the existing timeouts can be insufficient due to system load:

- Increase state wait loops from 20/50 to 100 iterations in
  _recover_ublk_dev(), __ublk_quiesce_dev(), and __ublk_kill_daemon()
  to handle slower state transitions under heavy load

- Add --timeout=20 to udevadm settle calls to prevent indefinite
  hangs when udev event queue is overwhelmed by rapid device
  creation/deletion

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/test_common.sh | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index bd27a6875c1a..c3afd00783a2 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -216,7 +216,7 @@ _create_ublk_dev() {
 	fi
 
 	if [ "$settle" = "yes" ]; then
-		udevadm settle
+		udevadm settle --timeout=20
 	fi
 
 	if [[ "$dev_id" =~ ^[0-9]+$ ]]; then
@@ -240,7 +240,7 @@ _recover_ublk_dev() {
 	local state
 
 	dev_id=$(_create_ublk_dev "recover" "yes" "$@")
-	for ((j=0;j<20;j++)); do
+	for ((j=0;j<100;j++)); do
 		state=$(_get_ublk_dev_state "${dev_id}")
 		[ "$state" == "LIVE" ] && break
 		sleep 1
@@ -260,7 +260,7 @@ __ublk_quiesce_dev()
 		return "$state"
 	fi
 
-	for ((j=0;j<50;j++)); do
+	for ((j=0;j<100;j++)); do
 		state=$(_get_ublk_dev_state "${dev_id}")
 		[ "$state" == "$exp_state" ] && break
 		sleep 1
@@ -279,7 +279,7 @@ __ublk_kill_daemon()
 	daemon_pid=$(_get_ublk_daemon_pid "${dev_id}")
 	state=$(_get_ublk_dev_state "${dev_id}")
 
-	for ((j=0;j<50;j++)); do
+	for ((j=0;j<100;j++)); do
 		[ "$state" == "$exp_state" ] && break
 		kill -9 "$daemon_pid" > /dev/null 2>&1
 		sleep 1
@@ -304,7 +304,7 @@ __remove_ublk_dev_return() {
 
 	_ublk_del_dev "${dev_id}"
 	local res=$?
-	udevadm settle
+	udevadm settle --timeout=20
 	return ${res}
 }
 
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 8/9] selftests: ublk: reorganize tests into integrity and recover groups
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (6 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 7/9] selftests: ublk: increase timeouts for parallel test execution Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 16:23 ` [PATCH 9/9] selftests: ublk: improve I/O ordering test with bpftrace Ming Lei
  2026-01-31 21:56 ` [PATCH 0/9] selftests: ublk: test infrastructure improvements Jens Axboe
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Move integrity-focused tests into new 'integrity' group:
- test_null_04.sh -> test_integrity_01.sh
- test_loop_08.sh -> test_integrity_02.sh

Move recovery-focused tests into new 'recover' group:
- test_generic_04.sh -> test_recover_01.sh
- test_generic_05.sh -> test_recover_02.sh
- test_generic_11.sh -> test_recover_03.sh
- test_generic_14.sh -> test_recover_04.sh

Update Makefile to reflect the reorganization.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile              | 14 ++++++++------
 .../ublk/{test_null_04.sh => test_integrity_01.sh} |  0
 .../ublk/{test_loop_08.sh => test_integrity_02.sh} |  0
 .../{test_generic_04.sh => test_recover_01.sh}     |  0
 .../{test_generic_05.sh => test_recover_02.sh}     |  0
 .../{test_generic_11.sh => test_recover_03.sh}     |  0
 .../{test_generic_14.sh => test_recover_04.sh}     |  0
 7 files changed, 8 insertions(+), 6 deletions(-)
 rename tools/testing/selftests/ublk/{test_null_04.sh => test_integrity_01.sh} (100%)
 rename tools/testing/selftests/ublk/{test_loop_08.sh => test_integrity_02.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_04.sh => test_recover_01.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_05.sh => test_recover_02.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_11.sh => test_recover_03.sh} (100%)
 rename tools/testing/selftests/ublk/{test_generic_14.sh => test_recover_04.sh} (100%)

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 1ceae611acb7..a62a06e13006 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -10,18 +10,14 @@ LDLIBS += -lpthread -lm -luring
 TEST_PROGS := test_generic_01.sh
 TEST_PROGS += test_generic_02.sh
 TEST_PROGS += test_generic_03.sh
-TEST_PROGS += test_generic_04.sh
-TEST_PROGS += test_generic_05.sh
 TEST_PROGS += test_generic_06.sh
 TEST_PROGS += test_generic_07.sh
 
 TEST_PROGS += test_generic_08.sh
 TEST_PROGS += test_generic_09.sh
 TEST_PROGS += test_generic_10.sh
-TEST_PROGS += test_generic_11.sh
 TEST_PROGS += test_generic_12.sh
 TEST_PROGS += test_generic_13.sh
-TEST_PROGS += test_generic_14.sh
 TEST_PROGS += test_generic_16.sh
 
 TEST_PROGS += test_batch_01.sh
@@ -31,7 +27,6 @@ TEST_PROGS += test_batch_03.sh
 TEST_PROGS += test_null_01.sh
 TEST_PROGS += test_null_02.sh
 TEST_PROGS += test_null_03.sh
-TEST_PROGS += test_null_04.sh
 TEST_PROGS += test_loop_01.sh
 TEST_PROGS += test_loop_02.sh
 TEST_PROGS += test_loop_03.sh
@@ -39,7 +34,14 @@ TEST_PROGS += test_loop_04.sh
 TEST_PROGS += test_loop_05.sh
 TEST_PROGS += test_loop_06.sh
 TEST_PROGS += test_loop_07.sh
-TEST_PROGS += test_loop_08.sh
+
+TEST_PROGS += test_integrity_01.sh
+TEST_PROGS += test_integrity_02.sh
+
+TEST_PROGS += test_recover_01.sh
+TEST_PROGS += test_recover_02.sh
+TEST_PROGS += test_recover_03.sh
+TEST_PROGS += test_recover_04.sh
 TEST_PROGS += test_stripe_01.sh
 TEST_PROGS += test_stripe_02.sh
 TEST_PROGS += test_stripe_03.sh
diff --git a/tools/testing/selftests/ublk/test_null_04.sh b/tools/testing/selftests/ublk/test_integrity_01.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_null_04.sh
rename to tools/testing/selftests/ublk/test_integrity_01.sh
diff --git a/tools/testing/selftests/ublk/test_loop_08.sh b/tools/testing/selftests/ublk/test_integrity_02.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_loop_08.sh
rename to tools/testing/selftests/ublk/test_integrity_02.sh
diff --git a/tools/testing/selftests/ublk/test_generic_04.sh b/tools/testing/selftests/ublk/test_recover_01.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_generic_04.sh
rename to tools/testing/selftests/ublk/test_recover_01.sh
diff --git a/tools/testing/selftests/ublk/test_generic_05.sh b/tools/testing/selftests/ublk/test_recover_02.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_generic_05.sh
rename to tools/testing/selftests/ublk/test_recover_02.sh
diff --git a/tools/testing/selftests/ublk/test_generic_11.sh b/tools/testing/selftests/ublk/test_recover_03.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_generic_11.sh
rename to tools/testing/selftests/ublk/test_recover_03.sh
diff --git a/tools/testing/selftests/ublk/test_generic_14.sh b/tools/testing/selftests/ublk/test_recover_04.sh
similarity index 100%
rename from tools/testing/selftests/ublk/test_generic_14.sh
rename to tools/testing/selftests/ublk/test_recover_04.sh
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 9/9] selftests: ublk: improve I/O ordering test with bpftrace
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (7 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 8/9] selftests: ublk: reorganize tests into integrity and recover groups Ming Lei
@ 2026-01-31 16:23 ` Ming Lei
  2026-01-31 21:56 ` [PATCH 0/9] selftests: ublk: test infrastructure improvements Jens Axboe
  9 siblings, 0 replies; 11+ messages in thread
From: Ming Lei @ 2026-01-31 16:23 UTC (permalink / raw)
  To: Jens Axboe, linux-block
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan, Ming Lei

Remove test_generic_01.sh since block layer may reorder I/O, making
the test prone to false positives. Apply the improvements to
test_generic_02.sh instead, which supposes for covering ublk dispatch
io order.

Rework test_generic_02 to verify that ublk dispatch doesn't reorder I/O
by comparing request start order with completion order using bpftrace.

The bpftrace script now:
- Tracks each request's start sequence number in a map keyed by sector
- On completion, verifies the request's start order matches expected
  completion order
- Reports any out-of-order completions detected

The test script:
- Wait bpftrace BEGIN code block is run
- Pins fio to CPU 0 for deterministic behavior
- Uses block_io_start and block_rq_complete tracepoints
- Checks bpftrace output for reordering errors

Reported-and-tested-by: Alexander Atanasov <alex@zazolabs.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile         |  3 +-
 .../testing/selftests/ublk/test_generic_01.sh | 47 -------------------
 .../testing/selftests/ublk/test_generic_02.sh | 22 ++++++---
 tools/testing/selftests/ublk/trace/seq_io.bt  | 47 +++++++++++++++----
 4 files changed, 54 insertions(+), 65 deletions(-)
 delete mode 100755 tools/testing/selftests/ublk/test_generic_01.sh

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index a62a06e13006..8ac2d4a682a1 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -7,8 +7,7 @@ endif
 
 LDLIBS += -lpthread -lm -luring
 
-TEST_PROGS := test_generic_01.sh
-TEST_PROGS += test_generic_02.sh
+TEST_PROGS := test_generic_02.sh
 TEST_PROGS += test_generic_03.sh
 TEST_PROGS += test_generic_06.sh
 TEST_PROGS += test_generic_07.sh
diff --git a/tools/testing/selftests/ublk/test_generic_01.sh b/tools/testing/selftests/ublk/test_generic_01.sh
deleted file mode 100755
index 26cf3c7ceeb5..000000000000
--- a/tools/testing/selftests/ublk/test_generic_01.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0
-
-. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh
-
-ERR_CODE=0
-
-if ! _have_program bpftrace; then
-	exit "$UBLK_SKIP_CODE"
-fi
-
-if ! _have_program fio; then
-	exit "$UBLK_SKIP_CODE"
-fi
-
-_prep_test "null" "sequential io order"
-
-dev_id=$(_add_ublk_dev -t null)
-_check_add_dev $TID $?
-
-dev_t=$(_get_disk_dev_t "$dev_id")
-bpftrace trace/seq_io.bt "$dev_t" "W" 1 > "$UBLK_TMP" 2>&1 &
-btrace_pid=$!
-sleep 2
-
-if ! kill -0 "$btrace_pid" > /dev/null 2>&1; then
-	_cleanup_test "null"
-	exit "$UBLK_SKIP_CODE"
-fi
-
-# run fio over this ublk disk
-fio --name=write_seq \
-    --filename=/dev/ublkb"${dev_id}" \
-    --ioengine=libaio --iodepth=16 \
-    --rw=write \
-    --size=512M \
-    --direct=1 \
-    --bs=4k > /dev/null 2>&1
-ERR_CODE=$?
-kill "$btrace_pid"
-wait
-if grep -q "io_out_of_order" "$UBLK_TMP"; then
-	cat "$UBLK_TMP"
-	ERR_CODE=255
-fi
-_cleanup_test "null"
-_show_result $TID $ERR_CODE
diff --git a/tools/testing/selftests/ublk/test_generic_02.sh b/tools/testing/selftests/ublk/test_generic_02.sh
index 1d4b1d6e059c..46b657143fd6 100755
--- a/tools/testing/selftests/ublk/test_generic_02.sh
+++ b/tools/testing/selftests/ublk/test_generic_02.sh
@@ -13,7 +13,7 @@ if ! _have_program fio; then
 	exit "$UBLK_SKIP_CODE"
 fi
 
-_prep_test "null" "sequential io order for MQ"
+_prep_test "null" "ublk dispatch won't reorder IO for MQ"
 
 dev_id=$(_add_ublk_dev -t null -q 2)
 _check_add_dev $TID $?
@@ -21,15 +21,20 @@ _check_add_dev $TID $?
 dev_t=$(_get_disk_dev_t "$dev_id")
 bpftrace trace/seq_io.bt "$dev_t" "W" 1 > "$UBLK_TMP" 2>&1 &
 btrace_pid=$!
-sleep 2
 
-if ! kill -0 "$btrace_pid" > /dev/null 2>&1; then
+# Wait for bpftrace probes to be attached (BEGIN block prints BPFTRACE_READY)
+for _ in $(seq 100); do
+	grep -q "BPFTRACE_READY" "$UBLK_TMP" 2>/dev/null && break
+	sleep 0.1
+done
+
+if ! kill -0 "$btrace_pid" 2>/dev/null; then
 	_cleanup_test "null"
 	exit "$UBLK_SKIP_CODE"
 fi
 
-# run fio over this ublk disk
-fio --name=write_seq \
+# run fio over this ublk disk (pinned to CPU 0)
+taskset -c 0 fio --name=write_seq \
     --filename=/dev/ublkb"${dev_id}" \
     --ioengine=libaio --iodepth=16 \
     --rw=write \
@@ -39,8 +44,11 @@ fio --name=write_seq \
 ERR_CODE=$?
 kill "$btrace_pid"
 wait
-if grep -q "io_out_of_order" "$UBLK_TMP"; then
-	cat "$UBLK_TMP"
+
+# Check for out-of-order completions detected by bpftrace
+if grep -q "^out_of_order:" "$UBLK_TMP"; then
+	echo "I/O reordering detected:"
+	grep "^out_of_order:" "$UBLK_TMP"
 	ERR_CODE=255
 fi
 _cleanup_test "null"
diff --git a/tools/testing/selftests/ublk/trace/seq_io.bt b/tools/testing/selftests/ublk/trace/seq_io.bt
index b2f60a92b118..9d36ba35468f 100644
--- a/tools/testing/selftests/ublk/trace/seq_io.bt
+++ b/tools/testing/selftests/ublk/trace/seq_io.bt
@@ -2,23 +2,52 @@
 	$1: 	dev_t
 	$2: 	RWBS
 	$3:     strlen($2)
+
+	Track request order between block_io_start and block_rq_complete.
+	Sequence starts at 1 so 0 means "never seen". On first valid
+	completion, sync complete_seq to handle probe attachment races.
+	block_rq_complete listed first to reduce missed completion window.
 */
+
 BEGIN {
-	@last_rw[$1, str($2)] = (uint64)0;
+	@start_seq = (uint64)1;
+	@complete_seq = (uint64)0;
+	@out_of_order = (uint64)0;
+	@start_order[0] = (uint64)0;
+	delete(@start_order[0]);
+	printf("BPFTRACE_READY\n");
 }
+
 tracepoint:block:block_rq_complete
+/(int64)args.dev == $1 && !strncmp(args.rwbs, str($2), $3)/
 {
-	$dev = $1;
-	if ((int64)args.dev == $1 && !strncmp(args.rwbs, str($2), $3)) {
-		$last = @last_rw[$dev, str($2)];
-		if ((uint64)args.sector != $last) {
-			printf("io_out_of_order: exp %llu actual %llu\n",
-				args.sector, $last);
+	$expected = @start_order[args.sector];
+	if ($expected > 0) {
+		if (@complete_seq == 0) {
+			@complete_seq = $expected;
+		}
+		if ($expected != @complete_seq) {
+			printf("out_of_order: sector %llu started at seq %llu but completed at seq %llu\n",
+				args.sector, $expected, @complete_seq);
+			@out_of_order = @out_of_order + 1;
 		}
-		@last_rw[$dev, str($2)] = (args.sector + args.nr_sector);
+		delete(@start_order[args.sector]);
+		@complete_seq = @complete_seq + 1;
 	}
 }
 
+tracepoint:block:block_io_start
+/(int64)args.dev == $1 && !strncmp(args.rwbs, str($2), $3)/
+{
+	@start_order[args.sector] = @start_seq;
+	@start_seq = @start_seq + 1;
+}
+
 END {
-	clear(@last_rw);
+	printf("total_start: %llu total_complete: %llu out_of_order: %llu\n",
+		@start_seq - 1, @complete_seq, @out_of_order);
+	clear(@start_order);
+	clear(@start_seq);
+	clear(@complete_seq);
+	clear(@out_of_order);
 }
-- 
2.47.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/9] selftests: ublk: test infrastructure improvements
  2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
                   ` (8 preceding siblings ...)
  2026-01-31 16:23 ` [PATCH 9/9] selftests: ublk: improve I/O ordering test with bpftrace Ming Lei
@ 2026-01-31 21:56 ` Jens Axboe
  9 siblings, 0 replies; 11+ messages in thread
From: Jens Axboe @ 2026-01-31 21:56 UTC (permalink / raw)
  To: linux-block, Ming Lei
  Cc: Caleb Sander Mateos, Uday Shankar, Alexander Atanasov,
	linux-kselftest, Shuah Khan


On Sun, 01 Feb 2026 00:23:31 +0800, Ming Lei wrote:
> This series improves the ublk selftests infrastructure with better
> organization and parallel test execution support.
> 
> Key improvements:
> 
> 1. Infrastructure enhancements:
>    - Add helper functions for device cleanup and parallel execution
>    - Implement group-based test targets for running test subsets
>    - Track created devices for proper per-test cleanup
>    - Increase timeouts to accommodate parallel test execution
>    - Test time can be reduced to 1/3 with parallel execution by passing JOBS=8
> 
> [...]

Applied, thanks!

[1/9] selftests: ublk: simplify UBLK_TEST_DIR handling
      commit: 5af302a15a1d628a025a78892001fe8afea90c60
[2/9] selftests: ublk: refactor test_loop_08 into separate functions
      commit: 842b6520e579b8bd7d6ea09937e1fb7729cce1c5
[3/9] selftests: ublk: add _ublk_del_dev helper function
      commit: 92734a4f3a7a5449b0c7d0160ba658a2b665c31b
[4/9] selftests: ublk: track created devices for per-test cleanup
      commit: 2021e6109de3e97adfce262c40a657ff206ef495
[5/9] selftests: ublk: add group-based test targets
      commit: b6bbc3bec19efd557f888d78865b627b80b37a32
[6/9] selftests: ublk: add _ublk_sleep helper for parallel execution
      commit: 64406dd2f69fe27921c7bf06088871c002cf6186
[7/9] selftests: ublk: increase timeouts for parallel test execution
      commit: 56a08b87f9f2a763cb5546f83b78ebe1e96260af
[8/9] selftests: ublk: reorganize tests into integrity and recover groups
      commit: d9a36ab302b1c90d8f03a3b13538b8676eb6ed3b
[9/9] selftests: ublk: improve I/O ordering test with bpftrace
      commit: 5314d25afbc44d0449fa2519d2c9d7f3c319f74c

Best regards,
-- 
Jens Axboe




^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2026-01-31 21:56 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-31 16:23 [PATCH 0/9] selftests: ublk: test infrastructure improvements Ming Lei
2026-01-31 16:23 ` [PATCH 1/9] selftests: ublk: simplify UBLK_TEST_DIR handling Ming Lei
2026-01-31 16:23 ` [PATCH 2/9] selftests: ublk: refactor test_loop_08 into separate functions Ming Lei
2026-01-31 16:23 ` [PATCH 3/9] selftests: ublk: add _ublk_del_dev helper function Ming Lei
2026-01-31 16:23 ` [PATCH 4/9] selftests: ublk: track created devices for per-test cleanup Ming Lei
2026-01-31 16:23 ` [PATCH 5/9] selftests: ublk: add group-based test targets Ming Lei
2026-01-31 16:23 ` [PATCH 6/9] selftests: ublk: add _ublk_sleep helper for parallel execution Ming Lei
2026-01-31 16:23 ` [PATCH 7/9] selftests: ublk: increase timeouts for parallel test execution Ming Lei
2026-01-31 16:23 ` [PATCH 8/9] selftests: ublk: reorganize tests into integrity and recover groups Ming Lei
2026-01-31 16:23 ` [PATCH 9/9] selftests: ublk: improve I/O ordering test with bpftrace Ming Lei
2026-01-31 21:56 ` [PATCH 0/9] selftests: ublk: test infrastructure improvements Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox