[PATCH 0/8] selftests: ublk: cleanup & more tests

[PATCH 0/8] selftests: ublk: cleanup & more tests

[Ming Lei]

Hi Jens,

The 1st patch adds generic_01.sh for checking if IO is dispatched in order.

The 2nd ~ 7th patches clean up and simplify target implementation, add zc
for for null, which is useful for evaluating/comparing perf.

The 8th patch adds ublk/stripe target and two tests, which will be useful
for verifying multiple IOs aiming at same fixed kernel buffer, also can
be used for verifying vectored fixed kernel buffer in future if this
feature can be supported.


Ming Lei (8):
  selftests: ublk: add generic_01 for verifying sequential IO order
  selftests: ublk: add single sqe allocator helper
  selftests: ublk: increase max buffer size to 1MB
  selftests: ublk: move common code into common.c
  selftests: ublk: prepare for supporting stripe target
  selftests: ublk: enable zero copy for null target
  selftests: ublk: simplify loop io completion
  selftests: ublk: add stripe target

 tools/testing/selftests/ublk/Makefile         |   9 +-
 tools/testing/selftests/ublk/common.c         |  55 +++
 tools/testing/selftests/ublk/file_backed.c    | 167 ++++-----
 tools/testing/selftests/ublk/kublk.c          |  33 +-
 tools/testing/selftests/ublk/kublk.h          |  85 +++--
 tools/testing/selftests/ublk/null.c           |  72 +++-
 tools/testing/selftests/ublk/stripe.c         | 318 ++++++++++++++++++
 tools/testing/selftests/ublk/test_common.sh   |  22 ++
 .../testing/selftests/ublk/test_generic_01.sh |  44 +++
 tools/testing/selftests/ublk/test_null_02.sh  |  20 ++
 .../testing/selftests/ublk/test_stripe_01.sh  |  34 ++
 .../testing/selftests/ublk/test_stripe_02.sh  |  24 ++
 tools/testing/selftests/ublk/trace/seq_io.bt  |  25 ++
 13 files changed, 759 insertions(+), 149 deletions(-)
 create mode 100644 tools/testing/selftests/ublk/common.c
 create mode 100644 tools/testing/selftests/ublk/stripe.c
 create mode 100755 tools/testing/selftests/ublk/test_generic_01.sh
 create mode 100755 tools/testing/selftests/ublk/test_null_02.sh
 create mode 100755 tools/testing/selftests/ublk/test_stripe_01.sh
 create mode 100755 tools/testing/selftests/ublk/test_stripe_02.sh
 create mode 100644 tools/testing/selftests/ublk/trace/seq_io.bt

-- 
2.47.0

[PATCH 1/8] selftests: ublk: add generic_01 for verifying sequential IO order

[Ming Lei]

block layer, ublk and io_uring might re-order IO in the past

- plug

- queue ublk io command via task work

Add one test for verifying if sequential WRITE IO is dispatched in order.

- null target is taken, so we can just observe io order from
`tracepoint:block:block_rq_complete` which represents the dispatch order

- WRITE IO is taken because READ may come from system-wide utility

Cc: Uday Shankar <ushankar@purestorage.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile         |  4 +-
 tools/testing/selftests/ublk/test_common.sh   | 22 ++++++++++
 .../testing/selftests/ublk/test_generic_01.sh | 44 +++++++++++++++++++
 tools/testing/selftests/ublk/trace/seq_io.bt  | 25 +++++++++++
 4 files changed, 94 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/ublk/test_generic_01.sh
 create mode 100644 tools/testing/selftests/ublk/trace/seq_io.bt

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 5d8d5939f051..652ab40adb73 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -3,7 +3,9 @@
 CFLAGS += -O3 -Wl,-no-as-needed -Wall -I $(top_srcdir)
 LDLIBS += -lpthread -lm -luring
 
-TEST_PROGS := test_null_01.sh
+TEST_PROGS := test_generic_01.sh
+
+TEST_PROGS += test_null_01.sh
 TEST_PROGS += test_loop_01.sh
 TEST_PROGS += test_loop_02.sh
 TEST_PROGS += test_loop_03.sh
diff --git a/tools/testing/selftests/ublk/test_common.sh b/tools/testing/selftests/ublk/test_common.sh
index 48fca609e741..75f54ac6b1c4 100755
--- a/tools/testing/selftests/ublk/test_common.sh
+++ b/tools/testing/selftests/ublk/test_common.sh
@@ -3,6 +3,26 @@
 
 UBLK_SKIP_CODE=4
 
+_have_program() {
+	if command -v "$1" >/dev/null 2>&1; then
+		return 0
+	fi
+	return 1
+}
+
+_get_disk_dev_t() {
+	local dev_id=$1
+	local dev
+	local major
+	local minor
+
+	dev=/dev/ublkb"${dev_id}"
+	major=$(stat -c '%Hr' "$dev")
+	minor=$(stat -c '%Lr' "$dev")
+
+	echo $(( (major & 0xfff) << 20 | (minor & 0xfffff) ))
+}
+
 _create_backfile() {
 	local my_size=$1
 	local my_file
@@ -121,6 +141,7 @@ _check_add_dev()
 
 _cleanup_test() {
 	"${UBLK_PROG}" del -a
+	rm -f "$UBLK_TMP"
 }
 
 _have_feature()
@@ -216,6 +237,7 @@ _ublk_test_top_dir()
 	cd "$(dirname "$0")" && pwd
 }
 
+UBLK_TMP=$(mktemp ublk_test_XXXXX)
 UBLK_PROG=$(_ublk_test_top_dir)/kublk
 UBLK_TEST_QUIET=1
 UBLK_TEST_SHOW_RESULT=1
diff --git a/tools/testing/selftests/ublk/test_generic_01.sh b/tools/testing/selftests/ublk/test_generic_01.sh
new file mode 100755
index 000000000000..9227a208ba53
--- /dev/null
+++ b/tools/testing/selftests/ublk/test_generic_01.sh
@@ -0,0 +1,44 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh
+
+TID="generic_01"
+ERR_CODE=0
+
+if ! _have_program bpftrace; then
+	exit "$UBLK_SKIP_CODE"
+fi
+
+_prep_test "null" "sequential io order"
+
+dev_id=$(_add_ublk_dev -t null)
+_check_add_dev $TID $?
+
+dev_t=$(_get_disk_dev_t "$dev_id")
+bpftrace trace/seq_io.bt "$dev_t" "W" 1 > "$UBLK_TMP" 2>&1 &
+btrace_pid=$!
+sleep 2
+
+if ! kill -0 "$btrace_pid" > /dev/null 2>&1; then
+	_cleanup_test "null"
+	exit "$UBLK_SKIP_CODE"
+fi
+
+# run fio over this ublk disk
+fio --name=write_seq \
+    --filename=/dev/ublkb"${dev_id}" \
+    --ioengine=libaio --iodepth=16 \
+    --rw=write \
+    --size=512M \
+    --direct=1 \
+    --bs=4k > /dev/null 2>&1
+ERR_CODE=$?
+kill "$btrace_pid"
+wait
+if grep -q "io_out_of_order" "$UBLK_TMP"; then
+	cat "$UBLK_TMP"
+	ERR_CODE=255
+fi
+_cleanup_test "null"
+_show_result $TID $ERR_CODE
diff --git a/tools/testing/selftests/ublk/trace/seq_io.bt b/tools/testing/selftests/ublk/trace/seq_io.bt
new file mode 100644
index 000000000000..272ac54c9d5f
--- /dev/null
+++ b/tools/testing/selftests/ublk/trace/seq_io.bt
@@ -0,0 +1,25 @@
+/*
+	$1: 	dev_t
+	$2: 	RWBS
+	$3:     strlen($2)
+*/
+BEGIN {
+	@last_rw[$1, str($2)] = 0;
+}
+tracepoint:block:block_rq_complete
+{
+	$dev = $1;
+	if ((int64)args.dev == $1 && !strncmp(args.rwbs, str($2), $3)) {
+		$last = @last_rw[$dev, str($2)];
+		if ((uint64)args.sector != $last) {
+			printf("io_out_of_order: exp %llu actual %llu\n",
+				args.sector, $last);
+		}
+		@last_rw[$dev, str($2)] = (args.sector + args.nr_sector);
+	}
+	@ios = count();
+}
+
+END {
+	clear(@last_rw);
+}
-- 
2.47.0

[PATCH 2/8] selftests: ublk: add single sqe allocator helper

[Ming Lei]

Unify the sqe allocator helper, and we will use it for supporting
more cases, such as ublk stripe, in which variable sqe allocation
is required.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/file_backed.c | 50 +++++++++++-----------
 tools/testing/selftests/ublk/kublk.c       | 20 ++++-----
 tools/testing/selftests/ublk/kublk.h       | 26 +++++------
 3 files changed, 44 insertions(+), 52 deletions(-)

diff --git a/tools/testing/selftests/ublk/file_backed.c b/tools/testing/selftests/ublk/file_backed.c
index 570a5158b665..f58fa4ec9b51 100644
--- a/tools/testing/selftests/ublk/file_backed.c
+++ b/tools/testing/selftests/ublk/file_backed.c
@@ -69,44 +69,42 @@ static int loop_queue_tgt_rw_io(struct ublk_queue *q, const struct ublksrv_io_de
 {
 	int zc = ublk_queue_use_zc(q);
 	enum io_uring_op op = ublk_to_uring_op(iod, zc);
-	struct io_uring_sqe *reg;
-	struct io_uring_sqe *rw;
-	struct io_uring_sqe *ureg;
+	struct io_uring_sqe *sqe[3];
 
 	if (!zc) {
-		rw = ublk_queue_alloc_sqe(q);
-		if (!rw)
+		ublk_queue_alloc_sqes(q, sqe, 1);
+		if (!sqe[0])
 			return -ENOMEM;
 
-		io_uring_prep_rw(op, rw, 1 /*fds[1]*/,
+		io_uring_prep_rw(op, sqe[0], 1 /*fds[1]*/,
 				(void *)iod->addr,
 				iod->nr_sectors << 9,
 				iod->start_sector << 9);
-		io_uring_sqe_set_flags(rw, IOSQE_FIXED_FILE);
+		io_uring_sqe_set_flags(sqe[0], IOSQE_FIXED_FILE);
 		q->io_inflight++;
 		/* bit63 marks us as tgt io */
-		rw->user_data = build_user_data(tag, op, UBLK_IO_TGT_NORMAL, 1);
+		sqe[0]->user_data = build_user_data(tag, op, UBLK_IO_TGT_NORMAL, 1);
 		return 0;
 	}
 
-	ublk_queue_alloc_sqe3(q, &reg, &rw, &ureg);
+	ublk_queue_alloc_sqes(q, sqe, 3);
 
-	io_uring_prep_buf_register(reg, 0, tag, q->q_id, tag);
-	reg->user_data = build_user_data(tag, 0xfe, 1, 1);
-	reg->flags |= IOSQE_CQE_SKIP_SUCCESS;
-	reg->flags |= IOSQE_IO_LINK;
+	io_uring_prep_buf_register(sqe[0], 0, tag, q->q_id, tag);
+	sqe[0]->user_data = build_user_data(tag, 0xfe, 1, 1);
+	sqe[0]->flags |= IOSQE_CQE_SKIP_SUCCESS;
+	sqe[0]->flags |= IOSQE_IO_LINK;
 
-	io_uring_prep_rw(op, rw, 1 /*fds[1]*/, 0,
+	io_uring_prep_rw(op, sqe[1], 1 /*fds[1]*/, 0,
 		iod->nr_sectors << 9,
 		iod->start_sector << 9);
-	rw->buf_index = tag;
-	rw->flags |= IOSQE_FIXED_FILE;
-	rw->flags |= IOSQE_IO_LINK;
-	rw->user_data = build_user_data(tag, op, UBLK_IO_TGT_ZC_OP, 1);
+	sqe[1]->buf_index = tag;
+	sqe[1]->flags |= IOSQE_FIXED_FILE;
+	sqe[1]->flags |= IOSQE_IO_LINK;
+	sqe[1]->user_data = build_user_data(tag, op, UBLK_IO_TGT_ZC_OP, 1);
 	q->io_inflight++;
 
-	io_uring_prep_buf_unregister(ureg, 0, tag, q->q_id, tag);
-	ureg->user_data = build_user_data(tag, 0xff, UBLK_IO_TGT_ZC_BUF, 1);
+	io_uring_prep_buf_unregister(sqe[2], 0, tag, q->q_id, tag);
+	sqe[2]->user_data = build_user_data(tag, 0xff, UBLK_IO_TGT_ZC_BUF, 1);
 	q->io_inflight++;
 
 	return 0;
@@ -116,17 +114,17 @@ static int loop_queue_tgt_io(struct ublk_queue *q, int tag)
 {
 	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
 	unsigned ublk_op = ublksrv_get_op(iod);
-	struct io_uring_sqe *sqe;
+	struct io_uring_sqe *sqe[1];
 
 	switch (ublk_op) {
 	case UBLK_IO_OP_FLUSH:
-		sqe = ublk_queue_alloc_sqe(q);
-		if (!sqe)
+		ublk_queue_alloc_sqes(q, sqe, 1);
+		if (!sqe[0])
 			return -ENOMEM;
-		io_uring_prep_fsync(sqe, 1 /*fds[1]*/, IORING_FSYNC_DATASYNC);
-		io_uring_sqe_set_flags(sqe, IOSQE_FIXED_FILE);
+		io_uring_prep_fsync(sqe[0], 1 /*fds[1]*/, IORING_FSYNC_DATASYNC);
+		io_uring_sqe_set_flags(sqe[0], IOSQE_FIXED_FILE);
 		q->io_inflight++;
-		sqe->user_data = build_user_data(tag, ublk_op, UBLK_IO_TGT_NORMAL, 1);
+		sqe[0]->user_data = build_user_data(tag, ublk_op, UBLK_IO_TGT_NORMAL, 1);
 		break;
 	case UBLK_IO_OP_WRITE_ZEROES:
 	case UBLK_IO_OP_DISCARD:
diff --git a/tools/testing/selftests/ublk/kublk.c b/tools/testing/selftests/ublk/kublk.c
index 11005a87bcfa..0080cad1f3ae 100644
--- a/tools/testing/selftests/ublk/kublk.c
+++ b/tools/testing/selftests/ublk/kublk.c
@@ -420,7 +420,7 @@ static void ublk_dev_unprep(struct ublk_dev *dev)
 int ublk_queue_io_cmd(struct ublk_queue *q, struct ublk_io *io, unsigned tag)
 {
 	struct ublksrv_io_cmd *cmd;
-	struct io_uring_sqe *sqe;
+	struct io_uring_sqe *sqe[1];
 	unsigned int cmd_op = 0;
 	__u64 user_data;
 
@@ -441,24 +441,24 @@ int ublk_queue_io_cmd(struct ublk_queue *q, struct ublk_io *io, unsigned tag)
 	if (io_uring_sq_space_left(&q->ring) < 1)
 		io_uring_submit(&q->ring);
 
-	sqe = ublk_queue_alloc_sqe(q);
-	if (!sqe) {
+	ublk_queue_alloc_sqes(q, sqe, 1);
+	if (!sqe[0]) {
 		ublk_err("%s: run out of sqe %d, tag %d\n",
 				__func__, q->q_id, tag);
 		return -1;
 	}
 
-	cmd = (struct ublksrv_io_cmd *)ublk_get_sqe_cmd(sqe);
+	cmd = (struct ublksrv_io_cmd *)ublk_get_sqe_cmd(sqe[0]);
 
 	if (cmd_op == UBLK_U_IO_COMMIT_AND_FETCH_REQ)
 		cmd->result = io->result;
 
 	/* These fields should be written once, never change */
-	ublk_set_sqe_cmd_op(sqe, cmd_op);
-	sqe->fd		= 0;	/* dev->fds[0] */
-	sqe->opcode	= IORING_OP_URING_CMD;
-	sqe->flags	= IOSQE_FIXED_FILE;
-	sqe->rw_flags	= 0;
+	ublk_set_sqe_cmd_op(sqe[0], cmd_op);
+	sqe[0]->fd		= 0;	/* dev->fds[0] */
+	sqe[0]->opcode	= IORING_OP_URING_CMD;
+	sqe[0]->flags	= IOSQE_FIXED_FILE;
+	sqe[0]->rw_flags	= 0;
 	cmd->tag	= tag;
 	cmd->q_id	= q->q_id;
 	if (!(q->state & UBLKSRV_NO_BUF))
@@ -467,7 +467,7 @@ int ublk_queue_io_cmd(struct ublk_queue *q, struct ublk_io *io, unsigned tag)
 		cmd->addr	= 0;
 
 	user_data = build_user_data(tag, _IOC_NR(cmd_op), 0, 0);
-	io_uring_sqe_set_data64(sqe, user_data);
+	io_uring_sqe_set_data64(sqe[0], user_data);
 
 	io->flags = 0;
 
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 3ff9ac5104a7..9cd7ab62f258 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -221,28 +221,22 @@ static inline void ublk_dbg(int level, const char *fmt, ...)
 	}
 }
 
-static inline struct io_uring_sqe *ublk_queue_alloc_sqe(struct ublk_queue *q)
+static inline int ublk_queue_alloc_sqes(struct ublk_queue *q,
+		struct io_uring_sqe *sqes[], int nr_sqes)
 {
 	unsigned left = io_uring_sq_space_left(&q->ring);
+	int i;
 
-	if (left < 1)
+	if (left < nr_sqes)
 		io_uring_submit(&q->ring);
-	return io_uring_get_sqe(&q->ring);
-}
-
-static inline void ublk_queue_alloc_sqe3(struct ublk_queue *q,
-		struct io_uring_sqe **sqe1, struct io_uring_sqe **sqe2,
-		struct io_uring_sqe **sqe3)
-{
-	struct io_uring *r = &q->ring;
-	unsigned left = io_uring_sq_space_left(r);
 
-	if (left < 3)
-		io_uring_submit(r);
+	for (i = 0; i < nr_sqes; i++) {
+		sqes[i] = io_uring_get_sqe(&q->ring);
+		if (!sqes[i])
+			return i;
+	}
 
-	*sqe1 = io_uring_get_sqe(r);
-	*sqe2 = io_uring_get_sqe(r);
-	*sqe3 = io_uring_get_sqe(r);
+	return nr_sqes;
 }
 
 static inline void io_uring_prep_buf_register(struct io_uring_sqe *sqe,
-- 
2.47.0

[PATCH 3/8] selftests: ublk: increase max buffer size to 1MB

[Ming Lei]

Increase max buffer size to 1MB, and 64KB is too small to evaluate
performance with builtin ublk server implementation.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/kublk.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 9cd7ab62f258..40b89dcf0704 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -40,7 +40,7 @@
 /* queue idle timeout */
 #define UBLKSRV_IO_IDLE_SECS		20
 
-#define UBLK_IO_MAX_BYTES               65536
+#define UBLK_IO_MAX_BYTES               (1 << 20)
 #define UBLK_MAX_QUEUES                 4
 #define UBLK_QUEUE_DEPTH                128
 
-- 
2.47.0

[PATCH 4/8] selftests: ublk: move common code into common.c

[Ming Lei]

Move two functions for initializing & de-initializing backing file
into common.c.

Also move one common helper into kublk.h.

Prepare for supporting ublk-stripe.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile      |  2 +-
 tools/testing/selftests/ublk/common.c      | 55 ++++++++++++++++++++++
 tools/testing/selftests/ublk/file_backed.c | 52 --------------------
 tools/testing/selftests/ublk/kublk.h       |  2 +
 4 files changed, 58 insertions(+), 53 deletions(-)
 create mode 100644 tools/testing/selftests/ublk/common.c

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 652ab40adb73..03dae5184d08 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -18,7 +18,7 @@ TEST_GEN_PROGS_EXTENDED = kublk
 
 include ../lib.mk
 
-$(TEST_GEN_PROGS_EXTENDED): kublk.c null.c file_backed.c
+$(TEST_GEN_PROGS_EXTENDED): kublk.c null.c file_backed.c common.c
 
 check:
 	shellcheck -x -f gcc *.sh
diff --git a/tools/testing/selftests/ublk/common.c b/tools/testing/selftests/ublk/common.c
new file mode 100644
index 000000000000..01580a6f8519
--- /dev/null
+++ b/tools/testing/selftests/ublk/common.c
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "kublk.h"
+
+void backing_file_tgt_deinit(struct ublk_dev *dev)
+{
+	int i;
+
+	for (i = 1; i < dev->nr_fds; i++) {
+		fsync(dev->fds[i]);
+		close(dev->fds[i]);
+	}
+}
+
+int backing_file_tgt_init(struct ublk_dev *dev)
+{
+	int fd, i;
+
+	assert(dev->nr_fds == 1);
+
+	for (i = 0; i < dev->tgt.nr_backing_files; i++) {
+		char *file = dev->tgt.backing_file[i];
+		unsigned long bytes;
+		struct stat st;
+
+		ublk_dbg(UBLK_DBG_DEV, "%s: file %d: %s\n", __func__, i, file);
+
+		fd = open(file, O_RDWR | O_DIRECT);
+		if (fd < 0) {
+			ublk_err("%s: backing file %s can't be opened: %s\n",
+					__func__, file, strerror(errno));
+			return -EBADF;
+		}
+
+		if (fstat(fd, &st) < 0) {
+			close(fd);
+			return -EBADF;
+		}
+
+		if (S_ISREG(st.st_mode))
+			bytes = st.st_size;
+		else if (S_ISBLK(st.st_mode)) {
+			if (ioctl(fd, BLKGETSIZE64, &bytes) != 0)
+				return -1;
+		} else {
+			return -EINVAL;
+		}
+
+		dev->tgt.backing_file_size[i] = bytes;
+		dev->fds[dev->nr_fds] = fd;
+		dev->nr_fds += 1;
+	}
+
+	return 0;
+}
diff --git a/tools/testing/selftests/ublk/file_backed.c b/tools/testing/selftests/ublk/file_backed.c
index f58fa4ec9b51..a2e8793390a8 100644
--- a/tools/testing/selftests/ublk/file_backed.c
+++ b/tools/testing/selftests/ublk/file_backed.c
@@ -2,58 +2,6 @@
 
 #include "kublk.h"
 
-static void backing_file_tgt_deinit(struct ublk_dev *dev)
-{
-	int i;
-
-	for (i = 1; i < dev->nr_fds; i++) {
-		fsync(dev->fds[i]);
-		close(dev->fds[i]);
-	}
-}
-
-static int backing_file_tgt_init(struct ublk_dev *dev)
-{
-	int fd, i;
-
-	assert(dev->nr_fds == 1);
-
-	for (i = 0; i < dev->tgt.nr_backing_files; i++) {
-		char *file = dev->tgt.backing_file[i];
-		unsigned long bytes;
-		struct stat st;
-
-		ublk_dbg(UBLK_DBG_DEV, "%s: file %d: %s\n", __func__, i, file);
-
-		fd = open(file, O_RDWR | O_DIRECT);
-		if (fd < 0) {
-			ublk_err("%s: backing file %s can't be opened: %s\n",
-					__func__, file, strerror(errno));
-			return -EBADF;
-		}
-
-		if (fstat(fd, &st) < 0) {
-			close(fd);
-			return -EBADF;
-		}
-
-		if (S_ISREG(st.st_mode))
-			bytes = st.st_size;
-		else if (S_ISBLK(st.st_mode)) {
-			if (ioctl(fd, BLKGETSIZE64, &bytes) != 0)
-				return -1;
-		} else {
-			return -EINVAL;
-		}
-
-		dev->tgt.backing_file_size[i] = bytes;
-		dev->fds[dev->nr_fds] = fd;
-		dev->nr_fds += 1;
-	}
-
-	return 0;
-}
-
 static enum io_uring_op ublk_to_uring_op(const struct ublksrv_io_desc *iod, int zc)
 {
 	unsigned ublk_op = ublksrv_get_op(iod);
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 40b89dcf0704..eaadd7364e25 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -320,4 +320,6 @@ static inline int ublk_queue_use_zc(const struct ublk_queue *q)
 extern const struct ublk_tgt_ops null_tgt_ops;
 extern const struct ublk_tgt_ops loop_tgt_ops;
 
+void backing_file_tgt_deinit(struct ublk_dev *dev);
+int backing_file_tgt_init(struct ublk_dev *dev);
 #endif
-- 
2.47.0

[PATCH 5/8] selftests: ublk: prepare for supporting stripe target

[Ming Lei]

- pass 'truct dev_ctx *ctx' to target init function

- add 'private_data' to 'struct ublk_dev' for storing target specific data

- add 'private_data' to 'struct ublk_io' for storing per-IO data

- add 'tgt_ios' to 'struct ublk_io' for counting how many io_uring ios
for handling the current io command

- add helper ublk_get_io() for supporting stripe target

- add two helpers for simplifying target io handling

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/file_backed.c |  2 +-
 tools/testing/selftests/ublk/kublk.c       |  6 ++--
 tools/testing/selftests/ublk/kublk.h       | 34 +++++++++++++++++++++-
 tools/testing/selftests/ublk/null.c        |  2 +-
 4 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/ublk/file_backed.c b/tools/testing/selftests/ublk/file_backed.c
index a2e8793390a8..e2287eedaac8 100644
--- a/tools/testing/selftests/ublk/file_backed.c
+++ b/tools/testing/selftests/ublk/file_backed.c
@@ -123,7 +123,7 @@ static void ublk_loop_io_done(struct ublk_queue *q, int tag,
 	q->io_inflight--;
 }
 
-static int ublk_loop_tgt_init(struct ublk_dev *dev)
+static int ublk_loop_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
 {
 	unsigned long long bytes;
 	int ret;
diff --git a/tools/testing/selftests/ublk/kublk.c b/tools/testing/selftests/ublk/kublk.c
index 0080cad1f3ae..2dd17663ef30 100644
--- a/tools/testing/selftests/ublk/kublk.c
+++ b/tools/testing/selftests/ublk/kublk.c
@@ -381,7 +381,7 @@ static int ublk_queue_init(struct ublk_queue *q)
 
 #define WAIT_USEC 	100000
 #define MAX_WAIT_USEC 	(3 * 1000000)
-static int ublk_dev_prep(struct ublk_dev *dev)
+static int ublk_dev_prep(const struct dev_ctx *ctx, struct ublk_dev *dev)
 {
 	int dev_id = dev->dev_info.dev_id;
 	unsigned int wait_usec = 0;
@@ -404,7 +404,7 @@ static int ublk_dev_prep(struct ublk_dev *dev)
 
 	dev->fds[0] = fd;
 	if (dev->tgt.ops->init_tgt)
-		ret = dev->tgt.ops->init_tgt(dev);
+		ret = dev->tgt.ops->init_tgt(ctx, dev);
 	if (ret)
 		close(dev->fds[0]);
 	return ret;
@@ -666,7 +666,7 @@ static int ublk_start_daemon(const struct dev_ctx *ctx, struct ublk_dev *dev)
 
 	ublk_dbg(UBLK_DBG_DEV, "%s enter\n", __func__);
 
-	ret = ublk_dev_prep(dev);
+	ret = ublk_dev_prep(ctx, dev);
 	if (ret)
 		return ret;
 
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index eaadd7364e25..4eee9ad2bead 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -94,11 +94,14 @@ struct ublk_io {
 	unsigned short refs;		/* used by target code only */
 
 	int result;
+
+	unsigned short tgt_ios;
+	void *private_data;
 };
 
 struct ublk_tgt_ops {
 	const char *name;
-	int (*init_tgt)(struct ublk_dev *);
+	int (*init_tgt)(const struct dev_ctx *ctx, struct ublk_dev *);
 	void (*deinit_tgt)(struct ublk_dev *);
 
 	int (*queue_io)(struct ublk_queue *, int tag);
@@ -146,6 +149,8 @@ struct ublk_dev {
 	int nr_fds;
 	int ctrl_fd;
 	struct io_uring ring;
+
+	void *private_data;
 };
 
 #ifndef offsetof
@@ -303,6 +308,11 @@ static inline void ublk_set_sqe_cmd_op(struct io_uring_sqe *sqe, __u32 cmd_op)
 	addr[1] = 0;
 }
 
+static inline struct ublk_io *ublk_get_io(struct ublk_queue *q, unsigned tag)
+{
+	return &q->ios[tag];
+}
+
 static inline int ublk_complete_io(struct ublk_queue *q, unsigned tag, int res)
 {
 	struct ublk_io *io = &q->ios[tag];
@@ -312,6 +322,28 @@ static inline int ublk_complete_io(struct ublk_queue *q, unsigned tag, int res)
 	return ublk_queue_io_cmd(q, io, tag);
 }
 
+static inline void ublk_queued_tgt_io(struct ublk_queue *q, unsigned tag, int queued)
+{
+	if (queued < 0)
+		ublk_complete_io(q, tag, queued);
+	else {
+		struct ublk_io *io = ublk_get_io(q, tag);
+
+		q->io_inflight += queued;
+		io->tgt_ios = queued;
+		io->result = 0;
+	}
+}
+
+static inline int ublk_completed_tgt_io(struct ublk_queue *q, unsigned tag)
+{
+	struct ublk_io *io = ublk_get_io(q, tag);
+
+	q->io_inflight--;
+
+	return --io->tgt_ios == 0;
+}
+
 static inline int ublk_queue_use_zc(const struct ublk_queue *q)
 {
 	return q->state & UBLKSRV_ZC;
diff --git a/tools/testing/selftests/ublk/null.c b/tools/testing/selftests/ublk/null.c
index b6ef16a8f514..975a11db22fd 100644
--- a/tools/testing/selftests/ublk/null.c
+++ b/tools/testing/selftests/ublk/null.c
@@ -2,7 +2,7 @@
 
 #include "kublk.h"
 
-static int ublk_null_tgt_init(struct ublk_dev *dev)
+static int ublk_null_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
 {
 	const struct ublksrv_ctrl_dev_info *info = &dev->dev_info;
 	unsigned long dev_size = 250UL << 30;
-- 
2.47.0

[PATCH 6/8] selftests: ublk: enable zero copy for null target

[Ming Lei]

Enable zero copy for null target so that we can evaluate performance
from zero copy or not.

Also this should be the simplest ublk zero copy implementation, which
can be served as zc example.

Add test for covering 'add -t null -z'.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile        |  1 +
 tools/testing/selftests/ublk/kublk.h         |  5 ++
 tools/testing/selftests/ublk/null.c          | 70 +++++++++++++++++++-
 tools/testing/selftests/ublk/test_null_02.sh | 20 ++++++
 4 files changed, 95 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/ublk/test_null_02.sh

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 03dae5184d08..36f50c000e55 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -6,6 +6,7 @@ LDLIBS += -lpthread -lm -luring
 TEST_PROGS := test_generic_01.sh
 
 TEST_PROGS += test_null_01.sh
+TEST_PROGS += test_null_02.sh
 TEST_PROGS += test_loop_01.sh
 TEST_PROGS += test_loop_02.sh
 TEST_PROGS += test_loop_03.sh
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 4eee9ad2bead..48ca16055710 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -198,6 +198,11 @@ static inline unsigned int user_data_to_tgt_data(__u64 user_data)
 	return (user_data >> 24) & 0xffff;
 }
 
+static inline unsigned short ublk_cmd_op_nr(unsigned int op)
+{
+	return _IOC_NR(op);
+}
+
 static inline void ublk_err(const char *fmt, ...)
 {
 	va_list ap;
diff --git a/tools/testing/selftests/ublk/null.c b/tools/testing/selftests/ublk/null.c
index 975a11db22fd..899875ff50fe 100644
--- a/tools/testing/selftests/ublk/null.c
+++ b/tools/testing/selftests/ublk/null.c
@@ -2,6 +2,14 @@
 
 #include "kublk.h"
 
+#ifndef IORING_NOP_INJECT_RESULT
+#define IORING_NOP_INJECT_RESULT        (1U << 0)
+#endif
+
+#ifndef IORING_NOP_FIXED_BUFFER
+#define IORING_NOP_FIXED_BUFFER         (1U << 3)
+#endif
+
 static int ublk_null_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
 {
 	const struct ublksrv_ctrl_dev_info *info = &dev->dev_info;
@@ -20,14 +28,73 @@ static int ublk_null_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
 		},
 	};
 
+	if (info->flags & UBLK_F_SUPPORT_ZERO_COPY)
+		dev->tgt.sq_depth = dev->tgt.cq_depth = 2 * info->queue_depth;
 	return 0;
 }
 
+static int null_queue_zc_io(struct ublk_queue *q, int tag)
+{
+	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
+	unsigned ublk_op = ublksrv_get_op(iod);
+	struct io_uring_sqe *sqe[3];
+
+	ublk_queue_alloc_sqes(q, sqe, 3);
+
+	io_uring_prep_buf_register(sqe[0], 0, tag, q->q_id, tag);
+	sqe[0]->user_data = build_user_data(tag,
+			ublk_cmd_op_nr(sqe[0]->cmd_op), 0, 1);
+	sqe[0]->flags |= IOSQE_CQE_SKIP_SUCCESS | IOSQE_IO_HARDLINK;
+
+	io_uring_prep_nop(sqe[1]);
+	sqe[1]->buf_index = tag;
+	sqe[1]->flags |= IOSQE_FIXED_FILE | IOSQE_IO_HARDLINK;
+	sqe[1]->rw_flags = IORING_NOP_FIXED_BUFFER | IORING_NOP_INJECT_RESULT;
+	sqe[1]->len = iod->nr_sectors << 9; 	/* injected result */
+	sqe[1]->user_data = build_user_data(tag, ublk_op, 0, 1);
+
+	io_uring_prep_buf_unregister(sqe[2], 0, tag, q->q_id, tag);
+	sqe[2]->user_data = build_user_data(tag, ublk_cmd_op_nr(sqe[2]->cmd_op), 0, 1);
+
+	// buf register is marked as IOSQE_CQE_SKIP_SUCCESS
+	return 2;
+}
+
+static void ublk_null_io_done(struct ublk_queue *q, int tag,
+		const struct io_uring_cqe *cqe)
+{
+	unsigned op = user_data_to_op(cqe->user_data);
+	struct ublk_io *io = ublk_get_io(q, tag);
+
+	if (cqe->res < 0 || op != ublk_cmd_op_nr(UBLK_U_IO_UNREGISTER_IO_BUF)) {
+		if (!io->result)
+			io->result = cqe->res;
+		if (cqe->res < 0)
+			ublk_err("%s: io failed op %x user_data %lx\n",
+					__func__, op, cqe->user_data);
+	}
+
+	/* buffer register op is IOSQE_CQE_SKIP_SUCCESS */
+	if (op == ublk_cmd_op_nr(UBLK_U_IO_REGISTER_IO_BUF))
+		io->tgt_ios += 1;
+
+	if (ublk_completed_tgt_io(q, tag))
+		ublk_complete_io(q, tag, io->result);
+}
+
 static int ublk_null_queue_io(struct ublk_queue *q, int tag)
 {
 	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
+	int zc = ublk_queue_use_zc(q);
+	int queued;
+
+	if (!zc) {
+		ublk_complete_io(q, tag, iod->nr_sectors << 9);
+		return 0;
+	}
 
-	ublk_complete_io(q, tag, iod->nr_sectors << 9);
+	queued = null_queue_zc_io(q, tag);
+	ublk_queued_tgt_io(q, tag, queued);
 	return 0;
 }
 
@@ -35,4 +102,5 @@ const struct ublk_tgt_ops null_tgt_ops = {
 	.name = "null",
 	.init_tgt = ublk_null_tgt_init,
 	.queue_io = ublk_null_queue_io,
+	.tgt_io_done = ublk_null_io_done,
 };
diff --git a/tools/testing/selftests/ublk/test_null_02.sh b/tools/testing/selftests/ublk/test_null_02.sh
new file mode 100755
index 000000000000..5633ca876655
--- /dev/null
+++ b/tools/testing/selftests/ublk/test_null_02.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh
+
+TID="null_02"
+ERR_CODE=0
+
+_prep_test "null" "basic IO test with zero copy"
+
+dev_id=$(_add_ublk_dev -t null -z)
+_check_add_dev $TID $?
+
+# run fio over the two disks
+fio --name=job1 --filename=/dev/ublkb"${dev_id}" --ioengine=libaio --rw=readwrite --iodepth=32 --size=256M > /dev/null 2>&1
+ERR_CODE=$?
+
+_cleanup_test "null"
+
+_show_result $TID $ERR_CODE
-- 
2.47.0

[PATCH 7/8] selftests: ublk: simplify loop io completion

[Ming Lei]

Use the added target io handling helpers for simplifying loop io
completion.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/file_backed.c | 91 +++++++++++-----------
 tools/testing/selftests/ublk/kublk.h       |  4 -
 2 files changed, 47 insertions(+), 48 deletions(-)

diff --git a/tools/testing/selftests/ublk/file_backed.c b/tools/testing/selftests/ublk/file_backed.c
index e2287eedaac8..6f34eabfae97 100644
--- a/tools/testing/selftests/ublk/file_backed.c
+++ b/tools/testing/selftests/ublk/file_backed.c
@@ -13,8 +13,22 @@ static enum io_uring_op ublk_to_uring_op(const struct ublksrv_io_desc *iod, int
 	assert(0);
 }
 
+static int loop_queue_flush_io(struct ublk_queue *q, const struct ublksrv_io_desc *iod, int tag)
+{
+	unsigned ublk_op = ublksrv_get_op(iod);
+	struct io_uring_sqe *sqe[1];
+
+	ublk_queue_alloc_sqes(q, sqe, 1);
+	io_uring_prep_fsync(sqe[0], 1 /*fds[1]*/, IORING_FSYNC_DATASYNC);
+	io_uring_sqe_set_flags(sqe[0], IOSQE_FIXED_FILE);
+	/* bit63 marks us as tgt io */
+	sqe[0]->user_data = build_user_data(tag, ublk_op, 0, 1);
+	return 1;
+}
+
 static int loop_queue_tgt_rw_io(struct ublk_queue *q, const struct ublksrv_io_desc *iod, int tag)
 {
+	unsigned ublk_op = ublksrv_get_op(iod);
 	int zc = ublk_queue_use_zc(q);
 	enum io_uring_op op = ublk_to_uring_op(iod, zc);
 	struct io_uring_sqe *sqe[3];
@@ -29,98 +43,87 @@ static int loop_queue_tgt_rw_io(struct ublk_queue *q, const struct ublksrv_io_de
 				iod->nr_sectors << 9,
 				iod->start_sector << 9);
 		io_uring_sqe_set_flags(sqe[0], IOSQE_FIXED_FILE);
-		q->io_inflight++;
 		/* bit63 marks us as tgt io */
-		sqe[0]->user_data = build_user_data(tag, op, UBLK_IO_TGT_NORMAL, 1);
-		return 0;
+		sqe[0]->user_data = build_user_data(tag, ublk_op, 0, 1);
+		return 1;
 	}
 
 	ublk_queue_alloc_sqes(q, sqe, 3);
 
 	io_uring_prep_buf_register(sqe[0], 0, tag, q->q_id, tag);
-	sqe[0]->user_data = build_user_data(tag, 0xfe, 1, 1);
-	sqe[0]->flags |= IOSQE_CQE_SKIP_SUCCESS;
-	sqe[0]->flags |= IOSQE_IO_LINK;
+	sqe[0]->flags |= IOSQE_CQE_SKIP_SUCCESS | IOSQE_IO_HARDLINK;
+	sqe[0]->user_data = build_user_data(tag,
+			ublk_cmd_op_nr(sqe[0]->cmd_op), 0, 1);
 
 	io_uring_prep_rw(op, sqe[1], 1 /*fds[1]*/, 0,
 		iod->nr_sectors << 9,
 		iod->start_sector << 9);
 	sqe[1]->buf_index = tag;
-	sqe[1]->flags |= IOSQE_FIXED_FILE;
-	sqe[1]->flags |= IOSQE_IO_LINK;
-	sqe[1]->user_data = build_user_data(tag, op, UBLK_IO_TGT_ZC_OP, 1);
-	q->io_inflight++;
+	sqe[1]->flags |= IOSQE_FIXED_FILE | IOSQE_IO_HARDLINK;
+	sqe[1]->user_data = build_user_data(tag, ublk_op, 0, 1);
 
 	io_uring_prep_buf_unregister(sqe[2], 0, tag, q->q_id, tag);
-	sqe[2]->user_data = build_user_data(tag, 0xff, UBLK_IO_TGT_ZC_BUF, 1);
-	q->io_inflight++;
+	sqe[2]->user_data = build_user_data(tag, ublk_cmd_op_nr(sqe[2]->cmd_op), 0, 1);
 
-	return 0;
+	return 2;
 }
 
 static int loop_queue_tgt_io(struct ublk_queue *q, int tag)
 {
 	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
 	unsigned ublk_op = ublksrv_get_op(iod);
-	struct io_uring_sqe *sqe[1];
+	int ret;
 
 	switch (ublk_op) {
 	case UBLK_IO_OP_FLUSH:
-		ublk_queue_alloc_sqes(q, sqe, 1);
-		if (!sqe[0])
-			return -ENOMEM;
-		io_uring_prep_fsync(sqe[0], 1 /*fds[1]*/, IORING_FSYNC_DATASYNC);
-		io_uring_sqe_set_flags(sqe[0], IOSQE_FIXED_FILE);
-		q->io_inflight++;
-		sqe[0]->user_data = build_user_data(tag, ublk_op, UBLK_IO_TGT_NORMAL, 1);
+		ret = loop_queue_flush_io(q, iod, tag);
 		break;
 	case UBLK_IO_OP_WRITE_ZEROES:
 	case UBLK_IO_OP_DISCARD:
-		return -ENOTSUP;
+		ret = -ENOTSUP;
+		break;
 	case UBLK_IO_OP_READ:
 	case UBLK_IO_OP_WRITE:
-		loop_queue_tgt_rw_io(q, iod, tag);
+		ret = loop_queue_tgt_rw_io(q, iod, tag);
 		break;
 	default:
-		return -EINVAL;
+		ret = -EINVAL;
+		break;
 	}
 
 	ublk_dbg(UBLK_DBG_IO, "%s: tag %d ublk io %x %llx %u\n", __func__, tag,
 			iod->op_flags, iod->start_sector, iod->nr_sectors << 9);
-	return 1;
+	return ret;
 }
 
 static int ublk_loop_queue_io(struct ublk_queue *q, int tag)
 {
 	int queued = loop_queue_tgt_io(q, tag);
 
-	if (queued < 0)
-		ublk_complete_io(q, tag, queued);
-
+	ublk_queued_tgt_io(q, tag, queued);
 	return 0;
 }
 
 static void ublk_loop_io_done(struct ublk_queue *q, int tag,
 		const struct io_uring_cqe *cqe)
 {
-	int cqe_tag = user_data_to_tag(cqe->user_data);
-	unsigned tgt_data = user_data_to_tgt_data(cqe->user_data);
-	int res = cqe->res;
+	unsigned op = user_data_to_op(cqe->user_data);
+	struct ublk_io *io = ublk_get_io(q, tag);
+
+	if (cqe->res < 0 || op != ublk_cmd_op_nr(UBLK_U_IO_UNREGISTER_IO_BUF)) {
+		if (!io->result)
+			io->result = cqe->res;
+		if (cqe->res < 0)
+			ublk_err("%s: io failed op %x user_data %lx\n",
+					__func__, op, cqe->user_data);
+	}
 
-	if (res < 0 || tgt_data == UBLK_IO_TGT_NORMAL)
-		goto complete;
+	/* buffer register op is IOSQE_CQE_SKIP_SUCCESS */
+	if (op == ublk_cmd_op_nr(UBLK_U_IO_REGISTER_IO_BUF))
+		io->tgt_ios += 1;
 
-	if (tgt_data == UBLK_IO_TGT_ZC_OP) {
-		ublk_set_io_res(q, tag, cqe->res);
-		goto exit;
-	}
-	assert(tgt_data == UBLK_IO_TGT_ZC_BUF);
-	res = ublk_get_io_res(q, tag);
-complete:
-	assert(tag == cqe_tag);
-	ublk_complete_io(q, tag, res);
-exit:
-	q->io_inflight--;
+	if (ublk_completed_tgt_io(q, tag))
+		ublk_complete_io(q, tag, io->result);
 }
 
 static int ublk_loop_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 48ca16055710..02f0bff7918c 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -44,10 +44,6 @@
 #define UBLK_MAX_QUEUES                 4
 #define UBLK_QUEUE_DEPTH                128
 
-#define UBLK_IO_TGT_NORMAL 		0
-#define UBLK_IO_TGT_ZC_BUF 		1
-#define UBLK_IO_TGT_ZC_OP 		2
-
 #define UBLK_DBG_DEV            (1U << 0)
 #define UBLK_DBG_QUEUE          (1U << 1)
 #define UBLK_DBG_IO_CMD         (1U << 2)
-- 
2.47.0

[PATCH 8/8] selftests: ublk: add stripe target

[Ming Lei]

Add ublk stripe target which can take 1~4 underlying backing files
or block device, with stripe size 4k ~ 512K.

Add two basic tests(write verify & mkfs/mount/umount) over ublk/stripe.

This target is helpful to cover multiple IOs aiming at same
fixed/registered IO kernel buffer.

It is also capable of verifying vectored registered (kernel)buffers
in future for zero copy, so far it isn't supported yet.

Todo: support vectored registered kernel buffer for ublk/zc.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 tools/testing/selftests/ublk/Makefile         |   4 +-
 tools/testing/selftests/ublk/kublk.c          |   7 +-
 tools/testing/selftests/ublk/kublk.h          |  12 +
 tools/testing/selftests/ublk/stripe.c         | 318 ++++++++++++++++++
 .../testing/selftests/ublk/test_stripe_01.sh  |  34 ++
 .../testing/selftests/ublk/test_stripe_02.sh  |  24 ++
 6 files changed, 397 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/ublk/stripe.c
 create mode 100755 tools/testing/selftests/ublk/test_stripe_01.sh
 create mode 100755 tools/testing/selftests/ublk/test_stripe_02.sh

diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 36f50c000e55..7817afe29005 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -11,6 +11,8 @@ TEST_PROGS += test_loop_01.sh
 TEST_PROGS += test_loop_02.sh
 TEST_PROGS += test_loop_03.sh
 TEST_PROGS += test_loop_04.sh
+TEST_PROGS += test_stripe_01.sh
+TEST_PROGS += test_stripe_02.sh
 
 TEST_PROGS += test_stress_01.sh
 TEST_PROGS += test_stress_02.sh
@@ -19,7 +21,7 @@ TEST_GEN_PROGS_EXTENDED = kublk
 
 include ../lib.mk
 
-$(TEST_GEN_PROGS_EXTENDED): kublk.c null.c file_backed.c common.c
+$(TEST_GEN_PROGS_EXTENDED): kublk.c null.c file_backed.c common.c stripe.c
 
 check:
 	shellcheck -x -f gcc *.sh
diff --git a/tools/testing/selftests/ublk/kublk.c b/tools/testing/selftests/ublk/kublk.c
index 2dd17663ef30..05147b53c361 100644
--- a/tools/testing/selftests/ublk/kublk.c
+++ b/tools/testing/selftests/ublk/kublk.c
@@ -9,6 +9,7 @@ unsigned int ublk_dbg_mask = UBLK_LOG;
 static const struct ublk_tgt_ops *tgt_ops_list[] = {
 	&null_tgt_ops,
 	&loop_tgt_ops,
+	&stripe_tgt_ops,
 };
 
 static const struct ublk_tgt_ops *ublk_find_tgt(const char *name)
@@ -1060,8 +1061,9 @@ int main(int argc, char *argv[])
 		{ "depth",		1,	NULL, 'd' },
 		{ "debug_mask",		1,	NULL,  0  },
 		{ "quiet",		0,	NULL,  0  },
-		{ "zero_copy",          1,      NULL, 'z' },
+		{ "zero_copy",          0,      NULL, 'z' },
 		{ "foreground",		0,	NULL,  0  },
+		{ "chunk_size", 	1,	NULL,  0  },
 		{ 0, 0, 0, 0 }
 	};
 	int option_idx, opt;
@@ -1071,6 +1073,7 @@ int main(int argc, char *argv[])
 		.nr_hw_queues	=	2,
 		.dev_id		=	-1,
 		.tgt_type	=	"unknown",
+		.chunk_size 	= 	65536, 	/* def chunk size is 64K */
 	};
 	int ret = -EINVAL, i;
 
@@ -1107,6 +1110,8 @@ int main(int argc, char *argv[])
 				ublk_dbg_mask = 0;
 			if (!strcmp(longopts[option_idx].name, "foreground"))
 				ctx.fg = 1;
+			if (!strcmp(longopts[option_idx].name, "chunk_size"))
+				ctx.chunk_size = strtol(optarg, NULL, 10);
 		}
 	}
 
diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests/ublk/kublk.h
index 02f0bff7918c..f31a5c4d4143 100644
--- a/tools/testing/selftests/ublk/kublk.h
+++ b/tools/testing/selftests/ublk/kublk.h
@@ -19,6 +19,7 @@
 #include <sys/inotify.h>
 #include <sys/wait.h>
 #include <sys/eventfd.h>
+#include <sys/uio.h>
 #include <liburing.h>
 #include <linux/ublk_cmd.h>
 #include "ublk_dep.h"
@@ -66,6 +67,9 @@ struct dev_ctx {
 	unsigned int	all:1;
 	unsigned int	fg:1;
 
+	/* stripe */
+	unsigned int    chunk_size;
+
 	int _evtfd;
 };
 
@@ -352,7 +356,15 @@ static inline int ublk_queue_use_zc(const struct ublk_queue *q)
 
 extern const struct ublk_tgt_ops null_tgt_ops;
 extern const struct ublk_tgt_ops loop_tgt_ops;
+extern const struct ublk_tgt_ops stripe_tgt_ops;
 
 void backing_file_tgt_deinit(struct ublk_dev *dev);
 int backing_file_tgt_init(struct ublk_dev *dev);
+
+static inline unsigned int ilog2(unsigned int x)
+{
+	if (x == 0)
+		return 0;
+	return (sizeof(x) * 8 - 1) - __builtin_clz(x);
+}
 #endif
diff --git a/tools/testing/selftests/ublk/stripe.c b/tools/testing/selftests/ublk/stripe.c
new file mode 100644
index 000000000000..98c564b12f3c
--- /dev/null
+++ b/tools/testing/selftests/ublk/stripe.c
@@ -0,0 +1,318 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "kublk.h"
+
+#define NR_STRIPE  MAX_BACK_FILES
+
+struct stripe_conf {
+	unsigned nr_files;
+	unsigned shift;
+};
+
+struct stripe {
+	loff_t 		start;
+	unsigned 	nr_sects;
+	int 		seq;
+
+	struct iovec 	*vec;
+	unsigned 	nr_vec;
+	unsigned 	cap;
+};
+
+struct stripe_array {
+	struct stripe 	s[NR_STRIPE];
+	unsigned 	nr;
+	struct iovec 	_vec[];
+};
+
+static inline const struct stripe_conf *get_chunk_shift(const struct ublk_queue *q)
+{
+	return (struct stripe_conf *)q->dev->private_data;
+}
+
+static inline unsigned calculate_nr_vec(const struct stripe_conf *conf,
+		const struct ublksrv_io_desc *iod)
+{
+	const unsigned shift = conf->shift - 9;
+	const unsigned unit_sects = conf->nr_files << shift;
+	loff_t start = iod->start_sector;
+	loff_t end = start + iod->nr_sectors;
+
+	return (end / unit_sects) - (start / unit_sects) + 1;
+}
+
+static struct stripe_array *alloc_stripe_array(const struct stripe_conf *conf,
+		const struct ublksrv_io_desc *iod)
+{
+	unsigned nr_vecs = calculate_nr_vec(conf, iod);
+	unsigned total = nr_vecs * conf->nr_files;
+	struct stripe_array *s;
+	int i;
+
+	s = malloc(sizeof(*s) + total * sizeof(struct iovec));
+
+	s->nr = 0;
+	for (i = 0; i < conf->nr_files; i++) {
+		struct stripe *t = &s->s[i];
+
+		t->nr_vec = 0;
+		t->vec = &s->_vec[i * nr_vecs];
+		t->nr_sects = 0;
+		t->cap = nr_vecs;
+	}
+
+	return s;
+}
+
+static void free_stripe_array(struct stripe_array *s)
+{
+	free(s);
+}
+
+static void calculate_stripe_array(const struct stripe_conf *conf,
+		const struct ublksrv_io_desc *iod, struct stripe_array *s)
+{
+	const unsigned shift = conf->shift - 9;
+	const unsigned chunk_sects = 1 << shift;
+	const unsigned unit_sects = conf->nr_files << shift;
+	off64_t start = iod->start_sector;
+	off64_t end = start + iod->nr_sectors;
+	unsigned long done = 0;
+	unsigned idx = 0;
+
+	while (start < end) {
+		unsigned nr_sects = chunk_sects - (start & (chunk_sects - 1));
+		loff_t unit_off = (start / unit_sects) * unit_sects;
+		unsigned seq = (start - unit_off) >> shift;
+		struct stripe *this = &s->s[idx];
+		loff_t stripe_off = (unit_off / conf->nr_files) +
+			(start & (chunk_sects - 1));
+
+		if (nr_sects > end - start)
+			nr_sects = end - start;
+		if (this->nr_sects == 0) {
+			this->nr_sects = nr_sects;
+			this->start = stripe_off;
+			this->seq = seq;
+			s->nr += 1;
+		} else {
+			assert(seq == this->seq);
+			assert(this->start + this->nr_sects == stripe_off);
+			this->nr_sects += nr_sects;
+		}
+
+		assert(this->nr_vec < this->cap);
+		this->vec[this->nr_vec].iov_base = (void *)(iod->addr + done);
+		this->vec[this->nr_vec++].iov_len = nr_sects << 9;
+
+		start += nr_sects;
+		done += nr_sects << 9;
+		idx = (idx + 1) % conf->nr_files;
+	}
+}
+
+static inline enum io_uring_op stripe_to_uring_op(const struct ublksrv_io_desc *iod)
+{
+	unsigned ublk_op = ublksrv_get_op(iod);
+
+	if (ublk_op == UBLK_IO_OP_READ)
+		return IORING_OP_READV;
+	else if (ublk_op == UBLK_IO_OP_WRITE)
+		return IORING_OP_WRITEV;
+	assert(0);
+}
+
+static int stripe_queue_tgt_rw_io(struct ublk_queue *q, const struct ublksrv_io_desc *iod, int tag)
+{
+	const struct stripe_conf *conf = get_chunk_shift(q);
+	enum io_uring_op op = stripe_to_uring_op(iod);
+	struct io_uring_sqe *sqe[NR_STRIPE];
+	struct stripe_array *s = alloc_stripe_array(conf, iod);
+	struct ublk_io *io = ublk_get_io(q, tag);
+	int i;
+
+	io->private_data = s;
+	calculate_stripe_array(conf, iod, s);
+
+	ublk_queue_alloc_sqes(q, sqe, s->nr);
+	for (i = 0; i < s->nr; i++) {
+		struct stripe *t = &s->s[i];
+
+		io_uring_prep_rw(op, sqe[i],
+				t->seq + 1,
+				(void *)t->vec,
+				t->nr_vec,
+				t->start << 9);
+		io_uring_sqe_set_flags(sqe[i], IOSQE_FIXED_FILE);
+		/* bit63 marks us as tgt io */
+		sqe[i]->user_data = build_user_data(tag, ublksrv_get_op(iod), i, 1);
+	}
+	return s->nr;
+}
+
+static int handle_flush(struct ublk_queue *q, const struct ublksrv_io_desc *iod, int tag)
+{
+	const struct stripe_conf *conf = get_chunk_shift(q);
+	struct io_uring_sqe *sqe[NR_STRIPE];
+	int i;
+
+	ublk_queue_alloc_sqes(q, sqe, conf->nr_files);
+	for (i = 0; i < conf->nr_files; i++) {
+		io_uring_prep_fsync(sqe[i], i + 1, IORING_FSYNC_DATASYNC);
+		io_uring_sqe_set_flags(sqe[i], IOSQE_FIXED_FILE);
+		sqe[i]->user_data = build_user_data(tag, UBLK_IO_OP_FLUSH, 0, 1);
+	}
+	return conf->nr_files;
+}
+
+static int stripe_queue_tgt_io(struct ublk_queue *q, int tag)
+{
+	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
+	unsigned ublk_op = ublksrv_get_op(iod);
+	int ret = 0;
+
+	switch (ublk_op) {
+	case UBLK_IO_OP_FLUSH:
+		ret = handle_flush(q, iod, tag);
+		break;
+	case UBLK_IO_OP_WRITE_ZEROES:
+	case UBLK_IO_OP_DISCARD:
+		ret = -ENOTSUP;
+		break;
+	case UBLK_IO_OP_READ:
+	case UBLK_IO_OP_WRITE:
+		ret = stripe_queue_tgt_rw_io(q, iod, tag);
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+	ublk_dbg(UBLK_DBG_IO, "%s: tag %d ublk io %x %llx %u ret %d\n", __func__, tag,
+			iod->op_flags, iod->start_sector, iod->nr_sectors << 9, ret);
+	return ret;
+}
+
+static int ublk_stripe_queue_io(struct ublk_queue *q, int tag)
+{
+	int queued = stripe_queue_tgt_io(q, tag);
+
+	ublk_queued_tgt_io(q, tag, queued);
+	return 0;
+}
+
+static void ublk_stripe_io_done(struct ublk_queue *q, int tag,
+		const struct io_uring_cqe *cqe)
+{
+	const struct ublksrv_io_desc *iod = ublk_get_iod(q, tag);
+	unsigned op = user_data_to_op(cqe->user_data);
+	struct ublk_io *io = ublk_get_io(q, tag);
+	int res = cqe->res;
+
+	if (res < 0) {
+		if (!io->result)
+			io->result = res;
+		ublk_err("%s: io failure %d tag %u\n", __func__, res, tag);
+	}
+
+	/* fail short READ/WRITE simply */
+	if (op == UBLK_IO_OP_READ || op == UBLK_IO_OP_WRITE) {
+		unsigned seq = user_data_to_tgt_data(cqe->user_data);
+		struct stripe_array *s = io->private_data;
+
+		if (res < s->s[seq].vec->iov_len)
+			io->result = -EIO;
+	}
+
+	if (ublk_completed_tgt_io(q, tag)) {
+		int res = io->result;
+
+		if (!res)
+			res = iod->nr_sectors << 9;
+
+		ublk_complete_io(q, tag, res);
+
+		free_stripe_array(io->private_data);
+		io->private_data = NULL;
+	}
+}
+
+static int ublk_stripe_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev)
+{
+	struct ublk_params p = {
+		.types = UBLK_PARAM_TYPE_BASIC,
+		.basic = {
+			.attrs = UBLK_ATTR_VOLATILE_CACHE,
+			.logical_bs_shift	= 9,
+			.physical_bs_shift	= 12,
+			.io_opt_shift	= 12,
+			.io_min_shift	= 9,
+			.max_sectors = dev->dev_info.max_io_buf_bytes >> 9,
+		},
+	};
+	unsigned chunk_size = ctx->chunk_size;
+	struct stripe_conf *conf;
+	unsigned chunk_shift;
+	loff_t bytes = 0;
+	int ret, i;
+
+	if ((chunk_size & (chunk_size - 1)) || !chunk_size) {
+		ublk_err("invalid chunk size %u\n", chunk_size);
+		return -EINVAL;
+	}
+
+	if (chunk_size < 4096 || chunk_size > 512 * 1024) {
+		ublk_err("invalid chunk size %u\n", chunk_size);
+		return -EINVAL;
+	}
+
+	chunk_shift = ilog2(chunk_size);
+
+	ret = backing_file_tgt_init(dev);
+	if (ret)
+		return ret;
+
+	if (!dev->tgt.nr_backing_files || dev->tgt.nr_backing_files > NR_STRIPE)
+		return -EINVAL;
+
+	assert(dev->nr_fds == dev->tgt.nr_backing_files + 1);
+
+	for (i = 0; i < dev->tgt.nr_backing_files; i++)
+		dev->tgt.backing_file_size[i] &= ~((1 << chunk_shift) - 1);
+
+	for (i = 0; i < dev->tgt.nr_backing_files; i++) {
+		unsigned long size = dev->tgt.backing_file_size[i];
+
+		if (size != dev->tgt.backing_file_size[0])
+			return -EINVAL;
+		bytes += size;
+	}
+
+	conf = malloc(sizeof(*conf));
+	conf->shift = chunk_shift;
+	conf->nr_files = dev->tgt.nr_backing_files;
+
+	dev->private_data = conf;
+	dev->tgt.dev_size = bytes;
+	p.basic.dev_sectors = bytes >> 9;
+	dev->tgt.params = p;
+	dev->tgt.sq_depth = dev->dev_info.queue_depth * conf->nr_files;
+	dev->tgt.cq_depth = dev->dev_info.queue_depth * conf->nr_files;
+
+	printf("%s: shift %u files %u\n", __func__, conf->shift, conf->nr_files);
+
+	return 0;
+}
+
+static void ublk_stripe_tgt_deinit(struct ublk_dev *dev)
+{
+	free(dev->private_data);
+	backing_file_tgt_deinit(dev);
+}
+
+const struct ublk_tgt_ops stripe_tgt_ops = {
+	.name = "stripe",
+	.init_tgt = ublk_stripe_tgt_init,
+	.deinit_tgt = ublk_stripe_tgt_deinit,
+	.queue_io = ublk_stripe_queue_io,
+	.tgt_io_done = ublk_stripe_io_done,
+};
diff --git a/tools/testing/selftests/ublk/test_stripe_01.sh b/tools/testing/selftests/ublk/test_stripe_01.sh
new file mode 100755
index 000000000000..c01f3dc325ab
--- /dev/null
+++ b/tools/testing/selftests/ublk/test_stripe_01.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh
+
+TID="stripe_01"
+ERR_CODE=0
+
+_prep_test "stripe" "write and verify test"
+
+backfile_0=$(_create_backfile 256M)
+backfile_1=$(_create_backfile 256M)
+
+dev_id=$(_add_ublk_dev -t stripe "$backfile_0" "$backfile_1")
+_check_add_dev $TID $? "${backfile_0}"
+
+# run fio over the ublk disk
+fio --name=write_and_verify \
+    --filename=/dev/ublkb"${dev_id}" \
+    --ioengine=libaio --iodepth=32 \
+    --rw=write \
+    --size=512M \
+    --direct=1 \
+    --verify=crc32c \
+    --do_verify=1 \
+    --bs=4k > /dev/null 2>&1
+ERR_CODE=$?
+
+_cleanup_test "stripe"
+
+_remove_backfile "$backfile_0"
+_remove_backfile "$backfile_1"
+
+_show_result $TID $ERR_CODE
diff --git a/tools/testing/selftests/ublk/test_stripe_02.sh b/tools/testing/selftests/ublk/test_stripe_02.sh
new file mode 100755
index 000000000000..e8a45fa82dde
--- /dev/null
+++ b/tools/testing/selftests/ublk/test_stripe_02.sh
@@ -0,0 +1,24 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh
+
+TID="stripe_02"
+ERR_CODE=0
+
+_prep_test "stripe" "mkfs & mount & umount"
+
+backfile_0=$(_create_backfile 256M)
+backfile_1=$(_create_backfile 256M)
+dev_id=$(_add_ublk_dev -t stripe "$backfile_0" "$backfile_1")
+_check_add_dev $TID $? "$backfile_0" "$backfile_1"
+
+_mkfs_mount_test /dev/ublkb"${dev_id}"
+ERR_CODE=$?
+
+_cleanup_test "stripe"
+
+_remove_backfile "$backfile_0"
+_remove_backfile "$backfile_1"
+
+_show_result $TID $ERR_CODE
-- 
2.47.0

Re: [PATCH 0/8] selftests: ublk: cleanup & more tests

[Jens Axboe]

On Sat, 22 Mar 2025 17:32:08 +0800, Ming Lei wrote:
> The 1st patch adds generic_01.sh for checking if IO is dispatched in order.
> 
> The 2nd ~ 7th patches clean up and simplify target implementation, add zc
> for for null, which is useful for evaluating/comparing perf.
> 
> The 8th patch adds ublk/stripe target and two tests, which will be useful
> for verifying multiple IOs aiming at same fixed kernel buffer, also can
> be used for verifying vectored fixed kernel buffer in future if this
> feature can be supported.
> 
> [...]

Applied, thanks!

[1/8] selftests: ublk: add generic_01 for verifying sequential IO order
      commit: 723977cab4c0fdcf5ba08da9e30a6ad72efa2464
[2/8] selftests: ublk: add single sqe allocator helper
      commit: f2639ed11e256b957690e241bb04ec9912367d60
[3/8] selftests: ublk: increase max buffer size to 1MB
      commit: 9413c0ca8e455efb16b81f2c99061f6eb3d38281
[4/8] selftests: ublk: move common code into common.c
      commit: 10d962dae2f6b4a7d86579cc6fe9d8987117fa8f
[5/8] selftests: ublk: prepare for supporting stripe target
      commit: 8842b72a821d4cd49281fa096c35f9fa630ec981
[6/8] selftests: ublk: enable zero copy for null target
      commit: 8cb9b971e2b6103c72faf765f64239f86ec9328f
[7/8] selftests: ublk: simplify loop io completion
      commit: 263846eb431f31ca3f38846c374377b732abb26e
[8/8] selftests: ublk: add stripe target
      commit: 0f3ebf2d4bc0296c61543b2a729151d89c60e1ec

Best regards,
-- 
Jens Axboe