* [PATCH blktests v2 1/4] block/049: add a direct I/O write fallback race test for block devices
2026-10-09 23:53 [PATCH blktests v2 0/4] blktests: tests for the direct I/O fallback, splice, bio trimming, and atomic write fixes Tal Zussman
@ 2026-10-09 23:53 ` Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 2/4] block/050: add a splice read " Tal Zussman
` (2 subsequent siblings)
3 siblings, 0 replies; 8+ messages in thread
From: Tal Zussman @ 2026-10-09 23:53 UTC (permalink / raw)
To: Shin'ichiro Kawasaki, linux-block
Cc: Christoph Hellwig, John Garry, Tal Zussman
A direct write to a block device that completes only partially finishes
as a buffered write through the direct I/O fallback. That buffered write
has to hold i_rwsem like the plain buffered write path, so that it does
not race set_blocksize() raising the mapping's minimum folio order and
add a folio that is too small for the mapping.
This is a regression test for that issue, fixed in the kernel patch
"block: take i_rwsem for the direct I/O write fallback" [1].
Issue O_DIRECT pwritev() to a memory-backed null_blk device with a
two-segment iovec whose second segment is an unreadable PROT_NONE
mapping, so that the direct path returns short and enters the fallback.
Meanwhile toggle the second segment's protection so that some fallbacks
get past the fault-in and reach the page cache, populate the page cache
with folios of the current block size, and toggle the block size between
512 bytes and 64K with BLKBSZSET. A CONFIG_DEBUG_VM kernel reports the
folio order mismatch as a BUG, which blktests picks up from dmesg.
The minimum folio order only moves with block sizes above the page size,
i.e. with CONFIG_TRANSPARENT_HUGEPAGE raising BLK_MAX_BLOCK_SIZE to 64K.
[1]: https://lore.kernel.org/linux-block/20261009-blkdev-fixes-v6-2-307939c387df@columbia.edu/
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
src/.gitignore | 1 +
src/Makefile | 7 +-
src/dio-fallback-race.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++
tests/block/049 | 57 ++++++++++++
tests/block/049.out | 2 +
5 files changed, 295 insertions(+), 1 deletion(-)
diff --git a/src/.gitignore b/src/.gitignore
index e9869e1..754beef 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -1,5 +1,6 @@
/bio-bounce-read
/bio-full-trim
+/dio-fallback-race
/dio-offsets
/discontiguous-io
/loblksize
diff --git a/src/Makefile b/src/Makefile
index dd64694..4590639 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -33,6 +33,7 @@ C_TARGETS := \
C_URING_TARGETS := metadata \
nvme-passthru-admin-uring
C_UBLK_TARGETS := miniublk
+C_THREAD_TARGETS := dio-fallback-race
HAVE_LIBURING := $(call HAVE_C_MACRO,liburing.h,IORING_OP_URING_CMD)
HAVE_UBLK_HEADER := $(call HAVE_C_HEADER,linux/ublk_cmd.h,1)
@@ -44,7 +45,7 @@ CXX_TARGETS := \
SYZKALLER_TARGETS := \
sg/syzkaller1
-TARGETS := $(C_TARGETS) $(CXX_TARGETS) $(SYZKALLER_TARGETS)
+TARGETS := $(C_TARGETS) $(C_THREAD_TARGETS) $(CXX_TARGETS) $(SYZKALLER_TARGETS)
ifeq ($(HAVE_UBLK_HEADER), 1)
ifeq ($(HAVE_NEW_UBLK_INTF), 1)
@@ -69,6 +70,7 @@ override CXXFLAGS := -O2 -std=c++11 -Wall -Wextra -Wshadow -Wno-sign-compare \
-Werror $(CXXFLAGS) $(CONFIG_DEFS)
URING_FLAGS := -D_GNU_SOURCE
URING_LIBS := -lpthread -luring
+THREAD_LIBS := -lpthread
LDFLAGS ?=
all: $(TARGETS)
@@ -89,6 +91,9 @@ $(CXX_TARGETS): %: %.cpp
$(SYZKALLER_TARGETS): %: %.c
$(CC) $(CFLAGS) -Wno-unused-but-set-variable $(LDFLAGS) -o $@ $^
+$(C_THREAD_TARGETS): %: %.c
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ $^ $(THREAD_LIBS)
+
$(C_URING_TARGETS): %: %.c
$(CC) $(CFLAGS) $(LDFLAGS) $(URING_FLAGS) -o $@ $^ $(URING_LIBS)
diff --git a/src/dio-fallback-race.c b/src/dio-fallback-race.c
new file mode 100644
index 0000000..e5221c5
--- /dev/null
+++ b/src/dio-fallback-race.c
@@ -0,0 +1,229 @@
+// SPDX-License-Identifier: GPL-3.0+
+/*
+ * Copyright (C) 2026 Tal Zussman
+ *
+ * Race partial O_DIRECT writes to a block device against BLKBSZSET.
+ *
+ * Writer threads issue O_DIRECT pwritev() with a two-segment iovec whose
+ * second segment is an unreadable PROT_NONE mapping. The direct path writes
+ * the first segment, fails to pin the second and returns short, so the write
+ * finishes as a buffered write through the direct I/O fallback. A second
+ * thread toggles the second segment's protection so that some fallbacks get
+ * past fault_in_iov_iter_readable() and reach the page cache, a third
+ * populates the page cache with folios of the current block size, and a
+ * fourth toggles the block size between 512 bytes and 64K with BLKBSZSET.
+ *
+ * The fallback has to run under i_rwsem like the plain buffered write path.
+ * If it does not, it races set_blocksize() raising the mapping's minimum
+ * folio order and adds a folio that is too small for the mapping, which a
+ * CONFIG_DEBUG_VM kernel reports as a BUG. The caller checks dmesg.
+ *
+ * usage: dio-fallback-race <blockdev> <seconds>
+ *
+ * exit: 0 = ran for <seconds>
+ * 1 = setup error
+ */
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/uio.h>
+#include <unistd.h>
+
+#include <linux/fs.h>
+
+#define GOOD (8 * 1024) /* written by the direct path */
+#define BAD (64 * 1024) /* unreadable, forces a short write */
+#define RANGE (2 * 1024 * 1024) /* keep the race on a few folios */
+#define NR_WRITERS 2
+
+#define SMALL_BS 512
+#define LARGE_BS (64 * 1024)
+
+static const char *dev;
+static long pgsz;
+static char *badseg;
+static int bszfd;
+static volatile int stop;
+static int failed;
+
+/* partial direct write, finished as a buffered write by the fallback */
+static void *writer(void *arg)
+{
+ struct iovec iov[2];
+ off_t off = 0;
+ char *good;
+ int fd;
+
+ fd = open(dev, O_RDWR | O_DIRECT);
+ if (fd < 0) {
+ perror("open");
+ failed = 1;
+ return NULL;
+ }
+
+ if (posix_memalign((void **)&good, pgsz, GOOD)) {
+ perror("posix_memalign");
+ failed = 1;
+ return NULL;
+ }
+ memset(good, 'A', GOOD);
+
+ iov[0].iov_base = good;
+ iov[0].iov_len = GOOD;
+ iov[1].iov_base = badseg;
+ iov[1].iov_len = BAD;
+
+ while (!stop) {
+ if (pwritev(fd, iov, 2, off) < 0) {
+ perror("pwritev");
+ failed = 1;
+ break;
+ }
+ off = (off + GOOD) % RANGE;
+ }
+
+ return NULL;
+}
+
+/* let some fallbacks get past the fault-in and into the page cache */
+static void *flipper(void *arg)
+{
+ while (!stop) {
+ if (mprotect(badseg, BAD, PROT_READ | PROT_WRITE) ||
+ mprotect(badseg, BAD, PROT_NONE)) {
+ perror("mprotect");
+ failed = 1;
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+/* populate the page cache with folios sized for the current block size */
+static void *reader(void *arg)
+{
+ off_t off = 0;
+ char *buf;
+ int fd;
+
+ fd = open(dev, O_RDONLY);
+ if (fd < 0) {
+ perror("open");
+ failed = 1;
+ return NULL;
+ }
+
+ buf = malloc(GOOD);
+ if (!buf) {
+ perror("malloc");
+ failed = 1;
+ return NULL;
+ }
+
+ while (!stop) {
+ if (pread(fd, buf, GOOD, off) < 0) {
+ perror("pread");
+ failed = 1;
+ break;
+ }
+ readahead(fd, off, RANGE / 4);
+ off = (off + GOOD) % RANGE;
+ }
+
+ return NULL;
+}
+
+/* change i_blkbits and the mapping's minimum folio order underneath them */
+static void *resizer(void *arg)
+{
+ int bs = SMALL_BS;
+
+ while (!stop) {
+ if (ioctl(bszfd, BLKBSZSET, &bs)) {
+ perror("BLKBSZSET");
+ failed = 1;
+ break;
+ }
+ bs = bs == SMALL_BS ? LARGE_BS : SMALL_BS;
+ }
+
+ return NULL;
+}
+
+static int spawn(pthread_t *t, void *(*fn)(void *))
+{
+ int err = pthread_create(t, NULL, fn, NULL);
+
+ if (err)
+ fprintf(stderr, "pthread_create: %s\n", strerror(err));
+
+ return err;
+}
+
+int main(int argc, char **argv)
+{
+ pthread_t writers[NR_WRITERS];
+ pthread_t flipper_t, reader_t, resizer_t;
+ int bs = LARGE_BS;
+ int i;
+
+ if (argc != 3) {
+ fprintf(stderr, "usage: %s <blockdev> <seconds>\n", argv[0]);
+ return EXIT_FAILURE;
+ }
+
+ dev = argv[1];
+
+ pgsz = sysconf(_SC_PAGESIZE);
+ if (pgsz < 0) {
+ perror("sysconf");
+ return EXIT_FAILURE;
+ }
+
+ bszfd = open(dev, O_RDONLY);
+ if (bszfd < 0) {
+ perror("open");
+ return EXIT_FAILURE;
+ }
+
+ /*
+ * The minimum folio order only moves with block sizes above the page
+ * size, which needs BLK_MAX_BLOCK_SIZE above PAGE_SIZE, i.e.
+ * CONFIG_TRANSPARENT_HUGEPAGE.
+ */
+ if (ioctl(bszfd, BLKBSZSET, &bs)) {
+ perror("BLKBSZSET");
+ return EXIT_FAILURE;
+ }
+
+ badseg = mmap(NULL, BAD, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (badseg == MAP_FAILED) {
+ perror("mmap");
+ return EXIT_FAILURE;
+ }
+
+ for (i = 0; i < NR_WRITERS; i++) {
+ if (spawn(&writers[i], writer))
+ return EXIT_FAILURE;
+ }
+ if (spawn(&flipper_t, flipper) || spawn(&reader_t, reader) ||
+ spawn(&resizer_t, resizer))
+ return EXIT_FAILURE;
+
+ sleep(atoi(argv[2]));
+ stop = 1;
+
+ for (i = 0; i < NR_WRITERS; i++)
+ pthread_join(writers[i], NULL);
+ pthread_join(flipper_t, NULL);
+ pthread_join(reader_t, NULL);
+ pthread_join(resizer_t, NULL);
+
+ return failed ? EXIT_FAILURE : EXIT_SUCCESS;
+}
diff --git a/tests/block/049 b/tests/block/049
new file mode 100755
index 0000000..aa5622a
--- /dev/null
+++ b/tests/block/049
@@ -0,0 +1,57 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2026 Tal Zussman
+#
+# Race partial O_DIRECT writes to a block device against BLKBSZSET. A direct
+# write that completes only partially finishes as a buffered write through the
+# direct I/O fallback, which has to hold i_rwsem like the plain buffered write
+# path so that it does not race set_blocksize() raising the mapping's minimum
+# folio order. Without it, the fallback adds a folio that is too small for the
+# mapping, which a CONFIG_DEBUG_VM kernel reports as a BUG.
+#
+# Regression test for patch "block: take i_rwsem for the direct I/O write
+# fallback".
+
+. tests/block/rc
+. common/null_blk
+
+DESCRIPTION="race the direct I/O write fallback against BLKBSZSET"
+TIMED=1
+
+requires() {
+ _have_null_blk
+ _have_kernel_option TRANSPARENT_HUGEPAGE
+ _have_kernel_option DEBUG_VM
+ _have_src_program dio-fallback-race
+ if (( $(_get_page_size) >= 65536 )); then
+ SKIP_REASONS+=("a 64K block size is not above the page size")
+ return 1
+ fi
+}
+
+test() {
+ echo "Running ${TEST_NAME}"
+
+ # the race hits about once a minute on an unfixed kernel
+ : "${TIMEOUT:=60}"
+
+ if ! _configure_null_blk nullb1 blocksize=512 memory_backed=1 \
+ size=64 power=1; then
+ echo "configuring null_blk failed"
+ return 1
+ fi
+
+ if ! blockdev --setbsz 65536 /dev/nullb1; then
+ SKIP_REASONS+=("kernel does not support a 64K block size")
+ _exit_null_blk
+ return
+ fi
+
+ if ! src/dio-fallback-race /dev/nullb1 "${TIMEOUT}" >>"${FULL}" 2>&1; then
+ echo "dio-fallback-race helper failed"
+ fi
+
+ _exit_null_blk
+
+ echo "Test complete"
+}
diff --git a/tests/block/049.out b/tests/block/049.out
new file mode 100644
index 0000000..c88edfe
--- /dev/null
+++ b/tests/block/049.out
@@ -0,0 +1,2 @@
+Running block/049
+Test complete
--
2.39.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH blktests v2 2/4] block/050: add a splice read race test for block devices
2026-10-09 23:53 [PATCH blktests v2 0/4] blktests: tests for the direct I/O fallback, splice, bio trimming, and atomic write fixes Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 1/4] block/049: add a direct I/O write fallback race test for block devices Tal Zussman
@ 2026-10-09 23:53 ` Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 3/4] block/051: add a pinned page leak test for trimmed direct writes Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices Tal Zussman
3 siblings, 0 replies; 8+ messages in thread
From: Tal Zussman @ 2026-10-09 23:53 UTC (permalink / raw)
To: Shin'ichiro Kawasaki, linux-block
Cc: Christoph Hellwig, John Garry, Tal Zussman
The block device splice read path has to hold i_rwsem like the plain
read path, so that it does not race set_blocksize() raising the
mapping's minimum folio order and add a folio that is too small for the
mapping.
This is a regression test for that issue, fixed in the kernel patch
"block: take i_rwsem for the splice read path" [1].
splice() from a memory-backed null_blk device into a pipe while toggling
the block size between 512 bytes and 64K with BLKBSZSET. A
CONFIG_DEBUG_VM kernel reports the folio order mismatch as a BUG, which
blktests picks up from dmesg.
The minimum folio order only moves with block sizes above the page size,
i.e. with CONFIG_TRANSPARENT_HUGEPAGE raising BLK_MAX_BLOCK_SIZE to 64K.
[1]: https://lore.kernel.org/linux-block/20261009-blkdev-fixes-v6-3-307939c387df@columbia.edu/
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
src/.gitignore | 1 +
src/Makefile | 3 +-
src/splice-race.c | 171 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/block/050 | 54 +++++++++++++++++
tests/block/050.out | 2 +
5 files changed, 230 insertions(+), 1 deletion(-)
diff --git a/src/.gitignore b/src/.gitignore
index 754beef..dbebc22 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -11,6 +11,7 @@
/mount_clear_sock
/nbdsetsize
/openclose
+/splice-race
/sg/dxfer-from-dev
/sg/syzkaller1
/zbdioctl
diff --git a/src/Makefile b/src/Makefile
index 4590639..b7e8ca9 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -33,7 +33,8 @@ C_TARGETS := \
C_URING_TARGETS := metadata \
nvme-passthru-admin-uring
C_UBLK_TARGETS := miniublk
-C_THREAD_TARGETS := dio-fallback-race
+C_THREAD_TARGETS := dio-fallback-race \
+ splice-race
HAVE_LIBURING := $(call HAVE_C_MACRO,liburing.h,IORING_OP_URING_CMD)
HAVE_UBLK_HEADER := $(call HAVE_C_HEADER,linux/ublk_cmd.h,1)
diff --git a/src/splice-race.c b/src/splice-race.c
new file mode 100644
index 0000000..e411752
--- /dev/null
+++ b/src/splice-race.c
@@ -0,0 +1,171 @@
+// SPDX-License-Identifier: GPL-3.0+
+/*
+ * Copyright (C) 2026 Tal Zussman
+ *
+ * Race splice() from a block device against BLKBSZSET.
+ *
+ * Splicer threads splice from the device into a pipe while another thread
+ * toggles the block size between 512 bytes and 64K with BLKBSZSET.
+ *
+ * The splice read path has to run under i_rwsem like the plain read path.
+ * If it does not, it races set_blocksize() raising the mapping's minimum
+ * folio order and adds a folio that is too small for the mapping, which a
+ * CONFIG_DEBUG_VM kernel reports as a BUG. The caller checks dmesg.
+ *
+ * usage: splice-race <blockdev> <seconds>
+ *
+ * exit: 0 = ran for <seconds>
+ * 1 = setup error
+ */
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+
+#include <linux/fs.h>
+
+#define CHUNK (64 * 1024)
+#define RANGE (2 * 1024 * 1024) /* keep the race on a few folios */
+#define NR_SPLICERS 4
+
+#define SMALL_BS 512
+#define LARGE_BS (64 * 1024)
+
+static const char *dev;
+static int bszfd;
+static volatile int stop;
+static int failed;
+
+/* filemap_splice_read() from the device */
+static void *splicer(void *arg)
+{
+ int pipefd[2];
+ loff_t off = 0;
+ char *sink;
+ int fd;
+
+ fd = open(dev, O_RDONLY);
+ if (fd < 0) {
+ perror("open");
+ failed = 1;
+ return NULL;
+ }
+
+ if (pipe(pipefd)) {
+ perror("pipe");
+ failed = 1;
+ return NULL;
+ }
+
+ sink = malloc(CHUNK);
+ if (!sink) {
+ perror("malloc");
+ failed = 1;
+ return NULL;
+ }
+
+ while (!stop) {
+ ssize_t n = splice(fd, &off, pipefd[1], NULL, CHUNK, 0);
+
+ if (n < 0) {
+ perror("splice");
+ failed = 1;
+ break;
+ }
+
+ /* drain the pipe so the next splice does not block on it */
+ while (n > 0) {
+ ssize_t d = read(pipefd[0], sink, n);
+
+ if (d <= 0) {
+ perror("read");
+ failed = 1;
+ return NULL;
+ }
+ n -= d;
+ }
+
+ if (off >= RANGE)
+ off = 0;
+ }
+
+ return NULL;
+}
+
+/* change i_blkbits and the mapping's minimum folio order underneath them */
+static void *resizer(void *arg)
+{
+ int bs = SMALL_BS;
+
+ while (!stop) {
+ if (ioctl(bszfd, BLKBSZSET, &bs)) {
+ perror("BLKBSZSET");
+ failed = 1;
+ break;
+ }
+ bs = bs == SMALL_BS ? LARGE_BS : SMALL_BS;
+ }
+
+ return NULL;
+}
+
+static int spawn(pthread_t *t, void *(*fn)(void *))
+{
+ int err = pthread_create(t, NULL, fn, NULL);
+
+ if (err)
+ fprintf(stderr, "pthread_create: %s\n", strerror(err));
+
+ return err;
+}
+
+int main(int argc, char **argv)
+{
+ pthread_t splicers[NR_SPLICERS];
+ pthread_t resizer_t;
+ int bs = LARGE_BS;
+ int i;
+
+ if (argc != 3) {
+ fprintf(stderr, "usage: %s <blockdev> <seconds>\n", argv[0]);
+ return EXIT_FAILURE;
+ }
+
+ dev = argv[1];
+
+ bszfd = open(dev, O_RDONLY);
+ if (bszfd < 0) {
+ perror("open");
+ return EXIT_FAILURE;
+ }
+
+ /*
+ * The minimum folio order only moves with block sizes above the page
+ * size, which needs BLK_MAX_BLOCK_SIZE above PAGE_SIZE, i.e.
+ * CONFIG_TRANSPARENT_HUGEPAGE.
+ */
+ if (ioctl(bszfd, BLKBSZSET, &bs)) {
+ perror("BLKBSZSET");
+ return EXIT_FAILURE;
+ }
+
+ for (i = 0; i < NR_SPLICERS; i++) {
+ if (spawn(&splicers[i], splicer))
+ return EXIT_FAILURE;
+ }
+ if (spawn(&resizer_t, resizer))
+ return EXIT_FAILURE;
+
+ sleep(atoi(argv[2]));
+ stop = 1;
+
+ for (i = 0; i < NR_SPLICERS; i++)
+ pthread_join(splicers[i], NULL);
+ pthread_join(resizer_t, NULL);
+
+ return failed ? EXIT_FAILURE : EXIT_SUCCESS;
+}
diff --git a/tests/block/050 b/tests/block/050
new file mode 100755
index 0000000..64c0406
--- /dev/null
+++ b/tests/block/050
@@ -0,0 +1,54 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2026 Tal Zussman
+#
+# Race splice() from a block device against BLKBSZSET. The splice read path
+# has to hold i_rwsem like the plain read path so that it does not race
+# set_blocksize() raising the mapping's minimum folio order. Without it, the
+# splice adds a folio that is too small for the mapping, which a
+# CONFIG_DEBUG_VM kernel reports as a BUG.
+#
+# Regression test for patch "block: take i_rwsem for the splice read path".
+
+. tests/block/rc
+. common/null_blk
+
+DESCRIPTION="race splice() from a block device against BLKBSZSET"
+TIMED=1
+
+requires() {
+ _have_null_blk
+ _have_kernel_option TRANSPARENT_HUGEPAGE
+ _have_kernel_option DEBUG_VM
+ _have_src_program splice-race
+ if (( $(_get_page_size) >= 65536 )); then
+ SKIP_REASONS+=("a 64K block size is not above the page size")
+ return 1
+ fi
+}
+
+test() {
+ echo "Running ${TEST_NAME}"
+
+ : "${TIMEOUT:=30}"
+
+ if ! _configure_null_blk nullb1 blocksize=512 memory_backed=1 \
+ size=64 power=1; then
+ echo "configuring null_blk failed"
+ return 1
+ fi
+
+ if ! blockdev --setbsz 65536 /dev/nullb1; then
+ SKIP_REASONS+=("kernel does not support a 64K block size")
+ _exit_null_blk
+ return
+ fi
+
+ if ! src/splice-race /dev/nullb1 "${TIMEOUT}" >>"${FULL}" 2>&1; then
+ echo "splice-race helper failed"
+ fi
+
+ _exit_null_blk
+
+ echo "Test complete"
+}
diff --git a/tests/block/050.out b/tests/block/050.out
new file mode 100644
index 0000000..fc4e537
--- /dev/null
+++ b/tests/block/050.out
@@ -0,0 +1,2 @@
+Running block/050
+Test complete
--
2.39.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH blktests v2 3/4] block/051: add a pinned page leak test for trimmed direct writes
2026-10-09 23:53 [PATCH blktests v2 0/4] blktests: tests for the direct I/O fallback, splice, bio trimming, and atomic write fixes Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 1/4] block/049: add a direct I/O write fallback race test for block devices Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 2/4] block/050: add a splice read " Tal Zussman
@ 2026-10-09 23:53 ` Tal Zussman
2026-10-09 23:53 ` [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices Tal Zussman
3 siblings, 0 replies; 8+ messages in thread
From: Tal Zussman @ 2026-10-09 23:53 UTC (permalink / raw)
To: Shin'ichiro Kawasaki, linux-block
Cc: Christoph Hellwig, John Garry, Tal Zussman
bio_iov_iter_align_down() trims a direct I/O bio down to the logical
block size and unpins the pages it drops. A bvec can span several pages
of one folio, each with its own pin, and dropping or shrinking such a
bvec has to release all of them. When it does not, the folio is never
freed.
This is a regression test for that issue, fixed in the kernel patch
"block: unpin all pages of a bvec in bio_iov_iter_align_down()" [1].
Issue O_DIRECT pwritev() from a hugetlb mapping to a memory-backed
null_blk device with a 64K logical block size, with a first segment that
ends half a block past a block boundary and an unreadable second
segment, so that the bio is trimmed by several pages of one huge page.
Place the first segment across two huge pages so the tail is its own
bvec and gets dropped, and inside one huge page so the tail is the end
of a larger bvec and gets shrunk, to cover both paths. Compare
HugePages_Free before and after, and fail if huge pages leaked.
The leak needs a logical block size above the page size, i.e.
CONFIG_TRANSPARENT_HUGEPAGE raising BLK_MAX_BLOCK_SIZE to 64K, and a
large folio backing the buffer, which hugetlb provides. Reserve huge
pages for the test and restore the previous count afterwards.
[1]: https://lore.kernel.org/linux-block/20261009-blkdev-fixes-v6-8-307939c387df@columbia.edu/
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
src/.gitignore | 1 +
src/Makefile | 1 +
src/bio-trim-pin-leak.c | 191 ++++++++++++++++++++++++++++++++++++++++++++++++
tests/block/051 | 64 ++++++++++++++++
tests/block/051.out | 2 +
5 files changed, 259 insertions(+)
diff --git a/src/.gitignore b/src/.gitignore
index dbebc22..92c08d1 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -1,5 +1,6 @@
/bio-bounce-read
/bio-full-trim
+/bio-trim-pin-leak
/dio-fallback-race
/dio-offsets
/discontiguous-io
diff --git a/src/Makefile b/src/Makefile
index b7e8ca9..0bdfd05 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -16,6 +16,7 @@ HAVE_C_DEF = $(shell if printf '$(H)include <%s>\n$(H)ifdef %s\nHAVE_%s\n$(H)end
C_TARGETS := \
bio-bounce-read \
bio-full-trim \
+ bio-trim-pin-leak \
dio-offsets \
loblksize \
loop_change_fd \
diff --git a/src/bio-trim-pin-leak.c b/src/bio-trim-pin-leak.c
new file mode 100644
index 0000000..54af181
--- /dev/null
+++ b/src/bio-trim-pin-leak.c
@@ -0,0 +1,191 @@
+// SPDX-License-Identifier: GPL-3.0+
+/*
+ * Copyright (C) 2026 Tal Zussman
+ *
+ * Check that a direct write trimmed to the logical block size unpins all the
+ * pages it drops.
+ *
+ * Issue O_DIRECT pwritev() from a hugetlb mapping with a two-segment iovec
+ * whose first segment ends half a block past a block boundary and whose
+ * second segment is an unreadable PROT_NONE mapping. The direct path pins the
+ * first segment, fails to pin the second and trims the bio down to a block
+ * boundary. The trimmed tail spans several pages of one huge page, each with
+ * its own pin. If the trim releases at most one of them, the huge page
+ * never returns to the pool, which shows up as a drop in HugePages_Free.
+ *
+ * The tail is either its own bvec, when the first segment straddles two huge
+ * pages, or the end of a larger one, when it sits inside a single huge page.
+ * Each iteration issues one write of each kind. The direct I/O fallback
+ * finishes the trimmed half block through the page cache, so each write
+ * returns one and a half blocks.
+ *
+ * The tail only holds several pins when it spans several pages of a large
+ * folio, so the device needs a logical block size of at least four pages and
+ * the buffer comes from a hugetlb mapping.
+ *
+ * usage: bio-trim-pin-leak <blockdev> <iterations>
+ *
+ * exit: 0 = no huge pages leaked
+ * 1 = setup error
+ * 2 = huge pages leaked
+ */
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/uio.h>
+#include <unistd.h>
+
+#include <linux/fs.h>
+
+#define EXIT_LEAKED 2
+
+static long meminfo(const char *key)
+{
+ char line[256];
+ long val = -1;
+ FILE *f;
+
+ f = fopen("/proc/meminfo", "r");
+ if (!f) {
+ perror("fopen");
+ return -1;
+ }
+
+ while (fgets(line, sizeof(line), f)) {
+ if (!strncmp(line, key, strlen(key))) {
+ val = strtol(line + strlen(key), NULL, 10);
+ break;
+ }
+ }
+
+ fclose(f);
+ return val;
+}
+
+int main(int argc, char **argv)
+{
+ struct iovec iov[2];
+ long pgsz, hpsz;
+ long before, after;
+ char *badseg;
+ char *map;
+ int lbs, iters;
+ int fd, i;
+
+ if (argc != 3) {
+ fprintf(stderr, "usage: %s <blockdev> <iterations>\n", argv[0]);
+ return EXIT_FAILURE;
+ }
+
+ iters = atoi(argv[2]);
+
+ pgsz = sysconf(_SC_PAGESIZE);
+ if (pgsz < 0) {
+ perror("sysconf");
+ return EXIT_FAILURE;
+ }
+
+ hpsz = meminfo("Hugepagesize:") * 1024;
+ if (hpsz <= 0) {
+ fprintf(stderr, "no hugetlb page size in /proc/meminfo\n");
+ return EXIT_FAILURE;
+ }
+
+ fd = open(argv[1], O_RDWR | O_DIRECT);
+ if (fd < 0) {
+ perror("open");
+ return EXIT_FAILURE;
+ }
+
+ if (ioctl(fd, BLKSSZGET, &lbs)) {
+ perror("BLKSSZGET");
+ return EXIT_FAILURE;
+ }
+ printf("logical block size: %d, page size: %ld, huge page size: %ld\n",
+ lbs, pgsz, hpsz);
+
+ /*
+ * The trimmed tail is half a block, and it has to span at least two
+ * pages for a pin to leak.
+ */
+ if (lbs < 4 * pgsz) {
+ fprintf(stderr, "logical block size %d is below four pages\n",
+ lbs);
+ return EXIT_FAILURE;
+ }
+ if (hpsz < 2 * lbs) {
+ fprintf(stderr, "huge page size %ld is below two blocks\n", hpsz);
+ return EXIT_FAILURE;
+ }
+
+ badseg = mmap(NULL, lbs, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (badseg == MAP_FAILED) {
+ perror("mmap");
+ return EXIT_FAILURE;
+ }
+
+ before = meminfo("HugePages_Free:");
+ if (before < 2 * iters + 2) {
+ fprintf(stderr, "HugePages_Free is %ld, need at least %d\n",
+ before, 2 * iters + 2);
+ return EXIT_FAILURE;
+ }
+
+ for (i = 0; i < iters; i++) {
+ map = mmap(NULL, 2 * hpsz, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
+ if (map == MAP_FAILED) {
+ perror("mmap");
+ return EXIT_FAILURE;
+ }
+ memset(map, 'A', 2 * hpsz);
+
+ iov[0].iov_len = lbs + lbs / 2;
+ iov[1].iov_base = badseg;
+ iov[1].iov_len = lbs / 2;
+
+ /*
+ * One block from the end of the first huge page and half a
+ * block from the start of the second, then an unreadable
+ * segment: the bio is trimmed by half a block, dropping the
+ * bvec pinned from the second huge page.
+ */
+ iov[0].iov_base = map + hpsz - lbs;
+ if (pwritev(fd, iov, 2, 0) != lbs + lbs / 2) {
+ perror("pwritev");
+ return EXIT_FAILURE;
+ }
+
+ /*
+ * The same write from the start of the first huge page: the
+ * pinned segment is a single bvec, and the trim shrinks it
+ * by half a block instead of dropping one.
+ */
+ iov[0].iov_base = map;
+ if (pwritev(fd, iov, 2, 0) != lbs + lbs / 2) {
+ perror("pwritev");
+ return EXIT_FAILURE;
+ }
+
+ if (munmap(map, 2 * hpsz)) {
+ perror("munmap");
+ return EXIT_FAILURE;
+ }
+ }
+
+ after = meminfo("HugePages_Free:");
+ printf("HugePages_Free: %ld -> %ld over %d iterations\n",
+ before, after, iters);
+
+ if (after < before) {
+ printf("%ld huge pages leaked\n", before - after);
+ return EXIT_LEAKED;
+ }
+
+ printf("no huge pages leaked\n");
+ return EXIT_SUCCESS;
+}
diff --git a/tests/block/051 b/tests/block/051
new file mode 100755
index 0000000..a4edec2
--- /dev/null
+++ b/tests/block/051
@@ -0,0 +1,64 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2026 Tal Zussman
+#
+# Issue partial O_DIRECT writes from a hugetlb mapping to a block device with
+# a 64K logical block size and check that no huge pages leak. The bio of such
+# a write is trimmed down to a block boundary, and the pages it drops each
+# hold their own pin when a large folio backs the buffer. If the trim
+# releases at most one of them, whether it drops a whole bvec or shrinks one,
+# the huge page never returns to the pool.
+#
+# Regression test for patch "block: unpin all pages of a bvec in
+# bio_iov_iter_align_down()".
+
+. tests/block/rc
+. common/null_blk
+
+DESCRIPTION="check for pinned page leaks from trimmed direct writes"
+QUICK=1
+
+requires() {
+ _have_null_blk
+ _have_kernel_option TRANSPARENT_HUGEPAGE
+ _have_kernel_option HUGETLBFS
+ _have_src_program bio-trim-pin-leak
+ if (( $(_get_page_size) > 16384 )); then
+ SKIP_REASONS+=("a 64K block size is below four pages")
+ return 1
+ fi
+}
+
+test() {
+ echo "Running ${TEST_NAME}"
+
+ local nr_hugepages ret
+
+ if ! _configure_null_blk nullb1 blocksize=65536 memory_backed=1 \
+ size=64 power=1; then
+ SKIP_REASONS+=("null_blk does not support a 64K block size")
+ return
+ fi
+
+ nr_hugepages=$(cat /proc/sys/vm/nr_hugepages)
+ echo $((nr_hugepages + 40)) > /proc/sys/vm/nr_hugepages
+
+ src/bio-trim-pin-leak /dev/nullb1 16 >>"${FULL}" 2>&1
+ ret=$?
+
+ echo "${nr_hugepages}" > /proc/sys/vm/nr_hugepages
+ _exit_null_blk
+
+ case $ret in
+ 0)
+ ;;
+ 2)
+ echo "huge pages leaked by trimmed direct writes"
+ ;;
+ *)
+ echo "bio-trim-pin-leak helper failed"
+ ;;
+ esac
+
+ echo "Test complete"
+}
diff --git a/tests/block/051.out b/tests/block/051.out
new file mode 100644
index 0000000..e9a2745
--- /dev/null
+++ b/tests/block/051.out
@@ -0,0 +1,2 @@
+Running block/051
+Test complete
--
2.39.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices
2026-10-09 23:53 [PATCH blktests v2 0/4] blktests: tests for the direct I/O fallback, splice, bio trimming, and atomic write fixes Tal Zussman
` (2 preceding siblings ...)
2026-10-09 23:53 ` [PATCH blktests v2 3/4] block/051: add a pinned page leak test for trimmed direct writes Tal Zussman
@ 2026-10-09 23:53 ` Tal Zussman
2026-10-12 8:12 ` John Garry
2026-10-12 16:04 ` John Garry
3 siblings, 2 replies; 8+ messages in thread
From: Tal Zussman @ 2026-10-09 23:53 UTC (permalink / raw)
To: Shin'ichiro Kawasaki, linux-block
Cc: Christoph Hellwig, John Garry, Tal Zussman
An atomic write is submitted as a single bio, so when its buffer can't
be fully pinned, the direct I/O path has to fail the write rather than
submit the pinned part. Otherwise, it submits what was pinned with
REQ_ATOMIC set and leaves the rest to the buffered fallback, tearing the
write at the pin boundary.
This is a regression test for that issue, fixed in the kernel patch
"block: fail a short atomic pin in bio_iov_iter_get_pages()" [1].
Issue pwritev2() with RWF_ATOMIC to a scsi_debug device with atomic_wr=1
from a buffer whose last page is PROT_NONE, so that pinning the buffer
stops one page short, then read the range back with O_DIRECT and fail if
it holds a mix of old and new data.
The write has to span at least two pages so that one can be left
unpinnable, and the device has to support it atomically. Use the largest
power of two up to 16K that the device's atomic write unit max allows,
and skip the test if that is below two pages.
[1]: https://lore.kernel.org/linux-block/20261009-blkdev-fixes-v6-5-307939c387df@columbia.edu/
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
src/.gitignore | 1 +
src/Makefile | 1 +
src/atomic-write-torn.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++++
tests/block/052 | 70 ++++++++++++++++++++++++++
tests/block/052.out | 2 +
5 files changed, 205 insertions(+)
diff --git a/src/.gitignore b/src/.gitignore
index 92c08d1..76a1c5e 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -1,3 +1,4 @@
+/atomic-write-torn
/bio-bounce-read
/bio-full-trim
/bio-trim-pin-leak
diff --git a/src/Makefile b/src/Makefile
index 0bdfd05..54335d9 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -14,6 +14,7 @@ HAVE_C_DEF = $(shell if printf '$(H)include <%s>\n$(H)ifdef %s\nHAVE_%s\n$(H)end
then echo 1;else echo 0; fi)
C_TARGETS := \
+ atomic-write-torn \
bio-bounce-read \
bio-full-trim \
bio-trim-pin-leak \
diff --git a/src/atomic-write-torn.c b/src/atomic-write-torn.c
new file mode 100644
index 0000000..e8ebba3
--- /dev/null
+++ b/src/atomic-write-torn.c
@@ -0,0 +1,131 @@
+// SPDX-License-Identifier: GPL-3.0+
+/*
+ * Copyright (C) 2026 Tal Zussman
+ *
+ * Check that a RWF_ATOMIC direct write to a block device is all or nothing
+ * when only part of its buffer can be pinned.
+ *
+ * Fill a range of the device with one byte value, then issue a pwritev2()
+ * with RWF_ATOMIC of a buffer holding another value whose last page is
+ * PROT_NONE, so that pinning the buffer stops one page short. The direct
+ * I/O path must fail the write outright rather than submit what it pinned.
+ *
+ * Read the range back with O_DIRECT and count the bytes of each value. A
+ * mix of old and new bytes means the write was torn.
+ *
+ * usage: atomic-write-torn <blockdev> <length>
+ *
+ * <length> is the size of the atomic write, a power of two of at least two
+ * pages that the device supports. The first <length> bytes of <blockdev>
+ * are overwritten.
+ *
+ * exit: 0 = the write was all or nothing
+ * 1 = setup error
+ * 2 = the write was torn
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/uio.h>
+#include <unistd.h>
+
+#ifndef RWF_ATOMIC
+#define RWF_ATOMIC 0x00000040
+#endif
+
+#define EXIT_TORN 2
+
+#define OLD 'A'
+#define NEW 'B'
+
+int main(int argc, char **argv)
+{
+ size_t old_bytes = 0, new_bytes = 0;
+ size_t len, pgsz, i;
+ struct iovec iov;
+ char *buf, *check;
+ ssize_t ret;
+ int fd;
+
+ if (argc != 3) {
+ fprintf(stderr, "usage: %s <blockdev> <length>\n", argv[0]);
+ return 1;
+ }
+
+ pgsz = sysconf(_SC_PAGESIZE);
+ len = strtoul(argv[2], NULL, 0);
+ if (len < 2 * pgsz || (len & (len - 1))) {
+ fprintf(stderr, "length must be a power of two of at least two pages\n");
+ return 1;
+ }
+
+ fd = open(argv[1], O_RDWR | O_DIRECT);
+ if (fd < 0) {
+ perror("open");
+ return 1;
+ }
+
+ buf = mmap(NULL, len, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (buf == MAP_FAILED) {
+ perror("mmap");
+ return 1;
+ }
+
+ /* Write the old contents. */
+ memset(buf, OLD, len);
+ if (pwrite(fd, buf, len, 0) != (ssize_t)len) {
+ perror("pwrite");
+ return 1;
+ }
+
+ /* Fill in the new contents and make the last page unreadable. */
+ memset(buf, NEW, len);
+ if (mprotect(buf + len - pgsz, pgsz, PROT_NONE)) {
+ perror("mprotect");
+ return 1;
+ }
+
+ iov.iov_base = buf;
+ iov.iov_len = len;
+ ret = pwritev2(fd, &iov, 1, 0, RWF_ATOMIC);
+ if (ret < 0)
+ printf("pwritev2(RWF_ATOMIC) failed: %s\n", strerror(errno));
+ else
+ printf("pwritev2(RWF_ATOMIC) returned %zd of %zu\n", ret, len);
+
+ /* Read the range back past the page cache. */
+ if (posix_memalign((void **)&check, pgsz, len)) {
+ perror("posix_memalign");
+ return 1;
+ }
+ if (pread(fd, check, len, 0) != (ssize_t)len) {
+ perror("pread");
+ return 1;
+ }
+
+ for (i = 0; i < len; i++) {
+ if (check[i] == OLD)
+ old_bytes++;
+ else if (check[i] == NEW)
+ new_bytes++;
+ }
+ printf("on disk: %zu old bytes, %zu new bytes, %zu other\n",
+ old_bytes, new_bytes, len - old_bytes - new_bytes);
+
+ free(check);
+ munmap(buf, len);
+ close(fd);
+
+ if (old_bytes != len) {
+ printf("atomic write was torn: %zu of %zu bytes written\n",
+ new_bytes, len);
+ return EXIT_TORN;
+ }
+ printf("atomic write was all or nothing\n");
+ return 0;
+}
diff --git a/tests/block/052 b/tests/block/052
new file mode 100755
index 0000000..8ec115a
--- /dev/null
+++ b/tests/block/052
@@ -0,0 +1,70 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2026 Tal Zussman
+#
+# Issue a RWF_ATOMIC direct write to a block device from a buffer whose last
+# page cannot be pinned and check that the write is all or nothing. An atomic
+# write is submitted as a single bio, so the direct I/O path has to fail the
+# write outright when pinning its buffer comes up short. Without that, it
+# submits what was pinned with REQ_ATOMIC set and leaves the rest to the
+# buffered fallback, tearing the write at the pin boundary.
+#
+# Regression test for patch "block: fail a short atomic pin in
+# bio_iov_iter_get_pages()".
+
+. tests/block/rc
+. common/scsi_debug
+
+DESCRIPTION="check that a partially pinned atomic write is not torn"
+QUICK=1
+
+requires() {
+ _have_scsi_debug
+ _have_module_param scsi_debug atomic_wr
+ _have_src_program atomic-write-torn
+}
+
+test() {
+ echo "Running ${TEST_NAME}"
+
+ local dev len page_size unit_max ret
+
+ if ! _configure_scsi_debug delay=0 atomic_wr=1 dev_size_mb=16; then
+ echo "configuring scsi_debug failed"
+ return 1
+ fi
+ dev="/dev/${SCSI_DEBUG_DEVICES[0]}"
+
+ # The largest power of two the device can write atomically, capped at
+ # 16K. It has to span at least two pages so that one can be left
+ # unpinnable.
+ page_size=$(_get_page_size)
+ unit_max=$(< "/sys/block/${SCSI_DEBUG_DEVICES[0]}/queue/atomic_write_unit_max_bytes")
+ len=16384
+ while (( len > unit_max )); do
+ len=$(( len / 2 ))
+ done
+ if (( len < 2 * page_size )); then
+ SKIP_REASONS+=("atomic write unit max ${unit_max} is below two pages")
+ _exit_scsi_debug
+ return
+ fi
+
+ src/atomic-write-torn "${dev}" "${len}" >>"${FULL}" 2>&1
+ ret=$?
+
+ _exit_scsi_debug
+
+ case $ret in
+ 0)
+ ;;
+ 2)
+ echo "atomic write was torn"
+ ;;
+ *)
+ echo "atomic-write-torn helper failed"
+ ;;
+ esac
+
+ echo "Test complete"
+}
diff --git a/tests/block/052.out b/tests/block/052.out
new file mode 100644
index 0000000..71c1f0a
--- /dev/null
+++ b/tests/block/052.out
@@ -0,0 +1,2 @@
+Running block/052
+Test complete
--
2.39.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices
2026-10-09 23:53 ` [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices Tal Zussman
@ 2026-10-12 8:12 ` John Garry
2026-10-12 15:15 ` Tal Zussman
2026-10-12 16:04 ` John Garry
1 sibling, 1 reply; 8+ messages in thread
From: John Garry @ 2026-10-12 8:12 UTC (permalink / raw)
To: Tal Zussman, Shin'ichiro Kawasaki, linux-block; +Cc: Christoph Hellwig
On 10/10/26 00:53, Tal Zussman wrote:
> An atomic write is submitted as a single bio,
nit: A HW-based atomic write is always a single bio. A SW-based atomic
write - like which xfs supports - does not have such a limit.
> so when its buffer can't
> be fully pinned, the direct I/O path has to fail the write rather than
> submit the pinned part. Otherwise, it submits what was pinned with
> REQ_ATOMIC set and leaves the rest to the buffered fallback, tearing the
> write at the pin boundary.
>
> This is a regression test for that issue, fixed in the kernel patch
> "block: fail a short atomic pin in bio_iov_iter_get_pages()" [1].
>
> Issue pwritev2() with RWF_ATOMIC to a scsi_debug device with atomic_wr=1
> from a buffer whose last page is PROT_NONE, so that pinning the buffer
> stops one page short, then read the range back with O_DIRECT and fail if
> it holds a mix of old and new data.
>
> The write has to span at least two pages so that one can be left
> unpinnable, and the device has to support it atomically. Use the largest
> power of two up to 16K that the device's atomic write unit max allows,
> and skip the test if that is below two pages.
As you probably saw, we have other atomic writes tests and they use
xfsprogs (xfs_io, specifically) to issue the atomic write and statx.
Is there scope to use xfsprogs again here? I suppose that the gap is the
mmap and mprotect call.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices
2026-10-12 8:12 ` John Garry
@ 2026-10-12 15:15 ` Tal Zussman
0 siblings, 0 replies; 8+ messages in thread
From: Tal Zussman @ 2026-10-12 15:15 UTC (permalink / raw)
To: John Garry, Shin'ichiro Kawasaki, linux-block; +Cc: Christoph Hellwig
On 10/12/26 4:12 AM, John Garry wrote:
> On 10/10/26 00:53, Tal Zussman wrote:
>> An atomic write is submitted as a single bio,
>
> nit: A HW-based atomic write is always a single bio. A SW-based atomic
> write - like which xfs supports - does not have such a limit.
>
Will clarify this here and in the comments to specify that block devices
use hardware atomic writes.
>> so when its buffer can't
>> be fully pinned, the direct I/O path has to fail the write rather than
>> submit the pinned part. Otherwise, it submits what was pinned with
>> REQ_ATOMIC set and leaves the rest to the buffered fallback, tearing the
>> write at the pin boundary.
>>
>> This is a regression test for that issue, fixed in the kernel patch
>> "block: fail a short atomic pin in bio_iov_iter_get_pages()" [1].
>>
>> Issue pwritev2() with RWF_ATOMIC to a scsi_debug device with atomic_wr=1
>> from a buffer whose last page is PROT_NONE, so that pinning the buffer
>> stops one page short, then read the range back with O_DIRECT and fail if
>> it holds a mix of old and new data.
>>
>> The write has to span at least two pages so that one can be left
>> unpinnable, and the device has to support it atomically. Use the largest
>> power of two up to 16K that the device's atomic write unit max allows,
>> and skip the test if that is below two pages.
>
> As you probably saw, we have other atomic writes tests and they use
> xfsprogs (xfs_io, specifically) to issue the atomic write and statx.
>
> Is there scope to use xfsprogs again here? I suppose that the gap is the
> mmap and mprotect call.
>
I don't think xfs_io can do this, since we need to control the permissions
of the buffer used. So yes, the gap is the mmap() + mprotect().
Thanks,
Tal
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices
2026-10-09 23:53 ` [PATCH blktests v2 4/4] block/052: add a torn atomic write test for block devices Tal Zussman
2026-10-12 8:12 ` John Garry
@ 2026-10-12 16:04 ` John Garry
1 sibling, 0 replies; 8+ messages in thread
From: John Garry @ 2026-10-12 16:04 UTC (permalink / raw)
To: Tal Zussman, Shin'ichiro Kawasaki, linux-block; +Cc: Christoph Hellwig
On 10/10/26 00:53, Tal Zussman wrote:
> An atomic write is submitted as a single bio, so when its buffer can't
> be fully pinned, the direct I/O path has to fail the write rather than
> submit the pinned part. Otherwise, it submits what was pinned with
> REQ_ATOMIC set and leaves the rest to the buffered fallback, tearing the
> write at the pin boundary.
>
> This is a regression test for that issue, fixed in the kernel patch
> "block: fail a short atomic pin in bio_iov_iter_get_pages()" [1].
>
> Issue pwritev2() with RWF_ATOMIC to a scsi_debug device with atomic_wr=1
> from a buffer whose last page is PROT_NONE, so that pinning the buffer
> stops one page short, then read the range back with O_DIRECT and fail if
> it holds a mix of old and new data.
>
> The write has to span at least two pages so that one can be left
> unpinnable, and the device has to support it atomically. Use the largest
> power of two up to 16K that the device's atomic write unit max allows,
> and skip the test if that is below two pages.
>
> [1]: https://lore.kernel.org/linux-block/20261009-blkdev-fixes-v6-5-307939c387df@columbia.edu/
>
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---
> src/.gitignore | 1 +
> src/Makefile | 1 +
> src/atomic-write-torn.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++++
> tests/block/052 | 70 ++++++++++++++++++++++++++
> tests/block/052.out | 2 +
> 5 files changed, 205 insertions(+)
>
> diff --git a/src/.gitignore b/src/.gitignore
> index 92c08d1..76a1c5e 100644
> --- a/src/.gitignore
> +++ b/src/.gitignore
> @@ -1,3 +1,4 @@
> +/atomic-write-torn
> /bio-bounce-read
> /bio-full-trim
> /bio-trim-pin-leak
> diff --git a/src/Makefile b/src/Makefile
> index 0bdfd05..54335d9 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -14,6 +14,7 @@ HAVE_C_DEF = $(shell if printf '$(H)include <%s>\n$(H)ifdef %s\nHAVE_%s\n$(H)end
> then echo 1;else echo 0; fi)
>
> C_TARGETS := \
> + atomic-write-torn \
> bio-bounce-read \
> bio-full-trim \
> bio-trim-pin-leak \
> diff --git a/src/atomic-write-torn.c b/src/atomic-write-torn.c
> new file mode 100644
> index 0000000..e8ebba3
> --- /dev/null
> +++ b/src/atomic-write-torn.c
> @@ -0,0 +1,131 @@
> +// SPDX-License-Identifier: GPL-3.0+
> +/*
> + * Copyright (C) 2026 Tal Zussman
> + *
> + * Check that a RWF_ATOMIC direct write to a block device is all or nothing
> + * when only part of its buffer can be pinned.
> + *
> + * Fill a range of the device with one byte value, then issue a pwritev2()
> + * with RWF_ATOMIC of a buffer holding another value whose last page is
> + * PROT_NONE, so that pinning the buffer stops one page short. The direct
> + * I/O path must fail the write outright rather than submit what it pinned.
> + *
> + * Read the range back with O_DIRECT and count the bytes of each value. A
> + * mix of old and new bytes means the write was torn.
> + *
> + * usage: atomic-write-torn <blockdev> <length>
> + *
> + * <length> is the size of the atomic write, a power of two of at least two
> + * pages that the device supports. The first <length> bytes of <blockdev>
> + * are overwritten.
> + *
> + * exit: 0 = the write was all or nothing
> + * 1 = setup error
> + * 2 = the write was torn
> + */
> +#define _GNU_SOURCE
> +#include <errno.h>
> +#include <fcntl.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/mman.h>
> +#include <sys/uio.h>
> +#include <unistd.h>
> +
> +#ifndef RWF_ATOMIC
> +#define RWF_ATOMIC 0x00000040
> +#endif
> +
> +#define EXIT_TORN 2
> +
> +#define OLD 'A'
> +#define NEW 'B'
> +
> +int main(int argc, char **argv)
This code should really test that the block device is atomically
writable, ideally via statx.
> +{
> + size_t old_bytes = 0, new_bytes = 0;
> + size_t len, pgsz, i;
> + struct iovec iov;
> + char *buf, *check;
> + ssize_t ret;
> + int fd;
> +
> + if (argc != 3) {
> + fprintf(stderr, "usage: %s <blockdev> <length>\n", argv[0]);
> + return 1;
> + }
> +
> + pgsz = sysconf(_SC_PAGESIZE);
> + len = strtoul(argv[2], NULL, 0);
> + if (len < 2 * pgsz || (len & (len - 1))) {
> + fprintf(stderr, "length must be a power of two of at least two pages\n");
> + return 1;
> + }
> +
> + fd = open(argv[1], O_RDWR | O_DIRECT);
> + if (fd < 0) {
> + perror("open");
> + return 1;
> + }
> +
> + buf = mmap(NULL, len, PROT_READ | PROT_WRITE,
> + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> + if (buf == MAP_FAILED) {
Are you intentionally not handling close(fd)?
> + perror("mmap");
> + return 1;
> + }
> +
> + /* Write the old contents. */
> + memset(buf, OLD, len);
> + if (pwrite(fd, buf, len, 0) != (ssize_t)len) {
> + perror("pwrite");
> + return 1;> + }
I suppose that it would be total paranoia to re-read and test vs expected.
> +
> + /* Fill in the new contents and make the last page unreadable. */
> + memset(buf, NEW, len);
> + if (mprotect(buf + len - pgsz, pgsz, PROT_NONE)) {
> + perror("mprotect");
> + return 1;
> + }
> +
> + iov.iov_base = buf;
> + iov.iov_len = len;
> + ret = pwritev2(fd, &iov, 1, 0, RWF_ATOMIC);
> + if (ret < 0)
> + printf("pwritev2(RWF_ATOMIC) failed: %s\n", strerror(errno));
> + else
> + printf("pwritev2(RWF_ATOMIC) returned %zd of %zu\n", ret, len);
As a sanity check we could test the error code is -EINVAL. That's always
expected, right?
> +
> + /* Read the range back past the page cache. */
> + if (posix_memalign((void **)&check, pgsz, len)) {
Can you just re-use buf?
> + perror("posix_memalign");
> + return 1;
> + }
> + if (pread(fd, check, len, 0) != (ssize_t)len) {
> + perror("pread");
> + return 1;
> + }
> +
> + for (i = 0; i < len; i++) {
> + if (check[i] == OLD)
> + old_bytes++;
> + else if (check[i] == NEW)
> + new_bytes++;
> + }
> + printf("on disk: %zu old bytes, %zu new bytes, %zu other\n",
> + old_bytes, new_bytes, len - old_bytes - new_bytes);
> +
> + free(check);
> + munmap(buf, len);
> + close(fd);
> +
> + if (old_bytes != len) {
> + printf("atomic write was torn: %zu of %zu bytes written\n",
> + new_bytes, len);
So what happens if we did write all the new bytes (even though it should
not happen)? That's not a torn write. This really goes back to testing
pwritev2() error code, above.
> + return EXIT_TORN;
> + }
> + printf("atomic write was all or nothing\n");
> + return 0;
> +}
> diff --git a/tests/block/052 b/tests/block/052
> new file mode 100755
> index 0000000..8ec115a
> --- /dev/null
> +++ b/tests/block/052
> @@ -0,0 +1,70 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-3.0+
> +# Copyright (C) 2026 Tal Zussman
> +#
> +# Issue a RWF_ATOMIC direct write to a block device from a buffer whose last
> +# page cannot be pinned and check that the write is all or nothing. An atomic
> +# write is submitted as a single bio, so the direct I/O path has to fail the
> +# write outright when pinning its buffer comes up short. Without that, it
> +# submits what was pinned with REQ_ATOMIC set and leaves the rest to the
> +# buffered fallback, tearing the write at the pin boundary.
> +#
> +# Regression test for patch "block: fail a short atomic pin in
> +# bio_iov_iter_get_pages()".
Please run shellcheck on this if not already done so. JFYI, the
maintainer asks for this.
> +
> +. tests/block/rc
> +. common/scsi_debug
> +
> +DESCRIPTION="check that a partially pinned atomic write is not torn"
> +QUICK=1
> +
> +requires() {
> + _have_scsi_debug
> + _have_module_param scsi_debug atomic_wr
> + _have_src_program atomic-write-torn
> +}
> +
> +test() {
> + echo "Running ${TEST_NAME}"
> +
> + local dev len page_size unit_max ret
> +
> + if ! _configure_scsi_debug delay=0 atomic_wr=1 dev_size_mb=16; then
I think that delay defaults to zero now.
> + echo "configuring scsi_debug failed"
> + return 1
> + fi
> + dev="/dev/${SCSI_DEBUG_DEVICES[0]}"
> +
> + # The largest power of two the device can write atomically, capped at
> + # 16K. It has to span at least two pages so that one can be left
> + # unpinnable.
> + page_size=$(_get_page_size)
> + unit_max=$(< "/sys/block/${SCSI_DEBUG_DEVICES[0]}/queue/atomic_write_unit_max_bytes")
> + len=16384
> + while (( len > unit_max )); do
> + len=$(( len / 2 ))
> + done
> + if (( len < 2 * page_size )); then
> + SKIP_REASONS+=("atomic write unit max ${unit_max} is below two pages")
Can you set params for scsi_debug such that this does not happen?
> + _exit_scsi_debug
> + return
> + fi
> +
> + src/atomic-write-torn "${dev}" "${len}" >>"${FULL}" 2>&1
> + ret=$?
> +
> + _exit_scsi_debug
> +
> + case $ret in
> + 0)
> + ;;
> + 2)
> + echo "atomic write was torn"
> + ;;
> + *)
> + echo "atomic-write-torn helper failed"
> + ;;
> + esac
> +
> + echo "Test complete"
> +}
> diff --git a/tests/block/052.out b/tests/block/052.out
> new file mode 100644
> index 0000000..71c1f0a
> --- /dev/null
> +++ b/tests/block/052.out
> @@ -0,0 +1,2 @@
> +Running block/052
> +Test complete
>
^ permalink raw reply [flat|nested] 8+ messages in thread