[PATCHSET 1/2] fstests: more random fixes for v2025.10.05

[PATCHSET 1/2] fstests: more random fixes for v2025.10.05

[Darrick J. Wong]

Hi all,

Here's the usual odd fixes for fstests.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
Commits in this patchset:
 * generic/427: try to ensure there's some free space before we do the aio test
 * common/rc: fix _require_xfs_io_shutdown
 * generic/742: avoid infinite loop if no fiemap results
 * generic/{482,757}: skip test if there are no FUA writes
 * generic/772: actually check for file_getattr special file support
 * common/filter: fix _filter_file_attributes to handle src/file_attr.c file flags
 * common/attr: fix _require_noattr2
 * common: fix _require_xfs_io_command pwrite -A for various blocksizes
 * generic/778: fix severe performance problems
 * check: line up stdout columns
 * fsx: don't print messages when atomic writes are explicitly disabled
---
 check              |    8 +++++++-
 common/attr        |    4 ++++
 common/filter      |   33 +++++++++++++++++++++++++--------
 common/rc          |   21 +++++++++++++++++----
 ltp/fsx.c          |    1 -
 src/fiemap-fault.c |   14 +++++++++++++-
 tests/generic/427  |    3 +++
 tests/generic/482  |    2 +-
 tests/generic/757  |    2 +-
 tests/generic/772  |   41 ++++++++++++++++++++++-------------------
 tests/generic/778  |   39 ++++++++++++++++++++++++++-------------
 tests/xfs/648      |    3 +++
 12 files changed, 122 insertions(+), 49 deletions(-)

[PATCH 01/11] generic/427: try to ensure there's some free space before we do the aio test

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

On a filesystem configured like this:
MKFS_OPTIONS="-m metadir=1,autofsck=1,uquota,gquota,pquota -d rtinherit=1 -r zoned=1"

This test fails like this:

 --- a/tests/generic/427.out      2025-04-30 16:20:44.584246582 -0700
 +++ b/tests/generic/427.out.bad        2025-07-14 10:47:07.605377287 -0700
 @@ -1,2 +1,2 @@
  QA output created by 427
 -Success, all done.
 +pwrite: No space left on device

The pwrite failure comes from the aio-dio-eof-race.c program because the
filesystem ran out of space.  There are no speculative posteof
preallocations on a zoned filesystem, so let's skip this test on those
setups.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 tests/generic/427 |    3 +++
 1 file changed, 3 insertions(+)


diff --git a/tests/generic/427 b/tests/generic/427
index bddfdb8714e9a7..bb20d9f44a2b5a 100755
--- a/tests/generic/427
+++ b/tests/generic/427
@@ -28,6 +28,9 @@ _require_no_compress
 _scratch_mkfs_sized $((256 * 1024 * 1024)) >>$seqres.full 2>&1
 _scratch_mount
 
+# Zoned filesystems don't support speculative preallocations
+_require_inplace_writes $SCRATCH_MNT
+
 # try to write more bytes than filesystem size to fill the filesystem,
 # then remove all these data. If we still can find these stale data in
 # a file' eofblock, then it's a bug

[PATCH 02/11] common/rc: fix _require_xfs_io_shutdown

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Capturing the output of _scratch_shutdown_handle requires one to enclose
the callsite with $(), otherwise you're comparing the literal string
"_scratch_shutdown_handle" to $SCRATCH_MNT, which always fails.

Also fix _require_xfs_io_command to handle testing the shutdown command
correctly.

Cc: <fstests@vger.kernel.org> # v2025.06.22
Fixes: 4b1cf3df009b22 ("fstests: add helper _require_xfs_io_shutdown")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 common/rc |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)


diff --git a/common/rc b/common/rc
index 8fd7876a9b95c5..2e06e375c2cff2 100644
--- a/common/rc
+++ b/common/rc
@@ -619,7 +619,7 @@ _scratch_shutdown_and_syncfs()
 # requirement down to _require_scratch_shutdown.
 _require_xfs_io_shutdown()
 {
-	if [ _scratch_shutdown_handle != $SCRATCH_MNT ]; then
+	if [ $(_scratch_shutdown_handle) != $SCRATCH_MNT ]; then
 		# Most likely overlayfs
 		_notrun "xfs_io -c shutdown not supported on $FSTYP"
 	fi
@@ -3073,6 +3073,11 @@ _require_xfs_io_command()
 		rm -f $testfile.1
 		param_checked="$param"
 		;;
+	"shutdown")
+		testio=$($XFS_IO_PROG -f -x -c "$command $param" $testfile 2>&1)
+		param_checked="$param"
+		_test_cycle_mount
+		;;
 	"utimes" )
 		testio=`$XFS_IO_PROG -f -c "utimes 0 0 0 0" $testfile 2>&1`
 		;;

[PATCH 03/11] generic/742: avoid infinite loop if no fiemap results

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The fiemap-fault program employed this program looks for deadlocks
between FIEMAP and file page faults by calling the FIEMAP ioctl with a
buffer pointer that is mmaped file range.  Unfortunately, the FIEMAP
loop can enter an infinite loop if FIEMAP reports zero extents because
it never changes the last variable.

This shouldn't happen if the filesystem is working correctly, but it
turns out that there's a bug in libext2fs' punch-range code that causes
punch-alternating to unmap all the double-indirect blocks in the file.
This causes the while loop to run forever because last never increases,
which then means that testing fuse2fs with ext2/ext3 grinds to a halt
because fstests doesn't enforce a per-testcase time limit.

Avoid this situation by bailing out if the loop doesn't make forward
progress.

Cc: <fstests@vger.kernel.org> # v2024.03.31
Fixes: 34cdaf0831ee42 ("generic: add a regression test for fiemap into an mmap range")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 src/fiemap-fault.c |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)


diff --git a/src/fiemap-fault.c b/src/fiemap-fault.c
index 73260068054ede..2bb474c083986d 100644
--- a/src/fiemap-fault.c
+++ b/src/fiemap-fault.c
@@ -55,17 +55,29 @@ int main(int argc, char *argv[])
 				  sizeof(struct fiemap_extent);
 
 	while (last < sz) {
+		size_t old_start;
 		int i;
 
-		fiemap->fm_start = last;
+		fiemap->fm_start = old_start = last;
 		fiemap->fm_length = sz - last;
 
 		ret = ioctl(fd, FS_IOC_FIEMAP, (unsigned long)fiemap);
 		if (ret < 0)
 			err(1, "fiemap failed %d", errno);
+		if (fiemap->fm_mapped_extents == 0) {
+			fprintf(stderr, "%s: fiemap returned 0 extents!\n",
+				argv[0]);
+			return 1;
+		}
 		for (i = 0; i < fiemap->fm_mapped_extents; i++)
 		       last = fiemap->fm_extents[i].fe_logical +
 			       fiemap->fm_extents[i].fe_length;
+
+		if (last <= old_start) {
+			fprintf(stderr, "%s: fiemap made no progress!\n",
+				argv[0]);
+			return 1;
+		}
 	}
 
 	munmap(buf, sz);

[PATCH 04/11] generic/{482,757}: skip test if there are no FUA writes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Both of these tests fail if the filesystem doesn't issue a FUA write to
a device, but ... there's no requirement that filesystems actually use
FUA at all.  For example, a fuse filesystem that writes to the block
device's page cache and issues fsync() will not cause the block layer to
issue FUA writes for the dirty pages.  Change that to _notrun.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/482 |    2 +-
 tests/generic/757 |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/tests/generic/482 b/tests/generic/482
index 8c114ee03058c6..3728112ff3f480 100755
--- a/tests/generic/482
+++ b/tests/generic/482
@@ -82,7 +82,7 @@ _log_writes_remove
 prev=$(_log_writes_mark_to_entry_number mkfs)
 [ -z "$prev" ] && _fail "failed to locate entry mark 'mkfs'"
 cur=$(_log_writes_find_next_fua $prev)
-[ -z "$cur" ] && _fail "failed to locate next FUA write"
+[ -z "$cur" ] && _notrun "could not locate any FUA write"
 
 while [ ! -z "$cur" ]; do
 	_log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full
diff --git a/tests/generic/757 b/tests/generic/757
index 6c13c6af41c57c..11398d8677f72b 100755
--- a/tests/generic/757
+++ b/tests/generic/757
@@ -71,7 +71,7 @@ _log_writes_remove
 prev=$(_log_writes_mark_to_entry_number mkfs)
 [ -z "$prev" ] && _fail "failed to locate entry mark 'mkfs'"
 cur=$(_log_writes_find_next_fua $prev)
-[ -z "$cur" ] && _fail "failed to locate next FUA write"
+[ -z "$cur" ] && _notrun "could not locate any FUA write"
 
 while _soak_loop_running $((100 * TIME_FACTOR)); do
 	_log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full

Re: [PATCH 04/11] generic/{482,757}: skip test if there are no FUA writes

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 05/11] generic/772: actually check for file_getattr special file support

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

On XFS in 6.17, this test fails with:

 --- /run/fstests/bin/tests/generic/772.out	2025-10-06 08:27:10.834318149 -0700
 +++ /var/tmp/fstests/generic/772.out.bad	2025-10-08 18:00:34.713388178 -0700
 @@ -9,29 +9,34 @@ Can not get fsxattr on ./foo: Invalid ar
  Can not set fsxattr on ./foo: Invalid argument
  Initial attributes state
  ----------------- SCRATCH_MNT/prj
 ------------------ ./fifo
 ------------------ ./chardev
 ------------------ ./blockdev
 ------------------ ./socket
 ------------------ ./foo
 ------------------ ./symlink
 +Can not get fsxattr on ./fifo: Inappropriate ioctl for device
 +Can not get fsxattr on ./chardev: Inappropriate ioctl for device
 +Can not get fsxattr on ./blockdev: Inappropriate ioctl for device
 +Can not get fsxattr on ./socket: Inappropriate ioctl for device

This is a result of XFS' file_getattr implementation rejecting special
files prior to 6.18.  Therefore, skip this new test on old kernels.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 tests/generic/772 |    3 +++
 tests/xfs/648     |    3 +++
 2 files changed, 6 insertions(+)


diff --git a/tests/generic/772 b/tests/generic/772
index cc1a1bb5bf655c..e68a6724654450 100755
--- a/tests/generic/772
+++ b/tests/generic/772
@@ -43,6 +43,9 @@ touch $projectdir/bar
 ln -s $projectdir/bar $projectdir/broken-symlink
 rm -f $projectdir/bar
 
+file_attr --get $projectdir ./fifo &>/dev/null || \
+	_notrun "file_getattr not supported on $FSTYP"
+
 echo "Error codes"
 # wrong AT_ flags
 file_attr --get --invalid-at $projectdir ./foo
diff --git a/tests/xfs/648 b/tests/xfs/648
index 215c809887b609..e3c2fbe00b666a 100755
--- a/tests/xfs/648
+++ b/tests/xfs/648
@@ -47,6 +47,9 @@ touch $projectdir/bar
 ln -s $projectdir/bar $projectdir/broken-symlink
 rm -f $projectdir/bar
 
+$here/src/file_attr --get $projectdir ./fifo &>/dev/null || \
+	_notrun "file_getattr not supported on $FSTYP"
+
 $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 	-c "project -sp $projectdir $id" $SCRATCH_DEV | filter_quota
 $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \

[PATCH 06/11] common/filter: fix _filter_file_attributes to handle src/file_attr.c file flags

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Currently, _filter_file_attributes filters ext4 lsattr flags, but it's
only callers (all introduced in 4eb4017) are using it to filter the file
attribute letters from src/file_attr.c.  These correspond to the flags
defined by the new file_getattr system call, so let's rename the helper
to make it clear we're filtering *VFS* attribute characters, and change
the code to filter all known characters from that interface.

While we're at it, make filtering of multiple characters work
consistently whether we're masking specific flags or everything *except*
those flags.

Cc: <fstests@vger.kernel.org> # v2025.10.05
Fixes: 4eb40174d77c1b ("generic: introduce test to test file_getattr/file_setattr syscalls")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/filter     |   33 +++++++++++++++++++++++++--------
 tests/generic/772 |   38 +++++++++++++++++++-------------------
 2 files changed, 44 insertions(+), 27 deletions(-)


diff --git a/common/filter b/common/filter
index b330b27827d005..b27ed7eda61a83 100644
--- a/common/filter
+++ b/common/filter
@@ -683,20 +683,37 @@ _filter_sysfs_error()
 	sed 's/.*: \(.*\)$/\1/'
 }
 
-# Filter file attributes (aka lsattr/chattr)
-# To filter X:
-# 	... | _filter_file_attributes X
-# Or to filter all except X
-# 	... | _filter_file_attributes ~X
-_filter_file_attributes()
+# General filter of file attribute characters, do not call directly
+__filter_file_attributes()
 {
+	local filter="$1"
+	shift
+
 	if [[ $1 == ~* ]]; then
-		regex=$(echo "[aAcCdDeEFijmNPsStTuxVX]" | tr -d "$1")
+		regex=$(echo "[${filter}]" | tr -d "$1")
 	else
-		regex="$1"
+		regex="[$1]"
 	fi
 	awk "{ printf \"%s \", gensub(\"$regex\", \"-\", \"g\", \$1) } {print \$2}"
 }
 
+# Filter src/file_attr.c file attribute characters, which for now match the
+# xfs_io lsattr file attribute characters.
+#
+# WARNING: Be careful, this does not filter ext4 lsattr characters!
+#
+# To filter X:
+# 	... | _filter_vfs_file_attributes X
+# To filter X and Y:
+# 	... | _filter_vfs_file_attributes XY
+# Or to filter all except X
+# 	... | _filter_vfs_file_attributes ~X
+# Or to filter all except X and Y
+# 	... | _filter_vfs_file_attributes ~XY
+_filter_vfs_file_attributes()
+{
+	__filter_file_attributes aACdeEfinpPrsStxX "$@"
+}
+
 # make sure this script returns success
 /bin/true
diff --git a/tests/generic/772 b/tests/generic/772
index e68a6724654450..cdadf09ff26158 100755
--- a/tests/generic/772
+++ b/tests/generic/772
@@ -59,13 +59,13 @@ file_attr --set --too-small-arg $projectdir ./foo
 file_attr --set --new-fsx-flag $projectdir ./foo
 
 echo "Initial attributes state"
-file_attr --get $projectdir | _filter_scratch | _filter_file_attributes ~d
-file_attr --get $projectdir ./fifo | _filter_file_attributes ~d
-file_attr --get $projectdir ./chardev | _filter_file_attributes ~d
-file_attr --get $projectdir ./blockdev | _filter_file_attributes ~d
-file_attr --get $projectdir ./socket | _filter_file_attributes ~d
-file_attr --get $projectdir ./foo | _filter_file_attributes ~d
-file_attr --get $projectdir ./symlink | _filter_file_attributes ~d
+file_attr --get $projectdir | _filter_scratch | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./fifo | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./chardev | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./blockdev | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./socket | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./foo | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./symlink | _filter_vfs_file_attributes ~d
 
 echo "Set FS_XFLAG_NODUMP (d)"
 file_attr --set --set-nodump $projectdir
@@ -77,35 +77,35 @@ file_attr --set --set-nodump $projectdir ./foo
 file_attr --set --set-nodump $projectdir ./symlink
 
 echo "Read attributes"
-file_attr --get $projectdir | _filter_scratch | _filter_file_attributes ~d
-file_attr --get $projectdir ./fifo | _filter_file_attributes ~d
-file_attr --get $projectdir ./chardev | _filter_file_attributes ~d
-file_attr --get $projectdir ./blockdev | _filter_file_attributes ~d
-file_attr --get $projectdir ./socket | _filter_file_attributes ~d
-file_attr --get $projectdir ./foo | _filter_file_attributes ~d
-file_attr --get $projectdir ./symlink | _filter_file_attributes ~d
+file_attr --get $projectdir | _filter_scratch | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./fifo | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./chardev | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./blockdev | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./socket | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./foo | _filter_vfs_file_attributes ~d
+file_attr --get $projectdir ./symlink | _filter_vfs_file_attributes ~d
 
 echo "Set attribute on broken link with AT_SYMLINK_NOFOLLOW"
 file_attr --set --set-nodump $projectdir ./broken-symlink
 file_attr --get $projectdir ./broken-symlink
 
 file_attr --set --no-follow --set-nodump $projectdir ./broken-symlink
-file_attr --get --no-follow $projectdir ./broken-symlink | _filter_file_attributes ~d
+file_attr --get --no-follow $projectdir ./broken-symlink | _filter_vfs_file_attributes ~d
 
 cd $SCRATCH_MNT
 touch ./foo2
 echo "Initial state of foo2"
-file_attr --get --at-cwd ./foo2 | _filter_file_attributes ~d
+file_attr --get --at-cwd ./foo2 | _filter_vfs_file_attributes ~d
 echo "Set attribute relative to AT_FDCWD"
 file_attr --set --at-cwd --set-nodump ./foo2
-file_attr --get --at-cwd ./foo2 | _filter_file_attributes ~d
+file_attr --get --at-cwd ./foo2 | _filter_vfs_file_attributes ~d
 
 echo "Set attribute on AT_FDCWD"
 mkdir ./bar
-file_attr --get --at-cwd ./bar | _filter_file_attributes ~d
+file_attr --get --at-cwd ./bar | _filter_vfs_file_attributes ~d
 cd ./bar
 file_attr --set --at-cwd --set-nodump ""
-file_attr --get --at-cwd . | _filter_file_attributes ~d
+file_attr --get --at-cwd . | _filter_vfs_file_attributes ~d
 
 # success, all done
 status=0

Re: [PATCH 06/11] common/filter: fix _filter_file_attributes to handle src/file_attr.c file flags

[Christoph Hellwig]

On Tue, Oct 21, 2025 at 11:40:46AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Currently, _filter_file_attributes filters ext4 lsattr flags, but it's
> only callers (all introduced in 4eb4017) are using it to filter the file
> attribute letters from src/file_attr.c.  These correspond to the flags
> defined by the new file_getattr system call, so let's rename the helper
> to make it clear we're filtering *VFS* attribute characters, and change
> the code to filter all known characters from that interface.

Thanks, with this commit message even me understands what's going on now
:)

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 07/11] common/attr: fix _require_noattr2

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

attr2/noattr2 doesn't do anything anymore and aren't reported in
/proc/mounts, so we need to check /proc/mounts and _notrun as a result.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 common/attr |    4 ++++
 1 file changed, 4 insertions(+)


diff --git a/common/attr b/common/attr
index 1c1de63e9d5465..eb5c862556d3be 100644
--- a/common/attr
+++ b/common/attr
@@ -241,7 +241,11 @@ _require_noattr2()
 		|| _fail "_try_scratch_mkfs_xfs failed on $SCRATCH_DEV"
 	_try_scratch_mount -o noattr2 > /dev/null 2>&1 \
 		|| _notrun "noattr2 mount option not supported on $SCRATCH_DEV"
+	grep -qw noattr2 <(findmnt -rncv -M "$SCRATCH_MNT" -o OPTIONS)
+	local res=$?
 	_scratch_unmount
+	test $res -eq 0 \
+		|| _notrun "noattr2 mount option no longer functional"
 }
 
 # getfattr -R returns info in readdir order which varies from fs to fs.

[PATCH 08/11] common: fix _require_xfs_io_command pwrite -A for various blocksizes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In this predicate, we should test an atomic write of the minimum
supported size, not just 4k.  This fixes a problem where none of the
atomic write tests actually run on a 32k-fsblock xfs because you can't
do a sub-fsblock atomic write.

Cc: <fstests@vger.kernel.org> # v2025.04.13
Fixes: d90ee3b6496346 ("generic: add a test for atomic writes")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 common/rc |   14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)


diff --git a/common/rc b/common/rc
index 2e06e375c2cff2..462f433197a3c2 100644
--- a/common/rc
+++ b/common/rc
@@ -3030,16 +3030,24 @@ _require_xfs_io_command()
 	"pwrite")
 		# -N (RWF_NOWAIT) only works with direct vectored I/O writes
 		local pwrite_opts=" "
+		local write_size="4k"
 		if [ "$param" == "-N" ]; then
 			opts+=" -d"
-			pwrite_opts+="-V 1 -b 4k"
+			pwrite_opts+="-V 1 -b $write_size"
 		fi
 		if [ "$param" == "-A" ]; then
 			opts+=" -d"
-			pwrite_opts+="-V 1 -b 4k"
+			# try to write the minimum supported atomic write size
+			write_size="$($XFS_IO_PROG -f -c "statx -r -m $STATX_WRITE_ATOMIC" $testfile 2>/dev/null | \
+				grep atomic_write_unit_min | \
+				grep -o '[0-9]\+')"
+			if [ -z "$write_size" ] || [ "$write_size" = "0" ]; then
+				write_size="0 --not-supported"
+			fi
+			pwrite_opts+="-V 1 -b $write_size"
 		fi
 		testio=`$XFS_IO_PROG -f $opts -c \
-		        "pwrite $pwrite_opts $param 0 4k" $testfile 2>&1`
+		        "pwrite $pwrite_opts $param 0 $write_size" $testfile 2>&1`
 		param_checked="$pwrite_opts $param"
 		;;
 	"scrub"|"repair")

[PATCH 09/11] generic/778: fix severe performance problems

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test takes 4800s to run, which is horrible.  AFAICT it starts out
by timing how much can be written atomically to a new file in 0.2
seconds, then scales up the file size by 3x.  On not very fast storage,
this can result in file_size being set to ~250MB on a 4k fsblock
filesystem.  That's about 64,000 blocks.

The next thing this test does is try to create a file of that size
(250MB) of alternating written and unwritten blocks.  For some reason,
it sets up this file by invoking xfs_io 64,000 times to write small
amounts of data, which takes 3+ minutes on the author's system because
exec overhead is pretty high when you do that.

As a result, one loop through the test takes almost 4 minutes.  The test
loops 20 times, so it runs for 80 minutes(!!) which is a really long
time.

So the first thing we do is observe that the giant slow loop is being
run as a single thread on an empty filesystem.  Most of the time the
allocator generates a mostly physically contiguous file.  We could
fallocate the whole file instead of fallocating one block every other
time through the loop.  This halves the setup time.

Next, we can also stuff the remaining pwrite commands into a bash array
and only invoke xfs_io once every 128x through the loop.  This amortizes
the xfs_io startup time, which reduces the test loop runtime to about 20
seconds.

Finally, replace the 20x loop with a _soak_loop_running 5x loop because
5 seems like enough.  Anyone who wants more can set TIME_FACTOR or
SOAK_DURATION to get more intensive testing.  On my system this cuts the
runtime to 75 seconds.

Cc: <fstests@vger.kernel.org> # v2025.10.20
Fixes: ca954527ff9d97 ("generic: Add sudden shutdown tests for multi block atomic writes")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/778 |   39 ++++++++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 13 deletions(-)


diff --git a/tests/generic/778 b/tests/generic/778
index 8cb1d8d4cad45d..3e070fc7d157d5 100755
--- a/tests/generic/778
+++ b/tests/generic/778
@@ -113,21 +113,32 @@ create_mixed_mappings() {
 	local off=0
 	local operations=("W" "U")
 
+	# fallocate the whole file once because preallocating single blocks
+	# with individual xfs_io invocations is really slow and the allocator
+	# usually gives out consecutive blocks anyway
+	$XFS_IO_PROG -f -c "falloc 0 $size_bytes" $file
+
+	local cmds=()
 	for ((i=0; i<$((size_bytes / blksz )); i++)); do
-		index=$(($i % ${#operations[@]}))
-		map="${operations[$index]}"
+		if (( i % 2 == 0 )); then
+			cmds+=("pwrite -b $blksz $off $blksz")
+		fi
+
+		# batch the write commands into larger xfs_io invocations to
+		# amortize the fork overhead
+		if [ "${#cmds[@]}" -ge 128 ]; then
+			$XFS_IO_PROG -f -c "${cmds[@]}" "$file" >> /dev/null
+			cmds=()
+		fi
 
-		case "$map" in
-		    "W")
-			$XFS_IO_PROG -fc "pwrite -b $blksz $off $blksz" $file  >> /dev/null
-			;;
-		    "U")
-			$XFS_IO_PROG -fc "falloc $off $blksz" $file >> /dev/null
-			;;
-		esac
 		off=$((off + blksz))
 	done
 
+	if [ "${#cmds[@]}" -gt 0 ]; then
+		$XFS_IO_PROG -f -c "${cmds[@]}" "$file" >> /dev/null
+		cmds=()
+	fi
+
 	sync $file
 }
 
@@ -336,9 +347,9 @@ echo >> $seqres.full
 echo "# Populating expected data buffers" >> $seqres.full
 populate_expected_data
 
-# Loop 20 times to shake out any races due to shutdown
-for ((iter=0; iter<20; iter++))
-do
+# Loop 5 times(ish) to shake out any races due to shutdown
+iter=0
+while _soak_loop_running $((5 * TIME_FACTOR)); do
 	echo >> $seqres.full
 	echo "------ Iteration $iter ------" >> $seqres.full
 
@@ -361,6 +372,8 @@ do
 	echo >> $seqres.full
 	echo "# Starting shutdown torn write test for append atomic writes" >> $seqres.full
 	test_append_torn_write
+
+	iter=$((iter + 1))
 done
 
 echo "Silence is golden"

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Christoph Hellwig]

On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> As a result, one loop through the test takes almost 4 minutes.  The test
> loops 20 times, so it runs for 80 minutes(!!) which is a really long
> time.

Heh.  I'm glade none of my usual test setups even supports atomics I
guess :)

> So the first thing we do is observe that the giant slow loop is being
> run as a single thread on an empty filesystem.  Most of the time the
> allocator generates a mostly physically contiguous file.  We could
> fallocate the whole file instead of fallocating one block every other
> time through the loop.  This halves the setup time.
> 
> Next, we can also stuff the remaining pwrite commands into a bash array
> and only invoke xfs_io once every 128x through the loop.  This amortizes
> the xfs_io startup time, which reduces the test loop runtime to about 20
> seconds.

Wouldn't it make sense to adopt src/punch-alternating.c to also be
able to create unwritten extents instead of holes for the punched
range and run all of this from a C program?

Otherwise this looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Darrick J. Wong]

On Tue, Oct 21, 2025 at 09:20:55PM -0700, Christoph Hellwig wrote:
> On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> > As a result, one loop through the test takes almost 4 minutes.  The test
> > loops 20 times, so it runs for 80 minutes(!!) which is a really long
> > time.
> 
> Heh.  I'm glade none of my usual test setups even supports atomics I
> guess :)

FWIW the failure was on a regular xfs, no hw atomics.  So in theory
you're affected, but only if you pulled the 20 Oct next branch.

> > So the first thing we do is observe that the giant slow loop is being
> > run as a single thread on an empty filesystem.  Most of the time the
> > allocator generates a mostly physically contiguous file.  We could
> > fallocate the whole file instead of fallocating one block every other
> > time through the loop.  This halves the setup time.
> > 
> > Next, we can also stuff the remaining pwrite commands into a bash array
> > and only invoke xfs_io once every 128x through the loop.  This amortizes
> > the xfs_io startup time, which reduces the test loop runtime to about 20
> > seconds.
> 
> Wouldn't it make sense to adopt src/punch-alternating.c to also be
> able to create unwritten extents instead of holes for the punched
> range and run all of this from a C program?

For the write sizes it comes up with I'm guessing that this test will
almost always be poking the software fallbacks so it probably doesn't
matter if the file is full of holes.

> Otherwise this looks good:
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>

Thanks!

--D

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Darrick J. Wong]

On Tue, Oct 21, 2025 at 09:27:31PM -0700, Darrick J. Wong wrote:
> On Tue, Oct 21, 2025 at 09:20:55PM -0700, Christoph Hellwig wrote:
> > On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> > > As a result, one loop through the test takes almost 4 minutes.  The test
> > > loops 20 times, so it runs for 80 minutes(!!) which is a really long
> > > time.
> > 
> > Heh.  I'm glade none of my usual test setups even supports atomics I
> > guess :)
> 
> FWIW the failure was on a regular xfs, no hw atomics.  So in theory
> you're affected, but only if you pulled the 20 Oct next branch.
> 
> > > So the first thing we do is observe that the giant slow loop is being
> > > run as a single thread on an empty filesystem.  Most of the time the
> > > allocator generates a mostly physically contiguous file.  We could
> > > fallocate the whole file instead of fallocating one block every other
> > > time through the loop.  This halves the setup time.
> > > 
> > > Next, we can also stuff the remaining pwrite commands into a bash array
> > > and only invoke xfs_io once every 128x through the loop.  This amortizes
> > > the xfs_io startup time, which reduces the test loop runtime to about 20
> > > seconds.
> > 
> > Wouldn't it make sense to adopt src/punch-alternating.c to also be
> > able to create unwritten extents instead of holes for the punched
> > range and run all of this from a C program?
> 
> For the write sizes it comes up with I'm guessing that this test will
> almost always be poking the software fallbacks so it probably doesn't
> matter if the file is full of holes.

...and now running this with 32k-fsblocks reveals that the
atomic_write_loop code actually writes the wrong value into $tmp.aw and
only runs the loop once, so the test fails because dry_run thinks the
file size should be 0.

Also the cmds+=() line needs to insert its own -c or else you end up
writing huge files to $here.  Ooops.

Will send a v2 once the brownpaperbag testing finishes.

--D

> > Otherwise this looks good:
> > 
> > Reviewed-by: Christoph Hellwig <hch@lst.de>
> 
> Thanks!
> 
> --D
> 

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Zorro Lang]

On Fri, Oct 31, 2025 at 10:47:34AM -0700, Darrick J. Wong wrote:
> On Tue, Oct 21, 2025 at 09:27:31PM -0700, Darrick J. Wong wrote:
> > On Tue, Oct 21, 2025 at 09:20:55PM -0700, Christoph Hellwig wrote:
> > > On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> > > > As a result, one loop through the test takes almost 4 minutes.  The test
> > > > loops 20 times, so it runs for 80 minutes(!!) which is a really long
> > > > time.
> > > 
> > > Heh.  I'm glade none of my usual test setups even supports atomics I
> > > guess :)
> > 
> > FWIW the failure was on a regular xfs, no hw atomics.  So in theory
> > you're affected, but only if you pulled the 20 Oct next branch.
> > 
> > > > So the first thing we do is observe that the giant slow loop is being
> > > > run as a single thread on an empty filesystem.  Most of the time the
> > > > allocator generates a mostly physically contiguous file.  We could
> > > > fallocate the whole file instead of fallocating one block every other
> > > > time through the loop.  This halves the setup time.
> > > > 
> > > > Next, we can also stuff the remaining pwrite commands into a bash array
> > > > and only invoke xfs_io once every 128x through the loop.  This amortizes
> > > > the xfs_io startup time, which reduces the test loop runtime to about 20
> > > > seconds.
> > > 
> > > Wouldn't it make sense to adopt src/punch-alternating.c to also be
> > > able to create unwritten extents instead of holes for the punched
> > > range and run all of this from a C program?
> > 
> > For the write sizes it comes up with I'm guessing that this test will
> > almost always be poking the software fallbacks so it probably doesn't
> > matter if the file is full of holes.
> 
> ...and now running this with 32k-fsblocks reveals that the
> atomic_write_loop code actually writes the wrong value into $tmp.aw and
> only runs the loop once, so the test fails because dry_run thinks the
> file size should be 0.
> 
> Also the cmds+=() line needs to insert its own -c or else you end up
> writing huge files to $here.  Ooops.
> 
> Will send a v2 once the brownpaperbag testing finishes.

Hi Darrick,

JFYI:) If you don't mind, as I haven't seen your v2, I'll try to merge this patchset
without this [09/11] at first. Then I can merge another patchset "[PATCH 0/3] generic/772:
split and fix" which bases on this patchset, and give them a test.

Thanks,
Zorro

> 
> --D
> 
> > > Otherwise this looks good:
> > > 
> > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > 
> > Thanks!
> > 
> > --D
> > 
> 

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Darrick J. Wong]

On Sat, Nov 01, 2025 at 05:34:18PM +0800, Zorro Lang wrote:
> On Fri, Oct 31, 2025 at 10:47:34AM -0700, Darrick J. Wong wrote:
> > On Tue, Oct 21, 2025 at 09:27:31PM -0700, Darrick J. Wong wrote:
> > > On Tue, Oct 21, 2025 at 09:20:55PM -0700, Christoph Hellwig wrote:
> > > > On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> > > > > As a result, one loop through the test takes almost 4 minutes.  The test
> > > > > loops 20 times, so it runs for 80 minutes(!!) which is a really long
> > > > > time.
> > > > 
> > > > Heh.  I'm glade none of my usual test setups even supports atomics I
> > > > guess :)
> > > 
> > > FWIW the failure was on a regular xfs, no hw atomics.  So in theory
> > > you're affected, but only if you pulled the 20 Oct next branch.
> > > 
> > > > > So the first thing we do is observe that the giant slow loop is being
> > > > > run as a single thread on an empty filesystem.  Most of the time the
> > > > > allocator generates a mostly physically contiguous file.  We could
> > > > > fallocate the whole file instead of fallocating one block every other
> > > > > time through the loop.  This halves the setup time.
> > > > > 
> > > > > Next, we can also stuff the remaining pwrite commands into a bash array
> > > > > and only invoke xfs_io once every 128x through the loop.  This amortizes
> > > > > the xfs_io startup time, which reduces the test loop runtime to about 20
> > > > > seconds.
> > > > 
> > > > Wouldn't it make sense to adopt src/punch-alternating.c to also be
> > > > able to create unwritten extents instead of holes for the punched
> > > > range and run all of this from a C program?
> > > 
> > > For the write sizes it comes up with I'm guessing that this test will
> > > almost always be poking the software fallbacks so it probably doesn't
> > > matter if the file is full of holes.
> > 
> > ...and now running this with 32k-fsblocks reveals that the
> > atomic_write_loop code actually writes the wrong value into $tmp.aw and
> > only runs the loop once, so the test fails because dry_run thinks the
> > file size should be 0.
> > 
> > Also the cmds+=() line needs to insert its own -c or else you end up
> > writing huge files to $here.  Ooops.
> > 
> > Will send a v2 once the brownpaperbag testing finishes.
> 
> Hi Darrick,
> 
> JFYI:) If you don't mind, as I haven't seen your v2, I'll try to merge this patchset
> without this [09/11] at first. Then I can merge another patchset "[PATCH 0/3] generic/772:
> split and fix" which bases on this patchset, and give them a test.

Sounds good!

The 778 changes are going to take a while longer because it keeps
breaking down in weird ways on 64k fsblock filesystems.

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > > Otherwise this looks good:
> > > > 
> > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > 
> > > Thanks!
> > > 
> > > --D
> > > 
> > 
> 

Re: [PATCH 09/11] generic/778: fix severe performance problems

[Zorro Lang]

On Sat, Nov 01, 2025 at 08:42:35AM -0700, Darrick J. Wong wrote:
> On Sat, Nov 01, 2025 at 05:34:18PM +0800, Zorro Lang wrote:
> > On Fri, Oct 31, 2025 at 10:47:34AM -0700, Darrick J. Wong wrote:
> > > On Tue, Oct 21, 2025 at 09:27:31PM -0700, Darrick J. Wong wrote:
> > > > On Tue, Oct 21, 2025 at 09:20:55PM -0700, Christoph Hellwig wrote:
> > > > > On Tue, Oct 21, 2025 at 11:41:33AM -0700, Darrick J. Wong wrote:
> > > > > > As a result, one loop through the test takes almost 4 minutes.  The test
> > > > > > loops 20 times, so it runs for 80 minutes(!!) which is a really long
> > > > > > time.
> > > > > 
> > > > > Heh.  I'm glade none of my usual test setups even supports atomics I
> > > > > guess :)
> > > > 
> > > > FWIW the failure was on a regular xfs, no hw atomics.  So in theory
> > > > you're affected, but only if you pulled the 20 Oct next branch.
> > > > 
> > > > > > So the first thing we do is observe that the giant slow loop is being
> > > > > > run as a single thread on an empty filesystem.  Most of the time the
> > > > > > allocator generates a mostly physically contiguous file.  We could
> > > > > > fallocate the whole file instead of fallocating one block every other
> > > > > > time through the loop.  This halves the setup time.
> > > > > > 
> > > > > > Next, we can also stuff the remaining pwrite commands into a bash array
> > > > > > and only invoke xfs_io once every 128x through the loop.  This amortizes
> > > > > > the xfs_io startup time, which reduces the test loop runtime to about 20
> > > > > > seconds.
> > > > > 
> > > > > Wouldn't it make sense to adopt src/punch-alternating.c to also be
> > > > > able to create unwritten extents instead of holes for the punched
> > > > > range and run all of this from a C program?
> > > > 
> > > > For the write sizes it comes up with I'm guessing that this test will
> > > > almost always be poking the software fallbacks so it probably doesn't
> > > > matter if the file is full of holes.
> > > 
> > > ...and now running this with 32k-fsblocks reveals that the
> > > atomic_write_loop code actually writes the wrong value into $tmp.aw and
> > > only runs the loop once, so the test fails because dry_run thinks the
> > > file size should be 0.
> > > 
> > > Also the cmds+=() line needs to insert its own -c or else you end up
> > > writing huge files to $here.  Ooops.
> > > 
> > > Will send a v2 once the brownpaperbag testing finishes.
> > 
> > Hi Darrick,
> > 
> > JFYI:) If you don't mind, as I haven't seen your v2, I'll try to merge this patchset
> > without this [09/11] at first. Then I can merge another patchset "[PATCH 0/3] generic/772:
> > split and fix" which bases on this patchset, and give them a test.
> 
> Sounds good!
> 
> The 778 changes are going to take a while longer because it keeps
> breaking down in weird ways on 64k fsblock filesystems.

Thanks for this detail, Darrick. I've pushed "ready to merge" patches into
patches-in-queue branch, feel free to check. This week we have many scattered
patches, I hope to take more time to give them enough test.

Thanks,
Zorro

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > 
> > > --D
> > > 
> > > > > Otherwise this looks good:
> > > > > 
> > > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > > 
> > > > Thanks!
> > > > 
> > > > --D
> > > > 
> > > 
> > 
> 

[PATCH 10/11] check: line up stdout columns

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The output columns don't really line up:

generic/768       [not run] xfs_io pwrite doesn't support -A
ext4/049        6s
generic/325       [not run] fuse4fs does not support metadata journaling on fuse.ext4

Make them line up properly.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)


diff --git a/check b/check
index ce7eacb7c45d9e..2d089d351380d2 100755
--- a/check
+++ b/check
@@ -669,6 +669,10 @@ _stash_test_status() {
 	esac
 }
 
+# Figure out the maximum test name length, e.g. "generic/1212" => 12
+max_test_namelen=$(ls "$SRC_DIR"/*/* | \
+	awk 'BEGIN {x = 0} /\/[0-9]*$/ {l = length($0) - 6; if (l > x) x = l;} END {print x}')
+
 # Can we run systemd scopes?
 HAVE_SYSTEMD_SCOPES=
 systemctl reset-failed "fstests-check" &>/dev/null
@@ -880,7 +884,9 @@ function run_section()
 					     "" &> /dev/null
 		fi
 
-		echo -n "$seqnum"
+		# Print test name and leave the cursor at a consistent column
+		# number for later reporting of test outcome.
+		printf "%-*s" "$max_test_namelen" "$seqnum"
 
 		if $showme; then
 			if _expunge_test $seqnum; then

Re: [PATCH 10/11] check: line up stdout columns

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 11/11] fsx: don't print messages when atomic writes are explicitly disabled

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The user knows they disabled atomic writes, no need to clutter up the
output with a message that also causes golden output failures if they
start fstests with FSX_AVOID='-a'.

Cc: <fstests@vger.kernel.org> # v2025.10.20
Fixes: e2bc78e9d340a6 ("ltp/fsx.c: Add atomic writes support to fsx")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 ltp/fsx.c |    1 -
 1 file changed, 1 deletion(-)


diff --git a/ltp/fsx.c b/ltp/fsx.c
index 0a035b37b6bd7e..626976dd4f9f27 100644
--- a/ltp/fsx.c
+++ b/ltp/fsx.c
@@ -3164,7 +3164,6 @@ main(int argc, char **argv)
 				 longopts, NULL)) != EOF)
 		switch (ch) {
 		case 'a':
-			prt("main(): Atomic writes disabled\n");
 			do_atomic_writes = 0;
 			break;
 		case 'b':

Re: [PATCH 11/11] fsx: don't print messages when atomic writes are explicitly disabled

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>