[PATCH 0/3] fstests: test that RWF_DONTCACHE I/O drops its folios

[PATCH 0/3] fstests: test that RWF_DONTCACHE I/O drops its folios

[Tal Zussman]

Add a test for RWF_DONTCACHE, as suggested by Christoph. [1]

generic/810 writes and reads a 1M file with and without RWF_DONTCACHE
and checks with mincore how much of it is still cached after each step.
On ext4 without the fix, the dontcache overwrite leaves the whole file
cached.

The first two patches make _require_xfs_io_command able to probe
"pwrite -U" and "pread -U".

[1] https://lore.kernel.org/linux-ext4/20261012-ext4-dontcache-complete-in-task-v1-1-6ac801557041@columbia.edu/

---
Tal Zussman (3):
      common/rc: probe pread options with a real read
      common/rc: pass -V 1 when probing pwrite and pread -U
      generic/810: check that RWF_DONTCACHE I/O drops its folios

 common/rc             | 14 +++++++++
 tests/generic/810     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/810.out | 11 +++++++
 3 files changed, 107 insertions(+)
---
base-commit: d07dca72d2455f4ba8c660ad4c008541ccccb707
change-id: 20261015-dontcache-test-be6094aa3f20

Best regards,
--  
Tal Zussman <tz2294@columbia.edu>

[PATCH 1/3] common/rc: probe pread options with a real read

[Tal Zussman]

_require_xfs_io_command has no pread case, so a pread option falls back
to a grep of the help text, and that only matches options that take an
argument. A bare -U is reported as unsupported even when xfs_io has it.
Add a pread case that does a real read with the option, like the pwrite
case does.

Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 common/rc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/common/rc b/common/rc
index 1eccd60c..945c9e42 100644
--- a/common/rc
+++ b/common/rc
@@ -3120,6 +3120,11 @@ _require_xfs_io_command()
 		        "pwrite $pwrite_opts $param 0 $write_size" $testfile 2>&1`
 		param_checked="$pwrite_opts $param"
 		;;
+	"pread")
+		testio=`$XFS_IO_PROG -f -c "pwrite 0 4k" \
+			-c "pread $param 0 4k" $testfile 2>&1`
+		param_checked="$param"
+		;;
 	"scrub"|"repair")
 		test -z "$param" && param="probe"
 		testio=`$XFS_IO_PROG -x -c "$command $param" $TEST_DIR 2>&1`

-- 
2.39.5

Re: [PATCH 1/3] common/rc: probe pread options with a real read

[Brian Foster]

On Thu, Oct 15, 2026 at 04:54:02PM -0400, Tal Zussman wrote:
> _require_xfs_io_command has no pread case, so a pread option falls back
> to a grep of the help text, and that only matches options that take an
> argument. A bare -U is reported as unsupported even when xfs_io has it.
> Add a pread case that does a real read with the option, like the pwrite
> case does.
> 
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  common/rc | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/common/rc b/common/rc
> index 1eccd60c..945c9e42 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -3120,6 +3120,11 @@ _require_xfs_io_command()
>  		        "pwrite $pwrite_opts $param 0 $write_size" $testfile 2>&1`
>  		param_checked="$pwrite_opts $param"
>  		;;
> +	"pread")
> +		testio=`$XFS_IO_PROG -f -c "pwrite 0 4k" \
> +			-c "pread $param 0 4k" $testfile 2>&1`
> +		param_checked="$param"
> +		;;
>  	"scrub"|"repair")
>  		test -z "$param" && param="probe"
>  		testio=`$XFS_IO_PROG -x -c "$command $param" $TEST_DIR 2>&1`
> 
> -- 
> 2.39.5
> 
> 

Re: [PATCH 1/3] common/rc: probe pread options with a real read

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 2/3] common/rc: pass -V 1 when probing pwrite and pread -U

[Tal Zussman]

xfs_io refuses pwritev2() and preadv2() flags unless -V is given, so
probing -U without it always fails. Pass -V 1 for -U in both the pwrite
and pread cases.

Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 common/rc | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/common/rc b/common/rc
index 945c9e42..a0239034 100644
--- a/common/rc
+++ b/common/rc
@@ -3105,6 +3105,10 @@ _require_xfs_io_command()
 			opts+=" -d"
 			pwrite_opts+="-V 1 -b $write_size"
 		fi
+		# -U (RWF_DONTCACHE) only works with vectored I/O writes
+		if [ "$param" == "-U" ]; then
+			pwrite_opts+="-V 1 -b $write_size"
+		fi
 		if [ "$param" == "-A" ]; then
 			opts+=" -d"
 			# try to write the minimum supported atomic write size
@@ -3121,9 +3125,14 @@ _require_xfs_io_command()
 		param_checked="$pwrite_opts $param"
 		;;
 	"pread")
+		# -U (RWF_DONTCACHE) only works with vectored I/O reads
+		local pread_opts=" "
+		if [ "$param" == "-U" ]; then
+			pread_opts+="-V 1 -b 4k"
+		fi
 		testio=`$XFS_IO_PROG -f -c "pwrite 0 4k" \
-			-c "pread $param 0 4k" $testfile 2>&1`
-		param_checked="$param"
+			-c "pread $pread_opts $param 0 4k" $testfile 2>&1`
+		param_checked="$pread_opts $param"
 		;;
 	"scrub"|"repair")
 		test -z "$param" && param="probe"

-- 
2.39.5

Re: [PATCH 2/3] common/rc: pass -V 1 when probing pwrite and pread -U

[Brian Foster]

On Thu, Oct 15, 2026 at 04:54:03PM -0400, Tal Zussman wrote:
> xfs_io refuses pwritev2() and preadv2() flags unless -V is given, so
> probing -U without it always fails. Pass -V 1 for -U in both the pwrite
> and pread cases.
> 
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---
>  common/rc | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index 945c9e42..a0239034 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -3105,6 +3105,10 @@ _require_xfs_io_command()
>  			opts+=" -d"
>  			pwrite_opts+="-V 1 -b $write_size"
>  		fi
> +		# -U (RWF_DONTCACHE) only works with vectored I/O writes
> +		if [ "$param" == "-U" ]; then
> +			pwrite_opts+="-V 1 -b $write_size"
> +		fi
>  		if [ "$param" == "-A" ]; then
>  			opts+=" -d"
>  			# try to write the minimum supported atomic write size
> @@ -3121,9 +3125,14 @@ _require_xfs_io_command()
>  		param_checked="$pwrite_opts $param"
>  		;;
>  	"pread")
> +		# -U (RWF_DONTCACHE) only works with vectored I/O reads
> +		local pread_opts=" "
> +		if [ "$param" == "-U" ]; then
> +			pread_opts+="-V 1 -b 4k"
> +		fi
>  		testio=`$XFS_IO_PROG -f -c "pwrite 0 4k" \
> -			-c "pread $param 0 4k" $testfile 2>&1`
> -		param_checked="$param"
> +			-c "pread $pread_opts $param 0 4k" $testfile 2>&1`
> +		param_checked="$pread_opts $param"
>  		;;

I find it a little odd that we auto add command related options here
even though the test still needs to know to use those options for the
commands to work correctly. But I guess there's precedent for it, so oh
well:

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  	"scrub"|"repair")
>  		test -z "$param" && param="probe"
> 
> -- 
> 2.39.5
> 
> 

Re: [PATCH 2/3] common/rc: pass -V 1 when probing pwrite and pread -U

[Christoph Hellwig]

On Thu, Oct 22, 2026 at 09:25:34AM -0400, Brian Foster wrote:
> > -			-c "pread $param 0 4k" $testfile 2>&1`
> > -		param_checked="$param"
> > +			-c "pread $pread_opts $param 0 4k" $testfile 2>&1`
> > +		param_checked="$pread_opts $param"
> >  		;;
> 
> I find it a little odd that we auto add command related options here
> even though the test still needs to know to use those options for the
> commands to work correctly. But I guess there's precedent for it, so oh
> well:

Same.  Given how annoying the required -V 1 is, maybe this defaulting
should go into xfs_io itself so that -U, -D and co can be used without
manually specifying -V ?

[PATCH 3/3] generic/810: check that RWF_DONTCACHE I/O drops its folios

[Tal Zussman]

A dontcache write drops its folios from the page cache once writeback
completes, and a dontcache read drops them once the read completes.

Test this by writing and reading a 1M file with and without
RWF_DONTCACHE and checking with mincore how much of it is cached
afterwards: all of it for plain I/O, none of it for dontcache I/O.

The overwrite is a separate case because the drop is skipped when
writeback completes in interrupt context, which is what happens on ext4
without commit "ext4: complete dropbehind writeback in task context"
[1].

Link: https://lore.kernel.org/linux-ext4/20261012-ext4-dontcache-complete-in-task-v1-1-6ac801557041@columbia.edu/ [1]
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 tests/generic/810     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/810.out | 11 +++++++
 2 files changed, 93 insertions(+)

diff --git a/tests/generic/810 b/tests/generic/810
new file mode 100755
index 00000000..9022c26a
--- /dev/null
+++ b/tests/generic/810
@@ -0,0 +1,82 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Tal Zussman.  All Rights Reserved.
+#
+# FS QA Test No. 810
+#
+# Check that RWF_DONTCACHE I/O drops its folios from the page cache: a
+# dontcache write once its writeback completes, and a dontcache read once the
+# read completes. Plain buffered I/O on the same file has to leave them cached.
+#
+. ./common/preamble
+_begin_fstest auto quick rw
+
+_require_scratch
+_require_xfs_io_command "pwrite" "-U"
+_require_xfs_io_command "pread" "-U"
+_require_xfs_io_command "mincore"
+_exclude_scratch_mount_option "dax"
+
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount
+
+size=$((1024 * 1024))
+file=$SCRATCH_MNT/file
+pagesize=$(_get_page_size)
+
+# Count the bytes of the file that are in the page cache.
+cached_bytes()
+{
+	local pages
+
+	pages=$($XFS_IO_PROG -c "mmap -r 0 $size" -c "mincore" $file | \
+		awk '{ n += $4 } END { print n + 0 }')
+	echo $((pages * pagesize))
+}
+
+report_cached()
+{
+	echo "$(cached_bytes) of $size bytes cached"
+}
+
+# Dontcache writeback drops a folio right after it clears the writeback flag,
+# so a check immediately after fsync returns can run a moment too early. Give
+# the drop a little time before reporting.
+report_cached_after_writeback()
+{
+	local i
+
+	for ((i = 0; i < 50; i++)); do
+		[ "$(cached_bytes)" -eq 0 ] && break
+		sleep 0.1
+	done
+	report_cached
+}
+
+echo "plain write"
+$XFS_IO_PROG -f -c "pwrite -q 0 $size" -c fsync $file
+report_cached
+
+echo "dontcache write to a new file"
+rm -f $file
+$XFS_IO_PROG -f -c "pwrite -q -U -V 1 -b $size 0 $size" -c fsync $file
+report_cached_after_writeback
+
+# Start each of the remaining cases from an empty page cache.
+echo "dontcache overwrite"
+_scratch_cycle_mount
+$XFS_IO_PROG -c "pwrite -q -U -V 1 -b $size 0 $size" -c fsync $file
+report_cached_after_writeback
+
+echo "dontcache read"
+_scratch_cycle_mount
+$XFS_IO_PROG -c "pread -q -U -V 1 -b $size 0 $size" $file
+report_cached
+
+echo "plain read"
+_scratch_cycle_mount
+$XFS_IO_PROG -c "pread -q 0 $size" $file
+report_cached
+
+_scratch_unmount
+_exit 0
diff --git a/tests/generic/810.out b/tests/generic/810.out
new file mode 100644
index 00000000..cb6c3f2a
--- /dev/null
+++ b/tests/generic/810.out
@@ -0,0 +1,11 @@
+QA output created by 810
+plain write
+1048576 of 1048576 bytes cached
+dontcache write to a new file
+0 of 1048576 bytes cached
+dontcache overwrite
+0 of 1048576 bytes cached
+dontcache read
+0 of 1048576 bytes cached
+plain read
+1048576 of 1048576 bytes cached

-- 
2.39.5

Re: [PATCH 3/3] generic/810: check that RWF_DONTCACHE I/O drops its folios

[Brian Foster]

On Thu, Oct 15, 2026 at 04:54:04PM -0400, Tal Zussman wrote:
> A dontcache write drops its folios from the page cache once writeback
> completes, and a dontcache read drops them once the read completes.
> 
> Test this by writing and reading a 1M file with and without
> RWF_DONTCACHE and checking with mincore how much of it is cached
> afterwards: all of it for plain I/O, none of it for dontcache I/O.
> 
> The overwrite is a separate case because the drop is skipped when
> writeback completes in interrupt context, which is what happens on ext4
> without commit "ext4: complete dropbehind writeback in task context"
> [1].
> 
> Link: https://lore.kernel.org/linux-ext4/20261012-ext4-dontcache-complete-in-task-v1-1-6ac801557041@columbia.edu/ [1]
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---
>  tests/generic/810     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/810.out | 11 +++++++
>  2 files changed, 93 insertions(+)
> 
> diff --git a/tests/generic/810 b/tests/generic/810
> new file mode 100755
> index 00000000..9022c26a
> --- /dev/null
> +++ b/tests/generic/810
> @@ -0,0 +1,82 @@
...
> +# Count the bytes of the file that are in the page cache.
> +cached_bytes()
> +{
> +	local pages
> +
> +	pages=$($XFS_IO_PROG -c "mmap -r 0 $size" -c "mincore" $file | \
> +		awk '{ n += $4 } END { print n + 0 }')
> +	echo $((pages * pagesize))
> +}
> +
> +report_cached()
> +{
> +	echo "$(cached_bytes) of $size bytes cached"
> +}
> +
> +# Dontcache writeback drops a folio right after it clears the writeback flag,
> +# so a check immediately after fsync returns can run a moment too early. Give
> +# the drop a little time before reporting.
> +report_cached_after_writeback()
> +{
> +	local i
> +
> +	for ((i = 0; i < 50; i++)); do
> +		[ "$(cached_bytes)" -eq 0 ] && break
> +		sleep 0.1
> +	done
> +	report_cached
> +}

So if I understand this right, the fsync completion isn't technically
sufficient to prevent the don't cache drops from racing with the mincore
check, right?

If so, then we give the system up to 5 seconds or so to report the don't
cache results. That seems reasonable to me, but I find the interface a
little unclear since it seems not universally used for write(back)
cases.

IOW (and I realize this is a wonky example), suppose don't cache breaks
in some way where it starts to apply to cases where it shouldn't.. do we
want to try and test for that?

This is kind of a nit and I'm not opposed to the test as is, so FWIW:

Reviewed-by: Brian Foster <bfoster@redhat.com>

But I do wonder whether this might be a little more robust if we just
gave the writeback case some unconditional settle time before checking
the result. The caveat I suppose is reclaim could still technically
occur at any time and cause false results either way, so *shrug*.

An idea that comes to mind could be to perhaps combine these two
report_cached helpers into one with an optional delay parameter. If not
specified then return the immediate result. If a delay is specified then
still check the initial result, but wait up to $delay seconds for the
result to change. Then maybe the don't cache case could use a little
more aggressive of a delay, but we could still have a short one for the
normal cached case(s). Hm?

Brian

> +
> +echo "plain write"
> +$XFS_IO_PROG -f -c "pwrite -q 0 $size" -c fsync $file
> +report_cached
> +
> +echo "dontcache write to a new file"
> +rm -f $file
> +$XFS_IO_PROG -f -c "pwrite -q -U -V 1 -b $size 0 $size" -c fsync $file
> +report_cached_after_writeback
> +
> +# Start each of the remaining cases from an empty page cache.
> +echo "dontcache overwrite"
> +_scratch_cycle_mount
> +$XFS_IO_PROG -c "pwrite -q -U -V 1 -b $size 0 $size" -c fsync $file
> +report_cached_after_writeback
> +
> +echo "dontcache read"
> +_scratch_cycle_mount
> +$XFS_IO_PROG -c "pread -q -U -V 1 -b $size 0 $size" $file
> +report_cached
> +
> +echo "plain read"
> +_scratch_cycle_mount
> +$XFS_IO_PROG -c "pread -q 0 $size" $file
> +report_cached
> +
> +_scratch_unmount
> +_exit 0
> diff --git a/tests/generic/810.out b/tests/generic/810.out
> new file mode 100644
> index 00000000..cb6c3f2a
> --- /dev/null
> +++ b/tests/generic/810.out
> @@ -0,0 +1,11 @@
> +QA output created by 810
> +plain write
> +1048576 of 1048576 bytes cached
> +dontcache write to a new file
> +0 of 1048576 bytes cached
> +dontcache overwrite
> +0 of 1048576 bytes cached
> +dontcache read
> +0 of 1048576 bytes cached
> +plain read
> +1048576 of 1048576 bytes cached
> 
> -- 
> 2.39.5
> 
> 

Re: [PATCH 3/3] generic/810: check that RWF_DONTCACHE I/O drops its folios

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>