[PATCH 0/2] fstests: expect and test truthful FIDEDUPERANGE progress

[PATCH 0/2] fstests: expect and test truthful FIDEDUPERANGE progress

[Matthias Goergens]

This series pairs with the kernel series "vfs: report truthful
FIDEDUPERANGE progress safely", posted separately to linux-fsdevel:

  https://lore.kernel.org/linux-fsdevel/20260805071414.3414870-1-matthias.goergens@gmail.com/

Patch 1 corrects generic/517's expected output: an unaligned 131172-byte
request is shortened to 131072 bytes, and the 100-byte remainder that
xfs_io retries now fails per destination with -EINVAL instead of
reporting success. These expectations encode the paired kernel change;
on an unpatched kernel the test fails on exactly these corrected lines,
as it did when the original kernel fix was briefly merged in 2022.

Patch 2 adds generic/806, a raw FIDEDUPERANGE ioctl test covering the
cases the retrying xfs_io interface cannot express precisely: an
explicit zero-length request succeeds with bytes_deduped=0, and mixed
multi-destination requests report the shortened count for an interior
destination and the full count for a destination ending at EOF. The new
number follows for-next's generic/798 and the publicly claimed numbers
through 805.

generic/517 and generic/806 both pass on Btrfs and XFS against the
paired kernel.

Matthias Goergens (2):
  generic/517: expect truthful dedupe progress
  generic: test FIDEDUPERANGE result reporting

 src/Makefile          |   2 +-
 src/fideduperange.c   | 138 ++++++++++++++++++++++++++++++++++++++++++
 tests/generic/517     |   6 ++
 tests/generic/517.out |   6 +-
 tests/generic/806     |  83 +++++++++++++++++++++++++
 tests/generic/806.out |   4 ++
 6 files changed, 235 insertions(+), 4 deletions(-)
 create mode 100644 src/fideduperange.c
 create mode 100755 tests/generic/806
 create mode 100644 tests/generic/806.out

-- 
2.55.0

[PATCH 1/2] generic/517: expect truthful dedupe progress

[Matthias Goergens]

The VFS currently reports the requested FIDEDUPERANGE length even when it
shortens the operation to avoid sharing a partial EOF block into the middle
of the destination.  Once the kernel reports its actual progress, the first
dedupe in this test reports 131072 of 131172 bytes.  xfs_io then retries the
100-byte remainder, which cannot make progress and returns -EINVAL.  The
separate 100-byte request returns the same error.

Update the golden output for those truthful results.  This is the output
change that caused commit 4a57a8400075 ("vf/remap: return the amount of
bytes actually deduplicated") to be reverted.  The paired kernel series that
determines the new output is linked from the cover letter.

The updated test passes on btrfs and XFS with the paired kernel changes.

Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
---
 tests/generic/517     | 6 ++++++
 tests/generic/517.out | 6 +++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/tests/generic/517 b/tests/generic/517
index 3f7027a8..ab5d17e7 100755
--- a/tests/generic/517
+++ b/tests/generic/517
@@ -11,6 +11,12 @@
 . ./common/preamble
 _begin_fstest auto quick dedupe clone
 
+# The dedupe amounts this test records are only correct once the kernel
+# reports the number of bytes it actually deduplicated rather than the number
+# requested. Both dedupe calls below target a range that does not reach the
+# destination's EOF, so generic_remap_check_len() shortens them under
+# REMAP_FILE_CAN_SHORTEN.
+
 # Import common functions.
 . ./common/filter
 . ./common/reflink
diff --git a/tests/generic/517.out b/tests/generic/517.out
index b9b63207..6ba753d8 100644
--- a/tests/generic/517.out
+++ b/tests/generic/517.out
@@ -13,7 +13,8 @@ File content before first deduplication:
 *
 0786528 ae ae ae ae
 0786532
-deduped 131172/131172 bytes at offset 65536
+XFS_IOC_FILE_EXTENT_SAME: Invalid argument
+deduped 131072/131172 bytes at offset 65536
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 File content after first deduplication and before unmounting:
 0000000 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
@@ -33,8 +34,7 @@ File content after first unmount:
 0786532
 wrote 100/100 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-deduped 100/100 bytes at offset 655360
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XFS_IOC_FILE_EXTENT_SAME: Invalid argument
 File content after second deduplication:
 0000000 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
 *
-- 
2.55.0

[PATCH 2/2] generic: test FIDEDUPERANGE result reporting

[Matthias Goergens]

xfs_io retries shortened dedupe operations, so its output cannot verify one
raw multi-destination ioctl or distinguish an explicit zero-length request
from a nonzero request shortened to zero.

Add a small FIDEDUPERANGE helper that reports each destination's raw
bytes_deduped and status fields.  Use it in generic/806 to check an explicit
zero-length success, a request that is shortened for an interior destination
but remains full-length at EOF, and a sub-block request that returns -EINVAL
for the interior destination while succeeding at EOF.

Compute the request lengths from the filesystem block size so the expected
semantics do not assume 4K blocks.  The paired kernel series is linked from
the cover letter.

The test passes on btrfs and XFS with the paired kernel changes.

Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
---
 src/Makefile          |   2 +-
 src/fideduperange.c   | 138 ++++++++++++++++++++++++++++++++++++++++++
 tests/generic/806     |  83 +++++++++++++++++++++++++
 tests/generic/806.out |   4 ++
 4 files changed, 226 insertions(+), 1 deletion(-)
 create mode 100644 src/fideduperange.c
 create mode 100755 tests/generic/806
 create mode 100644 tests/generic/806.out

diff --git a/src/Makefile b/src/Makefile
index 76cf50c3..e686fe27 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -25,7 +25,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \
 
 LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
-	locktest unwritten_mmap bulkstat_unlink_test deduperace \
+	locktest unwritten_mmap bulkstat_unlink_test deduperace fideduperange \
 	bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
 	stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
 	seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \
diff --git a/src/fideduperange.c b/src/fideduperange.c
new file mode 100644
index 00000000..24813de8
--- /dev/null
+++ b/src/fideduperange.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Issue a raw FIDEDUPERANGE ioctl and report each destination's result.
+ */
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <linux/fs.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+
+#ifndef FIDEDUPERANGE
+/* These definitions must match the kernel UAPI. */
+#define FILE_DEDUPE_RANGE_SAME		0
+#define FILE_DEDUPE_RANGE_DIFFERS	1
+
+struct file_dedupe_range_info {
+	__s64 dest_fd;
+	__u64 dest_offset;
+	__u64 bytes_deduped;
+	__s32 status;
+	__u32 reserved;
+};
+
+struct file_dedupe_range {
+	__u64 src_offset;
+	__u64 src_length;
+	__u16 dest_count;
+	__u16 reserved1;
+	__u32 reserved2;
+	struct file_dedupe_range_info info[0];
+};
+
+#define FIDEDUPERANGE _IOWR(0x94, 54, struct file_dedupe_range)
+#endif
+
+static void
+usage(const char *progname)
+{
+	fprintf(stderr,
+		"Usage: %s src src_offset length dest dest_offset "
+		"[dest dest_offset ...]\n",
+		progname);
+	exit(EXIT_FAILURE);
+}
+
+static uint64_t
+parse_u64(const char *arg, const char *name)
+{
+	char *end;
+	unsigned long long value;
+
+	errno = 0;
+	value = strtoull(arg, &end, 0);
+	if (errno || end == arg || *end != '\0') {
+		fprintf(stderr, "Invalid %s: %s\n", name, arg);
+		exit(EXIT_FAILURE);
+	}
+
+	return value;
+}
+
+int
+main(int argc, char **argv)
+{
+	struct file_dedupe_range *range;
+	size_t range_size;
+	unsigned int dest_count;
+	unsigned int i;
+	int src_fd;
+	int ret;
+
+	if (argc < 6 || (argc - 4) % 2 != 0)
+		usage(argv[0]);
+
+	dest_count = (argc - 4) / 2;
+	if (dest_count > UINT16_MAX) {
+		fprintf(stderr, "Too many destinations: %u\n", dest_count);
+		return EXIT_FAILURE;
+	}
+
+	range_size = sizeof(*range) +
+			dest_count * sizeof(struct file_dedupe_range_info);
+	range = calloc(1, range_size);
+	if (!range) {
+		perror("calloc");
+		return EXIT_FAILURE;
+	}
+
+	src_fd = open(argv[1], O_RDONLY);
+	if (src_fd < 0) {
+		perror(argv[1]);
+		free(range);
+		return EXIT_FAILURE;
+	}
+
+	range->src_offset = parse_u64(argv[2], "source offset");
+	range->src_length = parse_u64(argv[3], "length");
+	range->dest_count = dest_count;
+
+	for (i = 0; i < dest_count; i++) {
+		const char *dest_path = argv[4 + i * 2];
+
+		range->info[i].dest_fd = open(dest_path, O_RDWR);
+		if (range->info[i].dest_fd < 0) {
+			perror(dest_path);
+			ret = EXIT_FAILURE;
+			goto close_destinations;
+		}
+		range->info[i].dest_offset =
+			parse_u64(argv[5 + i * 2], "destination offset");
+	}
+
+	if (ioctl(src_fd, FIDEDUPERANGE, range) < 0) {
+		perror("FIDEDUPERANGE");
+		ret = EXIT_FAILURE;
+		goto close_destinations;
+	}
+
+	for (i = 0; i < dest_count; i++)
+		printf("destination %u: bytes=%" PRIu64 " status=%d\n", i,
+		       (uint64_t)range->info[i].bytes_deduped,
+		       range->info[i].status);
+	ret = EXIT_SUCCESS;
+
+close_destinations:
+	while (i > 0) {
+		i--;
+		close(range->info[i].dest_fd);
+	}
+	close(src_fd);
+	free(range);
+	return ret;
+}
diff --git a/tests/generic/806 b/tests/generic/806
new file mode 100755
index 00000000..98254b79
--- /dev/null
+++ b/tests/generic/806
@@ -0,0 +1,83 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Matthias Goergens. All Rights Reserved.
+#
+# FS QA Test No. 806
+#
+# Check FIDEDUPERANGE result reporting for zero-length requests and for a
+# multi-destination request where the VFS shortens only some destinations.
+#
+. ./common/preamble
+_begin_fstest auto quick dedupe clone
+
+. ./common/reflink
+
+_require_scratch_dedupe
+_require_test_program "fideduperange"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+block_size=$(_get_block_size "$SCRATCH_MNT")
+short_length=$((2 * block_size))
+full_length=$((short_length + 100))
+
+src=$SCRATCH_MNT/src
+src_short=$SCRATCH_MNT/src-short
+dest_interior=$SCRATCH_MNT/dest-interior
+dest_eof=$SCRATCH_MNT/dest-eof
+dest_short_eof=$SCRATCH_MNT/dest-short-eof
+
+$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 $full_length" "$src" \
+	>>$seqres.full 2>&1
+$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 100" "$src_short" \
+	>>$seqres.full 2>&1
+$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 $((full_length + block_size))" \
+	"$dest_interior" >>$seqres.full 2>&1
+$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 $full_length" "$dest_eof" \
+	>>$seqres.full 2>&1
+$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 100" "$dest_short_eof" \
+	>>$seqres.full 2>&1
+
+check_results()
+{
+	local description="$1"
+	local expected="$2"
+	shift 2
+	local result
+	local ret
+
+	result=$($here/src/fideduperange "$@" 2>&1)
+	ret=$?
+	{
+		echo "$description"
+		echo "$result"
+	} >>$seqres.full
+
+	if [ $ret -ne 0 ]; then
+		_fail "$description: ioctl helper failed"
+	fi
+	if [ "$result" != "$expected" ]; then
+		_fail "$description: unexpected ioctl result"
+	fi
+
+	echo "$description"
+}
+
+expected="destination 0: bytes=0 status=0"
+check_results "zero length: 0 bytes, SAME" "$expected" \
+	"$src" 0 0 "$dest_interior" 0
+
+expected="destination 0: bytes=$short_length status=0
+destination 1: bytes=$full_length status=0"
+check_results "mixed partial request: shortened, full" "$expected" \
+	"$src" 0 "$full_length" "$dest_interior" 0 "$dest_eof" 0
+
+expected="destination 0: bytes=0 status=-22
+destination 1: bytes=100 status=0"
+check_results "mixed sub-block request: EINVAL, full" "$expected" \
+	"$src_short" 0 100 "$dest_interior" 0 "$dest_short_eof" 0
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/806.out b/tests/generic/806.out
new file mode 100644
index 00000000..d7226e34
--- /dev/null
+++ b/tests/generic/806.out
@@ -0,0 +1,4 @@
+QA output created by 806
+zero length: 0 bytes, SAME
+mixed partial request: shortened, full
+mixed sub-block request: EINVAL, full
-- 
2.55.0

Re: [PATCH 1/2] generic/517: expect truthful dedupe progress

[Zorro Lang]

On Wed, Aug 05, 2026 at 03:18:38PM +0800, Matthias Goergens wrote:
> The VFS currently reports the requested FIDEDUPERANGE length even when it
> shortens the operation to avoid sharing a partial EOF block into the middle
> of the destination.  Once the kernel reports its actual progress, the first
> dedupe in this test reports 131072 of 131172 bytes.  xfs_io then retries the
> 100-byte remainder, which cannot make progress and returns -EINVAL.  The
> separate 100-byte request returns the same error.
> 
> Update the golden output for those truthful results.  This is the output
> change that caused commit 4a57a8400075 ("vf/remap: return the amount of
> bytes actually deduplicated") to be reverted.  The paired kernel series that
> determines the new output is linked from the cover letter.
> 
> The updated test passes on btrfs and XFS with the paired kernel changes.

To avoid panic in downstream testing caused by sudden breaking changes in
expected behavior, we'd better to reference the commit or patch that
introduced the change.

Rather than using _fixed_by_kernel_commit helper, _wants_kernel_commit
might be more appropriate here.

CC'ing Filipe, the original author of this test case, for further review.

Thanks,
Zorro

> 
> Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
> ---
>  tests/generic/517     | 6 ++++++
>  tests/generic/517.out | 6 +++---
>  2 files changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/generic/517 b/tests/generic/517
> index 3f7027a8..ab5d17e7 100755
> --- a/tests/generic/517
> +++ b/tests/generic/517
> @@ -11,6 +11,12 @@
>  . ./common/preamble
>  _begin_fstest auto quick dedupe clone
>  
> +# The dedupe amounts this test records are only correct once the kernel
> +# reports the number of bytes it actually deduplicated rather than the number
> +# requested. Both dedupe calls below target a range that does not reach the
> +# destination's EOF, so generic_remap_check_len() shortens them under
> +# REMAP_FILE_CAN_SHORTEN.
> +
>  # Import common functions.
>  . ./common/filter
>  . ./common/reflink
> diff --git a/tests/generic/517.out b/tests/generic/517.out
> index b9b63207..6ba753d8 100644
> --- a/tests/generic/517.out
> +++ b/tests/generic/517.out
> @@ -13,7 +13,8 @@ File content before first deduplication:
>  *
>  0786528 ae ae ae ae
>  0786532
> -deduped 131172/131172 bytes at offset 65536
> +XFS_IOC_FILE_EXTENT_SAME: Invalid argument
> +deduped 131072/131172 bytes at offset 65536
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  File content after first deduplication and before unmounting:
>  0000000 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> @@ -33,8 +34,7 @@ File content after first unmount:
>  0786532
>  wrote 100/100 bytes at offset 0
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -deduped 100/100 bytes at offset 655360
> -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XFS_IOC_FILE_EXTENT_SAME: Invalid argument
>  File content after second deduplication:
>  0000000 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
>  *
> -- 
> 2.55.0
>