[PATCH] btrfs: test an incremental send scenario with hole punching

[PATCH] btrfs: test an incremental send scenario with hole punching

[fdmanana]

From: Filipe Manana <fdmanana@suse.com>

Test an incremental send scenario where we punched a hole into a tail
section of a file extent item that is the last item in a leaf and the
next leaf in the subvolume tree was not changed (remains shared between
the parent and send snapshots).

This currently fails and is fixed by the following kernel patch:

  btrfs: send: fix missing hole punching during incremental send

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/362     | 84 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/362.out |  7 ++++
 2 files changed, 91 insertions(+)
 create mode 100755 tests/btrfs/362
 create mode 100644 tests/btrfs/362.out

diff --git a/tests/btrfs/362 b/tests/btrfs/362
new file mode 100755
index 00000000..aca56540
--- /dev/null
+++ b/tests/btrfs/362
@@ -0,0 +1,84 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 SUSE S.A.  All Rights Reserved.
+#
+# FS QA Test 362
+#
+# Test an incremental send scenario where we punched a hole into a tail section
+# of a file extent item that is the last item in a leaf and the next leaf in the
+# subvolume tree was not changed (remains shared between the parent and send
+# snapshots).
+#
+. ./common/preamble
+_begin_fstest auto quick send
+
+_cleanup()
+{
+	cd /
+	rm -fr $tmp.*
+	rm -fr $send_files_dir
+}
+
+. ./common/filter
+
+_require_test
+_require_scratch
+_require_btrfs_support_sectorsize 4096
+_require_xfs_io_command "fpunch"
+
+_fixed_by_kernel_commit xxxxxxxxxx \
+	"btrfs: send: fix missing hole punching during incremental send"
+
+send_files_dir=$TEST_DIR/btrfs-test-$seq
+
+rm -fr $send_files_dir
+mkdir $send_files_dir
+
+first_stream="$send_files_dir/1.send"
+second_stream="$send_files_dir/2.send"
+
+# We need a very specific leaf layout where we punch a hole in the last file
+# extent item of a leaf and the next leaf was not modified in the send snapshot,
+# so we set a fixed size of 16K.
+_scratch_mkfs -n 16K >> $seqres.full 2>&1 || _fail "first mkfs failed"
+_scratch_mount
+
+touch $SCRATCH_MNT/foo
+# 205 extents of 64K each with a 64K hole between them.
+for ((i = 0; i < 410; i += 2)); do
+	offset=$((i * 64 * 1024))
+	$XFS_IO_PROG -c "pwrite -b 64K -S 0xab $offset 64K" \
+		$SCRATCH_MNT/foo > /dev/null
+done
+
+_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
+
+# Punch a hole in the last 16K of the last file extent item of the first leaf,
+# which starts at file offset 12845056, so 12845056 + 64K - 16K = 12894208.
+# This does not cause any change in the second leaf of the fs tree, so it will
+# be shared between the snapshots.
+$XFS_IO_PROG -c "fpunch 12894208 16K" $SCRATCH_MNT/foo
+
+_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
+
+echo "File digest in the original fs: $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
+
+_btrfs send -f $first_stream $SCRATCH_MNT/snap1
+_btrfs send -f $second_stream -p $SCRATCH_MNT/snap1 $SCRATCH_MNT/snap2
+
+# Create a new fs and apply both send streams.
+_scratch_unmount
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
+_scratch_mount
+
+_btrfs receive -f $first_stream $SCRATCH_MNT
+_btrfs receive -f $second_stream $SCRATCH_MNT
+
+echo "File digest in the new fs:      $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
+
+# Check the region we punched a hole is full of zeroes.
+echo "Hole data:"
+od -A d -t x1 -j 12894208 -N $((16 * 1024)) $SCRATCH_MNT/snap2/foo
+
+# success, all done
+_exit 0
diff --git a/tests/btrfs/362.out b/tests/btrfs/362.out
new file mode 100644
index 00000000..789abb53
--- /dev/null
+++ b/tests/btrfs/362.out
@@ -0,0 +1,7 @@
+QA output created by 362
+File digest in the original fs: a5a06694d9f7fc3245afb2f391636e44
+File digest in the new fs:      a5a06694d9f7fc3245afb2f391636e44
+Hole data:
+12894208 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+*
+12910592
-- 
2.53.0

Re: [PATCH] btrfs: test an incremental send scenario with hole punching

[Johannes Thumshirn]

Looks good to me,
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>

Re: [PATCH] btrfs: test an incremental send scenario with hole punching

[Zorro Lang]

On Wed, Oct 21, 2026 at 09:17:25PM +0100, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Test an incremental send scenario where we punched a hole into a tail
> section of a file extent item that is the last item in a leaf and the
> next leaf in the subvolume tree was not changed (remains shared between
> the parent and send snapshots).
> 
> This currently fails and is fixed by the following kernel patch:
> 
>   btrfs: send: fix missing hole punching during incremental send
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
> ---
>  tests/btrfs/362     | 84 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/362.out |  7 ++++
>  2 files changed, 91 insertions(+)
>  create mode 100755 tests/btrfs/362
>  create mode 100644 tests/btrfs/362.out
> 
> diff --git a/tests/btrfs/362 b/tests/btrfs/362
> new file mode 100755
> index 00000000..aca56540
> --- /dev/null
> +++ b/tests/btrfs/362
> @@ -0,0 +1,84 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2026 SUSE S.A.  All Rights Reserved.
> +#
> +# FS QA Test 362
> +#
> +# Test an incremental send scenario where we punched a hole into a tail section
> +# of a file extent item that is the last item in a leaf and the next leaf in the
> +# subvolume tree was not changed (remains shared between the parent and send
> +# snapshots).
> +#
> +. ./common/preamble
> +_begin_fstest auto quick send
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -fr $tmp.*
> +	rm -fr $send_files_dir
> +}
> +
> +. ./common/filter
> +
> +_require_test
> +_require_scratch
> +_require_btrfs_support_sectorsize 4096
> +_require_xfs_io_command "fpunch"
> +
> +_fixed_by_kernel_commit xxxxxxxxxx \
> +	"btrfs: send: fix missing hole punching during incremental send"
> +
> +send_files_dir=$TEST_DIR/btrfs-test-$seq
> +
> +rm -fr $send_files_dir
> +mkdir $send_files_dir
> +
> +first_stream="$send_files_dir/1.send"
> +second_stream="$send_files_dir/2.send"
> +
> +# We need a very specific leaf layout where we punch a hole in the last file
> +# extent item of a leaf and the next leaf was not modified in the send snapshot,
> +# so we set a fixed size of 16K.
> +_scratch_mkfs -n 16K >> $seqres.full 2>&1 || _fail "first mkfs failed"
> +_scratch_mount
> +
> +touch $SCRATCH_MNT/foo
> +# 205 extents of 64K each with a 64K hole between them.
> +for ((i = 0; i < 410; i += 2)); do
> +	offset=$((i * 64 * 1024))
> +	$XFS_IO_PROG -c "pwrite -b 64K -S 0xab $offset 64K" \
> +		$SCRATCH_MNT/foo > /dev/null
> +done
> +
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> +
> +# Punch a hole in the last 16K of the last file extent item of the first leaf,
> +# which starts at file offset 12845056, so 12845056 + 64K - 16K = 12894208.
> +# This does not cause any change in the second leaf of the fs tree, so it will
> +# be shared between the snapshots.
> +$XFS_IO_PROG -c "fpunch 12894208 16K" $SCRATCH_MNT/foo
> +
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> +
> +echo "File digest in the original fs: $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> +
> +_btrfs send -f $first_stream $SCRATCH_MNT/snap1
> +_btrfs send -f $second_stream -p $SCRATCH_MNT/snap1 $SCRATCH_MNT/snap2
> +
> +# Create a new fs and apply both send streams.
> +_scratch_unmount
> +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> +_scratch_mount
> +
> +_btrfs receive -f $first_stream $SCRATCH_MNT
> +_btrfs receive -f $second_stream $SCRATCH_MNT
> +
> +echo "File digest in the new fs:      $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> +
> +# Check the region we punched a hole is full of zeroes.
> +echo "Hole data:"
> +od -A d -t x1 -j 12894208 -N $((16 * 1024)) $SCRATCH_MNT/snap2/foo


Can this be replaced with common helper: _hexdump ?

Thanks,
Zorro

> +
> +# success, all done
> +_exit 0
> diff --git a/tests/btrfs/362.out b/tests/btrfs/362.out
> new file mode 100644
> index 00000000..789abb53
> --- /dev/null
> +++ b/tests/btrfs/362.out
> @@ -0,0 +1,7 @@
> +QA output created by 362
> +File digest in the original fs: a5a06694d9f7fc3245afb2f391636e44
> +File digest in the new fs:      a5a06694d9f7fc3245afb2f391636e44
> +Hole data:
> +12894208 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> +*
> +12910592
> -- 
> 2.53.0
> 
> 

Re: [PATCH] btrfs: test an incremental send scenario with hole punching

[Filipe Manana]

On Sat, Nov 14, 2026 at 3:05 PM Zorro Lang <zlang@kernel.org> wrote:
>
> On Wed, Oct 21, 2026 at 09:17:25PM +0100, fdmanana@kernel.org wrote:
> > From: Filipe Manana <fdmanana@suse.com>
> >
> > Test an incremental send scenario where we punched a hole into a tail
> > section of a file extent item that is the last item in a leaf and the
> > next leaf in the subvolume tree was not changed (remains shared between
> > the parent and send snapshots).
> >
> > This currently fails and is fixed by the following kernel patch:
> >
> >   btrfs: send: fix missing hole punching during incremental send
> >
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > ---
> >  tests/btrfs/362     | 84 +++++++++++++++++++++++++++++++++++++++++++++
> >  tests/btrfs/362.out |  7 ++++
> >  2 files changed, 91 insertions(+)
> >  create mode 100755 tests/btrfs/362
> >  create mode 100644 tests/btrfs/362.out
> >
> > diff --git a/tests/btrfs/362 b/tests/btrfs/362
> > new file mode 100755
> > index 00000000..aca56540
> > --- /dev/null
> > +++ b/tests/btrfs/362
> > @@ -0,0 +1,84 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2026 SUSE S.A.  All Rights Reserved.
> > +#
> > +# FS QA Test 362
> > +#
> > +# Test an incremental send scenario where we punched a hole into a tail section
> > +# of a file extent item that is the last item in a leaf and the next leaf in the
> > +# subvolume tree was not changed (remains shared between the parent and send
> > +# snapshots).
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick send
> > +
> > +_cleanup()
> > +{
> > +     cd /
> > +     rm -fr $tmp.*
> > +     rm -fr $send_files_dir
> > +}
> > +
> > +. ./common/filter
> > +
> > +_require_test
> > +_require_scratch
> > +_require_btrfs_support_sectorsize 4096
> > +_require_xfs_io_command "fpunch"
> > +
> > +_fixed_by_kernel_commit xxxxxxxxxx \
> > +     "btrfs: send: fix missing hole punching during incremental send"
> > +
> > +send_files_dir=$TEST_DIR/btrfs-test-$seq
> > +
> > +rm -fr $send_files_dir
> > +mkdir $send_files_dir
> > +
> > +first_stream="$send_files_dir/1.send"
> > +second_stream="$send_files_dir/2.send"
> > +
> > +# We need a very specific leaf layout where we punch a hole in the last file
> > +# extent item of a leaf and the next leaf was not modified in the send snapshot,
> > +# so we set a fixed size of 16K.
> > +_scratch_mkfs -n 16K >> $seqres.full 2>&1 || _fail "first mkfs failed"
> > +_scratch_mount
> > +
> > +touch $SCRATCH_MNT/foo
> > +# 205 extents of 64K each with a 64K hole between them.
> > +for ((i = 0; i < 410; i += 2)); do
> > +     offset=$((i * 64 * 1024))
> > +     $XFS_IO_PROG -c "pwrite -b 64K -S 0xab $offset 64K" \
> > +             $SCRATCH_MNT/foo > /dev/null
> > +done
> > +
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> > +
> > +# Punch a hole in the last 16K of the last file extent item of the first leaf,
> > +# which starts at file offset 12845056, so 12845056 + 64K - 16K = 12894208.
> > +# This does not cause any change in the second leaf of the fs tree, so it will
> > +# be shared between the snapshots.
> > +$XFS_IO_PROG -c "fpunch 12894208 16K" $SCRATCH_MNT/foo
> > +
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> > +
> > +echo "File digest in the original fs: $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> > +
> > +_btrfs send -f $first_stream $SCRATCH_MNT/snap1
> > +_btrfs send -f $second_stream -p $SCRATCH_MNT/snap1 $SCRATCH_MNT/snap2
> > +
> > +# Create a new fs and apply both send streams.
> > +_scratch_unmount
> > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> > +_scratch_mount
> > +
> > +_btrfs receive -f $first_stream $SCRATCH_MNT
> > +_btrfs receive -f $second_stream $SCRATCH_MNT
> > +
> > +echo "File digest in the new fs:      $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> > +
> > +# Check the region we punched a hole is full of zeroes.
> > +echo "Hole data:"
> > +od -A d -t x1 -j 12894208 -N $((16 * 1024)) $SCRATCH_MNT/snap2/foo
>
>
> Can this be replaced with common helper: _hexdump ?

I don't have a strong objection to that, but honestly I prefer it as
it is because it prints the addresses in decimal format, which is more
human readable and directly matches the value passed to the hole
punching operation above, making it easier to read.

Do you want to make the change yourself, or should I send a new
version just for that?

>
> Thanks,
> Zorro
>
> > +
> > +# success, all done
> > +_exit 0
> > diff --git a/tests/btrfs/362.out b/tests/btrfs/362.out
> > new file mode 100644
> > index 00000000..789abb53
> > --- /dev/null
> > +++ b/tests/btrfs/362.out
> > @@ -0,0 +1,7 @@
> > +QA output created by 362
> > +File digest in the original fs: a5a06694d9f7fc3245afb2f391636e44
> > +File digest in the new fs:      a5a06694d9f7fc3245afb2f391636e44
> > +Hole data:
> > +12894208 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > +*
> > +12910592
> > --
> > 2.53.0
> >
> >

Re: [PATCH] btrfs: test an incremental send scenario with hole punching

[Zorro Lang]

On Sat, Nov 14, 2026 at 04:51:27PM +0000, Filipe Manana wrote:
> On Sat, Nov 14, 2026 at 3:05 PM Zorro Lang <zlang@kernel.org> wrote:
> >
> > On Wed, Oct 21, 2026 at 09:17:25PM +0100, fdmanana@kernel.org wrote:
> > > From: Filipe Manana <fdmanana@suse.com>
> > >
> > > Test an incremental send scenario where we punched a hole into a tail
> > > section of a file extent item that is the last item in a leaf and the
> > > next leaf in the subvolume tree was not changed (remains shared between
> > > the parent and send snapshots).
> > >
> > > This currently fails and is fixed by the following kernel patch:
> > >
> > >   btrfs: send: fix missing hole punching during incremental send
> > >
> > > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > > ---
> > >  tests/btrfs/362     | 84 +++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/btrfs/362.out |  7 ++++
> > >  2 files changed, 91 insertions(+)
> > >  create mode 100755 tests/btrfs/362
> > >  create mode 100644 tests/btrfs/362.out
> > >
> > > diff --git a/tests/btrfs/362 b/tests/btrfs/362
> > > new file mode 100755
> > > index 00000000..aca56540
> > > --- /dev/null
> > > +++ b/tests/btrfs/362
> > > @@ -0,0 +1,84 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2026 SUSE S.A.  All Rights Reserved.
> > > +#
> > > +# FS QA Test 362
> > > +#
> > > +# Test an incremental send scenario where we punched a hole into a tail section
> > > +# of a file extent item that is the last item in a leaf and the next leaf in the
> > > +# subvolume tree was not changed (remains shared between the parent and send
> > > +# snapshots).
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest auto quick send
> > > +
> > > +_cleanup()
> > > +{
> > > +     cd /
> > > +     rm -fr $tmp.*
> > > +     rm -fr $send_files_dir
> > > +}
> > > +
> > > +. ./common/filter
> > > +
> > > +_require_test
> > > +_require_scratch
> > > +_require_btrfs_support_sectorsize 4096
> > > +_require_xfs_io_command "fpunch"
> > > +
> > > +_fixed_by_kernel_commit xxxxxxxxxx \
> > > +     "btrfs: send: fix missing hole punching during incremental send"
> > > +
> > > +send_files_dir=$TEST_DIR/btrfs-test-$seq
> > > +
> > > +rm -fr $send_files_dir
> > > +mkdir $send_files_dir
> > > +
> > > +first_stream="$send_files_dir/1.send"
> > > +second_stream="$send_files_dir/2.send"
> > > +
> > > +# We need a very specific leaf layout where we punch a hole in the last file
> > > +# extent item of a leaf and the next leaf was not modified in the send snapshot,
> > > +# so we set a fixed size of 16K.
> > > +_scratch_mkfs -n 16K >> $seqres.full 2>&1 || _fail "first mkfs failed"
> > > +_scratch_mount
> > > +
> > > +touch $SCRATCH_MNT/foo
> > > +# 205 extents of 64K each with a 64K hole between them.
> > > +for ((i = 0; i < 410; i += 2)); do
> > > +     offset=$((i * 64 * 1024))
> > > +     $XFS_IO_PROG -c "pwrite -b 64K -S 0xab $offset 64K" \
> > > +             $SCRATCH_MNT/foo > /dev/null
> > > +done
> > > +
> > > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> > > +
> > > +# Punch a hole in the last 16K of the last file extent item of the first leaf,
> > > +# which starts at file offset 12845056, so 12845056 + 64K - 16K = 12894208.
> > > +# This does not cause any change in the second leaf of the fs tree, so it will
> > > +# be shared between the snapshots.
> > > +$XFS_IO_PROG -c "fpunch 12894208 16K" $SCRATCH_MNT/foo
> > > +
> > > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> > > +
> > > +echo "File digest in the original fs: $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> > > +
> > > +_btrfs send -f $first_stream $SCRATCH_MNT/snap1
> > > +_btrfs send -f $second_stream -p $SCRATCH_MNT/snap1 $SCRATCH_MNT/snap2
> > > +
> > > +# Create a new fs and apply both send streams.
> > > +_scratch_unmount
> > > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> > > +_scratch_mount
> > > +
> > > +_btrfs receive -f $first_stream $SCRATCH_MNT
> > > +_btrfs receive -f $second_stream $SCRATCH_MNT
> > > +
> > > +echo "File digest in the new fs:      $(_md5_checksum $SCRATCH_MNT/snap2/foo)"
> > > +
> > > +# Check the region we punched a hole is full of zeroes.
> > > +echo "Hole data:"
> > > +od -A d -t x1 -j 12894208 -N $((16 * 1024)) $SCRATCH_MNT/snap2/foo
> >
> >
> > Can this be replaced with common helper: _hexdump ?
> 
> I don't have a strong objection to that, but honestly I prefer it as
> it is because it prints the addresses in decimal format, which is more
> human readable and directly matches the value passed to the hole
> punching operation above, making it easier to read.
> 
> Do you want to make the change yourself, or should I send a new
> version just for that?

If I were to change it, I'd have to merge the kernel patch above, rebuild and
install the kernel, run the tests, and then update the .out file.
Never mind, just using od works fine too, since the _hexdump helper was
originally introduced to replace the widespread use of the hexdump command
anyway. Anyway, no need to send a new version. If I find some time, I'll
make the change myself, or merging it directly :)

Thanks,
Zorro

> 
> >
> > Thanks,
> > Zorro
> >
> > > +
> > > +# success, all done
> > > +_exit 0
> > > diff --git a/tests/btrfs/362.out b/tests/btrfs/362.out
> > > new file mode 100644
> > > index 00000000..789abb53
> > > --- /dev/null
> > > +++ b/tests/btrfs/362.out
> > > @@ -0,0 +1,7 @@
> > > +QA output created by 362
> > > +File digest in the original fs: a5a06694d9f7fc3245afb2f391636e44
> > > +File digest in the new fs:      a5a06694d9f7fc3245afb2f391636e44
> > > +Hole data:
> > > +12894208 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > +*
> > > +12910592
> > > --
> > > 2.53.0
> > >
> > >
>