Incremental btrfs send/receive fails if file is unlinked and cloned afterwards

Incremental btrfs send/receive fails if file is unlinked and cloned afterwards

[Martin Raiber]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

Hi,

the commit "Btrfs: incremental send, check if orphanized dir inode needs
delayed rename" causes incremental send/receive to fail if a file is
unlinked and then reflinked to the same location from the parent
snapshot. An xfstest reproducing the issue is attached.

Regards,
Martin

[-- Attachment #2: 0001-btrfs-test-for-incremental-send-after-file-unlink-an.patch --]
[-- Type: text/plain, Size: 4694 bytes --]

From ebc5e8721264823a0df92b31e5fb1381f7f5e6f8 Mon Sep 17 00:00:00 2001
From: Martin Raiber <martin@urbackup.org>
Date: Fri, 25 Sep 2015 13:24:13 +0200
Subject: [PATCH 1/1] btrfs: test for incremental send after file unlink and
 then cloning

Creating a snapshot, then removing a file and cloning it back to its
original location, causes btrfs send/receive to fail, because
it doesn't use the correct path for the file unlink.
---
 tests/btrfs/104     | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/104.out |  7 ++++
 tests/btrfs/group   |  1 +
 3 files changed, 102 insertions(+)
 create mode 100755 tests/btrfs/104
 create mode 100644 tests/btrfs/104.out

diff --git a/tests/btrfs/104 b/tests/btrfs/104
new file mode 100755
index 0000000..976228f
--- /dev/null
+++ b/tests/btrfs/104
@@ -0,0 +1,94 @@
+#! /bin/bash
+# FS QA Test No. btrfs/104
+#
+# Test that an incremental send works after a files gets unlinked
+# and then cloned back from the previous snapshot.
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2015 SUSE Linux Products GmbH. All Rights Reserved.
+# Copyright (C) 2015 Martin Raiber <martin@urbackup.org>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -fr $send_files_dir
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_cloner
+_need_to_be_root
+_require_cp_reflink
+
+send_files_dir=$TEST_DIR/btrfs-test-$seq
+
+rm -f $seqres.full
+rm -fr $send_files_dir
+mkdir $send_files_dir
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# Create our test file with a single extent of 64K starting at file offset 128K.
+mkdir -p $SCRATCH_MNT/foo
+$XFS_IO_PROG -f -c "pwrite -S 0xaa 128K 64K" $SCRATCH_MNT/foo/bar | _filter_xfs_io
+
+_run_btrfs_util_prog subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/mysnap1
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT $SCRATCH_MNT/mysnap2
+
+#Remove the file and then reflink it back from the original snapshot
+rm $SCRATCH_MNT/mysnap2/foo/bar
+cp --reflink=always $SCRATCH_MNT/foo/bar $SCRATCH_MNT/mysnap2/foo/bar
+
+_run_btrfs_util_prog subvolume snapshot -r $SCRATCH_MNT/mysnap2 $SCRATCH_MNT/mysnap2_ro
+
+_run_btrfs_util_prog send $SCRATCH_MNT/mysnap1 -f $send_files_dir/1.snap
+_run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 $SCRATCH_MNT/mysnap2_ro \
+	-f $send_files_dir/2.snap
+
+echo "File digest in the original filesystem:"
+md5sum $SCRATCH_MNT/mysnap2_ro/foo/bar | _filter_scratch
+
+# Now recreate the filesystem by receiving both send streams and verify we get
+# the same file contents that the original filesystem had.
+_scratch_unmount
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_run_btrfs_util_prog receive -vv $SCRATCH_MNT -f $send_files_dir/1.snap
+_run_btrfs_util_prog receive -vv $SCRATCH_MNT -f $send_files_dir/2.snap
+
+echo "File digest in the new filesystem:"
+md5sum $SCRATCH_MNT/mysnap2_ro/foo/bar | _filter_scratch
+
+status=0
+exit
diff --git a/tests/btrfs/104.out b/tests/btrfs/104.out
new file mode 100644
index 0000000..6d18932
--- /dev/null
+++ b/tests/btrfs/104.out
@@ -0,0 +1,7 @@
+QA output created by 104
+wrote 65536/65536 bytes at offset 131072
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+File digest in the original filesystem:
+eb4de91c30abc45b27bc4c00a653d84e  SCRATCH_MNT/mysnap2_ro/foo/bar
+File digest in the new filesystem:
+eb4de91c30abc45b27bc4c00a653d84e  SCRATCH_MNT/mysnap2_ro/foo/bar
diff --git a/tests/btrfs/group b/tests/btrfs/group
index e92a65a..91580b9 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -106,3 +106,4 @@
 101 auto quick replace
 102 auto quick metadata enospc
 103 auto quick clone compress
+104 auto quick send clone
-- 
2.5.1

Re: Incremental btrfs send/receive fails if file is unlinked and cloned afterwards

[Filipe Manana]

On Fri, Sep 25, 2015 at 1:20 PM, Martin Raiber <martin@urbackup.org> wrote:
> Hi,
>
> the commit "Btrfs: incremental send, check if orphanized dir inode needs
> delayed rename" causes incremental send/receive to fail if a file is
> unlinked and then reflinked to the same location from the parent
> snapshot. An xfstest reproducing the issue is attached.

Thanks for the report and test.
The problem is not related at all to reflinked files (or extent
cloning/dedup). You run into this problem by just creating a file with
the same name and at the same location, and it needs to be the file
with highest inode number in the snapshot.

I've sent a fix and modified your test too, as it didn't conform to
the xfstests coding style, had unnecessary requirements (cloner
program, cp with reflink support) and added a description of the
problem.

thanks

>
> Regards,
> Martin



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."