* [PATCH v4 1/6] fstests: rename _require_btrfs to _require_btrfs_subcommand
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 2/6] fstests: btrfs: Add basic test for btrfs in-band de-duplication Qu Wenruo
` (5 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
Rename _require_btrfs() to _require_btrfs_subcommand() to avoid
confusion, as all other _require_btrfs_* has a quite clear suffix, like
_require_btrfs_mkfs_feature() or _require_btrfs_fs_feature().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
common/rc | 2 +-
tests/btrfs/004 | 2 +-
tests/btrfs/048 | 2 +-
tests/btrfs/059 | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/common/rc b/common/rc
index 16f5a43..957182f 100644
--- a/common/rc
+++ b/common/rc
@@ -2706,7 +2706,7 @@ _require_deletable_scratch_dev_pool()
}
# We check for btrfs and (optionally) features of the btrfs command
-_require_btrfs()
+_require_btrfs_subcommand()
{
cmd=$1
_require_command "$BTRFS_UTIL_PROG" btrfs
diff --git a/tests/btrfs/004 b/tests/btrfs/004
index 905770a..2ce628e 100755
--- a/tests/btrfs/004
+++ b/tests/btrfs/004
@@ -51,7 +51,7 @@ _supported_fs btrfs
_supported_os Linux
_require_scratch
_require_no_large_scratch_dev
-_require_btrfs inspect-internal
+_require_btrfs_subcommand inspect-internal
_require_command "/usr/sbin/filefrag" filefrag
rm -f $seqres.full
diff --git a/tests/btrfs/048 b/tests/btrfs/048
index c2cb4a6..38089f6 100755
--- a/tests/btrfs/048
+++ b/tests/btrfs/048
@@ -47,7 +47,7 @@ _supported_fs btrfs
_supported_os Linux
_require_test
_require_scratch
-_require_btrfs "property"
+_require_btrfs_subcommand "property"
send_files_dir=$TEST_DIR/btrfs-test-$seq
diff --git a/tests/btrfs/059 b/tests/btrfs/059
index b9a6ef4..f6c2e27 100755
--- a/tests/btrfs/059
+++ b/tests/btrfs/059
@@ -50,7 +50,7 @@ _supported_fs btrfs
_supported_os Linux
_require_test
_require_scratch
-_require_btrfs "property"
+_require_btrfs_subcommand "property"
rm -f $seqres.full
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v4 2/6] fstests: btrfs: Add basic test for btrfs in-band de-duplication
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 1/6] fstests: rename _require_btrfs to _require_btrfs_subcommand Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 3/6] fstests: btrfs: Add testcase for btrfs dedup enable disable race test Qu Wenruo
` (4 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
Add basic test for btrfs in-band de-duplication, including:
1) Enable
2) Re-enable
3) Dedup
4) File correctness
5) Disable
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
common/defrag | 13 +++++++
tests/btrfs/200 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/200.out | 21 ++++++++++
tests/btrfs/group | 1 +
4 files changed, 144 insertions(+)
create mode 100755 tests/btrfs/200
create mode 100644 tests/btrfs/200.out
diff --git a/common/defrag b/common/defrag
index 6fbf830..756a76d 100644
--- a/common/defrag
+++ b/common/defrag
@@ -49,6 +49,19 @@ _extent_count()
$XFS_IO_PROG -c "fiemap" $1 | tail -n +2 | grep -v hole | wc -l| $AWK_PROG '{print $1}'
}
+# Get the number of unique file extents
+# Unique file extents means they have different ondisk bytenr
+# Some filesystem supports reflinkat() or in-band de-dup can create
+# a file whose all file extents points to the same ondisk bytenr
+# this can be used to test if such reflinkat() or in-band de-dup works
+_extent_count_uniq()
+{
+ file=$1
+ $XFS_IO_PROG -c "fiemap" $file >> $seqres.full 2>&1
+ $XFS_IO_PROG -c "fiemap" $file | tail -n +2 | grep -v hole |\
+ $AWK_PROG '{print $3}' | sort | uniq | wc -l
+}
+
_check_extent_count()
{
min=$1
diff --git a/tests/btrfs/200 b/tests/btrfs/200
new file mode 100755
index 0000000..240b7db
--- /dev/null
+++ b/tests/btrfs/200
@@ -0,0 +1,109 @@
+#! /bin/bash
+# FS QA Test 200
+#
+# Basic btrfs inband dedup test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/defrag
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_subcommand dedup
+_require_btrfs_fs_feature dedup
+_require_btrfs_mkfs_feature dedup
+
+# File size is twice the maximum file extent of btrfs
+# So even fallbacked to non-dedup, it will have at least 2 extents
+file_size=256m
+
+_scratch_mkfs "-O dedup" >> $seqres.full 2>&1
+_scratch_mount
+
+do_dedup_test()
+{
+ backend=$1
+ dedup_bs=$2
+
+ _run_btrfs_util_prog dedup enable -s $backend -b $dedup_bs $SCRATCH_MNT
+ # do sync write to ensure dedup hash is added into dedup pool
+ $XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $dedup_bs" -c "fsync"\
+ $SCRATCH_MNT/initial_block | _filter_xfs_io
+
+ # do sync write to ensure we can get stable fiemap later
+ $XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $file_size" -c "fsync"\
+ $SCRATCH_MNT/real_file | _filter_xfs_io
+
+ # Test if real_file is de-duplicated
+ nr_uniq_extents=$(_extent_count_uniq $SCRATCH_MNT/real_file)
+ nr_total_extents=$(_extent_count $SCRATCH_MNT/real_file)
+
+ echo "uniq/total: $nr_uniq_extents/$nr_total_extents" >> $seqres.full
+ # Allow a small amount of dedup miss, as commit interval or
+ # memory pressure may break a dedup_bs block and cause
+ # small extent which won't go through dedup routine
+ _within_tolerance "number of uniq extents" $nr_uniq_extents \
+ $nr_total_extents $(($nr_total_extents - 1)) 5%
+
+ # Also check the md5sum to ensure data is not corrupted
+ md5=$(_md5_checksum $SCRATCH_MNT/real_file)
+ echo "md5sum: $md5"
+}
+
+# Test inmemory dedup first, use 64K dedup bs to keep compatibility
+# with 64K page size
+do_dedup_test inmemory 64K
+
+# Test ondisk backend, and re-enable function
+do_dedup_test ondisk 64K
+
+# Test 128K(default) dedup bs
+do_dedup_test inmemory 128K
+do_dedup_test ondisk 128K
+
+# Check dedup disable
+_run_btrfs_util_prog dedup disable $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/200.out b/tests/btrfs/200.out
new file mode 100644
index 0000000..e965f8a
--- /dev/null
+++ b/tests/btrfs/200.out
@@ -0,0 +1,21 @@
+QA output created by 200
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+md5sum: a30e0f3f1b0884081de11d4357811c2e
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+md5sum: a30e0f3f1b0884081de11d4357811c2e
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+md5sum: a30e0f3f1b0884081de11d4357811c2e
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+md5sum: a30e0f3f1b0884081de11d4357811c2e
diff --git a/tests/btrfs/group b/tests/btrfs/group
index a2fa412..0b7354b 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -119,3 +119,4 @@
116 auto quick metadata
117 auto quick send clone
118 auto quick snapshot metadata
+200 auto dedup
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v4 3/6] fstests: btrfs: Add testcase for btrfs dedup enable disable race test
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 1/6] fstests: rename _require_btrfs to _require_btrfs_subcommand Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 2/6] fstests: btrfs: Add basic test for btrfs in-band de-duplication Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 4/6] fstests: btrfs: Add per inode dedup flag test Qu Wenruo
` (3 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
Add test case to check btrfs dedup enable/disable race.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/btrfs/201 | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/201.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 101 insertions(+)
create mode 100755 tests/btrfs/201
create mode 100644 tests/btrfs/201.out
diff --git a/tests/btrfs/201 b/tests/btrfs/201
new file mode 100755
index 0000000..1c523c5
--- /dev/null
+++ b/tests/btrfs/201
@@ -0,0 +1,98 @@
+#! /bin/bash
+# FS QA Test 201
+#
+# Basic btrfs inband dedup enable/disable race test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+ killall $FSSTRESS_PROG &> /dev/null
+ kill $fsstress_pid &> /dev/null
+ wait
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_subcommand dedup
+_require_btrfs_fs_feature dedup
+_require_btrfs_mkfs_feature dedup
+
+# Use 64K dedup size to keep compatibility for 64K page size
+dedup_bs=64K
+
+_scratch_mkfs "-O dedup" >> $seqres.full 2>&1
+_scratch_mount
+
+mkdir -p $SCRATCH_MNT/stressdir
+
+runtime=$((60 * $TIME_FACTOR))
+
+trigger_work()
+{
+ while true; do
+ _run_btrfs_util_prog dedup enable -s inmemory \
+ -b $dedup_bs $SCRATCH_MNT
+ sleep 5
+ _run_btrfs_util_prog dedup disable $SCRATCH_MNT
+ sleep 5
+ _run_btrfs_util_prog dedup enable -s ondisk \
+ -b $dedup_bs $SCRATCH_MNT
+ sleep 5
+ _run_btrfs_util_prog dedup disable $SCRATCH_MNT
+ sleep 5
+ done
+}
+
+$FSSTRESS_PROG $(_scale_fsstress_args -p 8 -n 10000000) $FSSTRESS_AVOID \
+ -d $SCRATCH_MNT/stressdir > /dev/null 2>&1 &
+
+trigger_work &
+trigger_pid=$!
+
+sleep $runtime
+killall $FSSTRESS_PROG
+kill $trigger_pid
+wait
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/201.out b/tests/btrfs/201.out
new file mode 100644
index 0000000..5ac973f
--- /dev/null
+++ b/tests/btrfs/201.out
@@ -0,0 +1,2 @@
+QA output created by 201
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 0b7354b..76ebea7 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -120,3 +120,4 @@
117 auto quick send clone
118 auto quick snapshot metadata
200 auto dedup
+201 auto dedup
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v4 4/6] fstests: btrfs: Add per inode dedup flag test
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
` (2 preceding siblings ...)
2016-03-09 8:33 ` [PATCH v4 3/6] fstests: btrfs: Add testcase for btrfs dedup enable disable race test Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 5/6] fstests: btrfs: Test inband dedup with balance Qu Wenruo
` (2 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
This test will check per inode dedup flag.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/btrfs/202 | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/202.out | 15 ++++++++
tests/btrfs/group | 1 +
3 files changed, 124 insertions(+)
create mode 100755 tests/btrfs/202
create mode 100644 tests/btrfs/202.out
diff --git a/tests/btrfs/202 b/tests/btrfs/202
new file mode 100755
index 0000000..33ad51d
--- /dev/null
+++ b/tests/btrfs/202
@@ -0,0 +1,108 @@
+#! /bin/bash
+# FS QA Test 202
+#
+# Btrfs per inode dedup flag test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/defrag
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_subcommand dedup
+_require_btrfs_subcommand property
+_require_btrfs_fs_feature dedup
+_require_btrfs_mkfs_feature dedup
+
+# File size is twice the maximum file extent of btrfs
+# So even fallbacked to non-dedup, it will have at least 2 extents
+file_size=256m
+dedup_bs=64k
+
+_scratch_mkfs "-O dedup" >> $seqres.full 2>&1
+_scratch_mount
+
+# Return 0 for not deduped at all , return 1 for part or full deduped
+test_file_deduped () {
+ file=$1
+
+ nr_uniq_extents=$(_extent_count_uniq $file)
+ nr_total_extents=$(_extent_count $file)
+
+ if [ $nr_uniq_extents -eq $nr_total_extents ]; then
+ echo "$file: not de-duplicated"
+ else
+ echo "$file: de-duplicated"
+ fi
+}
+
+_run_btrfs_util_prog dedup enable -b $dedup_bs $SCRATCH_MNT
+touch $SCRATCH_MNT/dedup_file
+touch $SCRATCH_MNT/no_dedup_file
+mkdir $SCRATCH_MNT/dedup_dir
+mkdir $SCRATCH_MNT/no_dedup_dir
+
+_run_btrfs_util_prog property set $SCRATCH_MNT/no_dedup_file dedup disable
+_run_btrfs_util_prog property set $SCRATCH_MNT/no_dedup_dir dedup disable
+
+# use fsync write to ensure hash is added to dedup pool
+$XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $dedup_bs" -c "fsync" \
+ $SCRATCH_MNT/tmp | _filter_xfs_io
+
+$XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $file_size" \
+ $SCRATCH_MNT/dedup_file | _filter_xfs_io
+$XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $file_size" \
+ $SCRATCH_MNT/no_dedup_file | _filter_xfs_io
+$XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $file_size" \
+ $SCRATCH_MNT/dedup_dir/default_file | _filter_xfs_io
+$XFS_IO_PROG -f -c "pwrite -b $dedup_bs 0 $file_size" \
+ $SCRATCH_MNT/no_dedup_dir/default_file | _filter_xfs_io
+
+test_file_deduped $SCRATCH_MNT/dedup_file | _filter_scratch
+test_file_deduped $SCRATCH_MNT/no_dedup_file | _filter_scratch
+test_file_deduped $SCRATCH_MNT/dedup_dir/default_file | _filter_scratch
+test_file_deduped $SCRATCH_MNT/no_dedup_dir/default_file | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/202.out b/tests/btrfs/202.out
new file mode 100644
index 0000000..6c1b4cb
--- /dev/null
+++ b/tests/btrfs/202.out
@@ -0,0 +1,15 @@
+QA output created by 202
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 268435456/268435456 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+SCRATCH_MNT/dedup_file: de-duplicated
+SCRATCH_MNT/no_dedup_file: not de-duplicated
+SCRATCH_MNT/dedup_dir/default_file: de-duplicated
+SCRATCH_MNT/no_dedup_dir/default_file: not de-duplicated
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 76ebea7..0c03cf1 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -121,3 +121,4 @@
118 auto quick snapshot metadata
200 auto dedup
201 auto dedup
+202 auto dedup
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v4 5/6] fstests: btrfs: Test inband dedup with balance.
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
` (3 preceding siblings ...)
2016-03-09 8:33 ` [PATCH v4 4/6] fstests: btrfs: Add per inode dedup flag test Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-09 8:33 ` [PATCH v4 6/6] fstests: btrfs: Test for btrfs dedup tree balance bug Qu Wenruo
2016-03-14 7:56 ` [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
Btrfs balance will reloate date extent, but its hash is removed too late
at run_delayed_ref() time, which will cause extent ref increased
increased during balance, cause either find_data_references() gives
WARN_ON() or even run_delayed_refs() fails and cause transaction abort.
Add such concurrency test for inband dedup and balance.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/btrfs/203 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/203.out | 5 +++
tests/btrfs/group | 1 +
3 files changed, 115 insertions(+)
create mode 100755 tests/btrfs/203
create mode 100644 tests/btrfs/203.out
diff --git a/tests/btrfs/203 b/tests/btrfs/203
new file mode 100755
index 0000000..ad4a871
--- /dev/null
+++ b/tests/btrfs/203
@@ -0,0 +1,109 @@
+#! /bin/bash
+# FS QA Test 203
+#
+# Btrfs reflink with balance concurrency test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ kill $populate_pid &> /dev/null
+ kill $balance_pid &> /dev/null
+ wait
+ # Check later comment for reason
+ $BTRFS_UTIL_PROG balance cancel $SCRATCH_MNT &> /dev/null
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_cp_reflink
+_require_btrfs_subcommand dedup
+_require_btrfs_fs_feature dedup
+_require_btrfs_mkfs_feature dedup
+
+dedup_bs=128k
+file_size_in_kilo=4096
+init_file=$SCRATCH_MNT/foo
+run_time=$((60 * $TIME_FACTOR))
+
+_scratch_mkfs "-O dedup" >> $seqres.full 2>&1
+_scratch_mount
+
+do_dedup_balance_test()
+{
+ backend=$1
+ _run_btrfs_util_prog dedup enable -b $dedup_bs -s $backend $SCRATCH_MNT
+
+ # create the initial file and fill hash pool
+ $XFS_IO_PROG -f -c "pwrite -S 0x0 -b $dedup_bs 0 $dedup_bs" -c "fsync" \
+ $init_file | _filter_xfs_io
+
+ _btrfs_stress_balance $SCRATCH_MNT >/dev/null 2>&1 &
+ balance_pid=$!
+
+ # Populate fs with all 0 data, to trigger enough in-band dedup work
+ # to race with balance
+ _populate_fs -n 5 -f 10000000 -d 1 -r $SCRATCH_MNT \
+ -s $file_size_in_kilo &> /dev/null &
+ populate_pid=$!
+
+ sleep $run_time
+
+ kill $populate_pid
+ kill $balance_pid
+ wait
+
+ # Sometimes even we killed $balance_pid and wait returned,
+ # balance may still be running, use balance cancel to wait it.
+ # As this is just a workaround, we don't want it pollute seqres
+ # so call $BTRFS_UTIL_PROG directly
+ $BTRFS_UTIL_PROG balance cancel $SCRATCH_MNT &> /dev/null
+
+ rm $SCRATCH_MNT/* -rf &> /dev/null
+ _run_btrfs_util_prog dedup disable $SCRATCH_MNT
+}
+
+do_dedup_balance_test inmemory
+do_dedup_balance_test ondisk
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/203.out b/tests/btrfs/203.out
new file mode 100644
index 0000000..af036b6
--- /dev/null
+++ b/tests/btrfs/203.out
@@ -0,0 +1,5 @@
+QA output created by 203
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 0c03cf1..fa90f33 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -122,3 +122,4 @@
200 auto dedup
201 auto dedup
202 auto dedup
+203 auto dedup balance
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v4 6/6] fstests: btrfs: Test for btrfs dedup tree balance bug
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
` (4 preceding siblings ...)
2016-03-09 8:33 ` [PATCH v4 5/6] fstests: btrfs: Test inband dedup with balance Qu Wenruo
@ 2016-03-09 8:33 ` Qu Wenruo
2016-03-14 7:56 ` [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
6 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-09 8:33 UTC (permalink / raw)
To: fstests, linux-btrfs
Btrfs in-band de-duplication has a bug when dedup tree metadata is
balanced, it will corrupted btrfs delayed ref and cause the following
kernel warning:
BTRFS error (device sdb6): unable to find ref byte nr 29736960 parent 0
root 11 owner 0 offset 0
------------[ cut here ]------------
WARNING: CPU: 1 PID: 19113 at fs/btrfs/extent-tree.c:6636
__btrfs_free_extent.isra.66+0xb6d/0xd20 [btrfs]()
BTRFS: Transaction aborted (error -2)
Modules linked in: btrfs(O) xor zlib_deflate raid6_pq xfs [last
unloaded: btrfs]
CPU: 1 PID: 19113 Comm: btrfs Tainted: G W O 4.5.0-rc5+ #2
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox
12/01/2006
0000000000000000 ffff880035b0ba18 ffffffff813771ff ffff880035b0ba60
ffffffffa06a810a ffff880035b0ba50 ffffffff810bcb81 ffff88003c45c528
0000000001c5c000 00000000fffffffe ffff88003dc8c520 0000000000000000
Call Trace:
[<ffffffff813771ff>] dump_stack+0x67/0x98
[<ffffffff810bcb81>] warn_slowpath_common+0x81/0xc0
[<ffffffff810bcc07>] warn_slowpath_fmt+0x47/0x50
[<ffffffffa06028fd>] __btrfs_free_extent.isra.66+0xb6d/0xd20 [btrfs]
[<ffffffffa0606d4d>] __btrfs_run_delayed_refs.constprop.71+0x96d/0x1560
[btrfs]
[<ffffffff81202ad9>] ? cmpxchg_double_slab.isra.68+0x149/0x160
[<ffffffff81106a1d>] ? trace_hardirqs_on+0xd/0x10
[<ffffffffa060a5ce>] btrfs_run_delayed_refs+0x8e/0x2d0 [btrfs]
[<ffffffffa06209fe>] btrfs_commit_transaction+0x3e/0xb50 [btrfs]
[<ffffffffa069f26e>] ? btrfs_dedup_disable+0x28e/0x2c0 [btrfs]
[<ffffffff812035c3>] ? kfree+0x223/0x270
[<ffffffffa069f27a>] btrfs_dedup_disable+0x29a/0x2c0 [btrfs]
[<ffffffffa065e403>] btrfs_ioctl+0x2363/0x2a40 [btrfs]
[<ffffffff8116b12a>] ? __audit_syscall_entry+0xaa/0xf0
[<ffffffff81137ce6>] ? current_kernel_time64+0x56/0xa0
[<ffffffff8122080e>] do_vfs_ioctl+0x8e/0x690
[<ffffffff8116b12a>] ? __audit_syscall_entry+0xaa/0xf0
[<ffffffff8122c181>] ? __fget_light+0x61/0x90
[<ffffffff81220e84>] SyS_ioctl+0x74/0x80
[<ffffffff8180ad57>] entry_SYSCALL_64_fastpath+0x12/0x6f
---[ end trace 618d5a5bc21d6a7c ]---
Add regression test case for it.
Reported-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/btrfs/204 | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/204.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 77 insertions(+)
create mode 100755 tests/btrfs/204
create mode 100644 tests/btrfs/204.out
diff --git a/tests/btrfs/204 b/tests/btrfs/204
new file mode 100755
index 0000000..5baecf8
--- /dev/null
+++ b/tests/btrfs/204
@@ -0,0 +1,74 @@
+#! /bin/bash
+# FS QA Test 204
+#
+# Regression test for dedup tree metadata balance
+# If btrfs balanced dedup tree block twice, delayed ref will be corrupted
+# and cause kernel warning with RO fs.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_subcommand dedup
+_require_btrfs_fs_feature dedup
+_require_btrfs_mkfs_feature dedup
+
+_scratch_mkfs "-O dedup" >> $seqres.full 2>&1
+_scratch_mount
+
+_run_btrfs_util_prog dedup enable $SCRATCH_MNT
+
+# Balance btrfs metadata twice, this will ensure the second balance
+# to drop refence on old dedup root bytenr if kernel has the dedup bug
+_run_btrfs_util_prog balance start -m $SCRATCH_MNT
+_run_btrfs_util_prog balance start -m $SCRATCH_MNT
+
+# Disable will force a sync_fs() and cause kernel warning
+_run_btrfs_util_prog dedup disable $SCRATCH_MNT
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/204.out b/tests/btrfs/204.out
new file mode 100644
index 0000000..1035eae
--- /dev/null
+++ b/tests/btrfs/204.out
@@ -0,0 +1,2 @@
+QA output created by 204
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index fa90f33..089e4a8 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -123,3 +123,4 @@
201 auto dedup
202 auto dedup
203 auto dedup balance
+204 auto quick dedup
--
2.7.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH v4 0/6] Btrfs in-band de-duplication test cases
2016-03-09 8:33 [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
` (5 preceding siblings ...)
2016-03-09 8:33 ` [PATCH v4 6/6] fstests: btrfs: Test for btrfs dedup tree balance bug Qu Wenruo
@ 2016-03-14 7:56 ` Qu Wenruo
2016-03-14 17:24 ` Darrick J. Wong
6 siblings, 1 reply; 11+ messages in thread
From: Qu Wenruo @ 2016-03-14 7:56 UTC (permalink / raw)
To: fstests, linux-btrfs
Please don't merge this patchset.
As the there is some naming undecided recently.
The abbreviation 'dedup' may be changed to 'dedupe'.
I'll update them when all related parts is settled down.
Thanks,
Qu
Qu Wenruo wrote on 2016/03/09 16:33 +0800:
> Since we are push btrfs in-band de-duplication for v4.6, it's better to
> add test cases for this new feature.
>
> Except the first basic function test, the rest are all regression test
> which we found during the development.
> We also found some bugs from the generic test, but we need some fstests
> option allowing us to enable dedup for any test case.
> (We did it by hack _scratch_mount and _test_mount to enable dedup for
> any test case)
>
> Use the sequence number starts from 200 to avoid any possible conflicts.
> The './new' script returns some hole number which is not proper for
> such related test case set.
> Hopes it's not too hard for maintainer to modify the sequence number.
>
> v1:
> Btrfs mail list internal use only
> v2:
> Add test case 203 to test balance race
> v3:
> Follow Dave's comment to use more existing fstest infrastructure.
> Also fix a small bug in the 1st which forgot to remove old macros
> v4:
> Enhance test 203 to cover both dedup backend
> Add new test 204, a specific regression test for balance and dedup.
>
> Qu Wenruo (6):
> fstests: rename _require_btrfs to _require_btrfs_subcommand
> fstests: btrfs: Add basic test for btrfs in-band de-duplication
> fstests: btrfs: Add testcase for btrfs dedup enable disable race test
> fstests: btrfs: Add per inode dedup flag test
> fstests: btrfs: Test inband dedup with balance.
> fstests: btrfs: Test for btrfs dedup tree balance bug
>
> common/defrag | 13 +++++++
> common/rc | 2 +-
> tests/btrfs/004 | 2 +-
> tests/btrfs/048 | 2 +-
> tests/btrfs/059 | 2 +-
> tests/btrfs/200 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/200.out | 21 ++++++++++
> tests/btrfs/201 | 98 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/201.out | 2 +
> tests/btrfs/202 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/202.out | 15 ++++++++
> tests/btrfs/203 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/203.out | 5 +++
> tests/btrfs/204 | 74 +++++++++++++++++++++++++++++++++++
> tests/btrfs/204.out | 2 +
> tests/btrfs/group | 5 +++
> 16 files changed, 565 insertions(+), 4 deletions(-)
> create mode 100755 tests/btrfs/200
> create mode 100644 tests/btrfs/200.out
> create mode 100755 tests/btrfs/201
> create mode 100644 tests/btrfs/201.out
> create mode 100755 tests/btrfs/202
> create mode 100644 tests/btrfs/202.out
> create mode 100755 tests/btrfs/203
> create mode 100644 tests/btrfs/203.out
> create mode 100755 tests/btrfs/204
> create mode 100644 tests/btrfs/204.out
>
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v4 0/6] Btrfs in-band de-duplication test cases
2016-03-14 7:56 ` [PATCH v4 0/6] Btrfs in-band de-duplication test cases Qu Wenruo
@ 2016-03-14 17:24 ` Darrick J. Wong
2016-03-15 1:23 ` Qu Wenruo
2016-03-15 2:42 ` Dave Chinner
0 siblings, 2 replies; 11+ messages in thread
From: Darrick J. Wong @ 2016-03-14 17:24 UTC (permalink / raw)
To: Qu Wenruo; +Cc: fstests, linux-btrfs
On Mon, Mar 14, 2016 at 03:56:47PM +0800, Qu Wenruo wrote:
> Please don't merge this patchset.
>
> As the there is some naming undecided recently.
>
> The abbreviation 'dedup' may be changed to 'dedupe'.
> I'll update them when all related parts is settled down.
There's already a 'dedupe' group in xfstests for testing the out-of-band ioctl
that duperemove uses. I wondered if that factored into your decision to use
'dedup' as the group name for the inband tests.
Seeing as other filesystems are beginning to support the OOB ioctls and might
never support the in-band stuff btrfs is doing, what do people think about
keeping the out-of and in-band dedup tests in separate groups to make it clear
which dedupe feature each test is aiming to validate?
There's no technical requirement to do this since we can always add feature
tests if the inband tests ever become generic, but I do see value in being able
to test one interface without having the tests for the other interface piling
on the output (and being able to tell which interface at a glance).
--D
>
> Thanks,
> Qu
>
>
> Qu Wenruo wrote on 2016/03/09 16:33 +0800:
> >Since we are push btrfs in-band de-duplication for v4.6, it's better to
> >add test cases for this new feature.
> >
> >Except the first basic function test, the rest are all regression test
> >which we found during the development.
> >We also found some bugs from the generic test, but we need some fstests
> >option allowing us to enable dedup for any test case.
> >(We did it by hack _scratch_mount and _test_mount to enable dedup for
> >any test case)
> >
> >Use the sequence number starts from 200 to avoid any possible conflicts.
> >The './new' script returns some hole number which is not proper for
> >such related test case set.
> >Hopes it's not too hard for maintainer to modify the sequence number.
> >
> >v1:
> > Btrfs mail list internal use only
> >v2:
> > Add test case 203 to test balance race
> >v3:
> > Follow Dave's comment to use more existing fstest infrastructure.
> > Also fix a small bug in the 1st which forgot to remove old macros
> >v4:
> > Enhance test 203 to cover both dedup backend
> > Add new test 204, a specific regression test for balance and dedup.
> >
> >Qu Wenruo (6):
> > fstests: rename _require_btrfs to _require_btrfs_subcommand
> > fstests: btrfs: Add basic test for btrfs in-band de-duplication
> > fstests: btrfs: Add testcase for btrfs dedup enable disable race test
> > fstests: btrfs: Add per inode dedup flag test
> > fstests: btrfs: Test inband dedup with balance.
> > fstests: btrfs: Test for btrfs dedup tree balance bug
> >
> > common/defrag | 13 +++++++
> > common/rc | 2 +-
> > tests/btrfs/004 | 2 +-
> > tests/btrfs/048 | 2 +-
> > tests/btrfs/059 | 2 +-
> > tests/btrfs/200 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/200.out | 21 ++++++++++
> > tests/btrfs/201 | 98 ++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/201.out | 2 +
> > tests/btrfs/202 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/202.out | 15 ++++++++
> > tests/btrfs/203 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/203.out | 5 +++
> > tests/btrfs/204 | 74 +++++++++++++++++++++++++++++++++++
> > tests/btrfs/204.out | 2 +
> > tests/btrfs/group | 5 +++
> > 16 files changed, 565 insertions(+), 4 deletions(-)
> > create mode 100755 tests/btrfs/200
> > create mode 100644 tests/btrfs/200.out
> > create mode 100755 tests/btrfs/201
> > create mode 100644 tests/btrfs/201.out
> > create mode 100755 tests/btrfs/202
> > create mode 100644 tests/btrfs/202.out
> > create mode 100755 tests/btrfs/203
> > create mode 100644 tests/btrfs/203.out
> > create mode 100755 tests/btrfs/204
> > create mode 100644 tests/btrfs/204.out
> >
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v4 0/6] Btrfs in-band de-duplication test cases
2016-03-14 17:24 ` Darrick J. Wong
@ 2016-03-15 1:23 ` Qu Wenruo
2016-03-15 2:42 ` Dave Chinner
1 sibling, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2016-03-15 1:23 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: fstests, linux-btrfs
Darrick J. Wong wrote on 2016/03/14 10:24 -0700:
> On Mon, Mar 14, 2016 at 03:56:47PM +0800, Qu Wenruo wrote:
>> Please don't merge this patchset.
>>
>> As the there is some naming undecided recently.
>>
>> The abbreviation 'dedup' may be changed to 'dedupe'.
>> I'll update them when all related parts is settled down.
>
> There's already a 'dedupe' group in xfstests for testing the out-of-band ioctl
> that duperemove uses. I wondered if that factored into your decision to use
> 'dedup' as the group name for the inband tests.
Not exactly. The largest reason I chose 'dedup' is that zfs uses the
same abbreviation.
(And I forgot other kernel part is already using 'dedupe')
Although I did want to make some difference with out-of-band dedupe, but
the extra 'e' is not distinguishable enough.
And since btrfs community choose to use 'dedupe' and kernel conversion
is already done, it has already affected the test cases.
For example, since new kernel uses 'dedupe' in /sys/fs/btrfs/features/,
old "_require_btrfs_fs_feature dedup" will just fail and skipped the
test case.
>
> Seeing as other filesystems are beginning to support the OOB ioctls and might
> never support the in-band stuff btrfs is doing, what do people think about
> keeping the out-of and in-band dedup tests in separate groups to make it clear
> which dedupe feature each test is aiming to validate?
This is a good idea.
IMHO it can be done by changing group naming only.
Like naming inband dedupe group to "inband_dedupe" instead of current
"dedupe"? (OOB not touched and uses the old "dedupe")
BTW, if I understand it right, for a fs to support OOB ioctl, the fs
must support CoW.
And if a fs support CoW, it's not that hard to implement inband dedupe.
Btrfs just adds about 2K lines to implement it.
If removing btrfs specific on-disk backend related codes, it would be
about 1K ~ 1.5K new lines.
(Yes, btrfs inband dedupe supports 2 different hash storage backends)
Thanks,
Qu
>
> There's no technical requirement to do this since we can always add feature
> tests if the inband tests ever become generic, but I do see value in being able
> to test one interface without having the tests for the other interface piling
> on the output (and being able to tell which interface at a glance).
>
> --D
>
>>
>> Thanks,
>> Qu
>>
>>
>> Qu Wenruo wrote on 2016/03/09 16:33 +0800:
>>> Since we are push btrfs in-band de-duplication for v4.6, it's better to
>>> add test cases for this new feature.
>>>
>>> Except the first basic function test, the rest are all regression test
>>> which we found during the development.
>>> We also found some bugs from the generic test, but we need some fstests
>>> option allowing us to enable dedup for any test case.
>>> (We did it by hack _scratch_mount and _test_mount to enable dedup for
>>> any test case)
>>>
>>> Use the sequence number starts from 200 to avoid any possible conflicts.
>>> The './new' script returns some hole number which is not proper for
>>> such related test case set.
>>> Hopes it's not too hard for maintainer to modify the sequence number.
>>>
>>> v1:
>>> Btrfs mail list internal use only
>>> v2:
>>> Add test case 203 to test balance race
>>> v3:
>>> Follow Dave's comment to use more existing fstest infrastructure.
>>> Also fix a small bug in the 1st which forgot to remove old macros
>>> v4:
>>> Enhance test 203 to cover both dedup backend
>>> Add new test 204, a specific regression test for balance and dedup.
>>>
>>> Qu Wenruo (6):
>>> fstests: rename _require_btrfs to _require_btrfs_subcommand
>>> fstests: btrfs: Add basic test for btrfs in-band de-duplication
>>> fstests: btrfs: Add testcase for btrfs dedup enable disable race test
>>> fstests: btrfs: Add per inode dedup flag test
>>> fstests: btrfs: Test inband dedup with balance.
>>> fstests: btrfs: Test for btrfs dedup tree balance bug
>>>
>>> common/defrag | 13 +++++++
>>> common/rc | 2 +-
>>> tests/btrfs/004 | 2 +-
>>> tests/btrfs/048 | 2 +-
>>> tests/btrfs/059 | 2 +-
>>> tests/btrfs/200 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/200.out | 21 ++++++++++
>>> tests/btrfs/201 | 98 ++++++++++++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/201.out | 2 +
>>> tests/btrfs/202 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/202.out | 15 ++++++++
>>> tests/btrfs/203 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/203.out | 5 +++
>>> tests/btrfs/204 | 74 +++++++++++++++++++++++++++++++++++
>>> tests/btrfs/204.out | 2 +
>>> tests/btrfs/group | 5 +++
>>> 16 files changed, 565 insertions(+), 4 deletions(-)
>>> create mode 100755 tests/btrfs/200
>>> create mode 100644 tests/btrfs/200.out
>>> create mode 100755 tests/btrfs/201
>>> create mode 100644 tests/btrfs/201.out
>>> create mode 100755 tests/btrfs/202
>>> create mode 100644 tests/btrfs/202.out
>>> create mode 100755 tests/btrfs/203
>>> create mode 100644 tests/btrfs/203.out
>>> create mode 100755 tests/btrfs/204
>>> create mode 100644 tests/btrfs/204.out
>>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v4 0/6] Btrfs in-band de-duplication test cases
2016-03-14 17:24 ` Darrick J. Wong
2016-03-15 1:23 ` Qu Wenruo
@ 2016-03-15 2:42 ` Dave Chinner
1 sibling, 0 replies; 11+ messages in thread
From: Dave Chinner @ 2016-03-15 2:42 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Qu Wenruo, fstests, linux-btrfs
On Mon, Mar 14, 2016 at 10:24:30AM -0700, Darrick J. Wong wrote:
> On Mon, Mar 14, 2016 at 03:56:47PM +0800, Qu Wenruo wrote:
> > Please don't merge this patchset.
> >
> > As the there is some naming undecided recently.
> >
> > The abbreviation 'dedup' may be changed to 'dedupe'.
> > I'll update them when all related parts is settled down.
>
> There's already a 'dedupe' group in xfstests for testing the out-of-band ioctl
> that duperemove uses. I wondered if that factored into your decision to use
> 'dedup' as the group name for the inband tests.
>
> Seeing as other filesystems are beginning to support the OOB ioctls and might
> never support the in-band stuff btrfs is doing, what do people think about
> keeping the out-of and in-band dedup tests in separate groups to make it clear
> which dedupe feature each test is aiming to validate?
So just name the two groups appropriately: "ib-dedupe" and
"oob-dedupe" or something like that.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 11+ messages in thread