[PATCH 0/7] xfstests: minor fixes for the reflink/dedupe tests

[PATCH 0/7] xfstests: minor fixes for the reflink/dedupe tests

[Darrick J. Wong]

This is a patch set for xfstests that adds some more reflink tests
and fixes problems with some of the existing reflink tests.  Most
notably, I fixed the problems with the ENOSPC tests trying to use
the ENOSPC'd fs for loop control -- bad bad bad.

Comments and questions are, as always, welcome.  The fixes are
available on github[1].

--D

[1] https://github.com/djwong/xfstests/tree/djwong-devel

[PATCH 1/7] xfs/104: don't enospc when ag metadata overhead grows

[Darrick J. Wong]

Adapt to different metadata overhead sizes by trying to reserve
decreasing amounts of disk space until we actually succeed at it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/104 |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/104 b/tests/xfs/104
index 17f9b62..785027e 100755
--- a/tests/xfs/104
+++ b/tests/xfs/104
@@ -88,9 +88,14 @@ sizeb=`expr $size / $dbsize`	# in data blocks
 echo "*** creating scratch filesystem"
 _create_scratch -lsize=10m -dsize=${size} -dagcount=${nags}
 
-fillsize=`expr 110 \* 1048576`	# 110 megabytes of filling
 echo "*** using some initial space on scratch filesystem"
-_fill_scratch $fillsize
+for i in `seq 125 -1 90`; do
+	fillsize=`expr $i \* 1048576`
+	out="$(_fill_scratch $fillsize 2>&1)"
+	echo "$out" | grep -q 'No space left on device' && continue
+	test -n "${out}" && echo "$out"
+	break
+done
 
 #
 # Grow the filesystem while actively stressing it...

Re: [PATCH 1/7] xfs/104: don't enospc when ag metadata overhead grows

[Christoph Hellwig]

> +for i in `seq 125 -1 90`; do
> +	fillsize=`expr $i \* 1048576`
> +	out="$(_fill_scratch $fillsize 2>&1)"
> +	echo "$out" | grep -q 'No space left on device' && continue
> +	test -n "${out}" && echo "$out"
> +	break
> +done

That's a bit of an odd loop, and it would seem an inverted grep
would seem more obvious.  But otherwise this looks fine to me:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 2/7] tests: don't put loop control files on the scratch mount

[Darrick J. Wong]

If we're doing write/overwrite/snapshot/resource exhaustion tests on
the scratch device, use the test directory to hold the loop
termination signal files.  This way we don't run infinitely because we
can't create the flag due to ENOSPC.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/162 |    8 +++++---
 tests/generic/163 |    8 +++++---
 tests/generic/164 |    8 +++++---
 tests/generic/165 |    8 +++++---
 tests/generic/166 |    8 +++++---
 tests/generic/167 |    8 +++++---
 tests/generic/168 |    8 +++++---
 tests/generic/170 |    8 +++++---
 tests/generic/333 |   20 ++++++++++++--------
 tests/generic/334 |   20 ++++++++++++--------
 10 files changed, 64 insertions(+), 40 deletions(-)


diff --git a/tests/generic/162 b/tests/generic/162
index 5ff2ee2..9142d53 100755
--- a/tests/generic/162
+++ b/tests/generic/162
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -51,6 +51,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -64,7 +66,7 @@ _pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
 _scratch_cycle_mount
 
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x61 $((i * blksz)) $blksz $testdir/file2 >> $seqres.full
 		done
@@ -81,7 +83,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished dedupeing"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/163 b/tests/generic/163
index fcba677..ebf4831 100755
--- a/tests/generic/163
+++ b/tests/generic/163
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -51,6 +51,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -64,7 +66,7 @@ _pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
 _scratch_cycle_mount
 
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x61 $((i * blksz)) $blksz $testdir/file1 >> $seqres.full
 		done
@@ -81,7 +83,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished dedupeing"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/164 b/tests/generic/164
index acacb7f..76f419f 100755
--- a/tests/generic/164
+++ b/tests/generic/164
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -71,7 +73,7 @@ fbytes() {
 }
 
 reader() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		_read_range $testdir/file3 0 $((loops * blksz)) | fbytes
 	done
 }
@@ -91,7 +93,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished reflinking"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/165 b/tests/generic/165
index b742d19..326127b 100755
--- a/tests/generic/165
+++ b/tests/generic/165
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -54,6 +54,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -72,7 +74,7 @@ fbytes() {
 }
 
 reader() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		_read_range $testdir/file3 0 $((loops * blksz)) -d | fbytes
 	done
 }
@@ -92,7 +94,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished reflinking"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/166 b/tests/generic/166
index 0e31f2e..f4d7210 100755
--- a/tests/generic/166
+++ b/tests/generic/166
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -54,6 +54,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -68,7 +70,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo "$out" | grep -q "No space left" && break
@@ -83,7 +85,7 @@ snappy &
 seq $nr_loops -1 0 | while read i; do
 	_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file1 >> $seqres.full
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/167 b/tests/generic/167
index b2d6c87..0db2109 100755
--- a/tests/generic/167
+++ b/tests/generic/167
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +69,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo "$out" | grep -q "No space left" && break
@@ -82,7 +84,7 @@ snappy &
 seq $nr_loops -1 0 | while read i; do
 	_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file1 >> $seqres.full
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/168 b/tests/generic/168
index 7047d10..9d95261 100755
--- a/tests/generic/168
+++ b/tests/generic/168
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -52,6 +52,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -66,7 +68,7 @@ _scratch_cycle_mount
 
 # Direct I/O overwriter...
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file2 >> $seqres.full
 		done
@@ -82,7 +84,7 @@ seq 1 10 | while read j; do
 		[ $? -ne 0 ] && exit
 	done
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/170 b/tests/generic/170
index 4861052..742f03a 100755
--- a/tests/generic/170
+++ b/tests/generic/170
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +69,7 @@ _scratch_cycle_mount
 
 # Direct I/O overwriter...
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file2 >> $seqres.full
 		done
@@ -83,7 +85,7 @@ seq 1 10 | while read j; do
 		[ $? -ne 0 ] && exit
 	done
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/333 b/tests/generic/333
index 8db86e3..3e7b600 100755
--- a/tests/generic/333
+++ b/tests/generic/333
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file $abort_file
     wait
 }
 
@@ -54,6 +54,10 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
+abort_file=$TEST_DIR/abort
+rm -rf $abort_file
 mkdir $testdir
 
 loops=1024
@@ -68,7 +72,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo $out | grep -q "No space left" && break
@@ -76,7 +80,7 @@ snappy() {
 		test $res -ne 0 && break
 		n=$((n + 1))
 	done
-	touch $testdir/abort
+	touch $abort_file
 }
 
 echo "Snapshot a file undergoing directio rewrite"
@@ -85,13 +89,13 @@ seq 1 1000 | while read j; do
 	seq $nr_loops -1 0 | while read i; do
 		out="$(_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file1 2>&1)"
 		echo $out >> $seqres.full
-		echo $out | grep -q "No space left" && touch $testdir/abort
-		echo $out | grep -qi "error" && touch $testdir/abort
-		test -e $testdir/abort && break
+		echo $out | grep -q "No space left" && touch $abort_file
+		echo $out | grep -qi "error" && touch $abort_file
+		test -e $abort_file && break
 	done
-	test -e $testdir/abort && break
+	test -e $abort_file && break
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/334 b/tests/generic/334
index 5714125..e45502a 100755
--- a/tests/generic/334
+++ b/tests/generic/334
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file $abort_file
     wait
 }
 
@@ -53,6 +53,10 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=$TEST_DIR/finished
+rm -rf $finished_file
+abort_file=$TEST_DIR/abort
+rm -rf $abort_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +71,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo $out | grep -q "No space left" && break
@@ -75,7 +79,7 @@ snappy() {
 		test $res -ne 0 && break
 		n=$((n + 1))
 	done
-	touch $testdir/abort
+	touch $abort_file
 }
 
 echo "Snapshot a file undergoing buffered rewrite"
@@ -84,13 +88,13 @@ seq 1 1000 | while read j; do
 	seq $nr_loops -1 0 | while read i; do
 		out="$(_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file1 2>&1)"
 		echo $out >> $seqres.full
-		echo $out | grep -q "No space left" && touch $testdir/abort
-		echo $out | grep -qi "error" && touch $testdir/abort
-		test -e $testdir/abort && break
+		echo $out | grep -q "No space left" && touch $abort_file
+		echo $out | grep -qi "error" && touch $abort_file
+		test -e $abort_file && break
 	done
-	test -e $testdir/abort && break
+	test -e $abort_file && break
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done

Re: [PATCH 2/7] tests: don't put loop control files on the scratch mount

[Christoph Hellwig]

Shouldn't these tests also add a _require_test now?  But we probably
should just move them to /tmp instead?

Re: [PATCH 2/7] tests: don't put loop control files on the scratch mount

[Darrick J. Wong]

On Thu, May 26, 2016 at 01:09:48AM -0700, Christoph Hellwig wrote:
> Shouldn't these tests also add a _require_test now?  But we probably
> should just move them to /tmp instead?

Yes.  I'll move 'em to /tmp.

--D

[PATCH v2 2/7] tests: don't put loop control files on the scratch mount

[Darrick J. Wong]

If we're doing write/overwrite/snapshot/resource exhaustion tests on
the scratch device, use the test directory to hold the loop
termination signal files.  This way we don't run infinitely because we
can't create the flag due to ENOSPC.

v2: put the control files in /tmp, not $TEST_DIR

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/162 |    8 +++++---
 tests/generic/163 |    8 +++++---
 tests/generic/164 |    8 +++++---
 tests/generic/165 |    8 +++++---
 tests/generic/166 |    8 +++++---
 tests/generic/167 |    8 +++++---
 tests/generic/168 |    8 +++++---
 tests/generic/170 |    8 +++++---
 tests/generic/333 |   20 ++++++++++++--------
 tests/generic/334 |   20 ++++++++++++--------
 10 files changed, 64 insertions(+), 40 deletions(-)

diff --git a/tests/generic/162 b/tests/generic/162
index 5ff2ee2..61cf463 100755
--- a/tests/generic/162
+++ b/tests/generic/162
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -51,6 +51,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -64,7 +66,7 @@ _pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
 _scratch_cycle_mount
 
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x61 $((i * blksz)) $blksz $testdir/file2 >> $seqres.full
 		done
@@ -81,7 +83,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished dedupeing"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/163 b/tests/generic/163
index fcba677..e2c6042 100755
--- a/tests/generic/163
+++ b/tests/generic/163
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -51,6 +51,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -64,7 +66,7 @@ _pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
 _scratch_cycle_mount
 
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x61 $((i * blksz)) $blksz $testdir/file1 >> $seqres.full
 		done
@@ -81,7 +83,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished dedupeing"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/164 b/tests/generic/164
index acacb7f..78ea787 100755
--- a/tests/generic/164
+++ b/tests/generic/164
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -71,7 +73,7 @@ fbytes() {
 }
 
 reader() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		_read_range $testdir/file3 0 $((loops * blksz)) | fbytes
 	done
 }
@@ -91,7 +93,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished reflinking"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/165 b/tests/generic/165
index b742d19..ca11e1e 100755
--- a/tests/generic/165
+++ b/tests/generic/165
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -54,6 +54,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=512
@@ -72,7 +74,7 @@ fbytes() {
 }
 
 reader() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		_read_range $testdir/file3 0 $((loops * blksz)) -d | fbytes
 	done
 }
@@ -92,7 +94,7 @@ for i in `seq 1 2`; do
 	done
 done
 echo "Finished reflinking"
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/166 b/tests/generic/166
index 0e31f2e..8600a13 100755
--- a/tests/generic/166
+++ b/tests/generic/166
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -54,6 +54,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -68,7 +70,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo "$out" | grep -q "No space left" && break
@@ -83,7 +85,7 @@ snappy &
 seq $nr_loops -1 0 | while read i; do
 	_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file1 >> $seqres.full
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/167 b/tests/generic/167
index b2d6c87..261506b 100755
--- a/tests/generic/167
+++ b/tests/generic/167
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +69,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo "$out" | grep -q "No space left" && break
@@ -82,7 +84,7 @@ snappy &
 seq $nr_loops -1 0 | while read i; do
 	_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file1 >> $seqres.full
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/168 b/tests/generic/168
index 7047d10..ceee264 100755
--- a/tests/generic/168
+++ b/tests/generic/168
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -52,6 +52,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -66,7 +68,7 @@ _scratch_cycle_mount
 
 # Direct I/O overwriter...
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file2 >> $seqres.full
 		done
@@ -82,7 +84,7 @@ seq 1 10 | while read j; do
 		[ $? -ne 0 ] && exit
 	done
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/170 b/tests/generic/170
index 4861052..6559243 100755
--- a/tests/generic/170
+++ b/tests/generic/170
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file
     wait
 }
 
@@ -53,6 +53,8 @@ _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +69,7 @@ _scratch_cycle_mount
 
 # Direct I/O overwriter...
 overwrite() {
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		seq $nr_loops -1 0 | while read i; do
 			_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file2 >> $seqres.full
 		done
@@ -83,7 +85,7 @@ seq 1 10 | while read j; do
 		[ $? -ne 0 ] && exit
 	done
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/333 b/tests/generic/333
index 8db86e3..3d874d7 100755
--- a/tests/generic/333
+++ b/tests/generic/333
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file $abort_file
     wait
 }
 
@@ -54,6 +54,10 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
+abort_file=/tmp/abort
+rm -rf $abort_file
 mkdir $testdir
 
 loops=1024
@@ -68,7 +72,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo $out | grep -q "No space left" && break
@@ -76,7 +80,7 @@ snappy() {
 		test $res -ne 0 && break
 		n=$((n + 1))
 	done
-	touch $testdir/abort
+	touch $abort_file
 }
 
 echo "Snapshot a file undergoing directio rewrite"
@@ -85,13 +89,13 @@ seq 1 1000 | while read j; do
 	seq $nr_loops -1 0 | while read i; do
 		out="$(_pwrite_byte 0x63 $((i * blksz)) $blksz -d $testdir/file1 2>&1)"
 		echo $out >> $seqres.full
-		echo $out | grep -q "No space left" && touch $testdir/abort
-		echo $out | grep -qi "error" && touch $testdir/abort
-		test -e $testdir/abort && break
+		echo $out | grep -q "No space left" && touch $abort_file
+		echo $out | grep -qi "error" && touch $abort_file
+		test -e $abort_file && break
 	done
-	test -e $testdir/abort && break
+	test -e $abort_file && break
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done
diff --git a/tests/generic/334 b/tests/generic/334
index 5714125..fb21d4f 100755
--- a/tests/generic/334
+++ b/tests/generic/334
@@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 7 15
 _cleanup()
 {
     cd /
-    rm -rf $tmp.*
+    rm -rf $tmp.* $finished_file $abort_file
     wait
 }
 
@@ -53,6 +53,10 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
 testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
+abort_file=/tmp/abort
+rm -rf $abort_file
 mkdir $testdir
 
 loops=1024
@@ -67,7 +71,7 @@ _scratch_cycle_mount
 # Snapshot creator...
 snappy() {
 	n=0
-	while [ ! -e $testdir/finished ]; do
+	while [ ! -e $finished_file ]; do
 		out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
 		res=$?
 		echo $out | grep -q "No space left" && break
@@ -75,7 +79,7 @@ snappy() {
 		test $res -ne 0 && break
 		n=$((n + 1))
 	done
-	touch $testdir/abort
+	touch $abort_file
 }
 
 echo "Snapshot a file undergoing buffered rewrite"
@@ -84,13 +88,13 @@ seq 1 1000 | while read j; do
 	seq $nr_loops -1 0 | while read i; do
 		out="$(_pwrite_byte 0x63 $((i * blksz)) $blksz $testdir/file1 2>&1)"
 		echo $out >> $seqres.full
-		echo $out | grep -q "No space left" && touch $testdir/abort
-		echo $out | grep -qi "error" && touch $testdir/abort
-		test -e $testdir/abort && break
+		echo $out | grep -q "No space left" && touch $abort_file
+		echo $out | grep -qi "error" && touch $abort_file
+		test -e $abort_file && break
 	done
-	test -e $testdir/abort && break
+	test -e $abort_file && break
 done
-touch $testdir/finished
+touch $finished_file
 wait
 
 # success, all done

Re: [PATCH v2 2/7] tests: don't put loop control files on the scratch mount

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 3/7] xfs: test copy-on-write leftover recovery

[Darrick J. Wong]

Test recovery of CoW leftovers in xfs_repair.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/853     |  179 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/853.out |   13 ++++
 tests/xfs/854     |  180 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/854.out |   13 ++++
 tests/xfs/group   |    2 +
 5 files changed, 387 insertions(+)
 create mode 100755 tests/xfs/853
 create mode 100644 tests/xfs/853.out
 create mode 100755 tests/xfs/854
 create mode 100644 tests/xfs/854.out


diff --git a/tests/xfs/853 b/tests/xfs/853
new file mode 100755
index 0000000..14a6940
--- /dev/null
+++ b/tests/xfs/853
@@ -0,0 +1,179 @@
+#! /bin/bash
+# FS QA Test No. 853
+#
+# Test recovery of "lost" CoW blocks:
+# - Use the debugger to fake a leftover CoW extent
+# - See if xfs_repair fixes it
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch_reflink
+
+rm -f $seqres.full
+
+echo "Format"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full
+is_rmap=$(xfs_info $SCRATCH_MNT | grep -c "rmapbt=1")
+umount $SCRATCH_MNT
+
+_get_agf_data() {
+	field="$1"
+	shift
+
+	xfs_db -c 'agf 1' "$@" -c "p $field" $SCRATCH_DEV | awk '{print $3}'
+}
+
+_set_agf_data() {
+	field="$1"
+	value="$2"
+	shift; shift
+
+	xfs_db -x -c 'agf 1' "$@" -c "write $field -- $value" $SCRATCH_DEV >> $seqres.full
+}
+
+_get_sb_data() {
+	field="$1"
+	shift
+
+	xfs_db -c 'sb 0' "$@" -c "p $field" $SCRATCH_DEV | awk '{print $3}'
+}
+
+_set_sb_data() {
+	field="$1"
+	value="$2"
+	shift; shift
+
+	xfs_db -x -c 'sb 0' "$@" -c "write $field -- $value" $SCRATCH_DEV >> $seqres.full
+}
+
+_filter_leftover() {
+	grep "^leftover" | sed -e "s/[0-9]\+/NR/g"
+}
+
+_dump_status() {
+	echo "** " "$@"
+	xfs_db -c 'sb 0' -c p $SCRATCH_DEV
+	echo "** agf header"
+	xfs_db -c 'agf 1' -c p $SCRATCH_DEV
+	echo "** refcntbt"
+	xfs_db -c 'agf 1' -c 'addr refcntroot' -c p $SCRATCH_DEV
+	echo "** rmapbt"
+	test $is_rmap -gt 0 && xfs_db -c 'agf 1' -c 'addr rmaproot' -c p $SCRATCH_DEV
+	echo "** bnobt"
+	xfs_db -c 'agf 1' -c 'addr bnoroot' -c p $SCRATCH_DEV
+	echo "** cntbt"
+	xfs_db -c 'agf 1' -c 'addr cntroot' -c p $SCRATCH_DEV
+}
+
+echo "We need AG1 to have a single free extent"
+bno_lvl=$(_get_agf_data level -c 'addr bnoroot')
+bno_nr=$(_get_agf_data numrecs -c 'addr bnoroot')
+refc_lvl=$(_get_agf_data level -c 'addr refcntroot')
+refc_nr=$(_get_agf_data numrecs -c 'addr refcntroot')
+
+test $bno_lvl -eq 0 || echo "  AG 1 bnobt must only have one level"
+test $bno_nr -eq 1 || echo "  AG 1 bnobt must only have one record"
+test $refc_lvl -eq 0 || echo "  AG 1 refcountbt must only have one level"
+test $refc_nr -eq 0 || echo "  AG 1 refcountbt must only have one record"
+
+if [ $is_rmap -gt 0 ]; then
+	rmap_lvl=$(_get_agf_data level -c 'addr rmaproot')
+	rmap_nr=$(_get_agf_data numrecs -c 'addr rmaproot')
+	test $rmap_lvl -eq 0 || echo "  AG 1 rmapbt must only have one level"
+fi
+
+echo "Find our extent and old counter values"
+bno=$(_get_agf_data "recs[1].startblock" -c 'addr bnoroot')
+len=$(_get_agf_data "recs[1].blockcount" -c 'addr bnoroot')
+agf_freeblks=$(_get_agf_data freeblks)
+sb_fdblocks=$(_get_sb_data fdblocks)
+
+test $len -ge 200 || echo "  AG 1 doesn't have enough free blocks"
+
+# Take the last 100 blocks of the free extent
+debris_len=100
+debris_bno=$((bno + len - debris_len))
+
+echo "Remove the extent from the freesp btrees"
+_set_agf_data "recs[1].blockcount" $((len - debris_len)) -c 'addr bnoroot'
+_set_agf_data "recs[1].blockcount" $((len - debris_len)) -c 'addr cntroot'
+_set_agf_data freeblks $((agf_freeblks - debris_len))
+_set_agf_data longest $((len - debris_len))
+_set_sb_data fdblocks $((sb_fdblocks - debris_len))
+
+echo "Add the extent to the refcount btree"
+_set_agf_data numrecs 1 -c 'addr refcntroot'
+_set_agf_data "recs[1].startblock" $debris_bno -c 'addr refcntroot'
+_set_agf_data "recs[1].blockcount" $debris_len -c 'addr refcntroot'
+_set_agf_data "recs[1].refcount" 1 -c 'addr refcntroot'
+
+if [ $is_rmap -gt 0 ]; then
+	rmap_nr=$((rmap_nr + 1))
+	_set_agf_data numrecs $rmap_nr -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].startblock" $debris_bno -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].blockcount" $debris_len -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].owner" -9 -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].offset" 0 -c 'addr rmaproot'
+fi
+
+_dump_status "broken fs config" >> $seqres.full
+
+echo "Look for leftover warning in xfs_check"
+_scratch_xfs_check | _filter_leftover
+
+echo "Look for leftover warning in xfs_repair"
+_scratch_xfs_repair -n 2>&1 | _filter_leftover
+
+echo "Fix filesystem"
+_scratch_xfs_repair >> $seqres.full 2>&1 || echo "xfs_repair failed?"
+
+_dump_status "supposedly fixed fs config" >> $seqres.full
+
+echo "Look for no more leftover warning in xfs_check"
+_scratch_xfs_check | _filter_leftover
+
+echo "Look for no more leftover warning in xfs_repair"
+_scratch_xfs_repair -n 2>&1 | _filter_leftover
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/853.out b/tests/xfs/853.out
new file mode 100644
index 0000000..3224e8d
--- /dev/null
+++ b/tests/xfs/853.out
@@ -0,0 +1,13 @@
+QA output created by 853
+Format
+We need AG1 to have a single free extent
+Find our extent and old counter values
+Remove the extent from the freesp btrees
+Add the extent to the refcount btree
+Look for leftover warning in xfs_check
+leftover CoW extent (NR/NR) len NR
+Look for leftover warning in xfs_repair
+leftover CoW extent (NR/NR) len NR
+Fix filesystem
+Look for no more leftover warning in xfs_check
+Look for no more leftover warning in xfs_repair
diff --git a/tests/xfs/854 b/tests/xfs/854
new file mode 100755
index 0000000..5c6bec9
--- /dev/null
+++ b/tests/xfs/854
@@ -0,0 +1,180 @@
+#! /bin/bash
+# FS QA Test No. 854
+#
+# Test recovery of "lost" CoW blocks:
+# - Use the debugger to fake a leftover CoW extent
+# - See if mount/umount fixes it
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch_reflink
+
+rm -f $seqres.full
+
+echo "Format"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full
+is_rmap=$(xfs_info $SCRATCH_MNT | grep -c "rmapbt=1")
+umount $SCRATCH_MNT
+
+_get_agf_data() {
+	field="$1"
+	shift
+
+	xfs_db -c 'agf 1' "$@" -c "p $field" $SCRATCH_DEV | awk '{print $3}'
+}
+
+_set_agf_data() {
+	field="$1"
+	value="$2"
+	shift; shift
+
+	xfs_db -x -c 'agf 1' "$@" -c "write $field -- $value" $SCRATCH_DEV >> $seqres.full
+}
+
+_get_sb_data() {
+	field="$1"
+	shift
+
+	xfs_db -c 'sb 0' "$@" -c "p $field" $SCRATCH_DEV | awk '{print $3}'
+}
+
+_set_sb_data() {
+	field="$1"
+	value="$2"
+	shift; shift
+
+	xfs_db -x -c 'sb 0' "$@" -c "write $field -- $value" $SCRATCH_DEV >> $seqres.full
+}
+
+_filter_leftover() {
+	grep "^leftover" | sed -e "s/[0-9]\+/NR/g"
+}
+
+_dump_status() {
+	echo "** " "$@"
+	xfs_db -c 'sb 0' -c p $SCRATCH_DEV
+	echo "** agf header"
+	xfs_db -c 'agf 1' -c p $SCRATCH_DEV
+	echo "** refcntbt"
+	xfs_db -c 'agf 1' -c 'addr refcntroot' -c p $SCRATCH_DEV
+	echo "** rmapbt"
+	test $is_rmap -gt 0 && xfs_db -c 'agf 1' -c 'addr rmaproot' -c p $SCRATCH_DEV
+	echo "** bnobt"
+	xfs_db -c 'agf 1' -c 'addr bnoroot' -c p $SCRATCH_DEV
+	echo "** cntbt"
+	xfs_db -c 'agf 1' -c 'addr cntroot' -c p $SCRATCH_DEV
+}
+
+echo "We need AG1 to have a single free extent"
+bno_lvl=$(_get_agf_data level -c 'addr bnoroot')
+bno_nr=$(_get_agf_data numrecs -c 'addr bnoroot')
+refc_lvl=$(_get_agf_data level -c 'addr refcntroot')
+refc_nr=$(_get_agf_data numrecs -c 'addr refcntroot')
+
+test $bno_lvl -eq 0 || echo "  AG 1 bnobt must only have one level"
+test $bno_nr -eq 1 || echo "  AG 1 bnobt must only have one record"
+test $refc_lvl -eq 0 || echo "  AG 1 refcountbt must only have one level"
+test $refc_nr -eq 0 || echo "  AG 1 refcountbt must only have one record"
+
+if [ $is_rmap -gt 0 ]; then
+	rmap_lvl=$(_get_agf_data level -c 'addr rmaproot')
+	rmap_nr=$(_get_agf_data numrecs -c 'addr rmaproot')
+	test $rmap_lvl -eq 0 || echo "  AG 1 rmapbt must only have one level"
+fi
+
+echo "Find our extent and old counter values"
+bno=$(_get_agf_data "recs[1].startblock" -c 'addr bnoroot')
+len=$(_get_agf_data "recs[1].blockcount" -c 'addr bnoroot')
+agf_freeblks=$(_get_agf_data freeblks)
+sb_fdblocks=$(_get_sb_data fdblocks)
+
+test $len -ge 200 || echo "  AG 1 doesn't have enough free blocks"
+
+# Take the last 100 blocks of the free extent
+debris_len=100
+debris_bno=$((bno + len - debris_len))
+
+echo "Remove the extent from the freesp btrees"
+_set_agf_data "recs[1].blockcount" $((len - debris_len)) -c 'addr bnoroot'
+_set_agf_data "recs[1].blockcount" $((len - debris_len)) -c 'addr cntroot'
+_set_agf_data freeblks $((agf_freeblks - debris_len))
+_set_agf_data longest $((len - debris_len))
+_set_sb_data fdblocks $((sb_fdblocks - debris_len))
+
+echo "Add the extent to the refcount btree"
+_set_agf_data numrecs 1 -c 'addr refcntroot'
+_set_agf_data "recs[1].startblock" $debris_bno -c 'addr refcntroot'
+_set_agf_data "recs[1].blockcount" $debris_len -c 'addr refcntroot'
+_set_agf_data "recs[1].refcount" 1 -c 'addr refcntroot'
+
+if [ $is_rmap -gt 0 ]; then
+	rmap_nr=$((rmap_nr + 1))
+	_set_agf_data numrecs $rmap_nr -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].startblock" $debris_bno -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].blockcount" $debris_len -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].owner" -9 -c 'addr rmaproot'
+	_set_agf_data "recs[$rmap_nr].offset" 0 -c 'addr rmaproot'
+fi
+
+_dump_status "broken fs config" >> $seqres.full
+
+echo "Look for leftover warning in xfs_check"
+_scratch_xfs_check | _filter_leftover
+
+echo "Look for leftover warning in xfs_repair"
+_scratch_xfs_repair -n 2>&1 | _filter_leftover
+
+echo "Mount filesystem"
+_scratch_mount
+umount $SCRATCH_MNT
+
+_dump_status "supposedly fixed fs config" >> $seqres.full
+
+echo "Look for no more leftover warning in xfs_check"
+_scratch_xfs_check | _filter_leftover
+
+echo "Look for no more leftover warning in xfs_repair"
+_scratch_xfs_repair -n 2>&1 | _filter_leftover
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/854.out b/tests/xfs/854.out
new file mode 100644
index 0000000..aae2fd7
--- /dev/null
+++ b/tests/xfs/854.out
@@ -0,0 +1,13 @@
+QA output created by 854
+Format
+We need AG1 to have a single free extent
+Find our extent and old counter values
+Remove the extent from the freesp btrees
+Add the extent to the refcount btree
+Look for leftover warning in xfs_check
+leftover CoW extent (NR/NR) len NR
+Look for leftover warning in xfs_repair
+leftover CoW extent (NR/NR) len NR
+Mount filesystem
+Look for no more leftover warning in xfs_check
+Look for no more leftover warning in xfs_repair
diff --git a/tests/xfs/group b/tests/xfs/group
index f4c6816..4ac62c1 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -285,3 +285,5 @@
 303 auto quick quota
 304 auto quick quota
 305 auto quota
+853 auto quick clone
+854 auto quick clone

Re: [PATCH 3/7] xfs: test copy-on-write leftover recovery

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 4/7] xfs: test per-ag allocation accounting during truncate-caused refcountbt expansion

[Darrick J. Wong]

Ensure that refcountbt allocations during truncate operations come
from the per-AG reservation and are not charged to the transaction.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/855     |   95 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/855.out |   10 ++++++
 tests/xfs/group   |    1 +
 3 files changed, 106 insertions(+)
 create mode 100755 tests/xfs/855
 create mode 100644 tests/xfs/855.out


diff --git a/tests/xfs/855 b/tests/xfs/855
new file mode 100755
index 0000000..93bd286
--- /dev/null
+++ b/tests/xfs/855
@@ -0,0 +1,95 @@
+#! /bin/bash
+# FS QA Test No. 855
+#
+# Ensure that we can create enough distinct reflink entries to force creation
+# of a multi-level refcount btree by reflinking a file a number of times and
+# truncating the copies at successively lower sizes.  Delete and recreate a few
+# times to exercise the refcount btree grow/shrink functions.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    umount $SCRATCH_MNT > /dev/null 2>&1
+    rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch_reflink
+_require_cp_reflink
+
+rm -f $seqres.full
+
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original file blocks"
+blksz="$(stat -f $testdir -c '%S')"
+nr_blks=$((2 * blksz / 12))
+
+for i in 1 2 x; do
+	_pwrite_byte 0x61 0 $((blksz * nr_blks)) $testdir/file1 >> $seqres.full
+
+	echo "$i: Reflink a bunch of times"
+	seq 1 $nr_blks | while read nr; do
+		_cp_reflink $testdir/file1 $testdir/file1.$nr >> $seqres.full
+	done
+	sync
+
+	echo "$i: Truncate files"
+	seq 1 $nr_blks | while read nr; do
+		truncate -s $((blksz * (nr_blks - nr))) $testdir/file1.$nr >> $seqres.full
+	done
+
+	umount $SCRATCH_MNT
+	_check_scratch_fs
+	_scratch_mount
+
+	test $i = "x" && break
+
+	echo "$i: Delete both files"
+	rm -rf $testdir
+	mkdir -p $testdir
+	umount $SCRATCH_MNT
+	_check_scratch_fs
+	_scratch_mount
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/855.out b/tests/xfs/855.out
new file mode 100644
index 0000000..d196352
--- /dev/null
+++ b/tests/xfs/855.out
@@ -0,0 +1,10 @@
+QA output created by 855
+Create the original file blocks
+1: Reflink a bunch of times
+1: Truncate files
+1: Delete both files
+2: Reflink a bunch of times
+2: Truncate files
+2: Delete both files
+x: Reflink a bunch of times
+x: Truncate files
diff --git a/tests/xfs/group b/tests/xfs/group
index 4ac62c1..f0ca410 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -287,3 +287,4 @@
 305 auto quota
 853 auto quick clone
 854 auto quick clone
+855 auto clone

Re: [PATCH 4/7] xfs: test per-ag allocation accounting during truncate-caused refcountbt expansion

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 5/7] reflink: test interaction with swap files

[Darrick J. Wong]

Since none of the current filesystems support reflinked swap files,
make sure that we prohibit reflinking of swapfiles and swapon of
reflinked files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
---
 tests/generic/923     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/923.out |    6 ++++
 tests/generic/924     |   73 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/924.out |    5 +++
 tests/generic/group   |    2 +
 5 files changed, 160 insertions(+)
 create mode 100755 tests/generic/923
 create mode 100644 tests/generic/923.out
 create mode 100755 tests/generic/924
 create mode 100644 tests/generic/924.out


diff --git a/tests/generic/923 b/tests/generic/923
new file mode 100755
index 0000000..73c1621
--- /dev/null
+++ b/tests/generic/923
@@ -0,0 +1,74 @@
+#! /bin/bash
+# FS QA Test No. 923
+#
+# Check that we can't reflink a swapfile.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Oracle, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 7 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+    wait
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_require_scratch_reflink
+_require_cp_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blocks=160
+blksz=65536
+
+echo "Initialize file"
+echo > $seqres.full
+_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file1 >> $seqres.full
+mkswap -U 27376b42-ff65-42ca-919f-6c9b62292a5c $testdir/file1 >> $seqres.full
+swapon $testdir/file1
+
+echo "Try to reflink"
+_cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch
+
+echo "Tear it down"
+swapoff $testdir/file1
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/923.out b/tests/generic/923.out
new file mode 100644
index 0000000..97c18cf
--- /dev/null
+++ b/tests/generic/923.out
@@ -0,0 +1,6 @@
+QA output created by 923
+Format and mount
+Initialize file
+Try to reflink
+cp: failed to clone 'SCRATCH_MNT/test-923/file2' from 'SCRATCH_MNT/test-923/file1': Text file busy
+Tear it down
diff --git a/tests/generic/924 b/tests/generic/924
new file mode 100755
index 0000000..86c2a39
--- /dev/null
+++ b/tests/generic/924
@@ -0,0 +1,73 @@
+#! /bin/bash
+# FS QA Test No. 924
+#
+# Check that we can't swapon a reflinked file.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Oracle, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 7 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+    wait
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_require_scratch_reflink
+_require_cp_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blocks=160
+blksz=65536
+
+echo "Initialize file"
+echo > $seqres.full
+_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file1 >> $seqres.full
+mkswap -U 27376b42-ff65-42ca-919f-6c9b62292a5c $testdir/file1 >> $seqres.full
+_cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch
+
+echo "Try to swapon"
+swapon $testdir/file1 2>&1 | _filter_scratch
+
+swapoff $testdir/file1 >> $seqres.full 2>&1
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/924.out b/tests/generic/924.out
new file mode 100644
index 0000000..b67e6fa
--- /dev/null
+++ b/tests/generic/924.out
@@ -0,0 +1,5 @@
+QA output created by 924
+Format and mount
+Initialize file
+Try to swapon
+swapon: SCRATCH_MNT/test-924/file1: swapon failed: Invalid argument
diff --git a/tests/generic/group b/tests/generic/group
index 36fb759..6fdf109 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -354,3 +354,5 @@
 349 blockdev quick rw
 350 blockdev quick rw
 351 blockdev quick rw
+923 auto quick clone
+924 auto quick clone

Re: [PATCH 5/7] reflink: test interaction with swap files

[Christoph Hellwig]

On Wed, May 25, 2016 at 10:57:36PM -0700, Darrick J. Wong wrote:
> Since none of the current filesystems support reflinked swap files,
> make sure that we prohibit reflinking of swapfiles and swapon of
> reflinked files.

Ah, thanks.  I've actually prepared a patch to fix swapon in the kernel
and was about to start writing test cases for it.  I hope you haven't
also done the kernel work already :)

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 6/7] xfs: test rmap behavior when multiple bmbt records map to a single rmapbt record

[Darrick J. Wong]

Make sure that we can handle multiple bmbt records mapping to a
single rmapbt record.  This can happen if you fallocate more than
2^21 contiguous blocks to a file.

(Also add some helpers that can create huge devices with some dm-zero
and dm-snapshot fakery.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/dmhugedisk |   61 +++++++++++++++++++++++++++
 src/t_immutable.c |    2 -
 tests/xfs/856     |  119 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/856.out |   13 ++++++
 tests/xfs/group   |    1 
 5 files changed, 195 insertions(+), 1 deletion(-)
 create mode 100644 common/dmhugedisk
 create mode 100755 tests/xfs/856
 create mode 100644 tests/xfs/856.out


diff --git a/common/dmhugedisk b/common/dmhugedisk
new file mode 100644
index 0000000..9c3c7d5
--- /dev/null
+++ b/common/dmhugedisk
@@ -0,0 +1,61 @@
+##/bin/bash
+# Routines for creating huge (fake) disks
+#-----------------------------------------------------------------------
+#  Copyright (c) 2016 Oracle.  All Rights Reserved.
+#  This program is free software; you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation; either version 2 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the Free Software
+#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307
+#  USA
+#
+#  Contact information: Oracle Corporation, 500 Oracle Parkway,
+#  Redwood Shores, CA 94065, USA, or: http://www.oracle.com/
+#-----------------------------------------------------------------------
+
+_require_dmhugedisk()
+{
+	_require_dm_target zero
+	_require_dm_target snapshot
+}
+
+_dmhugedisk_init()
+{
+	test -z "$1" && _fatal "must specify sector count to _dmhugedisk_init"
+	local dm_backing_dev=$SCRATCH_DEV
+
+	$DMSETUP_PROG remove huge-test > /dev/null 2>&1
+	$DMSETUP_PROG remove huge-test-zero > /dev/null 2>&1
+
+	local blk_dev_size=$1
+
+	DMHUGEDISK_ZERO='/dev/mapper/huge-test-zero'
+	DMHUGEDISK_DEV='/dev/mapper/huge-test'
+
+	DMHUGEDISK_ZERO_TABLE="0 $blk_dev_size zero"
+	DMHUGEDISK_DEV_TABLE="0 $blk_dev_size snapshot $DMHUGEDISK_ZERO $SCRATCH_DEV N 512"
+
+	$DMSETUP_PROG create huge-test-zero --table "$DMHUGEDISK_ZERO_TABLE" || \
+		_fatal "failed to create dm huge zero device"
+	$DMSETUP_PROG create huge-test --table "$DMHUGEDISK_DEV_TABLE" || \
+		_fatal "failed to create dm huge device"
+}
+
+_dmhugedisk_cleanup()
+{
+	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
+	# wait for device to be fully settled so that 'dmsetup remove' doesn't
+	# fail due to EBUSY
+	$UDEV_SETTLE_PROG >/dev/null 2>&1
+	$DMSETUP_PROG remove huge-test > /dev/null 2>&1
+	$DMSETUP_PROG remove huge-test-zero > /dev/null 2>&1
+}
+
diff --git a/src/t_immutable.c b/src/t_immutable.c
index 87ffc75..90b62fb 100644
--- a/src/t_immutable.c
+++ b/src/t_immutable.c
@@ -38,10 +38,10 @@
 #include <libgen.h>
 #include <sys/acl.h>
 #include <attr/xattr.h>
+#include <linux/fs.h>
 #include <xfs/xfs.h>
 #include <xfs/handle.h>
 #include <xfs/jdm.h>
-#include <linux/fs.h>
 #include <linux/magic.h>
 
 #ifndef XFS_SUPER_MAGIC
diff --git a/tests/xfs/856 b/tests/xfs/856
new file mode 100755
index 0000000..9186278
--- /dev/null
+++ b/tests/xfs/856
@@ -0,0 +1,119 @@
+#! /bin/bash
+# FS QA Test No. 856
+#
+# Create a file with more than 2^21 extents (the max length of a bmbt record).
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    umount $SCRATCH_MNT > /dev/null 2>&1
+    _dmhugedisk_cleanup
+    rm -rf $tmp.*
+    _scratch_mkfs >/dev/null 2>&1
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmhugedisk
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch
+_require_xfs_io_command "falloc"
+
+rm -f $seqres.full
+
+# Figure out block size
+echo "Figure out block size"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount >> $seqres.full
+
+is_rmap=$(xfs_info $SCRATCH_MNT | grep -c "rmapbt=1")
+test $is_rmap -gt 0 || _notrun "rmap not supported on scratch fs"
+
+testdir=$SCRATCH_MNT/test-$seq
+blksz="$(stat -f $SCRATCH_MNT -c '%S')"
+
+umount $SCRATCH_MNT
+
+echo "Format huge device"
+_dmhugedisk_init $((blksz * 2 * 4400))	# a little over 2^22 blocks
+_mkfs_dev -d agcount=2 $DMHUGEDISK_DEV
+_mount $DMHUGEDISK_DEV $SCRATCH_MNT 
+xfs_info $SCRATCH_MNT >> $seqres.full
+
+echo "Create the original file blocks"
+mkdir $testdir
+blksz="$(stat -f $testdir -c '%S')"
+nr_blks=2100000	# 2^21 plus a little more
+$XFS_IO_PROG -f -c "falloc 0 $((nr_blks * blksz))" $testdir/file1 >> $seqres.full
+
+echo "Check extent count"
+xfs_bmap -l -p -v $testdir/file1 | grep '^[[:space:]]*1:' -q && xfs_bmap -l -p -v $testdir/file1
+inum=$(stat -c '%i' $testdir/file1)
+umount $SCRATCH_MNT
+
+echo "Check bmap count"
+nr_bmaps=$(xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV | grep 'data offset' | wc -l)
+test $nr_bmaps -gt 1 || xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+#xfs_db -c "agf 0" -c p -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+
+echo "Check rmap count"
+nr_rmaps=$(xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0" | wc -l)
+test $nr_rmaps -eq 1 || xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0"
+
+echo "Check and fake-repair huge filesystem" | tee -a $seqres.full
+$XFS_DB_PROG -c 'check' $DMHUGEDISK_DEV
+$XFS_REPAIR_PROG -n $DMHUGEDISK_DEV >> $seqres.full 2>&1
+test $? -eq 0 || echo "xfs_repair -n failed, see $seqres.full"
+
+echo "Real repair huge filesystem" | tee -a $seqres.full
+$XFS_REPAIR_PROG $DMHUGEDISK_DEV >> $seqres.full 2>&1
+test $? -eq 0 || echo "xfs_repair failed, see $seqres.full"
+
+echo "Check bmap count again"
+nr_bmaps=$(xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV | grep 'data offset' | wc -l)
+test $nr_bmaps -gt 1 || xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+
+echo "Check rmap count again"
+nr_rmaps=$(xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0" | wc -l)
+test $nr_rmaps -eq 1 || xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0"
+
+echo "Check and fake-repair huge filesystem again" | tee -a $seqres.full
+$XFS_DB_PROG -c 'check' $DMHUGEDISK_DEV
+$XFS_REPAIR_PROG -n $DMHUGEDISK_DEV >> $seqres.full 2>&1
+_dmhugedisk_cleanup
+
+echo "Done"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/856.out b/tests/xfs/856.out
new file mode 100644
index 0000000..b24ebf2
--- /dev/null
+++ b/tests/xfs/856.out
@@ -0,0 +1,13 @@
+QA output created by 856
+Figure out block size
+Format huge device
+Create the original file blocks
+Check extent count
+Check bmap count
+Check rmap count
+Check and fake-repair huge filesystem
+Real repair huge filesystem
+Check bmap count again
+Check rmap count again
+Check and fake-repair huge filesystem again
+Done
diff --git a/tests/xfs/group b/tests/xfs/group
index f0ca410..0e34f9a 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -288,3 +288,4 @@
 853 auto quick clone
 854 auto quick clone
 855 auto clone
+856 auto clone rmap reflink

Re: [PATCH 6/7] xfs: test rmap behavior when multiple bmbt records map to a single rmapbt record

[Christoph Hellwig]

> --- a/src/t_immutable.c
> +++ b/src/t_immutable.c
> @@ -38,10 +38,10 @@
>  #include <libgen.h>
>  #include <sys/acl.h>
>  #include <attr/xattr.h>
> +#include <linux/fs.h>
>  #include <xfs/xfs.h>
>  #include <xfs/handle.h>
>  #include <xfs/jdm.h>
> -#include <linux/fs.h>
>  #include <linux/magic.h>
>  
>  #ifndef XFS_SUPER_MAGIC

How does this belong into the patch?

> diff --git a/tests/xfs/group b/tests/xfs/group
> index f0ca410..0e34f9a 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -288,3 +288,4 @@
>  853 auto quick clone
>  854 auto quick clone
>  855 auto clone
> +856 auto clone rmap reflink

This is the only test in the reflink group, how is this group different
from the clone one?

Re: [PATCH 6/7] xfs: test rmap behavior when multiple bmbt records map to a single rmapbt record

[Darrick J. Wong]

On Thu, May 26, 2016 at 01:04:24AM -0700, Christoph Hellwig wrote:
> > --- a/src/t_immutable.c
> > +++ b/src/t_immutable.c
> > @@ -38,10 +38,10 @@
> >  #include <libgen.h>
> >  #include <sys/acl.h>
> >  #include <attr/xattr.h>
> > +#include <linux/fs.h>
> >  #include <xfs/xfs.h>
> >  #include <xfs/handle.h>
> >  #include <xfs/jdm.h>
> > -#include <linux/fs.h>
> >  #include <linux/magic.h>
> >  
> >  #ifndef XFS_SUPER_MAGIC
> 
> How does this belong into the patch?

It doesn't, this somehow jumped patches when I rebased. :/

> > diff --git a/tests/xfs/group b/tests/xfs/group
> > index f0ca410..0e34f9a 100644
> > --- a/tests/xfs/group
> > +++ b/tests/xfs/group
> > @@ -288,3 +288,4 @@
> >  853 auto quick clone
> >  854 auto quick clone
> >  855 auto clone
> > +856 auto clone rmap reflink
> 
> This is the only test in the reflink group, how is this group different
> from the clone one?

I'm just gonna say 'thinko' and respin. :)

/me pours a larger cup of coffee...

(Thx for the review!)

--D

[PATCH v2 6/7] xfs: test rmap behavior when multiple bmbt records map to a single rmapbt record

[Darrick J. Wong]

Make sure that we can handle multiple bmbt records mapping to a
single rmapbt record.  This can happen if you fallocate more than
2^21 contiguous blocks to a file.

(Also add some helpers that can create huge devices with some dm-zero
and dm-snapshot fakery.)

v2: remove irrelevant t_immutable changes, put test in correct group

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/dmhugedisk |   61 +++++++++++++++++++++++++++
 tests/xfs/856     |  119 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/856.out |   13 ++++++
 tests/xfs/group   |    1 
 4 files changed, 194 insertions(+)
 create mode 100644 common/dmhugedisk
 create mode 100755 tests/xfs/856
 create mode 100644 tests/xfs/856.out

diff --git a/common/dmhugedisk b/common/dmhugedisk
new file mode 100644
index 0000000..9c3c7d5
--- /dev/null
+++ b/common/dmhugedisk
@@ -0,0 +1,61 @@
+##/bin/bash
+# Routines for creating huge (fake) disks
+#-----------------------------------------------------------------------
+#  Copyright (c) 2016 Oracle.  All Rights Reserved.
+#  This program is free software; you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation; either version 2 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the Free Software
+#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307
+#  USA
+#
+#  Contact information: Oracle Corporation, 500 Oracle Parkway,
+#  Redwood Shores, CA 94065, USA, or: http://www.oracle.com/
+#-----------------------------------------------------------------------
+
+_require_dmhugedisk()
+{
+	_require_dm_target zero
+	_require_dm_target snapshot
+}
+
+_dmhugedisk_init()
+{
+	test -z "$1" && _fatal "must specify sector count to _dmhugedisk_init"
+	local dm_backing_dev=$SCRATCH_DEV
+
+	$DMSETUP_PROG remove huge-test > /dev/null 2>&1
+	$DMSETUP_PROG remove huge-test-zero > /dev/null 2>&1
+
+	local blk_dev_size=$1
+
+	DMHUGEDISK_ZERO='/dev/mapper/huge-test-zero'
+	DMHUGEDISK_DEV='/dev/mapper/huge-test'
+
+	DMHUGEDISK_ZERO_TABLE="0 $blk_dev_size zero"
+	DMHUGEDISK_DEV_TABLE="0 $blk_dev_size snapshot $DMHUGEDISK_ZERO $SCRATCH_DEV N 512"
+
+	$DMSETUP_PROG create huge-test-zero --table "$DMHUGEDISK_ZERO_TABLE" || \
+		_fatal "failed to create dm huge zero device"
+	$DMSETUP_PROG create huge-test --table "$DMHUGEDISK_DEV_TABLE" || \
+		_fatal "failed to create dm huge device"
+}
+
+_dmhugedisk_cleanup()
+{
+	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
+	# wait for device to be fully settled so that 'dmsetup remove' doesn't
+	# fail due to EBUSY
+	$UDEV_SETTLE_PROG >/dev/null 2>&1
+	$DMSETUP_PROG remove huge-test > /dev/null 2>&1
+	$DMSETUP_PROG remove huge-test-zero > /dev/null 2>&1
+}
+
diff --git a/tests/xfs/856 b/tests/xfs/856
new file mode 100755
index 0000000..9186278
--- /dev/null
+++ b/tests/xfs/856
@@ -0,0 +1,119 @@
+#! /bin/bash
+# FS QA Test No. 856
+#
+# Create a file with more than 2^21 extents (the max length of a bmbt record).
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    umount $SCRATCH_MNT > /dev/null 2>&1
+    _dmhugedisk_cleanup
+    rm -rf $tmp.*
+    _scratch_mkfs >/dev/null 2>&1
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmhugedisk
+
+# real QA test starts here
+_supported_os Linux
+_supported_fs xfs
+_require_scratch
+_require_xfs_io_command "falloc"
+
+rm -f $seqres.full
+
+# Figure out block size
+echo "Figure out block size"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount >> $seqres.full
+
+is_rmap=$(xfs_info $SCRATCH_MNT | grep -c "rmapbt=1")
+test $is_rmap -gt 0 || _notrun "rmap not supported on scratch fs"
+
+testdir=$SCRATCH_MNT/test-$seq
+blksz="$(stat -f $SCRATCH_MNT -c '%S')"
+
+umount $SCRATCH_MNT
+
+echo "Format huge device"
+_dmhugedisk_init $((blksz * 2 * 4400))	# a little over 2^22 blocks
+_mkfs_dev -d agcount=2 $DMHUGEDISK_DEV
+_mount $DMHUGEDISK_DEV $SCRATCH_MNT 
+xfs_info $SCRATCH_MNT >> $seqres.full
+
+echo "Create the original file blocks"
+mkdir $testdir
+blksz="$(stat -f $testdir -c '%S')"
+nr_blks=2100000	# 2^21 plus a little more
+$XFS_IO_PROG -f -c "falloc 0 $((nr_blks * blksz))" $testdir/file1 >> $seqres.full
+
+echo "Check extent count"
+xfs_bmap -l -p -v $testdir/file1 | grep '^[[:space:]]*1:' -q && xfs_bmap -l -p -v $testdir/file1
+inum=$(stat -c '%i' $testdir/file1)
+umount $SCRATCH_MNT
+
+echo "Check bmap count"
+nr_bmaps=$(xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV | grep 'data offset' | wc -l)
+test $nr_bmaps -gt 1 || xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+#xfs_db -c "agf 0" -c p -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+
+echo "Check rmap count"
+nr_rmaps=$(xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0" | wc -l)
+test $nr_rmaps -eq 1 || xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0"
+
+echo "Check and fake-repair huge filesystem" | tee -a $seqres.full
+$XFS_DB_PROG -c 'check' $DMHUGEDISK_DEV
+$XFS_REPAIR_PROG -n $DMHUGEDISK_DEV >> $seqres.full 2>&1
+test $? -eq 0 || echo "xfs_repair -n failed, see $seqres.full"
+
+echo "Real repair huge filesystem" | tee -a $seqres.full
+$XFS_REPAIR_PROG $DMHUGEDISK_DEV >> $seqres.full 2>&1
+test $? -eq 0 || echo "xfs_repair failed, see $seqres.full"
+
+echo "Check bmap count again"
+nr_bmaps=$(xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV | grep 'data offset' | wc -l)
+test $nr_bmaps -gt 1 || xfs_db -c "inode $inum" -c "bmap" $DMHUGEDISK_DEV
+
+echo "Check rmap count again"
+nr_rmaps=$(xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0" | wc -l)
+test $nr_rmaps -eq 1 || xfs_db -c 'agf 0' -c 'addr rmaproot' -c 'p' $DMHUGEDISK_DEV | grep ",$inum,[0-9]*,1,0,0"
+
+echo "Check and fake-repair huge filesystem again" | tee -a $seqres.full
+$XFS_DB_PROG -c 'check' $DMHUGEDISK_DEV
+$XFS_REPAIR_PROG -n $DMHUGEDISK_DEV >> $seqres.full 2>&1
+_dmhugedisk_cleanup
+
+echo "Done"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/856.out b/tests/xfs/856.out
new file mode 100644
index 0000000..b24ebf2
--- /dev/null
+++ b/tests/xfs/856.out
@@ -0,0 +1,13 @@
+QA output created by 856
+Figure out block size
+Format huge device
+Create the original file blocks
+Check extent count
+Check bmap count
+Check rmap count
+Check and fake-repair huge filesystem
+Real repair huge filesystem
+Check bmap count again
+Check rmap count again
+Check and fake-repair huge filesystem again
+Done
diff --git a/tests/xfs/group b/tests/xfs/group
index f0ca410..6ff24d9 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -288,3 +288,4 @@
 853 auto quick clone
 854 auto quick clone
 855 auto clone
+856 auto clone rmap

Re: [PATCH v2 6/7] xfs: test rmap behavior when multiple bmbt records map to a single rmapbt record

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 7/7] reflink: test changing sharers of a block while keeping refcount the same

[Darrick J. Wong]

Ensure that we can handle the case where the refcount stays the same
even though the actual sharers changes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/925     |   80 ++++++++++++++++++++++++++++++++++++++++++
 tests/generic/925.out |   69 ++++++++++++++++++++++++++++++++++++
 tests/generic/926     |   94 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/926.out |   20 ++++++++++
 tests/generic/group   |    2 +
 5 files changed, 265 insertions(+)
 create mode 100755 tests/generic/925
 create mode 100644 tests/generic/925.out
 create mode 100755 tests/generic/926
 create mode 100644 tests/generic/926.out


diff --git a/tests/generic/925 b/tests/generic/925
new file mode 100755
index 0000000..9fa4ba9
--- /dev/null
+++ b/tests/generic/925
@@ -0,0 +1,80 @@
+#! /bin/bash
+# FS QA Test No. 925
+#
+# Share an extent amongst a bunch of files such that the refcount
+# stays the same while the rate of change of the set of owners is
+# steadily increasing.  For example, an extent of 32 blocks is owned
+# by 32 files.  At block 1, change one of the owners.  At block 2,
+# change 2 of the owners, and so on.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Oracle, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 7 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+    wait
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_require_scratch_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blocks=64
+blksz=65536
+
+echo "Initialize file"
+_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file >> $seqres.full
+
+echo "Share the file n-ways"
+seq 1 $((blocks - 1)) | while read nr; do
+	_reflink_range $testdir/file 0 $testdir/file$nr.0 0 $((nr * blksz)) >> $seqres.full
+	seq $nr $((blocks - 1)) | while read nnr; do
+		_reflink_range $testdir/file $((nnr * blksz)) $testdir/file$nr.$nnr $((nnr * blksz)) $blksz >> $seqres.full
+	done
+done
+_scratch_cycle_mount
+
+echo "Check output"
+md5sum $testdir/file $testdir/file*.0 | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/925.out b/tests/generic/925.out
new file mode 100644
index 0000000..099d6d3
--- /dev/null
+++ b/tests/generic/925.out
@@ -0,0 +1,69 @@
+QA output created by 925
+Format and mount
+Initialize file
+Share the file n-ways
+Check output
+bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-925/file
+2d61aa54b58c2e94403fb092c3dbc027  SCRATCH_MNT/test-925/file1.0
+399c239a8d72b5b2beb15565cf14f6d6  SCRATCH_MNT/test-925/file10.0
+7f783291615a5db41ba33d6c50318095  SCRATCH_MNT/test-925/file11.0
+4feff1bdae53d042e572a5e240b1183f  SCRATCH_MNT/test-925/file12.0
+ac2b0a907cd84c091c1cc36384246e42  SCRATCH_MNT/test-925/file13.0
+ff420a8abc25ff0a012638ddd9b81e6c  SCRATCH_MNT/test-925/file14.0
+3eb1751bcb7cff809df2ec376e45fe6f  SCRATCH_MNT/test-925/file15.0
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-925/file16.0
+7b0fea2a208aa4738d8b9433c7b8421d  SCRATCH_MNT/test-925/file17.0
+c6b6ec99b28741ffb3be2d7ff361ad89  SCRATCH_MNT/test-925/file18.0
+d609dcbdc972b8cff16acd74414a8dc9  SCRATCH_MNT/test-925/file19.0
+81615449a98aaaad8dc179b3bec87f38  SCRATCH_MNT/test-925/file2.0
+d97ae5fdb25c4622d5a2f313460a2b96  SCRATCH_MNT/test-925/file20.0
+afae74f7a13edf57f43bab5da6e411d6  SCRATCH_MNT/test-925/file21.0
+0ed53c341d0154d68c336eaa9d05990d  SCRATCH_MNT/test-925/file22.0
+ab20d0025b58cbf9fc49e436b7d7acef  SCRATCH_MNT/test-925/file23.0
+5279106ed91359051fbdb61e87d5077d  SCRATCH_MNT/test-925/file24.0
+9a8719c84c1a42d030fc6d700046eaf5  SCRATCH_MNT/test-925/file25.0
+d9b21a065727fd8118d493f7c62e7fb2  SCRATCH_MNT/test-925/file26.0
+d74ee01c55b2474f91a3723a01c83591  SCRATCH_MNT/test-925/file27.0
+ba7202bfc4b9cda564156562d8672a3f  SCRATCH_MNT/test-925/file28.0
+1b3d697355ca662451eb12419cab8eef  SCRATCH_MNT/test-925/file29.0
+998b4ba52f2940dc515001e75926b19f  SCRATCH_MNT/test-925/file3.0
+6b2648f84c57f58786cf75ac06e37217  SCRATCH_MNT/test-925/file30.0
+c400b872b0fb3fb58920c4e570937a32  SCRATCH_MNT/test-925/file31.0
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-925/file32.0
+cac15c2eeaf1b63614469f675ec71c5f  SCRATCH_MNT/test-925/file33.0
+ebd2d819d56725bd1a2a35ababd06cad  SCRATCH_MNT/test-925/file34.0
+a26ebf37da9976dfd7c86b4271c59501  SCRATCH_MNT/test-925/file35.0
+11f61aaa1f5f726929c91c6d434edf72  SCRATCH_MNT/test-925/file36.0
+1f73d21c457843ea20048726d461648e  SCRATCH_MNT/test-925/file37.0
+9ed53cefd1548008ed1948286e471233  SCRATCH_MNT/test-925/file38.0
+af5195351947da43e7a57f69086aa7d7  SCRATCH_MNT/test-925/file39.0
+c946b71bb69c07daf25470742c967e7c  SCRATCH_MNT/test-925/file4.0
+f31ecd4106278b5fb1aeabdedd43036a  SCRATCH_MNT/test-925/file40.0
+e8a8d1a0873e25697171ebb87125d794  SCRATCH_MNT/test-925/file41.0
+5920fd86bc7be6a0ef5d7ea283e16307  SCRATCH_MNT/test-925/file42.0
+8360c3c368143821045f9b7994ab8344  SCRATCH_MNT/test-925/file43.0
+6c76a6c9e2b039c82e62d985479f73ee  SCRATCH_MNT/test-925/file44.0
+8b5c6cab2f0da5ba2d3b2d3ca205ca75  SCRATCH_MNT/test-925/file45.0
+b4ef05d705b8e99d8824d85e993bf8fa  SCRATCH_MNT/test-925/file46.0
+0db1f2d5ad692ccd8941a8bc308cdba2  SCRATCH_MNT/test-925/file47.0
+6a11a8872b36343799a15617bea78cef  SCRATCH_MNT/test-925/file48.0
+015ab57c35525eeb776308e14fe2e731  SCRATCH_MNT/test-925/file49.0
+17af09af790a9b4c79cddf72f6b642cb  SCRATCH_MNT/test-925/file5.0
+fbc6203955f9c6e9010f831971a073b2  SCRATCH_MNT/test-925/file50.0
+aae01cd767e508bdf2d01f647d10c4b8  SCRATCH_MNT/test-925/file51.0
+4d942a936328991bcd9821072ac0fd9e  SCRATCH_MNT/test-925/file52.0
+b77f243bb207746743aeaafbdef4203a  SCRATCH_MNT/test-925/file53.0
+9fc7f158bd2ded9df3965ef4fe2fa932  SCRATCH_MNT/test-925/file54.0
+36dff6091853c946411bc40fdae79e84  SCRATCH_MNT/test-925/file55.0
+82a2f6d28f1bff4ce7edc1a7a31b02cb  SCRATCH_MNT/test-925/file56.0
+b0d123bf023332bba783abb5b68e4c98  SCRATCH_MNT/test-925/file57.0
+26b49b145b128c0abef0f92b3ccaf1d6  SCRATCH_MNT/test-925/file58.0
+b5ef5d86d6ef15a535ee95b8069d29c9  SCRATCH_MNT/test-925/file59.0
+cd3eccbf2f0126c4141740007ba70c3f  SCRATCH_MNT/test-925/file6.0
+205cb48615014f6f2377345b62442609  SCRATCH_MNT/test-925/file60.0
+89691c001b3a3d5473081687ae519a18  SCRATCH_MNT/test-925/file61.0
+a040ce9774652bd00c3cbf493a006e00  SCRATCH_MNT/test-925/file62.0
+970b42855c15ee8caa824cd5e177759a  SCRATCH_MNT/test-925/file63.0
+eb74da58c6e4f93b4bf82a62fef4ab93  SCRATCH_MNT/test-925/file7.0
+30c2557e8302a5beb290c71520d87f42  SCRATCH_MNT/test-925/file8.0
+6dc42f5ed1007a3e20d08af71e198cdc  SCRATCH_MNT/test-925/file9.0
diff --git a/tests/generic/926 b/tests/generic/926
new file mode 100755
index 0000000..7dedc3f
--- /dev/null
+++ b/tests/generic/926
@@ -0,0 +1,94 @@
+#! /bin/bash
+# FS QA Test No. 926
+#
+# Make sure that the reference counting mechanism can handle the case
+# where we share the first 1/4 of an extent with a file, share the
+# last 1/4 of the extent with a second file, share the first half of
+# the extent with N files, and share the second half of the extent
+# with a different set of N files.  The key point here is to test that
+# we handle the case where a refcount extent record doesn't coincide
+# exactly with the block mapping records.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Oracle, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 7 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $tmp.*
+    wait
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_os Linux
+_require_scratch_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+blocks=64
+blksz=65536
+nr=4
+halfway=$((blocks / 2 * blksz))
+quarter=$((blocks / 4 * blksz))
+
+echo "Initialize file"
+_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file >> $seqres.full
+
+echo "Share the first half of the extent"
+seq 1 $nr | while read nr; do
+	_reflink_range $testdir/file 0 $testdir/file$nr.0 0 $halfway >> $seqres.full
+done
+
+echo "Share the last half of the extent"
+seq 1 $nr | while read nr; do
+	_reflink_range $testdir/file $halfway $testdir/file$nr.1 0 $halfway >> $seqres.full
+done
+
+echo "Share the first quarter of the extent"
+_reflink_range $testdir/file 0 $testdir/file.2 0 $quarter >> $seqres.full
+
+echo "Share the last quarter of the extent"
+_reflink_range $testdir/file $((quarter * 3)) $testdir/file.3 0 $quarter >> $seqres.full
+
+_scratch_cycle_mount
+
+echo "Check output"
+md5sum $testdir/file $testdir/file* | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/926.out b/tests/generic/926.out
new file mode 100644
index 0000000..d3e6b59
--- /dev/null
+++ b/tests/generic/926.out
@@ -0,0 +1,20 @@
+QA output created by 926
+Format and mount
+Initialize file
+Share the first half of the extent
+Share the last half of the extent
+Share the first quarter of the extent
+Share the last quarter of the extent
+Check output
+bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-926/file
+bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-926/file
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-926/file.2
+7202826a7791073fe2787f0c94603278  SCRATCH_MNT/test-926/file.3
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file1.0
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file1.1
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file2.0
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file2.1
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file3.0
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file3.1
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file4.0
+de89461b64701958984c95d1bfb0065a  SCRATCH_MNT/test-926/file4.1
diff --git a/tests/generic/group b/tests/generic/group
index 6fdf109..2bd40a1 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -356,3 +356,5 @@
 351 blockdev quick rw
 923 auto quick clone
 924 auto quick clone
+925 auto quick clone
+926 auto quick clone

Re: [PATCH 7/7] reflink: test changing sharers of a block while keeping refcount the same

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 8/7] xfs/122: don't break on old xfsprogs

[Darrick J. Wong]

If we're running against a old version of xfsprogs that lacks some of
the structures that the golden output knows about, copy the structure
size definition from the golden output to the program output.  This
way we can check for structure size mutations on old xfsprogs without
generating false error reports for structs that don't exist in the
old release.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/122     |    8 ++++++++
 tests/xfs/122.out |    1 +
 2 files changed, 9 insertions(+)

diff --git a/tests/xfs/122 b/tests/xfs/122
index 845cdd2..c944942 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -73,6 +73,7 @@ _attribute_filter()
 cprog=$tmp.get_structs.c
 oprog=$tmp.get_structs
 progout=$tmp.output
+keyfile=$tmp.keys
 
 cat >$cprog <<EOF
 #define _GNU_SOURCE
@@ -239,6 +240,13 @@ if [ $XFSPROGS_VERSION -lt 30000 ]; then
 	echo 'sizeof( xfs_dsb_t ) = 208' >>$progout;
 fi
 
+# Find all the items that only exist in the golden output
+comm -23 <(grep '=' $0.out | sed -e 's/ =.*$//g' | LC_COLLATE=POSIX sort) \
+	 <(sed -e 's/ =.*$//g' < $progout | LC_COLLATE=POSIX sort) > $keyfile
+
+# Copy those items to the program output
+grep -F -f $keyfile $0.out >> $progout
+
 LC_COLLATE=POSIX sort $progout
 
 status=0
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 451871e..46d1dd4 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -147,3 +147,4 @@ sizeof(xfs_lookup_t) = 4
 sizeof(xfs_qoff_logformat_t) = 20
 sizeof(xfs_timestamp_t) = 8
 sizeof(xfs_trans_header_t) = 16
+sizeof(xfs_zzzz_test_the_test_program) = -47

Re: [PATCH 8/7] xfs/122: don't break on old xfsprogs

[Christoph Hellwig]

Looks fine, but can't we also remove the workaround at the end of the
the test as well?

Re: [PATCH 8/7] xfs/122: don't break on old xfsprogs

[Darrick J. Wong]

On Tue, May 31, 2016 at 05:51:09AM -0700, Christoph Hellwig wrote:
> Looks fine, but can't we also remove the workaround at the end of the
> the test as well?

Oh, heh, yes, those can go.  The whitespace is wrong on them anyway.

--D

> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

[PATCH v2 8/7] xfs/122: don't break on old xfsprogs

[Darrick J. Wong]

If we're running against a old version of xfsprogs that lacks some of
the structures that the golden output knows about, copy the structure
size definition from the golden output to the program output.  This
way we can check for structure size mutations on old xfsprogs without
generating false error reports for structs that don't exist in the
old release.

v2: Remove the xfsprogs 2.9.8 compatibility cruft.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/122     |   17 ++++++-----------
 tests/xfs/122.out |    1 +
 2 files changed, 7 insertions(+), 11 deletions(-)

diff --git a/tests/xfs/122 b/tests/xfs/122
index 845cdd2..330d3ac 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -73,6 +73,7 @@ _attribute_filter()
 cprog=$tmp.get_structs.c
 oprog=$tmp.get_structs
 progout=$tmp.output
+keyfile=$tmp.keys
 
 cat >$cprog <<EOF
 #define _GNU_SOURCE
@@ -226,18 +227,12 @@ cc -o $oprog $cprog >> $seqres.full 2>&1 || \
   _notrun "Could not compile test program (see end of $seqres.full)"
 $oprog | _type_size_filter | _type_name_filter > $progout
 
-#
-# add addition sizes and xfs_sb_t fields that don't exist in the version
-# being tested.
-#
+# Find all the items that only exist in the golden output
+comm -23 <(grep '=' $0.out | sed -e 's/ =.*$//g' | LC_COLLATE=POSIX sort) \
+	 <(sed -e 's/ =.*$//g' < $progout | LC_COLLATE=POSIX sort) > $keyfile
 
-# xfsprogs 2.9.8: sb_bad_features2 in pv 978822
-if [ $XFSPROGS_VERSION -lt 20908 ]; then
-	echo 'offsetof(xfs_sb_t, sb_bad_features2 ) = 204' >>$progout
-fi
-if [ $XFSPROGS_VERSION -lt 30000 ]; then
-	echo 'sizeof( xfs_dsb_t ) = 208' >>$progout;
-fi
+# Copy those items to the program output
+grep -F -f $keyfile $0.out >> $progout
 
 LC_COLLATE=POSIX sort $progout
 
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 451871e..46d1dd4 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -147,3 +147,4 @@ sizeof(xfs_lookup_t) = 4
 sizeof(xfs_qoff_logformat_t) = 20
 sizeof(xfs_timestamp_t) = 8
 sizeof(xfs_trans_header_t) = 16
+sizeof(xfs_zzzz_test_the_test_program) = -47

Re: [PATCH v2 8/7] xfs/122: don't break on old xfsprogs

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>