* [PATCH v3 01/17] generic: add mmap CoW test for ranges of two shared files
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
@ 2021-12-14 8:18 ` Shiyang Ruan
2021-12-14 8:18 ` [PATCH v3 02/17] generic: add mmap CoW test for regular&destination extents Shiyang Ruan
` (16 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:18 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents. (MMAP version of generic/185,183)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/900 | 69 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/900.out | 14 +++++++++
2 files changed, 83 insertions(+)
create mode 100755 tests/generic/900
create mode 100644 tests/generic/900.out
diff --git a/tests/generic/900 b/tests/generic/900
new file mode 100755
index 00000000..de76cd00
--- /dev/null
+++ b/tests/generic/900
@@ -0,0 +1,69 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 900
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents. (MMAP version of generic/185,183)
+# - Create two files
+# - Reflink the odd blocks of the first file into a third file.
+# - Reflink the even blocks of the second file into the third file.
+# - mmap CoW across the halfway mark.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_pwrite_byte 0x61 0 $filesize $testdir/file1 >> $seqres.full
+_pwrite_byte 0x62 0 $filesize $testdir/file2 >> $seqres.full
+seq 0 2 $((nr-1)) | while read f; do
+ _reflink_range $testdir/file1 $((blksz * f)) $testdir/file3 $((blksz * f)) $blksz >> $seqres.full
+ _pwrite_byte 0x61 $((blksz * f)) $blksz $testdir/file3.chk >> $seqres.full
+done
+seq 1 2 $((nr-1)) | while read f; do
+ _reflink_range $testdir/file2 $((blksz * f)) $testdir/file3 $((blksz * f)) $blksz >> $seqres.full
+ _pwrite_byte 0x62 $((blksz * f)) $blksz $testdir/file3.chk >> $seqres.full
+done
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file2 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/900.out b/tests/generic/900.out
new file mode 100644
index 00000000..b2569ab0
--- /dev/null
+++ b/tests/generic/900.out
@@ -0,0 +1,14 @@
+QA output created by 900
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-900/file1
+b83f9394092e15bdcda585cd8e776dc6 SCRATCH_MNT/test-900/file2
+d3959a68638c50af07f0258e032cb554 SCRATCH_MNT/test-900/file3
+d3959a68638c50af07f0258e032cb554 SCRATCH_MNT/test-900/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-900/file1
+b83f9394092e15bdcda585cd8e776dc6 SCRATCH_MNT/test-900/file2
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-900/file3
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-900/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 02/17] generic: add mmap CoW test for regular&destination extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
2021-12-14 8:18 ` [PATCH v3 01/17] generic: add mmap CoW test for ranges of two shared files Shiyang Ruan
@ 2021-12-14 8:18 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 03/17] generic: add mmap CoW test for unwritten&destination extents Shiyang Ruan
` (15 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:18 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents, some regular, some not.
(MMAP version of generic/197,196)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/901 | 59 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/901.out | 12 +++++++++
2 files changed, 71 insertions(+)
create mode 100755 tests/generic/901
create mode 100644 tests/generic/901.out
diff --git a/tests/generic/901 b/tests/generic/901
new file mode 100755
index 00000000..b9f4fbc0
--- /dev/null
+++ b/tests/generic/901
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 901
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, some regular, some not.
+# (MMAP version of generic/197,196)
+# - Create two files.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_weave_reflink_regular $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/901.out b/tests/generic/901.out
new file mode 100644
index 00000000..c3983e87
--- /dev/null
+++ b/tests/generic/901.out
@@ -0,0 +1,12 @@
+QA output created by 901
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-901/file1
+d3959a68638c50af07f0258e032cb554 SCRATCH_MNT/test-901/file3
+d3959a68638c50af07f0258e032cb554 SCRATCH_MNT/test-901/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-901/file1
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-901/file3
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-901/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 03/17] generic: add mmap CoW test for unwritten&destination extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
2021-12-14 8:18 ` [PATCH v3 01/17] generic: add mmap CoW test for ranges of two shared files Shiyang Ruan
2021-12-14 8:18 ` [PATCH v3 02/17] generic: add mmap CoW test for regular&destination extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 04/17] generic: add mmap CoW test for holes&destination extents Shiyang Ruan
` (14 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents, some unwritten, some not.
(MMAP version of generic/189,188)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/902 | 59 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/902.out | 12 +++++++++
2 files changed, 71 insertions(+)
create mode 100755 tests/generic/902
create mode 100644 tests/generic/902.out
diff --git a/tests/generic/902 b/tests/generic/902
new file mode 100755
index 00000000..457b039a
--- /dev/null
+++ b/tests/generic/902
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 902
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, some unwritten, some not.
+# (MMAP version of generic/189,188)
+# - Create a file and fallocate a second file.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_weave_reflink_unwritten $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/902.out b/tests/generic/902.out
new file mode 100644
index 00000000..bc951255
--- /dev/null
+++ b/tests/generic/902.out
@@ -0,0 +1,12 @@
+QA output created by 902
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-902/file1
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-902/file3
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-902/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-902/file1
+09101629908f9bdd5d178e7ce20bb1bb SCRATCH_MNT/test-902/file3
+09101629908f9bdd5d178e7ce20bb1bb SCRATCH_MNT/test-902/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 04/17] generic: add mmap CoW test for holes&destination extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (2 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 03/17] generic: add mmap CoW test for unwritten&destination extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 05/17] common/rc: Introduce _require_scratch_delalloc() Shiyang Ruan
` (13 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents, some holes, some not.
(MMAP version of generic/191,190)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/903 | 59 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/903.out | 12 +++++++++
2 files changed, 71 insertions(+)
create mode 100755 tests/generic/903
create mode 100644 tests/generic/903.out
diff --git a/tests/generic/903 b/tests/generic/903
new file mode 100755
index 00000000..d0de3f26
--- /dev/null
+++ b/tests/generic/903
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 903
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, some holes, some not.
+# (MMAP version of generic/191,190)
+# - Create a file and truncate a second file.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_weave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/903.out b/tests/generic/903.out
new file mode 100644
index 00000000..bf57acc7
--- /dev/null
+++ b/tests/generic/903.out
@@ -0,0 +1,12 @@
+QA output created by 903
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-903/file1
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-903/file3
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-903/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-903/file1
+09101629908f9bdd5d178e7ce20bb1bb SCRATCH_MNT/test-903/file3
+09101629908f9bdd5d178e7ce20bb1bb SCRATCH_MNT/test-903/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 05/17] common/rc: Introduce _require_scratch_delalloc()
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (3 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 04/17] generic: add mmap CoW test for holes&destination extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 06/17] generic: add mmap CoW test for delalloc&destination extents Shiyang Ruan
` (12 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Check whether delalloc is supported on current device.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
common/rc | 13 +++++++++++++
tests/generic/194 | 1 +
tests/generic/195 | 1 +
tests/generic/199 | 1 +
tests/generic/200 | 1 +
tests/generic/293 | 1 +
tests/generic/295 | 1 +
tests/generic/540 | 1 +
tests/generic/541 | 1 +
tests/generic/542 | 1 +
tests/generic/543 | 1 +
tests/xfs/172 | 12 +-----------
tests/xfs/223 | 1 +
tests/xfs/224 | 1 +
tests/xfs/228 | 1 +
tests/xfs/230 | 1 +
tests/xfs/257 | 1 +
tests/xfs/258 | 1 +
18 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/common/rc b/common/rc
index 7973ceb5..a419b3ba 100644
--- a/common/rc
+++ b/common/rc
@@ -1748,6 +1748,19 @@ _require_scratch_16T_support()
esac
}
+# Require scratch fs supports delay allocation.
+_require_scratch_delalloc()
+{
+ _require_command "$FILEFRAG_PROG" filefrag
+
+ _scratch_mkfs > $seqres.full
+ _scratch_mount
+ $XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
+ $FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
+ _notrun "test requires delayed allocation buffered writes"
+ _scratch_unmount
+}
+
# this test needs a test partition - check we're ok & mount it
#
_require_test()
diff --git a/tests/generic/194 b/tests/generic/194
index ce8c31ac..ff76438d 100755
--- a/tests/generic/194
+++ b/tests/generic/194
@@ -28,6 +28,7 @@ _cleanup()
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_odirect
diff --git a/tests/generic/195 b/tests/generic/195
index da502c39..e087b99c 100755
--- a/tests/generic/195
+++ b/tests/generic/195
@@ -28,6 +28,7 @@ _cleanup()
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/generic/199 b/tests/generic/199
index ab13f58e..2a8cafcc 100755
--- a/tests/generic/199
+++ b/tests/generic/199
@@ -31,6 +31,7 @@ _cleanup()
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "fpunch"
_require_cp_reflink
diff --git a/tests/generic/200 b/tests/generic/200
index deadfeaa..a1a78ef4 100755
--- a/tests/generic/200
+++ b/tests/generic/200
@@ -31,6 +31,7 @@ _cleanup()
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "fpunch"
_require_cp_reflink
diff --git a/tests/generic/293 b/tests/generic/293
index 74231b4d..0f1d8416 100755
--- a/tests/generic/293
+++ b/tests/generic/293
@@ -24,6 +24,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/generic/295 b/tests/generic/295
index b921c79b..f66c1805 100755
--- a/tests/generic/295
+++ b/tests/generic/295
@@ -24,6 +24,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_odirect
diff --git a/tests/generic/540 b/tests/generic/540
index 69cf635f..38e00f97 100755
--- a/tests/generic/540
+++ b/tests/generic/540
@@ -26,6 +26,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_supported_fs generic
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/generic/541 b/tests/generic/541
index 925f346e..89b2adad 100755
--- a/tests/generic/541
+++ b/tests/generic/541
@@ -26,6 +26,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_supported_fs generic
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/generic/542 b/tests/generic/542
index e3e627a8..e7682f59 100755
--- a/tests/generic/542
+++ b/tests/generic/542
@@ -26,6 +26,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_supported_fs generic
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/generic/543 b/tests/generic/543
index 047bedd5..624cfc41 100755
--- a/tests/generic/543
+++ b/tests/generic/543
@@ -26,6 +26,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_supported_fs generic
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
echo "Format and mount"
diff --git a/tests/xfs/172 b/tests/xfs/172
index c0495305..56c2583b 100755
--- a/tests/xfs/172
+++ b/tests/xfs/172
@@ -16,7 +16,6 @@ _begin_fstest rw filestreams
# real QA test starts here
_supported_fs xfs
-_require_command "$FILEFRAG_PROG" filefrag
_require_scratch
# The first _test_streams call sets up the filestreams allocator to fail and
@@ -36,16 +35,7 @@ _require_scratch
# rescan thresholds and will keep pushing out the expiration time, with the
# result that the filestreams allocator succeeds in maintaining the streams.
# The test considers this a failure.
-#
-# Make sure that a regular buffered write produces delalloc reservations.
-# This effectively disables the test for files with extent size hints or DAX
-# mode set.
-_scratch_mkfs > $seqres.full
-_scratch_mount
-$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
-$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
- _notrun "test requires delayed allocation buffered writes"
-_scratch_unmount
+_require_scratch_delalloc
_check_filestreams_support || _notrun "filestreams not available"
diff --git a/tests/xfs/223 b/tests/xfs/223
index f002f1de..11dbad14 100755
--- a/tests/xfs/223
+++ b/tests/xfs/223
@@ -22,6 +22,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
_require_odirect
diff --git a/tests/xfs/224 b/tests/xfs/224
index 11a54fa3..f8bab07e 100755
--- a/tests/xfs/224
+++ b/tests/xfs/224
@@ -22,6 +22,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
diff --git a/tests/xfs/228 b/tests/xfs/228
index 5c644c0f..f2f2f6a9 100755
--- a/tests/xfs/228
+++ b/tests/xfs/228
@@ -25,6 +25,7 @@ _begin_fstest auto quick clone punch
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
_require_xfs_io_command "fpunch"
diff --git a/tests/xfs/230 b/tests/xfs/230
index 53e8dd78..15f6b684 100755
--- a/tests/xfs/230
+++ b/tests/xfs/230
@@ -25,6 +25,7 @@ _begin_fstest auto quick clone punch
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
_require_xfs_io_command "fpunch"
diff --git a/tests/xfs/257 b/tests/xfs/257
index acd55c22..6a58f0ac 100755
--- a/tests/xfs/257
+++ b/tests/xfs/257
@@ -25,6 +25,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
diff --git a/tests/xfs/258 b/tests/xfs/258
index 25e8eb60..2865cdf9 100755
--- a/tests/xfs/258
+++ b/tests/xfs/258
@@ -25,6 +25,7 @@ _begin_fstest auto quick clone
# real QA test starts here
_require_scratch_reflink
+_require_scratch_delalloc
_require_xfs_io_command "falloc"
_require_xfs_io_command "cowextsize"
_require_odirect
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 06/17] generic: add mmap CoW test for delalloc&destination extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (4 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 05/17] common/rc: Introduce _require_scratch_delalloc() Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 07/17] generic: add mmap CoW test for mixed&destination extents Shiyang Ruan
` (11 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents, some delalloc, some not.
(MMAP version of generic/195,194)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
tests/generic/904 | 62 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/904.out | 12 +++++++++
2 files changed, 74 insertions(+)
create mode 100755 tests/generic/904
create mode 100644 tests/generic/904.out
diff --git a/tests/generic/904 b/tests/generic/904
new file mode 100755
index 00000000..428dbbf4
--- /dev/null
+++ b/tests/generic/904
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 904
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, some delalloc, some not.
+# (MMAP version of generic/195,194)
+# - Create a file.
+# - Reflink the odd blocks of the first file into the second file.
+# - Buffered write the even blocks of the second file.
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_weave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_weave_reflink_holes_delalloc $blksz $nr $testdir/file3 >> $seqres.full
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/904.out b/tests/generic/904.out
new file mode 100644
index 00000000..cc993394
--- /dev/null
+++ b/tests/generic/904.out
@@ -0,0 +1,12 @@
+QA output created by 904
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-904/file1
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-904/file3
+fa50dba51826899c372464a153cb2117 SCRATCH_MNT/test-904/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-904/file1
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-904/file3
+55968a95dfc0120df4e8485576514320 SCRATCH_MNT/test-904/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 07/17] generic: add mmap CoW test for mixed&destination extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (5 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 06/17] generic: add mmap CoW test for delalloc&destination extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 08/17] generic: add mmap CoW test for regular&source extents Shiyang Ruan
` (10 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Ensuring that copy on write in mmap mode works when the CoW range originally
covers multiple extents, mixed with reflinked, unwritten, hole, regular and
delalloc blocks.
(MMAP version of generic/200,199)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
tests/generic/905 | 67 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/905.out | 12 ++++++++
2 files changed, 79 insertions(+)
create mode 100755 tests/generic/905
create mode 100644 tests/generic/905.out
diff --git a/tests/generic/905 b/tests/generic/905
new file mode 100755
index 00000000..8178f630
--- /dev/null
+++ b/tests/generic/905
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 905
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, mixed with reflinked, unwritten,
+# hole, regular and delalloc blocks.
+# (MMAP version of generic/200,199)
+# - Create a file with the following repeating sequence of blocks:
+# 1. reflinked
+# 2. unwritten
+# 3. hole
+# 4. regular block
+# 5. delalloc
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone punch
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_weave_reflink_rainbow $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_weave_reflink_rainbow_delalloc $blksz $nr $testdir/file3 >> $seqres.full
+# now cow
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file3.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file3.chk | _filter_scratch
+
+# success, all done
+status=0
diff --git a/tests/generic/905.out b/tests/generic/905.out
new file mode 100644
index 00000000..6fb0be5f
--- /dev/null
+++ b/tests/generic/905.out
@@ -0,0 +1,12 @@
+QA output created by 905
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-905/file1
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-905/file3
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-905/file3.chk
+mmap CoW across the transition
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-905/file1
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-905/file3
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-905/file3.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 08/17] generic: add mmap CoW test for regular&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (6 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 07/17] generic: add mmap CoW test for mixed&destination extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 09/17] generic: add mmap CoW test for unwritten&source extents Shiyang Ruan
` (9 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode to the source file when the CoW range
covers regular unshared and regular shared blocks.
(MMAP version of generic/284,287)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/906 | 60 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/906.out | 12 +++++++++
2 files changed, 72 insertions(+)
create mode 100755 tests/generic/906
create mode 100644 tests/generic/906.out
diff --git a/tests/generic/906 b/tests/generic/906
new file mode 100755
index 00000000..953348e8
--- /dev/null
+++ b/tests/generic/906
@@ -0,0 +1,60 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 906
+#
+# Ensuring that copy on write in mmap mode to the source file when the
+# CoW range covers regular unshared and regular shared blocks.
+# (MMAP version of generic/284,287)
+# - Create two files.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW the first file across the halfway mark, starting with the
+# regular extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_regular $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/906.out b/tests/generic/906.out
new file mode 100644
index 00000000..4949769b
--- /dev/null
+++ b/tests/generic/906.out
@@ -0,0 +1,12 @@
+QA output created by 906
+Format and mount
+Create the original files
+Compare files
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-906/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-906/file3
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-906/file1.chk
+mmap CoW across the transition
+Compare files
+2dc2b131303b2f70ddb480778caac8b7 SCRATCH_MNT/test-906/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-906/file3
+2dc2b131303b2f70ddb480778caac8b7 SCRATCH_MNT/test-906/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 09/17] generic: add mmap CoW test for unwritten&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (7 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 08/17] generic: add mmap CoW test for regular&source extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 10/17] generic: add mmap CoW test for holes&source extents Shiyang Ruan
` (8 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode to the source file when the CoW range
covers unwritten and regular shared blocks.
(MMAP version of generic/289,290)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/907 | 62 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/907.out | 12 +++++++++
2 files changed, 74 insertions(+)
create mode 100755 tests/generic/907
create mode 100644 tests/generic/907.out
diff --git a/tests/generic/907 b/tests/generic/907
new file mode 100755
index 00000000..61c9293a
--- /dev/null
+++ b/tests/generic/907
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 907
+#
+# Ensuring that copy on write in mmap mode to the source file when the
+# CoW range covers unwritten and regular shared blocks.
+# (MMAP version of generic/289,290)
+# - Create two files.
+# - fallocate the first file.
+# - Write the odd blocks of the first file.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW the first file across the halfway mark, starting with the
+# regular extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_unwritten $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/907.out b/tests/generic/907.out
new file mode 100644
index 00000000..c18b849c
--- /dev/null
+++ b/tests/generic/907.out
@@ -0,0 +1,12 @@
+QA output created by 907
+Format and mount
+Create the original files
+Compare files
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-907/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-907/file3
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-907/file1.chk
+mmap CoW across the transition
+Compare files
+e8cf59797f0b3758c8878c368bb4cf7e SCRATCH_MNT/test-907/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-907/file3
+e8cf59797f0b3758c8878c368bb4cf7e SCRATCH_MNT/test-907/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 10/17] generic: add mmap CoW test for holes&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (8 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 09/17] generic: add mmap CoW test for unwritten&source extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 11/17] generic: add mmap CoW test for delalloc&source extents Shiyang Ruan
` (7 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode to the source file when the CoW range
covers holes and regular shared blocks.
(MMAP version of generic/291,292)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/908 | 62 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/908.out | 12 +++++++++
2 files changed, 74 insertions(+)
create mode 100755 tests/generic/908
create mode 100644 tests/generic/908.out
diff --git a/tests/generic/908 b/tests/generic/908
new file mode 100755
index 00000000..ad8de699
--- /dev/null
+++ b/tests/generic/908
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 908
+#
+# Ensuring that copy on write in mmap mode to the source file when the
+# CoW range covers holes and regular shared blocks.
+# (MMAP version of generic/291,292)
+# - Create two files.
+# - Truncate the first file.
+# - Write the odd blocks of the first file.
+# - Reflink the odd blocks of the first file into the second file.
+# - mmap CoW the first file across the halfway mark, starting with the
+# regular extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/908.out b/tests/generic/908.out
new file mode 100644
index 00000000..c8c670ac
--- /dev/null
+++ b/tests/generic/908.out
@@ -0,0 +1,12 @@
+QA output created by 908
+Format and mount
+Create the original files
+Compare files
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-908/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-908/file3
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-908/file1.chk
+mmap CoW across the transition
+Compare files
+e8cf59797f0b3758c8878c368bb4cf7e SCRATCH_MNT/test-908/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-908/file3
+e8cf59797f0b3758c8878c368bb4cf7e SCRATCH_MNT/test-908/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 11/17] generic: add mmap CoW test for delalloc&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (9 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 10/17] generic: add mmap CoW test for holes&source extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 12/17] generic: add buffered-io CoW test for mixed&source extents Shiyang Ruan
` (6 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Ensuring that copy on write in mmap mode to the source file when the CoW range
covers delalloc blocks and regular shared blocks.
(MMAP version of generic/293,295)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
tests/generic/909 | 65 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/909.out | 12 ++++++++
2 files changed, 77 insertions(+)
create mode 100755 tests/generic/909
create mode 100644 tests/generic/909.out
diff --git a/tests/generic/909 b/tests/generic/909
new file mode 100755
index 00000000..88e032b6
--- /dev/null
+++ b/tests/generic/909
@@ -0,0 +1,65 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 909
+#
+# Ensuring that copy on write in mmap mode to the source file when the
+# CoW range covers delalloc blocks and regular shared blocks.
+# (MMAP version of generic/293,295)
+# - Create two files.
+# - Truncate the first file.
+# - Write the odd blocks of the first file.
+# - Reflink the odd blocks of the first file into the second file.
+# - Write the even blocks of the first file.
+# - mmap CoW the first file across the halfway mark, starting with the
+# regular extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_sweave_reflink_holes_delalloc $blksz $nr $testdir/file1 >> $seqres.full
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/909.out b/tests/generic/909.out
new file mode 100644
index 00000000..4f9a5d33
--- /dev/null
+++ b/tests/generic/909.out
@@ -0,0 +1,12 @@
+QA output created by 909
+Format and mount
+Create the original files
+Compare files
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-909/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-909/file3
+b8a8a88d4c143f79900c4b4e79aa3e37 SCRATCH_MNT/test-909/file1.chk
+mmap CoW across the transition
+Compare files
+35e2170e14665b780c6dec328d6a263b SCRATCH_MNT/test-909/file1
+ce2023f765eba19677517c511886d5c9 SCRATCH_MNT/test-909/file3
+35e2170e14665b780c6dec328d6a263b SCRATCH_MNT/test-909/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 12/17] generic: add buffered-io CoW test for mixed&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (10 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 11/17] generic: add mmap CoW test for delalloc&source extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-25 13:27 ` Eryu Guan
2021-12-14 8:19 ` [PATCH v3 13/17] generic: add direct-io " Shiyang Ruan
` (5 subsequent siblings)
17 siblings, 1 reply; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Ensuring that copy on write in buffered mode works when the CoW
range originally covers multiple extents, mixed with reflinked, unwritten,
hole, regular and delalloc blocks.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
common/reflink | 51 +++++++++++++++++++++++++++++++++
tests/generic/910 | 65 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/910.out | 12 ++++++++
3 files changed, 128 insertions(+)
create mode 100755 tests/generic/910
create mode 100644 tests/generic/910.out
diff --git a/common/reflink b/common/reflink
index ee60398e..68dbdedd 100644
--- a/common/reflink
+++ b/common/reflink
@@ -426,3 +426,54 @@ _sweave_reflink_holes_delalloc() {
_pwrite_byte 0x64 $((blksz * i)) $blksz $sfile.chk
done
}
+
+# Create a file of interleaved holes, unwritten blocks, regular blocks, and
+# reflinked blocks
+_sweave_reflink_rainbow() {
+ blksz=$1
+ nr=$2
+ sfile=$3
+ dfile=$4
+
+ $XFS_IO_PROG -f -c "truncate $((blksz * nr))" $sfile
+ _pwrite_byte 0x00 0 $((blksz * nr)) $sfile.chk
+ _pwrite_byte 0x61 0 $((blksz * nr)) $dfile
+ seq 0 5 $((nr - 1)) | while read i; do
+ _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile
+ _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile.chk
+ done
+ # 0 blocks are reflinked
+ seq 0 5 $((nr - 1)) | while read i; do
+ _reflink_range $sfile $((blksz * i)) $dfile $((blksz * i)) $blksz
+ _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile.chk
+ done
+ # 1 blocks are unwritten
+ seq 1 5 $((nr - 1)) | while read i; do
+ $XFS_IO_PROG -f -c "falloc $((blksz * i)) $blksz" $sfile
+ _pwrite_byte 0x00 $((blksz * i)) $blksz $sfile.chk
+ done
+ # 2 blocks are holes
+ seq 2 5 $((nr - 1)) | while read i; do
+ _pwrite_byte 0x00 $((blksz * i)) $blksz $sfile.chk
+ done
+ # 3 blocks are regular
+ seq 3 5 $((nr - 1)) | while read i; do
+ _pwrite_byte 0x71 $((blksz * i)) $blksz $sfile
+ _pwrite_byte 0x71 $((blksz * i)) $blksz $sfile.chk
+ done
+ # 4 blocks will be delalloc later
+}
+
+# For a file created with _sweave_reflink_rainbow, fill the holes with delalloc
+# extents
+_sweave_reflink_rainbow_delalloc() {
+ blksz=$1
+ nr=$2
+ dfile=$3
+
+ # 4 blocks are delalloc (do later)
+ seq 4 5 $((nr - 1)) | while read i; do
+ _pwrite_byte 0x62 $((blksz * i)) $blksz $sfile
+ _pwrite_byte 0x62 $((blksz * i)) $blksz $sfile.chk
+ done
+}
diff --git a/tests/generic/910 b/tests/generic/910
new file mode 100755
index 00000000..32628060
--- /dev/null
+++ b/tests/generic/910
@@ -0,0 +1,65 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 910
+#
+# Ensuring that copy on write in buffered mode works when the CoW
+# range originally covers multiple extents, mixed with reflinked, unwritten,
+# hole, regular and delalloc blocks.
+# - Create a file with the following repeating sequence of blocks:
+# 1. reflinked
+# 2. unwritten
+# 3. hole
+# 4. regular block
+# 5. delalloc
+# - CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone punch
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_rainbow $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_sweave_reflink_rainbow_delalloc $blksz $nr $testdir/file1 >> $seqres.full
+# now cow
+$XFS_IO_PROG -f -c "pwrite -S 0x63 -b $cowsz $cowoff $cowsz" $testdir/file1 >> $seqres.full
+_pwrite_byte 0x63 $cowoff $cowsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
diff --git a/tests/generic/910.out b/tests/generic/910.out
new file mode 100644
index 00000000..12956584
--- /dev/null
+++ b/tests/generic/910.out
@@ -0,0 +1,12 @@
+QA output created by 910
+Format and mount
+Create the original files
+Compare files
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-910/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-910/file3
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-910/file1.chk
+CoW across the transition
+Compare files
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-910/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-910/file3
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-910/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH v3 12/17] generic: add buffered-io CoW test for mixed&source extents
2021-12-14 8:19 ` [PATCH v3 12/17] generic: add buffered-io CoW test for mixed&source extents Shiyang Ruan
@ 2021-12-25 13:27 ` Eryu Guan
0 siblings, 0 replies; 22+ messages in thread
From: Eryu Guan @ 2021-12-25 13:27 UTC (permalink / raw)
To: Shiyang Ruan; +Cc: fstests
On Tue, Dec 14, 2021 at 04:19:09PM +0800, Shiyang Ruan wrote:
> Ensuring that copy on write in buffered mode works when the CoW
> range originally covers multiple extents, mixed with reflinked, unwritten,
> hole, regular and delalloc blocks.
>
> Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
> ---
> common/reflink | 51 +++++++++++++++++++++++++++++++++
> tests/generic/910 | 65 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/910.out | 12 ++++++++
> 3 files changed, 128 insertions(+)
> create mode 100755 tests/generic/910
> create mode 100644 tests/generic/910.out
>
> diff --git a/common/reflink b/common/reflink
> index ee60398e..68dbdedd 100644
> --- a/common/reflink
> +++ b/common/reflink
> @@ -426,3 +426,54 @@ _sweave_reflink_holes_delalloc() {
> _pwrite_byte 0x64 $((blksz * i)) $blksz $sfile.chk
> done
> }
> +
> +# Create a file of interleaved holes, unwritten blocks, regular blocks, and
> +# reflinked blocks
> +_sweave_reflink_rainbow() {
> + blksz=$1
> + nr=$2
> + sfile=$3
> + dfile=$4
These variables should be declared as local.
> +
> + $XFS_IO_PROG -f -c "truncate $((blksz * nr))" $sfile
> + _pwrite_byte 0x00 0 $((blksz * nr)) $sfile.chk
> + _pwrite_byte 0x61 0 $((blksz * nr)) $dfile
> + seq 0 5 $((nr - 1)) | while read i; do
> + _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile
> + _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile.chk
> + done
> + # 0 blocks are reflinked
> + seq 0 5 $((nr - 1)) | while read i; do
> + _reflink_range $sfile $((blksz * i)) $dfile $((blksz * i)) $blksz
> + _pwrite_byte 0x61 $((blksz * i)) $blksz $sfile.chk
> + done
> + # 1 blocks are unwritten
> + seq 1 5 $((nr - 1)) | while read i; do
> + $XFS_IO_PROG -f -c "falloc $((blksz * i)) $blksz" $sfile
> + _pwrite_byte 0x00 $((blksz * i)) $blksz $sfile.chk
> + done
> + # 2 blocks are holes
> + seq 2 5 $((nr - 1)) | while read i; do
> + _pwrite_byte 0x00 $((blksz * i)) $blksz $sfile.chk
> + done
> + # 3 blocks are regular
> + seq 3 5 $((nr - 1)) | while read i; do
> + _pwrite_byte 0x71 $((blksz * i)) $blksz $sfile
> + _pwrite_byte 0x71 $((blksz * i)) $blksz $sfile.chk
> + done
> + # 4 blocks will be delalloc later
> +}
> +
> +# For a file created with _sweave_reflink_rainbow, fill the holes with delalloc
> +# extents
> +_sweave_reflink_rainbow_delalloc() {
> + blksz=$1
> + nr=$2
> + dfile=$3
Same here.
> +
> + # 4 blocks are delalloc (do later)
> + seq 4 5 $((nr - 1)) | while read i; do
> + _pwrite_byte 0x62 $((blksz * i)) $blksz $sfile
> + _pwrite_byte 0x62 $((blksz * i)) $blksz $sfile.chk
And there's no 'sfile' in this function, should be $dfile and $dfile.chk
I've fixed them all on commit.
Thanks,
Eryu
> + done
> +}
> diff --git a/tests/generic/910 b/tests/generic/910
> new file mode 100755
> index 00000000..32628060
> --- /dev/null
> +++ b/tests/generic/910
> @@ -0,0 +1,65 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 910
> +#
> +# Ensuring that copy on write in buffered mode works when the CoW
> +# range originally covers multiple extents, mixed with reflinked, unwritten,
> +# hole, regular and delalloc blocks.
> +# - Create a file with the following repeating sequence of blocks:
> +# 1. reflinked
> +# 2. unwritten
> +# 3. hole
> +# 4. regular block
> +# 5. delalloc
> +# - CoW across the halfway mark, starting with the unwritten extent.
> +# - Check that the files are now different where we say they're different.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick clone punch
> +
> +# Import common functions.
> +. ./common/filter
> +. ./common/reflink
> +
> +# real QA test starts here
> +_require_scratch_reflink
> +_require_scratch_delalloc
> +_require_xfs_io_command "falloc"
> +_require_xfs_io_command "fpunch"
> +
> +echo "Format and mount"
> +_scratch_mkfs > $seqres.full 2>&1
> +_scratch_mount >> $seqres.full 2>&1
> +
> +testdir=$SCRATCH_MNT/test-$seq
> +mkdir $testdir
> +
> +echo "Create the original files"
> +blksz=65536
> +nr=64
> +filesize=$((blksz * nr))
> +_sweave_reflink_rainbow $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
> +_scratch_cycle_mount
> +
> +echo "Compare files"
> +md5sum $testdir/file1 | _filter_scratch
> +md5sum $testdir/file3 | _filter_scratch
> +md5sum $testdir/file1.chk | _filter_scratch
> +
> +echo "CoW across the transition"
> +cowoff=$((filesize / 4))
> +cowsz=$((filesize / 2))
> +_sweave_reflink_rainbow_delalloc $blksz $nr $testdir/file1 >> $seqres.full
> +# now cow
> +$XFS_IO_PROG -f -c "pwrite -S 0x63 -b $cowsz $cowoff $cowsz" $testdir/file1 >> $seqres.full
> +_pwrite_byte 0x63 $cowoff $cowsz $testdir/file1.chk >> $seqres.full
> +_scratch_cycle_mount
> +
> +echo "Compare files"
> +md5sum $testdir/file1 | _filter_scratch
> +md5sum $testdir/file3 | _filter_scratch
> +md5sum $testdir/file1.chk | _filter_scratch
> +
> +# success, all done
> +status=0
> diff --git a/tests/generic/910.out b/tests/generic/910.out
> new file mode 100644
> index 00000000..12956584
> --- /dev/null
> +++ b/tests/generic/910.out
> @@ -0,0 +1,12 @@
> +QA output created by 910
> +Format and mount
> +Create the original files
> +Compare files
> +6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-910/file1
> +bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-910/file3
> +6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-910/file1.chk
> +CoW across the transition
> +Compare files
> +26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-910/file1
> +bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-910/file3
> +26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-910/file1.chk
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 13/17] generic: add direct-io CoW test for mixed&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (11 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 12/17] generic: add buffered-io CoW test for mixed&source extents Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 14/17] generic: add mmap " Shiyang Ruan
` (4 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Ensuring that copy on write in direct-io mode works when the CoW
range originally covers multiple extents, mixed with reflinked, unwritten,
hole, regular and delalloc blocks.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
tests/generic/911 | 66 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/911.out | 12 ++++++++
2 files changed, 78 insertions(+)
create mode 100755 tests/generic/911
create mode 100644 tests/generic/911.out
diff --git a/tests/generic/911 b/tests/generic/911
new file mode 100755
index 00000000..01baa49a
--- /dev/null
+++ b/tests/generic/911
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 911
+#
+# Ensuring that copy on write in direct-io mode works when the CoW
+# range originally covers multiple extents, mixed with reflinked, unwritten,
+# hole, regular and delalloc blocks.
+# - Create a file with the following repeating sequence of blocks:
+# 1. reflinked
+# 2. unwritten
+# 3. hole
+# 4. regular block
+# 5. delalloc
+# - directio CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone punch
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
+_require_odirect
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_rainbow $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "directio CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_sweave_reflink_rainbow_delalloc $blksz $nr $testdir/file1 >> $seqres.full
+# now cow
+$XFS_IO_PROG -d -f -c "pwrite -S 0x63 -b $cowsz $cowoff $cowsz" $testdir/file1 >> $seqres.full
+_pwrite_byte 0x63 $cowoff $cowsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
diff --git a/tests/generic/911.out b/tests/generic/911.out
new file mode 100644
index 00000000..e098fe3c
--- /dev/null
+++ b/tests/generic/911.out
@@ -0,0 +1,12 @@
+QA output created by 911
+Format and mount
+Create the original files
+Compare files
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-911/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-911/file3
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-911/file1.chk
+directio CoW across the transition
+Compare files
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-911/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-911/file3
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-911/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 14/17] generic: add mmap CoW test for mixed&source extents
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (12 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 13/17] generic: add direct-io " Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 15/17] generic: add race test between reflink and mmap read Shiyang Ruan
` (3 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Ensuring that copy on write in mmap mode works when the CoW
range originally covers multiple extents, mixed with reflinked, unwritten,
hole, regular and delalloc blocks.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/912 | 66 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/912.out | 12 ++++++++
2 files changed, 78 insertions(+)
create mode 100755 tests/generic/912
create mode 100644 tests/generic/912.out
diff --git a/tests/generic/912 b/tests/generic/912
new file mode 100755
index 00000000..83e08482
--- /dev/null
+++ b/tests/generic/912
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 912
+#
+# Ensuring that copy on write in mmap mode works when the CoW
+# range originally covers multiple extents, mixed with reflinked, unwritten,
+# hole, regular and delalloc blocks.
+# - Create a file with the following repeating sequence of blocks:
+# 1. reflinked
+# 2. unwritten
+# 3. hole
+# 4. regular block
+# 5. delalloc
+# - mmap CoW across the halfway mark, starting with the unwritten extent.
+# - Check that the files are now different where we say they're different.
+#
+. ./common/preamble
+_begin_fstest auto quick clone punch
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_scratch_delalloc
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+mkdir $testdir
+
+echo "Create the original files"
+blksz=65536
+nr=64
+filesize=$((blksz * nr))
+_sweave_reflink_rainbow $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+echo "mmap CoW across the transition"
+cowoff=$((filesize / 4))
+cowsz=$((filesize / 2))
+_sweave_reflink_rainbow_delalloc $blksz $nr $testdir/file1 >> $seqres.full
+# now cow
+mmapsz=$((cowoff + cowsz))
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1 >> $seqres.full
+_mwrite_byte 0x63 $cowoff $cowsz $mmapsz $testdir/file1.chk >> $seqres.full
+_scratch_cycle_mount
+
+echo "Compare files"
+md5sum $testdir/file1 | _filter_scratch
+md5sum $testdir/file3 | _filter_scratch
+md5sum $testdir/file1.chk | _filter_scratch
+
+# success, all done
+status=0
diff --git a/tests/generic/912.out b/tests/generic/912.out
new file mode 100644
index 00000000..32f25a1d
--- /dev/null
+++ b/tests/generic/912.out
@@ -0,0 +1,12 @@
+QA output created by 912
+Format and mount
+Create the original files
+Compare files
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-912/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-912/file3
+6366fd359371414186688a0ef6988893 SCRATCH_MNT/test-912/file1.chk
+mmap CoW across the transition
+Compare files
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-912/file1
+bdbcf02ee0aa977795a79d25fcfdccb1 SCRATCH_MNT/test-912/file3
+26aa3a0749b867ec58363c8539ee5471 SCRATCH_MNT/test-912/file1.chk
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 15/17] generic: add race test between reflink and mmap read
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (13 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 14/17] generic: add mmap " Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2022-01-11 18:55 ` Darrick J. Wong
2021-12-14 8:19 ` [PATCH v3 16/17] generic: add race test that mmap write to source of reflink Shiyang Ruan
` (2 subsequent siblings)
17 siblings, 1 reply; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst
Test for races or FS corruption between reflink and mmap reading the
target file. (MMAP version of generic/164,165)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
---
common/reflink | 11 +++++++
tests/generic/913 | 72 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/913.out | 5 +++
3 files changed, 88 insertions(+)
create mode 100755 tests/generic/913
create mode 100644 tests/generic/913.out
diff --git a/common/reflink b/common/reflink
index 68dbdedd..455260c6 100644
--- a/common/reflink
+++ b/common/reflink
@@ -186,6 +186,17 @@ _read_range() {
$XFS_IO_PROG $xfs_io_args -f -c "pread -q -v $offset $len" "$file" | cut -d ' ' -f '3-18'
}
+# Prints a range of a file as a hex dump
+_mread_range() {
+ file="$1"
+ offset="$2"
+ len="$3"
+ xfs_io_args="$4"
+
+ $XFS_IO_PROG $xfs_io_args -f -c "mmap -rw 0 $((offset + len))" \
+ -c "mread -v $offset $len" "$file" | cut -d ' ' -f '3-18'
+}
+
# Compare ranges of two files
_compare_range() {
file1="$1"
diff --git a/tests/generic/913 b/tests/generic/913
new file mode 100755
index 00000000..f709c36c
--- /dev/null
+++ b/tests/generic/913
@@ -0,0 +1,72 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 913
+#
+# Test for races or FS corruption between reflink and mmap reading the
+# target file. (MMAP version of generic/164,165)
+#
+. ./common/preamble
+_begin_fstest auto clone
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_cp_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
+mkdir $testdir
+
+loops=512
+nr_loops=$((loops - 1))
+blksz=65536
+
+echo "Initialize files"
+echo >> $seqres.full
+_pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file1 >> $seqres.full
+_pwrite_byte 0x62 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
+_cp_reflink $testdir/file1 $testdir/file3
+_scratch_cycle_mount
+
+fbytes() {
+ egrep -v '(61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61|62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62)'
+}
+
+reader() {
+ while [ ! -e $finished_file ]; do
+ _mread_range $testdir/file3 0 $((loops * blksz)) | fbytes
+ done
+}
+
+echo "Reflink and mmap reread the files!"
+reader &
+for i in `seq 1 2`; do
+ seq $nr_loops -1 0 | while read i; do
+ _reflink_range $testdir/file1 $((i * blksz)) \
+ $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
+ [ $? -ne 0 ] && break
+ done
+ seq $nr_loops -1 0 | while read i; do
+ _reflink_range $testdir/file2 $((i * blksz)) \
+ $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
+ [ $? -ne 0 ] && break
+ done
+done
+echo "Finished reflinking"
+touch $finished_file
+wait
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/913.out b/tests/generic/913.out
new file mode 100644
index 00000000..a34df6ce
--- /dev/null
+++ b/tests/generic/913.out
@@ -0,0 +1,5 @@
+QA output created by 913
+Format and mount
+Initialize files
+Reflink and mmap reread the files!
+Finished reflinking
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH v3 15/17] generic: add race test between reflink and mmap read
2021-12-14 8:19 ` [PATCH v3 15/17] generic: add race test between reflink and mmap read Shiyang Ruan
@ 2022-01-11 18:55 ` Darrick J. Wong
2022-01-12 2:38 ` Shiyang Ruan
0 siblings, 1 reply; 22+ messages in thread
From: Darrick J. Wong @ 2022-01-11 18:55 UTC (permalink / raw)
To: Shiyang Ruan; +Cc: fstests
On Tue, Dec 14, 2021 at 04:19:12PM +0800, Shiyang Ruan wrote:
> Test for races or FS corruption between reflink and mmap reading the
> target file. (MMAP version of generic/164,165)
Hi, now that this test has been running for a couple of weeks, I have
observed periodic false positives from this test:
QA output created by 670
Format and mount
Initialize files
Reflink and mmap reread the files!
00001000: 61 61 61 61 61 61 61 61 62 62 62 62 62 62 62 62 aaaabbbb
Finished reflinking
I suspect that if the _mread_range of file3 races with the page cache
invalidation that FICLONERANGE performs, it is possible that the mread
dump will contain a mix of 0x61 and 0x62. Looking at mread_f, it looks
like it does a byte-at-a-time copy of the mmap...
if (rflag) {
for (tmp = length - 1, c = 0; tmp >= 0; tmp--, c = 1) {
*bp = *(((char *)mapping->addr) + dumpoffset + tmp);
cnt++;
...which is a sufficient window for the page cache mapping to get
invalidated such that the mread will block on the page fault until the
reflink operation finishes.
I think the solution here is to adjust the egrep regexp above to find
any line that does /not/ contain a's or b's, since (in principle) the
reflink could run fast enough that every byte read hits a pgae fault.
What do you think?
--D
> Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
> ---
> common/reflink | 11 +++++++
> tests/generic/913 | 72 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/913.out | 5 +++
> 3 files changed, 88 insertions(+)
> create mode 100755 tests/generic/913
> create mode 100644 tests/generic/913.out
>
> diff --git a/common/reflink b/common/reflink
> index 68dbdedd..455260c6 100644
> --- a/common/reflink
> +++ b/common/reflink
> @@ -186,6 +186,17 @@ _read_range() {
> $XFS_IO_PROG $xfs_io_args -f -c "pread -q -v $offset $len" "$file" | cut -d ' ' -f '3-18'
> }
>
> +# Prints a range of a file as a hex dump
> +_mread_range() {
> + file="$1"
> + offset="$2"
> + len="$3"
> + xfs_io_args="$4"
> +
> + $XFS_IO_PROG $xfs_io_args -f -c "mmap -rw 0 $((offset + len))" \
> + -c "mread -v $offset $len" "$file" | cut -d ' ' -f '3-18'
> +}
> +
> # Compare ranges of two files
> _compare_range() {
> file1="$1"
> diff --git a/tests/generic/913 b/tests/generic/913
> new file mode 100755
> index 00000000..f709c36c
> --- /dev/null
> +++ b/tests/generic/913
> @@ -0,0 +1,72 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 913
> +#
> +# Test for races or FS corruption between reflink and mmap reading the
> +# target file. (MMAP version of generic/164,165)
> +#
> +. ./common/preamble
> +_begin_fstest auto clone
> +
> +_register_cleanup "_cleanup" BUS
> +
> +# Import common functions.
> +. ./common/filter
> +. ./common/reflink
> +
> +# real QA test starts here
> +_require_scratch_reflink
> +_require_cp_reflink
> +
> +echo "Format and mount"
> +_scratch_mkfs > $seqres.full 2>&1
> +_scratch_mount >> $seqres.full 2>&1
> +
> +testdir=$SCRATCH_MNT/test-$seq
> +finished_file=/tmp/finished
> +rm -rf $finished_file
> +mkdir $testdir
> +
> +loops=512
> +nr_loops=$((loops - 1))
> +blksz=65536
> +
> +echo "Initialize files"
> +echo >> $seqres.full
> +_pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file1 >> $seqres.full
> +_pwrite_byte 0x62 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
> +_cp_reflink $testdir/file1 $testdir/file3
> +_scratch_cycle_mount
> +
> +fbytes() {
> + egrep -v '(61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61|62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62)'
> +}
> +
> +reader() {
> + while [ ! -e $finished_file ]; do
> + _mread_range $testdir/file3 0 $((loops * blksz)) | fbytes
> + done
> +}
> +
> +echo "Reflink and mmap reread the files!"
> +reader &
> +for i in `seq 1 2`; do
> + seq $nr_loops -1 0 | while read i; do
> + _reflink_range $testdir/file1 $((i * blksz)) \
> + $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
> + [ $? -ne 0 ] && break
> + done
> + seq $nr_loops -1 0 | while read i; do
> + _reflink_range $testdir/file2 $((i * blksz)) \
> + $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
> + [ $? -ne 0 ] && break
> + done
> +done
> +echo "Finished reflinking"
> +touch $finished_file
> +wait
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/913.out b/tests/generic/913.out
> new file mode 100644
> index 00000000..a34df6ce
> --- /dev/null
> +++ b/tests/generic/913.out
> @@ -0,0 +1,5 @@
> +QA output created by 913
> +Format and mount
> +Initialize files
> +Reflink and mmap reread the files!
> +Finished reflinking
> --
> 2.34.1
>
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v3 15/17] generic: add race test between reflink and mmap read
2022-01-11 18:55 ` Darrick J. Wong
@ 2022-01-12 2:38 ` Shiyang Ruan
0 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2022-01-12 2:38 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: fstests, ruansy.fnst
在 2022/1/12 2:55, Darrick J. Wong 写道:
> On Tue, Dec 14, 2021 at 04:19:12PM +0800, Shiyang Ruan wrote:
>> Test for races or FS corruption between reflink and mmap reading the
>> target file. (MMAP version of generic/164,165)
>
> Hi, now that this test has been running for a couple of weeks, I have
> observed periodic false positives from this test:
>
> QA output created by 670
> Format and mount
> Initialize files
> Reflink and mmap reread the files!
> 00001000: 61 61 61 61 61 61 61 61 62 62 62 62 62 62 62 62 aaaabbbb
> Finished reflinking
>
> I suspect that if the _mread_range of file3 races with the page cache
> invalidation that FICLONERANGE performs, it is possible that the mread
> dump will contain a mix of 0x61 and 0x62. Looking at mread_f, it looks
> like it does a byte-at-a-time copy of the mmap...
>
> if (rflag) {
> for (tmp = length - 1, c = 0; tmp >= 0; tmp--, c = 1) {
> *bp = *(((char *)mapping->addr) + dumpoffset + tmp);
> cnt++;
>
> ...which is a sufficient window for the page cache mapping to get
> invalidated such that the mread will block on the page fault until the
> reflink operation finishes.
>
> I think the solution here is to adjust the egrep regexp above to find
> any line that does /not/ contain a's or b's, since (in principle) the
> reflink could run fast enough that every byte read hits a pgae fault.
> What do you think?
Reasonable. I couldn't reproduce it in my test environment, so I didn't
take that situation into consideration.
I'll change the regexp as below:
fbytes() {
- egrep -v '(61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61|62 62 62 62
62 62 62 62 62 62 62 62 62 62 62 62)'
+ egrep -v '((61|62) ){15}(61|62)'
}
--
Thanks,
Ruan.
>
> --D
>
>> Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
>> ---
>> common/reflink | 11 +++++++
>> tests/generic/913 | 72 +++++++++++++++++++++++++++++++++++++++++++
>> tests/generic/913.out | 5 +++
>> 3 files changed, 88 insertions(+)
>> create mode 100755 tests/generic/913
>> create mode 100644 tests/generic/913.out
>>
>> diff --git a/common/reflink b/common/reflink
>> index 68dbdedd..455260c6 100644
>> --- a/common/reflink
>> +++ b/common/reflink
>> @@ -186,6 +186,17 @@ _read_range() {
>> $XFS_IO_PROG $xfs_io_args -f -c "pread -q -v $offset $len" "$file" | cut -d ' ' -f '3-18'
>> }
>>
>> +# Prints a range of a file as a hex dump
>> +_mread_range() {
>> + file="$1"
>> + offset="$2"
>> + len="$3"
>> + xfs_io_args="$4"
>> +
>> + $XFS_IO_PROG $xfs_io_args -f -c "mmap -rw 0 $((offset + len))" \
>> + -c "mread -v $offset $len" "$file" | cut -d ' ' -f '3-18'
>> +}
>> +
>> # Compare ranges of two files
>> _compare_range() {
>> file1="$1"
>> diff --git a/tests/generic/913 b/tests/generic/913
>> new file mode 100755
>> index 00000000..f709c36c
>> --- /dev/null
>> +++ b/tests/generic/913
>> @@ -0,0 +1,72 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +#
>> +# FS QA Test No. 913
>> +#
>> +# Test for races or FS corruption between reflink and mmap reading the
>> +# target file. (MMAP version of generic/164,165)
>> +#
>> +. ./common/preamble
>> +_begin_fstest auto clone
>> +
>> +_register_cleanup "_cleanup" BUS
>> +
>> +# Import common functions.
>> +. ./common/filter
>> +. ./common/reflink
>> +
>> +# real QA test starts here
>> +_require_scratch_reflink
>> +_require_cp_reflink
>> +
>> +echo "Format and mount"
>> +_scratch_mkfs > $seqres.full 2>&1
>> +_scratch_mount >> $seqres.full 2>&1
>> +
>> +testdir=$SCRATCH_MNT/test-$seq
>> +finished_file=/tmp/finished
>> +rm -rf $finished_file
>> +mkdir $testdir
>> +
>> +loops=512
>> +nr_loops=$((loops - 1))
>> +blksz=65536
>> +
>> +echo "Initialize files"
>> +echo >> $seqres.full
>> +_pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file1 >> $seqres.full
>> +_pwrite_byte 0x62 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
>> +_cp_reflink $testdir/file1 $testdir/file3
>> +_scratch_cycle_mount
>> +
>> +fbytes() {
>> + egrep -v '(61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61|62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62)'
>> +}
>> +
>> +reader() {
>> + while [ ! -e $finished_file ]; do
>> + _mread_range $testdir/file3 0 $((loops * blksz)) | fbytes
>> + done
>> +}
>> +
>> +echo "Reflink and mmap reread the files!"
>> +reader &
>> +for i in `seq 1 2`; do
>> + seq $nr_loops -1 0 | while read i; do
>> + _reflink_range $testdir/file1 $((i * blksz)) \
>> + $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
>> + [ $? -ne 0 ] && break
>> + done
>> + seq $nr_loops -1 0 | while read i; do
>> + _reflink_range $testdir/file2 $((i * blksz)) \
>> + $testdir/file3 $((i * blksz)) $blksz >> $seqres.full
>> + [ $? -ne 0 ] && break
>> + done
>> +done
>> +echo "Finished reflinking"
>> +touch $finished_file
>> +wait
>> +
>> +# success, all done
>> +status=0
>> +exit
>> diff --git a/tests/generic/913.out b/tests/generic/913.out
>> new file mode 100644
>> index 00000000..a34df6ce
>> --- /dev/null
>> +++ b/tests/generic/913.out
>> @@ -0,0 +1,5 @@
>> +QA output created by 913
>> +Format and mount
>> +Initialize files
>> +Reflink and mmap reread the files!
>> +Finished reflinking
>> --
>> 2.34.1
>>
>>
>>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 16/17] generic: add race test that mmap write to source of reflink
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (14 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 15/17] generic: add race test between reflink and mmap read Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-14 8:19 ` [PATCH v3 17/17] generic: add race test that mmap write to target " Shiyang Ruan
2021-12-19 16:04 ` [PATCH v3 00/17] generic: add some mmap CoW tests Eryu Guan
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Test for races or FS corruption when mmap writing to a file that's also
the source of a reflink operation. (MMAP version of generic/167,166)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/914 | 64 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/914.out | 4 +++
2 files changed, 68 insertions(+)
create mode 100755 tests/generic/914
create mode 100644 tests/generic/914.out
diff --git a/tests/generic/914 b/tests/generic/914
new file mode 100755
index 00000000..c51c44c7
--- /dev/null
+++ b/tests/generic/914
@@ -0,0 +1,64 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 914
+#
+# Test for races or FS corruption when mmap writing to a file that's also
+# the source of a reflink operation. (MMAP version of generic/167,166)
+#
+. ./common/preamble
+_begin_fstest auto clone
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+_require_cp_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
+mkdir $testdir
+
+loops=1024
+nr_loops=$((loops - 1))
+blksz=65536
+
+echo "Initialize file"
+echo >> $seqres.full
+_pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file1 >> $seqres.full
+_scratch_cycle_mount
+
+# Snapshot creator...
+snappy() {
+ n=0
+ while [ ! -e $finished_file ]; do
+ out="$(_cp_reflink $testdir/file1 $testdir/snap_$n 2>&1)"
+ res=$?
+ echo "$out" | grep -q "No space left" && break
+ test -n "$out" && echo "$out"
+ test $res -ne 0 && break
+ n=$((n + 1))
+ done
+}
+
+echo "Snapshot a file undergoing mmap rewrite"
+snappy &
+seq $nr_loops -1 0 | while read i; do
+ $XFS_IO_PROG -f -c "mmap -rw $((i * blksz)) $blksz" \
+ -c "mwrite -S 0x63 $((i * blksz)) $blksz" $testdir/file1 >> $seqres.full
+done
+touch $finished_file
+wait
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/914.out b/tests/generic/914.out
new file mode 100644
index 00000000..1472055f
--- /dev/null
+++ b/tests/generic/914.out
@@ -0,0 +1,4 @@
+QA output created by 914
+Format and mount
+Initialize file
+Snapshot a file undergoing mmap rewrite
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH v3 17/17] generic: add race test that mmap write to target of reflink
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (15 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 16/17] generic: add race test that mmap write to source of reflink Shiyang Ruan
@ 2021-12-14 8:19 ` Shiyang Ruan
2021-12-19 16:04 ` [PATCH v3 00/17] generic: add some mmap CoW tests Eryu Guan
17 siblings, 0 replies; 22+ messages in thread
From: Shiyang Ruan @ 2021-12-14 8:19 UTC (permalink / raw)
To: fstests; +Cc: ruansy.fnst, Darrick J . Wong
Test for races or FS corruption when mmap writing to a file that's also
the target of a reflink operation. (MMAP version of generic/168,170)
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/915 | 64 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/915.out | 4 +++
2 files changed, 68 insertions(+)
create mode 100755 tests/generic/915
create mode 100644 tests/generic/915.out
diff --git a/tests/generic/915 b/tests/generic/915
new file mode 100755
index 00000000..365f304c
--- /dev/null
+++ b/tests/generic/915
@@ -0,0 +1,64 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 915
+#
+# Test for races or FS corruption when mmap writing to a file that's also
+# the target of a reflink operation. (MMAP version of generic/168,170)
+#
+. ./common/preamble
+_begin_fstest auto clone
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_require_scratch_reflink
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+testdir=$SCRATCH_MNT/test-$seq
+finished_file=/tmp/finished
+rm -rf $finished_file
+mkdir $testdir
+
+loops=1024
+nr_loops=$((loops - 1))
+blksz=65536
+
+echo "Initialize files"
+echo >> $seqres.full
+_pwrite_byte 0x61 0 $((loops * blksz)) $testdir/file1 >> $seqres.full
+_pwrite_byte 0x62 0 $((loops * blksz)) $testdir/file2 >> $seqres.full
+_scratch_cycle_mount
+
+# mmap overwriter...
+overwrite() {
+ while [ ! -e $finished_file ]; do
+ seq $nr_loops -1 0 | while read i; do
+ $XFS_IO_PROG -f -c "mmap -rw $((i * blksz)) $blksz" \
+ -c "mwrite -S 0x63 $((i * blksz)) $blksz" $testdir/file2 >> $seqres.full
+ done
+ done
+}
+
+echo "Reflink and mmap write the target"
+overwrite &
+seq 1 10 | while read j; do
+ seq 0 $nr_loops | while read i; do
+ _reflink_range $testdir/file1 $((i * blksz)) \
+ $testdir/file2 $((i * blksz)) $blksz >> $seqres.full
+ [ $? -ne 0 ] && exit
+ done
+done
+touch $finished_file
+wait
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/915.out b/tests/generic/915.out
new file mode 100644
index 00000000..4e77847b
--- /dev/null
+++ b/tests/generic/915.out
@@ -0,0 +1,4 @@
+QA output created by 915
+Format and mount
+Initialize files
+Reflink and mmap write the target
--
2.34.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH v3 00/17] generic: add some mmap CoW tests
2021-12-14 8:18 [PATCH v3 00/17] generic: add some mmap CoW tests Shiyang Ruan
` (16 preceding siblings ...)
2021-12-14 8:19 ` [PATCH v3 17/17] generic: add race test that mmap write to target " Shiyang Ruan
@ 2021-12-19 16:04 ` Eryu Guan
17 siblings, 0 replies; 22+ messages in thread
From: Eryu Guan @ 2021-12-19 16:04 UTC (permalink / raw)
To: Shiyang Ruan; +Cc: fstests
On Tue, Dec 14, 2021 at 04:18:57PM +0800, Shiyang Ruan wrote:
> Changes since v1:
> - Introduce _require_scratch_delalloc() to check if supports delay allocation
> - Remove useless _require_odirect and _require_cp_reflink in some cases
>
> This series is to add missing mmap CoW tests in some sets of testcases:
> Set 1. CoW on shared blocks which are interleaving reflinked by two files.
> - P1: add mmap test
> Set 2. CoW a range covers shared blocks and multiple kind of unshared blocks of
> the destination file.
> - P2 ~ P4, P6: add mmap test
> Set 3. CoW a range covers shared blocks and multiple kind of unshared blocks of
> the source file.
> - P12, P13: add missing buffered-io and direct-io test for mixed blocks
> - P8 ~ P11, P14: add mmap test
> Set 4. races tests
> - P15: race between reflink and mmap reading
> - P16: race between mmap writing to and reflink source file
> - P17: race between mmap writing to and reflink target file
Looks good to me overall, thanks!. But it's a large patchset, I'd like
wait for another week for feedbacks, and will merge it next week if
there's no further comments
Thanks,
Eryu
>
> Shiyang Ruan (17):
> generic: add mmap CoW test for ranges of two shared files
> generic: add mmap CoW test for regular&destination extents
> generic: add mmap CoW test for unwritten&destination extents
> generic: add mmap CoW test for holes&destination extents
> common/rc: Introduce _require_scratch_delalloc()
> generic: add mmap CoW test for delalloc&destination extents
> generic: add mmap CoW test for mixed&destination extents
> generic: add mmap CoW test for regular&source extents
> generic: add mmap CoW test for unwritten&source extents
> generic: add mmap CoW test for holes&source extents
> generic: add mmap CoW test for delalloc&source extents
> generic: add buffered-io CoW test for mixed&source extents
> generic: add direct-io CoW test for mixed&source extents
> generic: add mmap CoW test for mixed&source extents
> generic: add race test between reflink and mmap read
> generic: add race test that mmap write to source of reflink
> generic: add race test that mmap write to target of reflink
>
> common/rc | 13 ++++++++
> common/reflink | 62 +++++++++++++++++++++++++++++++++++++
> tests/generic/194 | 1 +
> tests/generic/195 | 1 +
> tests/generic/199 | 1 +
> tests/generic/200 | 1 +
> tests/generic/293 | 1 +
> tests/generic/295 | 1 +
> tests/generic/540 | 1 +
> tests/generic/541 | 1 +
> tests/generic/542 | 1 +
> tests/generic/543 | 1 +
> tests/generic/900 | 69 +++++++++++++++++++++++++++++++++++++++++
> tests/generic/900.out | 14 +++++++++
> tests/generic/901 | 59 +++++++++++++++++++++++++++++++++++
> tests/generic/901.out | 12 ++++++++
> tests/generic/902 | 59 +++++++++++++++++++++++++++++++++++
> tests/generic/902.out | 12 ++++++++
> tests/generic/903 | 59 +++++++++++++++++++++++++++++++++++
> tests/generic/903.out | 12 ++++++++
> tests/generic/904 | 62 +++++++++++++++++++++++++++++++++++++
> tests/generic/904.out | 12 ++++++++
> tests/generic/905 | 67 ++++++++++++++++++++++++++++++++++++++++
> tests/generic/905.out | 12 ++++++++
> tests/generic/906 | 60 ++++++++++++++++++++++++++++++++++++
> tests/generic/906.out | 12 ++++++++
> tests/generic/907 | 62 +++++++++++++++++++++++++++++++++++++
> tests/generic/907.out | 12 ++++++++
> tests/generic/908 | 62 +++++++++++++++++++++++++++++++++++++
> tests/generic/908.out | 12 ++++++++
> tests/generic/909 | 65 ++++++++++++++++++++++++++++++++++++++
> tests/generic/909.out | 12 ++++++++
> tests/generic/910 | 65 ++++++++++++++++++++++++++++++++++++++
> tests/generic/910.out | 12 ++++++++
> tests/generic/911 | 66 +++++++++++++++++++++++++++++++++++++++
> tests/generic/911.out | 12 ++++++++
> tests/generic/912 | 66 +++++++++++++++++++++++++++++++++++++++
> tests/generic/912.out | 12 ++++++++
> tests/generic/913 | 72 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/913.out | 5 +++
> tests/generic/914 | 64 ++++++++++++++++++++++++++++++++++++++
> tests/generic/914.out | 4 +++
> tests/generic/915 | 64 ++++++++++++++++++++++++++++++++++++++
> tests/generic/915.out | 4 +++
> tests/xfs/172 | 12 +-------
> tests/xfs/223 | 1 +
> tests/xfs/224 | 1 +
> tests/xfs/228 | 1 +
> tests/xfs/230 | 1 +
> tests/xfs/257 | 1 +
> tests/xfs/258 | 1 +
> 51 files changed, 1284 insertions(+), 11 deletions(-)
> create mode 100755 tests/generic/900
> create mode 100644 tests/generic/900.out
> create mode 100755 tests/generic/901
> create mode 100644 tests/generic/901.out
> create mode 100755 tests/generic/902
> create mode 100644 tests/generic/902.out
> create mode 100755 tests/generic/903
> create mode 100644 tests/generic/903.out
> create mode 100755 tests/generic/904
> create mode 100644 tests/generic/904.out
> create mode 100755 tests/generic/905
> create mode 100644 tests/generic/905.out
> create mode 100755 tests/generic/906
> create mode 100644 tests/generic/906.out
> create mode 100755 tests/generic/907
> create mode 100644 tests/generic/907.out
> create mode 100755 tests/generic/908
> create mode 100644 tests/generic/908.out
> create mode 100755 tests/generic/909
> create mode 100644 tests/generic/909.out
> create mode 100755 tests/generic/910
> create mode 100644 tests/generic/910.out
> create mode 100755 tests/generic/911
> create mode 100644 tests/generic/911.out
> create mode 100755 tests/generic/912
> create mode 100644 tests/generic/912.out
> create mode 100755 tests/generic/913
> create mode 100644 tests/generic/913.out
> create mode 100755 tests/generic/914
> create mode 100644 tests/generic/914.out
> create mode 100755 tests/generic/915
> create mode 100644 tests/generic/915.out
>
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread