* xfs post-EOF block freeing fixes
@ 2024-06-23 5:38 Christoph Hellwig
2024-06-23 5:38 ` [PATCH] xfs: new EOF fragmentation tests Christoph Hellwig
0 siblings, 1 reply; 4+ messages in thread
From: Christoph Hellwig @ 2024-06-23 5:38 UTC (permalink / raw)
To: Zorro Lang; +Cc: Darrick J. Wong, Dave Chinner, linux-xfs, fstests
Hi all,
this is a rebased of the tests dave sent 5 years ago for the xfs post-EOF
block freeing.
Without the patches here:
https://lore.kernel.org/linux-xfs/20240623053532.857496-1-hch@lst.de/
all of the new tests will fail for the default configuration.
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH] xfs: new EOF fragmentation tests
2024-06-23 5:38 xfs post-EOF block freeing fixes Christoph Hellwig
@ 2024-06-23 5:38 ` Christoph Hellwig
2024-06-23 13:03 ` Zorro Lang
0 siblings, 1 reply; 4+ messages in thread
From: Christoph Hellwig @ 2024-06-23 5:38 UTC (permalink / raw)
To: Zorro Lang; +Cc: Darrick J. Wong, Dave Chinner, linux-xfs, fstests
From: Dave Chinner <dchinner@redhat.com>
These tests create substantial file fragmentation as a result of
application actions that defeat post-EOF preallocation
optimisations. They are intended to replicate known vectors for
these problems, and provide a check that the fragmentation levels
have been controlled. The mitigations we make may not completely
remove fragmentation (e.g. they may demonstrate speculative delalloc
related extent size growth) so the checks don't assume we'll end up
with perfect layouts and hence check for an exceptable level of
fragmentation rather than none.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
[move to different test number, update to current xfstest APIs]
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/xfs/1500 | 59 ++++++++++++++++++++++++++++++++++++++
tests/xfs/1500.out | 9 ++++++
tests/xfs/1501 | 61 ++++++++++++++++++++++++++++++++++++++++
tests/xfs/1501.out | 9 ++++++
tests/xfs/1502 | 61 ++++++++++++++++++++++++++++++++++++++++
tests/xfs/1502.out | 9 ++++++
tests/xfs/1503 | 70 ++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/1503.out | 33 ++++++++++++++++++++++
8 files changed, 311 insertions(+)
create mode 100755 tests/xfs/1500
create mode 100644 tests/xfs/1500.out
create mode 100755 tests/xfs/1501
create mode 100644 tests/xfs/1501.out
create mode 100755 tests/xfs/1502
create mode 100644 tests/xfs/1502.out
create mode 100755 tests/xfs/1503
create mode 100644 tests/xfs/1503.out
diff --git a/tests/xfs/1500 b/tests/xfs/1500
new file mode 100755
index 000000000..222e90d6c
--- /dev/null
+++ b/tests/xfs/1500
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test xfs/500
+#
+# Post-EOF preallocation defeat test for O_SYNC buffered I/O.
+#
+
+. ./common/preamble
+_begin_fstest auto quick prealloc rw
+
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to
+# interleave allocations to fragment the files. Synchronous writes defeat the
+# open/write/close heuristics in xfs_file_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of >900 (almost worse case) whilst fixed behaviour
+# typically shows extent counts in the low 20s.
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+ idx=$1
+
+ for ((cnt=0; cnt<$wcnt; cnt++)); do
+ $XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+ done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+ write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+ count=$(_count_extents $workfile.$n)
+ # Acceptible extent count range is 1-40
+ _within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+status=0
+exit
diff --git a/tests/xfs/1500.out b/tests/xfs/1500.out
new file mode 100644
index 000000000..414df87ed
--- /dev/null
+++ b/tests/xfs/1500.out
@@ -0,0 +1,9 @@
+QA output created by 1500
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
diff --git a/tests/xfs/1501 b/tests/xfs/1501
new file mode 100755
index 000000000..beae49bff
--- /dev/null
+++ b/tests/xfs/1501
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test xfs/501
+#
+# Post-EOF preallocation defeat test for buffered I/O with extent size hints.
+#
+
+. ./common/preamble
+_begin_fstest auto quick prealloc rw
+
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using buffered writes with extent size hints.
+# Aim is to interleave allocations to fragment the files. Writes w/ extent size
+# hints set defeat the open/write/close heuristics in xfs_file_release() that
+# prevent EOF block removal, so this should fragment badly. Typical problematic
+# behaviour shows per-file extent counts of 1000 (worst case!) whilst
+# fixed behaviour should show very few extents (almost best case).
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+extent_size=16m
+
+write_extsz_file()
+{
+ idx=$1
+
+ $XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
+ for ((cnt=0; cnt<$wcnt; cnt++)); do
+ $XFS_IO_PROG -f -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+ done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+ write_extsz_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+ count=$(_count_extents $workfile.$n)
+ # Acceptible extent count range is 1-10
+ _within_tolerance "file.$n extent count" $count 2 1 8 -v
+done
+
+status=0
+exit
diff --git a/tests/xfs/1501.out b/tests/xfs/1501.out
new file mode 100644
index 000000000..a266ef74b
--- /dev/null
+++ b/tests/xfs/1501.out
@@ -0,0 +1,9 @@
+QA output created by 1501
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
diff --git a/tests/xfs/1502 b/tests/xfs/1502
new file mode 100755
index 000000000..9d303ced7
--- /dev/null
+++ b/tests/xfs/1502
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test xfs/502
+#
+# Post-EOF preallocation defeat test for direct I/O with extent size hints.
+#
+
+. ./common/preamble
+_begin_fstest auto quick prealloc rw
+
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
+# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
+# the open/write/close heuristics in xfs_file_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
+# shows extent counts in the low single digits (almost best case)
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+extent_size=16m
+
+write_direct_file()
+{
+ idx=$1
+
+ $XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
+ for ((cnt=0; cnt<$wcnt; cnt++)); do
+ $XFS_IO_PROG -f -d -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+ done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+ write_direct_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+ count=$(_count_extents $workfile.$n)
+ # Acceptible extent count range is 1-10
+ _within_tolerance "file.$n extent count" $count 2 1 8 -v
+done
+
+status=0
+exit
diff --git a/tests/xfs/1502.out b/tests/xfs/1502.out
new file mode 100644
index 000000000..82c8760a3
--- /dev/null
+++ b/tests/xfs/1502.out
@@ -0,0 +1,9 @@
+QA output created by 1502
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
diff --git a/tests/xfs/1503 b/tests/xfs/1503
new file mode 100755
index 000000000..41f4035ad
--- /dev/null
+++ b/tests/xfs/1503
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test xfs/503
+#
+# Post-EOF preallocation defeat test with O_SYNC buffered I/O that repeatedly
+# closes and reopens the files.
+#
+
+. ./common/preamble
+_begin_fstest auto prealloc rw
+
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes that
+# repeatedly close and reopen the fails. Aim is to interleave allocations to
+# fragment the files. Assuming we've fixed the synchronous write defeat, we can
+# still trigger the same issue with a open/read/close on O_RDONLY files. We
+# should not be triggering EOF preallocation removal on files we don't have
+# permission to write, so until this is fixed it should fragment badly. Typical
+# problematic behaviour shows per-file extent counts of 50-350 whilst fixed
+# behaviour typically demonstrates post-eof speculative delalloc growth in
+# extent size (~6 extents for 50MB file).
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=32
+wsize=4096
+wcnt=1000
+
+write_file()
+{
+ idx=$1
+
+ $XFS_IO_PROG -f -s -c "pwrite -b 64k 0 50m" $workfile.$idx
+}
+
+read_file()
+{
+ idx=$1
+
+ for ((cnt=0; cnt<$wcnt; cnt++)); do
+ $XFS_IO_PROG -f -r -c "pread 0 28" $workfile.$idx
+ done
+}
+
+rm -f $workdir/file*
+for ((n=0; n<$((nfiles)); n++)); do
+ write_file $n > /dev/null 2>&1 &
+ read_file $n > /dev/null 2>&1 &
+done
+wait
+
+for ((n=0; n<$nfiles; n++)); do
+ count=$(_count_extents $workfile.$n)
+ # Acceptible extent count range is 1-40
+ _within_tolerance "file.$n extent count" $count 6 5 10 -v
+done
+
+status=0
+exit
diff --git a/tests/xfs/1503.out b/tests/xfs/1503.out
new file mode 100644
index 000000000..1780b16df
--- /dev/null
+++ b/tests/xfs/1503.out
@@ -0,0 +1,33 @@
+QA output created by 1503
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
+file.8 extent count is in range
+file.9 extent count is in range
+file.10 extent count is in range
+file.11 extent count is in range
+file.12 extent count is in range
+file.13 extent count is in range
+file.14 extent count is in range
+file.15 extent count is in range
+file.16 extent count is in range
+file.17 extent count is in range
+file.18 extent count is in range
+file.19 extent count is in range
+file.20 extent count is in range
+file.21 extent count is in range
+file.22 extent count is in range
+file.23 extent count is in range
+file.24 extent count is in range
+file.25 extent count is in range
+file.26 extent count is in range
+file.27 extent count is in range
+file.28 extent count is in range
+file.29 extent count is in range
+file.30 extent count is in range
+file.31 extent count is in range
--
2.43.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] xfs: new EOF fragmentation tests
2024-06-23 5:38 ` [PATCH] xfs: new EOF fragmentation tests Christoph Hellwig
@ 2024-06-23 13:03 ` Zorro Lang
2024-06-24 8:49 ` Christoph Hellwig
0 siblings, 1 reply; 4+ messages in thread
From: Zorro Lang @ 2024-06-23 13:03 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Zorro Lang, Darrick J. Wong, Dave Chinner, linux-xfs, fstests
On Sun, Jun 23, 2024 at 07:38:53AM +0200, Christoph Hellwig wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> These tests create substantial file fragmentation as a result of
> application actions that defeat post-EOF preallocation
> optimisations. They are intended to replicate known vectors for
> these problems, and provide a check that the fragmentation levels
> have been controlled. The mitigations we make may not completely
> remove fragmentation (e.g. they may demonstrate speculative delalloc
> related extent size growth) so the checks don't assume we'll end up
> with perfect layouts and hence check for an exceptable level of
> fragmentation rather than none.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> [move to different test number, update to current xfstest APIs]
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> tests/xfs/1500 | 59 ++++++++++++++++++++++++++++++++++++++
> tests/xfs/1500.out | 9 ++++++
> tests/xfs/1501 | 61 ++++++++++++++++++++++++++++++++++++++++
> tests/xfs/1501.out | 9 ++++++
> tests/xfs/1502 | 61 ++++++++++++++++++++++++++++++++++++++++
> tests/xfs/1502.out | 9 ++++++
> tests/xfs/1503 | 70 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/1503.out | 33 ++++++++++++++++++++++
> 8 files changed, 311 insertions(+)
> create mode 100755 tests/xfs/1500
> create mode 100644 tests/xfs/1500.out
> create mode 100755 tests/xfs/1501
> create mode 100644 tests/xfs/1501.out
> create mode 100755 tests/xfs/1502
> create mode 100644 tests/xfs/1502.out
> create mode 100755 tests/xfs/1503
> create mode 100644 tests/xfs/1503.out
>
> diff --git a/tests/xfs/1500 b/tests/xfs/1500
> new file mode 100755
> index 000000000..222e90d6c
> --- /dev/null
> +++ b/tests/xfs/1500
> @@ -0,0 +1,59 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
^^^^
Should I keep "2019" or change it to 2024, or no matter :)
> +#
> +# FS QA Test xfs/500
> +#
> +# Post-EOF preallocation defeat test for O_SYNC buffered I/O.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick prealloc rw
> +
> +. ./common/rc
> +. ./common/filter
> +
> +_supported_fs xfs
> +_require_scratch
> +
> +_scratch_mkfs 2>&1 >> $seqres.full
Wrong order:) ">>$seqres.full 2>&1"
> +_scratch_mount
> +
> +# Write multiple files in parallel using synchronous buffered writes. Aim is to
> +# interleave allocations to fragment the files. Synchronous writes defeat the
> +# open/write/close heuristics in xfs_file_release() that prevent EOF block
> +# removal, so this should fragment badly. Typical problematic behaviour shows
> +# per-file extent counts of >900 (almost worse case) whilst fixed behaviour
> +# typically shows extent counts in the low 20s.
> +#
> +# Failure is determined by golden output mismatch from _within_tolerance().
> +
> +workfile=$SCRATCH_MNT/file
> +nfiles=8
> +wsize=4096
> +wcnt=1000
> +
> +write_sync_file()
> +{
> + idx=$1
> +
> + for ((cnt=0; cnt<$wcnt; cnt++)); do
> + $XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
> + done
> +}
> +
> +rm -f $workfile*
> +for ((n=0; n<$nfiles; n++)); do
> + write_sync_file $n > /dev/null 2>&1 &
> +done
> +wait
If a case has background process, better to do kill&wait in _cleanup,
for unexpected Ctrl^C when fstests is running. e.g.
_cleanup{}
{
# try to kill all background processes
wait
cd /
rm -r -f $tmp.*
}
I'm not sure if there's a good to way to do "kill all background
processes" in this case, or if it's necessary. But a "wait" might
be worth, to avoid the "Device Busy" error. Any thoughts?
Thanks,
Zorro
> +sync
> +
> +for ((n=0; n<$nfiles; n++)); do
> + count=$(_count_extents $workfile.$n)
> + # Acceptible extent count range is 1-40
> + _within_tolerance "file.$n extent count" $count 21 19 -v
> +done
> +
> +status=0
> +exit
> diff --git a/tests/xfs/1500.out b/tests/xfs/1500.out
> new file mode 100644
> index 000000000..414df87ed
> --- /dev/null
> +++ b/tests/xfs/1500.out
> @@ -0,0 +1,9 @@
> +QA output created by 1500
> +file.0 extent count is in range
> +file.1 extent count is in range
> +file.2 extent count is in range
> +file.3 extent count is in range
> +file.4 extent count is in range
> +file.5 extent count is in range
> +file.6 extent count is in range
> +file.7 extent count is in range
> diff --git a/tests/xfs/1501 b/tests/xfs/1501
> new file mode 100755
> index 000000000..beae49bff
> --- /dev/null
> +++ b/tests/xfs/1501
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test xfs/501
> +#
> +# Post-EOF preallocation defeat test for buffered I/O with extent size hints.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick prealloc rw
> +
> +. ./common/rc
> +. ./common/filter
> +
> +_supported_fs xfs
> +_require_scratch
> +
> +_scratch_mkfs 2>&1 >> $seqres.full
> +_scratch_mount
> +
> +# Write multiple files in parallel using buffered writes with extent size hints.
> +# Aim is to interleave allocations to fragment the files. Writes w/ extent size
> +# hints set defeat the open/write/close heuristics in xfs_file_release() that
> +# prevent EOF block removal, so this should fragment badly. Typical problematic
> +# behaviour shows per-file extent counts of 1000 (worst case!) whilst
> +# fixed behaviour should show very few extents (almost best case).
> +#
> +# Failure is determined by golden output mismatch from _within_tolerance().
> +
> +workfile=$SCRATCH_MNT/file
> +nfiles=8
> +wsize=4096
> +wcnt=1000
> +extent_size=16m
> +
> +write_extsz_file()
> +{
> + idx=$1
> +
> + $XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
> + for ((cnt=0; cnt<$wcnt; cnt++)); do
> + $XFS_IO_PROG -f -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
> + done
> +}
> +
> +rm -f $workfile*
> +for ((n=0; n<$nfiles; n++)); do
> + write_extsz_file $n > /dev/null 2>&1 &
> +done
> +wait
> +sync
> +
> +for ((n=0; n<$nfiles; n++)); do
> + count=$(_count_extents $workfile.$n)
> + # Acceptible extent count range is 1-10
> + _within_tolerance "file.$n extent count" $count 2 1 8 -v
> +done
> +
> +status=0
> +exit
> diff --git a/tests/xfs/1501.out b/tests/xfs/1501.out
> new file mode 100644
> index 000000000..a266ef74b
> --- /dev/null
> +++ b/tests/xfs/1501.out
> @@ -0,0 +1,9 @@
> +QA output created by 1501
> +file.0 extent count is in range
> +file.1 extent count is in range
> +file.2 extent count is in range
> +file.3 extent count is in range
> +file.4 extent count is in range
> +file.5 extent count is in range
> +file.6 extent count is in range
> +file.7 extent count is in range
> diff --git a/tests/xfs/1502 b/tests/xfs/1502
> new file mode 100755
> index 000000000..9d303ced7
> --- /dev/null
> +++ b/tests/xfs/1502
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test xfs/502
> +#
> +# Post-EOF preallocation defeat test for direct I/O with extent size hints.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick prealloc rw
> +
> +. ./common/rc
> +. ./common/filter
> +
> +_supported_fs xfs
> +_require_scratch
> +
> +_scratch_mkfs 2>&1 >> $seqres.full
> +_scratch_mount
> +
> +# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
> +# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
> +# the open/write/close heuristics in xfs_file_release() that prevent EOF block
> +# removal, so this should fragment badly. Typical problematic behaviour shows
> +# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
> +# shows extent counts in the low single digits (almost best case)
> +#
> +# Failure is determined by golden output mismatch from _within_tolerance().
> +
> +workfile=$SCRATCH_MNT/file
> +nfiles=8
> +wsize=4096
> +wcnt=1000
> +extent_size=16m
> +
> +write_direct_file()
> +{
> + idx=$1
> +
> + $XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
> + for ((cnt=0; cnt<$wcnt; cnt++)); do
> + $XFS_IO_PROG -f -d -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
> + done
> +}
> +
> +rm -f $workfile*
> +for ((n=0; n<$nfiles; n++)); do
> + write_direct_file $n > /dev/null 2>&1 &
> +done
> +wait
> +sync
> +
> +for ((n=0; n<$nfiles; n++)); do
> + count=$(_count_extents $workfile.$n)
> + # Acceptible extent count range is 1-10
> + _within_tolerance "file.$n extent count" $count 2 1 8 -v
> +done
> +
> +status=0
> +exit
> diff --git a/tests/xfs/1502.out b/tests/xfs/1502.out
> new file mode 100644
> index 000000000..82c8760a3
> --- /dev/null
> +++ b/tests/xfs/1502.out
> @@ -0,0 +1,9 @@
> +QA output created by 1502
> +file.0 extent count is in range
> +file.1 extent count is in range
> +file.2 extent count is in range
> +file.3 extent count is in range
> +file.4 extent count is in range
> +file.5 extent count is in range
> +file.6 extent count is in range
> +file.7 extent count is in range
> diff --git a/tests/xfs/1503 b/tests/xfs/1503
> new file mode 100755
> index 000000000..41f4035ad
> --- /dev/null
> +++ b/tests/xfs/1503
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test xfs/503
> +#
> +# Post-EOF preallocation defeat test with O_SYNC buffered I/O that repeatedly
> +# closes and reopens the files.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto prealloc rw
> +
> +. ./common/rc
> +. ./common/filter
> +
> +_supported_fs xfs
> +_require_scratch
> +
> +_scratch_mkfs 2>&1 >> $seqres.full
> +_scratch_mount
> +
> +# Write multiple files in parallel using synchronous buffered writes that
> +# repeatedly close and reopen the fails. Aim is to interleave allocations to
> +# fragment the files. Assuming we've fixed the synchronous write defeat, we can
> +# still trigger the same issue with a open/read/close on O_RDONLY files. We
> +# should not be triggering EOF preallocation removal on files we don't have
> +# permission to write, so until this is fixed it should fragment badly. Typical
> +# problematic behaviour shows per-file extent counts of 50-350 whilst fixed
> +# behaviour typically demonstrates post-eof speculative delalloc growth in
> +# extent size (~6 extents for 50MB file).
> +#
> +# Failure is determined by golden output mismatch from _within_tolerance().
> +
> +workfile=$SCRATCH_MNT/file
> +nfiles=32
> +wsize=4096
> +wcnt=1000
> +
> +write_file()
> +{
> + idx=$1
> +
> + $XFS_IO_PROG -f -s -c "pwrite -b 64k 0 50m" $workfile.$idx
> +}
> +
> +read_file()
> +{
> + idx=$1
> +
> + for ((cnt=0; cnt<$wcnt; cnt++)); do
> + $XFS_IO_PROG -f -r -c "pread 0 28" $workfile.$idx
> + done
> +}
> +
> +rm -f $workdir/file*
> +for ((n=0; n<$((nfiles)); n++)); do
> + write_file $n > /dev/null 2>&1 &
> + read_file $n > /dev/null 2>&1 &
> +done
> +wait
> +
> +for ((n=0; n<$nfiles; n++)); do
> + count=$(_count_extents $workfile.$n)
> + # Acceptible extent count range is 1-40
> + _within_tolerance "file.$n extent count" $count 6 5 10 -v
> +done
> +
> +status=0
> +exit
> diff --git a/tests/xfs/1503.out b/tests/xfs/1503.out
> new file mode 100644
> index 000000000..1780b16df
> --- /dev/null
> +++ b/tests/xfs/1503.out
> @@ -0,0 +1,33 @@
> +QA output created by 1503
> +file.0 extent count is in range
> +file.1 extent count is in range
> +file.2 extent count is in range
> +file.3 extent count is in range
> +file.4 extent count is in range
> +file.5 extent count is in range
> +file.6 extent count is in range
> +file.7 extent count is in range
> +file.8 extent count is in range
> +file.9 extent count is in range
> +file.10 extent count is in range
> +file.11 extent count is in range
> +file.12 extent count is in range
> +file.13 extent count is in range
> +file.14 extent count is in range
> +file.15 extent count is in range
> +file.16 extent count is in range
> +file.17 extent count is in range
> +file.18 extent count is in range
> +file.19 extent count is in range
> +file.20 extent count is in range
> +file.21 extent count is in range
> +file.22 extent count is in range
> +file.23 extent count is in range
> +file.24 extent count is in range
> +file.25 extent count is in range
> +file.26 extent count is in range
> +file.27 extent count is in range
> +file.28 extent count is in range
> +file.29 extent count is in range
> +file.30 extent count is in range
> +file.31 extent count is in range
> --
> 2.43.0
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] xfs: new EOF fragmentation tests
2024-06-23 13:03 ` Zorro Lang
@ 2024-06-24 8:49 ` Christoph Hellwig
0 siblings, 0 replies; 4+ messages in thread
From: Christoph Hellwig @ 2024-06-24 8:49 UTC (permalink / raw)
To: Zorro Lang
Cc: Christoph Hellwig, Zorro Lang, Darrick J. Wong, Dave Chinner,
linux-xfs, fstests
On Sun, Jun 23, 2024 at 09:03:01PM +0800, Zorro Lang wrote:
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> ^^^^
> Should I keep "2019" or change it to 2024, or no matter :)
No Red Hat employee has touched this since 2019. And my changes mostly
removed boilerplate code, so there isn't much of a point in adding
my copyright notice here.
> > +_scratch_mkfs 2>&1 >> $seqres.full
>
> Wrong order:) ">>$seqres.full 2>&1"
Ok.
> If a case has background process, better to do kill&wait in _cleanup,
> for unexpected Ctrl^C when fstests is running. e.g.
Ok.
>
> _cleanup{}
> {
> # try to kill all background processes
> wait
> cd /
> rm -r -f $tmp.*
> }
>
> I'm not sure if there's a good to way to do "kill all background
> processes" in this case, or if it's necessary. But a "wait" might
> be worth, to avoid the "Device Busy" error. Any thoughts?
No really sure, but I'll play around with a few variants.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-06-24 8:49 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-06-23 5:38 xfs post-EOF block freeing fixes Christoph Hellwig
2024-06-23 5:38 ` [PATCH] xfs: new EOF fragmentation tests Christoph Hellwig
2024-06-23 13:03 ` Zorro Lang
2024-06-24 8:49 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).