[PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Christoph Hellwig]

These fail for various non-default configs like DAX, alwayscow and
small block sizes.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 tests/xfs/629 | 2 +-
 tests/xfs/630 | 2 +-
 tests/xfs/631 | 2 +-
 tests/xfs/632 | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/xfs/629 b/tests/xfs/629
index 58beedc03a8b..e2f5af085b5f 100755
--- a/tests/xfs/629
+++ b/tests/xfs/629
@@ -8,7 +8,7 @@
 #
 
 . ./common/preamble
-_begin_fstest auto quick prealloc rw
+_begin_fstest prealloc rw
 
 . ./common/filter
 
diff --git a/tests/xfs/630 b/tests/xfs/630
index 939d8a4ac37f..df7ca60111d6 100755
--- a/tests/xfs/630
+++ b/tests/xfs/630
@@ -8,7 +8,7 @@
 #
 
 . ./common/preamble
-_begin_fstest auto quick prealloc rw
+_begin_fstest prealloc rw
 
 . ./common/filter
 
diff --git a/tests/xfs/631 b/tests/xfs/631
index 55a74297918a..1e50bc033f7c 100755
--- a/tests/xfs/631
+++ b/tests/xfs/631
@@ -8,7 +8,7 @@
 #
 
 . ./common/preamble
-_begin_fstest auto quick prealloc rw
+_begin_fstest prealloc rw
 
 . ./common/filter
 
diff --git a/tests/xfs/632 b/tests/xfs/632
index 61041d45a706..3b1c61fdc129 100755
--- a/tests/xfs/632
+++ b/tests/xfs/632
@@ -9,7 +9,7 @@
 #
 
 . ./common/preamble
-_begin_fstest auto prealloc rw
+_begin_fstest prealloc rw
 
 . ./common/filter
 
-- 
2.45.2

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Darrick J. Wong]

On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> These fail for various non-default configs like DAX, alwayscow and
> small block sizes.

Shouldn't we selectively _notrun these tests for configurations where
speculative/delayed allocations don't work?

I had started on a helper to try to detect the situations where the
tests cannot ever pass, but never quite finished it:

diff --git a/common/xfs b/common/xfs
index 557017c716e32c..5cb2c102e2c04f 100644
--- a/common/xfs
+++ b/common/xfs
@@ -2238,3 +2238,34 @@ _scratch_xfs_scrubbed() {
 
 	$XFS_SCRUBBED_PROG "${scrubbed_args[@]}" "$@" $SCRATCH_MNT
 }
+
+# Will this filesystem create speculative post-EOF preallocations for a file?
+_require_speculative_prealloc()
+{
+	local file="$1"
+	local tries
+	local overage
+
+	# Now that we have background garbage collection processes that can be
+	# triggered by low space/quota conditions, it's possible that we won't
+	# succeed in creating a speculative preallocation on the first try.
+	for ((tries = 0; tries < 5; tries++)); do
+		rm -f $file
+
+		# a few file extending open-write-close cycles should be enough
+		# to trigger the fs to retain preallocation. write 256k in 32k
+		# intervals to be sure
+		for i in $(seq 0 32768 262144); do
+			$XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full
+
+			# Do we have more blocks allocated than what we've
+			# written so far?
+			overage="$(stat -c '%b * %B - %s' $file | bc)"
+			test "$overage" -gt 0 && return 0
+		done
+	done
+
+	_notrun "Warning: No speculative preallocation for $file after " \
+			"$tries iterations." \
+			"Check use of the allocsize= mount option."
+}

--D

> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  tests/xfs/629 | 2 +-
>  tests/xfs/630 | 2 +-
>  tests/xfs/631 | 2 +-
>  tests/xfs/632 | 2 +-
>  4 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/xfs/629 b/tests/xfs/629
> index 58beedc03a8b..e2f5af085b5f 100755
> --- a/tests/xfs/629
> +++ b/tests/xfs/629
> @@ -8,7 +8,7 @@
>  #
>  
>  . ./common/preamble
> -_begin_fstest auto quick prealloc rw
> +_begin_fstest prealloc rw
>  
>  . ./common/filter
>  
> diff --git a/tests/xfs/630 b/tests/xfs/630
> index 939d8a4ac37f..df7ca60111d6 100755
> --- a/tests/xfs/630
> +++ b/tests/xfs/630
> @@ -8,7 +8,7 @@
>  #
>  
>  . ./common/preamble
> -_begin_fstest auto quick prealloc rw
> +_begin_fstest prealloc rw
>  
>  . ./common/filter
>  
> diff --git a/tests/xfs/631 b/tests/xfs/631
> index 55a74297918a..1e50bc033f7c 100755
> --- a/tests/xfs/631
> +++ b/tests/xfs/631
> @@ -8,7 +8,7 @@
>  #
>  
>  . ./common/preamble
> -_begin_fstest auto quick prealloc rw
> +_begin_fstest prealloc rw
>  
>  . ./common/filter
>  
> diff --git a/tests/xfs/632 b/tests/xfs/632
> index 61041d45a706..3b1c61fdc129 100755
> --- a/tests/xfs/632
> +++ b/tests/xfs/632
> @@ -9,7 +9,7 @@
>  #
>  
>  . ./common/preamble
> -_begin_fstest auto prealloc rw
> +_begin_fstest prealloc rw
>  
>  . ./common/filter
>  
> -- 
> 2.45.2
> 
> 

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Dave Chinner]

On Wed, Oct 23, 2024 at 10:23:51AM -0700, Darrick J. Wong wrote:
> On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> > These fail for various non-default configs like DAX, alwayscow and
> > small block sizes.
> 
> Shouldn't we selectively _notrun these tests for configurations where
> speculative/delayed allocations don't work?

Perhaps turning off speculative delalloc by adding the mount
option "-o allocsize=<1 fsb>" to these tests would result in them
always have the same behaviour?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Christoph Hellwig]

On Thu, Oct 24, 2024 at 08:40:42AM +1100, Dave Chinner wrote:
> On Wed, Oct 23, 2024 at 10:23:51AM -0700, Darrick J. Wong wrote:
> > On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> > > These fail for various non-default configs like DAX, alwayscow and
> > > small block sizes.
> > 
> > Shouldn't we selectively _notrun these tests for configurations where
> > speculative/delayed allocations don't work?
> 
> Perhaps turning off speculative delalloc by adding the mount
> option "-o allocsize=<1 fsb>" to these tests would result in them
> always have the same behaviour?

But it also kinda defeats what they are trying to test?  At least
that's my understanding, but your wrote them, though..

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Christoph Hellwig]

On Wed, Oct 23, 2024 at 10:23:51AM -0700, Darrick J. Wong wrote:
> On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> > These fail for various non-default configs like DAX, alwayscow and
> > small block sizes.
> 
> Shouldn't we selectively _notrun these tests for configurations where
> speculative/delayed allocations don't work?
> 
> I had started on a helper to try to detect the situations where the
> tests cannot ever pass, but never quite finished it:

If you get that to fully work it seems like a good idea, yes.

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Zorro Lang]

On Wed, Oct 23, 2024 at 10:23:51AM -0700, Darrick J. Wong wrote:
> On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> > These fail for various non-default configs like DAX, alwayscow and
> > small block sizes.
> 
> Shouldn't we selectively _notrun these tests for configurations where
> speculative/delayed allocations don't work?
> 
> I had started on a helper to try to detect the situations where the
> tests cannot ever pass, but never quite finished it:
> 
> diff --git a/common/xfs b/common/xfs
> index 557017c716e32c..5cb2c102e2c04f 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -2238,3 +2238,34 @@ _scratch_xfs_scrubbed() {
>  
>  	$XFS_SCRUBBED_PROG "${scrubbed_args[@]}" "$@" $SCRATCH_MNT
>  }
> +
> +# Will this filesystem create speculative post-EOF preallocations for a file?
> +_require_speculative_prealloc()
> +{
> +	local file="$1"
> +	local tries
> +	local overage
> +
> +	# Now that we have background garbage collection processes that can be
> +	# triggered by low space/quota conditions, it's possible that we won't
> +	# succeed in creating a speculative preallocation on the first try.
> +	for ((tries = 0; tries < 5; tries++)); do
> +		rm -f $file
> +
> +		# a few file extending open-write-close cycles should be enough
> +		# to trigger the fs to retain preallocation. write 256k in 32k
> +		# intervals to be sure
> +		for i in $(seq 0 32768 262144); do
> +			$XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full
> +
> +			# Do we have more blocks allocated than what we've
> +			# written so far?
> +			overage="$(stat -c '%b * %B - %s' $file | bc)"
> +			test "$overage" -gt 0 && return 0
> +		done
> +	done
> +
> +	_notrun "Warning: No speculative preallocation for $file after " \
> +			"$tries iterations." \
> +			"Check use of the allocsize= mount option."
> +}

Before we remove these cases from auto group, if above function can help these
test cases to be stable passed as expected. I'm glad to consider it at first :)

Thanks,
Zorro

> 
> --D
> 
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> >  tests/xfs/629 | 2 +-
> >  tests/xfs/630 | 2 +-
> >  tests/xfs/631 | 2 +-
> >  tests/xfs/632 | 2 +-
> >  4 files changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/tests/xfs/629 b/tests/xfs/629
> > index 58beedc03a8b..e2f5af085b5f 100755
> > --- a/tests/xfs/629
> > +++ b/tests/xfs/629
> > @@ -8,7 +8,7 @@
> >  #
> >  
> >  . ./common/preamble
> > -_begin_fstest auto quick prealloc rw
> > +_begin_fstest prealloc rw
> >  
> >  . ./common/filter
> >  
> > diff --git a/tests/xfs/630 b/tests/xfs/630
> > index 939d8a4ac37f..df7ca60111d6 100755
> > --- a/tests/xfs/630
> > +++ b/tests/xfs/630
> > @@ -8,7 +8,7 @@
> >  #
> >  
> >  . ./common/preamble
> > -_begin_fstest auto quick prealloc rw
> > +_begin_fstest prealloc rw
> >  
> >  . ./common/filter
> >  
> > diff --git a/tests/xfs/631 b/tests/xfs/631
> > index 55a74297918a..1e50bc033f7c 100755
> > --- a/tests/xfs/631
> > +++ b/tests/xfs/631
> > @@ -8,7 +8,7 @@
> >  #
> >  
> >  . ./common/preamble
> > -_begin_fstest auto quick prealloc rw
> > +_begin_fstest prealloc rw
> >  
> >  . ./common/filter
> >  
> > diff --git a/tests/xfs/632 b/tests/xfs/632
> > index 61041d45a706..3b1c61fdc129 100755
> > --- a/tests/xfs/632
> > +++ b/tests/xfs/632
> > @@ -9,7 +9,7 @@
> >  #
> >  
> >  . ./common/preamble
> > -_begin_fstest auto prealloc rw
> > +_begin_fstest prealloc rw
> >  
> >  . ./common/filter
> >  
> > -- 
> > 2.45.2
> > 
> > 
> 

Re: [PATCH] xfs: remove the post-EOF prealloc tests from the auto and quick groups

[Christoph Hellwig]

On Wed, Oct 23, 2024 at 10:23:51AM -0700, Darrick J. Wong wrote:
> On Wed, Oct 23, 2024 at 12:39:30PM +0200, Christoph Hellwig wrote:
> > These fail for various non-default configs like DAX, alwayscow and
> > small block sizes.
> 
> Shouldn't we selectively _notrun these tests for configurations where
> speculative/delayed allocations don't work?
> 
> I had started on a helper to try to detect the situations where the
> tests cannot ever pass, but never quite finished it:

Did you ever finish this off?