[PATCHSET v3] fstests: random fixes for v2024.12.01

[PATCHSET v3] fstests: random fixes for v2024.12.01

[Darrick J. Wong]

Hi all,

Here's the usual odd fixes for fstests.  Most of these are cleanups and
bug fixes that have been aging in my djwong-wtf branch forever.

v2: Now with more cleanups to the logwrites code and better loop control
    for 251, as discussed on the v1 patchset.
v3: Add more acks, kick out some of the logwrites stuff, add more
    bugfixes.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
Commits in this patchset:
 * xfs/032: try running on blocksize > pagesize filesystems
 * xfs/43[4-6]: implement impatient module reloading
---
 common/module |   11 +++++++++++
 tests/xfs/032 |   11 +++++++++++
 tests/xfs/434 |    2 +-
 tests/xfs/435 |    2 +-
 tests/xfs/436 |    2 +-
 5 files changed, 25 insertions(+), 3 deletions(-)

[PATCH 1/2] xfs/032: try running on blocksize > pagesize filesystems

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we're no longer limited to blocksize <= pagesize, let's make
sure that mkfs, fsstress, and copy work on such things.  This is also a
subtle way to get more people running at least one test with that
config.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/032 |   11 +++++++++++
 1 file changed, 11 insertions(+)


diff --git a/tests/xfs/032 b/tests/xfs/032
index 75edf0e9c7268d..52d66ea182d47e 100755
--- a/tests/xfs/032
+++ b/tests/xfs/032
@@ -25,6 +25,17 @@ IMGFILE=$TEST_DIR/${seq}_copy.img
 
 echo "Silence is golden."
 
+# Can we mount blocksize > pagesize filesystems?
+for ((blocksize = PAGESIZE; blocksize <= 65536; blocksize *= 2)); do
+	_scratch_mkfs -b size=$blocksize -d size=1g >> $seqres.full 2>&1 || \
+		continue
+
+	_try_scratch_mount || continue
+	mounted_blocksize="$(stat -f -c '%S' $SCRATCH_MNT)"
+	_scratch_unmount
+	test "$blocksize" -eq "$mounted_blocksize" && PAGESIZE=$blocksize
+done
+
 do_copy()
 {
 	local opts="$*"

Re: [PATCH 1/2] xfs/032: try running on blocksize > pagesize filesystems

[Zorro Lang]

On Tue, Dec 03, 2024 at 07:45:49PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Now that we're no longer limited to blocksize <= pagesize, let's make
> sure that mkfs, fsstress, and copy work on such things.  This is also a
> subtle way to get more people running at least one test with that
> config.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---

Hi Darrick, sorry for missing this patchset long time :-D

>  tests/xfs/032 |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> 
> diff --git a/tests/xfs/032 b/tests/xfs/032
> index 75edf0e9c7268d..52d66ea182d47e 100755
> --- a/tests/xfs/032
> +++ b/tests/xfs/032
> @@ -25,6 +25,17 @@ IMGFILE=$TEST_DIR/${seq}_copy.img
>  
>  echo "Silence is golden."
>  
> +# Can we mount blocksize > pagesize filesystems?
> +for ((blocksize = PAGESIZE; blocksize <= 65536; blocksize *= 2)); do
> +	_scratch_mkfs -b size=$blocksize -d size=1g >> $seqres.full 2>&1 || \
> +		continue
> +
> +	_try_scratch_mount || continue
> +	mounted_blocksize="$(stat -f -c '%S' $SCRATCH_MNT)"

_get_block_size $SCRATCH_MNT

> +	_scratch_unmount
> +	test "$blocksize" -eq "$mounted_blocksize" && PAGESIZE=$blocksize
> +done

I'm wondering if we can have a helper likes _has_lbs_support(), if it
returns 0, then set PAGESIZE to 65536 directly? (and we'd better to
change name of PAGESIZE, e.g. MAX_BLOCKSIZE)

Thanks,
Zorro

> +
>  do_copy()
>  {
>  	local opts="$*"
> 

Re: [PATCH 1/2] xfs/032: try running on blocksize > pagesize filesystems

[Darrick J. Wong]

On Sun, Dec 22, 2024 at 08:44:21PM +0800, Zorro Lang wrote:
> On Tue, Dec 03, 2024 at 07:45:49PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Now that we're no longer limited to blocksize <= pagesize, let's make
> > sure that mkfs, fsstress, and copy work on such things.  This is also a
> > subtle way to get more people running at least one test with that
> > config.
> > 
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > ---
> 
> Hi Darrick, sorry for missing this patchset long time :-D

No worries.

> >  tests/xfs/032 |   11 +++++++++++
> >  1 file changed, 11 insertions(+)
> > 
> > 
> > diff --git a/tests/xfs/032 b/tests/xfs/032
> > index 75edf0e9c7268d..52d66ea182d47e 100755
> > --- a/tests/xfs/032
> > +++ b/tests/xfs/032
> > @@ -25,6 +25,17 @@ IMGFILE=$TEST_DIR/${seq}_copy.img
> >  
> >  echo "Silence is golden."
> >  
> > +# Can we mount blocksize > pagesize filesystems?
> > +for ((blocksize = PAGESIZE; blocksize <= 65536; blocksize *= 2)); do
> > +	_scratch_mkfs -b size=$blocksize -d size=1g >> $seqres.full 2>&1 || \
> > +		continue
> > +
> > +	_try_scratch_mount || continue
> > +	mounted_blocksize="$(stat -f -c '%S' $SCRATCH_MNT)"
> 
> _get_block_size $SCRATCH_MNT

Fixed, thanks.

> > +	_scratch_unmount
> > +	test "$blocksize" -eq "$mounted_blocksize" && PAGESIZE=$blocksize
> > +done
> 
> I'm wondering if we can have a helper likes _has_lbs_support(), if it
> returns 0, then set PAGESIZE to 65536 directly? (and we'd better to
> change name of PAGESIZE, e.g. MAX_BLOCKSIZE)

I suppose we could, though how do we detect large block size support?
If it's just mkfs+mount then that's not a lot better than the loop that
exists now.

Another approach might be to change the loop to:

while [ $SECTORSIZE -le 65536 ]; do
	while [ $BLOCKSIZE -le 65536 ]; do
		...
	done
done

But break out of the loop if _scratch_mount fails and BLOCKSIZE >
PAGESIZE?  Then we don't need the detector loop.

--D

> Thanks,
> Zorro
> 
> > +
> >  do_copy()
> >  {
> >  	local opts="$*"
> > 
> 
> 

[PATCH 2/2] xfs/43[4-6]: implement impatient module reloading

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

These three tests try to reload the xfs module as a cheap way to detect
leaked inode and dquot objects when the slabs for those object are torn
down during rmmod.  Removal might not succeed, and we don't really care
for that case because we still want to exercise the log recovery code.

However, if (say) the root filesystem is xfs, then removal will never
succeed.  There's no way that waiting 50 seconds(!) per test is going
to change that.  Add a silly helper to do it fast or go home.

Reported-by: sandeen@sandeen.net
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/module |   11 +++++++++++
 tests/xfs/434 |    2 +-
 tests/xfs/435 |    2 +-
 tests/xfs/436 |    2 +-
 4 files changed, 14 insertions(+), 3 deletions(-)


diff --git a/common/module b/common/module
index a8d5f492d3f416..697d76ba718bbc 100644
--- a/common/module
+++ b/common/module
@@ -214,3 +214,14 @@ _patient_rmmod()
 
 	return $mod_ret
 }
+
+# Try to reload a filesystem driver.  Don't wait if we can't remove the module,
+# and don't let failures related to removing the module escape.  The caller
+# doesn't care if removal doesn't work.
+_optional_reload_fs_module()
+{
+	MODPROBE_PATIENT_RM_TIMEOUT_SECONDS=0 \
+		MODPROBE_REMOVE_PATIENT="" \
+		_test_loadable_fs_module "$@" 2>&1 | \
+		sed -e '/patient module removal/d'
+}
diff --git a/tests/xfs/434 b/tests/xfs/434
index c5122884324eb0..fe609b138d732b 100755
--- a/tests/xfs/434
+++ b/tests/xfs/434
@@ -74,7 +74,7 @@ _scratch_unmount 2> /dev/null
 rm -f ${RESULT_DIR}/require_scratch
 
 echo "See if we leak"
-_test_loadable_fs_module "xfs"
+_optional_reload_fs_module "xfs"
 
 # success, all done
 status=0
diff --git a/tests/xfs/435 b/tests/xfs/435
index 0bb5675e1dba23..22c02fbd1289bb 100755
--- a/tests/xfs/435
+++ b/tests/xfs/435
@@ -52,7 +52,7 @@ _scratch_unmount 2> /dev/null
 rm -f ${RESULT_DIR}/require_scratch
 
 echo "See if we leak"
-_test_loadable_fs_module "xfs"
+_optional_reload_fs_module "xfs"
 
 # success, all done
 status=0
diff --git a/tests/xfs/436 b/tests/xfs/436
index 1f7eb329e1394e..6a9d93d95f432f 100755
--- a/tests/xfs/436
+++ b/tests/xfs/436
@@ -69,7 +69,7 @@ _scratch_unmount 2> /dev/null
 rm -f ${RESULT_DIR}/require_scratch
 
 echo "See if we leak"
-_test_loadable_fs_module "xfs"
+_optional_reload_fs_module "xfs"
 
 # success, all done
 status=0

Re: [PATCH 2/2] xfs/43[4-6]: implement impatient module reloading

[Zorro Lang]

On Tue, Dec 03, 2024 at 07:46:05PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> These three tests try to reload the xfs module as a cheap way to detect
> leaked inode and dquot objects when the slabs for those object are torn
> down during rmmod.  Removal might not succeed, and we don't really care
> for that case because we still want to exercise the log recovery code.
> 
> However, if (say) the root filesystem is xfs, then removal will never
> succeed.  There's no way that waiting 50 seconds(!) per test is going
> to change that.  Add a silly helper to do it fast or go home.
> 
> Reported-by: sandeen@sandeen.net
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  common/module |   11 +++++++++++
>  tests/xfs/434 |    2 +-
>  tests/xfs/435 |    2 +-
>  tests/xfs/436 |    2 +-
>  4 files changed, 14 insertions(+), 3 deletions(-)
> 
> 
> diff --git a/common/module b/common/module
> index a8d5f492d3f416..697d76ba718bbc 100644
> --- a/common/module
> +++ b/common/module
> @@ -214,3 +214,14 @@ _patient_rmmod()
>  
>  	return $mod_ret
>  }
> +
> +# Try to reload a filesystem driver.  Don't wait if we can't remove the module,
> +# and don't let failures related to removing the module escape.  The caller
> +# doesn't care if removal doesn't work.
> +_optional_reload_fs_module()
> +{
> +	MODPROBE_PATIENT_RM_TIMEOUT_SECONDS=0 \
> +		MODPROBE_REMOVE_PATIENT="" \
> +		_test_loadable_fs_module "$@" 2>&1 | \
> +		sed -e '/patient module removal/d'
> +}

Thanks for fixing this error report, it looks good to me,

Reviewed-by: Zorro Lang <zlang@redhat.com>

> diff --git a/tests/xfs/434 b/tests/xfs/434
> index c5122884324eb0..fe609b138d732b 100755
> --- a/tests/xfs/434
> +++ b/tests/xfs/434
> @@ -74,7 +74,7 @@ _scratch_unmount 2> /dev/null
>  rm -f ${RESULT_DIR}/require_scratch
>  
>  echo "See if we leak"
> -_test_loadable_fs_module "xfs"
> +_optional_reload_fs_module "xfs"
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/435 b/tests/xfs/435
> index 0bb5675e1dba23..22c02fbd1289bb 100755
> --- a/tests/xfs/435
> +++ b/tests/xfs/435
> @@ -52,7 +52,7 @@ _scratch_unmount 2> /dev/null
>  rm -f ${RESULT_DIR}/require_scratch
>  
>  echo "See if we leak"
> -_test_loadable_fs_module "xfs"
> +_optional_reload_fs_module "xfs"
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/436 b/tests/xfs/436
> index 1f7eb329e1394e..6a9d93d95f432f 100755
> --- a/tests/xfs/436
> +++ b/tests/xfs/436
> @@ -69,7 +69,7 @@ _scratch_unmount 2> /dev/null
>  rm -f ${RESULT_DIR}/require_scratch
>  
>  echo "See if we leak"
> -_test_loadable_fs_module "xfs"
> +_optional_reload_fs_module "xfs"
>  
>  # success, all done
>  status=0
> 

Re: [PATCHSET v3] fstests: random fixes for v2024.12.01

[Darrick J. Wong]

Any review comments?

--D

On Tue, Dec 03, 2024 at 07:45:43PM -0800, Darrick J. Wong wrote:
> Hi all,
> 
> Here's the usual odd fixes for fstests.  Most of these are cleanups and
> bug fixes that have been aging in my djwong-wtf branch forever.
> 
> v2: Now with more cleanups to the logwrites code and better loop control
>     for 251, as discussed on the v1 patchset.
> v3: Add more acks, kick out some of the logwrites stuff, add more
>     bugfixes.
> 
> If you're going to start using this code, I strongly recommend pulling
> from my git trees, which are linked below.
> 
> With a bit of luck, this should all go splendidly.
> Comments and questions are, as always, welcome.
> 
> --D
> 
> kernel git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes
> 
> xfsprogs git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes
> 
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> ---
> Commits in this patchset:
>  * xfs/032: try running on blocksize > pagesize filesystems
>  * xfs/43[4-6]: implement impatient module reloading
> ---
>  common/module |   11 +++++++++++
>  tests/xfs/032 |   11 +++++++++++
>  tests/xfs/434 |    2 +-
>  tests/xfs/435 |    2 +-
>  tests/xfs/436 |    2 +-
>  5 files changed, 25 insertions(+), 3 deletions(-)
> 
>