[PATCH] generic/388: randomly recover via read-only mounts

[PATCH] generic/388: randomly recover via read-only mounts

[Brian Foster]

XFS has an issue where superblock counters may not be properly
synced when recovery occurs via a read-only mount. This causes the
filesystem to become inconsistent after unmount. To cover this test
case, update generic/388 to switch between read-only and read-write
mounts to perform log recovery.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---

I didn't think it was worth duplicating generic/388 to a whole new test
just to invoke log recovery from a read-only mount. generic/388 is a
rather general log recovery test and this preserves historical behavior
of the test.

A prospective fix for the issue this reproduces on XFS is posted here:

https://lore.kernel.org/linux-xfs/20201217145334.2512475-1-bfoster@redhat.com/

Brian

 tests/generic/388 | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tests/generic/388 b/tests/generic/388
index 451a6be2..cdd547f4 100755
--- a/tests/generic/388
+++ b/tests/generic/388
@@ -66,8 +66,14 @@ for i in $(seq 1 $((50 * TIME_FACTOR)) ); do
 		ps -e | grep fsstress > /dev/null 2>&1
 	done
 
-	# quit if mount fails so we don't shutdown the host fs
-	_scratch_cycle_mount || _fail "cycle mount failed"
+	# Toggle between rw and ro mounts for recovery. Quit if any mount
+	# attempt fails so we don't shutdown the host fs.
+	if [ $((RANDOM % 2)) -eq 0 ]; then
+		_scratch_cycle_mount || _fail "cycle mount failed"
+	else
+		_scratch_cycle_mount "ro" || _fail "cycle ro mount failed"
+		_scratch_cycle_mount || _fail "cycle mount failed"
+	fi
 done
 
 # success, all done
-- 
2.26.2

Re: [PATCH] generic/388: randomly recover via read-only mounts

[Darrick J. Wong]

On Thu, Dec 17, 2020 at 09:59:41AM -0500, Brian Foster wrote:
> XFS has an issue where superblock counters may not be properly
> synced when recovery occurs via a read-only mount. This causes the
> filesystem to become inconsistent after unmount. To cover this test
> case, update generic/388 to switch between read-only and read-write
> mounts to perform log recovery.
> 
> Signed-off-by: Brian Foster <bfoster@redhat.com>
> ---
> 
> I didn't think it was worth duplicating generic/388 to a whole new test
> just to invoke log recovery from a read-only mount. generic/388 is a
> rather general log recovery test and this preserves historical behavior
> of the test.
> 
> A prospective fix for the issue this reproduces on XFS is posted here:
> 
> https://lore.kernel.org/linux-xfs/20201217145334.2512475-1-bfoster@redhat.com/
> 
> Brian
> 
>  tests/generic/388 | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/generic/388 b/tests/generic/388
> index 451a6be2..cdd547f4 100755
> --- a/tests/generic/388
> +++ b/tests/generic/388
> @@ -66,8 +66,14 @@ for i in $(seq 1 $((50 * TIME_FACTOR)) ); do
>  		ps -e | grep fsstress > /dev/null 2>&1
>  	done
>  
> -	# quit if mount fails so we don't shutdown the host fs
> -	_scratch_cycle_mount || _fail "cycle mount failed"
> +	# Toggle between rw and ro mounts for recovery. Quit if any mount
> +	# attempt fails so we don't shutdown the host fs.
> +	if [ $((RANDOM % 2)) -eq 0 ]; then
> +		_scratch_cycle_mount || _fail "cycle mount failed"
> +	else
> +		_scratch_cycle_mount "ro" || _fail "cycle ro mount failed"
> +		_scratch_cycle_mount || _fail "cycle mount failed"

I would change that third failure message to something distinct, like:

_fail "cycle remount failed"

To give us extra clues as to which branch encountered failure.
This looks like a fun way to find new bugs. :)

--D

> +	fi
>  done
>  
>  # success, all done
> -- 
> 2.26.2
> 

Re: [PATCH] generic/388: randomly recover via read-only mounts

[Brian Foster]

On Mon, Jan 04, 2021 at 10:34:24AM -0800, Darrick J. Wong wrote:
> On Thu, Dec 17, 2020 at 09:59:41AM -0500, Brian Foster wrote:
> > XFS has an issue where superblock counters may not be properly
> > synced when recovery occurs via a read-only mount. This causes the
> > filesystem to become inconsistent after unmount. To cover this test
> > case, update generic/388 to switch between read-only and read-write
> > mounts to perform log recovery.
> > 
> > Signed-off-by: Brian Foster <bfoster@redhat.com>
> > ---
> > 
> > I didn't think it was worth duplicating generic/388 to a whole new test
> > just to invoke log recovery from a read-only mount. generic/388 is a
> > rather general log recovery test and this preserves historical behavior
> > of the test.
> > 
> > A prospective fix for the issue this reproduces on XFS is posted here:
> > 
> > https://lore.kernel.org/linux-xfs/20201217145334.2512475-1-bfoster@redhat.com/
> > 
> > Brian
> > 
> >  tests/generic/388 | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tests/generic/388 b/tests/generic/388
> > index 451a6be2..cdd547f4 100755
> > --- a/tests/generic/388
> > +++ b/tests/generic/388
> > @@ -66,8 +66,14 @@ for i in $(seq 1 $((50 * TIME_FACTOR)) ); do
> >  		ps -e | grep fsstress > /dev/null 2>&1
> >  	done
> >  
> > -	# quit if mount fails so we don't shutdown the host fs
> > -	_scratch_cycle_mount || _fail "cycle mount failed"
> > +	# Toggle between rw and ro mounts for recovery. Quit if any mount
> > +	# attempt fails so we don't shutdown the host fs.
> > +	if [ $((RANDOM % 2)) -eq 0 ]; then
> > +		_scratch_cycle_mount || _fail "cycle mount failed"
> > +	else
> > +		_scratch_cycle_mount "ro" || _fail "cycle ro mount failed"
> > +		_scratch_cycle_mount || _fail "cycle mount failed"
> 
> I would change that third failure message to something distinct, like:
> 
> _fail "cycle remount failed"
> 
> To give us extra clues as to which branch encountered failure.
> This looks like a fun way to find new bugs. :)
> 

Sure.. I tweaked it to "cycle rw mount failed" so it's distinct, yet
more consistent with the preceding ro cycle.

Brian

> --D
> 
> > +	fi
> >  done
> >  
> >  # success, all done
> > -- 
> > 2.26.2
> > 
>