public inbox for linux-btrfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0
@ 2022-11-09 11:43 fdmanana
  2022-11-09 11:43 ` [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
                   ` (3 more replies)
  0 siblings, 4 replies; 13+ messages in thread
From: fdmanana @ 2022-11-09 11:43 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

The output of some commands changed with btrfs-progs 5.19 and 6.0, as well
as the --leafsize/-l command line option was removed from mkfs. These make
some tests fail. This patchset changes the tests so that they work with the
new btrfs-progs versions as well as older versions.

Filipe Manana (3):
  btrfs/003: fix failure on new btrfs-progs versions
  btrfs/053: fix test failure when running with btrfs-progs v6.0+
  btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+

 tests/btrfs/003 | 5 +++--
 tests/btrfs/021 | 4 +++-
 tests/btrfs/053 | 4 ++--
 tests/btrfs/256 | 4 +++-
 4 files changed, 11 insertions(+), 6 deletions(-)

-- 
2.35.1


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions
  2022-11-09 11:43 [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
@ 2022-11-09 11:43 ` fdmanana
  2022-11-09 15:37   ` Zorro Lang
  2022-11-09 11:43 ` [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+ fdmanana
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 13+ messages in thread
From: fdmanana @ 2022-11-09 11:43 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

Starting with btrfs-progs version 5.19, the output of 'filesystem show'
command changed when we have a missing device. The old output was like the
following:

    Label: none  uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
            Total devices 2 FS bytes used 128.00KiB
            devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
            *** Some devices missing

While the new output (btrfs-progs 5.19+) is like the following:

    Label: none  uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
            Total devices 2 FS bytes used 128.00KiB
            devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
            devid    2 size 0 used 0 path /dev/loop1 MISSING

More specifically it happened in the following btrfs-progs commit:

    957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")

This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
filter in btrfs/003 so that it works with both output formats.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/003 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tests/btrfs/003 b/tests/btrfs/003
index cf605730..fae6d9d1 100755
--- a/tests/btrfs/003
+++ b/tests/btrfs/003
@@ -141,8 +141,9 @@ _test_replace()
 	_devmgt_remove ${removed_dev_htl} $ds
 	dev_removed=1
 
-	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \
-							"btrfs did not report device missing"
+	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \
+		grep -ie '\bmissing\b' >> $seqres.full || \
+		_fail "btrfs did not report device missing"
 
 	# add a new disk to btrfs
 	ds=${devs[@]:$(($n)):1}
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+
  2022-11-09 11:43 [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
  2022-11-09 11:43 ` [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
@ 2022-11-09 11:43 ` fdmanana
  2022-11-09 15:55   ` Zorro Lang
  2022-11-09 11:43 ` [PATCH 3/3] btrfs: fix failure of tests that use defrag on " fdmanana
  2022-11-09 16:44 ` [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
  3 siblings, 1 reply; 13+ messages in thread
From: fdmanana @ 2022-11-09 11:43 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

In btrfs-progs v6.0 the --leafsize (-l) command line option was removed,
so btrfs/053 always fails with v6.0+.

The change was introduced by the following btrfs-progs commit:

  f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")

Change the test to use --nodesize (-n) instead, since it exists in both
old and new btrfs-progs versions.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/053 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/btrfs/053 b/tests/btrfs/053
index fbd2e7d9..c0446257 100755
--- a/tests/btrfs/053
+++ b/tests/btrfs/053
@@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq
 rm -fr $send_files_dir
 mkdir $send_files_dir
 
-_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
+_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
 _scratch_mount
 
 echo "hello world" > $SCRATCH_MNT/foobar
@@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \
 _scratch_unmount
 _check_scratch_fs
 
-_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
+_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
 _scratch_mount
 
 _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNT
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
  2022-11-09 11:43 [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
  2022-11-09 11:43 ` [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
  2022-11-09 11:43 ` [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+ fdmanana
@ 2022-11-09 11:43 ` fdmanana
  2022-11-09 16:01   ` Zorro Lang
  2022-11-09 16:44 ` [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
  3 siblings, 1 reply; 13+ messages in thread
From: fdmanana @ 2022-11-09 11:43 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

Starting with btrfs-progs v6.0, the defrag command now prints to stdout
the full path of the files it processes. This makes test cases btrfs/021
and btrfs/256 fail because they don't expect any output from the defrag
command.

The change happened with the following commit in btrfs-progs:

  dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")

So update the tests to ignore the stdout of the defrag command.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/021 | 4 +++-
 tests/btrfs/256 | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/btrfs/021 b/tests/btrfs/021
index 5943da2f..1b55834a 100755
--- a/tests/btrfs/021
+++ b/tests/btrfs/021
@@ -22,8 +22,10 @@ run_test()
 
 	sleep 0.5
 
+	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
+	# stdout the path of the files it operates on. So ignore that.
 	find $SCRATCH_MNT -type f -print0 | xargs -0 \
-	$BTRFS_UTIL_PROG filesystem defrag -f
+		$BTRFS_UTIL_PROG filesystem defrag -f > /dev/null
 
 	sync
 	wait
diff --git a/tests/btrfs/256 b/tests/btrfs/256
index 1360c2c2..acbbc6fa 100755
--- a/tests/btrfs/256
+++ b/tests/btrfs/256
@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT"
 # Now defrag each file.
 for sz in ${file_sizes[@]}; do
 	echo "Defragging file with $sz bytes..." >> $seqres.full
-	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz"
+	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
+	# stdout the path of the files it operates on. So ignore that.
+	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null
 done
 
 # Verify the checksums after the defrag operations.
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions
  2022-11-09 11:43 ` [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
@ 2022-11-09 15:37   ` Zorro Lang
  0 siblings, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2022-11-09 15:37 UTC (permalink / raw)
  To: fdmanana; +Cc: fstests, linux-btrfs, Filipe Manana

On Wed, Nov 09, 2022 at 11:43:34AM +0000, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Starting with btrfs-progs version 5.19, the output of 'filesystem show'
> command changed when we have a missing device. The old output was like the
> following:
> 
>     Label: none  uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
>             Total devices 2 FS bytes used 128.00KiB
>             devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
>             *** Some devices missing
> 
> While the new output (btrfs-progs 5.19+) is like the following:
> 
>     Label: none  uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
>             Total devices 2 FS bytes used 128.00KiB
>             devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
>             devid    2 size 0 used 0 path /dev/loop1 MISSING
> 
> More specifically it happened in the following btrfs-progs commit:
> 
>     957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
> 
> This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
> filter in btrfs/003 so that it works with both output formats.
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
> ---

OK, make sense,

Reviewed-by: Zorro Lang <zlang@redhat.com>

>  tests/btrfs/003 | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/btrfs/003 b/tests/btrfs/003
> index cf605730..fae6d9d1 100755
> --- a/tests/btrfs/003
> +++ b/tests/btrfs/003
> @@ -141,8 +141,9 @@ _test_replace()
>  	_devmgt_remove ${removed_dev_htl} $ds
>  	dev_removed=1
>  
> -	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \
> -							"btrfs did not report device missing"
> +	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \
> +		grep -ie '\bmissing\b' >> $seqres.full || \
> +		_fail "btrfs did not report device missing"
>  
>  	# add a new disk to btrfs
>  	ds=${devs[@]:$(($n)):1}
> -- 
> 2.35.1
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+
  2022-11-09 11:43 ` [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+ fdmanana
@ 2022-11-09 15:55   ` Zorro Lang
  2022-11-09 16:43     ` Filipe Manana
  0 siblings, 1 reply; 13+ messages in thread
From: Zorro Lang @ 2022-11-09 15:55 UTC (permalink / raw)
  To: fdmanana; +Cc: fstests, linux-btrfs, Filipe Manana

On Wed, Nov 09, 2022 at 11:43:35AM +0000, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> In btrfs-progs v6.0 the --leafsize (-l) command line option was removed,
> so btrfs/053 always fails with v6.0+.
> 
> The change was introduced by the following btrfs-progs commit:
> 
>   f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")
> 
> Change the test to use --nodesize (-n) instead, since it exists in both
> old and new btrfs-progs versions.
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
> ---
>  tests/btrfs/053 | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/btrfs/053 b/tests/btrfs/053
> index fbd2e7d9..c0446257 100755
> --- a/tests/btrfs/053
> +++ b/tests/btrfs/053
> @@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq
>  rm -fr $send_files_dir
>  mkdir $send_files_dir
>  
> -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
> +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1

We you said this case starts to fail on btrfs-progs v6.0, I'm wondering how it
fail (I don't doubt it fails), at least not fails at here right?

Actually I recommend mkfs output to .full file at here, especially when you use
specified mkfs options to _scratch_mkfs helper. That really might fail, and
the case might keep running (with old fs on SCRATCH_DEV), and we hard to notice
that if no message output and no return value checking.

The _scratch_mkfs doesn't _fail if it fails, it just return nonzero and output
error message. So generally I recommend writting likes this (or other proper way
which can detect mkfs failure):

  _scratch_mkfs "--nodesize $leaf_size" >>$seqres.full 2>&1 || _fail "mkfs failed"

especially if there's specified mkfs options.

Thanks,
Zorro

>  _scratch_mount
>  
>  echo "hello world" > $SCRATCH_MNT/foobar
> @@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \
>  _scratch_unmount
>  _check_scratch_fs
>  
> -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
> +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
>  _scratch_mount
>  
>  _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNT
> -- 
> 2.35.1
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
  2022-11-09 11:43 ` [PATCH 3/3] btrfs: fix failure of tests that use defrag on " fdmanana
@ 2022-11-09 16:01   ` Zorro Lang
  0 siblings, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2022-11-09 16:01 UTC (permalink / raw)
  To: fdmanana; +Cc: fstests, linux-btrfs, Filipe Manana

On Wed, Nov 09, 2022 at 11:43:36AM +0000, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Starting with btrfs-progs v6.0, the defrag command now prints to stdout
> the full path of the files it processes. This makes test cases btrfs/021
> and btrfs/256 fail because they don't expect any output from the defrag
> command.
> 
> The change happened with the following commit in btrfs-progs:
> 
>   dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
> 
> So update the tests to ignore the stdout of the defrag command.
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
> ---

Good to me,
Reviewed-by: Zorro Lang <zlang@redhat.com>

>  tests/btrfs/021 | 4 +++-
>  tests/btrfs/256 | 4 +++-
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/btrfs/021 b/tests/btrfs/021
> index 5943da2f..1b55834a 100755
> --- a/tests/btrfs/021
> +++ b/tests/btrfs/021
> @@ -22,8 +22,10 @@ run_test()
>  
>  	sleep 0.5
>  
> +	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
> +	# stdout the path of the files it operates on. So ignore that.
>  	find $SCRATCH_MNT -type f -print0 | xargs -0 \
> -	$BTRFS_UTIL_PROG filesystem defrag -f
> +		$BTRFS_UTIL_PROG filesystem defrag -f > /dev/null
>  
>  	sync
>  	wait
> diff --git a/tests/btrfs/256 b/tests/btrfs/256
> index 1360c2c2..acbbc6fa 100755
> --- a/tests/btrfs/256
> +++ b/tests/btrfs/256
> @@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT"
>  # Now defrag each file.
>  for sz in ${file_sizes[@]}; do
>  	echo "Defragging file with $sz bytes..." >> $seqres.full
> -	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz"
> +	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
> +	# stdout the path of the files it operates on. So ignore that.
> +	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null
>  done
>  
>  # Verify the checksums after the defrag operations.
> -- 
> 2.35.1
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+
  2022-11-09 15:55   ` Zorro Lang
@ 2022-11-09 16:43     ` Filipe Manana
  0 siblings, 0 replies; 13+ messages in thread
From: Filipe Manana @ 2022-11-09 16:43 UTC (permalink / raw)
  To: Zorro Lang; +Cc: fstests, linux-btrfs

On Wed, Nov 9, 2022 at 3:55 PM Zorro Lang <zlang@redhat.com> wrote:
>
> On Wed, Nov 09, 2022 at 11:43:35AM +0000, fdmanana@kernel.org wrote:
> > From: Filipe Manana <fdmanana@suse.com>
> >
> > In btrfs-progs v6.0 the --leafsize (-l) command line option was removed,
> > so btrfs/053 always fails with v6.0+.
> >
> > The change was introduced by the following btrfs-progs commit:
> >
> >   f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")
> >
> > Change the test to use --nodesize (-n) instead, since it exists in both
> > old and new btrfs-progs versions.
> >
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > ---
> >  tests/btrfs/053 | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/tests/btrfs/053 b/tests/btrfs/053
> > index fbd2e7d9..c0446257 100755
> > --- a/tests/btrfs/053
> > +++ b/tests/btrfs/053
> > @@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq
> >  rm -fr $send_files_dir
> >  mkdir $send_files_dir
> >
> > -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
> > +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
>
> We you said this case starts to fail on btrfs-progs v6.0, I'm wondering how it
> fail (I don't doubt it fails), at least not fails at here right?

It fails during mount in case no previous mkfs on the scratch device happened.
First time after booting a vm and without running any other test case.

>
> Actually I recommend mkfs output to .full file at here, especially when you use
> specified mkfs options to _scratch_mkfs helper. That really might fail, and
> the case might keep running (with old fs on SCRATCH_DEV), and we hard to notice
> that if no message output and no return value checking.
>
> The _scratch_mkfs doesn't _fail if it fails, it just return nonzero and output
> error message. So generally I recommend writting likes this (or other proper way
> which can detect mkfs failure):
>
>   _scratch_mkfs "--nodesize $leaf_size" >>$seqres.full 2>&1 || _fail "mkfs failed"

Yes, I remember some years ago a thread discussing silent mkfs failures being
ignored and getting unexpected results or tests passing when they shouldn't.

I'll add that, thanks.

>
> especially if there's specified mkfs options.
>
> Thanks,
> Zorro
>
> >  _scratch_mount
> >
> >  echo "hello world" > $SCRATCH_MNT/foobar
> > @@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \
> >  _scratch_unmount
> >  _check_scratch_fs
> >
> > -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1
> > +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
> >  _scratch_mount
> >
> >  _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNT
> > --
> > 2.35.1
> >
>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0
  2022-11-09 11:43 [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
                   ` (2 preceding siblings ...)
  2022-11-09 11:43 ` [PATCH 3/3] btrfs: fix failure of tests that use defrag on " fdmanana
@ 2022-11-09 16:44 ` fdmanana
  2022-11-09 16:44   ` [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
  2022-11-09 16:44   ` [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+ fdmanana
  3 siblings, 2 replies; 13+ messages in thread
From: fdmanana @ 2022-11-09 16:44 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

The output of some commands changed with btrfs-progs 5.19 and 6.0, as well
as the --leafsize/-l command line option was removed from mkfs. These make
some tests fail. This patchset changes the tests so that they work with the
new btrfs-progs versions as well as older versions.

v2: Updated patch 02/03, redirect mkfs output to log file and explicitly
    fail if mkfs failed.

Filipe Manana (3):
  btrfs/003: fix failure on new btrfs-progs versions
  btrfs/053: fix test failure when running with btrfs-progs v6.0+
  btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+

 tests/btrfs/003 | 5 +++--
 tests/btrfs/021 | 4 +++-
 tests/btrfs/053 | 4 ++--
 tests/btrfs/256 | 4 +++-
 4 files changed, 11 insertions(+), 6 deletions(-)

-- 
2.35.1


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions
  2022-11-09 16:44 ` [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
@ 2022-11-09 16:44   ` fdmanana
  2022-11-14 11:19     ` Qu Wenruo
  2022-11-09 16:44   ` [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+ fdmanana
  1 sibling, 1 reply; 13+ messages in thread
From: fdmanana @ 2022-11-09 16:44 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

Starting with btrfs-progs version 5.19, the output of 'filesystem show'
command changed when we have a missing device. The old output was like the
following:

    Label: none  uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
            Total devices 2 FS bytes used 128.00KiB
            devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
            *** Some devices missing

While the new output (btrfs-progs 5.19+) is like the following:

    Label: none  uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
            Total devices 2 FS bytes used 128.00KiB
            devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
            devid    2 size 0 used 0 path /dev/loop1 MISSING

More specifically it happened in the following btrfs-progs commit:

    957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")

This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
filter in btrfs/003 so that it works with both output formats.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/003 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tests/btrfs/003 b/tests/btrfs/003
index cf605730..fae6d9d1 100755
--- a/tests/btrfs/003
+++ b/tests/btrfs/003
@@ -141,8 +141,9 @@ _test_replace()
 	_devmgt_remove ${removed_dev_htl} $ds
 	dev_removed=1
 
-	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \
-							"btrfs did not report device missing"
+	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \
+		grep -ie '\bmissing\b' >> $seqres.full || \
+		_fail "btrfs did not report device missing"
 
 	# add a new disk to btrfs
 	ds=${devs[@]:$(($n)):1}
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
  2022-11-09 16:44 ` [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
  2022-11-09 16:44   ` [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
@ 2022-11-09 16:44   ` fdmanana
  2022-11-14 11:09     ` Qu Wenruo
  1 sibling, 1 reply; 13+ messages in thread
From: fdmanana @ 2022-11-09 16:44 UTC (permalink / raw)
  To: fstests; +Cc: linux-btrfs, Filipe Manana

From: Filipe Manana <fdmanana@suse.com>

Starting with btrfs-progs v6.0, the defrag command now prints to stdout
the full path of the files it processes. This makes test cases btrfs/021
and btrfs/256 fail because they don't expect any output from the defrag
command.

The change happened with the following commit in btrfs-progs:

  dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")

So update the tests to ignore the stdout of the defrag command.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/021 | 4 +++-
 tests/btrfs/256 | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/btrfs/021 b/tests/btrfs/021
index 5943da2f..1b55834a 100755
--- a/tests/btrfs/021
+++ b/tests/btrfs/021
@@ -22,8 +22,10 @@ run_test()
 
 	sleep 0.5
 
+	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
+	# stdout the path of the files it operates on. So ignore that.
 	find $SCRATCH_MNT -type f -print0 | xargs -0 \
-	$BTRFS_UTIL_PROG filesystem defrag -f
+		$BTRFS_UTIL_PROG filesystem defrag -f > /dev/null
 
 	sync
 	wait
diff --git a/tests/btrfs/256 b/tests/btrfs/256
index 1360c2c2..acbbc6fa 100755
--- a/tests/btrfs/256
+++ b/tests/btrfs/256
@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT"
 # Now defrag each file.
 for sz in ${file_sizes[@]}; do
 	echo "Defragging file with $sz bytes..." >> $seqres.full
-	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz"
+	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
+	# stdout the path of the files it operates on. So ignore that.
+	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null
 done
 
 # Verify the checksums after the defrag operations.
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
  2022-11-09 16:44   ` [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+ fdmanana
@ 2022-11-14 11:09     ` Qu Wenruo
  0 siblings, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2022-11-14 11:09 UTC (permalink / raw)
  To: fdmanana, fstests; +Cc: linux-btrfs, Filipe Manana



On 2022/11/10 00:44, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Starting with btrfs-progs v6.0, the defrag command now prints to stdout
> the full path of the files it processes. This makes test cases btrfs/021
> and btrfs/256 fail because they don't expect any output from the defrag
> command.
> 
> The change happened with the following commit in btrfs-progs:
> 
>    dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
> 
> So update the tests to ignore the stdout of the defrag command.
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>

Reviewed-by: Qu Wenruo <wqu@suse.com>

Thanks,
Qu
> ---
>   tests/btrfs/021 | 4 +++-
>   tests/btrfs/256 | 4 +++-
>   2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/btrfs/021 b/tests/btrfs/021
> index 5943da2f..1b55834a 100755
> --- a/tests/btrfs/021
> +++ b/tests/btrfs/021
> @@ -22,8 +22,10 @@ run_test()
>   
>   	sleep 0.5
>   
> +	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
> +	# stdout the path of the files it operates on. So ignore that.
>   	find $SCRATCH_MNT -type f -print0 | xargs -0 \
> -	$BTRFS_UTIL_PROG filesystem defrag -f
> +		$BTRFS_UTIL_PROG filesystem defrag -f > /dev/null
>   
>   	sync
>   	wait
> diff --git a/tests/btrfs/256 b/tests/btrfs/256
> index 1360c2c2..acbbc6fa 100755
> --- a/tests/btrfs/256
> +++ b/tests/btrfs/256
> @@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT"
>   # Now defrag each file.
>   for sz in ${file_sizes[@]}; do
>   	echo "Defragging file with $sz bytes..." >> $seqres.full
> -	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz"
> +	# In new versions of btrfs-progs (6.0+), the defrag command outputs to
> +	# stdout the path of the files it operates on. So ignore that.
> +	$BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null
>   done
>   
>   # Verify the checksums after the defrag operations.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions
  2022-11-09 16:44   ` [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
@ 2022-11-14 11:19     ` Qu Wenruo
  0 siblings, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2022-11-14 11:19 UTC (permalink / raw)
  To: fdmanana, fstests; +Cc: linux-btrfs, Filipe Manana



On 2022/11/10 00:44, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Starting with btrfs-progs version 5.19, the output of 'filesystem show'
> command changed when we have a missing device. The old output was like the
> following:
> 
>      Label: none  uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
>              Total devices 2 FS bytes used 128.00KiB
>              devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
>              *** Some devices missing
> 
> While the new output (btrfs-progs 5.19+) is like the following:
> 
>      Label: none  uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
>              Total devices 2 FS bytes used 128.00KiB
>              devid    1 size 5.00GiB used 1.26GiB path /dev/loop0
>              devid    2 size 0 used 0 path /dev/loop1 MISSING
> 
> More specifically it happened in the following btrfs-progs commit:
> 
>      957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
> 
> This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
> filter in btrfs/003 so that it works with both output formats.
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>

Reviewed-by: Qu Wenruo <wqu@suse.com>

After more digging, it shows that the test case itself will skip 
non-deletable disks, thus it doesn't cover my LVM based test environment 
at all.

Thanks,
Qu

> ---
>   tests/btrfs/003 | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/btrfs/003 b/tests/btrfs/003
> index cf605730..fae6d9d1 100755
> --- a/tests/btrfs/003
> +++ b/tests/btrfs/003
> @@ -141,8 +141,9 @@ _test_replace()
>   	_devmgt_remove ${removed_dev_htl} $ds
>   	dev_removed=1
>   
> -	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \
> -							"btrfs did not report device missing"
> +	$BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \
> +		grep -ie '\bmissing\b' >> $seqres.full || \
> +		_fail "btrfs did not report device missing"
>   
>   	# add a new disk to btrfs
>   	ds=${devs[@]:$(($n)):1}

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-11-14 11:24 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-11-09 11:43 [PATCH 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
2022-11-09 11:43 ` [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
2022-11-09 15:37   ` Zorro Lang
2022-11-09 11:43 ` [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+ fdmanana
2022-11-09 15:55   ` Zorro Lang
2022-11-09 16:43     ` Filipe Manana
2022-11-09 11:43 ` [PATCH 3/3] btrfs: fix failure of tests that use defrag on " fdmanana
2022-11-09 16:01   ` Zorro Lang
2022-11-09 16:44 ` [PATCH v2 0/3] btrfs: fix some test failures with btrfs-progs 5.19 and 6.0 fdmanana
2022-11-09 16:44   ` [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions fdmanana
2022-11-14 11:19     ` Qu Wenruo
2022-11-09 16:44   ` [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+ fdmanana
2022-11-14 11:09     ` Qu Wenruo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox