[PATCH v2] fstests: btrfs: check conversion of zoned fileystems

[PATCH v2] fstests: btrfs: check conversion of zoned fileystems

[Johannes Thumshirn]

Recently we had a bug where a zoned filesystem could be converted to a
higher data redundancy profile than supported.

Add a test-case to check the conversion on zoned filesystems.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/310     | 75 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/310.out | 12 ++++++++
 2 files changed, 87 insertions(+)
 create mode 100755 tests/btrfs/310
 create mode 100644 tests/btrfs/310.out

diff --git a/tests/btrfs/310 b/tests/btrfs/310
new file mode 100755
index 00000000..6b0846f0
--- /dev/null
+++ b/tests/btrfs/310
@@ -0,0 +1,75 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Test that btrfs convert can ony be run to convert to supported profiles on a
+# zoned filesystem
+#
+. ./common/preamble
+_begin_fstest volume raid convert
+
+_fixed_by_kernel_commit XXXXXXXXXX \
+	"btrfs: zoned: don't skip block group profile checks on conv zones"
+
+. common/filter
+. common/filter.btrfs
+
+_supported_fs btrfs
+_require_scratch_dev_pool 4
+_require_zoned_device "$SCRATCH_DEV"
+
+
+_filter_convert()
+{
+	_filter_scratch | \
+	sed -e "s/relocate [0-9]\+ out of [0-9]\+ chunks/relocate X out of X chunks/g"
+}
+
+_filter_add()
+{
+	_filter_scratch | _filter_scratch_pool | \
+	sed -e "s/Resetting device zones SCRATCH_DEV ([0-9]\+/Resetting device zones SCRATCH_DEV (XXX/g"
+
+}
+
+devs=( $SCRATCH_DEV_POOL )
+
+# Create and mount single device FS
+_scratch_mkfs -msingle -dsingle 2>&1 > /dev/null
+_scratch_mount
+
+# Convert FS to metadata/system DUP
+$BTRFS_UTIL_PROG balance start -f -mconvert=dup -sconvert=dup $SCRATCH_MNT 2>&1 | _filter_convert
+
+# Convert FS to data DUP, must fail
+$BTRFS_UTIL_PROG balance start -dconvert=dup $SCRATCH_MNT 2>&1 | _filter_convert
+
+# Add device
+$BTRFS_UTIL_PROG device add ${devs[1]} $SCRATCH_MNT | _filter_add
+
+# Convert FS to data RAID1, must fail
+$BTRFS_UTIL_PROG balance start -dconvert=raid1 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
+
+# Convert FS to data RAID0, must fail
+$BTRFS_UTIL_PROG balance start -dconvert=raid0 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
+
+# Add device
+$BTRFS_UTIL_PROG device add ${devs[2]} $SCRATCH_MNT | _filter_add
+
+# Convert FS to data RAID5, must fail
+$BTRFS_UTIL_PROG balance start -f -dconvert=raid5 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
+
+# Add device
+$BTRFS_UTIL_PROG device add ${devs[3]} $SCRATCH_MNT | _filter_add
+
+# Convert FS to data RAID10, must fail
+$BTRFS_UTIL_PROG balance start -dconvert=raid10 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
+
+# Convert FS to data RAID6, must fail
+$BTRFS_UTIL_PROG balance start -f -dconvert=raid6 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/310.out b/tests/btrfs/310.out
new file mode 100644
index 00000000..bc06b29e
--- /dev/null
+++ b/tests/btrfs/310.out
@@ -0,0 +1,12 @@
+QA output created by 310
+Done, had to relocate X out of X chunks
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
+There may be more info in syslog - try dmesg | tail
+Resetting device zones SCRATCH_DEV (XXX zones) ...
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
+Resetting device zones SCRATCH_DEV (XXX zones) ...
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
+Resetting device zones SCRATCH_DEV (XXX zones) ...
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
+ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
-- 
2.35.3

Re: [PATCH v2] fstests: btrfs: check conversion of zoned fileystems

[Anand Jain]

On 2/13/24 13:50, Johannes Thumshirn wrote:
> Recently we had a bug where a zoned filesystem could be converted to a
> higher data redundancy profile than supported.
> 
> Add a test-case to check the conversion on zoned filesystems.
> 
> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> ---
>   tests/btrfs/310     | 75 +++++++++++++++++++++++++++++++++++++++++++++
>   tests/btrfs/310.out | 12 ++++++++
>   2 files changed, 87 insertions(+)
>   create mode 100755 tests/btrfs/310
>   create mode 100644 tests/btrfs/310.out
> 
> diff --git a/tests/btrfs/310 b/tests/btrfs/310
> new file mode 100755
> index 00000000..6b0846f0
> --- /dev/null
> +++ b/tests/btrfs/310
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
> +#
> +# FS QA Test 310
> +#
> +# Test that btrfs convert can ony be run to convert to supported profiles on a
> +# zoned filesystem
> +#
> +. ./common/preamble
> +_begin_fstest volume raid convert
> +
> +_fixed_by_kernel_commit XXXXXXXXXX \
> +	"btrfs: zoned: don't skip block group profile checks on conv zones"
> +

> +. common/filter
> +. common/filter.btrfs

common/filter.btrfs includes common/filter;
So common/filter can be dropped.

> +
> +_supported_fs btrfs
> +_require_scratch_dev_pool 4

> +_require_zoned_device "$SCRATCH_DEV"

So, only the first device has to be a zone device?

> +
> +
> +_filter_convert()
> +{
> +	_filter_scratch | \
> +	sed -e "s/relocate [0-9]\+ out of [0-9]\+ chunks/relocate X out of X chunks/g"
> +}
> +
> +_filter_add()
> +{
> +	_filter_scratch | _filter_scratch_pool | \
> +	sed -e "s/Resetting device zones SCRATCH_DEV ([0-9]\+/Resetting device zones SCRATCH_DEV (XXX/g"
> +
> +}
> +

Can we add the prefix 'balance' to these functions since they filter
the balance output? Also, imo, it is better to move them to
filter.btrfs.

and..

> +devs=( $SCRATCH_DEV_POOL )
> +
> +# Create and mount single device FS
> +_scratch_mkfs -msingle -dsingle 2>&1 > /dev/null
> +_scratch_mount
> +
> +# Convert FS to metadata/system DUP
> +$BTRFS_UTIL_PROG balance start -f -mconvert=dup -sconvert=dup $SCRATCH_MNT 2>&1 | _filter_convert
> +

Why not update _run_btrfs_balance_start() to support argument passing
and pass the options to it to run balance using the helper function?

Thanks, Anand


> +# Convert FS to data DUP, must fail
> +$BTRFS_UTIL_PROG balance start -dconvert=dup $SCRATCH_MNT 2>&1 | _filter_convert
> +
> +# Add device
> +$BTRFS_UTIL_PROG device add ${devs[1]} $SCRATCH_MNT | _filter_add
> +
> +# Convert FS to data RAID1, must fail
> +$BTRFS_UTIL_PROG balance start -dconvert=raid1 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
> +
> +# Convert FS to data RAID0, must fail
> +$BTRFS_UTIL_PROG balance start -dconvert=raid0 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
> +
> +# Add device
> +$BTRFS_UTIL_PROG device add ${devs[2]} $SCRATCH_MNT | _filter_add
> +
> +# Convert FS to data RAID5, must fail
> +$BTRFS_UTIL_PROG balance start -f -dconvert=raid5 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
> +
> +# Add device
> +$BTRFS_UTIL_PROG device add ${devs[3]} $SCRATCH_MNT | _filter_add
> +
> +# Convert FS to data RAID10, must fail
> +$BTRFS_UTIL_PROG balance start -dconvert=raid10 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
> +
> +# Convert FS to data RAID6, must fail
> +$BTRFS_UTIL_PROG balance start -f -dconvert=raid6 $SCRATCH_MNT 2>&1 | _filter_convert | head -1
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/310.out b/tests/btrfs/310.out
> new file mode 100644
> index 00000000..bc06b29e
> --- /dev/null
> +++ b/tests/btrfs/310.out
> @@ -0,0 +1,12 @@
> +QA output created by 310
> +Done, had to relocate X out of X chunks
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
> +There may be more info in syslog - try dmesg | tail
> +Resetting device zones SCRATCH_DEV (XXX zones) ...
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
> +Resetting device zones SCRATCH_DEV (XXX zones) ...
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
> +Resetting device zones SCRATCH_DEV (XXX zones) ...
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument
> +ERROR: error during balancing 'SCRATCH_MNT': Invalid argument

Re: [PATCH v2] fstests: btrfs: check conversion of zoned fileystems

[Johannes Thumshirn]

On 13.02.24 09:46, Anand Jain wrote:
> 
> 
> On 2/13/24 13:50, Johannes Thumshirn wrote:
>> Recently we had a bug where a zoned filesystem could be converted to a
>> higher data redundancy profile than supported.
>>
>> Add a test-case to check the conversion on zoned filesystems.
>>
>> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
>> Reviewed-by: Filipe Manana <fdmanana@suse.com>
>> ---
>>    tests/btrfs/310     | 75 +++++++++++++++++++++++++++++++++++++++++++++
>>    tests/btrfs/310.out | 12 ++++++++
>>    2 files changed, 87 insertions(+)
>>    create mode 100755 tests/btrfs/310
>>    create mode 100644 tests/btrfs/310.out
>>
>> diff --git a/tests/btrfs/310 b/tests/btrfs/310
>> new file mode 100755
>> index 00000000..6b0846f0
>> --- /dev/null
>> +++ b/tests/btrfs/310
>> @@ -0,0 +1,75 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
>> +#
>> +# FS QA Test 310
>> +#
>> +# Test that btrfs convert can ony be run to convert to supported profiles on a
>> +# zoned filesystem
>> +#
>> +. ./common/preamble
>> +_begin_fstest volume raid convert
>> +
>> +_fixed_by_kernel_commit XXXXXXXXXX \
>> +	"btrfs: zoned: don't skip block group profile checks on conv zones"
>> +
> 
>> +. common/filter
>> +. common/filter.btrfs
> 
> common/filter.btrfs includes common/filter;
> So common/filter can be dropped.

Sure.

> 
>> +
>> +_supported_fs btrfs
>> +_require_scratch_dev_pool 4
> 
>> +_require_zoned_device "$SCRATCH_DEV"
> 
> So, only the first device has to be a zone device?

Nope, but _require_zoned_device only accepts a single device ATM and if 
device 1 is a zoned device, the FS is zoend, so adding non zoned devices 
will treat them as zoned devices using the zone emulation layer. So I 
think it is fine.

> 
>> +
>> +
>> +_filter_convert()
>> +{
>> +	_filter_scratch | \
>> +	sed -e "s/relocate [0-9]\+ out of [0-9]\+ chunks/relocate X out of X chunks/g"
>> +}
>> +
>> +_filter_add()
>> +{
>> +	_filter_scratch | _filter_scratch_pool | \
>> +	sed -e "s/Resetting device zones SCRATCH_DEV ([0-9]\+/Resetting device zones SCRATCH_DEV (XXX/g"
>> +
>> +}
>> +
> 
> Can we add the prefix 'balance' to these functions since they filter
> the balance output? Also, imo, it is better to move them to
> filter.btrfs.
> 

Sure.

> and..
> 
>> +devs=( $SCRATCH_DEV_POOL )
>> +
>> +# Create and mount single device FS
>> +_scratch_mkfs -msingle -dsingle 2>&1 > /dev/null
>> +_scratch_mount
>> +
>> +# Convert FS to metadata/system DUP
>> +$BTRFS_UTIL_PROG balance start -f -mconvert=dup -sconvert=dup $SCRATCH_MNT 2>&1 | _filter_convert
>> +
> 
> Why not update _run_btrfs_balance_start() to support argument passing
> and pass the options to it to run balance using the helper function?
>

I'll look into it.