[PATCH 0/4] btrfs-progs: move RST feature back to experimental

[PATCH 0/4] btrfs-progs: move RST feature back to experimental

[Qu Wenruo]

Although we have exported raid-stripe-tree feature to end users,
the feature is still experimental as only kernels with
CONFIG_BTRFS_DEBUG can even mount it.

This results a feature mismatch in btrfs-progs and kernels, and can lead
to complains/confusion from end users.

This patchset would hide RST feature back behind experimental builds for
mkfs, and completely disable rst for convert, so end users won't and
can't enable RST by accident.

Now RST related test cases (mkfs/029, mkfs/030) are skipped for
regular builds, and a new test case (mkfs/033) is skipped for
experimental builds to make sure there is really no way to enable RST
support for non-experimental builds.

The new experimental build detection also provides the basis for
other experimental features like csum conversion in the future (recent
days).

Both regular and experimental builds have passed the full test suite
(except random misc/055 failure, which is caused by a clear-stale
behavior change, and will be addressed separately)

Qu Wenruo (4):
  btrfs-progs: convert: remove raid-stripe-tree support
  btrfs-progs: hide rst related mkfs tests behind experimental builds
  btrfs-progs: fsfeatures: move RST back to experimental
  btrfs-progs: mkfs-tests: ensure regular builds won't enable rst
    feature

 common/fsfeatures.c                           |  2 ++
 common/fsfeatures.h                           |  3 +-
 mkfs/main.c                                   |  2 ++
 tests/common                                  | 29 ++++++++++++++++
 tests/convert-tests/001-ext2-basic/test.sh    |  2 +-
 tests/convert-tests/003-ext4-basic/test.sh    |  2 +-
 .../005-delete-all-rollback/test.sh           |  2 +-
 .../convert-tests/010-reiserfs-basic/test.sh  |  2 +-
 .../011-reiserfs-delete-all-rollback/test.sh  |  2 +-
 tests/convert-tests/024-ntfs-basic/test.sh    |  2 +-
 tests/mkfs-tests/029-raid-stripe-tree/test.sh |  1 +
 tests/mkfs-tests/030-zoned-rst/test.sh        |  1 +
 tests/mkfs-tests/033-zoned-reject-rst/test.sh | 34 +++++++++++++++++++
 13 files changed, 76 insertions(+), 8 deletions(-)
 create mode 100755 tests/mkfs-tests/033-zoned-reject-rst/test.sh

--
2.45.2

[PATCH 1/4] btrfs-progs: convert: remove raid-stripe-tree support

[Qu Wenruo]

The raid-stripe-tree (RST) feature is for zoned devices to support extra
data profiles, and is not yet a stable feature (still requires
CONFIG_BTRFS_DEBUG enabled kernel to support it).

Furthermore the supported filesystems (ext*, reiserfs and ntfs) doesn't
even support zoned devices, and even we force RST support for
btrfs-convert, we would only create an empty tree for RST, as btrfs
convert would only result SINGLE data profile with SINGLE/DUP metadata
profile, neither needs RST at all.

Meanwhile enabling RST for btrfs-convert would only cause problems for
false test failures as we incorrectly allow RST feature for
btrfs-convert.

This patch fixes the problem by remove raid-stripe-tree support from
btrfs-convert and remove the test cases support for RST.

This patch is mostly reverting commit 346a3819237b ("btrfs-progs:
convert: add raid-stripe-tree to allowed features"), but keeps the test
infrastructure to support bgt features for convert.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 common/fsfeatures.h                                          | 3 +--
 tests/convert-tests/001-ext2-basic/test.sh                   | 2 +-
 tests/convert-tests/003-ext4-basic/test.sh                   | 2 +-
 tests/convert-tests/005-delete-all-rollback/test.sh          | 2 +-
 tests/convert-tests/010-reiserfs-basic/test.sh               | 2 +-
 tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh | 2 +-
 tests/convert-tests/024-ntfs-basic/test.sh                   | 2 +-
 7 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/common/fsfeatures.h b/common/fsfeatures.h
index 5b241bdf9122..f636171f6bd9 100644
--- a/common/fsfeatures.h
+++ b/common/fsfeatures.h
@@ -68,8 +68,7 @@ static const struct btrfs_mkfs_features btrfs_convert_allowed_features = {
 			   BTRFS_FEATURE_INCOMPAT_BIG_METADATA |
 			   BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF |
 			   BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA |
-			   BTRFS_FEATURE_INCOMPAT_NO_HOLES |
-			   BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE,
+			   BTRFS_FEATURE_INCOMPAT_NO_HOLES,
 	.runtime_flags   = BTRFS_FEATURE_RUNTIME_QUOTA,
 };
 
diff --git a/tests/convert-tests/001-ext2-basic/test.sh b/tests/convert-tests/001-ext2-basic/test.sh
index 461ff4a5f1a9..d905ed4aeecb 100755
--- a/tests/convert-tests/001-ext2-basic/test.sh
+++ b/tests/convert-tests/001-ext2-basic/test.sh
@@ -12,7 +12,7 @@ check_kernel_support_acl
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices
-for feature in '' 'block-group-tree' 'raid-stripe-tree'; do
+for feature in '' 'block-group-tree'; do
 	convert_test ext2 "$feature" "ext2 4k nodesize" 4096 mke2fs -b 4096
 	convert_test ext2 "$feature" "ext2 16k nodesize" 16384 mke2fs -b 4096
 	convert_test ext2 "$feature" "ext2 64k nodesize" 65536 mke2fs -b 4096
diff --git a/tests/convert-tests/003-ext4-basic/test.sh b/tests/convert-tests/003-ext4-basic/test.sh
index 14637fc852db..83a2cf12425c 100755
--- a/tests/convert-tests/003-ext4-basic/test.sh
+++ b/tests/convert-tests/003-ext4-basic/test.sh
@@ -12,7 +12,7 @@ check_kernel_support_acl
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices
-for feature in '' 'block-group-tree' 'raid-stripe-tree' ; do
+for feature in '' 'block-group-tree'; do
 	convert_test ext4 "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096
 	convert_test ext4 "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096
 	convert_test ext4 "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096
diff --git a/tests/convert-tests/005-delete-all-rollback/test.sh b/tests/convert-tests/005-delete-all-rollback/test.sh
index 5603d3078bc8..536d0313ffdc 100755
--- a/tests/convert-tests/005-delete-all-rollback/test.sh
+++ b/tests/convert-tests/005-delete-all-rollback/test.sh
@@ -68,7 +68,7 @@ do_test() {
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices
-for feature in '' 'block-group-tree' 'raid-stripe-tree' ; do
+for feature in '' 'block-group-tree'; do
 	do_test "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096
 	do_test "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096
 	do_test "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096
diff --git a/tests/convert-tests/010-reiserfs-basic/test.sh b/tests/convert-tests/010-reiserfs-basic/test.sh
index 6ab02b31d176..67e818b9827d 100755
--- a/tests/convert-tests/010-reiserfs-basic/test.sh
+++ b/tests/convert-tests/010-reiserfs-basic/test.sh
@@ -15,7 +15,7 @@ prepare_test_dev
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices
-for feature in '' 'block-group-tree' 'raid-stripe-tree'; do
+for feature in '' 'block-group-tree'; do
 	convert_test reiserfs "$feature" "reiserfs 4k nodesize" 4096 mkreiserfs -b 4096
 	convert_test reiserfs "$feature" "reiserfs 16k nodesize" 16384 mkreiserfs -b 4096
 	convert_test reiserfs "$feature" "reiserfs 64k nodesize" 65536 mkreiserfs -b 4096
diff --git a/tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh b/tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
index 688613c2c6ad..d874422659a6 100755
--- a/tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
+++ b/tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
@@ -70,7 +70,7 @@ do_test() {
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices
-for feature in '' 'block-group-tree' 'raid-stripe-tree'; do
+for feature in '' 'block-group-tree'; do
 	do_test "$feature" "reiserfs 4k nodesize" 4096 mkreiserfs -b 4096
 	do_test "$feature" "reiserfs 16k nodesize" 16384 mkreiserfs -b 4096
 	do_test "$feature" "reiserfs 64k nodesize" 65536 mkreiserfs -b 4096
diff --git a/tests/convert-tests/024-ntfs-basic/test.sh b/tests/convert-tests/024-ntfs-basic/test.sh
index a93f60070674..b88dbe29c356 100755
--- a/tests/convert-tests/024-ntfs-basic/test.sh
+++ b/tests/convert-tests/024-ntfs-basic/test.sh
@@ -17,6 +17,6 @@ prepare_test_dev
 
 # Iterate over defaults and options that are not tied to hardware capabilities
 # or number of devices. Test only 4K block size as minimum.
-for feature in '' 'block-group-tree' 'raid-stripe-tree'; do
+for feature in '' 'block-group-tree'; do
 	convert_test ntfs "$feature" "ntfs 4k nodesize" 4096 mkfs.ntfs -s 4096
 done
-- 
2.45.2

[PATCH 2/4] btrfs-progs: hide rst related mkfs tests behind experimental builds

[Qu Wenruo]

Currently we unconditionally run mkfs/029 and mkfs/030 as we export
RST feature through mkfs.btrfs as supported features.

But considering the mkfs.btrfs features are mismatch with kernel support
(only a BTRFS_DEBUG kernel can support that), we're going to hide RST
behind experimental builds.

In that case RST related tests cases also need to be behind
experimental builds as regular builds of mkfs.btrfs will no longer
support RST feature.

This patch would introduce two helpers:

- _test_config()
  To verify if certain config is set to 1

- check_experimental_build()
  A wrapper of "_test_config EXPERIMENTAL", and skip the test if
  btrfs-progs has no experimental features.

So that test cases can be skipped for non-experimental builds.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/common                                  | 22 +++++++++++++++++++
 tests/mkfs-tests/029-raid-stripe-tree/test.sh |  1 +
 tests/mkfs-tests/030-zoned-rst/test.sh        |  1 +
 3 files changed, 24 insertions(+)

diff --git a/tests/common b/tests/common
index 7a23f6dde1a5..e9456c1d96b8 100644
--- a/tests/common
+++ b/tests/common
@@ -6,6 +6,21 @@
 # Temporary array for building the final command, injecting arguments as needed
 declare -a cmd_array
 
+# Check if certain config is set to 1
+_test_config()
+{
+	local feature="$1"
+
+	if [ ! -f "${TOP}/include/config.h" ]; then
+		echo "include/config.h not exists"
+		exit 1
+	fi
+	if grep -q "${feature}.*1" "${TOP}/include/config.h"; then
+		return 0
+	fi
+	return 1
+}
+
 # assert that argument is not empty and is an existing path (file or directory)
 _assert_path()
 {
@@ -370,6 +385,13 @@ run_mustfail_stdout()
 	fi
 }
 
+check_experimental_build()
+{
+	if ! _test_config "EXPERIMENTAL"; then
+		_not_run "This test requires experimental build"
+	fi
+}
+
 check_prereq()
 {
 	# Internal tools for testing, not shipped with the package
diff --git a/tests/mkfs-tests/029-raid-stripe-tree/test.sh b/tests/mkfs-tests/029-raid-stripe-tree/test.sh
index c3edd36d8761..f200d068fe79 100755
--- a/tests/mkfs-tests/029-raid-stripe-tree/test.sh
+++ b/tests/mkfs-tests/029-raid-stripe-tree/test.sh
@@ -3,6 +3,7 @@
 
 source "$TEST_TOP/common" || exit
 
+check_experimental_build
 check_prereq mkfs.btrfs
 check_prereq btrfs
 
diff --git a/tests/mkfs-tests/030-zoned-rst/test.sh b/tests/mkfs-tests/030-zoned-rst/test.sh
index b1c696c96eb7..b03c691f1b69 100755
--- a/tests/mkfs-tests/030-zoned-rst/test.sh
+++ b/tests/mkfs-tests/030-zoned-rst/test.sh
@@ -3,6 +3,7 @@
 
 source "$TEST_TOP/common" || exit
 
+check_experimental_build
 setup_root_helper
 setup_nullbdevs 4 128 4
 prepare_nullbdevs
-- 
2.45.2

[PATCH 3/4] btrfs-progs: fsfeatures: move RST back to experimental

[Qu Wenruo]

Currently raid-stripe-tree feature is still experimental as it requires
a BTRFS_DEBUG kernel to recognize it.

To avoid confusion move it back to experimental so regular users won't
incorrectly set it.

And since RST is no longer supported by default, also change the RST
profile detection so that for non-experimental build we won't enable RST
according to the data profiles.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 common/fsfeatures.c | 2 ++
 mkfs/main.c         | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/common/fsfeatures.c b/common/fsfeatures.c
index c5e81629ccea..7aaddab6a192 100644
--- a/common/fsfeatures.c
+++ b/common/fsfeatures.c
@@ -222,6 +222,7 @@ static const struct btrfs_feature mkfs_features[] = {
 		VERSION_NULL(default),
 		.desc		= "block group tree, more efficient block group tracking to reduce mount time"
 	},
+#if EXPERIMENTAL
 	{
 		.name		= "rst",
 		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE,
@@ -238,6 +239,7 @@ static const struct btrfs_feature mkfs_features[] = {
 		VERSION_NULL(default),
 		.desc		= "raid stripe tree, enhanced file extent tracking"
 	},
+#endif
 	{
 		.name		= "squota",
 		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA,
diff --git a/mkfs/main.c b/mkfs/main.c
index 0b9d1d78663d..8d0924ea3b0b 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -1693,7 +1693,9 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 		case BTRFS_BLOCK_GROUP_RAID1C4:
 		case BTRFS_BLOCK_GROUP_RAID0:
 		case BTRFS_BLOCK_GROUP_RAID10:
+#if EXPERIMENTAL
 			features.incompat_flags |= BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE;
+#endif
 			break;
 		default:
 			break;
-- 
2.45.2

[PATCH 4/4] btrfs-progs: mkfs-tests: ensure regular builds won't enable rst feature

[Qu Wenruo]

This test case will check the following behaivors:

- Regular zoned supported data/metadata profiles
  Should success

- RST with zoned feature
  Should fail for non-experimental builds

- zoned with data profiles that only RST supports
  Should fail for non-experimental builds.

  Meanwhile for experimental builds it should success and enable RST
  feature automatically (verified in mkfs/030)

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/common                                  |  7 ++++
 tests/mkfs-tests/033-zoned-reject-rst/test.sh | 34 +++++++++++++++++++
 2 files changed, 41 insertions(+)
 create mode 100755 tests/mkfs-tests/033-zoned-reject-rst/test.sh

diff --git a/tests/common b/tests/common
index e9456c1d96b8..12ec1ad92b85 100644
--- a/tests/common
+++ b/tests/common
@@ -392,6 +392,13 @@ check_experimental_build()
 	fi
 }
 
+check_regular_build()
+{
+	if _test_config "EXPERIMENTAL"; then
+		_not_run "This test requires non-experimental build"
+	fi
+}
+
 check_prereq()
 {
 	# Internal tools for testing, not shipped with the package
diff --git a/tests/mkfs-tests/033-zoned-reject-rst/test.sh b/tests/mkfs-tests/033-zoned-reject-rst/test.sh
new file mode 100755
index 000000000000..698551eccd59
--- /dev/null
+++ b/tests/mkfs-tests/033-zoned-reject-rst/test.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+# Verify mkfs for all currently supported profiles of zoned + raid-stripe-tree
+
+source "$TEST_TOP/common" || exit
+
+check_regular_build
+setup_root_helper
+setup_nullbdevs 4 128 4
+prepare_nullbdevs
+TEST_DEV=${nullb_devs[1]}
+
+profiles="dup raid1 raid1c3 raid1c4 raid10"
+
+# The existing supported profiles.
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d single -m single "${nullb_devs[@]}"
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d single -m DUP "${nullb_devs[@]}"
+
+# RST feature is rejected
+run_mustfail "RST feature created" \
+	$SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned,raid-stripe-tree -d single -m DUP "${nullb_devs[@]}"
+
+for dprofile in $profiles; do
+	# make sure extra data profiles won't enable RST for non-experimental build
+	run_mustfail "unsupported profile created" \
+		$SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d "$dprofile" -m DUP "${nullb_devs[@]}"
+done
+
+# The old unsupported profiles should fail no matter experimental build or not.
+run_mustfail "unsupported profile raid56 created" \
+	$SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d raid5 -m raid5 "${nullb_devs[@]}"
+run_mustfail "unsupported profile raid56 created" \
+	$SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d raid6 -m raid6 "${nullb_devs[@]}"
+
+cleanup_nullbdevs
-- 
2.45.2

Re: [PATCH 0/4] btrfs-progs: move RST feature back to experimental

[HAN Yuwei]

[-- Attachment #1.1: Type: text/plain, Size: 361 bytes --]

> This patchset would hide RST feature back behind experimental builds for
> mkfs, and completely disable rst for convert, so end users won't and
> can't enable RST by accident.
How about introduce a option named like "--experimental" to enable these 
feature?

It can work perfectly and don't need me to re-compiling another btrfs-progs.

HAN Yuwei


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 0/4] btrfs-progs: move RST feature back to experimental

[Johannes Thumshirn]

For the series,
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>

Thanks

Re: [PATCH 0/4] btrfs-progs: move RST feature back to experimental

[Qu Wenruo]

在 2024/6/13 17:17, HAN Yuwei 写道:
>> This patchset would hide RST feature back behind experimental builds for
>> mkfs, and completely disable rst for convert, so end users won't and
>> can't enable RST by accident.
> How about introduce a option named like "--experimental" to enable these 
> feature?

Well, that's exactly what we (at least me) want, to make it harder for 
end uers to reach it.

For you, with your advanced subpage and zoned setup, compiling 
btrfs-progs should only be a small hassle, and you only need to do it once.

Thanks,
Qu
> 
> It can work perfectly and don't need me to re-compiling another 
> btrfs-progs.
> 
> HAN Yuwei
> 

Re: [PATCH 0/4] btrfs-progs: move RST feature back to experimental

[Johannes Thumshirn]

On 13.06.24 10:43, Qu Wenruo wrote:
> 
> 
> 在 2024/6/13 17:17, HAN Yuwei 写道:
>>> This patchset would hide RST feature back behind experimental builds for
>>> mkfs, and completely disable rst for convert, so end users won't and
>>> can't enable RST by accident.
>> How about introduce a option named like "--experimental" to enable these
>> feature?
> 
> Well, that's exactly what we (at least me) want, to make it harder for
> end uers to reach it.

I completely agree with Qu here. As much as I love you being an early 
adopter for my RST stuff, it is not in a shape where I want it to be 
used by people.

Also there will be a change in the on-disk format as well, so please do 
not (yet) use it.


> For you, with your advanced subpage and zoned setup, compiling
> btrfs-progs should only be a small hassle, and you only need to do it once.
> 
> Thanks,
> Qu
>>
>> It can work perfectly and don't need me to re-compiling another
>> btrfs-progs.
>>
>> HAN Yuwei
>>
> 
>