[PATCH] btrfs/277: specify protocol version 3 for verity send

[PATCH] btrfs/277: specify protocol version 3 for verity send

[Boris Burkov]

This test uses btrfs send with fs-verity which relies on protocol
version 3. The default in progs is version 2, so we need to explicitly
specify the protocol version. Note that the max protocol version in
progs is also currently broken (not properly gated by EXPERIMENTAL) so
that needs fixing as well.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 tests/btrfs/277 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/btrfs/277 b/tests/btrfs/277
index f5684fde1..3898bda4d 100755
--- a/tests/btrfs/277
+++ b/tests/btrfs/277
@@ -84,7 +84,7 @@ _test_send_verity() {
 	echo "set subvolume read only"
 	$BTRFS_UTIL_PROG property set $subv ro true
 	echo "send subvolume"
-	$BTRFS_UTIL_PROG send $subv -f $stream -q >> $seqres.full
+	$BTRFS_UTIL_PROG send $subv -f $stream -q --proto=3 >> $seqres.full
 
 	echo "blow away fs"
 	_scratch_unmount
-- 
2.43.0

[PATCH] btrfs/316: use rescan wrapper

[Boris Burkov]

btrfs/316 is broken on the squota configuration because it uses a raw
rescan call which fails, instead of using the rescan wrapper. The test
passes with squota, so run it (instead of requiring rescan) though I
suspect it isn't the most meaningful test.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 tests/btrfs/316 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/btrfs/316 b/tests/btrfs/316
index 07a94334a..36fcad7f8 100755
--- a/tests/btrfs/316
+++ b/tests/btrfs/316
@@ -24,7 +24,8 @@ _scratch_mkfs >> $seqres.full
 _scratch_mount
 
 $BTRFS_UTIL_PROG quota enable $SCRATCH_MNT
-$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full
+#$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full
+_qgroup_rescan $SCRATCH_MNT >> $seqres.full
 
 $BTRFS_UTIL_PROG qgroup create 1/0 $SCRATCH_MNT >> $seqres.full
 $BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/subv1 >> $seqres.full
-- 
2.43.0

[PATCH] btrfs: fix _require_btrfs_mkfs_uuid_option

[Boris Burkov]

currently, this helper causes tests to fail on my system due to
unexpected warning messages about dangling '\' characters from grep.
Change the invocation to not escape the '-' characters but instead to
use -- to instruct grep that the options list is over. Fixes
btrfs/31[345] for me (and I believe the btrfs github ci)

Signed-off-by: Boris Burkov <boris@bur.io>
---
 common/btrfs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/common/btrfs b/common/btrfs
index 3eb2a91b7..aa344706c 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -93,7 +93,7 @@ _require_btrfs_mkfs_uuid_option()
 	local cnt
 
 	cnt=$($MKFS_BTRFS_PROG --help 2>&1 | \
-				grep -E --count "\-\-uuid|\-\-device-uuid")
+				grep -E --count -- "--uuid|--device-uuid")
 	if [ $cnt != 2 ]; then
 		_notrun "Require $MKFS_BTRFS_PROG with --uuid and --device-uuid options"
 	fi
-- 
2.43.0

Re: [PATCH] btrfs: fix _require_btrfs_mkfs_uuid_option

[Filipe Manana]

On Wed, Mar 13, 2024 at 11:46 PM Boris Burkov <boris@bur.io> wrote:
>
> currently, this helper causes tests to fail on my system due to
> unexpected warning messages about dangling '\' characters from grep.
> Change the invocation to not escape the '-' characters but instead to
> use -- to instruct grep that the options list is over. Fixes
> btrfs/31[345] for me (and I believe the btrfs github ci)
>
> Signed-off-by: Boris Burkov <boris@bur.io>
> ---
>  common/btrfs | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/common/btrfs b/common/btrfs
> index 3eb2a91b7..aa344706c 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -93,7 +93,7 @@ _require_btrfs_mkfs_uuid_option()
>         local cnt
>
>         cnt=$($MKFS_BTRFS_PROG --help 2>&1 | \
> -                               grep -E --count "\-\-uuid|\-\-device-uuid")
> +                               grep -E --count -- "--uuid|--device-uuid")

This was already fixed and it's in the patches-in-queue branch:

https://lore.kernel.org/fstests/ef2df19486ef71adccd14b3df0bf475ecc7f3b38.1709737287.git.fdmanana@suse.com/

https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/log/?h=patches-in-queue

>         if [ $cnt != 2 ]; then
>                 _notrun "Require $MKFS_BTRFS_PROG with --uuid and --device-uuid options"
>         fi
> --
> 2.43.0
>
>

Re: [PATCH] btrfs/316: use rescan wrapper

[Filipe Manana]

On Wed, Mar 13, 2024 at 11:45 PM Boris Burkov <boris@bur.io> wrote:
>
> btrfs/316 is broken on the squota configuration because it uses a raw
> rescan call which fails, instead of using the rescan wrapper. The test
> passes with squota, so run it (instead of requiring rescan) though I
> suspect it isn't the most meaningful test.
>
> Signed-off-by: Boris Burkov <boris@bur.io>
> ---
>  tests/btrfs/316 | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tests/btrfs/316 b/tests/btrfs/316
> index 07a94334a..36fcad7f8 100755
> --- a/tests/btrfs/316
> +++ b/tests/btrfs/316
> @@ -24,7 +24,8 @@ _scratch_mkfs >> $seqres.full
>  _scratch_mount
>
>  $BTRFS_UTIL_PROG quota enable $SCRATCH_MNT
> -$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full
> +#$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full

Why the comment and not remove the line? Seems like unintentional.

> +_qgroup_rescan $SCRATCH_MNT >> $seqres.full

With that fixed:

Reviewed-by: Filipe Manana <fdmanana@suse.com>

>
>  $BTRFS_UTIL_PROG qgroup create 1/0 $SCRATCH_MNT >> $seqres.full
>  $BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/subv1 >> $seqres.full
> --
> 2.43.0
>
>

Re: [PATCH] btrfs/277: specify protocol version 3 for verity send

[Filipe Manana]

On Wed, Mar 13, 2024 at 11:45 PM Boris Burkov <boris@bur.io> wrote:
>
> This test uses btrfs send with fs-verity which relies on protocol
> version 3. The default in progs is version 2, so we need to explicitly
> specify the protocol version. Note that the max protocol version in
> progs is also currently broken (not properly gated by EXPERIMENTAL) so
> that needs fixing as well.
>
> Signed-off-by: Boris Burkov <boris@bur.io>

Reviewed-by: Filipe Manana <fdmanana@suse.com>

Looks good, thanks.

> ---
>  tests/btrfs/277 | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/btrfs/277 b/tests/btrfs/277
> index f5684fde1..3898bda4d 100755
> --- a/tests/btrfs/277
> +++ b/tests/btrfs/277
> @@ -84,7 +84,7 @@ _test_send_verity() {
>         echo "set subvolume read only"
>         $BTRFS_UTIL_PROG property set $subv ro true
>         echo "send subvolume"
> -       $BTRFS_UTIL_PROG send $subv -f $stream -q >> $seqres.full
> +       $BTRFS_UTIL_PROG send $subv -f $stream -q --proto=3 >> $seqres.full
>
>         echo "blow away fs"
>         _scratch_unmount
> --
> 2.43.0
>
>

Re: [PATCH] btrfs/316: use rescan wrapper

[Anand Jain]

On 3/14/24 05:16, Boris Burkov wrote:
> btrfs/316 is broken on the squota configuration because it uses a raw
> rescan call which fails, instead of using the rescan wrapper. The test
> passes with squota, so run it (instead of requiring rescan) though I
> suspect it isn't the most meaningful test.
> 
> Signed-off-by: Boris Burkov <boris@bur.io>
> ---
>   tests/btrfs/316 | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/btrfs/316 b/tests/btrfs/316
> index 07a94334a..36fcad7f8 100755
> --- a/tests/btrfs/316
> +++ b/tests/btrfs/316
> @@ -24,7 +24,8 @@ _scratch_mkfs >> $seqres.full
>   _scratch_mount
>   
>   $BTRFS_UTIL_PROG quota enable $SCRATCH_MNT
> -$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full

> +#$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full


Delete instead, I'll fix this locally.

> +_qgroup_rescan $SCRATCH_MNT >> $seqres.full
>   

However, we also need.

_require_qgroup_rescan ??

Could you pls confirm. Thx.


>   $BTRFS_UTIL_PROG qgroup create 1/0 $SCRATCH_MNT >> $seqres.full
>   $BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/subv1 >> $seqres.full

Re: [PATCH] btrfs/277: specify protocol version 3 for verity send

[Anand Jain]

On 3/14/24 05:16, Boris Burkov wrote:
> This test uses btrfs send with fs-verity which relies on protocol
> version 3. The default in progs is version 2, so we need to explicitly
> specify the protocol version. Note that the max protocol version in
> progs is also currently broken (not properly gated by EXPERIMENTAL) so
> that needs fixing as well.
> 
> Signed-off-by: Boris Burkov <boris@bur.io>
> ---
>   tests/btrfs/277 | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/btrfs/277 b/tests/btrfs/277
> index f5684fde1..3898bda4d 100755
> --- a/tests/btrfs/277
> +++ b/tests/btrfs/277
> @@ -84,7 +84,7 @@ _test_send_verity() {
>   	echo "set subvolume read only"
>   	$BTRFS_UTIL_PROG property set $subv ro true
>   	echo "send subvolume"
> -	$BTRFS_UTIL_PROG send $subv -f $stream -q >> $seqres.full
> +	$BTRFS_UTIL_PROG send $subv -f $stream -q --proto=3 >> $seqres.full
>   
>   	echo "blow away fs"
>   	_scratch_unmount


As of now, the test fails if v3 support isn't in the Send.
Instead, we need to create _require_btrfs_send_v3() to call _notrun().

Thanks, Anand

Re: [PATCH] btrfs/277: specify protocol version 3 for verity send

[David Sterba]

On Wed, Mar 13, 2024 at 04:46:29PM -0700, Boris Burkov wrote:
> This test uses btrfs send with fs-verity which relies on protocol
> version 3. The default in progs is version 2, so we need to explicitly
> specify the protocol version. Note that the max protocol version in
> progs is also currently broken (not properly gated by EXPERIMENTAL) so
> that needs fixing as well.

What do you mean? Progs are ready for protocol 3 as its availability
depends on kernel and if it's enabled then it's assumed that progs match
the protocol specification.

Re: [PATCH] btrfs/277: specify protocol version 3 for verity send

[Boris Burkov]

On Thu, Mar 14, 2024 at 04:01:22PM +0100, David Sterba wrote:
> On Wed, Mar 13, 2024 at 04:46:29PM -0700, Boris Burkov wrote:
> > This test uses btrfs send with fs-verity which relies on protocol
> > version 3. The default in progs is version 2, so we need to explicitly
> > specify the protocol version. Note that the max protocol version in
> > progs is also currently broken (not properly gated by EXPERIMENTAL) so
> > that needs fixing as well.
> 
> What do you mean? Progs are ready for protocol 3 as its availability
> depends on kernel and if it's enabled then it's assumed that progs match
> the protocol specification.

I'm not sure exactly how it happened, but I currently also can't get
btrfs-progs to actually put proto=3 in the send ioctl. I believe that
the #ifdef CONFIG_BTRFS_DEBUG used to set the protocol version doesn't
pick up the config value in the kernel (should it?!)

I was able to fix it by switching to using EXPERIMENTAL as the flag but
haven't sent that yet.