* [PATCH 1/3] common/btrfs: add _require_btrfs_sysfs_fsid helper
2021-10-20 7:16 [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Anand Jain
@ 2021-10-20 7:16 ` Anand Jain
2021-10-20 7:16 ` [PATCH 2/3] btrfs/248: validate sysfs fsid Anand Jain
` (2 subsequent siblings)
3 siblings, 0 replies; 8+ messages in thread
From: Anand Jain @ 2021-10-20 7:16 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, josef
It checks if the kernel has the following patch
btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
that added sysfs interface to get fsid.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
common/btrfs | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/common/btrfs b/common/btrfs
index ac880bddf524..5d938c19b56a 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -445,3 +445,17 @@ _scratch_btrfs_is_zoned()
[ `_zone_type ${SCRATCH_DEV}` != "none" ] && return 0
return 1
}
+
+_require_btrfs_sysfs_fsid()
+{
+ local fsid
+
+ fsid=$($BTRFS_UTIL_PROG filesystem show $TEST_DIR |grep uuid: |\
+ awk '{print $NF}')
+
+ # Check if the kernel has sysfs fsid support.
+ # Following kernel patch adds it:
+ # btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+ test -f /sys/fs/btrfs/$fsid/devinfo/1/fsid ||\
+ _notrun "Need btrfs sysfs fsid support"
+}
--
2.27.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 2/3] btrfs/248: validate sysfs fsid
2021-10-20 7:16 [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Anand Jain
2021-10-20 7:16 ` [PATCH 1/3] common/btrfs: add _require_btrfs_sysfs_fsid helper Anand Jain
@ 2021-10-20 7:16 ` Anand Jain
2021-10-20 7:16 ` [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device Anand Jain
2021-10-20 18:45 ` [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Josef Bacik
3 siblings, 0 replies; 8+ messages in thread
From: Anand Jain @ 2021-10-20 7:16 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, josef
Validate if the sysfs fsid is the same as fsid as obtained from the
superblock.
Suggested-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
tests/btrfs/248 | 66 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/248.out | 2 ++
2 files changed, 68 insertions(+)
create mode 100755 tests/btrfs/248
create mode 100644 tests/btrfs/248.out
diff --git a/tests/btrfs/248 b/tests/btrfs/248
new file mode 100755
index 000000000000..856cd489d4e4
--- /dev/null
+++ b/tests/btrfs/248
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain. All Rights Reserved.
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test 248
+#
+# Validate if the sysfs devinfo/<devid>/fsid behaves properly
+# Steps:
+# Create a sprout filesystem (an rw device on top of a seed device)
+# Read the seed and sprout devices fsid from the superblock
+# Validate with the sysfs fsid
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_test
+_require_scratch_dev_pool 2
+_require_btrfs_forget_or_module_loadable
+_require_command "$BTRFS_TUNE_PROG" btrfstune
+_require_btrfs_command inspect-internal dump-super
+_require_btrfs_sysfs_fsid
+
+_scratch_dev_pool_get 1
+# use the scratch device as a seed device
+seed_dev=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+
+# create seed device
+_scratch_pool_mkfs >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev
+_scratch_mount >> $seqres.full 2>&1
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+_scratch_unmount
+
+# record the fsid of both seed and sprout devices
+seedfsid=$($BTRFS_UTIL_PROG inspect-internal dump-super $seed_dev |grep ^fsid |\
+ $AWK_PROG '{print $2}')
+sproutfsid=$($BTRFS_UTIL_PROG inspect-internal dump-super $SPARE_DEV |\
+ grep ^fsid |$AWK_PROG '{print $2}')
+
+# validate it with the fsid as shown in the sysfs
+_mount -o device=$seed_dev $SPARE_DEV $SCRATCH_MNT
+
+cat /sys/fs/btrfs/$sproutfsid/devinfo/2/fsid | grep -v $sproutfsid
+cat /sys/fs/btrfs/$sproutfsid/devinfo/1/fsid | grep -v $seedfsid
+
+_scratch_unmount
+_spare_dev_put
+_scratch_dev_pool_put
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/248.out b/tests/btrfs/248.out
new file mode 100644
index 000000000000..58af9173bea3
--- /dev/null
+++ b/tests/btrfs/248.out
@@ -0,0 +1,2 @@
+QA output created by 248
+Silence is golden
--
2.27.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device
2021-10-20 7:16 [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Anand Jain
2021-10-20 7:16 ` [PATCH 1/3] common/btrfs: add _require_btrfs_sysfs_fsid helper Anand Jain
2021-10-20 7:16 ` [PATCH 2/3] btrfs/248: validate sysfs fsid Anand Jain
@ 2021-10-20 7:16 ` Anand Jain
2021-10-20 13:53 ` Josef Bacik
2021-10-20 18:45 ` [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Josef Bacik
3 siblings, 1 reply; 8+ messages in thread
From: Anand Jain @ 2021-10-20 7:16 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, josef
If there is a missing seed device in a sprout, the btrfs filesystem usage
command fails, which is fixed by the following patches:
btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
btrfs-progs: read fsid from the sysfs in device_is_seed
Test if it works now after these patches in the kernel and in the
btrfs-progs respectively.
Suggested-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
tests/btrfs/249 | 67 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/249.out | 2 ++
2 files changed, 69 insertions(+)
create mode 100755 tests/btrfs/249
create mode 100644 tests/btrfs/249.out
diff --git a/tests/btrfs/249 b/tests/btrfs/249
new file mode 100755
index 000000000000..f8f2f07052c6
--- /dev/null
+++ b/tests/btrfs/249
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain. All Rights Reserved.
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test 249
+#
+# Validate if the command 'btrfs filesystem usage' works with missing seed
+# device
+# Steps:
+# Create a degraded raid1 seed device
+# Create a sprout filesystem (an rw device on top of a seed device)
+# Dump 'btrfs filesystem usage', check it didn't fail
+#
+# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
+# btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+# btrfs-progs: read fsid from the sysfs in device_is_seed
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+_require_command "$WIPEFS_PROG" wipefs
+_require_btrfs_forget_or_module_loadable
+
+_scratch_dev_pool_get 2
+# use the scratch devices as seed devices
+seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+sprout_dev=$SPARE_DEV
+
+# create raid1 seed filesystem
+_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev1
+$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
+_btrfs_forget_or_module_reload
+_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+
+# dump filesystem usage if it fails error goes to the bad.out file
+$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
+# also check for the error code
+ret=$?
+if [ $ret != 0 ]; then
+ _fail "FAILED: btrfs filesystem usage, ret $ret"
+fi
+
+_scratch_unmount
+_spare_dev_put
+_scratch_dev_pool_put
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/249.out b/tests/btrfs/249.out
new file mode 100644
index 000000000000..b79a5dca8820
--- /dev/null
+++ b/tests/btrfs/249.out
@@ -0,0 +1,2 @@
+QA output created by 249
+Silence is golden
--
2.27.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device
2021-10-20 7:16 ` [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device Anand Jain
@ 2021-10-20 13:53 ` Josef Bacik
2021-10-20 14:58 ` Anand Jain
0 siblings, 1 reply; 8+ messages in thread
From: Josef Bacik @ 2021-10-20 13:53 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Wed, Oct 20, 2021 at 03:16:44PM +0800, Anand Jain wrote:
> If there is a missing seed device in a sprout, the btrfs filesystem usage
> command fails, which is fixed by the following patches:
>
> btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
> btrfs-progs: read fsid from the sysfs in device_is_seed
>
> Test if it works now after these patches in the kernel and in the
> btrfs-progs respectively.
>
> Suggested-by: Josef Bacik <josef@toxicpanda.com>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
Shouldn't this use
as well? I wish there was a way to detect that btrfs-progs had support for
reading it but I suppose this is a good enough gate. Maybe add a
> ---
> tests/btrfs/249 | 67 +++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/249.out | 2 ++
> 2 files changed, 69 insertions(+)
> create mode 100755 tests/btrfs/249
> create mode 100644 tests/btrfs/249.out
>
> diff --git a/tests/btrfs/249 b/tests/btrfs/249
> new file mode 100755
> index 000000000000..f8f2f07052c6
> --- /dev/null
> +++ b/tests/btrfs/249
> @@ -0,0 +1,67 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Anand Jain. All Rights Reserved.
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 249
> +#
> +# Validate if the command 'btrfs filesystem usage' works with missing seed
> +# device
> +# Steps:
> +# Create a degraded raid1 seed device
> +# Create a sprout filesystem (an rw device on top of a seed device)
> +# Dump 'btrfs filesystem usage', check it didn't fail
> +#
> +# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
> +# btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
> +# btrfs-progs: read fsid from the sysfs in device_is_seed
> +
> +. ./common/preamble
> +_begin_fstest auto quick seed volume
> +
> +# Import common functions.
> +# . ./common/filter
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_require_scratch_dev_pool 3
> +_require_command "$WIPEFS_PROG" wipefs
> +_require_btrfs_forget_or_module_loadable
Need
_require_btrfs_sysfs_fsid
here I think.
> +
> +_scratch_dev_pool_get 2
> +# use the scratch devices as seed devices
> +seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
> +seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
> +
> +# use the spare device as a sprout device
> +_spare_dev_get
> +sprout_dev=$SPARE_DEV
> +
> +# create raid1 seed filesystem
> +_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
> +$BTRFS_TUNE_PROG -S 1 $seed_dev1
> +$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
> +_btrfs_forget_or_module_reload
> +_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1
> +
> +# create a sprout device
> +$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
> +
> +# dump filesystem usage if it fails error goes to the bad.out file
> +$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
> +# also check for the error code
> +ret=$?
> +if [ $ret != 0 ]; then
> + _fail "FAILED: btrfs filesystem usage, ret $ret"
Can you add "check your btrfs-progs version" here or something? In case I'm an
idiot and forget to update btrfs-progs on the overnight xfstests boxes? Thanks,
Josef
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device
2021-10-20 13:53 ` Josef Bacik
@ 2021-10-20 14:58 ` Anand Jain
2021-10-20 18:44 ` Josef Bacik
0 siblings, 1 reply; 8+ messages in thread
From: Anand Jain @ 2021-10-20 14:58 UTC (permalink / raw)
To: Josef Bacik; +Cc: fstests, linux-btrfs
On 20/10/2021 21:53, Josef Bacik wrote:
> On Wed, Oct 20, 2021 at 03:16:44PM +0800, Anand Jain wrote:
>> If there is a missing seed device in a sprout, the btrfs filesystem usage
>> command fails, which is fixed by the following patches:
>>
>> btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
>> btrfs-progs: read fsid from the sysfs in device_is_seed
>>
>> Test if it works now after these patches in the kernel and in the
>> btrfs-progs respectively.
>>
>> Suggested-by: Josef Bacik <josef@toxicpanda.com>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>
> Shouldn't this use
>
>
> as well? I wish there was a way to detect that btrfs-progs had support for
> reading it but I suppose this is a good enough gate. Maybe add a
_require_btrfs_sysfs_fsid ?
The problem is about the nonexistence of the sysfs interface to read
fsid. Adding it will fail to reproduce the problem.
So if there is no sysfs interface, then btrfs-progs will fail back to
the read-sb method, which shall expose it to fail.
>
>
>> ---
>> tests/btrfs/249 | 67 +++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/249.out | 2 ++
>> 2 files changed, 69 insertions(+)
>> create mode 100755 tests/btrfs/249
>> create mode 100644 tests/btrfs/249.out
>>
>> diff --git a/tests/btrfs/249 b/tests/btrfs/249
>> new file mode 100755
>> index 000000000000..f8f2f07052c6
>> --- /dev/null
>> +++ b/tests/btrfs/249
>> @@ -0,0 +1,67 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2021 Anand Jain. All Rights Reserved.
>> +# Copyright (c) 2021 Oracle. All Rights Reserved.
>> +#
>> +# FS QA Test 249
>> +#
>> +# Validate if the command 'btrfs filesystem usage' works with missing seed
>> +# device
>> +# Steps:
>> +# Create a degraded raid1 seed device
>> +# Create a sprout filesystem (an rw device on top of a seed device)
>> +# Dump 'btrfs filesystem usage', check it didn't fail
>> +#
>> +# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
>> +# btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
>> +# btrfs-progs: read fsid from the sysfs in device_is_seed
>> +
>> +. ./common/preamble
>> +_begin_fstest auto quick seed volume
>> +
>> +# Import common functions.
>> +# . ./common/filter
>> +
>> +# real QA test starts here
>> +
>> +# Modify as appropriate.
>> +_supported_fs btrfs
>> +_require_scratch_dev_pool 3
>> +_require_command "$WIPEFS_PROG" wipefs
>> +_require_btrfs_forget_or_module_loadable
>
> Need
>
> _require_btrfs_sysfs_fsid
>
> here I think.
As above.
>
>> +
>> +_scratch_dev_pool_get 2
>> +# use the scratch devices as seed devices
>> +seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
>> +seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
>> +
>> +# use the spare device as a sprout device
>> +_spare_dev_get
>> +sprout_dev=$SPARE_DEV
>> +
>> +# create raid1 seed filesystem
>> +_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
>> +$BTRFS_TUNE_PROG -S 1 $seed_dev1
>> +$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
>> +_btrfs_forget_or_module_reload
>> +_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1
>> +
>> +# create a sprout device
>> +$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
>> +
>> +# dump filesystem usage if it fails error goes to the bad.out file
>> +$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
>> +# also check for the error code
>> +ret=$?
>> +if [ $ret != 0 ]; then
>> + _fail "FAILED: btrfs filesystem usage, ret $ret"
>
> Can you add "check your btrfs-progs version" here or something? In case I'm an
> idiot and forget to update btrfs-progs on the overnight xfstests boxes? Thanks,
>
Sure.
Thanks, Anand
> Josef
>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device
2021-10-20 14:58 ` Anand Jain
@ 2021-10-20 18:44 ` Josef Bacik
0 siblings, 0 replies; 8+ messages in thread
From: Josef Bacik @ 2021-10-20 18:44 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Wed, Oct 20, 2021 at 10:58:56PM +0800, Anand Jain wrote:
>
>
> On 20/10/2021 21:53, Josef Bacik wrote:
> > On Wed, Oct 20, 2021 at 03:16:44PM +0800, Anand Jain wrote:
> > > If there is a missing seed device in a sprout, the btrfs filesystem usage
> > > command fails, which is fixed by the following patches:
> > >
> > > btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
> > > btrfs-progs: read fsid from the sysfs in device_is_seed
> > >
> > > Test if it works now after these patches in the kernel and in the
> > > btrfs-progs respectively.
> > >
> > > Suggested-by: Josef Bacik <josef@toxicpanda.com>
> > > Signed-off-by: Anand Jain <anand.jain@oracle.com>
> >
> > Shouldn't this use
> >
> >
> > as well? I wish there was a way to detect that btrfs-progs had support for
> > reading it but I suppose this is a good enough gate. Maybe add a
>
> _require_btrfs_sysfs_fsid ?
>
> The problem is about the nonexistence of the sysfs interface to read fsid.
> Adding it will fail to reproduce the problem.
>
> So if there is no sysfs interface, then btrfs-progs will fail back to the
> read-sb method, which shall expose it to fail.
>
Sigh sorry, I typed this out and then decided I wanted to comment below, and
forgot to delete this bit.
Yeah sorry I'm thinking about it in terms of the CI testing for us, we'll start
failing this test once its merged without the fix merged, but I guess that's
kinda what we want. Let's just leave this as it is. Thanks,
Josef
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage
2021-10-20 7:16 [PATCH 0/3] fstests: btrfs validate sysfs fsid and usage Anand Jain
` (2 preceding siblings ...)
2021-10-20 7:16 ` [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device Anand Jain
@ 2021-10-20 18:45 ` Josef Bacik
3 siblings, 0 replies; 8+ messages in thread
From: Josef Bacik @ 2021-10-20 18:45 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Wed, Oct 20, 2021 at 03:16:41PM +0800, Anand Jain wrote:
> Patch 1 adds a helper to check if the kernel is latest enough to provide
> the fsid from the sysfs.
> Patch 2 validates the sysfs fsid against the fsid from the sb
> Patch 3 is a test case to make sure if the btrfs filesystem usage is
> successful on a sprout filesystem with a seed device missing.
>
> Anand Jain (3):
> common/btrfs: add _require_btrfs_sysfs_fsid helper
> btrfs/248: validate sysfs fsid
> btrfs/249: test btrfs filesystem usage command on missing seed device
>
You can add
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Thanks,
Josef
^ permalink raw reply [flat|nested] 8+ messages in thread