[PATCH] btrfs/zoned: test premature ENOSPC because of reclaim being too slow

[PATCH] btrfs/zoned: test premature ENOSPC because of reclaim being too slow

[Johannes Thumshirn]

Add a test writing a file of 60% the drive size on a zoned btrfs and then
overwriting the file again.

On fast drives this will cause premature ENOSPC because the reclaim
process isn't triggered fast enough.

The kernel patch for this issue is:
 btrfs: zoned: wake up cleaner sooner if needed

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 tests/btrfs/310     | 54 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/310.out |  2 ++
 2 files changed, 56 insertions(+)
 create mode 100755 tests/btrfs/310
 create mode 100644 tests/btrfs/310.out

diff --git a/tests/btrfs/310 b/tests/btrfs/310
new file mode 100755
index 000000000000..6f6f5542f73f
--- /dev/null
+++ b/tests/btrfs/310
@@ -0,0 +1,54 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Write a single file with 60% disk size to a zoned btrfs and then overwrite
+# it again. On kernels without the fix this results in ENOSPC.
+#
+# This issue is fixed by the following kernel patch:
+#    btrfs: zoned: wake up cleaner sooner if needed
+
+. ./common/preamble
+_begin_fstest auto enospc rw zone
+
+# real QA test starts here
+
+_supported_fs btrfs
+_require_scratch
+_require_zoned_device "$SCRATCH_DEV"
+
+devsize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/size)
+devsize=$(expr $devsize \* 512)
+filesize=$(expr $devsize \* 60 / 100)
+
+fio_config=$tmp.fio
+
+# Override the default cleanup function.
+_cleanup()
+{
+	rm -f $tmp.*
+}
+
+cat >$fio_config <<EOF
+[test]
+filename=$SCRATCH_MNT/test
+readwrite=write
+loops=2
+filesize=$filesize
+EOF
+
+_require_fio $fio_config
+
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount
+
+$FIO_PROG $fio_config >> $seqres.full
+
+_scratch_unmount
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/310.out b/tests/btrfs/310.out
new file mode 100644
index 000000000000..7b9eaf78a07a
--- /dev/null
+++ b/tests/btrfs/310.out
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden
-- 
2.43.0

Re: [PATCH] btrfs/zoned: test premature ENOSPC because of reclaim being too slow

[David Disseldorp]

Hi Johannes,

On Mon, 22 Jan 2024 02:55:54 -0800, Johannes Thumshirn wrote:

> Add a test writing a file of 60% the drive size on a zoned btrfs and then
> overwriting the file again.
> 
> On fast drives this will cause premature ENOSPC because the reclaim
> process isn't triggered fast enough.
> 
> The kernel patch for this issue is:
>  btrfs: zoned: wake up cleaner sooner if needed
> 
> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> ---
>  tests/btrfs/310     | 54 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/310.out |  2 ++
>  2 files changed, 56 insertions(+)
>  create mode 100755 tests/btrfs/310
>  create mode 100644 tests/btrfs/310.out
> 
> diff --git a/tests/btrfs/310 b/tests/btrfs/310
> new file mode 100755
> index 000000000000..6f6f5542f73f
> --- /dev/null
> +++ b/tests/btrfs/310
> @@ -0,0 +1,54 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
> +#
> +# FS QA Test 310
> +#
> +# Write a single file with 60% disk size to a zoned btrfs and then overwrite
> +# it again. On kernels without the fix this results in ENOSPC.
> +#
> +# This issue is fixed by the following kernel patch:
> +#    btrfs: zoned: wake up cleaner sooner if needed

To-be-queued fixes are often flagged in the test via:
_fixed_by_kernel_commit XXXXXXXXXXXX \
	"btrfs: zoned: wake up cleaner sooner if needed"

> +
> +. ./common/preamble
> +_begin_fstest auto enospc rw zone
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs

I don't see anything btrfs specific here, aside from the actual bug
being triggered.
Would it make sense to move this to generic, or would that be a waste of
cycles for non-zone envs?

> +_require_scratch
> +_require_zoned_device "$SCRATCH_DEV"
> +
> +devsize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/size)
> +devsize=$(expr $devsize \* 512)
> +filesize=$(expr $devsize \* 60 / 100)
> +
> +fio_config=$tmp.fio
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	rm -f $tmp.*
> +}

nit: looks like there's no need to override the default cleanup.

Test looks fine otherwise.

Cheers, David

Re: [PATCH] btrfs/zoned: test premature ENOSPC because of reclaim being too slow

[Johannes Thumshirn]

On 22.01.24 13:26, David Disseldorp wrote:
>> +# This issue is fixed by the following kernel patch:
>> +#    btrfs: zoned: wake up cleaner sooner if needed
> 
> To-be-queued fixes are often flagged in the test via:
> _fixed_by_kernel_commit XXXXXXXXXXXX \
> 	"btrfs: zoned: wake up cleaner sooner if needed"

OK will update.

> 
>> +
>> +. ./common/preamble
>> +_begin_fstest auto enospc rw zone
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs btrfs
> 
> I don't see anything btrfs specific here, aside from the actual bug
> being triggered.
> Would it make sense to move this to generic, or would that be a waste of
> cycles for non-zone envs?

Of cause it could, I'm not sure how beneficial this is to other FSes 
though. TBH I'm not aware of any generic test that specifically targets 
issues related to zoned devices, which doesn't mean that it shouldn't.

I'm totally fine either way.

>> +_require_scratch
>> +_require_zoned_device "$SCRATCH_DEV"
>> +
>> +devsize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/size)
>> +devsize=$(expr $devsize \* 512)
>> +filesize=$(expr $devsize \* 60 / 100)
>> +
>> +fio_config=$tmp.fio
>> +
>> +# Override the default cleanup function.
>> +_cleanup()
>> +{
>> +	rm -f $tmp.*
>> +}
> 
> nit: looks like there's no need to override the default cleanup.

Oops.

Re: [PATCH] btrfs/zoned: test premature ENOSPC because of reclaim being too slow

[Filipe Manana]

On Mon, Jan 22, 2024 at 10:56 AM Johannes Thumshirn
<johannes.thumshirn@wdc.com> wrote:
>
> Add a test writing a file of 60% the drive size on a zoned btrfs and then
> overwriting the file again.
>
> On fast drives this will cause premature ENOSPC because the reclaim
> process isn't triggered fast enough.
>
> The kernel patch for this issue is:
>  btrfs: zoned: wake up cleaner sooner if needed
>
> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> ---
>  tests/btrfs/310     | 54 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/310.out |  2 ++
>  2 files changed, 56 insertions(+)
>  create mode 100755 tests/btrfs/310
>  create mode 100644 tests/btrfs/310.out
>
> diff --git a/tests/btrfs/310 b/tests/btrfs/310
> new file mode 100755
> index 000000000000..6f6f5542f73f
> --- /dev/null
> +++ b/tests/btrfs/310
> @@ -0,0 +1,54 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2024 Western Digital Corporation.  All Rights Reserved.
> +#
> +# FS QA Test 310
> +#
> +# Write a single file with 60% disk size to a zoned btrfs and then overwrite
> +# it again. On kernels without the fix this results in ENOSPC.
> +#
> +# This issue is fixed by the following kernel patch:
> +#    btrfs: zoned: wake up cleaner sooner if needed
> +
> +. ./common/preamble
> +_begin_fstest auto enospc rw zone

Why the rw group?
The test only exercises writes.

Also the enospc group, I think it's meant to exercise cases that are
expected to hit ENOSPC.
Here we want to verify that we don't hit ENOSPC (or any other error).

Thanks.

> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_require_scratch
> +_require_zoned_device "$SCRATCH_DEV"
> +
> +devsize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/size)
> +devsize=$(expr $devsize \* 512)
> +filesize=$(expr $devsize \* 60 / 100)
> +
> +fio_config=$tmp.fio
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +       rm -f $tmp.*
> +}
> +
> +cat >$fio_config <<EOF
> +[test]
> +filename=$SCRATCH_MNT/test
> +readwrite=write
> +loops=2
> +filesize=$filesize
> +EOF
> +
> +_require_fio $fio_config
> +
> +_scratch_mkfs >/dev/null 2>&1
> +_scratch_mount
> +
> +$FIO_PROG $fio_config >> $seqres.full
> +
> +_scratch_unmount
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/310.out b/tests/btrfs/310.out
> new file mode 100644
> index 000000000000..7b9eaf78a07a
> --- /dev/null
> +++ b/tests/btrfs/310.out
> @@ -0,0 +1,2 @@
> +QA output created by 310
> +Silence is golden
> --
> 2.43.0
>
>