[PATCH 0/4] btrfs-progs: tests false alerts fixes

[PATCH 0/4] btrfs-progs: tests false alerts fixes

[Qu Wenruo]

This is the resent and aggregate version of the fixes I sent but mostly
eaten by the recent vger down time.

Those are small fixes for the false alerts I hit during my local runs.

Most of them are subtle fixes, only the last one is more like an
optimization.

Qu Wenruo (4):
  btrfs-progs: tests/mkfs/005: use udevadm settle to avoid false alerts
  btrfs-progs: tests/misc/030: do not require v1 cache for the test case
  btrfs-progs: tests/misc/046: fix false alerts on write detection
  btrfs-progs: tests/misc/058: reduce the space requirement and speed up
    the test

 tests/misc-tests/030-missing-device-image/test.sh     | 8 --------
 tests/misc-tests/046-seed-multi-mount/test.sh         | 4 +++-
 tests/misc-tests/058-replace-start-enqueue/test.sh    | 5 ++---
 tests/mkfs-tests/005-long-device-name-for-ssd/test.sh | 4 +++-
 4 files changed, 8 insertions(+), 13 deletions(-)

--
2.41.0

[PATCH 1/4] btrfs-progs: tests/mkfs/005: use udevadm settle to avoid false alerts

[Qu Wenruo]

[BUG]
During my test runs of mkfs-tests, 005-long-device-name-for-ssd failed
with the following error messages:

  ====== RUN CHECK dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
  device-mapper: remove ioctl on btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc  failed: Device or resource busy
  Command failed.
  failed: dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
  test failed for case 005-long-device-name-for-ssd

[CAUSE]
There seems to be a race between "btrfs inspect dump-super" and the
dmsetup removal.

[FIX]
Add a "udevadm settle" before removing the dm devices.

Also since we're here, use the same "udevadm settle" instead of the
manual sleep to wait for the new dm device to showup.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/mkfs-tests/005-long-device-name-for-ssd/test.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh b/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh
index 208698dce243..c32eb3a42def 100755
--- a/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh
+++ b/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh
@@ -24,7 +24,7 @@ loopdev=`run_check_stdout $SUDO_HELPER losetup --find --show img`
 run_check $SUDO_HELPER dmsetup create "$dmname" --table "0 1048576 linear $loopdev 0"
 
 # Setting up the device may need some time to appear
-sleep 5
+run_check $SUDO_HELPER udevadm settle
 if ! [ -b "$dmdev" ]; then
 	_not_run "dm device created but not visible in /dev/mapper"
 fi
@@ -43,6 +43,8 @@ run_check_stdout $SUDO_HELPER "$TOP/mkfs.btrfs" -f "$@" "$dmdev" |
 	grep -q 'SSD detected:.*yes' || _fail 'SSD not detected'
 run_check $SUDO_HELPER "$TOP/btrfs" inspect-internal dump-super "$dmdev"
 
+run_check $SUDO_HELPER udevadm settle
+
 # cleanup
 run_check $SUDO_HELPER dmsetup remove "$dmname"
 run_mayfail $SUDO_HELPER losetup -d "$loopdev"
-- 
2.41.0

[PATCH 2/4] btrfs-progs: tests/misc/030: do not require v1 cache for the test case

[Qu Wenruo]

[PROBLEM]
Since we have migrated to default v2 cache, the test case
misc/030-missing-device-image is no longer executed:

    [TEST/misc]   030-missing-device-image
    [NOTRUN] unable to create v1 space cache

[CAUSE]
The test case itself is trying its best to cover all paths, including
the data extent read path.

Thus the test case is requiring v1 cache, as that's the only way to
cover the data read path.

[FIX]
Just remove the v1 space cache requirement, it's still better to run the
test even it only exercises the metadata read path.

The good news is, after commit 3ff9d352576b ("btrfs-progs: use
read_data_from_disk() to replace read_extent_from_disk() and replace
read_extent_data()"), all data/metadata read paths are unified.
They only differ in the verification part.

Thus even if we didn't fully exercise the data read path, we didn't lose
much coverage anyway.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/misc-tests/030-missing-device-image/test.sh | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/tests/misc-tests/030-missing-device-image/test.sh b/tests/misc-tests/030-missing-device-image/test.sh
index be022c4bb9eb..1438f8a45229 100755
--- a/tests/misc-tests/030-missing-device-image/test.sh
+++ b/tests/misc-tests/030-missing-device-image/test.sh
@@ -37,14 +37,6 @@ test_missing()
 	run_check $SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/b" bs=4k count=1000 conv=sync
 	run_check $SUDO_HELPER umount "$TEST_MNT"
 
-	# make sure we have space cache
-	if ! run_check_stdout "$TOP/btrfs" inspect dump-tree -t root "$dev1" |
-		grep -q "EXTENT_DATA"; then
-		# Normally the above operation should create the space cache.
-		# If not, it may mean we have migrated to v2 cache by default
-		_not_run "unable to create v1 space cache"
-	fi
-
 	# now wipe the device
 	run_check wipefs -fa "$bad_dev"
 
-- 
2.41.0

[PATCH 3/4] btrfs-progs: tests/misc/046: fix false alerts on write detection

[Qu Wenruo]

[BUG]
Test case misc/046-seed-multi-mount would always fail with the following
error:

    [TEST]   misc-tests.sh
    [TEST/misc]   046-seed-multi-mount
unexpected success: writable file despite read-only mount
test failed for case 046-seed-multi-mount

[CAUSE]
Although mounting seed device is indeed read-only, sprouting it with a
new device would always make it read-write by itself.

The behavior is already there for a long time, thus expecting a new
behavior (not changing the read-only flag) is a little weird.

[FIX]
Instead of doing the write check after the sprout, do it before the
sprout.

This looks more correct, and would not rely on the kernel behavior
change (if we determine to go that path).

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/misc-tests/046-seed-multi-mount/test.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/misc-tests/046-seed-multi-mount/test.sh b/tests/misc-tests/046-seed-multi-mount/test.sh
index 83f7a91effdd..95654fd904ff 100755
--- a/tests/misc-tests/046-seed-multi-mount/test.sh
+++ b/tests/misc-tests/046-seed-multi-mount/test.sh
@@ -42,9 +42,11 @@ nextdevice() {
 	run_check mkdir -p "$mnt"
 	TEST_MNT="$mnt"
 	run_check_mount_test_dev
-	run_check $SUDO_HELPER "$TOP/btrfs" device add ${loopdevs[$nextdev]} "$TEST_MNT"
 	run_mustfail "writable file despite read-only mount" \
 		$SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/file$nextdevice" bs=1M count=1 status=none
+	run_check $SUDO_HELPER "$TOP/btrfs" device add ${loopdevs[$nextdev]} "$TEST_MNT"
+	# Although seed sprout would make the fs RW, explicitly remount it RW
+	# just in case of future behavior change.
 	run_check $SUDO_HELPER mount -o remount,rw "$TEST_MNT"
 	# Rewrite the file
 	md5sum=$(run_check_stdout md5sum "$TEST_MNT/file$nextdev" | awk '{print $1}')
-- 
2.41.0

[PATCH 4/4] btrfs-progs: tests/misc/058: reduce the space requirement and speed up the test

[Qu Wenruo]

[BUG]
When I was testing misc/058, the fs still has around 7GiB free space,
but during that test case, btrfs kernel module reports write failures
and even git commands failed inside that fs.

And obviously the test case failed.

[CAUSE]
It turns out that, the test case itself would require 6GiB (4 data
disks) + 1.5GiB x 2 (the two replace target), thus it requires 9 GiB
free space.

And obviously my partition is not that large and failed.

[FIX]
In fact, we really don't need that much space at all.

Our objective is to test "btrfs device replace --enqueue" functionality,
there is not much need to wait for 1 second, we can just do the enqueue
immediately.

So this patch would reduce the file size to a more sane (and rounded)
2GiB, and do the enqueue immediately.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/misc-tests/058-replace-start-enqueue/test.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tests/misc-tests/058-replace-start-enqueue/test.sh b/tests/misc-tests/058-replace-start-enqueue/test.sh
index 1a24d5ec7ecb..bdbc87b4090d 100755
--- a/tests/misc-tests/058-replace-start-enqueue/test.sh
+++ b/tests/misc-tests/058-replace-start-enqueue/test.sh
@@ -21,16 +21,15 @@ run_check_mount_test_dev
 run_check $SUDO_HELPER "$TOP/btrfs" device remove "$REPLACE1" "$TEST_MNT"
 run_check $SUDO_HELPER "$TOP/btrfs" device remove "$REPLACE2" "$TEST_MNT"
 
-for i in `seq 48`; do
+for i in `seq 16`; do
 	run_check $SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/file$i" bs=1M count=128 status=noxfer
 done
 # Sync so replace start does not block in unwritten IO
 run_check "$TOP/btrfs" filesystem sync "$TEST_MNT"
 run_check "$TOP/btrfs" filesystem usage -T "$TEST_MNT"
 
-# Go background, should not be that fast, estimated 10 seconds
+# Go background, should not be that fast.
 run_check $SUDO_HELPER "$TOP/btrfs" replace start 2 "$REPLACE1" "$TEST_MNT"
-run_check sleep 1
 # No background, should wait
 run_check $SUDO_HELPER "$TOP/btrfs" replace start --enqueue 3 "$REPLACE2" "$TEST_MNT"
 run_check $SUDO_HELPER "$TOP/btrfs" replace status "$TEST_MNT"
-- 
2.41.0

Re: [PATCH 0/4] btrfs-progs: tests false alerts fixes

[David Sterba]

On Wed, Aug 09, 2023 at 06:40:40AM +0800, Qu Wenruo wrote:
> This is the resent and aggregate version of the fixes I sent but mostly
> eaten by the recent vger down time.
> 
> Those are small fixes for the false alerts I hit during my local runs.
> 
> Most of them are subtle fixes, only the last one is more like an
> optimization.
> 
> Qu Wenruo (4):
>   btrfs-progs: tests/mkfs/005: use udevadm settle to avoid false alerts
>   btrfs-progs: tests/misc/030: do not require v1 cache for the test case
>   btrfs-progs: tests/misc/046: fix false alerts on write detection

This one was not in the first batch so I've added it now, thanks.

>   btrfs-progs: tests/misc/058: reduce the space requirement and speed up
>     the test