FS/XFS testing framework
 help / color / mirror / Atom feed
From: Pankaj Raghav <p.raghav@samsung.com>
To: fstests@vger.kernel.org
Cc: Johannes.Thumshirn@wdc.com, damien.lemoal@opensource.wdc.com,
	pankydev8@gmail.com, naohiro.aota@wdc.com, gost.dev@samsung.com,
	mcgrof@kernel.org, dsterba@suse.cz,
	Pankaj Raghav <p.raghav@samsung.com>
Subject: [RFC 1/1] btrfs/237: adapt the test to work with the new reclaim algorithm
Date: Fri, 19 Aug 2022 13:53:37 +0200	[thread overview]
Message-ID: <20220819115337.35681-2-p.raghav@samsung.com> (raw)
In-Reply-To: <20220819115337.35681-1-p.raghav@samsung.com>

Since 3687fcb0752a ("btrfs: zoned: make auto-reclaim less aggressive")
commit, reclaim algorithm has been changed to trigger auto-reclaim once
the fs used size is more than certain threshold. This change breaks this
test.

The test has been adapted so that the new auto-reclaim algorithm can be
tested along with relocation.
---
 tests/btrfs/237 | 80 +++++++++++++++++++++++++++++++++++--------------
 1 file changed, 57 insertions(+), 23 deletions(-)

diff --git a/tests/btrfs/237 b/tests/btrfs/237
index f96031d5..18945e78 100755
--- a/tests/btrfs/237
+++ b/tests/btrfs/237
@@ -54,46 +54,80 @@ if [[ "$uuid" == "" ]]; then
 	exit 1
 fi
 
+fssize=$($BTRFS_UTIL_PROG fi usage -b $SCRATCH_MNT |grep "Device size" |\
+	grep -Eo "[0-9]+")
+
+allocated_fssize=$($BTRFS_UTIL_PROG fi usage -b $SCRATCH_MNT |grep "Device allocated" |\
+	grep -Eo "[0-9]+")
+
+
 start_data_bg_phy=$(get_data_bg_physical)
 start_data_bg_phy=$((start_data_bg_phy >> 9))
 
-size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\
+zone_cap=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\
 	_filter_blkzone_report |\
 	grep -Po "cap 0x[[:xdigit:]]+" | cut -d ' ' -f 2)
-size=$((size << 9))
+zone_cap=$((zone_cap << 9))
 
-reclaim_threshold=75
-echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold
-fill_percent=$((reclaim_threshold + 2))
-rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG
-fill_size=$((size * fill_percent / 100))
-rest=$((size * rest_percent / 100))
+fs_reclaim_threshold=60
+bg_reclaim_threshold=75
+echo $fs_reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold
+echo $bg_reclaim_threshold > /sys/fs/btrfs/"$uuid"/allocation/data/bg_reclaim_threshold
 
-# step 1, fill FS over $fillsize
-$XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full
-$XFS_IO_PROG -fc "pwrite 0 $rest" $SCRATCH_MNT/$seq.test2 >> $seqres.full
+fs_fill_percent=$((fs_reclaim_threshold + 2))
+fill_size=$((fssize * fs_fill_percent / 100))
+
+# Remove the allocated size from the $fill_size
+fill_size=$((fill_size - allocated_fssize))
+
+bg_fill_percent=$((bg_reclaim_threshold + 2))
+zone_fill_size=$((zone_cap * bg_fill_percent / 100))
+
+# $fill_size might not cover the last zone block group with threshold
+# for reclaim. Add the remaining bytes so that it can also be reclaimed
+last_zone_offset=$((fill_size % zone_cap))
+
+if [ $last_zone_offset -lt $zone_fill_size ]; then
+	fill_size=$((fill_size + zone_fill_size - last_zone_offset))
+fi
+
+# This small file will be used to verify the relocation
+relocate_file_size=$((zone_cap * 2 / 100))
+
+# step 1, fill FS over $relocated_file_size and $fill_size
+$XFS_IO_PROG -fc "pwrite 0 $relocate_file_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full
 $BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT
 
-zones_before=$($BLKZONE_PROG report $SCRATCH_DEV | grep -v -e em -e nw | wc -l)
-echo "Before reclaim: $zones_before zones open" >> $seqres.full
 old_data_zone=$(get_data_bg)
 old_data_zone=$((old_data_zone >> 9))
 printf "Old data zone 0x%x\n" $old_data_zone >> $seqres.full
 
-# step 2, delete the 1st $fill_size sized file to trigger reclaim
-rm $SCRATCH_MNT/$seq.test1
+$XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test2 >> $seqres.full
 $BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT
-sleep 2 # 1 transaction commit for 'rm' and 1 for balance
+
+open_zones_before_reclaim=$($BLKZONE_PROG report --offset $start_data_bg_phy $SCRATCH_DEV |\
+	grep -v -e em -e nw | wc -l)
+
+# sanity check
+if [ $open_zones_before_reclaim -eq 0 ]; then
+	echo "Error writing to the device"
+fi
+
+echo "Before reclaim: $open_zones_before_reclaim zones open" >> $seqres.full
+
+# step 2, delete the $fill_size sized file to trigger reclaim
+rm $SCRATCH_MNT/$seq.test2
+$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT
+sleep 5 # sleep for transaction commit for 'rm' and for balance
 
 # check that we don't have more zones open than before
-zones_after=$($BLKZONE_PROG report $SCRATCH_DEV | grep -v -e em -e nw | wc -l)
-echo "After reclaim: $zones_after zones open" >> $seqres.full
+open_zones_after_reclaim=$($BLKZONE_PROG report --offset $start_data_bg_phy $SCRATCH_DEV |\
+	grep -v -e em -e nw | wc -l)
+echo "After reclaim: $open_zones_after_reclaim zones open" >> $seqres.full
 
-# Check that old data zone was reset
-old_wptr=$($BLKZONE_PROG report -o $old_data_zone -c 1 $SCRATCH_DEV |\
-	grep -Eo "wptr 0x[[:xdigit:]]+" | cut -d ' ' -f 2)
-if [ "$old_wptr" != "0x000000" ]; then
-	_fail "Old wptr still at $old_wptr"
+# Check that data was really relocated to a different zone
+if [ $open_zones_after_reclaim != 1 ]; then
+	echo "Error relocating the data"
 fi
 
 new_data_zone=$(get_data_bg)
-- 
2.25.1


  reply	other threads:[~2022-08-19 11:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220819115338eucas1p11b916296213572e97a03241ebdc399d0@eucas1p1.samsung.com>
2022-08-19 11:53 ` [RFC 0/1] adapting btrfs/237 to work with the new reclaim algorithm Pankaj Raghav
2022-08-19 11:53   ` Pankaj Raghav [this message]
2022-08-22  9:40     ` [RFC 1/1] btrfs/237: adapt the test " Johannes Thumshirn
2022-08-22 10:49       ` Pankaj Raghav
2022-08-22 12:22         ` Johannes Thumshirn
2022-08-22 14:29   ` [RFC 0/1] adapting btrfs/237 " Johannes Thumshirn
2022-08-23 11:46     ` Pankaj Raghav
2022-12-05 14:53       ` Pankaj Raghav
2022-12-05 16:04         ` Johannes Thumshirn
2022-12-07 16:01           ` Pankaj Raghav
2022-12-13 13:35           ` Pankaj Raghav
2022-12-05  7:56   ` Johannes Thumshirn

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220819115337.35681-2-p.raghav@samsung.com \
    --to=p.raghav@samsung.com \
    --cc=Johannes.Thumshirn@wdc.com \
    --cc=damien.lemoal@opensource.wdc.com \
    --cc=dsterba@suse.cz \
    --cc=fstests@vger.kernel.org \
    --cc=gost.dev@samsung.com \
    --cc=mcgrof@kernel.org \
    --cc=naohiro.aota@wdc.com \
    --cc=pankydev8@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox