public inbox for fstests@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: fstests@vger.kernel.org
Subject: [PATCH 14/40] fstests: xfs/227 is really slow
Date: Wed, 27 Nov 2024 15:51:44 +1100	[thread overview]
Message-ID: <20241127045403.3665299-15-david@fromorbit.com> (raw)
In-Reply-To: <20241127045403.3665299-1-david@fromorbit.com>

From: Dave Chinner <dchinner@redhat.com>

The slowest test tto run on my test VMs is xfs/227:

...
xfs/227 826
...

It is doing nested iteration on created filesets that are explicitly
defined, so separate the inner loop filesets and run the outer loops
in parallel.

Also reduce the number of times we have to execute setfattr and
xfs_io to once per created file instead of once per xattr/extent
count per file.

The result is test runtime reduction of ~60%.

....
xfs/227 336
....

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 tests/xfs/227 | 57 ++++++++++++++++++++++++++++++++-------------------
 1 file changed, 36 insertions(+), 21 deletions(-)

diff --git a/tests/xfs/227 b/tests/xfs/227
index 2ad910cbd..7f8ee2dba 100755
--- a/tests/xfs/227
+++ b/tests/xfs/227
@@ -41,7 +41,7 @@ fragment_freespace()
 	# allocate inode space
 	mkdir -p $_dir
 	for i in `seq 0 1 1000`; do
-		touch $_file.$i
+		echo -n > $_file.$i
 	done
 	for i in `seq 0 63 1000`; do
 		mv $_file.$i $_dir
@@ -68,19 +68,23 @@ fragment_freespace()
 
 create_attrs()
 {
-	for foo in `seq 0 1 $1`; do
-		$SETFATTR_PROG -n user.$foo -v 0xbabe $2
-	done
+	( echo "# file: $2"
+	for i in `seq 0 1 $1`; do
+		echo "user.$foo=\"0xbabe\""
+	done ) > $tmp.$1.attrs
+	$SETFATTR_PROG --restore=$tmp.$1.attrs $2
 }
 
 create_data()
 {
-	size=`expr \( $1 + 1 \) \* 4096`
-	$XFS_IO_PROG -f -c "truncate $size" $2 > /dev/null 2>&1
-	for foo in `seq $1 -1 0`; do
-		let offset=$foo*4096
-		$XFS_IO_PROG -f -c "resvsp $offset 4096" $2 > /dev/null 2>&1
+	local blocks=$1
+	local cmd_str
+
+	for off in `seq $blocks -1 0`; do
+		cmd_str="-c \"resvsp $((off * 4096)) 4096\" $cmd_str"
 	done
+	$XFS_IO_PROG -f -c "truncate $((($blocks + 1) * 4096))" \
+			$cmd_str $2 >> seqres.full 2>&1
 }
 
 # create the designated file with a certain number of attributes and a certain
@@ -119,6 +123,27 @@ create_target_attr_last()
 	create_attrs $nattrs $target
 }
 
+do_fsr()
+{
+	local n=$1
+	local i=$2
+	local j
+
+	for j in `seq 5 1 20`; do
+		create_target_attr_first $i $j $targ.$i.$j >> $seqres.full 2>&1
+	done
+	xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
+	FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1
+	xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
+	for j in `seq 5 1 20`; do
+		create_target_attr_last $i $j $targ.$i.$j >> $seqres.full 2>&1
+	done
+	xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
+	FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1
+	xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
+}
+
+
 # use a small filesystem so we can control freespace easily
 _scratch_mkfs_sized $((50 * 1024 * 1024)) >> $seqres.full 2>&1
 _scratch_mount
@@ -154,19 +179,9 @@ targ=$SCRATCH_MNT/fsr_test_file.$$
 for n in `seq 4 1 12`; do
 	echo "*** n == $n ***" >> $seqres.full
 	for i in `seq 5 1 15`; do
-		for j in `seq 5 1 20`; do
-			create_target_attr_first $i $j $targ.$i.$j >> $seqres.full 2>&1
-		done
-		xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
-		FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1
-		xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
-		for j in `seq 5 1 20`; do
-			create_target_attr_last $i $j $targ.$i.$j >> $seqres.full 2>&1
-		done
-		xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
-		FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1
-		xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1
+		do_fsr $n $i &
 	done
+	wait
 done
 
 _scratch_unmount
-- 
2.45.2


  parent reply	other threads:[~2024-11-27  4:54 UTC|newest]

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-27  4:51 [RFC PATCH 00/40] fstests: concurrent test execution Dave Chinner
2024-11-27  4:51 ` [PATCH 01/40] xfs/448: get rid of assert-on-failure Dave Chinner
2024-11-27  4:51 ` [PATCH 02/40] fstests: cleanup fsstress process management Dave Chinner
2024-11-29  4:03   ` Zorro Lang
2024-12-04 17:57   ` Zorro Lang
2024-12-05  4:42     ` Dave Chinner
2024-12-05  9:57       ` Zorro Lang
2024-12-04 18:04   ` Zorro Lang
2024-12-05  4:55     ` Dave Chinner
2024-12-05 10:05       ` Zorro Lang
2024-11-27  4:51 ` [PATCH 03/40] fuzzy: don't use killall Dave Chinner
2024-11-27  4:51 ` [PATCH 04/40] fstests: per-test dmflakey instances Dave Chinner
2024-11-27  4:51 ` [PATCH 05/40] fstests: per-test dmerror instances Dave Chinner
2024-11-27  4:51 ` [PATCH 06/40] fstests: per-test dmhuge instances Dave Chinner
2024-11-27  4:51 ` [PATCH 07/40] fstests: per-test dmthin instances Dave Chinner
2024-11-27  4:51 ` [PATCH 08/40] fstests: per-test dmdust instances Dave Chinner
2024-11-27  4:51 ` [PATCH 09/40] fstests: per-test dmdelay instances Dave Chinner
2024-11-27  4:51 ` [PATCH 10/40] fstests: fix DM device creation/removal vs udev races Dave Chinner
2024-11-27  4:51 ` [PATCH 11/40] fstests: use syncfs rather than sync Dave Chinner
2024-11-27  4:51 ` [PATCH 12/40] fstests: clean up mount and unmount operations Dave Chinner
2024-11-27  4:51 ` [PATCH 13/40] fstests: clean up loop device instantiation Dave Chinner
2024-12-01 12:31   ` Zorro Lang
2024-12-01 12:50   ` Zorro Lang
2024-12-07 12:44   ` Zorro Lang
2024-12-07 18:59     ` Zorro Lang
2024-12-07 19:51       ` Zorro Lang
2024-11-27  4:51 ` Dave Chinner [this message]
2024-11-27  4:51 ` [PATCH 15/40] fstests: mark tests that are unreliable when run in parallel Dave Chinner
2024-11-27  4:51 ` [PATCH 16/40] fstests: use udevadm wait in preference to settle Dave Chinner
2024-11-29 17:10   ` Darrick J. Wong
2024-11-29 22:33     ` Dave Chinner
2024-11-30  2:34       ` Zorro Lang
2024-11-27  4:51 ` [PATCH 17/40] xfs/442: rescale load so it's not exponential Dave Chinner
2024-11-27  4:51 ` [PATCH 18/40] xfs/176: fix broken setup code Dave Chinner
2024-11-27  4:51 ` [PATCH 19/40] xfs/177: remove unused slab object count location checks Dave Chinner
2024-11-27  4:51 ` [PATCH 20/40] fstests: remove uses of killall where possible Dave Chinner
2024-11-27  4:51 ` [PATCH 21/40] generic/127: reduce runtime Dave Chinner
2024-11-27  4:51 ` [PATCH 22/40] quota: system project quota files need to be shared Dave Chinner
2024-11-27  4:51 ` [PATCH 23/40] dmesg: reduce noise from other tests Dave Chinner
2024-11-27  4:51 ` [PATCH 24/40] fstests: stop using /tmp directly Dave Chinner
2024-11-27  4:51 ` [PATCH 25/40] fstests: scale some tests for high CPU count sanity Dave Chinner
2024-11-29  3:34   ` Zorro Lang
2024-11-27  4:51 ` [PATCH 26/40] generic/310: cleanup killing background processes Dave Chinner
2024-11-27  4:51 ` [PATCH 27/40] filter: handle mount errors from CONFIG_BLK_DEV_WRITE_MOUNTED=y Dave Chinner
2024-11-27  4:51 ` [PATCH 28/40] filters: add a filter that accepts EIO instead of other errors Dave Chinner
2024-11-27  4:51 ` [PATCH 29/40] generic/085: general cleanup for reliability and debugging Dave Chinner
2024-11-27  4:52 ` [PATCH 30/40] fstests: don't use directory stacks Dave Chinner
2024-12-01 12:10   ` Zorro Lang
2024-12-01 21:37     ` Dave Chinner
2024-11-27  4:52 ` [PATCH 31/40] fstests: clean up a couple of dm-flakey tests Dave Chinner
2024-11-27  4:52 ` [PATCH 32/40] fstests: clean up termination of various tests Dave Chinner
2024-11-27  4:52 ` [PATCH 33/40] vfstests: some tests require the testdir to be shared Dave Chinner
2024-11-27  4:52 ` [PATCH 34/40] xfs/629: single extent files should be within tolerance Dave Chinner
2024-11-27  4:52 ` [PATCH 35/40] xfs/076: fix broken mkfs filtering Dave Chinner
2024-11-27  4:52 ` [PATCH 36/40] fstests: capture some failures to seqres.full Dave Chinner
2024-11-27  4:52 ` [PATCH 37/40] fstests: always use fail-at-unmount semantics for XFS Dave Chinner
2024-11-27  4:52 ` [PATCH 38/40] generic/062: don't leave debug files in $here on failure Dave Chinner
2024-11-27  4:52 ` [PATCH 39/40] fstests: quota grace periods unreliable under load Dave Chinner
2024-11-27  4:52 ` [PATCH 40/40] fstests: check-parallel Dave Chinner
2024-11-29  4:22 ` [RFC PATCH 00/40] fstests: concurrent test execution Zorro Lang
2024-12-07  0:09   ` Darrick J. Wong
2024-12-07  9:38     ` Zorro Lang
2024-12-08  0:02     ` Dave Chinner
2024-12-08  6:15       ` Zorro Lang
2024-12-10  0:55         ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241127045403.3665299-15-david@fromorbit.com \
    --to=david@fromorbit.com \
    --cc=fstests@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox