Re: [PATCH 09/12] generic/251: constrain runtime via time/load/soak factors

[Dave Chinner <david@fromorbit.com>]

On Mon, Nov 18, 2024 at 03:03:43PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> On my test fleet, this test can run for well in excess of 20 minutes:
> 
>    613 generic/251
>    616 generic/251
>    624 generic/251
>    630 generic/251
>    634 generic/251
>    652 generic/251
>    675 generic/251
>    749 generic/251
>    777 generic/251
>    808 generic/251
>    832 generic/251
>    946 generic/251
>   1082 generic/251
>   1221 generic/251
>   1241 generic/251
>   1254 generic/251
>   1305 generic/251
>   1366 generic/251
>   1646 generic/251
>   1936 generic/251
>   1952 generic/251
>   2358 generic/251
>   4359 generic/251
>   5325 generic/251
>  34046 generic/251
> 
> because it hardcodes 20 threads and 10 copies.  It's not great to have a
> test that results in a significant fraction of the total test runtime.
> Fix the looping and load on this test to use LOAD and TIME_FACTOR to
> scale up its operations, along with the usual SOAK_DURATION override.
> That brings the default runtime down to less than a minute.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>

Question for you: Does your $here directory contain a .git subdir?

One of the causes of long runtime for me has been that $here might
only contain 30MB of files, but the .git subdir balloons to several
hundred MB over time, resulting is really long runtimes because it's
copying GBs of data from the .git subdir.

I have this patch in my tree:

--- a/tests/generic/251
+++ b/tests/generic/251
@@ -175,9 +175,12 @@ nproc=20
 # Copy $here to the scratch fs and make coipes of the replica.  The fstests
 # output (and hence $seqres.full) could be in $here, so we need to snapshot
 # $here before computing file checksums.
+#
+# $here/* as the files to copy so we avoid any .git directory that might be
+# much, much larger than the rest of the fstests source tree we are copying.
 content=$SCRATCH_MNT/orig
 mkdir -p $content
-cp -axT $here/ $content/
+cp -ax $here/* $content/

 mkdir -p $tmp

And that's made the runtime drop from (typically) 10-15 minutes
down to around 5 minutes....

Does this have any impact on the runtime on your test systems?

-Dave.

-- 
Dave Chinner
david@fromorbit.com