Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

["Darrick J. Wong" <djwong@kernel.org>]

On Wed, Jan 22, 2025 at 09:15:48AM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 08:00:27AM -0500, Theodore Ts'o wrote:
> > On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> > > I probably misunderstood how -n nr_ops vs --duration=30 interact;
> > > I expected it to run until either were exhausted, not for duration
> > > to override nr_ops as implied by this:
> > 
> > There are (at least) two ways that a soak duration is being used
> > today; one is where someone wants to run a very long soak for hours
> > and where if you go long by an hour or two it's no big deals.  The
> > other is where you are specifying a soak duration as part of a smoke
> > test (using the smoketest group), where you might be hoping to keep
> > the overall run time to 15-20 minutes and so you set SOAK_DURATION to
> > 3m.
> 
> check-parallel on my 64p machine runs the full auto group test in
> under 10 minutes.
> 
> i.e. if you have a typical modern server (64-128p, 256GB RAM and a
> couple of NVMe SSDs), then check-parallel allows a full test run in
> the same time that './check -g smoketest' will run....
> 
> > (This was based on some research that Darrick did which showed that
> > running the original 5 tests in the smoketest group gave you most of
> > the code coverage of running all of the quick group, which had
> > ballooned from 15 minutes many years ago to an hour or more.  I just
> > noticed that we've since added two more tests to the smoketest group;
> > it might be worth checking whether those two new tests addded to thhe
> > smoketest groups significantly improves code coverage or not.  It
> > would be unfortunate if the runtime bloat that happened to the quick
> > group also happens to the smoketest group...)
> 
> Yes, and I've previously made the point about how check-parallel
> changes the way we should be looking at dev-test cycles. We no
> longer have to care that auto group testing takes 4 hours to run and
> have to work around that with things like smoketest groups. If you
> can run the whole auto test group in 10-15 minutes, then we don't
> need "quick", "smoketest", etc to reduce dev-test cycle time
> anymore...
> 
> > The bottom line is in addition to trying to design semantics for users
> > who might be at either end of the CPU count spectrum, we should also
> > consider that SOAK_DURATION could be set for values ranging from
> > minutes to hours.
> 
> I don't see much point in testing for hours with check-parallel. The
> whole point of it is to enable iteration across the entire fs test
> matrix as fast as possible.

I do -- running all the soak tests in parallel on a (probably old lower
spec) machine.  Parallelism is all right for a lot of things.

--D

> If you want to do long running soak tests, then keep using check for
> that. If you want to run the auto group test across 100 different
> mkfs option combinations, then that is where check-parallel comes in
> - it'll take a few hours to do this instead of a week.
> 
> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
>