Re: [LSF/MM TOPIC] FS, MM, and stable trees

[Luis Chamberlain <mcgrof@kernel.org>]

On Tue, Mar 08, 2022 at 11:40:18AM -0500, Theodore Ts'o wrote:
> One of my team members has been working with Darrick to set up a set
> of xfs configs[1] recommended by Darrick, and she's stood up an
> automated test spinner using gce-xfstests which can watch a git branch
> and automatically kick off a set of tests whenever it is updated.

I think its important to note, as we would all know, that contrary to
most other subsystems, in so far as blktests and fstests is concerned,
simply passing a test once does not mean there is no issue given that
some test can fail with a failure rate of 1/1,000 for instance.

How many times you want to run a full set of fstests against a
filesystem varies depending on your filesystem, requirements and also
what resources you have. It also varies depending on how long you want
to dedicate time towards this.

To help with these concepts I ended up calling this a kernel-ci steady state
goal on kdevops:

  │ CONFIG_KERNEL_CI_STEADY_STATE_GOAL:
  │  
  │ The maximum number of possitive successes to have before bailing out
  │ a kernel-ci loop and report success. This value is currently used for
  │ all workflows. A value of 100 means 100 tests will run before we
  │ bail out and report we have achieved steady state for the workflow
  │ being tested. 

For fstests for XFS and btrfs, when testing for enterprise, I ended up going
with a steady state test goal of 500. That is, 500 consecutive runs of fstests
without any failure. This takes about 1 full week to run and one of my
eventual goals is to reduce this time. Perhaps it makes more sense to
talk generally how to optimize these sorts of tests, or share
information on experiences like these.

Do we want to define a steady state goal for stable for XFS?

  Luis