Re: [RFC: kdevops] Standardizing on failure rate nomenclature for expunges

["Theodore Ts'o" <tytso@mit.edu>]

On Sun, Jul 03, 2022 at 05:22:17PM +0300, Amir Goldstein wrote:
> 
> To be clear, when I wrote deterministic, what I meant was deterministic
> results empirically, in the same sense that Bart meant - a test should
> always pass.

Well all of the tests in the auto group pass at 100% of the time for
the ext4/4k and xfs/4k groups.  (Well, at least if use the HDD and SSD
as the storage device.  If you are using eMMC flash, or Luis's loop
device config, there would be more failures.)

But if we're talking about btrfs/4k, f2fs/4k, xfs/realtime,
xfs/realtime_28k/logdev, ext4/bigalloc, etc. there would be a *lot* of
tests that would need to be removed from the auto group.

So what "non-determinsitic tests" should we remove from the auto
group?  For what file systems, file system configs, and storage
devices?  What would you propose?

Remember, Matthew wants something that he can use to test "dozens" of
file systems that he's touching for the folio patches.  If we have to
remove all of the tests that fail if you are using nfs, vfat, hfs,
msdos, etc., then the auto group would be pretty anemic.  Let's not do
that.

If you want a "always pass" group, we could do that, but let's not
call that the "auto" group, please.

						- Ted