From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69409C43334 for ; Sun, 3 Jul 2022 16:30:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232446AbiGCQay (ORCPT ); Sun, 3 Jul 2022 12:30:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229574AbiGCQax (ORCPT ); Sun, 3 Jul 2022 12:30:53 -0400 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13DC35FB1; Sun, 3 Jul 2022 09:30:51 -0700 (PDT) Received: from cwcc.thunk.org (pool-173-48-118-63.bstnma.fios.verizon.net [173.48.118.63]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 263GUG1U006323 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 3 Jul 2022 12:30:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1656865823; bh=uKC0mfydFMX4nUGBHuT9jkHWP7ClJOQjzQ31vgmMI5E=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=ovcz2irxhBKksjbFBj4coBed2FkyJqDL5tkts1iix6B+4/yL/g8C9U5LwjN631HSa aMA2tibyXYr5MEVfNxBWpEPRH9wW7Xi0Ae4JasXNq3OhGwzBJv0JkgN/9tQ4Js0fvn zrWezqpIT0c9tr5wEO6XeolPAYbo0ybOMplXHi/VZX9vbTJyhLGy6X8Tf1nyJvgXSY InsEIKp9piRbk8bkV+fXdSDsj6GpzSPfscaaGBK+R8XmLroIzmtYcshgXZ+svbfnTq pzrRuAzvSmJCrzmz/iVNJydcJhkv8BxhyytpZxmz9Jb0YZ3vuezcLpdzaG4EeLsXOr rbmJVJnL0K74A== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 62C5215C3E94; Sun, 3 Jul 2022 12:30:16 -0400 (EDT) Date: Sun, 3 Jul 2022 12:30:16 -0400 From: "Theodore Ts'o" To: Amir Goldstein Cc: Bart Van Assche , "Darrick J. Wong" , Luis Chamberlain , linux-fsdevel , linux-block , Pankaj Raghav , Josef Bacik , jmeneghi@redhat.com, Jan Kara , Davidlohr Bueso , Dan Williams , Jake Edge , Klaus Jensen , fstests , Zorro Lang , Matthew Wilcox , Dave Chinner Subject: Re: [RFC: kdevops] Standardizing on failure rate nomenclature for expunges Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Sun, Jul 03, 2022 at 05:22:17PM +0300, Amir Goldstein wrote: > > To be clear, when I wrote deterministic, what I meant was deterministic > results empirically, in the same sense that Bart meant - a test should > always pass. Well all of the tests in the auto group pass at 100% of the time for the ext4/4k and xfs/4k groups. (Well, at least if use the HDD and SSD as the storage device. If you are using eMMC flash, or Luis's loop device config, there would be more failures.) But if we're talking about btrfs/4k, f2fs/4k, xfs/realtime, xfs/realtime_28k/logdev, ext4/bigalloc, etc. there would be a *lot* of tests that would need to be removed from the auto group. So what "non-determinsitic tests" should we remove from the auto group? For what file systems, file system configs, and storage devices? What would you propose? Remember, Matthew wants something that he can use to test "dozens" of file systems that he's touching for the folio patches. If we have to remove all of the tests that fail if you are using nfs, vfat, hfs, msdos, etc., then the auto group would be pretty anemic. Let's not do that. If you want a "always pass" group, we could do that, but let's not call that the "auto" group, please. - Ted