Re: [PATCH 2/2] generic/042: add to auto group

[Dave Chinner <david@fromorbit.com>]

On Tue, Jun 21, 2016 at 04:12:42PM +0800, Eryu Guan wrote:
> On Tue, Jun 14, 2016 at 06:51:25PM +0200, Christoph Hellwig wrote:
> > On Tue, Jun 14, 2016 at 06:44:40PM +0300, Omer Zilberberg wrote:
> > > 
> > > 
> > > On 06/14/2016 06:08 PM, Christoph Hellwig wrote:
> > > > This is an odd one as it's in the quick group but not the auto one.
> > > > And it actually fails on XFS for me..
> > > Please note that this was removed from 'auto' in commit:
> > > 7721b85 generic/042: remove from the 'auto' group
> > > I don't know whether the original reasons for this are still relevant...
> > > just FYI
> > 
> > I tend to disagree with the rationale, but even if we believe it it
> > should have been removed from the quick group as well.
> 
> So the question is how do we define auto group, 7721b85 removed auto
> group because "the auto group is for tests that *are* expected to
> succeed".
> 
> I searched for Dave's explainations on 'auto' group in his reviews, and
> got the following definitions:
> 
> - it should be a valid & reliable test (it's finished and have
>   deterministic output) [1]
> - it passes on current upstream kernels, if it fails, it's likely to be
>   resolved in forseeable future [2]
> - it should take no longer than 5 minutes to finish [3]
> 
> generic/042 is a valid & reliable regression test, and it finishes
> within 5 seconds for me, but it fails on current upstream kernels, and I
> don't see when it could be fixed (the test was added in Oct. 2014, it's
> not fixed in almost two years).
> 
> So removing it from 'auto' group seems fine to me, but I'm not sure
> about the 'quick' group. Is it related to auto?

Yes. The only difference between quick and auto group criteria is
the test runtime.

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com