Re: xfstests results over NFS

[Jeff Layton <jlayton@kernel.org>]

On Thu, 2023-08-17 at 14:04 +0000, Chuck Lever III wrote:
> 
> > On Aug 17, 2023, at 7:21 AM, Jeff Layton <jlayton@kernel.org> wrote:
> > 
> > I finally got my kdevops (https://github.com/linux-kdevops/kdevops) test
> > rig working well enough to get some publishable results. To run fstests,
> > kdevops will spin up a server and (in this case) 2 clients to run
> > xfstests' auto group. One client mounts with default options, and the
> > other uses NFSv3.
> > 
> > I tested 3 kernels:
> > 
> > v6.4.0 (stock release)
> > 6.5.0-rc6-g4853c74bd7ab (Linus' tree as of a couple of days ago)
> > 6.5.0-rc6-next-20230816-gef66bf8aeb91 (linux-next as of yesterday morning)
> > 
> > Here are the results summary of all 3:
> > 
> > KERNEL:    6.4.0
> > CPUS:      8
> > 
> > nfs_v3: 727 tests, 12 failures, 569 skipped, 14863 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/124 
> >    generic/193 generic/258 generic/294 generic/318 generic/319 
> >    generic/444 generic/528 generic/529 
> > nfs_default: 727 tests, 18 failures, 452 skipped, 21899 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/186 
> >    generic/187 generic/193 generic/294 generic/318 generic/319 
> >    generic/357 generic/444 generic/486 generic/513 generic/528 
> >    generic/529 generic/578 generic/675 generic/688 
> > Totals: 1454 tests, 1021 skipped, 30 failures, 0 errors, 35096s
> > 
> > KERNEL:    6.5.0-rc6-g4853c74bd7ab
> > CPUS:      8
> > 
> > nfs_v3: 727 tests, 9 failures, 570 skipped, 14775 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/258 
> >    generic/294 generic/318 generic/319 generic/444 generic/529 
> > nfs_default: 727 tests, 16 failures, 453 skipped, 22326 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/186 
> >    generic/187 generic/294 generic/318 generic/319 generic/357 
> >    generic/444 generic/486 generic/513 generic/529 generic/578 
> >    generic/675 generic/688 
> > Totals: 1454 tests, 1023 skipped, 25 failures, 0 errors, 35396s
> > 
> > KERNEL:    6.5.0-rc6-next-20230816-gef66bf8aeb91
> > CPUS:      8
> > 
> > nfs_v3: 727 tests, 9 failures, 570 skipped, 14657 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/258 
> >    generic/294 generic/318 generic/319 generic/444 generic/529 
> > nfs_default: 727 tests, 18 failures, 453 skipped, 21757 seconds
> >  Failures: generic/053 generic/099 generic/105 generic/186 
> >    generic/187 generic/294 generic/318 generic/319 generic/357 
> >    generic/444 generic/486 generic/513 generic/529 generic/578 
> >    generic/675 generic/683 generic/684 generic/688 
> > Totals: 1454 tests, 1023 skipped, 27 failures, 0 errors, 34870s
> > 
> > With NFSv4.2, v6.4.0 has 2 extra failures that the current mainline
> > kernel doesn't:
> > 
> >    generic/193 (some sort of setattr problem)
> >    generic/528 (known problem with btime handling in client that has been fixed)
> > 
> > While I haven't investigated, I'm assuming the 193 bug is also something
> > that has been fixed in recent kernels. There are also 3 other NFSv3
> > tests that started passing since v6.4.0. I haven't looked into those.
> > 
> > With the linux-next kernel there are 2 new regressions:
> > 
> >    generic/683
> >    generic/684
> > 
> > Both of these look like problems with setuid/setgid stripping, and still
> > need to be investigated. I have more verbose result info on the test
> > failures if anyone is interested.
> 
> 100% awesome sauce. Out of curiosity:
> 
> Does kdevops have a way of publishing (via an autonomous web site)
> and archiving these results?
> 

There's nothing much prewritten for this. There is some support for
sending emails when you run a "ci" loop. I need to do more investigation
here.

Note that there has been some parallel effort toward CI in the SMB space
using buildbot. It may worthwhile to consider combining efforts somehow.

> Does the "auto" group include tests that require a SCRATCH_DEV?
> 

Yes. The nfs server is configured with 2 exported fs', so I have it
mounting a directory under one as "test" and the other as "scratch".
-- 
Jeff Layton <jlayton@kernel.org>