Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

["Darrick J. Wong" <djwong@kernel.org>]

On Wed, Sep 01, 2021 at 07:46:01AM +0300, Amir Goldstein wrote:
> On Wed, Sep 1, 2021 at 3:37 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > Create a file to document the purpose of each test group that is
> > currently defined in fstests, and add a build script to check that every
> > group mentioned in the tests is also mentioned in the documentation.
> >
> 
> This is awesome and long due.
> Thanks for doing that!
> 
> Minor nits about overlayfs groups below...

Heh, yeah, thanks for making corrections. :)

> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  include/buildgrouplist |    1
> >  tools/check-groups     |   33 ++++++++++++
> >  3 files changed, 170 insertions(+)
> >  create mode 100644 doc/group-names.txt
> >  create mode 100755 tools/check-groups
> >
> >
> > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > new file mode 100644
> > index 00000000..ae517328
> > --- /dev/null
> > +++ b/doc/group-names.txt
> > @@ -0,0 +1,136 @@
> > +======================= =======================================================
> > +Group Name:            Description:
> > +======================= =======================================================
> > +all                    All known tests, automatically generated by ./check at
> > +                       runtime
> > +auto                   Tests that should be run automatically.  These should
> > +                       not require more than ~5 minutes to run.
> > +quick                  Tests that should run in under 30 seconds.
> > +deprecated             Old tests that should not be run.
> > +
> > +acl                    Access Control Lists
> > +admin                  xfs_admin functionality
> > +aio                    general libaio async io tests
> > +atime                  file access time
> > +attr                   extended attributes
> > +attr2                  xfs v2 extended aributes
> > +balance                        btrfs tree rebalance
> > +bigtime                        timestamps beyond the year 2038
> > +blockdev               block device functionality
> > +broken                 broken tests
> > +cap                    Linux capabilities
> > +casefold               directory name casefolding
> > +ci                     ASCII case-insensitive directory name lookups
> > +clone                  FICLONE/FICLONERANGE ioctls
> > +clone_stress           stress testing FICLONE/FICLONERANGE
> > +collapse               fallocate collapse_range
> > +compress               file compression
> > +convert                        btrfs ext[34] conversion tool
> > +copy                   xfs_copy functionality
> > +copy_range             copy_file_range syscall
> > +copyup                 overlayfs copyup support
> 
> The tests in this group exercise copy up.
> There is no such thing as overlayfs without "copyup support",
> so guess my point is - remove the word "support"

OK.

> > +dangerous              dangerous test that can crash the system
> > +dangerous_bothrepair   fuzzers to evaluate xfs_scrub + xfs_repair repair
> > +dangerous_fuzzers      fuzzers that can crash your computer
> > +dangerous_norepair     fuzzers to evaluate kernel metadata verifiers
> > +dangerous_online_repair        fuzzers to evaluate xfs_scrub online repair
> > +dangerous_repair       fuzzers to evaluate xfs_repair offline repair
> > +dangerous_scrub                fuzzers to evaluate xfs_scrub checking
> > +data                   data loss checkers
> > +dax                    direct access mode for persistent memory files
> > +db                     xfs_db functional tests
> > +dedupe                 FIEDEDUPERANGE ioctl
> > +defrag                 filesystem defragmenters
> > +dir                    directory test functions
> > +dump                   dump and restore utilities
> > +eio                    IO error reporting
> > +encrypt                        encrypted file contents
> > +enospc                 ENOSPC error reporting
> > +exportfs               file handles
> > +filestreams            XFS filestreams allocator
> > +freeze                 filesystem freeze tests
> > +fsck                   general fsck tests
> > +fsmap                  FS_IOC_GETFSMAP ioctl
> > +fsr                    XFS free space reorganizer
> > +fuzzers                        filesystem fuzz tests
> > +growfs                 increasing the size of a filesystem
> > +hardlink               hardlinks
> > +health                 XFS health reporting
> > +idmapped               idmapped mount functionality
> > +inobtcount             XFS inode btree count tests
> > +insert                 fallocate insert_range
> > +ioctl                  general ioctl tests
> > +io_uring               general io_uring async io tests
> > +label                  filesystem labelling
> > +limit                  resource limits
> > +locks                  file locking
> > +log                    metadata logging
> > +logprint               xfs_logprint functional tests
> > +long_rw                        long-soak read write IO path exercisers
> > +metacopy               overlayfs metadata-only copy-up
> > +metadata               filesystem metadata update exercisers
> > +metadump               xfs_metadump/xfs_mdrestore functionality
> > +mkfs                   filesystem formatting tools
> > +mount                  mount option and functionality checks
> > +nested                 nested overlayfs instances
> > +nfs4_acl               NFSv4 access control lists
> > +nonsamefs              overlayfs layers on different filesystems
> > +online_repair          online repair functionality tests
> > +other                  dumping ground, do not add more tests to this group
> > +overlay                        using overlayfs on top of FSTYP
> 
> This description is a bit confusing, because the recommended
> way to run overlayfs tests as described in README.overlay is
> to set FSTYP=xfs and run ./check -overlay
> 
> I'm struggling for a better description but perhaps:
> "using overlayfs regardless of ./check -overlay flag"?

Hmm.  Since I'm the author of the only test that uses this tag, I guess
I'm the authority (ha!) on what the name actually means.

That test (generic/631) is a regression test for a XFS whiteout handling
bug that can only be reproduced by layering overlayfs atop xfs.
Overlayfs is incidental to reproducing the XFS bug, but AFAIK overlayfs
is the only in-kernel user of whiteout, which is why it's critical here.

It's not right to make it "_supported_fs overlay" because we're not
testing overlayfs functionality; we're merely using it as a stick to
poke another filesystem.

How about: "regression tests that require the use of overlayfs in a
targetted configuration" ?

> > +pattern                        specific IO pattern tests
> > +perms                  access control and permission checking
> > +pipe                   pipe functionality
> > +pnfs                   PNFS
> > +posix                  POSIX behavior conformance
> > +prealloc               fallocate
> > +preallocrw             fallocate, then read and write
> > +punch                  fallocate punch_hole
> > +qgroup                 btrfs qgroup feature
> > +quota                  filesystem usage quotas
> > +raid                   btrfs RAID
> > +realtime               XFS realtime volumes
> > +recoveryloop           crash recovery loops
> > +redirect               overlayfs redirect_dir feature
> > +remote                 dump and restore with a remote tape
> > +remount                        remounting filesystems
> > +rename                 rename system call
> > +repair                 xfs_repair functional tests
> > +replace                        btrfs device replace
> > +replay                 dm-logwrites replays
> > +resize                 resize2fs functionality tests
> > +richacl                        rich ACL feature
> > +rmap                   XFS reverse mapping exercisers
> > +rotate                 overlayfs feature of some sort"
> 
> I guess that works :-D
> but to be accurate, this is actually a unionmount testsuite feature -
> at selected test points in the workload, a new upper layer is stacked
> on to overlayfs, so maybe:
> "upper layer rotate tests from the unionmount test suite"?

I've changed it to 'overlayfs upper layer rotate tests from the
unionmount test suite'.

> > +rw                     read/write IO tests
> > +samefs                 overlayfs when all layers are on the same fs
> > +scrub                  filesystem metadata scrubbers
> > +seed                   btrfs seeded filesystems
> > +seek                   llseek functionality
> > +send                   btrfs send/receive
> > +shrinkfs               decreasing the size of a filesystem
> > +shutdown               FS_IOC_SHUTDOWN ioctl
> > +snapshot               btrfs snapshots
> > +soak                   long soak tests of any kind
> > +spaceman               xfs_spaceman functional tests
> > +splice                 splice system call
> > +stress                 fsstress filesystem exerciser
> > +subvol                 btrfs subvolumes
> > +subvolume              btrfs subvolumes (again?)
> 
> A cleanup patch to fix this typo in btrfs/233?

Will do.

--D

> 
> Thanks,
> Amir.