Re: [PATCHSET RFC 00/10] fstests: move test group lists into test files

["Darrick J. Wong" <djwong@kernel.org>]

On Sun, Jun 06, 2021 at 10:44:39PM +0800, Eryu Guan wrote:
> On Tue, May 25, 2021 at 06:46:42PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Test group files (e.g. tests/generic/group) are a pain to keep up.
> > Every week I rebase on Eryu's latest upstream, and every week I have to
> > slog through dozens of trivial merge conflicts because of the
> > groupfiles.  Moving tests is annoying because we have to maintain all
> > this code to move the group associations from one /group file to
> > another.
> 
> Thanks for doing this! It looks find overall from a quick look. Mind
> adding some words in README file as well?

Ok.

> > It doesn't need to be this way -- we could move each test's group
> > information into the test itself, and automatically generate the group
> > files as part of the make process.  This series does exactly that.
> > 
> > The first few patches add some convenient anchors for the new
> > per-testfile group tagging and a conversion script to migrate existing
> > test files.  Next there's a huge patch that is the results of running
> > the conversion script, followed by cleanup of the golden outputs.  After
> > that comes the build infrastructure to generate group files and other
> 
> The group files are auto-generated correctly upon "make", but "make
> group" printed tons of errors like:
> 
> [root@fedoravm xfstests]# make group
>  [GROUP] /root/workspace/xfstests/group
> /root/workspace/xfstests/tests/btrfs/001: line 9: ./common/test_names: No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 10: _set_seq_and_groups: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 21: ./common/rc: No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 22: ./common/filter: No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 23: ./common/filter.btrfs: No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 26: _supported_fs: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 27: _require_scratch: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 30: _scratch_mount: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 38: _filter_scratch: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 38: subvolume: command not found
> ls: cannot access '/snap': No such file or directory
> ls: cannot access '/snap': No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 50: _filter_scratch: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 50: subvolume: command not found
> ls: cannot access '/subvol': No such file or directory
> /root/workspace/xfstests/tests/btrfs/001: line 60: _btrfs_get_subvolid: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 61: _filter_scratch: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 61: subvolume: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 62: _scratch_cycle_mount: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 65: _scratch_unmount: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 67: _scratch_mount: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 71: _filter_scratch: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 71: subvolume: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 72: _scratch_cycle_mount: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 78: subvolume: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 81: _filter_btrfs_subvol_delete: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 81: subvolume: command not found
> /root/workspace/xfstests/tests/btrfs/001: line 84: _scratch_cycle_mount: command not found
> 
> I think it's better to let 'make group' work as well, it might be easier
> to re-generated just group files.

Yikes.  I will investigate this before reposting.

> > tweaks to the existing maintainer scripts to use the new infrastructure.
> > Finally, remove the group files themselves and the (now unnecessary)
> > code that maintained them.
> > 
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> > 
> > This is an extraordinary way to destroy everything.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=autogenerate-groupfiles
> 
> I noticed that your patches are based on your internal base, and this
> branch couldn't be pulled into current master branch. But this is fine
> for RFC patches I think :)

Yep.  Since this is a treewide change I'll put this in a special branch
and rebase my dev tree after it lands.

--D

> 
> Thanks,
> Eryu