Re: [PATCH 1/2] check: add a -smoketest option

[Zorro Lang <zlang@redhat.com>]

On Thu, Jul 27, 2023 at 10:33:26AM -0400, Theodore Ts'o wrote:
> On Thu, Jul 27, 2023 at 11:25:37AM +0800, Zorro Lang wrote:
> > > SOAK_DURATION=4m ./check -g smoketest
> > 
> > Now we provide two ways to help to customize testing in fstests:
> > 
> > 1)
> > https://lore.kernel.org/fstests/20230727030529.r4ivp6dmtrht5zo2@zlang-mailbox/T/#mc5cdb59344f4cd681515bf0fab501d7f30f1e263
> > 
> > 2)
> > https://lore.kernel.org/fstests/169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs/T/#u
> > 
> > Which one do you like to use? I'd like to hear more review points before I
> > choose one to merge.
> 
> (1) is the "./check -t smoketest" option, and it provides a more
> generic way of adding new templates.  On the positive side it allows
> more of this kind of simple "configuration" style options where "-t
> smoketest" is essentially syntactic sugar for:
> 
> 	SOAK_DURATION=${SOAK_DURATION:-4m} ./check -g smoketest"
> 
> The potential disadvantage of (1) is that it seems like extra
> complexity for what is really simple.
> 
> 
> (2) is "./check -smoketest" option.  Its advantage is that it might
> easier for a drive-by patcher to type.  The disadvantage is that it's
> adding Yet Another Option to the ./check script.
> 
> I also will note that we have some "long options" which use a single
> hypen (e.g., -overlay and -udiff) but newer "long options" seem to use
> the double hypehn approach (e.g., --exact-order and --large-fs).  My
> personal preference is for the newer GNU getopt style of using double
> hyphens, but the fact that we have both types of long options
> is... unfortunate.

Yeah, I'd like to tidy the ./check, include the option names. But change the
check option format will affect many users, cause most of their scripts go
wrong suddently, then they need to check and use new option format. That's
why I still not touch this part.

> 
> 
> I guess I have a slight preference for (1), but I'm really not sure
> either is really necessary.  My view is that for a drive-by tester,
> trying to set up xfstests is Too Hard.  So the reality is they will be
> using some kind of wrapper script --- either one that they've written
> for their own, such as what Darrick (and I assume other XFS developers
> have their own), or they're using something like kdevops or
> kvm-xfstests.

Sure, you're right. fstests can be used to do a simple test, but for regular
test, a wrapper is needed. Darrick has his wrapper, Dave might has his wrapper
too. My team also has our wrapper, (we also have wrappers to run ltp and others).
Different users might have different testing environment, so they might build
different wrappers to connect fstests with their own environment/requirement.

So I'd like to keep fstests as simple underlying test cases, provide basic
testing functions to anyone who would like to run it, try to not limit much
things. But I'd like to let fstests provides more help to each of your testing
requirement. That's my current crude thought :-P

> 
> From *my* perspective, I have absolutely *no* problem with having my
> wrapper script use:
> 
> 	SOACK_DURATION=4m ./check -g smoketest
> 
> because I only have to do it once, and no end-user is ever going to
> see it.  They will just use "kvm-xfstests smoke", and all of the magic
> will be hidden from them.
> 
> The main advantage of having some kind of "official" top-level way of
> specifying the smoke test is that it makes it more likely that
> different wrapper scripts will converge on the same kind of smoke
> test, and it becomes easier for fstests developers to communicate with
> each other because the concept of what a "smoke test" is has been well
> defined in the fstests source code.  And for that purpose, I think the
> "./check -t smoketest" approach works just fine.

OK, thanks for your reply. I'll double check with Darrick, then merge
one of them :)

Thanks,
Zorro

> 
> But really, I can live with either.   :-)
> 
> Cheers,
> 
> 						- Ted
>