Re: [PATCH] selftests: Add a taint selftest

[Greg KH <gregkh@linuxfoundation.org>]

On Tue, Aug 23, 2022 at 10:50:33PM -0700, Andrew Delgadillo wrote:
> On Tue, Aug 23, 2022 at 10:06 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Tue, Aug 23, 2022 at 09:19:58PM +0000, Andrew Delgadilo wrote:
> > > From: Andrew Delgadillo <adelg@google.com>
> > >
> > > When testing a kernel, one of the earliest signals one can get is if a
> > > kernel has become tainted. For example, an organization might be
> > > interested in mass testing commits on their hardware. An obvious first
> > > step would be to make sure every commit boots, and a next step would be
> > > to make sure there are no warnings/crashes/lockups, hence the utility of
> > > a taint test.
> >
> > What's wrong with the tools/debugging/kernel-chktaint script?
> >
> > Why do we need another "get what the taint status is" program?
> 
> The main functionality that this selftests has that kernel-chktaint
> does not is that it exits with a non-zero status code if the kernel is
> tainted. kernel-chktaint outputs information to stdout based on the
> taint status, but will always exit 0.

Great, then change that, don't create a whole new script :)

> The issue with this is that it cannot be plugged into a test runner
> that checks the exit code of a test script. In other words, if I
> wanted to plug it into git bisect, I would have to wrap
> kernel-chktaint in a command that transformed the output to an exit
> code. Sure that is doable, but it is not as simple as it could be.
> 
> More concretely, I am setting kselftest runs against kernel commits
> (with a harness that logs kselftest runs into some other
> infrastructure), and such a test that is missing is a kselftest that
> checks the kernel's taint status. One could argue that one should just
> create a kselftest target that calls into kernel-chktaint and parse
> the output there to determine what the exit status is, but that seems
> fragile as a change in the underlying script could break it. For
> example, if I want to test for taint #18, and I am grepping for the
> string " * an in-kernel test has been run (#18)", I will actually get
> a false positive because the underlying script does not check for
> taint #18. Contrived example yes, but I think it shows that textual
> grepping for errors is error prone (as an aside, I'll send a patch to
> update the script to check for the new taint bit).

Then modify the existing script to handle your use case, let's not have
duplicate ones in the tree.

thanks,

greg k-h