Re: [PATCH v6 02/12] clk: Introduce Kunit Tests for the framework

[Stephen Boyd <sboyd@kernel.org>]

Quoting Daniel Latypov (2022-02-24 15:21:57)
> On Thu, Feb 24, 2022 at 2:54 PM Stephen Boyd <sboyd@kernel.org> wrote:
> >
> > Quoting Daniel Latypov (2022-02-23 14:50:59)
> > > On Wed, Feb 23, 2022 at 2:56 AM Maxime Ripard <maxime@cerno.tech> wrote:
> > > >
> > > > Let's test various parts of the rate-related clock API with the kunit
> > > > testing framework.
> > > >
> > > > Cc: kunit-dev@googlegroups.com
> > > > Suggested-by: Stephen Boyd <sboyd@kernel.org>
> > > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > >
> > > Tested-by: Daniel Latypov <dlatypov@google.com>
> > >
> > > Looks good to me on the KUnit side.
> > > Two small nits below.
> > >
> > > FYI, I computed the incremental coverage for this series, i.e.:
> > > 1) applied the full series
> > > 2) computed the absolute coverage
> > >
> > > $  ./tools/testing/kunit/kunit.py run  --kunitconfig=drivers/clk
> > > --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_KERNEL=y
> > > --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_GCOV=y
> > > $ lcov -t "clk_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
> >
> > This is cool. Thanks! Is it possible to add some 'coverage' command to
> > kunit so we don't have to recall this invocation?
> 
> This is documented at
> https://www.kernel.org/doc/html/latest/dev-tools/kunit/running_tips.html#generating-code-coverage-reports-under-uml
> It also includes pointers on how to use lcov to process the .gcda files.
> I wrote it before --kconfig_add existed, so it just looks a bit different.
> 
> The main blockers to directly supporting this in kunit.py are
> 1.) this only works on UML
> 2.) it needs gcc-6 or lower (and the kernel's min version is 5.1, iirc)...
> 3.) in kernels older than 5.14, this requires some more hacks to get
> working. So for the large portion of us stuck dealing with somewhat
> older kernels, we'd have to do stuff manually anyway.
> 
> For #1, we'd need different kconfig options and kunit.py's QEMU would
> need some sort of userspace (busybox should be sufficient).
> For #2, I don't recall what the precise issues were anymore. But I
> think there were some more issues in gcc 8 or 9... :(
> 
> >
> > >
> > > 3) intersected that with the total diff
> >
> > This would also be cool to do automatically with a revision range.
> 
> Hmm, can you elaborate?
> I assume you mean other revision ranges beyond this patch set?

I mean somehow to tell kunit.py that I want incremental coverage
information for a git revision range so that I can say something like

	kunit.py incremental HEAD~3..HEAD

and have it tell me the line coverage.