Re: [PATCH v6 02/12] clk: Introduce Kunit Tests for the framework

[Maxime Ripard <maxime@cerno.tech>]

[-- Attachment #1: Type: text/plain, Size: 5424 bytes --]

Hi Daniel,

On Mon, Mar 28, 2022 at 02:36:25PM -0500, Daniel Latypov wrote:
> On Mon, Mar 28, 2022 at 2:57 AM Maxime Ripard <maxime@cerno.tech> wrote:
> > On Fri, Mar 25, 2022 at 05:36:25PM -0500, Daniel Latypov wrote:
> > > On Mon, Feb 28, 2022 at 4:47 AM Maxime Ripard <maxime@cerno.tech> wrote:
> > > >
> > > > On Fri, Feb 25, 2022 at 01:29:03PM -0800, Daniel Latypov wrote:
> > > > > On Fri, Feb 25, 2022 at 5:23 AM Maxime Ripard <maxime@cerno.tech> wrote:
> > > > > >
> > > > > > Hi Daniel,
> > > > > >
> > > > > > On Wed, Feb 23, 2022 at 02:50:59PM -0800, Daniel Latypov wrote:
> > > > > > > On Wed, Feb 23, 2022 at 2:56 AM Maxime Ripard <maxime@cerno.tech> wrote:
> > > > > > > >
> > > > > > > > Let's test various parts of the rate-related clock API with the kunit
> > > > > > > > testing framework.
> > > > > > > >
> > > > > > > > Cc: kunit-dev@googlegroups.com
> > > > > > > > Suggested-by: Stephen Boyd <sboyd@kernel.org>
> > > > > > > > Signed-off-by: Maxime Ripard <maxime@cerno.tech>
> > > > > > >
> > > > > > > Tested-by: Daniel Latypov <dlatypov@google.com>
> > > > > > >
> > > > > > > Looks good to me on the KUnit side.
> > > > > > > Two small nits below.
> > > > > > >
> > > > > > > FYI, I computed the incremental coverage for this series, i.e.:
> > > > > > > 1) applied the full series
> > > > > > > 2) computed the absolute coverage
> > > > > > >
> > > > > > > $  ./tools/testing/kunit/kunit.py run  --kunitconfig=drivers/clk
> > > > > > > --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_KERNEL=y
> > > > > > > --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_GCOV=y
> > > > > >
> > > > > > I built a docker container based on ubuntu 18.04 to have gcc6 and
> > > > > > python3.7, but this doesn't seem to be working, I'm not entirely sure why:
> > > > > >
> > > > > > [13:11:22] Configuring KUnit Kernel ...
> > > > > > Regenerating .config ...
> > > > > > Populating config with:
> > > > > > $ make ARCH=um olddefconfig CC=/usr/bin/gcc-6 O=.kunit
> > > > > > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
> > > > > > This is probably due to unsatisfied dependencies.
> > > > > > Missing: CONFIG_DEBUG_INFO=y, CONFIG_GCOV=y
> > > > > > Note: many Kconfig options aren't available on UML. You can try running on a different architecture with something like "--arch=x86_64".
> > > > >
> > > > > Did you perhaps drop CONFIG_DEBUG_KERNEL=y?
> > > > > Need to add 3 config options in total for coverage.
> > > > >
> > > > > If I tweak the command I ran above but drop CONFIG_DEBUG_KERNEL=y, I
> > > > > get the error message you get:
> > > > >
> > > > > $  ./tools/testing/kunit/kunit.py run  --kunitconfig=drivers/clk
> > > > > --make_options=CC=/usr/bin/gcc-6  --kconfig_add=CONFIG_DEBUG_INFO=y
> > > > > --kconfig_add=CONFIG_GCOV=y
> > > > > ...
> > > > > Missing: CONFIG_DEBUG_INFO=y, CONFIG_GCOV=y
> > > > > Note: many Kconfig options aren't available on UML. You can try
> > > > > running on a different architecture with something like
> > > > > "--arch=x86_64".
> > > >
> > > > It looks to me that it's more that DEBUG_INFO isn't enabled.
> > >
> > > Sorry for the very delayed response.
> > > I was largely getting internet over mobile data around when this email
> > > came in and didn't want to try and download docker images over that.
> > >
> > > It looks like that there was another change that is now merged into
> > > Linus' tree that causes this.
> > >
> > > I found that adding this helped (thanks David Gow)
> > >   --kconfig_add=DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT
> > >
> > > Running against --kunitconfig=lib/kunit, my final coverage result is
> > >
> > > Overall coverage rate:
> > >   lines......: 13.6% (18004 of 132055 lines)
> > >   functions..: 15.7% (1885 of 12010 functions)
> > >
> > > Can you give that a shot and see if it works?
> >
> > It does fix the configuration issue, but I'm not able to run the tests either:
> >
> > [07:53:51] Configuring KUnit Kernel ...
> > Generating .config ...
> > Populating config with:
> > $ make ARCH=um olddefconfig O=/home/max/out
> > [07:53:53] Building KUnit Kernel ...
> > Populating config with:
> > $ make ARCH=um olddefconfig O=/home/max/out
> > Building with:
> > $ make ARCH=um --jobs=16 O=/home/max/out
> > [07:54:09] Starting KUnit Kernel (1/1)...
> > [07:54:09] ============================================================
> > [07:54:09] [ERROR] Test : invalid KTAP input!
> > [07:54:09] ============================================================
> > [07:54:09] Testing complete. Passed: 0, Failed: 0, Crashed: 0, Skipped: 0, Errors: 1
> > [07:54:09] Elapsed time: 18.486s total, 2.430s configuring, 16.052s building, 0.003s running
> >
> >
> > I've tried to remove all the coverage from the equation, and I get the
> > same issue if I only run kunit run from inside the container, but it
> > works fine outside. So I guess it's my setup that is broken. Is there
> > some way to debug what could be going wrong there?
> 
> kunit.py is failing to find any test results from the raw kernel dmesg output.
> That is stored in $BUILD_DIR/test.log, so /home/max/out/test.log.
> (You can also have kunit.py print this out instead with `kunit.py run
> --raw_output`)

I was missing CAP_SYS_PTRACE in my container, once set it works just fine, thanks!
Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]