Enabling additional KUnit tests in KernelCI?

Enabling additional KUnit tests in KernelCI?

[Eric Biggers]

Hi,

I would like the KUnit tests in lib/crypto/tests/ and lib/crc/tests/ to
run in KernelCI and have any failures reported to
linux-crypto@vger.kernel.org.  Preferably on all platforms, not just
UML, since UML doesn't cover all the code.  Is there a way to do this?
What is the status of KUnit in KernelCI?  I'm having a hard time finding
information about this, and looking at the latest mainline test results
(https://dashboard.kernelci.org/tree/mainline/master/4ae12d8bd9a830799db335ee661d6cbc6597f838?p=t)
there seem to be no KUnit tests running at all.

Thanks,

- Eric

Re: Enabling additional KUnit tests in KernelCI?

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On Fri, Mar 06, 2026 at 11:55:46PM -0800, Eric Biggers wrote:

> I would like the KUnit tests in lib/crypto/tests/ and lib/crc/tests/ to
> run in KernelCI and have any failures reported to
> linux-crypto@vger.kernel.org.  Preferably on all platforms, not just
> UML, since UML doesn't cover all the code.  Is there a way to do this?
> What is the status of KUnit in KernelCI?  I'm having a hard time finding
> information about this, and looking at the latest mainline test results
> (https://dashboard.kernelci.org/tree/mainline/master/4ae12d8bd9a830799db335ee661d6cbc6597f838?p=t)
> there seem to be no KUnit tests running at all.

We should be running KUnit tests:

  https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/jobs.yaml#L2153

  https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/scheduler.yaml#L1750

though like you say something seems to be going AWOL with at least the
reporting, I can't see any results either.  The job that's configured
there is to run on x86_64 rather than UML so is probably what you're
looking for in terms of the tests?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Enabling additional KUnit tests in KernelCI?

[Eric Biggers]

On Thu, Mar 12, 2026 at 08:21:26PM +0000, Mark Brown wrote:
> On Fri, Mar 06, 2026 at 11:55:46PM -0800, Eric Biggers wrote:
> 
> > I would like the KUnit tests in lib/crypto/tests/ and lib/crc/tests/ to
> > run in KernelCI and have any failures reported to
> > linux-crypto@vger.kernel.org.  Preferably on all platforms, not just
> > UML, since UML doesn't cover all the code.  Is there a way to do this?
> > What is the status of KUnit in KernelCI?  I'm having a hard time finding
> > information about this, and looking at the latest mainline test results
> > (https://dashboard.kernelci.org/tree/mainline/master/4ae12d8bd9a830799db335ee661d6cbc6597f838?p=t)
> > there seem to be no KUnit tests running at all.
> 
> We should be running KUnit tests:
> 
>   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/jobs.yaml#L2153
> 
>   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/scheduler.yaml#L1750

Which ones?  Does anything need to be done to add new tests to the list?

Note that even if "all" tests are being run (via
CONFIG_KUNIT_ALL_TESTS=y), that includes only the tests whose
dependencies are met.  There can be additional kconfig dependencies.

> though like you say something seems to be going AWOL with at least the
> reporting, I can't see any results either.  The job that's configured
> there is to run on x86_64 rather than UML so is probably what you're
> looking for in terms of the tests?

The kernel has optimized crypto and/or CRC code for the following
architectures: arm, arm64, mips, powerpc, riscv, s390, sparc, x86.  In
many cases there's also a finer division based on CPU features.

So I'm looking for testing on as many platforms as possible.  But some
x86_64 platform is a good start, certainly better than nothing.

- Eric

Re: Enabling additional KUnit tests in KernelCI?

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 2060 bytes --]

On Thu, Mar 12, 2026 at 02:51:45PM -0700, Eric Biggers wrote:
> On Thu, Mar 12, 2026 at 08:21:26PM +0000, Mark Brown wrote:

> > We should be running KUnit tests:

> >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/jobs.yaml#L2153

> >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/scheduler.yaml#L1750

> Which ones?  Does anything need to be done to add new tests to the list?

> Note that even if "all" tests are being run (via
> CONFIG_KUNIT_ALL_TESTS=y), that includes only the tests whose
> dependencies are met.  There can be additional kconfig dependencies.

Whatever is enabled by the KUnit runner tools/testing/kunit/kunit.py
(which makes x86_64 a bit of an unfortunate choice given how slim theilr
defconfig is).  The theory is that the KUnit configs will do the right
thing, it looks like we're not currently pasing --alltests though which
we should - I'll look at a patch for that tomorrow.  If you need arch
specific options that aren't enabled by the platform's defconfig that
gets a bit awkward.

> > though like you say something seems to be going AWOL with at least the
> > reporting, I can't see any results either.  The job that's configured
> > there is to run on x86_64 rather than UML so is probably what you're
> > looking for in terms of the tests?

> The kernel has optimized crypto and/or CRC code for the following
> architectures: arm, arm64, mips, powerpc, riscv, s390, sparc, x86.  In
> many cases there's also a finer division based on CPU features.

> So I'm looking for testing on as many platforms as possible.  But some
> x86_64 platform is a good start, certainly better than nothing.

The KUnit runner uses qemu as standard which I guess is probably fine
for an initial pass, we would need to do a bunch more plumbing to run on
actual hardware and pull the results out.  We'd also be restricted by
what's available to us, at the minute we've got arm, arm64, riscv and
x86 systems with a bit of an embedded focus.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Enabling additional KUnit tests in KernelCI?

[Eric Biggers]

On Thu, Mar 12, 2026 at 11:34:51PM +0000, Mark Brown wrote:
> On Thu, Mar 12, 2026 at 02:51:45PM -0700, Eric Biggers wrote:
> > On Thu, Mar 12, 2026 at 08:21:26PM +0000, Mark Brown wrote:
> 
> > > We should be running KUnit tests:
> 
> > >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/jobs.yaml#L2153
> 
> > >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/scheduler.yaml#L1750
> 
> > Which ones?  Does anything need to be done to add new tests to the list?
> 
> > Note that even if "all" tests are being run (via
> > CONFIG_KUNIT_ALL_TESTS=y), that includes only the tests whose
> > dependencies are met.  There can be additional kconfig dependencies.
> 
> Whatever is enabled by the KUnit runner tools/testing/kunit/kunit.py
> (which makes x86_64 a bit of an unfortunate choice given how slim theilr
> defconfig is).  The theory is that the KUnit configs will do the right
> thing, it looks like we're not currently pasing --alltests though which
> we should - I'll look at a patch for that tomorrow.  If you need arch
> specific options that aren't enabled by the platform's defconfig that
> gets a bit awkward.

Yes, the tests run by kunit.py depend on the args passed to it.  It
looks like without any arguments it runs all tests available with
defconfig + tools/testing/kunit/configs/default.config.  With
--alltests, it runs all tests available with defconfig +
tools/testing/kunit/configs/all_tests.config.  Both set
"CONFIG_KUNIT_ALL_TESTS=y", so the difference is just what additional
options get enabled to fulfill the test dependencies.  all_tests.config
enables more dependencies than default.config.

So I think KernelCI should start passing --alltests to kunit.py, and I
should get the dependencies for all the crypto and CRC tests added to
tools/testing/kunit/configs/all_tests.config.  Then KernelCI would pick
them up automatically.

Same for any other kernel subsystem: it seems maintainers should get
their test dependencies added to all_tests.config.  (Unless those
options are arch-specific, as you mentioned.  The crypto and CRC options
are all generic though; just the implementations differ across arches.)

> > > though like you say something seems to be going AWOL with at least the
> > > reporting, I can't see any results either.  The job that's configured
> > > there is to run on x86_64 rather than UML so is probably what you're
> > > looking for in terms of the tests?
> 
> > The kernel has optimized crypto and/or CRC code for the following
> > architectures: arm, arm64, mips, powerpc, riscv, s390, sparc, x86.  In
> > many cases there's also a finer division based on CPU features.
> 
> > So I'm looking for testing on as many platforms as possible.  But some
> > x86_64 platform is a good start, certainly better than nothing.
> 
> The KUnit runner uses qemu as standard which I guess is probably fine
> for an initial pass, we would need to do a bunch more plumbing to run on
> actual hardware and pull the results out.  We'd also be restricted by
> what's available to us, at the minute we've got arm, arm64, riscv and
> x86 systems with a bit of an embedded focus.

It seems kunit.py uses UML by default, though it accepts an --arch
argument which enables QEMU.  So something to think about for KernelCI
would be to run all KUnit tests with different combinations of --arch.

Of course, running the tests natively on different hardware would be
even better, when such hardware is available.

- Eric

Re: Enabling additional KUnit tests in KernelCI?

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1840 bytes --]

On Fri, Mar 13, 2026 at 01:04:56AM +0000, Eric Biggers wrote:
> On Thu, Mar 12, 2026 at 11:34:51PM +0000, Mark Brown wrote:

> > Whatever is enabled by the KUnit runner tools/testing/kunit/kunit.py
> > (which makes x86_64 a bit of an unfortunate choice given how slim theilr
> > defconfig is).  The theory is that the KUnit configs will do the right

> Yes, the tests run by kunit.py depend on the args passed to it.  It
> looks like without any arguments it runs all tests available with
> defconfig + tools/testing/kunit/configs/default.config.  With
> --alltests, it runs all tests available with defconfig +
> tools/testing/kunit/configs/all_tests.config.  Both set
> "CONFIG_KUNIT_ALL_TESTS=y", so the difference is just what additional
> options get enabled to fulfill the test dependencies.  all_tests.config
> enables more dependencies than default.config.

Yes, exactly.

> So I think KernelCI should start passing --alltests to kunit.py, and I
> should get the dependencies for all the crypto and CRC tests added to
> tools/testing/kunit/configs/all_tests.config.  Then KernelCI would pick
> them up automatically.

That sounds like a plan.

> > The KUnit runner uses qemu as standard which I guess is probably fine
> > for an initial pass, we would need to do a bunch more plumbing to run on
> > actual hardware and pull the results out.  We'd also be restricted by
> > what's available to us, at the minute we've got arm, arm64, riscv and
> > x86 systems with a bit of an embedded focus.

> It seems kunit.py uses UML by default, though it accepts an --arch
> argument which enables QEMU.  So something to think about for KernelCI
> would be to run all KUnit tests with different combinations of --arch.

Right, and UML only works with an x86 host which is fun.  qemu is the
standard way of running for a specific architecture.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Enabling additional KUnit tests in KernelCI?

[Eric Biggers]

On Fri, Mar 13, 2026 at 12:31:41PM +0000, Mark Brown wrote:
> On Fri, Mar 13, 2026 at 01:04:56AM +0000, Eric Biggers wrote:
> > On Thu, Mar 12, 2026 at 11:34:51PM +0000, Mark Brown wrote:
> 
> > > Whatever is enabled by the KUnit runner tools/testing/kunit/kunit.py
> > > (which makes x86_64 a bit of an unfortunate choice given how slim theilr
> > > defconfig is).  The theory is that the KUnit configs will do the right
> 
> > Yes, the tests run by kunit.py depend on the args passed to it.  It
> > looks like without any arguments it runs all tests available with
> > defconfig + tools/testing/kunit/configs/default.config.  With
> > --alltests, it runs all tests available with defconfig +
> > tools/testing/kunit/configs/all_tests.config.  Both set
> > "CONFIG_KUNIT_ALL_TESTS=y", so the difference is just what additional
> > options get enabled to fulfill the test dependencies.  all_tests.config
> > enables more dependencies than default.config.
> 
> Yes, exactly.
> 
> > So I think KernelCI should start passing --alltests to kunit.py, and I
> > should get the dependencies for all the crypto and CRC tests added to
> > tools/testing/kunit/configs/all_tests.config.  Then KernelCI would pick
> > them up automatically.
> 
> That sounds like a plan.
> 
> > > The KUnit runner uses qemu as standard which I guess is probably fine
> > > for an initial pass, we would need to do a bunch more plumbing to run on
> > > actual hardware and pull the results out.  We'd also be restricted by
> > > what's available to us, at the minute we've got arm, arm64, riscv and
> > > x86 systems with a bit of an embedded focus.
> 
> > It seems kunit.py uses UML by default, though it accepts an --arch
> > argument which enables QEMU.  So something to think about for KernelCI
> > would be to run all KUnit tests with different combinations of --arch.
> 
> Right, and UML only works with an x86 host which is fun.  qemu is the
> standard way of running for a specific architecture.

Just to follow up on this, in mainline 'kunit.py run --alltests' now
runs all the CRC and crypto library KUnit tests.

It looks like the kunit job on KernelCI is still running only on the 6.1
kernel.  Will it be enabled on newer kernels?

- Eric

Re: Enabling additional KUnit tests in KernelCI?

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On Sat, Apr 18, 2026 at 05:50:19PM -0700, Eric Biggers wrote:
> On Fri, Mar 13, 2026 at 12:31:41PM +0000, Mark Brown wrote:

> > Right, and UML only works with an x86 host which is fun.  qemu is the
> > standard way of running for a specific architecture.

> Just to follow up on this, in mainline 'kunit.py run --alltests' now
> runs all the CRC and crypto library KUnit tests.

Excellent, that'll be being picked up as part of the -next merges.

> It looks like the kunit job on KernelCI is still running only on the 6.1
> kernel.  Will it be enabled on newer kernels?

Huh.  That's weird and not intentional, looking at the config I can't
see why it's happening.  Thanks for spotting this, I've filed an issue:

   https://github.com/kernelci/kernelci-pipeline/issues/1466

so it doesn't get lost, hopefully it'll be something obvious but it's
not obvious enough for me right now :/ .

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Enabling additional KUnit tests in KernelCI?

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]

On Mon, Apr 20, 2026 at 03:16:54PM +0100, Mark Brown wrote:
> On Sat, Apr 18, 2026 at 05:50:19PM -0700, Eric Biggers wrote:

> > It looks like the kunit job on KernelCI is still running only on the 6.1
> > kernel.  Will it be enabled on newer kernels?

> Huh.  That's weird and not intentional, looking at the config I can't
> see why it's happening.  Thanks for spotting this, I've filed an issue:

>    https://github.com/kernelci/kernelci-pipeline/issues/1466

> so it doesn't get lost, hopefully it'll be something obvious but it's
> not obvious enough for me right now :/ .

This should be sorting itself now - Denys realised that the results were
getting produced but stuck in processing, it's one of the biggest suites
which triggered some infrastructure problems.  I'm seeing results
appearing for mainline and -next now, hopefully everything else should
start populating as well.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]