Automation with 0-day & kdevops

Automation with 0-day & kdevops

[Luis Chamberlain]

Dear 0-day developers,

kdevops [0] has evolved over the years now to a full automation suite
for kernel development and testing. As for the later aspects of it, we
use it to enable complicated subsystem tests such as filesystems
testing. Our automated filesystem coverage has been rather reduced
given the complexity, and so one of its goals was to tackle this. It
also has support to automate testing complex subsystems involving
custom non-upstream yet for things like qemu as well.

While long term we'd like to aim towards automating most of the things
tested under kdevops, it makes sense to start slow with a few simpler
targets. Since kdevops supports kselftests as well, my recommendation
is we start with a few selftests for components we have kernel
maintainers willing to help with either review or help tune up. The
same applies to filesystems. While we have support to test most
popular filesystems it makes sense to start with something simple.

To this end I'd like to see if we can collaborate with 0-day so enable
automation of testing for the following components, the first 3 of
which I help maintain:

With kdevops using its kernel selftests support:

  * Linux kernel modules: using kernel selftests and userspace kmod tests
  * Linux firmware loader: firmware selftests
  * Linux sysctl

As for filesystems I'd like to start with tmpfs as we have a developer
who already has a good baseline for it, and is helping to fix some
fstests bugs found, Daniel Gomez. We also have created different
target profiles to test tmpfs for the different mount options it
supports.

What would this collaboration consist of? Using 0-day's automated to
git clone kdevops, spawn some resouces and run a series of make
commands. If git diff returns non-empty we have a new failure.

[0] https://github.com/linux-kdevops/kdevops

 Luis

Re: Automation with 0-day & kdevops

[Philip Li]

On Fri, Feb 23, 2024 at 07:44:12AM -0800, Luis Chamberlain wrote:
> Dear 0-day developers,
> 
> kdevops [0] has evolved over the years now to a full automation suite
> for kernel development and testing. As for the later aspects of it, we
> use it to enable complicated subsystem tests such as filesystems
> testing. Our automated filesystem coverage has been rather reduced
> given the complexity, and so one of its goals was to tackle this. It
> also has support to automate testing complex subsystems involving
> custom non-upstream yet for things like qemu as well.
> 
> While long term we'd like to aim towards automating most of the things
> tested under kdevops, it makes sense to start slow with a few simpler
> targets. Since kdevops supports kselftests as well, my recommendation
> is we start with a few selftests for components we have kernel
> maintainers willing to help with either review or help tune up. The
> same applies to filesystems. While we have support to test most
> popular filesystems it makes sense to start with something simple.
> 
> To this end I'd like to see if we can collaborate with 0-day so enable
> automation of testing for the following components, the first 3 of
> which I help maintain:
> 
> With kdevops using its kernel selftests support:
> 
>   * Linux kernel modules: using kernel selftests and userspace kmod tests
>   * Linux firmware loader: firmware selftests
>   * Linux sysctl
> 
> As for filesystems I'd like to start with tmpfs as we have a developer
> who already has a good baseline for it, and is helping to fix some
> fstests bugs found, Daniel Gomez. We also have created different
> target profiles to test tmpfs for the different mount options it
> supports.
> 
> What would this collaboration consist of? Using 0-day's automated to
> git clone kdevops, spawn some resouces and run a series of make
> commands. If git diff returns non-empty we have a new failure.

Thanks Luis, we are glad to collaborate on this. We will learn the detail of
kdevops that the website contains a lot of useful materials. And we will update
our progress to you next month for the questions and next steps.

Thanks

> 
> [0] https://github.com/linux-kdevops/kdevops
> 
>  Luis
>