kunit alltests runs broken in mainline

kunit alltests runs broken in mainline

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]

Hi,

Commit 28b3df1fe6ba2cb4 ("kunit: add wireless unit tests") which I can't
seem to find on lore breaks full kunit runs on non-UML builds and is now
present in mainline.  If I run:

   ./tools/testing/kunit/kunit.py run --alltests --cross_compile x86_64-linux-gnu- --arch x86_64

on a clean tree then I get:

   [15:09:20] Configuring KUnit Kernel ...
   Generating .config ...
   Populating config with:
   $ make ARCH=x86_64 O=.kunit olddefconfig CROSS_COMPILE=x86_64-linux-gnu-
   ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
   This is probably due to unsatisfied dependencies.
   Missing: CONFIG_IWLWIFI=y, CONFIG_WLAN_VENDOR_INTEL=y

UML works fine, but other real architectures (eg, arm64) seem similarly
broken.  I've not looked properly yet, I'm a bit confused given that
there's not even any dependencies for WLAN_VENDOR_INTEL and it's not
mentoned in the defconfig.

Thanks,
Mark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: kunit alltests runs broken in mainline

[Johannes Berg]

On Mon, 2024-03-25 at 15:21 +0000, Mark Brown wrote:
> Hi,
> 
> Commit 28b3df1fe6ba2cb4 ("kunit: add wireless unit tests") which I can't
> seem to find on lore breaks full kunit runs on non-UML builds and is now
> present in mainline.  If I run:
> 
>    ./tools/testing/kunit/kunit.py run --alltests --cross_compile x86_64-linux-gnu- --arch x86_64
> 
> on a clean tree then I get:
> 
>    [15:09:20] Configuring KUnit Kernel ...
>    Generating .config ...
>    Populating config with:
>    $ make ARCH=x86_64 O=.kunit olddefconfig CROSS_COMPILE=x86_64-linux-gnu-
>    ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
>    This is probably due to unsatisfied dependencies.
>    Missing: CONFIG_IWLWIFI=y, CONFIG_WLAN_VENDOR_INTEL=y
> 
> UML works fine, but other real architectures (eg, arm64) seem similarly
> broken.

Hmm, strange.

> I've not looked properly yet, I'm a bit confused given that
> there's not even any dependencies for WLAN_VENDOR_INTEL and it's not
> mentoned in the defconfig.

Well it's in the directory dependencies or something I think, this seems
to help:

--- a/tools/testing/kunit/configs/all_tests.config
+++ b/tools/testing/kunit/configs/all_tests.config
@@ -28,6 +28,8 @@ CONFIG_MCTP_FLOWS=y
 CONFIG_INET=y
 CONFIG_MPTCP=y
 
+CONFIG_NETDEVICES=y
+CONFIG_WLAN=y
 CONFIG_CFG80211=y
 CONFIG_MAC80211=y
 CONFIG_WLAN_VENDOR_INTEL=y


But I'm not sure why ARCH=um is different?

johannes

Re: kunit alltests runs broken in mainline

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 237 bytes --]

On Mon, Mar 25, 2024 at 04:29:53PM +0100, Johannes Berg wrote:

> But I'm not sure why ARCH=um is different?

It's probably something to do with it lacking a bunch of features of
normal architectures, especially around hardware support.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: kunit alltests runs broken in mainline

[Johannes Berg]

On Mon, 2024-03-25 at 15:55 +0000, Mark Brown wrote:
> On Mon, Mar 25, 2024 at 04:29:53PM +0100, Johannes Berg wrote:
> 
> > But I'm not sure why ARCH=um is different?
> 
> It's probably something to do with it lacking a bunch of features of
> normal architectures, especially around hardware support.

Feels though that should make it *more* likely to not have support for
some hardware driver like iwlwifi, not *less* :-)

Anyway, does it fix it for you as well? if yes, where/how should we get
that in?

johannes

Re: kunit alltests runs broken in mainline

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 357 bytes --]

On Mon, Mar 25, 2024 at 05:16:35PM +0100, Johannes Berg wrote:

> Anyway, does it fix it for you as well? if yes, where/how should we get
> that in?

Seems to.  Given the breakage I'd expect it to go to Linus urgently, I
don't know that it super matters exactly how but I guess net will be
sending a pull anyway and is the source of the breakage?  

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: kunit alltests runs broken in mainline

[Johannes Berg]

On Mon, 2024-03-25 at 16:23 +0000, Mark Brown wrote:
> On Mon, Mar 25, 2024 at 05:16:35PM +0100, Johannes Berg wrote:
> 
> > Anyway, does it fix it for you as well? if yes, where/how should we get
> > that in?
> 
> Seems to.  Given the breakage I'd expect it to go to Linus urgently, I
> don't know that it super matters exactly how but I guess net will be
> sending a pull anyway and is the source of the breakage?  

Hmm, yeah ... after the previous discussion I had just put the patch in
there directly in wireless, so I'll just add this fix too.

johannes

Re: kunit alltests runs broken in mainline

[Jakub Kicinski]

On Mon, 25 Mar 2024 15:21:33 +0000 Mark Brown wrote:
> Hi,
> 
> Commit 28b3df1fe6ba2cb4 ("kunit: add wireless unit tests") which I can't
> seem to find on lore breaks full kunit runs on non-UML builds and is now
> present in mainline.  If I run:
> 
>    ./tools/testing/kunit/kunit.py run --alltests --cross_compile x86_64-linux-gnu- --arch x86_64
> 
> on a clean tree then I get:
> 
>    [15:09:20] Configuring KUnit Kernel ...
>    Generating .config ...
>    Populating config with:
>    $ make ARCH=x86_64 O=.kunit olddefconfig CROSS_COMPILE=x86_64-linux-gnu-
>    ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config.
>    This is probably due to unsatisfied dependencies.
>    Missing: CONFIG_IWLWIFI=y, CONFIG_WLAN_VENDOR_INTEL=y
> 
> UML works fine, but other real architectures (eg, arm64) seem similarly
> broken.  I've not looked properly yet, I'm a bit confused given that
> there's not even any dependencies for WLAN_VENDOR_INTEL and it's not
> mentoned in the defconfig.

I'm late to the party, but FWIW I had to toss this into netdev testing
tree as a local patch:

CONFIG_NETDEVICES=y
CONFIG_WLAN=y
CONFIG_DAMON_DBGFS_DEPRECATED=y
https://github.com/linux-netdev/testing/commit/9a632301bf3dbc9ba553562a0ba2657af6fa84d1

We run using:

	./tools/testing/kunit/kunit.py run --alltests

it got broken after we fast-forwarded to Linus on Thu, so I assumed it
was kunit itself that changed. Not that I found the relevant commit.

The DAMON config was also breaking UML for us, BTW, and I don't see
any fix for that in Linus's tree. Strangeness.

Re: kunit alltests runs broken in mainline

[Johannes Berg]

On Tue, 2024-03-26 at 01:52 +0000, Jakub Kicinski wrote:
> 
> I'm late to the party, but FWIW I had to toss this into netdev testing
> tree as a local patch:
> 
> CONFIG_NETDEVICES=y
> CONFIG_WLAN=y

I'll send this in the next wireless pull, soon.

> CONFIG_DAMON_DBGFS_DEPRECATED=y

> The DAMON config was also breaking UML for us, BTW, and I don't see
> any fix for that in Linus's tree. Strangeness.

I noticed that too (though didn't actually find the fix) against net-
next, wireless trees are still a bit behind. I guess it'll get fixed
eventually.

johannes

Re: kunit alltests runs broken in mainline

[David Gow]

On Tue, 26 Mar 2024 at 15:55, Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2024-03-26 at 01:52 +0000, Jakub Kicinski wrote:
> >
> > I'm late to the party, but FWIW I had to toss this into netdev testing
> > tree as a local patch:
> >
> > CONFIG_NETDEVICES=y
> > CONFIG_WLAN=y
>
> I'll send this in the next wireless pull, soon.
>
> > CONFIG_DAMON_DBGFS_DEPRECATED=y
>
> > The DAMON config was also breaking UML for us, BTW, and I don't see
> > any fix for that in Linus's tree. Strangeness.
>
> I noticed that too (though didn't actually find the fix) against net-
> next, wireless trees are still a bit behind. I guess it'll get fixed
> eventually.
>

+ Shuah, sj

Thanks for fixing this. I've sent out a fix (though I'm not 100% sure
it's the right one) to the DAMON issue here:
https://lore.kernel.org/linux-kselftest/20240326100740.178594-1-davidgow@google.com/

I don't think it'd conflict with the wireless fix, but if so, I'm
happy for them both to go in via KUnit if that's easier.

Sorry for the breakage!
-- David

Re: kunit alltests runs broken in mainline

[Shuah Khan]

On 3/26/24 04:09, David Gow wrote:
> On Tue, 26 Mar 2024 at 15:55, Johannes Berg <johannes@sipsolutions.net> wrote:
>>
>> On Tue, 2024-03-26 at 01:52 +0000, Jakub Kicinski wrote:
>>>
>>> I'm late to the party, but FWIW I had to toss this into netdev testing
>>> tree as a local patch:
>>>
>>> CONFIG_NETDEVICES=y
>>> CONFIG_WLAN=y
>>
>> I'll send this in the next wireless pull, soon.

You are welcome to send this with wireless pull if you like
or I can include it in my pull request.

Either way let me know:

Acked-by: Shuah Khan <skhan@linuxfoundation.org>

>>
>>> CONFIG_DAMON_DBGFS_DEPRECATED=y
>>
>>> The DAMON config was also breaking UML for us, BTW, and I don't see
>>> any fix for that in Linus's tree. Strangeness.
>>
>> I noticed that too (though didn't actually find the fix) against net-
>> next, wireless trees are still a bit behind. I guess it'll get fixed
>> eventually.
>>
> 
> + Shuah, sj
> 
> Thanks for fixing this. I've sent out a fix (though I'm not 100% sure
> it's the right one) to the DAMON issue here:
> https://lore.kernel.org/linux-kselftest/20240326100740.178594-1-davidgow@google.com/
> 

I applied this to linux-kselftest kunit-fixes branch

I am planning to send this up tomorrow.

> I don't think it'd conflict with the wireless fix, but if so, I'm
> happy for them both to go in via KUnit if that's easier.
> 

thanks,
-- Shuah

Re: kunit alltests runs broken in mainline

[Johannes Berg]

On Thu, 2024-03-28 at 14:08 -0600, Shuah Khan wrote:
> On 3/26/24 04:09, David Gow wrote:
> > On Tue, 26 Mar 2024 at 15:55, Johannes Berg <johannes@sipsolutions.net> wrote:
> > > 
> > > On Tue, 2024-03-26 at 01:52 +0000, Jakub Kicinski wrote:
> > > > 
> > > > I'm late to the party, but FWIW I had to toss this into netdev testing
> > > > tree as a local patch:
> > > > 
> > > > CONFIG_NETDEVICES=y
> > > > CONFIG_WLAN=y
> > > 
> > > I'll send this in the next wireless pull, soon.
> 
> You are welcome to send this with wireless pull if you like
> or I can include it in my pull request.

It's already in net now, and in the pull request to Linus here:
https://lore.kernel.org/netdev/20240328143117.26574-1-pabeni@redhat.com

johannes