Re: [ANN] Containers to run Netdev CI tests locally

[Matthieu Baerts <matttbe@kernel.org>]

Hi Hangbin,

On 04/09/2026 13:44, Hangbin Liu wrote:
> On Fri, Sep 04, 2026 at 10:33:18AM +0200, Matthieu Baerts wrote:
>> Hi Hangbin,
>>
>> On 04/09/2026 08:59, Hangbin Liu wrote:
>>> Hi Matthieu,
>>>
>>> On Thu, Sep 03, 2026 at 06:19:25PM +0200, Matthieu Baerts wrote:
>>>> Hello,
>>>>
>>>> TL;DR: containers are now available with all dependencies to easily
>>>> execute all static, unit, and functional tests in an environment closed
>>>> to the Netdev CI one:
>>>>
>>>> - https://github.com/orgs/linux-netdev/packages
>>>>
>>>> - https://github.com/linux-netdev/nipa/wiki/Running-Netdev-CI-tests-locally
>>>
>>> Great! I've been struggling to set up the local NIPA docker. I'm following
>>> the steps from your git commit description. Now this article saved me.
>>
>> I'm glad it helped!
>>
> 
> From a user‑experience perspective, I think the static tests work well.
> They display the check results clearly and save outputs to the result directory.
> 
> However, for selftests, I do not see any difference compared to running vng
> directly on my local machine, i.e.
> 
>   vng -v --run . --user root -a mitigations=off --cpus 4 -- \
>   	make -C tools/testing/selftests TARGETS=net run_tests
> 
> Do you know how to make the result looks like the CI shows[1], that each test
> has a separate log dir and shows stdout/stderr.

Mmh, I understand the issue, but that's mainly kselftests related I think.

We could reuse python scripts from the NIPA repo to generate .json files
that will need to be processed to be presented in a local HTML page. But
that seems overcomplicated to put in place. At the end, the 'run_tests'
command will output a TAP format, which can be parsed with different
consumers [1] (e.g. tappy) or a simple "grep" command, e.g.

  grep -e "^not ok" -e "ok .* # SKIP"

An alternative I suggested a few months ago [2], is to print a summary
at the end of the "run_tests" command, e.g.

  # Totals: pass:1 fail:1 xfail:0 xpass:0 skip:1 error:0

Can you try it on your side to see if it improves your user experience?

Maybe this simple patch should be taken in the net-next tree. Often,
patches modifying the kselftest framework have to be taken via a
different tree I think...

[1] https://testanything.org/consumers.html
[2]
https://lore.kernel.org/20260625-sft-run-tests-print-totals-v1-1-bfe588266249@kernel.org

Cheers,
Matt