Re: [RFC net-next 0/6] selftests: net: add performance metric reporting

[Jakub Kicinski <kuba@kernel.org>]

Adding Sheena, Willem, Przemek, and Adrian, who I have in my notes 
as folks responsible for the Google and Intel NIPA runners.
(Please LMK if I should update my notes!)

On Wed, 16 Sep 2026 12:04:03 -0700 Stanislav Fomichev wrote:
> Sharing as an RFC to get the feedback on the overall approach. Current
> model is where the ktap side drives system monitoring and nipa only
> collects/interprets/draws. Another way we can do it is to move system
> monitoring stuff to nipa.
> 
> ktap-side monitoring pros/cons:
> - pro: same format for system vs test metrics (same parser, same
>   aggregator, etc)
> - pro: test controls when the system collection starts
> - pro: test controls what the system collects (say, on multi-nic
>   machines, we collect only the things that matter)
> - pro: local runs produce the same output (and we can add some tools to
>   produce the aggregates for local analysis)

I'm CCing folks from other runners, because I wonder how much it would
help them. I don't think lifting the code form NIPA would be a major
effort, with LLMs. But it is work..

> - con: bespoke metrics aggregation format defined/exported by the test (but
>   I think we still need it for non-system metrics regardless?)
> - con: extra code/complexity on ksft side (although it's only SystemMonitor,
>   the rest of this patch series still relevant)

That's a big one for me, hand up who is able to read all netdev@
traffic as is, this will be more emails and review :(

> nipa-side monitoring pros/cons:
> - pro: more code stays on nipa side

yes, which aligns with where runners are and we can fix things much
more quickly

> - pro: tests don't care about system side of things, it's always
>   collected on nipa side
> - pro: each nipa runner can define its own policy/thresholds (although,
>   why would we want it only for the system side and not the test side?)

pro: the system metrics require no semantic annotations for the UI,
since NIPA produces them it knows them

> - con: separate collection & parsing (system vs test)
> - con: local runs need nipa if we want to observe the metrics

On the code organization - should we not keep the perf tests
separate? 
 - the perf tests will probably need to run ~30 sec each
   and it's very easy to multiply the variants. I'm worried 
   we may need to sooner or later split them out and only run
   a portion of the tests for each branch if they take long?
 - makes no sense to run perf tests for debug kernels.
 - extra infra and history is required to make sense of results,
   no point running them during 95% of development..


That's my $.50, I don't love the ksft.py changes, but I'm open
to trying if others think this is the right move. So please share 
your opinions? :S