[TEST] google runner needs nested cases

[TEST] google runner needs nested cases

[Jakub Kicinski]

Hi!

Another ask, please parse the TAP output of the tests and provide 
a nested array of results per-subcase, see:

https://github.com/linux-netdev/nipa/wiki/Netdev-CI-system/#results

right now you only seem to report top level pass/fail but most tests
have at least one sub-case the driver doesn't support.

Re: [TEST] google runner needs nested cases

[Sheena Mohan]

Thanks, Jakub. We plan to merge the changes to support nested arrays
for sub-test results by next week.


On Tue, Jul 21, 2026 at 6:38 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Hi!
>
> Another ask, please parse the TAP output of the tests and provide
> a nested array of results per-subcase, see:
>
> https://github.com/linux-netdev/nipa/wiki/Netdev-CI-system/#results
>
> right now you only seem to report top level pass/fail but most tests
> have at least one sub-case the driver doesn't support.

Re: [TEST] google runner needs nested cases

[Jakub Kicinski]

On Thu, 23 Jul 2026 10:14:20 -0700 Sheena Mohan wrote:
> Thanks, Jakub. We plan to merge the changes to support nested arrays
> for sub-test results by next week.

SG, please ping again when you do, so I can check and close the request
on my side.

Re: [TEST] google runner needs nested cases

[Sheena Mohan]

Hello Jakub,

The recent updates have been merged, and the tests should now report
sub-case results.
Please let us know if any further improvements are needed.

Thanks,
Sheena

On Thu, Jul 23, 2026 at 10:36 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 23 Jul 2026 10:14:20 -0700 Sheena Mohan wrote:
> > Thanks, Jakub. We plan to merge the changes to support nested arrays
> > for sub-test results by next week.
>
> SG, please ping again when you do, so I can check and close the request
> on my side.

Re: [TEST] google runner needs nested cases

[Jakub Kicinski]

On Tue, 28 Jul 2026 10:07:15 -0700 Sheena Mohan wrote:
> The recent updates have been merged, and the tests should now report
> sub-case results.
> Please let us know if any further improvements are needed.

Looks good, thanks!

I'm still waiting on the device node, but I assume we're stacking that
separately.

Re: [TEST] google runner needs nested cases

[Sheena Mohan]

On Tue, Jul 28, 2026 at 3:13 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 28 Jul 2026 10:07:15 -0700 Sheena Mohan wrote:
> > The recent updates have been merged, and the tests should now report
> > sub-case results.
> > Please let us know if any further improvements are needed.
>
> Looks good, thanks!
>
> I'm still waiting on the device node, but I assume we're stacking that
> separately.
Yes, we are tracking that separately. I will share an update on
devlink support once the patches are submitted for upstream review.

Re: [TEST] google runner needs nested cases

[Jakub Kicinski]

On Tue, 28 Jul 2026 15:38:56 -0700 Sheena Mohan wrote:
> On Tue, Jul 28, 2026 at 3:13 PM Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > On Tue, 28 Jul 2026 10:07:15 -0700 Sheena Mohan wrote:  
> > > The recent updates have been merged, and the tests should now report
> > > sub-case results.
> > > Please let us know if any further improvements are needed.  
> >
> > Looks good, thanks!
> >
> > I'm still waiting on the device node, but I assume we're stacking that
> > separately.  
> Yes, we are tracking that separately. I will share an update on
> devlink support once the patches are submitted for upstream review.

To be clear -- it's not required to be real devlink.
The format has to be like devlink (JSON keys).
For QEMU virtio for example we have a two liner
bash script in NIPA:

qver=$(qemu-system-x86_64 --version | head -1)

echo '{"driver":"virtio_net","versions":{"fixed":{},"stored":{},"running":{"fw":"'"$qver"'"}}}'

https://github.com/linux-netdev/nipa/blob/main/contest/scripts/vm-virtio-dev-info.sh

Of course real devlink reporting is better because
it may also help users correlate problems in production.