Re: kernelci test results

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2026-09-04 09:44:31 [+0100], Ben Copeland wrote:
> Hi Sebastian,
Hi Ben,

> >  > > Later there is the cyclictest invocation, arguments and the result.
> >  > > There is no max-value set for the test so it passes. Would there be a
> >  > > notification if the test failed and if so, where?
> 
> I cannot find any thresholds set, which isn't ideal.
> parse_rt_tests_results.py prints the literal string "pass" for every
> latency line; only cyclictest's return code decides the overall
> result. So a max-value parameter on our side would change nothing
> until the parser learns about thresholds.

Regarding the thresholds. Assuming we agree on a default threshold of
100us.

There is
   test-definitions/automated/lib/parse_rt_tests_results.py

which has this piece:
|     if int(rawdata["return_code"]) == 0:
|         print("{} pass".format(testname))
|     else:
|         print("{} fail".format(testname))

Would that be the right place for the decision (test passed vs failed)?
If so, we could invoke cyclictest with the '-b' argument passing the max
value for the expected latency and then cyclictest would terminate if
the value is reached with the exit code 2. This is however not entirely
possible with the current binary, there is an outstanding patch
	https://lore.kernel.org/linux-rt-users/20260824101852.2237494-1-costa.shul@redhat.com/

to be merged. But other than that… :)

> Ben

Sebastian