Re: [PATCH 6/7] tools/rtla: Add test engine support for unexpected output

[Crystal Wood <crwood@redhat.com>]

On Thu, 2025-08-28 at 10:09 +0200, Tomas Glozar wrote:
> čt 21. 8. 2025 v 5:58 odesílatel Crystal Wood <crwood@redhat.com> napsal:
> > 
> > Simplify the code so that we can print failures as they happen rather
> > than trying to figure out what went wrong after printing "not ok".  This
> > also means that "not ok" gets printed after the info rather than before,
> > which seems more intuitive anyway.
> > 
> 
> This change is causing some formatting issues, e.g.:
> 
> tests/hwnoise.t ... 2/6 # Output match failed: "rtla hit stop tracing"
> # Error iterating on events
> # Oops, error disabling tracer
> #
> # exit code 1
> tests/hwnoise.t ... 3/6 not ok 3 - set the automatic trace mode
> 
> (the comment is on the same line as the previous test)

I'm not seeing the "tests/hwnoise.t ... 2/6" part.  What is printing
that?

> Furthermore, it's standard to first print "not ok" and then the
> comments, see the documentation to Test::More [1], so I think we
> should keep that.
> 
> [1] https://metacpan.org/pod/Test::More#ok

https://xkcd.com/927/ :-P

I'm not a Perler, so I didn't recognize it as anything standardized. 
Still seems backwards to me, both in terms of making it easier to see
which test failed, and in terms of being a pain to implement.  And we
don't even get the benefit of seeing the test name printed before it
runs, to make it easier to see what's taking a long time.

Seems like Linux uses a variant of this, though:
Documentation/dev-tools/ktap.rst

I'll fix it, hopefully by sticking the error output into a variable or
something rather than duplicating logic.  Or should we be using some
existing test infrastructure?

-Crystal