Re: [LTP] LTP in valgrind :)

[Cyril Hrubis <chrubis@suse.cz>]

Hi!
> > > > > It also opens some interesting questions, i.e. how do we make comparing
> > > > > results from two different tests easier. Currently they grep the test
> > > > > results for a summary, but maybe we can do better.
> >
> > > > One option is to extract all TPASS/TFAIL/TWARN/TBROK/TCONF messages,
> > > > discard any message contents past the file:line header and then compare
> > > > whether the sanitized output is identical. That'll take care of random
> > > > values in the output while ensuring that the test went through the same
> > > > code paths as before. We could provide a sanitizer script for that.
> >
> > > Maybe we can even add an option to the test library to supress the
> > > messages in output, that would be fairly simple.
> >
> > @Martin @Mark: feel free to comment what we can do for you :).
> > Whole thread:
> > https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#t
> 
> Nice to see you noticed!  Primarily we're trying to identify
> valgrind gaps and bugs.  For our use-case it's good to have
> reproducible test runs in a sense of identical test results for
> repeated test runs.  We got pretty close to this by only taking
> into account the testcase summaries at the very end of the test
> logs.  Any steps to get closer to reproducible test runs on the
> LTP side are interesting for us, and we'll try to reflect those!

I've proposed a simple solution that would cut the message from tests
only to contain the line numbers and results in:

https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#m98317262b25ede62704b005fcefc63c62198bb90

If that works for you we can get this in LTP before the next release.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp