Re: [PATCH] boost: add ptest support

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2018-08-27 at 20:57 -0400, Randy MacLeod wrote:
> On 08/27/2018 06:17 PM, Yang Wang wrote:
> > Not sure if it's worth to run the Ptest on QEMU though, I also run
> > Ptest 
> > on SIMICS simulators, thousands of tests didn't get run, looks like
> > the 
> > result was not good as well.
> > 
> > Now my nightly Ptest runs on x86 device and gets consistent result
> > every 
> > day:
> > 
> >      2018-08-27T06:26 -- 2018-08-27T09:52
> >      Passed:  40518
> >      Failed:  289
> >      Skipped: 1876
> 
> Consistent results are good and > 90% pass rate is very good.
> What are the stats using qemux86-64 and/or simics?
> 
> I don't expect that qemu results would be as close to real hardware
> as Simics but it is quite good and freely available.

I think this is an example of where we may need to start adding "stage
2" testing to the autobuilder. The first stage tests would be enough to
get patches merged and be the faster ones, the second stage would be
the longer running things which we'd only trigger when stage one had
passed, maybe on milestone releases.

We'd put off doing this due to the old creaking codebase. With the new
autobuilder codebase, this should be much more straightforward to do...

boost ptest would be something I'd put in the second stage.

Cheers,

Richard