Re: [OSSTEST Nested PATCH v11 6/7] Compose the main recipe of nested test job

[Robert Hu <robert.hu@vmm.sh.intel.com>]

On Fri, 2015-06-12 at 09:44 +0100, Ian Campbell wrote:
> On Fri, 2015-06-12 at 11:42 +0800, Robert Hu wrote:
> > Hi Ian J., because nested Xen isn't that matured at present; it is
> > currently 'tech preview' phase. We now just add some sanity test case to
> > defend current work fruits. We can add more complicated test cases later
> > as nested Xen development moves forward.
> 
> I don't think it needs doing as part of this series, but I do think it
> would be worth adding the standard suite of tests steps to L2 guests
> soon after.
Agree. We can separately add those part in a later patch series.  
> 
> The way osstest deals with test failures i.e. classifying them into "has
> always failed" vs. "regression" means that it is fine to add tests for
> features which aren't mature yet, since they will fall into the "has
> always failed" set and not block pushes. In fact it is in some sense
> good to do this since it gives a concrete set of (necessary but not
> necessarily sufficient) things which need to be fixed for the feature to
> reach maturity.
We would like this test job's failure to be marked 'regression', which
shall block related breaking code's commitment.
We can consider to contribute more code later, including proved/success
test cases (which will fall into regression as this case), and fail test
cases (which shall soon be fixed/developed).
> 
> Ian.
>