Re: [xen-unstable test] 65141: regressions - FAIL

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 02/12/15 10:52, Jan Beulich wrote:
>>>> On 02.12.15 at 11:34, <ian.campbell@citrix.com> wrote:
>> On Mon, 2015-11-30 at 05:35 +0000, Hu, Robert wrote:
>>
>> Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not
>> sure if there is a dedicated nested-vmx maintainer. This failure has been
>> blocking the xen-unstable push gate for a week now so it really does need
>> looking into.
>>
>> Also CC the arch X86 maintainers, for-their-info.
> I was actually waiting for the bisector to point at something. Also
> looking at the history this might be machine specific (the sole
> initial pass was on italia0, all failures are on godello1). Should new
> tests perhaps be capable of causing regressions only once they
> passed on every host they may (usefully) run on?

As a general improvement, OSSTest should not make new tests blocking by
default.  They should need to be proved stable (5 consecutive passes?)
before they become blocking, to prevent a new test spuriously passing
and subsequently blocking pushes.

During this time, the author of the new test has the onus to ensure test
stability; either modifications to the test, or bugfixes to master.

~Andrew