Re: [OSSTEST PATCH 2/2] make-flight: create the vNUMA HVM test job

[Ian Campbell <ian.campbell@citrix.com>]

On Tue, 2015-10-06 at 10:33 +0200, Dario Faggioli wrote:
> On Tue, 2015-10-06 at 09:23 +0100, Ian Campbell wrote:
> > On Mon, 2015-10-05 at 17:41 +0100, Wei Liu wrote:
> 
> > > We don't need to make ts-migrate-support-check fail. It is fine for
> > > the
> > > actual migration test to fail at the beginning as it won't block
> > > the
> > > push gate. It's conceivable that vNUMA guest will be able to
> > > migrate in
> > > the future. When that comes true, the actual migration test will
> > > pass.
> > 
> > I think the point was that if the migration tests fails then all
> > subsequent
> > test steps won't get run at all (apart from leak check & log
> > collection
> > etc).
> > 
> By "test steps" you mean things like other ts-* within the same (vNUMA)
> job? Or something different, e.g., other tests on the same host, etc?

Effectively rows in the results results chart
http://logs.test-lab.xenproject.org/osstest/logs/62609/test-amd64-amd64-xl/info.html
http://logs.test-lab.xenproject.org/osstest/logs/62609/test-amd64-amd64-xl-qemuu-debianhvm-amd64/info.html

Which to a first approximation correspond to the ts-* scripts except a
given ts-* might be used by multiple steps.

> If the former (which I think is the case), that's not really a big
> deal, as there are no other steps. :-)

AFAICT looking at your patch to make-flight the jobs you are adding are
using the test-debianhvm recipe (corresponding to run-job/test-debianhvm in
sg-run-job) which is the same as the debianhvm job linked above, which does
have steps after the migration one, specifically guest-start, guest-stop.2
and guest-start/debianhvm.repeat.

> > Whereas if ts-migrate-support-check fails then the migrations will be
> > skipped and those other tests will be run.
> > 
> The above being said, I wasn't sure how to procede myself. I went for
> this approach, following Wei's advice (on IRC), and I still think it's
> a valid one, in line with how new tests have been handled since now...
> unless there are downsides that I'm not seeing. For example, would the
> failure be sticky, i.e., this test will be kept on the same host,
> preventing other tests to run there?

Yes, although that is an issue which needs solving more generically (I
happened to be talking to Ian about it yesterday) and not something you
should worry about.

> In any case, I'd be fine with tweaking ts-saverestore-support-check
> (it's that one that fails, even before the migration-check one), just
> let me know what you prefer. :-)
> 
> Regards,
> Dario