Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Campbell <ian.campbell@citrix.com>]

On Mon, 2015-07-20 at 16:07 +0100, Anthony PERARD wrote:
> > >   I have not done it, but we could have some smoke test before
> > > Tempest where osstest tryied to start a guest.
> > 
> > A test can have a dependency on a build job, but I'm not sure about
> > another test, but that would seem generally useful and would allow your
> > new test to depend on test-ARCH-ARCH-libvirt.
> 
> I was thinking of an extra steps within the test which would just start a
> guest via OpenStack/Nova before running the full test suite from Tempest.

ISWYM.

That might be nice, if it were very little osstest-development effort to
make it happen, but I think it isn't going to be a small effort and
running tempest and seeing what happens seems like it would be good
enough anyway.

> > > Then later, there will be the question of which tree to track, devstack?
> > > nova? Or don't track any and just test with the master branch from time to
> > > time.
> > 
> > This is a complicated one, especially for things which don't fit into
> > the "one tree one branch" model of things...
> > 
> > In terms of gating (which matters to us for regression tracking even if
> > we don't care about the output of the gate) is it the case that if you
> > clone $rootthing (whatever that is) and select a given revision of that
> > you will always get the same thing?
> 
> > Or is it like raisin where you clone $rootthing and select a revision of
> > that and it will clone the latest version of everything at that time?
> > 
> > I'm expecting the second one?
> 
> Yes, I think your description of raisin would match the description of
> devstack, the $rootthing.
> So, devstack will clone master of every other tree by default. But we
> can select a specific revision for everything that is going to be cloned.
> 
> Also, if one clone a stable/* branch of devstack, then we'll get the same
> stable branch of every other trees.

But in the presence of stable backports we may not get the exact same
set of commits from one day to the next, despite cloning the exact same
devstack revision, I think?

> > I think the important thing is we would want to be testing stuff which
> > has already gone through openstack testing?
> 
> Yes. Everything in OpenStack trees have been tested and have gone through
> the gate anyway.
> 
> > Maybe we want to track particular OStack releases?
> 
> I think tracking master at first would be fine.

Right.