Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD <anthony.perard@citrix.com>]

On Fri, Jul 17, 2015 at 05:22:10PM +0100, Ian Campbell wrote:
> On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> 
> > I've introduce an extra Osstest::Toolstack which help to install extra
> > package, 
> 
> I've commented on this.
> 
> > and use ballonning for Dom0, 500MB for Dom0 is definetly not
> > enough.
> 
> This is for overriding Dom0MemFixed I think?

Yes.

> I think going to ballooning is a bit too far, I think instead the actual
> size of dom0 ought to be something which can be overridden via a runvar,
> so that the openstack tests can ask for a bigger one.

I will look into that.

> > The ts-devstack script does prepare a bit more the host, clone devstack,
> > then run ./stack.sh, which is a bit like raisin. Once the machine ready,
> > the integration test suite from OpenStack, Tempest, is started. Do you
> > think those two step should be in separate test, one for devstack, and one
> > for Tempest?
> 
> I don't really understand the difference between them well enough, but I
> would say if they are separate things then different steps would be good
> -- since if nothing else it gives a clearer indication in the test
> report what had failed.

I'll seperated them in different step.

> >   I have not done it, but we could have some smoke test before
> > Tempest where osstest tryied to start a guest.
> 
> A test can have a dependency on a build job, but I'm not sure about
> another test, but that would seem generally useful and would allow your
> new test to depend on test-ARCH-ARCH-libvirt.

I was thinking of an extra steps within the test which would just start a
guest via OpenStack/Nova before running the full test suite from Tempest.

> However I think we don't actually want to do that:
> 
> For the openstack branch we will be using known good versions of all the
> other branches, so that test-ARCH-ARCH-libvirt will have happened
> already (when the libvirt branch got pushed).
> 
> For the other branches then we already have loads of tests which might
> all fail for the same reason which you could serialise to prevent this,
> but it would make the whole flight take much longer and it isn't really
> needed.
> 
> > Then later, there will be the question of which tree to track, devstack?
> > nova? Or don't track any and just test with the master branch from time to
> > time.
> 
> This is a complicated one, especially for things which don't fit into
> the "one tree one branch" model of things...
> 
> In terms of gating (which matters to us for regression tracking even if
> we don't care about the output of the gate) is it the case that if you
> clone $rootthing (whatever that is) and select a given revision of that
> you will always get the same thing?

> Or is it like raisin where you clone $rootthing and select a revision of
> that and it will clone the latest version of everything at that time?
> 
> I'm expecting the second one?

Yes, I think your description of raisin would match the description of
devstack, the $rootthing.
So, devstack will clone master of every other tree by default. But we
can select a specific revision for everything that is going to be cloned.

Also, if one clone a stable/* branch of devstack, then we'll get the same
stable branch of every other trees.

> I think the important thing is we would want to be testing stuff which
> has already gone through openstack testing?

Yes. Everything in OpenStack trees have been tested and have gone through
the gate anyway.

> Maybe we want to track particular OStack releases?

I think tracking master at first would be fine.

-- 
Anthony PERARD