[PATCH OSSTEST v3 0/3] Have OpenStack tested on top of xen's master and libvirt's master.

[PATCH OSSTEST v3 0/3] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

Hi,

I have looked into getting OpenStack been tested on the latest Xen via
osstest.

The ts-openstack-deploy script does prepare a bit more the host, clone
devstack and other OpenStack trees, then run ./stack.sh, which is a bit
like raisin and deploy OpenStack on the host. Once the machine is ready,
the integration test suite from OpenStack, Tempest, is started by
ts-openstack-tempest.

For the last patch that create a flight plan, I've tested only with
`./standalone make-flight openstack` and looked into the database to check
that only the necessary build job and the only test job are there.

Thanks.

Change in V3:
  - Track Nova tree instead of devstack.
      Nova is the service we care about from a Xen point of view.
      Also it is updated much more often than devstack.
  - Cleanups, see change log in patches.

Changes in V2:
  - no more Osstest::Toolstack::OpenStack.
  - osstest now clone every single tree that devstack is going to need.
    And ./stack.sh should fail if one tree is missing.
  - avoid build-*-xsm for an openstack flight
  - rename ts-devstack to ts-openstack-devstack
  - New test script ts-openstack-tempest
  - Add CONFIG_CRYPTO_XTS=m to the kernel build.
  - new possible runvar $dom0_mem to control dom0 memory
  - several fix to have volume tests working.
  - have OpenStack deploy from it's builddir instead of /opt/stack
  - have 4GB for dom0 instead of relying on balloning.

Anthony PERARD (3):
  ts-openstack-deploy: Deploy OpenStack on a host with devstack
  ts-openstack-tempest: Run Tempest to check OpenStack
  Create a flight to test OpenStack with xen-unstable and libvirt

 ap-common            |  12 ++
 ap-fetch-version     |   4 +
 ap-fetch-version-old |   5 +
 ap-print-url         |   3 +
 ap-push              |   5 +
 cr-daily-branch      |   8 ++
 cr-for-branches      |   2 +-
 cri-common           |   1 +
 make-flight          |  34 ++++++
 mfi-common           |   5 +
 sg-run-job           |   6 +
 ts-openstack-deploy  | 304 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ts-openstack-tempest |  53 +++++++++
 13 files changed, 441 insertions(+), 1 deletion(-)
 create mode 100755 ts-openstack-deploy
 create mode 100755 ts-openstack-tempest

-- 
Anthony PERARD

[PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Anthony PERARD]

This script installs any necessary packages and clones all of the OpenStack
trees which are used by devstack to deploy OpenStack.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

---
Change in V3:
- Use host as argument to run the job.
- Use selectjob() and get rid of the unused $gho.
- Use target_jobdir() instead of builddirsprops().
- Remove GIT_BASE from devstack config.
- Rename the script to ts-openstack-deploy (from ts-openstack-devstack).
---
 sg-run-job          |   5 +
 ts-openstack-deploy | 304 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 309 insertions(+)
 create mode 100755 ts-openstack-deploy

diff --git a/sg-run-job b/sg-run-job
index 591ac54..4f87f30 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -373,6 +373,11 @@ proc run-job/test-rumpuserxen {} {
                  ts-guest-destroy-hard            host   $g   +
 }
 
+proc need-hosts/test-devstack {} { return host }
+proc run-job/test-devstack {} {
+  run-ts . = ts-openstack-deploy host
+}
+
 if {[file exists sg-run-job-adhoc]} {
     source sg-run-job-adhoc
 }
diff --git a/ts-openstack-deploy b/ts-openstack-deploy
new file mode 100755
index 0000000..25a83fa
--- /dev/null
+++ b/ts-openstack-deploy
@@ -0,0 +1,304 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tgt_init ();
+
+sub packages () {
+  # Install open-iscsi ahead of devstack ...
+  target_install_packages($ho, qw(git sudo open-iscsi));
+  # ... and start open-iscsi to have /etc/iscsi/initiatorname.iscsi
+  # generated. This is done on install on Ubuntu.
+  target_cmd_root($ho, 'service open-iscsi start');
+}
+
+sub checkout () {
+  prepbuilddirs();
+  build_clone($ho, 'cinder', $builddir, 'cinder');
+  build_clone($ho, 'glance', $builddir, 'glance');
+  build_clone($ho, 'keystone', $builddir, 'keystone');
+  build_clone($ho, 'nova', $builddir, 'nova');
+  build_clone($ho, 'requirements', $builddir, 'requirements');
+  build_clone($ho, 'tempest', $builddir, 'tempest');
+
+  build_clone($ho, 'devstack', $builddir, 'devstack');
+  my $vg = target_choose_vg($ho, 10*1024); # 10GB
+  target_putfilecontents_stash($ho, 60, <<END, $builddir.'/devstack/local.conf');
+[[local|localrc]]
+# Everything should be cloned by osstest, so devstack don't have to do it
+ERROR_ON_CLONE=True
+USE_SCREEN=False
+ADMIN_PASSWORD=secretadmin
+DATABASE_PASSWORD=secretdatabase
+RABBIT_PASSWORD=secretrabbit
+SERVICE_PASSWORD=secretservice
+SERVICE_TOKEN=atokenserviced
+# make it small because there is no way to not have this lvm volume created
+VOLUME_BACKING_FILE_SIZE=500M
+DEST=$builddir
+LOGFILE=\$DEST/logs/stack.sh.log
+# stackrc set this but don't take \$DEST into account
+SERVICE_DIR=\${DEST}/status
+LOG_COLOR=False
+LIBVIRT_TYPE=xen
+disable_service horizon
+disable_service n-novnc
+enable_service n-obj
+[[post-config|\$CINDER_CONF]]
+[lvmdriver-1]
+volume_group = $vg
+END
+
+  # libvirt is already installed, but not as a package, so avoid installation of
+  # the libvirt package with devstack
+  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
+      while (<EI>) {
+        next if m/.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+  target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions-libvirt", sub {
+      while (<EI>) {
+        next if m/install_package.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+
+  # OpenStack needs access to libvirt from a user.
+  target_cmd_root($ho, <<END);
+    cat >> /etc/libvirt/libvirtd.conf <<EOF
+unix_sock_group = "libvirt"
+unix_sock_ro_perms = "0777"
+unix_sock_rw_perms = "0770"
+EOF
+END
+
+  # devstack is going to setup the host, install some dependency.
+  target_putfilecontents_root_stash($ho, 100, <<END,"/etc/sudoers.d/devstack");
+osstest ALL=(ALL) NOPASSWD:ALL
+END
+
+  target_putfilecontents_root_stash($ho, 60, tgt_init(), "/etc/init.d/tgt");
+  target_cmd_root($ho, <<END);
+    chmod +x /etc/init.d/tgt
+END
+}
+
+sub deploy() {
+  target_cmd($ho, <<END, 1800);
+    cd $builddir/devstack
+    ./stack.sh
+END
+}
+
+packages();
+checkout();
+deploy();
+
+# This is missing from Debian but required by devstack
+# Got it from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577925
+sub tgt_init () {
+  return <<'END';
+#!/bin/bash
+### BEGIN INIT INFO
+# Provides:          tgtd
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Should-Start:      zfs
+# Should-Stop:       zfs
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: iscsi target daemon
+# Description:       iscsi target daemon
+### END INIT INFO
+
+DESC="target framework daemon"
+NAME=tgtd
+DAEMON=/usr/sbin/${NAME}
+
+TGTD_CONFIG=/etc/tgt/targets.conf
+
+TASK=$1
+
+. /lib/lsb/init-functions
+
+[ -x $DAEMON ] || exit 0
+
+start()
+{
+        log_daemon_msg "Starting $DESC" "$NAME"
+        # Start tgtd first.
+        tgtd &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                exit 1
+        else
+                log_end_msg 0
+        fi
+        # Put tgtd into "offline" state until all the targets are configured.
+        # We don't want initiators to (re)connect and fail the connection
+        # if it's not ready.
+        tgtadm --op update --mode sys --name State -v offline
+        # Configure the targets.
+        tgt-admin -e -c $TGTD_CONFIG
+        # Put tgtd into "ready" state.
+        tgtadm --op update --mode sys --name State -v ready
+}
+
+stop()
+{
+        if [ "$RUNLEVEL" == 0 -o "$RUNLEVEL" == 6 ] ; then
+            forcedstop
+        fi
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        # Remove all targets. It only removes targets which are not in use.
+        tgt-admin --update ALL -c /dev/null &>/dev/null
+        # tgtd will exit if all targets were removed
+        tgtadm --op delete --mode system &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                if [ "$TASK" != "restart" ] ; then
+                        log_end_msg 1
+                        exit 1
+                else
+                        log_end_msg 0
+                fi
+        elif [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                echo "Some initiators are still connected - could not stop tgtd"
+                exit 2
+        else
+                log_end_msg 0
+        fi
+        echo -n
+}
+
+forcedstop()
+{
+        # NOTE: Forced shutdown of the iscsi target may cause data corruption
+        # for initiators that are connected.
+        echo "Force-stopping target framework daemon"
+        # Offline everything first. May be needed if we're rebooting, but
+        # expect the initiators to reconnect cleanly when we boot again
+        # (i.e. we don't want them to reconnect to a tgtd which is still
+        # working, but the target is gone).
+        tgtadm --op update --mode sys --name State -v offline &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+            echo "tgtd is not running"
+            [ "$TASK" != "restart" ] && exit 1
+        else
+            tgt-admin --offline ALL
+            # Remove all targets, even if they are still in use.
+            tgt-admin --update ALL -c /dev/null -f
+            # It will shut down tgtd only after all targets were removed.
+            tgtadm --op delete --mode system
+            RETVAL=$?
+            if [ "$RETVAL" -ne 0 ] ; then
+                echo "Failed to shutdown tgtd"
+                exit 1
+            fi
+        fi
+        echo -n
+}
+
+reload()
+{
+        log_daemon_msg "Reloading configuration of $DESC" "$NAME"
+        # Update configuration for targets. Only targets which
+        # are not in use will be updated.
+        tgt-admin --update ALL -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        fi
+        log_end_msg 0
+}
+
+forcedreload()
+{
+        log_daemon_msg "Forced-reload configuration of $DESC" "$NAME"
+        # Update configuration for targets, even those in use.
+        tgt-admin --update ALL -f -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        else
+                log_end_msg 0
+        fi
+}
+
+status()
+{
+        # Don't name this script "tgtd"...
+        TGTD_PROC=$(ps -C tgtd | grep -c tgtd)
+        if [ "$TGTD_PROC" -eq 2 ] ; then
+            echo "tgtd is running. Run 'tgt-admin -s' to see detailed target info."
+        else
+            echo "tgtd is NOT running."
+        fi
+}
+
+case $1 in
+        start)
+                start
+                ;;
+        stop)
+                stop
+                ;;
+        forcedstop)
+                forcedstop
+                ;;
+        restart)
+                TASK=restart
+                stop && start
+                ;;
+        forcedrestart)
+                TASK=restart
+                forcedstop && start
+                ;;
+        reload)
+                reload
+                ;;
+        force-reload)
+                forcedreload
+                ;;
+        status)
+                status
+                ;;
+        *)
+                echo "Usage: $0 {start|stop|forcedstop|restart|forcedrestart|reload|force-reload|status}"
+                exit 2
+                ;;
+esac
+END
+}
-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Campbell]

On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> This script installs any necessary packages and clones all of the
> OpenStack
> trees which are used by devstack to deploy OpenStack.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

This mostly looks good to me. A few comments.

> +  # libvirt is already installed, but not as a package, so avoid
> installation of
> +  # the libvirt package with devstack
> +  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
> +      while (<EI>) {
> +        next if m/.*libvirt.*/;
> +        print EO or die $!;
> +      }
> +  });
> +  target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions
> -libvirt", sub {
> +      while (<EI>) {
> +        next if m/install_package.*libvirt.*/;
> +        print EO or die $!;
> +      }
> +  });

Do these end up causing the built_revision_foo to say "-dirty" or anything
like that?

Maybe not because I don't see any calls to store_vcs_revision, which I
think we discussed on an earlier revision?

> +  # OpenStack needs access to libvirt from a user.
> +  target_cmd_root($ho, <<END);

You could use target_putfilecontents_root_stash for this too I think?

Ian.

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Anthony PERARD]

On Tue, Sep 29, 2015 at 04:34:44PM +0100, Ian Campbell wrote:
> On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > This script installs any necessary packages and clones all of the
> > OpenStack
> > trees which are used by devstack to deploy OpenStack.
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> This mostly looks good to me. A few comments.
> 
> > +  # libvirt is already installed, but not as a package, so avoid
> > installation of
> > +  # the libvirt package with devstack
> > +  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
> > +      while (<EI>) {
> > +        next if m/.*libvirt.*/;
> > +        print EO or die $!;
> > +      }
> > +  });
> > +  target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions
> > -libvirt", sub {
> > +      while (<EI>) {
> > +        next if m/install_package.*libvirt.*/;
> > +        print EO or die $!;
> > +      }
> > +  });
> 
> Do these end up causing the built_revision_foo to say "-dirty" or anything
> like that?
> 
> Maybe not because I don't see any calls to store_vcs_revision, which I
> think we discussed on an earlier revision?

I'm only using build_clone() from osstest to clone any git tree. devstack
is not allowed to clone anything (because of ERROR_ON_CLONE=True in its
config file). So I don't need to call store_vcs_revision, and the tree will
not be seen as -dirty from osstest.

> > +  # OpenStack needs access to libvirt from a user.
> > +  target_cmd_root($ho, <<END);
> 
> You could use target_putfilecontents_root_stash for this too I think?

Well, this append things to the config files, and we thought bash HEREDOC was
"the right answer" ( <1437402707.17368.39.camel@citrix.com> ).

The next line of the patch is:
> +    cat >> /etc/libvirt/libvirtd.conf <<EOF

Thanks,

-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Jackson]

Anthony PERARD writes ("Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack"):
> On Tue, Sep 29, 2015 at 04:34:44PM +0100, Ian Campbell wrote:
> > Maybe not because I don't see any calls to store_vcs_revision, which I
> > think we discussed on an earlier revision?
> 
> I'm only using build_clone() from osstest to clone any git tree. devstack
> is not allowed to clone anything (because of ERROR_ON_CLONE=True in its
> config file). So I don't need to call store_vcs_revision, and the tree will
> not be seen as -dirty from osstest.

Oh, I see.  Sorry, yes.  build_clone calls store_vcs_revision itself.

Ian.

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Campbell]

On Tue, 2015-09-29 at 17:19 +0100, Anthony PERARD wrote:
> On Tue, Sep 29, 2015 at 04:34:44PM +0100, Ian Campbell wrote:
> > On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > > This script installs any necessary packages and clones all of the
> > > OpenStack
> > > trees which are used by devstack to deploy OpenStack.
> > > 
> > > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > 
> > This mostly looks good to me. A few comments.
> > 
> > > +  # libvirt is already installed, but not as a package, so avoid
> > > installation of
> > > +  # the libvirt package with devstack
> > > +  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
> > > +      while (<EI>) {
> > > +        next if m/.*libvirt.*/;
> > > +        print EO or die $!;
> > > +      }
> > > +  });
> > > +  target_editfile($ho,
> > > "$builddir/devstack/lib/nova_plugins/functions
> > > -libvirt", sub {
> > > +      while (<EI>) {
> > > +        next if m/install_package.*libvirt.*/;
> > > +        print EO or die $!;
> > > +      }
> > > +  });
> > 
> > Do these end up causing the built_revision_foo to say "-dirty" or
> > anything
> > like that?
> > 
> > Maybe not because I don't see any calls to store_vcs_revision, which I
> > think we discussed on an earlier revision?
> 
> I'm only using build_clone() from osstest to clone any git tree. devstack
> is not allowed to clone anything (because of ERROR_ON_CLONE=True in its
> config file). So I don't need to call store_vcs_revision, and the tree
> will
> not be seen as -dirty from osstest.

I'd forgotten that build_clone also called store_vcs_clone, sorry.


> > +  # OpenStack needs access to libvirt from a user.
> > > +  target_cmd_root($ho, <<END);
> > 
> > You could use target_putfilecontents_root_stash for this too I think?
> 
> Well, this append things to the config files, and we thought bash HEREDOC
> was
> "the right answer" ( <1437402707.17368.39.camel@citrix.com> ).
> 
> The next line of the patch is:
> > +    cat >> /etc/libvirt/libvirtd.conf <<EOF

Ah yes, I'd forgotten about that, sorry.

So given that I think what I meant to say last time was:

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack"):
> On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > This script installs any necessary packages and clones all of the
> > OpenStack
> > trees which are used by devstack to deploy OpenStack.
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> This mostly looks good to me. A few comments.
...
> Do these end up causing the built_revision_foo to say "-dirty" or anything
> like that?

I imagine they would.

I think it might be necessary to extend store_vcs_revision to be able
to pass some kind of ignore list, or perhaps to restore the un-edited
files from the stash, or something.

> Maybe not because I don't see any calls to store_vcs_revision, which I
> think we discussed on an earlier revision?

This is indeed needed I think.

Anthony, do you want tests to be subject to bisection ?

Ian.

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Campbell]

On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> +  # OpenStack needs access to libvirt from a user.
> +  target_cmd_root($ho, <<END);
> +    cat >> /etc/libvirt/libvirtd.conf <<EOF
> +unix_sock_group = "libvirt"
> +unix_sock_ro_perms = "0777"
> +unix_sock_rw_perms = "0770"
> +EOF
> +END

One small nit on this, which is that it isn't idempotent, i.e. running this
step on the same host repeatedly (common in standalone mode) will keep
adding those lines to the conf file.

If you care then possible solutions:

Use target_editfile_root() to filter the lines out and then re-add them.
See setupboot_grub2  and the handling of GRUB_ENABLE_XEN_UEFI_CHAINLOAD

Use grep (3 times?) to decide if you need to append the given line.

Modify the file in the dist file in ts-libvirt-build so it applies to all
test cases and not just this one.

Ian.

Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST v3 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack"):
> On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > +  # OpenStack needs access to libvirt from a user.
> > +  target_cmd_root($ho, <<END);
> > +    cat >> /etc/libvirt/libvirtd.conf <<EOF
> > +unix_sock_group = "libvirt"
> > +unix_sock_ro_perms = "0777"
> > +unix_sock_rw_perms = "0770"
> > +EOF
> > +END
> 
> One small nit on this, which is that it isn't idempotent, i.e. running this
> step on the same host repeatedly (common in standalone mode) will keep
> adding those lines to the conf file.
> 
> If you care then possible solutions:

I agree with Ian C that this is a nit.  However, provided that libvirt
doesn't mind if the lines are duplicated, I think this is tolerable.
It will only occur in by-hand testing.

Ian.

[PATCH OSSTEST v3 2/3] ts-openstack-tempest: Run Tempest to check OpenStack

[Anthony PERARD]

This script runs the OpenStack integration test suite, Tempest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

---
Change in V3:
- Use host as argument of the script.
- Use selecthost() and get rid of $gho.
- Use target_jobdir() instead of builddirsprops().
- Put the ignored Tempest tests into a var and try to explain why there are
  skip.
---
 sg-run-job           |  1 +
 ts-openstack-tempest | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
 create mode 100755 ts-openstack-tempest

diff --git a/sg-run-job b/sg-run-job
index 4f87f30..cb48b2b 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -376,6 +376,7 @@ proc run-job/test-rumpuserxen {} {
 proc need-hosts/test-devstack {} { return host }
 proc run-job/test-devstack {} {
   run-ts . = ts-openstack-deploy host
+  run-ts . = ts-openstack-tempest host
 }
 
 if {[file exists sg-run-job-adhoc]} {
diff --git a/ts-openstack-tempest b/ts-openstack-tempest
new file mode 100755
index 0000000..e2266d3
--- /dev/null
+++ b/ts-openstack-tempest
@@ -0,0 +1,53 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tempest() {
+  my $ignored_tests = '';
+
+  # Ignore these tests:
+  # tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
+  #   It try to start a guest with /dev/vda as boot device name.
+  $ignored_tests .= '|.*TestVolumeBootPattern.test_volume_boot_pattern';
+  # tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance
+  #   It try to start a guest with /dev/vda as boot device name.
+  $ignored_tests .= '|.*TestShelveInstance.test_shelve_volume_backed_instance';
+  # scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
+  #   It access a volume through iSCSI. This does not work when both the
+  #   server and client of iSCSI are on the same Xen host, Linux 4.0 is the
+  #   first Linux to have a fix.
+  $ignored_tests .= '|.*TestVolumeBootPatternV2.test_volume_boot_pattern';
+
+  # Adding the tests to skip to the default regex.
+  my $regex = "(?!.*\\[.*\\bslow\\b.*\\]$ignored_tests)(^tempest\\.(api|scenario|thirdparty))";
+
+  target_cmd($ho, <<END, 7200);
+    $builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+END
+}
+
+tempest();
-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 2/3] ts-openstack-tempest: Run Tempest to check OpenStack

[Ian Campbell]

On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:

> +  # Ignore these tests:
> +  # tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
> +  #   It try to start a guest with /dev/vda as boot device name.
> +  $ignored_tests .= '|.*TestVolumeBootPattern.test_volume_boot_pattern';

Don't you need to escape the second "." for the regex context in the
tempest script? \Q and \E might be helpful for this?

Also, you actually ignore any test ending with that string, not just the
exact one in the comment, is that deliberate? Also any test with a longer
name which has this substring will also be ignored.

If not then you could put the common prefix into a variable to avoid
repeating it (and the very long lines) and maybe add some anchors?

You might also want to consider

    push @ignored_tests, "^$prefix\QTestVolumeBootPattern.test_volume_boot_pattern$";

Then later:
    $ignored_tests = join("|", @ignored_tests);

(I'm not sure, but I don't think you want qr{} [0] here, I might be wrong
though)

> +  # tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance
> +  #   It try to start a guest with /dev/vda as boot device name.
> +  $ignored_tests .= '|.*TestShelveInstance.test_shelve_volume_backed_instance';
> +  # scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
> +  #   It access a volume through iSCSI. This does not work when both the
> +  #   server and client of iSCSI are on the same Xen host, Linux 4.0 is the
> +  #   first Linux to have a fix.
> +  $ignored_tests .= '|.*TestVolumeBootPatternV2.test_volume_boot_pattern';
> +
> +  # Adding the tests to skip to the default regex.
> +  my $regex = "(?!.*\\[.*\\bslow\\b.*\\]$ignored_tests)(^tempest\\.(api|scenario|thirdparty))";

Where the default regex is:
    "(?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty))";
?

This is a pretty rubbish interface which tempest has given you. What a
pain!

Ian

[0] http://perldoc.perl.org/perlop.html#Regexp-Quote-Like-Operators

Re: [PATCH OSSTEST v3 2/3] ts-openstack-tempest: Run Tempest to check OpenStack

[Anthony PERARD]

On Tue, Sep 29, 2015 at 04:43:50PM +0100, Ian Campbell wrote:
> On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> 
> > +  # Ignore these tests:
> > +  # tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
> > +  #   It try to start a guest with /dev/vda as boot device name.
> > +  $ignored_tests .= '|.*TestVolumeBootPattern.test_volume_boot_pattern';
> 
> Don't you need to escape the second "." for the regex context in the
> tempest script? \Q and \E might be helpful for this?

Yes, I should probably escape the dot. \Q and \E is for perl, isn't it?
The regex is going to be interpreted by python.

> Also, you actually ignore any test ending with that string, not just the
> exact one in the comment, is that deliberate? Also any test with a longer
> name which has this substring will also be ignored.
> 
> If not then you could put the common prefix into a variable to avoid
> repeating it (and the very long lines) and maybe add some anchors?

I'll clean that up, and use a prefix.

What are you speaking about when you say "anchors"?

> You might also want to consider
> 
>     push @ignored_tests, "^$prefix\QTestVolumeBootPattern.test_volume_boot_pattern$";
> 
> Then later:
>     $ignored_tests = join("|", @ignored_tests);

I'll look into that.

> (I'm not sure, but I don't think you want qr{} [0] here, I might be wrong
> though)
> 
> > +  # tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance
> > +  #   It try to start a guest with /dev/vda as boot device name.
> > +  $ignored_tests .= '|.*TestShelveInstance.test_shelve_volume_backed_instance';
> > +  # scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
> > +  #   It access a volume through iSCSI. This does not work when both the
> > +  #   server and client of iSCSI are on the same Xen host, Linux 4.0 is the
> > +  #   first Linux to have a fix.
> > +  $ignored_tests .= '|.*TestVolumeBootPatternV2.test_volume_boot_pattern';
> > +
> > +  # Adding the tests to skip to the default regex.
> > +  my $regex = "(?!.*\\[.*\\bslow\\b.*\\]$ignored_tests)(^tempest\\.(api|scenario|thirdparty))";
> 
> Where the default regex is:
>     "(?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty))";
> ?

This regex of tests to execute can be found in tempest.git/tox.ini, in the
section "[testenv:full]". That what's been used the OpenStack gate.

I'll add a comment about it in the next patch version.

> This is a pretty rubbish interface which tempest has given you. What a
> pain!

Yeah, that the only way to ignore a test on a specific third-party
openstack CI loop...

-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 2/3] ts-openstack-tempest: Run Tempest to check OpenStack

[Ian Campbell]

On Tue, 2015-09-29 at 18:15 +0100, Anthony PERARD wrote:
> On Tue, Sep 29, 2015 at 04:43:50PM +0100, Ian Campbell wrote:
> > On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > 
> > > +  # Ignore these tests:
> > > +  #
> > > tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_
> > > volume_boot_pattern
> > > +  #   It try to start a guest with /dev/vda as boot device name.
> > > +  $ignored_tests .=
> > > '|.*TestVolumeBootPattern.test_volume_boot_pattern';
> > 
> > Don't you need to escape the second "." for the regex context in the
> > tempest script? \Q and \E might be helpful for this?
> 
> Yes, I should probably escape the dot. \Q and \E is for perl, isn't it?
> The regex is going to be interpreted by python.

$foo = "\Qtest.case\E" ends up with $foo having the a literal value which
is escaped i.e. "test\.case", which then gets passed to whichever regex
(including Pythons) and does the right thing.

e.g. 

$ perl -e 'print "\Qtest.case\E\n";'
test\.case
$ perl -e 'print "\Qtest.case*\E\n";'
test\.case\*

There is a subtle reliance on the Perl quoting rules and the Python quoting
rules being similar enough, but since these are regexp's I think that is
safe enough. We also assume Perl quoting is compatible with Shell quoting
in various places.

> > Also, you actually ignore any test ending with that string, not just
> > the
> > exact one in the comment, is that deliberate? Also any test with a
> > longer
> > name which has this substring will also be ignored.
> > 
> > If not then you could put the common prefix into a variable to avoid
> > repeating it (and the very long lines) and maybe add some anchors?
> 
> I'll clean that up, and use a prefix.
> 
> What are you speaking about when you say "anchors"?

I mean anchoring the match to the start of the string with ^ or to the end
with $. i.e. /^foo$/ only matches "foo" and not "foobar" or "bazfoo" etc.


> > > +  #
> > > tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_
> > > volume_backed_instance
> > > +  #   It try to start a guest with /dev/vda as boot device name.
> > > +  $ignored_tests .=
> > > '|.*TestShelveInstance.test_shelve_volume_backed_instance';
> > > +  #
> > > scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume
> > > _boot_pattern
> > > +  #   It access a volume through iSCSI. This does not work when both
> > > the
> > > +  #   server and client of iSCSI are on the same Xen host, Linux 4.0
> > > is the
> > > +  #   first Linux to have a fix.
> > > +  $ignored_tests .=
> > > '|.*TestVolumeBootPatternV2.test_volume_boot_pattern';
> > > +
> > > +  # Adding the tests to skip to the default regex.
> > > +  my $regex =
> > > "(?!.*\\[.*\\bslow\\b.*\\]$ignored_tests)(^tempest\\.(api|scenario|th
> > > irdparty))";
> > 
> > Where the default regex is:
> >     "(?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty))";
> > ?
> 
> This regex of tests to execute can be found in tempest.git/tox.ini, in
> the
> section "[testenv:full]". That what's been used the OpenStack gate.
> 
> I'll add a comment about it in the next patch version.

Given that the upstream default might change under our feet maybe we should
just consider this to be our own decision to make, even if it happens to
match the default today?

> > This is a pretty rubbish interface which tempest has given you. What a
> > pain!
> 
> Yeah, that the only way to ignore a test on a specific third-party
> openstack CI loop...

Yuck. Oh well.

Ian.

[PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

---
Change in V3:
- Switch to "track" Nova tree instead of devstack.
    Nova is the service we care about from a Xen point of view.
    Also it is updated much more often than devstack.
- Use TREE_OPENSTACK_ as prefix for all trees variables.
- Change the filter, keep only *-devstack jobs.
- Add stuff into ./ap-push
- Add stuff into ./cr-daily-branch.
- Add 'openstack' into ./cr-for-branches.
---
 ap-common            | 12 ++++++++++++
 ap-fetch-version     |  4 ++++
 ap-fetch-version-old |  5 +++++
 ap-print-url         |  3 +++
 ap-push              |  5 +++++
 cr-daily-branch      |  8 ++++++++
 cr-for-branches      |  2 +-
 cri-common           |  1 +
 make-flight          | 34 ++++++++++++++++++++++++++++++++++
 mfi-common           |  5 +++++
 10 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/ap-common b/ap-common
index 91425a9..a65e2fa 100644
--- a/ap-common
+++ b/ap-common
@@ -59,6 +59,17 @@ xenbranch_forqemu=${xenbranch/xen-unstable-smoke/xen-unstable}
 : ${PUSH_TREE_OVMF:=$XENBITS:/home/xen/git/osstest/ovmf.git}
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
+: ${GIT_OPENSTACK_ORG:=git://git.openstack.org}
+: ${TREE_OPENSTACK_CINDER:=$GIT_OPENSTACK_ORG/openstack/cinder.git}
+: ${TREE_OPENSTACK_DEVSTACK:=$GIT_OPENSTACK_ORG/openstack-dev/devstack.git}
+: ${TREE_OPENSTACK_GLANCE:=$GIT_OPENSTACK_ORG/openstack/glance.git}
+: ${TREE_OPENSTACK_KEYSTONE:=$GIT_OPENSTACK_ORG/openstack/keystone.git}
+: ${TREE_OPENSTACK_NOVA:=$GIT_OPENSTACK_ORG/openstack/nova.git}
+: ${TREE_OPENSTACK_REQUIREMENTS:=$GIT_OPENSTACK_ORG/openstack/requirements.git}
+: ${TREE_OPENSTACK_TEMPEST:=$GIT_OPENSTACK_ORG/openstack/tempest.git}
+: ${PUSH_TREE_OPENSTACK_NOVA:=$XENBITS:/home/xen/git/osstest/openstack-nova.git}
+: ${BASE_TREE_OPENSTACK_NOVA:=git://xenbits.xen.org/osstest/openstack-nova.git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -86,6 +97,7 @@ fi
 : ${LOCALREV_RUMPUSERXEN:=daily-cron.$branch}
 : ${LOCALREV_SEABIOS:=daily-cron.$branch}
 : ${LOCALREV_OVMF:=daily-cron.$branch}
+: ${LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index 62adf79..ebcac0e 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -91,6 +91,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
+openstack)
+	repo_tree_rev_fetch_git openstack-nova \
+		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 716fc8f..1b2ebb7 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -31,6 +31,7 @@ select_xenbranch
 : ${BASE_LOCALREV_RUMPUSERXEN:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_SEABIOS:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old}
+: ${BASE_LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch.old}
 
 : ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}}
 
@@ -99,6 +100,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack)
+	repo_tree_rev_fetch_git openstack-nova \
+		$BASE_TREE_OPENSTACK_NOVA xen-tested-master $BASE_LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
 	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch -f $HOME/testing.git production:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index c161169..57dc9d0 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -64,6 +64,9 @@ ovmf)
 osstest)
 	echo none:;
 	;;
+openstack)
+	echo $TREE_OPENSTACK_NOVA
+	;;
 *)
 	echo >&2 "branch $branch ?"
 	exit 1
diff --git a/ap-push b/ap-push
index aa0ec3d..826832c 100755
--- a/ap-push
+++ b/ap-push
@@ -37,6 +37,7 @@ TREE_LIBVIRT=$PUSH_TREE_LIBVIRT
 TREE_RUMPUSERXEN=$PUSH_TREE_RUMPUSERXEN
 TREE_SEABIOS=$PUSH_TREE_SEABIOS
 TREE_OVMF=$PUSH_TREE_OVMF
+TREE_OPENSTACK_NOVA=$PUSH_TREE_OPENSTACK_NOVA
 
 if info_linux_tree "$branch"; then
 	cd $repos/linux
@@ -101,6 +102,10 @@ ovmf)
 	cd $repos/ovmf
 	git push $TREE_OVMF $revision:refs/heads/xen-tested-master
 	;;
+openstack)
+	cd $repos/openstack-nova
+	git push $TREE_OPENSTACK_NOVA $revision:refs/heads/xen-tested-master
+	;;
 osstest)
 	git push $HOME/testing.git $revision:production
 	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index dd9c30a..ac13677 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -197,6 +197,10 @@ if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
 	determine_version REVISION_LINUXFIRMWARE linuxfirmware LINUXFIRMWARE
         export REVISION_LINUXFIRMWARE
 fi
+if [ "x$REVISION_OPENSTACK_NOVA" = x ]; then
+        determine_version REVISION_OPENSTACK_NOVA openstack OPENSTACK_NOVA
+        export REVISION_OPENSTACK_NOVA
+fi
 
 if [ "x$branch" != "xxen-unstable" ]; then
         export REVISION_LINUX_OLD=disable
@@ -246,6 +250,10 @@ ovmf)
 	realtree=ovmf
 	NEW_REVISION=$REVISION_OVMF
 	;;
+openstack)
+	realtree=openstack-nova
+	NEW_REVISION=$REVISION_OPENSTACK_NOVA
+	;;
 *)
 	NEW_REVISION=''
 	wantpush=false
diff --git a/cr-for-branches b/cr-for-branches
index 8d5748b..2726ea4 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf ${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf openstack ${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index c874ff9..66d13d1 100644
--- a/cri-common
+++ b/cri-common
@@ -79,6 +79,7 @@ select_xenbranch () {
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	distros-*)		tree=none;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
+	openstack)		tree=devstack;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
diff --git a/make-flight b/make-flight
index 70c909d..e5fe0d6 100755
--- a/make-flight
+++ b/make-flight
@@ -54,6 +54,11 @@ job_create_build_filter_callback () {
         *)			return 1 ;;
       esac
     ;;
+    openstack)
+      case "$job" in
+        *-xsm) return 1;;
+      esac
+    ;;
   esac
   return 0
 }
@@ -129,6 +134,16 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
+    openstack)
+      case "$job" in
+        *-devstack) ;;
+        *) return 1;;
+      esac
+      case $dom0arch in
+          amd64) ;;
+          *) return 1;;
+      esac
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -621,6 +636,25 @@ test_matrix_do_one () {
 
   do_pygrub_tests
   do_pvgrub_tests
+
+  job_create_test test-$xenarch$kern-$dom0arch-devstack \
+    test-devstack libvirt $xenarch $dom0arch \
+    tree_devstack=$TREE_OPENSTACK_DEVSTACK \
+    tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
+    tree_keystone=$TREE_OPENSTACK_KEYSTONE \
+    tree_glance=$TREE_OPENSTACK_GLANCE \
+    tree_cinder=$TREE_OPENSTACK_CINDER \
+    tree_nova=$TREE_OPENSTACK_NOVA \
+    tree_tempest=$TREE_OPENSTACK_TEMPEST \
+    revision_devstack=master \
+    revision_requirements=master \
+    revision_keystone=master \
+    revision_glance=master \
+    revision_cinder=master \
+    revision_nova=master \
+    revision_tempest=master \
+    dom0_mem=4000 \
+    all_hostflags=$most_hostflags
 }
 
 test_matrix_iterate
diff --git a/mfi-common b/mfi-common
index 01a5c9e..7e30cc7 100644
--- a/mfi-common
+++ b/mfi-common
@@ -59,6 +59,7 @@ xenbranch_xsm_variants () {
     xen-4.3-testing) echo "false";;
     xen-4.4-testing) echo "false";;
     xen-4.5-testing) echo "false";;
+    openstack)       echo "false";;
     *) echo "false true";
     esac
 }
@@ -102,6 +103,7 @@ create_build_jobs () {
       rumpuserxen) continue;;
       seabios) continue;;
       ovmf) continue;;
+      openstack) continue;;
       esac
       case "$xenbranch" in
       xen-3.*-testing) continue;;
@@ -127,6 +129,9 @@ create_build_jobs () {
       "
       ;;
     esac
+    if [ "$arch" = i386 ] && [ "$branch" = openstack ]; then
+      continue
+    fi
 
     case "$arch" in
     *)     suite=$defsuite;;
-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Campbell]

On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> diff --git a/cri-common b/cri-common
> index c874ff9..66d13d1 100644
> --- a/cri-common
> +++ b/cri-common
> @@ -79,6 +79,7 @@ select_xenbranch () {
>  > 	> ovmf)> 	> 	> 	> tree=ovmf;> 	> xenbranch=xen-unstable ;;
>  > 	> distros-*)> 	> 	> tree=none;> 	> xenbranch=xen-unstable ;;
>  > 	> osstest)> 	> 	> tree=osstest;> 	> xenbranch=xen-unstable ;;
> +> 	> openstack)> 	> 	> tree=devstack;> 	> xenbranch=xen-unstable ;;

Is this "devstack" a left over which should have become openstack-nova like
the others?

@@ -621,6 +636,25 @@ test_matrix_do_one () {
>  
>    do_pygrub_tests
>    do_pvgrub_tests
> +
> +  job_create_test test-$xenarch$kern-$dom0arch-devstack \
> +    test-devstack libvirt $xenarch $dom0arch \
> +    tree_devstack=$TREE_OPENSTACK_DEVSTACK \
> +    tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
> +    tree_keystone=$TREE_OPENSTACK_KEYSTONE \
> +    tree_glance=$TREE_OPENSTACK_GLANCE \
> +    tree_cinder=$TREE_OPENSTACK_CINDER \
> +    tree_nova=$TREE_OPENSTACK_NOVA \
> +    tree_tempest=$TREE_OPENSTACK_TEMPEST \
> +    revision_devstack=master \
> +    revision_requirements=master \
> +    revision_keystone=master \
> +    revision_glance=master \
> +    revision_cinder=master \
> +    revision_nova=master \

This one wants to be $REVISION_OPENSTACK_NOVA which comes from the new call
to determine_revision which you added.

If what you get when you do a bare git clone of $TREE_OPENSTACK_NOVA is not
master then to support standalone mode you might want
${REVISION_OPENSTACK_NOVA:-master} to set the default for non-cr-daily
-branch callers.

Ian.

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

On Tue, Sep 29, 2015 at 04:49:29PM +0100, Ian Campbell wrote:
> On Mon, 2015-09-28 at 16:56 +0100, Anthony PERARD wrote:
> > diff --git a/cri-common b/cri-common
> > index c874ff9..66d13d1 100644
> > --- a/cri-common
> > +++ b/cri-common
> > @@ -79,6 +79,7 @@ select_xenbranch () {
> >  > 	> ovmf)> 	> 	> 	> tree=ovmf;> 	> xenbranch=xen-unstable ;;
> >  > 	> distros-*)> 	> 	> tree=none;> 	> xenbranch=xen-unstable ;;
> >  > 	> osstest)> 	> 	> tree=osstest;> 	> xenbranch=xen-unstable ;;
> > +> 	> openstack)> 	> 	> tree=devstack;> 	> xenbranch=xen-unstable ;;
> 
> Is this "devstack" a left over which should have become openstack-nova like
> the others?

Yes, I should change this one as well.

> @@ -621,6 +636,25 @@ test_matrix_do_one () {
> >  
> >    do_pygrub_tests
> >    do_pvgrub_tests
> > +
> > +  job_create_test test-$xenarch$kern-$dom0arch-devstack \
> > +    test-devstack libvirt $xenarch $dom0arch \
> > +    tree_devstack=$TREE_OPENSTACK_DEVSTACK \
> > +    tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
> > +    tree_keystone=$TREE_OPENSTACK_KEYSTONE \
> > +    tree_glance=$TREE_OPENSTACK_GLANCE \
> > +    tree_cinder=$TREE_OPENSTACK_CINDER \
> > +    tree_nova=$TREE_OPENSTACK_NOVA \
> > +    tree_tempest=$TREE_OPENSTACK_TEMPEST \
> > +    revision_devstack=master \
> > +    revision_requirements=master \
> > +    revision_keystone=master \
> > +    revision_glance=master \
> > +    revision_cinder=master \
> > +    revision_nova=master \
> 
> This one wants to be $REVISION_OPENSTACK_NOVA which comes from the new call
> to determine_revision which you added.

Ok, I'll change that.

> If what you get when you do a bare git clone of $TREE_OPENSTACK_NOVA is not
> master then to support standalone mode you might want
> ${REVISION_OPENSTACK_NOVA:-master} to set the default for non-cr-daily
> -branch callers.

I think the default is master.

Also, I guest I should remove the other revision_* runvar which are not
very usefull right now.

-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Jackson]

Anthony PERARD writes ("[PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> ---
> Change in V3:
> - Switch to "track" Nova tree instead of devstack.
>     Nova is the service we care about from a Xen point of view.
>     Also it is updated much more often than devstack.
> - Use TREE_OPENSTACK_ as prefix for all trees variables.
> - Change the filter, keep only *-devstack jobs.
> - Add stuff into ./ap-push
> - Add stuff into ./cr-daily-branch.
> - Add 'openstack' into ./cr-for-branches.
...
> +  job_create_test test-$xenarch$kern-$dom0arch-devstack \
> +    test-devstack libvirt $xenarch $dom0arch \
> +    tree_devstack=$TREE_OPENSTACK_DEVSTACK \
> +    tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
> +    tree_keystone=$TREE_OPENSTACK_KEYSTONE \
> +    tree_glance=$TREE_OPENSTACK_GLANCE \
> +    tree_cinder=$TREE_OPENSTACK_CINDER \
> +    tree_nova=$TREE_OPENSTACK_NOVA \
> +    tree_tempest=$TREE_OPENSTACK_TEMPEST \
> +    revision_devstack=master \
> +    revision_requirements=master \
> +    revision_keystone=master \
> +    revision_glance=master \
> +    revision_cinder=master \
> +    revision_nova=master \
> +    revision_tempest=master \

All these revision_FOO=master are rather odd.  You seem to be creating
a push gate which maintains a tested revision of `openstack-nova'.  Is
that what you want ?  If so, then it's not really clear to me that
what you want to do is to simply pick up new master revisions of these
other git trees.  It might be better to obtain a bunch of revision
ids, test them all together, and push them together ?  (This is not
something that osstest can do right now but it doesn't seem
difficult.)

If you don't want to make a push gate, then you probably want to do
away with the changes to ap-fetch and ap-push.

Ian.

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Campbell]

On Tue, 2015-09-29 at 17:52 +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[PATCH OSSTEST v3 3/3] Create a flight to test
> OpenStack with xen-unstable and libvirt"):
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > 
> > ---
> > Change in V3:
> > - Switch to "track" Nova tree instead of devstack.
> >     Nova is the service we care about from a Xen point of view.
> >     Also it is updated much more often than devstack.
> > - Use TREE_OPENSTACK_ as prefix for all trees variables.
> > - Change the filter, keep only *-devstack jobs.
> > - Add stuff into ./ap-push
> > - Add stuff into ./cr-daily-branch.
> > - Add 'openstack' into ./cr-for-branches.
> ...
> > +  job_create_test test-$xenarch$kern-$dom0arch-devstack \
> > +    test-devstack libvirt $xenarch $dom0arch \
> > +    tree_devstack=$TREE_OPENSTACK_DEVSTACK \
> > +    tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
> > +    tree_keystone=$TREE_OPENSTACK_KEYSTONE \
> > +    tree_glance=$TREE_OPENSTACK_GLANCE \
> > +    tree_cinder=$TREE_OPENSTACK_CINDER \
> > +    tree_nova=$TREE_OPENSTACK_NOVA \
> > +    tree_tempest=$TREE_OPENSTACK_TEMPEST \
> > +    revision_devstack=master \
> > +    revision_requirements=master \
> > +    revision_keystone=master \
> > +    revision_glance=master \
> > +    revision_cinder=master \
> > +    revision_nova=master \
> > +    revision_tempest=master \
> 
> All these revision_FOO=master are rather odd.

I think the revision_nova one was supposed to be $REVISION_OPENSTACK_NOVA?

AIUI Nova is the component which we care about (it's the "compute" bit of
openstack) and changes frequently compared with devstack (the overall
driver).

>   You seem to be creating
> a push gate which maintains a tested revision of `openstack-nova'.  Is
> that what you want ?  If so, then it's not really clear to me that
> what you want to do is to simply pick up new master revisions of these
> other git trees.  It might be better to obtain a bunch of revision
> ids, test them all together, and push them together ?

i.e. to have multiple output gates, one for each of those trees, each
pushed from a single flight?

I'm assuming you aren't talking about having N openstack flights, one for
each tree.

>   (This is not
> something that osstest can do right now but it doesn't seem
> difficult.)

It seems like quite a lot of faff in cr-daily-branch (what does $tree mean,
which $NEW_REVISION and $OLD_REVISION do we look at), ap-push now needs to
operate on a list or something (arguments?).

Getting this stuff right (i.e. testing) is quite hard even with access to a
production instance.

The series today records the actual built versions, so we still get
bisections, what would having multiple output gates buy us in practice over
that?

Maybe the explicit =master stuff above should be removed, or replaced with
$REVISION_OPENSTACK_FOO which apart from NOVA are _not_ set by cr-daily
-branch (resulting in cloning master)?

> If you don't want to make a push gate, then you probably want to do
> away with the changes to ap-fetch and ap-push.

I think we want a push gate just for the regression tracking, but not
really for its own sake.

Ian.

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt"):
> On Tue, 2015-09-29 at 17:52 +0100, Ian Jackson wrote:
> > All these revision_FOO=master are rather odd.
> 
> I think the revision_nova one was supposed to be $REVISION_OPENSTACK_NOVA?
> 
> AIUI Nova is the component which we care about (it's the "compute" bit of
> openstack) and changes frequently compared with devstack (the overall
> driver).

And the others we don't care about ?  Are they likely to break things
anyway, is my point ?  Do they have uniform intercompatibility ?

> >   You seem to be creating
> > a push gate which maintains a tested revision of `openstack-nova'.  Is
> > that what you want ?  If so, then it's not really clear to me that
> > what you want to do is to simply pick up new master revisions of these
> > other git trees.  It might be better to obtain a bunch of revision
> > ids, test them all together, and push them together ?
> 
> i.e. to have multiple output gates, one for each of those trees, each
> pushed from a single flight?

Yes.

> I'm assuming you aren't talking about having N openstack flights, one for
> each tree.
> 
> >   (This is not
> > something that osstest can do right now but it doesn't seem
> > difficult.)
> 
> It seems like quite a lot of faff in cr-daily-branch (what does $tree mean,
> which $NEW_REVISION and $OLD_REVISION do we look at), ap-push now needs to
> operate on a list or something (arguments?).
> 
> Getting this stuff right (i.e. testing) is quite hard even with access to a
> production instance.

I can think of ways of doing it.  I wasn't expecting Anthony to
produce them :-).  Right now I'm trying to understand the situation,
and if I think we want to push multiple trees at once I'll write the
code.

> The series today records the actual built versions, so we still get
> bisections, what would having multiple output gates buy us in practice over
> that?

I guess the main advantage is that other osstest `branches' could have
a stable set of openstack things, and that the reporting of `what
changed' would be accurate.

> Maybe the explicit =master stuff above should be removed, or replaced with
> $REVISION_OPENSTACK_FOO which apart from NOVA are _not_ set by cr-daily
> -branch (resulting in cloning master)?

If we do want to just ignore this issue of the other trees, I do
indeed see no particular reason to explicitly set all the runvars to
master.

> I think we want a push gate just for the regression tracking, but not
> really for its own sake.

In that case we I think yes don't really need to push multiple branches.

Ian.

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

On Thu, Oct 08, 2015 at 05:42:56PM +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt"):
> > On Tue, 2015-09-29 at 17:52 +0100, Ian Jackson wrote:
> > > All these revision_FOO=master are rather odd.
> > 
> > I think the revision_nova one was supposed to be $REVISION_OPENSTACK_NOVA?
> > 
> > AIUI Nova is the component which we care about (it's the "compute" bit of
> > openstack) and changes frequently compared with devstack (the overall
> > driver).
> 
> And the others we don't care about ?  Are they likely to break things
> anyway, is my point ?  Do they have uniform intercompatibility ?
> 
> > >   You seem to be creating
> > > a push gate which maintains a tested revision of `openstack-nova'.  Is
> > > that what you want ?  If so, then it's not really clear to me that
> > > what you want to do is to simply pick up new master revisions of these
> > > other git trees.  It might be better to obtain a bunch of revision
> > > ids, test them all together, and push them together ?
> > 
> > i.e. to have multiple output gates, one for each of those trees, each
> > pushed from a single flight?
> 
> Yes.
> 
> > I'm assuming you aren't talking about having N openstack flights, one for
> > each tree.
> > 
> > >   (This is not
> > > something that osstest can do right now but it doesn't seem
> > > difficult.)
> > 
> > It seems like quite a lot of faff in cr-daily-branch (what does $tree mean,
> > which $NEW_REVISION and $OLD_REVISION do we look at), ap-push now needs to
> > operate on a list or something (arguments?).
> > 
> > Getting this stuff right (i.e. testing) is quite hard even with access to a
> > production instance.
> 
> I can think of ways of doing it.  I wasn't expecting Anthony to
> produce them :-).  Right now I'm trying to understand the situation,
> and if I think we want to push multiple trees at once I'll write the
> code.
> 
> > The series today records the actual built versions, so we still get
> > bisections, what would having multiple output gates buy us in practice over
> > that?
> 
> I guess the main advantage is that other osstest `branches' could have
> a stable set of openstack things, and that the reporting of `what
> changed' would be accurate.
> 
> > Maybe the explicit =master stuff above should be removed, or replaced with
> > $REVISION_OPENSTACK_FOO which apart from NOVA are _not_ set by cr-daily
> > -branch (resulting in cloning master)?
> 
> If we do want to just ignore this issue of the other trees, I do
> indeed see no particular reason to explicitly set all the runvars to
> master.
> 
> > I think we want a push gate just for the regression tracking, but not
> > really for its own sake.

Yes, that is what I had in mind, testing OpenStack with tips of xen and
libvirt.

I'll remove all those revision_* runvars.

> In that case we I think yes don't really need to push multiple branches.

-- 
Anthony PERARD

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Jackson]

Anthony PERARD writes ("Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt"):
> On Thu, Oct 08, 2015 at 05:42:56PM +0100, Ian Jackson wrote:
> > If we do want to just ignore this issue of the other trees, I do
> > indeed see no particular reason to explicitly set all the runvars to
> > master.
> > 
> > > I think we want a push gate just for the regression tracking, but not
> > > really for its own sake.
> 
> Yes, that is what I had in mind, testing OpenStack with tips of xen and
> libvirt.
> > 
> > In that case we I think yes don't really need to push multiple branches.
>
> I'll remove all those revision_* runvars.

Do you mean to remove the settings of them all to `master', or the
code to honour them ?

The machinery in the ts-* script to honour revision_xyz and to set
built_revision_xyz is useful because the bisector can use it to finger
individual tress.

Unless, of course, you don't want to run the bisector on these tests.

Ian.

Re: [PATCH OSSTEST v3 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Campbell]

On Tue, 2015-10-13 at 12:02 +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [PATCH OSSTEST v3 3/3] Create a flight to
> test OpenStack with xen-unstable and libvirt"):
> > On Thu, Oct 08, 2015 at 05:42:56PM +0100, Ian Jackson wrote:
> > > If we do want to just ignore this issue of the other trees, I do
> > > indeed see no particular reason to explicitly set all the runvars to
> > > master.
> > > 
> > > > I think we want a push gate just for the regression tracking, but
> > > > not
> > > > really for its own sake.
> > 
> > Yes, that is what I had in mind, testing OpenStack with tips of xen and
> > libvirt.
> > > 
> > > In that case we I think yes don't really need to push multiple
> > > branches.
> > 
> > I'll remove all those revision_* runvars.
> 
> Do you mean to remove the settings of them all to `master', or the
> code to honour them ?
> 
> The machinery in the ts-* script to honour revision_xyz and to set
> built_revision_xyz is useful because the bisector can use it to finger
> individual tress.

That all comes for free via build_clone I think, IOW Anthony would need to
go a good deal out of his way to not honour them I think.

> Unless, of course, you don't want to run the bisector on these tests.

I don't think we should rule it out.

Ian.