[PATCH v5 0/3] Have OpenStack tested on top of xen's master and libvirt's master.

[PATCH v5 0/3] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

Hi,

I have looked into getting OpenStack been tested on the latest Xen via
osstest.

The ts-openstack-deploy script does prepare a bit more the host, clone
devstack and other OpenStack trees, then run ./stack.sh, which is a bit
like raisin and deploy OpenStack on the host. Once the machine is ready,
the integration test suite from OpenStack, Tempest, is started by
ts-openstack-tempest.

For the last patch that create a flight plan, I've tested only with
`./standalone make-flight openstack` and looked into the database to check
that only the necessary build job and the only test job are there.
(build-amd64{,-pvops,-libvirt} test-amd64-amd64-devstack)

Thanks.

Change in V5:
  - on small change in patch 1, but I keeped the acked-by
  - few changes in the way the flight is created (in the last patch)

Change in V4:
  few changes listed in second and third patch, mostly cleanup.

Change in V3:
  - Track Nova tree instead of devstack.
      Nova is the service we care about from a Xen point of view.
      Also it is updated much more often than devstack.
  - Cleanups, see change log in patches.

Changes in V2:
  - no more Osstest::Toolstack::OpenStack.
  - osstest now clone every single tree that devstack is going to need.
    And ./stack.sh should fail if one tree is missing.
  - avoid build-*-xsm for an openstack flight
  - rename ts-devstack to ts-openstack-devstack
  - New test script ts-openstack-tempest
  - Add CONFIG_CRYPTO_XTS=m to the kernel build.
  - new possible runvar $dom0_mem to control dom0 memory
  - several fix to have volume tests working.
  - have OpenStack deploy from it's builddir instead of /opt/stack
  - have 4GB for dom0 instead of relying on balloning.

Anthony PERARD (3):
  ts-openstack-deploy: Deploy OpenStack on a host with devstack
  ts-openstack-tempest: Run Tempest to check OpenStack
  Create a flight to test OpenStack with xen-unstable and libvirt

 ap-common            |  12 ++
 ap-fetch-version     |   4 +
 ap-fetch-version-old |   5 +
 ap-print-url         |   3 +
 ap-push              |   5 +
 cr-daily-branch      |   8 ++
 cr-for-branches      |   2 +-
 cri-common           |   1 +
 make-flight          |  52 +++++++++
 sg-run-job           |   6 +
 ts-openstack-deploy  | 314 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ts-openstack-tempest |  65 +++++++++++
 12 files changed, 476 insertions(+), 1 deletion(-)
 create mode 100755 ts-openstack-deploy
 create mode 100755 ts-openstack-tempest

-- 
Anthony PERARD

[PATCH v5 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Anthony PERARD]

This script installs any necessary packages and clones all of the OpenStack
trees which are used by devstack to deploy OpenStack.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

---
Only change in V5:
- edit stackrc from devstack file to change the hardcoded path DEST

No change in V4:
- acked

Change in V3:
- Use host as argument to run the job.
- Use selectjob() and get rid of the unused $gho.
- Use target_jobdir() instead of builddirsprops().
- Remove GIT_BASE from devstack config.
- Rename the script to ts-openstack-deploy (from ts-openstack-devstack).
---
 sg-run-job          |   5 +
 ts-openstack-deploy | 314 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 319 insertions(+)
 create mode 100755 ts-openstack-deploy

diff --git a/sg-run-job b/sg-run-job
index ca9a621..a995db5 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -410,6 +410,11 @@ proc run-job/test-rumpuserxen {} {
                  ts-guest-destroy-hard            host   $g   +
 }
 
+proc need-hosts/test-devstack {} { return host }
+proc run-job/test-devstack {} {
+  run-ts . = ts-openstack-deploy host
+}
+
 if {[file exists sg-run-job-adhoc]} {
     source sg-run-job-adhoc
 }
diff --git a/ts-openstack-deploy b/ts-openstack-deploy
new file mode 100755
index 0000000..f42778b
--- /dev/null
+++ b/ts-openstack-deploy
@@ -0,0 +1,314 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tgt_init ();
+
+sub packages () {
+  # Install open-iscsi ahead of devstack ...
+  target_install_packages($ho, qw(git sudo open-iscsi));
+  # ... and start open-iscsi to have /etc/iscsi/initiatorname.iscsi
+  # generated. This is done on install on Ubuntu.
+  target_cmd_root($ho, 'service open-iscsi start');
+}
+
+sub checkout () {
+  prepbuilddirs();
+  build_clone($ho, 'cinder', $builddir, 'cinder');
+  build_clone($ho, 'devstack', $builddir, 'devstack');
+  build_clone($ho, 'glance', $builddir, 'glance');
+  build_clone($ho, 'keystone', $builddir, 'keystone');
+  build_clone($ho, 'nova', $builddir, 'nova');
+  build_clone($ho, 'requirements', $builddir, 'requirements');
+  build_clone($ho, 'tempest', $builddir, 'tempest');
+
+  my $vg = target_choose_vg($ho, 10*1024); # 10GB
+  target_putfilecontents_stash($ho, 60, <<END, $builddir.'/devstack/local.conf');
+[[local|localrc]]
+# Everything should be cloned by osstest, so devstack don't have to do it
+ERROR_ON_CLONE=True
+USE_SCREEN=False
+ADMIN_PASSWORD=secretadmin
+DATABASE_PASSWORD=secretdatabase
+RABBIT_PASSWORD=secretrabbit
+SERVICE_PASSWORD=secretservice
+SERVICE_TOKEN=atokenserviced
+# make it small because there is no way to not have this lvm volume created
+VOLUME_BACKING_FILE_SIZE=500M
+DEST=$builddir
+LOGFILE=\$DEST/logs/stack.sh.log
+# stackrc set this but don't take \$DEST into account
+SERVICE_DIR=\${DEST}/status
+LOG_COLOR=False
+LIBVIRT_TYPE=xen
+disable_service horizon
+disable_service n-novnc
+enable_service n-obj
+[[post-config|\$CINDER_CONF]]
+[lvmdriver-1]
+volume_group = $vg
+END
+
+  # stackrc does not take $DEST from local.conf into account, so fix it here
+  target_editfile($ho, "$builddir/devstack/stackrc", sub {
+      while (<EI>) {
+        if (m/^DEST=\/opt\/stack$/) {
+          s/DEST=\/opt\/stack/DEST=$builddir/;
+        }
+        print EO or die $!;
+      }
+  });
+
+  # libvirt is already installed, but not as a package, so avoid installation of
+  # the libvirt package with devstack
+  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
+      while (<EI>) {
+        next if m/.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+  target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions-libvirt", sub {
+      while (<EI>) {
+        next if m/install_package.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+
+  # OpenStack needs access to libvirt from a user.
+  target_cmd_root($ho, <<END);
+    cat >> /etc/libvirt/libvirtd.conf <<EOF
+unix_sock_group = "libvirt"
+unix_sock_ro_perms = "0777"
+unix_sock_rw_perms = "0770"
+EOF
+END
+
+  # devstack is going to setup the host, install some dependency.
+  target_putfilecontents_root_stash($ho, 100, <<END,"/etc/sudoers.d/devstack");
+osstest ALL=(ALL) NOPASSWD:ALL
+END
+
+  target_putfilecontents_root_stash($ho, 60, tgt_init(), "/etc/init.d/tgt");
+  target_cmd_root($ho, <<END);
+    chmod +x /etc/init.d/tgt
+END
+}
+
+sub deploy() {
+  target_cmd($ho, <<END, 1800);
+    cd $builddir/devstack
+    ./stack.sh
+END
+}
+
+packages();
+checkout();
+deploy();
+
+# This is missing from Debian but required by devstack
+# Got it from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577925
+sub tgt_init () {
+  return <<'END';
+#!/bin/bash
+### BEGIN INIT INFO
+# Provides:          tgtd
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Should-Start:      zfs
+# Should-Stop:       zfs
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: iscsi target daemon
+# Description:       iscsi target daemon
+### END INIT INFO
+
+DESC="target framework daemon"
+NAME=tgtd
+DAEMON=/usr/sbin/${NAME}
+
+TGTD_CONFIG=/etc/tgt/targets.conf
+
+TASK=$1
+
+. /lib/lsb/init-functions
+
+[ -x $DAEMON ] || exit 0
+
+start()
+{
+        log_daemon_msg "Starting $DESC" "$NAME"
+        # Start tgtd first.
+        tgtd &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                exit 1
+        else
+                log_end_msg 0
+        fi
+        # Put tgtd into "offline" state until all the targets are configured.
+        # We don't want initiators to (re)connect and fail the connection
+        # if it's not ready.
+        tgtadm --op update --mode sys --name State -v offline
+        # Configure the targets.
+        tgt-admin -e -c $TGTD_CONFIG
+        # Put tgtd into "ready" state.
+        tgtadm --op update --mode sys --name State -v ready
+}
+
+stop()
+{
+        if [ "$RUNLEVEL" == 0 -o "$RUNLEVEL" == 6 ] ; then
+            forcedstop
+        fi
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        # Remove all targets. It only removes targets which are not in use.
+        tgt-admin --update ALL -c /dev/null &>/dev/null
+        # tgtd will exit if all targets were removed
+        tgtadm --op delete --mode system &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                if [ "$TASK" != "restart" ] ; then
+                        log_end_msg 1
+                        exit 1
+                else
+                        log_end_msg 0
+                fi
+        elif [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                echo "Some initiators are still connected - could not stop tgtd"
+                exit 2
+        else
+                log_end_msg 0
+        fi
+        echo -n
+}
+
+forcedstop()
+{
+        # NOTE: Forced shutdown of the iscsi target may cause data corruption
+        # for initiators that are connected.
+        echo "Force-stopping target framework daemon"
+        # Offline everything first. May be needed if we're rebooting, but
+        # expect the initiators to reconnect cleanly when we boot again
+        # (i.e. we don't want them to reconnect to a tgtd which is still
+        # working, but the target is gone).
+        tgtadm --op update --mode sys --name State -v offline &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+            echo "tgtd is not running"
+            [ "$TASK" != "restart" ] && exit 1
+        else
+            tgt-admin --offline ALL
+            # Remove all targets, even if they are still in use.
+            tgt-admin --update ALL -c /dev/null -f
+            # It will shut down tgtd only after all targets were removed.
+            tgtadm --op delete --mode system
+            RETVAL=$?
+            if [ "$RETVAL" -ne 0 ] ; then
+                echo "Failed to shutdown tgtd"
+                exit 1
+            fi
+        fi
+        echo -n
+}
+
+reload()
+{
+        log_daemon_msg "Reloading configuration of $DESC" "$NAME"
+        # Update configuration for targets. Only targets which
+        # are not in use will be updated.
+        tgt-admin --update ALL -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        fi
+        log_end_msg 0
+}
+
+forcedreload()
+{
+        log_daemon_msg "Forced-reload configuration of $DESC" "$NAME"
+        # Update configuration for targets, even those in use.
+        tgt-admin --update ALL -f -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        else
+                log_end_msg 0
+        fi
+}
+
+status()
+{
+        # Don't name this script "tgtd"...
+        TGTD_PROC=$(ps -C tgtd | grep -c tgtd)
+        if [ "$TGTD_PROC" -eq 2 ] ; then
+            echo "tgtd is running. Run 'tgt-admin -s' to see detailed target info."
+        else
+            echo "tgtd is NOT running."
+        fi
+}
+
+case $1 in
+        start)
+                start
+                ;;
+        stop)
+                stop
+                ;;
+        forcedstop)
+                forcedstop
+                ;;
+        restart)
+                TASK=restart
+                stop && start
+                ;;
+        forcedrestart)
+                TASK=restart
+                forcedstop && start
+                ;;
+        reload)
+                reload
+                ;;
+        force-reload)
+                forcedreload
+                ;;
+        status)
+                status
+                ;;
+        *)
+                echo "Usage: $0 {start|stop|forcedstop|restart|forcedrestart|reload|force-reload|status}"
+                exit 2
+                ;;
+esac
+END
+}
-- 
Anthony PERARD

[PATCH v5 2/3] ts-openstack-tempest: Run Tempest to check OpenStack

[Anthony PERARD]

This script runs the OpenStack integration test suite, Tempest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

---
No change in V5

Change in V4:
- use \Q\E for tests names
- write the full name of the tests to skip
- use push @ignored_test then join()
- use variables to store common prefix
- rewrite comments

Change in V3:
- Use host as argument of the script.
- Use selecthost() and get rid of $gho.
- Use target_jobdir() instead of builddirsprops().
- Put the ignored Tempest tests into a var and try to explain why there are
  skip.
---
 sg-run-job           |  1 +
 ts-openstack-tempest | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+)
 create mode 100755 ts-openstack-tempest

diff --git a/sg-run-job b/sg-run-job
index a995db5..8695ae8 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -413,6 +413,7 @@ proc run-job/test-rumpuserxen {} {
 proc need-hosts/test-devstack {} { return host }
 proc run-job/test-devstack {} {
   run-ts . = ts-openstack-deploy host
+  run-ts . = ts-openstack-tempest host
 }
 
 if {[file exists sg-run-job-adhoc]} {
diff --git a/ts-openstack-tempest b/ts-openstack-tempest
new file mode 100755
index 0000000..91c5f3a
--- /dev/null
+++ b/ts-openstack-tempest
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tempest() {
+  my @ignored_tests;
+  my $scenario = 'tempest.scenario';
+  my $volume_boot_pattern =
+      "$scenario.test_volume_boot_pattern.TestVolumeBootPattern";
+  my $shelve_instance = "$scenario.test_shelve_instance.TestShelveInstance";
+
+  # Ignore tests which try to boot a guest with /dev/vda as boot device name.
+  push @ignored_tests,
+      "^\Q$volume_boot_pattern.test_volume_boot_pattern\E";
+  push @ignored_tests,
+      "^\Q$volume_boot_pattern.test_create_ebs_image_and_check_boot\E";
+  push @ignored_tests,
+      "^\Q$shelve_instance.test_shelve_volume_backed_instance\E";
+
+  # Those tests access a volume through iSCSI. This does not work when both the
+  # server and client of iSCSI are on the same Xen host, Linux 4.0 is the first
+  # Linux to have a fix.
+  push @ignored_tests,
+      "^\Q${volume_boot_pattern}V2.test_volume_boot_pattern\E";
+  push @ignored_tests,
+      "^\Q${volume_boot_pattern}V2.test_create_ebs_image_and_check_boot\E";
+
+  # This regex below select the tests to run and exclude the ones marked as
+  # slow as well as the explicit tests listed above.
+  # It is based on the one that can be found in tempest.git/tox.ini
+  # in the section [testenv:full].
+  my $ignored_tests = join("|", @ignored_tests);
+  my $regex =
+      "(?!.*\\[.*\\bslow\\b.*\\]|$ignored_tests)(^tempest\\.(api|scenario|thirdparty))";
+
+  target_cmd($ho, <<END, 7200);
+    $builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+END
+}
+
+tempest();
-- 
Anthony PERARD

[PATCH v5 3/3] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

This patch should create a flight "openstack-nova", with those jobs:
  build-amd64
  build-amd64-xsm
  build-amd64-pvops
  build-amd64-libvirt
  test-amd64-amd64-devstack
  test-amd64-amd64-devstack-xsm

About the runvars revision_* of test-*-*-devstack:
  only REVISION_OPENSTACK_NOVA is set, the others are unset.
  Empty revision_* runvar would clone the default branch, which should
  be master for every openstack repos.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

---
Change in V5:
- rename flight openstack to openstack-nova
- add -xsm variant of the test
- run test-devstack only on openstack-nova flight

Change in V4:
- also skip build-*-oldkern in make flight
- fix select_xenbranch
- set revision_*=$REVISION_OPENSTACK_* in make-flight
  (was revision_*=master before)
  only REVISION_OPENSTACK_NOVA is set, the others are unset.
  empty revision_* runvar would clone the default branch, which should
  be master for every openstack repos

Change in V3:
- Switch to "track" Nova tree instead of devstack.
    Nova is the service we care about from a Xen point of view.
    Also it is updated much more often than devstack.
- Use TREE_OPENSTACK_ as prefix for all trees variables.
- Change the filter, keep only *-devstack jobs.
- Add stuff into ./ap-push
- Add stuff into ./cr-daily-branch.
- Add 'openstack' into ./cr-for-branches.
---
 ap-common            | 12 ++++++++++++
 ap-fetch-version     |  4 ++++
 ap-fetch-version-old |  5 +++++
 ap-print-url         |  3 +++
 ap-push              |  5 +++++
 cr-daily-branch      |  8 ++++++++
 cr-for-branches      |  2 +-
 cri-common           |  1 +
 make-flight          | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 9 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/ap-common b/ap-common
index 19c7580..5ea0891 100644
--- a/ap-common
+++ b/ap-common
@@ -55,6 +55,17 @@
 : ${PUSH_TREE_OVMF:=$XENBITS:/home/xen/git/osstest/ovmf.git}
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
+: ${GIT_OPENSTACK_ORG:=git://git.openstack.org}
+: ${TREE_OPENSTACK_CINDER:=$GIT_OPENSTACK_ORG/openstack/cinder.git}
+: ${TREE_OPENSTACK_DEVSTACK:=$GIT_OPENSTACK_ORG/openstack-dev/devstack.git}
+: ${TREE_OPENSTACK_GLANCE:=$GIT_OPENSTACK_ORG/openstack/glance.git}
+: ${TREE_OPENSTACK_KEYSTONE:=$GIT_OPENSTACK_ORG/openstack/keystone.git}
+: ${TREE_OPENSTACK_NOVA:=$GIT_OPENSTACK_ORG/openstack/nova.git}
+: ${TREE_OPENSTACK_REQUIREMENTS:=$GIT_OPENSTACK_ORG/openstack/requirements.git}
+: ${TREE_OPENSTACK_TEMPEST:=$GIT_OPENSTACK_ORG/openstack/tempest.git}
+: ${PUSH_TREE_OPENSTACK_NOVA:=$XENBITS:/home/xen/git/osstest/openstack-nova.git}
+: ${BASE_TREE_OPENSTACK_NOVA:=git://xenbits.xen.org/osstest/openstack-nova.git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -82,6 +93,7 @@ fi
 : ${LOCALREV_RUMPUSERXEN:=daily-cron.$branch}
 : ${LOCALREV_SEABIOS:=daily-cron.$branch}
 : ${LOCALREV_OVMF:=daily-cron.$branch}
+: ${LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index a7b658b..6d98de4 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -99,6 +99,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
+openstack-nova)
+	repo_tree_rev_fetch_git openstack-nova \
+		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index e2c6b3b..c832aca 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -33,6 +33,7 @@ check_ap_fetch_placeholders
 : ${BASE_LOCALREV_RUMPUSERXEN:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_SEABIOS:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old}
+: ${BASE_LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch.old}
 
 if info_linux_tree "$branch"; then
 	repo_tree_rev_fetch_git linux \
@@ -105,6 +106,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack-nova)
+	repo_tree_rev_fetch_git openstack-nova \
+		$BASE_TREE_OPENSTACK_NOVA xen-tested-master $BASE_LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
 	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch -f $HOME/testing.git production:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index 4088852..423556e 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -64,6 +64,9 @@ ovmf)
 osstest)
 	echo none:;
 	;;
+openstack-nova)
+	echo $TREE_OPENSTACK_NOVA
+	;;
 *)
 	echo >&2 "branch $branch ?"
 	exit 1
diff --git a/ap-push b/ap-push
index 8def652..aeb7d75 100755
--- a/ap-push
+++ b/ap-push
@@ -40,6 +40,7 @@ TREE_LIBVIRT=$PUSH_TREE_LIBVIRT
 TREE_RUMPUSERXEN=$PUSH_TREE_RUMPUSERXEN
 TREE_SEABIOS=$PUSH_TREE_SEABIOS
 TREE_OVMF=$PUSH_TREE_OVMF
+TREE_OPENSTACK_NOVA=$PUSH_TREE_OPENSTACK_NOVA
 
 if info_linux_tree "$branch"; then
 	cd $repos/linux
@@ -120,6 +121,10 @@ ovmf)
 	cd $repos/ovmf
 	git push $TREE_OVMF $revision:refs/heads/xen-tested-master
 	;;
+openstack-nova)
+	cd $repos/openstack-nova
+	git push $TREE_OPENSTACK_NOVA $revision:refs/heads/xen-tested-master
+	;;
 osstest)
 	git push $HOME/testing.git $revision:production
 	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index 364238c..5e371b0 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -197,6 +197,10 @@ if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
 	determine_version REVISION_LINUXFIRMWARE linuxfirmware LINUXFIRMWARE
         export REVISION_LINUXFIRMWARE
 fi
+if [ "x$REVISION_OPENSTACK_NOVA" = x ]; then
+        determine_version REVISION_OPENSTACK_NOVA openstack-nova OPENSTACK_NOVA
+        export REVISION_OPENSTACK_NOVA
+fi
 
 if [ "x$branch" != "xxen-unstable" ]; then
         export REVISION_LINUX_OLD=disable
@@ -246,6 +250,10 @@ ovmf)
 	realtree=ovmf
 	NEW_REVISION=$REVISION_OVMF
 	;;
+openstack-nova)
+	realtree=openstack-nova
+	NEW_REVISION=$REVISION_OPENSTACK_NOVA
+	;;
 *)
 	NEW_REVISION=''
 	wantpush=false
diff --git a/cr-for-branches b/cr-for-branches
index 757045d..a07d0d5 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf ${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf openstack-nova ${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index 6dfe8df..7eaea0d 100644
--- a/cri-common
+++ b/cri-common
@@ -79,6 +79,7 @@ select_xenbranch () {
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	distros-*)		tree=none;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
+	openstack-nova)	   tree=openstack-nova;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
diff --git a/make-flight b/make-flight
index 6b2b3ea..442d24e 100755
--- a/make-flight
+++ b/make-flight
@@ -119,6 +119,17 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
+    openstack-nova)
+      case "$job" in
+        *-devstack) ;;
+        *-devstack-xsm) ;;
+        *) return 1;;
+      esac
+      case $dom0arch in
+          amd64) ;;
+          *) return 1;;
+      esac
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -150,6 +161,12 @@ arch_branch_filter_callback () {
         rumpuserxen) return 1;;
         seabios) return 1;;
         ovmf) return 1;;
+        openstack-nova) return 1;;
+        esac
+        ;;
+  i386)
+        case "$branch" in
+        openstack-nova) return 1;;
         esac
         ;;
   i386|amd64)
@@ -534,6 +551,39 @@ do_pv_debian_tests () {
   $endfmt$dom0arch
 }
 
+do_openstack_tests () {
+  local xsms=$(xenbranch_xsm_variants)
+
+  if [ $dom0arch != amd64 ]; then
+    return
+  fi
+  if [ $branch != openstack-nova ]; then
+    return
+  fi
+
+  for xsm in $xsms ; do
+    job_create_test test-$xenarch$kern-$dom0arch-devstack \
+      test-devstack libvirt $xenarch $dom0arch \
+      tree_cinder=$TREE_OPENSTACK_CINDER \
+      tree_devstack=$TREE_OPENSTACK_DEVSTACK \
+      tree_glance=$TREE_OPENSTACK_GLANCE \
+      tree_keystone=$TREE_OPENSTACK_KEYSTONE \
+      tree_nova=$TREE_OPENSTACK_NOVA \
+      tree_requirements=$TREE_OPENSTACK_REQUIREMENTS \
+      tree_tempest=$TREE_OPENSTACK_TEMPEST \
+      revision_nova=$REVISION_OPENSTACK_NOVA \
+      revision_cinder=$REVISION_OPENSTACK_CINDER \
+      revision_devstack=$REVISION_OPENSTACK_DEVSTACK \
+      revision_glance=$REVISION_OPENSTACK_GLANCE \
+      revision_keystone=$REVISION_OPENSTACK_KEYSTONE \
+      revision_requirements=$REVISION_OPENSTACK_REQUIREMENTS \
+      revision_tempest=$REVISION_OPENSTACK_TEMPEST \
+      dom0_mem=4000 \
+      enable_xsm=$xsm \
+      all_hostflags=$most_hostflags
+  done
+}
+
 test_matrix_do_one () {
 
   do_pv_debian_tests
@@ -679,6 +729,8 @@ test_matrix_do_one () {
 
   do_pygrub_tests
   do_pvgrub_tests
+
+  do_openstack_tests
 }
 
 if [ x$buildflight = x ]; then
-- 
Anthony PERARD