* [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances
@ 2015-04-15 10:04 Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
` (5 more replies)
0 siblings, 6 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 2368 bytes --]
Now that the Cambridge osstest instance is no longer the XenProject
master instance it needs some tweaks to allow us to keep using it for
local testing.
Since v2 I have now tested this in as close to how the osstest user in
Cambridge would, but in my own $HOME/testing.git and it appears to do
the right things. As part of that I made various fixes, in particular to
the ap-fetch-version stuff.
A test report email from my adhoc flight is below. (the Cambridge VM is
current knackered so the link doesn't work and it is only internally
available now).
WRT applying this I think normally we should apply such changes to the
upstream production instance and let them filter through, but in this
case I think it would be better to push my ad-hoc tested branch (based
on upstream "cambridge: Set HostProp_DhcpWatchMethod back to woking",
reference by pull req below) to the local Cambridge pretest, allow it to
automerge upstream master into that and test the result. Once that
passes I will send a pull-request to merge the result back into the
mainline.
Ian.
The following changes since commit eff0c7780505a610a39f995fb132318bd3f8647b:
cambridge: Set HostProp_DhcpWatchMethod back to woking (2015-04-02 12:03:58 +0100)
are available in the git repository at:
git://xenbits.xen.org/people/ianc/osstest.git non-prod-instances-v2
for you to fetch changes up to 76c499329a6a47117c2c397433b1f950f35dd741:
cambridge: Stop publishing logs to chiark (2015-04-04 08:53:06 +0100)
----------------------------------------------------------------
Ian Campbell (5):
cri-args-hostslists: allow instance specific settings
Handle osstest's own local push gate in non-master production instances
cambridge: Add a crontab
cambridge: Do not try to push harness to XenProject instance output
cambridge: Stop publishing logs to chiark
Osstest.pm | 2 ++
ap-common | 2 ++
ap-fetch-version | 25 ++++++++++++++++++++++---
ap-push | 8 +++++++-
cr-ensure-disk-space | 2 ++
cr-publish-flight-logs | 6 ++++--
cri-args-hostlists | 4 +++-
crontab-cambridge | 5 +++++
production-config | 2 ++
production-config-cambridge | 9 +--------
10 files changed, 50 insertions(+), 15 deletions(-)
create mode 100644 crontab-cambridge
[-- Attachment #2: Attached message - [osstest test] 37075: tolerable trouble: broken/fail/pass - PUSHED --]
[-- Type: message/rfc822, Size: 16256 bytes --]
From: Ian Campbell <ianc@eu.citrix.com>
To: <ian.jackson@eu.citrix.com>, <ian.campbell@eu.citrix.com>
Subject: [osstest test] 37075: tolerable trouble: broken/fail/pass - PUSHED
Date: Sat, 4 Apr 2015 19:16:17 +0100
Message-ID: <osstest-37075-mainreport@xen.org>
flight 37075 osstest play [play]
http://osstest.cam.xci-test.com/~osstest/testlogs/logs/37075/
Failures :-/ but no regressions.
Regressions which are regarded as allowable (not blocking):
test-amd64-amd64-xl-sedf 9 debian-install fail REGR. vs. 36584
test-armhf-armhf-xl-xsm 6 xen-boot fail like 36584
test-amd64-i386-pair 21 guest-migrate/src_host/dst_host fail like 36584
Tests which did not succeed, but are not blocking:
test-armhf-armhf-xl-multivcpu 18 capture-logs(18) broken never pass
test-armhf-armhf-xl 18 capture-logs(18) broken never pass
test-armhf-armhf-xl-sedf 18 capture-logs(18) broken never pass
test-armhf-armhf-libvirt-xsm 7 capture-logs(7) broken never pass
test-armhf-armhf-xl-sedf-pin 18 capture-logs(18) broken never pass
test-armhf-armhf-xl-credit2 18 capture-logs(18) broken never pass
test-armhf-armhf-xl-midway 18 capture-logs(18) broken never pass
test-amd64-amd64-rumpuserxen-amd64 15 rumpuserxen-demo-xenstorels/xenstorels.repeat fail never pass
test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
test-amd64-amd64-libvirt-xsm 11 guest-start fail never pass
test-amd64-amd64-xl-xsm 11 guest-start fail never pass
test-amd64-i386-xl-xsm 11 guest-start fail never pass
test-amd64-i386-libvirt-xsm 11 guest-start fail never pass
test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
test-amd64-amd64-xl-pvh-amd 11 guest-start fail never pass
test-amd64-amd64-xl-pvh-intel 11 guest-start fail never pass
test-amd64-i386-libvirt 12 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass
test-armhf-armhf-xl 12 migrate-support-check fail never pass
test-armhf-armhf-xl-sedf 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-xsm 6 xen-boot fail never pass
test-armhf-armhf-xl-sedf-pin 12 migrate-support-check fail never pass
test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt 15 guest-start/debian.repeat fail never pass
test-armhf-armhf-libvirt 12 migrate-support-check fail never pass
test-armhf-armhf-xl-midway 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt 12 migrate-support-check fail never pass
test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail never pass
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 16 guest-stop fail never pass
test-amd64-i386-xl-qemut-winxpsp3 16 guest-stop fail never pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 16 guest-stop fail never pass
test-amd64-amd64-xl-qemuu-winxpsp3 16 guest-stop fail never pass
test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail never pass
test-amd64-i386-xl-win7-amd64 16 guest-stop fail never pass
test-amd64-amd64-xl-win7-amd64 16 guest-stop fail never pass
test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
test-amd64-i386-xl-qemuu-winxpsp3 16 guest-stop fail never pass
test-amd64-i386-xl-winxpsp3-vcpus1 16 guest-stop fail never pass
test-amd64-i386-xl-winxpsp3 16 guest-stop fail never pass
test-amd64-amd64-xl-winxpsp3 16 guest-stop fail never pass
test-amd64-amd64-xl-qemut-winxpsp3 16 guest-stop fail never pass
version targeted for testing:
osstest 2d675fb0de966bae8046b936fe0afe574c6cd9a1
baseline version:
osstest a623769cbac25ae716ff9d8251116f95e2ee2531
jobs:
build-amd64-xsm pass
build-armhf-xsm pass
build-i386-xsm pass
build-amd64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
build-amd64-rumpuserxen pass
build-i386-rumpuserxen pass
test-amd64-amd64-xl pass
test-armhf-armhf-xl broken
test-amd64-i386-xl pass
test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm fail
test-amd64-i386-xl-qemut-debianhvm-amd64-xsm fail
test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm fail
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm fail
test-amd64-amd64-libvirt-xsm fail
test-armhf-armhf-libvirt-xsm broken
test-amd64-i386-libvirt-xsm fail
test-amd64-amd64-xl-xsm fail
test-armhf-armhf-xl-xsm fail
test-amd64-i386-xl-xsm fail
test-amd64-amd64-xl-pvh-amd fail
test-amd64-i386-rhel6hvm-amd pass
test-amd64-i386-qemut-rhel6hvm-amd pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemut-debianhvm-amd64 pass
test-amd64-i386-xl-qemut-debianhvm-amd64 pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-freebsd10-amd64 pass
test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
test-amd64-i386-xl-qemuu-ovmf-amd64 pass
test-amd64-amd64-rumpuserxen-amd64 fail
test-amd64-amd64-xl-qemut-win7-amd64 fail
test-amd64-i386-xl-qemut-win7-amd64 fail
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-amd64-amd64-xl-win7-amd64 fail
test-amd64-i386-xl-win7-amd64 fail
test-amd64-amd64-xl-credit2 pass
test-armhf-armhf-xl-credit2 broken
test-amd64-i386-freebsd10-i386 pass
test-amd64-i386-rumpuserxen-i386 pass
test-amd64-amd64-xl-pvh-intel fail
test-amd64-i386-rhel6hvm-intel pass
test-amd64-i386-qemut-rhel6hvm-intel pass
test-amd64-i386-qemuu-rhel6hvm-intel pass
test-amd64-amd64-libvirt pass
test-armhf-armhf-libvirt fail
test-amd64-i386-libvirt pass
test-armhf-armhf-xl-midway broken
test-amd64-amd64-xl-multivcpu pass
test-armhf-armhf-xl-multivcpu broken
test-amd64-amd64-pair pass
test-amd64-i386-pair fail
test-amd64-amd64-xl-sedf-pin pass
test-armhf-armhf-xl-sedf-pin broken
test-amd64-amd64-xl-sedf fail
test-armhf-armhf-xl-sedf broken
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
test-amd64-i386-xl-winxpsp3-vcpus1 fail
test-amd64-amd64-xl-qemut-winxpsp3 fail
test-amd64-i386-xl-qemut-winxpsp3 fail
test-amd64-amd64-xl-qemuu-winxpsp3 fail
test-amd64-i386-xl-qemuu-winxpsp3 fail
test-amd64-amd64-xl-winxpsp3 fail
test-amd64-i386-xl-winxpsp3 fail
------------------------------------------------------------
sg-report-flight on osstest.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://osstest.cam.xci-test.com/~osstest/testlogs/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Pushing revision :
+ branch=osstest
+ revision=2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/ianc/repos
++ repos_lock=/export/home/ianc/repos/lock
++ '[' x '!=' x/export/home/ianc/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/export/home/ianc/repos/lock
++ exec with-lock-ex -w /export/home/ianc/repos/lock ./ap-push osstest 2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ branch=osstest
+ revision=2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/ianc/repos
++ repos_lock=/export/home/ianc/repos/lock
++ '[' x/export/home/ianc/repos/lock '!=' x/export/home/ianc/repos/lock ']'
+ . cri-common
++ . cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=osstest
+ xenbranch=xen-unstable
+ '[' xosstest = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ : tested/2.6.39.x
+ . ap-common
++ : osstest@xenbits.xen.org
++ : /export/home/ianc/testing.git remote-pretest
++ : git://xenbits.xen.org/xen.git
++ : osstest@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/staging/qemu-xen-unstable.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osstest@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osstest@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src '[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
++++ getconfig GitCacheProxy
++++ perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://drall.uk.xensource.com:9419/
+++ '[' xgit://drall.uk.xensource.com:9419/ '!=' x ']'
+++ echo 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : git
++ : git://git.seabios.org/seabios.git
++ : osstest@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osstest@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osstest@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osstest@xenbits.xen.org:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' xgit://xenbits.xen.org/linux-pvops.git = x ']'
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27
++ : git://xenbits.xen.org/staging/qemu-upstream-unstable.git
++ : daily-cron.osstest
++ : git://git.qemu.org/qemu.git
++ : git://xenbits.xen.org/osstest/qemu.git
++ : osstest@xenbits.xen.org:/home/xen/git/osstest/qemu.git
++ : daily-cron.osstest
+ TREE_LINUX=osstest@xenbits.xen.org:/home/xen/git/linux-pvops.git
+ TREE_QEMU_MAINLINE=osstest@xenbits.xen.org:/home/xen/git/osstest/qemu.git
+ TREE_QEMU_UPSTREAM=osstest@xenbits.xen.org:/home/xen/git/qemu-upstream-unstable.git
+ TREE_XEN=osstest@xenbits.xen.org:/home/xen/git/xen.git
+ TREE_LIBVIRT=osstest@xenbits.xen.org:/home/xen/git/libvirt.git
+ TREE_RUMPUSERXEN=osstest@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+ TREE_SEABIOS=osstest@xenbits.xen.org:/home/xen/git/osstest/seabios.git
+ TREE_OVMF=osstest@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
+ info_linux_tree osstest
+ case $1 in
+ return 1
+ case "$branch" in
+ git push /export/home/ianc/testing.git 2d675fb0de966bae8046b936fe0afe574c6cd9a1:production
To /export/home/ianc/testing.git
a623769..2d675fb 2d675fb0de966bae8046b936fe0afe574c6cd9a1 -> production
+ '[' 'x/export/home/ianc/testing.git remote-pretest' '!=' x ']'
+ :
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
@ 2015-04-15 10:04 ` Ian Campbell
2015-04-15 11:32 ` Ian Jackson
2015-04-15 10:04 ` [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
` (4 subsequent siblings)
5 siblings, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
can contain things like "export
OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
particular instance of osstest running in production mode.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
cri-args-hostlists | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/cri-args-hostlists b/cri-args-hostlists
index 04cd6be..bf847f8 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -16,7 +16,9 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
+if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
+ source $HOME/.xen-osstest/cri-args-hostslists.settings
+fi
export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
check_stop_core () {
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
@ 2015-04-15 10:04 ` Ian Campbell
2015-04-15 11:29 ` Ian Jackson
2015-04-20 13:14 ` Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 3/5] cambridge: Add a crontab Ian Campbell
` (3 subsequent siblings)
5 siblings, 2 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
We want to arrange that the master XenProject instance continues to
test its own pretest branch while any downstream instances will pickup
changes from the master instance's production (i.e. tested) branch,
which is published at git://xenbits.xen.org/osstest.git#master.
Add a new configuration option OsstestUpstream which by default is
"git://xenbits.xen.org/osstest.git master" and which is cleared to
nothing on the master instance via production-config.
If the option is not set then the existing behaviour is unchanged.
If the option is set then osstest branch flights proceed by merging,
the referenced branch into the local pretest branch and testing the
result.
This must be done:
- in a clone not in the main testing.git in order to avoid inserting
merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.
$repos/osstest is a bare repo which we want to keep that way because
using repo_tree_rev_fetch_git to fetch the remote branch is
convenient.
So we use $repos/osstest-merge as a temporary merge repo and reclone
from the active local repo each time.
All of this happens in ap-fetch version which also fetches the result
into FETCH_HEAD into the active repo (the one running the test), such
that the existing hard reset in cr-daily-branch can find the commit.
If the merge fails then manual intervention (i.e. a manual merge and
push to the _local_ pretest) will be required.
In ap-push we stop pushing to xenbits#master except for the master
instance if an upstream is defined. At some point it might be useful
to add a configuration option for where to push to but I don't have
that requirement right now.
ap-fetch-version-old requires no changes.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2:
- Arrange for $OSSTEST_USE_HEAD=y to take precendence
- drop LOCALREV (which was wrong anyway) in favour of inline
branchname
- Rename OSSTEST_REVISION_MERGE as revision_merge to avoid implying
it can be set and will be honoured.
- Git in Debian Squeeze lacks -C and --no-edit, adjust accordingly.
---
Osstest.pm | 2 ++
ap-common | 2 ++
ap-fetch-version | 25 ++++++++++++++++++++++---
ap-push | 8 +++++++-
production-config | 2 ++
5 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/Osstest.pm b/Osstest.pm
index 7989129..7f72bc6 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -102,6 +102,8 @@ sub readglobalconfig () {
return if $readglobalconfig_done;
$readglobalconfig_done=1;
+ $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
$c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
$c{AuthorizedKeysFiles} = '';
$c{AuthorizedKeysAppend} = '';
diff --git a/ap-common b/ap-common
index 64749e3..de7bb1d 100644
--- a/ap-common
+++ b/ap-common
@@ -21,6 +21,8 @@
: ${XENBITS:=osstest@xenbits.xen.org}
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
: ${TREE_XEN:=git://xenbits.xen.org/xen.git}
: ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
diff --git a/ap-fetch-version b/ap-fetch-version
index 33aaf00..ee502c6 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -90,11 +90,30 @@ ovmf)
$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
;;
osstest)
- if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+ if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
+ git rev-parse HEAD^0
+ elif [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+ revision_merge=`repo_tree_rev_fetch_git osstest \
+ $TREEBRANCH_OSSTEST_UPSTREAM daily-cron.$branch`
+ echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $revision_merge"
+
+ rm -rf $repos/osstest-merge >&2
+ git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+ (
+ cd $repos/osstest-merge
+ git fetch $repos/osstest daily-cron.$branch:ap-merge >&2
+ git merge \
+ -m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into pretest" \
+ ap-merge >&2
+ )
+
+ # cr-daily-branch will reset the local tree to this version.
+ git fetch $repos/osstest-merge pretest
+ git rev-parse FETCH_HEAD^0
+ else
git fetch $HOME/testing.git pretest:ap-fetch >&2
git rev-parse ap-fetch^0
- else
- git rev-parse HEAD^0
fi
;;
*)
diff --git a/ap-push b/ap-push
index c141464..5967b42 100755
--- a/ap-push
+++ b/ap-push
@@ -99,7 +99,13 @@ ovmf)
;;
osstest)
git push $HOME/testing.git $revision:production
- git push $XENBITS:/home/xen/git/osstest.git $revision:master
+ if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+ # could push to instance specific location, but
+ # certainly not to master instance's xenbits repo!
+ :
+ else
+ git push $XENBITS:/home/xen/git/osstest.git $revision:master
+ fi
;;
*)
echo >&2 "branch $branch ?"
diff --git a/production-config b/production-config
index 8f20c27..84a3bb7 100644
--- a/production-config
+++ b/production-config
@@ -15,6 +15,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
JobDB Executive
HostDB Executive
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH OSSTEST v2 3/5] cambridge: Add a crontab
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
@ 2015-04-15 10:04 ` Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
` (2 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Right now all it does is keep osstest itself up to date.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
crontab-cambridge | 5 +++++
1 file changed, 5 insertions(+)
create mode 100644 crontab-cambridge
diff --git a/crontab-cambridge b/crontab-cambridge
new file mode 100644
index 0000000..ef0d234
--- /dev/null
+++ b/crontab-cambridge
@@ -0,0 +1,5 @@
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
+# m h dom mon dow command
+49 1 * * * cd testing.git && BRANCHES=osstest ./cr-for-branches branches -w "./cr-daily-branch --real"
+3 4 * * * savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH OSSTEST v2 4/5] cambridge: Do not try to push harness to XenProject instance output
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
` (2 preceding siblings ...)
2015-04-15 10:04 ` [PATCH OSSTEST v2 3/5] cambridge: Add a crontab Ian Campbell
@ 2015-04-15 10:04 ` Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
2015-04-15 11:33 ` [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Jackson
5 siblings, 0 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
By arranging for cr-publish-flight-logs to ignore --push-harness if
either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are
not specified
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2:
- Avoid logm which isn't available here, wasn't saying much of use
anyway.
- Syntax fix (is not a function, so exit not return)
Perhaps we should have our own tree for such things, but for now just
nobble it.
---
cr-publish-flight-logs | 2 ++
production-config-cambridge | 3 ---
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/cr-publish-flight-logs b/cr-publish-flight-logs
index 1feb52d..2e96f60 100755
--- a/cr-publish-flight-logs
+++ b/cr-publish-flight-logs
@@ -43,6 +43,8 @@ if ($push_harness) {
my $githost= $c{HarnessPublishGitUserHost};
my $gitdir= $c{HarnessPublishGitRepoDir};
+ exit 0 if (!$githost || !$gitdir);
+
system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight");
system_checked("ssh $githost 'cd $gitdir && git update-server-info'");
}
diff --git a/production-config-cambridge b/production-config-cambridge
index 1404d10..e44f3b0 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -50,9 +50,6 @@ GlobalLockDir /export/home/osstest/testing.git
LogsPublish= "$c{Publish}/logs"
ResultsPublish= "$c{Publish}/results"
-HarnessPublishGitUserHost xen@xenbits.xensource.com
-HarnessPublishGitRepoDir git/osstest.git
-
ReportTrailer= <<END
Logs, config files, etc. are available at
$c{ReportHtmlPubBaseUrl}
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
` (3 preceding siblings ...)
2015-04-15 10:04 ` [PATCH OSSTEST v2 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
@ 2015-04-15 10:04 ` Ian Campbell
2015-04-15 11:31 ` Ian Jackson
2015-04-15 11:33 ` [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Jackson
5 siblings, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
points to the live logs directory, so switch PubBaseUrl to that in the
Cambridge config such that email reports etc contain it. This won't be
externally accessible but I think that won't matter now that the
master production instance is elsewhere.
Arrange that cr-publish-flight-logs doesn't publish the corresponding
thing if either LogsPublish or ResultsPublish is not set, and unset
them in the Cambridge config.
Likewise arrange that cr-ensure-disk-space doesn't do anything if the
configuration variable passed as an option is not set, and unset
Publish (the base for {Logs,Results}Publish) in the Cambridge config.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Check the config variable and not its name.
---
cr-ensure-disk-space | 2 ++
cr-publish-flight-logs | 4 ++--
production-config-cambridge | 6 +-----
3 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
index 0ee84c7..3788e63 100755
--- a/cr-ensure-disk-space
+++ b/cr-ensure-disk-space
@@ -42,6 +42,8 @@ our ($cfgbase) = @ARGV;
csreadconfig();
+exit 0 unless $c{$cfgbase};
+
open LOCK, "> $c{GlobalLockDir}/publish-lock" or die $!;
flock LOCK, LOCK_EX or die $!;
diff --git a/cr-publish-flight-logs b/cr-publish-flight-logs
index 2e96f60..94c74c8 100755
--- a/cr-publish-flight-logs
+++ b/cr-publish-flight-logs
@@ -59,5 +59,5 @@ sub copydir ($$) {
$!=0; $?=0; system @cmd; die "rsync $? $!" if $? or $!;
}
-copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight");
-copydir("$c{Results}/", "$c{ResultsPublish}");
+copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight") if $c{LogsPublish};
+copydir("$c{Results}/", "$c{ResultsPublish}") if $c{ResultsPublish};
diff --git a/production-config-cambridge b/production-config-cambridge
index e44f3b0..a144cef 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -40,16 +40,12 @@ TestHostKeypairPath /export/home/osstest/.ssh/id_rsa_osstest
GitCacheProxy git://drall.uk.xensource.com:9419/
-PubBaseUrl http://www.chiark.greenend.org.uk/~xensrcts
+PubBaseUrl http://osstest.cam.xci-test.com/~osstest/testlogs
ReportHtmlPubBaseUrl="$c{PubBaseUrl}/logs"
ResultsHtmlPubBaseUrl="$c{PubBaseUrl}/results"
-Publish xensrcts@login.chiark.greenend.org.uk:/home/ian/work/xc_osstest
GlobalLockDir /export/home/osstest/testing.git
-LogsPublish= "$c{Publish}/logs"
-ResultsPublish= "$c{Publish}/results"
-
ReportTrailer= <<END
Logs, config files, etc. are available at
$c{ReportHtmlPubBaseUrl}
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances
2015-04-15 10:04 ` [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
@ 2015-04-15 11:29 ` Ian Jackson
2015-04-20 13:14 ` Ian Campbell
1 sibling, 0 replies; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 11:29 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances"):
> We want to arrange that the master XenProject instance continues to
> test its own pretest branch while any downstream instances will pickup
> changes from the master instance's production (i.e. tested) branch,
> which is published at git://xenbits.xen.org/osstest.git#master.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark
2015-04-15 10:04 ` [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
@ 2015-04-15 11:31 ` Ian Jackson
0 siblings, 0 replies; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 11:31 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark"):
> http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
> points to the live logs directory, so switch PubBaseUrl to that in the
> Cambridge config such that email reports etc contain it. This won't be
> externally accessible but I think that won't matter now that the
> master production instance is elsewhere.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings
2015-04-15 10:04 ` [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
@ 2015-04-15 11:32 ` Ian Jackson
2015-04-15 11:57 ` Ian Campbell
0 siblings, 1 reply; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 11:32 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings"):
> In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
> can contain things like "export
> OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
> particular instance of osstest running in production mode.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
...
> +if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
> + source $HOME/.xen-osstest/cri-args-hostslists.settings
> +fi
> export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
At some point we should remove this last line and put it in various
~/.xen-osstest/cri-args-hostslists.settings files in the test lab.
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
` (4 preceding siblings ...)
2015-04-15 10:04 ` [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
@ 2015-04-15 11:33 ` Ian Jackson
2015-04-15 11:58 ` Ian Campbell
5 siblings, 1 reply; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 11:33 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):
> WRT applying this I think normally we should apply such changes to the
> upstream production instance and let them filter through, but in this
> case I think it would be better to push my ad-hoc tested branch (based
> on upstream "cambridge: Set HostProp_DhcpWatchMethod back to woking",
> reference by pull req below) to the local Cambridge pretest, allow it to
> automerge upstream master into that and test the result. Once that
> passes I will send a pull-request to merge the result back into the
> mainline.
I think you are probably right, since this particular series contains
the merging-from-an-upsream machinery.
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings
2015-04-15 11:32 ` Ian Jackson
@ 2015-04-15 11:57 ` Ian Campbell
2015-04-15 13:47 ` Ian Jackson
0 siblings, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 11:57 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Wed, 2015-04-15 at 12:32 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings"):
> > In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
> > can contain things like "export
> > OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
> > particular instance of osstest running in production mode.
> >
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ...
> > +if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
> > + source $HOME/.xen-osstest/cri-args-hostslists.settings
> > +fi
> > export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
>
> At some point we should remove this last line and put it in various
> ~/.xen-osstest/cri-args-hostslists.settings files in the test lab.
Makes sense. I'd be happy to do that straightaway if you would prefer.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances
2015-04-15 11:33 ` [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Jackson
@ 2015-04-15 11:58 ` Ian Campbell
2015-04-15 13:47 ` Ian Jackson
0 siblings, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-15 11:58 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Wed, 2015-04-15 at 12:33 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):
> > WRT applying this I think normally we should apply such changes to the
> > upstream production instance and let them filter through, but in this
> > case I think it would be better to push my ad-hoc tested branch (based
> > on upstream "cambridge: Set HostProp_DhcpWatchMethod back to woking",
> > reference by pull req below) to the local Cambridge pretest, allow it to
> > automerge upstream master into that and test the result. Once that
> > passes I will send a pull-request to merge the result back into the
> > mainline.
>
> I think you are probably right, since this particular series contains
> the merging-from-an-upsream machinery.
Since you have now acked everything may I push to the Cambridge branch
(once the VM is working again)?
(modulo if you want me to make the minor change discussed on 1/5 now or
not)
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings
2015-04-15 11:57 ` Ian Campbell
@ 2015-04-15 13:47 ` Ian Jackson
0 siblings, 0 replies; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 13:47 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("Re: [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings"):
> On Wed, 2015-04-15 at 12:32 +0100, Ian Jackson wrote:
> > At some point we should remove this last line and put it in various
> > ~/.xen-osstest/cri-args-hostslists.settings files in the test lab.
>
> Makes sense. I'd be happy to do that straightaway if you would prefer.
I don't mind. Feel free to make a 6/5 which does this, or fold it in,
or whatever.
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances
2015-04-15 11:58 ` Ian Campbell
@ 2015-04-15 13:47 ` Ian Jackson
2015-04-17 9:32 ` Ian Campbell
0 siblings, 1 reply; 18+ messages in thread
From: Ian Jackson @ 2015-04-15 13:47 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):
> Since you have now acked everything may I push to the Cambridge branch
> (once the VM is working again)?
Yes.
> (modulo if you want me to make the minor change discussed on 1/5 now or
> not)
Either way is fine.
Thanks,
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances
2015-04-15 13:47 ` Ian Jackson
@ 2015-04-17 9:32 ` Ian Campbell
2015-04-17 9:40 ` [PATCH OSSTEST] cri-args-hostlists: No longer export $OSSTEST_CONFIG Ian Campbell
0 siblings, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-17 9:32 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Wed, 2015-04-15 at 14:47 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):
> > Since you have now acked everything may I push to the Cambridge branch
> > (once the VM is working again)?
>
> Yes.
>
> > (modulo if you want me to make the minor change discussed on 1/5 now or
> > not)
>
> Either way is fine.
I created the file on both sites, but I didn't change this patch since
we are pushing this to Cambridge first if it does cause an issue when
merged back into mainline it'll be annoying to cleanup, better to let a
future change go through the usual mainline gate alone.
I've just pushed the following, which is a bunch of stuff from upstream
osstest and these five patches, to Cambridge pretest and then:
$ rm branches/for-osstest.git/stop
Lets see if the VM stays up long enough to test them!
Ian.
$ git log --oneline a623769..41ac504
41ac504 cambridge: Stop publishing logs to chiark
bcc9c68 cambridge: Do not try to push harness to XenProject instance output
6196543 cambridge: Add a crontab
7007b0a Handle osstest's own local push gate in non-master production instances
5926203 cri-args-hostslists: allow instance specific settings
eff0c77 cambridge: Set HostProp_DhcpWatchMethod back to woking
8f1dc3f ts-host-ping-check: Do not run if host is being reused
ba691ef standalone: Add --dry-run option for run-job.
effe70a tcl: Handle environment variables which are unset.
4a938b2 Merge branch 'production' of osstest.test-lab:/home/osstest/testing into wip
2d1aa25 production-config: Adjust LogsMinSpaceMby to not be half of the VM's disk
5ae027f Merge branch 'colo.2015-03-31' of xenbits.xen.org:/home/iwj/ext/osstest into queue
4bf33f8 production-config*: Set HostnameSortSwapWords as appropriate
e2008dc (Re)Add production config for Cambridge instance
f89d6ac Osstest.pm: Add "use IO::File"
0ab790c ts-host-install: honour arch-specific boot append properties
e654ecc ts-host-ping-check: New ubiquitous test step
a80d215 sg-report-flight: Produce better output for running jobs.
6ec412e sg-report-flight: Sort email output by results, not job name
6cbaf65 cs-adjust-flight: Support negation of foo-specs
fdddba7 config: turn off HostnameSortSwapWords by default
ea335bc start/stop the guest 10 times in the standard test jobs
4d6bbe0 README.dev: Document steps after restarting ms-ownerdaemon
97fa466 uboot: make flask loading address host property
ab441f3 uboot: do not use readlink on xsm policy
dce7456 Merge remote-tracking branch 'origin/incoming' into wip.colo
bf3a468 Remove passthrough tests
7d87cf0 Empty commit to force test run
bc50a13 Configs: Replace xenbits.xensource.com with xenbits.xen.org
da7c22d mg-branch-setup: Make initial push optional in mg-branch-setup
831c94e ts-debian-install: Honour DebianMirrorProxy, HttpProxy
a141aca production-config: Configuration for new production colo
310c5b5 Branches: Rename "incoming" branch to "production"
b294e14 config defaults: Do not use woking for dhcp leases
2fedf11 ts-kernel-build: enable CONFIG_SCSI_HPSA
347eadf ts-kernel-build: enable CONFIG_SCSI_ISCI
591ed47 ts-kernel-build: Document a hacky way to get CONFIG_... options
4d0dba4 HostnameSortSwapWords: Make name order mangling configurable
a4debc0 mg-execute-flight: New -P (publish) option
41e9bf0 cr-ensure-disk-space: Honour different config for published log expiry
2ad7b3f cr-ensure-disk-space: Operate locally entirely separately.
2563b91 cr-ensure-disk-space: Permit argument to specify local directory.
dd6be0f cr-ensure-disk-space: Fix -n option
43a0974 cr-ensure-disk-space: Take argument for what to clean
1d25a8c cr-ensure-disk-space: Do listing of flights on target.
d6edb5d cr-ensure-disk-space: Do df check on target.
438637a cr-ensure-disk-space: Break out ontarget
fdc2659 cr-ensure-disk-space: Move LogsPublish decomposition up
d8b13ac cr-ensure-disk-space: Emacs perl-mode syntax workaround
f16e621 daemons: Allow QueueDaemon and OwnerDaemon to run on different hosts
0ff3bb1 target_cmd_build: Honour $c{HttpProxy}
a11fc63 HostDB Executive: Option to suppress "configdb"
2b5aa29 readglobalconfig: Support Include
f5b354a readglobalconfig: Break out $readcfg as a sub
bf2b666 Debian mirrors: Support proxy, and default to ftp.debian.org if specified
eb805e8 pxe setup: mg-hosts mkpxedir has -n (dry run) option
a865cc2 pxe setup: New TftpPxeTemplatesReal feature
14e0e8d Osstest: Default config: Use foreach for empty defaults
bbcce4d mg-hosts: Provide new "power" subcommand
68f35a1 pxe setup: Define %name% in templates
e8a5b82 PDU: pdu-msw: Support APC v6 firmware
9705c37 PDU: pdu-msw: Split $read_oid and $write_oid
0fb0a00 PDU: msw: support arbitrary arguments
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH OSSTEST] cri-args-hostlists: No longer export $OSSTEST_CONFIG
2015-04-17 9:32 ` Ian Campbell
@ 2015-04-17 9:40 ` Ian Campbell
0 siblings, 0 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-17 9:40 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
All sites now have a suitable
$HOME/.xen-osstest/cri-args-hostslists.settings in place which does
this.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
To be applied once the following commit passes Cambridge push gate, is
merged into the master instance and passes the push gate there.
commit 5926203d0851792e4f6ce26df5a48ba5c79456e1
Author: Ian Campbell <ian.campbell@citrix.com>
Date: Wed Apr 15 11:04:53 2015 +0100
cri-args-hostslists: allow instance specific settings
In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
can contain things like "export
OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
particular instance of osstest running in production mode.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
cri-args-hostlists | 1 -
1 file changed, 1 deletion(-)
diff --git a/cri-args-hostlists b/cri-args-hostlists
index bf847f8..6aa6927 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -19,7 +19,6 @@
if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
source $HOME/.xen-osstest/cri-args-hostslists.settings
fi
-export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
check_stop_core () {
if [ "x$OSSTEST_IGNORE_STOP" = xy ]; then return; fi
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances
2015-04-15 10:04 ` [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
2015-04-15 11:29 ` Ian Jackson
@ 2015-04-20 13:14 ` Ian Campbell
2015-04-20 13:15 ` Ian Campbell
1 sibling, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2015-04-20 13:14 UTC (permalink / raw)
To: ian.jackson; +Cc: xen-devel
On Wed, 2015-04-15 at 11:04 +0100, Ian Campbell wrote:
> If the option is set then osstest branch flights proceed by merging,
> the referenced branch into the local pretest branch and testing the
> result.
This turned out to not be quite right in practice. Since pretest is not
updated, in the absence of anything actually being changed on each
iteration, git would create a new merge, identical to the last time
around but for the date, test that and push the result to production.
Creating lots of little branches branches in the history. Not what we
want at all.
After some discussion IRL and drawing some pictures on a whiteboard here
is a replacement version.
I'm about to kick off an adhoc test on the Cambridge VM, but thought I'd
best send this out while it was fresh in our heads...
Ian.
>From 0ad809cacd931d68d92dd9dd0f8a94f7cb01c94d Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Wed, 1 Apr 2015 13:50:50 +0100
Subject: [PATCH] Handle osstest's own local push gate in non-master production
instances
We want to arrange that the master XenProject instance continues to
test its own pretest branch while any downstream instances will pickup
changes from the master instance's production (i.e. tested) branch,
which is published at git://xenbits.xen.org/osstest.git#master. We
want to also be able to use local pretest for local changes (which may
or may not get merged back upstream).
Add a new configuration option OsstestUpstream which by default is
"git://xenbits.xen.org/osstest.git master" and which is cleared to
nothing on the master instance via production-config.
If the option is not set then the existing behaviour is unchanged.
If the option is set then osstest branch flights will still prefer to
test the local pretest branch, but if nothing is pending there then it
will proceed by merging the upstream branch into the local production
branch and testing the result.
This merge must be done:
- in a clone not in the main testing.git in order to avoid inserting
merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.
$repos/osstest is a bare repo which we want to keep that way because
using repo_tree_rev_fetch_git to fetch the remote branch is
convenient.
So we use $repos/osstest-merge as a temporary merge repo and reclone
from the active local repo each time.
All of this happens in ap-fetch-version.
As part of this arrange that the result is always left in the ap-fetch
branch of the for-osstest.git repo (even for existing cases) and the
sha1 is produced as output. Resetting to that revision is handled by
cr-daily-branch.
If the merge fails then manual intervention (i.e. a manual merge and
push to the _local_ pretest) will be required. Likewise if local
pretest and local production have diverged manual intervention will be
required.
In ap-push we stop pushing to xenbits#master except for the master
instance if an upstream is defined. At some point it might be useful
to add a configuration option for where to push to but I don't have
that requirement right now.
ap-fetch-version-old requires no changes.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3:
- Only merge from upstream if there is nothing pending locally.
- Always update ap-fetch.
v2:
- Arrange for $OSSTEST_USE_HEAD=y to take precendence
- drop LOCALREV (which was wrong anyway) in favour of inline
branchname
- Rename OSSTEST_REVISION_MERGE as revision_merge to avoid implying
it can be set and will be honoured.
- Git in Debian Squeeze lacks -C and --no-edit, adjust accordingly.
---
Osstest.pm | 2 ++
ap-common | 2 ++
ap-fetch-version | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
ap-push | 8 +++++-
production-config | 2 ++
5 files changed, 82 insertions(+), 5 deletions(-)
diff --git a/Osstest.pm b/Osstest.pm
index 7989129..7f72bc6 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -102,6 +102,8 @@ sub readglobalconfig () {
return if $readglobalconfig_done;
$readglobalconfig_done=1;
+ $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
$c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
$c{AuthorizedKeysFiles} = '';
$c{AuthorizedKeysAppend} = '';
diff --git a/ap-common b/ap-common
index 64749e3..de7bb1d 100644
--- a/ap-common
+++ b/ap-common
@@ -21,6 +21,8 @@
: ${XENBITS:=osstest@xenbits.xen.org}
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
: ${TREE_XEN:=git://xenbits.xen.org/xen.git}
: ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
diff --git a/ap-fetch-version b/ap-fetch-version
index 33aaf00..41486b3 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -90,12 +90,77 @@ ovmf)
$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
;;
osstest)
- if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
- git fetch $HOME/testing.git pretest:ap-fetch >&2
- git rev-parse ap-fetch^0
+ if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
+ git update-ref -m "Arranging to test HEAD" \
+ ap-fetch HEAD^0
+ elif [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+ # Deciding what to test here is somewhat complex because
+ # we would prefer to avoid octopus merges and to try and
+ # keep the change history as clean as reasonably possible.
+ #
+ # Therefore we prefer to test anything which is in local
+ # pretest before testing anything which is in
+ # upstream. Whether or not there is anything in local
+ # pretest is determined by using "git merge base"
+
+ git fetch $HOME/testing.git +production:production
+ git fetch $HOME/testing.git +pretest:pretest
+
+ local_production=$(git rev-parse production^0)
+ local_pretest=$(git rev-parse pretest^0)
+
+ merge_base=$(git merge-base $local_production $local_pretest)
+
+ echo >&2 "Considering merge from upstream"
+ echo >&2 "Local production: $local_production"
+ echo >&2 "Local pretest: $local_pretest"
+ echo >&2 "Merge base: $merge_base"
+
+ if [ x$merge_base = x$local_pretest ] ; then
+ # local production is a ff of local pretest
+ # (i.e. there is nothing local to test). So merge from
+ # upstream master into production and test that.
+
+ revision_merge=`repo_tree_rev_fetch_git osstest \
+ $TREEBRANCH_OSSTEST_UPSTREAM daily-cron.$branch`
+ echo >&2 "Testing $TREEBRANCH_OSSTEST_UPSTREAM = $revision_merge"
+
+ rm -rf $repos/osstest-merge >&2
+ git clone -b production $HOME/testing.git $repos/osstest-merge >&2
+
+ (
+ cd $repos/osstest-merge
+ git fetch $repos/osstest daily-cron.$branch:ap-merge >&2
+ git merge \
+ -m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into production" \
+ ap-merge >&2
+ )
+
+ # Fetch the result of the above merge into local tree
+ # ready for use. cr-daily-branch handles reseting the
+ # working tree to this version
+ git fetch $repos/osstest-merge +production:ap-fetch
+ elif [ x$merge_base = x$local_production ] ; then
+ # there are commits in pretest which we should test in
+ # an attempt to progress production to this point.
+ echo >&2 "Testing local pretest"
+ git update-ref -m "Arranging to test local pretest" \
+ ap-fetch $local_pretest
+ else
+ # It's too hard for to deal with automatically. This
+ # might happen if a push to pretest races with a
+ # successful flight which merges from upstream. We
+ # expect pushes to pretest to be rare in practice.
+ echo >&2 "************************************************"
+ echo >&2 "production and pretest have diverged."
+ echo >&2 "need manual fixup/rebase and push to pretest."
+ echo >&2 "************************************************"
+ fi
else
- git rev-parse HEAD^0
+ git fetch $HOME/testing.git pretest:ap-fetch >&2
fi
+
+ git rev-parse ap-fetch^0
;;
*)
echo >&2 "branch $branch ?"
diff --git a/ap-push b/ap-push
index c141464..5967b42 100755
--- a/ap-push
+++ b/ap-push
@@ -99,7 +99,13 @@ ovmf)
;;
osstest)
git push $HOME/testing.git $revision:production
- git push $XENBITS:/home/xen/git/osstest.git $revision:master
+ if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+ # could push to instance specific location, but
+ # certainly not to master instance's xenbits repo!
+ :
+ else
+ git push $XENBITS:/home/xen/git/osstest.git $revision:master
+ fi
;;
*)
echo >&2 "branch $branch ?"
diff --git a/production-config b/production-config
index 8f20c27..84a3bb7 100644
--- a/production-config
+++ b/production-config
@@ -15,6 +15,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
JobDB Executive
HostDB Executive
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances
2015-04-20 13:14 ` Ian Campbell
@ 2015-04-20 13:15 ` Ian Campbell
0 siblings, 0 replies; 18+ messages in thread
From: Ian Campbell @ 2015-04-20 13:15 UTC (permalink / raw)
To: ian.jackson; +Cc: xen-devel
On Mon, 2015-04-20 at 14:14 +0100, Ian Campbell wrote:
> I'm about to kick off an adhoc test on the Cambridge VM
Except the filer seems to have just died for the second time today. I
suspect I'll be putting a hold on this work until we can switch over to
reliable infrastructure (which is in progress AIUI)
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-04-20 13:15 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-15 10:04 [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
2015-04-15 11:32 ` Ian Jackson
2015-04-15 11:57 ` Ian Campbell
2015-04-15 13:47 ` Ian Jackson
2015-04-15 10:04 ` [PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
2015-04-15 11:29 ` Ian Jackson
2015-04-20 13:14 ` Ian Campbell
2015-04-20 13:15 ` Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 3/5] cambridge: Add a crontab Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
2015-04-15 10:04 ` [PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
2015-04-15 11:31 ` Ian Jackson
2015-04-15 11:33 ` [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances Ian Jackson
2015-04-15 11:58 ` Ian Campbell
2015-04-15 13:47 ` Ian Jackson
2015-04-17 9:32 ` Ian Campbell
2015-04-17 9:40 ` [PATCH OSSTEST] cri-args-hostlists: No longer export $OSSTEST_CONFIG Ian Campbell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.