From: Ian Campbell <Ian.Campbell@citrix.com>
To: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: xen-devel <xen-devel@lists.xen.org>
Subject: Re: [PATCH OSSTEST v3 00/19] add distro domU testing flight
Date: Mon, 1 Dec 2014 13:24:06 +0000 [thread overview]
Message-ID: <1417440246.29138.26.camel@citrix.com> (raw)
In-Reply-To: <1417438571.29138.22.camel@citrix.com>
On Mon, 2014-12-01 at 12:56 +0000, Ian Campbell wrote:
> The exception to that good news is that sg-report-flight
> --that-flight=NNN MMM doesn't seem to be considering failures in MMM
> regressions vs NNN. e.g. test-amd64-i386-i386-daily-netboot-pvgrub
> should appear as a regression from 31961 to 31964 but isn't. I'm going
> to have a poke around that.
This turned out to be as easy as passing --blessings=play to
sg-report-flight (the default is ["real", "real-bisect"]), which
produced the output below (previously everything was "Tests which did
not succeed, but are not blocking")
I have a feeling this is deliberate and that only real+bisects are
supposed to be considered here on purpose.
If not then I think the right answer is just to add this to sgr_args in
the relevant case:
--blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING}
Ian.
------
31964: regressions - FAIL
flight 31964 distros play [play]
http://www.chiark.greenend.org.uk/~xensrcts/logs/31964/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-i386-i386-current-netinst-pygrub 5 xen-boot fail REGR. vs. 31961
test-amd64-i386-i386-daily-netboot-pvgrub 7 debian-di-install fail REGR. vs. 31961
test-amd64-i386-i386-jessie-netboot-pvgrub 5 xen-boot fail REGR. vs. 31961
Regressions which are regarded as allowable (not blocking):
test-amd64-i386-i386-weekly-netinst-pygrub 7 debian-di-install fail like 31961
Tests which did not succeed, but are not blocking:
test-amd64-amd64-i386-sid-netboot-pvgrub 7 debian-di-install fail never pass
test-amd64-amd64-amd64-jessie-netboot-pvgrub 7 debian-di-install fail never pass
test-armhf-armhf-armhf-sid-netboot-pygrub 7 debian-di-install fail never pass
test-amd64-i386-i386-sid-netboot-pygrub 7 debian-di-install fail never pass
test-armhf-armhf-armhf-jessie-netboot-pygrub 7 debian-di-install fail never pass
test-armhf-armhf-armhf-daily-netboot-pygrub 7 debian-di-install fail never pass
test-amd64-amd64-i386-jessie-netboot-pygrub 7 debian-di-install fail never pass
test-amd64-amd64-amd64-sid-netboot-pygrub 7 debian-di-install fail never pass
test-amd64-i386-amd64-jessie-netboot-pygrub 7 debian-di-install fail never pass
test-amd64-i386-amd64-sid-netboot-pygrub 7 debian-di-install fail never pass
baseline version:
flight 31961
jobs:
build-amd64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
test-amd64-amd64-amd64-daily-netboot-pvgrub pass
test-amd64-i386-i386-daily-netboot-pvgrub fail
test-amd64-amd64-amd64-jessie-netboot-pvgrub fail
test-amd64-i386-i386-jessie-netboot-pvgrub fail
test-amd64-amd64-i386-sid-netboot-pvgrub fail
test-amd64-amd64-amd64-wheezy-netboot-pvgrub pass
test-amd64-i386-i386-wheezy-netboot-pvgrub pass
test-amd64-i386-amd64-daily-netboot-pygrub pass
test-armhf-armhf-armhf-daily-netboot-pygrub fail
test-amd64-amd64-i386-daily-netboot-pygrub pass
test-amd64-i386-amd64-jessie-netboot-pygrub fail
test-armhf-armhf-armhf-jessie-netboot-pygrub fail
test-amd64-amd64-i386-jessie-netboot-pygrub fail
test-amd64-amd64-amd64-sid-netboot-pygrub fail
test-amd64-i386-amd64-sid-netboot-pygrub fail
test-armhf-armhf-armhf-sid-netboot-pygrub fail
test-amd64-i386-i386-sid-netboot-pygrub fail
test-amd64-amd64-amd64-squeeze-netboot-pygrub pass
test-amd64-i386-amd64-squeeze-netboot-pygrub pass
test-amd64-amd64-i386-squeeze-netboot-pygrub pass
test-amd64-i386-i386-squeeze-netboot-pygrub pass
test-amd64-i386-amd64-wheezy-netboot-pygrub pass
test-amd64-amd64-i386-wheezy-netboot-pygrub pass
test-amd64-amd64-amd64-current-netinst-pygrub pass
test-amd64-i386-amd64-current-netinst-pygrub pass
test-amd64-amd64-i386-current-netinst-pygrub pass
test-amd64-i386-i386-current-netinst-pygrub fail
test-amd64-amd64-amd64-weekly-netinst-pygrub pass
test-amd64-i386-amd64-weekly-netinst-pygrub pass
test-amd64-amd64-i386-weekly-netinst-pygrub pass
test-amd64-i386-i386-weekly-netinst-pygrub fail
------------------------------------------------------------
sg-report-flight on kazak.uk.xensource.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
prev parent reply other threads:[~2014-12-01 13:24 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-01 12:56 [PATCH OSSTEST v3 00/19] add distro domU testing flight Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 01/19] TestSupport: Add helper to fetch a URL on a host Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 02/19] TestSupport: Add helper to wait for a guest to shutdown Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 03/19] TestSupport: allow overring of on_* in prepareguest_part_xencfg Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 04/19] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 05/19] create_webfile: Support use with guests as well as hosts Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 06/19] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 07/19] Debian: refactor preseeding of .ssh directories Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 08/19] Debian: ensure preseed base comment remains at actual end of base Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 09/19] Debian: Refactor installation of overlays, so it can be used for guests too Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 10/19] Debian: add preseed_create_guest helper Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 11/19] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs() Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 12/19] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 13/19] distros: support booting Debian PV (d-i installed) guests with pvgrub Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 14/19] distros: Support pvgrub for Wheezy too Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 15/19] distros: support PV guest install from Debian netinst media Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 16/19] Test pygrub and pvgrub on the regular flights Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 17/19] distros: add branch infrastructure Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 18/19] distros: Run a flight over the weekend Ian Campbell
2014-12-01 12:57 ` [PATCH OSSTEST v3 19/19] Debian: Handle lack of bootloader support in d-i on ARM Ian Campbell
2014-12-01 13:24 ` Ian Campbell [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1417440246.29138.26.camel@citrix.com \
--to=ian.campbell@citrix.com \
--cc=Ian.Jackson@eu.citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.