* [OSSTEST PATCH 2/4] README: Better documentation of recipes, db, etc.
2017-09-11 9:54 [OSSTEST PATCH 1/4] examine: Do not try to find old version Ian Jackson
@ 2017-09-11 9:54 ` Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 3/4] README.dev: Miscellaneous minor improvements Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 4/4] README.dev: Improve instructions for new machine commissioning Ian Jackson
2 siblings, 0 replies; 4+ messages in thread
From: Ian Jackson @ 2017-09-11 9:54 UTC (permalink / raw)
To: xen-devel; +Cc: Ian Jackson, George Dunlap
CC: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
README | 75 ++++++++++++++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 60 insertions(+), 15 deletions(-)
diff --git a/README b/README
index ffe0018..93129e3 100644
--- a/README
+++ b/README
@@ -8,32 +8,55 @@ Terminology
"flight":
- Each run of osstest is referred to as a "flight". Each flight is
- given a unique ID (a number or name).
+ Each run of osstest is referred to as a "flight". Each flight is
+ given a unique ID.
+
+ Standalone mode generally uses a flight named "standalone", which
+ is frequently erased and reused. In "Executive" mode (used for
+ production osstest instances) flights are numbered and the
+ metadata is permanently recorded.
"job":
- Each flight consists of one or more "jobs". These are a sequence
+ Each flight consists of one or more "jobs". These are a sequence
of test steps run in order and correspond to a column in the test
- report grid. They have names like "build-amd64" or
- "test-amd64-amd64-pv". A job can depend on the output of another
+ report grid. They have names like "build-amd64" or
+ "test-amd64-amd64-pv". A job can depend on the output of another
job in the flight -- e.g. most test-* jobs depend on one or more
build-* jobs.
+ A job has a named "recipe", which indicates what things need to be
+ done in what order: generally, a sequence of ts-* scripts. The
+ recipes are defined by their implementations in sg-run-job.
+
+ The set of jobs for any particular kind of flight is defined in
+ make-flight, make-*-flight, and mfi-*.
+
"step":
- Each job consists of multiple "steps" which is an individual test
- operation, such as "build the hypervisor", "install a guest",
- "start a guest", "migrate a guest", etc. A step corresponds to a
- cell in the results grid. A given step can be reused in multiple
- different jobs, e.g. the "xen build" step is used in several
- different build-* jobs. This reuse can be seen in the rows of the
- results grid.
+ Running a job consists of running its individual "steps". Each
+ step is an individual test operation, such as "build the
+ hypervisor", "install a guest", "start a guest", "migrate a
+ guest", etc.
+
+ Generally a step corresponds to one execution of some ts-* script.
+
+ steps have a "testid" which is used to uniquely identify the same
+ operation in different test runs. This is used for identifying
+ regressions, bisection, and so on.
+
+ The steps for a job, including the testids, are defined by the
+ recipe in sg-run-job (see "job", above).
+
+ Each step run in a job becomes a cell in the HTML results grid. A
+ given step might appear in multiple different jobs, e.g. the "xen
+ build" step is used in several different build-* jobs. This can
+ be seen in the rows of the results grid.
"runvar":
- A runvar is a named textual variable associated with each job in a
- given flight. They serve as both the inputs and outputs to the
+ A runvar is a named textual variable associated with a given job
+ and flight. They serve as both the inputs and outputs to the
job.
For example a Xen build job may have input runvars "tree_xen" (the
@@ -55,6 +78,15 @@ Terminology
* the parameters of the guest to test (e.g. distro, PV vs HVM
etc).
+ Runvar names often have structure, being assembled out of various
+ pieces; and the names are sometimes parsed, too.
+
+flights, jobs and runvars are kept in the database (in standalone
+mode, "standalone.db"). As for steps, only steps which have been
+started are recorded in the db; steps which have yet to be attempted
+are not in the database. Also, if a ts-* script is run by hand, this
+is not recorded as a step.
+
Operation
=========
@@ -73,12 +105,25 @@ referenced by each job's configuration. It then runs each of these in
turn, taking into account the prerequisites etc, by calling the
relevant "ts-*" scripts.
+Each ts-* is a convenient unit of test execution and reporting. Most
+ts-* scripts are written in perl, although simple shell wrappers are
+sometimes used. ts-* scripts expect OSSTEST_FLIGHT and OSSTEST_JOB to
+be set in the environment. Most of them also expect to be told extra
+information on the command line - notably, which host(s) to operate
+on.
+
+In automatic operation, the command line arguments for each test
+step's ts-* script invocation are fixed by the recipe defined in
+sg-run-job.
+
When running in standalone mode it is possible to run any of these
steps by hand, ("mg-execute-flight", "sg-run-job", "ts-*") although
you will need to find the correct inputs (some of which are documented
below) and perhaps take care of prerequisites yourself (e.g. running
"./sg-run-job test-armhf-armhf-xl" means you must have done
-"./sg-runjob build-armhf" and "build-armhf-pvops" first.
+"./sg-runjob build-armhf" and "build-armhf-pvops" first. When running
+a ts-* script manually one often wants to specify "host=<hostname>".
+(See the head comment for "selecthost" in Osstest/TestSupport.pm.)
Results
=======
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 4+ messages in thread* [OSSTEST PATCH 3/4] README.dev: Miscellaneous minor improvements
2017-09-11 9:54 [OSSTEST PATCH 1/4] examine: Do not try to find old version Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 2/4] README: Better documentation of recipes, db, etc Ian Jackson
@ 2017-09-11 9:54 ` Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 4/4] README.dev: Improve instructions for new machine commissioning Ian Jackson
2 siblings, 0 replies; 4+ messages in thread
From: Ian Jackson @ 2017-09-11 9:54 UTC (permalink / raw)
To: xen-devel; +Cc: Ian Jackson
Clarify documentation on mg-blockage, and also suggest using
allocation instead.
No longer advise removing crontab as a way to shut osstest down. The
stop file approach works fine and is less disruptive.
Minor formatting changes.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
README.dev | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/README.dev b/README.dev
index 09b9531..6d37ada 100644
--- a/README.dev
+++ b/README.dev
@@ -22,11 +22,14 @@ Removing machines for servicing/outage
======================================
mg-blockage <date-from> <date-to> <hostflag> [<extra-xinfo>]
-OSSTEST_CONFIG=production-config ./mg-blockage '2014-05-21 0955' '2014-05-21 1400' equiv-marilith
+Eg
+ OSSTEST_CONFIG=production-config ./mg-blockage '2014-05-21 0955' '2014-05-21 1400' equiv-marilith
Keeps running for the duration, so run it in a screen on the osstest VM.
+Or you can use mg-allocate.
+
Commisioning a new machine
==========================
@@ -70,17 +73,16 @@ blessing and add the production ones, e.g.
$ mg-hosts setflags HOSTA HOSTB -- \!blessed-commission blessed-{real,play,adhoc}
-Shutting down
-=============
-crontab -r (remove)
-crontab < crontab (restore afterwards)
+Shutting down the whole system
+==============================
-you can create a file "stop" in ~osstest/testing.git or
+You can create a file "stop" in ~osstest/testing.git or
~osstest/for-blah/something.git to cause that cron job to not start
any new things. Put a note in the stop file to say why... (in the
former case, disables everything; in the latter, just that "branch")
+
Creating a new branch
=====================
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 4+ messages in thread* [OSSTEST PATCH 4/4] README.dev: Improve instructions for new machine commissioning
2017-09-11 9:54 [OSSTEST PATCH 1/4] examine: Do not try to find old version Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 2/4] README: Better documentation of recipes, db, etc Ian Jackson
2017-09-11 9:54 ` [OSSTEST PATCH 3/4] README.dev: Miscellaneous minor improvements Ian Jackson
@ 2017-09-11 9:54 ` Ian Jackson
2 siblings, 0 replies; 4+ messages in thread
From: Ian Jackson @ 2017-09-11 9:54 UTC (permalink / raw)
To: xen-devel; +Cc: Xudong Hao, Ian Jackson
Discuss preliminary setup (hardware and BIOS).
Invent "mudcake0" and "mudcake1" as machines being added and use them
consistently in the example runes.
Recommend to run a host examination flight first. (NB I am still
testing that this actually works as expected. Host examination is a
bit new.)
Consistently suggest use of a $basis, and use of $flight variable.
Do not prepend runes with OSSTEST_CONFIG setting. Instead, we expect
the user to have it set already (and mention this).
Use "cs-adjust-flight new:" rather than the two-step process.
Formatting improvements etc.
CC: Xudong Hao <xudong.hao@intel.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
README.dev | 113 ++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 90 insertions(+), 23 deletions(-)
diff --git a/README.dev b/README.dev
index 6d37ada..dc0dafd 100644
--- a/README.dev
+++ b/README.dev
@@ -33,45 +33,112 @@ Or you can use mg-allocate.
Commisioning a new machine
==========================
+Firstly, arrange that it is hooked up to network, serial, and pdu.
+
+(In the Xen Project Massachusetts test lab, make sure that ansible
+playbook hosts/ has the wiring information; running `make install'
+will then arrange for /root/osstest-hosts-config to contain the Ether,
+Serial and PowerMethod configuration, and also configure the DNS and
+the DHCP server.)
+
+Go through the BIOS set up, using the serial console. Check that:
+ - After AC power loss, machine is set to turn on
+ (NOT "last state" which is usually the default)
+ - Serial console is enabled (usually with "redirection after boot")
+ - Boot order is 1. primary network 2. primary disk
+ - Virtualisation extensions (and VT-d if applicable) are enabled
+ - ILOM is disabled (or, us using a not-connected network port)
+
+In the Massachusetts instance you access the machines' serial consoles
+with something like
+ ssh -vt root@serial0.test-lab sympathy -r mudcake0
+
+Most BIOSes can be entered by hammering on, alternately, ESC 2
+or ESC 0 or DEL or something.
+
+The program "xduplic-copier" can be useful when used with xterm to
+type into two serial consoles at once - that lets you make sure the
+two BIOSes are set up completely identically.
+
+If the machine's ethernet address is not already in the relevant
+database, it can be collected via the BIOS and should be recorded.
+(In Massachusetts, in ansible playbook/hosts/testnet.info.)
+
+Secondly, set it up in the osstest database
+-------------------------------------------
+
+Most of these tools can operate on multiple hosts, and we like to have
+hosts in at least pairs. Let us imagine we are adding two hosts
+mudcake0 and mudcake1.
+
Base it on an existing machine:
-$ mg-hosts create-like marilith-n5 marilith-n4
+ $ ./mg-hosts create-like merlot0 mudcake0,mudcake1
+
+Allocate it for the duration of setup:
+
+ $ ./mg-allocate mudcake{0,1} # this shell syntax provides
+ # two arguments, mudcake0 mudcake1
+
+Set up the host flags:
+
+ $ ./mg-hosts setflags mudcake{0,1} -- blessed-commission-mudcake equiv-mudcake arch-{amd64,i386} arch-xen-{amd64,i386} hvm hvm-intel purpose-{build,test} suite-{jessie,stretch}
+
+(Consult mg-hosts showflags to see which flags are appropriate.)
+
+Set up any necessary host properties:
+
+ $ ./mg-hosts setprops mudcake{0,1} -- ....
+
+For example, one might need something like this:
+
+ $ ./mg-hosts setprops marilith-n4 -- dhcp-watch-method 'leases dhcp3 dns-cam1.uk.xensource.com:5556'
+
+(Many of these things are not needed in Massachusetts as the ansible
+playbook will provide it via an autogenerated config file - see above.)
+
+Create the tftp directory:
-Allocate it for the duration of setup
-$ mg-allocate HOST
+ $ ./mg-hosts mknetbootdir mudcake{0,1}
-Set up the props and flags
-$ mg-hosts setprops ...
-$ mg-hosts setflags ...
+Run commissioning test flights
+------------------------------
-$ mg-hosts setprops marilith-n4 -- dhcp-watch-method 'leases dhcp3 dns-cam1.uk.xensource.com:5556'
-Handles multiple hosts:
-$ mg-hosts setprops HOSTA HOSTB -- "PROP" "VALUE"
+Firstly, a basic "host examination" that checks that we can install
+and boot Xen:
-$ mg-hosts mknetbootdir HOSTA HOSTB
+ $ basis=113124 # pick last good xen-unstable or osstest flight
+ $ flight=`./make-hosts-flight play xen-unstable blessed-commission-mudcake commission-mudcake $basis`; echo $flight
+ 113155
+ $ ./mg-execute-flight -Bcommission-mudcake -Eian.jackson@citrix.com $flight
-Create a flight:
-$ FLIGHT=`OSSTEST_CONFIG=production-config ./cs-flight-create commission xen-unstable`
-$ echo $FLIGHT
-20974
+This will email the specified address. The examination should pass,
+completely. If it does not then you may need to change the BIOS
+serial arrangements, or do other debugging.
-Clone an existing flight, e.g. 20855 was previous successful flight
-$ OSSTEST_CONFIG=production-config ./cs-adjust-flight $FLIGHT copy 20855
+It is best to run mg-execute-flight in screen, so that it will survive
+ssh session disconnection. You should be running out of an osstest
+tree (testing.git clone) with good recent code, with an appropriate
+OSSTEST_CONFIG setting (eg OSSTEST_CONFIG=production-config, in
+Massachusetts). You can run it as yourself, though, if you have the
+right permissions set up.
-"commission" is a blessing, hosts should be blessed with it, e.g.
+If that works, a more thorough test:
-$ mg-hosts setflags HOSTA HOSTB -- blessed-commission
+ $ basis=113124 # pick last good xen-unstable or osstest flight
+ $ flight=`./cs-adjust-flight new:commission-mudcake copy $basis`; echo $flight
+ $ ./mg-execute-flight -Bcommission-mudcake -Eian.jackson@citrix.com -f$basis $flight
-Can use anything e.g. commission-$classofmachine if doing multiple
-commissions in parallel.
+This should show no regressions. (Or, at least, none that are a cause
+for concern.)
-Run it, e.g. in a screen session:
-OSSTEST_CONFIG=production-config ./mg-execute-flight -Bcommission -Eian.campbell@citrix.com -f20855 $FLIGHT
+Bless
+-----
Once machines are ready for production use remove the commission
blessing and add the production ones, e.g.
-$ mg-hosts setflags HOSTA HOSTB -- \!blessed-commission blessed-{real,play,adhoc}
+ $ ./mg-hosts setflags mudcake{0,1} -- \!blessed-commission blessed-{real,play,adhoc}
Shutting down the whole system
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 4+ messages in thread