Now available: xm-test-0.1.8

Now available: xm-test-0.1.8

[Dan Smith]

I have posted the latest version of xm-test here:

  http://static.danplanet.com/xm-test/xm-test-0.1.8.tar.gz

Many tests have been updated.  Many tests have been added.

The restore, save, and migrate tests are now enabled and timed to
prevent a run-away tests.

The reporting scripts used in Dave Barrera's daily emails are
included.

Feedback is welcome.  We would love for people to post their report
files so we can see what is and is not working "out there".

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com

 Makefile.am                                     |    2 
 README                                          |   33 ++++++++-----
 configure.ac                                    |    4 +
 lib/XmTestLib/Console.py                        |   30 ++++++++++--
 lib/XmTestLib/Test.py                           |   57 +++++++++++++++++++++---
 lib/XmTestLib/XenDomain.py                      |   52 ++++++++++++++++-----
 lib/XmTestLib/Xm.py                             |   44 +++++++++++++++++-
 lib/XmTestLib/__init__.py                       |    3 +
 ramdisk/Makefile.am                             |    5 +-
 tests/Makefile.am                               |    9 ++-
 tests/create/01_create_basic_pos.py             |    4 +
 tests/create/02_create_noparm_neg.py            |    4 -
 tests/create/03_create_badparm_neg.py           |    2 
 tests/create/04_create_conflictname_neg.py      |    9 ++-
 tests/create/06_create_mem_neg.py               |    4 -
 tests/create/07_create_mem64_pos.py             |    4 -
 tests/create/08_create_mem128_pos.py            |    4 -
 tests/create/09_create_mem256_pos.py            |    4 -
 tests/create/10_create_fastdestroy.py           |   38 ++++++++++++----
 tests/create/Makefile.am                        |    3 -
 tests/destroy/01_destroy_basic_pos.py           |    4 -
 tests/destroy/02_destroy_noparm_neg.py          |    4 -
 tests/destroy/03_destroy_nonexist_neg.py        |    4 -
 tests/destroy/04_destroy_badparm_neg.py         |    4 -
 tests/destroy/Makefile.am                       |    4 +
 tests/dmesg/01_dmesg_basic_pos.py               |    4 -
 tests/dmesg/02_dmesg_basic_neg.py               |    4 -
 tests/domid/01_domid_basic_pos.py               |    6 +-
 tests/domid/02_domid_basic_neg.py               |    4 -
 tests/domname/01_domname_basic_pos.py           |    6 +-
 tests/domname/02_domname_basic_neg.py           |    4 -
 tests/help/01_help_basic_pos.py                 |    2 
 tests/help/02_help_basic_neg.py                 |    2 
 tests/help/03_help_badparm_neg.py               |    2 
 tests/help/04_help_long_pos.py                  |    2 
 tests/info/Makefile.am                          |    3 -
 tests/list/01_list_basic_pos.py                 |    4 -
 tests/list/02_list_badparm_neg.py               |    4 -
 tests/list/03_list_nonexist_neg.py              |    4 -
 tests/list/04_list_goodparm_pos.py              |    4 -
 tests/memmax/01_memmax_badparm_neg.py           |    8 +--
 tests/memset/01_memset_basic_pos.py             |   55 ++++++++++-------------
 tests/memset/02_memset_badparm_neg.py           |   16 +++---
 tests/memset/Makefile.am                        |    4 +
 tests/migrate/01_migrate_localhost_pos.py       |   11 +++-
 tests/pause/01_pause_basic_pos.py               |    4 -
 tests/reboot/Makefile.am                        |    4 +
 tests/restore/01_restore_basic_pos.py           |   13 ++++-
 tests/restore/Makefile.am                       |    7 ++
 tests/save/01_save_basic_pos.py                 |    7 ++
 tests/save/Makefile.am                          |    7 ++
 tests/sedf/01_sedf_multi_pos.py                 |    4 -
 tests/shutdown/01_shutdown_basic_pos.py         |    2 
 tests/shutdown/02_shutdown_badparm_neg.py       |    4 -
 tests/shutdown/03_shutdown_nonexist_neg.py      |    4 -
 tests/sysrq/01_sysrq_basic_neg.py               |    5 +-
 tests/sysrq/02_sysrq_sync_pos.py                |    2 
 tests/sysrq/Makefile.am                         |    4 +
 tests/unpause/01_unpause_basic_pos.py           |    6 +-
 tests/vcpu-disable/01_vcpu-disable_basic_pos.py |    4 -
 tests/vcpu-pin/01_vcpu-pin_basic_pos.py         |    4 -
 tests/vcpu-pin/Makefile.am                      |    2 
 62 files changed, 388 insertions(+), 179 deletions(-)

Re: Now available: xm-test-0.1.8

[Dan Smith]

I meant to include this in my previous email:

* For those who are not familiar with xm-test:

Several of us here at IBM have been working on a framework for testing
the xen tools, specifically xm.  Our goal is to provide a way for
developers to _easily_ write tests for new and existing xm commands.
We believe that such a test suite will help reduce breakages in the
user-facing tools when developers modify xm and/or xend.

We would like some feedback from the community on the usefulness of
our framework, in hopes that it might be hosted by xensource so that
everyone can contribute tests to help harden xm and xend.

* Details:

The framework tests (as well as the support libraries) are written in
python, which are executed by the standard automake "make check"
facilities.  We build a standardized ramdisk that can be used for
portable test writing, therefore reducing dependencies on the test
host machine.

The framework library provides several abstractions to make common and
complex tasks easier for the test writer.  For example, we provide a
domain and console abstraction that allows a test writer to start a
domU and execute arbitrary commands, retrieving the status and output
of each.  This allows a decent amount of automation for verifying that
(for example) "xm sysrq mydomain s" actually sent the sysrq.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com