[PATCH kvm-unit-tests 00/15] Introduce an errata framework

[Andrew Jones <drjones@redhat.com>]

When using kvm-unit-tests to verify distributions (which use older,
stable KVM and QEMU), it's possible to get FAILs for tests of features
not implemented, or FAILs for fixes that are targeted for later
releases. This framework empowers unit test developers to more
generously apply report_skip and report_xfail by giving the user
run-time modifiable booleans to control the flow of tests. For example,
we can now do something like

  if (!ERRATA(PPC64_TM_01))
    report_skip("ERRATA_PPC64_TM_01 not set 'y', skipping...");
  else
    report(...);

ERRATA_PPC64_TM_01 is simply a unit test environment variable which
can have its value changed easily. E.g.

  $ echo ERRATA_PPC64_TM_01=y > ppc64.env
  $ ENV=ppc64.env ./run_tests.sh

will run all ppc64 unit tests, and any test that checks
ERRATA(PPC64_TM_01) will get a return value of true. Another
example is

  report_xfail("APIC test", !ERRATA_RELAXED(X86_APIC_01), ...);

allowing an APIC test to report FAIL when ERRATA_X86_APIC_01 is
'y', or non-exist, and the test fails, or XFAIL when it fails
with ERRATA_X86_APIC_01=n.

Getting report_skip/xfail patches upstream, and then using an environ
with the appropriate errata set downstream, should help avoid automated
test suites from flagging too many false alarms, and also avoid the
need to carry too many downstream patches.

Thanks,
drew


Andrew Jones (15):
  devicetree: improve dt_get_bootargs interface
  devicetree: introduce dt_get_initrd
  arm/arm64: better document setup
  powerpc: better document setup
  arm/arm64: import initrd
  powerpc: import initrd
  x86_64: mbi-cmdline is a 4 byte addr
  x86: import initrd
  lib/argv: fix coding style
  lib/argv: introduce setup_env and getenv
  arm/arm64: enable environ
  powerpc: enable environ
  x86: enable environ
  README: reserve some environment variables
  Introduce lib/errata.h

 arm/run             |   1 +
 powerpc/run         |   1 +
 x86/run             |   4 +-
 x86/Makefile.common |   1 +
 lib/devicetree.h    |  15 +++++-
 lib/errata.h        |  24 +++++++++
 lib/libcflat.h      |   1 +
 lib/argv.c          | 143 +++++++++++++++++++++++++++++++++++++++++-----------
 lib/arm/setup.c     |  55 +++++++++++++++++---
 lib/devicetree.c    |  39 ++++++++++++--
 lib/powerpc/setup.c |  55 +++++++++++++++++---
 lib/string.c        |  15 ++++++
 lib/x86/setup.c     |  47 +++++++++++++++++
 arm/cstart.S        |   1 +
 arm/cstart64.S      |   1 +
 powerpc/cstart64.S  |   5 +-
 x86/cstart.S        |   4 ++
 x86/cstart64.S      |   8 ++-
 README.md           |  24 +++++++++
 19 files changed, 391 insertions(+), 53 deletions(-)
 create mode 100644 lib/errata.h
 create mode 100644 lib/x86/setup.c

-- 
2.9.3