kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH kvm-unit-tests 00/15] Introduce an errata framework
@ 2017-01-13 18:15 Andrew Jones
  2017-01-13 18:15 ` [PATCH kvm-unit-tests 01/15] devicetree: improve dt_get_bootargs interface Andrew Jones
                   ` (14 more replies)
  0 siblings, 15 replies; 17+ messages in thread
From: Andrew Jones @ 2017-01-13 18:15 UTC (permalink / raw)
  To: kvm; +Cc: rkrcmar, pbonzini, lvivier, thuth

When using kvm-unit-tests to verify distributions (which use older,
stable KVM and QEMU), it's possible to get FAILs for tests of features
not implemented, or FAILs for fixes that are targeted for later
releases. This framework empowers unit test developers to more
generously apply report_skip and report_xfail by giving the user
run-time modifiable booleans to control the flow of tests. For example,
we can now do something like

  if (!ERRATA(PPC64_TM_01))
    report_skip("ERRATA_PPC64_TM_01 not set 'y', skipping...");
  else
    report(...);

ERRATA_PPC64_TM_01 is simply a unit test environment variable which
can have its value changed easily. E.g.

  $ echo ERRATA_PPC64_TM_01=y > ppc64.env
  $ ENV=ppc64.env ./run_tests.sh

will run all ppc64 unit tests, and any test that checks
ERRATA(PPC64_TM_01) will get a return value of true. Another
example is

  report_xfail("APIC test", !ERRATA_RELAXED(X86_APIC_01), ...);

allowing an APIC test to report FAIL when ERRATA_X86_APIC_01 is
'y', or non-exist, and the test fails, or XFAIL when it fails
with ERRATA_X86_APIC_01=n.

Getting report_skip/xfail patches upstream, and then using an environ
with the appropriate errata set downstream, should help avoid automated
test suites from flagging too many false alarms, and also avoid the
need to carry too many downstream patches.

Thanks,
drew


Andrew Jones (15):
  devicetree: improve dt_get_bootargs interface
  devicetree: introduce dt_get_initrd
  arm/arm64: better document setup
  powerpc: better document setup
  arm/arm64: import initrd
  powerpc: import initrd
  x86_64: mbi-cmdline is a 4 byte addr
  x86: import initrd
  lib/argv: fix coding style
  lib/argv: introduce setup_env and getenv
  arm/arm64: enable environ
  powerpc: enable environ
  x86: enable environ
  README: reserve some environment variables
  Introduce lib/errata.h

 arm/run             |   1 +
 powerpc/run         |   1 +
 x86/run             |   4 +-
 x86/Makefile.common |   1 +
 lib/devicetree.h    |  15 +++++-
 lib/errata.h        |  24 +++++++++
 lib/libcflat.h      |   1 +
 lib/argv.c          | 143 +++++++++++++++++++++++++++++++++++++++++-----------
 lib/arm/setup.c     |  55 +++++++++++++++++---
 lib/devicetree.c    |  39 ++++++++++++--
 lib/powerpc/setup.c |  55 +++++++++++++++++---
 lib/string.c        |  15 ++++++
 lib/x86/setup.c     |  47 +++++++++++++++++
 arm/cstart.S        |   1 +
 arm/cstart64.S      |   1 +
 powerpc/cstart64.S  |   5 +-
 x86/cstart.S        |   4 ++
 x86/cstart64.S      |   8 ++-
 README.md           |  24 +++++++++
 19 files changed, 391 insertions(+), 53 deletions(-)
 create mode 100644 lib/errata.h
 create mode 100644 lib/x86/setup.c

-- 
2.9.3


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2017-03-03 14:47 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-01-13 18:15 [PATCH kvm-unit-tests 00/15] Introduce an errata framework Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 01/15] devicetree: improve dt_get_bootargs interface Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 02/15] devicetree: introduce dt_get_initrd Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 03/15] arm/arm64: better document setup Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 04/15] powerpc: " Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 05/15] arm/arm64: import initrd Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 06/15] powerpc: " Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 07/15] x86_64: mbi-cmdline is a 4 byte addr Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 08/15] x86: import initrd Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 09/15] lib/argv: fix coding style Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 10/15] lib/argv: introduce setup_env and getenv Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 11/15] arm/arm64: enable environ Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 12/15] powerpc: " Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 13/15] x86: " Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 14/15] README: reserve some environment variables Andrew Jones
2017-01-13 18:15 ` [PATCH kvm-unit-tests 15/15] Introduce lib/errata.h Andrew Jones
2017-03-03 14:40   ` Radim Krčmář

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).