From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Subject: [kvm-unit-tests PATCH v4 01/13] lib/report: allow test skipping Date: Wed, 11 May 2016 18:12:43 +0200 Message-ID: <1462983171-4208-2-git-send-email-rkrcmar@redhat.com> References: <1462983171-4208-1-git-send-email-rkrcmar@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Paolo Bonzini , Andrew Jones To: kvm@vger.kernel.org Return-path: Received: from mx1.redhat.com ([209.132.183.28]:40539 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932165AbcEKQNK (ORCPT ); Wed, 11 May 2016 12:13:10 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 12C7DC09726D for ; Wed, 11 May 2016 16:13:10 +0000 (UTC) In-Reply-To: <1462983171-4208-1-git-send-email-rkrcmar@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: We can now explicitly mark a unit-test as skipped. If all unit-tests were skipped, the whole test is reported as skipped a= s well. This also includes the case where no tests were run, but still ended with report_summary(). When the whole test is skipped, ./run_tests.sh prints yellow "SKIP" instead of green "PASS". Return value of 77 is used to please Autotools. I also renamed few things in reporting code and chose to refactor a logic while at it. Reviewed-by: Andrew Jones Signed-off-by: Radim Kr=C4=8Dm=C3=A1=C5=99 --- lib/libcflat.h | 1 + lib/report.c | 45 +++++++++++++++++++++++++++++++-------------= - scripts/runtime.bash | 2 ++ 3 files changed, 34 insertions(+), 14 deletions(-) diff --git a/lib/libcflat.h b/lib/libcflat.h index df50615b4366..582e3fc60e28 100644 --- a/lib/libcflat.h +++ b/lib/libcflat.h @@ -79,6 +79,7 @@ extern void report_prefix_pop(void); extern void report(const char *msg_fmt, bool pass, ...); extern void report_xfail(const char *msg_fmt, bool xfail, bool pass, .= =2E.); extern void report_abort(const char *msg_fmt, ...); +extern void report_skip(const char *msg_fmt, ...); extern int report_summary(void); =20 extern void dump_stack(void); diff --git a/lib/report.c b/lib/report.c index 9e45781ee2ab..430b2aeaecbd 100644 --- a/lib/report.c +++ b/lib/report.c @@ -13,7 +13,7 @@ #include "libcflat.h" #include "asm/spinlock.h" =20 -static unsigned int tests, failures, xfailures; +static unsigned int tests, failures, xfailures, skipped; static char prefixes[256]; static struct spinlock lock; =20 @@ -43,23 +43,25 @@ void report_prefix_pop(void) spin_unlock(&lock); } =20 -void va_report_xfail(const char *msg_fmt, bool xfail, bool cond, va_li= st va) +static void va_report(const char *msg_fmt, + bool pass, bool xfail, bool skip, va_list va) { - char *pass =3D xfail ? "XPASS" : "PASS"; - char *fail =3D xfail ? "XFAIL" : "FAIL"; + char *prefix =3D skip ? "SKIP" + : xfail ? (pass ? "XPASS" : "XFAIL") + : (pass ? "PASS" : "FAIL"); =20 spin_lock(&lock); =20 tests++; - printf("%s: ", cond ? pass : fail); + printf("%s: ", prefix); puts(prefixes); vprintf(msg_fmt, va); puts("\n"); - if (xfail && cond) - failures++; - else if (xfail) + if (skip) + skipped++; + else if (xfail && !pass) xfailures++; - else if (!cond) + else if (xfail || !pass) failures++; =20 spin_unlock(&lock); @@ -69,7 +71,7 @@ void report(const char *msg_fmt, bool pass, ...) { va_list va; va_start(va, pass); - va_report_xfail(msg_fmt, false, pass, va); + va_report(msg_fmt, pass, false, false, va); va_end(va); } =20 @@ -77,7 +79,15 @@ void report_xfail(const char *msg_fmt, bool xfail, b= ool pass, ...) { va_list va; va_start(va, pass); - va_report_xfail(msg_fmt, xfail, pass, va); + va_report(msg_fmt, pass, xfail, false, va); + va_end(va); +} + +void report_skip(const char *msg_fmt, ...) +{ + va_list va; + va_start(va, msg_fmt); + va_report(msg_fmt, false, false, true, va); va_end(va); } =20 @@ -87,9 +97,16 @@ int report_summary(void) =20 printf("\nSUMMARY: %d tests, %d unexpected failures", tests, failures= ); if (xfailures) - printf(", %d expected failures\n", xfailures); - else - printf("\n"); + printf(", %d expected failures", xfailures); + if (skipped) + printf(", %d skipped", skipped); + printf("\n"); + + if (tests =3D=3D skipped) + /* Blame AUTOTOOLS for using 77 for skipped test and QEMU for + * mangling error codes in a way that gets 77 if we ... */ + return 77 >> 1; + return failures > 0 ? 1 : 0; =20 spin_unlock(&lock); diff --git a/scripts/runtime.bash b/scripts/runtime.bash index 0e055f0dddc2..ed073721216c 100644 --- a/scripts/runtime.bash +++ b/scripts/runtime.bash @@ -51,6 +51,8 @@ function run() =20 if [ $ret -eq 0 ]; then echo -e "\e[32mPASS\e[0m $1" + elif [ $ret -eq 77 ]; then + echo -e "\e[33mSKIP\e[0m $1" elif [ $ret -eq 124 ]; then echo -e "\e[31mFAIL\e[0m $1 (timeout; duration=3D$timeout)" else --=20 2.8.2