* [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile)
@ 2016-07-15 15:39 Thomas Huth
2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include Thomas Huth
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Thomas Huth @ 2016-07-15 15:39 UTC (permalink / raw)
To: qemu-devel, Paolo Bonzini
Cc: david, Eric Blake, Markus Armbruster, Peter Maydell,
Richard Henderson, Peter Crosthwaite, Eduardo Habkost
Since some of the machines that we support in QEMU sometimes get
broken by other changes, and it then takes a while 'till somebody
notices the breakage, it would be nice to get some more automatic
test coverage for various machines during "make check". The
second patch adds such a test for the machines where we've got
a firmware image for and thus can test for some magic strings
in the serial output of the firmware.
The first patch is just a clean-up for the Makefile to avoid that
new tests get added with "=" instead of "+=" (and thus completely
overwrite the correspond variable by accident).
v2:
- Some more clean-ups for the Makefile (i.e. some additional empty
lines)
- Check return value of read() to make sure that there is no error
Thomas Huth (2):
tests: Resort check-qtest entries in Makefile.include
tests: Check serial output of firmware boot of some machines
tests/Makefile.include | 46 ++++++++++++++------
tests/boot-serial-test.c | 111 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 144 insertions(+), 13 deletions(-)
create mode 100644 tests/boot-serial-test.c
--
1.8.3.1
^ permalink raw reply [flat|nested] 8+ messages in thread* [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include 2016-07-15 15:39 [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) Thomas Huth @ 2016-07-15 15:39 ` Thomas Huth 2016-07-18 0:44 ` David Gibson 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines Thomas Huth 2016-07-18 10:02 ` [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) David Gibson 2 siblings, 1 reply; 8+ messages in thread From: Thomas Huth @ 2016-07-15 15:39 UTC (permalink / raw) To: qemu-devel, Paolo Bonzini Cc: david, Eric Blake, Markus Armbruster, Peter Maydell, Richard Henderson, Peter Crosthwaite, Eduardo Habkost The rather random list of check-qtest-xxx entries caused some confusion in the past, where to use "=" and where to use "+=" (see commits 0ccac16f59462b8e2b9afbc1 and 1f5c1cfbaec0792cd2e5da for example). Sorting the check-qtest-xxx entries by architecure instead and using some empty lines inbetween should help to ease this situation a little bit, so that it is hopefully now obvious that new tests should be added with "+=" instead of "=". While we are at it, this patch also comments out two of the "gcov-files-..." lines since the corresponding m48t59-test is disabled for sparc and sparc64, too. Signed-off-by: Thomas Huth <thuth@redhat.com> --- tests/Makefile.include | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 2010b11..3d76cf4 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -240,33 +240,45 @@ check-qtest-i386-y += tests/postcopy-test$(EXESUF) check-qtest-x86_64-y += $(check-qtest-i386-y) gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y)) + check-qtest-mips-y = tests/endianness-test$(EXESUF) + check-qtest-mips64-y = tests/endianness-test$(EXESUF) + check-qtest-mips64el-y = tests/endianness-test$(EXESUF) + check-qtest-ppc-y = tests/endianness-test$(EXESUF) -check-qtest-ppc64-y = tests/endianness-test$(EXESUF) +check-qtest-ppc-y += tests/boot-order-test$(EXESUF) +check-qtest-ppc-y += tests/prom-env-test$(EXESUF) + +check-qtest-ppc64-y = tests/spapr-phb-test$(EXESUF) +gcov-files-ppc64-y = ppc64-softmmu/hw/ppc/spapr_pci.c +check-qtest-ppc64-y += tests/endianness-test$(EXESUF) +check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) +check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) + check-qtest-sh4-y = tests/endianness-test$(EXESUF) + check-qtest-sh4eb-y = tests/endianness-test$(EXESUF) + +check-qtest-sparc-y = tests/prom-env-test$(EXESUF) +#check-qtest-sparc-y += tests/m48t59-test$(EXESUF) +#gcov-files-sparc-y = hw/timer/m48t59.c + check-qtest-sparc64-y = tests/endianness-test$(EXESUF) -#check-qtest-sparc-y = tests/m48t59-test$(EXESUF) #check-qtest-sparc64-y += tests/m48t59-test$(EXESUF) -gcov-files-sparc-y += hw/timer/m48t59.c -gcov-files-sparc64-y += hw/timer/m48t59.c +#gcov-files-sparc64-y += hw/timer/m48t59.c +#Disabled for now, triggers a TCG bug on 32-bit hosts +#check-qtest-sparc64-y += tests/prom-env-test$(EXESUF) + check-qtest-arm-y = tests/tmp105-test$(EXESUF) check-qtest-arm-y += tests/ds1338-test$(EXESUF) gcov-files-arm-y += hw/misc/tmp105.c check-qtest-arm-y += tests/virtio-blk-test$(EXESUF) gcov-files-arm-y += arm-softmmu/hw/block/virtio-blk.c -check-qtest-ppc-y += tests/boot-order-test$(EXESUF) -check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) -check-qtest-ppc64-y += tests/spapr-phb-test$(EXESUF) -gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c -check-qtest-ppc-y += tests/prom-env-test$(EXESUF) -check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) -check-qtest-sparc-y += tests/prom-env-test$(EXESUF) -#Disabled for now, triggers a TCG bug on 32-bit hosts -#check-qtest-sparc64-y += tests/prom-env-test$(EXESUF) + check-qtest-microblazeel-y = $(check-qtest-microblaze-y) + check-qtest-xtensaeb-y = $(check-qtest-xtensa-y) check-qtest-generic-y += tests/qom-test$(EXESUF) -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include Thomas Huth @ 2016-07-18 0:44 ` David Gibson 0 siblings, 0 replies; 8+ messages in thread From: David Gibson @ 2016-07-18 0:44 UTC (permalink / raw) To: Thomas Huth Cc: qemu-devel, Paolo Bonzini, Eric Blake, Markus Armbruster, Peter Maydell, Richard Henderson, Peter Crosthwaite, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 3950 bytes --] On Fri, Jul 15, 2016 at 05:39:38PM +0200, Thomas Huth wrote: > The rather random list of check-qtest-xxx entries caused some > confusion in the past, where to use "=" and where to use "+=" > (see commits 0ccac16f59462b8e2b9afbc1 and 1f5c1cfbaec0792cd2e5da > for example). > Sorting the check-qtest-xxx entries by architecure instead and > using some empty lines inbetween should help to ease this > situation a little bit, so that it is hopefully now obvious > that new tests should be added with "+=" instead of "=". > While we are at it, this patch also comments out two of the > "gcov-files-..." lines since the corresponding m48t59-test is > disabled for sparc and sparc64, too. > > Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> > --- > tests/Makefile.include | 38 +++++++++++++++++++++++++------------- > 1 file changed, 25 insertions(+), 13 deletions(-) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 2010b11..3d76cf4 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -240,33 +240,45 @@ check-qtest-i386-y += tests/postcopy-test$(EXESUF) > check-qtest-x86_64-y += $(check-qtest-i386-y) > gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c > gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y)) > + > check-qtest-mips-y = tests/endianness-test$(EXESUF) > + > check-qtest-mips64-y = tests/endianness-test$(EXESUF) > + > check-qtest-mips64el-y = tests/endianness-test$(EXESUF) > + > check-qtest-ppc-y = tests/endianness-test$(EXESUF) > -check-qtest-ppc64-y = tests/endianness-test$(EXESUF) > +check-qtest-ppc-y += tests/boot-order-test$(EXESUF) > +check-qtest-ppc-y += tests/prom-env-test$(EXESUF) > + > +check-qtest-ppc64-y = tests/spapr-phb-test$(EXESUF) > +gcov-files-ppc64-y = ppc64-softmmu/hw/ppc/spapr_pci.c > +check-qtest-ppc64-y += tests/endianness-test$(EXESUF) > +check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) > +check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) > + > check-qtest-sh4-y = tests/endianness-test$(EXESUF) > + > check-qtest-sh4eb-y = tests/endianness-test$(EXESUF) > + > +check-qtest-sparc-y = tests/prom-env-test$(EXESUF) > +#check-qtest-sparc-y += tests/m48t59-test$(EXESUF) > +#gcov-files-sparc-y = hw/timer/m48t59.c > + > check-qtest-sparc64-y = tests/endianness-test$(EXESUF) > -#check-qtest-sparc-y = tests/m48t59-test$(EXESUF) > #check-qtest-sparc64-y += tests/m48t59-test$(EXESUF) > -gcov-files-sparc-y += hw/timer/m48t59.c > -gcov-files-sparc64-y += hw/timer/m48t59.c > +#gcov-files-sparc64-y += hw/timer/m48t59.c > +#Disabled for now, triggers a TCG bug on 32-bit hosts > +#check-qtest-sparc64-y += tests/prom-env-test$(EXESUF) > + > check-qtest-arm-y = tests/tmp105-test$(EXESUF) > check-qtest-arm-y += tests/ds1338-test$(EXESUF) > gcov-files-arm-y += hw/misc/tmp105.c > check-qtest-arm-y += tests/virtio-blk-test$(EXESUF) > gcov-files-arm-y += arm-softmmu/hw/block/virtio-blk.c > -check-qtest-ppc-y += tests/boot-order-test$(EXESUF) > -check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) > -check-qtest-ppc64-y += tests/spapr-phb-test$(EXESUF) > -gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c > -check-qtest-ppc-y += tests/prom-env-test$(EXESUF) > -check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) > -check-qtest-sparc-y += tests/prom-env-test$(EXESUF) > -#Disabled for now, triggers a TCG bug on 32-bit hosts > -#check-qtest-sparc64-y += tests/prom-env-test$(EXESUF) > + > check-qtest-microblazeel-y = $(check-qtest-microblaze-y) > + > check-qtest-xtensaeb-y = $(check-qtest-xtensa-y) > > check-qtest-generic-y += tests/qom-test$(EXESUF) -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines 2016-07-15 15:39 [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) Thomas Huth 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include Thomas Huth @ 2016-07-15 15:39 ` Thomas Huth 2016-07-18 0:45 ` David Gibson 2016-07-18 10:02 ` [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) David Gibson 2 siblings, 1 reply; 8+ messages in thread From: Thomas Huth @ 2016-07-15 15:39 UTC (permalink / raw) To: qemu-devel, Paolo Bonzini Cc: david, Eric Blake, Markus Armbruster, Peter Maydell, Richard Henderson, Peter Crosthwaite, Eduardo Habkost Some of the machines that we have got a firmware image for write some output to the serial console while booting up. We can use this output to make sure that the machine is basically working, so this adds a test that checks the output of these machines for some well-known "magic" strings. Signed-off-by: Thomas Huth <thuth@redhat.com> --- tests/Makefile.include | 8 ++++ tests/boot-serial-test.c | 111 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+) create mode 100644 tests/boot-serial-test.c diff --git a/tests/Makefile.include b/tests/Makefile.include index 3d76cf4..3986093 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -194,6 +194,7 @@ check-qtest-i386-y += tests/hd-geo-test$(EXESUF) gcov-files-i386-y += hw/block/hd-geometry.c check-qtest-i386-y += tests/boot-order-test$(EXESUF) check-qtest-i386-y += tests/bios-tables-test$(EXESUF) +check-qtest-i386-y += tests/boot-serial-test$(EXESUF) check-qtest-i386-y += tests/pxe-test$(EXESUF) check-qtest-i386-y += tests/rtc-test$(EXESUF) check-qtest-i386-y += tests/ipmi-kcs-test$(EXESUF) @@ -241,6 +242,8 @@ check-qtest-x86_64-y += $(check-qtest-i386-y) gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y)) +check-qtest-alpha-y = tests/boot-serial-test$(EXESUF) + check-qtest-mips-y = tests/endianness-test$(EXESUF) check-qtest-mips64-y = tests/endianness-test$(EXESUF) @@ -250,12 +253,14 @@ check-qtest-mips64el-y = tests/endianness-test$(EXESUF) check-qtest-ppc-y = tests/endianness-test$(EXESUF) check-qtest-ppc-y += tests/boot-order-test$(EXESUF) check-qtest-ppc-y += tests/prom-env-test$(EXESUF) +check-qtest-ppc-y += tests/boot-serial-test$(EXESUF) check-qtest-ppc64-y = tests/spapr-phb-test$(EXESUF) gcov-files-ppc64-y = ppc64-softmmu/hw/ppc/spapr_pci.c check-qtest-ppc64-y += tests/endianness-test$(EXESUF) check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) +check-qtest-ppc64-y += tests/boot-serial-test$(EXESUF) check-qtest-sh4-y = tests/endianness-test$(EXESUF) @@ -281,6 +286,8 @@ check-qtest-microblazeel-y = $(check-qtest-microblaze-y) check-qtest-xtensaeb-y = $(check-qtest-xtensa-y) +check-qtest-s390x-y = tests/boot-serial-test$(EXESUF) + check-qtest-generic-y += tests/qom-test$(EXESUF) qapi-schema += alternate-any.json @@ -579,6 +586,7 @@ tests/ipmi-kcs-test$(EXESUF): tests/ipmi-kcs-test.o tests/ipmi-bt-test$(EXESUF): tests/ipmi-bt-test.o tests/hd-geo-test$(EXESUF): tests/hd-geo-test.o tests/boot-order-test$(EXESUF): tests/boot-order-test.o $(libqos-obj-y) +tests/boot-serial-test$(EXESUF): tests/boot-serial-test.o $(libqos-obj-y) tests/bios-tables-test$(EXESUF): tests/bios-tables-test.o \ tests/boot-sector.o $(libqos-obj-y) tests/pxe-test$(EXESUF): tests/pxe-test.o tests/boot-sector.o $(libqos-obj-y) diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c new file mode 100644 index 0000000..fd60337 --- /dev/null +++ b/tests/boot-serial-test.c @@ -0,0 +1,111 @@ +/* + * Test serial output of some machines. + * + * Copyright 2016 Thomas Huth, Red Hat Inc. + * + * This work is licensed under the terms of the GNU GPL, version 2 + * or later. See the COPYING file in the top-level directory. + * + * This test is used to check that the serial output of the firmware + * (that we provide for some machines) contains an expected string. + * Thus we check that the firmware still boots at least to a certain + * point and so we know that the machine is not completely broken. + */ + +#include "qemu/osdep.h" +#include "libqtest.h" + +typedef struct testdef { + const char *arch; /* Target architecture */ + const char *machine; /* Name of the machine */ + const char *extra; /* Additional parameters */ + const char *expect; /* Expected string in the serial output */ +} testdef_t; + +static testdef_t tests[] = { + { "alpha", "clipper", "", "PCI:" }, + { "ppc", "ppce500", "", "U-Boot" }, + { "ppc", "prep", "", "Open Hack'Ware BIOS" }, + { "ppc64", "ppce500", "", "U-Boot" }, + { "ppc64", "prep", "", "Open Hack'Ware BIOS" }, + { "ppc64", "pseries", "", "Open Firmware" }, + { "i386", "isapc", "-device sga", "SGABIOS" }, + { "i386", "pc", "-device sga", "SGABIOS" }, + { "i386", "q35", "-device sga", "SGABIOS" }, + { "x86_64", "isapc", "-device sga", "SGABIOS" }, + { "x86_64", "pc", "-device sga", "SGABIOS" }, + { "x86_64", "q35", "-device sga", "SGABIOS" }, + { "s390x", "s390-ccw-virtio", + "-nodefaults -device sclpconsole,chardev=serial0", "virtio device" }, + { NULL } +}; + +static void check_guest_output(const testdef_t *test, int fd) +{ + bool output_ok = false; + int i, nbr, pos = 0; + char ch; + + /* Poll serial output... Wait at most 60 seconds */ + for (i = 0; i < 6000; ++i) { + while ((nbr = read(fd, &ch, 1)) == 1) { + if (ch == test->expect[pos]) { + pos += 1; + if (test->expect[pos] == '\0') { + /* We've reached the end of the expected string! */ + output_ok = true; + goto done; + } + } else { + pos = 0; + } + } + g_assert(nbr >= 0); + g_usleep(10000); + } + +done: + g_assert(output_ok); +} + +static void test_machine(const void *data) +{ + const testdef_t *test = data; + char *args; + char tmpname[] = "/tmp/qtest-boot-serial-XXXXXX"; + int fd; + + fd = mkstemp(tmpname); + g_assert(fd != -1); + + args = g_strdup_printf("-M %s,accel=tcg -chardev file,id=serial0,path=%s" + " -serial chardev:serial0 %s", test->machine, + tmpname, test->extra); + + qtest_start(args); + unlink(tmpname); + + check_guest_output(test, fd); + qtest_quit(global_qtest); + + g_free(args); + close(fd); +} + +int main(int argc, char *argv[]) +{ + const char *arch = qtest_get_arch(); + int i; + + g_test_init(&argc, &argv, NULL); + + for (i = 0; tests[i].arch != NULL; i++) { + if (strcmp(arch, tests[i].arch) == 0) { + char *name = g_strdup_printf("boot-serial/%s", tests[i].machine); + qtest_add_data_func(name, &tests[i], test_machine); + g_free(name); + } + } + + return g_test_run(); +} -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines Thomas Huth @ 2016-07-18 0:45 ` David Gibson 0 siblings, 0 replies; 8+ messages in thread From: David Gibson @ 2016-07-18 0:45 UTC (permalink / raw) To: Thomas Huth Cc: qemu-devel, Paolo Bonzini, Eric Blake, Markus Armbruster, Peter Maydell, Richard Henderson, Peter Crosthwaite, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 7344 bytes --] On Fri, Jul 15, 2016 at 05:39:39PM +0200, Thomas Huth wrote: > Some of the machines that we have got a firmware image for write > some output to the serial console while booting up. We can use > this output to make sure that the machine is basically working, > so this adds a test that checks the output of these machines > for some well-known "magic" strings. > > Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> > --- > tests/Makefile.include | 8 ++++ > tests/boot-serial-test.c | 111 +++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 119 insertions(+) > create mode 100644 tests/boot-serial-test.c > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 3d76cf4..3986093 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -194,6 +194,7 @@ check-qtest-i386-y += tests/hd-geo-test$(EXESUF) > gcov-files-i386-y += hw/block/hd-geometry.c > check-qtest-i386-y += tests/boot-order-test$(EXESUF) > check-qtest-i386-y += tests/bios-tables-test$(EXESUF) > +check-qtest-i386-y += tests/boot-serial-test$(EXESUF) > check-qtest-i386-y += tests/pxe-test$(EXESUF) > check-qtest-i386-y += tests/rtc-test$(EXESUF) > check-qtest-i386-y += tests/ipmi-kcs-test$(EXESUF) > @@ -241,6 +242,8 @@ check-qtest-x86_64-y += $(check-qtest-i386-y) > gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c > gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y)) > > +check-qtest-alpha-y = tests/boot-serial-test$(EXESUF) > + > check-qtest-mips-y = tests/endianness-test$(EXESUF) > > check-qtest-mips64-y = tests/endianness-test$(EXESUF) > @@ -250,12 +253,14 @@ check-qtest-mips64el-y = tests/endianness-test$(EXESUF) > check-qtest-ppc-y = tests/endianness-test$(EXESUF) > check-qtest-ppc-y += tests/boot-order-test$(EXESUF) > check-qtest-ppc-y += tests/prom-env-test$(EXESUF) > +check-qtest-ppc-y += tests/boot-serial-test$(EXESUF) > > check-qtest-ppc64-y = tests/spapr-phb-test$(EXESUF) > gcov-files-ppc64-y = ppc64-softmmu/hw/ppc/spapr_pci.c > check-qtest-ppc64-y += tests/endianness-test$(EXESUF) > check-qtest-ppc64-y += tests/boot-order-test$(EXESUF) > check-qtest-ppc64-y += tests/prom-env-test$(EXESUF) > +check-qtest-ppc64-y += tests/boot-serial-test$(EXESUF) > > check-qtest-sh4-y = tests/endianness-test$(EXESUF) > > @@ -281,6 +286,8 @@ check-qtest-microblazeel-y = $(check-qtest-microblaze-y) > > check-qtest-xtensaeb-y = $(check-qtest-xtensa-y) > > +check-qtest-s390x-y = tests/boot-serial-test$(EXESUF) > + > check-qtest-generic-y += tests/qom-test$(EXESUF) > > qapi-schema += alternate-any.json > @@ -579,6 +586,7 @@ tests/ipmi-kcs-test$(EXESUF): tests/ipmi-kcs-test.o > tests/ipmi-bt-test$(EXESUF): tests/ipmi-bt-test.o > tests/hd-geo-test$(EXESUF): tests/hd-geo-test.o > tests/boot-order-test$(EXESUF): tests/boot-order-test.o $(libqos-obj-y) > +tests/boot-serial-test$(EXESUF): tests/boot-serial-test.o $(libqos-obj-y) > tests/bios-tables-test$(EXESUF): tests/bios-tables-test.o \ > tests/boot-sector.o $(libqos-obj-y) > tests/pxe-test$(EXESUF): tests/pxe-test.o tests/boot-sector.o $(libqos-obj-y) > diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c > new file mode 100644 > index 0000000..fd60337 > --- /dev/null > +++ b/tests/boot-serial-test.c > @@ -0,0 +1,111 @@ > +/* > + * Test serial output of some machines. > + * > + * Copyright 2016 Thomas Huth, Red Hat Inc. > + * > + * This work is licensed under the terms of the GNU GPL, version 2 > + * or later. See the COPYING file in the top-level directory. > + * > + * This test is used to check that the serial output of the firmware > + * (that we provide for some machines) contains an expected string. > + * Thus we check that the firmware still boots at least to a certain > + * point and so we know that the machine is not completely broken. > + */ > + > +#include "qemu/osdep.h" > +#include "libqtest.h" > + > +typedef struct testdef { > + const char *arch; /* Target architecture */ > + const char *machine; /* Name of the machine */ > + const char *extra; /* Additional parameters */ > + const char *expect; /* Expected string in the serial output */ > +} testdef_t; > + > +static testdef_t tests[] = { > + { "alpha", "clipper", "", "PCI:" }, > + { "ppc", "ppce500", "", "U-Boot" }, > + { "ppc", "prep", "", "Open Hack'Ware BIOS" }, > + { "ppc64", "ppce500", "", "U-Boot" }, > + { "ppc64", "prep", "", "Open Hack'Ware BIOS" }, > + { "ppc64", "pseries", "", "Open Firmware" }, > + { "i386", "isapc", "-device sga", "SGABIOS" }, > + { "i386", "pc", "-device sga", "SGABIOS" }, > + { "i386", "q35", "-device sga", "SGABIOS" }, > + { "x86_64", "isapc", "-device sga", "SGABIOS" }, > + { "x86_64", "pc", "-device sga", "SGABIOS" }, > + { "x86_64", "q35", "-device sga", "SGABIOS" }, > + { "s390x", "s390-ccw-virtio", > + "-nodefaults -device sclpconsole,chardev=serial0", "virtio device" }, > + { NULL } > +}; > + > +static void check_guest_output(const testdef_t *test, int fd) > +{ > + bool output_ok = false; > + int i, nbr, pos = 0; > + char ch; > + > + /* Poll serial output... Wait at most 60 seconds */ > + for (i = 0; i < 6000; ++i) { > + while ((nbr = read(fd, &ch, 1)) == 1) { > + if (ch == test->expect[pos]) { > + pos += 1; > + if (test->expect[pos] == '\0') { > + /* We've reached the end of the expected string! */ > + output_ok = true; > + goto done; > + } > + } else { > + pos = 0; > + } > + } > + g_assert(nbr >= 0); > + g_usleep(10000); > + } > + > +done: > + g_assert(output_ok); > +} > + > +static void test_machine(const void *data) > +{ > + const testdef_t *test = data; > + char *args; > + char tmpname[] = "/tmp/qtest-boot-serial-XXXXXX"; > + int fd; > + > + fd = mkstemp(tmpname); > + g_assert(fd != -1); > + > + args = g_strdup_printf("-M %s,accel=tcg -chardev file,id=serial0,path=%s" > + " -serial chardev:serial0 %s", test->machine, > + tmpname, test->extra); > + > + qtest_start(args); > + unlink(tmpname); > + > + check_guest_output(test, fd); > + qtest_quit(global_qtest); > + > + g_free(args); > + close(fd); > +} > + > +int main(int argc, char *argv[]) > +{ > + const char *arch = qtest_get_arch(); > + int i; > + > + g_test_init(&argc, &argv, NULL); > + > + for (i = 0; tests[i].arch != NULL; i++) { > + if (strcmp(arch, tests[i].arch) == 0) { > + char *name = g_strdup_printf("boot-serial/%s", tests[i].machine); > + qtest_add_data_func(name, &tests[i], test_machine); > + g_free(name); > + } > + } > + > + return g_test_run(); > +} -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) 2016-07-15 15:39 [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) Thomas Huth 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include Thomas Huth 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines Thomas Huth @ 2016-07-18 10:02 ` David Gibson 2016-09-02 22:11 ` Thomas Huth 2 siblings, 1 reply; 8+ messages in thread From: David Gibson @ 2016-07-18 10:02 UTC (permalink / raw) To: Thomas Huth Cc: qemu-devel, Paolo Bonzini, Eric Blake, Markus Armbruster, Peter Maydell, Richard Henderson, Peter Crosthwaite, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 1640 bytes --] On Fri, Jul 15, 2016 at 05:39:37PM +0200, Thomas Huth wrote: > Since some of the machines that we support in QEMU sometimes get > broken by other changes, and it then takes a while 'till somebody > notices the breakage, it would be nice to get some more automatic > test coverage for various machines during "make check". The > second patch adds such a test for the machines where we've got > a firmware image for and thus can test for some magic strings > in the serial output of the firmware. > > The first patch is just a clean-up for the Makefile to avoid that > new tests get added with "=" instead of "+=" (and thus completely > overwrite the correspond variable by accident). Anyone have an ideas whose tree this should go through? I'm happy to take it through mine, if there are no better ideas, although there's nothing really ppc specific about it. > > v2: > - Some more clean-ups for the Makefile (i.e. some additional empty > lines) > - Check return value of read() to make sure that there is no error > > Thomas Huth (2): > tests: Resort check-qtest entries in Makefile.include > tests: Check serial output of firmware boot of some machines > > tests/Makefile.include | 46 ++++++++++++++------ > tests/boot-serial-test.c | 111 +++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 144 insertions(+), 13 deletions(-) > create mode 100644 tests/boot-serial-test.c > -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) 2016-07-18 10:02 ` [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) David Gibson @ 2016-09-02 22:11 ` Thomas Huth 2016-09-03 4:43 ` David Gibson 0 siblings, 1 reply; 8+ messages in thread From: Thomas Huth @ 2016-09-02 22:11 UTC (permalink / raw) To: David Gibson Cc: Peter Maydell, Eduardo Habkost, Peter Crosthwaite, qemu-devel, Markus Armbruster, Paolo Bonzini, Richard Henderson [-- Attachment #1: Type: text/plain, Size: 1069 bytes --] On 18.07.2016 12:02, David Gibson wrote: > On Fri, Jul 15, 2016 at 05:39:37PM +0200, Thomas Huth wrote: >> Since some of the machines that we support in QEMU sometimes get >> broken by other changes, and it then takes a while 'till somebody >> notices the breakage, it would be nice to get some more automatic >> test coverage for various machines during "make check". The >> second patch adds such a test for the machines where we've got >> a firmware image for and thus can test for some magic strings >> in the serial output of the firmware. >> >> The first patch is just a clean-up for the Makefile to avoid that >> new tests get added with "=" instead of "+=" (and thus completely >> overwrite the correspond variable by accident). > > Anyone have an ideas whose tree this should go through? > > I'm happy to take it through mine, if there are no better ideas, > although there's nothing really ppc specific about it. Since nobody else picked these patches up yet, could you maybe take them through your tree, David? Thanks! Thomas [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) 2016-09-02 22:11 ` Thomas Huth @ 2016-09-03 4:43 ` David Gibson 0 siblings, 0 replies; 8+ messages in thread From: David Gibson @ 2016-09-03 4:43 UTC (permalink / raw) To: Thomas Huth Cc: Peter Maydell, Eduardo Habkost, Peter Crosthwaite, qemu-devel, Markus Armbruster, Paolo Bonzini, Richard Henderson [-- Attachment #1: Type: text/plain, Size: 1411 bytes --] On Sat, Sep 03, 2016 at 12:11:53AM +0200, Thomas Huth wrote: > On 18.07.2016 12:02, David Gibson wrote: > > On Fri, Jul 15, 2016 at 05:39:37PM +0200, Thomas Huth wrote: > >> Since some of the machines that we support in QEMU sometimes get > >> broken by other changes, and it then takes a while 'till somebody > >> notices the breakage, it would be nice to get some more automatic > >> test coverage for various machines during "make check". The > >> second patch adds such a test for the machines where we've got > >> a firmware image for and thus can test for some magic strings > >> in the serial output of the firmware. > >> > >> The first patch is just a clean-up for the Makefile to avoid that > >> new tests get added with "=" instead of "+=" (and thus completely > >> overwrite the correspond variable by accident). > > > > Anyone have an ideas whose tree this should go through? > > > > I'm happy to take it through mine, if there are no better ideas, > > although there's nothing really ppc specific about it. > > Since nobody else picked these patches up yet, could you maybe take them > through your tree, David? Thanks! Uh.. sorry, I lost track of this one, can you resend. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-09-03 4:48 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-07-15 15:39 [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) Thomas Huth 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 1/2] tests: Resort check-qtest entries in Makefile.include Thomas Huth 2016-07-18 0:44 ` David Gibson 2016-07-15 15:39 ` [Qemu-devel] [PATCH v2 2/2] tests: Check serial output of firmware boot of some machines Thomas Huth 2016-07-18 0:45 ` David Gibson 2016-07-18 10:02 ` [Qemu-devel] [PATCH v2 0/2] Add a test for serial output (and sort tests by architecture in the Makefile) David Gibson 2016-09-02 22:11 ` Thomas Huth 2016-09-03 4:43 ` David Gibson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).