[Qemu-devel] [RFC] tests: Add ivshmem qtest

[Qemu-devel] [RFC] tests: Add ivshmem qtest

[Andreas Färber]

Note that it launches two instances to as sharing memory is the purpose
of Nahanni/ivshmem.

Cc: Cam Macdonell <cam@cs.ualberta.ca>
Signed-off-by: Andreas Färber <afaerber@suse.de>
---
 This test demonstrates a use case of running two QEMU instances in qtest.

 However, similar to mst's proposed KVM acpi-test, it fails on systems
 without KVM support, and we cannot use $(CONFIG_KVM) to make it conditional
 since that is a per-target define rather than host-wide.

 I wonder if libqtest should expose an API to start a QEMU instance and
 inquire via QMP whether the executable supports KVM, returning true or false?

 tests/Makefile       |  3 +++
 tests/ivshmem-test.c | 40 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)
 create mode 100644 tests/ivshmem-test.c

diff --git a/tests/Makefile b/tests/Makefile
index 6086f68..e4c3b12 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -137,6 +137,8 @@ check-qtest-i386-y += tests/pvpanic-test$(EXESUF)
 gcov-files-i386-y += i386-softmmu/hw/misc/pvpanic.c
 check-qtest-i386-y += tests/i82801b11-test$(EXESUF)
 gcov-files-i386-y += hw/pci-bridge/i82801b11.c
+check-qtest-i386-y += tests/ivshmem-test$(EXESUF)
+gcov-files-i386-y += i386-softmmu/hw/misc/ivshmem.c
 check-qtest-x86_64-y = $(check-qtest-i386-y)
 gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
 gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
@@ -287,6 +289,7 @@ tests/qdev-monitor-test$(EXESUF): tests/qdev-monitor-test.o $(libqos-pc-obj-y)
 tests/nvme-test$(EXESUF): tests/nvme-test.o
 tests/pvpanic-test$(EXESUF): tests/pvpanic-test.o
 tests/i82801b11-test$(EXESUF): tests/i82801b11-test.o
+tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o
 tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_scm_helper.o
 
 # QTest rules
diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
new file mode 100644
index 0000000..e981ad3
--- /dev/null
+++ b/tests/ivshmem-test.c
@@ -0,0 +1,40 @@
+/*
+ * QTest testcase for Nahanni
+ *
+ * Copyright (c) 2014 SUSE LINUX Products GmbH
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include <glib.h>
+#include <string.h>
+#include "libqtest.h"
+#include "qemu/osdep.h"
+
+/* Tests only initialization so far. TODO: Replace with functional tests */
+static void nop(void)
+{
+}
+
+int main(int argc, char **argv)
+{
+    QTestState *s1, *s2;
+    char *cmd;
+    int ret;
+
+    g_test_init(&argc, &argv, NULL);
+    qtest_add_func("/ivshmem/nop", nop);
+
+    cmd = g_strdup_printf("-device ivshmem,shm=%s,size=1M", "qtest");
+    s1 = qtest_start(cmd);
+    s2 = qtest_start(cmd);
+    g_free(cmd);
+
+    ret = g_test_run();
+
+    qtest_quit(s1);
+    qtest_quit(s2);
+
+    return ret;
+}
-- 
1.8.4.5

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Michael S. Tsirkin]

On Wed, Apr 02, 2014 at 04:57:48PM +0200, Andreas Färber wrote:
> Note that it launches two instances to as sharing memory is the purpose
> of Nahanni/ivshmem.
> 
> Cc: Cam Macdonell <cam@cs.ualberta.ca>
> Signed-off-by: Andreas Färber <afaerber@suse.de>
> ---
>  This test demonstrates a use case of running two QEMU instances in qtest.
> 
>  However, similar to mst's proposed KVM acpi-test, it fails on systems
>  without KVM support,

Hmm, acpi-test no longer depends on kvm.


> and we cannot use $(CONFIG_KVM) to make it conditional
>  since that is a per-target define rather than host-wide.
> 
>  I wonder if libqtest should expose an API to start a QEMU instance and
>  inquire via QMP whether the executable supports KVM, returning true or false?
> 
>  tests/Makefile       |  3 +++
>  tests/ivshmem-test.c | 40 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 43 insertions(+)
>  create mode 100644 tests/ivshmem-test.c
> 
> diff --git a/tests/Makefile b/tests/Makefile
> index 6086f68..e4c3b12 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -137,6 +137,8 @@ check-qtest-i386-y += tests/pvpanic-test$(EXESUF)
>  gcov-files-i386-y += i386-softmmu/hw/misc/pvpanic.c
>  check-qtest-i386-y += tests/i82801b11-test$(EXESUF)
>  gcov-files-i386-y += hw/pci-bridge/i82801b11.c
> +check-qtest-i386-y += tests/ivshmem-test$(EXESUF)
> +gcov-files-i386-y += i386-softmmu/hw/misc/ivshmem.c
>  check-qtest-x86_64-y = $(check-qtest-i386-y)
>  gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
>  gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
> @@ -287,6 +289,7 @@ tests/qdev-monitor-test$(EXESUF): tests/qdev-monitor-test.o $(libqos-pc-obj-y)
>  tests/nvme-test$(EXESUF): tests/nvme-test.o
>  tests/pvpanic-test$(EXESUF): tests/pvpanic-test.o
>  tests/i82801b11-test$(EXESUF): tests/i82801b11-test.o
> +tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o
>  tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_scm_helper.o
>  
>  # QTest rules
> diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
> new file mode 100644
> index 0000000..e981ad3
> --- /dev/null
> +++ b/tests/ivshmem-test.c
> @@ -0,0 +1,40 @@
> +/*
> + * QTest testcase for Nahanni
> + *
> + * Copyright (c) 2014 SUSE LINUX Products GmbH
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#include <glib.h>
> +#include <string.h>
> +#include "libqtest.h"
> +#include "qemu/osdep.h"
> +
> +/* Tests only initialization so far. TODO: Replace with functional tests */
> +static void nop(void)
> +{
> +}
> +
> +int main(int argc, char **argv)
> +{
> +    QTestState *s1, *s2;
> +    char *cmd;
> +    int ret;
> +
> +    g_test_init(&argc, &argv, NULL);
> +    qtest_add_func("/ivshmem/nop", nop);
> +
> +    cmd = g_strdup_printf("-device ivshmem,shm=%s,size=1M", "qtest");
> +    s1 = qtest_start(cmd);
> +    s2 = qtest_start(cmd);
> +    g_free(cmd);
> +
> +    ret = g_test_run();
> +
> +    qtest_quit(s1);
> +    qtest_quit(s2);
> +
> +    return ret;
> +}
> -- 
> 1.8.4.5

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Andreas Färber]

Am 02.04.2014 17:06, schrieb Michael S. Tsirkin:
> On Wed, Apr 02, 2014 at 04:57:48PM +0200, Andreas Färber wrote:
>> Note that it launches two instances to as sharing memory is the purpose
>> of Nahanni/ivshmem.
>>
>> Cc: Cam Macdonell <cam@cs.ualberta.ca>
>> Signed-off-by: Andreas Färber <afaerber@suse.de>
>> ---
>>  This test demonstrates a use case of running two QEMU instances in qtest.
>>
>>  However, similar to mst's proposed KVM acpi-test, it fails on systems
>>  without KVM support,
> 
> Hmm, acpi-test no longer depends on kvm.

You only committed the TCG version due to someone raising the issue. But
maybe I'm mistaken...

Andreas

>> and we cannot use $(CONFIG_KVM) to make it conditional
>>  since that is a per-target define rather than host-wide.
>>
>>  I wonder if libqtest should expose an API to start a QEMU instance and
>>  inquire via QMP whether the executable supports KVM, returning true or false?

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Michael S. Tsirkin]

On Wed, Apr 02, 2014 at 05:07:09PM +0200, Andreas Färber wrote:
> Am 02.04.2014 17:06, schrieb Michael S. Tsirkin:
> > On Wed, Apr 02, 2014 at 04:57:48PM +0200, Andreas Färber wrote:
> >> Note that it launches two instances to as sharing memory is the purpose
> >> of Nahanni/ivshmem.
> >>
> >> Cc: Cam Macdonell <cam@cs.ualberta.ca>
> >> Signed-off-by: Andreas Färber <afaerber@suse.de>
> >> ---
> >>  This test demonstrates a use case of running two QEMU instances in qtest.
> >>
> >>  However, similar to mst's proposed KVM acpi-test, it fails on systems
> >>  without KVM support,
> > 
> > Hmm, acpi-test no longer depends on kvm.
> 
> You only committed the TCG version due to someone raising the issue. But
> maybe I'm mistaken...
> 
> Andreas

Yes and I think this someone was you actually.
Can something similar be done for this test?

> >> and we cannot use $(CONFIG_KVM) to make it conditional
> >>  since that is a per-target define rather than host-wide.
> >>
> >>  I wonder if libqtest should expose an API to start a QEMU instance and
> >>  inquire via QMP whether the executable supports KVM, returning true or false?
> 
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Paolo Bonzini]

Il 02/04/2014 17:15, Michael S. Tsirkin ha scritto:
> Yes and I think this someone was you actually.
> Can something similar be done for this test?
>

I think ivshmem depends on ioeventfd, doesn't it?  So short of 
implementing ioeventfd for TCG, no.

Paolo

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Stefan Hajnoczi]

On Wed, Apr 02, 2014 at 04:57:48PM +0200, Andreas Färber wrote:
> +int main(int argc, char **argv)
> +{
> +    QTestState *s1, *s2;
> +    char *cmd;
> +    int ret;
> +
> +    g_test_init(&argc, &argv, NULL);
> +    qtest_add_func("/ivshmem/nop", nop);
> +
> +    cmd = g_strdup_printf("-device ivshmem,shm=%s,size=1M", "qtest");

This test leaks the "qtest" shm object that gets created by the first
QEMU process.  The name is constant so two instances of the test cannot
run in parallel on the same machine (the objects go in /dev/shm).

The name should be unique and we should clean up in both success and
failure (abort(3)) cases.

Stefan

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Michael S. Tsirkin]

On Wed, Apr 02, 2014 at 05:15:59PM +0200, Paolo Bonzini wrote:
> Il 02/04/2014 17:15, Michael S. Tsirkin ha scritto:
> >Yes and I think this someone was you actually.
> >Can something similar be done for this test?
> >
> 
> I think ivshmem depends on ioeventfd, doesn't it?  So short of
> implementing ioeventfd for TCG, no.
> 
> Paolo

Okay so will make check just fail on a non kvm host?

-- 
MST

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Andreas Färber]

Am 03.04.2014 13:16, schrieb Michael S. Tsirkin:
> On Wed, Apr 02, 2014 at 05:15:59PM +0200, Paolo Bonzini wrote:
>> Il 02/04/2014 17:15, Michael S. Tsirkin ha scritto:
>>> Yes and I think this someone was you actually.
>>> Can something similar be done for this test?
>>>
>>
>> I think ivshmem depends on ioeventfd, doesn't it?  So short of
>> implementing ioeventfd for TCG, no.
>>
>> Paolo
> 
> Okay so will make check just fail on a non kvm host?

Yes, this test fails on FreeBSD for instance, therefore this is a
non-polished RFC. I'm hoping for comments how to best make it not crash.

Cheers,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Andreas Färber]

Am 03.04.2014 10:46, schrieb Stefan Hajnoczi:
> On Wed, Apr 02, 2014 at 04:57:48PM +0200, Andreas Färber wrote:
>> +int main(int argc, char **argv)
>> +{
>> +    QTestState *s1, *s2;
>> +    char *cmd;
>> +    int ret;
>> +
>> +    g_test_init(&argc, &argv, NULL);
>> +    qtest_add_func("/ivshmem/nop", nop);
>> +
>> +    cmd = g_strdup_printf("-device ivshmem,shm=%s,size=1M", "qtest");
> 
> This test leaks the "qtest" shm object that gets created by the first
> QEMU process.  The name is constant so two instances of the test cannot
> run in parallel on the same machine (the objects go in /dev/shm).

Right. Therefore it's already using %s and not a fully hardcoded string.

> The name should be unique and we should clean up in both success and
> failure (abort(3)) cases.

The recipe for unique filenames elsewhere is mkstemp(), which I was
planning to use here as well using "/dev/shm/qtest.XXXXXX" and then to
somehow strip the /dev/shm/ part, e.g. &...[9].

However I do not see any existing test specially cleaning up such
temporary files on SIGABRT rather than just on success. Do you have a
pointer or suggestion how to do that?

Still the actual question of this RFC is, how do I detect whether we may
run this test at all?

1) Relying on being run in the build directory, we could try to peek at
$arch-softmmu/config-target.mak, looking for CONFIG_KVM. Ugly.

2) Determine via QMP whether KVM is available in the QEMU binary. How?
Involves an additional process before launching the actual process with
-device ivshmem.

3) Determine via QMP whether the device QOM type is available in the
QEMU binary. Involves an additional process.

4) Make ivshmem build independent of CONFIG_KVM. Too much work for a
single out of many test cases.

5) ???

Thanks,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Stefan Hajnoczi]

On Thu, Apr 03, 2014 at 01:45:28PM +0200, Andreas Färber wrote:
> Am 03.04.2014 10:46, schrieb Stefan Hajnoczi:
> > The name should be unique and we should clean up in both success and
> > failure (abort(3)) cases.
> 
> The recipe for unique filenames elsewhere is mkstemp(), which I was
> planning to use here as well using "/dev/shm/qtest.XXXXXX" and then to
> somehow strip the /dev/shm/ part, e.g. &...[9].
> 
> However I do not see any existing test specially cleaning up such
> temporary files on SIGABRT rather than just on success. Do you have a
> pointer or suggestion how to do that?

We currently don't have a atabort() mechanism (like atexit(3)) but I
think this is needed.  This would be a separate libqtest patch.

> Still the actual question of this RFC is, how do I detect whether we may
> run this test at all?
> 
> 1) Relying on being run in the build directory, we could try to peek at
> $arch-softmmu/config-target.mak, looking for CONFIG_KVM. Ugly.
> 
> 2) Determine via QMP whether KVM is available in the QEMU binary. How?
> Involves an additional process before launching the actual process with
> -device ivshmem.
> 
> 3) Determine via QMP whether the device QOM type is available in the
> QEMU binary. Involves an additional process.
> 
> 4) Make ivshmem build independent of CONFIG_KVM. Too much work for a
> single out of many test cases.
> 
> 5) ???

Another ugly option, add the test to the i386/x86_64 softmmu target and
only run on Linux hosts.

Stefan

Re: [Qemu-devel] [RFC] tests: Add ivshmem qtest

[Michael S. Tsirkin]

On Thu, Apr 03, 2014 at 01:23:08PM +0200, Andreas Färber wrote:
> Am 03.04.2014 13:16, schrieb Michael S. Tsirkin:
> > On Wed, Apr 02, 2014 at 05:15:59PM +0200, Paolo Bonzini wrote:
> >> Il 02/04/2014 17:15, Michael S. Tsirkin ha scritto:
> >>> Yes and I think this someone was you actually.
> >>> Can something similar be done for this test?
> >>>
> >>
> >> I think ivshmem depends on ioeventfd, doesn't it?  So short of
> >> implementing ioeventfd for TCG, no.
> >>
> >> Paolo
> > 
> > Okay so will make check just fail on a non kvm host?
> 
> Yes, this test fails on FreeBSD for instance, therefore this is a
> non-polished RFC. I'm hoping for comments how to best make it not crash.
> 
> Cheers,
> Andreas

Well we could work around this
	- start machine with kvm or without if not supported
	- query ioeventfd support


But maybe it's easier to implement eventfd APIs ...

> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg