[RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[Alex Bennée]

When trying to work out what the virtio-net-tests where doing it was
hard because the g_test_trap_subprocess redirects all output to
/dev/null. Lift this restriction by using the appropriate flags so you
can see something similar to what the vhost-user-blk tests show when
running.

While we are at it remove the g_test_verbose() check so we always show
how the QEMU is run.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/qtest/qos-test.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tests/qtest/qos-test.c b/tests/qtest/qos-test.c
index f97d0a08fd..c6c196cc95 100644
--- a/tests/qtest/qos-test.c
+++ b/tests/qtest/qos-test.c
@@ -89,9 +89,7 @@ static void qos_set_machines_devices_available(void)
 
 static void restart_qemu_or_continue(char *path)
 {
-    if (g_test_verbose()) {
-        qos_printf("Run QEMU with: '%s'\n", path);
-    }
+    qos_printf("Run QEMU with: '%s'\n", path);
     /* compares the current command line with the
      * one previously executed: if they are the same,
      * don't restart QEMU, if they differ, stop previous
@@ -185,7 +183,8 @@ static void run_one_test(const void *arg)
 static void subprocess_run_one_test(const void *arg)
 {
     const gchar *path = arg;
-    g_test_trap_subprocess(path, 0, 0);
+    g_test_trap_subprocess(path, 0,
+                           G_TEST_SUBPROCESS_INHERIT_STDOUT | G_TEST_SUBPROCESS_INHERIT_STDERR);
     g_test_trap_assert_passed();
 }
 
-- 
2.30.2

Re: [RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[Eric Auger]

Hi Alex,

On 4/7/22 5:00 PM, Alex Bennée wrote:
> When trying to work out what the virtio-net-tests where doing it was
> hard because the g_test_trap_subprocess redirects all output to
> /dev/null. Lift this restriction by using the appropriate flags so you
> can see something similar to what the vhost-user-blk tests show when
> running.
> 
> While we are at it remove the g_test_verbose() check so we always show
> how the QEMU is run.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/qtest/qos-test.c | 7 +++----
>  1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/qtest/qos-test.c b/tests/qtest/qos-test.c
> index f97d0a08fd..c6c196cc95 100644
> --- a/tests/qtest/qos-test.c
> +++ b/tests/qtest/qos-test.c
> @@ -89,9 +89,7 @@ static void qos_set_machines_devices_available(void)
>  
>  static void restart_qemu_or_continue(char *path)
>  {
> -    if (g_test_verbose()) {
> -        qos_printf("Run QEMU with: '%s'\n", path);
> -    }
> +    qos_printf("Run QEMU with: '%s'\n", path);
>      /* compares the current command line with the
>       * one previously executed: if they are the same,
>       * don't restart QEMU, if they differ, stop previous
> @@ -185,7 +183,8 @@ static void run_one_test(const void *arg)
>  static void subprocess_run_one_test(const void *arg)
>  {
>      const gchar *path = arg;
> -    g_test_trap_subprocess(path, 0, 0);
> +    g_test_trap_subprocess(path, 0,
> +                           G_TEST_SUBPROCESS_INHERIT_STDOUT | G_TEST_SUBPROCESS_INHERIT_STDERR);
While workling on libqos/pci tests on aarch64 I also did that but I
noticed there were a bunch of errors such as:

/aarch64/virt/generic-pcihost/pci-bus-generic/pci-bus/virtio-net-pci/virtio-net/virtio-net-tests/vhost-user/multiqueue:
qemu-system-aarch64: Failed to set msg fds.
qemu-system-aarch64: vhost VQ 0 ring restore failed: -22: Invalid
argument (22)
qemu-system-aarch64: Failed to set msg fds.
qemu-system-aarch64: vhost VQ 1 ring restore failed: -22: Invalid
argument (22)
qemu-system-aarch64: Failed to set msg fds.
qemu-system-aarch64: vhost VQ 2 ring restore failed: -22: Invalid
argument (22)
qemu-system-aarch64: Failed to set msg fds.
qemu-system-aarch64: vhost VQ 3 ring restore failed: -22: Invalid
argument (22)

I see those also when running with x86_64-softmmu/qemu-system-x86_64
(this is no aarch64 specific).

I don't know if it is an issue to get those additional errors?

Thanks

Eric

>      g_test_trap_assert_passed();
>  }
>  
> 

Re: [RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 3198 bytes --]

On Thu, Apr 14, 2022 at 07:25:54PM +0200, Eric Auger wrote:
> Hi Alex,
> 
> On 4/7/22 5:00 PM, Alex Bennée wrote:
> > When trying to work out what the virtio-net-tests where doing it was
> > hard because the g_test_trap_subprocess redirects all output to
> > /dev/null. Lift this restriction by using the appropriate flags so you
> > can see something similar to what the vhost-user-blk tests show when
> > running.
> > 
> > While we are at it remove the g_test_verbose() check so we always show
> > how the QEMU is run.
> > 
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > ---
> >  tests/qtest/qos-test.c | 7 +++----
> >  1 file changed, 3 insertions(+), 4 deletions(-)
> > 
> > diff --git a/tests/qtest/qos-test.c b/tests/qtest/qos-test.c
> > index f97d0a08fd..c6c196cc95 100644
> > --- a/tests/qtest/qos-test.c
> > +++ b/tests/qtest/qos-test.c
> > @@ -89,9 +89,7 @@ static void qos_set_machines_devices_available(void)
> >  
> >  static void restart_qemu_or_continue(char *path)
> >  {
> > -    if (g_test_verbose()) {
> > -        qos_printf("Run QEMU with: '%s'\n", path);
> > -    }
> > +    qos_printf("Run QEMU with: '%s'\n", path);
> >      /* compares the current command line with the
> >       * one previously executed: if they are the same,
> >       * don't restart QEMU, if they differ, stop previous
> > @@ -185,7 +183,8 @@ static void run_one_test(const void *arg)
> >  static void subprocess_run_one_test(const void *arg)
> >  {
> >      const gchar *path = arg;
> > -    g_test_trap_subprocess(path, 0, 0);
> > +    g_test_trap_subprocess(path, 0,
> > +                           G_TEST_SUBPROCESS_INHERIT_STDOUT | G_TEST_SUBPROCESS_INHERIT_STDERR);
> While workling on libqos/pci tests on aarch64 I also did that but I
> noticed there were a bunch of errors such as:
> 
> /aarch64/virt/generic-pcihost/pci-bus-generic/pci-bus/virtio-net-pci/virtio-net/virtio-net-tests/vhost-user/multiqueue:
> qemu-system-aarch64: Failed to set msg fds.
> qemu-system-aarch64: vhost VQ 0 ring restore failed: -22: Invalid
> argument (22)
> qemu-system-aarch64: Failed to set msg fds.
> qemu-system-aarch64: vhost VQ 1 ring restore failed: -22: Invalid
> argument (22)
> qemu-system-aarch64: Failed to set msg fds.
> qemu-system-aarch64: vhost VQ 2 ring restore failed: -22: Invalid
> argument (22)
> qemu-system-aarch64: Failed to set msg fds.
> qemu-system-aarch64: vhost VQ 3 ring restore failed: -22: Invalid
> argument (22)
> 
> I see those also when running with x86_64-softmmu/qemu-system-x86_64
> (this is no aarch64 specific).
> 
> I don't know if it is an issue to get those additional errors?

I see the same errors on
/x86_64/pc/i440FX-pcihost/pci-bus-pc/pci-bus/virtio-net-pci/virtio-net/virtio-net-tests/vhost-user/.

On the other hand, "make check" is happy (and silent) when run on the
command-line.

If the CI enables more verbose output then these messages might be
diffed and interpreted as failures, but I didn't check the CI scripts.

As long as GitLab CI is happy I think it's okay to merge this patch, but
it would be interesting to investigate the reason for these messages.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[Alex Bennée]

Stefan Hajnoczi <stefanha@redhat.com> writes:

> [[PGP Signed Part:Undecided]]
> On Thu, Apr 14, 2022 at 07:25:54PM +0200, Eric Auger wrote:
>> Hi Alex,
>> 
>> On 4/7/22 5:00 PM, Alex Bennée wrote:
>> > When trying to work out what the virtio-net-tests where doing it was
>> > hard because the g_test_trap_subprocess redirects all output to
>> > /dev/null. Lift this restriction by using the appropriate flags so you
>> > can see something similar to what the vhost-user-blk tests show when
>> > running.
>> > 
>> > While we are at it remove the g_test_verbose() check so we always show
>> > how the QEMU is run.
>> > 
>> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> > ---
>> >  tests/qtest/qos-test.c | 7 +++----
>> >  1 file changed, 3 insertions(+), 4 deletions(-)
>> > 
>> > diff --git a/tests/qtest/qos-test.c b/tests/qtest/qos-test.c
>> > index f97d0a08fd..c6c196cc95 100644
>> > --- a/tests/qtest/qos-test.c
>> > +++ b/tests/qtest/qos-test.c
>> > @@ -89,9 +89,7 @@ static void qos_set_machines_devices_available(void)
>> >  
>> >  static void restart_qemu_or_continue(char *path)
>> >  {
>> > -    if (g_test_verbose()) {
>> > -        qos_printf("Run QEMU with: '%s'\n", path);
>> > -    }
>> > +    qos_printf("Run QEMU with: '%s'\n", path);
>> >      /* compares the current command line with the
>> >       * one previously executed: if they are the same,
>> >       * don't restart QEMU, if they differ, stop previous
>> > @@ -185,7 +183,8 @@ static void run_one_test(const void *arg)
>> >  static void subprocess_run_one_test(const void *arg)
>> >  {
>> >      const gchar *path = arg;
>> > -    g_test_trap_subprocess(path, 0, 0);
>> > +    g_test_trap_subprocess(path, 0,
>> > +                           G_TEST_SUBPROCESS_INHERIT_STDOUT | G_TEST_SUBPROCESS_INHERIT_STDERR);
>> While workling on libqos/pci tests on aarch64 I also did that but I
>> noticed there were a bunch of errors such as:
>> 
>> /aarch64/virt/generic-pcihost/pci-bus-generic/pci-bus/virtio-net-pci/virtio-net/virtio-net-tests/vhost-user/multiqueue:
>> qemu-system-aarch64: Failed to set msg fds.
>> qemu-system-aarch64: vhost VQ 0 ring restore failed: -22: Invalid
>> argument (22)
>> qemu-system-aarch64: Failed to set msg fds.
>> qemu-system-aarch64: vhost VQ 1 ring restore failed: -22: Invalid
>> argument (22)
>> qemu-system-aarch64: Failed to set msg fds.
>> qemu-system-aarch64: vhost VQ 2 ring restore failed: -22: Invalid
>> argument (22)
>> qemu-system-aarch64: Failed to set msg fds.
>> qemu-system-aarch64: vhost VQ 3 ring restore failed: -22: Invalid
>> argument (22)
>> 
>> I see those also when running with x86_64-softmmu/qemu-system-x86_64
>> (this is no aarch64 specific).

I think it's a case of things not being cleanly taken down leaving
dangling sockets. I suspect the qemu instance should shutdown before the
fake vhost-user daemon.

>> 
>> I don't know if it is an issue to get those additional errors?
>
> I see the same errors on
> /x86_64/pc/i440FX-pcihost/pci-bus-pc/pci-bus/virtio-net-pci/virtio-net/virtio-net-tests/vhost-user/.
>
> On the other hand, "make check" is happy (and silent) when run on the
> command-line.
>
> If the CI enables more verbose output then these messages might be
> diffed and interpreted as failures, but I didn't check the CI scripts.
>
> As long as GitLab CI is happy I think it's okay to merge this patch, but
> it would be interesting to investigate the reason for these messages.

Well this is all part of trying to get a working test case for the gpio
vhost-user device. So far I'm adding verbose output to try and divine
all the secret moving parts that go to make the test work.

>
> Stefan
>
> [[End of PGP Signed Part]]


-- 
Alex Bennée

Re: [RFC PATCH] tests/qtest: pass stdout/stderr down to subtests

[Thomas Huth]

On 07/04/2022 17.00, Alex Bennée wrote:
> When trying to work out what the virtio-net-tests where doing it was
> hard because the g_test_trap_subprocess redirects all output to
> /dev/null. Lift this restriction by using the appropriate flags so you
> can see something similar to what the vhost-user-blk tests show when
> running.
> 
> While we are at it remove the g_test_verbose() check so we always show
> how the QEMU is run.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/qtest/qos-test.c | 7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/qtest/qos-test.c b/tests/qtest/qos-test.c
> index f97d0a08fd..c6c196cc95 100644
> --- a/tests/qtest/qos-test.c
> +++ b/tests/qtest/qos-test.c
> @@ -89,9 +89,7 @@ static void qos_set_machines_devices_available(void)
>   
>   static void restart_qemu_or_continue(char *path)
>   {
> -    if (g_test_verbose()) {
> -        qos_printf("Run QEMU with: '%s'\n", path);
> -    }
> +    qos_printf("Run QEMU with: '%s'\n", path);

I think I'd rather drop this hunk since it breaks the usual output of the 
qtests. And adding a --verbose when running the test isn't that hard either, 
is it?

  Thomas