[PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Bin Meng]

In preparation to adding virtio-9p support on Windows, this series
enables running qtest on Windows, so that we can run the virtio-9p
tests on Windows to make sure it does not break accidently.

Changes in v6:
- drop patches that are already in Alex and Daniel's tree
- remove CONFIG_POSIX from meson.build
- include <qemu/sockets.h> in libqtest.c
- move documentation comments of qemu_send_full() from util/osdep.c
  to qemu/sockets.h
- save the "exit_code" in struct QTestState
- new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"
- new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
- new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
- change to use qtest_wait_qemu() API
- new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"

Changes in v5:
- restore to v1 version which does not touch the posix implementation
- Drop patches that are already merged

Changes in v3:
- Add a usleep(1) in the busy wait loop
- Drop the host test

Changes in v2:
- Introduce qemu_send_full() and use it
- Move the enabling of building qtests on Windows to a separate
  patch to keep bisectablity
- Call socket_init() unconditionally
- Add a missing CloseHandle() call
- Change to a busy wait after migration is canceled
- Change the timeout limit to 90 minutes
- new patch: "tests/qtest: Enable qtest build on Windows"

Bin Meng (8):
  tests/qtest: Support libqtest to build and run on Windows
  tests/qtest: device-plug-test: Reverse the usage of double/single
    quotes
  tests/qtest: Use EXIT_FAILURE instead of magic number
  tests/qtest: libqtest: Introduce qtest_wait_qemu()
  tests/qtest: libqos: Do not build virtio-9p unconditionally
  tests/qtest: libqtest: Correct the timeout unit of blocking receive
    calls for win32
  .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
  tests/qtest: Enable qtest build on Windows

Xuzhou Cheng (3):
  accel/qtest: Support qtest accelerator for Windows
  tests/qtest: Use send/recv for socket communication
  tests/qtest: migration-test: Make sure QEMU process "to" exited after
    migration is canceled

 include/hw/core/cpu.h           |   1 +
 include/qemu/sockets.h          |  13 +++
 tests/qtest/libqtest.h          |   9 ++
 accel/dummy-cpus.c              |  14 ++-
 softmmu/cpus.c                  |   9 +-
 tests/qtest/dbus-vmstate-test.c |   2 +-
 tests/qtest/device-plug-test.c  |  16 ++--
 tests/qtest/libqmp.c            |   5 +-
 tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
 tests/qtest/migration-test.c    |   8 +-
 util/osdep.c                    |  22 +++++
 .gitlab-ci.d/windows.yml        |   4 +-
 accel/meson.build               |   2 +-
 accel/qtest/meson.build         |   3 +-
 tests/qtest/libqos/meson.build  |   6 +-
 tests/qtest/meson.build         |   6 --
 16 files changed, 221 insertions(+), 50 deletions(-)

-- 
2.25.1

[PATCH v6 01/11] accel/qtest: Support qtest accelerator for Windows

[Bin Meng]

From: Xuzhou Cheng <xuzhou.cheng@windriver.com>

Currently signal SIGIPI [=SIGUSR1] is used to kick the dummy CPU
when qtest accelerator is used. However SIGUSR1 is unsupported on
Windows. To support Windows, we add a QemuSemaphore CPUState::sem
to kick the dummy CPU instead for Windows.

Signed-off-by: Xuzhou Cheng <xuzhou.cheng@windriver.com>
Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

---

Changes in v6:
- remove CONFIG_POSIX from meson.build

Changes in v5:
- restore to v1 version which does not touch the posix implementation

 include/hw/core/cpu.h   |  1 +
 accel/dummy-cpus.c      | 14 ++++++++++++--
 softmmu/cpus.c          |  9 +++++----
 accel/meson.build       |  2 +-
 accel/qtest/meson.build |  3 +--
 5 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index f9b58773f7..8830546121 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -333,6 +333,7 @@ struct CPUState {
     struct QemuThread *thread;
 #ifdef _WIN32
     HANDLE hThread;
+    QemuSemaphore sem;
 #endif
     int thread_id;
     bool running, has_waiter;
diff --git a/accel/dummy-cpus.c b/accel/dummy-cpus.c
index 10429fdfb2..d6a1b8d0a2 100644
--- a/accel/dummy-cpus.c
+++ b/accel/dummy-cpus.c
@@ -21,8 +21,6 @@
 static void *dummy_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
-    sigset_t waitset;
-    int r;
 
     rcu_register_thread();
 
@@ -32,8 +30,13 @@ static void *dummy_cpu_thread_fn(void *arg)
     cpu->can_do_io = 1;
     current_cpu = cpu;
 
+#ifndef _WIN32
+    sigset_t waitset;
+    int r;
+
     sigemptyset(&waitset);
     sigaddset(&waitset, SIG_IPI);
+#endif
 
     /* signal CPU creation */
     cpu_thread_signal_created(cpu);
@@ -41,6 +44,7 @@ static void *dummy_cpu_thread_fn(void *arg)
 
     do {
         qemu_mutex_unlock_iothread();
+#ifndef _WIN32
         do {
             int sig;
             r = sigwait(&waitset, &sig);
@@ -49,6 +53,9 @@ static void *dummy_cpu_thread_fn(void *arg)
             perror("sigwait");
             exit(1);
         }
+#else
+        qemu_sem_wait(&cpu->sem);
+#endif
         qemu_mutex_lock_iothread();
         qemu_wait_io_event(cpu);
     } while (!cpu->unplug);
@@ -69,4 +76,7 @@ void dummy_start_vcpu_thread(CPUState *cpu)
              cpu->cpu_index);
     qemu_thread_create(cpu->thread, thread_name, dummy_cpu_thread_fn, cpu,
                        QEMU_THREAD_JOINABLE);
+#ifdef _WIN32
+    qemu_sem_init(&cpu->sem, 0);
+#endif
 }
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 61b27ff59d..9dd1a4dc17 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -437,18 +437,19 @@ void qemu_wait_io_event(CPUState *cpu)
 
 void cpus_kick_thread(CPUState *cpu)
 {
-#ifndef _WIN32
-    int err;
-
     if (cpu->thread_kicked) {
         return;
     }
     cpu->thread_kicked = true;
-    err = pthread_kill(cpu->thread->thread, SIG_IPI);
+
+#ifndef _WIN32
+    int err = pthread_kill(cpu->thread->thread, SIG_IPI);
     if (err && err != ESRCH) {
         fprintf(stderr, "qemu:%s: %s", __func__, strerror(err));
         exit(1);
     }
+#else
+    qemu_sem_post(&cpu->sem);
 #endif
 }
 
diff --git a/accel/meson.build b/accel/meson.build
index b9a963cf80..259c35c4c8 100644
--- a/accel/meson.build
+++ b/accel/meson.build
@@ -16,5 +16,5 @@ dummy_ss.add(files(
   'dummy-cpus.c',
 ))
 
-specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: dummy_ss)
+specific_ss.add_all(when: ['CONFIG_SOFTMMU'], if_true: dummy_ss)
 specific_ss.add_all(when: ['CONFIG_XEN'], if_true: dummy_ss)
diff --git a/accel/qtest/meson.build b/accel/qtest/meson.build
index 4c65600293..176d990ae1 100644
--- a/accel/qtest/meson.build
+++ b/accel/qtest/meson.build
@@ -1,2 +1 @@
-qtest_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'],
-                    if_true: files('qtest.c'))
+qtest_module_ss.add(when: ['CONFIG_SOFTMMU'], if_true: files('qtest.c'))
-- 
2.25.1

[PATCH v6 02/11] tests/qtest: Use send/recv for socket communication

[Bin Meng]

From: Xuzhou Cheng <xuzhou.cheng@windriver.com>

Socket communication in the libqtest and libqmp codes uses read()
and write() which work on any file descriptor on *nix, and sockets
in *nix are an example of a file descriptor.

However sockets on Windows do not use *nix-style file descriptors,
so read() and write() cannot be used on sockets on Windows.
Switch over to use send() and recv() instead which work on both
Windows and *nix.

Signed-off-by: Xuzhou Cheng <xuzhou.cheng@windriver.com>
Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

---

Changes in v6:
- include <qemu/sockets.h> in libqtest.c
- move documentation comments of qemu_send_full() from util/osdep.c
  to qemu/sockets.h

Changes in v2:
- Introduce qemu_send_full() and use it

 include/qemu/sockets.h | 13 +++++++++++++
 tests/qtest/libqmp.c   |  5 +++--
 tests/qtest/libqtest.c |  5 +++--
 util/osdep.c           | 22 ++++++++++++++++++++++
 4 files changed, 41 insertions(+), 4 deletions(-)

diff --git a/include/qemu/sockets.h b/include/qemu/sockets.h
index 036745e586..61648f3f3c 100644
--- a/include/qemu/sockets.h
+++ b/include/qemu/sockets.h
@@ -33,6 +33,19 @@ int qemu_socketpair(int domain, int type, int protocol, int sv[2]);
 #endif
 
 int qemu_accept(int s, struct sockaddr *addr, socklen_t *addrlen);
+/*
+ * A variant of send(2) which handles partial send.
+ *
+ * Return the number of bytes transferred over the socket.
+ * Set errno if fewer than `count' bytes are sent.
+ *
+ * This function don't work with non-blocking socket's.
+ * Any of the possibilities with non-blocking socket's is bad:
+ *   - return a short write (then name is wrong)
+ *   - busy wait adding (errno == EAGAIN) to the loop
+ */
+ssize_t qemu_send_full(int s, const void *buf, size_t count)
+    G_GNUC_WARN_UNUSED_RESULT;
 int socket_set_cork(int fd, int v);
 int socket_set_nodelay(int fd);
 void qemu_socket_set_block(int fd);
diff --git a/tests/qtest/libqmp.c b/tests/qtest/libqmp.c
index ade26c15f0..2b08382e5d 100644
--- a/tests/qtest/libqmp.c
+++ b/tests/qtest/libqmp.c
@@ -23,6 +23,7 @@
 #endif
 
 #include "qemu/cutils.h"
+#include "qemu/sockets.h"
 #include "qapi/error.h"
 #include "qapi/qmp/json-parser.h"
 #include "qapi/qmp/qjson.h"
@@ -36,7 +37,7 @@ typedef struct {
 
 static void socket_send(int fd, const char *buf, size_t size)
 {
-    size_t res = qemu_write_full(fd, buf, size);
+    ssize_t res = qemu_send_full(fd, buf, size);
 
     assert(res == size);
 }
@@ -69,7 +70,7 @@ QDict *qmp_fd_receive(int fd)
         ssize_t len;
         char c;
 
-        len = read(fd, &c, 1);
+        len = recv(fd, &c, 1, 0);
         if (len == -1 && errno == EINTR) {
             continue;
         }
diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index b23eb3edc3..b01846fd98 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -27,6 +27,7 @@
 #include "libqmp.h"
 #include "qemu/ctype.h"
 #include "qemu/cutils.h"
+#include "qemu/sockets.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/qlist.h"
@@ -428,7 +429,7 @@ void qtest_quit(QTestState *s)
 
 static void socket_send(int fd, const char *buf, size_t size)
 {
-    size_t res = qemu_write_full(fd, buf, size);
+    ssize_t res = qemu_send_full(fd, buf, size);
 
     assert(res == size);
 }
@@ -460,7 +461,7 @@ static GString *qtest_client_socket_recv_line(QTestState *s)
         ssize_t len;
         char buffer[1024];
 
-        len = read(s->fd, buffer, sizeof(buffer));
+        len = recv(s->fd, buffer, sizeof(buffer), 0);
         if (len == -1 && errno == EINTR) {
             continue;
         }
diff --git a/util/osdep.c b/util/osdep.c
index 746d5f7d71..77c1a6c562 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -502,6 +502,28 @@ int qemu_accept(int s, struct sockaddr *addr, socklen_t *addrlen)
     return ret;
 }
 
+ssize_t qemu_send_full(int s, const void *buf, size_t count)
+{
+    ssize_t ret = 0;
+    ssize_t total = 0;
+
+    while (count) {
+        ret = send(s, buf, count, 0);
+        if (ret < 0) {
+            if (errno == EINTR) {
+                continue;
+            }
+            break;
+        }
+
+        count -= ret;
+        buf += ret;
+        total += ret;
+    }
+
+    return total;
+}
+
 void qemu_set_hw_version(const char *version)
 {
     hw_version = version;
-- 
2.25.1

[PATCH v6 03/11] tests/qtest: Support libqtest to build and run on Windows

[Bin Meng]

At present the libqtest codes were written to depend on several
POSIX APIs, including fork(), kill() and waitpid(). Unfortunately
these APIs are not available on Windows.

This commit implements the corresponding functionalities using
win32 native APIs. With this change, all qtest cases can build
successfully on a Windows host, and we can start qtest testing
on Windows now.

Signed-off-by: Xuzhou Cheng <xuzhou.cheng@windriver.com>
Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- save the "exit_code" in struct QTestState

Changes in v2:
- Move the enabling of building qtests on Windows to a separate
  patch to keep bisectablity
- Call socket_init() unconditionally
- Add a missing CloseHandle() call

 tests/qtest/libqtest.c | 96 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 94 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index b01846fd98..d12a604d78 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -16,9 +16,11 @@
 
 #include "qemu/osdep.h"
 
+#ifndef _WIN32
 #include <sys/socket.h>
 #include <sys/wait.h>
 #include <sys/un.h>
+#endif /* _WIN32 */
 #ifdef __linux__
 #include <sys/prctl.h>
 #endif /* __linux__ */
@@ -36,6 +38,16 @@
 #define MAX_IRQ 256
 #define SOCKET_TIMEOUT 50
 
+#ifndef _WIN32
+# define CMD_EXEC   "exec "
+# define DEV_STDERR "/dev/fd/2"
+# define DEV_NULL   "/dev/null"
+#else
+# define CMD_EXEC   ""
+# define DEV_STDERR "2"
+# define DEV_NULL   "nul"
+#endif
+
 typedef void (*QTestSendFn)(QTestState *s, const char *buf);
 typedef void (*ExternalSendFn)(void *s, const char *buf);
 typedef GString* (*QTestRecvFn)(QTestState *);
@@ -58,6 +70,9 @@ struct QTestState
     int qmp_fd;
     pid_t qemu_pid;  /* our child QEMU process */
     int wstatus;
+#ifdef _WIN32
+    DWORD exit_code;
+#endif
     int expected_status;
     bool big_endian;
     bool irq_level[MAX_IRQ];
@@ -119,10 +134,18 @@ bool qtest_probe_child(QTestState *s)
     pid_t pid = s->qemu_pid;
 
     if (pid != -1) {
+#ifndef _WIN32
         pid = waitpid(pid, &s->wstatus, WNOHANG);
         if (pid == 0) {
             return true;
         }
+#else
+        GetExitCodeProcess((HANDLE)pid, &s->exit_code);
+        if (s->exit_code == STILL_ACTIVE) {
+            return true;
+        }
+        CloseHandle((HANDLE)pid);
+#endif
         s->qemu_pid = -1;
     }
     return false;
@@ -136,13 +159,25 @@ void qtest_set_expected_status(QTestState *s, int status)
 void qtest_kill_qemu(QTestState *s)
 {
     pid_t pid = s->qemu_pid;
+#ifndef _WIN32
     int wstatus;
+#else
+    DWORD ret;
+#endif
 
     /* Skip wait if qtest_probe_child already reaped.  */
     if (pid != -1) {
+#ifndef _WIN32
         kill(pid, SIGTERM);
         TFR(pid = waitpid(s->qemu_pid, &s->wstatus, 0));
         assert(pid == s->qemu_pid);
+#else
+        TerminateProcess((HANDLE)pid, s->expected_status);
+        ret = WaitForSingleObject((HANDLE)pid, INFINITE);
+        assert(ret == WAIT_OBJECT_0);
+        GetExitCodeProcess((HANDLE)pid, &s->exit_code);
+        CloseHandle((HANDLE)pid);
+#endif
         s->qemu_pid = -1;
     }
 
@@ -150,6 +185,7 @@ void qtest_kill_qemu(QTestState *s)
      * Check whether qemu exited with expected exit status; anything else is
      * fishy and should be logged with as much detail as possible.
      */
+#ifndef _WIN32
     wstatus = s->wstatus;
     if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) != s->expected_status) {
         fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
@@ -166,6 +202,14 @@ void qtest_kill_qemu(QTestState *s)
                 __FILE__, __LINE__, sig, signame, dump);
         abort();
     }
+#else
+    if (s->exit_code != s->expected_status) {
+        fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
+                "process but encountered exit status %ld (expected %d)\n",
+                __FILE__, __LINE__, s->exit_code, s->expected_status);
+        abort();
+    }
+#endif
 }
 
 static void kill_qemu_hook_func(void *s)
@@ -244,6 +288,38 @@ static const char *qtest_qemu_binary(void)
     return qemu_bin;
 }
 
+#ifdef _WIN32
+static pid_t qtest_create_process(char *cmd)
+{
+    STARTUPINFO si;
+    PROCESS_INFORMATION pi;
+    BOOL ret;
+
+    ZeroMemory(&si, sizeof(si));
+    si.cb = sizeof(si);
+    ZeroMemory(&pi, sizeof(pi));
+
+    ret = CreateProcess(NULL,   /* module name */
+                        cmd,    /* command line */
+                        NULL,   /* process handle not inheritable */
+                        NULL,   /* thread handle not inheritable */
+                        FALSE,  /* set handle inheritance to FALSE */
+                        0,      /* No creation flags */
+                        NULL,   /* use parent's environment block */
+                        NULL,   /* use parent's starting directory */
+                        &si,    /* pointer to STARTUPINFO structure */
+                        &pi     /* pointer to PROCESS_INFORMATION structure */
+                        );
+    if (ret == 0) {
+        fprintf(stderr, "%s:%d: unable to create a new process (%s)\n",
+                __FILE__, __LINE__, strerror(GetLastError()));
+        abort();
+    }
+
+    return (pid_t)pi.hProcess;
+}
+#endif /* _WIN32 */
+
 QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
 {
     QTestState *s;
@@ -271,6 +347,7 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
     unlink(socket_path);
     unlink(qmp_socket_path);
 
+    socket_init();
     sock = init_socket(socket_path);
     qmpsock = init_socket(qmp_socket_path);
 
@@ -279,7 +356,7 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
 
     qtest_add_abrt_handler(kill_qemu_hook_func, s);
 
-    command = g_strdup_printf("exec %s %s"
+    command = g_strdup_printf(CMD_EXEC "%s %s"
                               "-qtest unix:%s "
                               "-qtest-log %s "
                               "-chardev socket,path=%s,id=char0 "
@@ -288,7 +365,7 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
                               "%s"
                               " -accel qtest",
                               qemu_binary, tracearg, socket_path,
-                              getenv("QTEST_LOG") ? "/dev/fd/2" : "/dev/null",
+                              getenv("QTEST_LOG") ? DEV_STDERR : DEV_NULL,
                               qmp_socket_path,
                               extra_args ?: "");
 
@@ -297,6 +374,7 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
     s->pending_events = NULL;
     s->wstatus = 0;
     s->expected_status = 0;
+#ifndef _WIN32
     s->qemu_pid = fork();
     if (s->qemu_pid == 0) {
 #ifdef __linux__
@@ -319,6 +397,9 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
         execlp("/bin/sh", "sh", "-c", command, NULL);
         exit(1);
     }
+#else
+    s->qemu_pid = qtest_create_process(command);
+#endif /* _WIN32 */
 
     g_free(command);
     s->fd = socket_accept(sock);
@@ -337,9 +418,19 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
         s->irq_level[i] = false;
     }
 
+    /*
+     * Stopping QEMU for debugging is not supported on Windows.
+     *
+     * Using DebugActiveProcess() API can suspend the QEMU process,
+     * but gdb cannot attach to the process. Using the undocumented
+     * NtSuspendProcess() can suspend the QEMU process and gdb can
+     * attach to the process, but gdb cannot resume it.
+     */
+#ifndef _WIN32
     if (getenv("QTEST_STOP")) {
         kill(s->qemu_pid, SIGSTOP);
     }
+#endif
 
     /* ask endianness of the target */
 
@@ -393,6 +484,7 @@ QTestState *qtest_init_with_serial(const char *extra_args, int *sock_fd)
     g_assert_true(sock_dir != NULL);
     sock_path = g_strdup_printf("%s/sock", sock_dir);
 
+    socket_init();
     sock_fd_init = init_socket(sock_path);
 
     qts = qtest_initf("-chardev socket,id=s0,path=%s -serial chardev:s0 %s",
-- 
2.25.1

[PATCH v6 04/11] tests/qtest: device-plug-test: Reverse the usage of double/single quotes

[Bin Meng]

The usage of double/single quotes in test_q35_pci_unplug_json_request()
should be reversed to work on both win32 and non-win32 platforms:

- The value of -device parameter needs to be surrounded by "" as
  Windows does not drop '' when passing it to QEMU which causes
  QEMU command line option parser failure.
- The JSON key/value pairs need to be surrounded by '' to make the
  JSON parser happy on Windows.

Fixes: a12f1a7e56b7 ("tests/x86: Add subtest with 'q35' machine type to device-plug-test")
Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"

 tests/qtest/device-plug-test.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/qtest/device-plug-test.c b/tests/qtest/device-plug-test.c
index 3f44f731d1..5a6afa2b57 100644
--- a/tests/qtest/device-plug-test.c
+++ b/tests/qtest/device-plug-test.c
@@ -112,16 +112,16 @@ static void test_pci_unplug_json_request(void)
 
 static void test_q35_pci_unplug_json_request(void)
 {
-    const char *port = "-device '{\"driver\": \"pcie-root-port\", "
-                                      "\"id\": \"p1\"}'";
+    const char *port = "-device \"{'driver': 'pcie-root-port', "
+                                  "'id': 'p1'}\"";
 
-    const char *bridge = "-device '{\"driver\": \"pcie-pci-bridge\", "
-                                   "\"id\": \"b1\", "
-                                   "\"bus\": \"p1\"}'";
+    const char *bridge = "-device \"{'driver': 'pcie-pci-bridge', "
+                                    "'id': 'b1', "
+                                    "'bus': 'p1'}\"";
 
-    const char *device = "-device '{\"driver\": \"virtio-mouse-pci\", "
-                                   "\"bus\": \"b1\", "
-                                   "\"id\": \"dev0\"}'";
+    const char *device = "-device \"{'driver': 'virtio-mouse-pci', "
+                                    "'bus': 'b1', "
+                                    "'id': 'dev0'}\"";
 
     QTestState *qtest = qtest_initf("-machine q35 %s %s %s",
                                     port, bridge, device);
-- 
2.25.1

[PATCH v6 05/11] tests/qtest: Use EXIT_FAILURE instead of magic number

[Bin Meng]

When migration fails, QEMU exits with a status code EXIT_FAILURE.
Change qtests to use the well-defined macro instead of magic number.

Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"

 tests/qtest/dbus-vmstate-test.c | 2 +-
 tests/qtest/migration-test.c    | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/qtest/dbus-vmstate-test.c b/tests/qtest/dbus-vmstate-test.c
index 74ede651f6..6c990864e3 100644
--- a/tests/qtest/dbus-vmstate-test.c
+++ b/tests/qtest/dbus-vmstate-test.c
@@ -233,7 +233,7 @@ test_dbus_vmstate(Test *test)
     test->src_qemu = src_qemu;
     if (test->migrate_fail) {
         wait_for_migration_fail(src_qemu, true);
-        qtest_set_expected_status(dst_qemu, 1);
+        qtest_set_expected_status(dst_qemu, EXIT_FAILURE);
     } else {
         wait_for_migration_complete(src_qemu);
     }
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index aa1ba179fa..28a06d8170 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1342,7 +1342,7 @@ static void test_precopy_common(MigrateCommon *args)
         wait_for_migration_fail(from, allow_active);
 
         if (args->result == MIG_TEST_FAIL_DEST_QUIT_ERR) {
-            qtest_set_expected_status(to, 1);
+            qtest_set_expected_status(to, EXIT_FAILURE);
         }
     } else {
         if (args->iterations) {
@@ -1738,7 +1738,7 @@ static void do_test_validate_uuid(MigrateStart *args, bool should_fail)
     migrate_qmp(from, uri, "{}");
 
     if (should_fail) {
-        qtest_set_expected_status(to, 1);
+        qtest_set_expected_status(to, EXIT_FAILURE);
         wait_for_migration_fail(from, true);
     } else {
         wait_for_migration_complete(from);
-- 
2.25.1

[PATCH v6 06/11] tests/qtest: libqtest: Introduce qtest_wait_qemu()

[Bin Meng]

Introduce an API for qtest to wait for the QEMU process to terminate.

Suggested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"

 tests/qtest/libqtest.h |  9 ++++++
 tests/qtest/libqtest.c | 63 +++++++++++++++++++++++++-----------------
 2 files changed, 47 insertions(+), 25 deletions(-)

diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
index 65c040e504..91a5f7edd9 100644
--- a/tests/qtest/libqtest.h
+++ b/tests/qtest/libqtest.h
@@ -75,6 +75,15 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args);
  */
 QTestState *qtest_init_with_serial(const char *extra_args, int *sock_fd);
 
+/**
+ * qtest_wait_qemu:
+ * @s: #QTestState instance to operate on.
+ *
+ * Wait for the QEMU process to terminate. It is safe to call this function
+ * multiple times.
+ */
+void qtest_wait_qemu(QTestState *s);
+
 /**
  * qtest_kill_qemu:
  * @s: #QTestState instance to operate on.
diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index d12a604d78..e1e2d39a6e 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -156,37 +156,14 @@ void qtest_set_expected_status(QTestState *s, int status)
     s->expected_status = status;
 }
 
-void qtest_kill_qemu(QTestState *s)
+static void qtest_check_status(QTestState *s)
 {
-    pid_t pid = s->qemu_pid;
-#ifndef _WIN32
-    int wstatus;
-#else
-    DWORD ret;
-#endif
-
-    /* Skip wait if qtest_probe_child already reaped.  */
-    if (pid != -1) {
-#ifndef _WIN32
-        kill(pid, SIGTERM);
-        TFR(pid = waitpid(s->qemu_pid, &s->wstatus, 0));
-        assert(pid == s->qemu_pid);
-#else
-        TerminateProcess((HANDLE)pid, s->expected_status);
-        ret = WaitForSingleObject((HANDLE)pid, INFINITE);
-        assert(ret == WAIT_OBJECT_0);
-        GetExitCodeProcess((HANDLE)pid, &s->exit_code);
-        CloseHandle((HANDLE)pid);
-#endif
-        s->qemu_pid = -1;
-    }
-
     /*
      * Check whether qemu exited with expected exit status; anything else is
      * fishy and should be logged with as much detail as possible.
      */
 #ifndef _WIN32
-    wstatus = s->wstatus;
+    int wstatus = s->wstatus;
     if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) != s->expected_status) {
         fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
                 "process but encountered exit status %d (expected %d)\n",
@@ -212,6 +189,42 @@ void qtest_kill_qemu(QTestState *s)
 #endif
 }
 
+void qtest_wait_qemu(QTestState *s)
+{
+#ifndef _WIN32
+    pid_t pid;
+
+    TFR(pid = waitpid(s->qemu_pid, &s->wstatus, 0));
+    assert(pid == s->qemu_pid);
+#else
+    DWORD ret;
+
+    ret = WaitForSingleObject((HANDLE)s->qemu_pid, INFINITE);
+    assert(ret == WAIT_OBJECT_0);
+    GetExitCodeProcess((HANDLE)s->qemu_pid, &s->exit_code);
+    CloseHandle((HANDLE)s->qemu_pid);
+#endif
+
+    qtest_check_status(s);
+}
+
+void qtest_kill_qemu(QTestState *s)
+{
+    /* Skip wait if qtest_probe_child() already reaped */
+    if (s->qemu_pid != -1) {
+#ifndef _WIN32
+        kill(s->qemu_pid, SIGTERM);
+#else
+        TerminateProcess((HANDLE)s->qemu_pid, s->expected_status);
+#endif
+        qtest_wait_qemu(s);
+        s->qemu_pid = -1;
+        return;
+    }
+
+    qtest_check_status(s);
+}
+
 static void kill_qemu_hook_func(void *s)
 {
     qtest_kill_qemu(s);
-- 
2.25.1

[PATCH v6 07/11] tests/qtest: migration-test: Make sure QEMU process "to" exited after migration is canceled

[Bin Meng]

From: Xuzhou Cheng <xuzhou.cheng@windriver.com>

Make sure QEMU process "to" exited before launching another target
for migration in the test_multifd_tcp_cancel case.

Signed-off-by: Xuzhou Cheng <xuzhou.cheng@windriver.com>
Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- change to use qtest_wait_qemu() API

Changes in v3:
- Add a usleep(1) in the busy wait loop

Changes in v2:
- Change to a busy wait after migration is canceled

 tests/qtest/migration-test.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 28a06d8170..d2eb107f0c 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -2141,6 +2141,10 @@ static void test_multifd_tcp_cancel(void)
 
     migrate_cancel(from);
 
+    /* Make sure QEMU process "to" exited */
+    qtest_set_expected_status(to, EXIT_FAILURE);
+    qtest_wait_qemu(to);
+
     args = (MigrateStart){
         .only_target = true,
     };
-- 
2.25.1

[PATCH v6 08/11] tests/qtest: libqos: Do not build virtio-9p unconditionally

[Bin Meng]

At present the virtio-9p related codes are built into libqos
unconditionally. Change to build them conditionally by testing
the 'virtfs' config option.

Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

Changes in v6:
- new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"

 tests/qtest/libqos/meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
index 113c80b4e4..32f028872c 100644
--- a/tests/qtest/libqos/meson.build
+++ b/tests/qtest/libqos/meson.build
@@ -33,8 +33,6 @@ libqos_srcs = files(
         'sdhci.c',
         'tpci200.c',
         'virtio.c',
-        'virtio-9p.c',
-        'virtio-9p-client.c',
         'virtio-balloon.c',
         'virtio-blk.c',
         'vhost-user-blk.c',
@@ -62,6 +60,10 @@ libqos_srcs = files(
         'x86_64_pc-machine.c',
 )
 
+if have_virtfs
+  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
+endif
+
 libqos = static_library('qos', libqos_srcs + genh,
                         name_suffix: 'fa',
                         build_by_default: false)
-- 
2.25.1

[PATCH v6 09/11] tests/qtest: libqtest: Correct the timeout unit of blocking receive calls for win32

[Bin Meng]

Some qtest cases don't get response from the QEMU executable under
test in time on Windows. It turns out that the socket receive call
got timeout before it receive the complete response.

The timeout value is supposed to be set to 50 seconds via the
setsockopt() call, but there is a difference among platforms.
The timeout unit of blocking receive calls is measured in
seconds on non-Windows platforms but milliseconds on Windows.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---

(no changes since v1)

 tests/qtest/libqtest.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index e1e2d39a6e..2fbc3b88f3 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -36,13 +36,14 @@
 #include "qapi/qmp/qstring.h"
 
 #define MAX_IRQ 256
-#define SOCKET_TIMEOUT 50
 
 #ifndef _WIN32
+# define SOCKET_TIMEOUT 50
 # define CMD_EXEC   "exec "
 # define DEV_STDERR "/dev/fd/2"
 # define DEV_NULL   "/dev/null"
 #else
+# define SOCKET_TIMEOUT 50000
 # define CMD_EXEC   ""
 # define DEV_STDERR "2"
 # define DEV_NULL   "nul"
@@ -106,8 +107,16 @@ static int socket_accept(int sock)
     struct sockaddr_un addr;
     socklen_t addrlen;
     int ret;
+    /*
+     * timeout unit of blocking receive calls is different among platfoms.
+     * It's in seconds on non-Windows platforms but milliseconds on Windows.
+     */
+#ifndef _WIN32
     struct timeval timeout = { .tv_sec = SOCKET_TIMEOUT,
                                .tv_usec = 0 };
+#else
+    DWORD timeout = SOCKET_TIMEOUT;
+#endif
 
     if (setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,
                    (void *)&timeout, sizeof(timeout))) {
-- 
2.25.1

[PATCH v6 10/11] .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes

[Bin Meng]

commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices"
changed to compile QEMU with the --without-default-devices switch for
the msys2-64bit job, due to the build could not complete within the
project timeout (1h), and also mentioned that a bigger timeout was
getting ignored on the shared Gitlab-CI Windows runners.

However as of today it seems the shared Gitlab-CI Windows runners does
honor the job timeout, and the runner has the timeout limit of 2h, so
let's increase the timeout to 90 minutes and drop the configure switch
"--without-default-devices" to get a larger build coverage.

Signed-off-by: Bin Meng <bin.meng@windriver.com>

---

(no changes since v2)

Changes in v2:
- Change the timeout limit to 90 minutes

 .gitlab-ci.d/windows.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index a3e7a37022..093276ddbc 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -10,7 +10,7 @@
       - ${CI_PROJECT_DIR}/msys64/var/cache
   needs: []
   stage: build
-  timeout: 70m
+  timeout: 90m
   before_script:
   - If ( !(Test-Path -Path msys64\var\cache ) ) {
       mkdir msys64\var\cache
@@ -60,7 +60,7 @@ msys2-64bit:
   - $env:MSYSTEM = 'MINGW64'     # Start a 64 bit Mingw environment
   - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
-      --enable-capstone --without-default-devices'
+      --enable-capstone'
   - .\msys64\usr\bin\bash -lc 'make'
   - .\msys64\usr\bin\bash -lc 'make check || { cat build/meson-logs/testlog.txt; exit 1; } ;'
 
-- 
2.25.1

[PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Bin Meng]

Now that we have fixed various test case issues as seen when running
on Windows, let's enable the qtest build on Windows.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>

---

Changes in v5:
- Drop patches that are already merged

Changes in v3:
- Drop the host test

Changes in v2:
- new patch: "tests/qtest: Enable qtest build on Windows"

 tests/qtest/meson.build | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index c07a5b1a5f..f0ebb5fac6 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -1,9 +1,3 @@
-# All QTests for now are POSIX-only, but the dependencies are
-# really in libqtest, not in the testcases themselves.
-if not config_host.has_key('CONFIG_POSIX')
-  subdir_done()
-endif
-
 slow_qtests = {
   'ahci-test' : 60,
   'bios-tables-test' : 120,
-- 
2.25.1

Re: [PATCH v6 04/11] tests/qtest: device-plug-test: Reverse the usage of double/single quotes

[Thomas Huth]

On 28/10/2022 06.57, Bin Meng wrote:
> The usage of double/single quotes in test_q35_pci_unplug_json_request()
> should be reversed to work on both win32 and non-win32 platforms:
> 
> - The value of -device parameter needs to be surrounded by "" as
>    Windows does not drop '' when passing it to QEMU which causes
>    QEMU command line option parser failure.
> - The JSON key/value pairs need to be surrounded by '' to make the
>    JSON parser happy on Windows.
> 
> Fixes: a12f1a7e56b7 ("tests/x86: Add subtest with 'q35' machine type to device-plug-test")
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> 
> ---
> 
> Changes in v6:
> - new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"
> 
>   tests/qtest/device-plug-test.c | 16 ++++++++--------
>   1 file changed, 8 insertions(+), 8 deletions(-)
> 
> diff --git a/tests/qtest/device-plug-test.c b/tests/qtest/device-plug-test.c
> index 3f44f731d1..5a6afa2b57 100644
> --- a/tests/qtest/device-plug-test.c
> +++ b/tests/qtest/device-plug-test.c
> @@ -112,16 +112,16 @@ static void test_pci_unplug_json_request(void)
>   
>   static void test_q35_pci_unplug_json_request(void)
>   {
> -    const char *port = "-device '{\"driver\": \"pcie-root-port\", "
> -                                      "\"id\": \"p1\"}'";
> +    const char *port = "-device \"{'driver': 'pcie-root-port', "
> +                                  "'id': 'p1'}\"";
>   
> -    const char *bridge = "-device '{\"driver\": \"pcie-pci-bridge\", "
> -                                   "\"id\": \"b1\", "
> -                                   "\"bus\": \"p1\"}'";
> +    const char *bridge = "-device \"{'driver': 'pcie-pci-bridge', "
> +                                    "'id': 'b1', "
> +                                    "'bus': 'p1'}\"";
>   
> -    const char *device = "-device '{\"driver\": \"virtio-mouse-pci\", "
> -                                   "\"bus\": \"b1\", "
> -                                   "\"id\": \"dev0\"}'";
> +    const char *device = "-device \"{'driver': 'virtio-mouse-pci', "
> +                                    "'bus': 'b1', "
> +                                    "'id': 'dev0'}\"";
>   
>       QTestState *qtest = qtest_initf("-machine q35 %s %s %s",
>                                       port, bridge, device);

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v6 08/11] tests/qtest: libqos: Do not build virtio-9p unconditionally

[Thomas Huth]

On 28/10/2022 06.57, Bin Meng wrote:
> At present the virtio-9p related codes are built into libqos
> unconditionally. Change to build them conditionally by testing
> the 'virtfs' config option.
> 
> Signed-off-by: Bin Meng <bin.meng@windriver.com>

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v6 05/11] tests/qtest: Use EXIT_FAILURE instead of magic number

[Thomas Huth]

On 28/10/2022 06.57, Bin Meng wrote:
> When migration fails, QEMU exits with a status code EXIT_FAILURE.
> Change qtests to use the well-defined macro instead of magic number.
> 
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> 
> ---
> 
> Changes in v6:
> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
> 
>   tests/qtest/dbus-vmstate-test.c | 2 +-
>   tests/qtest/migration-test.c    | 4 ++--
>   2 files changed, 3 insertions(+), 3 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v6 06/11] tests/qtest: libqtest: Introduce qtest_wait_qemu()

[Thomas Huth]

On 28/10/2022 06.57, Bin Meng wrote:
> Introduce an API for qtest to wait for the QEMU process to terminate.
> 
> Suggested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> 
> ---
> 
> Changes in v6:
> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
> 
>   tests/qtest/libqtest.h |  9 ++++++
>   tests/qtest/libqtest.c | 63 +++++++++++++++++++++++++-----------------
>   2 files changed, 47 insertions(+), 25 deletions(-)
> 
> diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
> index 65c040e504..91a5f7edd9 100644
> --- a/tests/qtest/libqtest.h
> +++ b/tests/qtest/libqtest.h
> @@ -75,6 +75,15 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args);
>    */
>   QTestState *qtest_init_with_serial(const char *extra_args, int *sock_fd);
>   
> +/**
> + * qtest_wait_qemu:
> + * @s: #QTestState instance to operate on.
> + *
> + * Wait for the QEMU process to terminate. It is safe to call this function
> + * multiple times.
> + */
> +void qtest_wait_qemu(QTestState *s);
> +
>   /**
>    * qtest_kill_qemu:
>    * @s: #QTestState instance to operate on.
> diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
> index d12a604d78..e1e2d39a6e 100644
> --- a/tests/qtest/libqtest.c
> +++ b/tests/qtest/libqtest.c
> @@ -156,37 +156,14 @@ void qtest_set_expected_status(QTestState *s, int status)
>       s->expected_status = status;
>   }
>   
> -void qtest_kill_qemu(QTestState *s)
> +static void qtest_check_status(QTestState *s)
>   {
> -    pid_t pid = s->qemu_pid;
> -#ifndef _WIN32
> -    int wstatus;
> -#else
> -    DWORD ret;
> -#endif
> -
> -    /* Skip wait if qtest_probe_child already reaped.  */
> -    if (pid != -1) {
> -#ifndef _WIN32
> -        kill(pid, SIGTERM);
> -        TFR(pid = waitpid(s->qemu_pid, &s->wstatus, 0));
> -        assert(pid == s->qemu_pid);
> -#else
> -        TerminateProcess((HANDLE)pid, s->expected_status);
> -        ret = WaitForSingleObject((HANDLE)pid, INFINITE);
> -        assert(ret == WAIT_OBJECT_0);
> -        GetExitCodeProcess((HANDLE)pid, &s->exit_code);
> -        CloseHandle((HANDLE)pid);
> -#endif
> -        s->qemu_pid = -1;
> -    }
> -
>       /*
>        * Check whether qemu exited with expected exit status; anything else is
>        * fishy and should be logged with as much detail as possible.
>        */
>   #ifndef _WIN32
> -    wstatus = s->wstatus;
> +    int wstatus = s->wstatus;
>       if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) != s->expected_status) {
>           fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
>                   "process but encountered exit status %d (expected %d)\n",
> @@ -212,6 +189,42 @@ void qtest_kill_qemu(QTestState *s)
>   #endif
>   }
>   
> +void qtest_wait_qemu(QTestState *s)
> +{
> +#ifndef _WIN32
> +    pid_t pid;

Should we have a check for  s->qemu_pid != -1 here ?

> +    TFR(pid = waitpid(s->qemu_pid, &s->wstatus, 0));
> +    assert(pid == s->qemu_pid);
> +#else
> +    DWORD ret;
> +
> +    ret = WaitForSingleObject((HANDLE)s->qemu_pid, INFINITE);
> +    assert(ret == WAIT_OBJECT_0);
> +    GetExitCodeProcess((HANDLE)s->qemu_pid, &s->exit_code);
> +    CloseHandle((HANDLE)s->qemu_pid);
> +#endif
> +
> +    qtest_check_status(s);
> +}
> +
> +void qtest_kill_qemu(QTestState *s)
> +{
> +    /* Skip wait if qtest_probe_child() already reaped */
> +    if (s->qemu_pid != -1) {
> +#ifndef _WIN32
> +        kill(s->qemu_pid, SIGTERM);
> +#else
> +        TerminateProcess((HANDLE)s->qemu_pid, s->expected_status);
> +#endif
> +        qtest_wait_qemu(s);
> +        s->qemu_pid = -1;
> +        return;
> +    }
> +
> +    qtest_check_status(s);
> +}
> +
>   static void kill_qemu_hook_func(void *s)
>   {
>       qtest_kill_qemu(s);

  Thomas

Re: [PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 3698 bytes --]

Hi

On Fri, Oct 28, 2022 at 8:58 AM Bin Meng <bin.meng@windriver.com> wrote:

> In preparation to adding virtio-9p support on Windows, this series
> enables running qtest on Windows, so that we can run the virtio-9p
> tests on Windows to make sure it does not break accidently.
>
> Changes in v6:
> - drop patches that are already in Alex and Daniel's tree
> - remove CONFIG_POSIX from meson.build
> - include <qemu/sockets.h> in libqtest.c
> - move documentation comments of qemu_send_full() from util/osdep.c
>   to qemu/sockets.h
> - save the "exit_code" in struct QTestState
> - new patch: "tests/qtest: device-plug-test: Reverse the usage of
> double/single quotes"
> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
> - change to use qtest_wait_qemu() API
> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p
> unconditionally"
>
> Changes in v5:
> - restore to v1 version which does not touch the posix implementation
> - Drop patches that are already merged
>
> Changes in v3:
> - Add a usleep(1) in the busy wait loop
> - Drop the host test
>
> Changes in v2:
> - Introduce qemu_send_full() and use it
> - Move the enabling of building qtests on Windows to a separate
>   patch to keep bisectablity
> - Call socket_init() unconditionally
> - Add a missing CloseHandle() call
> - Change to a busy wait after migration is canceled
> - Change the timeout limit to 90 minutes
> - new patch: "tests/qtest: Enable qtest build on Windows"
>
> Bin Meng (8):
>   tests/qtest: Support libqtest to build and run on Windows
>   tests/qtest: device-plug-test: Reverse the usage of double/single
>     quotes
>   tests/qtest: Use EXIT_FAILURE instead of magic number
>   tests/qtest: libqtest: Introduce qtest_wait_qemu()
>   tests/qtest: libqos: Do not build virtio-9p unconditionally
>   tests/qtest: libqtest: Correct the timeout unit of blocking receive
>     calls for win32
>   .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
>   tests/qtest: Enable qtest build on Windows
>
> Xuzhou Cheng (3):
>   accel/qtest: Support qtest accelerator for Windows
>   tests/qtest: Use send/recv for socket communication
>   tests/qtest: migration-test: Make sure QEMU process "to" exited after
>     migration is canceled
>
>  include/hw/core/cpu.h           |   1 +
>  include/qemu/sockets.h          |  13 +++
>  tests/qtest/libqtest.h          |   9 ++
>  accel/dummy-cpus.c              |  14 ++-
>  softmmu/cpus.c                  |   9 +-
>  tests/qtest/dbus-vmstate-test.c |   2 +-
>  tests/qtest/device-plug-test.c  |  16 ++--
>  tests/qtest/libqmp.c            |   5 +-
>  tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
>  tests/qtest/migration-test.c    |   8 +-
>  util/osdep.c                    |  22 +++++
>  .gitlab-ci.d/windows.yml        |   4 +-
>  accel/meson.build               |   2 +-
>  accel/qtest/meson.build         |   3 +-
>  tests/qtest/libqos/meson.build  |   6 +-
>  tests/qtest/meson.build         |   6 --
>  16 files changed, 221 insertions(+), 50 deletions(-)
>
> --
> 2.25.1
>
>
Series looks good to me:
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>


However, many qtests are flaky on Windows. I get a number of "broken pipe"
exit code 3 & timeout. Should gitlab ignore windows test failures ? Or
perhaps have a new "ignored" job for the windows qtests. What's your
experience running gitlab CI with this series? Can you share results? (I
kicked off one here
https://gitlab.com/marcandre.lureau/qemu/-/pipelines/679511572)

[-- Attachment #2: Type: text/html, Size: 4607 bytes --]

Re: [PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Bin Meng]

On Fri, Oct 28, 2022 at 4:09 PM Marc-André Lureau
<marcandre.lureau@redhat.com> wrote:
>
> Hi
>
> On Fri, Oct 28, 2022 at 8:58 AM Bin Meng <bin.meng@windriver.com> wrote:
>>
>> In preparation to adding virtio-9p support on Windows, this series
>> enables running qtest on Windows, so that we can run the virtio-9p
>> tests on Windows to make sure it does not break accidently.
>>
>> Changes in v6:
>> - drop patches that are already in Alex and Daniel's tree
>> - remove CONFIG_POSIX from meson.build
>> - include <qemu/sockets.h> in libqtest.c
>> - move documentation comments of qemu_send_full() from util/osdep.c
>>   to qemu/sockets.h
>> - save the "exit_code" in struct QTestState
>> - new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"
>> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
>> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
>> - change to use qtest_wait_qemu() API
>> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
>>
>> Changes in v5:
>> - restore to v1 version which does not touch the posix implementation
>> - Drop patches that are already merged
>>
>> Changes in v3:
>> - Add a usleep(1) in the busy wait loop
>> - Drop the host test
>>
>> Changes in v2:
>> - Introduce qemu_send_full() and use it
>> - Move the enabling of building qtests on Windows to a separate
>>   patch to keep bisectablity
>> - Call socket_init() unconditionally
>> - Add a missing CloseHandle() call
>> - Change to a busy wait after migration is canceled
>> - Change the timeout limit to 90 minutes
>> - new patch: "tests/qtest: Enable qtest build on Windows"
>>
>> Bin Meng (8):
>>   tests/qtest: Support libqtest to build and run on Windows
>>   tests/qtest: device-plug-test: Reverse the usage of double/single
>>     quotes
>>   tests/qtest: Use EXIT_FAILURE instead of magic number
>>   tests/qtest: libqtest: Introduce qtest_wait_qemu()
>>   tests/qtest: libqos: Do not build virtio-9p unconditionally
>>   tests/qtest: libqtest: Correct the timeout unit of blocking receive
>>     calls for win32
>>   .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
>>   tests/qtest: Enable qtest build on Windows
>>
>> Xuzhou Cheng (3):
>>   accel/qtest: Support qtest accelerator for Windows
>>   tests/qtest: Use send/recv for socket communication
>>   tests/qtest: migration-test: Make sure QEMU process "to" exited after
>>     migration is canceled
>>
>>  include/hw/core/cpu.h           |   1 +
>>  include/qemu/sockets.h          |  13 +++
>>  tests/qtest/libqtest.h          |   9 ++
>>  accel/dummy-cpus.c              |  14 ++-
>>  softmmu/cpus.c                  |   9 +-
>>  tests/qtest/dbus-vmstate-test.c |   2 +-
>>  tests/qtest/device-plug-test.c  |  16 ++--
>>  tests/qtest/libqmp.c            |   5 +-
>>  tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
>>  tests/qtest/migration-test.c    |   8 +-
>>  util/osdep.c                    |  22 +++++
>>  .gitlab-ci.d/windows.yml        |   4 +-
>>  accel/meson.build               |   2 +-
>>  accel/qtest/meson.build         |   3 +-
>>  tests/qtest/libqos/meson.build  |   6 +-
>>  tests/qtest/meson.build         |   6 --
>>  16 files changed, 221 insertions(+), 50 deletions(-)
>>
>> --
>> 2.25.1
>>
>
> Series looks good to me:
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>
>
> However, many qtests are flaky on Windows. I get a number of "broken pipe" exit code 3 & timeout. Should gitlab ignore windows test failures ? Or perhaps have a new "ignored" job for the windows qtests. What's your experience running gitlab CI with this series? Can you share results? (I kicked off one here https://gitlab.com/marcandre.lureau/qemu/-/pipelines/679511572)
>

This "broken pipe" error was fixed by [1] which is currently in
Daniel's tree. Please apply it in your tree and it should have a 100%
pass rate.

[1] http://patchwork.ozlabs.org/project/qemu-devel/patch/20221006151927.2079583-17-bmeng.cn@gmail.com/

Regards,
Bin

Re: [PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 4698 bytes --]

Hi

On Fri, Oct 28, 2022 at 1:21 PM Bin Meng <bmeng.cn@gmail.com> wrote:

> On Fri, Oct 28, 2022 at 4:09 PM Marc-André Lureau
> <marcandre.lureau@redhat.com> wrote:
> >
> > Hi
> >
> > On Fri, Oct 28, 2022 at 8:58 AM Bin Meng <bin.meng@windriver.com> wrote:
> >>
> >> In preparation to adding virtio-9p support on Windows, this series
> >> enables running qtest on Windows, so that we can run the virtio-9p
> >> tests on Windows to make sure it does not break accidently.
> >>
> >> Changes in v6:
> >> - drop patches that are already in Alex and Daniel's tree
> >> - remove CONFIG_POSIX from meson.build
> >> - include <qemu/sockets.h> in libqtest.c
> >> - move documentation comments of qemu_send_full() from util/osdep.c
> >>   to qemu/sockets.h
> >> - save the "exit_code" in struct QTestState
> >> - new patch: "tests/qtest: device-plug-test: Reverse the usage of
> double/single quotes"
> >> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
> >> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
> >> - change to use qtest_wait_qemu() API
> >> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p
> unconditionally"
> >>
> >> Changes in v5:
> >> - restore to v1 version which does not touch the posix implementation
> >> - Drop patches that are already merged
> >>
> >> Changes in v3:
> >> - Add a usleep(1) in the busy wait loop
> >> - Drop the host test
> >>
> >> Changes in v2:
> >> - Introduce qemu_send_full() and use it
> >> - Move the enabling of building qtests on Windows to a separate
> >>   patch to keep bisectablity
> >> - Call socket_init() unconditionally
> >> - Add a missing CloseHandle() call
> >> - Change to a busy wait after migration is canceled
> >> - Change the timeout limit to 90 minutes
> >> - new patch: "tests/qtest: Enable qtest build on Windows"
> >>
> >> Bin Meng (8):
> >>   tests/qtest: Support libqtest to build and run on Windows
> >>   tests/qtest: device-plug-test: Reverse the usage of double/single
> >>     quotes
> >>   tests/qtest: Use EXIT_FAILURE instead of magic number
> >>   tests/qtest: libqtest: Introduce qtest_wait_qemu()
> >>   tests/qtest: libqos: Do not build virtio-9p unconditionally
> >>   tests/qtest: libqtest: Correct the timeout unit of blocking receive
> >>     calls for win32
> >>   .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
> >>   tests/qtest: Enable qtest build on Windows
> >>
> >> Xuzhou Cheng (3):
> >>   accel/qtest: Support qtest accelerator for Windows
> >>   tests/qtest: Use send/recv for socket communication
> >>   tests/qtest: migration-test: Make sure QEMU process "to" exited after
> >>     migration is canceled
> >>
> >>  include/hw/core/cpu.h           |   1 +
> >>  include/qemu/sockets.h          |  13 +++
> >>  tests/qtest/libqtest.h          |   9 ++
> >>  accel/dummy-cpus.c              |  14 ++-
> >>  softmmu/cpus.c                  |   9 +-
> >>  tests/qtest/dbus-vmstate-test.c |   2 +-
> >>  tests/qtest/device-plug-test.c  |  16 ++--
> >>  tests/qtest/libqmp.c            |   5 +-
> >>  tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
> >>  tests/qtest/migration-test.c    |   8 +-
> >>  util/osdep.c                    |  22 +++++
> >>  .gitlab-ci.d/windows.yml        |   4 +-
> >>  accel/meson.build               |   2 +-
> >>  accel/qtest/meson.build         |   3 +-
> >>  tests/qtest/libqos/meson.build  |   6 +-
> >>  tests/qtest/meson.build         |   6 --
> >>  16 files changed, 221 insertions(+), 50 deletions(-)
> >>
> >> --
> >> 2.25.1
> >>
> >
> > Series looks good to me:
> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> >
> >
> > However, many qtests are flaky on Windows. I get a number of "broken
> pipe" exit code 3 & timeout. Should gitlab ignore windows test failures ?
> Or perhaps have a new "ignored" job for the windows qtests. What's your
> experience running gitlab CI with this series? Can you share results? (I
> kicked off one here
> https://gitlab.com/marcandre.lureau/qemu/-/pipelines/679511572)
> >
>
> This "broken pipe" error was fixed by [1] which is currently in
> Daniel's tree. Please apply it in your tree and it should have a 100%
> pass rate.
>
> [1]
> http://patchwork.ozlabs.org/project/qemu-devel/patch/20221006151927.2079583-17-bmeng.cn@gmail.com/
>
>
Ok I have seen other tests randomly failing. Furthermore:
https://gitlab.com/marcandre.lureau/qemu/-/jobs/3241465230
ERROR: Job failed: execution took longer than 1h30m0s seconds


I think we should drop the last 2 patches for now, until CI testing is
under control...

[-- Attachment #2: Type: text/html, Size: 6575 bytes --]

Re: [PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Bin Meng]

On Fri, Oct 28, 2022 at 5:41 PM Marc-André Lureau
<marcandre.lureau@redhat.com> wrote:
>
> Hi
>
> On Fri, Oct 28, 2022 at 1:21 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>>
>> On Fri, Oct 28, 2022 at 4:09 PM Marc-André Lureau
>> <marcandre.lureau@redhat.com> wrote:
>> >
>> > Hi
>> >
>> > On Fri, Oct 28, 2022 at 8:58 AM Bin Meng <bin.meng@windriver.com> wrote:
>> >>
>> >> In preparation to adding virtio-9p support on Windows, this series
>> >> enables running qtest on Windows, so that we can run the virtio-9p
>> >> tests on Windows to make sure it does not break accidently.
>> >>
>> >> Changes in v6:
>> >> - drop patches that are already in Alex and Daniel's tree
>> >> - remove CONFIG_POSIX from meson.build
>> >> - include <qemu/sockets.h> in libqtest.c
>> >> - move documentation comments of qemu_send_full() from util/osdep.c
>> >>   to qemu/sockets.h
>> >> - save the "exit_code" in struct QTestState
>> >> - new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"
>> >> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
>> >> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
>> >> - change to use qtest_wait_qemu() API
>> >> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
>> >>
>> >> Changes in v5:
>> >> - restore to v1 version which does not touch the posix implementation
>> >> - Drop patches that are already merged
>> >>
>> >> Changes in v3:
>> >> - Add a usleep(1) in the busy wait loop
>> >> - Drop the host test
>> >>
>> >> Changes in v2:
>> >> - Introduce qemu_send_full() and use it
>> >> - Move the enabling of building qtests on Windows to a separate
>> >>   patch to keep bisectablity
>> >> - Call socket_init() unconditionally
>> >> - Add a missing CloseHandle() call
>> >> - Change to a busy wait after migration is canceled
>> >> - Change the timeout limit to 90 minutes
>> >> - new patch: "tests/qtest: Enable qtest build on Windows"
>> >>
>> >> Bin Meng (8):
>> >>   tests/qtest: Support libqtest to build and run on Windows
>> >>   tests/qtest: device-plug-test: Reverse the usage of double/single
>> >>     quotes
>> >>   tests/qtest: Use EXIT_FAILURE instead of magic number
>> >>   tests/qtest: libqtest: Introduce qtest_wait_qemu()
>> >>   tests/qtest: libqos: Do not build virtio-9p unconditionally
>> >>   tests/qtest: libqtest: Correct the timeout unit of blocking receive
>> >>     calls for win32
>> >>   .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
>> >>   tests/qtest: Enable qtest build on Windows
>> >>
>> >> Xuzhou Cheng (3):
>> >>   accel/qtest: Support qtest accelerator for Windows
>> >>   tests/qtest: Use send/recv for socket communication
>> >>   tests/qtest: migration-test: Make sure QEMU process "to" exited after
>> >>     migration is canceled
>> >>
>> >>  include/hw/core/cpu.h           |   1 +
>> >>  include/qemu/sockets.h          |  13 +++
>> >>  tests/qtest/libqtest.h          |   9 ++
>> >>  accel/dummy-cpus.c              |  14 ++-
>> >>  softmmu/cpus.c                  |   9 +-
>> >>  tests/qtest/dbus-vmstate-test.c |   2 +-
>> >>  tests/qtest/device-plug-test.c  |  16 ++--
>> >>  tests/qtest/libqmp.c            |   5 +-
>> >>  tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
>> >>  tests/qtest/migration-test.c    |   8 +-
>> >>  util/osdep.c                    |  22 +++++
>> >>  .gitlab-ci.d/windows.yml        |   4 +-
>> >>  accel/meson.build               |   2 +-
>> >>  accel/qtest/meson.build         |   3 +-
>> >>  tests/qtest/libqos/meson.build  |   6 +-
>> >>  tests/qtest/meson.build         |   6 --
>> >>  16 files changed, 221 insertions(+), 50 deletions(-)
>> >>
>> >> --
>> >> 2.25.1
>> >>
>> >
>> > Series looks good to me:
>> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >
>> >
>> > However, many qtests are flaky on Windows. I get a number of "broken pipe" exit code 3 & timeout. Should gitlab ignore windows test failures ? Or perhaps have a new "ignored" job for the windows qtests. What's your experience running gitlab CI with this series? Can you share results? (I kicked off one here https://gitlab.com/marcandre.lureau/qemu/-/pipelines/679511572)
>> >
>>
>> This "broken pipe" error was fixed by [1] which is currently in
>> Daniel's tree. Please apply it in your tree and it should have a 100%
>> pass rate.
>>
>> [1] http://patchwork.ozlabs.org/project/qemu-devel/patch/20221006151927.2079583-17-bmeng.cn@gmail.com/
>>
>
> Ok I have seen other tests randomly failing. Furthermore:
> https://gitlab.com/marcandre.lureau/qemu/-/jobs/3241465230
> ERROR: Job failed: execution took longer than 1h30m0s seconds
>
>
> I think we should drop the last 2 patches for now, until CI testing is under control...

2 hours is the maximum time supported by the gitlab shared runners
which should be enough.

However people may feel that it takes too long ...

Regards,
Bin

Re: [PATCH v6 00/11] tests/qtest: Enable running qtest on Windows

[Thomas Huth]

On 28/10/2022 11.43, Bin Meng wrote:
> On Fri, Oct 28, 2022 at 5:41 PM Marc-André Lureau
> <marcandre.lureau@redhat.com> wrote:
>>
>> Hi
>>
>> On Fri, Oct 28, 2022 at 1:21 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>>>
>>> On Fri, Oct 28, 2022 at 4:09 PM Marc-André Lureau
>>> <marcandre.lureau@redhat.com> wrote:
>>>>
>>>> Hi
>>>>
>>>> On Fri, Oct 28, 2022 at 8:58 AM Bin Meng <bin.meng@windriver.com> wrote:
>>>>>
>>>>> In preparation to adding virtio-9p support on Windows, this series
>>>>> enables running qtest on Windows, so that we can run the virtio-9p
>>>>> tests on Windows to make sure it does not break accidently.
>>>>>
>>>>> Changes in v6:
>>>>> - drop patches that are already in Alex and Daniel's tree
>>>>> - remove CONFIG_POSIX from meson.build
>>>>> - include <qemu/sockets.h> in libqtest.c
>>>>> - move documentation comments of qemu_send_full() from util/osdep.c
>>>>>    to qemu/sockets.h
>>>>> - save the "exit_code" in struct QTestState
>>>>> - new patch: "tests/qtest: device-plug-test: Reverse the usage of double/single quotes"
>>>>> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
>>>>> - new patch: "tests/qtest: libqtest: Introduce qtest_wait_qemu()"
>>>>> - change to use qtest_wait_qemu() API
>>>>> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
>>>>>
>>>>> Changes in v5:
>>>>> - restore to v1 version which does not touch the posix implementation
>>>>> - Drop patches that are already merged
>>>>>
>>>>> Changes in v3:
>>>>> - Add a usleep(1) in the busy wait loop
>>>>> - Drop the host test
>>>>>
>>>>> Changes in v2:
>>>>> - Introduce qemu_send_full() and use it
>>>>> - Move the enabling of building qtests on Windows to a separate
>>>>>    patch to keep bisectablity
>>>>> - Call socket_init() unconditionally
>>>>> - Add a missing CloseHandle() call
>>>>> - Change to a busy wait after migration is canceled
>>>>> - Change the timeout limit to 90 minutes
>>>>> - new patch: "tests/qtest: Enable qtest build on Windows"
>>>>>
>>>>> Bin Meng (8):
>>>>>    tests/qtest: Support libqtest to build and run on Windows
>>>>>    tests/qtest: device-plug-test: Reverse the usage of double/single
>>>>>      quotes
>>>>>    tests/qtest: Use EXIT_FAILURE instead of magic number
>>>>>    tests/qtest: libqtest: Introduce qtest_wait_qemu()
>>>>>    tests/qtest: libqos: Do not build virtio-9p unconditionally
>>>>>    tests/qtest: libqtest: Correct the timeout unit of blocking receive
>>>>>      calls for win32
>>>>>    .gitlab-ci.d/windows.yml: Increase the timeout to 90 minutes
>>>>>    tests/qtest: Enable qtest build on Windows
>>>>>
>>>>> Xuzhou Cheng (3):
>>>>>    accel/qtest: Support qtest accelerator for Windows
>>>>>    tests/qtest: Use send/recv for socket communication
>>>>>    tests/qtest: migration-test: Make sure QEMU process "to" exited after
>>>>>      migration is canceled
>>>>>
>>>>>   include/hw/core/cpu.h           |   1 +
>>>>>   include/qemu/sockets.h          |  13 +++
>>>>>   tests/qtest/libqtest.h          |   9 ++
>>>>>   accel/dummy-cpus.c              |  14 ++-
>>>>>   softmmu/cpus.c                  |   9 +-
>>>>>   tests/qtest/dbus-vmstate-test.c |   2 +-
>>>>>   tests/qtest/device-plug-test.c  |  16 ++--
>>>>>   tests/qtest/libqmp.c            |   5 +-
>>>>>   tests/qtest/libqtest.c          | 151 ++++++++++++++++++++++++++++----
>>>>>   tests/qtest/migration-test.c    |   8 +-
>>>>>   util/osdep.c                    |  22 +++++
>>>>>   .gitlab-ci.d/windows.yml        |   4 +-
>>>>>   accel/meson.build               |   2 +-
>>>>>   accel/qtest/meson.build         |   3 +-
>>>>>   tests/qtest/libqos/meson.build  |   6 +-
>>>>>   tests/qtest/meson.build         |   6 --
>>>>>   16 files changed, 221 insertions(+), 50 deletions(-)
>>>>>
>>>>> --
>>>>> 2.25.1
>>>>>
>>>>
>>>> Series looks good to me:
>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>>
>>>>
>>>> However, many qtests are flaky on Windows. I get a number of "broken pipe" exit code 3 & timeout. Should gitlab ignore windows test failures ? Or perhaps have a new "ignored" job for the windows qtests. What's your experience running gitlab CI with this series? Can you share results? (I kicked off one here https://gitlab.com/marcandre.lureau/qemu/-/pipelines/679511572)
>>>>
>>>
>>> This "broken pipe" error was fixed by [1] which is currently in
>>> Daniel's tree. Please apply it in your tree and it should have a 100%
>>> pass rate.
>>>
>>> [1] http://patchwork.ozlabs.org/project/qemu-devel/patch/20221006151927.2079583-17-bmeng.cn@gmail.com/
>>>
>>
>> Ok I have seen other tests randomly failing. Furthermore:
>> https://gitlab.com/marcandre.lureau/qemu/-/jobs/3241465230
>> ERROR: Job failed: execution took longer than 1h30m0s seconds
>>
>>
>> I think we should drop the last 2 patches for now, until CI testing is under control...
> 
> 2 hours is the maximum time supported by the gitlab shared runners
> which should be enough.
> 
> However people may feel that it takes too long ...

Most jobs run fine within the default timeout of 60 minutes, we've got some 
few exceptions where we bumped it to 70, 80 or even 90 minutes. But I think 
that should really be the uppermost limit - using even more than 90 minutes 
for the slow-running Windows job just feels wrong.

Aren't there other possibilities to cut the run time again? Maybe compile 
with --disable-tools? ... searching for "mingw compilation slow" in the 
internet also reveals that a lot of other people are seeing problems with 
the compilation speed of MinGW ... but there does not seem to be one 
ultimate solution that worked for everybody to speed it up...

I'll queue patches 1-9 for now so that we get the important parts in before 
the soft freeze starts ... if we finally figure out how to deal with the 
slow CI jobs best, I think it's also still OK if we enable it later before 
the hard freeze starts.

  Thomas

Re: [PATCH v6 05/11] tests/qtest: Use EXIT_FAILURE instead of magic number

[Juan Quintela]

Bin Meng <bin.meng@windriver.com> wrote:
> When migration fails, QEMU exits with a status code EXIT_FAILURE.
> Change qtests to use the well-defined macro instead of magic number.
>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>


>
> ---
>
> Changes in v6:
> - new patch: "tests/qtest: Use EXIT_FAILURE instead of magic number"
>
>  tests/qtest/dbus-vmstate-test.c | 2 +-
>  tests/qtest/migration-test.c    | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/tests/qtest/dbus-vmstate-test.c b/tests/qtest/dbus-vmstate-test.c
> index 74ede651f6..6c990864e3 100644
> --- a/tests/qtest/dbus-vmstate-test.c
> +++ b/tests/qtest/dbus-vmstate-test.c
> @@ -233,7 +233,7 @@ test_dbus_vmstate(Test *test)
>      test->src_qemu = src_qemu;
>      if (test->migrate_fail) {
>          wait_for_migration_fail(src_qemu, true);
> -        qtest_set_expected_status(dst_qemu, 1);
> +        qtest_set_expected_status(dst_qemu, EXIT_FAILURE);
>      } else {
>          wait_for_migration_complete(src_qemu);
>      }
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index aa1ba179fa..28a06d8170 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -1342,7 +1342,7 @@ static void test_precopy_common(MigrateCommon *args)
>          wait_for_migration_fail(from, allow_active);
>  
>          if (args->result == MIG_TEST_FAIL_DEST_QUIT_ERR) {
> -            qtest_set_expected_status(to, 1);
> +            qtest_set_expected_status(to, EXIT_FAILURE);
>          }
>      } else {
>          if (args->iterations) {
> @@ -1738,7 +1738,7 @@ static void do_test_validate_uuid(MigrateStart *args, bool should_fail)
>      migrate_qmp(from, uri, "{}");
>  
>      if (should_fail) {
> -        qtest_set_expected_status(to, 1);
> +        qtest_set_expected_status(to, EXIT_FAILURE);
>          wait_for_migration_fail(from, true);
>      } else {
>          wait_for_migration_complete(from);

Re: [PATCH v6 08/11] tests/qtest: libqos: Do not build virtio-9p unconditionally

[Christian Schoenebeck]

On Friday, October 28, 2022 6:57:33 AM CEST Bin Meng wrote:
> At present the virtio-9p related codes are built into libqos
> unconditionally. Change to build them conditionally by testing
> the 'virtfs' config option.
> 
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> 
> ---
> 
> Changes in v6:
> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
> 
>  tests/qtest/libqos/meson.build | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
> index 113c80b4e4..32f028872c 100644
> --- a/tests/qtest/libqos/meson.build
> +++ b/tests/qtest/libqos/meson.build
> @@ -33,8 +33,6 @@ libqos_srcs = files(
>          'sdhci.c',
>          'tpci200.c',
>          'virtio.c',
> -        'virtio-9p.c',
> -        'virtio-9p-client.c',
>          'virtio-balloon.c',
>          'virtio-blk.c',
>          'vhost-user-blk.c',
> @@ -62,6 +60,10 @@ libqos_srcs = files(
>          'x86_64_pc-machine.c',
>  )
>  
> +if have_virtfs
> +  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> +endif
> +
>  libqos = static_library('qos', libqos_srcs + genh,
>                          name_suffix: 'fa',
>                          build_by_default: false)
> 

I wondered why this change would no longer execute the 9p tests here.
Apparently because it changes the order of tests being executed, i.e. 9p tests
would then be scheduled after:

  # Start of vhost-user-blk-pci tests
  # Start of vhost-user-blk-pci-tests tests
  Environment variable QTEST_QEMU_STORAGE_DAEMON_BINARY required
  [EXIT]

and I never cared about QEMU storage binary. Can we make a hack like the
following to not change the order of the tests?

diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
index 32f028872c..389bca9804 100644
--- a/tests/qtest/libqos/meson.build
+++ b/tests/qtest/libqos/meson.build
@@ -1,7 +1,13 @@
 libqos_srcs = files(
         '../libqtest.c',
         '../libqmp.c',
+)
 
+if have_virtfs
+  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
+endif
+
+libqos_srcs += files(
         'qgraph.c',
         'qos_external.c',
         'pci.c',
@@ -60,10 +66,6 @@ libqos_srcs = files(
         'x86_64_pc-machine.c',
 )
 
-if have_virtfs
-  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
-endif
-
 libqos = static_library('qos', libqos_srcs + genh,
                         name_suffix: 'fa',
                         build_by_default: false)

Too ugly?

Best regards,
Christian Schoenebeck

Re: [PATCH v6 08/11] tests/qtest: libqos: Do not build virtio-9p unconditionally

[Thomas Huth]

On 28/10/2022 14.59, Christian Schoenebeck wrote:
> On Friday, October 28, 2022 6:57:33 AM CEST Bin Meng wrote:
>> At present the virtio-9p related codes are built into libqos
>> unconditionally. Change to build them conditionally by testing
>> the 'virtfs' config option.
>>
>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
>>
>> ---
>>
>> Changes in v6:
>> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
>>
>>   tests/qtest/libqos/meson.build | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
>> index 113c80b4e4..32f028872c 100644
>> --- a/tests/qtest/libqos/meson.build
>> +++ b/tests/qtest/libqos/meson.build
>> @@ -33,8 +33,6 @@ libqos_srcs = files(
>>           'sdhci.c',
>>           'tpci200.c',
>>           'virtio.c',
>> -        'virtio-9p.c',
>> -        'virtio-9p-client.c',
>>           'virtio-balloon.c',
>>           'virtio-blk.c',
>>           'vhost-user-blk.c',
>> @@ -62,6 +60,10 @@ libqos_srcs = files(
>>           'x86_64_pc-machine.c',
>>   )
>>   
>> +if have_virtfs
>> +  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
>> +endif
>> +
>>   libqos = static_library('qos', libqos_srcs + genh,
>>                           name_suffix: 'fa',
>>                           build_by_default: false)
>>
> 
> I wondered why this change would no longer execute the 9p tests here.
> Apparently because it changes the order of tests being executed, i.e. 9p tests
> would then be scheduled after:
> 
>    # Start of vhost-user-blk-pci tests
>    # Start of vhost-user-blk-pci-tests tests
>    Environment variable QTEST_QEMU_STORAGE_DAEMON_BINARY required
>    [EXIT]
> 
> and I never cared about QEMU storage binary. Can we make a hack like the
> following to not change the order of the tests?
> 
> diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
> index 32f028872c..389bca9804 100644
> --- a/tests/qtest/libqos/meson.build
> +++ b/tests/qtest/libqos/meson.build
> @@ -1,7 +1,13 @@
>   libqos_srcs = files(
>           '../libqtest.c',
>           '../libqmp.c',
> +)
>   
> +if have_virtfs
> +  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> +endif
> +
> +libqos_srcs += files(
>           'qgraph.c',
>           'qos_external.c',
>           'pci.c',
> @@ -60,10 +66,6 @@ libqos_srcs = files(
>           'x86_64_pc-machine.c',
>   )
>   
> -if have_virtfs
> -  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> -endif
> -
>   libqos = static_library('qos', libqos_srcs + genh,
>                           name_suffix: 'fa',
>                           build_by_default: false)
> 
> Too ugly?

Looks a little bit ugly, indeed. What about marking the vhost-user-blk-pci 
test as skipped instead of exiting? (i.e. use g_test_skip() instead of 
exit()). Would that work for you?

  Thomas

Re: [PATCH v6 07/11] tests/qtest: migration-test: Make sure QEMU process "to" exited after migration is canceled

[Juan Quintela]

Bin Meng <bin.meng@windriver.com> wrote:
> From: Xuzhou Cheng <xuzhou.cheng@windriver.com>
>
> Make sure QEMU process "to" exited before launching another target
> for migration in the test_multifd_tcp_cancel case.
>
> Signed-off-by: Xuzhou Cheng <xuzhou.cheng@windriver.com>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
>
> ---
>
> Changes in v6:
> - change to use qtest_wait_qemu() API
>
> Changes in v3:
> - Add a usleep(1) in the busy wait loop
>
> Changes in v2:
> - Change to a busy wait after migration is canceled
>
>  tests/qtest/migration-test.c | 4 ++++
>  1 file changed, 4 insertions(+)

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [PATCH v6 08/11] tests/qtest: libqos: Do not build virtio-9p unconditionally

[Christian Schoenebeck]

On Friday, October 28, 2022 3:09:06 PM CEST Thomas Huth wrote:
> On 28/10/2022 14.59, Christian Schoenebeck wrote:
> > On Friday, October 28, 2022 6:57:33 AM CEST Bin Meng wrote:
> >> At present the virtio-9p related codes are built into libqos
> >> unconditionally. Change to build them conditionally by testing
> >> the 'virtfs' config option.
> >>
> >> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> >>
> >> ---
> >>
> >> Changes in v6:
> >> - new patch: "test/qtest/libqos: meson.build: Do not build virtio-9p unconditionally"
> >>
> >>   tests/qtest/libqos/meson.build | 6 ++++--
> >>   1 file changed, 4 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
> >> index 113c80b4e4..32f028872c 100644
> >> --- a/tests/qtest/libqos/meson.build
> >> +++ b/tests/qtest/libqos/meson.build
> >> @@ -33,8 +33,6 @@ libqos_srcs = files(
> >>           'sdhci.c',
> >>           'tpci200.c',
> >>           'virtio.c',
> >> -        'virtio-9p.c',
> >> -        'virtio-9p-client.c',
> >>           'virtio-balloon.c',
> >>           'virtio-blk.c',
> >>           'vhost-user-blk.c',
> >> @@ -62,6 +60,10 @@ libqos_srcs = files(
> >>           'x86_64_pc-machine.c',
> >>   )
> >>   
> >> +if have_virtfs
> >> +  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> >> +endif
> >> +
> >>   libqos = static_library('qos', libqos_srcs + genh,
> >>                           name_suffix: 'fa',
> >>                           build_by_default: false)
> >>
> > 
> > I wondered why this change would no longer execute the 9p tests here.
> > Apparently because it changes the order of tests being executed, i.e. 9p tests
> > would then be scheduled after:
> > 
> >    # Start of vhost-user-blk-pci tests
> >    # Start of vhost-user-blk-pci-tests tests
> >    Environment variable QTEST_QEMU_STORAGE_DAEMON_BINARY required
> >    [EXIT]
> > 
> > and I never cared about QEMU storage binary. Can we make a hack like the
> > following to not change the order of the tests?
> > 
> > diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
> > index 32f028872c..389bca9804 100644
> > --- a/tests/qtest/libqos/meson.build
> > +++ b/tests/qtest/libqos/meson.build
> > @@ -1,7 +1,13 @@
> >   libqos_srcs = files(
> >           '../libqtest.c',
> >           '../libqmp.c',
> > +)
> >   
> > +if have_virtfs
> > +  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> > +endif
> > +
> > +libqos_srcs += files(
> >           'qgraph.c',
> >           'qos_external.c',
> >           'pci.c',
> > @@ -60,10 +66,6 @@ libqos_srcs = files(
> >           'x86_64_pc-machine.c',
> >   )
> >   
> > -if have_virtfs
> > -  libqos_srcs += files('virtio-9p.c', 'virtio-9p-client.c')
> > -endif
> > -
> >   libqos = static_library('qos', libqos_srcs + genh,
> >                           name_suffix: 'fa',
> >                           build_by_default: false)
> > 
> > Too ugly?
> 
> Looks a little bit ugly, indeed. What about marking the vhost-user-blk-pci 
> test as skipped instead of exiting? (i.e. use g_test_skip() instead of 
> exit()). Would that work for you?

In general, sure!

But it seems it needs a bit more tweaking, simply placing

  g_test_skip(...);
  return NULL;

there causes a hang.

Best regards,
Christian Schoenebeck

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Marc-André Lureau]

Hi Bin

On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
>
> Now that we have fixed various test case issues as seen when running
> on Windows, let's enable the qtest build on Windows.
>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> Reviewed-by: Thomas Huth <thuth@redhat.com>

We haven't solved the CI timing out or eating all the CPU time, right?

Can we simply exclude it from CI for now, ie add to this patch

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 093276ddbc..ba9045ec38 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -62,7 +62,7 @@ msys2-64bit:
   - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
       --enable-capstone'
   - .\msys64\usr\bin\bash -lc 'make'
-  - .\msys64\usr\bin\bash -lc 'make check || { cat
build/meson-logs/testlog.txt; exit 1; } ;'
+  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'

 msys2-32bit:
   extends: .shared_msys2_builder
@@ -96,4 +96,4 @@ msys2-32bit:
   - cd output
   - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
   - ..\msys64\usr\bin\bash -lc 'make'
-  - ..\msys64\usr\bin\bash -lc 'make check || { cat
meson-logs/testlog.txt; exit 1; } ;'
+  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'


Could you resubmit your missing win test patches and check if gitlab is happy?

thanks

>
> ---
>
> Changes in v5:
> - Drop patches that are already merged
>
> Changes in v3:
> - Drop the host test
>
> Changes in v2:
> - new patch: "tests/qtest: Enable qtest build on Windows"
>
>  tests/qtest/meson.build | 6 ------
>  1 file changed, 6 deletions(-)
>
> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index c07a5b1a5f..f0ebb5fac6 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -1,9 +1,3 @@
> -# All QTests for now are POSIX-only, but the dependencies are
> -# really in libqtest, not in the testcases themselves.
> -if not config_host.has_key('CONFIG_POSIX')
> -  subdir_done()
> -endif
> -
>  slow_qtests = {
>    'ahci-test' : 60,
>    'bios-tables-test' : 120,
> --
> 2.25.1
>
>


-- 
Marc-André Lureau

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Thomas Huth]

On 23/11/2022 15.13, Marc-André Lureau wrote:
> Hi Bin
> 
> On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
>>
>> Now that we have fixed various test case issues as seen when running
>> on Windows, let's enable the qtest build on Windows.
>>
>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
> 
> We haven't solved the CI timing out or eating all the CPU time, right?
> 
> Can we simply exclude it from CI for now, ie add to this patch
> 
> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> index 093276ddbc..ba9045ec38 100644
> --- a/.gitlab-ci.d/windows.yml
> +++ b/.gitlab-ci.d/windows.yml
> @@ -62,7 +62,7 @@ msys2-64bit:
>     - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
>         --enable-capstone'
>     - .\msys64\usr\bin\bash -lc 'make'
> -  - .\msys64\usr\bin\bash -lc 'make check || { cat
> build/meson-logs/testlog.txt; exit 1; } ;'
> +  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'
> 
>   msys2-32bit:
>     extends: .shared_msys2_builder
> @@ -96,4 +96,4 @@ msys2-32bit:
>     - cd output
>     - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
>     - ..\msys64\usr\bin\bash -lc 'make'
> -  - ..\msys64\usr\bin\bash -lc 'make check || { cat
> meson-logs/testlog.txt; exit 1; } ;'
> +  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'

I think it's only the 64-bit job that is really problematic, so we could 
still run the qtests in the 32-bit job?

Alternatively, what about switching the 64-bit to another target that does 
not have so many qtests enabled? Some mips-softmmu or riscv-softmmu maybe? 
... we still check x86_64-softmmu in the .cirrus.yml builds, so this is 
hopefully not such a big loss...

  Thomas

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Marc-André Lureau]

Hi Thomas

On Wed, Nov 23, 2022 at 6:19 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 23/11/2022 15.13, Marc-André Lureau wrote:
> > Hi Bin
> >
> > On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
> >>
> >> Now that we have fixed various test case issues as seen when running
> >> on Windows, let's enable the qtest build on Windows.
> >>
> >> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> >> Reviewed-by: Thomas Huth <thuth@redhat.com>
> >
> > We haven't solved the CI timing out or eating all the CPU time, right?
> >
> > Can we simply exclude it from CI for now, ie add to this patch
> >
> > diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> > index 093276ddbc..ba9045ec38 100644
> > --- a/.gitlab-ci.d/windows.yml
> > +++ b/.gitlab-ci.d/windows.yml
> > @@ -62,7 +62,7 @@ msys2-64bit:
> >     - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
> >         --enable-capstone'
> >     - .\msys64\usr\bin\bash -lc 'make'
> > -  - .\msys64\usr\bin\bash -lc 'make check || { cat
> > build/meson-logs/testlog.txt; exit 1; } ;'
> > +  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> > qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'
> >
> >   msys2-32bit:
> >     extends: .shared_msys2_builder
> > @@ -96,4 +96,4 @@ msys2-32bit:
> >     - cd output
> >     - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
> >     - ..\msys64\usr\bin\bash -lc 'make'
> > -  - ..\msys64\usr\bin\bash -lc 'make check || { cat
> > meson-logs/testlog.txt; exit 1; } ;'
> > +  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> > qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'
>
> I think it's only the 64-bit job that is really problematic, so we could
> still run the qtests in the 32-bit job?
>
> Alternatively, what about switching the 64-bit to another target that does
> not have so many qtests enabled? Some mips-softmmu or riscv-softmmu maybe?
> ... we still check x86_64-softmmu in the .cirrus.yml builds, so this is
> hopefully not such a big loss...
>

The change I propose above is to simply skip the qtests on msys CI
builds. They are not running today on !POSIX.


-- 
Marc-André Lureau

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Thomas Huth]

On 24/11/2022 12.17, Marc-André Lureau wrote:
> Hi Thomas
> 
> On Wed, Nov 23, 2022 at 6:19 PM Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 23/11/2022 15.13, Marc-André Lureau wrote:
>>> Hi Bin
>>>
>>> On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
>>>>
>>>> Now that we have fixed various test case issues as seen when running
>>>> on Windows, let's enable the qtest build on Windows.
>>>>
>>>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
>>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>>
>>> We haven't solved the CI timing out or eating all the CPU time, right?
>>>
>>> Can we simply exclude it from CI for now, ie add to this patch
>>>
>>> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
>>> index 093276ddbc..ba9045ec38 100644
>>> --- a/.gitlab-ci.d/windows.yml
>>> +++ b/.gitlab-ci.d/windows.yml
>>> @@ -62,7 +62,7 @@ msys2-64bit:
>>>      - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
>>>          --enable-capstone'
>>>      - .\msys64\usr\bin\bash -lc 'make'
>>> -  - .\msys64\usr\bin\bash -lc 'make check || { cat
>>> build/meson-logs/testlog.txt; exit 1; } ;'
>>> +  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
>>> qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'
>>>
>>>    msys2-32bit:
>>>      extends: .shared_msys2_builder
>>> @@ -96,4 +96,4 @@ msys2-32bit:
>>>      - cd output
>>>      - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
>>>      - ..\msys64\usr\bin\bash -lc 'make'
>>> -  - ..\msys64\usr\bin\bash -lc 'make check || { cat
>>> meson-logs/testlog.txt; exit 1; } ;'
>>> +  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
>>> qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'
>>
>> I think it's only the 64-bit job that is really problematic, so we could
>> still run the qtests in the 32-bit job?
>>
>> Alternatively, what about switching the 64-bit to another target that does
>> not have so many qtests enabled? Some mips-softmmu or riscv-softmmu maybe?
>> ... we still check x86_64-softmmu in the .cirrus.yml builds, so this is
>> hopefully not such a big loss...
>>
> 
> The change I propose above is to simply skip the qtests on msys CI
> builds. They are not running today on !POSIX.

Sure ... question is what would be more valuable in the gitlab-CI ... only 
compile-testing x86_64-softmmu on msys2 (since that also compile-tests the 
HAX and WHPX stuff), but without qtests, or also run a (limited) set of 
qtests with another smaller softmmu target?
I don't mind either way, I just wanted to suggest an alternative to consider.

  Thomas

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Marc-André Lureau]

Hi

On Thu, Nov 24, 2022 at 3:49 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 24/11/2022 12.17, Marc-André Lureau wrote:
> > Hi Thomas
> >
> > On Wed, Nov 23, 2022 at 6:19 PM Thomas Huth <thuth@redhat.com> wrote:
> >>
> >> On 23/11/2022 15.13, Marc-André Lureau wrote:
> >>> Hi Bin
> >>>
> >>> On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
> >>>>
> >>>> Now that we have fixed various test case issues as seen when running
> >>>> on Windows, let's enable the qtest build on Windows.
> >>>>
> >>>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> >>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
> >>>
> >>> We haven't solved the CI timing out or eating all the CPU time, right?
> >>>
> >>> Can we simply exclude it from CI for now, ie add to this patch
> >>>
> >>> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> >>> index 093276ddbc..ba9045ec38 100644
> >>> --- a/.gitlab-ci.d/windows.yml
> >>> +++ b/.gitlab-ci.d/windows.yml
> >>> @@ -62,7 +62,7 @@ msys2-64bit:
> >>>      - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
> >>>          --enable-capstone'
> >>>      - .\msys64\usr\bin\bash -lc 'make'
> >>> -  - .\msys64\usr\bin\bash -lc 'make check || { cat
> >>> build/meson-logs/testlog.txt; exit 1; } ;'
> >>> +  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> >>> qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'
> >>>
> >>>    msys2-32bit:
> >>>      extends: .shared_msys2_builder
> >>> @@ -96,4 +96,4 @@ msys2-32bit:
> >>>      - cd output
> >>>      - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
> >>>      - ..\msys64\usr\bin\bash -lc 'make'
> >>> -  - ..\msys64\usr\bin\bash -lc 'make check || { cat
> >>> meson-logs/testlog.txt; exit 1; } ;'
> >>> +  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> >>> qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'
> >>
> >> I think it's only the 64-bit job that is really problematic, so we could
> >> still run the qtests in the 32-bit job?
> >>
> >> Alternatively, what about switching the 64-bit to another target that does
> >> not have so many qtests enabled? Some mips-softmmu or riscv-softmmu maybe?
> >> ... we still check x86_64-softmmu in the .cirrus.yml builds, so this is
> >> hopefully not such a big loss...
> >>
> >
> > The change I propose above is to simply skip the qtests on msys CI
> > builds. They are not running today on !POSIX.
>
> Sure ... question is what would be more valuable in the gitlab-CI ... only
> compile-testing x86_64-softmmu on msys2 (since that also compile-tests the
> HAX and WHPX stuff), but without qtests, or also run a (limited) set of
> qtests with another smaller softmmu target?
> I don't mind either way, I just wanted to suggest an alternative to consider.
>

Yes, we can do that on top though. I would want qtest to run on my
local msys build first, hence Bin's patch :)


-- 
Marc-André Lureau

Re: [PATCH v6 11/11] tests/qtest: Enable qtest build on Windows

[Bin Meng]

Hi Marc-André,

On Wed, Nov 23, 2022 at 10:14 PM Marc-André Lureau
<marcandre.lureau@gmail.com> wrote:
>
> Hi Bin
>
> On Fri, Oct 28, 2022 at 9:06 AM Bin Meng <bin.meng@windriver.com> wrote:
> >
> > Now that we have fixed various test case issues as seen when running
> > on Windows, let's enable the qtest build on Windows.
> >
> > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > Reviewed-by: Thomas Huth <thuth@redhat.com>
>
> We haven't solved the CI timing out or eating all the CPU time, right?

Correct

>
> Can we simply exclude it from CI for now, ie add to this patch

Thanks for your suggestion!

>
> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> index 093276ddbc..ba9045ec38 100644
> --- a/.gitlab-ci.d/windows.yml
> +++ b/.gitlab-ci.d/windows.yml
> @@ -62,7 +62,7 @@ msys2-64bit:
>    - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
>        --enable-capstone'
>    - .\msys64\usr\bin\bash -lc 'make'
> -  - .\msys64\usr\bin\bash -lc 'make check || { cat
> build/meson-logs/testlog.txt; exit 1; } ;'
> +  - .\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> qtest" || { cat build/meson-logs/testlog.txt; exit 1; } ;'

The double quote should be escaped.

>
>  msys2-32bit:
>    extends: .shared_msys2_builder
> @@ -96,4 +96,4 @@ msys2-32bit:
>    - cd output
>    - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
>    - ..\msys64\usr\bin\bash -lc 'make'
> -  - ..\msys64\usr\bin\bash -lc 'make check || { cat
> meson-logs/testlog.txt; exit 1; } ;'
> +  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS="--no-suite
> qtest" || { cat meson-logs/testlog.txt; exit 1; } ;'

I don't think we need to touch 32-bit.

> Could you resubmit your missing win test patches and check if gitlab is happy?
>

Sure, will do soon.

Regards,
Bin