[Qemu-devel] [RFC v2 0/6] qtest unit test framework

[Qemu-devel] [RFC v2 0/6] qtest unit test framework

[Anthony Liguori]

This series is still pretty rough but I wanted to get an idea of what people
thought about it before polishing it.

The general idea is outlined in the first test.  The main advantage of this
type of test framework compared to something like kvm-unit-test is that you
don't need a build environment for what you're trying to test.

Since your tests also link against the host environment, it potentially makes
tests much simplier to write (as you aren't reinventing an OS).  I think this
makes this style of test more appropriate for something like QEMU.

Anthony Liguori (6):
  qtest: add test framework
  qtest: add support for target-i386 -M pc
  Add core python test framework
  Add uart test case
  Add RTC test case
  Add C version of rtc-test

 Makefile        |    4 +
 Makefile.objs   |    2 +
 hw/pc.c         |    7 +-
 hw/pc_piix.c    |    9 +-
 qemu-options.hx |    8 ++
 qtest.c         |  357 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qtest.h         |   37 ++++++
 qtest.py        |   69 +++++++++++
 rtc-test.c      |  201 +++++++++++++++++++++++++++++++
 rtc-test.py     |  105 ++++++++++++++++
 serial-test.py  |   24 ++++
 vl.c            |    8 ++
 12 files changed, 827 insertions(+), 4 deletions(-)
 create mode 100644 qtest.c
 create mode 100644 qtest.h
 create mode 100644 qtest.py
 create mode 100644 rtc-test.c
 create mode 100644 rtc-test.py
 create mode 100644 serial-test.py

-- 
1.7.4.1

[Qemu-devel] [RFC v2 1/6] qtest: add test framework

[Anthony Liguori]

The idea behind qtest is pretty simple.  Instead of executing a CPU via TCG or
KVM, rely on an external process to send events to the device model that the CPU
would normally generate.

qtest presents itself as an accelerator.  In addition, a new option is added to
establish a qtest server (-qtest) that takes a character device.  This is what
allows the external process to send CPU events to the device model.

This is currently modelled after Xen since the Xen device model does something
very similar.  Instead of hooking cpu_exec, Xen sticks the CPU in the halted
state making sure it never gets to execute.  In addition, Xen replaces the LAPIC
with a dummy interrupt controller that forwards interrupt requests.

qtest does the exact same thing and uses a simple line based protocol to send
the events.  Documentation of that protocol is in qtest.c.

I considered reusing the monitor for this job.  Adding interrupts would be a bit
difficult.  In addition, logging would also be difficult.

qtest has extensive logging support.  All protocol commands are logged with
time stamps using a new command line option (-qtest-log).  Logging is important
since ultimately, this is a feature for debugging.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 Makefile.objs   |    2 +
 qemu-options.hx |    8 ++
 qtest.c         |  357 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qtest.h         |   37 ++++++
 vl.c            |    8 ++
 5 files changed, 412 insertions(+), 0 deletions(-)
 create mode 100644 qtest.c
 create mode 100644 qtest.h

diff --git a/Makefile.objs b/Makefile.objs
index d7a6539..8364ea2 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -293,6 +293,8 @@ hw-obj-$(CONFIG_DP8393X) += dp8393x.o
 hw-obj-$(CONFIG_DS1225Y) += ds1225y.o
 hw-obj-$(CONFIG_MIPSNET) += mipsnet.o
 
+hw-obj-y += qtest.o
+
 # Sound
 sound-obj-y =
 sound-obj-$(CONFIG_SB16) += sb16.o
diff --git a/qemu-options.hx b/qemu-options.hx
index 681eaf1..ca50988 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2598,6 +2598,14 @@ the @var{simple} tracing backend.
 @end table
 ETEXI
 
+DEF("qtest", HAS_ARG, QEMU_OPTION_qtest,
+    "-qtest CHR      specify tracing options\n",
+    QEMU_ARCH_ALL)
+
+DEF("qtest-log", HAS_ARG, QEMU_OPTION_qtest_log,
+    "-qtest-log LOG  specify tracing options\n",
+    QEMU_ARCH_ALL)
+
 HXCOMM This is the last statement. Insert new options before this line!
 STEXI
 @end table
diff --git a/qtest.c b/qtest.c
new file mode 100644
index 0000000..f41a9c3
--- /dev/null
+++ b/qtest.c
@@ -0,0 +1,357 @@
+/*
+ * Test Server
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Anthony Liguori   <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qtest.h"
+#include "qemu-char.h"
+#include "ioport.h"
+#include "memory.h"
+#include "hw/irq.h"
+#include "sysemu.h"
+
+#define MAX_IRQ 256
+
+const char *qtest_chrdev;
+const char *qtest_log;
+int qtest_allowed = 0;
+
+static FILE *qtest_log_fp;
+static CharDriverState *qtest_chr;
+static GString *inbuf;
+static int irq_levels[MAX_IRQ];
+static struct timeval start_time;
+static bool qtest_opened;
+
+#define FMT_timeval "%" PRId64 ".%06" PRId64
+
+/**
+ * QTest Protocol
+ *
+ * Line based protocol, request/response based.  Server can send async messages
+ * so clients should always handle many async messages before the response
+ * comes in.
+ *
+ * Valid requests
+ *
+ *  > outb ADDR VALUE
+ *  < OK
+ *
+ *  > outw ADDR VALUE
+ *  < OK
+ *
+ *  > outl ADDR VALUE
+ *  < OK
+ *
+ *  > inb ADDR
+ *  < OK VALUE
+ *
+ *  > inw ADDR
+ *  < OK VALUE
+ *
+ *  > inl ADDR
+ *  < OK VALUE
+ *
+ *  > read ADDR SIZE
+ *  < OK DATA
+ *
+ *  > write ADDR SIZE DATA
+ *  < OK
+ *
+ * Valid async messages:
+ *
+ *  IRQ raise NUM
+ *  IRQ lower NUM
+ *
+ * ADDR, SIZE, VALUE are all integers parsed with strtoul() with a base of 0.
+ *
+ * DATA is an arbitrarily long hex number prefixed with '0x'.  If it's smaller
+ * than the expected size, the value will be zero filled at the end of the data
+ * sequence.
+ *
+ * NUM is an IRQ number.
+ */
+
+static int hex2nib(char ch)
+{
+    if (ch >= '0' && ch <= '9') {
+        return ch - '0';
+    } else if (ch >= 'a' && ch <= 'f') {
+        return 10 + (ch - 'a');
+    } else if (ch >= 'A' && ch <= 'F') {
+        return 10 + (ch - 'a');
+    } else {
+        return -1;
+    }
+}
+
+static void qtest_get_time(struct timeval *tv)
+{
+    gettimeofday(tv, NULL);
+    tv->tv_sec -= start_time.tv_sec;
+    tv->tv_usec -= start_time.tv_usec;
+    if (tv->tv_usec < 0) {
+        tv->tv_usec += 1000000;
+        tv->tv_sec -= 1;
+    }
+}
+
+static void qtest_send_prefix(CharDriverState *chr)
+{
+    struct timeval tv;
+
+    if (!qtest_log_fp || !qtest_opened) {
+        return;
+    }
+
+    qtest_get_time(&tv);
+    fprintf(qtest_log_fp, "[S +" FMT_timeval "] ",
+            tv.tv_sec, tv.tv_usec);
+}
+
+static void qtest_send(CharDriverState *chr, const char *fmt, ...)
+{
+    va_list ap;
+    char buffer[1024];
+    size_t len;
+
+    va_start(ap, fmt);
+    len = vsnprintf(buffer, sizeof(buffer), fmt, ap);
+    va_end(ap);
+
+    qemu_chr_fe_write(chr, (uint8_t *)buffer, len);
+    if (qtest_log_fp && qtest_opened) {
+        fprintf(qtest_log_fp, "%s", buffer);
+    }
+}
+
+static void qtest_process_command(CharDriverState *chr, gchar **words)
+{
+    const gchar *command;
+
+    g_assert(words);
+
+    command = words[0];
+
+    if (qtest_log_fp) {
+        struct timeval tv;
+        int i;
+
+        qtest_get_time(&tv);
+        fprintf(qtest_log_fp, "[R +" FMT_timeval "]",
+                tv.tv_sec, tv.tv_usec);
+        for (i = 0; words[i]; i++) {
+            fprintf(qtest_log_fp, " %s", words[i]);
+        }
+        fprintf(qtest_log_fp, "\n");
+    }
+
+    g_assert(command);
+    if (strcmp(words[0], "outb") == 0 ||
+        strcmp(words[0], "outw") == 0 ||
+        strcmp(words[0], "outl") == 0) {
+        uint16_t addr;
+        uint32_t value;
+
+        g_assert(words[1] && words[2]);
+        addr = strtol(words[1], NULL, 0);
+        value = strtol(words[2], NULL, 0);
+
+        if (words[0][3] == 'b') {
+            cpu_outb(addr, value);
+        } else if (words[0][3] == 'w') {
+            cpu_outw(addr, value);
+        } else if (words[0][3] == 'l') {
+            cpu_outl(addr, value);
+        }
+        qtest_send_prefix(chr);
+        qtest_send(chr, "OK\n");
+    } else if (strcmp(words[0], "inb") == 0 ||
+        strcmp(words[0], "inw") == 0 ||
+        strcmp(words[0], "inl") == 0) {
+        uint16_t addr;
+        uint32_t value = -1U;
+
+        g_assert(words[1]);
+        addr = strtol(words[1], NULL, 0);
+
+        if (words[0][2] == 'b') {
+            value = cpu_inb(addr);
+        } else if (words[0][2] == 'w') {
+            value = cpu_inw(addr);
+        } else if (words[0][2] == 'l') {
+            value = cpu_inl(addr);
+        }
+        qtest_send_prefix(chr);
+        qtest_send(chr, "OK 0x%04x\n", value);
+    } else if (strcmp(words[0], "read") == 0) {
+        uint64_t addr, len, i;
+        uint8_t *data;
+
+        g_assert(words[1] && words[2]);
+        addr = strtoul(words[1], NULL, 0);
+        len = strtoul(words[2], NULL, 0);
+
+        data = g_malloc(len);
+        cpu_physical_memory_read(addr, data, len);
+
+        qtest_send_prefix(chr);
+        qtest_send(chr, "OK 0x");
+        for (i = 0; i < len; i++) {
+            qtest_send(chr, "%02x", data[i]);
+        }
+        qtest_send(chr, "\n");
+
+        g_free(data);
+    } else if (strcmp(words[0], "write") == 0) {
+        uint64_t addr, len, i;
+        uint8_t *data;
+        size_t data_len;
+
+        g_assert(words[1] && words[2] && words[3]);
+        addr = strtoul(words[1], NULL, 0);
+        len = strtoul(words[2], NULL, 0);
+
+        data_len = strlen(words[3]);
+        if (data_len < 3) {
+            qtest_send(chr, "ERR invalid argument size\n");
+            return;
+        }
+
+        data = g_malloc(len);
+        for (i = 0; i < len; i++) {
+            if ((i * 2 + 4) <= data_len) {
+                data[i] = hex2nib(words[3][i * 2 + 2]) << 4;
+                data[i] |= hex2nib(words[3][i * 2 + 3]);
+            } else {
+                data[i] = 0;
+            }
+        }
+        cpu_physical_memory_write(addr, data, len);
+        g_free(data);
+
+        qtest_send_prefix(chr);
+        qtest_send(chr, "OK\n");
+    } else {
+        fprintf(stderr, "Unknown command `%s'\n", words[0]);
+    }
+}
+
+static void qtest_process_inbuf(CharDriverState *chr, GString *inbuf)
+{
+    char *end;
+
+    while ((end = strchr(inbuf->str, '\n')) != NULL) {
+        size_t offset;
+        GString *cmd;
+        gchar **words;
+
+        offset = end - inbuf->str;
+
+        cmd = g_string_new_len(inbuf->str, offset);
+        g_string_erase(inbuf, 0, offset + 1);
+
+        words = g_strsplit(cmd->str, " ", 0);
+        qtest_process_command(chr, words);
+        g_strfreev(words);
+
+        g_string_free(cmd, TRUE);
+    }
+}
+
+static void qtest_read(void *opaque, const uint8_t *buf, int size)
+{
+    CharDriverState *chr = opaque;
+
+    g_string_append_len(inbuf, (const gchar *)buf, size);
+    qtest_process_inbuf(chr, inbuf);
+}
+
+static int qtest_can_read(void *opaque)
+{
+    return 1024;
+}
+
+static void qtest_event(void *opaque, int event)
+{
+    int i;
+
+    switch (event) {
+    case CHR_EVENT_OPENED:
+        qemu_system_reset(false);
+        for (i = 0; i < ARRAY_SIZE(irq_levels); i++) {
+            irq_levels[i] = 0;
+        }
+        gettimeofday(&start_time, NULL);
+        qtest_opened = true;
+        if (qtest_log_fp) {
+            fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n",
+                    start_time.tv_sec, start_time.tv_usec);
+        }
+        break;
+    case CHR_EVENT_CLOSED:
+        qtest_opened = false;
+        if (qtest_log_fp) {
+            struct timeval tv;
+            qtest_get_time(&tv);
+            fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n",
+                    tv.tv_sec, tv.tv_usec);
+        }
+        break;
+    default:
+        break;
+    }
+}
+
+static void qtest_set_irq(void *opaque, int irq, int level)
+{
+    CharDriverState *chr = qtest_chr;
+    bool changed;
+
+    changed = (irq_levels[irq] != level);
+    irq_levels[irq] = level;
+
+    if (changed) {
+        qtest_send_prefix(chr);
+        qtest_send(chr, "IRQ %s %d\n",
+                   level ? "raise" : "lower", irq);
+    }
+}
+
+qemu_irq *qtest_interrupt_controller_init(void)
+{
+    return qemu_allocate_irqs(qtest_set_irq, NULL, MAX_IRQ);
+}
+
+int qtest_init(void)
+{
+    CharDriverState *chr;
+
+    g_assert(qtest_chrdev != NULL);
+
+    chr = qemu_chr_new("qtest", qtest_chrdev, NULL);
+
+    qemu_chr_add_handlers(chr, qtest_can_read, qtest_read, qtest_event, chr);
+
+    inbuf = g_string_new("");
+
+    if (qtest_log) {
+        if (strcmp(qtest_log, "none") != 0) {
+            qtest_log_fp = fopen(qtest_log, "w+");
+        }
+    } else {
+        qtest_log_fp = stderr;
+    }
+
+    qtest_chr = chr;
+
+    return 0;
+}
diff --git a/qtest.h b/qtest.h
new file mode 100644
index 0000000..f0e1377
--- /dev/null
+++ b/qtest.h
@@ -0,0 +1,37 @@
+/*
+ * Test Server
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Anthony Liguori   <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef QTEST_H
+#define QTEST_H
+
+#include "qemu-common.h"
+
+extern int qtest_allowed;
+extern const char *qtest_chrdev;
+extern const char *qtest_log;
+
+static inline bool qtest_enabled(void)
+{
+    return qtest_allowed;
+}
+
+static inline int qtest_available(void)
+{
+    return 1;
+}
+
+int qtest_init(void);
+
+qemu_irq *qtest_interrupt_controller_init(void);
+
+#endif
diff --git a/vl.c b/vl.c
index a50842b..1c299e4 100644
--- a/vl.c
+++ b/vl.c
@@ -152,6 +152,7 @@ int main(int argc, char **argv)
 #ifdef CONFIG_VIRTFS
 #include "fsdev/qemu-fsdev.h"
 #endif
+#include "qtest.h"
 
 #include "disas.h"
 
@@ -2001,6 +2002,7 @@ static struct {
     { "tcg", "tcg", tcg_available, tcg_init, &tcg_allowed },
     { "xen", "Xen", xen_available, xen_init, &xen_allowed },
     { "kvm", "KVM", kvm_available, kvm_init, &kvm_allowed },
+    { "qtest", "QTest", qtest_available, qtest_init, &qtest_allowed },
 };
 
 static int configure_accelerator(void)
@@ -3057,6 +3059,12 @@ int main(int argc, char **argv, char **envp)
                     fclose(fp);
                     break;
                 }
+            case QEMU_OPTION_qtest:
+                qtest_chrdev = optarg;
+                break;
+            case QEMU_OPTION_qtest_log:
+                qtest_log = optarg;
+                break;
             default:
                 os_parse_cmd_args(popt->index, optarg);
             }
-- 
1.7.4.1

[Qemu-devel] [RFC v2 2/6] qtest: add support for target-i386 -M pc

[Anthony Liguori]

This involves forcing the CPU into the halted state if qtest is enabled and
replacing the local APIC with the qtest interrupt controller.

It should be pretty straight forward to do the same for other machine types on
other architectures.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 hw/pc.c      |    7 ++++++-
 hw/pc_piix.c |    9 ++++++---
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/hw/pc.c b/hw/pc.c
index 33778fe..bba8adc 100644
--- a/hw/pc.c
+++ b/hw/pc.c
@@ -43,6 +43,7 @@
 #include "ui/qemu-spice.h"
 #include "memory.h"
 #include "exec-memory.h"
+#include "qtest.h"
 
 /* output Bochs bios info messages */
 //#define DEBUG_BIOS
@@ -926,7 +927,11 @@ static void pc_cpu_reset(void *opaque)
     CPUState *env = opaque;
 
     cpu_reset(env);
-    env->halted = !cpu_is_bsp(env);
+    if (qtest_enabled()) {
+        env->halted = 1;
+    } else {
+        env->halted = !cpu_is_bsp(env);
+    }
 }
 
 static CPUState *pc_new_cpu(const char *cpu_model)
diff --git a/hw/pc_piix.c b/hw/pc_piix.c
index 970f43c..570d5fb 100644
--- a/hw/pc_piix.c
+++ b/hw/pc_piix.c
@@ -46,6 +46,7 @@
 #ifdef CONFIG_XEN
 #  include <xen/hvm/hvm_info_table.h>
 #endif
+#include "qtest.h"
 
 #define MAX_IDE_BUS 2
 
@@ -152,11 +153,13 @@ static void pc_init1(MemoryRegion *system_memory,
     }
     isa_bus_irqs(gsi);
 
-    if (!xen_enabled()) {
+    if (xen_enabled()) {
+        i8259 = xen_interrupt_controller_init();
+    } else if (qtest_enabled()) {
+        i8259 = qtest_interrupt_controller_init();
+    } else {
         cpu_irq = pc_allocate_cpu_irq();
         i8259 = i8259_init(cpu_irq[0]);
-    } else {
-        i8259 = xen_interrupt_controller_init();
     }
 
     for (i = 0; i < ISA_NUM_IRQS; i++) {
-- 
1.7.4.1

[Qemu-devel] [RFC v2 3/6] Add core python test framework

[Anthony Liguori]

---
 qtest.py |   69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 69 insertions(+), 0 deletions(-)
 create mode 100644 qtest.py

diff --git a/qtest.py b/qtest.py
new file mode 100644
index 0000000..5ac2e9b
--- /dev/null
+++ b/qtest.py
@@ -0,0 +1,69 @@
+import socket
+
+q = None
+
+class QTest(object):
+    def __init__(self, path):
+        self._sock = socket.socket(socket.AF_UNIX)
+        self._sock.connect(path)
+        self.inbuf = ''
+        self.irqs = {}
+        for i in range(16):
+            self.irqs[i] = False
+
+    def _recv(self):
+        while self.inbuf.find('\n') == -1:
+            self.inbuf += self._sock.recv(1024)
+
+        rsp, self.inbuf = self.inbuf.split('\n', 1)
+        return rsp.split()
+
+    def _send(self, command, *args):
+        outbuf = ' '.join([command] + map(str, args))
+        self._sock.sendall(outbuf + '\n')
+
+    def _cmd(self, command, *args):
+        self._send(command, *args)
+        while True:
+            rsp = self._recv()
+            if rsp[0] in ['IRQ']:
+                num = int(rsp[2], 0)
+                if rsp[1] in ['raise']:
+                    self.irqs[num] = True
+                else:
+                    self.irqs[num] = False
+                continue
+            if rsp[0] != 'OK':
+                raise Exception('Bad response')
+            break
+        return rsp[1:]
+
+    def get_irq(self, num):
+        return self.irqs[num]
+
+# Helpers to add expected platform functions in the current namespace
+
+def init(path):
+    global q
+    q = QTest(path)
+
+def outb(addr, value):
+    q._cmd('outb', addr, value)
+
+def outw(addr, value):
+    q._cmd('outw', addr, value)
+
+def outl(addr, value):
+    q._cmd('outl', addr, value)
+
+def inb(addr):
+    return int(q._cmd('inb', addr)[0], 0)
+
+def inw(addr):
+    return int(q._cmd('inw', addr)[0], 0)
+
+def inl(addr):
+    return int(q._cmd('inl', addr)[0], 0)
+
+def get_irq(num):
+    return q.get_irq(num)
-- 
1.7.4.1

[Qemu-devel] [RFC v2 4/6] Add uart test case

[Anthony Liguori]

---
 serial-test.py |   24 ++++++++++++++++++++++++
 1 files changed, 24 insertions(+), 0 deletions(-)
 create mode 100644 serial-test.py

diff --git a/serial-test.py b/serial-test.py
new file mode 100644
index 0000000..7aed0cb
--- /dev/null
+++ b/serial-test.py
@@ -0,0 +1,24 @@
+from qtest import outb, inb
+import qtest, sys
+
+def main(args):
+    if len(args) != 1:
+        raise Exception('Missing argument')
+
+    qtest.init(args[0])
+
+    base = 0x3f8
+
+    # disable THRE and RDA interrupt
+    outb(base + 1, 0x00)
+
+    for ch in "Hello, World!\r\n":
+        # wait for THRE
+        while (inb(base + 5) & 0x20) == 0:
+            pass
+
+        outb(base + 0, ord(ch))
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
-- 
1.7.4.1

[Qemu-devel] [RFC v2 5/6] Add RTC test case

[Anthony Liguori]

---
 rtc-test.py |  105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 105 insertions(+), 0 deletions(-)
 create mode 100644 rtc-test.py

diff --git a/rtc-test.py b/rtc-test.py
new file mode 100644
index 0000000..3159795
--- /dev/null
+++ b/rtc-test.py
@@ -0,0 +1,105 @@
+from qtest import inb, outb
+import qtest, time, calendar
+
+base = 0x70
+
+def bcd2dec(value):
+    return (((value >> 4) & 0x0F) * 10) + (value & 0x0F)
+
+def dec2bcd(value):
+    return ((value / 10) << 4) | (value % 10)
+
+def cmos_read(reg):
+    outb(base + 0, reg)
+    return inb(base + 1)
+
+def cmos_write(reg, val):
+    outb(base + 0, reg)
+    outb(base + 1, val)
+
+def cmos_get_date_time():
+    base_year = 2000
+
+    sec = cmos_read(0x00)
+    min = cmos_read(0x02)
+    hour = cmos_read(0x04)
+    mday = cmos_read(0x07)
+    mon = cmos_read(0x08)
+    year = cmos_read(0x09)
+
+    if (cmos_read(0x0B) & 4) == 0:
+        sec = bcd2dec(sec)
+        min = bcd2dec(min)
+        hour = bcd2dec(hour)
+        mday = bcd2dec(mday)
+        mon = bcd2dec(mon)
+        year = bcd2dec(year)
+        hour_offset = 80
+    else:
+        hour_offset = 0x80
+
+    if (cmos_read(0x0B) & 2) == 0:
+        if hour >= hour_offset:
+            hour -= hour_offset
+            hour += 12
+
+    return time.gmtime(calendar.timegm(((base_year + year), mon, mday, hour, min, sec)))
+
+def check_time():
+    # This check assumes a few things.  First, we cannot guarantee that we get
+    # a consistent reading from the wall clock because we may hit an edge of
+    # the clock while reading.  To work around this, we read four clock readings
+    # such that at least two of them should match.  We need to assume that one
+    # reading is corrupt so we need four readings to ensure that we have at
+    # least two consecutive identical readings
+    #
+    # It's also possible that we'll cross an edge reading the host clock so
+    # simply check to make sure that the clock reading is within the period of
+    # when we expect it to be.
+
+    start = time.gmtime()
+    date1 = cmos_get_date_time()
+    date2 = cmos_get_date_time()
+    date3 = cmos_get_date_time()
+    date4 = cmos_get_date_time()
+    end = time.gmtime()
+
+    if date1 == date2:
+        date = date1
+    elif date2 == date3:
+        date = date2
+    elif date3 == date4:
+        date = date4
+    else:
+        print 'Could not read RTC fast enough'
+        return False
+
+    if not start <= date <= end:
+        t = calendar.timegm(date)
+        s = calendar.timegm(start)
+        if t < s:
+            print 'RTC is %d second(s) behind wall-clock' % (s - t)
+        else:
+            print 'RTC is %d second(s) ahead of wall-clock' % (t - s)
+        return False
+
+    return True
+
+def main(args):
+    qtest.init(args[0])
+
+    # Set BCD mode
+    cmos_write(0x0B, cmos_read(0x0B) | 0x02)
+    if not check_time():
+        return 1
+
+    # Set DEC mode
+    cmos_write(0x0B, cmos_read(0x0B) & ~0x02)
+    if not check_time():
+        return 1
+
+    return 0
+
+if __name__ == '__main__':
+    import sys
+    sys.exit(main(sys.argv[1:]))
-- 
1.7.4.1

[Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Anthony Liguori]

---
 Makefile   |    4 +
 rtc-test.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 205 insertions(+), 0 deletions(-)
 create mode 100644 rtc-test.c

diff --git a/Makefile b/Makefile
index 301c75e..838cb01 100644
--- a/Makefile
+++ b/Makefile
@@ -215,6 +215,10 @@ $(qga-obj-y) qemu-ga.o: $(QGALIB_GEN) $(GENERATED_HEADERS)
 
 qemu-ga$(EXESUF): qemu-ga.o $(qga-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qobject-obj-y) $(version-obj-y) $(QGALIB_OBJ)
 
+libqtest.o: libqtest.c
+
+rtc-test$(EXESUF): rtc-test.o libqtest.o
+
 QEMULIBS=libhw32 libhw64 libuser libdis libdis-user
 
 clean:
diff --git a/rtc-test.c b/rtc-test.c
new file mode 100644
index 0000000..01e4e49
--- /dev/null
+++ b/rtc-test.c
@@ -0,0 +1,201 @@
+#include "libqtest.h"
+
+#include <glib.h>
+#include <stdio.h>
+#include <string.h>
+
+static uint8_t base = 0x70;
+
+static int bcd2dec(int value)
+{
+    return (((value >> 4) & 0x0F) * 10) + (value & 0x0F);
+}
+
+#if 0
+static int dec2bcd(int value)
+{
+    return ((value / 10) << 4) | (value % 10);
+}
+#endif
+
+static uint8_t cmos_read(uint8_t reg)
+{
+    outb(base + 0, reg);
+    return inb(base + 1);
+}
+
+static void cmos_write(uint8_t reg, uint8_t val)
+{
+    outb(base + 0, reg);
+    outb(base + 1, val);
+}
+
+static int tm_cmp(struct tm *lhs, struct tm *rhs)
+{
+    time_t a, b;
+    struct tm d1, d2;
+
+    memcpy(&d1, lhs, sizeof(d1));
+    memcpy(&d2, rhs, sizeof(d2));
+
+    a = mktime(&d1);
+    b = mktime(&d2);
+
+    if (a < b) {
+        return -1;
+    } else if (a > b) {
+        return 1;
+    }
+
+    return 0;
+}
+
+#if 0
+static void print_tm(struct tm *tm)
+{
+    printf("%04d-%02d-%02d %02d:%02d:%02d\n",
+           tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday, 
+           tm->tm_hour, tm->tm_min, tm->tm_sec, tm->tm_gmtoff);
+}
+#endif
+
+static void cmos_get_date_time(struct tm *date)
+{
+    int base_year = 2000, hour_offset;
+    int sec, min, hour, mday, mon, year;
+    time_t ts;
+    struct tm dummy;
+
+    sec = cmos_read(0x00);
+    min = cmos_read(0x02);
+    hour = cmos_read(0x04);
+    mday = cmos_read(0x07);
+    mon = cmos_read(0x08);
+    year = cmos_read(0x09);
+
+    if ((cmos_read(0x0B) & 4) == 0) {
+        sec = bcd2dec(sec);
+        min = bcd2dec(min);
+        hour = bcd2dec(hour);
+        mday = bcd2dec(mday);
+        mon = bcd2dec(mon);
+        year = bcd2dec(year);
+        hour_offset = 80;
+    } else {
+        hour_offset = 0x80;
+    }
+
+    if ((cmos_read(0x0B) & 2) == 0) {
+        if (hour >= hour_offset) {
+            hour -= hour_offset;
+            hour += 12;
+        }
+    }
+
+    ts = time(NULL);
+    localtime_r(&ts, &dummy);
+
+    date->tm_isdst = dummy.tm_isdst;
+    date->tm_sec = sec;
+    date->tm_min = min;
+    date->tm_hour = hour;
+    date->tm_mday = mday;
+    date->tm_mon = mon - 1;
+    date->tm_year = base_year + year - 1900;
+    date->tm_gmtoff = 0;
+
+    ts = mktime(date);
+}
+
+static bool check_time(int wiggle)
+{
+    struct tm start, date[4], end;
+    struct tm *datep;
+    time_t ts;
+
+    /*
+     * This check assumes a few things.  First, we cannot guarantee that we get
+     * a consistent reading from the wall clock because we may hit an edge of
+     * the clock while reading.  To work around this, we read four clock readings
+     * such that at least two of them should match.  We need to assume that one
+     * reading is corrupt so we need four readings to ensure that we have at
+     * least two consecutive identical readings
+     *
+     * It's also possible that we'll cross an edge reading the host clock so
+     * simply check to make sure that the clock reading is within the period of
+     * when we expect it to be.
+     */
+
+    ts = time(NULL);
+    gmtime_r(&ts, &start);
+
+    cmos_get_date_time(&date[0]);
+    cmos_get_date_time(&date[1]);
+    cmos_get_date_time(&date[2]);
+    cmos_get_date_time(&date[3]);
+
+    ts = time(NULL);
+    gmtime_r(&ts, &end);
+
+    if (tm_cmp(&date[0], &date[1]) == 0) {
+        datep = &date[0];
+    } else if (tm_cmp(&date[1], &date[2]) == 0) {
+        datep = &date[1];
+    } else if (tm_cmp(&date[2], &date[3]) == 0) {
+        datep = &date[2];
+    } else {
+        g_assert_not_reached();
+        return false;
+    }
+
+    if (!(tm_cmp(&start, datep) <= 0 && tm_cmp(datep, &end) <= 0)) {
+        time_t t, s;
+
+        start.tm_isdst = datep->tm_isdst;
+
+        t = mktime(datep);
+        s = mktime(&start);
+        if (t < s) {
+            fprintf(stderr, "RTC is %ld second(s) behind wall-clock\n", (s - t));
+        } else {
+            fprintf(stderr, "RTC is %ld second(s) ahead of wall-clock\n", (t - s));
+        }
+
+        if (ABS(t - s) <= wiggle) {
+            return true;
+        }
+
+        return false;
+    }
+
+    return true;
+}
+
+int main(int argc, char **argv)
+{
+    // We update the RTC based on a timer that fires every second.  But there's
+    // no guarantee that we'll get the timer at any point in time so when we
+    // read the CMOS time, the timer may be pending but not yet fired.  That
+    // means we can be a few seconds behind.
+    //
+    // We really should do a gettimeofday() when CMOS time is read to get an
+    // accurate clock time but until then, give ourselves a little bit of
+    // wiggle room.
+    int wiggle = 2;
+
+    qtest_start(argv[1]);
+
+    // Set BCD mode
+    cmos_write(0x0B, cmos_read(0x0B) | 0x02);
+    if (!check_time(wiggle)) {
+        return 1;
+    }
+
+    // Set DEC mode
+    cmos_write(0x0B, cmos_read(0x0B) & ~0x02);
+    if (!check_time(wiggle)) {
+        return 1;
+    }
+
+    return 0;
+}
-- 
1.7.4.1

Re: [Qemu-devel] [RFC v2 2/6] qtest: add support for target-i386 -M pc

[Paolo Bonzini]

On 12/01/2011 07:43 PM, Anthony Liguori wrote:
> This involves forcing the CPU into the halted state if qtest is enabled and
> replacing the local APIC with the qtest interrupt controller.
>
> It should be pretty straight forward to do the same for other machine types on
> other architectures.
>
> Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>

Can you do this in some per-accelerator function?  Is it different from 
just not creating the CPU threads?  (Also, whatever you do here 
generally you could do for Xen, too).

Paolo

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Kevin Wolf]

Am 01.12.2011 19:43, schrieb Anthony Liguori:
> ---
>  Makefile   |    4 +
>  rtc-test.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 205 insertions(+), 0 deletions(-)
>  create mode 100644 rtc-test.c
> 
> diff --git a/Makefile b/Makefile
> index 301c75e..838cb01 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -215,6 +215,10 @@ $(qga-obj-y) qemu-ga.o: $(QGALIB_GEN) $(GENERATED_HEADERS)
>  
>  qemu-ga$(EXESUF): qemu-ga.o $(qga-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qobject-obj-y) $(version-obj-y) $(QGALIB_OBJ)
>  
> +libqtest.o: libqtest.c

Did you forget to commit libqtest.c?

I think your series is a good start, but the examples only use port I/O.
Most other tests would probably need to access RAM (not sure if it's
convenient to do explicit read/write commands for that), interrupts and
some even QMP.

Should the framework and the tests live in the tests/ directory?

Kevin

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Anthony Liguori]

On 12/02/2011 11:25 AM, Kevin Wolf wrote:
> Am 01.12.2011 19:43, schrieb Anthony Liguori:
>> ---
>>   Makefile   |    4 +
>>   rtc-test.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 205 insertions(+), 0 deletions(-)
>>   create mode 100644 rtc-test.c
>>
>> diff --git a/Makefile b/Makefile
>> index 301c75e..838cb01 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -215,6 +215,10 @@ $(qga-obj-y) qemu-ga.o: $(QGALIB_GEN) $(GENERATED_HEADERS)
>>
>>   qemu-ga$(EXESUF): qemu-ga.o $(qga-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qobject-obj-y) $(version-obj-y) $(QGALIB_OBJ)
>>
>> +libqtest.o: libqtest.c
>
> Did you forget to commit libqtest.c?

Probably, sorry about that.

> I think your series is a good start, but the examples only use port I/O.
> Most other tests would probably need to access RAM (not sure if it's
> convenient to do explicit read/write commands for that), interrupts and
> some even QMP.

libqtest.h has:

bool qtest_get_irq(QTestState *s, int num);

void qtest_outb(QTestState *s, uint16_t addr, uint8_t value);

void qtest_outw(QTestState *s, uint16_t addr, uint16_t value);

void qtest_outl(QTestState *s, uint16_t addr, uint32_t value);

uint8_t qtest_inb(QTestState *s, uint16_t addr);

uint16_t qtest_inw(QTestState *s, uint16_t addr);

uint32_t qtest_inl(QTestState *s, uint16_t addr);

void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size);

void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size);

So that's how you read/write memory.  Likewise, for IRQs, you can poll the 
status of a given IRQ.  I thought about doing some sort of signal magic around 
but when writing tests, polling the IRQ seems easier to deal with.

> Should the framework and the tests live in the tests/ directory?

Probably, except that tests/ has it's own Makefile which is sort of awkward. 
Any objections to moving tests/* to tests/tcg-test and then moving all of the 
various gtest/check tests to tests/ along with qtest?

Regards,

Anthony Liguori

> Kevin
>

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Kevin Wolf]

Am 02.12.2011 18:26, schrieb Anthony Liguori:
> On 12/02/2011 11:25 AM, Kevin Wolf wrote:
>> Am 01.12.2011 19:43, schrieb Anthony Liguori:
>>> ---
>>>   Makefile   |    4 +
>>>   rtc-test.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   2 files changed, 205 insertions(+), 0 deletions(-)
>>>   create mode 100644 rtc-test.c
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 301c75e..838cb01 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -215,6 +215,10 @@ $(qga-obj-y) qemu-ga.o: $(QGALIB_GEN) $(GENERATED_HEADERS)
>>>
>>>   qemu-ga$(EXESUF): qemu-ga.o $(qga-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qobject-obj-y) $(version-obj-y) $(QGALIB_OBJ)
>>>
>>> +libqtest.o: libqtest.c
>>
>> Did you forget to commit libqtest.c?
> 
> Probably, sorry about that.
> 
>> I think your series is a good start, but the examples only use port I/O.
>> Most other tests would probably need to access RAM (not sure if it's
>> convenient to do explicit read/write commands for that), interrupts and
>> some even QMP.
> 
> libqtest.h has:
> 
> bool qtest_get_irq(QTestState *s, int num);
> 
> void qtest_outb(QTestState *s, uint16_t addr, uint8_t value);
> 
> void qtest_outw(QTestState *s, uint16_t addr, uint16_t value);
> 
> void qtest_outl(QTestState *s, uint16_t addr, uint32_t value);
> 
> uint8_t qtest_inb(QTestState *s, uint16_t addr);
> 
> uint16_t qtest_inw(QTestState *s, uint16_t addr);
> 
> uint32_t qtest_inl(QTestState *s, uint16_t addr);
> 
> void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size);
> 
> void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size);
> 
> So that's how you read/write memory.  Likewise, for IRQs, you can poll the 
> status of a given IRQ.  I thought about doing some sort of signal magic around 
> but when writing tests, polling the IRQ seems easier to deal with.

Okay, polling interrupts should be good enough for tests.

I guess the test still needs to do everything that a guest OS would have
to do, for example send an EOI to the PIC? We'll probably want to have a
library for such things then, but we can add it with the first test that
uses interrupts.

>> Should the framework and the tests live in the tests/ directory?
> 
> Probably, except that tests/ has it's own Makefile which is sort of awkward. 
> Any objections to moving tests/* to tests/tcg-test and then moving all of the 
> various gtest/check tests to tests/ along with qtest?

No objection, I think this is exactly what we should do.

Kevin

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Luiz Capitulino]

On Fri, 02 Dec 2011 18:45:27 +0100
Kevin Wolf <kwolf@redhat.com> wrote:

> Am 02.12.2011 18:26, schrieb Anthony Liguori:
> > On 12/02/2011 11:25 AM, Kevin Wolf wrote:
> >> Am 01.12.2011 19:43, schrieb Anthony Liguori:
> >>> ---
> >>>   Makefile   |    4 +
> >>>   rtc-test.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>   2 files changed, 205 insertions(+), 0 deletions(-)
> >>>   create mode 100644 rtc-test.c
> >>>
> >>> diff --git a/Makefile b/Makefile
> >>> index 301c75e..838cb01 100644
> >>> --- a/Makefile
> >>> +++ b/Makefile
> >>> @@ -215,6 +215,10 @@ $(qga-obj-y) qemu-ga.o: $(QGALIB_GEN) $(GENERATED_HEADERS)
> >>>
> >>>   qemu-ga$(EXESUF): qemu-ga.o $(qga-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qobject-obj-y) $(version-obj-y) $(QGALIB_OBJ)
> >>>
> >>> +libqtest.o: libqtest.c
> >>
> >> Did you forget to commit libqtest.c?
> > 
> > Probably, sorry about that.
> > 
> >> I think your series is a good start, but the examples only use port I/O.
> >> Most other tests would probably need to access RAM (not sure if it's
> >> convenient to do explicit read/write commands for that), interrupts and
> >> some even QMP.
> > 
> > libqtest.h has:
> > 
> > bool qtest_get_irq(QTestState *s, int num);
> > 
> > void qtest_outb(QTestState *s, uint16_t addr, uint8_t value);
> > 
> > void qtest_outw(QTestState *s, uint16_t addr, uint16_t value);
> > 
> > void qtest_outl(QTestState *s, uint16_t addr, uint32_t value);
> > 
> > uint8_t qtest_inb(QTestState *s, uint16_t addr);
> > 
> > uint16_t qtest_inw(QTestState *s, uint16_t addr);
> > 
> > uint32_t qtest_inl(QTestState *s, uint16_t addr);
> > 
> > void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size);
> > 
> > void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size);
> > 
> > So that's how you read/write memory.  Likewise, for IRQs, you can poll the 
> > status of a given IRQ.  I thought about doing some sort of signal magic around 
> > but when writing tests, polling the IRQ seems easier to deal with.
> 
> Okay, polling interrupts should be good enough for tests.
> 
> I guess the test still needs to do everything that a guest OS would have
> to do, for example send an EOI to the PIC? We'll probably want to have a
> library for such things then, but we can add it with the first test that
> uses interrupts.
> 
> >> Should the framework and the tests live in the tests/ directory?
> > 
> > Probably, except that tests/ has it's own Makefile which is sort of awkward. 
> > Any objections to moving tests/* to tests/tcg-test and then moving all of the 
> > various gtest/check tests to tests/ along with qtest?
> 
> No objection, I think this is exactly what we should do.

I fully agree.

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Anthony Liguori]

On 12/02/2011 11:45 AM, Kevin Wolf wrote:
> Am 02.12.2011 18:26, schrieb Anthony Liguori:
>> On 12/02/2011 11:25 AM, Kevin Wolf wrote:
>> So that's how you read/write memory.  Likewise, for IRQs, you can poll the
>> status of a given IRQ.  I thought about doing some sort of signal magic around
>> but when writing tests, polling the IRQ seems easier to deal with.
>
> Okay, polling interrupts should be good enough for tests.
>
> I guess the test still needs to do everything that a guest OS would have
> to do, for example send an EOI to the PIC? We'll probably want to have a
> library for such things then, but we can add it with the first test that
> uses interrupts.

No, right now we more or less create a fake I/O APIC.  We don't have to deal 
with masking in the local APIC, boot strapping, or anything like that.

It makes writing tests easier but I think it makes supporting MSI a bit more 
challenging.  I'm not sure how well it will generalize to other platforms either.

That's one of the reasons I wanted to get an early version out to get some 
feedback on this.

Regards,

Anthony Liguori

>>> Should the framework and the tests live in the tests/ directory?
>>
>> Probably, except that tests/ has it's own Makefile which is sort of awkward.
>> Any objections to moving tests/* to tests/tcg-test and then moving all of the
>> various gtest/check tests to tests/ along with qtest?
>
> No objection, I think this is exactly what we should do.
>
> Kevin
>

Re: [Qemu-devel] [RFC v2 0/6] qtest unit test framework

[Dor Laor]

On 12/01/2011 08:43 PM, Anthony Liguori wrote:
> This series is still pretty rough but I wanted to get an idea of what people
> thought about it before polishing it.
>
> The general idea is outlined in the first test.  The main advantage of this
> type of test framework compared to something like kvm-unit-test is that you
> don't need a build environment for what you're trying to test.

Luckily w/ qemu cpu emulation and few images it can be set once and be 
there for ever.

The advantage of kvm-unit-test is that the code actually does run. So we 
can test irq injections, io/mmio in the kernel too, dirty bit tracking 
and some more all together.

>
> Since your tests also link against the host environment, it potentially makes
> tests much simplier to write (as you aren't reinventing an OS).  I think this
> makes this style of test more appropriate for something like QEMU.
>
> Anthony Liguori (6):
>    qtest: add test framework
>    qtest: add support for target-i386 -M pc
>    Add core python test framework
>    Add uart test case
>    Add RTC test case
>    Add C version of rtc-test
>
>   Makefile        |    4 +
>   Makefile.objs   |    2 +
>   hw/pc.c         |    7 +-
>   hw/pc_piix.c    |    9 +-
>   qemu-options.hx |    8 ++
>   qtest.c         |  357 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   qtest.h         |   37 ++++++
>   qtest.py        |   69 +++++++++++
>   rtc-test.c      |  201 +++++++++++++++++++++++++++++++
>   rtc-test.py     |  105 ++++++++++++++++
>   serial-test.py  |   24 ++++
>   vl.c            |    8 ++
>   12 files changed, 827 insertions(+), 4 deletions(-)
>   create mode 100644 qtest.c
>   create mode 100644 qtest.h
>   create mode 100644 qtest.py
>   create mode 100644 rtc-test.c
>   create mode 100644 rtc-test.py
>   create mode 100644 serial-test.py
>

Re: [Qemu-devel] [RFC v2 6/6] Add C version of rtc-test

[Kevin Wolf]

Am 02.12.2011 19:43, schrieb Anthony Liguori:
> On 12/02/2011 11:45 AM, Kevin Wolf wrote:
>> Am 02.12.2011 18:26, schrieb Anthony Liguori:
>>> On 12/02/2011 11:25 AM, Kevin Wolf wrote:
>>> So that's how you read/write memory.  Likewise, for IRQs, you can poll the
>>> status of a given IRQ.  I thought about doing some sort of signal magic around
>>> but when writing tests, polling the IRQ seems easier to deal with.
>>
>> Okay, polling interrupts should be good enough for tests.
>>
>> I guess the test still needs to do everything that a guest OS would have
>> to do, for example send an EOI to the PIC? We'll probably want to have a
>> library for such things then, but we can add it with the first test that
>> uses interrupts.
> 
> No, right now we more or less create a fake I/O APIC.  We don't have to deal 
> with masking in the local APIC, boot strapping, or anything like that.
> 
> It makes writing tests easier but I think it makes supporting MSI a bit more 
> challenging.  I'm not sure how well it will generalize to other platforms either.
> 
> That's one of the reasons I wanted to get an early version out to get some 
> feedback on this.

Hm, if it's useful to have a fake controller probably depends on what
you're trying to do. For writing test code for interrupt controllers
it's probably not helpful.

Now I'm not familiar enough with the interrupt code to say how tight
it's coupled with the actual CPU implementation and how difficult it is
to reuse in the context of a test environment, but I would say that we
should use as much of the real emulation as possible.

Also note that this isn't the only thing that test cases would share:
Most tests will probably want to find some PCI device, virtio tests will
probably want to share some code, and I don't even want to think about
testing USB devices...

Kevin

Re: [Qemu-devel] [RFC v2 1/6] qtest: add test framework

[Luiz Capitulino]

On Thu,  1 Dec 2011 12:43:27 -0600
Anthony Liguori <aliguori@us.ibm.com> wrote:

> The idea behind qtest is pretty simple.  Instead of executing a CPU via TCG or
> KVM, rely on an external process to send events to the device model that the CPU
> would normally generate.
> 
> qtest presents itself as an accelerator.  In addition, a new option is added to
> establish a qtest server (-qtest) that takes a character device.  This is what
> allows the external process to send CPU events to the device model.
> 
> This is currently modelled after Xen since the Xen device model does something
> very similar.  Instead of hooking cpu_exec, Xen sticks the CPU in the halted
> state making sure it never gets to execute.  In addition, Xen replaces the LAPIC
> with a dummy interrupt controller that forwards interrupt requests.
> 
> qtest does the exact same thing and uses a simple line based protocol to send
> the events.  Documentation of that protocol is in qtest.c.
> 
> I considered reusing the monitor for this job.  Adding interrupts would be a bit
> difficult.  In addition, logging would also be difficult.
> 
> qtest has extensive logging support.  All protocol commands are logged with
> time stamps using a new command line option (-qtest-log).  Logging is important
> since ultimately, this is a feature for debugging.
> 
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> ---
>  Makefile.objs   |    2 +
>  qemu-options.hx |    8 ++
>  qtest.c         |  357 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qtest.h         |   37 ++++++
>  vl.c            |    8 ++
>  5 files changed, 412 insertions(+), 0 deletions(-)
>  create mode 100644 qtest.c
>  create mode 100644 qtest.h
> 
> diff --git a/Makefile.objs b/Makefile.objs
> index d7a6539..8364ea2 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -293,6 +293,8 @@ hw-obj-$(CONFIG_DP8393X) += dp8393x.o
>  hw-obj-$(CONFIG_DS1225Y) += ds1225y.o
>  hw-obj-$(CONFIG_MIPSNET) += mipsnet.o
>  
> +hw-obj-y += qtest.o
> +
>  # Sound
>  sound-obj-y =
>  sound-obj-$(CONFIG_SB16) += sb16.o
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 681eaf1..ca50988 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -2598,6 +2598,14 @@ the @var{simple} tracing backend.
>  @end table
>  ETEXI
>  
> +DEF("qtest", HAS_ARG, QEMU_OPTION_qtest,
> +    "-qtest CHR      specify tracing options\n",
> +    QEMU_ARCH_ALL)
> +
> +DEF("qtest-log", HAS_ARG, QEMU_OPTION_qtest_log,
> +    "-qtest-log LOG  specify tracing options\n",
> +    QEMU_ARCH_ALL)

Missing good descriptions.

> +
>  HXCOMM This is the last statement. Insert new options before this line!
>  STEXI
>  @end table
> diff --git a/qtest.c b/qtest.c
> new file mode 100644
> index 0000000..f41a9c3
> --- /dev/null
> +++ b/qtest.c
> @@ -0,0 +1,357 @@
> +/*
> + * Test Server
> + *
> + * Copyright IBM, Corp. 2011
> + *
> + * Authors:
> + *  Anthony Liguori   <aliguori@us.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qtest.h"
> +#include "qemu-char.h"
> +#include "ioport.h"
> +#include "memory.h"
> +#include "hw/irq.h"
> +#include "sysemu.h"
> +
> +#define MAX_IRQ 256
> +
> +const char *qtest_chrdev;
> +const char *qtest_log;
> +int qtest_allowed = 0;
> +
> +static FILE *qtest_log_fp;
> +static CharDriverState *qtest_chr;
> +static GString *inbuf;
> +static int irq_levels[MAX_IRQ];
> +static struct timeval start_time;
> +static bool qtest_opened;
> +
> +#define FMT_timeval "%" PRId64 ".%06" PRId64
> +
> +/**
> + * QTest Protocol
> + *
> + * Line based protocol, request/response based.  Server can send async messages
> + * so clients should always handle many async messages before the response
> + * comes in.
> + *
> + * Valid requests
> + *
> + *  > outb ADDR VALUE
> + *  < OK
> + *
> + *  > outw ADDR VALUE
> + *  < OK
> + *
> + *  > outl ADDR VALUE
> + *  < OK
> + *
> + *  > inb ADDR
> + *  < OK VALUE
> + *
> + *  > inw ADDR
> + *  < OK VALUE
> + *
> + *  > inl ADDR
> + *  < OK VALUE
> + *
> + *  > read ADDR SIZE
> + *  < OK DATA
> + *
> + *  > write ADDR SIZE DATA
> + *  < OK
> + *
> + * Valid async messages:
> + *
> + *  IRQ raise NUM
> + *  IRQ lower NUM
> + *
> + * ADDR, SIZE, VALUE are all integers parsed with strtoul() with a base of 0.
> + *
> + * DATA is an arbitrarily long hex number prefixed with '0x'.  If it's smaller
> + * than the expected size, the value will be zero filled at the end of the data
> + * sequence.
> + *
> + * NUM is an IRQ number.
> + */
> +
> +static int hex2nib(char ch)
> +{
> +    if (ch >= '0' && ch <= '9') {
> +        return ch - '0';
> +    } else if (ch >= 'a' && ch <= 'f') {
> +        return 10 + (ch - 'a');
> +    } else if (ch >= 'A' && ch <= 'F') {
> +        return 10 + (ch - 'a');
> +    } else {
> +        return -1;
> +    }
> +}
> +
> +static void qtest_get_time(struct timeval *tv)
> +{
> +    gettimeofday(tv, NULL);
> +    tv->tv_sec -= start_time.tv_sec;
> +    tv->tv_usec -= start_time.tv_usec;
> +    if (tv->tv_usec < 0) {
> +        tv->tv_usec += 1000000;
> +        tv->tv_sec -= 1;
> +    }
> +}
> +
> +static void qtest_send_prefix(CharDriverState *chr)
> +{
> +    struct timeval tv;
> +
> +    if (!qtest_log_fp || !qtest_opened) {
> +        return;
> +    }
> +
> +    qtest_get_time(&tv);
> +    fprintf(qtest_log_fp, "[S +" FMT_timeval "] ",
> +            tv.tv_sec, tv.tv_usec);
> +}
> +
> +static void qtest_send(CharDriverState *chr, const char *fmt, ...)
> +{
> +    va_list ap;
> +    char buffer[1024];
> +    size_t len;
> +
> +    va_start(ap, fmt);
> +    len = vsnprintf(buffer, sizeof(buffer), fmt, ap);
> +    va_end(ap);
> +
> +    qemu_chr_fe_write(chr, (uint8_t *)buffer, len);
> +    if (qtest_log_fp && qtest_opened) {
> +        fprintf(qtest_log_fp, "%s", buffer);
> +    }
> +}
> +
> +static void qtest_process_command(CharDriverState *chr, gchar **words)
> +{
> +    const gchar *command;
> +
> +    g_assert(words);
> +
> +    command = words[0];
> +
> +    if (qtest_log_fp) {
> +        struct timeval tv;
> +        int i;
> +
> +        qtest_get_time(&tv);
> +        fprintf(qtest_log_fp, "[R +" FMT_timeval "]",
> +                tv.tv_sec, tv.tv_usec);
> +        for (i = 0; words[i]; i++) {
> +            fprintf(qtest_log_fp, " %s", words[i]);
> +        }
> +        fprintf(qtest_log_fp, "\n");
> +    }
> +
> +    g_assert(command);
> +    if (strcmp(words[0], "outb") == 0 ||
> +        strcmp(words[0], "outw") == 0 ||
> +        strcmp(words[0], "outl") == 0) {
> +        uint16_t addr;
> +        uint32_t value;
> +
> +        g_assert(words[1] && words[2]);
> +        addr = strtol(words[1], NULL, 0);
> +        value = strtol(words[2], NULL, 0);
> +
> +        if (words[0][3] == 'b') {
> +            cpu_outb(addr, value);
> +        } else if (words[0][3] == 'w') {
> +            cpu_outw(addr, value);
> +        } else if (words[0][3] == 'l') {
> +            cpu_outl(addr, value);
> +        }
> +        qtest_send_prefix(chr);
> +        qtest_send(chr, "OK\n");
> +    } else if (strcmp(words[0], "inb") == 0 ||
> +        strcmp(words[0], "inw") == 0 ||
> +        strcmp(words[0], "inl") == 0) {
> +        uint16_t addr;
> +        uint32_t value = -1U;
> +
> +        g_assert(words[1]);
> +        addr = strtol(words[1], NULL, 0);
> +
> +        if (words[0][2] == 'b') {
> +            value = cpu_inb(addr);
> +        } else if (words[0][2] == 'w') {
> +            value = cpu_inw(addr);
> +        } else if (words[0][2] == 'l') {
> +            value = cpu_inl(addr);
> +        }
> +        qtest_send_prefix(chr);
> +        qtest_send(chr, "OK 0x%04x\n", value);
> +    } else if (strcmp(words[0], "read") == 0) {
> +        uint64_t addr, len, i;
> +        uint8_t *data;
> +
> +        g_assert(words[1] && words[2]);
> +        addr = strtoul(words[1], NULL, 0);
> +        len = strtoul(words[2], NULL, 0);
> +
> +        data = g_malloc(len);
> +        cpu_physical_memory_read(addr, data, len);
> +
> +        qtest_send_prefix(chr);
> +        qtest_send(chr, "OK 0x");
> +        for (i = 0; i < len; i++) {
> +            qtest_send(chr, "%02x", data[i]);
> +        }
> +        qtest_send(chr, "\n");
> +
> +        g_free(data);
> +    } else if (strcmp(words[0], "write") == 0) {
> +        uint64_t addr, len, i;
> +        uint8_t *data;
> +        size_t data_len;
> +
> +        g_assert(words[1] && words[2] && words[3]);
> +        addr = strtoul(words[1], NULL, 0);
> +        len = strtoul(words[2], NULL, 0);
> +
> +        data_len = strlen(words[3]);
> +        if (data_len < 3) {
> +            qtest_send(chr, "ERR invalid argument size\n");
> +            return;
> +        }
> +
> +        data = g_malloc(len);
> +        for (i = 0; i < len; i++) {
> +            if ((i * 2 + 4) <= data_len) {
> +                data[i] = hex2nib(words[3][i * 2 + 2]) << 4;
> +                data[i] |= hex2nib(words[3][i * 2 + 3]);
> +            } else {
> +                data[i] = 0;
> +            }
> +        }
> +        cpu_physical_memory_write(addr, data, len);
> +        g_free(data);
> +
> +        qtest_send_prefix(chr);
> +        qtest_send(chr, "OK\n");
> +    } else {
> +        fprintf(stderr, "Unknown command `%s'\n", words[0]);
> +    }
> +}
> +
> +static void qtest_process_inbuf(CharDriverState *chr, GString *inbuf)
> +{
> +    char *end;
> +
> +    while ((end = strchr(inbuf->str, '\n')) != NULL) {
> +        size_t offset;
> +        GString *cmd;
> +        gchar **words;
> +
> +        offset = end - inbuf->str;
> +
> +        cmd = g_string_new_len(inbuf->str, offset);
> +        g_string_erase(inbuf, 0, offset + 1);
> +
> +        words = g_strsplit(cmd->str, " ", 0);
> +        qtest_process_command(chr, words);
> +        g_strfreev(words);
> +
> +        g_string_free(cmd, TRUE);
> +    }
> +}
> +
> +static void qtest_read(void *opaque, const uint8_t *buf, int size)
> +{
> +    CharDriverState *chr = opaque;
> +
> +    g_string_append_len(inbuf, (const gchar *)buf, size);
> +    qtest_process_inbuf(chr, inbuf);
> +}
> +
> +static int qtest_can_read(void *opaque)
> +{
> +    return 1024;
> +}
> +
> +static void qtest_event(void *opaque, int event)
> +{
> +    int i;
> +
> +    switch (event) {
> +    case CHR_EVENT_OPENED:
> +        qemu_system_reset(false);
> +        for (i = 0; i < ARRAY_SIZE(irq_levels); i++) {
> +            irq_levels[i] = 0;
> +        }
> +        gettimeofday(&start_time, NULL);
> +        qtest_opened = true;
> +        if (qtest_log_fp) {
> +            fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n",
> +                    start_time.tv_sec, start_time.tv_usec);
> +        }
> +        break;
> +    case CHR_EVENT_CLOSED:
> +        qtest_opened = false;
> +        if (qtest_log_fp) {
> +            struct timeval tv;
> +            qtest_get_time(&tv);
> +            fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n",
> +                    tv.tv_sec, tv.tv_usec);
> +        }
> +        break;
> +    default:
> +        break;
> +    }
> +}
> +
> +static void qtest_set_irq(void *opaque, int irq, int level)
> +{
> +    CharDriverState *chr = qtest_chr;
> +    bool changed;
> +
> +    changed = (irq_levels[irq] != level);
> +    irq_levels[irq] = level;
> +
> +    if (changed) {
> +        qtest_send_prefix(chr);
> +        qtest_send(chr, "IRQ %s %d\n",
> +                   level ? "raise" : "lower", irq);
> +    }
> +}
> +
> +qemu_irq *qtest_interrupt_controller_init(void)
> +{
> +    return qemu_allocate_irqs(qtest_set_irq, NULL, MAX_IRQ);
> +}
> +
> +int qtest_init(void)
> +{
> +    CharDriverState *chr;
> +
> +    g_assert(qtest_chrdev != NULL);
> +
> +    chr = qemu_chr_new("qtest", qtest_chrdev, NULL);
> +
> +    qemu_chr_add_handlers(chr, qtest_can_read, qtest_read, qtest_event, chr);
> +
> +    inbuf = g_string_new("");
> +
> +    if (qtest_log) {
> +        if (strcmp(qtest_log, "none") != 0) {
> +            qtest_log_fp = fopen(qtest_log, "w+");
> +        }
> +    } else {
> +        qtest_log_fp = stderr;
> +    }
> +
> +    qtest_chr = chr;
> +
> +    return 0;
> +}
> diff --git a/qtest.h b/qtest.h
> new file mode 100644
> index 0000000..f0e1377
> --- /dev/null
> +++ b/qtest.h
> @@ -0,0 +1,37 @@
> +/*
> + * Test Server
> + *
> + * Copyright IBM, Corp. 2011
> + *
> + * Authors:
> + *  Anthony Liguori   <aliguori@us.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#ifndef QTEST_H
> +#define QTEST_H
> +
> +#include "qemu-common.h"
> +
> +extern int qtest_allowed;
> +extern const char *qtest_chrdev;
> +extern const char *qtest_log;
> +
> +static inline bool qtest_enabled(void)
> +{
> +    return qtest_allowed;
> +}
> +
> +static inline int qtest_available(void)
> +{
> +    return 1;
> +}
> +
> +int qtest_init(void);
> +
> +qemu_irq *qtest_interrupt_controller_init(void);
> +
> +#endif
> diff --git a/vl.c b/vl.c
> index a50842b..1c299e4 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -152,6 +152,7 @@ int main(int argc, char **argv)
>  #ifdef CONFIG_VIRTFS
>  #include "fsdev/qemu-fsdev.h"
>  #endif
> +#include "qtest.h"
>  
>  #include "disas.h"
>  
> @@ -2001,6 +2002,7 @@ static struct {
>      { "tcg", "tcg", tcg_available, tcg_init, &tcg_allowed },
>      { "xen", "Xen", xen_available, xen_init, &xen_allowed },
>      { "kvm", "KVM", kvm_available, kvm_init, &kvm_allowed },
> +    { "qtest", "QTest", qtest_available, qtest_init, &qtest_allowed },
>  };
>  
>  static int configure_accelerator(void)
> @@ -3057,6 +3059,12 @@ int main(int argc, char **argv, char **envp)
>                      fclose(fp);
>                      break;
>                  }
> +            case QEMU_OPTION_qtest:
> +                qtest_chrdev = optarg;
> +                break;
> +            case QEMU_OPTION_qtest_log:
> +                qtest_log = optarg;
> +                break;
>              default:
>                  os_parse_cmd_args(popt->index, optarg);
>              }

Re: [Qemu-devel] [RFC v2 5/6] Add RTC test case

[Luiz Capitulino]

On Thu,  1 Dec 2011 12:43:31 -0600
Anthony Liguori <aliguori@us.ibm.com> wrote:

I think it's a good idea to use python's unittest module and write tests as
unit-tests (eg. using asserts instead of writing results as strings).

> ---
>  rtc-test.py |  105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 105 insertions(+), 0 deletions(-)
>  create mode 100644 rtc-test.py
> 
> diff --git a/rtc-test.py b/rtc-test.py
> new file mode 100644
> index 0000000..3159795
> --- /dev/null
> +++ b/rtc-test.py
> @@ -0,0 +1,105 @@
> +from qtest import inb, outb
> +import qtest, time, calendar
> +
> +base = 0x70
> +
> +def bcd2dec(value):
> +    return (((value >> 4) & 0x0F) * 10) + (value & 0x0F)
> +
> +def dec2bcd(value):
> +    return ((value / 10) << 4) | (value % 10)
> +
> +def cmos_read(reg):
> +    outb(base + 0, reg)
> +    return inb(base + 1)
> +
> +def cmos_write(reg, val):
> +    outb(base + 0, reg)
> +    outb(base + 1, val)
> +
> +def cmos_get_date_time():
> +    base_year = 2000
> +
> +    sec = cmos_read(0x00)
> +    min = cmos_read(0x02)
> +    hour = cmos_read(0x04)
> +    mday = cmos_read(0x07)
> +    mon = cmos_read(0x08)
> +    year = cmos_read(0x09)
> +
> +    if (cmos_read(0x0B) & 4) == 0:
> +        sec = bcd2dec(sec)
> +        min = bcd2dec(min)
> +        hour = bcd2dec(hour)
> +        mday = bcd2dec(mday)
> +        mon = bcd2dec(mon)
> +        year = bcd2dec(year)
> +        hour_offset = 80
> +    else:
> +        hour_offset = 0x80
> +
> +    if (cmos_read(0x0B) & 2) == 0:
> +        if hour >= hour_offset:
> +            hour -= hour_offset
> +            hour += 12
> +
> +    return time.gmtime(calendar.timegm(((base_year + year), mon, mday, hour, min, sec)))
> +
> +def check_time():
> +    # This check assumes a few things.  First, we cannot guarantee that we get
> +    # a consistent reading from the wall clock because we may hit an edge of
> +    # the clock while reading.  To work around this, we read four clock readings
> +    # such that at least two of them should match.  We need to assume that one
> +    # reading is corrupt so we need four readings to ensure that we have at
> +    # least two consecutive identical readings
> +    #
> +    # It's also possible that we'll cross an edge reading the host clock so
> +    # simply check to make sure that the clock reading is within the period of
> +    # when we expect it to be.
> +
> +    start = time.gmtime()
> +    date1 = cmos_get_date_time()
> +    date2 = cmos_get_date_time()
> +    date3 = cmos_get_date_time()
> +    date4 = cmos_get_date_time()
> +    end = time.gmtime()
> +
> +    if date1 == date2:
> +        date = date1
> +    elif date2 == date3:
> +        date = date2
> +    elif date3 == date4:
> +        date = date4
> +    else:
> +        print 'Could not read RTC fast enough'
> +        return False
> +
> +    if not start <= date <= end:
> +        t = calendar.timegm(date)
> +        s = calendar.timegm(start)
> +        if t < s:
> +            print 'RTC is %d second(s) behind wall-clock' % (s - t)
> +        else:
> +            print 'RTC is %d second(s) ahead of wall-clock' % (t - s)
> +        return False
> +
> +    return True
> +
> +def main(args):
> +    qtest.init(args[0])
> +
> +    # Set BCD mode
> +    cmos_write(0x0B, cmos_read(0x0B) | 0x02)
> +    if not check_time():
> +        return 1
> +
> +    # Set DEC mode
> +    cmos_write(0x0B, cmos_read(0x0B) & ~0x02)
> +    if not check_time():
> +        return 1
> +
> +    return 0
> +
> +if __name__ == '__main__':
> +    import sys
> +    sys.exit(main(sys.argv[1:]))

Re: [Qemu-devel] [RFC v2 0/6] qtest unit test framework

[Anthony Liguori]

On 12/04/2011 04:03 AM, Dor Laor wrote:
> On 12/01/2011 08:43 PM, Anthony Liguori wrote:
>> This series is still pretty rough but I wanted to get an idea of what people
>> thought about it before polishing it.
>>
>> The general idea is outlined in the first test. The main advantage of this
>> type of test framework compared to something like kvm-unit-test is that you
>> don't need a build environment for what you're trying to test.
>
> Luckily w/ qemu cpu emulation and few images it can be set once and be there for
> ever.
>
> The advantage of kvm-unit-test is that the code actually does run. So we can
> test irq injections, io/mmio in the kernel too, dirty bit tracking and some more
> all together.

Yup, and kvm-unit-test will always need to exist for that.

But we need something that makes it easy to test ARM, PPC, s390, etc so that 
when we do things like the memory API, it's possible for a developer to do 
reasonable testing without having to track down 100 different images and tool 
chains.

Regards,

Anthony Liguori

>
>>
>> Since your tests also link against the host environment, it potentially makes
>> tests much simplier to write (as you aren't reinventing an OS). I think this
>> makes this style of test more appropriate for something like QEMU.
>>
>> Anthony Liguori (6):
>> qtest: add test framework
>> qtest: add support for target-i386 -M pc
>> Add core python test framework
>> Add uart test case
>> Add RTC test case
>> Add C version of rtc-test
>>
>> Makefile | 4 +
>> Makefile.objs | 2 +
>> hw/pc.c | 7 +-
>> hw/pc_piix.c | 9 +-
>> qemu-options.hx | 8 ++
>> qtest.c | 357 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> qtest.h | 37 ++++++
>> qtest.py | 69 +++++++++++
>> rtc-test.c | 201 +++++++++++++++++++++++++++++++
>> rtc-test.py | 105 ++++++++++++++++
>> serial-test.py | 24 ++++
>> vl.c | 8 ++
>> 12 files changed, 827 insertions(+), 4 deletions(-)
>> create mode 100644 qtest.c
>> create mode 100644 qtest.h
>> create mode 100644 qtest.py
>> create mode 100644 rtc-test.c
>> create mode 100644 rtc-test.py
>> create mode 100644 serial-test.py
>>
>
>

Re: [Qemu-devel] [RFC v2 2/6] qtest: add support for target-i386 -M pc

[Peter Maydell]

On 1 December 2011 18:43, Anthony Liguori <aliguori@us.ibm.com> wrote:
> This involves forcing the CPU into the halted state if qtest is enabled and
> replacing the local APIC with the qtest interrupt controller.
>
> It should be pretty straight forward to do the same for other machine types on
> other architectures.

Having to modify every machine type seems to me like a huge red flag that this
is the wrong approach for device level tests. Surely the right thing is to
(a) instantiate the device (b) manipulate it via its publicly exposed interfaces
(ie the memory regions, gpio signals, irqs, etc etc) ?

Alternatively if you're just testing features of the device as it
is wired up on the board then you want something that generates an
image that can just be run on the unmodified QEMU model; that then
lets you cross check by running your tests on real hardware.

-- PMM

Re: [Qemu-devel] [RFC v2 2/6] qtest: add support for target-i386 -M pc

[Anthony Liguori]

On 12/29/2011 11:40 AM, Peter Maydell wrote:
> On 1 December 2011 18:43, Anthony Liguori<aliguori@us.ibm.com>  wrote:
>> This involves forcing the CPU into the halted state if qtest is enabled and
>> replacing the local APIC with the qtest interrupt controller.
>>
>> It should be pretty straight forward to do the same for other machine types on
>> other architectures.
>
> Having to modify every machine type seems to me like a huge red flag that this
> is the wrong approach for device level tests. Surely the right thing is to
> (a) instantiate the device (b) manipulate it via its publicly exposed interfaces
> (ie the memory regions, gpio signals, irqs, etc etc) ?

The problem is the machine creation.  Once we convert everything to QOM 
including machine setup, yes, this is what we can do.

Regards,

Anthony Liguori

>
> Alternatively if you're just testing features of the device as it
> is wired up on the board then you want something that generates an
> image that can just be run on the unmodified QEMU model; that then
> lets you cross check by running your tests on real hardware.
>
> -- PMM
>