Re: [Qemu-devel] [PATCH 0/4] add "make check"

[Gerd Hoffmann <kraxel@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

  Hi,

>> I was hoping for more, but maybe we just need to start here and grow 
>> organically, I'll queue it again.
> 
> A while ago I played with some simple IDE tests. It basically was a
> small x86 kernel with an empty image that sends IDE commands and prints
> some results, and a script that invokes the guest and checks whether the
> test has passed or failed.

That reminds me that I've started toying with running tests inside a
guest too.  Stopped working on it a while back due to other priorities.
 Attached what I have so far.

> So at first I started with my own multiboot kernel and copied over some
> parts of kvm-unittest's libc. Clearly not the best idea once it's more
> than a couple of lines, so at some point I took the code and integrated
> with my real kvm-unittests repository.
> 
> Now I don't have to duplicate code any more, but at the same time
> there's no chance that a 'make check' in an upstream qemu tree could run
> this. Tests for other devices will have exactly the same problem.
> 
> Any suggestions on how to go forward with this kind of tests? Should
> this go into qemu or into kvm-unittests? Or should kvm-unittests be
> merged into the qemu tree? Or is the approach completely wrong?

I think we should have some framework to run tests inside the guest in
the qemu source tree.  I'm not sure kvm-unittests is the right tool for
the job though.  It is quite low-level and mainly targets the kvm bits
inside the linux kernel.  Testing -- for example -- usb device emulation
would pretty much require writing a usb stack for kvm-unitests ...

cheers,
  Gerd

[-- Attachment #2: 0001-initramfs-test-framework.patch --]
[-- Type: text/plain, Size: 16053 bytes --]

From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <kraxel@redhat.com>
Date: Tue, 13 Sep 2011 17:38:37 +0200
Subject: [PATCH] initramfs test framework

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 initramfs/.gitignore         |    3 +
 initramfs/10-qemu-udev.rules |    5 +
 initramfs/Makefile           |   36 +++++++
 initramfs/README             |   44 ++++++++
 initramfs/init.c             |  225 ++++++++++++++++++++++++++++++++++++++++++
 initramfs/initramfs-boot     |   32 ++++++
 initramfs/initramfs-create   |  111 +++++++++++++++++++++
 initramfs/test-ehci          |    3 +
 initramfs/test-ehci.good     |    8 ++
 initramfs/test-hello.c       |    7 ++
 initramfs/test-hello.good    |    1 +
 initramfs/test-uhci          |    3 +
 initramfs/test-uhci.good     |    3 +
 13 files changed, 481 insertions(+), 0 deletions(-)
 create mode 100644 initramfs/.gitignore
 create mode 100644 initramfs/10-qemu-udev.rules
 create mode 100644 initramfs/Makefile
 create mode 100644 initramfs/README
 create mode 100644 initramfs/init.c
 create mode 100755 initramfs/initramfs-boot
 create mode 100755 initramfs/initramfs-create
 create mode 100755 initramfs/test-ehci
 create mode 100644 initramfs/test-ehci.good
 create mode 100644 initramfs/test-hello.c
 create mode 100644 initramfs/test-hello.good
 create mode 100755 initramfs/test-uhci
 create mode 100644 initramfs/test-uhci.good

diff --git a/initramfs/.gitignore b/initramfs/.gitignore
new file mode 100644
index 0000000..8ece42c
--- /dev/null
+++ b/initramfs/.gitignore
@@ -0,0 +1,3 @@
+initramfs.cpio.gz
+init
+test-hello
diff --git a/initramfs/10-qemu-udev.rules b/initramfs/10-qemu-udev.rules
new file mode 100644
index 0000000..fb5cc0a
--- /dev/null
+++ b/initramfs/10-qemu-udev.rules
@@ -0,0 +1,5 @@
+# load modules
+DRIVER!="?*", ENV{MODALIAS}=="?*", RUN+="/sbin/modprobe -b $env{MODALIAS}"
+
+# virtio console
+KERNEL=="vport*", ATTR{name}=="?*", SYMLINK+="virtio-ports/$attr{name}"
diff --git a/initramfs/Makefile b/initramfs/Makefile
new file mode 100644
index 0000000..2db2b76
--- /dev/null
+++ b/initramfs/Makefile
@@ -0,0 +1,36 @@
+
+CC	:= gcc
+CFLAGS	:= -O2 -g -Wall
+LDFLAGS	:= -lutil
+
+TOOLS	:= init
+TEST_C	:= test-hello
+TEST_SH	:= test-uhci test-ehci
+
+TARGETS := $(TOOLS) $(TEST_C) initramfs.cpio.gz
+TESTS	:= $(TEST_C) $(TEST_SH)
+
+run-test-uhci : QEMU_ARGS := -usb -device usb-tablet
+run-test-ehci : QEMU_ARGS := -readconfig ../docs/ich9-ehci-uhci.cfg
+
+default all: $(TARGETS)
+
+clean:
+	rm -f $(TARGETS) *.o *~
+	rm -f org.qemu.initramfs.log
+
+init: init.o
+test-hello: test-hello.o
+
+initramfs.cpio.gz: $(TOOLS) $(TESTS) initramfs-create
+	./initramfs-create $@ $(TESTS)
+
+boot shell: $(TARGETS)
+	./initramfs-boot
+
+run-test-%: $(TARGETS)
+	./initramfs-boot /tests/test-$* $(QEMU_ARGS)
+	diff -u org.qemu.initramfs.log test-$*.good
+
+run-tests: $(patsubst %,run-%,$(TESTS))
+
diff --git a/initramfs/README b/initramfs/README
new file mode 100644
index 0000000..a9504ad
--- /dev/null
+++ b/initramfs/README
@@ -0,0 +1,44 @@
+
+This is an experimental test framework.
+
+Design goals
+------------
+
+  * Allow running tests within a guest.
+  * Be small enougth that it can easily be included in
+    the qemu source tree.
+  * Don't require setup and/or downloading stuff
+    (i.e. guest images) from the internet.
+  * Be easy to use.
+
+
+How it works
+------------
+
+It creates a linux initramfs from the bits found on the host machine,
+then boots the host kernel with the initramfs just created within
+qemu.  A special init handles the setup (create core devices, mount
+filesystems, start udev), command execution and shutdown.
+
+Obviously requires a linux host.  It also needs udev for device setup
+and module loading and virtio-serial support for logging.
+
+
+Getting started
+---------------
+
+Just type "make boot", a few seconds later you should be greeted by
+the guests bash prompt.  You can look around now.  Exiting the shell
+will shutdown the guest.  You'll find the shell output logged in the
+"org.qemu.initramfs.log" file.
+
+
+Run tests
+---------
+
+Type "make run-test-$name" for a single test or "make run-tests" to
+run all of them.  This will run the test program instead of the shell,
+then compare the actual output with the expected output.
+
+This can probably be refined ...
+
diff --git a/initramfs/init.c b/initramfs/init.c
new file mode 100644
index 0000000..94a8764
--- /dev/null
+++ b/initramfs/init.c
@@ -0,0 +1,225 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <unistd.h>
+#include <string.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <pty.h>
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <sys/mount.h>
+#include <sys/reboot.h>
+#include <sys/select.h>
+
+#define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
+
+static const char *logdev = "/dev/virtio-ports/org.qemu.initramfs.log";
+
+static struct {
+    mode_t     mode;
+    const char *path;
+    int        major;
+    int        minor;
+} cdevs[] = {
+    { 0666, "/dev/null",    1,  3 },
+    { 0600, "/dev/console", 5,  1 },
+    { 0666, "/dev/ptmx",    5,  2 },
+    { 0660, "/dev/kmsg",    1, 11 },
+};
+
+static struct {
+    const char *dest;
+    const char *type;
+} fs[] = {
+    { "/proc", "proc" },
+    { "/sys", "sysfs" },
+    { "/sys/kernel/debug", "debugfs" },
+    { "/dev/pts", "devpts" },
+    { "/dev/shm", "tmpfs" },
+};
+
+int run(const char *cmd, ...)
+{
+    va_list args;
+    char *argv[16], *arg;
+    int i, pid;
+
+    va_start(args, cmd);
+    argv[0] = (char*)cmd;
+    for (i = 1; i < ARRAY_SIZE(argv)-1; i++) {
+        arg = va_arg(args, char*);
+        if (arg == NULL) {
+            break;
+        }
+        argv[i] = arg;
+    }
+    argv[i] = NULL;
+    va_end(args);
+
+    pid = fork();
+    if (pid != 0) {
+        /* parent */
+        if (pid < 0) {
+            perror("fork");
+        }
+        return pid;
+    }
+    /* child */
+    execv(cmd, argv);
+    fprintf(stderr, "exec %s: %s\n", cmd, strerror(errno));
+    exit(1);
+}
+
+int forward(int from_fd, int to_fd, int log_fd)
+{
+    int len, pos, rc;
+    char buf[512];
+
+    len = read(from_fd, buf, sizeof(buf));
+    if (len < 0) {
+        return len;
+    }
+    for (pos = 0; pos < len; pos += rc) {
+        rc = write(to_fd, buf + pos, len - pos);
+        if (rc < 0) {
+            return rc;
+        }
+    }
+    if (log_fd != -1) {
+        for (pos = 0; pos < len; pos += rc) {
+            rc = write(log_fd, buf + pos, len - pos);
+            if (rc < 0) {
+                return rc;
+            }
+        }
+    }
+    return len;
+}
+
+static void
+tty_raw(int fd)
+{
+    struct termios tattr;
+
+    tcgetattr(fd, &tattr);
+    tattr.c_lflag &= ~(ICANON|ECHO);
+    tattr.c_cc[VMIN] = 1;
+    tattr.c_cc[VTIME] = 0;
+    tcsetattr(fd, TCSAFLUSH, &tattr);
+}
+
+int main(int argc, char *argv[])
+{
+    const char *command;
+    pid_t cmdpid, pid;
+    int status, i, rc, pty, log;
+    bool cmd_exit, eof_seen;
+
+    /* say hi */
+    fprintf(stderr, "-*- qemu initramfs starting -*-\n");
+
+    /* mount filesystems */
+    for (i = 0; i < ARRAY_SIZE(fs); i++) {
+        rc = mount(fs[i].type, fs[i].dest, fs[i].type, 0, NULL);
+        if (rc < 0) {
+            fprintf(stderr, "mount %s at %s: %s\n",
+                    fs[i].type, fs[i].dest, strerror(errno));
+        }
+    }
+
+    /* create basic chardevs */
+    for (i = 0; i < ARRAY_SIZE(cdevs); i++) {
+        rc = mknod(cdevs[i].path, cdevs[i].mode | S_IFCHR,
+                   makedev(cdevs[i].major, cdevs[i].minor));
+        if (rc < 0 && errno != EEXIST) {
+            fprintf(stderr, "mknod %s : %s\n",
+                    cdevs[i].path, strerror(errno));
+        }
+    }
+
+    /* start udev, let it create devices */
+    fprintf(stderr, "-*- starting udev -*-\n");
+    run("/sbin/depmod", "-a", NULL);
+    run("/sbin/udevd", "--daemon", "--resolve-names=never", NULL);
+    run("/sbin/udevadm", "trigger", "--action=add", NULL);
+
+    /* wait until virtio-serial is up'n'running */
+    for (log = -1, i = 0; log == -1 && i < 32; i++) {
+        sleep(1);
+        run("/sbin/udevadm", "settle", NULL);
+        log = open(logdev, O_WRONLY);
+    }
+    if (log == -1) {
+        fprintf(stderr, "open %s: %s\n", logdev, strerror(errno));
+    }
+
+    /* setup environment */
+    setenv("PATH", "/sbin:/bin", 1);
+
+    /* run our command in a pseuto tty */
+    command = getenv("QEMU_RUN");
+    if (command == NULL) {
+        command = "/bin/bash";
+    }
+    fprintf(stderr, "-*- running %s -*-\n", command);
+
+    tty_raw(0);
+    cmdpid = forkpty(&pty, NULL, NULL, NULL);
+    if (cmdpid == 0) {
+        /* child */
+        execl(command, command, NULL);
+        fprintf(stderr, "exec %s: %s\n", command, strerror(errno));
+        exit(1);
+    }
+
+    /* main loop */
+    cmd_exit = false;
+    eof_seen = false;
+    do {
+        fd_set rd;
+
+        /* reap children and zombies */
+        for (;;) {
+            pid = waitpid(-1, &status, WNOHANG);
+            if (pid <= 0) {
+                break;
+            }
+            if (pid == cmdpid) {
+                cmd_exit = true;
+            }
+        }
+
+        /* forward data between stdio (aka /dev/console) and pseudo tty */
+        if (!eof_seen) {
+            FD_ZERO(&rd);
+            FD_SET(0, &rd);
+            FD_SET(pty, &rd);
+            rc = select(pty+1, &rd, NULL, NULL, NULL);
+            if (rc > 0) {
+                if (FD_ISSET(0, &rd)) {
+                    /* stdin -> pseudo tty */
+                    if (forward(0, pty, -1) <= 0) {
+                        eof_seen = 1;
+                    }
+                }
+                if (FD_ISSET(pty, &rd)) {
+                    /* pseudo tty -> stdout + log */
+                    if (forward(pty, 1, log) <= 0) {
+                        eof_seen = 1;
+                    }
+                }
+            }
+        }
+    } while (!cmd_exit || !eof_seen);
+
+    /* powerdown vm */
+    fprintf(stderr, "-*- qemu initramfs done -*-\n");
+    reboot(RB_POWER_OFF);
+
+    /* keep gcc happy ;) */
+    return 0;
+}
diff --git a/initramfs/initramfs-boot b/initramfs/initramfs-boot
new file mode 100755
index 0000000..4884a6d
--- /dev/null
+++ b/initramfs/initramfs-boot
@@ -0,0 +1,32 @@
+#!/bin/sh
+
+base="$(dirname $0)"
+command="$1"
+shift;
+
+case "$(uname -m)" in
+	x86_64)	qemu="x86_64-softmmu/qemu-system-x86_64"
+		;;
+	i?86)	qemu="i386-softmmu/qemu-system-i386"
+		;;
+	*)	echo "unknown arch: $(uname -m)"
+		exit 1
+		;;
+esac
+
+kernel="$(echo /boot/vmlinu*$(uname -r)*)"
+initrd="initramfs.cpio.gz"
+append="console=ttyS0"
+if test "$command" != ""; then
+	append="$append QEMU_RUN=$command"
+fi
+
+exec ../$qemu -nographic -no-reboot \
+	-machine accel=kvm:tcg \
+	-kernel "$kernel" \
+	-initrd "$initrd" \
+	-append "$append" \
+	-chardev file,id=cmdlog,path=org.qemu.initramfs.log \
+	-device virtio-serial \
+	-device virtserialport,name=org.qemu.initramfs.log,chardev=cmdlog \
+	"$@"
diff --git a/initramfs/initramfs-create b/initramfs/initramfs-create
new file mode 100755
index 0000000..4525c11
--- /dev/null
+++ b/initramfs/initramfs-create
@@ -0,0 +1,111 @@
+#!/bin/bash
+#
+# Create a simple linux initramfs with the stuff found on the host.
+# Intented to be used for a quick boot test with the host kernel.
+#
+
+base="$(dirname $0)"
+file="${1-initramfs.cpio.gz}"; shift
+tests="$*"
+
+# create work dir
+WORK="${TMPDIR-/tmp}/${0##*/}-$$"
+mkdir "$WORK" || exit 1
+trap 'rm -rf "$WORK"' EXIT
+
+
+##############################################################################
+# helper functions
+
+function add_dirs() {
+	local dest="$WORK/fs"
+	local dir
+	for dir in dev dev/pts dev/shm etc proc sys \
+		   var var/tmp /var/log tmp
+	do
+		mkdir -p "$dest/$dir"
+	done
+}
+
+function add_binary() {
+	local dest="$WORK/fs$1"; shift
+	local item
+
+       	mkdir -p "$dest"
+	for item in $*; do
+		cp -L "$(which $item)" "$dest" || exit 1
+	done
+}
+
+function add_symlink() {
+	local target="$1"
+	local name="$2"
+	ln -s "$target" "$WORK/fs$name"
+}
+
+function add_data() {
+	local dest="$WORK/fs"
+	local item
+
+	for item in $*; do
+	       	mkdir -p $(dirname "$dest/$item")
+		cp -L "$item" "$dest/$item" || exit 1
+	done
+}
+
+function add_data_to_dir() {
+	local dest="$WORK/fs$1"; shift
+	local item
+
+       	mkdir -p "$dest"
+	for item in $*; do
+		cp -L "$item" "$dest" || exit 1
+	done
+}
+
+function add_libs() {
+	local item dest
+	for item in $(ldd $WORK/fs/init $WORK/fs/bin/* $WORK/fs/sbin/* \
+			$WORK/fs/tests/* $WORK/fs/lib/udev/*id); do
+		test -f "$item" || continue
+		test -f "$WORK/fs$item" && continue
+		dest=$(dirname "$WORK/fs$item")
+		mkdir -p "$dest"
+		cp -L "$item" "$dest"
+	done
+}
+
+function add_modules() {
+	local item
+
+	echo -n > "$WORK/modules"
+	for item in $*; do
+		modprobe --show-depends $item 2>/dev/null \
+			| awk '{ print $2 }' >> "$WORK/modules"
+	done
+	add_data $(sort "$WORK/modules" | uniq)
+}
+
+
+##############################################################################
+# main
+
+add_dirs
+add_binary / $base/init
+for t in $tests; do
+	add_binary /tests $base/$t;
+done
+add_binary /bin bash ls cat more dmesg ps uname find sort grep uniq
+add_binary /sbin lspci lsusb mount umount udevd udevadm blkid
+add_binary /sbin depmod insmod lsmod modinfo modprobe rmmod ip
+add_symlink bash /bin/sh
+add_data /usr/share/hwdata/*.ids
+add_data /lib/udev/*id
+add_data /lib/udev/rules.d/60-persistent-storage.rules
+add_data_to_dir /lib/udev/rules.d $base/10-qemu-udev.rules
+add_libs
+add_modules virtio_pci virtio_blk virtio_net virtio_console \
+	virtio_balloon virtio-rng 9pnet_virtio 9p \
+	ata_piix ahci sd_mod sr_mod sg e1000 8139cp
+
+(cd $WORK/fs; find -print | cpio -o -R 0:0 -H newc) | gzip > "$file"
diff --git a/initramfs/test-ehci b/initramfs/test-ehci
new file mode 100755
index 0000000..bd4bb5c
--- /dev/null
+++ b/initramfs/test-ehci
@@ -0,0 +1,3 @@
+#!/bin/sh
+/sbin/lspci -s1d
+/sbin/lsusb | sort
diff --git a/initramfs/test-ehci.good b/initramfs/test-ehci.good
new file mode 100644
index 0000000..dd9c6b8
--- /dev/null
+++ b/initramfs/test-ehci.good
@@ -0,0 +1,8 @@
+00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
+00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
+00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
+00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
+Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
+Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
+Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
+Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
diff --git a/initramfs/test-hello.c b/initramfs/test-hello.c
new file mode 100644
index 0000000..e5ce2d2
--- /dev/null
+++ b/initramfs/test-hello.c
@@ -0,0 +1,7 @@
+#include <stdio.h>
+
+int main(int argc, char *argv[])
+{
+    printf("Hello world!\n");
+    return 0;
+}
diff --git a/initramfs/test-hello.good b/initramfs/test-hello.good
new file mode 100644
index 0000000..dfd6895
--- /dev/null
+++ b/initramfs/test-hello.good
@@ -0,0 +1 @@
+Hello world!
diff --git a/initramfs/test-uhci b/initramfs/test-uhci
new file mode 100755
index 0000000..0af70e0
--- /dev/null
+++ b/initramfs/test-uhci
@@ -0,0 +1,3 @@
+#!/bin/sh
+/sbin/lspci -s1.2
+/sbin/lsusb | sort
\ No newline at end of file
diff --git a/initramfs/test-uhci.good b/initramfs/test-uhci.good
new file mode 100644
index 0000000..c87271e
--- /dev/null
+++ b/initramfs/test-uhci.good
@@ -0,0 +1,3 @@
+00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
+Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
+Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd 
-- 
1.7.1