[PATCH kvm-unit-tests v2 00/14] API test framework

[PATCH kvm-unit-tests v2 00/14] API test framework

[Avi Kivity]

This patchset adds an API test framework.  Rather than driving kvm from qemu,
we now have a way of calling the kvm API directly and observing the results.
We can switch to guest mode and back at will and see any micro effects such
as the result of executing particular instructions.

A first test is added, for commit edde99ce0 ("KVM: Write protect memory
after slot swap").  This would be pretty hard to test for using the current
qemu-driven tests, but detects 5 million failures in about two minutes on
my machine.

As an experiment, the framework is coded in C++.

Avi Kivity (14):
  Makefile: add support for C++
  Improve autodepend includes
    v2: new
  Add exception class for kernel errors (errno)
    v2: new
  Add try_main() for running a program under an exception handler
    v2: new
  Introduce a C++ wrapper for the kvm APIs
    v2: throw std::exception exceptions, not ints, for libc failures
        better msr list management
  Add support for calling a function in guest mode
    v2: use tr1::function instead of boost::function
  Add sample test using the api test harness
  api: add support for KVM_SET_USER_MEMORY_REGION flags field
    v2: new
  api: support KVM_GET_DIRTY_LOG ioctl
    v2: new
  api: add memory map management
    v2: new
  Build tests with debug information
    v2: new
  api: Add support for creating an identity map with a hole
    v2: new
  Introduce libapi.a to avoid long Makefile recipes
    v2: new
  Add dirty log test
    v2: new

 Makefile              |    8 +-
 api/api-sample.cc     |   30 ++++++++
 api/dirty-log.cc      |   78 ++++++++++++++++++++
 api/exception.cc      |   33 ++++++++
 api/exception.hh      |   19 +++++
 api/identity.cc       |   95 ++++++++++++++++++++++++
 api/identity.hh       |   43 +++++++++++
 api/kvmxx.cc          |  194 +++++++++++++++++++++++++++++++++++++++++++++++++
 api/kvmxx.hh          |   85 +++++++++++++++++++++
 api/memmap.cc         |   76 +++++++++++++++++++
 api/memmap.hh         |   43 +++++++++++
 config-x86-common.mak |   15 ++++-
 12 files changed, 715 insertions(+), 4 deletions(-)
 create mode 100644 api/api-sample.cc
 create mode 100644 api/dirty-log.cc
 create mode 100644 api/exception.cc
 create mode 100644 api/exception.hh
 create mode 100644 api/identity.cc
 create mode 100644 api/identity.hh
 create mode 100644 api/kvmxx.cc
 create mode 100644 api/kvmxx.hh
 create mode 100644 api/memmap.cc
 create mode 100644 api/memmap.hh

[PATCH kvm-unit-tests v2 01/14] Makefile: add support for C++

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 Makefile |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index d25e6f2..9b0256d 100644
--- a/Makefile
+++ b/Makefile
@@ -30,11 +30,13 @@ CFLAGS += -O1
 CFLAGS += $(autodepend-flags) -g -fomit-frame-pointer -Wall
 CFLAGS += $(call cc-option, -fno-stack-protector, "")
 CFLAGS += $(call cc-option, -fno-stack-protector-all, "")
+CFLAGS += -I.
 
-CXXFLAGS = $(autodepend-flags)
+CXXFLAGS += $(CFLAGS)
 
 autodepend-flags = -MMD -MF $(dir $*).$(notdir $*).d
 
+LDFLAGS += $(CFLAGS)
 LDFLAGS += -pthread -lrt
 
 kvmtrace_objs= kvmtrace.o
-- 
1.7.1

[PATCH kvm-unit-tests v2 02/14] Improve autodepend includes

[Avi Kivity]

Instead of listing all directories explicitly, include all autodepend files.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 Makefile              |    2 +-
 config-x86-common.mak |    1 -
 2 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 9b0256d..85ebd37 100644
--- a/Makefile
+++ b/Makefile
@@ -50,7 +50,7 @@ $(libcflat): $(cflatobjs)
 %.o: %.S
 	$(CC) $(CFLAGS) -c -nostdlib -o $@ $<
 
--include .*.d
+-include .*.d */.*.d */*/.*.d
 
 install:
 	mkdir -p $(DESTDIR)
diff --git a/config-x86-common.mak b/config-x86-common.mak
index c5508b3..d22df17 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -81,4 +81,3 @@ arch_clean:
 	$(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \
 	$(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o
 
--include $(TEST_DIR)/.*.d lib/.*.d lib/x86/.*.d
-- 
1.7.1

[PATCH kvm-unit-tests v2 03/14] Add exception class for kernel errors (errno)

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/exception.cc |   20 ++++++++++++++++++++
 api/exception.hh |   16 ++++++++++++++++
 2 files changed, 36 insertions(+), 0 deletions(-)
 create mode 100644 api/exception.cc
 create mode 100644 api/exception.hh

diff --git a/api/exception.cc b/api/exception.cc
new file mode 100644
index 0000000..500569a
--- /dev/null
+++ b/api/exception.cc
@@ -0,0 +1,20 @@
+#include "exception.hh"
+#include <cstdio>
+#include <cstring>
+
+errno_exception::errno_exception(int errno)
+    : _errno(errno)
+{
+}
+
+int errno_exception::errno() const
+{
+    return _errno;
+}
+
+const char *errno_exception::what()
+{
+    std::snprintf(_buf, sizeof _buf, "error: %s (%d)",
+		  std::strerror(_errno), _errno);
+    return _buf;
+}
diff --git a/api/exception.hh b/api/exception.hh
new file mode 100644
index 0000000..4672760
--- /dev/null
+++ b/api/exception.hh
@@ -0,0 +1,16 @@
+#ifndef EXCEPTION_HH
+#define EXCEPTION_HH
+
+#include <exception>
+
+class errno_exception : public std::exception {
+public:
+    explicit errno_exception(int err_no);
+    int errno() const;
+    virtual const char *what();
+private:
+    int _errno;
+    char _buf[1000];
+};
+
+#endif
-- 
1.7.1

[PATCH kvm-unit-tests v2 04/14] Add try_main() for running a program under an exception handler

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/exception.cc |   13 +++++++++++++
 api/exception.hh |    3 +++
 2 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/api/exception.cc b/api/exception.cc
index 500569a..910bdff 100644
--- a/api/exception.cc
+++ b/api/exception.cc
@@ -18,3 +18,16 @@ const char *errno_exception::what()
 		  std::strerror(_errno), _errno);
     return _buf;
 }
+
+int try_main(int (*main)(int argc, char** argv), int argc, char** argv,
+	     int ret_on_exception)
+{
+    try {
+        return main(argc, argv);
+    } catch (std::exception& e) {
+        std::fprintf(stderr, "exception: %s\n", e.what());
+    } catch (...) {
+        std::fprintf(stderr, "unknown exception\n");
+    }
+    return ret_on_exception;
+}
diff --git a/api/exception.hh b/api/exception.hh
index 4672760..f78d9a1 100644
--- a/api/exception.hh
+++ b/api/exception.hh
@@ -13,4 +13,7 @@ private:
     char _buf[1000];
 };
 
+int try_main(int (*main)(int argc, char** argv), int argc, char** argv,
+	     int ret_on_exception = 127);
+
 #endif
-- 
1.7.1

[PATCH kvm-unit-tests v2 05/14] Introduce a C++ wrapper for the kvm APIs

[Avi Kivity]

Introduce exception-safe objects for calling system, vm, and vcpu ioctls.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/kvmxx.cc |  185 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 api/kvmxx.hh |   83 ++++++++++++++++++++++++++
 2 files changed, 268 insertions(+), 0 deletions(-)
 create mode 100644 api/kvmxx.cc
 create mode 100644 api/kvmxx.hh

diff --git a/api/kvmxx.cc b/api/kvmxx.cc
new file mode 100644
index 0000000..ad27907
--- /dev/null
+++ b/api/kvmxx.cc
@@ -0,0 +1,185 @@
+#include "kvmxx.hh"
+#include "exception.hh"
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <stdlib.h>
+#include <memory>
+#include <algorithm>
+
+namespace kvm {
+
+static long check_error(long r)
+{
+    if (r == -1) {
+	throw errno_exception(errno);
+    }
+    return r;
+}
+
+fd::fd(int fd)
+    : _fd(fd)
+{
+}
+
+fd::fd(const fd& other)
+    : _fd(::dup(other._fd))
+{
+    check_error(_fd);
+}
+
+fd::fd(std::string device_node, int flags)
+    : _fd(::open(device_node.c_str(), flags))
+{
+    check_error(_fd);
+}
+
+long fd::ioctl(unsigned nr, long arg)
+{
+    return check_error(::ioctl(_fd, nr, arg));
+}
+
+vcpu::vcpu(vm& vm, int id)
+    : _vm(vm), _fd(vm._fd.ioctl(KVM_CREATE_VCPU, id)), _shared(NULL)
+    , _mmap_size(_vm._system._fd.ioctl(KVM_GET_VCPU_MMAP_SIZE, 0))
+
+{
+    kvm_run *shared = static_cast<kvm_run*>(::mmap(NULL, _mmap_size,
+						   PROT_READ | PROT_WRITE,
+						   MAP_SHARED,
+						   _fd.get(), 0));
+    if (shared == MAP_FAILED) {
+	throw errno_exception(errno);
+    }
+    _shared = shared;
+}
+
+vcpu::~vcpu()
+{
+    munmap(_shared, _mmap_size);
+}
+
+void vcpu::run()
+{
+    _fd.ioctl(KVM_RUN, 0);
+}
+
+kvm_regs vcpu::regs()
+{
+    kvm_regs regs;
+    _fd.ioctlp(KVM_GET_REGS, &regs);
+    return regs;
+}
+
+void vcpu::set_regs(const kvm_regs& regs)
+{
+    _fd.ioctlp(KVM_SET_REGS, const_cast<kvm_regs*>(&regs));
+}
+
+kvm_sregs vcpu::sregs()
+{
+    kvm_sregs sregs;
+    _fd.ioctlp(KVM_GET_SREGS, &sregs);
+    return sregs;
+}
+
+void vcpu::set_sregs(const kvm_sregs& sregs)
+{
+    _fd.ioctlp(KVM_SET_SREGS, const_cast<kvm_sregs*>(&sregs));
+}
+
+class vcpu::kvm_msrs_ptr {
+public:
+    explicit kvm_msrs_ptr(size_t nmsrs);
+    ~kvm_msrs_ptr() { ::free(_kvm_msrs); }
+    kvm_msrs* operator->() { return _kvm_msrs; }
+    kvm_msrs* get() { return _kvm_msrs; }
+private:
+    kvm_msrs* _kvm_msrs;
+};
+
+vcpu::kvm_msrs_ptr::kvm_msrs_ptr(size_t nmsrs)
+    : _kvm_msrs(0)
+{
+    size_t size = sizeof(kvm_msrs) + sizeof(kvm_msr_entry) * nmsrs;
+    _kvm_msrs = static_cast<kvm_msrs*>(::malloc(size));
+    if (!_kvm_msrs) {
+	throw std::bad_alloc();
+    }
+}
+
+std::vector<kvm_msr_entry> vcpu::msrs(std::vector<uint32_t> indices)
+{
+    kvm_msrs_ptr msrs(indices.size());
+    msrs->nmsrs = indices.size();
+    for (unsigned i = 0; i < msrs->nmsrs; ++i) {
+	msrs->entries[i].index = indices[i];
+    }
+    _fd.ioctlp(KVM_GET_MSRS, msrs.get());
+    return std::vector<kvm_msr_entry>(msrs->entries,
+				      msrs->entries + msrs->nmsrs);
+}
+
+void vcpu::set_msrs(const std::vector<kvm_msr_entry>& msrs)
+{
+    kvm_msrs_ptr _msrs(msrs.size());
+    _msrs->nmsrs = msrs.size();
+    std::copy(msrs.begin(), msrs.end(), _msrs->entries);
+    _fd.ioctlp(KVM_SET_MSRS, _msrs.get());
+}
+
+void vcpu::set_debug(uint64_t dr[8], bool enabled, bool singlestep)
+{
+    kvm_guest_debug gd;
+
+    gd.control = 0;
+    if (enabled) {
+	gd.control |= KVM_GUESTDBG_ENABLE;
+    }
+    if (singlestep) {
+	gd.control |= KVM_GUESTDBG_SINGLESTEP;
+    }
+    for (int i = 0; i < 8; ++i) {
+	gd.arch.debugreg[i] = dr[i];
+    }
+    _fd.ioctlp(KVM_SET_GUEST_DEBUG, &gd);
+}
+
+vm::vm(system& system)
+    : _system(system), _fd(system._fd.ioctl(KVM_CREATE_VM, 0))
+{
+}
+
+void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len)
+{
+    struct kvm_userspace_memory_region umr;
+
+    umr.slot = slot;
+    umr.flags = 0;
+    umr.guest_phys_addr = gpa;
+    umr.memory_size = len;
+    umr.userspace_addr = reinterpret_cast<uint64_t>(addr);
+    _fd.ioctlp(KVM_SET_USER_MEMORY_REGION, &umr);
+}
+
+void vm::set_tss_addr(uint32_t addr)
+{
+    _fd.ioctl(KVM_SET_TSS_ADDR, addr);
+}
+
+system::system(std::string device_node)
+    : _fd(device_node, O_RDWR)
+{
+}
+
+bool system::check_extension(int extension)
+{
+    return _fd.ioctl(KVM_CHECK_EXTENSION, extension);
+}
+
+int system::get_extension_int(int extension)
+{
+    return _fd.ioctl(KVM_CHECK_EXTENSION, extension);
+}
+
+};
diff --git a/api/kvmxx.hh b/api/kvmxx.hh
new file mode 100644
index 0000000..51dbe7a
--- /dev/null
+++ b/api/kvmxx.hh
@@ -0,0 +1,83 @@
+#ifndef KVMXX_H
+#define KVMXX_H
+
+#include <string>
+#include <signal.h>
+#include <unistd.h>
+#include <vector>
+#include <errno.h>
+#include <linux/kvm.h>
+#include <stdint.h>
+
+namespace kvm {
+
+class system;
+class vm;
+class vcpu;
+class fd;
+
+class fd {
+public:
+    explicit fd(int n);
+    explicit fd(std::string path, int flags);
+    fd(const fd& other);
+    ~fd() { ::close(_fd); }
+    int get() { return _fd; }
+    long ioctl(unsigned nr, long arg);
+    long ioctlp(unsigned nr, void *arg) {
+	return ioctl(nr, reinterpret_cast<long>(arg));
+    }
+private:
+    int _fd;
+};
+
+class vcpu {
+public:
+    vcpu(vm& vm, int fd);
+    ~vcpu();
+    void run();
+    kvm_run *shared();
+    kvm_regs regs();
+    void set_regs(const kvm_regs& regs);
+    kvm_sregs sregs();
+    void set_sregs(const kvm_sregs& sregs);
+    std::vector<kvm_msr_entry> msrs(std::vector<uint32_t> indices);
+    void set_msrs(const std::vector<kvm_msr_entry>& msrs);
+    void set_debug(uint64_t dr[8], bool enabled, bool singlestep);
+private:
+    class kvm_msrs_ptr;
+private:
+    vm& _vm;
+    fd _fd;
+    kvm_run *_shared;
+    unsigned _mmap_size;
+    friend class vm;
+};
+
+class vm {
+public:
+    explicit vm(system& system);
+    void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len);
+    void set_tss_addr(uint32_t addr);
+    system& sys() { return _system; }
+private:
+    system& _system;
+    fd _fd;
+    friend class system;
+    friend class vcpu;
+};
+
+class system {
+public:
+    explicit system(std::string device_node = "/dev/kvm");
+    bool check_extension(int extension);
+    int get_extension_int(int extension);
+private:
+    fd _fd;
+    friend class vcpu;
+    friend class vm;
+};
+
+};
+
+#endif
-- 
1.7.1

[PATCH kvm-unit-tests v2 06/14] Add support for calling a function in guest mode

[Avi Kivity]

This patch provides a way to establish an "identity" guest which has
a 1:1 gva->hva translation.  This allows the host to switch to guest
mode, call a function in the same address space, and return.

Because long mode virtual addresses are 47 bits long, and some hosts
have smaller physical addresses, we target 32-bit mode only.  On
x86_64 the code needs to be run with 'setarch i386 -3' to limit the
address space to 3GB, so the address space occupied by the local
APIC is left unused.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/identity.cc       |   76 +++++++++++++++++++++++++++++++++++++++++++++++++
 api/identity.hh       |   28 ++++++++++++++++++
 config-x86-common.mak |    1 +
 3 files changed, 105 insertions(+), 0 deletions(-)
 create mode 100644 api/identity.cc
 create mode 100644 api/identity.hh

diff --git a/api/identity.cc b/api/identity.cc
new file mode 100644
index 0000000..de52f68
--- /dev/null
+++ b/api/identity.cc
@@ -0,0 +1,76 @@
+
+#include "identity.hh"
+#include <stdio.h>
+
+namespace identity {
+
+typedef unsigned long ulong;
+
+void setup_vm(kvm::vm& vm)
+{
+    vm.set_memory_region(0, NULL, 0, 3UL << 30);
+    vm.set_tss_addr(3UL << 30);
+}
+
+void vcpu::setup_sregs()
+{
+    kvm_sregs sregs = { };
+    kvm_segment dseg = { };
+    dseg.base = 0; dseg.limit = -1U; dseg.type = 3; dseg.present = 1;
+    dseg.dpl = 3; dseg.db = 1; dseg.s = 1; dseg.l = 0; dseg.g = 1;
+    kvm_segment cseg = dseg;
+    cseg.type = 11;
+
+    sregs.cs = cseg; asm ("mov %%cs, %0" : "=rm"(sregs.cs.selector));
+    sregs.ds = dseg; asm ("mov %%ds, %0" : "=rm"(sregs.ds.selector));
+    sregs.es = dseg; asm ("mov %%es, %0" : "=rm"(sregs.es.selector));
+    sregs.fs = dseg; asm ("mov %%fs, %0" : "=rm"(sregs.fs.selector));
+    sregs.gs = dseg; asm ("mov %%gs, %0" : "=rm"(sregs.gs.selector));
+    sregs.ss = dseg; asm ("mov %%ss, %0" : "=rm"(sregs.ss.selector));
+
+    uint32_t gsbase;
+    asm ("mov %%gs:0, %0" : "=r"(gsbase));
+    sregs.gs.base = gsbase;
+
+    sregs.tr.base = reinterpret_cast<ulong>(&*_stack.begin());
+    sregs.tr.type = 11;
+    sregs.tr.s = 0;
+    sregs.tr.present = 1;
+
+    sregs.cr0 = 0x11; /* PE, ET, !PG */
+    sregs.cr4 = 0;
+    sregs.efer = 0;
+    sregs.apic_base = 0xfee00000;
+    _vcpu.set_sregs(sregs);
+}
+
+void vcpu::thunk(vcpu* zis)
+{
+    zis->_guest_func();
+    asm volatile("outb %%al, %%dx" : : "a"(0), "d"(0));
+}
+
+void vcpu::setup_regs()
+{
+    kvm_regs regs = {};
+    regs.rflags = 0x3202;
+    regs.rsp = reinterpret_cast<ulong>(&*_stack.end());
+    regs.rsp &= ~15UL;
+    ulong* sp = reinterpret_cast<ulong *>(regs.rsp);
+    *--sp = reinterpret_cast<ulong>((char*)this);
+    *--sp = 0;
+    regs.rsp = reinterpret_cast<ulong>(sp);
+    regs.rip = reinterpret_cast<ulong>(&vcpu::thunk);
+    printf("rip %llx\n", regs.rip);
+    _vcpu.set_regs(regs);
+}
+
+vcpu::vcpu(kvm::vcpu& vcpu, std::tr1::function<void ()> guest_func,
+           unsigned long stack_size)
+    : _vcpu(vcpu), _guest_func(guest_func), _stack(stack_size)
+{
+    setup_sregs();
+    setup_regs();
+}
+
+}
diff --git a/api/identity.hh b/api/identity.hh
new file mode 100644
index 0000000..7401826
--- /dev/null
+++ b/api/identity.hh
@@ -0,0 +1,28 @@
+#ifndef API_IDENTITY_HH
+#define API_IDENTITY_HH
+
+#include "kvmxx.hh"
+#include <tr1/functional>
+#include <vector>
+
+namespace identity {
+
+void setup_vm(kvm::vm& vm);
+
+class vcpu {
+public:
+    vcpu(kvm::vcpu& vcpu, std::tr1::function<void ()> guest_func,
+	 unsigned long stack_size = 256 * 1024);
+private:
+    static void thunk(vcpu* vcpu);
+    void setup_regs();
+    void setup_sregs();
+private:
+    kvm::vcpu& _vcpu;
+    std::tr1::function<void ()> _guest_func;
+    std::vector<char> _stack;
+};
+
+}
+
+#endif
diff --git a/config-x86-common.mak b/config-x86-common.mak
index d22df17..dde4f67 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -81,3 +81,4 @@ arch_clean:
 	$(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \
 	$(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o
 
+api/%.o: CFLAGS += -m32
-- 
1.7.1

[PATCH kvm-unit-tests v2 07/14] Add sample test using the api test harness

[Avi Kivity]

Call a function setting a global variable.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/api-sample.cc     |   29 +++++++++++++++++++++++++++++
 config-x86-common.mak |    7 +++++++
 2 files changed, 36 insertions(+), 0 deletions(-)
 create mode 100644 api/api-sample.cc

diff --git a/api/api-sample.cc b/api/api-sample.cc
new file mode 100644
index 0000000..8d57c09
--- /dev/null
+++ b/api/api-sample.cc
@@ -0,0 +1,29 @@
+
+#include "api/kvmxx.hh"
+#include "api/identity.hh"
+#include "api/exception.hh"
+#include "stdio.h"
+
+static int global = 0;
+
+static void set_global()
+{
+    global = 1;
+}
+
+int test_main(int ac, char** av)
+{
+    kvm::system system;
+    kvm::vm vm(system);
+    identity::setup_vm(vm);
+    kvm::vcpu vcpu(vm, 0);
+    identity::vcpu thread(vcpu, set_global);
+    vcpu.run();
+    printf("global %d\n", global);
+    return global == 1 ? 0 : 1;
+}
+
+int main(int ac, char** av)
+{
+    return try_main(test_main, ac, av);
+}
diff --git a/config-x86-common.mak b/config-x86-common.mak
index dde4f67..3e8e641 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -32,6 +32,8 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \
                $(TEST_DIR)/hypercall.flat $(TEST_DIR)/sieve.flat \
                $(TEST_DIR)/kvmclock_test.flat
 
+tests-common += api/api-sample
+
 tests_and_config = $(TEST_DIR)/*.flat $(TEST_DIR)/unittests.cfg
 
 test_cases: $(tests-common) $(tests)
@@ -82,3 +84,8 @@ arch_clean:
 	$(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o
 
 api/%.o: CFLAGS += -m32
+
+api/api-sample: LDLIBS += -lstdc++
+api/api-sample: LDFLAGS += -m32
+
+api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o
-- 
1.7.1

[PATCH kvm-unit-tests v2 08/14] api: add support for KVM_SET_USER_MEMORY_REGION flags field

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/kvmxx.cc |    5 +++--
 api/kvmxx.hh |    3 ++-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/api/kvmxx.cc b/api/kvmxx.cc
index ad27907..42e8781 100644
--- a/api/kvmxx.cc
+++ b/api/kvmxx.cc
@@ -150,12 +150,13 @@ vm::vm(system& system)
 {
 }
 
-void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len)
+void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len,
+                           uint32_t flags)
 {
     struct kvm_userspace_memory_region umr;
 
     umr.slot = slot;
-    umr.flags = 0;
+    umr.flags = flags;
     umr.guest_phys_addr = gpa;
     umr.memory_size = len;
     umr.userspace_addr = reinterpret_cast<uint64_t>(addr);
diff --git a/api/kvmxx.hh b/api/kvmxx.hh
index 51dbe7a..958d36f 100644
--- a/api/kvmxx.hh
+++ b/api/kvmxx.hh
@@ -57,7 +57,8 @@ private:
 class vm {
 public:
     explicit vm(system& system);
-    void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len);
+    void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len,
+                           uint32_t flags = 0);
     void set_tss_addr(uint32_t addr);
     system& sys() { return _system; }
 private:
-- 
1.7.1

[PATCH kvm-unit-tests v2 09/14] api: support KVM_GET_DIRTY_LOG ioctl

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/kvmxx.cc |    8 ++++++++
 api/kvmxx.hh |    1 +
 2 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/api/kvmxx.cc b/api/kvmxx.cc
index 42e8781..7ebebb5 100644
--- a/api/kvmxx.cc
+++ b/api/kvmxx.cc
@@ -163,6 +163,14 @@ void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len,
     _fd.ioctlp(KVM_SET_USER_MEMORY_REGION, &umr);
 }
 
+void vm::get_dirty_log(int slot, void *log)
+{
+    struct kvm_dirty_log kdl;
+    kdl.slot = slot;
+    kdl.dirty_bitmap = log;
+    _fd.ioctlp(KVM_GET_DIRTY_LOG, &kdl);
+}
+
 void vm::set_tss_addr(uint32_t addr)
 {
     _fd.ioctl(KVM_SET_TSS_ADDR, addr);
diff --git a/api/kvmxx.hh b/api/kvmxx.hh
index 958d36f..1dcb41d 100644
--- a/api/kvmxx.hh
+++ b/api/kvmxx.hh
@@ -59,6 +59,7 @@ public:
     explicit vm(system& system);
     void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len,
                            uint32_t flags = 0);
+    void get_dirty_log(int slot, void *log);
     void set_tss_addr(uint32_t addr);
     system& sys() { return _system; }
 private:
-- 
1.7.1

[PATCH kvm-unit-tests v2 10/14] api: add memory map management

[Avi Kivity]

Add a class to manage the memory map and a class to represent
a memory slot.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/memmap.cc |   76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 api/memmap.hh |   43 ++++++++++++++++++++++++++++++++
 2 files changed, 119 insertions(+), 0 deletions(-)
 create mode 100644 api/memmap.cc
 create mode 100644 api/memmap.hh

diff --git a/api/memmap.cc b/api/memmap.cc
new file mode 100644
index 0000000..c625852
--- /dev/null
+++ b/api/memmap.cc
@@ -0,0 +1,76 @@
+
+#include "memmap.hh"
+
+mem_slot::mem_slot(mem_map& map, uint64_t gpa, uint64_t size, void* hva)
+    : _map(map)
+    , _slot(map._free_slots.top())
+    , _gpa(gpa)
+    , _size(size)
+    , _hva(hva)
+    , _dirty_log_enabled(false)
+    , _log()
+{
+    map._free_slots.pop();
+    update();
+}
+
+mem_slot::~mem_slot()
+{
+    _size = 0;
+    try {
+        update();
+        _map._free_slots.push(_slot);
+    } catch (...) {
+        // can't do much if we can't undo slot registration - leak the slot
+    }
+}
+
+void mem_slot::set_dirty_logging(bool enabled)
+{
+    if (_dirty_log_enabled != enabled) {
+        _dirty_log_enabled = enabled;
+        if (enabled) {
+            int logsize = ((_size >> 12) + bits_per_word - 1) / bits_per_word;
+            _log.resize(logsize);
+        } else {
+            _log.resize(0);
+        }
+        update();
+    }
+}
+
+void mem_slot::update()
+{
+    uint32_t flags = 0;
+    if (_dirty_log_enabled) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
+    _map._vm.set_memory_region(_slot, _hva, _gpa, _size, flags);
+}
+
+bool mem_slot::dirty_logging() const
+{
+    return _dirty_log_enabled;
+}
+
+void mem_slot::update_dirty_log()
+{
+    _map._vm.get_dirty_log(_slot, &_log[0]);
+}
+
+bool mem_slot::is_dirty(uint64_t gpa) const
+{
+    uint64_t pagenr = (gpa - _gpa) >> 12;
+    ulong wordnr = pagenr / bits_per_word;
+    ulong bit = 1ULL << (pagenr % bits_per_word);
+    return _log[wordnr] & bit;
+}
+
+mem_map::mem_map(kvm::vm& vm)
+    : _vm(vm)
+{
+    int nr_slots = vm.sys().get_extension_int(KVM_CAP_NR_MEMSLOTS);
+    for (int i = 0; i < nr_slots; ++i) {
+        _free_slots.push(i);
+    }
+}
diff --git a/api/memmap.hh b/api/memmap.hh
new file mode 100644
index 0000000..59ec619
--- /dev/null
+++ b/api/memmap.hh
@@ -0,0 +1,43 @@
+#ifndef MEMMAP_HH
+#define MEMMAP_HH
+
+#include "kvmxx.hh"
+#include <stdint.h>
+#include <vector>
+#include <stack>
+
+class mem_map;
+class mem_slot;
+
+class mem_slot {
+public:
+    mem_slot(mem_map& map, uint64_t gpa, uint64_t size, void *hva);
+    ~mem_slot();
+    void set_dirty_logging(bool enabled);
+    bool dirty_logging() const;
+    void update_dirty_log();
+    bool is_dirty(uint64_t gpa) const;
+private:
+    void update();
+private:
+    typedef unsigned long ulong;
+    static const int bits_per_word = sizeof(ulong) * 8;
+    mem_map& _map;
+    int _slot;
+    uint64_t _gpa;
+    uint64_t _size;
+    void *_hva;
+    bool _dirty_log_enabled;
+    std::vector<ulong> _log;
+};
+
+class mem_map {
+public:
+    mem_map(kvm::vm& vm);
+private:
+    kvm::vm& _vm;
+    std::stack<int> _free_slots;
+    friend class mem_slot;
+};
+
+#endif
-- 
1.7.1

[PATCH kvm-unit-tests v2 11/14] Build tests with debug information

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index 85ebd37..b6e8759 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,7 @@ DESTDIR := $(PREFIX)/share/qemu/tests
 .PHONY: arch_clean clean
 
 #make sure env CFLAGS variable is not used
-CFLAGS =
+CFLAGS = -g
 
 libgcc := $(shell $(CC) --print-libgcc-file-name)
 
-- 
1.7.1

[PATCH kvm-unit-tests v2 12/14] api: Add support for creating an identity map with a hole

[Avi Kivity]

The hole may be used for mmio or dirty logging.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/api-sample.cc     |    3 ++-
 api/identity.cc       |   23 +++++++++++++++++++++--
 api/identity.hh       |   17 ++++++++++++++++-
 config-x86-common.mak |    1 +
 4 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/api/api-sample.cc b/api/api-sample.cc
index 8d57c09..524ad7b 100644
--- a/api/api-sample.cc
+++ b/api/api-sample.cc
@@ -15,7 +15,8 @@ int test_main(int ac, char** av)
 {
     kvm::system system;
     kvm::vm vm(system);
-    identity::setup_vm(vm);
+    mem_map memmap(vm);
+    identity::vm ident_vm(vm, memmap);
     kvm::vcpu vcpu(vm, 0);
     identity::vcpu thread(vcpu, set_global);
     vcpu.run();
diff --git a/api/identity.cc b/api/identity.cc
index de52f68..e04231b 100644
--- a/api/identity.cc
+++ b/api/identity.cc
@@ -6,9 +6,28 @@ namespace identity {
 
 typedef unsigned long ulong;
 
-void setup_vm(kvm::vm& vm)
+hole::hole()
+    : address(), size()
 {
-    vm.set_memory_region(0, NULL, 0, 3UL << 30);
+}
+
+hole::hole(void* address, size_t size)
+    : address(address), size(size)
+{
+}
+
+vm::vm(kvm::vm& vm, mem_map& mmap, hole h)
+{
+    uint64_t hole_gpa = reinterpret_cast<uint64_t>(h.address);
+    char* hole_hva = static_cast<char*>(h.address);
+    if (h.address) {
+        _slots.push_back(mem_slot_ptr(new mem_slot(mmap, 0, hole_gpa, NULL)));
+    }
+    uint64_t hole_end = hole_gpa + h.size;
+    uint64_t end = 3U << 30;
+    _slots.push_back(mem_slot_ptr(new mem_slot(mmap, hole_end,
+                                               end - hole_end,
+                                               hole_hva + h.size)));
     vm.set_tss_addr(3UL << 30);
 }
 
diff --git a/api/identity.hh b/api/identity.hh
index 7401826..4491043 100644
--- a/api/identity.hh
+++ b/api/identity.hh
@@ -2,12 +2,27 @@
 #define API_IDENTITY_HH
 
 #include "kvmxx.hh"
+#include "memmap.hh"
 #include <tr1/functional>
+#include <tr1/memory>
 #include <vector>
 
 namespace identity {
 
-void setup_vm(kvm::vm& vm);
+struct hole {
+    hole();
+    hole(void* address, size_t size);
+    void* address;
+    size_t size;
+};
+
+class vm {
+public:
+    vm(kvm::vm& vm, mem_map& mmap, hole address_space_hole = hole());
+private:
+    typedef std::tr1::shared_ptr<mem_slot> mem_slot_ptr;
+    std::vector<mem_slot_ptr> _slots;
+};
 
 class vcpu {
 public:
diff --git a/config-x86-common.mak b/config-x86-common.mak
index 3e8e641..436f4bd 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -89,3 +89,4 @@ api/api-sample: LDLIBS += -lstdc++
 api/api-sample: LDFLAGS += -m32
 
 api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o
+api/api-sample: api/memmap.o
\ No newline at end of file
-- 
1.7.1

[PATCH kvm-unit-tests v2 13/14] Introduce libapi.a to avoid long Makefile recipes

[Avi Kivity]

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 config-x86-common.mak |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/config-x86-common.mak b/config-x86-common.mak
index 436f4bd..ce36cde 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -85,8 +85,10 @@ arch_clean:
 
 api/%.o: CFLAGS += -m32
 
-api/api-sample: LDLIBS += -lstdc++
-api/api-sample: LDFLAGS += -m32
+api/%: LDLIBS += -lstdc++
+api/%: LDFLAGS += -m32
 
-api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o
-api/api-sample: api/memmap.o
\ No newline at end of file
+api/libapi.a: api/kvmxx.o api/identity.o api/exception.o api/memmap.o
+	$(AR) rcs $@ $^
+
+api/api-sample: api/api-sample.o api/libapi.a
\ No newline at end of file
-- 
1.7.1

[PATCH kvm-unit-tests v2 14/14] Add dirty log test

[Avi Kivity]

This checks the failure that was fixed by kernel commit edde99ce0529
("KVM: Write protect memory after slot swap").  Two threads are used;
a guest thread continuously updates a shared variable, which is also
sampled by a host thread that also checks if dirty logging marked it
as dirty.

It detects about 5 million failures with the fix reverted, and 0 failures
with the fix applied.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 api/dirty-log.cc      |   78 +++++++++++++++++++++++++++++++++++++++++++++++++
 config-x86-common.mak |    7 +++-
 2 files changed, 83 insertions(+), 2 deletions(-)
 create mode 100644 api/dirty-log.cc

diff --git a/api/dirty-log.cc b/api/dirty-log.cc
new file mode 100644
index 0000000..1e4ef9e
--- /dev/null
+++ b/api/dirty-log.cc
@@ -0,0 +1,78 @@
+#include "kvmxx.hh"
+#include "memmap.hh"
+#include "identity.hh"
+#include <boost/thread/thread.hpp>
+#include <stdlib.h>
+#include <stdio.h>
+
+namespace {
+
+void delay_loop(unsigned n)
+{
+    for (unsigned i = 0; i < n; ++i) {
+        asm volatile("pause");
+    }
+ }
+
+void write_mem(volatile bool& running, volatile int* shared_var)
+{
+    while (running) {
+        ++*shared_var;
+        delay_loop(1000);
+    }
+}
+
+void check_dirty_log(mem_slot& slot,
+                     volatile bool& running,
+                     volatile int* shared_var,
+                     int& nr_fail)
+{
+    uint64_t shared_var_gpa = reinterpret_cast<uint64_t>(shared_var);
+    slot.set_dirty_logging(true);
+    slot.update_dirty_log();
+    for (int i = 0; i < 10000000; ++i) {
+        int sample1 = *shared_var;
+        delay_loop(600);
+        int sample2 = *shared_var;
+        slot.update_dirty_log();
+        if (!slot.is_dirty(shared_var_gpa) && sample1 != sample2) {
+            ++nr_fail;
+        }
+    }
+    running = false;
+    slot.set_dirty_logging(false);
+}
+
+}
+
+using boost::ref;
+using std::tr1::bind;
+
+int main(int ac, char **av)
+{
+    kvm::system sys;
+    kvm::vm vm(sys);
+    mem_map memmap(vm);
+    void* logged_slot_virt;
+    posix_memalign(&logged_slot_virt, 4096, 4096);
+    int* shared_var = static_cast<int*>(logged_slot_virt);
+    identity::hole hole(logged_slot_virt, 4096);
+    identity::vm ident_vm(vm, memmap, hole);
+    kvm::vcpu vcpu(vm, 0);
+    bool running = true;
+    int nr_fail = 0;
+    mem_slot logged_slot(memmap,
+                         reinterpret_cast<uint64_t>(logged_slot_virt),
+                         4096, logged_slot_virt);
+    boost::thread host_poll_thread(check_dirty_log, ref(logged_slot),
+                                   ref(running),
+                                   ref(shared_var), ref(nr_fail));
+    identity::vcpu guest_write_thread(vcpu,
+                                      bind(write_mem,
+                                           ref(running),
+                                           ref(shared_var)));
+    vcpu.run();
+    host_poll_thread.join();
+    printf("Dirty bitmap failures: %d\n", nr_fail);
+    return nr_fail == 0 ? 0 : 1;
+}
diff --git a/config-x86-common.mak b/config-x86-common.mak
index ce36cde..b5c49f4 100644
--- a/config-x86-common.mak
+++ b/config-x86-common.mak
@@ -33,6 +33,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \
                $(TEST_DIR)/kvmclock_test.flat
 
 tests-common += api/api-sample
+tests-common += api/dirty-log
 
 tests_and_config = $(TEST_DIR)/*.flat $(TEST_DIR)/unittests.cfg
 
@@ -85,10 +86,12 @@ arch_clean:
 
 api/%.o: CFLAGS += -m32
 
-api/%: LDLIBS += -lstdc++
+api/%: LDLIBS += -lstdc++ -lboost_thread-mt -lpthread
 api/%: LDFLAGS += -m32
 
 api/libapi.a: api/kvmxx.o api/identity.o api/exception.o api/memmap.o
 	$(AR) rcs $@ $^
 
-api/api-sample: api/api-sample.o api/libapi.a
\ No newline at end of file
+api/api-sample: api/api-sample.o api/libapi.a
+
+api/dirty-log: api/dirty-log.o api/libapi.a
-- 
1.7.1

Re: [PATCH kvm-unit-tests v2 00/14] API test framework

[Marcelo Tosatti]

On Wed, Dec 15, 2010 at 06:09:29PM +0200, Avi Kivity wrote:
> This patchset adds an API test framework.  Rather than driving kvm from qemu,
> we now have a way of calling the kvm API directly and observing the results.
> We can switch to guest mode and back at will and see any micro effects such
> as the result of executing particular instructions.
> 
> A first test is added, for commit edde99ce0 ("KVM: Write protect memory
> after slot swap").  This would be pretty hard to test for using the current
> qemu-driven tests, but detects 5 million failures in about two minutes on
> my machine.
> 
> As an experiment, the framework is coded in C++.
> 
> Avi Kivity (14):
>   Makefile: add support for C++
>   Improve autodepend includes
>     v2: new
>   Add exception class for kernel errors (errno)
>     v2: new
>   Add try_main() for running a program under an exception handler
>     v2: new
>   Introduce a C++ wrapper for the kvm APIs
>     v2: throw std::exception exceptions, not ints, for libc failures
>         better msr list management
>   Add support for calling a function in guest mode
>     v2: use tr1::function instead of boost::function
>   Add sample test using the api test harness
>   api: add support for KVM_SET_USER_MEMORY_REGION flags field
>     v2: new
>   api: support KVM_GET_DIRTY_LOG ioctl
>     v2: new
>   api: add memory map management
>     v2: new
>   Build tests with debug information
>     v2: new
>   api: Add support for creating an identity map with a hole
>     v2: new
>   Introduce libapi.a to avoid long Makefile recipes
>     v2: new
>   Add dirty log test
>     v2: new
> 
>  Makefile              |    8 +-
>  api/api-sample.cc     |   30 ++++++++
>  api/dirty-log.cc      |   78 ++++++++++++++++++++
>  api/exception.cc      |   33 ++++++++
>  api/exception.hh      |   19 +++++
>  api/identity.cc       |   95 ++++++++++++++++++++++++
>  api/identity.hh       |   43 +++++++++++
>  api/kvmxx.cc          |  194 +++++++++++++++++++++++++++++++++++++++++++++++++
>  api/kvmxx.hh          |   85 +++++++++++++++++++++
>  api/memmap.cc         |   76 +++++++++++++++++++
>  api/memmap.hh         |   43 +++++++++++
>  config-x86-common.mak |   15 ++++-
>  12 files changed, 715 insertions(+), 4 deletions(-)
>  create mode 100644 api/api-sample.cc
>  create mode 100644 api/dirty-log.cc
>  create mode 100644 api/exception.cc
>  create mode 100644 api/exception.hh
>  create mode 100644 api/identity.cc
>  create mode 100644 api/identity.hh
>  create mode 100644 api/kvmxx.cc
>  create mode 100644 api/kvmxx.hh
>  create mode 100644 api/memmap.cc
>  create mode 100644 api/memmap.hh

Applied, thanks.