[Qemu-devel] [RFC 0/5] Support KVM on ARM

[Qemu-devel] [RFC 0/5] Support KVM on ARM

[Peter Maydell]

This patch series is an early RFC for the QEMU patches adding
support for KVM on ARM Cortex-A15 hardware. It's intended for
use with the kernel tree at
 git://github.com/virtualopensystems/linux-kvm-arm.git kvm-a15-v10-stage

There are two aims here:
 * early review for qemu-devel folk
 * resend a complete set of patches to kvmarm, since I've done a
   number of incremental changes and tweaks since Christoffer's
   original QEMU code

These patches depend on various cleanups to KVM and configure
which I've posted in the last couple of weeks:
 configure: Don't implicitly hardcode list of KVM architectures
 update-linux-headers.sh: Pull in asm-generic/kvm_para.h
 update-linux-headers.sh: Don't hard code list of architectures
 kvm-all.c: Move init of irqchip_inject_ioctl out of kvm_irqchip_create()
 kvm: Add documentation comment for kvm_irqchip_in_kernel()
 kvm: Decouple 'GSI routing' from 'kernel irqchip'
 kvm: Decouple 'MSI routing via irqfds' from 'kernel irqchip'
 kvm: Decouple 'irqfds usable' from 'kernel irqchip'
 kvm: Move kvm_allows_irq0_override() to target-i386, fix return type
 kvm: Rename kvm_irqchip_set_irq() to kvm_set_irq()
 kvm: Decouple 'async interrupt delivery' from 'kernel irqchip'

A git branch consisting of qemu master + these preliminary fixes
 + the ARM patches is available here:
 git://git.linaro.org/people/pmaydell/qemu-arm.git kvm-arm

with pointy-clicky version here:
 http://git.linaro.org/gitweb?p=people/pmaydell/qemu-arm.git;a=shortlog;h=refs/heads/kvm-arm

There are still a number of TODOs scattered through the code;
a quick summary:
 * a15mpcore should enforce vgic use (currently not done pending the
   VGIC patches landing in Christoffer's kernel tree)
 * the makefile change for the hw/kvm/arm_gic object is not right
   (see discussions on qemu-devel in the past about how to handle
   "only if architecture foo and KVM" object files)
 * kvm_arch_put/get_registers should drive register set/get from
   the cp15 hashtable
 * we should use an accessor function for c2_mask/base/control
 * breakpoint support is unimplemented
 * vgic register save/load from kernel is unimplemented (no kernel ABI)
 * fpu save/load unimplemented (no kernel ABI yet)
 * tell kernel the A15 peripheral base address (no kernel ABI)
 * the kernel ABI for sending per-CPU interrupts for VGIC vs non-VGIC
   is inconsistent (the former uses a vcpu ioctl, the latter encodes
   cpu number in the irq number), and we should standardise on one
   approach or the other

Christoffer Dall (1):
  ARM: KVM: Add support for KVM on ARM architecture

Peter Maydell (4):
  linux-headers: Add ARM KVM headers (not for upstream)
  hw/arm_gic: Add presave/postload hooks
  hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC
  configure: Enable KVM on ARM

 configure                            |    2 +-
 hw/a15mpcore.c                       |   11 +-
 hw/arm/Makefile.objs                 |    1 +
 hw/arm_gic_common.c                  |   10 ++
 hw/arm_gic_internal.h                |    2 +
 hw/arm_pic.c                         |   28 ++++
 hw/kvm/arm_gic.c                     |  153 +++++++++++++++++++
 linux-headers/asm-arm/kvm.h          |  119 +++++++++++++++
 linux-headers/asm-arm/kvm_para.h     |    1 +
 linux-headers/asm-generic/kvm_para.h |    5 +
 linux-headers/linux/kvm.h            |    3 +
 target-arm/Makefile.objs             |    1 +
 target-arm/cpu.h                     |    1 +
 target-arm/helper.c                  |    2 +-
 target-arm/kvm.c                     |  274 ++++++++++++++++++++++++++++++++++
 15 files changed, 610 insertions(+), 3 deletions(-)
 create mode 100644 hw/kvm/arm_gic.c
 create mode 100644 linux-headers/asm-arm/kvm.h
 create mode 100644 linux-headers/asm-arm/kvm_para.h
 create mode 100644 linux-headers/asm-generic/kvm_para.h
 create mode 100644 target-arm/kvm.c

-- 
1.7.9.5

[Qemu-devel] [RFC 1/5] linux-headers: Add ARM KVM headers (not for upstream)

[Peter Maydell]

This commit adds the ARM KVM headers. This is not to go to QEMU
upstream -- the correct path there is that the KVM code will be
committed to a mainline upstream kernel, and then upstream QEMU
can do a bulk header update from the upstream kernel, which will
allow us to drop this temporary commit.

This commit updates to the KVM ARM kernel tree commit
5196b1b58c, including the changes to the cp15 access
ioctls. It is the result of an update-linux-headers.sh run
with the non-ARM changes removed.
---
 linux-headers/asm-arm/kvm.h          |  119 ++++++++++++++++++++++++++++++++++
 linux-headers/asm-arm/kvm_para.h     |    1 +
 linux-headers/asm-generic/kvm_para.h |    5 ++
 linux-headers/linux/kvm.h            |    3 +
 4 files changed, 128 insertions(+)
 create mode 100644 linux-headers/asm-arm/kvm.h
 create mode 100644 linux-headers/asm-arm/kvm_para.h
 create mode 100644 linux-headers/asm-generic/kvm_para.h

diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h
new file mode 100644
index 0000000..d040a2a
--- /dev/null
+++ b/linux-headers/asm-arm/kvm.h
@@ -0,0 +1,119 @@
+/*
+ * Copyright (C) 2012 - Virtual Open Systems and Columbia University
+ * Author: Christoffer Dall <c.dall@virtualopensystems.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ */
+
+#ifndef __ARM_KVM_H__
+#define __ARM_KVM_H__
+
+#include <asm/types.h>
+
+#define __KVM_HAVE_GUEST_DEBUG
+#define __KVM_HAVE_IRQ_LINE
+
+/*
+ * KVM_IRQ_LINE macros to set/read IRQ/FIQ for specific VCPU index.
+ */
+enum KVM_ARM_IRQ_LINE_TYPE {
+	KVM_ARM_IRQ_LINE = 0,
+	KVM_ARM_FIQ_LINE = 1,
+};
+
+/*
+ * Modes used for short-hand mode determinition in the world-switch code and
+ * in emulation code.
+ *
+ * Note: These indices do NOT correspond to the value of the CPSR mode bits!
+ */
+enum vcpu_mode {
+	MODE_FIQ = 0,
+	MODE_IRQ,
+	MODE_SVC,
+	MODE_ABT,
+	MODE_UND,
+	MODE_USR,
+	MODE_SYS
+};
+
+struct kvm_regs {
+	__u32 regs0_7[8];	/* Unbanked regs. (r0 - r7)	   */
+	__u32 fiq_regs8_12[5];	/* Banked fiq regs. (r8 - r12)	   */
+	__u32 usr_regs8_12[5];	/* Banked usr registers (r8 - r12) */
+	__u32 reg13[6];		/* Banked r13, indexed by MODE_	   */
+	__u32 reg14[6];		/* Banked r13, indexed by MODE_	   */
+	__u32 reg15;
+	__u32 cpsr;
+	__u32 spsr[5];		/* Banked SPSR,  indexed by MODE_  */
+};
+
+/* Supported Processor Types */
+#define KVM_ARM_TARGET_CORTEX_A15	(0xC0F)
+
+struct kvm_vcpu_init {
+	__u32 target;
+	__u32 features[7];
+};
+
+struct kvm_sregs {
+};
+
+struct kvm_fpu {
+};
+
+struct kvm_guest_debug_arch {
+};
+
+struct kvm_debug_exit_arch {
+};
+
+struct kvm_sync_regs {
+};
+
+struct kvm_arch_memory_slot {
+};
+
+/* Based on x86, but we use KVM_GET_VCPU_MSR_INDEX_LIST. */
+struct kvm_msr_entry {
+	__u32 index;
+	__u32 reserved;
+	__u64 data;
+};
+
+/* for KVM_GET_MSRS and KVM_SET_MSRS */
+struct kvm_msrs {
+	__u32 nmsrs; /* number of msrs in entries */
+	__u32 pad;
+
+	struct kvm_msr_entry entries[0];
+};
+
+/* for KVM_VCPU_GET_MSR_INDEX_LIST */
+struct kvm_msr_list {
+	__u32 nmsrs; /* number of msrs in entries */
+	__u32 indices[0];
+};
+
+/* If you need to interpret the index values, here's the key. */
+#define KVM_ARM_MSR_COPROC_MASK		0xFFFF0000
+#define KVM_ARM_MSR_64_BIT_MASK		0x00008000
+#define KVM_ARM_MSR_64_OPC1_MASK	0x000000F0
+#define KVM_ARM_MSR_64_CRM_MASK		0x0000000F
+#define KVM_ARM_MSR_32_CRM_MASK		0x0000000F
+#define KVM_ARM_MSR_32_OPC2_MASK	0x00000070
+#define KVM_ARM_MSR_32_CRN_MASK		0x00000780
+#define KVM_ARM_MSR_32_OPC1_MASK	0x00003800
+
+#endif /* __ARM_KVM_H__ */
diff --git a/linux-headers/asm-arm/kvm_para.h b/linux-headers/asm-arm/kvm_para.h
new file mode 100644
index 0000000..14fab8f
--- /dev/null
+++ b/linux-headers/asm-arm/kvm_para.h
@@ -0,0 +1 @@
+#include <asm-generic/kvm_para.h>
diff --git a/linux-headers/asm-generic/kvm_para.h b/linux-headers/asm-generic/kvm_para.h
new file mode 100644
index 0000000..63df88b
--- /dev/null
+++ b/linux-headers/asm-generic/kvm_para.h
@@ -0,0 +1,5 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+#endif
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 5a9d4e3..0afbbab 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -111,6 +111,7 @@ struct kvm_irq_level {
 	 * ACPI gsi notion of irq.
 	 * For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
 	 * For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
+	 * For ARM: IRQ: irq = (2*vcpu_index). FIQ: irq = (2*vcpu_indx + 1).
 	 */
 	union {
 		__u32 irq;
@@ -901,6 +902,8 @@ struct kvm_s390_ucas_mapping {
 #define KVM_SET_ONE_REG		  _IOW(KVMIO,  0xac, struct kvm_one_reg)
 /* VM is being stopped by host */
 #define KVM_KVMCLOCK_CTRL	  _IO(KVMIO,   0xad)
+#define KVM_ARM_VCPU_INIT	  _IOW(KVMIO,  0xae, struct kvm_vcpu_init)
+#define KVM_VCPU_GET_MSR_INDEX_LIST    _IOWR(KVMIO, 0xaf, struct kvm_msr_list)
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
-- 
1.7.9.5

[Qemu-devel] [RFC 2/5] ARM: KVM: Add support for KVM on ARM architecture

[Peter Maydell]

From: Christoffer Dall <cdall@cs.columbia.edu>

Add basic support for KVM on ARM architecture.

Signed-off-by: Christoffer Dall <cdall@cs.columbia.edu>
[Rusty: updates to use KVM_ARM_VCPU_INIT, KVM_GET/SET_MSRS]
Signed-off-by: Rusty Russell <rusty.russell@linaro.org>
[PMM: Minor tweaks and code cleanup]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm_pic.c             |   28 +++++
 target-arm/Makefile.objs |    1 +
 target-arm/cpu.h         |    1 +
 target-arm/helper.c      |    2 +-
 target-arm/kvm.c         |  274 ++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 305 insertions(+), 1 deletion(-)
 create mode 100644 target-arm/kvm.c

diff --git a/hw/arm_pic.c b/hw/arm_pic.c
index ffb4d41..950e6c4 100644
--- a/hw/arm_pic.c
+++ b/hw/arm_pic.c
@@ -9,6 +9,7 @@
 
 #include "hw.h"
 #include "arm-misc.h"
+#include "kvm.h"
 
 /* Input 0 is IRQ and input 1 is FIQ.  */
 static void arm_pic_cpu_handler(void *opaque, int irq, int level)
@@ -34,7 +35,34 @@ static void arm_pic_cpu_handler(void *opaque, int irq, int level)
     }
 }
 
+#ifdef CONFIG_KVM
+static void kvm_arm_pic_cpu_handler(void *opaque, int irq, int level)
+{
+    ARMCPU *cpu = opaque;
+    CPUARMState *env = &cpu->env;
+    int kvm_irq;
+
+    switch (irq) {
+    case ARM_PIC_CPU_IRQ:
+        kvm_irq = KVM_ARM_IRQ_LINE;
+        break;
+    case ARM_PIC_CPU_FIQ:
+        kvm_irq = KVM_ARM_FIQ_LINE;
+        break;
+    default:
+        hw_error("kvm_arm_pic_cpu_handler: Bad interrupt line %d\n", irq);
+    }
+    kvm_irq |= (env->cpu_index << 1);
+    kvm_set_irq(kvm_state, kvm_irq, level ? 1 : 0);
+}
+#endif
+
 qemu_irq *arm_pic_init_cpu(ARMCPU *cpu)
 {
+#ifdef CONFIG_KVM
+    if (kvm_enabled()) {
+        return qemu_allocate_irqs(kvm_arm_pic_cpu_handler, cpu, 2);
+    }
+#endif
     return qemu_allocate_irqs(arm_pic_cpu_handler, cpu, 2);
 }
diff --git a/target-arm/Makefile.objs b/target-arm/Makefile.objs
index f447c4f..f906d20 100644
--- a/target-arm/Makefile.objs
+++ b/target-arm/Makefile.objs
@@ -1,5 +1,6 @@
 obj-y += arm-semi.o
 obj-$(CONFIG_SOFTMMU) += machine.o
+obj-$(CONFIG_KVM) += kvm.o
 obj-y += translate.o op_helper.o helper.o cpu.o
 obj-y += neon_helper.o iwmmxt_helper.o
 
diff --git a/target-arm/cpu.h b/target-arm/cpu.h
index 191895c..5194bad 100644
--- a/target-arm/cpu.h
+++ b/target-arm/cpu.h
@@ -236,6 +236,7 @@ ARMCPU *cpu_arm_init(const char *cpu_model);
 void arm_translate_init(void);
 int cpu_arm_exec(CPUARMState *s);
 void do_interrupt(CPUARMState *);
+int bank_number(CPUARMState *env, int mode);
 void switch_mode(CPUARMState *, int);
 uint32_t do_arm_semihosting(CPUARMState *env);
 
diff --git a/target-arm/helper.c b/target-arm/helper.c
index 5727da2..632a366 100644
--- a/target-arm/helper.c
+++ b/target-arm/helper.c
@@ -1619,7 +1619,7 @@ uint32_t HELPER(get_r13_banked)(CPUARMState *env, uint32_t mode)
 #else
 
 /* Map CPU modes onto saved register banks.  */
-static inline int bank_number(CPUARMState *env, int mode)
+int bank_number(CPUARMState *env, int mode)
 {
     switch (mode) {
     case ARM_CPU_MODE_USR:
diff --git a/target-arm/kvm.c b/target-arm/kvm.c
new file mode 100644
index 0000000..b9abee0
--- /dev/null
+++ b/target-arm/kvm.c
@@ -0,0 +1,274 @@
+/*
+ * ARM implementation of KVM hooks
+ *
+ * Copyright Christoffer Dall 2009-2010
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/kvm.h>
+
+#include "qemu-common.h"
+#include "qemu-timer.h"
+#include "sysemu.h"
+#include "kvm.h"
+#include "cpu.h"
+#include "device_tree.h"
+#include "hw/arm-misc.h"
+
+const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
+    KVM_CAP_LAST_INFO
+};
+
+int kvm_arch_init(KVMState *s)
+{
+    /* For ARM interrupt delivery is always asynchronous,
+     * whether we are using an in-kernel VGIC or not.
+     */
+    kvm_async_interrupts_allowed = true;
+    return 0;
+}
+
+int kvm_arch_init_vcpu(CPUARMState *env)
+{
+    struct kvm_vcpu_init init;
+
+    init.target = KVM_ARM_TARGET_CORTEX_A15;
+    memset(init.features, 0, sizeof(init.features));
+    return kvm_vcpu_ioctl(env, KVM_ARM_VCPU_INIT, &init);
+}
+
+#define MSR32_INDEX_OF(coproc, crn, opc1, crm, opc2) \
+    (((coproc)<<16) | ((opc1)<<11) | ((crn)<<7) | ((opc2)<<4) | (crm))
+
+int kvm_arch_put_registers(CPUARMState *env, int level)
+{
+    struct kvm_regs regs;
+    int mode, bn;
+    struct cp15 {
+        struct kvm_msrs hdr;
+        struct kvm_msr_entry e[2];
+    } cp15;
+    int ret;
+
+    ret = kvm_vcpu_ioctl(env, KVM_GET_REGS, &regs);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /* We make sure the banked regs are properly set */
+    mode = env->uncached_cpsr & CPSR_M;
+    bn = bank_number(env, mode);
+    if (mode == ARM_CPU_MODE_FIQ) {
+        memcpy(env->fiq_regs, env->regs + 8, 5 * sizeof(uint32_t));
+    } else {
+        memcpy(env->usr_regs, env->regs + 8, 5 * sizeof(uint32_t));
+    }
+    env->banked_r13[bn] = env->regs[13];
+    env->banked_r14[bn] = env->regs[14];
+    env->banked_spsr[bn] = env->spsr;
+
+    /* Now we can safely copy stuff down to the kernel */
+    memcpy(regs.regs0_7, env->regs, sizeof(uint32_t) * 8);
+    memcpy(regs.usr_regs8_12, env->usr_regs, sizeof(uint32_t) * 5);
+    memcpy(regs.fiq_regs8_12, env->fiq_regs, sizeof(uint32_t) * 5);
+    regs.reg13[MODE_FIQ] = env->banked_r13[5];
+    regs.reg13[MODE_IRQ] = env->banked_r13[4];
+    regs.reg13[MODE_SVC] = env->banked_r13[1];
+    regs.reg13[MODE_ABT] = env->banked_r13[2];
+    regs.reg13[MODE_UND] = env->banked_r13[3];
+    regs.reg13[MODE_USR] = env->banked_r13[0];
+    regs.reg14[MODE_FIQ] = env->banked_r14[5];
+    regs.reg14[MODE_IRQ] = env->banked_r14[4];
+    regs.reg14[MODE_SVC] = env->banked_r14[1];
+    regs.reg14[MODE_ABT] = env->banked_r14[2];
+    regs.reg14[MODE_UND] = env->banked_r14[3];
+    regs.reg14[MODE_USR] = env->banked_r14[0];
+    regs.reg15 = env->regs[15];
+    regs.cpsr = cpsr_read(env);
+    regs.spsr[MODE_FIQ] = env->banked_spsr[5];
+    regs.spsr[MODE_IRQ] = env->banked_spsr[4];
+    regs.spsr[MODE_SVC] = env->banked_spsr[1];
+    regs.spsr[MODE_ABT] = env->banked_spsr[2];
+    regs.spsr[MODE_UND] = env->banked_spsr[3];
+
+    cp15.hdr.nmsrs = ARRAY_SIZE(cp15.e);
+    cp15.e[0].index = MSR32_INDEX_OF(15, 0, 0, 0, 0); /* MIDR */
+    cp15.e[0].data = env->cp15.c0_cpuid;
+    cp15.e[1].index = MSR32_INDEX_OF(15, 1, 0, 0, 0); /* SCTLR */
+    cp15.e[1].data = env->cp15.c1_sys;
+
+    ret = kvm_vcpu_ioctl(env, KVM_SET_REGS, &regs);
+    if (ret == 0) {
+        ret = kvm_vcpu_ioctl(env, KVM_SET_MSRS, &cp15);
+    }
+    return ret;
+}
+
+int kvm_arch_get_registers(CPUARMState *env)
+{
+    struct kvm_regs regs;
+    int mode, bn;
+    int32_t ret;
+    struct cp15 {
+        struct kvm_msrs hdr;
+        struct kvm_msr_entry e[6];
+    } cp15;
+
+
+    ret = kvm_vcpu_ioctl(env, KVM_GET_REGS, &regs);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /* First, let's transfer the banked state */
+    cpsr_write(env, regs.cpsr, 0xFFFFFFFF);
+    memcpy(env->regs, regs.regs0_7, sizeof(uint32_t) * 8);
+    memcpy(env->usr_regs, regs.usr_regs8_12, sizeof(uint32_t) * 5);
+    memcpy(env->fiq_regs, regs.fiq_regs8_12, sizeof(uint32_t) * 5);
+
+    env->banked_r13[5] = regs.reg13[MODE_FIQ];
+    env->banked_r13[4] = regs.reg13[MODE_IRQ];
+    env->banked_r13[1] = regs.reg13[MODE_SVC];
+    env->banked_r13[2] = regs.reg13[MODE_ABT];
+    env->banked_r13[3] = regs.reg13[MODE_UND];
+    env->banked_r13[0] = regs.reg13[MODE_USR];
+    env->banked_r14[5] = regs.reg14[MODE_FIQ];
+    env->banked_r14[4] = regs.reg14[MODE_IRQ];
+    env->banked_r14[1] = regs.reg14[MODE_SVC];
+    env->banked_r14[2] = regs.reg14[MODE_ABT];
+    env->banked_r14[3] = regs.reg14[MODE_UND];
+    env->banked_r14[0] = regs.reg14[MODE_USR];
+    env->regs[15] = regs.reg15;
+    env->banked_spsr[5] = regs.spsr[MODE_FIQ];
+    env->banked_spsr[4] = regs.spsr[MODE_IRQ];
+    env->banked_spsr[1] = regs.spsr[MODE_SVC];
+    env->banked_spsr[2] = regs.spsr[MODE_ABT];
+    env->banked_spsr[3] = regs.spsr[MODE_UND];
+
+    /* We make sure the current mode regs are properly set */
+    mode = env->uncached_cpsr & CPSR_M;
+    bn = bank_number(env, mode);
+    if (mode == ARM_CPU_MODE_FIQ) {
+        memcpy(env->regs + 8, env->fiq_regs, 5 * sizeof(uint32_t));
+    } else {
+        memcpy(env->regs + 8, env->usr_regs, 5 * sizeof(uint32_t));
+    }
+    env->regs[13] = env->banked_r13[bn];
+    env->regs[14] = env->banked_r14[bn];
+    env->spsr = env->banked_spsr[bn];
+
+    /* TODO: investigate automatically getting all registers
+     * we know about via the ARMCPU cp_regs hashtable.
+     */
+    cp15.hdr.nmsrs = ARRAY_SIZE(cp15.e);
+    cp15.e[0].index = MSR32_INDEX_OF(15, 0, 0, 0, 0); /* MIDR */
+    cp15.e[1].index = MSR32_INDEX_OF(15, 1, 0, 0, 0); /* SCTLR */
+    cp15.e[2].index = MSR32_INDEX_OF(15, 2, 0, 0, 0); /* TTBR0 */
+    cp15.e[3].index = MSR32_INDEX_OF(15, 2, 0, 0, 1); /* TTBR1 */
+    cp15.e[4].index = MSR32_INDEX_OF(15, 2, 0, 0, 2); /* TTBCR */
+    cp15.e[5].index = MSR32_INDEX_OF(15, 3, 0, 0, 0); /* DACR */
+
+    ret = kvm_vcpu_ioctl(env, KVM_GET_MSRS, &cp15);
+    if (ret < 0) {
+        return ret;
+    }
+
+    env->cp15.c1_sys = cp15.e[1].data;
+    env->cp15.c2_base0 = cp15.e[2].data;
+    env->cp15.c2_base1 = cp15.e[3].data;
+
+    /* This is ugly, but necessary for GDB compatibility
+     * TODO: do this via an access function.
+     */
+    env->cp15.c2_control = cp15.e[4].data;
+    env->cp15.c2_mask = ~(((uint32_t)0xffffffffu) >> cp15.e[4].data);
+    env->cp15.c2_base_mask = ~((uint32_t)0x3fffu >> cp15.e[4].data);
+
+    env->cp15.c3 = cp15.e[5].data;
+    return 0;
+}
+
+void kvm_arch_pre_run(CPUARMState *env, struct kvm_run *run)
+{
+}
+
+void kvm_arch_post_run(CPUARMState *env, struct kvm_run *run)
+{
+}
+
+int kvm_arch_handle_exit(CPUARMState *env, struct kvm_run *run)
+{
+    int ret = 0;
+
+    return ret;
+}
+
+void kvm_arch_reset_vcpu(CPUARMState *env)
+{
+}
+
+bool kvm_arch_stop_on_emulation_error(CPUARMState *env)
+{
+    return true;
+}
+
+int kvm_arch_process_async_events(CPUARMState *env)
+{
+    return 0;
+}
+
+int kvm_arch_on_sigbus_vcpu(CPUARMState *env, int code, void *addr)
+{
+    return 1;
+}
+
+int kvm_arch_on_sigbus(int code, void *addr)
+{
+    return 1;
+}
+
+void kvm_arch_update_guest_debug(CPUARMState *env, struct kvm_guest_debug *dbg)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+}
+
+int kvm_arch_insert_sw_breakpoint(CPUARMState *env,
+                                  struct kvm_sw_breakpoint *bp)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_insert_hw_breakpoint(target_ulong addr,
+                                  target_ulong len, int type)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_remove_hw_breakpoint(target_ulong addr,
+                                  target_ulong len, int type)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_remove_sw_breakpoint(CPUARMState *env,
+                                  struct kvm_sw_breakpoint *bp)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+void kvm_arch_remove_all_hw_breakpoints(void)
+{
+    fprintf(stderr, "%s: not implemented\n", __func__);
+}
-- 
1.7.9.5

[Qemu-devel] [RFC 3/5] hw/arm_gic: Add presave/postload hooks

[Peter Maydell]

Add presave/postload hooks to the ARM GIC common base class.
These will be used by the KVM in-kernel GIC subclass to sync
state between kernel and userspace when migrating.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm_gic_common.c   |   10 ++++++++++
 hw/arm_gic_internal.h |    2 ++
 2 files changed, 12 insertions(+)

diff --git a/hw/arm_gic_common.c b/hw/arm_gic_common.c
index 360e782..d972755 100644
--- a/hw/arm_gic_common.c
+++ b/hw/arm_gic_common.c
@@ -23,9 +23,14 @@
 static void gic_save(QEMUFile *f, void *opaque)
 {
     gic_state *s = (gic_state *)opaque;
+    ARMGICCommonClass *c = ARM_GIC_COMMON_GET_CLASS(s);
     int i;
     int j;
 
+    if (c->pre_save) {
+        c->pre_save(s);
+    }
+
     qemu_put_be32(f, s->enabled);
     for (i = 0; i < s->num_cpu; i++) {
         qemu_put_be32(f, s->cpu_enabled[i]);
@@ -57,6 +62,7 @@ static void gic_save(QEMUFile *f, void *opaque)
 static int gic_load(QEMUFile *f, void *opaque, int version_id)
 {
     gic_state *s = (gic_state *)opaque;
+    ARMGICCommonClass *c = ARM_GIC_COMMON_GET_CLASS(s);
     int i;
     int j;
 
@@ -91,6 +97,10 @@ static int gic_load(QEMUFile *f, void *opaque, int version_id)
         s->irq_state[i].trigger = qemu_get_byte(f);
     }
 
+    if (c->post_load) {
+        c->post_load(s);
+    }
+
     return 0;
 }
 
diff --git a/hw/arm_gic_internal.h b/hw/arm_gic_internal.h
index db4fad5..183dca6 100644
--- a/hw/arm_gic_internal.h
+++ b/hw/arm_gic_internal.h
@@ -118,6 +118,8 @@ void gic_init_irqs_and_distributor(gic_state *s, int num_irq);
 
 typedef struct ARMGICCommonClass {
     SysBusDeviceClass parent_class;
+    void (*pre_save)(gic_state *s);
+    void (*post_load)(gic_state *s);
 } ARMGICCommonClass;
 
 #define TYPE_ARM_GIC "arm_gic"
-- 
1.7.9.5

[Qemu-devel] [RFC 4/5] hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC

[Peter Maydell]

Implement support for using the KVM in-kernel GIC for ARM.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/a15mpcore.c       |   11 +++-
 hw/arm/Makefile.objs |    1 +
 hw/kvm/arm_gic.c     |  153 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 164 insertions(+), 1 deletion(-)
 create mode 100644 hw/kvm/arm_gic.c

diff --git a/hw/a15mpcore.c b/hw/a15mpcore.c
index fc0a02a..e075849 100644
--- a/hw/a15mpcore.c
+++ b/hw/a15mpcore.c
@@ -19,6 +19,7 @@
  */
 
 #include "sysbus.h"
+#include "kvm.h"
 
 /* A15MP private memory region.  */
 
@@ -41,7 +42,15 @@ static int a15mp_priv_init(SysBusDevice *dev)
     A15MPPrivState *s = FROM_SYSBUS(A15MPPrivState, dev);
     SysBusDevice *busdev;
 
-    s->gic = qdev_create(NULL, "arm_gic");
+    /* TODO when the VGIC patches make it to Christoffer's kernel
+     * tree we can make !kvm_irqchip_in_kernel() a fatal error.
+     */
+    if (kvm_irqchip_in_kernel()) {
+        s->gic = qdev_create(NULL, "kvm-arm_gic");
+    } else {
+        s->gic = qdev_create(NULL, "arm_gic");
+    }
+
     qdev_prop_set_uint32(s->gic, "num-cpu", s->num_cpu);
     qdev_prop_set_uint32(s->gic, "num-irq", s->num_irq);
     qdev_prop_set_uint32(s->gic, "revision", 2);
diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
index c413780..881fffd 100644
--- a/hw/arm/Makefile.objs
+++ b/hw/arm/Makefile.objs
@@ -39,5 +39,6 @@ obj-y += imx_serial.o imx_ccm.o imx_timer.o imx_avic.o
 obj-y += kzm.o
 obj-y += pl041.o lm4549.o
 obj-$(CONFIG_FDT) += ../device_tree.o
+obj-$(CONFIG_KVM) += kvm/arm_gic.o
 
 obj-y := $(addprefix ../,$(obj-y))
diff --git a/hw/kvm/arm_gic.c b/hw/kvm/arm_gic.c
new file mode 100644
index 0000000..4535f90
--- /dev/null
+++ b/hw/kvm/arm_gic.c
@@ -0,0 +1,153 @@
+/*
+ * ARM Generic Interrupt Controller using KVM in-kernel support
+ *
+ * Copyright (c) 2012 Linaro Limited
+ * Written by Peter Maydell
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "hw/sysbus.h"
+#include "kvm.h"
+#include "hw/arm_gic_internal.h"
+
+#define TYPE_KVM_ARM_GIC "kvm-arm_gic"
+#define KVM_ARM_GIC(obj) \
+     OBJECT_CHECK(gic_state, (obj), TYPE_KVM_ARM_GIC)
+#define KVM_ARM_GIC_CLASS(klass) \
+     OBJECT_CLASS_CHECK(KVMARMGICClass, (klass), TYPE_KVM_ARM_GIC)
+#define KVM_ARM_GIC_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(KVMARMGICClass, (obj), TYPE_KVM_ARM_GIC)
+
+typedef struct KVMARMGICClass {
+    ARMGICCommonClass parent_class;
+    int (*parent_init)(SysBusDevice *dev);
+    void (*parent_reset)(DeviceState *dev);
+} KVMARMGICClass;
+
+static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
+{
+    /* Meaning of the 'irq' parameter:
+     *  [0..N-1] : external interrupts
+     *  [N..N+31] : PPI (internal) interrupts for CPU 0
+     *  [N+32..N+63] : PPI (internal interrupts for CPU 1
+     *  ...
+     */
+    gic_state *s = (gic_state *)opaque;
+
+
+    if (irq < (s->num_irq - GIC_INTERNAL)) {
+        /* External interrupt number 'irq' */
+        kvm_set_irq(kvm_state, irq + GIC_INTERNAL, !!level);
+    } else {
+        struct kvm_irq_level irq_level;
+        int cpu;
+        irq -= (s->num_irq - GIC_INTERNAL);
+        cpu = irq / GIC_INTERNAL;
+        irq %= GIC_INTERNAL;
+        /* Internal interrupt 'irq' for CPU 'cpu' */
+        irq_level.irq = irq;
+        irq_level.level = !!level;
+        kvm_vcpu_ioctl(qemu_get_cpu(cpu), KVM_IRQ_LINE, &irq_level);
+    }
+}
+
+static void kvm_arm_gic_put(gic_state *s)
+{
+    /* TODO: there isn't currently a kernel interface to set the GIC state */
+}
+
+static void kvm_arm_gic_get(gic_state *s)
+{
+    /* TODO: there isn't currently a kernel interface to get the GIC state */
+}
+
+static void kvm_arm_gic_reset(DeviceState *dev)
+{
+    gic_state *s = ARM_GIC_COMMON(dev);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_GET_CLASS(s);
+    kgc->parent_reset(dev);
+    kvm_arm_gic_put(s);
+}
+
+static int kvm_arm_gic_init(SysBusDevice *dev)
+{
+    /* Device instance init function for the GIC sysbus device */
+    int i;
+    gic_state *s = FROM_SYSBUS(gic_state, dev);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_GET_CLASS(s);
+
+    kgc->parent_init(dev);
+
+    i = s->num_irq - GIC_INTERNAL;
+    /* For the GIC, also expose incoming GPIO lines for PPIs for each CPU.
+     * GPIO array layout is thus:
+     *  [0..N-1] SPIs
+     *  [N..N+31] PPIs for CPU 0
+     *  [N+32..N+63] PPIs for CPU 1
+     *   ...
+     */
+    i += (GIC_INTERNAL * s->num_cpu);
+    qdev_init_gpio_in(&s->busdev.qdev, kvm_arm_gic_set_irq, i);
+    /* We never use our outbound IRQ lines but provide them so that
+     * we maintain the same interface as the non-KVM GIC.
+     */
+    for (i = 0; i < s->num_cpu; i++) {
+        sysbus_init_irq(&s->busdev, &s->parent_irq[i]);
+    }
+    /* Distributor */
+    memory_region_init_reservation(&s->iomem, "kvm-gic_dist", 0x1000);
+    sysbus_init_mmio(dev, &s->iomem);
+    /* CPU interface for current core. Unlike arm_gic, we don't
+     * provide the "interface for core #N" memory regions, because
+     * cores with a VGIC don't have those.
+     */
+    memory_region_init_reservation(&s->cpuiomem[0], "kvm-gic_cpu", 0x1000);
+    sysbus_init_mmio(dev, &s->cpuiomem[0]);
+    /* TODO: we should tell the kernel at some point the address
+     * of the private peripheral base. However we don't currently have
+     * any convenient infrastructure to do that, and in any case the
+     * kernel doesn't yet implement an ioctl to let us tell it.
+     */
+    return 0;
+}
+
+static void kvm_arm_gic_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    SysBusDeviceClass *sbc = SYS_BUS_DEVICE_CLASS(klass);
+    ARMGICCommonClass *agcc = ARM_GIC_COMMON_CLASS(klass);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_CLASS(klass);
+    agcc->pre_save = kvm_arm_gic_get;
+    agcc->post_load = kvm_arm_gic_put;
+    kgc->parent_init = sbc->init;
+    kgc->parent_reset = dc->reset;
+    sbc->init = kvm_arm_gic_init;
+    dc->reset = kvm_arm_gic_reset;
+    dc->no_user = 1;
+}
+
+static TypeInfo arm_gic_info = {
+    .name = TYPE_KVM_ARM_GIC,
+    .parent = TYPE_ARM_GIC_COMMON,
+    .instance_size = sizeof(gic_state),
+    .class_init = kvm_arm_gic_class_init,
+};
+
+static void arm_gic_register_types(void)
+{
+    type_register_static(&arm_gic_info);
+}
+
+type_init(arm_gic_register_types)
-- 
1.7.9.5

[Qemu-devel] [RFC 5/5] configure: Enable KVM on ARM

[Peter Maydell]

Enable KVM on ARM hosts, now that all the necessary components
for it exist.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 configure |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index b9a0b27..f7a825a 100755
--- a/configure
+++ b/configure
@@ -3797,7 +3797,7 @@ case "$target_arch2" in
     echo "CONFIG_NO_XEN=y" >> $config_target_mak
 esac
 case "$target_arch2" in
-  i386|x86_64|ppcemb|ppc|ppc64|s390x)
+  arm|i386|x86_64|ppcemb|ppc|ppc64|s390x)
     # Make sure the target and host cpus are compatible
     if test "$kvm" = "yes" -a "$target_softmmu" = "yes" -a \
       \( "$target_arch2" = "$cpu" -o \
-- 
1.7.9.5

Re: [Qemu-devel] [RFC 5/5] configure: Enable KVM on ARM

[Avi Kivity]

On 08/09/2012 07:33 PM, Peter Maydell wrote:
> Enable KVM on ARM hosts, now that all the necessary components
> for it exist.
> 
>  esac
>  case "$target_arch2" in
> -  i386|x86_64|ppcemb|ppc|ppc64|s390x)
> +  arm|i386|x86_64|ppcemb|ppc|ppc64|s390x)
>      # Make sure the target and host cpus are compatible

Nice to see the list grow... who's next?


-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [kvmarm] [RFC 5/5] configure: Enable KVM on ARM

[Alexander Graf]

On 12.08.2012, at 11:55, Avi Kivity wrote:

> On 08/09/2012 07:33 PM, Peter Maydell wrote:
>> Enable KVM on ARM hosts, now that all the necessary components
>> for it exist.
>> 
>> esac
>> case "$target_arch2" in
>> -  i386|x86_64|ppcemb|ppc|ppc64|s390x)
>> +  arm|i386|x86_64|ppcemb|ppc|ppc64|s390x)
>>     # Make sure the target and host cpus are compatible
> 
> Nice to see the list grow... who's next?

arm64 :)


Alex