[RFC v5 0/8] QEMU: Support KVM on ARM

[RFC v5 0/8] QEMU: Support KVM on ARM

[Peter Maydell]

Round 5 of the QEMU patches to support KVM for
ARM on Cortex-A15 hardware. It's intended for use with
the kernel tree at
 git://github.com/virtualopensystems/linux-kvm-arm.git kvm-arm-v17-vgic-timers

Still RFC pending the kernel patches actually being accepted
upstream...

Changes v4 to v5:
 * no changes of consequence, but rebased on current qemu master
   and resynced with v17 of the kernel patches (minor ABI changes
   caused by other people getting in first for ioctl numbers etc,
   and insertion of "_ARM_" in KVM_ARM_SET_DEVICE_ADDRESS related
   constants
Changes v3 to v4:
 * minor updates to match kernel ABI changes (ID field in
   kvm_device_address is now 64 bits, core register offsets now
   changed due to use of pt_regs struct)
 * squashed the two 'update kernel headers' patches, since the
   plan is for vgic support to go upstream at the same time as
   the baseline kernel patchset
 * added a new patch 8 which adds ARM to the list of Linux archs
   which prefer 2MB alignment so they can use transparent hugepages

Changes v2 to v3:
 * applied various minor tweaks suggested during review of v2
 * rebased on master, resynced with kernel headers for v13
 * new patch 6 which uses a MemoryListener to track where the
   VGIC memory regions are mapped, so we can tell the kernel
   where they live in the memory map (via new ioctl
   KVM_SET_DEVICE_ADDRESS)

Git tree available at
 git://git.linaro.org/people/pmaydell/qemu-arm.git kvm-arm-v17
with pointy-clicky interface at
 http://git.linaro.org/gitweb?p=people/pmaydell/qemu-arm.git;a=shortlog;h=refs/heads/kvm-arm-v17


Peter Maydell (8):
  oslib-posix: Align to permit transparent hugepages on ARM Linux
  linux-headers: Add ARM KVM headers (not for upstream)
  ARM: KVM: Add support for KVM on ARM architecture
  ARM KVM: save and load VFP registers from kernel
  hw/arm_gic: Add presave/postload hooks
  target-arm: Use MemoryListener to identify GIC base address for KVM
  hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC
  configure: Enable KVM on ARM

 configure                            |    2 +-
 hw/a15mpcore.c                       |    8 +-
 hw/arm/Makefile.objs                 |    1 +
 hw/arm_gic_common.c                  |   10 +
 hw/arm_gic_internal.h                |    2 +
 hw/arm_pic.c                         |   26 ++
 hw/kvm/arm_gic.c                     |  169 ++++++++++++
 linux-headers/asm-arm/kvm.h          |  180 +++++++++++++
 linux-headers/asm-arm/kvm_para.h     |    1 +
 linux-headers/asm-generic/kvm_para.h |    4 +
 linux-headers/linux/kvm.h            |   17 ++
 target-arm/Makefile.objs             |    1 +
 target-arm/cpu.h                     |    1 +
 target-arm/helper.c                  |    2 +-
 target-arm/kvm.c                     |  487 ++++++++++++++++++++++++++++++++++
 target-arm/kvm_arm.h                 |   32 +++
 util/oslib-posix.c                   |    2 +-
 17 files changed, 941 insertions(+), 4 deletions(-)
 create mode 100644 hw/kvm/arm_gic.c
 create mode 100644 linux-headers/asm-arm/kvm.h
 create mode 100644 linux-headers/asm-arm/kvm_para.h
 create mode 100644 linux-headers/asm-generic/kvm_para.h
 create mode 100644 target-arm/kvm.c
 create mode 100644 target-arm/kvm_arm.h

-- 
1.7.9.5

[RFC v5 1/8] oslib-posix: Align to permit transparent hugepages on ARM Linux

[Peter Maydell]

ARM Linux (like x86-64 Linux) can use transparent hugepages for
KVM if memory blocks are 2MiB aligned; set QEMU_VMALLOC_ALIGN
accordingly.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 util/oslib-posix.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 4f5ec67..4ad9940 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -35,7 +35,7 @@
 extern int daemon(int, int);
 #endif
 
-#if defined(__linux__) && defined(__x86_64__)
+#if defined(__linux__) && (defined(__x86_64__) || defined(__arm__))
    /* Use 2 MiB alignment so transparent hugepages can be used by KVM.
       Valgrind does not support alignments larger than 1 MiB,
       therefore we need special code which handles running on Valgrind. */
-- 
1.7.9.5

[RFC v5 2/8] linux-headers: Add ARM KVM headers (not for upstream)

[Peter Maydell]

This commit adds the ARM KVM headers. This is not to go to QEMU
upstream -- the correct path there is that the KVM code will be
committed to a mainline upstream kernel, and then upstream QEMU
can do a bulk header update from the upstream kernel, which will
allow us to drop this temporary commit.

This is the result of running update-headers on Christoffer's
kvm-arm-v17-vgic-timers branch (commit 4dcab3d) and then
hand-editing to avoid reverting some changes for other archs.
---
 linux-headers/asm-arm/kvm.h          |  180 ++++++++++++++++++++++++++++++++++
 linux-headers/asm-arm/kvm_para.h     |    1 +
 linux-headers/asm-generic/kvm_para.h |    4 +
 linux-headers/linux/kvm.h            |   17 ++++
 4 files changed, 202 insertions(+)
 create mode 100644 linux-headers/asm-arm/kvm.h
 create mode 100644 linux-headers/asm-arm/kvm_para.h
 create mode 100644 linux-headers/asm-generic/kvm_para.h

diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h
new file mode 100644
index 0000000..023bfeb
--- /dev/null
+++ b/linux-headers/asm-arm/kvm.h
@@ -0,0 +1,180 @@
+/*
+ * Copyright (C) 2012 - Virtual Open Systems and Columbia University
+ * Author: Christoffer Dall <c.dall@virtualopensystems.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ */
+
+#ifndef __ARM_KVM_H__
+#define __ARM_KVM_H__
+
+#include <linux/types.h>
+#include <asm/ptrace.h>
+
+#define __KVM_HAVE_GUEST_DEBUG
+#define __KVM_HAVE_IRQ_LINE
+
+#define KVM_REG_SIZE(id)						\
+	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
+
+/* Valid for svc_regs, abt_regs, und_regs, irq_regs in struct kvm_regs */
+#define KVM_ARM_SVC_sp		svc_regs[0]
+#define KVM_ARM_SVC_lr		svc_regs[1]
+#define KVM_ARM_SVC_spsr	svc_regs[2]
+#define KVM_ARM_ABT_sp		abt_regs[0]
+#define KVM_ARM_ABT_lr		abt_regs[1]
+#define KVM_ARM_ABT_spsr	abt_regs[2]
+#define KVM_ARM_UND_sp		und_regs[0]
+#define KVM_ARM_UND_lr		und_regs[1]
+#define KVM_ARM_UND_spsr	und_regs[2]
+#define KVM_ARM_IRQ_sp		irq_regs[0]
+#define KVM_ARM_IRQ_lr		irq_regs[1]
+#define KVM_ARM_IRQ_spsr	irq_regs[2]
+
+/* Valid only for fiq_regs in struct kvm_regs */
+#define KVM_ARM_FIQ_r8		fiq_regs[0]
+#define KVM_ARM_FIQ_r9		fiq_regs[1]
+#define KVM_ARM_FIQ_r10		fiq_regs[2]
+#define KVM_ARM_FIQ_fp		fiq_regs[3]
+#define KVM_ARM_FIQ_ip		fiq_regs[4]
+#define KVM_ARM_FIQ_sp		fiq_regs[5]
+#define KVM_ARM_FIQ_lr		fiq_regs[6]
+#define KVM_ARM_FIQ_spsr	fiq_regs[7]
+
+struct kvm_regs {
+	struct pt_regs usr_regs;/* R0_usr - R14_usr, PC, CPSR */
+	__u32 svc_regs[3];	/* SP_svc, LR_svc, SPSR_svc */
+	__u32 abt_regs[3];	/* SP_abt, LR_abt, SPSR_abt */
+	__u32 und_regs[3];	/* SP_und, LR_und, SPSR_und */
+	__u32 irq_regs[3];	/* SP_irq, LR_irq, SPSR_irq */
+	__u32 fiq_regs[8];	/* R8_fiq - R14_fiq, SPSR_fiq */
+};
+
+/* Supported Processor Types */
+#define KVM_ARM_TARGET_CORTEX_A15	0
+#define KVM_ARM_NUM_TARGETS		1
+
+/* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
+#define KVM_ARM_DEVICE_TYPE_SHIFT	0
+#define KVM_ARM_DEVICE_TYPE_MASK	(0xffff << KVM_ARM_DEVICE_TYPE_SHIFT)
+#define KVM_ARM_DEVICE_ID_SHIFT		16
+#define KVM_ARM_DEVICE_ID_MASK		(0xffff << KVM_ARM_DEVICE_ID_SHIFT)
+
+/* Supported device IDs */
+#define KVM_ARM_DEVICE_VGIC_V2		0
+
+/* Supported VGIC address types  */
+#define KVM_VGIC_V2_ADDR_TYPE_DIST	0
+#define KVM_VGIC_V2_ADDR_TYPE_CPU	1
+
+#define KVM_VGIC_V2_DIST_SIZE		0x1000
+#define KVM_VGIC_V2_CPU_SIZE		0x2000
+
+#define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
+
+struct kvm_vcpu_init {
+	__u32 target;
+	__u32 features[7];
+};
+
+struct kvm_sregs {
+};
+
+struct kvm_fpu {
+};
+
+struct kvm_guest_debug_arch {
+};
+
+struct kvm_debug_exit_arch {
+};
+
+struct kvm_sync_regs {
+};
+
+struct kvm_arch_memory_slot {
+};
+
+/* If you need to interpret the index values, here is the key: */
+#define KVM_REG_ARM_COPROC_MASK		0x000000000FFF0000
+#define KVM_REG_ARM_COPROC_SHIFT	16
+#define KVM_REG_ARM_32_OPC2_MASK	0x0000000000000007
+#define KVM_REG_ARM_32_OPC2_SHIFT	0
+#define KVM_REG_ARM_OPC1_MASK		0x0000000000000078
+#define KVM_REG_ARM_OPC1_SHIFT		3
+#define KVM_REG_ARM_CRM_MASK		0x0000000000000780
+#define KVM_REG_ARM_CRM_SHIFT		7
+#define KVM_REG_ARM_32_CRN_MASK		0x0000000000007800
+#define KVM_REG_ARM_32_CRN_SHIFT	11
+
+/* Normal registers are mapped as coprocessor 16. */
+#define KVM_REG_ARM_CORE		(0x0010 << KVM_REG_ARM_COPROC_SHIFT)
+#define KVM_REG_ARM_CORE_REG(name)	(offsetof(struct kvm_regs, name) / 4)
+
+/* Some registers need more space to represent values. */
+#define KVM_REG_ARM_DEMUX		(0x0011 << KVM_REG_ARM_COPROC_SHIFT)
+#define KVM_REG_ARM_DEMUX_ID_MASK	0x000000000000FF00
+#define KVM_REG_ARM_DEMUX_ID_SHIFT	8
+#define KVM_REG_ARM_DEMUX_ID_CCSIDR	(0x00 << KVM_REG_ARM_DEMUX_ID_SHIFT)
+#define KVM_REG_ARM_DEMUX_VAL_MASK	0x00000000000000FF
+#define KVM_REG_ARM_DEMUX_VAL_SHIFT	0
+
+/* VFP registers: we could overload CP10 like ARM does, but that's ugly. */
+#define KVM_REG_ARM_VFP			(0x0012 << KVM_REG_ARM_COPROC_SHIFT)
+#define KVM_REG_ARM_VFP_MASK		0x000000000000FFFF
+#define KVM_REG_ARM_VFP_BASE_REG	0x0
+#define KVM_REG_ARM_VFP_FPSID		0x1000
+#define KVM_REG_ARM_VFP_FPSCR		0x1001
+#define KVM_REG_ARM_VFP_MVFR1		0x1006
+#define KVM_REG_ARM_VFP_MVFR0		0x1007
+#define KVM_REG_ARM_VFP_FPEXC		0x1008
+#define KVM_REG_ARM_VFP_FPINST		0x1009
+#define KVM_REG_ARM_VFP_FPINST2		0x100A
+
+
+/* KVM_IRQ_LINE irq field index values */
+#define KVM_ARM_IRQ_TYPE_SHIFT		24
+#define KVM_ARM_IRQ_TYPE_MASK		0xff
+#define KVM_ARM_IRQ_VCPU_SHIFT		16
+#define KVM_ARM_IRQ_VCPU_MASK		0xff
+#define KVM_ARM_IRQ_NUM_SHIFT		0
+#define KVM_ARM_IRQ_NUM_MASK		0xffff
+
+/* irq_type field */
+#define KVM_ARM_IRQ_TYPE_CPU		0
+#define KVM_ARM_IRQ_TYPE_SPI		1
+#define KVM_ARM_IRQ_TYPE_PPI		2
+
+/* out-of-kernel GIC cpu interrupt injection irq_number field */
+#define KVM_ARM_IRQ_CPU_IRQ		0
+#define KVM_ARM_IRQ_CPU_FIQ		1
+
+/* Highest supported SPI, from VGIC_NR_IRQS */
+#define KVM_ARM_IRQ_GIC_MAX		127
+
+/* PSCI interface */
+#define KVM_PSCI_FN_BASE		0x95c1ba5e
+#define KVM_PSCI_FN(n)			(KVM_PSCI_FN_BASE + (n))
+
+#define KVM_PSCI_FN_CPU_SUSPEND		KVM_PSCI_FN(0)
+#define KVM_PSCI_FN_CPU_OFF		KVM_PSCI_FN(1)
+#define KVM_PSCI_FN_CPU_ON		KVM_PSCI_FN(2)
+#define KVM_PSCI_FN_MIGRATE		KVM_PSCI_FN(3)
+
+#define KVM_PSCI_RET_SUCCESS		0
+#define KVM_PSCI_RET_NI			((unsigned long)-1)
+#define KVM_PSCI_RET_INVAL		((unsigned long)-2)
+#define KVM_PSCI_RET_DENIED		((unsigned long)-3)
+
+#endif /* __ARM_KVM_H__ */
diff --git a/linux-headers/asm-arm/kvm_para.h b/linux-headers/asm-arm/kvm_para.h
new file mode 100644
index 0000000..14fab8f
--- /dev/null
+++ b/linux-headers/asm-arm/kvm_para.h
@@ -0,0 +1 @@
+#include <asm-generic/kvm_para.h>
diff --git a/linux-headers/asm-generic/kvm_para.h b/linux-headers/asm-generic/kvm_para.h
new file mode 100644
index 0000000..486f0af
--- /dev/null
+++ b/linux-headers/asm-generic/kvm_para.h
@@ -0,0 +1,4 @@
+/*
+ * There isn't anything here, but the file must not be empty or patch
+ * will delete it.
+ */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 5af9357..caca979 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -115,6 +115,7 @@ struct kvm_irq_level {
 	 * ACPI gsi notion of irq.
 	 * For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
 	 * For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
+	 * For ARM: See Documentation/virtual/kvm/api.txt
 	 */
 	union {
 		__u32 irq;
@@ -662,6 +663,8 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_PPC_HTAB_FD 84
 #define KVM_CAP_S390_CSS_SUPPORT 85
 #define KVM_CAP_PPC_EPR 86
+#define KVM_CAP_ARM_PSCI 87
+#define KVM_CAP_ARM_SET_DEVICE_ADDR 88
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -791,6 +794,11 @@ struct kvm_dirty_tlb {
 #define KVM_REG_SIZE_U512	0x0060000000000000ULL
 #define KVM_REG_SIZE_U1024	0x0070000000000000ULL
 
+struct kvm_reg_list {
+	__u64 n; /* number of regs */
+	__u64 reg[0];
+};
+
 struct kvm_one_reg {
 	__u64 id;
 	__u64 addr;
@@ -804,6 +812,11 @@ struct kvm_msi {
 	__u8  pad[16];
 };
 
+struct kvm_arm_device_addr {
+	__u64 id;
+	__u64 addr;
+};
+
 /*
  * ioctls for VM fds
  */
@@ -889,6 +902,8 @@ struct kvm_s390_ucas_mapping {
 #define KVM_ALLOCATE_RMA	  _IOR(KVMIO,  0xa9, struct kvm_allocate_rma)
 /* Available with KVM_CAP_PPC_HTAB_FD */
 #define KVM_PPC_GET_HTAB_FD	  _IOW(KVMIO,  0xaa, struct kvm_get_htab_fd)
+/* Available with KVM_CAP_ARM_SET_DEVICE_ADDR */
+#define KVM_ARM_SET_DEVICE_ADDR	  _IOW(KVMIO,  0xab, struct kvm_arm_device_addr)
 
 /*
  * ioctls for vcpu fds
@@ -959,6 +974,8 @@ struct kvm_s390_ucas_mapping {
 #define KVM_SET_ONE_REG		  _IOW(KVMIO,  0xac, struct kvm_one_reg)
 /* VM is being stopped by host */
 #define KVM_KVMCLOCK_CTRL	  _IO(KVMIO,   0xad)
+#define KVM_ARM_VCPU_INIT	  _IOW(KVMIO,  0xae, struct kvm_vcpu_init)
+#define KVM_GET_REG_LIST	  _IOWR(KVMIO, 0xb0, struct kvm_reg_list)
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
-- 
1.7.9.5

[RFC v5 3/8] ARM: KVM: Add support for KVM on ARM architecture

[Peter Maydell]

Add basic support for KVM on ARM architecture.

Signed-off-by: Christoffer Dall <cdall@cs.columbia.edu>
[PMM: Minor tweaks and code cleanup, switch to ONE_REG]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm_pic.c             |   26 ++++
 target-arm/Makefile.objs |    1 +
 target-arm/cpu.h         |    1 +
 target-arm/helper.c      |    2 +-
 target-arm/kvm.c         |  331 ++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 360 insertions(+), 1 deletion(-)
 create mode 100644 target-arm/kvm.c

diff --git a/hw/arm_pic.c b/hw/arm_pic.c
index ffb4d41..45ccb9f 100644
--- a/hw/arm_pic.c
+++ b/hw/arm_pic.c
@@ -9,6 +9,7 @@
 
 #include "hw.h"
 #include "arm-misc.h"
+#include "sysemu/kvm.h"
 
 /* Input 0 is IRQ and input 1 is FIQ.  */
 static void arm_pic_cpu_handler(void *opaque, int irq, int level)
@@ -34,7 +35,32 @@ static void arm_pic_cpu_handler(void *opaque, int irq, int level)
     }
 }
 
+static void kvm_arm_pic_cpu_handler(void *opaque, int irq, int level)
+{
+#ifdef CONFIG_KVM
+    ARMCPU *armcpu = opaque;
+    CPUState *cpu = CPU(armcpu);
+    int kvm_irq = KVM_ARM_IRQ_TYPE_CPU << KVM_ARM_IRQ_TYPE_SHIFT;
+
+    switch (irq) {
+    case ARM_PIC_CPU_IRQ:
+        kvm_irq |= KVM_ARM_IRQ_CPU_IRQ;
+        break;
+    case ARM_PIC_CPU_FIQ:
+        kvm_irq |= KVM_ARM_IRQ_CPU_FIQ;
+        break;
+    default:
+        hw_error("kvm_arm_pic_cpu_handler: Bad interrupt line %d\n", irq);
+    }
+    kvm_irq |= cpu->cpu_index << KVM_ARM_IRQ_VCPU_SHIFT;
+    kvm_set_irq(kvm_state, kvm_irq, level ? 1 : 0);
+#endif
+}
+
 qemu_irq *arm_pic_init_cpu(ARMCPU *cpu)
 {
+    if (kvm_enabled()) {
+        return qemu_allocate_irqs(kvm_arm_pic_cpu_handler, cpu, 2);
+    }
     return qemu_allocate_irqs(arm_pic_cpu_handler, cpu, 2);
 }
diff --git a/target-arm/Makefile.objs b/target-arm/Makefile.objs
index b6f1a9e..d89b57c 100644
--- a/target-arm/Makefile.objs
+++ b/target-arm/Makefile.objs
@@ -1,4 +1,5 @@
 obj-y += arm-semi.o
 obj-$(CONFIG_SOFTMMU) += machine.o
+obj-$(CONFIG_KVM) += kvm.o
 obj-y += translate.o op_helper.o helper.o cpu.o
 obj-y += neon_helper.o iwmmxt_helper.o
diff --git a/target-arm/cpu.h b/target-arm/cpu.h
index ffddfcb..21cee84 100644
--- a/target-arm/cpu.h
+++ b/target-arm/cpu.h
@@ -236,6 +236,7 @@ ARMCPU *cpu_arm_init(const char *cpu_model);
 void arm_translate_init(void);
 int cpu_arm_exec(CPUARMState *s);
 void do_interrupt(CPUARMState *);
+int bank_number(CPUARMState *env, int mode);
 void switch_mode(CPUARMState *, int);
 uint32_t do_arm_semihosting(CPUARMState *env);
 
diff --git a/target-arm/helper.c b/target-arm/helper.c
index 37c34a1..f3244e3 100644
--- a/target-arm/helper.c
+++ b/target-arm/helper.c
@@ -1610,7 +1610,7 @@ uint32_t HELPER(get_r13_banked)(CPUARMState *env, uint32_t mode)
 #else
 
 /* Map CPU modes onto saved register banks.  */
-static inline int bank_number(CPUARMState *env, int mode)
+int bank_number(CPUARMState *env, int mode)
 {
     switch (mode) {
     case ARM_CPU_MODE_USR:
diff --git a/target-arm/kvm.c b/target-arm/kvm.c
new file mode 100644
index 0000000..db13568
--- /dev/null
+++ b/target-arm/kvm.c
@@ -0,0 +1,331 @@
+/*
+ * ARM implementation of KVM hooks
+ *
+ * Copyright Christoffer Dall 2009-2010
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/kvm.h>
+
+#include "qemu-common.h"
+#include "qemu/timer.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
+#include "cpu.h"
+#include "hw/arm-misc.h"
+
+const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
+    KVM_CAP_LAST_INFO
+};
+
+int kvm_arch_init(KVMState *s)
+{
+    /* For ARM interrupt delivery is always asynchronous,
+     * whether we are using an in-kernel VGIC or not.
+     */
+    kvm_async_interrupts_allowed = true;
+    return 0;
+}
+
+int kvm_arch_init_vcpu(CPUState *cs)
+{
+    struct kvm_vcpu_init init;
+
+    init.target = KVM_ARM_TARGET_CORTEX_A15;
+    memset(init.features, 0, sizeof(init.features));
+    return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init);
+}
+
+typedef struct Reg {
+    uint64_t id;
+    int offset;
+} Reg;
+
+#define COREREG(KERNELNAME, QEMUFIELD)                       \
+    {                                                        \
+        KVM_REG_ARM | KVM_REG_SIZE_U32 |                     \
+        KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(KERNELNAME), \
+        offsetof(CPUARMState, QEMUFIELD)                     \
+    }
+
+#define CP15REG(CRN, CRM, OPC1, OPC2, QEMUFIELD) \
+    {                                            \
+        KVM_REG_ARM | KVM_REG_SIZE_U32 |         \
+        (15 << KVM_REG_ARM_COPROC_SHIFT) |       \
+        ((CRN) << KVM_REG_ARM_32_CRN_SHIFT) |    \
+        ((CRM) << KVM_REG_ARM_CRM_SHIFT) |       \
+        ((OPC1) << KVM_REG_ARM_OPC1_SHIFT) |     \
+        ((OPC2) << KVM_REG_ARM_32_OPC2_SHIFT),   \
+        offsetof(CPUARMState, QEMUFIELD)         \
+    }
+
+static const Reg regs[] = {
+    /* R0_usr .. R14_usr */
+    COREREG(usr_regs.uregs[0], regs[0]),
+    COREREG(usr_regs.uregs[1], regs[1]),
+    COREREG(usr_regs.uregs[2], regs[2]),
+    COREREG(usr_regs.uregs[3], regs[3]),
+    COREREG(usr_regs.uregs[4], regs[4]),
+    COREREG(usr_regs.uregs[5], regs[5]),
+    COREREG(usr_regs.uregs[6], regs[6]),
+    COREREG(usr_regs.uregs[7], regs[7]),
+    COREREG(usr_regs.uregs[8], usr_regs[0]),
+    COREREG(usr_regs.uregs[9], usr_regs[1]),
+    COREREG(usr_regs.uregs[10], usr_regs[2]),
+    COREREG(usr_regs.uregs[11], usr_regs[3]),
+    COREREG(usr_regs.uregs[12], usr_regs[4]),
+    COREREG(usr_regs.uregs[13], banked_r13[0]),
+    COREREG(usr_regs.uregs[14], banked_r14[0]),
+    /* R13, R14, SPSR for SVC, ABT, UND, IRQ banks */
+    COREREG(svc_regs[0], banked_r13[1]),
+    COREREG(svc_regs[1], banked_r14[1]),
+    COREREG(svc_regs[2], banked_spsr[1]),
+    COREREG(abt_regs[0], banked_r13[2]),
+    COREREG(abt_regs[1], banked_r14[2]),
+    COREREG(abt_regs[2], banked_spsr[2]),
+    COREREG(und_regs[0], banked_r13[3]),
+    COREREG(und_regs[1], banked_r14[3]),
+    COREREG(und_regs[2], banked_spsr[3]),
+    COREREG(irq_regs[0], banked_r13[4]),
+    COREREG(irq_regs[1], banked_r14[4]),
+    COREREG(irq_regs[2], banked_spsr[4]),
+    /* R8_fiq .. R14_fiq and SPSR_fiq */
+    COREREG(fiq_regs[0], fiq_regs[0]),
+    COREREG(fiq_regs[1], fiq_regs[1]),
+    COREREG(fiq_regs[2], fiq_regs[2]),
+    COREREG(fiq_regs[3], fiq_regs[3]),
+    COREREG(fiq_regs[4], fiq_regs[4]),
+    COREREG(fiq_regs[5], banked_r13[5]),
+    COREREG(fiq_regs[6], banked_r14[5]),
+    COREREG(fiq_regs[7], banked_spsr[5]),
+    /* R15 */
+    COREREG(usr_regs.uregs[15], regs[15]),
+    /* A non-comprehensive set of cp15 registers.
+     * TODO: drive this from the cp_regs hashtable instead.
+     */
+    CP15REG(1, 0, 0, 0, cp15.c1_sys), /* SCTLR */
+    CP15REG(2, 0, 0, 2, cp15.c2_control), /* TTBCR */
+    CP15REG(3, 0, 0, 0, cp15.c3), /* DACR */
+};
+
+int kvm_arch_put_registers(CPUState *cs, int level)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+    struct kvm_one_reg r;
+    int mode, bn;
+    int ret, i;
+    uint32_t cpsr;
+    uint64_t ttbr;
+
+    /* Make sure the banked regs are properly set */
+    mode = env->uncached_cpsr & CPSR_M;
+    bn = bank_number(env, mode);
+    if (mode == ARM_CPU_MODE_FIQ) {
+        memcpy(env->fiq_regs, env->regs + 8, 5 * sizeof(uint32_t));
+    } else {
+        memcpy(env->usr_regs, env->regs + 8, 5 * sizeof(uint32_t));
+    }
+    env->banked_r13[bn] = env->regs[13];
+    env->banked_r14[bn] = env->regs[14];
+    env->banked_spsr[bn] = env->spsr;
+
+    /* Now we can safely copy stuff down to the kernel */
+    for (i = 0; i < ARRAY_SIZE(regs); i++) {
+        r.id = regs[i].id;
+        r.addr = (uintptr_t)(env) + regs[i].offset;
+        ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    /* Special cases which aren't a single CPUARMState field */
+    cpsr = cpsr_read(env);
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U32 |
+        KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(usr_regs.ARM_cpsr);
+    r.addr = (uintptr_t)(&cpsr);
+    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+
+    /* TTBR0: cp15 crm=2 opc1=0 */
+    ttbr = ((uint64_t)env->cp15.c2_base0_hi << 32) | env->cp15.c2_base0;
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | (15 << KVM_REG_ARM_COPROC_SHIFT) |
+        (2 << KVM_REG_ARM_CRM_SHIFT) | (0 << KVM_REG_ARM_OPC1_SHIFT);
+    r.addr = (uintptr_t)(&ttbr);
+    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+
+    /* TTBR1: cp15 crm=2 opc1=1 */
+    ttbr = ((uint64_t)env->cp15.c2_base1_hi << 32) | env->cp15.c2_base1;
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | (15 << KVM_REG_ARM_COPROC_SHIFT) |
+        (2 << KVM_REG_ARM_CRM_SHIFT) | (1 << KVM_REG_ARM_OPC1_SHIFT);
+    r.addr = (uintptr_t)(&ttbr);
+    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+
+    return ret;
+}
+
+int kvm_arch_get_registers(CPUState *cs)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+    struct kvm_one_reg r;
+    int mode, bn;
+    int ret, i;
+    uint32_t cpsr;
+    uint64_t ttbr;
+
+    for (i = 0; i < ARRAY_SIZE(regs); i++) {
+        r.id = regs[i].id;
+        r.addr = (uintptr_t)(env) + regs[i].offset;
+        ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    /* Special cases which aren't a single CPUARMState field */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U32 |
+        KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(usr_regs.ARM_cpsr);
+    r.addr = (uintptr_t)(&cpsr);
+    ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+    cpsr_write(env, cpsr, 0xffffffff);
+
+    /* TTBR0: cp15 crm=2 opc1=0 */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | (15 << KVM_REG_ARM_COPROC_SHIFT) |
+        (2 << KVM_REG_ARM_CRM_SHIFT) | (0 << KVM_REG_ARM_OPC1_SHIFT);
+    r.addr = (uintptr_t)(&ttbr);
+    ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+    env->cp15.c2_base0_hi = ttbr >> 32;
+    env->cp15.c2_base0 = ttbr;
+
+    /* TTBR1: cp15 crm=2 opc1=1 */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | (15 << KVM_REG_ARM_COPROC_SHIFT) |
+        (2 << KVM_REG_ARM_CRM_SHIFT) | (1 << KVM_REG_ARM_OPC1_SHIFT);
+    r.addr = (uintptr_t)(&ttbr);
+    ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+    env->cp15.c2_base1_hi = ttbr >> 32;
+    env->cp15.c2_base1 = ttbr;
+
+    /* Make sure the current mode regs are properly set */
+    mode = env->uncached_cpsr & CPSR_M;
+    bn = bank_number(env, mode);
+    if (mode == ARM_CPU_MODE_FIQ) {
+        memcpy(env->regs + 8, env->fiq_regs, 5 * sizeof(uint32_t));
+    } else {
+        memcpy(env->regs + 8, env->usr_regs, 5 * sizeof(uint32_t));
+    }
+    env->regs[13] = env->banked_r13[bn];
+    env->regs[14] = env->banked_r14[bn];
+    env->spsr = env->banked_spsr[bn];
+
+    /* The main GET_ONE_REG loop above set c2_control, but we need to
+     * update some extra cached precomputed values too.
+     * When this is driven from the cp_regs hashtable then this ugliness
+     * can disappear because we'll use the access function which sets
+     * these values automatically.
+     */
+    env->cp15.c2_mask = ~(0xffffffffu >> env->cp15.c2_control);
+    env->cp15.c2_base_mask = ~(0x3fffu >> env->cp15.c2_control);
+
+    return 0;
+}
+
+void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run)
+{
+}
+
+void kvm_arch_post_run(CPUState *cs, struct kvm_run *run)
+{
+}
+
+int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
+{
+    int ret = 0;
+
+    return ret;
+}
+
+void kvm_arch_reset_vcpu(CPUState *cs)
+{
+}
+
+bool kvm_arch_stop_on_emulation_error(CPUState *cs)
+{
+    return true;
+}
+
+int kvm_arch_process_async_events(CPUState *cs)
+{
+    return 0;
+}
+
+int kvm_arch_on_sigbus_vcpu(CPUState *cs, int code, void *addr)
+{
+    return 1;
+}
+
+int kvm_arch_on_sigbus(int code, void *addr)
+{
+    return 1;
+}
+
+void kvm_arch_update_guest_debug(CPUState *cs, struct kvm_guest_debug *dbg)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+}
+
+int kvm_arch_insert_sw_breakpoint(CPUState *cs,
+                                  struct kvm_sw_breakpoint *bp)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_insert_hw_breakpoint(target_ulong addr,
+                                  target_ulong len, int type)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_remove_hw_breakpoint(target_ulong addr,
+                                  target_ulong len, int type)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+int kvm_arch_remove_sw_breakpoint(CPUState *cs,
+                                  struct kvm_sw_breakpoint *bp)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+    return -EINVAL;
+}
+
+void kvm_arch_remove_all_hw_breakpoints(void)
+{
+    qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__);
+}
-- 
1.7.9.5

[RFC v5 4/8] ARM KVM: save and load VFP registers from kernel

[Peter Maydell]

Add support for saving and restoring VFP register state from the
kernel. This includes a check that the KVM-created CPU has full
VFP support (as the TCG Cortex-A15 model always does), since for
the moment ARM QEMU doesn't have any way to tweak optional features
on created CPUs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target-arm/kvm.c |   78 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 75 insertions(+), 3 deletions(-)

diff --git a/target-arm/kvm.c b/target-arm/kvm.c
index db13568..13f7b0d 100644
--- a/target-arm/kvm.c
+++ b/target-arm/kvm.c
@@ -38,10 +38,28 @@ int kvm_arch_init(KVMState *s)
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     struct kvm_vcpu_init init;
+    int ret;
+    uint64_t v;
+    struct kvm_one_reg r;
 
     init.target = KVM_ARM_TARGET_CORTEX_A15;
     memset(init.features, 0, sizeof(init.features));
-    return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init);
+    ret = kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init);
+    if (ret) {
+        return ret;
+    }
+    /* Query the kernel to make sure it supports 32 VFP
+     * registers: QEMU's "cortex-a15" CPU is always a
+     * VFP-D32 core. The simplest way to do this is just
+     * to attempt to read register d31.
+     */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | KVM_REG_ARM_VFP | 31;
+    r.addr = (uintptr_t)(&v);
+    ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+    if (ret == ENOENT) {
+        return EINVAL;
+    }
+    return ret;
 }
 
 typedef struct Reg {
@@ -67,6 +85,13 @@ typedef struct Reg {
         offsetof(CPUARMState, QEMUFIELD)         \
     }
 
+#define VFPSYSREG(R)                                       \
+    {                                                      \
+        KVM_REG_ARM | KVM_REG_SIZE_U32 | KVM_REG_ARM_VFP | \
+        KVM_REG_ARM_VFP_##R,                               \
+        offsetof(CPUARMState, vfp.xregs[ARM_VFP_##R])      \
+    }
+
 static const Reg regs[] = {
     /* R0_usr .. R14_usr */
     COREREG(usr_regs.uregs[0], regs[0]),
@@ -114,6 +139,13 @@ static const Reg regs[] = {
     CP15REG(1, 0, 0, 0, cp15.c1_sys), /* SCTLR */
     CP15REG(2, 0, 0, 2, cp15.c2_control), /* TTBCR */
     CP15REG(3, 0, 0, 0, cp15.c3), /* DACR */
+    /* VFP system registers */
+    VFPSYSREG(FPSID),
+    VFPSYSREG(MVFR1),
+    VFPSYSREG(MVFR0),
+    VFPSYSREG(FPEXC),
+    VFPSYSREG(FPINST),
+    VFPSYSREG(FPINST2),
 };
 
 int kvm_arch_put_registers(CPUState *cs, int level)
@@ -123,7 +155,7 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     struct kvm_one_reg r;
     int mode, bn;
     int ret, i;
-    uint32_t cpsr;
+    uint32_t cpsr, fpscr;
     uint64_t ttbr;
 
     /* Make sure the banked regs are properly set */
@@ -174,6 +206,26 @@ int kvm_arch_put_registers(CPUState *cs, int level)
         (2 << KVM_REG_ARM_CRM_SHIFT) | (1 << KVM_REG_ARM_OPC1_SHIFT);
     r.addr = (uintptr_t)(&ttbr);
     ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+
+    /* VFP registers */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | KVM_REG_ARM_VFP;
+    for (i = 0; i < 32; i++) {
+        r.addr = (uintptr_t)(&env->vfp.regs[i]);
+        ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
+        if (ret) {
+            return ret;
+        }
+        r.id++;
+    }
+
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U32 | KVM_REG_ARM_VFP |
+        KVM_REG_ARM_VFP_FPSCR;
+    fpscr = vfp_get_fpscr(env);
+    r.addr = (uintptr_t)&fpscr;
+    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
 
     return ret;
 }
@@ -185,7 +237,7 @@ int kvm_arch_get_registers(CPUState *cs)
     struct kvm_one_reg r;
     int mode, bn;
     int ret, i;
-    uint32_t cpsr;
+    uint32_t cpsr, fpscr;
     uint64_t ttbr;
 
     for (i = 0; i < ARRAY_SIZE(regs); i++) {
@@ -250,6 +302,26 @@ int kvm_arch_get_registers(CPUState *cs)
     env->cp15.c2_mask = ~(0xffffffffu >> env->cp15.c2_control);
     env->cp15.c2_base_mask = ~(0x3fffu >> env->cp15.c2_control);
 
+    /* VFP registers */
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U64 | KVM_REG_ARM_VFP;
+    for (i = 0; i < 32; i++) {
+        r.addr = (uintptr_t)(&env->vfp.regs[i]);
+        ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+        if (ret) {
+            return ret;
+        }
+        r.id++;
+    }
+
+    r.id = KVM_REG_ARM | KVM_REG_SIZE_U32 | KVM_REG_ARM_VFP |
+        KVM_REG_ARM_VFP_FPSCR;
+    r.addr = (uintptr_t)&fpscr;
+    ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
+    if (ret) {
+        return ret;
+    }
+    vfp_set_fpscr(env, fpscr);
+
     return 0;
 }
 
-- 
1.7.9.5

[RFC v5 5/8] hw/arm_gic: Add presave/postload hooks

[Peter Maydell]

Add presave/postload hooks to the ARM GIC common base class.
These will be used by the KVM in-kernel GIC subclass to sync
state between kernel and userspace when migrating.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Andreas Färber <afaerber@suse.de>
---
 hw/arm_gic_common.c   |   10 ++++++++++
 hw/arm_gic_internal.h |    2 ++
 2 files changed, 12 insertions(+)

diff --git a/hw/arm_gic_common.c b/hw/arm_gic_common.c
index 40e8dd7..2947622 100644
--- a/hw/arm_gic_common.c
+++ b/hw/arm_gic_common.c
@@ -23,9 +23,14 @@
 static void gic_save(QEMUFile *f, void *opaque)
 {
     GICState *s = (GICState *)opaque;
+    ARMGICCommonClass *c = ARM_GIC_COMMON_GET_CLASS(s);
     int i;
     int j;
 
+    if (c->pre_save) {
+        c->pre_save(s);
+    }
+
     qemu_put_be32(f, s->enabled);
     for (i = 0; i < s->num_cpu; i++) {
         qemu_put_be32(f, s->cpu_enabled[i]);
@@ -57,6 +62,7 @@ static void gic_save(QEMUFile *f, void *opaque)
 static int gic_load(QEMUFile *f, void *opaque, int version_id)
 {
     GICState *s = (GICState *)opaque;
+    ARMGICCommonClass *c = ARM_GIC_COMMON_GET_CLASS(s);
     int i;
     int j;
 
@@ -91,6 +97,10 @@ static int gic_load(QEMUFile *f, void *opaque, int version_id)
         s->irq_state[i].trigger = qemu_get_byte(f);
     }
 
+    if (c->post_load) {
+        c->post_load(s);
+    }
+
     return 0;
 }
 
diff --git a/hw/arm_gic_internal.h b/hw/arm_gic_internal.h
index 699352c..3640be0 100644
--- a/hw/arm_gic_internal.h
+++ b/hw/arm_gic_internal.h
@@ -118,6 +118,8 @@ void gic_init_irqs_and_distributor(GICState *s, int num_irq);
 
 typedef struct ARMGICCommonClass {
     SysBusDeviceClass parent_class;
+    void (*pre_save)(GICState *s);
+    void (*post_load)(GICState *s);
 } ARMGICCommonClass;
 
 #define TYPE_ARM_GIC "arm_gic"
-- 
1.7.9.5

[RFC v5 6/8] target-arm: Use MemoryListener to identify GIC base address for KVM

[Peter Maydell]

When using an in-kernel GIC with KVM, we need to tell the kernel where
the GIC's memory mapped registers live. Do this by registering a
MemoryListener which tracks where the board model maps the A15's
private peripherals, so we can finish the GIC initialisation
when the GIC is actually mapped.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target-arm/kvm.c     |   84 ++++++++++++++++++++++++++++++++++++++++++++++++++
 target-arm/kvm_arm.h |   32 +++++++++++++++++++
 2 files changed, 116 insertions(+)
 create mode 100644 target-arm/kvm_arm.h

diff --git a/target-arm/kvm.c b/target-arm/kvm.c
index 13f7b0d..ee9367a 100644
--- a/target-arm/kvm.c
+++ b/target-arm/kvm.c
@@ -19,6 +19,7 @@
 #include "qemu/timer.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/kvm.h"
+#include "kvm_arm.h"
 #include "cpu.h"
 #include "hw/arm-misc.h"
 
@@ -62,6 +63,89 @@ int kvm_arch_init_vcpu(CPUState *cs)
     return ret;
 }
 
+/* We track all the KVM devices which need their memory addresses
+ * passing to the kernel in a list of these structures.
+ * When board init is complete we run through the list and
+ * tell the kernel the base addresses of the memory regions.
+ * We use a MemoryListener to track mapping and unmapping of
+ * the regions during board creation, so the board models don't
+ * need to do anything special for the KVM case.
+ */
+typedef struct KVMDevice {
+    struct kvm_arm_device_addr kda;
+    MemoryRegion *mr;
+    QSLIST_ENTRY(KVMDevice) entries;
+} KVMDevice;
+
+static QSLIST_HEAD(kvm_devices_head, KVMDevice) kvm_devices_head;
+
+static void kvm_arm_devlistener_add(MemoryListener *listener,
+                                    MemoryRegionSection *section)
+{
+    KVMDevice *kd;
+    QSLIST_FOREACH(kd, &kvm_devices_head, entries) {
+        if (section->mr == kd->mr) {
+            kd->kda.addr = section->offset_within_address_space;
+        }
+    }
+}
+
+static void kvm_arm_devlistener_del(MemoryListener *listener,
+                                    MemoryRegionSection *section)
+{
+    KVMDevice *kd;
+    QSLIST_FOREACH(kd, &kvm_devices_head, entries) {
+        if (section->mr == kd->mr) {
+            kd->kda.addr = -1;
+        }
+    }
+}
+
+static MemoryListener devlistener = {
+    .region_add = kvm_arm_devlistener_add,
+    .region_del = kvm_arm_devlistener_del,
+};
+
+static void kvm_arm_machine_init_done(Notifier *notifier, void *data)
+{
+    KVMDevice *kd, *tkd;
+    memory_listener_unregister(&devlistener);
+    QSLIST_FOREACH_SAFE(kd, &kvm_devices_head, entries, tkd) {
+        if (kd->kda.addr != -1) {
+            if (kvm_vm_ioctl(kvm_state, KVM_ARM_SET_DEVICE_ADDR,
+                             &kd->kda) < 0) {
+                fprintf(stderr, "KVM_ARM_SET_DEVICE_ADDRESS failed: %s\n",
+                        strerror(errno));
+                abort();
+            }
+        }
+        g_free(kd);
+    }
+}
+
+static Notifier notify = {
+    .notify = kvm_arm_machine_init_done,
+};
+
+void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid)
+{
+    KVMDevice *kd;
+
+    if (!kvm_irqchip_in_kernel()) {
+        return;
+    }
+
+    if (QSLIST_EMPTY(&kvm_devices_head)) {
+        memory_listener_register(&devlistener, NULL);
+        qemu_add_machine_init_done_notifier(&notify);
+    }
+    kd = g_new0(KVMDevice, 1);
+    kd->mr = mr;
+    kd->kda.id = devid;
+    kd->kda.addr = -1;
+    QSLIST_INSERT_HEAD(&kvm_devices_head, kd, entries);
+}
+
 typedef struct Reg {
     uint64_t id;
     int offset;
diff --git a/target-arm/kvm_arm.h b/target-arm/kvm_arm.h
new file mode 100644
index 0000000..b1c54ff
--- /dev/null
+++ b/target-arm/kvm_arm.h
@@ -0,0 +1,32 @@
+/*
+ * QEMU KVM support -- ARM specific functions.
+ *
+ * Copyright (c) 2012 Linaro Limited
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_KVM_ARM_H
+#define QEMU_KVM_ARM_H
+
+#include "sysemu/kvm.h"
+#include "exec/memory.h"
+
+/**
+ * kvm_arm_register_device:
+ * @mr: memory region for this device
+ * @devid: the KVM device ID
+ *
+ * Remember the memory region @mr, and when it is mapped by the
+ * machine model, tell the kernel that base address using the
+ * KVM_SET_DEVICE_ADDRESS ioctl. @devid should be the ID of
+ * the device as defined by KVM_SET_DEVICE_ADDRESS.
+ * The machine model may map and unmap the device multiple times;
+ * the kernel will only be told the final address at the point
+ * where machine init is complete.
+ */
+void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid);
+
+#endif
-- 
1.7.9.5

[RFC v5 7/8] hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC

[Peter Maydell]

Implement support for using the KVM in-kernel GIC for ARM.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/a15mpcore.c       |    8 ++-
 hw/arm/Makefile.objs |    1 +
 hw/kvm/arm_gic.c     |  169 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 hw/kvm/arm_gic.c

diff --git a/hw/a15mpcore.c b/hw/a15mpcore.c
index fe6c34c..1ca6f28 100644
--- a/hw/a15mpcore.c
+++ b/hw/a15mpcore.c
@@ -19,6 +19,7 @@
  */
 
 #include "sysbus.h"
+#include "sysemu/kvm.h"
 
 /* A15MP private memory region.  */
 
@@ -40,8 +41,13 @@ static int a15mp_priv_init(SysBusDevice *dev)
 {
     A15MPPrivState *s = FROM_SYSBUS(A15MPPrivState, dev);
     SysBusDevice *busdev;
+    const char *gictype = "arm-gic";
 
-    s->gic = qdev_create(NULL, "arm_gic");
+    if (kvm_irqchip_in_kernel()) {
+        gictype = "kvm-arm-gic";
+    }
+
+    s->gic = qdev_create(NULL, gictype);
     qdev_prop_set_uint32(s->gic, "num-cpu", s->num_cpu);
     qdev_prop_set_uint32(s->gic, "num-irq", s->num_irq);
     qdev_prop_set_uint32(s->gic, "revision", 2);
diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
index 6d049e7..38b10a8 100644
--- a/hw/arm/Makefile.objs
+++ b/hw/arm/Makefile.objs
@@ -31,5 +31,6 @@ obj-y += collie.o
 obj-y += imx_serial.o imx_ccm.o imx_timer.o imx_avic.o
 obj-y += kzm.o
 obj-$(CONFIG_FDT) += ../device_tree.o
+obj-$(CONFIG_KVM) += kvm/arm_gic.o
 
 obj-y := $(addprefix ../,$(obj-y))
diff --git a/hw/kvm/arm_gic.c b/hw/kvm/arm_gic.c
new file mode 100644
index 0000000..c3b73c4
--- /dev/null
+++ b/hw/kvm/arm_gic.c
@@ -0,0 +1,169 @@
+/*
+ * ARM Generic Interrupt Controller using KVM in-kernel support
+ *
+ * Copyright (c) 2012 Linaro Limited
+ * Written by Peter Maydell
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "hw/sysbus.h"
+#include "sysemu/kvm.h"
+#include "kvm_arm.h"
+#include "hw/arm_gic_internal.h"
+
+#define TYPE_KVM_ARM_GIC "kvm-arm-gic"
+#define KVM_ARM_GIC(obj) \
+     OBJECT_CHECK(GICState, (obj), TYPE_KVM_ARM_GIC)
+#define KVM_ARM_GIC_CLASS(klass) \
+     OBJECT_CLASS_CHECK(KVMARMGICClass, (klass), TYPE_KVM_ARM_GIC)
+#define KVM_ARM_GIC_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(KVMARMGICClass, (obj), TYPE_KVM_ARM_GIC)
+
+typedef struct KVMARMGICClass {
+    ARMGICCommonClass parent_class;
+    int (*parent_init)(SysBusDevice *dev);
+    void (*parent_reset)(DeviceState *dev);
+} KVMARMGICClass;
+
+static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
+{
+    /* Meaning of the 'irq' parameter:
+     *  [0..N-1] : external interrupts
+     *  [N..N+31] : PPI (internal) interrupts for CPU 0
+     *  [N+32..N+63] : PPI (internal interrupts for CPU 1
+     *  ...
+     * Convert this to the kernel's desired encoding, which
+     * has separate fields in the irq number for type,
+     * CPU number and interrupt number.
+     */
+    GICState *s = (GICState *)opaque;
+    int kvm_irq, irqtype, cpu;
+
+    if (irq < (s->num_irq - GIC_INTERNAL)) {
+        /* External interrupt. The kernel numbers these like the GIC
+         * hardware, with external interrupt IDs starting after the
+         * internal ones.
+         */
+        irqtype = KVM_ARM_IRQ_TYPE_SPI;
+        cpu = 0;
+        irq += GIC_INTERNAL;
+    } else {
+        /* Internal interrupt: decode into (cpu, interrupt id) */
+        irqtype = KVM_ARM_IRQ_TYPE_PPI;
+        irq -= (s->num_irq - GIC_INTERNAL);
+        cpu = irq / GIC_INTERNAL;
+        irq %= GIC_INTERNAL;
+    }
+    kvm_irq = (irqtype << KVM_ARM_IRQ_TYPE_SHIFT)
+        | (cpu << KVM_ARM_IRQ_VCPU_SHIFT) | irq;
+
+    kvm_set_irq(kvm_state, kvm_irq, !!level);
+}
+
+static void kvm_arm_gic_put(GICState *s)
+{
+    /* TODO: there isn't currently a kernel interface to set the GIC state */
+}
+
+static void kvm_arm_gic_get(GICState *s)
+{
+    /* TODO: there isn't currently a kernel interface to get the GIC state */
+}
+
+static void kvm_arm_gic_reset(DeviceState *dev)
+{
+    GICState *s = ARM_GIC_COMMON(dev);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_GET_CLASS(s);
+    kgc->parent_reset(dev);
+    kvm_arm_gic_put(s);
+}
+
+static int kvm_arm_gic_init(SysBusDevice *dev)
+{
+    /* Device instance init function for the GIC sysbus device */
+    int i;
+    GICState *s = FROM_SYSBUS(GICState, dev);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_GET_CLASS(s);
+
+    kgc->parent_init(dev);
+
+    i = s->num_irq - GIC_INTERNAL;
+    /* For the GIC, also expose incoming GPIO lines for PPIs for each CPU.
+     * GPIO array layout is thus:
+     *  [0..N-1] SPIs
+     *  [N..N+31] PPIs for CPU 0
+     *  [N+32..N+63] PPIs for CPU 1
+     *   ...
+     */
+    i += (GIC_INTERNAL * s->num_cpu);
+    qdev_init_gpio_in(&s->busdev.qdev, kvm_arm_gic_set_irq, i);
+    /* We never use our outbound IRQ lines but provide them so that
+     * we maintain the same interface as the non-KVM GIC.
+     */
+    for (i = 0; i < s->num_cpu; i++) {
+        sysbus_init_irq(&s->busdev, &s->parent_irq[i]);
+    }
+    /* Distributor */
+    memory_region_init_reservation(&s->iomem, "kvm-gic_dist", 0x1000);
+    sysbus_init_mmio(dev, &s->iomem);
+    kvm_arm_register_device(&s->iomem,
+                            (KVM_ARM_DEVICE_VGIC_V2 << KVM_ARM_DEVICE_ID_SHIFT)
+                            | KVM_VGIC_V2_ADDR_TYPE_DIST);
+    /* CPU interface for current core. Unlike arm_gic, we don't
+     * provide the "interface for core #N" memory regions, because
+     * cores with a VGIC don't have those.
+     */
+    memory_region_init_reservation(&s->cpuiomem[0], "kvm-gic_cpu", 0x1000);
+    sysbus_init_mmio(dev, &s->cpuiomem[0]);
+    kvm_arm_register_device(&s->cpuiomem[0],
+                            (KVM_ARM_DEVICE_VGIC_V2 << KVM_ARM_DEVICE_ID_SHIFT)
+                            | KVM_VGIC_V2_ADDR_TYPE_CPU);
+    /* TODO: we should tell the kernel at some point the address
+     * of the private peripheral base. However we don't currently have
+     * any convenient infrastructure to do that, and in any case the
+     * kernel doesn't yet implement an ioctl to let us tell it.
+     */
+    return 0;
+}
+
+static void kvm_arm_gic_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    SysBusDeviceClass *sbc = SYS_BUS_DEVICE_CLASS(klass);
+    ARMGICCommonClass *agcc = ARM_GIC_COMMON_CLASS(klass);
+    KVMARMGICClass *kgc = KVM_ARM_GIC_CLASS(klass);
+    agcc->pre_save = kvm_arm_gic_get;
+    agcc->post_load = kvm_arm_gic_put;
+    kgc->parent_init = sbc->init;
+    kgc->parent_reset = dc->reset;
+    sbc->init = kvm_arm_gic_init;
+    dc->reset = kvm_arm_gic_reset;
+    dc->no_user = 1;
+}
+
+static const TypeInfo kvm_arm_gic_info = {
+    .name = TYPE_KVM_ARM_GIC,
+    .parent = TYPE_ARM_GIC_COMMON,
+    .instance_size = sizeof(GICState),
+    .class_init = kvm_arm_gic_class_init,
+    .class_size = sizeof(KVMARMGICClass),
+};
+
+static void kvm_arm_gic_register_types(void)
+{
+    type_register_static(&kvm_arm_gic_info);
+}
+
+type_init(kvm_arm_gic_register_types)
-- 
1.7.9.5

[RFC v5 8/8] configure: Enable KVM on ARM

[Peter Maydell]

Enable KVM on ARM hosts, now that all the necessary components
for it exist.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 configure |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index c6172ef..b6e235d 100755
--- a/configure
+++ b/configure
@@ -4054,7 +4054,7 @@ case "$target_arch2" in
     echo "CONFIG_NO_XEN=y" >> $config_target_mak
 esac
 case "$target_arch2" in
-  i386|x86_64|ppcemb|ppc|ppc64|s390x)
+  arm|i386|x86_64|ppcemb|ppc|ppc64|s390x)
     # Make sure the target and host cpus are compatible
     if test "$kvm" = "yes" -a "$target_softmmu" = "yes" -a \
       \( "$target_arch2" = "$cpu" -o \
-- 
1.7.9.5

Re: [RFC v5 0/8] QEMU: Support KVM on ARM

[Paolo Bonzini]

Il 24/01/2013 16:43, Peter Maydell ha scritto:
> Round 5 of the QEMU patches to support KVM for
> ARM on Cortex-A15 hardware. It's intended for use with
> the kernel tree at
>  git://github.com/virtualopensystems/linux-kvm-arm.git kvm-arm-v17-vgic-timers
> 
> Still RFC pending the kernel patches actually being accepted
> upstream...

Apart from patch 2,

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Paolo

Re: [kvmarm] [RFC v5 7/8] hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC

[KONRAD Frédéric]

On 24/01/2013 16:43, Peter Maydell wrote:
> Implement support for using the KVM in-kernel GIC for ARM.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>   hw/a15mpcore.c       |    8 ++-
>   hw/arm/Makefile.objs |    1 +
>   hw/kvm/arm_gic.c     |  169 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 177 insertions(+), 1 deletion(-)
>   create mode 100644 hw/kvm/arm_gic.c
>
> diff --git a/hw/a15mpcore.c b/hw/a15mpcore.c
> index fe6c34c..1ca6f28 100644
> --- a/hw/a15mpcore.c
> +++ b/hw/a15mpcore.c
> @@ -19,6 +19,7 @@
>    */
>   
>   #include "sysbus.h"
> +#include "sysemu/kvm.h"
>   
>   /* A15MP private memory region.  */
>   
> @@ -40,8 +41,13 @@ static int a15mp_priv_init(SysBusDevice *dev)
>   {
>       A15MPPrivState *s = FROM_SYSBUS(A15MPPrivState, dev);
>       SysBusDevice *busdev;
> +    const char *gictype = "arm-gic";
s/arm-gic/arm_gic/                 ^^ ?

Christoffer and I had trouble with that:

qemu-system-arm: Unknown device 'arm-gic' for default sysbus

Fred
>   
> -    s->gic = qdev_create(NULL, "arm_gic");
> +    if (kvm_irqchip_in_kernel()) {
> +        gictype = "kvm-arm-gic";
> +    }
> +
> +    s->gic = qdev_create(NULL, gictype);
>       qdev_prop_set_uint32(s->gic, "num-cpu", s->num_cpu);
>       qdev_prop_set_uint32(s->gic, "num-irq", s->num_irq);

Re: [kvmarm] [RFC v5 7/8] hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC

[Andreas Färber]

Am 31.01.2013 11:52, schrieb KONRAD Frédéric:
> On 24/01/2013 16:43, Peter Maydell wrote:
>> Implement support for using the KVM in-kernel GIC for ARM.
>>
>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>> ---
>>   hw/a15mpcore.c       |    8 ++-
>>   hw/arm/Makefile.objs |    1 +
>>   hw/kvm/arm_gic.c     |  169
>> ++++++++++++++++++++++++++++++++++++++++++++++++++
>>   3 files changed, 177 insertions(+), 1 deletion(-)
>>   create mode 100644 hw/kvm/arm_gic.c
>>
>> diff --git a/hw/a15mpcore.c b/hw/a15mpcore.c
>> index fe6c34c..1ca6f28 100644
>> --- a/hw/a15mpcore.c
>> +++ b/hw/a15mpcore.c
>> @@ -19,6 +19,7 @@
>>    */
>>     #include "sysbus.h"
>> +#include "sysemu/kvm.h"
>>     /* A15MP private memory region.  */
>>   @@ -40,8 +41,13 @@ static int a15mp_priv_init(SysBusDevice *dev)
>>   {
>>       A15MPPrivState *s = FROM_SYSBUS(A15MPPrivState, dev);
>>       SysBusDevice *busdev;
>> +    const char *gictype = "arm-gic";
> s/arm-gic/arm_gic/                 ^^ ?
> 
> Christoffer and I had trouble with that:
> 
> qemu-system-arm: Unknown device 'arm-gic' for default sysbus

Since you already ran into issues here, even better would be to use a
TYPE_ARM_GIC constant or so.

Andreas

> 
> Fred
>>   -    s->gic = qdev_create(NULL, "arm_gic");
>> +    if (kvm_irqchip_in_kernel()) {
>> +        gictype = "kvm-arm-gic";
>> +    }
>> +
>> +    s->gic = qdev_create(NULL, gictype);
>>       qdev_prop_set_uint32(s->gic, "num-cpu", s->num_cpu);
>>       qdev_prop_set_uint32(s->gic, "num-irq", s->num_irq);
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [kvmarm] [RFC v5 7/8] hw/kvm/arm_gic: Implement support for KVM in-kernel ARM GIC

[Peter Maydell]

On 31 January 2013 10:54, Andreas Färber <afaerber@suse.de> wrote:
> Am 31.01.2013 11:52, schrieb KONRAD Frédéric:
>>> +    const char *gictype = "arm-gic";
>> s/arm-gic/arm_gic/                 ^^ ?
>>
>> Christoffer and I had trouble with that:
>>
>> qemu-system-arm: Unknown device 'arm-gic' for default sysbus

Oops, nice catch.

> Since you already ran into issues here, even better would be to use a
> TYPE_ARM_GIC constant or so.

Hmm, I kind of agree, but QOM idiom doesn't seem to encourage
having that define be publicly visible. Should we have a
hw/my_device.h [with the public bits like the TYPE_ and
FOO_CLASS/FOO_GET_CLASS macros] for every type? (and a
hw/my_device_priv.h if needed]

-- PMM