[Qemu-devel] [RFC 0/5] GICv3 state structs and migration

[Qemu-devel] [RFC 0/5] GICv3 state structs and migration

[Peter Maydell]

This is a very rough-and-ready RFC that takes Pavel's patch
series from last year:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg05284.html
and applies the various changes I recommended in code review.

I haven't tested this beyond compile-testing, but I think
that the general shape of it, and in particular the set
of information in the state structs, is correct. (In
particular I have removed several state struct fields that
should definitely not be present.)
There are a few fixme-type comments scattered in the code.

Shlomo, I think this should be a sensible thing to base your
GICv3 emulation patches on.

thanks
-- PMM

Pavel Fedin (4):
  hw/intc/arm_gicv3_common: Add state information
  kernel: Add definitions for GICv3 attributes
  hw/intc/arm_gicv3_kvm: Implement get/put functions
  hw/intc/arm_gicv3_common: Add vmstate descriptors

Peter Maydell (1):
  migration: Define VMSTATE_UINT64_2DARRAY

 hw/intc/arm_gicv3_common.c         | 161 +++++++++++++-
 hw/intc/arm_gicv3_kvm.c            | 437 ++++++++++++++++++++++++++++++++++++-
 hw/intc/gicv3_internal.h           | 147 +++++++++++++
 include/hw/intc/arm_gicv3_common.h | 106 ++++++++-
 include/migration/vmstate.h        |  15 ++
 linux-headers/asm-arm64/kvm.h      |  17 +-
 6 files changed, 869 insertions(+), 14 deletions(-)
 create mode 100644 hw/intc/gicv3_internal.h

-- 
1.9.1

[Qemu-devel] [RFC 1/5] hw/intc/arm_gicv3_common: Add state information

[Peter Maydell]

From: Pavel Fedin <p.fedin@samsung.com>

Add state information to GICv3 object structure and implement
arm_gicv3_common_reset(). Also, add some functions for registers which are
not stored directly but simulated.

State information includes not only pure GICv3 data, but also some legacy
registers. This will be useful for implementing software emulation of GICv3
with v2 backwards compatilibity mode.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
[PMM: significantly overhauled:
 * Add missing qom/cpu.h include
 * Remove legacy-only state fields (we can add them later if/when we add
   legacy emulation)
 * Add various missing register offset #defines
 * Accessor macros removed entirely
 * Fields in state structures renamed to match architectural register names
 * Corrected the reset value for GICR_IENABLER0 since we don't support
   legacy mode
 * Added ARM_LINUX_BOOT_IF interface for "we are directly booting a kernel in
   non-secure" so that we can fake up the firmware-mandated reconfiguration
   only when we need it
]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/intc/arm_gicv3_common.c         | 116 ++++++++++++++++++++++++++++-
 hw/intc/gicv3_internal.h           | 147 +++++++++++++++++++++++++++++++++++++
 include/hw/intc/arm_gicv3_common.h | 106 +++++++++++++++++++++++++-
 3 files changed, 364 insertions(+), 5 deletions(-)
 create mode 100644 hw/intc/gicv3_internal.h

diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index e4f0f5a..a1ce4c1 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -3,8 +3,9 @@
  *
  * Copyright (c) 2012 Linaro Limited
  * Copyright (c) 2015 Huawei.
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Written by Peter Maydell
- * Extended to 64 cores by Shlomo Pongratz
+ * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -21,7 +22,10 @@
  */
 
 #include "qemu/osdep.h"
+#include "qom/cpu.h"
 #include "hw/intc/arm_gicv3_common.h"
+#include "gicv3_internal.h"
+#include "hw/arm/linux-boot-if.h"
 
 static void gicv3_pre_save(void *opaque)
 {
@@ -89,6 +93,8 @@ void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
 static void arm_gicv3_common_realize(DeviceState *dev, Error **errp)
 {
     GICv3State *s = ARM_GICV3_COMMON(dev);
+    Object *cpu;
+    int i;
 
     /* revision property is actually reserved and currently used only in order
      * to keep the interface compatible with GICv2 code, avoiding extra
@@ -99,11 +105,111 @@ static void arm_gicv3_common_realize(DeviceState *dev, Error **errp)
         error_setg(errp, "unsupported GIC revision %d", s->revision);
         return;
     }
+
+    if (s->num_irq > GICV3_MAXIRQ) {
+        error_setg(errp,
+                   "requested %u interrupt lines exceeds GIC maximum %d",
+                   s->num_irq, GICV3_MAXIRQ);
+        return;
+    }
+
+    s->cpu = g_new0(GICv3CPUState, s->num_cpu);
+
+    for (i = 0; i < s->num_cpu; i++) {
+        cpu = OBJECT(qemu_get_cpu(i));
+        s->cpu[i].affinity_id = object_property_get_int(cpu, "mp-affinity",
+                                                        NULL);
+    }
 }
 
 static void arm_gicv3_common_reset(DeviceState *dev)
 {
-    /* TODO */
+    GICv3State *s = ARM_GICV3_COMMON(dev);
+    int i;
+
+    for (i = 0; i < s->num_cpu; i++) {
+        GICv3CPUState *c = &s->cpu[i];
+
+        c->gicr_waker = GICR_WAKER_ProcessorSleep | GICR_WAKER_ChildrenAsleep;
+        c->gicr_ctlr = 0;
+        c->gicr_propbaser = 0;
+        c->gicr_pendbaser = 0;
+        /* If we're resetting a TZ-aware GIC as if secure firmware
+         * had set it up ready to start a kernel in non-secure, we
+         * need to set interrupts to group 1 so the kernel can use them.
+         * Otherwise they reset to group 0 like the hardware.
+         */
+        if (s->security_extn && s->irq_reset_nonsecure) {
+            c->gicr_igroupr0 = 0xffffffff;
+        } else {
+            c->gicr_igroupr0 = 0;
+        }
+
+        c->gicr_ienabler0 = 0;
+        c->gicr_ipendr0 = 0;
+        c->gicr_iactiver0 = 0;
+        c->gicr_icfgr0 = 0xaaaaaaaa;
+        c->gicr_icfgr1 = 0;
+        memset(c->gicr_ipriorityr, 0, sizeof(c->gicr_ipriorityr));
+        c->level = 0;
+
+        c->icc_ctlr_el1[0] = 0;
+        c->icc_ctlr_el1[1] = 0;
+        c->icc_igrpen0_el1 = 0;
+        c->icc_igrpen1_el1 = 0;
+        c->icc_pmr_el1 = 0;
+        c->icc_bpr[0] = GIC_MIN_BPR0;
+        c->icc_bpr[1] = GIC_MIN_BPR1;
+        memset(c->icc_ap1r, 0, sizeof(c->icc_ap1r));
+    }
+
+    memset(s->group, 0, sizeof(s->group));
+    memset(s->enabled, 0, sizeof(s->enabled));
+    memset(s->pending, 0, sizeof(s->pending));
+    memset(s->active, 0, sizeof(s->active));
+    memset(s->level, 0, sizeof(s->level));
+    memset(s->edge_trigger, 0, sizeof(s->edge_trigger));
+
+    /* Workaround! (the same as c->group above) */
+    for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+        set_bit(i - GIC_INTERNAL, s->group);
+    }
+
+    /* By default all interrupts always target CPU #0 */
+    for (i = 0; i < GICV3_MAXSPI; i++) {
+        s->gicd_itargetsr[i] = 1;
+    }
+    memset(s->gicd_irouter, 0, sizeof(s->gicd_irouter));
+    memset(s->gicd_ipriority, 0, sizeof(s->gicd_ipriority));
+
+    /* With all configuration we don't support GICv2 backwards computability */
+    if (s->security_extn) {
+        /* GICv3 5.3.20 With two security So DS is RAZ/WI ARE_NS is RAO/WI
+         * and ARE_S is RAO/WI
+         */
+         s->gicd_ctlr = GICD_CTLR_ARE_S | GICD_CTLR_ARE_NS;
+    } else {
+        /* GICv3 5.3.20 With one security So DS is RAO/WI ARE is RAO/WI
+         */
+        s->gicd_ctlr = GICD_CTLR_DS | GICD_CTLR_ARE;
+    }
+}
+
+static void arm_gic_common_linux_init(ARMLinuxBootIf *obj,
+                                      bool secure_boot)
+{
+    GICState *s = ARM_GIC_COMMON(obj);
+
+    if (s->security_extn && !secure_boot) {
+        /* We're directly booting a kernel into NonSecure. If this GIC
+         * implements the security extensions then we must configure it
+         * to have all the interrupts be NonSecure (this is a job that
+         * is done by the Secure boot firmware in real hardware, and in
+         * this mode QEMU is acting as a minimalist firmware-and-bootloader
+         * equivalent).
+         */
+        s->irq_reset_nonsecure = true;
+    }
 }
 
 static Property arm_gicv3_common_properties[] = {
@@ -117,11 +223,13 @@ static Property arm_gicv3_common_properties[] = {
 static void arm_gicv3_common_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
+    ARMLinuxBootIfClass *albifc = ARM_LINUX_BOOT_IF_CLASS(klass);
 
     dc->reset = arm_gicv3_common_reset;
     dc->realize = arm_gicv3_common_realize;
     dc->props = arm_gicv3_common_properties;
     dc->vmsd = &vmstate_gicv3;
+    albifc->arm_linux_init = arm_gic_common_linux_init;
 }
 
 static const TypeInfo arm_gicv3_common_type = {
@@ -131,6 +239,10 @@ static const TypeInfo arm_gicv3_common_type = {
     .class_size = sizeof(ARMGICv3CommonClass),
     .class_init = arm_gicv3_common_class_init,
     .abstract = true,
+    .interfaces = (InterfaceInfo []) {
+        { TYPE_ARM_LINUX_BOOT_IF },
+        { },
+    },
 };
 
 static void register_types(void)
diff --git a/hw/intc/gicv3_internal.h b/hw/intc/gicv3_internal.h
new file mode 100644
index 0000000..9f9a45e
--- /dev/null
+++ b/hw/intc/gicv3_internal.h
@@ -0,0 +1,147 @@
+/*
+ * ARM GICv3 support - internal interfaces
+ *
+ * Copyright (c) 2012 Linaro Limited
+ * Copyright (c) 2015 Huawei.
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
+ * Written by Peter Maydell
+ * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_ARM_GICV3_INTERNAL_H
+#define QEMU_ARM_GICV3_INTERNAL_H
+
+#include "hw/intc/arm_gicv3_common.h"
+
+/* Distributor registers, as offsets from the distributor base address */
+#define GICD_CTLR            0x0000
+#define GICD_TYPER           0x0004
+#define GICD_IIDR            0x0008
+#define GICD_STATUSR         0x0010
+#define GICD_SETSPI_NSR      0x0040
+#define GICD_CLRSPI_NSR      0x0048
+#define GICD_SETSPI_SR       0x0050
+#define GICD_CLRSPI_SR       0x0058
+#define GICD_SEIR            0x0068
+#define GICD_IGROUPR         0x0080
+#define GICD_ISENABLER       0x0100
+#define GICD_ICENABLER       0x0180
+#define GICD_ISPENDR         0x0200
+#define GICD_ICPENDR         0x0280
+#define GICD_ISACTIVER       0x0300
+#define GICD_ICACTIVER       0x0380
+#define GICD_IPRIORITYR      0x0400
+#define GICD_ITARGETSR       0x0800
+#define GICD_ICFGR           0x0C00
+#define GICD_IGRPMODR        0x0D00
+#define GICD_NASCR           0x0E00
+#define GICD_SGIR            0x0F00
+#define GICD_CPENDSGIR       0x0F10
+#define GICD_SPENDSGIR       0x0F20
+#define GICD_IROUTER         0x6000
+#define GICD_PIDR2           0xFFE8
+
+/* GICD_CTLR fields  */
+#define GICD_CTLR_EN_GRP0           (1U << 0)
+#define GICD_CTLR_EN_GRP1NS         (1U << 1) /* GICv3 5.3.20 */
+#define GICD_CTLR_EN_GRP1S          (1U << 2)
+#define GICD_CTLR_EN_GRP1_ALL       (GICD_CTLR_EN_GRP1NS | GICD_CTLR_EN_GRP1S)
+/* Bit 4 is ARE if the system doesn't support TrustZone, ARE_S otherwise */
+#define GICD_CTLR_ARE               (1U << 4)
+#define GICD_CTLR_ARE_S             (1U << 4)
+#define GICD_CTLR_ARE_NS            (1U << 5)
+#define GICD_CTLR_DS                (1U << 6)
+#define GICD_CTLR_E1NWF             (1U << 7)
+#define GICD_CTLR_RWP               (1U << 31)
+
+/*
+ * Redistributor frame offsets from RD_base
+ */
+#define GICR_SGI_OFFSET 0x10000
+
+/*
+ * Redistributor registers, offsets from RD_base
+ */
+#define GICR_CTLR             0x0000
+#define GICR_IIDR             0x0004
+#define GICR_TYPER            0x0008
+#define GICR_STATUSR          0x0010
+#define GICR_WAKER            0x0014
+#define GICR_SETLPIR          0x0040
+#define GICR_CLRLPIR          0x0048
+#define GICR_PROPBASER        0x0070
+#define GICR_PENDBASER        0x0078
+#define GICR_INVLPIR          0x00A0
+#define GICR_INVALLR          0x00B0
+#define GICR_SYNCR            0x00C0
+#define GICR_PIDR2            0xFFE8
+
+/* SGI and PPI Redistributor registers, offsets from SGI_base */
+#define GICR_IGROUPR0         0x0080
+#define GICR_ISENABLER0       0x0100
+#define GICR_ICENABLER0       0x0180
+#define GICR_ISPENDR0         0x0200
+#define GICR_ICPENDR0         0x0280
+#define GICR_ISACTIVER0       0x0300
+#define GICR_ICACTIVER0       0x0380
+#define GICR_IPRIORITYR0      0x0400
+#define GICR_ICFGR0           0x0C00
+#define GICR_ICFGR1           0x0C04
+#define GICR_IGRPMODR0        0x0D00
+#define GICR_NSACR            0x0E00
+
+#define GICR_CTLR_ENABLE_LPIS        (1U << 0)
+#define GICR_CTLR_RWP                (1U << 3)
+#define GICR_CTLR_DPG0               (1U << 24)
+#define GICR_CTLR_DPG1NS             (1U << 25)
+#define GICR_CTLR_DPG1S              (1U << 26)
+#define GICR_CTLR_UWP                (1U << 31)
+
+#define GICR_TYPER_PLPIS             (1U << 0)
+#define GICR_TYPER_VLPIS             (1U << 1)
+#define GICR_TYPER_DIRECTLPI         (1U << 3)
+#define GICR_TYPER_LAST              (1U << 4)
+#define GICR_TYPER_DPGS              (1U << 5)
+#define GICR_TYPER_PROCNUM           (0xFFFFU << 8)
+#define GICR_TYPER_COMMONLPIAFF      (0x3 << 24)
+#define GICR_TYPER_AFFINITYVALUE     (0xFFFFFFFFULL << 32)
+
+#define GICR_WAKER_ProcessorSleep    (1U << 1)
+#define GICR_WAKER_ChildrenAsleep    (1U << 2)
+
+#define GICR_PROPBASER_OUTER_CACHEABILITY_MASK (7ULL << 56)
+#define GICR_PROPBASER_ADDR_MASK               (0xfffffffffULL << 12)
+#define GICR_PROPBASER_SHAREABILITY_MASK       (3U << 10)
+#define GICR_PROPBASER_CACHEABILITY_MASK       (7U << 7)
+#define GICR_PROPBASER_IDBITS_MASK             (0x1f)
+
+#define GICR_PENDBASER_PTZ                     (1ULL << 62)
+#define GICR_PENDBASER_OUTER_CACHEABILITY_MASK (7ULL << 56)
+#define GICR_PENDBASER_ADDR_MASK               (0xffffffffULL << 16)
+#define GICR_PENDBASER_SHAREABILITY_MASK       (3U << 10)
+#define GICR_PENDBASER_CACHEABILITY_MASK       (7U << 7)
+
+#define ICC_CTLR_EL1_CBPR           (1U << 0)
+#define ICC_CTLR_EL1_EOIMODE        (1U << 1)
+#define ICC_CTLR_EL1_PMHE           (1U << 6)
+#define ICC_CTLR_EL1_SEIS           (1U << 14)
+#define ICC_CTLR_EL1_A3V            (1U << 15)
+
+#define ICC_PMR_PRIORITY_MASK    0xff
+#define ICC_BPR_BINARYPOINT_MASK 0x07
+#define ICC_IGRPEN_ENABLE        0x01
+
+#endif /* !QEMU_ARM_GIC_INTERNAL_H */
diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index c2fd8da..84e9406 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -3,8 +3,9 @@
  *
  * Copyright (c) 2012 Linaro Limited
  * Copyright (c) 2015 Huawei.
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Written by Peter Maydell
- * Extended to 64 cores by Shlomo Pongratz
+ * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -26,7 +27,48 @@
 #include "hw/sysbus.h"
 #include "hw/intc/arm_gic_common.h"
 
-typedef struct GICv3State {
+/*
+ * Maximum number of possible interrupts, determined by the GIC architecture.
+ * Note that this does not include LPIs. When implemented, these should be
+ * dealt with separately.
+ */
+#define GICV3_MAXIRQ 1020
+#define GICV3_MAXSPI (GICV3_MAXIRQ - GIC_INTERNAL)
+
+#define GIC_MIN_BPR0 0
+#define GIC_MIN_BPR1 (GIC_MIN_BPR0 + 1)
+
+struct GICv3CPUState {
+    uint64_t affinity_id;            /* Cached affinity ID of the CPU */
+
+    /* Redistributor */
+    uint32_t level;                  /* Current IRQ level */
+    /* RD_base page registers */
+    uint32_t gicr_ctlr;
+    uint32_t gicr_waker;
+    uint64_t gicr_propbaser;
+    uint64_t gicr_pendbaser;
+    /* SGI_base page registers */
+    uint32_t gicr_igroupr0;
+    uint32_t gicr_ienabler0;
+    uint32_t gicr_ipendr0;
+    uint32_t gicr_iactiver0;
+    uint32_t gicr_icfgr0;
+    uint32_t gicr_icfgr1;
+    uint8_t gicr_ipriorityr[GIC_INTERNAL];
+
+    /* CPU interface */
+    uint64_t icc_ctlr_el1[2];
+    uint64_t icc_pmr_el1;
+    uint64_t icc_bpr[2];
+    uint64_t icc_ap1r[4][2];
+    uint64_t icc_igrpen0_el1;
+    uint64_t icc_igrpen1_el1;
+};
+
+typedef struct GICv3CPUState GICv3CPUState;
+
+struct GICv3State {
     /*< private >*/
     SysBusDevice parent_obj;
     /*< public >*/
@@ -41,9 +83,61 @@ typedef struct GICv3State {
     uint32_t num_irq;
     uint32_t revision;
     bool security_extn;
+    bool irq_reset_nonsecure;
 
     int dev_fd; /* kvm device fd if backed by kvm vgic support */
-} GICv3State;
+    Error *migration_blocker;
+
+    /* Distributor */
+
+    /* for a GIC with the security extensions the NS banked version of this
+     * register is just an alias of bit 1 of the S banked version.
+     */
+    uint32_t gicd_ctlr;
+    DECLARE_BITMAP(group, GICV3_MAXSPI);        /* GICD_IGROUPR */
+    DECLARE_BITMAP(enabled, GICV3_MAXSPI);      /* GICD_ISENABLER */
+    DECLARE_BITMAP(pending, GICV3_MAXSPI);      /* GICD_ISPENDR */
+    DECLARE_BITMAP(active, GICV3_MAXSPI);       /* GICD_ISACTIVER */
+    DECLARE_BITMAP(level, GICV3_MAXSPI);        /* Current level */
+    DECLARE_BITMAP(edge_trigger, GICV3_MAXSPI); /* GICD_ICFGR */
+    uint8_t gicd_ipriority[GICV3_MAXSPI];
+    uint8_t gicd_itargetsr[GICV3_MAXSPI];
+    uint64_t gicd_irouter[GICV3_MAXSPI];
+
+    GICv3CPUState *cpu;
+};
+
+typedef struct GICv3State GICv3State;
+
+#define GICV3_BITMAP_ACCESSORS(BMP)                                     \
+    static inline void gicv3_gicd_##BMP##_set(GICv3State *s, int irq)   \
+    {                                                                   \
+        set_bit(irq - GIC_INTERNAL, s->BMP);                            \
+    }                                                                   \
+    static inline int gicv3_gicd_##BMP##_test(GICv3State *s, int irq)   \
+    {                                                                   \
+        return test_bit(irq - GIC_INTERNAL, s->BMP);                    \
+    }                                                                   \
+    static inline void gicv3_gicd_##BMP##_clear(GICv3State *s, int irq) \
+    {                                                                   \
+        clear_bit(irq - GIC_INTERNAL, s->BMP);                          \
+    }                                                                   \
+    static inline void gicv3_gicd_##BMP##_replace(GICv3State *s,        \
+                                                  int irq, int value)   \
+    {                                                                   \
+        if (value) {                                                    \
+            gicv3_gicd_##BMP##_set(s, irq);                             \
+        } else {                                                        \
+            gicv3_gicd_##BMP##_clear(s, irq);                           \
+        }                                                               \
+    }
+
+GICV3_BITMAP_ACCESSORS(group)
+GICV3_BITMAP_ACCESSORS(enabled)
+GICV3_BITMAP_ACCESSORS(pending)
+GICV3_BITMAP_ACCESSORS(active)
+GICV3_BITMAP_ACCESSORS(level)
+GICV3_BITMAP_ACCESSORS(edge_trigger)
 
 #define TYPE_ARM_GICV3_COMMON "arm-gicv3-common"
 #define ARM_GICV3_COMMON(obj) \
@@ -65,4 +159,10 @@ typedef struct ARMGICv3CommonClass {
 void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
                               const MemoryRegionOps *ops);
 
+/* Accessors for simulated system registers */
+uint32_t gicv3_get_igrpen0(GICv3State *s, int cpuindex);
+void gicv3_set_igrpen0(GICv3State *s, int cpuindex, uint32_t val);
+uint32_t gicv3_get_igrpen1(GICv3State *s, int cpuindex);
+void gicv3_set_igrpen1(GICv3State *s, int cpuindex, uint32_t val);
+
 #endif
-- 
1.9.1

[Qemu-devel] [RFC 2/5] kernel: Add definitions for GICv3 attributes

[Peter Maydell]

From: Pavel Fedin <p.fedin@samsung.com>

This temporary patch adds kernel API definitions. Use proper header update
procedure after these features are released.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 linux-headers/asm-arm64/kvm.h | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
index a2fd4d9..f76a90c 100644
--- a/linux-headers/asm-arm64/kvm.h
+++ b/linux-headers/asm-arm64/kvm.h
@@ -179,14 +179,14 @@ struct kvm_arch_memory_slot {
 	KVM_REG_ARM64_SYSREG_ ## n ## _MASK)
 
 #define __ARM64_SYS_REG(op0,op1,crn,crm,op2) \
-	(KVM_REG_ARM64 | KVM_REG_ARM64_SYSREG | \
-	ARM64_SYS_REG_SHIFT_MASK(op0, OP0) | \
+	(ARM64_SYS_REG_SHIFT_MASK(op0, OP0) | \
 	ARM64_SYS_REG_SHIFT_MASK(op1, OP1) | \
 	ARM64_SYS_REG_SHIFT_MASK(crn, CRN) | \
 	ARM64_SYS_REG_SHIFT_MASK(crm, CRM) | \
 	ARM64_SYS_REG_SHIFT_MASK(op2, OP2))
 
-#define ARM64_SYS_REG(...) (__ARM64_SYS_REG(__VA_ARGS__) | KVM_REG_SIZE_U64)
+#define ARM64_SYS_REG(...) (__ARM64_SYS_REG(__VA_ARGS__) | KVM_REG_ARM64 | \
+			    KVM_REG_SIZE_U64 | KVM_REG_ARM64_SYSREG)
 
 #define KVM_REG_ARM_TIMER_CTL		ARM64_SYS_REG(3, 3, 14, 3, 1)
 #define KVM_REG_ARM_TIMER_CNT		ARM64_SYS_REG(3, 3, 14, 3, 2)
@@ -197,12 +197,21 @@ struct kvm_arch_memory_slot {
 #define KVM_DEV_ARM_VGIC_GRP_DIST_REGS	1
 #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS	2
 #define   KVM_DEV_ARM_VGIC_CPUID_SHIFT	32
-#define   KVM_DEV_ARM_VGIC_CPUID_MASK	(0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define   KVM_DEV_ARM_VGIC_CPUID_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
+#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
+#define KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS 6
+#define   KVM_DEV_ARM_VGIC_SYSREG_MASK	(KVM_REG_ARM64_SYSREG_OP0_MASK | \
+					 KVM_REG_ARM64_SYSREG_OP1_MASK | \
+					 KVM_REG_ARM64_SYSREG_CRN_MASK | \
+					 KVM_REG_ARM64_SYSREG_CRM_MASK | \
+					 KVM_REG_ARM64_SYSREG_OP2_MASK)
+#define   KVM_DEV_ARM_VGIC_SYSREG(op0,op1,crn,crm,op2) \
+					__ARM64_SYS_REG(op0,op1,crn,crm,op2)
 
 /* KVM_IRQ_LINE irq field index values */
 #define KVM_ARM_IRQ_TYPE_SHIFT		24
-- 
1.9.1

[Qemu-devel] [RFC 3/5] hw/intc/arm_gicv3_kvm: Implement get/put functions

[Peter Maydell]

From: Pavel Fedin <p.fedin@samsung.com>

This actually implements pre_save and post_load methods for in-kernel
vGICv3.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
[PMM:
 * use decimal, not 0bnnn
 * fixed typo in names of ICC_APR0R_EL1 and ICC_AP1R_EL1
 * completely rearranged the get and put functions to read and write
   the state in a natural order, rather than mixing distributor and
   redistributor state together]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/intc/arm_gicv3_kvm.c | 437 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 433 insertions(+), 4 deletions(-)

diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 90c7950..b674b01 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -22,8 +22,11 @@
 #include "qemu/osdep.h"
 #include "hw/intc/arm_gicv3_common.h"
 #include "hw/sysbus.h"
+#include "migration/migration.h"
+#include "qemu/error-report.h"
 #include "sysemu/kvm.h"
 #include "kvm_arm.h"
+#include "gicv3_internal.h"
 #include "vgic_common.h"
 
 #ifdef DEBUG_GICV3_KVM
@@ -42,6 +45,23 @@
 #define KVM_ARM_GICV3_GET_CLASS(obj) \
      OBJECT_GET_CLASS(KVMARMGICv3Class, (obj), TYPE_KVM_ARM_GICV3)
 
+#define ICC_PMR_EL1     \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 4, 6, 0)
+#define ICC_BPR0_EL1    \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 8, 3)
+#define ICC_AP0R_EL1(n) \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 8, 4 | n)
+#define ICC_AP1R_EL1(n) \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 9, n)
+#define ICC_BPR1_EL1    \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 12, 3)
+#define ICC_CTLR_EL1    \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 12, 4)
+#define ICC_IGRPEN0_EL1 \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 12, 6)
+#define ICC_IGRPEN1_EL1 \
+    KVM_DEV_ARM_VGIC_SYSREG(3, 0, 12, 12, 7)
+
 typedef struct KVMARMGICv3Class {
     ARMGICv3CommonClass parent_class;
     DeviceRealize parent_realize;
@@ -55,16 +75,412 @@ static void kvm_arm_gicv3_set_irq(void *opaque, int irq, int level)
     kvm_arm_gic_set_irq(s->num_irq, irq, level);
 }
 
+#define VGIC_CPUID(cpuid) ((((cpuid) & ARM_AFF3_MASK) >> 8) | \
+                           ((cpuid) & ARM32_AFFINITY_MASK))
+#define KVM_VGIC_ATTR(reg, cpuid) \
+    ((VGIC_CPUID(cpuid) << KVM_DEV_ARM_VGIC_CPUID_SHIFT) | (reg))
+
+static inline void kvm_gicd_access(GICv3State *s, int offset,
+                                   uint64_t *val, bool write)
+{
+    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_DIST_REGS,
+                      KVM_VGIC_ATTR(offset, 0),
+                      val, write);
+}
+
+static inline void kvm_gicr_access(GICv3State *s, int offset, int cpu,
+                                   uint64_t *val, bool write)
+{
+    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_REDIST_REGS,
+                      KVM_VGIC_ATTR(offset, s->cpu[cpu].affinity_id),
+                      val, write);
+}
+
+static inline void kvm_gicc_access(GICv3State *s, uint64_t reg, int cpu,
+                                   uint64_t *val, bool write)
+{
+    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+                      KVM_VGIC_ATTR(reg, s->cpu[cpu].affinity_id),
+                      val, write);
+}
+
+/*
+ * Translate from the in-kernel field for an IRQ value to/from the qemu
+ * representation. Note that these are only expected to be used for
+ * SPIs (that is, for interrupts whose state is in the distributor
+ * rather than the redistributor).
+ */
+typedef void (*vgic_translate_fn)(GICv3State *s, int irq,
+                                  uint32_t *field, bool to_kernel);
+
+/* synthetic translate function used for clear/set registers to completely
+ * clear a setting using a clear-register before setting the remaining bits
+ * using a set-register */
+static void translate_clear(GICv3State *s, int irq,
+                            uint32_t *field, bool to_kernel)
+{
+    if (to_kernel) {
+        *field = ~0;
+    } else {
+        /* does not make sense: qemu model doesn't use set/clear regs */
+        abort();
+    }
+}
+
+#define GIC_BMP_TRANSLATE_FN(BMP) \
+    static void translate_##BMP(GICv3State *s, int irq, \
+                                uint32_t *field, bool to_kernel) \
+    {                                                            \
+    if (to_kernel) {                                             \
+        *field = gicv3_gicd_##BMP##_test(s, irq);                \
+    } else {                                                     \
+        gicv3_gicd_##BMP##_replace(s, irq, *field);              \
+    }                                                            \
+}
+
+GIC_BMP_TRANSLATE_FN(group)
+GIC_BMP_TRANSLATE_FN(enabled)
+GIC_BMP_TRANSLATE_FN(active)
+GIC_BMP_TRANSLATE_FN(edge_trigger)
+
+static void translate_pending(GICv3State *s, int irq,
+                              uint32_t *field, bool to_kernel)
+{
+    if (to_kernel) {
+        *field = gicv3_gicd_pending_test(s, irq);
+    } else {
+        gicv3_gicd_pending_replace(s, irq, *field);
+        /* TODO: Capture if level-line is held high in the kernel */
+    }
+}
+static void translate_priority(GICv3State *s, int irq,
+                               uint32_t *field, bool to_kernel)
+{
+    if (to_kernel) {
+        *field = s->gicd_ipriority[irq - GIC_INTERNAL];
+    } else {
+        s->gicd_ipriority[irq - GIC_INTERNAL] = *field;
+    }
+}
+
+/* Loop through each distributor IRQ related register; since bits
+ * corresponding to SPIs and PPIs are RAZ/WI when affinity routing
+ * is enabled, we skip those.
+ */
+#define for_each_dist_irq_reg(_irq, _max, _field_width) \
+    for (_irq = GIC_INTERNAL; _irq < _max; _irq += (32 / _field_width))
+
+/* Read a register group from the kernel VGIC */
+static void kvm_dist_get(GICv3State *s, uint32_t offset, int width,
+                         vgic_translate_fn translate_fn)
+{
+    uint64_t reg;
+    int j;
+    int irq;
+    uint32_t field;
+    int regsz = 32 / width; /* irqs per kernel register */
+
+    for_each_dist_irq_reg(irq, s->num_irq, width) {
+        kvm_gicd_access(s, offset, &reg, false);
+
+        for (j = 0; j < regsz; j++) {
+            field = extract32(reg, j * width, width);
+            translate_fn(s, irq + j, &field, false);
+        }
+        offset += 4;
+    }
+}
+
+/* Write a register group to the kernel VGIC */
+static void kvm_dist_put(GICv3State *s, uint32_t offset, int width,
+                         vgic_translate_fn translate_fn)
+{
+    uint64_t reg;
+    int j;
+    int irq;
+    uint32_t field;
+    int regsz = 32 / width; /* irqs per kernel register */
+
+    for_each_dist_irq_reg(irq, s->num_irq, width) {
+        reg = 0;
+        for (j = 0; j < regsz; j++) {
+            translate_fn(s, irq + j, &field, true);
+            reg = deposit32(reg, j * width, width, field);
+        }
+        kvm_gicd_access(s, offset, &reg, true);
+        offset += 4;
+    }
+}
+
+static void kvm_arm_gicv3_check(GICv3State *s)
+{
+    uint64_t reg;
+    uint32_t num_irq;
+
+    /* Sanity checking s->num_irq */
+    kvm_gicd_access(s, GICD_TYPER, &reg, false);
+    num_irq = ((reg & 0x1f) + 1) * 32;
+
+    if (num_irq < s->num_irq) {
+        error_report("Model requests %u IRQs, but kernel supports max %u\n",
+                     s->num_irq, num_irq);
+        abort();
+    }
+
+    /* TODO: Consider checking compatibility with the IIDR ? */
+}
+
 static void kvm_arm_gicv3_put(GICv3State *s)
 {
-    /* TODO */
-    DPRINTF("Cannot put kernel gic state, no kernel interface\n");
+    uint64_t reg, redist_typer;
+    int ncpu, i;
+
+    kvm_arm_gicv3_check(s);
+
+    kvm_gicr_access(s, GICR_TYPER, 0, &redist_typer, false);
+
+    reg = s->gicd_ctlr;
+    kvm_gicd_access(s, GICD_CTLR, &reg, true);
+
+    if (redist_typer & GICR_TYPER_PLPIS) {
+        /* Set base addresses before LPIs are enabled by GICR_CTLR write */
+        for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+            GICv3CPUState *c = &s->cpu[ncpu];
+
+            reg = c->gicr_propbaser;
+            kvm_gicr_access(s, GICR_PROPBASER, ncpu, &reg, true);
+
+            reg = c->gicr_pendbaser;
+            if (!c->gicr_ctlr & GICR_CTLR_ENABLE_LPIS) {
+                /* Setting PTZ is advised if LPIs are disabled, to reduce
+                 * GIC initialization time.
+                 */
+                reg |= GICR_PENDBASER_PTZ;
+            }
+            kvm_gicr_access(s, GICR_PENDBASER, ncpu, &reg, true);
+        }
+    }
+
+    /* Redistributor state (one per CPU) */
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        GICv3CPUState *c = &s->cpu[ncpu];
+
+        reg = c->gicr_ctlr;
+        kvm_gicr_access(s, GICR_CTLR, ncpu, &reg, true);
+
+        reg = c->gicr_waker;
+        kvm_gicr_access(s, GICR_WAKER, ncpu, &reg, true);
+
+        reg = c->gicr_igroupr0;
+        kvm_gicr_access(s, GICR_IGROUPR0, ncpu, &reg, true);
+
+        reg = ~0;
+        kvm_gicr_access(s, GICR_ICENABLER0, ncpu, &reg, true);
+        reg = c->gicr_ienabler0;
+        kvm_gicr_access(s, GICR_ISENABLER0, ncpu, &reg, true);
+
+        /* Restore config before pending so we treat level/edge correctly */
+        reg = c->gicr_icfgr0;
+        kvm_gicr_access(s, GICR_ICFGR0, ncpu, &reg, true);
+        reg = c->gicr_icfgr1;
+        kvm_gicr_access(s, GICR_ICFGR1, ncpu, &reg, true);
+
+        reg = ~0;
+        kvm_gicr_access(s, GICR_ICPENDR0, ncpu, &reg, true);
+        // FIXME do we need the hack here for mixing level with pending ?
+        reg = c->gicr_ipendr0;
+        kvm_gicr_access(s, GICR_ISPENDR0, ncpu, &reg, true);
+
+        reg = ~0;
+        kvm_gicr_access(s, GICR_ICACTIVER0, ncpu, &reg, true);
+        reg = c->gicr_iactiver0;
+        kvm_gicr_access(s, GICR_ISACTIVER0, ncpu, &reg, true);
+
+        for (i = 0; i < GIC_INTERNAL; i += 4) {
+            reg = c->gicr_ipriorityr[i] |
+                (c->gicr_ipriorityr[i + 1] << 8) |
+                (c->gicr_ipriorityr[i + 2] << 16) |
+                (c->gicr_ipriorityr[i + 3] << 24);
+            kvm_gicr_access(s, GICR_IPRIORITYR0 + i, ncpu, &reg, true);
+        }
+    }
+
+    /* Distributor state (shared between all CPUs */
+
+    /* s->enable bitmap -> GICD_ISENABLERn */
+    kvm_dist_put(s, GICD_ICENABLER, 1, translate_clear);
+    kvm_dist_put(s, GICD_ISENABLER, 1, translate_enabled);
+
+    /* s->group bitmap -> GICD_IGROUPRn */
+    kvm_dist_put(s, GICD_IGROUPR, 1, translate_group);
+
+    /* Restore targets before pending to ensure the pending state is set on
+     * the appropriate CPU interfaces in the kernel
+     */
+
+    /* s->gicd_irouter[irq] -> GICD_IROUTERn
+     * We can't use kvm_dist_put() here because the registers are 64-bit
+     */
+    for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+        uint32_t offset = GICD_IROUTER + (sizeof(uint64_t) * i);
+
+        kvm_gicd_access(s, offset, &s->gicd_irouter[i - GIC_INTERNAL], true);
+    }
+
+    /* s->trigger bitmap -> GICD_ICFGRn
+     * (restore configuration registers before pending IRQs so we treat
+     * level/edge correctly)
+     */
+    kvm_dist_put(s, GICD_ICFGR, 2, translate_edge_trigger);
+
+    /* s->pending bitmap + s->level bitmap -> GICD_ISPENDRn */
+    kvm_dist_put(s, GICD_ICPENDR, 1, translate_clear);
+    kvm_dist_put(s, GICD_ISPENDR, 1, translate_pending);
+
+    /* s->active bitmap -> GICD_ISACTIVERn */
+    kvm_dist_put(s, GICD_ICACTIVER, 1, translate_clear);
+    kvm_dist_put(s, GICD_ISACTIVER, 1, translate_active);
+
+    /* s->gicd_ipriority[] -> GICD_IPRIORITYRn */
+    kvm_dist_put(s, GICD_IPRIORITYR, 8, translate_priority);
+
+    /* CPU Interface state (one per CPU) */
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        GICv3CPUState *c = &s->cpu[ncpu];
+
+        kvm_gicc_access(s, ICC_CTLR_EL1, ncpu, &c->icc_ctlr_el1[1], true);
+        kvm_gicc_access(s, ICC_IGRPEN0_EL1, ncpu, &c->icc_igrpen0_el1, true);
+        kvm_gicc_access(s, ICC_IGRPEN1_EL1, ncpu, &c->icc_igrpen1_el1, true);
+        kvm_gicc_access(s, ICC_PMR_EL1, ncpu, &c->icc_pmr_el1, true);
+        kvm_gicc_access(s, ICC_BPR0_EL1, ncpu, &c->icc_bpr[0], true);
+        kvm_gicc_access(s, ICC_BPR1_EL1, ncpu, &c->icc_bpr[1], true);
+
+        for (i = 0; i < 4; i++) {
+            reg = c->icc_ap1r[i][0];
+            kvm_gicc_access(s, ICC_AP0R_EL1(i), ncpu, &reg, true);
+        }
+
+        for (i = 0; i < 4; i++) {
+            reg = c->icc_ap1r[i][1];
+            kvm_gicc_access(s, ICC_AP1R_EL1(i), ncpu, &reg, true);
+        }
+    }
 }
 
 static void kvm_arm_gicv3_get(GICv3State *s)
 {
-    /* TODO */
-    DPRINTF("Cannot get kernel gic state, no kernel interface\n");
+    uint64_t reg, redist_typer;
+    int ncpu, i;
+
+    kvm_arm_gicv3_check(s);
+
+    kvm_gicr_access(s, GICR_TYPER, 0, &redist_typer, false);
+
+    kvm_gicd_access(s, GICD_CTLR, &reg, false);
+    s->gicd_ctlr = reg;
+
+    /* Redistributor state (one per CPU) */
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        GICv3CPUState *c = &s->cpu[ncpu];
+
+        kvm_gicr_access(s, GICR_CTLR, ncpu, &reg, false);
+        c->gicr_ctlr = reg;
+
+        kvm_gicr_access(s, GICR_WAKER, ncpu, &reg, false);
+        c->gicr_waker = reg;
+
+        kvm_gicr_access(s, GICR_IGROUPR0, ncpu, &reg, false);
+        c->gicr_igroupr0 = reg;
+        kvm_gicr_access(s, GICR_ISENABLER0, ncpu, &reg, false);
+        c->gicr_ienabler0 = reg;
+        kvm_gicr_access(s, GICR_ICFGR0, ncpu, &reg, false);
+        c->gicr_icfgr0 = reg;
+        kvm_gicr_access(s, GICR_ICFGR1, ncpu, &reg, false);
+        c->gicr_icfgr1 = reg;
+        kvm_gicr_access(s, GICR_ISPENDR0, ncpu, &reg, false);
+        c->gicr_ipendr0 = reg;
+        kvm_gicr_access(s, GICR_ISACTIVER0, ncpu, &reg, false);
+        c->gicr_iactiver0 = reg;
+
+        for (i = 0; i < GIC_INTERNAL; i += 4) {
+            kvm_gicr_access(s, GICR_IPRIORITYR0 + i, ncpu, &reg, false);
+            c->gicr_ipriorityr[i] = extract32(reg, 0, 8);
+            c->gicr_ipriorityr[i + 1] = extract32(reg, 8, 8);
+            c->gicr_ipriorityr[i + 2] = extract32(reg, 16, 8);
+            c->gicr_ipriorityr[i + 3] = extract32(reg, 24, 8);
+        }
+    }
+
+    if (redist_typer & GICR_TYPER_PLPIS) {
+        for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+            GICv3CPUState *c = &s->cpu[ncpu];
+
+            kvm_gicr_access(s, GICR_PROPBASER, ncpu, &reg, false);
+            c->gicr_propbaser = reg;
+
+            kvm_gicr_access(s, GICR_PENDBASER, ncpu, &reg, false);
+            c->gicr_pendbaser = reg;
+        }
+    }
+
+    /* Distributor state (shared between all CPUs */
+
+    /* GICD_IIDR -> ?   FIXME */
+    /* kvm_gicd_access(s, GICD_IIDR, 0, &reg, false); */
+
+    /* GICD_IGROUPRn -> s->group bitmap */
+    kvm_dist_get(s, GICD_IGROUPR, 1, translate_group);
+
+    /* GICD_ISENABLERn -> s->enabled bitmap */
+    kvm_dist_get(s, GICD_ISENABLER, 1, translate_enabled);
+
+    /* GICD_ISPENDRn -> s->pending + s->level bitmap */
+    kvm_dist_get(s, GICD_ISPENDR, 1, translate_pending);
+
+    /* GICD_ISACTIVERn -> s->active bitmap */
+    kvm_dist_get(s, GICD_ISACTIVER, 1, translate_active);
+
+    /* GICD_ICFRn -> s->trigger bitmap */
+    kvm_dist_get(s, GICD_ICFGR, 2, translate_edge_trigger);
+
+    /* GICD_IPRIORITYRn -> s->gicd_ipriority[] */
+    kvm_dist_get(s, GICD_IPRIORITYR, 8, translate_priority);
+
+    /* GICD_IROUTERn -> s->gicd_irouter[irq] */
+    for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+        uint32_t offset = GICD_IROUTER + (sizeof(reg) * i);
+
+        kvm_gicd_access(s, offset, &reg, false);
+        s->gicd_irouter[i - GIC_INTERNAL] = reg;
+    }
+
+    /*****************************************************************
+     * CPU Interface(s) State
+     */
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        GICv3CPUState *c = &s->cpu[ncpu];
+
+        kvm_gicc_access(s, ICC_CTLR_EL1, ncpu, &c->icc_ctlr_el1[1], false);
+        kvm_gicc_access(s, ICC_IGRPEN0_EL1, ncpu, &c->icc_igrpen0_el1, false);
+        kvm_gicc_access(s, ICC_IGRPEN1_EL1, ncpu, &c->icc_igrpen1_el1, false);
+        kvm_gicc_access(s, ICC_PMR_EL1, ncpu, &c->icc_pmr_el1, false);
+        kvm_gicc_access(s, ICC_BPR0_EL1, ncpu, &c->icc_bpr[0], false);
+        kvm_gicc_access(s, ICC_BPR1_EL1, ncpu, &c->icc_bpr[1], false);
+
+        for (i = 0; i < 4; i++) {
+            kvm_gicc_access(s, ICC_AP0R_EL1(i), ncpu, &reg, false);
+            c->icc_ap1r[i][0] = reg;
+        }
+
+        for (i = 0; i < 4; i++) {
+            kvm_gicc_access(s, ICC_AP1R_EL1(i), ncpu, &reg, false);
+            c->icc_ap1r[i][1] = reg;
+        }
+    }
 }
 
 static void kvm_arm_gicv3_reset(DeviceState *dev)
@@ -75,6 +491,12 @@ static void kvm_arm_gicv3_reset(DeviceState *dev)
     DPRINTF("Reset\n");
 
     kgc->parent_reset(dev);
+
+    if (s->migration_blocker) {
+        DPRINTF("Cannot put kernel gic state, no kernel interface\n");
+        return;
+    }
+
     kvm_arm_gicv3_put(s);
 }
 
@@ -118,6 +540,13 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
                             KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd);
     kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd);
+
+    if (!kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_DIST_REGS,
+                               GICD_CTLR)) {
+        error_setg(&s->migration_blocker, "This operating system kernel does "
+                                          "not support vGICv3 migration");
+        migrate_add_blocker(s->migration_blocker);
+    }
 }
 
 static void kvm_arm_gicv3_class_init(ObjectClass *klass, void *data)
-- 
1.9.1

[Qemu-devel] [RFC 4/5] migration: Define VMSTATE_UINT64_2DARRAY

[Peter Maydell]

Define a VMSTATE_UINT64_2DARRAY macro, to go with the ones we
already have for other type sizes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 include/migration/vmstate.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 7246f29..ecc4992 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -766,6 +766,12 @@ extern const VMStateInfo vmstate_info_bitmap;
 #define VMSTATE_UINT64_ARRAY(_f, _s, _n)                              \
     VMSTATE_UINT64_ARRAY_V(_f, _s, _n, 0)
 
+#define VMSTATE_UINT64_2DARRAY(_f, _s, _n1, _n2)                      \
+    VMSTATE_UINT64_2DARRAY_V(_f, _s, _n1, _n2, 0)
+
+#define VMSTATE_UINT64_2DARRAY_V(_f, _s, _n1, _n2, _v)                 \
+    VMSTATE_2DARRAY(_f, _s, _n1, _n2, _v, vmstate_info_uint64, uint64_t)
+
 #define VMSTATE_INT16_ARRAY_V(_f, _s, _n, _v)                         \
     VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_int16, int16_t)
 
-- 
1.9.1

[Qemu-devel] [RFC 5/5] hw/intc/arm_gicv3_common: Add vmstate descriptors

[Peter Maydell]

From: Pavel Fedin <p.fedin@samsung.com>

Add state structure descriptors and actually enable live migration.

In order to describe fixed-size bitmaps, VMSTATE_BITMAP_STATIC() macro is
introduced.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
[PMM: Adjust to renamed struct fields]
---
 hw/intc/arm_gicv3_common.c  | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 include/migration/vmstate.h |  9 +++++++++
 2 files changed, 53 insertions(+), 1 deletion(-)

diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index a1ce4c1..e1078a5 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -48,11 +48,54 @@ static int gicv3_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static const VMStateDescription vmstate_gicv3_cpu = {
+    .name = "arm_gicv3_cpu",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32(level, GICv3CPUState),
+        VMSTATE_UINT32(gicr_ctlr, GICv3CPUState),
+        VMSTATE_UINT32(gicr_waker, GICv3CPUState),
+        VMSTATE_UINT64(gicr_propbaser, GICv3CPUState),
+        VMSTATE_UINT64(gicr_pendbaser, GICv3CPUState),
+        VMSTATE_UINT32(gicr_igroupr0, GICv3CPUState),
+        VMSTATE_UINT32(gicr_ienabler0, GICv3CPUState),
+        VMSTATE_UINT32(gicr_ipendr0, GICv3CPUState),
+        VMSTATE_UINT32(gicr_iactiver0, GICv3CPUState),
+        VMSTATE_UINT32(gicr_icfgr0, GICv3CPUState),
+        VMSTATE_UINT32(gicr_icfgr1, GICv3CPUState),
+        VMSTATE_UINT8_ARRAY(gicr_ipriorityr, GICv3CPUState, GIC_INTERNAL),
+        VMSTATE_UINT64_ARRAY(icc_ctlr_el1, GICv3CPUState, 2),
+        VMSTATE_UINT64(icc_pmr_el1, GICv3CPUState),
+        VMSTATE_UINT64_ARRAY(icc_bpr, GICv3CPUState, 2),
+        VMSTATE_UINT64_2DARRAY(icc_ap1r, GICv3CPUState, 4, 2),
+        VMSTATE_UINT64(icc_igrpen0_el1, GICv3CPUState),
+        VMSTATE_UINT64(icc_igrpen1_el1, GICv3CPUState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static const VMStateDescription vmstate_gicv3 = {
     .name = "arm_gicv3",
-    .unmigratable = 1,
+    .version_id = 1,
+    .minimum_version_id = 1,
     .pre_save = gicv3_pre_save,
     .post_load = gicv3_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32(gicd_ctlr, GICv3State),
+        VMSTATE_BITMAP_STATIC(group, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_BITMAP_STATIC(enabled, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_BITMAP_STATIC(pending, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_BITMAP_STATIC(active, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_BITMAP_STATIC(level, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_BITMAP_STATIC(edge_trigger, GICv3State , 0, GICV3_MAXSPI),
+        VMSTATE_UINT8_ARRAY(gicd_ipriority, GICv3State, GICV3_MAXSPI),
+        VMSTATE_UINT8_ARRAY(gicd_itargetsr, GICv3State, GICV3_MAXSPI),
+        VMSTATE_UINT64_ARRAY(gicd_irouter, GICv3State, GICV3_MAXSPI),
+        VMSTATE_STRUCT_VARRAY_POINTER_UINT32(cpu, GICv3State, num_cpu,
+                                             vmstate_gicv3_cpu, GICv3CPUState),
+        VMSTATE_END_OF_LIST()
+    }
 };
 
 void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index ecc4992..934809a 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -564,6 +564,15 @@ extern const VMStateInfo vmstate_info_bitmap;
     .offset       = offsetof(_state, _field),                        \
 }
 
+#define VMSTATE_BITMAP_STATIC(_field, _state, _version, _size) {     \
+    .name         = (stringify(_field)),                             \
+    .version_id   = (_version),                                      \
+    .size         = (_size),                                         \
+    .info         = &vmstate_info_bitmap,                            \
+    .flags        = VMS_BUFFER,                                      \
+    .offset       = offsetof(_state, _field),                        \
+}
+
 /* _f : field name
    _f_n : num of elements field_name
    _n : num of elements
-- 
1.9.1

Re: [Qemu-devel] [RFC 1/5] hw/intc/arm_gicv3_common: Add state information

[Shlomo Pongratz]

[-- Attachment #1: Type: text/plain, Size: 20156 bytes --]

On Monday, February 22, 2016, Peter Maydell <peter.maydell@linaro.org>
wrote:

> From: Pavel Fedin <p.fedin@samsung.com <javascript:;>>
>
> Add state information to GICv3 object structure and implement
> arm_gicv3_common_reset(). Also, add some functions for registers which are
> not stored directly but simulated.
>
> State information includes not only pure GICv3 data, but also some legacy
> registers. This will be useful for implementing software emulation of GICv3
> with v2 backwards compatilibity mode.
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com <javascript:;>>
> [PMM: significantly overhauled:
>  * Add missing qom/cpu.h include
>  * Remove legacy-only state fields (we can add them later if/when we add
>    legacy emulation)
>  * Add various missing register offset #defines
>  * Accessor macros removed entirely
>  * Fields in state structures renamed to match architectural register names
>  * Corrected the reset value for GICR_IENABLER0 since we don't support
>    legacy mode
>  * Added ARM_LINUX_BOOT_IF interface for "we are directly booting a kernel
> in
>    non-secure" so that we can fake up the firmware-mandated reconfiguration
>    only when we need it
> ]
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org <javascript:;>>
> ---
>  hw/intc/arm_gicv3_common.c         | 116 ++++++++++++++++++++++++++++-
>  hw/intc/gicv3_internal.h           | 147
> +++++++++++++++++++++++++++++++++++++
>  include/hw/intc/arm_gicv3_common.h | 106 +++++++++++++++++++++++++-
>  3 files changed, 364 insertions(+), 5 deletions(-)
>  create mode 100644 hw/intc/gicv3_internal.h
>
> diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
> index e4f0f5a..a1ce4c1 100644
> --- a/hw/intc/arm_gicv3_common.c
> +++ b/hw/intc/arm_gicv3_common.c
> @@ -3,8 +3,9 @@
>   *
>   * Copyright (c) 2012 Linaro Limited
>   * Copyright (c) 2015 Huawei.
> + * Copyright (c) 2015 Samsung Electronics Co., Ltd.
>   * Written by Peter Maydell
> - * Extended to 64 cores by Shlomo Pongratz
> + * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
>   *
>   * This program is free software; you can redistribute it and/or modify
>   * it under the terms of the GNU General Public License as published by
> @@ -21,7 +22,10 @@
>   */
>
>  #include "qemu/osdep.h"
> +#include "qom/cpu.h"
>  #include "hw/intc/arm_gicv3_common.h"
> +#include "gicv3_internal.h"
> +#include "hw/arm/linux-boot-if.h"
>
>  static void gicv3_pre_save(void *opaque)
>  {
> @@ -89,6 +93,8 @@ void gicv3_init_irqs_and_mmio(GICv3State *s,
> qemu_irq_handler handler,
>  static void arm_gicv3_common_realize(DeviceState *dev, Error **errp)
>  {
>      GICv3State *s = ARM_GICV3_COMMON(dev);
> +    Object *cpu;
> +    int i;
>
>      /* revision property is actually reserved and currently used only in
> order
>       * to keep the interface compatible with GICv2 code, avoiding extra
> @@ -99,11 +105,111 @@ static void arm_gicv3_common_realize(DeviceState
> *dev, Error **errp)
>          error_setg(errp, "unsupported GIC revision %d", s->revision);
>          return;
>      }
> +
> +    if (s->num_irq > GICV3_MAXIRQ) {
> +        error_setg(errp,
> +                   "requested %u interrupt lines exceeds GIC maximum %d",
> +                   s->num_irq, GICV3_MAXIRQ);
> +        return;
> +    }
> +
> +    s->cpu = g_new0(GICv3CPUState, s->num_cpu);
> +
> +    for (i = 0; i < s->num_cpu; i++) {
> +        cpu = OBJECT(qemu_get_cpu(i));
> +        s->cpu[i].affinity_id = object_property_get_int(cpu,
> "mp-affinity",
> +                                                        NULL);
> +    }
>  }
>
>  static void arm_gicv3_common_reset(DeviceState *dev)
>  {
> -    /* TODO */
> +    GICv3State *s = ARM_GICV3_COMMON(dev);
> +    int i;
> +
> +    for (i = 0; i < s->num_cpu; i++) {
> +        GICv3CPUState *c = &s->cpu[i];
> +
> +        c->gicr_waker = GICR_WAKER_ProcessorSleep |
> GICR_WAKER_ChildrenAsleep;
> +        c->gicr_ctlr = 0;
> +        c->gicr_propbaser = 0;
> +        c->gicr_pendbaser = 0;
> +        /* If we're resetting a TZ-aware GIC as if secure firmware
> +         * had set it up ready to start a kernel in non-secure, we
> +         * need to set interrupts to group 1 so the kernel can use them.
> +         * Otherwise they reset to group 0 like the hardware.
> +         */
> +        if (s->security_extn && s->irq_reset_nonsecure) {
> +            c->gicr_igroupr0 = 0xffffffff;
> +        } else {
> +            c->gicr_igroupr0 = 0;
> +        }
> +
> +        c->gicr_ienabler0 = 0;
> +        c->gicr_ipendr0 = 0;
> +        c->gicr_iactiver0 = 0;
> +        c->gicr_icfgr0 = 0xaaaaaaaa;
> +        c->gicr_icfgr1 = 0;
> +        memset(c->gicr_ipriorityr, 0, sizeof(c->gicr_ipriorityr));
> +        c->level = 0;
> +
> +        c->icc_ctlr_el1[0] = 0;
> +        c->icc_ctlr_el1[1] = 0;
> +        c->icc_igrpen0_el1 = 0;
> +        c->icc_igrpen1_el1 = 0;
> +        c->icc_pmr_el1 = 0;
> +        c->icc_bpr[0] = GIC_MIN_BPR0;
> +        c->icc_bpr[1] = GIC_MIN_BPR1;
> +        memset(c->icc_ap1r, 0, sizeof(c->icc_ap1r));
> +    }
> +
> +    memset(s->group, 0, sizeof(s->group));
> +    memset(s->enabled, 0, sizeof(s->enabled));
> +    memset(s->pending, 0, sizeof(s->pending));
> +    memset(s->active, 0, sizeof(s->active));
> +    memset(s->level, 0, sizeof(s->level));
> +    memset(s->edge_trigger, 0, sizeof(s->edge_trigger));
> +
> +    /* Workaround! (the same as c->group above) */
> +    for (i = GIC_INTERNAL; i < s->num_irq; i++) {
> +        set_bit(i - GIC_INTERNAL, s->group);
> +    }
> +
> +    /* By default all interrupts always target CPU #0 */
> +    for (i = 0; i < GICV3_MAXSPI; i++) {
> +        s->gicd_itargetsr[i] = 1;
> +    }
> +    memset(s->gicd_irouter, 0, sizeof(s->gicd_irouter));
> +    memset(s->gicd_ipriority, 0, sizeof(s->gicd_ipriority));
> +
> +    /* With all configuration we don't support GICv2 backwards
> computability */
> +    if (s->security_extn) {
> +        /* GICv3 5.3.20 With two security So DS is RAZ/WI ARE_NS is RAO/WI
> +         * and ARE_S is RAO/WI
> +         */
> +         s->gicd_ctlr = GICD_CTLR_ARE_S | GICD_CTLR_ARE_NS;
> +    } else {
> +        /* GICv3 5.3.20 With one security So DS is RAO/WI ARE is RAO/WI
> +         */
> +        s->gicd_ctlr = GICD_CTLR_DS | GICD_CTLR_ARE;
> +    }
> +}
> +
> +static void arm_gic_common_linux_init(ARMLinuxBootIf *obj,
> +                                      bool secure_boot)
> +{
> +    GICState *s = ARM_GIC_COMMON(obj);
> +
> +    if (s->security_extn && !secure_boot) {
> +        /* We're directly booting a kernel into NonSecure. If this GIC
> +         * implements the security extensions then we must configure it
> +         * to have all the interrupts be NonSecure (this is a job that
> +         * is done by the Secure boot firmware in real hardware, and in
> +         * this mode QEMU is acting as a minimalist
> firmware-and-bootloader
> +         * equivalent).
> +         */
> +        s->irq_reset_nonsecure = true;
> +    }
>  }
>
>  static Property arm_gicv3_common_properties[] = {
> @@ -117,11 +223,13 @@ static Property arm_gicv3_common_properties[] = {
>  static void arm_gicv3_common_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> +    ARMLinuxBootIfClass *albifc = ARM_LINUX_BOOT_IF_CLASS(klass);
>
>      dc->reset = arm_gicv3_common_reset;
>      dc->realize = arm_gicv3_common_realize;
>      dc->props = arm_gicv3_common_properties;
>      dc->vmsd = &vmstate_gicv3;
> +    albifc->arm_linux_init = arm_gic_common_linux_init;
>  }
>
>  static const TypeInfo arm_gicv3_common_type = {
> @@ -131,6 +239,10 @@ static const TypeInfo arm_gicv3_common_type = {
>      .class_size = sizeof(ARMGICv3CommonClass),
>      .class_init = arm_gicv3_common_class_init,
>      .abstract = true,
> +    .interfaces = (InterfaceInfo []) {
> +        { TYPE_ARM_LINUX_BOOT_IF },
> +        { },
> +    },
>  };
>
>  static void register_types(void)
> diff --git a/hw/intc/gicv3_internal.h b/hw/intc/gicv3_internal.h
> new file mode 100644
> index 0000000..9f9a45e
> --- /dev/null
> +++ b/hw/intc/gicv3_internal.h
> @@ -0,0 +1,147 @@
> +/*
> + * ARM GICv3 support - internal interfaces
> + *
> + * Copyright (c) 2012 Linaro Limited
> + * Copyright (c) 2015 Huawei.
> + * Copyright (c) 2015 Samsung Electronics Co., Ltd.
> + * Written by Peter Maydell
> + * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation, either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License along
> + * with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef QEMU_ARM_GICV3_INTERNAL_H
> +#define QEMU_ARM_GICV3_INTERNAL_H
> +
> +#include "hw/intc/arm_gicv3_common.h"
> +
> +/* Distributor registers, as offsets from the distributor base address */
> +#define GICD_CTLR            0x0000
> +#define GICD_TYPER           0x0004
> +#define GICD_IIDR            0x0008
> +#define GICD_STATUSR         0x0010
> +#define GICD_SETSPI_NSR      0x0040
> +#define GICD_CLRSPI_NSR      0x0048
> +#define GICD_SETSPI_SR       0x0050
> +#define GICD_CLRSPI_SR       0x0058
> +#define GICD_SEIR            0x0068
> +#define GICD_IGROUPR         0x0080
> +#define GICD_ISENABLER       0x0100
> +#define GICD_ICENABLER       0x0180
> +#define GICD_ISPENDR         0x0200
> +#define GICD_ICPENDR         0x0280
> +#define GICD_ISACTIVER       0x0300
> +#define GICD_ICACTIVER       0x0380
> +#define GICD_IPRIORITYR      0x0400
> +#define GICD_ITARGETSR       0x0800
> +#define GICD_ICFGR           0x0C00
> +#define GICD_IGRPMODR        0x0D00
> +#define GICD_NASCR           0x0E00
> +#define GICD_SGIR            0x0F00
> +#define GICD_CPENDSGIR       0x0F10
> +#define GICD_SPENDSGIR       0x0F20
> +#define GICD_IROUTER         0x6000
> +#define GICD_PIDR2           0xFFE8
> +
> +/* GICD_CTLR fields  */
> +#define GICD_CTLR_EN_GRP0           (1U << 0)
> +#define GICD_CTLR_EN_GRP1NS         (1U << 1) /* GICv3 5.3.20 */
> +#define GICD_CTLR_EN_GRP1S          (1U << 2)
> +#define GICD_CTLR_EN_GRP1_ALL       (GICD_CTLR_EN_GRP1NS |
> GICD_CTLR_EN_GRP1S)
> +/* Bit 4 is ARE if the system doesn't support TrustZone, ARE_S otherwise
> */
> +#define GICD_CTLR_ARE               (1U << 4)
> +#define GICD_CTLR_ARE_S             (1U << 4)
> +#define GICD_CTLR_ARE_NS            (1U << 5)
> +#define GICD_CTLR_DS                (1U << 6)
> +#define GICD_CTLR_E1NWF             (1U << 7)
> +#define GICD_CTLR_RWP               (1U << 31)
> +
> +/*
> + * Redistributor frame offsets from RD_base
> + */
> +#define GICR_SGI_OFFSET 0x10000
> +
> +/*
> + * Redistributor registers, offsets from RD_base
> + */
> +#define GICR_CTLR             0x0000
> +#define GICR_IIDR             0x0004
> +#define GICR_TYPER            0x0008
> +#define GICR_STATUSR          0x0010
> +#define GICR_WAKER            0x0014
> +#define GICR_SETLPIR          0x0040
> +#define GICR_CLRLPIR          0x0048
> +#define GICR_PROPBASER        0x0070
> +#define GICR_PENDBASER        0x0078
> +#define GICR_INVLPIR          0x00A0
> +#define GICR_INVALLR          0x00B0
> +#define GICR_SYNCR            0x00C0
> +#define GICR_PIDR2            0xFFE8
> +
> +/* SGI and PPI Redistributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0         0x0080
> +#define GICR_ISENABLER0       0x0100
> +#define GICR_ICENABLER0       0x0180
> +#define GICR_ISPENDR0         0x0200
> +#define GICR_ICPENDR0         0x0280
> +#define GICR_ISACTIVER0       0x0300
> +#define GICR_ICACTIVER0       0x0380
> +#define GICR_IPRIORITYR0      0x0400
> +#define GICR_ICFGR0           0x0C00
> +#define GICR_ICFGR1           0x0C04
> +#define GICR_IGRPMODR0        0x0D00
> +#define GICR_NSACR            0x0E00
> +
> +#define GICR_CTLR_ENABLE_LPIS        (1U << 0)
> +#define GICR_CTLR_RWP                (1U << 3)
> +#define GICR_CTLR_DPG0               (1U << 24)
> +#define GICR_CTLR_DPG1NS             (1U << 25)
> +#define GICR_CTLR_DPG1S              (1U << 26)
> +#define GICR_CTLR_UWP                (1U << 31)
> +
> +#define GICR_TYPER_PLPIS             (1U << 0)
> +#define GICR_TYPER_VLPIS             (1U << 1)
> +#define GICR_TYPER_DIRECTLPI         (1U << 3)
> +#define GICR_TYPER_LAST              (1U << 4)
> +#define GICR_TYPER_DPGS              (1U << 5)
> +#define GICR_TYPER_PROCNUM           (0xFFFFU << 8)
> +#define GICR_TYPER_COMMONLPIAFF      (0x3 << 24)
> +#define GICR_TYPER_AFFINITYVALUE     (0xFFFFFFFFULL << 32)
> +
> +#define GICR_WAKER_ProcessorSleep    (1U << 1)
> +#define GICR_WAKER_ChildrenAsleep    (1U << 2)
> +
> +#define GICR_PROPBASER_OUTER_CACHEABILITY_MASK (7ULL << 56)
> +#define GICR_PROPBASER_ADDR_MASK               (0xfffffffffULL << 12)
> +#define GICR_PROPBASER_SHAREABILITY_MASK       (3U << 10)
> +#define GICR_PROPBASER_CACHEABILITY_MASK       (7U << 7)
> +#define GICR_PROPBASER_IDBITS_MASK             (0x1f)
> +
> +#define GICR_PENDBASER_PTZ                     (1ULL << 62)
> +#define GICR_PENDBASER_OUTER_CACHEABILITY_MASK (7ULL << 56)
> +#define GICR_PENDBASER_ADDR_MASK               (0xffffffffULL << 16)
> +#define GICR_PENDBASER_SHAREABILITY_MASK       (3U << 10)
> +#define GICR_PENDBASER_CACHEABILITY_MASK       (7U << 7)
> +
> +#define ICC_CTLR_EL1_CBPR           (1U << 0)
> +#define ICC_CTLR_EL1_EOIMODE        (1U << 1)
> +#define ICC_CTLR_EL1_PMHE           (1U << 6)
> +#define ICC_CTLR_EL1_SEIS           (1U << 14)
> +#define ICC_CTLR_EL1_A3V            (1U << 15)
> +
> +#define ICC_PMR_PRIORITY_MASK    0xff
> +#define ICC_BPR_BINARYPOINT_MASK 0x07
> +#define ICC_IGRPEN_ENABLE        0x01
> +
> +#endif /* !QEMU_ARM_GIC_INTERNAL_H */
> diff --git a/include/hw/intc/arm_gicv3_common.h
> b/include/hw/intc/arm_gicv3_common.h
> index c2fd8da..84e9406 100644
> --- a/include/hw/intc/arm_gicv3_common.h
> +++ b/include/hw/intc/arm_gicv3_common.h
> @@ -3,8 +3,9 @@
>   *
>   * Copyright (c) 2012 Linaro Limited
>   * Copyright (c) 2015 Huawei.
> + * Copyright (c) 2015 Samsung Electronics Co., Ltd.
>   * Written by Peter Maydell
> - * Extended to 64 cores by Shlomo Pongratz
> + * Reworked for GICv3 by Shlomo Pongratz and Pavel Fedin
>   *
>   * This program is free software; you can redistribute it and/or modify
>   * it under the terms of the GNU General Public License as published by
> @@ -26,7 +27,48 @@
>  #include "hw/sysbus.h"
>  #include "hw/intc/arm_gic_common.h"
>
> -typedef struct GICv3State {
> +/*
> + * Maximum number of possible interrupts, determined by the GIC
> architecture.
> + * Note that this does not include LPIs. When implemented, these should be
> + * dealt with separately.
> + */
> +#define GICV3_MAXIRQ 1020
> +#define GICV3_MAXSPI (GICV3_MAXIRQ - GIC_INTERNAL)
> +
> +#define GIC_MIN_BPR0 0
> +#define GIC_MIN_BPR1 (GIC_MIN_BPR0 + 1)
> +
> +struct GICv3CPUState {
> +    uint64_t affinity_id;            /* Cached affinity ID of the CPU */
> +
> +    /* Redistributor */
> +    uint32_t level;                  /* Current IRQ level */
> +    /* RD_base page registers */
> +    uint32_t gicr_ctlr;
> +    uint32_t gicr_waker;
> +    uint64_t gicr_propbaser;
> +    uint64_t gicr_pendbaser;
> +    /* SGI_base page registers */
> +    uint32_t gicr_igroupr0;
> +    uint32_t gicr_ienabler0;
> +    uint32_t gicr_ipendr0;
> +    uint32_t gicr_iactiver0;
> +    uint32_t gicr_icfgr0;
> +    uint32_t gicr_icfgr1;
> +    uint8_t gicr_ipriorityr[GIC_INTERNAL];
> +
> +    /* CPU interface */
> +    uint64_t icc_ctlr_el1[2];
> +    uint64_t icc_pmr_el1;
> +    uint64_t icc_bpr[2];
> +    uint64_t icc_ap1r[4][2];
> +    uint64_t icc_igrpen0_el1;
> +    uint64_t icc_igrpen1_el1;
> +};
> +
> +typedef struct GICv3CPUState GICv3CPUState;
> +
> +struct GICv3State {
>      /*< private >*/
>      SysBusDevice parent_obj;
>      /*< public >*/
> @@ -41,9 +83,61 @@ typedef struct GICv3State {
>      uint32_t num_irq;
>      uint32_t revision;
>      bool security_extn;
> +    bool irq_reset_nonsecure;
>
>      int dev_fd; /* kvm device fd if backed by kvm vgic support */
> -} GICv3State;
> +    Error *migration_blocker;
> +
> +    /* Distributor */
> +
> +    /* for a GIC with the security extensions the NS banked version of
> this
> +     * register is just an alias of bit 1 of the S banked version.
> +     */
> +    uint32_t gicd_ctlr;
> +    DECLARE_BITMAP(group, GICV3_MAXSPI);        /* GICD_IGROUPR */
> +    DECLARE_BITMAP(enabled, GICV3_MAXSPI);      /* GICD_ISENABLER */
> +    DECLARE_BITMAP(pending, GICV3_MAXSPI);      /* GICD_ISPENDR */
> +    DECLARE_BITMAP(active, GICV3_MAXSPI);       /* GICD_ISACTIVER */
> +    DECLARE_BITMAP(level, GICV3_MAXSPI);        /* Current level */
> +    DECLARE_BITMAP(edge_trigger, GICV3_MAXSPI); /* GICD_ICFGR */
> +    uint8_t gicd_ipriority[GICV3_MAXSPI];
> +    uint8_t gicd_itargetsr[GICV3_MAXSPI];
> +    uint64_t gicd_irouter[GICV3_MAXSPI];
> +
> +    GICv3CPUState *cpu;
> +};
> +
> +typedef struct GICv3State GICv3State;
> +
> +#define GICV3_BITMAP_ACCESSORS(BMP)                                     \
> +    static inline void gicv3_gicd_##BMP##_set(GICv3State *s, int irq)   \
> +    {                                                                   \
> +        set_bit(irq - GIC_INTERNAL, s->BMP);                            \
> +    }                                                                   \
> +    static inline int gicv3_gicd_##BMP##_test(GICv3State *s, int irq)   \
> +    {                                                                   \
> +        return test_bit(irq - GIC_INTERNAL, s->BMP);                    \
> +    }                                                                   \
> +    static inline void gicv3_gicd_##BMP##_clear(GICv3State *s, int irq) \
> +    {                                                                   \
> +        clear_bit(irq - GIC_INTERNAL, s->BMP);                          \
> +    }                                                                   \
> +    static inline void gicv3_gicd_##BMP##_replace(GICv3State *s,        \
> +                                                  int irq, int value)   \
> +    {                                                                   \
> +        if (value) {                                                    \
> +            gicv3_gicd_##BMP##_set(s, irq);                             \
> +        } else {                                                        \
> +            gicv3_gicd_##BMP##_clear(s, irq);                           \
> +        }                                                               \
> +    }
> +
> +GICV3_BITMAP_ACCESSORS(group)
> +GICV3_BITMAP_ACCESSORS(enabled)
> +GICV3_BITMAP_ACCESSORS(pending)
> +GICV3_BITMAP_ACCESSORS(active)
> +GICV3_BITMAP_ACCESSORS(level)
> +GICV3_BITMAP_ACCESSORS(edge_trigger)
>
>  #define TYPE_ARM_GICV3_COMMON "arm-gicv3-common"
>  #define ARM_GICV3_COMMON(obj) \
> @@ -65,4 +159,10 @@ typedef struct ARMGICv3CommonClass {
>  void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
>                                const MemoryRegionOps *ops);
>
> +/* Accessors for simulated system registers */
> +uint32_t gicv3_get_igrpen0(GICv3State *s, int cpuindex);
> +void gicv3_set_igrpen0(GICv3State *s, int cpuindex, uint32_t val);
> +uint32_t gicv3_get_igrpen1(GICv3State *s, int cpuindex);
> +void gicv3_set_igrpen1(GICv3State *s, int cpuindex, uint32_t val);
> +
>  #endif
> --
> 1.9.1
>
>
Hi,

I just wonder why the internal interrupts are excluded from the new
GICv3State and the new access API don't handle them?
Is it because they aren't required by KVM?
I see that they are only referenced in gicr_ipriority.

What am I missing?

Best regards,

S.P.

[-- Attachment #2: Type: text/html, Size: 24562 bytes --]

Re: [Qemu-devel] [RFC 1/5] hw/intc/arm_gicv3_common: Add state information

[Peter Maydell]

On 24 February 2016 at 12:50, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>
>
> On Monday, February 22, 2016, Peter Maydell <peter.maydell@linaro.org>
> wrote:
>>
>> +struct GICv3CPUState {
>> +    uint64_t affinity_id;            /* Cached affinity ID of the CPU */
>> +
>> +    /* Redistributor */
>> +    uint32_t level;                  /* Current IRQ level */
>> +    /* RD_base page registers */
>> +    uint32_t gicr_ctlr;
>> +    uint32_t gicr_waker;
>> +    uint64_t gicr_propbaser;
>> +    uint64_t gicr_pendbaser;
>> +    /* SGI_base page registers */
>> +    uint32_t gicr_igroupr0;
>> +    uint32_t gicr_ienabler0;
>> +    uint32_t gicr_ipendr0;
>> +    uint32_t gicr_iactiver0;
>> +    uint32_t gicr_icfgr0;
>> +    uint32_t gicr_icfgr1;
>> +    uint8_t gicr_ipriorityr[GIC_INTERNAL];
>> +
>> +    /* CPU interface */
>> +    uint64_t icc_ctlr_el1[2];
>> +    uint64_t icc_pmr_el1;
>> +    uint64_t icc_bpr[2];
>> +    uint64_t icc_ap1r[4][2];
>> +    uint64_t icc_igrpen0_el1;
>> +    uint64_t icc_igrpen1_el1;
>> +};
>> +
>> +typedef struct GICv3CPUState GICv3CPUState;
>> +
>> +struct GICv3State {
>>      /*< private >*/
>>      SysBusDevice parent_obj;
>>      /*< public >*/
>> @@ -41,9 +83,61 @@ typedef struct GICv3State {
>>      uint32_t num_irq;
>>      uint32_t revision;
>>      bool security_extn;
>> +    bool irq_reset_nonsecure;
>>
>>      int dev_fd; /* kvm device fd if backed by kvm vgic support */
>> -} GICv3State;
>> +    Error *migration_blocker;
>> +
>> +    /* Distributor */
>> +
>> +    /* for a GIC with the security extensions the NS banked version of
>> this
>> +     * register is just an alias of bit 1 of the S banked version.
>> +     */
>> +    uint32_t gicd_ctlr;
>> +    DECLARE_BITMAP(group, GICV3_MAXSPI);        /* GICD_IGROUPR */
>> +    DECLARE_BITMAP(enabled, GICV3_MAXSPI);      /* GICD_ISENABLER */
>> +    DECLARE_BITMAP(pending, GICV3_MAXSPI);      /* GICD_ISPENDR */
>> +    DECLARE_BITMAP(active, GICV3_MAXSPI);       /* GICD_ISACTIVER */
>> +    DECLARE_BITMAP(level, GICV3_MAXSPI);        /* Current level */
>> +    DECLARE_BITMAP(edge_trigger, GICV3_MAXSPI); /* GICD_ICFGR */
>> +    uint8_t gicd_ipriority[GICV3_MAXSPI];
>> +    uint8_t gicd_itargetsr[GICV3_MAXSPI];
>> +    uint64_t gicd_irouter[GICV3_MAXSPI];
>> +
>> +    GICv3CPUState *cpu;
>> +};

> I just wonder why the internal interrupts are excluded from the new
> GICv3State and the new access API don't handle them?
> Is it because they aren't required by KVM?
> I see that they are only referenced in gicr_ipriority.
>
> What am I missing?

Which internal interrupts did you have in mind? In GICv3
the PPIs and SGIs have state which lives in the redistributor
(in this data structure layout that's in the gicr_ipendr0,
gicr_ienabler0, etc in each GICv3CPUState struct). SPIs
have state which lives in the distributor (that's the
enabled/pending/etc bitmaps in the GICv3State struct).

I removed the confusing macros which tried to make the
two things appear the same (letting you mark an interrupt
as enabled whether the irq was for an SGI/PPI or an SPI)
because we shouldn't need them. The emulation code should
clearly separate the responsibilities of the distributor
from those of the redistributor (as happens in the real
hardware), and so it should in general already know what
kind of interrupt it's dealing with, and be able to just
work directly with the right data structure. If it looks
like you need a single accessor that works on both SGI/PPI
and SPI it's probably an indication that the code should
be restructured.

thanks
-- PMM