[PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

Hi,

When a MPCore cluster is used, the Cortex-A cores belong the the
cluster container, not to the board/soc layer. This series move
the creation of vCPUs to the MPCore private container.

Doing so we consolidate the QOM model, moving common code in a
central place (abstract MPCore parent).

This eventually allow removing one qemu_get_cpu() use, which we
want to remove in heterogeneous machines (machines using MPCore
are candidate for heterogeneous emulation).

Maybe these hw/cpu/arm/ files belong to hw/arm/...

Regards,

Phil.

Philippe Mathieu-Daudé (33):
  hw/arm/boot: Propagate vCPU to arm_load_dtb()
  hw/arm/fsl-imx6: Add a local 'gic' variable
  hw/arm/fsl-imx6ul: Add a local 'gic' variable
  hw/arm/fsl-imx7: Add a local 'gic' variable
  hw/cpu: Remove dead Kconfig
  hw/cpu/arm: Rename 'busdev' -> 'gicsbd' in a15mp_priv_realize()
  hw/cpu/arm: Alias 'num-cpu' property on TYPE_REALVIEW_MPCORE
  hw/cpu/arm: Declare CPU QOM types using DEFINE_TYPES() macro
  hw/cpu/arm: Merge {a9mpcore.h, a15mpcore.h} as cortex_mpcore.h
  hw/cpu/arm: Introduce abstract CORTEX_MPCORE_PRIV QOM type
  hw/cpu/arm: Have A9MPCORE/A15MPCORE inheritate common
    CORTEX_MPCORE_PRIV
  hw/cpu/arm: Create MPCore container in QOM parent
  hw/cpu/arm: Handle 'num_cores' property once in MPCore parent
  hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent
  hw/cpu/arm: Handle 'gic-irq' property once in MPCore parent
  hw/cpu/arm: Handle GIC once in MPCore parent
  hw/cpu/arm: Document more properties of CORTEX_MPCORE_PRIV QOM type
  hw/cpu/arm: Replace A15MPPrivState by CortexMPPrivState
  hw/cpu/arm: Introduce TYPE_A7MPCORE_PRIV for Cortex-A7 MPCore
  hw/cpu/arm: Consolidate check on max GIC spi supported
  hw/cpu/arm: Create CPUs once in MPCore parent
  hw/arm/aspeed_ast2600: Let the A7MPcore create/wire the CPU cores
  hw/arm/exynos4210: Let the A9MPcore create/wire the CPU cores
  hw/arm/fsl-imx6: Let the A9MPcore create/wire the CPU cores
  hw/arm/fsl-imx6ul: Let the A7MPcore create/wire the CPU cores
  hw/arm/fsl-imx7: Let the A7MPcore create/wire the CPU cores
  hw/arm/highbank: Let the A9/A15MPcore create/wire the CPU cores
  hw/arm/vexpress: Let the A9/A15MPcore create/wire the CPU cores
  hw/arm/xilinx_zynq: Let the A9MPcore create/wire the CPU cores
  hw/arm/npcm7xx: Let the A9MPcore create/wire the CPU cores
  hw/cpu/a9mpcore: Remove legacy code
  hw/cpu/arm: Remove 'num-cpu' property alias
  hw/cpu/arm: Remove use of qemu_get_cpu() in A7/A15 realize()

 MAINTAINERS                    |   3 +-
 include/hw/arm/aspeed_soc.h    |   5 +-
 include/hw/arm/boot.h          |   4 +-
 include/hw/arm/exynos4210.h    |   6 +-
 include/hw/arm/fsl-imx6.h      |   6 +-
 include/hw/arm/fsl-imx6ul.h    |   8 +-
 include/hw/arm/fsl-imx7.h      |   8 +-
 include/hw/arm/npcm7xx.h       |   3 +-
 include/hw/cpu/a15mpcore.h     |  44 -------
 include/hw/cpu/a9mpcore.h      |  39 -------
 include/hw/cpu/cortex_mpcore.h | 135 ++++++++++++++++++++++
 hw/arm/aspeed_ast2600.c        |  61 ++++------
 hw/arm/boot.c                  |  11 +-
 hw/arm/exynos4210.c            |  60 ++++------
 hw/arm/exynos4_boards.c        |   6 +-
 hw/arm/fsl-imx6.c              |  84 ++++----------
 hw/arm/fsl-imx6ul.c            |  65 ++++-------
 hw/arm/fsl-imx7.c              | 103 +++++------------
 hw/arm/highbank.c              |  56 ++-------
 hw/arm/mcimx6ul-evk.c          |   3 +-
 hw/arm/mcimx7d-sabre.c         |   3 +-
 hw/arm/npcm7xx.c               |  48 ++------
 hw/arm/realview.c              |   4 +-
 hw/arm/sabrelite.c             |   4 +-
 hw/arm/vexpress.c              |  60 +++-------
 hw/arm/virt.c                  |   2 +-
 hw/arm/xilinx_zynq.c           |  30 ++---
 hw/cpu/a15mpcore.c             | 179 +++++++++++++----------------
 hw/cpu/a9mpcore.c              | 138 +++++++++-------------
 hw/cpu/arm11mpcore.c           |  23 ++--
 hw/cpu/cortex_mpcore.c         | 202 +++++++++++++++++++++++++++++++++
 hw/cpu/realview_mpcore.c       |  30 ++---
 hw/arm/Kconfig                 |   8 +-
 hw/cpu/Kconfig                 |   8 --
 hw/cpu/meson.build             |   1 +
 35 files changed, 689 insertions(+), 761 deletions(-)
 delete mode 100644 include/hw/cpu/a15mpcore.h
 delete mode 100644 include/hw/cpu/a9mpcore.h
 create mode 100644 include/hw/cpu/cortex_mpcore.h
 create mode 100644 hw/cpu/cortex_mpcore.c
 delete mode 100644 hw/cpu/Kconfig

-- 
2.41.0

[PATCH 01/33] hw/arm/boot: Propagate vCPU to arm_load_dtb()

[Philippe Mathieu-Daudé]

In heterogeneous setup the first vCPU might not be
the one expected, better pass it explicitly.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/boot.h |  4 +++-
 hw/arm/boot.c         | 11 ++++++-----
 hw/arm/virt.c         |  2 +-
 3 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
index 80c492d742..b12bf61ca8 100644
--- a/include/hw/arm/boot.h
+++ b/include/hw/arm/boot.h
@@ -160,6 +160,7 @@ AddressSpace *arm_boot_address_space(ARMCPU *cpu,
  * @binfo:      struct describing the boot environment
  * @addr_limit: upper limit of the available memory area at @addr
  * @as:         address space to load image to
+ * @cpu:        ARM CPU object
  *
  * Load a device tree supplied by the machine or by the user  with the
  * '-dtb' command line option, and put it at offset @addr in target
@@ -176,7 +177,8 @@ AddressSpace *arm_boot_address_space(ARMCPU *cpu,
  * Note: Must not be called unless have_dtb(binfo) is true.
  */
 int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
-                 hwaddr addr_limit, AddressSpace *as, MachineState *ms);
+                 hwaddr addr_limit, AddressSpace *as, MachineState *ms,
+                 ARMCPU *cpu);
 
 /* Write a secure board setup routine with a dummy handler for SMCs */
 void arm_write_secure_board_setup_dummy_smc(ARMCPU *cpu,
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 84ea6a807a..334e4640fd 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -432,13 +432,12 @@ out:
     return ret;
 }
 
-static void fdt_add_psci_node(void *fdt)
+static void fdt_add_psci_node(void *fdt, ARMCPU *armcpu)
 {
     uint32_t cpu_suspend_fn;
     uint32_t cpu_off_fn;
     uint32_t cpu_on_fn;
     uint32_t migrate_fn;
-    ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(0));
     const char *psci_method;
     int64_t psci_conduit;
     int rc;
@@ -512,7 +511,8 @@ static void fdt_add_psci_node(void *fdt)
 }
 
 int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
-                 hwaddr addr_limit, AddressSpace *as, MachineState *ms)
+                 hwaddr addr_limit, AddressSpace *as, MachineState *ms,
+                 ARMCPU *cpu)
 {
     void *fdt = NULL;
     int size, rc, n = 0;
@@ -655,7 +655,7 @@ int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
         }
     }
 
-    fdt_add_psci_node(fdt);
+    fdt_add_psci_node(fdt, cpu);
 
     if (binfo->modify_dtb) {
         binfo->modify_dtb(binfo, fdt);
@@ -1321,7 +1321,8 @@ void arm_load_kernel(ARMCPU *cpu, MachineState *ms, struct arm_boot_info *info)
      * decided whether to enable PSCI and set the psci-conduit CPU properties.
      */
     if (!info->skip_dtb_autoload && have_dtb(info)) {
-        if (arm_load_dtb(info->dtb_start, info, info->dtb_limit, as, ms) < 0) {
+        if (arm_load_dtb(info->dtb_start, info, info->dtb_limit,
+                         as, ms, cpu) < 0) {
             exit(1);
         }
     }
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index be2856c018..4a1d1550f5 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1678,7 +1678,7 @@ void virt_machine_done(Notifier *notifier, void *data)
                                        vms->memmap[VIRT_PLATFORM_BUS].size,
                                        vms->irqmap[VIRT_PLATFORM_BUS]);
     }
-    if (arm_load_dtb(info->dtb_start, info, info->dtb_limit, as, ms) < 0) {
+    if (arm_load_dtb(info->dtb_start, info, info->dtb_limit, as, ms, cpu) < 0) {
         exit(1);
     }
 
-- 
2.41.0

Re: [PATCH 01/33] hw/arm/boot: Propagate vCPU to arm_load_dtb()

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> In heterogeneous setup the first vCPU might not be
> the one expected, better pass it explicitly.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 02/33] hw/arm/fsl-imx6: Add a local 'gic' variable

[Philippe Mathieu-Daudé]

The A9MPCore forward the IRQs from its internal GIC.
To make the code clearer, add a 'gic' variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/fsl-imx6.c | 37 +++++++++++++------------------------
 1 file changed, 13 insertions(+), 24 deletions(-)

diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
index 7dc42cbfe6..f6edbd7465 100644
--- a/hw/arm/fsl-imx6.c
+++ b/hw/arm/fsl-imx6.c
@@ -111,6 +111,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
     uint16_t i;
     Error *err = NULL;
     unsigned int smp_cpus = ms->smp.cpus;
+    DeviceState *gic = DEVICE(&s->a9mpcore);
 
     if (smp_cpus > FSL_IMX6_NUM_CPUS) {
         error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
@@ -186,8 +187,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->uart[i]), 0, serial_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->uart[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            serial_table[i].irq));
+                           qdev_get_gpio_in(gic, serial_table[i].irq));
     }
 
     s->gpt.ccm = IMX_CCM(&s->ccm);
@@ -198,8 +198,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->gpt), 0, FSL_IMX6_GPT_ADDR);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpt), 0,
-                       qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                        FSL_IMX6_GPT_IRQ));
+                       qdev_get_gpio_in(gic, FSL_IMX6_GPT_IRQ));
 
     /* Initialize all EPIT timers */
     for (i = 0; i < FSL_IMX6_NUM_EPITS; i++) {
@@ -219,8 +218,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->epit[i]), 0, epit_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->epit[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            epit_table[i].irq));
+                           qdev_get_gpio_in(gic, epit_table[i].irq));
     }
 
     /* Initialize all I2C */
@@ -240,8 +238,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->i2c[i]), 0, i2c_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->i2c[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            i2c_table[i].irq));
+                           qdev_get_gpio_in(gic, i2c_table[i].irq));
     }
 
     /* Initialize all GPIOs */
@@ -298,11 +295,9 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->gpio[i]), 0, gpio_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            gpio_table[i].irq_low));
+                           qdev_get_gpio_in(gic, gpio_table[i].irq_low));
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 1,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            gpio_table[i].irq_high));
+                           qdev_get_gpio_in(gic, gpio_table[i].irq_high));
     }
 
     /* Initialize all SDHC */
@@ -329,8 +324,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
         }
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->esdhc[i]), 0, esdhc_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->esdhc[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            esdhc_table[i].irq));
+                           qdev_get_gpio_in(gic, esdhc_table[i].irq));
     }
 
     /* USB */
@@ -351,8 +345,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->usb[i]), 0,
                         FSL_IMX6_USBOH3_USB_ADDR + i * 0x200);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->usb[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            FSL_IMX6_USBn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6_USBn_IRQ[i]));
     }
 
     /* Initialize all ECSPI */
@@ -375,8 +368,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->spi[i]), 0, spi_table[i].addr);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->spi[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            spi_table[i].irq));
+                           qdev_get_gpio_in(gic, spi_table[i].irq));
     }
 
     object_property_set_uint(OBJECT(&s->eth), "phy-num", s->phy_num,
@@ -387,11 +379,9 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
     }
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->eth), 0, FSL_IMX6_ENET_ADDR);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth), 0,
-                       qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                        FSL_IMX6_ENET_MAC_IRQ));
+                       qdev_get_gpio_in(gic, FSL_IMX6_ENET_MAC_IRQ));
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth), 1,
-                       qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                        FSL_IMX6_ENET_MAC_1588_IRQ));
+                       qdev_get_gpio_in(gic, FSL_IMX6_ENET_MAC_1588_IRQ));
 
     /*
      * SNVS
@@ -418,8 +408,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->wdt[i]), 0, FSL_IMX6_WDOGn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->wdt[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a9mpcore),
-                                            FSL_IMX6_WDOGn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6_WDOGn_IRQ[i]));
     }
 
     /* ROM memory */
-- 
2.41.0

Re: [PATCH 02/33] hw/arm/fsl-imx6: Add a local 'gic' variable

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> The A9MPCore forward the IRQs from its internal GIC.
> To make the code clearer, add a 'gic' variable.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 03/33] hw/arm/fsl-imx6ul: Add a local 'gic' variable

[Philippe Mathieu-Daudé]

The A7MPCore forward the IRQs from its internal GIC.
To make the code clearer, add a 'gic' variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/fsl-imx6ul.c | 38 ++++++++++++++------------------------
 1 file changed, 14 insertions(+), 24 deletions(-)

diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
index e37b69a5e1..59032263cf 100644
--- a/hw/arm/fsl-imx6ul.c
+++ b/hw/arm/fsl-imx6ul.c
@@ -156,6 +156,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
     FslIMX6ULState *s = FSL_IMX6UL(dev);
+    DeviceState *gic;
     int i;
     char name[NAME_SIZE];
     SysBusDevice *sbd;
@@ -177,6 +178,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                             FSL_IMX6UL_MAX_IRQ + GIC_INTERNAL, &error_abort);
     sysbus_realize(SYS_BUS_DEVICE(&s->a7mpcore), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a7mpcore), 0, FSL_IMX6UL_A7MPCORE_ADDR);
+    gic = DEVICE(&s->a7mpcore);
 
     sbd = SYS_BUS_DEVICE(&s->a7mpcore);
     d = DEVICE(&s->cpu);
@@ -213,8 +215,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_GPTn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpt[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_GPTn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_GPTn_IRQ[i]));
     }
 
     /*
@@ -238,8 +239,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_EPITn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->epit[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_EPITn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_EPITn_IRQ[i]));
     }
 
     /*
@@ -276,12 +276,10 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_GPIOn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_GPIOn_LOW_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_GPIOn_LOW_IRQ[i]));
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 1,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_GPIOn_HIGH_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_GPIOn_HIGH_IRQ[i]));
     }
 
     /*
@@ -335,8 +333,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_SPIn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->spi[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_SPIn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_SPIn_IRQ[i]));
     }
 
     /*
@@ -361,8 +358,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->i2c[i]), 0, FSL_IMX6UL_I2Cn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->i2c[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_I2Cn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_I2Cn_IRQ[i]));
     }
 
     /*
@@ -399,8 +395,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_UARTn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->uart[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_UARTn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_UARTn_IRQ[i]));
     }
 
     /*
@@ -449,12 +444,10 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_ENETn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_ENETn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_ENETn_IRQ[i]));
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth[i]), 1,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_ENETn_TIMER_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_ENETn_TIMER_IRQ[i]));
     }
 
     /*
@@ -490,8 +483,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->usb[i]), 0,
                         FSL_IMX6UL_USB02_USBn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->usb[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_USBn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_USBn_IRQ[i]));
     }
 
     /*
@@ -516,8 +508,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
                         FSL_IMX6UL_USDHCn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->usdhc[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_USDHCn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_USDHCn_IRQ[i]));
     }
 
     /*
@@ -549,8 +540,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->wdt[i]), 0,
                         FSL_IMX6UL_WDOGn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->wdt[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX6UL_WDOGn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX6UL_WDOGn_IRQ[i]));
     }
 
     /*
-- 
2.41.0

Re: [PATCH 03/33] hw/arm/fsl-imx6ul: Add a local 'gic' variable

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> The A7MPCore forward the IRQs from its internal GIC.
> To make the code clearer, add a 'gic' variable.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 04/33] hw/arm/fsl-imx7: Add a local 'gic' variable

[Philippe Mathieu-Daudé]

The A7MPCore forward the IRQs from its internal GIC.
To make the code clearer, add a 'gic' variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/fsl-imx7.c | 38 +++++++++++++++++---------------------
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
index 474cfdc87c..a283dcb45f 100644
--- a/hw/arm/fsl-imx7.c
+++ b/hw/arm/fsl-imx7.c
@@ -163,6 +163,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
     FslIMX7State *s = FSL_IMX7(dev);
+    DeviceState *gic;
     Object *o;
     int i;
     qemu_irq irq;
@@ -209,6 +210,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
 
     sysbus_realize(SYS_BUS_DEVICE(&s->a7mpcore), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a7mpcore), 0, FSL_IMX7_A7MPCORE_ADDR);
+    gic = DEVICE(&s->a7mpcore);
 
     for (i = 0; i < smp_cpus; i++) {
         SysBusDevice *sbd = SYS_BUS_DEVICE(&s->a7mpcore);
@@ -252,8 +254,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         sysbus_realize(SYS_BUS_DEVICE(&s->gpt[i]), &error_abort);
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->gpt[i]), 0, FSL_IMX7_GPTn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpt[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_GPTn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_GPTn_IRQ[i]));
     }
 
     /*
@@ -295,12 +296,10 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
                         FSL_IMX7_GPIOn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_GPIOn_LOW_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_GPIOn_LOW_IRQ[i]));
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio[i]), 1,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_GPIOn_HIGH_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_GPIOn_HIGH_IRQ[i]));
     }
 
     /*
@@ -352,8 +351,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->spi[i]), 0,
                         FSL_IMX7_SPIn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->spi[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_SPIn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_SPIn_IRQ[i]));
     }
 
     /*
@@ -378,8 +376,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->i2c[i]), 0, FSL_IMX7_I2Cn_ADDR[i]);
 
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->i2c[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_I2Cn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_I2Cn_IRQ[i]));
     }
 
     /*
@@ -413,7 +410,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->uart[i]), 0, FSL_IMX7_UARTn_ADDR[i]);
 
-        irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_UARTn_IRQ[i]);
+        irq = qdev_get_gpio_in(gic, FSL_IMX7_UARTn_IRQ[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->uart[i]), 0, irq);
     }
 
@@ -451,9 +448,9 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->eth[i]), 0, FSL_IMX7_ENETn_ADDR[i]);
 
-        irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_ENET_IRQ(i, 0));
+        irq = qdev_get_gpio_in(gic, FSL_IMX7_ENET_IRQ(i, 0));
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth[i]), 0, irq);
-        irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_ENET_IRQ(i, 3));
+        irq = qdev_get_gpio_in(gic, FSL_IMX7_ENET_IRQ(i, 3));
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->eth[i]), 1, irq);
     }
 
@@ -480,7 +477,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->usdhc[i]), 0,
                         FSL_IMX7_USDHCn_ADDR[i]);
 
-        irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_USDHCn_IRQ[i]);
+        irq = qdev_get_gpio_in(gic, FSL_IMX7_USDHCn_IRQ[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->usdhc[i]), 0, irq);
     }
 
@@ -519,8 +516,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
 
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->wdt[i]), 0, FSL_IMX7_WDOGn_ADDR[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->wdt[i]), 0,
-                           qdev_get_gpio_in(DEVICE(&s->a7mpcore),
-                                            FSL_IMX7_WDOGn_IRQ[i]));
+                           qdev_get_gpio_in(gic, FSL_IMX7_WDOGn_IRQ[i]));
     }
 
     /*
@@ -596,13 +592,13 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
     sysbus_realize(SYS_BUS_DEVICE(&s->pcie), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->pcie), 0, FSL_IMX7_PCIE_REG_ADDR);
 
-    irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_PCI_INTA_IRQ);
+    irq = qdev_get_gpio_in(gic, FSL_IMX7_PCI_INTA_IRQ);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->pcie), 0, irq);
-    irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_PCI_INTB_IRQ);
+    irq = qdev_get_gpio_in(gic, FSL_IMX7_PCI_INTB_IRQ);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->pcie), 1, irq);
-    irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_PCI_INTC_IRQ);
+    irq = qdev_get_gpio_in(gic, FSL_IMX7_PCI_INTC_IRQ);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->pcie), 2, irq);
-    irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_PCI_INTD_IRQ);
+    irq = qdev_get_gpio_in(gic, FSL_IMX7_PCI_INTD_IRQ);
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->pcie), 3, irq);
 
     /*
@@ -631,7 +627,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->usb[i]), 0,
                         FSL_IMX7_USBn_ADDR[i]);
 
-        irq = qdev_get_gpio_in(DEVICE(&s->a7mpcore), FSL_IMX7_USBn_IRQ[i]);
+        irq = qdev_get_gpio_in(gic, FSL_IMX7_USBn_IRQ[i]);
         sysbus_connect_irq(SYS_BUS_DEVICE(&s->usb[i]), 0, irq);
 
         snprintf(name, NAME_SIZE, "usbmisc%d", i);
-- 
2.41.0

Re: [PATCH 04/33] hw/arm/fsl-imx7: Add a local 'gic' variable

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> The A7MPCore forward the IRQs from its internal GIC.
> To make the code clearer, add a 'gic' variable.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 05/33] hw/cpu: Remove dead Kconfig

[Philippe Mathieu-Daudé]

ARM MPCore Kconfig are defined in hw/arm/Kconfig.
hw/cpu/Kconfig is never used, remove it.

Fixes: 82f5181777 ("kconfig: introduce kconfig files")
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/Kconfig | 8 --------
 1 file changed, 8 deletions(-)
 delete mode 100644 hw/cpu/Kconfig

diff --git a/hw/cpu/Kconfig b/hw/cpu/Kconfig
deleted file mode 100644
index 1767d028ac..0000000000
--- a/hw/cpu/Kconfig
+++ /dev/null
@@ -1,8 +0,0 @@
-config ARM11MPCORE
-    bool
-
-config A9MPCORE
-    bool
-
-config A15MPCORE
-    bool
-- 
2.41.0

Re: [PATCH 05/33] hw/cpu: Remove dead Kconfig

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> ARM MPCore Kconfig are defined in hw/arm/Kconfig.
> hw/cpu/Kconfig is never used, remove it.
> 
> Fixes: 82f5181777 ("kconfig: introduce kconfig files")
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 06/33] hw/cpu/arm: Rename 'busdev' -> 'gicsbd' in a15mp_priv_realize()

[Philippe Mathieu-Daudé]

'busdev' is the internal GIC as SysBus device.
Since we already have a 'gicdev' variable for the GIC as QDev,
rename 'busdev' as 'gicsbd' to make it clear we access the IRQ
lines from the GIC.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/a15mpcore.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index bfd8aa5644..a40f142128 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -51,7 +51,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A15MPPrivState *s = A15MPCORE_PRIV(dev);
     DeviceState *gicdev;
-    SysBusDevice *busdev;
+    SysBusDevice *gicsbd;
     int i;
     bool has_el3;
     bool has_el2 = false;
@@ -78,10 +78,10 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
         return;
     }
-    busdev = SYS_BUS_DEVICE(&s->gic);
+    gicsbd = SYS_BUS_DEVICE(&s->gic);
 
     /* Pass through outbound IRQ lines from the GIC */
-    sysbus_pass_irq(sbd, busdev);
+    sysbus_pass_irq(sbd, gicsbd);
 
     /* Pass through inbound GPIO lines to the GIC */
     qdev_init_gpio_in(dev, a15mp_priv_set_irq, s->num_irq - 32);
@@ -126,17 +126,17 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
      *  0x6000-0x7fff -- GIC virtual CPU interface
      */
     memory_region_add_subregion(&s->container, 0x1000,
-                                sysbus_mmio_get_region(busdev, 0));
+                                sysbus_mmio_get_region(gicsbd, 0));
     memory_region_add_subregion(&s->container, 0x2000,
-                                sysbus_mmio_get_region(busdev, 1));
+                                sysbus_mmio_get_region(gicsbd, 1));
     if (has_el2) {
         memory_region_add_subregion(&s->container, 0x4000,
-                                    sysbus_mmio_get_region(busdev, 2));
+                                    sysbus_mmio_get_region(gicsbd, 2));
         memory_region_add_subregion(&s->container, 0x6000,
-                                    sysbus_mmio_get_region(busdev, 3));
+                                    sysbus_mmio_get_region(gicsbd, 3));
         for (i = 0; i < s->num_cpu; i++) {
             hwaddr base = 0x5000 + i * 0x200;
-            MemoryRegion *mr = sysbus_mmio_get_region(busdev,
+            MemoryRegion *mr = sysbus_mmio_get_region(gicsbd,
                                                       4 + s->num_cpu + i);
             memory_region_add_subregion(&s->container, base, mr);
         }
-- 
2.41.0

Re: [PATCH 06/33] hw/cpu/arm: Rename 'busdev' -> 'gicsbd' in a15mp_priv_realize()

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> 'busdev' is the internal GIC as SysBus device.
> Since we already have a 'gicdev' variable for the GIC as QDev,
> rename 'busdev' as 'gicsbd' to make it clear we access the IRQ
> lines from the GIC.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 07/33] hw/cpu/arm: Alias 'num-cpu' property on TYPE_REALVIEW_MPCORE

[Philippe Mathieu-Daudé]

No need to duplicate and forward the 'num-cpu' property from
TYPE_ARM11MPCORE_PRIV to TYPE_REALVIEW_MPCORE, alias it with
QOM object_property_add_alias().

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/realview_mpcore.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/hw/cpu/realview_mpcore.c b/hw/cpu/realview_mpcore.c
index 72c792eef1..7480b38d1a 100644
--- a/hw/cpu/realview_mpcore.c
+++ b/hw/cpu/realview_mpcore.c
@@ -14,7 +14,6 @@
 #include "hw/cpu/arm11mpcore.h"
 #include "hw/intc/realview_gic.h"
 #include "hw/irq.h"
-#include "hw/qdev-properties.h"
 #include "qom/object.h"
 
 #define TYPE_REALVIEW_MPCORE_RIRQ "realview_mpcore"
@@ -68,7 +67,6 @@ static void realview_mpcore_realize(DeviceState *dev, Error **errp)
     int n;
     int i;
 
-    qdev_prop_set_uint32(priv, "num-cpu", s->num_cpu);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->priv), errp)) {
         return;
     }
@@ -100,6 +98,7 @@ static void mpcore_rirq_init(Object *obj)
     int i;
 
     object_initialize_child(obj, "a11priv", &s->priv, TYPE_ARM11MPCORE_PRIV);
+    object_property_add_alias(obj, "num-cpu", OBJECT(&s->priv), "num-cpu");
     privbusdev = SYS_BUS_DEVICE(&s->priv);
     sysbus_init_mmio(sbd, sysbus_mmio_get_region(privbusdev, 0));
 
@@ -108,17 +107,11 @@ static void mpcore_rirq_init(Object *obj)
     }
 }
 
-static Property mpcore_rirq_properties[] = {
-    DEFINE_PROP_UINT32("num-cpu", mpcore_rirq_state, num_cpu, 1),
-    DEFINE_PROP_END_OF_LIST(),
-};
-
 static void mpcore_rirq_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->realize = realview_mpcore_realize;
-    device_class_set_props(dc, mpcore_rirq_properties);
 }
 
 static const TypeInfo mpcore_rirq_info = {
-- 
2.41.0

Re: [PATCH 07/33] hw/cpu/arm: Alias 'num-cpu' property on TYPE_REALVIEW_MPCORE

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> No need to duplicate and forward the 'num-cpu' property from
> TYPE_ARM11MPCORE_PRIV to TYPE_REALVIEW_MPCORE, alias it with
> QOM object_property_add_alias().
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 08/33] hw/cpu/arm: Declare CPU QOM types using DEFINE_TYPES() macro

[Philippe Mathieu-Daudé]

When multiple QOM types are registered in the same file,
it is simpler to use the the DEFINE_TYPES() macro. In
particular because type array declared with such macro
are easier to review.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/a15mpcore.c       | 21 +++++++++------------
 hw/cpu/a9mpcore.c        | 21 +++++++++------------
 hw/cpu/arm11mpcore.c     | 21 +++++++++------------
 hw/cpu/realview_mpcore.c | 21 +++++++++------------
 4 files changed, 36 insertions(+), 48 deletions(-)

diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index a40f142128..c2ee985fb4 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -164,17 +164,14 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
     /* We currently have no saveable state */
 }
 
-static const TypeInfo a15mp_priv_info = {
-    .name  = TYPE_A15MPCORE_PRIV,
-    .parent = TYPE_SYS_BUS_DEVICE,
-    .instance_size  = sizeof(A15MPPrivState),
-    .instance_init = a15mp_priv_initfn,
-    .class_init = a15mp_priv_class_init,
+static const TypeInfo a15mp_types[] = {
+    {
+        .name           = TYPE_A15MPCORE_PRIV,
+        .parent         = TYPE_SYS_BUS_DEVICE,
+        .instance_size  = sizeof(A15MPPrivState),
+        .instance_init  = a15mp_priv_initfn,
+        .class_init     = a15mp_priv_class_init,
+    },
 };
 
-static void a15mp_register_types(void)
-{
-    type_register_static(&a15mp_priv_info);
-}
-
-type_init(a15mp_register_types)
+DEFINE_TYPES(a15mp_types)
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index d03f57e579..6b5ce01246 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -178,17 +178,14 @@ static void a9mp_priv_class_init(ObjectClass *klass, void *data)
     device_class_set_props(dc, a9mp_priv_properties);
 }
 
-static const TypeInfo a9mp_priv_info = {
-    .name          = TYPE_A9MPCORE_PRIV,
-    .parent        = TYPE_SYS_BUS_DEVICE,
-    .instance_size = sizeof(A9MPPrivState),
-    .instance_init = a9mp_priv_initfn,
-    .class_init    = a9mp_priv_class_init,
+static const TypeInfo a9mp_types[] = {
+    {
+        .name           = TYPE_A9MPCORE_PRIV,
+        .parent         = TYPE_SYS_BUS_DEVICE,
+        .instance_size  =  sizeof(A9MPPrivState),
+        .instance_init  = a9mp_priv_initfn,
+        .class_init     = a9mp_priv_class_init,
+    },
 };
 
-static void a9mp_register_types(void)
-{
-    type_register_static(&a9mp_priv_info);
-}
-
-type_init(a9mp_register_types)
+DEFINE_TYPES(a9mp_types)
diff --git a/hw/cpu/arm11mpcore.c b/hw/cpu/arm11mpcore.c
index 89c4e35143..74f44434de 100644
--- a/hw/cpu/arm11mpcore.c
+++ b/hw/cpu/arm11mpcore.c
@@ -153,17 +153,14 @@ static void mpcore_priv_class_init(ObjectClass *klass, void *data)
     device_class_set_props(dc, mpcore_priv_properties);
 }
 
-static const TypeInfo mpcore_priv_info = {
-    .name          = TYPE_ARM11MPCORE_PRIV,
-    .parent        = TYPE_SYS_BUS_DEVICE,
-    .instance_size = sizeof(ARM11MPCorePriveState),
-    .instance_init = mpcore_priv_initfn,
-    .class_init    = mpcore_priv_class_init,
+static const TypeInfo arm11mp_types[] = {
+    {
+        .name           = TYPE_ARM11MPCORE_PRIV,
+        .parent         = TYPE_SYS_BUS_DEVICE,
+        .instance_size  = sizeof(ARM11MPCorePriveState),
+        .instance_init  = mpcore_priv_initfn,
+        .class_init     = mpcore_priv_class_init,
+    },
 };
 
-static void arm11mpcore_register_types(void)
-{
-    type_register_static(&mpcore_priv_info);
-}
-
-type_init(arm11mpcore_register_types)
+DEFINE_TYPES(arm11mp_types)
diff --git a/hw/cpu/realview_mpcore.c b/hw/cpu/realview_mpcore.c
index 7480b38d1a..b140888618 100644
--- a/hw/cpu/realview_mpcore.c
+++ b/hw/cpu/realview_mpcore.c
@@ -114,17 +114,14 @@ static void mpcore_rirq_class_init(ObjectClass *klass, void *data)
     dc->realize = realview_mpcore_realize;
 }
 
-static const TypeInfo mpcore_rirq_info = {
-    .name          = TYPE_REALVIEW_MPCORE_RIRQ,
-    .parent        = TYPE_SYS_BUS_DEVICE,
-    .instance_size = sizeof(mpcore_rirq_state),
-    .instance_init = mpcore_rirq_init,
-    .class_init    = mpcore_rirq_class_init,
+static const TypeInfo realview_mpcore_types[] = {
+    {
+        .name           = TYPE_REALVIEW_MPCORE_RIRQ,
+        .parent         = TYPE_SYS_BUS_DEVICE,
+        .instance_size  = sizeof(mpcore_rirq_state),
+        .instance_init  = mpcore_rirq_init,
+        .class_init     = mpcore_rirq_class_init,
+    },
 };
 
-static void realview_mpcore_register_types(void)
-{
-    type_register_static(&mpcore_rirq_info);
-}
-
-type_init(realview_mpcore_register_types)
+DEFINE_TYPES(realview_mpcore_types)
-- 
2.41.0

Re: [PATCH 08/33] hw/cpu/arm: Declare CPU QOM types using DEFINE_TYPES() macro

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> When multiple QOM types are registered in the same file,
> it is simpler to use the the DEFINE_TYPES() macro. In
> particular because type array declared with such macro
> are easier to review.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 09/33] hw/cpu/arm: Merge {a9mpcore.h, a15mpcore.h} as cortex_mpcore.h

[Philippe Mathieu-Daudé]

Merge Cortex-A{9,15} MPCore devices in the same header.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 MAINTAINERS                                   |  3 +-
 include/hw/arm/aspeed_soc.h                   |  2 +-
 include/hw/arm/exynos4210.h                   |  2 +-
 include/hw/arm/fsl-imx6.h                     |  2 +-
 include/hw/arm/fsl-imx6ul.h                   |  2 +-
 include/hw/arm/fsl-imx7.h                     |  2 +-
 include/hw/arm/npcm7xx.h                      |  2 +-
 include/hw/cpu/a15mpcore.h                    | 44 -------------------
 .../hw/cpu/{a9mpcore.h => cortex_mpcore.h}    | 29 ++++++++----
 hw/arm/exynos4210.c                           |  2 +-
 hw/arm/highbank.c                             |  3 +-
 hw/arm/realview.c                             |  2 +-
 hw/arm/vexpress.c                             |  3 +-
 hw/arm/xilinx_zynq.c                          |  2 +-
 hw/cpu/a15mpcore.c                            |  2 +-
 hw/cpu/a9mpcore.c                             |  2 +-
 16 files changed, 35 insertions(+), 69 deletions(-)
 delete mode 100644 include/hw/cpu/a15mpcore.h
 rename include/hw/cpu/{a9mpcore.h => cortex_mpcore.h} (57%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 695e0bd34f..8a37e5962f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -203,7 +203,8 @@ F: tests/tcg/aarch64/
 F: tests/qtest/arm-cpu-features.c
 F: hw/arm/
 F: hw/cpu/a*mpcore.c
-F: include/hw/cpu/a*mpcore.h
+F: include/hw/cpu/arm11mpcore.h
+F: include/hw/cpu/cortex_mpcore.h
 F: docs/system/target-arm.rst
 F: docs/system/arm/cpu-features.rst
 
diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index cb832bc1ee..e0663ab50d 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -12,7 +12,7 @@
 #ifndef ASPEED_SOC_H
 #define ASPEED_SOC_H
 
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/arm/armv7m.h"
 #include "hw/intc/aspeed_vic.h"
 #include "hw/misc/aspeed_scu.h"
diff --git a/include/hw/arm/exynos4210.h b/include/hw/arm/exynos4210.h
index d33fe38586..f95e3232c5 100644
--- a/include/hw/arm/exynos4210.h
+++ b/include/hw/arm/exynos4210.h
@@ -26,7 +26,7 @@
 
 #include "hw/or-irq.h"
 #include "hw/sysbus.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/intc/exynos4210_gic.h"
 #include "hw/intc/exynos4210_combiner.h"
 #include "hw/core/split-irq.h"
diff --git a/include/hw/arm/fsl-imx6.h b/include/hw/arm/fsl-imx6.h
index 519b871014..21a3b035a4 100644
--- a/include/hw/arm/fsl-imx6.h
+++ b/include/hw/arm/fsl-imx6.h
@@ -17,7 +17,7 @@
 #ifndef FSL_IMX6_H
 #define FSL_IMX6_H
 
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/misc/imx6_ccm.h"
 #include "hw/misc/imx6_src.h"
 #include "hw/misc/imx7_snvs.h"
diff --git a/include/hw/arm/fsl-imx6ul.h b/include/hw/arm/fsl-imx6ul.h
index 14390f6014..5ed22004d1 100644
--- a/include/hw/arm/fsl-imx6ul.h
+++ b/include/hw/arm/fsl-imx6ul.h
@@ -17,7 +17,7 @@
 #ifndef FSL_IMX6UL_H
 #define FSL_IMX6UL_H
 
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/misc/imx6ul_ccm.h"
 #include "hw/misc/imx6_src.h"
 #include "hw/misc/imx7_snvs.h"
diff --git a/include/hw/arm/fsl-imx7.h b/include/hw/arm/fsl-imx7.h
index 411fa1c2e3..87faee33c2 100644
--- a/include/hw/arm/fsl-imx7.h
+++ b/include/hw/arm/fsl-imx7.h
@@ -19,7 +19,7 @@
 #ifndef FSL_IMX7_H
 #define FSL_IMX7_H
 
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/intc/imx_gpcv2.h"
 #include "hw/misc/imx7_ccm.h"
 #include "hw/misc/imx7_snvs.h"
diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h
index 72c7722096..7abbf85cbf 100644
--- a/include/hw/arm/npcm7xx.h
+++ b/include/hw/arm/npcm7xx.h
@@ -19,7 +19,7 @@
 #include "hw/boards.h"
 #include "hw/adc/npcm7xx_adc.h"
 #include "hw/core/split-irq.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/gpio/npcm7xx_gpio.h"
 #include "hw/i2c/npcm7xx_smbus.h"
 #include "hw/mem/npcm7xx_mc.h"
diff --git a/include/hw/cpu/a15mpcore.h b/include/hw/cpu/a15mpcore.h
deleted file mode 100644
index 75d39e5458..0000000000
--- a/include/hw/cpu/a15mpcore.h
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Cortex-A15MPCore internal peripheral emulation.
- *
- * Copyright (c) 2012 Linaro Limited.
- * Written by Peter Maydell.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef HW_CPU_A15MPCORE_H
-#define HW_CPU_A15MPCORE_H
-
-#include "hw/sysbus.h"
-#include "hw/intc/arm_gic.h"
-#include "qom/object.h"
-
-/* A15MP private memory region.  */
-
-#define TYPE_A15MPCORE_PRIV "a15mpcore_priv"
-OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
-
-struct A15MPPrivState {
-    /*< private >*/
-    SysBusDevice parent_obj;
-    /*< public >*/
-
-    uint32_t num_cpu;
-    uint32_t num_irq;
-    MemoryRegion container;
-
-    GICState gic;
-};
-
-#endif
diff --git a/include/hw/cpu/a9mpcore.h b/include/hw/cpu/cortex_mpcore.h
similarity index 57%
rename from include/hw/cpu/a9mpcore.h
rename to include/hw/cpu/cortex_mpcore.h
index e0396ab6af..9147f35c2a 100644
--- a/include/hw/cpu/a9mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -1,29 +1,27 @@
 /*
- * Cortex-A9MPCore internal peripheral emulation.
+ * Cortex-MPCore internal peripheral emulation.
  *
  * Copyright (c) 2009 CodeSourcery.
- * Copyright (c) 2011 Linaro Limited.
+ * Copyright (c) 2011, 2012, 2023 Linaro Limited.
  * Written by Paul Brook, Peter Maydell.
  *
- * This code is licensed under the GPL.
+ * SPDX-License-Identifier: GPL-2.0-or-later
  */
-#ifndef HW_CPU_A9MPCORE_H
-#define HW_CPU_A9MPCORE_H
+#ifndef HW_CPU_CORTEX_MPCORE_H
+#define HW_CPU_CORTEX_MPCORE_H
 
+#include "qom/object.h"
 #include "hw/sysbus.h"
 #include "hw/intc/arm_gic.h"
 #include "hw/misc/a9scu.h"
-#include "hw/timer/arm_mptimer.h"
 #include "hw/timer/a9gtimer.h"
-#include "qom/object.h"
+#include "hw/timer/arm_mptimer.h"
 
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
 OBJECT_DECLARE_SIMPLE_TYPE(A9MPPrivState, A9MPCORE_PRIV)
 
 struct A9MPPrivState {
-    /*< private >*/
     SysBusDevice parent_obj;
-    /*< public >*/
 
     uint32_t num_cpu;
     MemoryRegion container;
@@ -36,4 +34,17 @@ struct A9MPPrivState {
     ARMMPTimerState wdt;
 };
 
+#define TYPE_A15MPCORE_PRIV "a15mpcore_priv"
+OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
+
+struct A15MPPrivState {
+    SysBusDevice parent_obj;
+
+    uint32_t num_cpu;
+    uint32_t num_irq;
+    MemoryRegion container;
+
+    GICState gic;
+};
+
 #endif
diff --git a/hw/arm/exynos4210.c b/hw/arm/exynos4210.c
index de39fb0ece..2e6423f74d 100644
--- a/hw/arm/exynos4210.c
+++ b/hw/arm/exynos4210.c
@@ -24,7 +24,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "cpu.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/irq.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/sysemu.h"
diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index f12aacea6b..a184bee993 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -31,8 +31,7 @@
 #include "qemu/error-report.h"
 #include "hw/char/pl011.h"
 #include "hw/ide/ahci.h"
-#include "hw/cpu/a9mpcore.h"
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "qemu/log.h"
 #include "qom/object.h"
 #include "cpu.h"
diff --git a/hw/arm/realview.c b/hw/arm/realview.c
index 132217b2ed..a112a6b529 100644
--- a/hw/arm/realview.c
+++ b/hw/arm/realview.c
@@ -24,7 +24,7 @@
 #include "hw/i2c/i2c.h"
 #include "qemu/error-report.h"
 #include "hw/char/pl011.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/intc/realview_gic.h"
 #include "hw/irq.h"
 #include "hw/i2c/arm_sbcon_i2c.h"
diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index fd981f4c33..a320d1c181 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -39,8 +39,7 @@
 #include "qemu/error-report.h"
 #include <libfdt.h>
 #include "hw/char/pl011.h"
-#include "hw/cpu/a9mpcore.h"
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/i2c/arm_sbcon_i2c.h"
 #include "hw/sd/sd.h"
 #include "qapi/qmp/qlist.h"
diff --git a/hw/arm/xilinx_zynq.c b/hw/arm/xilinx_zynq.c
index dbb9793aa1..fc9c927d09 100644
--- a/hw/arm/xilinx_zynq.c
+++ b/hw/arm/xilinx_zynq.c
@@ -33,7 +33,7 @@
 #include "hw/sd/sdhci.h"
 #include "hw/char/cadence_uart.h"
 #include "hw/net/cadence_gem.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/qdev-clock.h"
 #include "sysemu/reset.h"
 #include "qom/object.h"
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index c2ee985fb4..0f56c40b86 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -21,7 +21,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
-#include "hw/cpu/a15mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/irq.h"
 #include "hw/qdev-properties.h"
 #include "sysemu/kvm.h"
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index 6b5ce01246..e9cd71e92f 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -11,7 +11,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
-#include "hw/cpu/a9mpcore.h"
+#include "hw/cpu/cortex_mpcore.h"
 #include "hw/irq.h"
 #include "hw/qdev-properties.h"
 #include "hw/core/cpu.h"
-- 
2.41.0

Re: [PATCH 09/33] hw/cpu/arm: Merge {a9mpcore.h, a15mpcore.h} as cortex_mpcore.h

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> Merge Cortex-A{9,15} MPCore devices in the same header.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 10/33] hw/cpu/arm: Introduce abstract CORTEX_MPCORE_PRIV QOM type

[Philippe Mathieu-Daudé]

This type will be common to A9MPCORE/A15MPCORE devices.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 21 +++++++++++++++++++++
 hw/cpu/cortex_mpcore.c         | 22 ++++++++++++++++++++++
 hw/arm/Kconfig                 |  3 +++
 hw/cpu/meson.build             |  1 +
 4 files changed, 47 insertions(+)
 create mode 100644 hw/cpu/cortex_mpcore.c

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 9147f35c2a..c5dd7200d0 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -17,6 +17,27 @@
 #include "hw/timer/a9gtimer.h"
 #include "hw/timer/arm_mptimer.h"
 
+/*
+ * This is a model of the Arm Cortex-A MPCore family of hardware.
+ *
+ * The A7 and A15 MPCore contain:
+ *  up to 4 Cortex-A cores
+ *  a GIC
+ * The A9 MPCore additionally contains:
+ *  a System Control Unit
+ *  some timers and watchdogs
+ */
+#define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
+OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
+
+struct CortexMPPrivClass {
+    SysBusDeviceClass parent_class;
+};
+
+struct CortexMPPrivState {
+    SysBusDevice parent_obj;
+};
+
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
 OBJECT_DECLARE_SIMPLE_TYPE(A9MPPrivState, A9MPCORE_PRIV)
 
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
new file mode 100644
index 0000000000..7d3796bd69
--- /dev/null
+++ b/hw/cpu/cortex_mpcore.c
@@ -0,0 +1,22 @@
+/*
+ * Cortex-MPCore internal peripheral emulation.
+ *
+ * Copyright (c) 2023 Linaro Limited.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cpu/cortex_mpcore.h"
+
+static const TypeInfo cortex_mpcore_types[] = {
+    {
+        .name           = TYPE_CORTEX_MPCORE_PRIV,
+        .parent         = TYPE_SYS_BUS_DEVICE,
+        .instance_size  = sizeof(CortexMPPrivState),
+        .class_size     = sizeof(CortexMPPrivClass),
+        .abstract       = true,
+    },
+};
+
+DEFINE_TYPES(cortex_mpcore_types)
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 3ada335a24..3040342fcb 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -643,6 +643,9 @@ config ZAURUS
     select NAND
     select ECC
 
+config CORTEX_MPCORE
+    bool
+
 config A9MPCORE
     bool
     select A9_GTIMER
diff --git a/hw/cpu/meson.build b/hw/cpu/meson.build
index 6d319947ca..8b376d6e7c 100644
--- a/hw/cpu/meson.build
+++ b/hw/cpu/meson.build
@@ -2,5 +2,6 @@ system_ss.add(files('core.c', 'cluster.c'))
 
 system_ss.add(when: 'CONFIG_ARM11MPCORE', if_true: files('arm11mpcore.c'))
 system_ss.add(when: 'CONFIG_REALVIEW', if_true: files('realview_mpcore.c'))
+specific_ss.add(when: 'CONFIG_CORTEX_MPCORE', if_true: files('cortex_mpcore.c'))
 specific_ss.add(when: 'CONFIG_A9MPCORE', if_true: files('a9mpcore.c'))
 specific_ss.add(when: 'CONFIG_A15MPCORE', if_true: files('a15mpcore.c'))
-- 
2.41.0

Re: [PATCH 10/33] hw/cpu/arm: Introduce abstract CORTEX_MPCORE_PRIV QOM type

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> This type will be common to A9MPCORE/A15MPCORE devices.
> 

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 11/33] hw/cpu/arm: Have A9MPCORE/A15MPCORE inheritate common CORTEX_MPCORE_PRIV

[Philippe Mathieu-Daudé]

TYPE_CORTEX_MPCORE_PRIV becomes the common parent to
TYPE_A9MPCORE_PRIV and TYPE_A15MPCORE_PRIV.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h |  6 ++++--
 hw/cpu/a15mpcore.c             | 14 ++++++++++++--
 hw/cpu/a9mpcore.c              | 13 +++++++++++--
 hw/arm/Kconfig                 |  2 ++
 4 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index c5dd7200d0..4084c6003a 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -32,6 +32,8 @@ OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
 
 struct CortexMPPrivClass {
     SysBusDeviceClass parent_class;
+
+    DeviceRealize parent_realize;
 };
 
 struct CortexMPPrivState {
@@ -42,7 +44,7 @@ struct CortexMPPrivState {
 OBJECT_DECLARE_SIMPLE_TYPE(A9MPPrivState, A9MPCORE_PRIV)
 
 struct A9MPPrivState {
-    SysBusDevice parent_obj;
+    CortexMPPrivState parent_obj;
 
     uint32_t num_cpu;
     MemoryRegion container;
@@ -59,7 +61,7 @@ struct A9MPPrivState {
 OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
 
 struct A15MPPrivState {
-    SysBusDevice parent_obj;
+    CortexMPPrivState parent_obj;
 
     uint32_t num_cpu;
     uint32_t num_irq;
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 0f56c40b86..5a57145179 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -48,15 +48,23 @@ static void a15mp_priv_initfn(Object *obj)
 
 static void a15mp_priv_realize(DeviceState *dev, Error **errp)
 {
+    CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A15MPPrivState *s = A15MPCORE_PRIV(dev);
     DeviceState *gicdev;
     SysBusDevice *gicsbd;
+    Error *local_err = NULL;
     int i;
     bool has_el3;
     bool has_el2 = false;
     Object *cpuobj;
 
+    cc->parent_realize(dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
     gicdev = DEVICE(&s->gic);
     qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cpu);
     qdev_prop_set_uint32(gicdev, "num-irq", s->num_irq);
@@ -158,8 +166,10 @@ static Property a15mp_priv_properties[] = {
 static void a15mp_priv_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
+    CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_CLASS(klass);
 
-    dc->realize = a15mp_priv_realize;
+    device_class_set_parent_realize(dc, a15mp_priv_realize,
+                                    &cc->parent_realize);
     device_class_set_props(dc, a15mp_priv_properties);
     /* We currently have no saveable state */
 }
@@ -167,7 +177,7 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
 static const TypeInfo a15mp_types[] = {
     {
         .name           = TYPE_A15MPCORE_PRIV,
-        .parent         = TYPE_SYS_BUS_DEVICE,
+        .parent         = TYPE_CORTEX_MPCORE_PRIV,
         .instance_size  = sizeof(A15MPPrivState),
         .instance_init  = a15mp_priv_initfn,
         .class_init     = a15mp_priv_class_init,
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index e9cd71e92f..d59e49126b 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -46,11 +46,13 @@ static void a9mp_priv_initfn(Object *obj)
 
 static void a9mp_priv_realize(DeviceState *dev, Error **errp)
 {
+    CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A9MPPrivState *s = A9MPCORE_PRIV(dev);
     DeviceState *scudev, *gicdev, *gtimerdev, *mptimerdev, *wdtdev;
     SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
                  *wdtbusdev;
+    Error *local_err = NULL;
     int i;
     bool has_el3;
     CPUState *cpu0;
@@ -65,6 +67,12 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    cc->parent_realize(dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
     scudev = DEVICE(&s->scu);
     qdev_prop_set_uint32(scudev, "num-cpu", s->num_cpu);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->scu), errp)) {
@@ -173,15 +181,16 @@ static Property a9mp_priv_properties[] = {
 static void a9mp_priv_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
+    CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_CLASS(klass);
 
-    dc->realize = a9mp_priv_realize;
+    device_class_set_parent_realize(dc, a9mp_priv_realize, &cc->parent_realize);
     device_class_set_props(dc, a9mp_priv_properties);
 }
 
 static const TypeInfo a9mp_types[] = {
     {
         .name           = TYPE_A9MPCORE_PRIV,
-        .parent         = TYPE_SYS_BUS_DEVICE,
+        .parent         = TYPE_CORTEX_MPCORE_PRIV,
         .instance_size  =  sizeof(A9MPPrivState),
         .instance_init  = a9mp_priv_initfn,
         .class_init     = a9mp_priv_class_init,
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 3040342fcb..ec6e585797 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -648,6 +648,7 @@ config CORTEX_MPCORE
 
 config A9MPCORE
     bool
+    select CORTEX_MPCORE
     select A9_GTIMER
     select A9SCU       # snoop control unit
     select ARM_GIC
@@ -655,6 +656,7 @@ config A9MPCORE
 
 config A15MPCORE
     bool
+    select CORTEX_MPCORE
     select ARM_GIC
 
 config ARM11MPCORE
-- 
2.41.0

Re: [PATCH 11/33] hw/cpu/arm: Have A9MPCORE/A15MPCORE inheritate common CORTEX_MPCORE_PRIV

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> TYPE_CORTEX_MPCORE_PRIV becomes the common parent to
> TYPE_A9MPCORE_PRIV and TYPE_A15MPCORE_PRIV.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 12/33] hw/cpu/arm: Create MPCore container in QOM parent

[Philippe Mathieu-Daudé]

Move the memory region container creation to the abstract QOM
parent. Children set the region size via the class 'container_size'
field.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 10 ++++++++--
 hw/cpu/a15mpcore.c             | 17 ++++++++---------
 hw/cpu/a9mpcore.c              | 18 +++++++++---------
 hw/cpu/cortex_mpcore.c         | 14 ++++++++++++++
 4 files changed, 39 insertions(+), 20 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 4084c6003a..1d94c8769d 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -30,14 +30,22 @@
 #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
 OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
 
+/**
+ * CortexMPPrivClass:
+ * @container_size - size of the device's MMIO region
+ */
 struct CortexMPPrivClass {
     SysBusDeviceClass parent_class;
 
     DeviceRealize parent_realize;
+
+    uint64_t container_size;
 };
 
 struct CortexMPPrivState {
     SysBusDevice parent_obj;
+
+    MemoryRegion container;
 };
 
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
@@ -47,7 +55,6 @@ struct A9MPPrivState {
     CortexMPPrivState parent_obj;
 
     uint32_t num_cpu;
-    MemoryRegion container;
     uint32_t num_irq;
 
     A9SCUState scu;
@@ -65,7 +72,6 @@ struct A15MPPrivState {
 
     uint32_t num_cpu;
     uint32_t num_irq;
-    MemoryRegion container;
 
     GICState gic;
 };
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 5a57145179..128941eb50 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -36,12 +36,8 @@ static void a15mp_priv_set_irq(void *opaque, int irq, int level)
 
 static void a15mp_priv_initfn(Object *obj)
 {
-    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
     A15MPPrivState *s = A15MPCORE_PRIV(obj);
 
-    memory_region_init(&s->container, obj, "a15mp-priv-container", 0x8000);
-    sysbus_init_mmio(sbd, &s->container);
-
     object_initialize_child(obj, "gic", &s->gic, gic_class_name());
     qdev_prop_set_uint32(DEVICE(&s->gic), "revision", 2);
 }
@@ -51,6 +47,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A15MPPrivState *s = A15MPCORE_PRIV(dev);
+    CortexMPPrivState *c = CORTEX_MPCORE_PRIV(dev);
     DeviceState *gicdev;
     SysBusDevice *gicsbd;
     Error *local_err = NULL;
@@ -133,20 +130,20 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
      *  0x5600-0x57ff -- GIC virtual interface control for CPU 3
      *  0x6000-0x7fff -- GIC virtual CPU interface
      */
-    memory_region_add_subregion(&s->container, 0x1000,
+    memory_region_add_subregion(&c->container, 0x1000,
                                 sysbus_mmio_get_region(gicsbd, 0));
-    memory_region_add_subregion(&s->container, 0x2000,
+    memory_region_add_subregion(&c->container, 0x2000,
                                 sysbus_mmio_get_region(gicsbd, 1));
     if (has_el2) {
-        memory_region_add_subregion(&s->container, 0x4000,
+        memory_region_add_subregion(&c->container, 0x4000,
                                     sysbus_mmio_get_region(gicsbd, 2));
-        memory_region_add_subregion(&s->container, 0x6000,
+        memory_region_add_subregion(&c->container, 0x6000,
                                     sysbus_mmio_get_region(gicsbd, 3));
         for (i = 0; i < s->num_cpu; i++) {
             hwaddr base = 0x5000 + i * 0x200;
             MemoryRegion *mr = sysbus_mmio_get_region(gicsbd,
                                                       4 + s->num_cpu + i);
-            memory_region_add_subregion(&s->container, base, mr);
+            memory_region_add_subregion(&c->container, base, mr);
         }
     }
 }
@@ -168,6 +165,8 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
     DeviceClass *dc = DEVICE_CLASS(klass);
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_CLASS(klass);
 
+    cc->container_size = 0x8000;
+
     device_class_set_parent_realize(dc, a15mp_priv_realize,
                                     &cc->parent_realize);
     device_class_set_props(dc, a15mp_priv_properties);
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index d59e49126b..08346b0049 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -30,9 +30,6 @@ static void a9mp_priv_initfn(Object *obj)
 {
     A9MPPrivState *s = A9MPCORE_PRIV(obj);
 
-    memory_region_init(&s->container, obj, "a9mp-priv-container", 0x2000);
-    sysbus_init_mmio(SYS_BUS_DEVICE(obj), &s->container);
-
     object_initialize_child(obj, "scu", &s->scu, TYPE_A9_SCU);
 
     object_initialize_child(obj, "gic", &s->gic, TYPE_ARM_GIC);
@@ -47,6 +44,7 @@ static void a9mp_priv_initfn(Object *obj)
 static void a9mp_priv_realize(DeviceState *dev, Error **errp)
 {
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
+    CortexMPPrivState *c = CORTEX_MPCORE_PRIV(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A9MPPrivState *s = A9MPCORE_PRIV(dev);
     DeviceState *scudev, *gicdev, *gtimerdev, *mptimerdev, *wdtdev;
@@ -134,21 +132,21 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
      *  0x0700-0x0fff -- nothing
      *  0x1000-0x1fff -- GIC Distributor
      */
-    memory_region_add_subregion(&s->container, 0,
+    memory_region_add_subregion(&c->container, 0,
                                 sysbus_mmio_get_region(scubusdev, 0));
     /* GIC CPU interface */
-    memory_region_add_subregion(&s->container, 0x100,
+    memory_region_add_subregion(&c->container, 0x100,
                                 sysbus_mmio_get_region(gicbusdev, 1));
-    memory_region_add_subregion(&s->container, 0x200,
+    memory_region_add_subregion(&c->container, 0x200,
                                 sysbus_mmio_get_region(gtimerbusdev, 0));
     /* Note that the A9 exposes only the "timer/watchdog for this core"
      * memory region, not the "timer/watchdog for core X" ones 11MPcore has.
      */
-    memory_region_add_subregion(&s->container, 0x600,
+    memory_region_add_subregion(&c->container, 0x600,
                                 sysbus_mmio_get_region(mptimerbusdev, 0));
-    memory_region_add_subregion(&s->container, 0x620,
+    memory_region_add_subregion(&c->container, 0x620,
                                 sysbus_mmio_get_region(wdtbusdev, 0));
-    memory_region_add_subregion(&s->container, 0x1000,
+    memory_region_add_subregion(&c->container, 0x1000,
                                 sysbus_mmio_get_region(gicbusdev, 0));
 
     /* Wire up the interrupt from each watchdog and timer.
@@ -183,6 +181,8 @@ static void a9mp_priv_class_init(ObjectClass *klass, void *data)
     DeviceClass *dc = DEVICE_CLASS(klass);
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_CLASS(klass);
 
+    cc->container_size = 0x2000;
+
     device_class_set_parent_realize(dc, a9mp_priv_realize, &cc->parent_realize);
     device_class_set_props(dc, a9mp_priv_properties);
 }
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index 7d3796bd69..0c12f99610 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -7,13 +7,27 @@
  */
 
 #include "qemu/osdep.h"
+#include "qapi/error.h"
 #include "hw/cpu/cortex_mpcore.h"
 
+static void cortex_mpcore_priv_instance_init(Object *obj)
+{
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+    CortexMPPrivState *s = CORTEX_MPCORE_PRIV(obj);
+    CortexMPPrivClass *k = CORTEX_MPCORE_PRIV_GET_CLASS(obj);
+
+    assert(k->container_size > 0);
+    memory_region_init(&s->container, obj,
+                       "mpcore-priv-container", k->container_size);
+    sysbus_init_mmio(sbd, &s->container);
+}
+
 static const TypeInfo cortex_mpcore_types[] = {
     {
         .name           = TYPE_CORTEX_MPCORE_PRIV,
         .parent         = TYPE_SYS_BUS_DEVICE,
         .instance_size  = sizeof(CortexMPPrivState),
+        .instance_init  = cortex_mpcore_priv_instance_init,
         .class_size     = sizeof(CortexMPPrivClass),
         .abstract       = true,
     },
-- 
2.41.0

Re: [PATCH 12/33] hw/cpu/arm: Create MPCore container in QOM parent

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> Move the memory region container creation to the abstract QOM
> parent. Children set the region size via the class 'container_size'
> field.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

[PATCH 13/33] hw/cpu/arm: Handle 'num_cores' property once in MPCore parent

[Philippe Mathieu-Daudé]

Move the 'num_cpu' property to the abstract QOM parent.
Rename it as 'num_cores', keeping it aliased as 'num_cpu'.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h |  9 +++++++--
 hw/arm/aspeed_ast2600.c        |  2 +-
 hw/arm/exynos4210.c            |  2 +-
 hw/arm/fsl-imx6.c              |  2 +-
 hw/arm/fsl-imx6ul.c            |  2 +-
 hw/arm/fsl-imx7.c              |  2 +-
 hw/arm/npcm7xx.c               |  2 +-
 hw/arm/realview.c              |  2 +-
 hw/cpu/a15mpcore.c             | 11 +++++------
 hw/cpu/a9mpcore.c              | 14 ++++++--------
 hw/cpu/arm11mpcore.c           |  3 ++-
 hw/cpu/cortex_mpcore.c         | 17 +++++++++++++++++
 hw/cpu/realview_mpcore.c       |  3 ++-
 13 files changed, 46 insertions(+), 25 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 1d94c8769d..0e7cca9e93 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -26,6 +26,10 @@
  * The A9 MPCore additionally contains:
  *  a System Control Unit
  *  some timers and watchdogs
+ *
+ * QEMU interface:
+ *  + QOM property "num-cores" which set the number of cores present in
+ *    the cluster.
  */
 #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
 OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
@@ -46,6 +50,9 @@ struct CortexMPPrivState {
     SysBusDevice parent_obj;
 
     MemoryRegion container;
+
+    /* Properties */
+    uint32_t num_cores;
 };
 
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
@@ -54,7 +61,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(A9MPPrivState, A9MPCORE_PRIV)
 struct A9MPPrivState {
     CortexMPPrivState parent_obj;
 
-    uint32_t num_cpu;
     uint32_t num_irq;
 
     A9SCUState scu;
@@ -70,7 +76,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
 struct A15MPPrivState {
     CortexMPPrivState parent_obj;
 
-    uint32_t num_cpu;
     uint32_t num_irq;
 
     GICState gic;
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index b965fbab5e..ca788e1cf0 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -330,7 +330,7 @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
     }
 
     /* A7MPCORE */
-    object_property_set_int(OBJECT(&a->a7mpcore), "num-cpu", sc->num_cpus,
+    object_property_set_int(OBJECT(&a->a7mpcore), "num-cores", sc->num_cpus,
                             &error_abort);
     object_property_set_int(OBJECT(&a->a7mpcore), "num-irq",
                             ROUND_UP(AST2600_MAX_IRQ + GIC_INTERNAL, 32),
diff --git a/hw/arm/exynos4210.c b/hw/arm/exynos4210.c
index 2e6423f74d..ea1364499d 100644
--- a/hw/arm/exynos4210.c
+++ b/hw/arm/exynos4210.c
@@ -582,7 +582,7 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
     }
 
     /* Private memory region and Internal GIC */
-    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cpu", EXYNOS4210_NCPUS);
+    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cores", EXYNOS4210_NCPUS);
     busdev = SYS_BUS_DEVICE(&s->a9mpcore);
     sysbus_realize(busdev, &error_fatal);
     sysbus_mmio_map(busdev, 0, EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
index f6edbd7465..65c7c1a0f9 100644
--- a/hw/arm/fsl-imx6.c
+++ b/hw/arm/fsl-imx6.c
@@ -138,7 +138,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
         }
     }
 
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-cpu", smp_cpus,
+    object_property_set_int(OBJECT(&s->a9mpcore), "num-cores", smp_cpus,
                             &error_abort);
 
     object_property_set_int(OBJECT(&s->a9mpcore), "num-irq",
diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
index 59032263cf..93908811c5 100644
--- a/hw/arm/fsl-imx6ul.c
+++ b/hw/arm/fsl-imx6ul.c
@@ -173,7 +173,7 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
     /*
      * A7MPCORE
      */
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-cpu", 1, &error_abort);
+    object_property_set_int(OBJECT(&s->a7mpcore), "num-cores", 1, &error_abort);
     object_property_set_int(OBJECT(&s->a7mpcore), "num-irq",
                             FSL_IMX6UL_MAX_IRQ + GIC_INTERNAL, &error_abort);
     sysbus_realize(SYS_BUS_DEVICE(&s->a7mpcore), &error_abort);
diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
index a283dcb45f..8a3e9933c4 100644
--- a/hw/arm/fsl-imx7.c
+++ b/hw/arm/fsl-imx7.c
@@ -203,7 +203,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
     /*
      * A7MPCORE
      */
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-cpu", smp_cpus,
+    object_property_set_int(OBJECT(&s->a7mpcore), "num-cores", smp_cpus,
                             &error_abort);
     object_property_set_int(OBJECT(&s->a7mpcore), "num-irq",
                             FSL_IMX7_MAX_IRQ + GIC_INTERNAL, &error_abort);
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index 15ff21d047..5b6e968fa9 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -491,7 +491,7 @@ static void npcm7xx_realize(DeviceState *dev, Error **errp)
     }
 
     /* A9MPCORE peripherals. Can only fail if we pass bad parameters here. */
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-cpu", nc->num_cpus,
+    object_property_set_int(OBJECT(&s->a9mpcore), "num-cores", nc->num_cpus,
                             &error_abort);
     object_property_set_int(OBJECT(&s->a9mpcore), "num-irq", NPCM7XX_NUM_IRQ,
                             &error_abort);
diff --git a/hw/arm/realview.c b/hw/arm/realview.c
index a112a6b529..bb6bab12c8 100644
--- a/hw/arm/realview.c
+++ b/hw/arm/realview.c
@@ -187,7 +187,7 @@ static void realview_init(MachineState *machine,
 
     if (is_mpcore) {
         dev = qdev_new(is_pb ? TYPE_A9MPCORE_PRIV : "realview_mpcore");
-        qdev_prop_set_uint32(dev, "num-cpu", smp_cpus);
+        qdev_prop_set_uint32(dev, "num-cores", smp_cpus);
         busdev = SYS_BUS_DEVICE(dev);
         sysbus_realize_and_unref(busdev, &error_fatal);
         sysbus_mmio_map(busdev, 0, periphbase);
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 128941eb50..16874426e1 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -63,7 +63,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     }
 
     gicdev = DEVICE(&s->gic);
-    qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
     qdev_prop_set_uint32(gicdev, "num-irq", s->num_irq);
 
     if (!kvm_irqchip_in_kernel()) {
@@ -94,7 +94,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     /* Wire the outputs from each CPU's generic timer to the
      * appropriate GIC PPI inputs
      */
-    for (i = 0; i < s->num_cpu; i++) {
+    for (i = 0; i < c->num_cores; i++) {
         DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
         int ppibase = s->num_irq - 32 + i * 32;
         int irq;
@@ -114,7 +114,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
         }
         if (has_el2) {
             /* Connect the GIC maintenance interrupt to PPI ID 25 */
-            sysbus_connect_irq(SYS_BUS_DEVICE(gicdev), i + 4 * s->num_cpu,
+            sysbus_connect_irq(SYS_BUS_DEVICE(gicdev), i + 4 * c->num_cores,
                                qdev_get_gpio_in(gicdev, ppibase + 25));
         }
     }
@@ -139,17 +139,16 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
                                     sysbus_mmio_get_region(gicsbd, 2));
         memory_region_add_subregion(&c->container, 0x6000,
                                     sysbus_mmio_get_region(gicsbd, 3));
-        for (i = 0; i < s->num_cpu; i++) {
+        for (i = 0; i < c->num_cores; i++) {
             hwaddr base = 0x5000 + i * 0x200;
             MemoryRegion *mr = sysbus_mmio_get_region(gicsbd,
-                                                      4 + s->num_cpu + i);
+                                                      4 + c->num_cores + i);
             memory_region_add_subregion(&c->container, base, mr);
         }
     }
 }
 
 static Property a15mp_priv_properties[] = {
-    DEFINE_PROP_UINT32("num-cpu", A15MPPrivState, num_cpu, 1),
     /* The Cortex-A15MP may have anything from 0 to 224 external interrupt
      * IRQ lines (with another 32 internal). We default to 128+32, which
      * is the number provided by the Cortex-A15MP test chip in the
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index 08346b0049..9c138f4442 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -51,7 +51,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
                  *wdtbusdev;
     Error *local_err = NULL;
-    int i;
     bool has_el3;
     CPUState *cpu0;
     Object *cpuobj;
@@ -72,14 +71,14 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     }
 
     scudev = DEVICE(&s->scu);
-    qdev_prop_set_uint32(scudev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(scudev, "num-cpu", c->num_cores);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->scu), errp)) {
         return;
     }
     scubusdev = SYS_BUS_DEVICE(&s->scu);
 
     gicdev = DEVICE(&s->gic);
-    qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
     qdev_prop_set_uint32(gicdev, "num-irq", s->num_irq);
     qdev_prop_set_uint32(gicdev, "num-priority-bits",
                          A9_GIC_NUM_PRIORITY_BITS);
@@ -103,21 +102,21 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     qdev_init_gpio_in(dev, a9mp_priv_set_irq, s->num_irq - 32);
 
     gtimerdev = DEVICE(&s->gtimer);
-    qdev_prop_set_uint32(gtimerdev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(gtimerdev, "num-cpu", c->num_cores);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->gtimer), errp)) {
         return;
     }
     gtimerbusdev = SYS_BUS_DEVICE(&s->gtimer);
 
     mptimerdev = DEVICE(&s->mptimer);
-    qdev_prop_set_uint32(mptimerdev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(mptimerdev, "num-cpu", c->num_cores);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->mptimer), errp)) {
         return;
     }
     mptimerbusdev = SYS_BUS_DEVICE(&s->mptimer);
 
     wdtdev = DEVICE(&s->wdt);
-    qdev_prop_set_uint32(wdtdev, "num-cpu", s->num_cpu);
+    qdev_prop_set_uint32(wdtdev, "num-cpu", c->num_cores);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->wdt), errp)) {
         return;
     }
@@ -153,7 +152,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
      * For each core the global timer is PPI 27, the private
      * timer is PPI 29 and the watchdog PPI 30.
      */
-    for (i = 0; i < s->num_cpu; i++) {
+    for (unsigned i = 0; i < c->num_cores; i++) {
         int ppibase = (s->num_irq - 32) + i * 32;
         sysbus_connect_irq(gtimerbusdev, i,
                            qdev_get_gpio_in(gicdev, ppibase + 27));
@@ -165,7 +164,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
 }
 
 static Property a9mp_priv_properties[] = {
-    DEFINE_PROP_UINT32("num-cpu", A9MPPrivState, num_cpu, 1),
     /* The Cortex-A9MP may have anything from 0 to 224 external interrupt
      * IRQ lines (with another 32 internal). We default to 64+32, which
      * is the number provided by the Cortex-A9MP test chip in the
diff --git a/hw/cpu/arm11mpcore.c b/hw/cpu/arm11mpcore.c
index 74f44434de..67c96a3a7c 100644
--- a/hw/cpu/arm11mpcore.c
+++ b/hw/cpu/arm11mpcore.c
@@ -132,7 +132,8 @@ static void mpcore_priv_initfn(Object *obj)
 }
 
 static Property mpcore_priv_properties[] = {
-    DEFINE_PROP_UINT32("num-cpu", ARM11MPCorePriveState, num_cpu, 1),
+    DEFINE_PROP_UINT32("num-cores", ARM11MPCorePriveState, num_cpu, 1),
+    DEFINE_PROP_UINT32("num-cpu", ARM11MPCorePriveState, num_cpu, 1), /* alias */
     /* The ARM11 MPCORE TRM says the on-chip controller may have
      * anything from 0 to 224 external interrupt IRQ lines (with another
      * 32 internal). We default to 32+32, which is the number provided by
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index 0c12f99610..d7ea633e4e 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -8,6 +8,7 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
+#include "hw/qdev-properties.h"
 #include "hw/cpu/cortex_mpcore.h"
 
 static void cortex_mpcore_priv_instance_init(Object *obj)
@@ -22,6 +23,21 @@ static void cortex_mpcore_priv_instance_init(Object *obj)
     sysbus_init_mmio(sbd, &s->container);
 }
 
+static Property cortex_mpcore_priv_properties[] = {
+    DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
+    DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
+
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void cortex_mpcore_priv_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    device_class_set_props(dc, cortex_mpcore_priv_properties);
+    /* We currently have no saveable state */
+}
+
 static const TypeInfo cortex_mpcore_types[] = {
     {
         .name           = TYPE_CORTEX_MPCORE_PRIV,
@@ -29,6 +45,7 @@ static const TypeInfo cortex_mpcore_types[] = {
         .instance_size  = sizeof(CortexMPPrivState),
         .instance_init  = cortex_mpcore_priv_instance_init,
         .class_size     = sizeof(CortexMPPrivClass),
+        .class_init     = cortex_mpcore_priv_class_init,
         .abstract       = true,
     },
 };
diff --git a/hw/cpu/realview_mpcore.c b/hw/cpu/realview_mpcore.c
index b140888618..e985d230e0 100644
--- a/hw/cpu/realview_mpcore.c
+++ b/hw/cpu/realview_mpcore.c
@@ -98,7 +98,8 @@ static void mpcore_rirq_init(Object *obj)
     int i;
 
     object_initialize_child(obj, "a11priv", &s->priv, TYPE_ARM11MPCORE_PRIV);
-    object_property_add_alias(obj, "num-cpu", OBJECT(&s->priv), "num-cpu");
+    object_property_add_alias(obj, "num-cores", OBJECT(&s->priv), "num-cores");
+    object_property_add_alias(obj, "num-cpu", OBJECT(&s->priv), "num-cores");
     privbusdev = SYS_BUS_DEVICE(&s->priv);
     sysbus_init_mmio(sbd, sysbus_mmio_get_region(privbusdev, 0));
 
-- 
2.41.0

[PATCH 14/33] hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent

[Philippe Mathieu-Daudé]

Move the 'has_el2' and 'has_el3' properties to the abstract
QOM parent.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h |  5 +++++
 hw/arm/exynos4210.c            | 10 ++++++++--
 hw/arm/vexpress.c              |  6 ++++++
 hw/arm/xilinx_zynq.c           |  6 ++++++
 hw/cpu/a15mpcore.c             | 18 ++++++------------
 hw/cpu/a9mpcore.c              |  5 +----
 hw/cpu/cortex_mpcore.c         |  3 +++
 7 files changed, 35 insertions(+), 18 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 0e7cca9e93..387552468c 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -30,6 +30,8 @@
  * QEMU interface:
  *  + QOM property "num-cores" which set the number of cores present in
  *    the cluster.
+ *  + QOM properties "cpu-has-el3", "cpu-has-el2" which set whether the CPUs
+ *    have the exception level features present.
  */
 #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
 OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
@@ -53,6 +55,9 @@ struct CortexMPPrivState {
 
     /* Properties */
     uint32_t num_cores;
+
+    bool cpu_has_el3;
+    bool cpu_has_el2;
 };
 
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
diff --git a/hw/arm/exynos4210.c b/hw/arm/exynos4210.c
index ea1364499d..7386a8fe57 100644
--- a/hw/arm/exynos4210.c
+++ b/hw/arm/exynos4210.c
@@ -548,7 +548,7 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
     Exynos4210State *s = EXYNOS4210_SOC(socdev);
     MemoryRegion *system_mem = get_system_memory();
     SysBusDevice *busdev;
-    DeviceState *dev, *uart[4], *pl330[3];
+    DeviceState *dev, *mpdev, *uart[4], *pl330[3];
     int i, n;
 
     for (n = 0; n < EXYNOS4210_NCPUS; n++) {
@@ -582,7 +582,13 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
     }
 
     /* Private memory region and Internal GIC */
-    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cores", EXYNOS4210_NCPUS);
+    mpdev = DEVICE(&s->a9mpcore);
+    qdev_prop_set_uint32(mpdev, "num-cores", EXYNOS4210_NCPUS);
+    /*
+     * By default A9 CPUs have EL3 enabled.  This board does not currently
+     * support EL3 so the CPU EL3 property is disabled before realization.
+     */
+    qdev_prop_set_bit(mpdev, "cpu-has-el3", false);
     busdev = SYS_BUS_DEVICE(&s->a9mpcore);
     sysbus_realize(busdev, &error_fatal);
     sysbus_mmio_map(busdev, 0, EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index a320d1c181..294b6f15f2 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -239,6 +239,12 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
      * wires itself up to the CPU's generic_timer gpio out lines.
      */
     dev = qdev_new(privdev);
+    if (!secure) {
+        qdev_prop_set_bit(dev, "cpu-has-el3", false);
+    }
+    if (!virt) {
+        qdev_prop_set_bit(dev, "cpu-has-el2", false);
+    }
     qdev_prop_set_uint32(dev, "num-cpu", smp_cpus);
     busdev = SYS_BUS_DEVICE(dev);
     sysbus_realize_and_unref(busdev, &error_fatal);
diff --git a/hw/arm/xilinx_zynq.c b/hw/arm/xilinx_zynq.c
index fc9c927d09..28430dcfba 100644
--- a/hw/arm/xilinx_zynq.c
+++ b/hw/arm/xilinx_zynq.c
@@ -240,6 +240,12 @@ static void zynq_init(MachineState *machine)
 
     dev = qdev_new(TYPE_A9MPCORE_PRIV);
     qdev_prop_set_uint32(dev, "num-cpu", 1);
+    /*
+     * By default A9 CPUs have EL3 enabled.  This board does not
+     * currently support EL3 so the CPU EL3 property is disabled before
+     * realization.
+     */
+    qdev_prop_set_bit(dev, "cpu-has-el3", false);
     busdev = SYS_BUS_DEVICE(dev);
     sysbus_realize_and_unref(busdev, &error_fatal);
     sysbus_mmio_map(busdev, 0, MPCORE_PERIPHBASE);
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 16874426e1..a16544fdde 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -52,9 +52,6 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     SysBusDevice *gicsbd;
     Error *local_err = NULL;
     int i;
-    bool has_el3;
-    bool has_el2 = false;
-    Object *cpuobj;
 
     cc->parent_realize(dev, &local_err);
     if (local_err) {
@@ -70,14 +67,11 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
         /* Make the GIC's TZ support match the CPUs. We assume that
          * either all the CPUs have TZ, or none do.
          */
-        cpuobj = OBJECT(qemu_get_cpu(0));
-        has_el3 = object_property_find(cpuobj, "has_el3") &&
-            object_property_get_bool(cpuobj, "has_el3", &error_abort);
-        qdev_prop_set_bit(gicdev, "has-security-extensions", has_el3);
+        qdev_prop_set_bit(gicdev, "has-security-extensions",
+                          c->cpu_has_el3);
         /* Similarly for virtualization support */
-        has_el2 = object_property_find(cpuobj, "has_el2") &&
-            object_property_get_bool(cpuobj, "has_el2", &error_abort);
-        qdev_prop_set_bit(gicdev, "has-virtualization-extensions", has_el2);
+        qdev_prop_set_bit(gicdev, "has-virtualization-extensions",
+                          c->cpu_has_el2);
     }
 
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
@@ -112,7 +106,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
                                   qdev_get_gpio_in(gicdev,
                                                    ppibase + timer_irq[irq]));
         }
-        if (has_el2) {
+        if (c->cpu_has_el2) {
             /* Connect the GIC maintenance interrupt to PPI ID 25 */
             sysbus_connect_irq(SYS_BUS_DEVICE(gicdev), i + 4 * c->num_cores,
                                qdev_get_gpio_in(gicdev, ppibase + 25));
@@ -134,7 +128,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
                                 sysbus_mmio_get_region(gicsbd, 0));
     memory_region_add_subregion(&c->container, 0x2000,
                                 sysbus_mmio_get_region(gicsbd, 1));
-    if (has_el2) {
+    if (c->cpu_has_el2) {
         memory_region_add_subregion(&c->container, 0x4000,
                                     sysbus_mmio_get_region(gicsbd, 2));
         memory_region_add_subregion(&c->container, 0x6000,
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index 9c138f4442..54949314f8 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -51,7 +51,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
                  *wdtbusdev;
     Error *local_err = NULL;
-    bool has_el3;
     CPUState *cpu0;
     Object *cpuobj;
 
@@ -86,9 +85,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     /* Make the GIC's TZ support match the CPUs. We assume that
      * either all the CPUs have TZ, or none do.
      */
-    has_el3 = object_property_find(cpuobj, "has_el3") &&
-        object_property_get_bool(cpuobj, "has_el3", &error_abort);
-    qdev_prop_set_bit(gicdev, "has-security-extensions", has_el3);
+    qdev_prop_set_bit(gicdev, "has-security-extensions", c->cpu_has_el3);
 
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
         return;
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index d7ea633e4e..549b81f708 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -27,6 +27,9 @@ static Property cortex_mpcore_priv_properties[] = {
     DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
     DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
 
+    DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
+    DEFINE_PROP_BOOL("cpu-has-el2", CortexMPPrivState, cpu_has_el2, false),
+
     DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.41.0

Re: [PATCH 14/33] hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent

[Fabiano Rosas]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> Move the 'has_el2' and 'has_el3' properties to the abstract
> QOM parent.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>  include/hw/cpu/cortex_mpcore.h |  5 +++++
>  hw/arm/exynos4210.c            | 10 ++++++++--
>  hw/arm/vexpress.c              |  6 ++++++
>  hw/arm/xilinx_zynq.c           |  6 ++++++
>  hw/cpu/a15mpcore.c             | 18 ++++++------------
>  hw/cpu/a9mpcore.c              |  5 +----
>  hw/cpu/cortex_mpcore.c         |  3 +++
>  7 files changed, 35 insertions(+), 18 deletions(-)
>
> diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
> index 0e7cca9e93..387552468c 100644
> --- a/include/hw/cpu/cortex_mpcore.h
> +++ b/include/hw/cpu/cortex_mpcore.h
> @@ -30,6 +30,8 @@
>   * QEMU interface:
>   *  + QOM property "num-cores" which set the number of cores present in
>   *    the cluster.
> + *  + QOM properties "cpu-has-el3", "cpu-has-el2" which set whether the CPUs
> + *    have the exception level features present.
>   */
>  #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
>  OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
> @@ -53,6 +55,9 @@ struct CortexMPPrivState {
>  
>      /* Properties */
>      uint32_t num_cores;
> +
> +    bool cpu_has_el3;
> +    bool cpu_has_el2;
>  };
>  
>  #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
> diff --git a/hw/arm/exynos4210.c b/hw/arm/exynos4210.c
> index ea1364499d..7386a8fe57 100644
> --- a/hw/arm/exynos4210.c
> +++ b/hw/arm/exynos4210.c
> @@ -548,7 +548,7 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
>      Exynos4210State *s = EXYNOS4210_SOC(socdev);
>      MemoryRegion *system_mem = get_system_memory();
>      SysBusDevice *busdev;
> -    DeviceState *dev, *uart[4], *pl330[3];
> +    DeviceState *dev, *mpdev, *uart[4], *pl330[3];
>      int i, n;
>  
>      for (n = 0; n < EXYNOS4210_NCPUS; n++) {
> @@ -582,7 +582,13 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
>      }
>  
>      /* Private memory region and Internal GIC */
> -    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cores", EXYNOS4210_NCPUS);
> +    mpdev = DEVICE(&s->a9mpcore);
> +    qdev_prop_set_uint32(mpdev, "num-cores", EXYNOS4210_NCPUS);
> +    /*
> +     * By default A9 CPUs have EL3 enabled.  This board does not currently
> +     * support EL3 so the CPU EL3 property is disabled before realization.
> +     */
> +    qdev_prop_set_bit(mpdev, "cpu-has-el3", false);
>      busdev = SYS_BUS_DEVICE(&s->a9mpcore);
>      sysbus_realize(busdev, &error_fatal);
>      sysbus_mmio_map(busdev, 0, EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
> diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
> index a320d1c181..294b6f15f2 100644
> --- a/hw/arm/vexpress.c
> +++ b/hw/arm/vexpress.c
> @@ -239,6 +239,12 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
>       * wires itself up to the CPU's generic_timer gpio out lines.
>       */
>      dev = qdev_new(privdev);
> +    if (!secure) {
> +        qdev_prop_set_bit(dev, "cpu-has-el3", false);
> +    }
> +    if (!virt) {
> +        qdev_prop_set_bit(dev, "cpu-has-el2", false);
> +    }
>      qdev_prop_set_uint32(dev, "num-cpu", smp_cpus);
>      busdev = SYS_BUS_DEVICE(dev);
>      sysbus_realize_and_unref(busdev, &error_fatal);
> diff --git a/hw/arm/xilinx_zynq.c b/hw/arm/xilinx_zynq.c
> index fc9c927d09..28430dcfba 100644
> --- a/hw/arm/xilinx_zynq.c
> +++ b/hw/arm/xilinx_zynq.c
> @@ -240,6 +240,12 @@ static void zynq_init(MachineState *machine)
>  
>      dev = qdev_new(TYPE_A9MPCORE_PRIV);
>      qdev_prop_set_uint32(dev, "num-cpu", 1);
> +    /*
> +     * By default A9 CPUs have EL3 enabled.  This board does not
> +     * currently support EL3 so the CPU EL3 property is disabled before
> +     * realization.
> +     */
> +    qdev_prop_set_bit(dev, "cpu-has-el3", false);
>      busdev = SYS_BUS_DEVICE(dev);
>      sysbus_realize_and_unref(busdev, &error_fatal);
>      sysbus_mmio_map(busdev, 0, MPCORE_PERIPHBASE);
> diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
> index 16874426e1..a16544fdde 100644
> --- a/hw/cpu/a15mpcore.c
> +++ b/hw/cpu/a15mpcore.c
> @@ -52,9 +52,6 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
>      SysBusDevice *gicsbd;
>      Error *local_err = NULL;
>      int i;
> -    bool has_el3;
> -    bool has_el2 = false;
> -    Object *cpuobj;
>  
>      cc->parent_realize(dev, &local_err);
>      if (local_err) {
> @@ -70,14 +67,11 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
>          /* Make the GIC's TZ support match the CPUs. We assume that
>           * either all the CPUs have TZ, or none do.
>           */
> -        cpuobj = OBJECT(qemu_get_cpu(0));
> -        has_el3 = object_property_find(cpuobj, "has_el3") &&
> -            object_property_get_bool(cpuobj, "has_el3", &error_abort);
> -        qdev_prop_set_bit(gicdev, "has-security-extensions", has_el3);
> +        qdev_prop_set_bit(gicdev, "has-security-extensions",
> +                          c->cpu_has_el3);
>          /* Similarly for virtualization support */
> -        has_el2 = object_property_find(cpuobj, "has_el2") &&
> -            object_property_get_bool(cpuobj, "has_el2", &error_abort);
> -        qdev_prop_set_bit(gicdev, "has-virtualization-extensions", has_el2);
> +        qdev_prop_set_bit(gicdev, "has-virtualization-extensions",
> +                          c->cpu_has_el2);
>      }
>  
>      if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
> @@ -112,7 +106,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
>                                    qdev_get_gpio_in(gicdev,
>                                                     ppibase + timer_irq[irq]));
>          }
> -        if (has_el2) {
> +        if (c->cpu_has_el2) {
>              /* Connect the GIC maintenance interrupt to PPI ID 25 */
>              sysbus_connect_irq(SYS_BUS_DEVICE(gicdev), i + 4 * c->num_cores,
>                                 qdev_get_gpio_in(gicdev, ppibase + 25));
> @@ -134,7 +128,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
>                                  sysbus_mmio_get_region(gicsbd, 0));
>      memory_region_add_subregion(&c->container, 0x2000,
>                                  sysbus_mmio_get_region(gicsbd, 1));
> -    if (has_el2) {
> +    if (c->cpu_has_el2) {
>          memory_region_add_subregion(&c->container, 0x4000,
>                                      sysbus_mmio_get_region(gicsbd, 2));
>          memory_region_add_subregion(&c->container, 0x6000,
> diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
> index 9c138f4442..54949314f8 100644
> --- a/hw/cpu/a9mpcore.c
> +++ b/hw/cpu/a9mpcore.c
> @@ -51,7 +51,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
>      SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
>                   *wdtbusdev;
>      Error *local_err = NULL;
> -    bool has_el3;
>      CPUState *cpu0;
>      Object *cpuobj;
>  
> @@ -86,9 +85,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
>      /* Make the GIC's TZ support match the CPUs. We assume that
>       * either all the CPUs have TZ, or none do.
>       */
> -    has_el3 = object_property_find(cpuobj, "has_el3") &&
> -        object_property_get_bool(cpuobj, "has_el3", &error_abort);
> -    qdev_prop_set_bit(gicdev, "has-security-extensions", has_el3);
> +    qdev_prop_set_bit(gicdev, "has-security-extensions", c->cpu_has_el3);
>  
>      if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
>          return;
> diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
> index d7ea633e4e..549b81f708 100644
> --- a/hw/cpu/cortex_mpcore.c
> +++ b/hw/cpu/cortex_mpcore.c
> @@ -27,6 +27,9 @@ static Property cortex_mpcore_priv_properties[] = {
>      DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
>      DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
>  
> +    DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
> +    DEFINE_PROP_BOOL("cpu-has-el2", CortexMPPrivState, cpu_has_el2, false),

Are we missing setting cpu_has_el2 somewhere else? This^ results in fewer
cpregs being registered and is what breaks migration.

You can test with:

$ (echo "migrate file:migfile"; echo "quit") | ./qemu-system-arm -M ast2600-evb -monitor stdio
$ ./scripts/analyze-migration.py -f migfile | grep array_len

Before series:
$ ./scripts/analyze-migration.py -f migfile | grep array_len
        "cpreg_vmstate_array_len": "0x0000010a",
        "cpreg_vmstate_array_len": "0x0000010a",

After series:
$ ./scripts/analyze-migration.py -f migfile | grep array_len
        "cpreg_vmstate_array_len": "0x000000df",
        "cpreg_vmstate_array_len": "0x000000df",

Re: [PATCH 14/33] hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent

[Philippe Mathieu-Daudé]

On 12/1/24 22:33, Fabiano Rosas wrote:
> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> 
>> Move the 'has_el2' and 'has_el3' properties to the abstract
>> QOM parent.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> ---
>>   include/hw/cpu/cortex_mpcore.h |  5 +++++
>>   hw/arm/exynos4210.c            | 10 ++++++++--
>>   hw/arm/vexpress.c              |  6 ++++++
>>   hw/arm/xilinx_zynq.c           |  6 ++++++
>>   hw/cpu/a15mpcore.c             | 18 ++++++------------
>>   hw/cpu/a9mpcore.c              |  5 +----
>>   hw/cpu/cortex_mpcore.c         |  3 +++
>>   7 files changed, 35 insertions(+), 18 deletions(-)
>>
>> diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
>> index 0e7cca9e93..387552468c 100644
>> --- a/include/hw/cpu/cortex_mpcore.h
>> +++ b/include/hw/cpu/cortex_mpcore.h
>> @@ -30,6 +30,8 @@
>>    * QEMU interface:
>>    *  + QOM property "num-cores" which set the number of cores present in
>>    *    the cluster.
>> + *  + QOM properties "cpu-has-el3", "cpu-has-el2" which set whether the CPUs
>> + *    have the exception level features present.
>>    */
>>   #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
>>   OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
>> @@ -53,6 +55,9 @@ struct CortexMPPrivState {
>>   
>>       /* Properties */
>>       uint32_t num_cores;
>> +
>> +    bool cpu_has_el3;
>> +    bool cpu_has_el2;
>>   };


>> diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
>> index 9c138f4442..54949314f8 100644
>> --- a/hw/cpu/a9mpcore.c
>> +++ b/hw/cpu/a9mpcore.c
>> @@ -51,7 +51,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
>>       SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
>>                    *wdtbusdev;
>>       Error *local_err = NULL;
>> -    bool has_el3;
>>       CPUState *cpu0;
>>       Object *cpuobj;
>>   
>> @@ -86,9 +85,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
>>       /* Make the GIC's TZ support match the CPUs. We assume that
>>        * either all the CPUs have TZ, or none do.
>>        */
>> -    has_el3 = object_property_find(cpuobj, "has_el3") &&
>> -        object_property_get_bool(cpuobj, "has_el3", &error_abort);
>> -    qdev_prop_set_bit(gicdev, "has-security-extensions", has_el3);
>> +    qdev_prop_set_bit(gicdev, "has-security-extensions", c->cpu_has_el3);
>>   
>>       if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
>>           return;
>> diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
>> index d7ea633e4e..549b81f708 100644
>> --- a/hw/cpu/cortex_mpcore.c
>> +++ b/hw/cpu/cortex_mpcore.c
>> @@ -27,6 +27,9 @@ static Property cortex_mpcore_priv_properties[] = {
>>       DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
>>       DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
>>   
>> +    DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
>> +    DEFINE_PROP_BOOL("cpu-has-el2", CortexMPPrivState, cpu_has_el2, false),
> 
> Are we missing setting cpu_has_el2 somewhere else? This^ results in fewer
> cpregs being registered and is what breaks migration.
> 
> You can test with:
> 
> $ (echo "migrate file:migfile"; echo "quit") | ./qemu-system-arm -M ast2600-evb -monitor stdio
> $ ./scripts/analyze-migration.py -f migfile | grep array_len
> 
> Before series:
> $ ./scripts/analyze-migration.py -f migfile | grep array_len
>          "cpreg_vmstate_array_len": "0x0000010a",
>          "cpreg_vmstate_array_len": "0x0000010a",
> 
> After series:
> $ ./scripts/analyze-migration.py -f migfile | grep array_len
>          "cpreg_vmstate_array_len": "0x000000df",
>          "cpreg_vmstate_array_len": "0x000000df",

Thank you Fabiano for helping debugging. Indeed there is a problem
with properties, so this patch is bogus.

The good news is the QOM reparenting happened 2 commits earlier,
so this discarded the doubts on qom-path change possibly affecting
migration :)

Regards,

Phil.

[PATCH 15/33] hw/cpu/arm: Handle 'gic-irq' property once in MPCore parent

[Philippe Mathieu-Daudé]

Move the 'num-irq' property to the abstract QOM parent.
Rename it as 'gic-irq', keeping it aliased as 'num-irq'.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 13 +++++++++----
 hw/cpu/a15mpcore.c             | 28 +++++++++++++---------------
 hw/cpu/a9mpcore.c              | 28 +++++++++++++---------------
 hw/cpu/cortex_mpcore.c         |  5 +++++
 4 files changed, 40 insertions(+), 34 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 387552468c..96da713be9 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -32,6 +32,8 @@
  *    the cluster.
  *  + QOM properties "cpu-has-el3", "cpu-has-el2" which set whether the CPUs
  *    have the exception level features present.
+ *  + QOM property "gic-spi-num" sets the number of GIC Shared Peripheral
+ *    Interrupts.
  */
 #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
 OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
@@ -39,6 +41,8 @@ OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
 /**
  * CortexMPPrivClass:
  * @container_size - size of the device's MMIO region
+ * @gic_spi_default - default number of GIC SPIs
+ * @gic_spi_max - maximum number of GIC SPIs
  */
 struct CortexMPPrivClass {
     SysBusDeviceClass parent_class;
@@ -46,6 +50,9 @@ struct CortexMPPrivClass {
     DeviceRealize parent_realize;
 
     uint64_t container_size;
+
+    unsigned gic_spi_default;
+    unsigned gic_spi_max;
 };
 
 struct CortexMPPrivState {
@@ -58,6 +65,8 @@ struct CortexMPPrivState {
 
     bool cpu_has_el3;
     bool cpu_has_el2;
+
+    uint32_t gic_spi_num;
 };
 
 #define TYPE_A9MPCORE_PRIV "a9mpcore_priv"
@@ -66,8 +75,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(A9MPPrivState, A9MPCORE_PRIV)
 struct A9MPPrivState {
     CortexMPPrivState parent_obj;
 
-    uint32_t num_irq;
-
     A9SCUState scu;
     GICState gic;
     A9GTimerState gtimer;
@@ -81,8 +88,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
 struct A15MPPrivState {
     CortexMPPrivState parent_obj;
 
-    uint32_t num_irq;
-
     GICState gic;
 };
 
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index a16544fdde..195355b589 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -61,7 +61,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
 
     gicdev = DEVICE(&s->gic);
     qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
-    qdev_prop_set_uint32(gicdev, "num-irq", s->num_irq);
+    qdev_prop_set_uint32(gicdev, "num-irq", c->gic_spi_num);
 
     if (!kvm_irqchip_in_kernel()) {
         /* Make the GIC's TZ support match the CPUs. We assume that
@@ -83,14 +83,14 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     sysbus_pass_irq(sbd, gicsbd);
 
     /* Pass through inbound GPIO lines to the GIC */
-    qdev_init_gpio_in(dev, a15mp_priv_set_irq, s->num_irq - 32);
+    qdev_init_gpio_in(dev, a15mp_priv_set_irq, c->gic_spi_num - 32);
 
     /* Wire the outputs from each CPU's generic timer to the
      * appropriate GIC PPI inputs
      */
     for (i = 0; i < c->num_cores; i++) {
         DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
-        int ppibase = s->num_irq - 32 + i * 32;
+        int ppibase = c->gic_spi_num - 32 + i * 32;
         int irq;
         /* Mapping from the output timer irq lines from the CPU to the
          * GIC PPI inputs used on the A15:
@@ -142,17 +142,6 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
     }
 }
 
-static Property a15mp_priv_properties[] = {
-    /* The Cortex-A15MP may have anything from 0 to 224 external interrupt
-     * IRQ lines (with another 32 internal). We default to 128+32, which
-     * is the number provided by the Cortex-A15MP test chip in the
-     * Versatile Express A15 development board.
-     * Other boards may differ and should set this property appropriately.
-     */
-    DEFINE_PROP_UINT32("num-irq", A15MPPrivState, num_irq, 160),
-    DEFINE_PROP_END_OF_LIST(),
-};
-
 static void a15mp_priv_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -160,9 +149,18 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
 
     cc->container_size = 0x8000;
 
+    /*
+     * The Cortex-A15MP may have anything from 0 to 224 external interrupt
+     * IRQ lines (with another 32 internal). We default to 128+32, which
+     * is the number provided by the Cortex-A15MP test chip in the
+     * Versatile Express A15 development board.
+     * Other boards may differ and should set this property appropriately.
+     */
+    cc->gic_spi_default = 160;
+    cc->gic_spi_max = 224;
+
     device_class_set_parent_realize(dc, a15mp_priv_realize,
                                     &cc->parent_realize);
-    device_class_set_props(dc, a15mp_priv_properties);
     /* We currently have no saveable state */
 }
 
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index 54949314f8..72621bce64 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -78,7 +78,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
 
     gicdev = DEVICE(&s->gic);
     qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
-    qdev_prop_set_uint32(gicdev, "num-irq", s->num_irq);
+    qdev_prop_set_uint32(gicdev, "num-irq", c->gic_spi_num);
     qdev_prop_set_uint32(gicdev, "num-priority-bits",
                          A9_GIC_NUM_PRIORITY_BITS);
 
@@ -96,7 +96,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     sysbus_pass_irq(sbd, gicbusdev);
 
     /* Pass through inbound GPIO lines to the GIC */
-    qdev_init_gpio_in(dev, a9mp_priv_set_irq, s->num_irq - 32);
+    qdev_init_gpio_in(dev, a9mp_priv_set_irq, c->gic_spi_num - 32);
 
     gtimerdev = DEVICE(&s->gtimer);
     qdev_prop_set_uint32(gtimerdev, "num-cpu", c->num_cores);
@@ -150,7 +150,7 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
      * timer is PPI 29 and the watchdog PPI 30.
      */
     for (unsigned i = 0; i < c->num_cores; i++) {
-        int ppibase = (s->num_irq - 32) + i * 32;
+        int ppibase = (c->gic_spi_num - 32) + i * 32;
         sysbus_connect_irq(gtimerbusdev, i,
                            qdev_get_gpio_in(gicdev, ppibase + 27));
         sysbus_connect_irq(mptimerbusdev, i,
@@ -160,17 +160,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     }
 }
 
-static Property a9mp_priv_properties[] = {
-    /* The Cortex-A9MP may have anything from 0 to 224 external interrupt
-     * IRQ lines (with another 32 internal). We default to 64+32, which
-     * is the number provided by the Cortex-A9MP test chip in the
-     * Realview PBX-A9 and Versatile Express A9 development boards.
-     * Other boards may differ and should set this property appropriately.
-     */
-    DEFINE_PROP_UINT32("num-irq", A9MPPrivState, num_irq, 96),
-    DEFINE_PROP_END_OF_LIST(),
-};
-
 static void a9mp_priv_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -178,8 +167,17 @@ static void a9mp_priv_class_init(ObjectClass *klass, void *data)
 
     cc->container_size = 0x2000;
 
+    /*
+     * The Cortex-A9MP may have anything from 0 to 224 external interrupt
+     * IRQ lines (with another 32 internal). We default to 64+32, which
+     * is the number provided by the Cortex-A9MP test chip in the
+     * Realview PBX-A9 and Versatile Express A9 development boards.
+     * Other boards may differ and should set this property appropriately.
+     */
+    cc->gic_spi_default = 96;
+    cc->gic_spi_max = 224;
+
     device_class_set_parent_realize(dc, a9mp_priv_realize, &cc->parent_realize);
-    device_class_set_props(dc, a9mp_priv_properties);
 }
 
 static const TypeInfo a9mp_types[] = {
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index 549b81f708..dee40a7bf2 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -21,6 +21,8 @@ static void cortex_mpcore_priv_instance_init(Object *obj)
     memory_region_init(&s->container, obj,
                        "mpcore-priv-container", k->container_size);
     sysbus_init_mmio(sbd, &s->container);
+
+    s->gic_spi_num = k->gic_spi_default;
 }
 
 static Property cortex_mpcore_priv_properties[] = {
@@ -30,6 +32,9 @@ static Property cortex_mpcore_priv_properties[] = {
     DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
     DEFINE_PROP_BOOL("cpu-has-el2", CortexMPPrivState, cpu_has_el2, false),
 
+    DEFINE_PROP_UINT32("gic-spi-num", CortexMPPrivState, gic_spi_num, 0),
+    DEFINE_PROP_UINT32("num-irq", CortexMPPrivState, gic_spi_num, 0), /* alias */
+
     DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.41.0

[PATCH 16/33] hw/cpu/arm: Handle GIC once in MPCore parent

[Philippe Mathieu-Daudé]

Move the GIC instance creation to the abstract QOM parent.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 31 +++++++++++++++++--
 hw/cpu/a15mpcore.c             | 56 ++++------------------------------
 hw/cpu/a9mpcore.c              | 43 +++++---------------------
 hw/cpu/cortex_mpcore.c         | 46 ++++++++++++++++++++++++++++
 hw/arm/Kconfig                 |  3 +-
 5 files changed, 88 insertions(+), 91 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 96da713be9..4697fd47c7 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -34,6 +34,28 @@
  *    have the exception level features present.
  *  + QOM property "gic-spi-num" sets the number of GIC Shared Peripheral
  *    Interrupts.
+ * QEMU interface forwarded from the GIC:
+ *  + unnamed GPIO inputs: (where P is number of GIC SPIs, i.e. num-irq - 32)
+ *    [0..P-1]  GIC SPIs
+ *    [P..P+31] PPIs for CPU 0
+ *    [P+32..P+63] PPIs for CPU 1
+ *    ...
+ *  + sysbus output IRQs: (in order; number will vary depending on number of
+ *    cores)
+ *    - IRQ for CPU 0
+ *    - IRQ for CPU 1
+ *      ...
+ *    - FIQ for CPU 0
+ *    - FIQ for CPU 1
+ *      ...
+ *    - VIRQ for CPU 0 (exists even if virt extensions not present)
+ *    - VIRQ for CPU 1 (exists even if virt extensions not present)
+ *      ...
+ *    - VFIQ for CPU 0 (exists even if virt extensions not present)
+ *    - VFIQ for CPU 1 (exists even if virt extensions not present)
+ *      ...
+ *    - maintenance IRQ for CPU i/f 0 (only if virt extensions present)
+ *    - maintenance IRQ for CPU i/f 1 (only if virt extensions present)
  */
 #define TYPE_CORTEX_MPCORE_PRIV "cortex_mpcore_priv"
 OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
@@ -41,8 +63,10 @@ OBJECT_DECLARE_TYPE(CortexMPPrivState, CortexMPPrivClass, CORTEX_MPCORE_PRIV)
 /**
  * CortexMPPrivClass:
  * @container_size - size of the device's MMIO region
+ * @gic_class_name - GIC QOM class name
  * @gic_spi_default - default number of GIC SPIs
  * @gic_spi_max - maximum number of GIC SPIs
+ * @gic_revision - revision of the GIC
  */
 struct CortexMPPrivClass {
     SysBusDeviceClass parent_class;
@@ -51,14 +75,18 @@ struct CortexMPPrivClass {
 
     uint64_t container_size;
 
+    const char *gic_class_name;
     unsigned gic_spi_default;
     unsigned gic_spi_max;
+    unsigned gic_revision;
+    uint32_t gic_priority_bits;
 };
 
 struct CortexMPPrivState {
     SysBusDevice parent_obj;
 
     MemoryRegion container;
+    GICState gic;
 
     /* Properties */
     uint32_t num_cores;
@@ -76,7 +104,6 @@ struct A9MPPrivState {
     CortexMPPrivState parent_obj;
 
     A9SCUState scu;
-    GICState gic;
     A9GTimerState gtimer;
     ARMMPTimerState mptimer;
     ARMMPTimerState wdt;
@@ -87,8 +114,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
 
 struct A15MPPrivState {
     CortexMPPrivState parent_obj;
-
-    GICState gic;
 };
 
 #endif
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 195355b589..ecd04b7af1 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -22,34 +22,15 @@
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "hw/cpu/cortex_mpcore.h"
-#include "hw/irq.h"
-#include "hw/qdev-properties.h"
-#include "sysemu/kvm.h"
-#include "kvm_arm.h"
-
-static void a15mp_priv_set_irq(void *opaque, int irq, int level)
-{
-    A15MPPrivState *s = (A15MPPrivState *)opaque;
-
-    qemu_set_irq(qdev_get_gpio_in(DEVICE(&s->gic), irq), level);
-}
-
-static void a15mp_priv_initfn(Object *obj)
-{
-    A15MPPrivState *s = A15MPCORE_PRIV(obj);
-
-    object_initialize_child(obj, "gic", &s->gic, gic_class_name());
-    qdev_prop_set_uint32(DEVICE(&s->gic), "revision", 2);
-}
+#include "hw/core/cpu.h"
+#include "target/arm/cpu.h"
 
 static void a15mp_priv_realize(DeviceState *dev, Error **errp)
 {
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
-    SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
-    A15MPPrivState *s = A15MPCORE_PRIV(dev);
     CortexMPPrivState *c = CORTEX_MPCORE_PRIV(dev);
-    DeviceState *gicdev;
-    SysBusDevice *gicsbd;
+    DeviceState *gicdev = DEVICE(&c->gic);
+    SysBusDevice *gicsbd = SYS_BUS_DEVICE(&c->gic);
     Error *local_err = NULL;
     int i;
 
@@ -59,32 +40,6 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    gicdev = DEVICE(&s->gic);
-    qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
-    qdev_prop_set_uint32(gicdev, "num-irq", c->gic_spi_num);
-
-    if (!kvm_irqchip_in_kernel()) {
-        /* Make the GIC's TZ support match the CPUs. We assume that
-         * either all the CPUs have TZ, or none do.
-         */
-        qdev_prop_set_bit(gicdev, "has-security-extensions",
-                          c->cpu_has_el3);
-        /* Similarly for virtualization support */
-        qdev_prop_set_bit(gicdev, "has-virtualization-extensions",
-                          c->cpu_has_el2);
-    }
-
-    if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
-        return;
-    }
-    gicsbd = SYS_BUS_DEVICE(&s->gic);
-
-    /* Pass through outbound IRQ lines from the GIC */
-    sysbus_pass_irq(sbd, gicsbd);
-
-    /* Pass through inbound GPIO lines to the GIC */
-    qdev_init_gpio_in(dev, a15mp_priv_set_irq, c->gic_spi_num - 32);
-
     /* Wire the outputs from each CPU's generic timer to the
      * appropriate GIC PPI inputs
      */
@@ -149,6 +104,8 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
 
     cc->container_size = 0x8000;
 
+    cc->gic_class_name = gic_class_name();
+    cc->gic_revision = 2;
     /*
      * The Cortex-A15MP may have anything from 0 to 224 external interrupt
      * IRQ lines (with another 32 internal). We default to 128+32, which
@@ -169,7 +126,6 @@ static const TypeInfo a15mp_types[] = {
         .name           = TYPE_A15MPCORE_PRIV,
         .parent         = TYPE_CORTEX_MPCORE_PRIV,
         .instance_size  = sizeof(A15MPPrivState),
-        .instance_init  = a15mp_priv_initfn,
         .class_init     = a15mp_priv_class_init,
     },
 };
diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index 72621bce64..ffdaf392fc 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -12,28 +12,18 @@
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "hw/cpu/cortex_mpcore.h"
-#include "hw/irq.h"
 #include "hw/qdev-properties.h"
 #include "hw/core/cpu.h"
 #include "cpu.h"
 
 #define A9_GIC_NUM_PRIORITY_BITS    5
 
-static void a9mp_priv_set_irq(void *opaque, int irq, int level)
-{
-    A9MPPrivState *s = (A9MPPrivState *)opaque;
-
-    qemu_set_irq(qdev_get_gpio_in(DEVICE(&s->gic), irq), level);
-}
-
 static void a9mp_priv_initfn(Object *obj)
 {
     A9MPPrivState *s = A9MPCORE_PRIV(obj);
 
     object_initialize_child(obj, "scu", &s->scu, TYPE_A9_SCU);
 
-    object_initialize_child(obj, "gic", &s->gic, TYPE_ARM_GIC);
-
     object_initialize_child(obj, "gtimer", &s->gtimer, TYPE_A9_GTIMER);
 
     object_initialize_child(obj, "mptimer", &s->mptimer, TYPE_ARM_MPTIMER);
@@ -45,11 +35,11 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
 {
     CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
     CortexMPPrivState *c = CORTEX_MPCORE_PRIV(dev);
-    SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     A9MPPrivState *s = A9MPCORE_PRIV(dev);
-    DeviceState *scudev, *gicdev, *gtimerdev, *mptimerdev, *wdtdev;
-    SysBusDevice *scubusdev, *gicbusdev, *gtimerbusdev, *mptimerbusdev,
-                 *wdtbusdev;
+    DeviceState *gicdev = DEVICE(&c->gic);
+    SysBusDevice *gicbusdev = SYS_BUS_DEVICE(&c->gic);
+    DeviceState *scudev, *gtimerdev, *mptimerdev, *wdtdev;
+    SysBusDevice *scubusdev, *gtimerbusdev, *mptimerbusdev, *wdtbusdev;
     Error *local_err = NULL;
     CPUState *cpu0;
     Object *cpuobj;
@@ -76,28 +66,6 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     }
     scubusdev = SYS_BUS_DEVICE(&s->scu);
 
-    gicdev = DEVICE(&s->gic);
-    qdev_prop_set_uint32(gicdev, "num-cpu", c->num_cores);
-    qdev_prop_set_uint32(gicdev, "num-irq", c->gic_spi_num);
-    qdev_prop_set_uint32(gicdev, "num-priority-bits",
-                         A9_GIC_NUM_PRIORITY_BITS);
-
-    /* Make the GIC's TZ support match the CPUs. We assume that
-     * either all the CPUs have TZ, or none do.
-     */
-    qdev_prop_set_bit(gicdev, "has-security-extensions", c->cpu_has_el3);
-
-    if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
-        return;
-    }
-    gicbusdev = SYS_BUS_DEVICE(&s->gic);
-
-    /* Pass through outbound IRQ lines from the GIC */
-    sysbus_pass_irq(sbd, gicbusdev);
-
-    /* Pass through inbound GPIO lines to the GIC */
-    qdev_init_gpio_in(dev, a9mp_priv_set_irq, c->gic_spi_num - 32);
-
     gtimerdev = DEVICE(&s->gtimer);
     qdev_prop_set_uint32(gtimerdev, "num-cpu", c->num_cores);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->gtimer), errp)) {
@@ -167,6 +135,9 @@ static void a9mp_priv_class_init(ObjectClass *klass, void *data)
 
     cc->container_size = 0x2000;
 
+    cc->gic_class_name = TYPE_ARM_GIC;
+    cc->gic_revision = 1;
+    cc->gic_priority_bits = A9_GIC_NUM_PRIORITY_BITS;
     /*
      * The Cortex-A9MP may have anything from 0 to 224 external interrupt
      * IRQ lines (with another 32 internal). We default to 64+32, which
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index dee40a7bf2..c9ba254818 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -10,6 +10,15 @@
 #include "qapi/error.h"
 #include "hw/qdev-properties.h"
 #include "hw/cpu/cortex_mpcore.h"
+#include "hw/irq.h"
+#include "sysemu/kvm.h"
+
+static void cortex_mpcore_priv_set_irq(void *opaque, int irq, int level)
+{
+    CortexMPPrivState *s = (CortexMPPrivState *)opaque;
+
+    qemu_set_irq(qdev_get_gpio_in(DEVICE(&s->gic), irq), level);
+}
 
 static void cortex_mpcore_priv_instance_init(Object *obj)
 {
@@ -23,6 +32,42 @@ static void cortex_mpcore_priv_instance_init(Object *obj)
     sysbus_init_mmio(sbd, &s->container);
 
     s->gic_spi_num = k->gic_spi_default;
+    object_initialize_child(obj, "gic", &s->gic, k->gic_class_name);
+    qdev_prop_set_uint32(DEVICE(&s->gic), "revision", k->gic_revision);
+}
+
+static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
+{
+    SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
+    CortexMPPrivState *s = CORTEX_MPCORE_PRIV(dev);
+    CortexMPPrivClass *k = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
+    DeviceState *gicdev = DEVICE(&s->gic);
+
+    qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cores);
+    qdev_prop_set_uint32(gicdev, "num-irq", s->gic_spi_num);
+    if (k->gic_priority_bits) {
+        qdev_prop_set_uint32(gicdev, "num-priority-bits", k->gic_priority_bits);
+    }
+    if (!kvm_irqchip_in_kernel()) {
+        /*
+         * Make the GIC's TZ support match the CPUs. We assume that
+         * either all the CPUs have TZ, or none do.
+         */
+        qdev_prop_set_bit(gicdev, "has-security-extensions",
+                          s->cpu_has_el3);
+        /* Similarly for virtualization support */
+        qdev_prop_set_bit(gicdev, "has-virtualization-extensions",
+                          s->cpu_has_el2);
+    }
+    if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
+        return;
+    }
+
+    /* Pass through outbound IRQ lines from the GIC */
+    sysbus_pass_irq(sbd, SYS_BUS_DEVICE(&s->gic));
+
+    /* Pass through inbound GPIO lines to the GIC */
+    qdev_init_gpio_in(dev, cortex_mpcore_priv_set_irq, s->gic_spi_num - 32);
 }
 
 static Property cortex_mpcore_priv_properties[] = {
@@ -42,6 +87,7 @@ static void cortex_mpcore_priv_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->realize = cortex_mpcore_priv_realize;
     device_class_set_props(dc, cortex_mpcore_priv_properties);
     /* We currently have no saveable state */
 }
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index ec6e585797..6c4fa5108b 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -645,19 +645,18 @@ config ZAURUS
 
 config CORTEX_MPCORE
     bool
+    select ARM_GIC
 
 config A9MPCORE
     bool
     select CORTEX_MPCORE
     select A9_GTIMER
     select A9SCU       # snoop control unit
-    select ARM_GIC
     select ARM_MPTIMER
 
 config A15MPCORE
     bool
     select CORTEX_MPCORE
-    select ARM_GIC
 
 config ARM11MPCORE
     bool
-- 
2.41.0

[PATCH 17/33] hw/cpu/arm: Document more properties of CORTEX_MPCORE_PRIV QOM type

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 4697fd47c7..73627bc415 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -28,10 +28,17 @@
  *  some timers and watchdogs
  *
  * QEMU interface:
+ *  + QOM property "cluster-id" which set the cluster ID and its affinity.
  *  + QOM property "num-cores" which set the number of cores present in
  *    the cluster.
+ *  + QOM property "cpu-type" is the CPU model typename.
  *  + QOM properties "cpu-has-el3", "cpu-has-el2" which set whether the CPUs
  *    have the exception level features present.
+ *  + QOM properties "cpu-has-vfp-d32", "cpu-has-neon" which set whether the
+ *    CPUs have the FPU features present.
+ *  + QOM property "cpu-freq-hz" is the frequency of each core
+ *  + QOM property "cpu-memory" is a MemoryRegion containing the devices
+ *    provided by the board model.
  *  + QOM property "gic-spi-num" sets the number of GIC Shared Peripheral
  *    Interrupts.
  * QEMU interface forwarded from the GIC:
-- 
2.41.0

[PATCH 18/33] hw/cpu/arm: Replace A15MPPrivState by CortexMPPrivState

[Philippe Mathieu-Daudé]

A15MPPrivState doesn't contain anything else but its parent,
CortexMPPrivState. Remove it in favor of the parent.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/aspeed_soc.h    | 2 +-
 include/hw/arm/fsl-imx6ul.h    | 2 +-
 include/hw/arm/fsl-imx7.h      | 2 +-
 include/hw/cpu/cortex_mpcore.h | 5 -----
 hw/cpu/a15mpcore.c             | 2 +-
 5 files changed, 4 insertions(+), 9 deletions(-)

diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index e0663ab50d..2f51d78e22 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -108,7 +108,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(Aspeed2400SoCState, ASPEED2400_SOC)
 struct Aspeed2600SoCState {
     AspeedSoCState parent;
 
-    A15MPPrivState a7mpcore;
+    CortexMPPrivState a7mpcore;
     ARMCPU cpu[ASPEED_CPUS_NUM]; /* XXX belong to a7mpcore */
 };
 
diff --git a/include/hw/arm/fsl-imx6ul.h b/include/hw/arm/fsl-imx6ul.h
index 5ed22004d1..b37d544319 100644
--- a/include/hw/arm/fsl-imx6ul.h
+++ b/include/hw/arm/fsl-imx6ul.h
@@ -68,7 +68,7 @@ struct FslIMX6ULState {
 
     /*< public >*/
     ARMCPU             cpu;
-    A15MPPrivState     a7mpcore;
+    CortexMPPrivState  a7mpcore;
     IMXGPTState        gpt[FSL_IMX6UL_NUM_GPTS];
     IMXEPITState       epit[FSL_IMX6UL_NUM_EPITS];
     IMXGPIOState       gpio[FSL_IMX6UL_NUM_GPIOS];
diff --git a/include/hw/arm/fsl-imx7.h b/include/hw/arm/fsl-imx7.h
index 87faee33c2..a6f3a96029 100644
--- a/include/hw/arm/fsl-imx7.h
+++ b/include/hw/arm/fsl-imx7.h
@@ -68,7 +68,7 @@ struct FslIMX7State {
 
     /*< public >*/
     ARMCPU             cpu[FSL_IMX7_NUM_CPUS];
-    A15MPPrivState     a7mpcore;
+    CortexMPPrivState  a7mpcore;
     IMXGPTState        gpt[FSL_IMX7_NUM_GPTS];
     IMXGPIOState       gpio[FSL_IMX7_NUM_GPIOS];
     IMX7CCMState       ccm;
diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 73627bc415..7822d5cbc4 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -117,10 +117,5 @@ struct A9MPPrivState {
 };
 
 #define TYPE_A15MPCORE_PRIV "a15mpcore_priv"
-OBJECT_DECLARE_SIMPLE_TYPE(A15MPPrivState, A15MPCORE_PRIV)
-
-struct A15MPPrivState {
-    CortexMPPrivState parent_obj;
-};
 
 #endif
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index ecd04b7af1..87b0786781 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -125,7 +125,7 @@ static const TypeInfo a15mp_types[] = {
     {
         .name           = TYPE_A15MPCORE_PRIV,
         .parent         = TYPE_CORTEX_MPCORE_PRIV,
-        .instance_size  = sizeof(A15MPPrivState),
+        .instance_size  = sizeof(CortexMPPrivState),
         .class_init     = a15mp_priv_class_init,
     },
 };
-- 
2.41.0

[PATCH 19/33] hw/cpu/arm: Introduce TYPE_A7MPCORE_PRIV for Cortex-A7 MPCore

[Philippe Mathieu-Daudé]

For QEMU modelling, the only difference between the A15 and A7
MPCore is the latter can have up to 480 SPIs.

In particular, since commit b151de69f6 ("hw/arm: ast2600: Set
AST2600_MAX_IRQ to value from datasheet") the AST2600 machine
initializes its GIC with 256 SPIs, which is more than the 224
maximum of the A15.

Since the A7 was not available, few boards were using the A15.
Replace them by a A7 MPCore.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
The comment in a7mp_priv_class_init() is a bit off.
---
 include/hw/cpu/cortex_mpcore.h |  2 ++
 hw/arm/aspeed_ast2600.c        |  3 +--
 hw/arm/fsl-imx6ul.c            |  3 +--
 hw/arm/fsl-imx7.c              |  3 +--
 hw/cpu/a15mpcore.c             | 30 ++++++++++++++++++++++++++++++
 5 files changed, 35 insertions(+), 6 deletions(-)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 7822d5cbc4..4e1aa9f7f7 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -118,4 +118,6 @@ struct A9MPPrivState {
 
 #define TYPE_A15MPCORE_PRIV "a15mpcore_priv"
 
+#define TYPE_A7MPCORE_PRIV "a7mpcore_priv"
+
 #endif
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index ca788e1cf0..88e2a23514 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -171,8 +171,7 @@ static void aspeed_soc_ast2600_init(Object *obj)
     object_property_add_alias(obj, "hw-prot-key", OBJECT(&s->scu),
                               "hw-prot-key");
 
-    object_initialize_child(obj, "a7mpcore", &a->a7mpcore,
-                            TYPE_A15MPCORE_PRIV);
+    object_initialize_child(obj, "a7mpcore", &a->a7mpcore, TYPE_A7MPCORE_PRIV);
 
     object_initialize_child(obj, "rtc", &s->rtc, TYPE_ASPEED_RTC);
 
diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
index 93908811c5..6e4343efaa 100644
--- a/hw/arm/fsl-imx6ul.c
+++ b/hw/arm/fsl-imx6ul.c
@@ -40,8 +40,7 @@ static void fsl_imx6ul_init(Object *obj)
     /*
      * A7MPCORE
      */
-    object_initialize_child(obj, "a7mpcore", &s->a7mpcore,
-                            TYPE_A15MPCORE_PRIV);
+    object_initialize_child(obj, "a7mpcore", &s->a7mpcore, TYPE_A7MPCORE_PRIV);
 
     /*
      * CCM
diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
index 8a3e9933c4..bd9266b8b5 100644
--- a/hw/arm/fsl-imx7.c
+++ b/hw/arm/fsl-imx7.c
@@ -48,8 +48,7 @@ static void fsl_imx7_init(Object *obj)
     /*
      * A7MPCORE
      */
-    object_initialize_child(obj, "a7mpcore", &s->a7mpcore,
-                            TYPE_A15MPCORE_PRIV);
+    object_initialize_child(obj, "a7mpcore", &s->a7mpcore, TYPE_A7MPCORE_PRIV);
 
     /*
      * GPIOs
diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 87b0786781..5f28a97adb 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -121,6 +121,30 @@ static void a15mp_priv_class_init(ObjectClass *klass, void *data)
     /* We currently have no saveable state */
 }
 
+static void a7mp_priv_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    CortexMPPrivClass *cc = CORTEX_MPCORE_PRIV_CLASS(klass);
+
+    cc->container_size = 0x8000;
+
+    cc->gic_class_name = gic_class_name();
+    cc->gic_revision = 2;
+    /*
+     * The Cortex-A7MP may have anything from 0 to 480 external interrupt
+     * IRQ lines (with another 32 internal). We default to 128+32, which
+     * is the number provided by the Cortex-A15MP test chip in the
+     * Versatile Express A15 development board.
+     * Other boards may differ and should set this property appropriately.
+     */
+    cc->gic_spi_default = 160;
+    cc->gic_spi_max = 480;
+
+    device_class_set_parent_realize(dc, a15mp_priv_realize,
+                                    &cc->parent_realize);
+    /* We currently have no saveable state */
+}
+
 static const TypeInfo a15mp_types[] = {
     {
         .name           = TYPE_A15MPCORE_PRIV,
@@ -128,6 +152,12 @@ static const TypeInfo a15mp_types[] = {
         .instance_size  = sizeof(CortexMPPrivState),
         .class_init     = a15mp_priv_class_init,
     },
+    {
+        .name           = TYPE_A7MPCORE_PRIV,
+        .parent         = TYPE_CORTEX_MPCORE_PRIV,
+        .instance_size  = sizeof(CortexMPPrivState),
+        .class_init     = a7mp_priv_class_init,
+    },
 };
 
 DEFINE_TYPES(a15mp_types)
-- 
2.41.0

[PATCH 20/33] hw/cpu/arm: Consolidate check on max GIC spi supported

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/cortex_mpcore.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index c9ba254818..75324268fa 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -43,6 +43,13 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
     CortexMPPrivClass *k = CORTEX_MPCORE_PRIV_GET_CLASS(dev);
     DeviceState *gicdev = DEVICE(&s->gic);
 
+    if (s->gic_spi_num > k->gic_spi_max) {
+        error_setg(errp,
+                   "At most %u GIC SPI are supported (requested %u)",
+                   k->gic_spi_max, s->gic_spi_num);
+        return;
+    }
+
     qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cores);
     qdev_prop_set_uint32(gicdev, "num-irq", s->gic_spi_num);
     if (k->gic_priority_bits) {
-- 
2.41.0

[PATCH 21/33] hw/cpu/arm: Create CPUs once in MPCore parent

[Philippe Mathieu-Daudé]

Add support for creating the MPCore CPU cluster in the
abstract TYPE_CORTEX_MPCORE_PRIV parent realize() handler.

Only do so if the 'cpu-type' property is set, so current
behavior is not modified. Boards will be converted by
setting this property.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/cpu/cortex_mpcore.h | 12 +++++
 hw/cpu/cortex_mpcore.c         | 88 ++++++++++++++++++++++++++++++++++
 2 files changed, 100 insertions(+)

diff --git a/include/hw/cpu/cortex_mpcore.h b/include/hw/cpu/cortex_mpcore.h
index 4e1aa9f7f7..9a4fc2404e 100644
--- a/include/hw/cpu/cortex_mpcore.h
+++ b/include/hw/cpu/cortex_mpcore.h
@@ -16,6 +16,7 @@
 #include "hw/misc/a9scu.h"
 #include "hw/timer/a9gtimer.h"
 #include "hw/timer/arm_mptimer.h"
+#include "target/arm/cpu-qom.h"
 
 /*
  * This is a model of the Arm Cortex-A MPCore family of hardware.
@@ -93,13 +94,24 @@ struct CortexMPPrivState {
     SysBusDevice parent_obj;
 
     MemoryRegion container;
+    ARMCPU *cpu[4];
     GICState gic;
 
     /* Properties */
+    uint8_t cluster_id;
     uint32_t num_cores;
 
+    char *cpu_type;
     bool cpu_has_el3;
     bool cpu_has_el2;
+    bool cpu_has_vfp_d32;
+    bool cpu_has_neon;
+    uint64_t cpu_freq_hz;
+    uint64_t cpu_midr;
+    uint32_t cpu_psci_conduit;
+    uint64_t cpu_reset_cbar;
+    bool cpu_reset_hivecs;
+    MemoryRegion *cpu_memory;
 
     uint32_t gic_spi_num;
 };
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index 75324268fa..65309636d7 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -12,6 +12,7 @@
 #include "hw/cpu/cortex_mpcore.h"
 #include "hw/irq.h"
 #include "sysemu/kvm.h"
+#include "target/arm/cpu.h"
 
 static void cortex_mpcore_priv_set_irq(void *opaque, int irq, int level)
 {
@@ -50,6 +51,12 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (s->num_cores > ARRAY_SIZE(s->cpu)) {
+        error_setg(errp,
+                   "At most %zu CPU cores are supported", ARRAY_SIZE(s->cpu));
+        return;
+    }
+
     qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cores);
     qdev_prop_set_uint32(gicdev, "num-irq", s->gic_spi_num);
     if (k->gic_priority_bits) {
@@ -75,14 +82,95 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
 
     /* Pass through inbound GPIO lines to the GIC */
     qdev_init_gpio_in(dev, cortex_mpcore_priv_set_irq, s->gic_spi_num - 32);
+
+
+    /* CPU */
+    if (!s->cpu_type) {
+        return;
+    }
+    for (int i = 0; i < s->num_cores; i++) {
+        Object *cpuobj;
+
+        cpuobj = object_new(s->cpu_type);
+        object_property_add_child(OBJECT(dev), "cpu[*]", OBJECT(cpuobj));
+        object_unref(cpuobj);
+        s->cpu[i] = ARM_CPU(cpuobj);
+
+        object_property_set_bool(cpuobj, "neon", s->cpu_has_neon,
+                                &error_abort);
+        object_property_set_bool(cpuobj, "vfp-d32", s->cpu_has_vfp_d32,
+                                &error_abort);
+        if (object_property_find(cpuobj, "has_el3")) {
+            object_property_set_bool(cpuobj, "has_el3", s->cpu_has_el3,
+                                     &error_abort);
+        }
+        if (object_property_find(cpuobj, "has_el2")) {
+            object_property_set_bool(cpuobj, "has_el2", s->cpu_has_el2,
+                                     &error_abort);
+        }
+        if (s->cpu_freq_hz) {
+            object_property_set_int(cpuobj, "cntfrq", s->cpu_freq_hz,
+                                    &error_abort);
+        }
+        object_property_set_int(cpuobj, "midr", s->cpu_midr, &error_abort);
+        object_property_set_bool(cpuobj, "reset-hivecs", s->cpu_reset_hivecs,
+                                 &error_abort);
+        if (s->num_cores == 1) {
+            /* On uniprocessor, the CBAR is set to 0 */
+        } else if (object_property_find(cpuobj, "reset-cbar")) {
+            object_property_set_int(cpuobj, "reset-cbar",
+                                    s->cpu_reset_cbar, &error_abort);
+        }
+        if (i > 0) {
+            /*
+             * Secondary CPUs start in powered-down state (and can be
+             * powered up via the SRC system reset controller)
+             */
+            object_property_set_bool(cpuobj, "start-powered-off", true,
+                                     &error_abort);
+        }
+        if (s->cluster_id) {
+            object_property_set_int(cpuobj, "mp-affinity",
+                                    (s->cluster_id << ARM_AFF1_SHIFT) | i,
+                                    &error_abort);
+        } else {
+            object_property_set_int(cpuobj, "mp-affinity",
+                                    arm_cpu_mp_affinity(i, s->num_cores),
+                                    &error_abort);
+        }
+        object_property_set_int(cpuobj, "psci-conduit",
+                                s->cpu_psci_conduit, &error_abort);
+        if (s->cpu_memory) {
+            object_property_set_link(cpuobj, "memory",
+                                     OBJECT(s->cpu_memory), &error_abort);
+        }
+
+        if (!qdev_realize(DEVICE(s->cpu[i]), NULL, errp)) {
+            return;
+        }
+    }
 }
 
 static Property cortex_mpcore_priv_properties[] = {
+    DEFINE_PROP_UINT8("cluster-id", CortexMPPrivState, cluster_id, 0),
     DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
     DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
 
+    DEFINE_PROP_STRING("cpu-type", CortexMPPrivState, cpu_type),
     DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
     DEFINE_PROP_BOOL("cpu-has-el2", CortexMPPrivState, cpu_has_el2, false),
+    DEFINE_PROP_BOOL("cpu-has-vfp-d32", CortexMPPrivState, cpu_has_vfp_d32,
+                     true),
+    DEFINE_PROP_BOOL("cpu-has-neon", CortexMPPrivState, cpu_has_neon, true),
+    DEFINE_PROP_UINT64("cpu-freq-hz", CortexMPPrivState, cpu_freq_hz, 0),
+    DEFINE_PROP_UINT64("cpu-midr", CortexMPPrivState, cpu_midr, 0),
+    DEFINE_PROP_UINT32("cpu-psci-conduit", CortexMPPrivState, cpu_psci_conduit,
+                       QEMU_PSCI_CONDUIT_DISABLED),
+    DEFINE_PROP_UINT64("cpu-reset-cbar", CortexMPPrivState, cpu_reset_cbar, 0),
+    DEFINE_PROP_BOOL("cpu-reset-hivecs", CortexMPPrivState, cpu_reset_hivecs,
+                     false),
+    DEFINE_PROP_LINK("cpu-memory", CortexMPPrivState, cpu_memory,
+                     TYPE_MEMORY_REGION, MemoryRegion *),
 
     DEFINE_PROP_UINT32("gic-spi-num", CortexMPPrivState, gic_spi_num, 0),
     DEFINE_PROP_UINT32("num-irq", CortexMPPrivState, gic_spi_num, 0), /* alias */
-- 
2.41.0

[PATCH 22/33] hw/arm/aspeed_ast2600: Let the A7MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the a7mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/aspeed_soc.h |  1 -
 hw/arm/aspeed_ast2600.c     | 58 ++++++++++++-------------------------
 2 files changed, 18 insertions(+), 41 deletions(-)

diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index 2f51d78e22..a824679b1e 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -109,7 +109,6 @@ struct Aspeed2600SoCState {
     AspeedSoCState parent;
 
     CortexMPPrivState a7mpcore;
-    ARMCPU cpu[ASPEED_CPUS_NUM]; /* XXX belong to a7mpcore */
 };
 
 #define TYPE_ASPEED2600_SOC "aspeed2600-soc"
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index 88e2a23514..1000fac675 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -156,10 +156,6 @@ static void aspeed_soc_ast2600_init(Object *obj)
         g_assert_not_reached();
     }
 
-    for (i = 0; i < sc->num_cpus; i++) {
-        object_initialize_child(obj, "cpu[*]", &a->cpu[i], sc->cpu_type);
-    }
-
     snprintf(typename, sizeof(typename), "aspeed.scu-%s", socname);
     object_initialize_child(obj, "scu", &s->scu, typename);
     qdev_prop_set_uint32(DEVICE(&s->scu), "silicon-rev",
@@ -270,11 +266,6 @@ static void aspeed_soc_ast2600_init(Object *obj)
  *
  * https://developer.arm.com/documentation/ddi0388/e/the-system-control-coprocessors/summary-of-system-control-coprocessor-registers/multiprocessor-affinity-register
  */
-static uint64_t aspeed_calc_affinity(int cpu)
-{
-    return (0xf << ARM_AFF1_SHIFT) | cpu;
-}
-
 static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
 {
     int i;
@@ -284,6 +275,8 @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
     Error *err = NULL;
     qemu_irq irq;
     g_autofree char *sram_name = NULL;
+    DeviceState *mpdev;
+    CortexMPPrivState *mppriv;
 
     /* Default boot region (SPI memory or ROMs) */
     memory_region_init(&s->spi_boot_container, OBJECT(s),
@@ -305,42 +298,26 @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
                                   "aspeed.emmc-boot-controller",
                                   sc->memmap[ASPEED_DEV_EMMC_BC], 0x1000);
 
-    /* CPU */
-    for (i = 0; i < sc->num_cpus; i++) {
-        if (sc->num_cpus > 1) {
-            object_property_set_int(OBJECT(&a->cpu[i]), "reset-cbar",
-                                    ASPEED_A7MPCORE_ADDR, &error_abort);
-        }
-        object_property_set_int(OBJECT(&a->cpu[i]), "mp-affinity",
-                                aspeed_calc_affinity(i), &error_abort);
-
-        object_property_set_int(OBJECT(&a->cpu[i]), "cntfrq", 1125000000,
-                                &error_abort);
-        object_property_set_bool(OBJECT(&a->cpu[i]), "neon", false,
-                                &error_abort);
-        object_property_set_bool(OBJECT(&a->cpu[i]), "vfp-d32", false,
-                                &error_abort);
-        object_property_set_link(OBJECT(&a->cpu[i]), "memory",
-                                 OBJECT(s->memory), &error_abort);
-
-        if (!qdev_realize(DEVICE(&a->cpu[i]), NULL, errp)) {
-            return;
-        }
-    }
-
     /* A7MPCORE */
-    object_property_set_int(OBJECT(&a->a7mpcore), "num-cores", sc->num_cpus,
-                            &error_abort);
-    object_property_set_int(OBJECT(&a->a7mpcore), "num-irq",
-                            ROUND_UP(AST2600_MAX_IRQ + GIC_INTERNAL, 32),
-                            &error_abort);
-
+    mpdev = DEVICE(&a->a7mpcore);
+    qdev_prop_set_uint32(mpdev, "cluster-id", 0xf);
+    qdev_prop_set_uint32(mpdev, "num-cores", sc->num_cpus);
+    qdev_prop_set_string(mpdev, "cpu-type", sc->cpu_type);
+    qdev_prop_set_uint64(mpdev, "cpu-freq-hz", 1125000000);
+    qdev_prop_set_bit(mpdev, "cpu-has-neon", false);
+    qdev_prop_set_bit(mpdev, "cpu-has-vfp-d32", false);
+    qdev_prop_set_uint64(mpdev, "cpu-reset-cbar", ASPEED_A7MPCORE_ADDR);
+    object_property_set_link(OBJECT(&a->a7mpcore), "cpu-memory",
+                             OBJECT(s->memory), &error_abort);
+    qdev_prop_set_uint32(mpdev, "gic-spi-num",
+                         ROUND_UP(AST2600_MAX_IRQ + GIC_INTERNAL, 32));
     sysbus_realize(SYS_BUS_DEVICE(&a->a7mpcore), &error_abort);
     aspeed_mmio_map(s, SYS_BUS_DEVICE(&a->a7mpcore), 0, ASPEED_A7MPCORE_ADDR);
+    mppriv = CORTEX_MPCORE_PRIV(&a->a7mpcore);
 
     for (i = 0; i < sc->num_cpus; i++) {
         SysBusDevice *sbd = SYS_BUS_DEVICE(&a->a7mpcore);
-        DeviceState  *d   = DEVICE(&a->cpu[i]);
+        DeviceState  *d   = DEVICE(mppriv->cpu[i]);
 
         irq = qdev_get_gpio_in(d, ARM_CPU_IRQ);
         sysbus_connect_irq(sbd, i, irq);
@@ -353,7 +330,8 @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
     }
 
     /* SRAM */
-    sram_name = g_strdup_printf("aspeed.sram.%d", CPU(&a->cpu[0])->cpu_index);
+    sram_name = g_strdup_printf("aspeed.sram.%d",
+                                CPU(mppriv->cpu[0])->cpu_index);
     memory_region_init_ram(&s->sram, OBJECT(s), sram_name, sc->sram_size, &err);
     if (err) {
         error_propagate(errp, err);
-- 
2.41.0

Re: [PATCH 22/33] hw/arm/aspeed_ast2600: Let the A7MPcore create/wire the CPU cores

[Andrew Jeffery]

On Tue, 2023-12-12 at 17:29 +0100, Philippe Mathieu-Daudé wrote:
> Set the properties on the a7mpcore object to let it create and
> wire the CPU cores. Remove the redundant code.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Acked-by: Andrew Jeffery <andrew@codeconstruct.com.au>

[PATCH 23/33] hw/arm/exynos4210: Let the A9MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the a9mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/exynos4210.h |  4 +--
 hw/arm/exynos4210.c         | 62 +++++++++++++------------------------
 hw/arm/exynos4_boards.c     |  6 ++--
 3 files changed, 26 insertions(+), 46 deletions(-)

diff --git a/include/hw/arm/exynos4210.h b/include/hw/arm/exynos4210.h
index f95e3232c5..28a64f275c 100644
--- a/include/hw/arm/exynos4210.h
+++ b/include/hw/arm/exynos4210.h
@@ -83,10 +83,8 @@
 #define EXYNOS4210_NUM_SPLITTERS (EXYNOS4210_MAX_EXT_COMBINER_IN_IRQ + 38)
 
 struct Exynos4210State {
-    /*< private >*/
     SysBusDevice parent_obj;
-    /*< public >*/
-    ARMCPU *cpu[EXYNOS4210_NCPUS];
+
     qemu_irq irq_table[EXYNOS4210_MAX_INT_COMBINER_IN_IRQ];
 
     MemoryRegion chipid_mem;
diff --git a/hw/arm/exynos4210.c b/hw/arm/exynos4210.c
index 7386a8fe57..c6da908961 100644
--- a/hw/arm/exynos4210.c
+++ b/hw/arm/exynos4210.c
@@ -501,12 +501,6 @@ void exynos4210_write_secondary(ARMCPU *cpu,
                        info->smp_loader_start);
 }
 
-static uint64_t exynos4210_calc_affinity(int cpu)
-{
-    /* Exynos4210 has 0x9 as cluster ID */
-    return (0x9 << ARM_AFF1_SHIFT) | cpu;
-}
-
 static DeviceState *pl330_create(uint32_t base, OrIRQState *orgate,
                                  qemu_irq irq, int nreq, int nevents, int width)
 {
@@ -549,26 +543,25 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
     MemoryRegion *system_mem = get_system_memory();
     SysBusDevice *busdev;
     DeviceState *dev, *mpdev, *uart[4], *pl330[3];
+    CortexMPPrivState *mpcore;
     int i, n;
 
-    for (n = 0; n < EXYNOS4210_NCPUS; n++) {
-        Object *cpuobj = object_new(ARM_CPU_TYPE_NAME("cortex-a9"));
-
-        /* By default A9 CPUs have EL3 enabled.  This board does not currently
-         * support EL3 so the CPU EL3 property is disabled before realization.
-         */
-        if (object_property_find(cpuobj, "has_el3")) {
-            object_property_set_bool(cpuobj, "has_el3", false, &error_fatal);
-        }
-
-        s->cpu[n] = ARM_CPU(cpuobj);
-        object_property_set_int(cpuobj, "mp-affinity",
-                                exynos4210_calc_affinity(n), &error_abort);
-        object_property_set_int(cpuobj, "reset-cbar",
-                                EXYNOS4210_SMP_PRIVATE_BASE_ADDR,
-                                &error_abort);
-        qdev_realize(DEVICE(cpuobj), NULL, &error_fatal);
-    }
+    /* Private memory region and Internal GIC */
+    mpdev = DEVICE(&s->a9mpcore);
+    mpcore = CORTEX_MPCORE_PRIV(&s->a9mpcore);
+    /* Exynos4210 has 0x9 as cluster ID */
+    qdev_prop_set_uint32(mpdev, "cluster-id", 0x9);
+    qdev_prop_set_uint32(mpdev, "num-cores", EXYNOS4210_NCPUS);
+    /*
+     * By default A9 CPUs have EL3 enabled.  This board does not currently
+     * support EL3 so the CPU EL3 property is disabled before realization.
+     */
+    qdev_prop_set_bit(mpdev, "cpu-has-el3", false);
+    qdev_prop_set_uint64(mpdev, "cpu-reset-cbar",
+                         EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
+    busdev = SYS_BUS_DEVICE(&s->a9mpcore);
+    sysbus_realize(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
 
     /* IRQ Gate */
     for (i = 0; i < EXYNOS4210_NCPUS; i++) {
@@ -578,23 +571,10 @@ static void exynos4210_realize(DeviceState *socdev, Error **errp)
                                 &error_abort);
         qdev_realize(orgate, NULL, &error_abort);
         qdev_connect_gpio_out(orgate, 0,
-                              qdev_get_gpio_in(DEVICE(s->cpu[i]), ARM_CPU_IRQ));
-    }
-
-    /* Private memory region and Internal GIC */
-    mpdev = DEVICE(&s->a9mpcore);
-    qdev_prop_set_uint32(mpdev, "num-cores", EXYNOS4210_NCPUS);
-    /*
-     * By default A9 CPUs have EL3 enabled.  This board does not currently
-     * support EL3 so the CPU EL3 property is disabled before realization.
-     */
-    qdev_prop_set_bit(mpdev, "cpu-has-el3", false);
-    busdev = SYS_BUS_DEVICE(&s->a9mpcore);
-    sysbus_realize(busdev, &error_fatal);
-    sysbus_mmio_map(busdev, 0, EXYNOS4210_SMP_PRIVATE_BASE_ADDR);
-    for (n = 0; n < EXYNOS4210_NCPUS; n++) {
-        sysbus_connect_irq(busdev, n,
-                           qdev_get_gpio_in(DEVICE(&s->cpu_irq_orgate[n]), 0));
+                              qdev_get_gpio_in(DEVICE(mpcore->cpu[i]),
+                                               ARM_CPU_IRQ));
+        sysbus_connect_irq(busdev, i,
+                           qdev_get_gpio_in(DEVICE(&s->cpu_irq_orgate[i]), 0));
     }
 
     /* Cache controller */
diff --git a/hw/arm/exynos4_boards.c b/hw/arm/exynos4_boards.c
index b0e13eb4f0..7eea66d027 100644
--- a/hw/arm/exynos4_boards.c
+++ b/hw/arm/exynos4_boards.c
@@ -136,18 +136,20 @@ static void nuri_init(MachineState *machine)
 {
     Exynos4BoardState *s = exynos4_boards_init_common(machine,
                                                       EXYNOS4_BOARD_NURI);
+    CortexMPPrivState *mp = CORTEX_MPCORE_PRIV(&s->soc.a9mpcore);
 
-    arm_load_kernel(s->soc.cpu[0], machine, &exynos4_board_binfo);
+    arm_load_kernel(mp->cpu[0], machine, &exynos4_board_binfo);
 }
 
 static void smdkc210_init(MachineState *machine)
 {
     Exynos4BoardState *s = exynos4_boards_init_common(machine,
                                                       EXYNOS4_BOARD_SMDKC210);
+    CortexMPPrivState *mp = CORTEX_MPCORE_PRIV(&s->soc.a9mpcore);
 
     lan9215_init(SMDK_LAN9118_BASE_ADDR,
             qemu_irq_invert(s->soc.irq_table[exynos4210_get_irq(37, 1)]));
-    arm_load_kernel(s->soc.cpu[0], machine, &exynos4_board_binfo);
+    arm_load_kernel(mp->cpu[0], machine, &exynos4_board_binfo);
 }
 
 static void nuri_class_init(ObjectClass *oc, void *data)
-- 
2.41.0

[PATCH 24/33] hw/arm/fsl-imx6: Let the A9MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the a9mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/fsl-imx6.h |  4 ----
 hw/arm/fsl-imx6.c         | 47 +++++++--------------------------------
 hw/arm/sabrelite.c        |  4 +++-
 3 files changed, 11 insertions(+), 44 deletions(-)

diff --git a/include/hw/arm/fsl-imx6.h b/include/hw/arm/fsl-imx6.h
index 21a3b035a4..643c3160c7 100644
--- a/include/hw/arm/fsl-imx6.h
+++ b/include/hw/arm/fsl-imx6.h
@@ -33,7 +33,6 @@
 #include "hw/usb/chipidea.h"
 #include "hw/usb/imx-usb-phy.h"
 #include "exec/memory.h"
-#include "cpu.h"
 #include "qom/object.h"
 
 #define TYPE_FSL_IMX6 "fsl-imx6"
@@ -51,11 +50,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(FslIMX6State, FSL_IMX6)
 #define FSL_IMX6_NUM_USBS 4
 
 struct FslIMX6State {
-    /*< private >*/
     DeviceState parent_obj;
 
-    /*< public >*/
-    ARMCPU         cpu[FSL_IMX6_NUM_CPUS];
     A9MPPrivState  a9mpcore;
     IMX6CCMState   ccm;
     IMX6SRCState   src;
diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
index 65c7c1a0f9..f05ea62351 100644
--- a/hw/arm/fsl-imx6.c
+++ b/hw/arm/fsl-imx6.c
@@ -29,6 +29,7 @@
 #include "chardev/char.h"
 #include "qemu/error-report.h"
 #include "qemu/module.h"
+#include "target/arm/cpu.h"
 
 #define IMX6_ESDHC_CAPABILITIES     0x057834b4
 
@@ -36,17 +37,10 @@
 
 static void fsl_imx6_init(Object *obj)
 {
-    MachineState *ms = MACHINE(qdev_get_machine());
     FslIMX6State *s = FSL_IMX6(obj);
     char name[NAME_SIZE];
     int i;
 
-    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX6_NUM_CPUS); i++) {
-        snprintf(name, NAME_SIZE, "cpu%d", i);
-        object_initialize_child(obj, name, &s->cpu[i],
-                                ARM_CPU_TYPE_NAME("cortex-a9"));
-    }
-
     object_initialize_child(obj, "a9mpcore", &s->a9mpcore, TYPE_A9MPCORE_PRIV);
 
     object_initialize_child(obj, "ccm", &s->ccm, TYPE_IMX6_CCM);
@@ -119,43 +113,18 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    for (i = 0; i < smp_cpus; i++) {
-
-        /* On uniprocessor, the CBAR is set to 0 */
-        if (smp_cpus > 1) {
-            object_property_set_int(OBJECT(&s->cpu[i]), "reset-cbar",
-                                    FSL_IMX6_A9MPCORE_ADDR, &error_abort);
-        }
-
-        /* All CPU but CPU 0 start in power off mode */
-        if (i) {
-            object_property_set_bool(OBJECT(&s->cpu[i]), "start-powered-off",
-                                     true, &error_abort);
-        }
-
-        if (!qdev_realize(DEVICE(&s->cpu[i]), NULL, errp)) {
-            return;
-        }
-    }
-
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-cores", smp_cpus,
-                            &error_abort);
-
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-irq",
-                            FSL_IMX6_MAX_IRQ + GIC_INTERNAL, &error_abort);
-
+    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cores", smp_cpus);
+    qdev_prop_set_string(DEVICE(&s->a9mpcore), "cpu-type",
+                         ARM_CPU_TYPE_NAME("cortex-a9"));
+    qdev_prop_set_uint64(DEVICE(&s->a9mpcore), "cpu-reset-cbar",
+                         FSL_IMX6_A9MPCORE_ADDR);
+    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "gic-spi-num",
+                         FSL_IMX6_MAX_IRQ + GIC_INTERNAL);
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->a9mpcore), errp)) {
         return;
     }
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a9mpcore), 0, FSL_IMX6_A9MPCORE_ADDR);
 
-    for (i = 0; i < smp_cpus; i++) {
-        sysbus_connect_irq(SYS_BUS_DEVICE(&s->a9mpcore), i,
-                           qdev_get_gpio_in(DEVICE(&s->cpu[i]), ARM_CPU_IRQ));
-        sysbus_connect_irq(SYS_BUS_DEVICE(&s->a9mpcore), i + smp_cpus,
-                           qdev_get_gpio_in(DEVICE(&s->cpu[i]), ARM_CPU_FIQ));
-    }
-
     if (!sysbus_realize(SYS_BUS_DEVICE(&s->ccm), errp)) {
         return;
     }
diff --git a/hw/arm/sabrelite.c b/hw/arm/sabrelite.c
index 56f184b9ae..751053e8e3 100644
--- a/hw/arm/sabrelite.c
+++ b/hw/arm/sabrelite.c
@@ -99,7 +99,9 @@ static void sabrelite_init(MachineState *machine)
     sabrelite_binfo.secondary_cpu_reset_hook = sabrelite_reset_secondary;
 
     if (!qtest_enabled()) {
-        arm_load_kernel(&s->cpu[0], machine, &sabrelite_binfo);
+        CortexMPPrivState *mp = CORTEX_MPCORE_PRIV(&s->a9mpcore);
+
+        arm_load_kernel(mp->cpu[0], machine, &sabrelite_binfo);
     }
 }
 
-- 
2.41.0

[PATCH 25/33] hw/arm/fsl-imx6ul: Let the A7MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the a7mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/fsl-imx6ul.h |  4 ----
 hw/arm/fsl-imx6ul.c         | 24 ++++++------------------
 hw/arm/mcimx6ul-evk.c       |  3 ++-
 3 files changed, 8 insertions(+), 23 deletions(-)

diff --git a/include/hw/arm/fsl-imx6ul.h b/include/hw/arm/fsl-imx6ul.h
index b37d544319..9957ab5be0 100644
--- a/include/hw/arm/fsl-imx6ul.h
+++ b/include/hw/arm/fsl-imx6ul.h
@@ -34,7 +34,6 @@
 #include "hw/usb/chipidea.h"
 #include "hw/usb/imx-usb-phy.h"
 #include "exec/memory.h"
-#include "cpu.h"
 #include "qom/object.h"
 #include "qemu/units.h"
 
@@ -63,11 +62,8 @@ enum FslIMX6ULConfiguration {
 };
 
 struct FslIMX6ULState {
-    /*< private >*/
     DeviceState    parent_obj;
 
-    /*< public >*/
-    ARMCPU             cpu;
     CortexMPPrivState  a7mpcore;
     IMXGPTState        gpt[FSL_IMX6UL_NUM_GPTS];
     IMXEPITState       epit[FSL_IMX6UL_NUM_EPITS];
diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
index 6e4343efaa..512973e3c1 100644
--- a/hw/arm/fsl-imx6ul.c
+++ b/hw/arm/fsl-imx6ul.c
@@ -25,6 +25,7 @@
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
 #include "qemu/module.h"
+#include "target/arm/cpu.h" /* qom */
 
 #define NAME_SIZE 20
 
@@ -34,9 +35,6 @@ static void fsl_imx6ul_init(Object *obj)
     char name[NAME_SIZE];
     int i;
 
-    object_initialize_child(obj, "cpu0", &s->cpu,
-                            ARM_CPU_TYPE_NAME("cortex-a7"));
-
     /*
      * A7MPCORE
      */
@@ -158,8 +156,6 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
     DeviceState *gic;
     int i;
     char name[NAME_SIZE];
-    SysBusDevice *sbd;
-    DeviceState *d;
 
     if (ms->smp.cpus > 1) {
         error_setg(errp, "%s: Only a single CPU is supported (%d requested)",
@@ -167,26 +163,18 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    qdev_realize(DEVICE(&s->cpu), NULL, &error_abort);
-
     /*
      * A7MPCORE
      */
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-cores", 1, &error_abort);
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-irq",
-                            FSL_IMX6UL_MAX_IRQ + GIC_INTERNAL, &error_abort);
+    qdev_prop_set_uint32(DEVICE(&s->a7mpcore), "num-cores", 1);
+    qdev_prop_set_string(DEVICE(&s->a7mpcore), "cpu-type",
+                         ARM_CPU_TYPE_NAME("cortex-a7"));
+    qdev_prop_set_uint32(DEVICE(&s->a7mpcore), "gic-spi-num",
+                         FSL_IMX6UL_MAX_IRQ + GIC_INTERNAL);
     sysbus_realize(SYS_BUS_DEVICE(&s->a7mpcore), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a7mpcore), 0, FSL_IMX6UL_A7MPCORE_ADDR);
     gic = DEVICE(&s->a7mpcore);
 
-    sbd = SYS_BUS_DEVICE(&s->a7mpcore);
-    d = DEVICE(&s->cpu);
-
-    sysbus_connect_irq(sbd, 0, qdev_get_gpio_in(d, ARM_CPU_IRQ));
-    sysbus_connect_irq(sbd, 1, qdev_get_gpio_in(d, ARM_CPU_FIQ));
-    sysbus_connect_irq(sbd, 2, qdev_get_gpio_in(d, ARM_CPU_VIRQ));
-    sysbus_connect_irq(sbd, 3, qdev_get_gpio_in(d, ARM_CPU_VFIQ));
-
     /*
      * A7MPCORE DAP
      */
diff --git a/hw/arm/mcimx6ul-evk.c b/hw/arm/mcimx6ul-evk.c
index 500427e94b..a19834930f 100644
--- a/hw/arm/mcimx6ul-evk.c
+++ b/hw/arm/mcimx6ul-evk.c
@@ -18,6 +18,7 @@
 #include "hw/qdev-properties.h"
 #include "qemu/error-report.h"
 #include "sysemu/qtest.h"
+#include "target/arm/cpu.h" /* qom */
 
 static void mcimx6ul_evk_init(MachineState *machine)
 {
@@ -64,7 +65,7 @@ static void mcimx6ul_evk_init(MachineState *machine)
     }
 
     if (!qtest_enabled()) {
-        arm_load_kernel(&s->cpu, machine, &boot_info);
+        arm_load_kernel(s->a7mpcore.cpu[0], machine, &boot_info);
     }
 }
 
-- 
2.41.0

[PATCH 26/33] hw/arm/fsl-imx7: Let the A7MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the a7mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/fsl-imx7.h |  4 ---
 hw/arm/fsl-imx7.c         | 62 +++++----------------------------------
 hw/arm/mcimx7d-sabre.c    |  3 +-
 3 files changed, 10 insertions(+), 59 deletions(-)

diff --git a/include/hw/arm/fsl-imx7.h b/include/hw/arm/fsl-imx7.h
index a6f3a96029..77771f916c 100644
--- a/include/hw/arm/fsl-imx7.h
+++ b/include/hw/arm/fsl-imx7.h
@@ -36,7 +36,6 @@
 #include "hw/net/imx_fec.h"
 #include "hw/pci-host/designware.h"
 #include "hw/usb/chipidea.h"
-#include "cpu.h"
 #include "qom/object.h"
 #include "qemu/units.h"
 
@@ -63,11 +62,8 @@ enum FslIMX7Configuration {
 };
 
 struct FslIMX7State {
-    /*< private >*/
     DeviceState    parent_obj;
 
-    /*< public >*/
-    ARMCPU             cpu[FSL_IMX7_NUM_CPUS];
     CortexMPPrivState  a7mpcore;
     IMXGPTState        gpt[FSL_IMX7_NUM_GPTS];
     IMXGPIOState       gpio[FSL_IMX7_NUM_GPIOS];
diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
index bd9266b8b5..f71ffe7910 100644
--- a/hw/arm/fsl-imx7.c
+++ b/hw/arm/fsl-imx7.c
@@ -26,25 +26,16 @@
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
 #include "qemu/module.h"
+#include "target/arm/cpu.h" /* qom */
 
 #define NAME_SIZE 20
 
 static void fsl_imx7_init(Object *obj)
 {
-    MachineState *ms = MACHINE(qdev_get_machine());
     FslIMX7State *s = FSL_IMX7(obj);
     char name[NAME_SIZE];
     int i;
 
-    /*
-     * CPUs
-     */
-    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX7_NUM_CPUS); i++) {
-        snprintf(name, NAME_SIZE, "cpu%d", i);
-        object_initialize_child(obj, name, &s->cpu[i],
-                                ARM_CPU_TYPE_NAME("cortex-a7"));
-    }
-
     /*
      * A7MPCORE
      */
@@ -163,7 +154,6 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
     MachineState *ms = MACHINE(qdev_get_machine());
     FslIMX7State *s = FSL_IMX7(dev);
     DeviceState *gic;
-    Object *o;
     int i;
     qemu_irq irq;
     char name[NAME_SIZE];
@@ -175,56 +165,20 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    /*
-     * CPUs
-     */
-    for (i = 0; i < smp_cpus; i++) {
-        o = OBJECT(&s->cpu[i]);
-
-        /* On uniprocessor, the CBAR is set to 0 */
-        if (smp_cpus > 1) {
-            object_property_set_int(o, "reset-cbar", FSL_IMX7_A7MPCORE_ADDR,
-                                    &error_abort);
-        }
-
-        if (i) {
-            /*
-             * Secondary CPUs start in powered-down state (and can be
-             * powered up via the SRC system reset controller)
-             */
-            object_property_set_bool(o, "start-powered-off", true,
-                                     &error_abort);
-        }
-
-        qdev_realize(DEVICE(o), NULL, &error_abort);
-    }
-
     /*
      * A7MPCORE
      */
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-cores", smp_cpus,
-                            &error_abort);
-    object_property_set_int(OBJECT(&s->a7mpcore), "num-irq",
-                            FSL_IMX7_MAX_IRQ + GIC_INTERNAL, &error_abort);
-
+    qdev_prop_set_uint32(DEVICE(&s->a7mpcore), "num-cores", ms->smp.cpus);
+    qdev_prop_set_string(DEVICE(&s->a7mpcore), "cpu-type",
+                         ARM_CPU_TYPE_NAME("cortex-a7"));
+    qdev_prop_set_uint64(DEVICE(&s->a7mpcore), "cpu-reset-cbar",
+                         FSL_IMX7_A7MPCORE_ADDR);
+    qdev_prop_set_uint32(DEVICE(&s->a7mpcore), "gic-spi-num",
+                         FSL_IMX7_MAX_IRQ + GIC_INTERNAL);
     sysbus_realize(SYS_BUS_DEVICE(&s->a7mpcore), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a7mpcore), 0, FSL_IMX7_A7MPCORE_ADDR);
     gic = DEVICE(&s->a7mpcore);
 
-    for (i = 0; i < smp_cpus; i++) {
-        SysBusDevice *sbd = SYS_BUS_DEVICE(&s->a7mpcore);
-        DeviceState  *d   = DEVICE(qemu_get_cpu(i));
-
-        irq = qdev_get_gpio_in(d, ARM_CPU_IRQ);
-        sysbus_connect_irq(sbd, i, irq);
-        irq = qdev_get_gpio_in(d, ARM_CPU_FIQ);
-        sysbus_connect_irq(sbd, i + smp_cpus, irq);
-        irq = qdev_get_gpio_in(d, ARM_CPU_VIRQ);
-        sysbus_connect_irq(sbd, i + 2 * smp_cpus, irq);
-        irq = qdev_get_gpio_in(d, ARM_CPU_VFIQ);
-        sysbus_connect_irq(sbd, i + 3 * smp_cpus, irq);
-    }
-
     /*
      * A7MPCORE DAP
      */
diff --git a/hw/arm/mcimx7d-sabre.c b/hw/arm/mcimx7d-sabre.c
index 693a1023b6..782f7591db 100644
--- a/hw/arm/mcimx7d-sabre.c
+++ b/hw/arm/mcimx7d-sabre.c
@@ -20,6 +20,7 @@
 #include "hw/qdev-properties.h"
 #include "qemu/error-report.h"
 #include "sysemu/qtest.h"
+#include "target/arm/cpu.h" /* qom */
 
 static void mcimx7d_sabre_init(MachineState *machine)
 {
@@ -64,7 +65,7 @@ static void mcimx7d_sabre_init(MachineState *machine)
     }
 
     if (!qtest_enabled()) {
-        arm_load_kernel(&s->cpu[0], machine, &boot_info);
+        arm_load_kernel(s->a7mpcore.cpu[0], machine, &boot_info);
     }
 }
 
-- 
2.41.0

[PATCH 27/33] hw/arm/highbank: Let the A9/A15MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/highbank.c | 53 +++++++----------------------------------------
 1 file changed, 8 insertions(+), 45 deletions(-)

diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index a184bee993..b4f8304146 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -180,47 +180,10 @@ static void calxeda_init(MachineState *machine, enum cxmachines machine_id)
     SysBusDevice *busdev;
     qemu_irq pic[128];
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
-    qemu_irq cpu_irq[4];
-    qemu_irq cpu_fiq[4];
-    qemu_irq cpu_virq[4];
-    qemu_irq cpu_vfiq[4];
     MemoryRegion *sysram;
     MemoryRegion *sysmem;
     char *sysboot_filename;
 
-    switch (machine_id) {
-    case CALXEDA_HIGHBANK:
-        machine->cpu_type = ARM_CPU_TYPE_NAME("cortex-a9");
-        break;
-    case CALXEDA_MIDWAY:
-        machine->cpu_type = ARM_CPU_TYPE_NAME("cortex-a15");
-        break;
-    default:
-        assert(0);
-    }
-
-    for (n = 0; n < smp_cpus; n++) {
-        Object *cpuobj;
-        ARMCPU *cpu;
-
-        cpuobj = object_new(machine->cpu_type);
-        cpu = ARM_CPU(cpuobj);
-
-        object_property_set_int(cpuobj, "psci-conduit", QEMU_PSCI_CONDUIT_SMC,
-                                &error_abort);
-
-        if (object_property_find(cpuobj, "reset-cbar")) {
-            object_property_set_int(cpuobj, "reset-cbar", MPCORE_PERIPHBASE,
-                                    &error_abort);
-        }
-        qdev_realize(DEVICE(cpuobj), NULL, &error_fatal);
-        cpu_irq[n] = qdev_get_gpio_in(DEVICE(cpu), ARM_CPU_IRQ);
-        cpu_fiq[n] = qdev_get_gpio_in(DEVICE(cpu), ARM_CPU_FIQ);
-        cpu_virq[n] = qdev_get_gpio_in(DEVICE(cpu), ARM_CPU_VIRQ);
-        cpu_vfiq[n] = qdev_get_gpio_in(DEVICE(cpu), ARM_CPU_VFIQ);
-    }
-
     sysmem = get_system_memory();
     /* SDRAM at address zero.  */
     memory_region_add_subregion(sysmem, 0, machine->ram);
@@ -251,22 +214,22 @@ static void calxeda_init(MachineState *machine, enum cxmachines machine_id)
         sysbus_mmio_map(busdev, 0, 0xfff12000);
 
         dev = qdev_new(TYPE_A9MPCORE_PRIV);
+        qdev_prop_set_string(dev, "cpu-type", ARM_CPU_TYPE_NAME("cortex-a9"));
         break;
     case CALXEDA_MIDWAY:
         dev = qdev_new(TYPE_A15MPCORE_PRIV);
+        qdev_prop_set_string(dev, "cpu-type", ARM_CPU_TYPE_NAME("cortex-a15"));
         break;
+    default:
+        g_assert_not_reached();
     }
-    qdev_prop_set_uint32(dev, "num-cpu", smp_cpus);
-    qdev_prop_set_uint32(dev, "num-irq", NIRQ_GIC);
+    qdev_prop_set_uint32(dev, "num-cores", machine->smp.cpus);
+    qdev_prop_set_uint32(dev, "cpu-psci-conduit", QEMU_PSCI_CONDUIT_SMC);
+    qdev_prop_set_uint64(dev, "cpu-reset-cbar", MPCORE_PERIPHBASE);
+    qdev_prop_set_uint32(dev, "gic-spi-num", NIRQ_GIC);
     busdev = SYS_BUS_DEVICE(dev);
     sysbus_realize_and_unref(busdev, &error_fatal);
     sysbus_mmio_map(busdev, 0, MPCORE_PERIPHBASE);
-    for (n = 0; n < smp_cpus; n++) {
-        sysbus_connect_irq(busdev, n, cpu_irq[n]);
-        sysbus_connect_irq(busdev, n + smp_cpus, cpu_fiq[n]);
-        sysbus_connect_irq(busdev, n + 2 * smp_cpus, cpu_virq[n]);
-        sysbus_connect_irq(busdev, n + 3 * smp_cpus, cpu_vfiq[n]);
-    }
 
     for (n = 0; n < 128; n++) {
         pic[n] = qdev_get_gpio_in(dev, n);
-- 
2.41.0

[PATCH 28/33] hw/arm/vexpress: Let the A9/A15MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/vexpress.c | 51 ++++++-----------------------------------------
 1 file changed, 6 insertions(+), 45 deletions(-)

diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index 294b6f15f2..c104759588 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -210,45 +210,19 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
                       qemu_irq *pic, bool secure, bool virt)
 {
     DeviceState *dev;
-    SysBusDevice *busdev;
-    int n;
-    unsigned int smp_cpus = ms->smp.cpus;
 
-    /* Create the actual CPUs */
-    for (n = 0; n < smp_cpus; n++) {
-        Object *cpuobj = object_new(cpu_type);
-
-        if (!secure) {
-            object_property_set_bool(cpuobj, "has_el3", false, NULL);
-        }
-        if (!virt) {
-            if (object_property_find(cpuobj, "has_el2")) {
-                object_property_set_bool(cpuobj, "has_el2", false, NULL);
-            }
-        }
-
-        if (object_property_find(cpuobj, "reset-cbar")) {
-            object_property_set_int(cpuobj, "reset-cbar", periphbase,
-                                    &error_abort);
-        }
-        qdev_realize(DEVICE(cpuobj), NULL, &error_fatal);
-    }
-
-    /* Create the private peripheral devices (including the GIC);
-     * this must happen after the CPUs are created because a15mpcore_priv
-     * wires itself up to the CPU's generic_timer gpio out lines.
-     */
     dev = qdev_new(privdev);
+    qdev_prop_set_uint32(dev, "num-cores", ms->smp.cpus);
+    qdev_prop_set_string(dev, "cpu-type", cpu_type);
     if (!secure) {
         qdev_prop_set_bit(dev, "cpu-has-el3", false);
     }
     if (!virt) {
         qdev_prop_set_bit(dev, "cpu-has-el2", false);
     }
-    qdev_prop_set_uint32(dev, "num-cpu", smp_cpus);
-    busdev = SYS_BUS_DEVICE(dev);
-    sysbus_realize_and_unref(busdev, &error_fatal);
-    sysbus_mmio_map(busdev, 0, periphbase);
+    qdev_prop_set_uint64(dev, "cpu-reset-cbar", periphbase);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, periphbase);
 
     /* Interrupts [42:0] are from the motherboard;
      * [47:43] are reserved; [63:48] are daughterboard
@@ -256,22 +230,9 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
      * external interrupts starting from 32 (because there
      * are internal interrupts 0..31).
      */
-    for (n = 0; n < 64; n++) {
+    for (int n = 0; n < 64; n++) {
         pic[n] = qdev_get_gpio_in(dev, n);
     }
-
-    /* Connect the CPUs to the GIC */
-    for (n = 0; n < smp_cpus; n++) {
-        DeviceState *cpudev = DEVICE(qemu_get_cpu(n));
-
-        sysbus_connect_irq(busdev, n, qdev_get_gpio_in(cpudev, ARM_CPU_IRQ));
-        sysbus_connect_irq(busdev, n + smp_cpus,
-                           qdev_get_gpio_in(cpudev, ARM_CPU_FIQ));
-        sysbus_connect_irq(busdev, n + 2 * smp_cpus,
-                           qdev_get_gpio_in(cpudev, ARM_CPU_VIRQ));
-        sysbus_connect_irq(busdev, n + 3 * smp_cpus,
-                           qdev_get_gpio_in(cpudev, ARM_CPU_VFIQ));
-    }
 }
 
 static void a9_daughterboard_init(VexpressMachineState *vms,
-- 
2.41.0

[PATCH 29/33] hw/arm/xilinx_zynq: Let the A9MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/xilinx_zynq.c | 22 +++++-----------------
 1 file changed, 5 insertions(+), 17 deletions(-)

diff --git a/hw/arm/xilinx_zynq.c b/hw/arm/xilinx_zynq.c
index 28430dcfba..736e2115a6 100644
--- a/hw/arm/xilinx_zynq.c
+++ b/hw/arm/xilinx_zynq.c
@@ -192,22 +192,6 @@ static void zynq_init(MachineState *machine)
         exit(EXIT_FAILURE);
     }
 
-    cpu = ARM_CPU(object_new(machine->cpu_type));
-
-    /* By default A9 CPUs have EL3 enabled.  This board does not
-     * currently support EL3 so the CPU EL3 property is disabled before
-     * realization.
-     */
-    if (object_property_find(OBJECT(cpu), "has_el3")) {
-        object_property_set_bool(OBJECT(cpu), "has_el3", false, &error_fatal);
-    }
-
-    object_property_set_int(OBJECT(cpu), "midr", ZYNQ_BOARD_MIDR,
-                            &error_fatal);
-    object_property_set_int(OBJECT(cpu), "reset-cbar", MPCORE_PERIPHBASE,
-                            &error_fatal);
-    qdev_realize(DEVICE(cpu), NULL, &error_fatal);
-
     /* DDR remapped to address zero.  */
     memory_region_add_subregion(address_space_mem, 0, machine->ram);
 
@@ -239,15 +223,19 @@ static void zynq_init(MachineState *machine)
     sysbus_mmio_map(SYS_BUS_DEVICE(slcr), 0, 0xF8000000);
 
     dev = qdev_new(TYPE_A9MPCORE_PRIV);
-    qdev_prop_set_uint32(dev, "num-cpu", 1);
+    qdev_prop_set_uint32(dev, "num-cores", 1);
+    qdev_prop_set_string(dev, "cpu-type", machine->cpu_type);
     /*
      * By default A9 CPUs have EL3 enabled.  This board does not
      * currently support EL3 so the CPU EL3 property is disabled before
      * realization.
      */
     qdev_prop_set_bit(dev, "cpu-has-el3", false);
+    qdev_prop_set_uint64(dev, "cpu-reset-cbar", MPCORE_PERIPHBASE);
+    qdev_prop_set_uint64(dev, "cpu-midr", ZYNQ_BOARD_MIDR);
     busdev = SYS_BUS_DEVICE(dev);
     sysbus_realize_and_unref(busdev, &error_fatal);
+    cpu = CORTEX_MPCORE_PRIV(dev)->cpu[0];
     sysbus_mmio_map(busdev, 0, MPCORE_PERIPHBASE);
     sysbus_connect_irq(busdev, 0,
                        qdev_get_gpio_in(DEVICE(cpu), ARM_CPU_IRQ));
-- 
2.41.0

[PATCH 30/33] hw/arm/npcm7xx: Let the A9MPcore create/wire the CPU cores

[Philippe Mathieu-Daudé]

Set the properties on the mpcore object to let it create and
wire the CPU cores. Remove the redundant code.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/arm/npcm7xx.h |  1 -
 hw/arm/npcm7xx.c         | 48 ++++++++++------------------------------
 2 files changed, 12 insertions(+), 37 deletions(-)

diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h
index 7abbf85cbf..e0737fa4de 100644
--- a/include/hw/arm/npcm7xx.h
+++ b/include/hw/arm/npcm7xx.h
@@ -80,7 +80,6 @@ typedef struct NPCM7xxMachineClass {
 struct NPCM7xxState {
     DeviceState         parent;
 
-    ARMCPU              cpu[NPCM7XX_MAX_NUM_CPUS];
     A9MPPrivState       a9mpcore;
 
     MemoryRegion        sram;
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index 5b6e968fa9..1154a0f0a5 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -363,9 +363,11 @@ static struct arm_boot_info npcm7xx_binfo = {
 
 void npcm7xx_load_kernel(MachineState *machine, NPCM7xxState *soc)
 {
+    CortexMPPrivState *mp = CORTEX_MPCORE_PRIV(&soc->a9mpcore);
+
     npcm7xx_binfo.ram_size = machine->ram_size;
 
-    arm_load_kernel(&soc->cpu[0], machine, &npcm7xx_binfo);
+    arm_load_kernel(mp->cpu[0], machine, &npcm7xx_binfo);
 }
 
 static void npcm7xx_init_fuses(NPCM7xxState *s)
@@ -400,11 +402,6 @@ static void npcm7xx_init(Object *obj)
     NPCM7xxState *s = NPCM7XX(obj);
     int i;
 
-    for (i = 0; i < NPCM7XX_MAX_NUM_CPUS; i++) {
-        object_initialize_child(obj, "cpu[*]", &s->cpu[i],
-                                ARM_CPU_TYPE_NAME("cortex-a9"));
-    }
-
     object_initialize_child(obj, "a9mpcore", &s->a9mpcore, TYPE_A9MPCORE_PRIV);
     object_initialize_child(obj, "gcr", &s->gcr, TYPE_NPCM7XX_GCR);
     object_property_add_alias(obj, "power-on-straps", OBJECT(&s->gcr),
@@ -471,40 +468,19 @@ static void npcm7xx_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    /* CPUs */
-    for (i = 0; i < nc->num_cpus; i++) {
-        object_property_set_int(OBJECT(&s->cpu[i]), "mp-affinity",
-                                arm_cpu_mp_affinity(i, NPCM7XX_MAX_NUM_CPUS),
-                                &error_abort);
-        object_property_set_int(OBJECT(&s->cpu[i]), "reset-cbar",
-                                NPCM7XX_GIC_CPU_IF_ADDR, &error_abort);
-        object_property_set_bool(OBJECT(&s->cpu[i]), "reset-hivecs", true,
-                                 &error_abort);
-
-        /* Disable security extensions. */
-        object_property_set_bool(OBJECT(&s->cpu[i]), "has_el3", false,
-                                 &error_abort);
-
-        if (!qdev_realize(DEVICE(&s->cpu[i]), NULL, errp)) {
-            return;
-        }
-    }
-
     /* A9MPCORE peripherals. Can only fail if we pass bad parameters here. */
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-cores", nc->num_cpus,
-                            &error_abort);
-    object_property_set_int(OBJECT(&s->a9mpcore), "num-irq", NPCM7XX_NUM_IRQ,
-                            &error_abort);
+    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "num-cores", nc->num_cpus);
+    qdev_prop_set_string(DEVICE(&s->a9mpcore), "cpu-type",
+                         ARM_CPU_TYPE_NAME("cortex-a9"));
+    /* Disable security extensions. */
+    qdev_prop_set_bit(DEVICE(&s->a9mpcore), "cpu-has-el3", false);
+    qdev_prop_set_uint64(DEVICE(&s->a9mpcore), "cpu-reset-cbar",
+                         NPCM7XX_GIC_CPU_IF_ADDR);
+    qdev_prop_set_bit(DEVICE(&s->a9mpcore), "cpu-reset-hivecs", true);
+    qdev_prop_set_uint32(DEVICE(&s->a9mpcore), "gic-spi-num", NPCM7XX_NUM_IRQ);
     sysbus_realize(SYS_BUS_DEVICE(&s->a9mpcore), &error_abort);
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->a9mpcore), 0, NPCM7XX_CPUP_BA);
 
-    for (i = 0; i < nc->num_cpus; i++) {
-        sysbus_connect_irq(SYS_BUS_DEVICE(&s->a9mpcore), i,
-                           qdev_get_gpio_in(DEVICE(&s->cpu[i]), ARM_CPU_IRQ));
-        sysbus_connect_irq(SYS_BUS_DEVICE(&s->a9mpcore), i + nc->num_cpus,
-                           qdev_get_gpio_in(DEVICE(&s->cpu[i]), ARM_CPU_FIQ));
-    }
-
     /* L2 cache controller */
     sysbus_create_simple("l2x0", NPCM7XX_L2C_BA, NULL);
 
-- 
2.41.0

[PATCH 31/33] hw/cpu/a9mpcore: Remove legacy code

[Philippe Mathieu-Daudé]

Now that all boards set the "cpu-type" property,
the legacy code can be removed.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/a9mpcore.c      | 8 +++-----
 hw/cpu/cortex_mpcore.c | 8 +++++---
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/hw/cpu/a9mpcore.c b/hw/cpu/a9mpcore.c
index ffdaf392fc..4e2fdb8321 100644
--- a/hw/cpu/a9mpcore.c
+++ b/hw/cpu/a9mpcore.c
@@ -41,12 +41,10 @@ static void a9mp_priv_realize(DeviceState *dev, Error **errp)
     DeviceState *scudev, *gtimerdev, *mptimerdev, *wdtdev;
     SysBusDevice *scubusdev, *gtimerbusdev, *mptimerbusdev, *wdtbusdev;
     Error *local_err = NULL;
-    CPUState *cpu0;
-    Object *cpuobj;
 
-    cpu0 = qemu_get_cpu(0);
-    cpuobj = OBJECT(cpu0);
-    if (strcmp(object_get_typename(cpuobj), ARM_CPU_TYPE_NAME("cortex-a9"))) {
+    if (!c->cpu_type) {
+        qdev_prop_set_string(dev, "cpu-type", ARM_CPU_TYPE_NAME("cortex-a9"));
+    } else if (strcmp(c->cpu_type, ARM_CPU_TYPE_NAME("cortex-a9"))) {
         /* We might allow Cortex-A5 once we model it */
         error_setg(errp,
                    "Cortex-A9MPCore peripheral can only use Cortex-A9 CPU");
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index 65309636d7..c5de6c3ae4 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -57,6 +57,11 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (!s->cpu_type) {
+        error_setg(errp, "'cpu-type' property is not set");
+        return;
+    }
+
     qdev_prop_set_uint32(gicdev, "num-cpu", s->num_cores);
     qdev_prop_set_uint32(gicdev, "num-irq", s->gic_spi_num);
     if (k->gic_priority_bits) {
@@ -85,9 +90,6 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
 
 
     /* CPU */
-    if (!s->cpu_type) {
-        return;
-    }
     for (int i = 0; i < s->num_cores; i++) {
         Object *cpuobj;
 
-- 
2.41.0

[PATCH 32/33] hw/cpu/arm: Remove 'num-cpu' property alias

[Philippe Mathieu-Daudé]

All callers access the 'num-cores' property.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
Better to squash earlier and replace in place, so no need for alias?
---
 hw/cpu/arm11mpcore.c     | 1 -
 hw/cpu/cortex_mpcore.c   | 2 --
 hw/cpu/realview_mpcore.c | 1 -
 3 files changed, 4 deletions(-)

diff --git a/hw/cpu/arm11mpcore.c b/hw/cpu/arm11mpcore.c
index 67c96a3a7c..32756e05e9 100644
--- a/hw/cpu/arm11mpcore.c
+++ b/hw/cpu/arm11mpcore.c
@@ -133,7 +133,6 @@ static void mpcore_priv_initfn(Object *obj)
 
 static Property mpcore_priv_properties[] = {
     DEFINE_PROP_UINT32("num-cores", ARM11MPCorePriveState, num_cpu, 1),
-    DEFINE_PROP_UINT32("num-cpu", ARM11MPCorePriveState, num_cpu, 1), /* alias */
     /* The ARM11 MPCORE TRM says the on-chip controller may have
      * anything from 0 to 224 external interrupt IRQ lines (with another
      * 32 internal). We default to 32+32, which is the number provided by
diff --git a/hw/cpu/cortex_mpcore.c b/hw/cpu/cortex_mpcore.c
index c5de6c3ae4..3cc9feef16 100644
--- a/hw/cpu/cortex_mpcore.c
+++ b/hw/cpu/cortex_mpcore.c
@@ -156,7 +156,6 @@ static void cortex_mpcore_priv_realize(DeviceState *dev, Error **errp)
 static Property cortex_mpcore_priv_properties[] = {
     DEFINE_PROP_UINT8("cluster-id", CortexMPPrivState, cluster_id, 0),
     DEFINE_PROP_UINT32("num-cores", CortexMPPrivState, num_cores, 1),
-    DEFINE_PROP_UINT32("num-cpu", CortexMPPrivState, num_cores, 1), /* alias */
 
     DEFINE_PROP_STRING("cpu-type", CortexMPPrivState, cpu_type),
     DEFINE_PROP_BOOL("cpu-has-el3", CortexMPPrivState, cpu_has_el3, true),
@@ -175,7 +174,6 @@ static Property cortex_mpcore_priv_properties[] = {
                      TYPE_MEMORY_REGION, MemoryRegion *),
 
     DEFINE_PROP_UINT32("gic-spi-num", CortexMPPrivState, gic_spi_num, 0),
-    DEFINE_PROP_UINT32("num-irq", CortexMPPrivState, gic_spi_num, 0), /* alias */
 
     DEFINE_PROP_END_OF_LIST(),
 };
diff --git a/hw/cpu/realview_mpcore.c b/hw/cpu/realview_mpcore.c
index e985d230e0..4d4965126b 100644
--- a/hw/cpu/realview_mpcore.c
+++ b/hw/cpu/realview_mpcore.c
@@ -99,7 +99,6 @@ static void mpcore_rirq_init(Object *obj)
 
     object_initialize_child(obj, "a11priv", &s->priv, TYPE_ARM11MPCORE_PRIV);
     object_property_add_alias(obj, "num-cores", OBJECT(&s->priv), "num-cores");
-    object_property_add_alias(obj, "num-cpu", OBJECT(&s->priv), "num-cores");
     privbusdev = SYS_BUS_DEVICE(&s->priv);
     sysbus_init_mmio(sbd, sysbus_mmio_get_region(privbusdev, 0));
 
-- 
2.41.0

[PATCH 33/33] hw/cpu/arm: Remove use of qemu_get_cpu() in A7/A15 realize()

[Philippe Mathieu-Daudé]

Previous commits (in particular "Create CPUs once in MPCore parent")
allowed creating the MPCore private container vCPUs within the
container. We don't need to call qemu_get_cpu(), which is unsafe in
hegerogeneous context.
Directly access the CortexMPPrivState::cpu array.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/cpu/a15mpcore.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/cpu/a15mpcore.c b/hw/cpu/a15mpcore.c
index 5f28a97adb..8dccf15103 100644
--- a/hw/cpu/a15mpcore.c
+++ b/hw/cpu/a15mpcore.c
@@ -44,7 +44,7 @@ static void a15mp_priv_realize(DeviceState *dev, Error **errp)
      * appropriate GIC PPI inputs
      */
     for (i = 0; i < c->num_cores; i++) {
-        DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
+        DeviceState *cpudev = DEVICE(c->cpu[i]);
         int ppibase = c->gic_spi_num - 32 + i * 32;
         int irq;
         /* Mapping from the output timer irq lines from the CPU to the
-- 
2.41.0

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

Hi,

ping for review?

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> When a MPCore cluster is used, the Cortex-A cores belong the the
> cluster container, not to the board/soc layer. This series move
> the creation of vCPUs to the MPCore private container.
> 
> Doing so we consolidate the QOM model, moving common code in a
> central place (abstract MPCore parent).
> 
> This eventually allow removing one qemu_get_cpu() use, which we
> want to remove in heterogeneous machines (machines using MPCore
> are candidate for heterogeneous emulation).
> 
> Maybe these hw/cpu/arm/ files belong to hw/arm/...
> 
> Regards,
> 
> Phil.
> 
> Philippe Mathieu-Daudé (33):
>    hw/arm/boot: Propagate vCPU to arm_load_dtb()
>    hw/arm/fsl-imx6: Add a local 'gic' variable
>    hw/arm/fsl-imx6ul: Add a local 'gic' variable
>    hw/arm/fsl-imx7: Add a local 'gic' variable
>    hw/cpu: Remove dead Kconfig
>    hw/cpu/arm: Rename 'busdev' -> 'gicsbd' in a15mp_priv_realize()
>    hw/cpu/arm: Alias 'num-cpu' property on TYPE_REALVIEW_MPCORE
>    hw/cpu/arm: Declare CPU QOM types using DEFINE_TYPES() macro
>    hw/cpu/arm: Merge {a9mpcore.h, a15mpcore.h} as cortex_mpcore.h
>    hw/cpu/arm: Introduce abstract CORTEX_MPCORE_PRIV QOM type
>    hw/cpu/arm: Have A9MPCORE/A15MPCORE inheritate common
>      CORTEX_MPCORE_PRIV
>    hw/cpu/arm: Create MPCore container in QOM parent
>    hw/cpu/arm: Handle 'num_cores' property once in MPCore parent
>    hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent
>    hw/cpu/arm: Handle 'gic-irq' property once in MPCore parent
>    hw/cpu/arm: Handle GIC once in MPCore parent
>    hw/cpu/arm: Document more properties of CORTEX_MPCORE_PRIV QOM type
>    hw/cpu/arm: Replace A15MPPrivState by CortexMPPrivState
>    hw/cpu/arm: Introduce TYPE_A7MPCORE_PRIV for Cortex-A7 MPCore
>    hw/cpu/arm: Consolidate check on max GIC spi supported
>    hw/cpu/arm: Create CPUs once in MPCore parent
>    hw/arm/aspeed_ast2600: Let the A7MPcore create/wire the CPU cores
>    hw/arm/exynos4210: Let the A9MPcore create/wire the CPU cores
>    hw/arm/fsl-imx6: Let the A9MPcore create/wire the CPU cores
>    hw/arm/fsl-imx6ul: Let the A7MPcore create/wire the CPU cores
>    hw/arm/fsl-imx7: Let the A7MPcore create/wire the CPU cores
>    hw/arm/highbank: Let the A9/A15MPcore create/wire the CPU cores
>    hw/arm/vexpress: Let the A9/A15MPcore create/wire the CPU cores
>    hw/arm/xilinx_zynq: Let the A9MPcore create/wire the CPU cores
>    hw/arm/npcm7xx: Let the A9MPcore create/wire the CPU cores
>    hw/cpu/a9mpcore: Remove legacy code
>    hw/cpu/arm: Remove 'num-cpu' property alias
>    hw/cpu/arm: Remove use of qemu_get_cpu() in A7/A15 realize()

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> When a MPCore cluster is used, the Cortex-A cores belong the the
> cluster container, not to the board/soc layer. This series move
> the creation of vCPUs to the MPCore private container.
> 
> Doing so we consolidate the QOM model, moving common code in a
> central place (abstract MPCore parent).

Changing the QOM hierarchy has an impact on the state of the machine
and some fixups are then required to maintain migration compatibility.
This can become a real headache for KVM machines like virt for which
migration compatibility is a feature, less for emulated ones.

I don't have a good solution to propose to overcome this problem :/

C.
  

> 
> This eventually allow removing one qemu_get_cpu() use, which we
> want to remove in heterogeneous machines (machines using MPCore
> are candidate for heterogeneous emulation).
> 
> Maybe these hw/cpu/arm/ files belong to hw/arm/...
> 
> Regards,
> 
> Phil.
> 
> Philippe Mathieu-Daudé (33):
>    hw/arm/boot: Propagate vCPU to arm_load_dtb()
>    hw/arm/fsl-imx6: Add a local 'gic' variable
>    hw/arm/fsl-imx6ul: Add a local 'gic' variable
>    hw/arm/fsl-imx7: Add a local 'gic' variable
>    hw/cpu: Remove dead Kconfig
>    hw/cpu/arm: Rename 'busdev' -> 'gicsbd' in a15mp_priv_realize()
>    hw/cpu/arm: Alias 'num-cpu' property on TYPE_REALVIEW_MPCORE
>    hw/cpu/arm: Declare CPU QOM types using DEFINE_TYPES() macro
>    hw/cpu/arm: Merge {a9mpcore.h, a15mpcore.h} as cortex_mpcore.h
>    hw/cpu/arm: Introduce abstract CORTEX_MPCORE_PRIV QOM type
>    hw/cpu/arm: Have A9MPCORE/A15MPCORE inheritate common
>      CORTEX_MPCORE_PRIV
>    hw/cpu/arm: Create MPCore container in QOM parent
>    hw/cpu/arm: Handle 'num_cores' property once in MPCore parent
>    hw/cpu/arm: Handle 'has_el2/3' properties once in MPCore parent
>    hw/cpu/arm: Handle 'gic-irq' property once in MPCore parent
>    hw/cpu/arm: Handle GIC once in MPCore parent
>    hw/cpu/arm: Document more properties of CORTEX_MPCORE_PRIV QOM type
>    hw/cpu/arm: Replace A15MPPrivState by CortexMPPrivState
>    hw/cpu/arm: Introduce TYPE_A7MPCORE_PRIV for Cortex-A7 MPCore
>    hw/cpu/arm: Consolidate check on max GIC spi supported
>    hw/cpu/arm: Create CPUs once in MPCore parent
>    hw/arm/aspeed_ast2600: Let the A7MPcore create/wire the CPU cores
>    hw/arm/exynos4210: Let the A9MPcore create/wire the CPU cores
>    hw/arm/fsl-imx6: Let the A9MPcore create/wire the CPU cores
>    hw/arm/fsl-imx6ul: Let the A7MPcore create/wire the CPU cores
>    hw/arm/fsl-imx7: Let the A7MPcore create/wire the CPU cores
>    hw/arm/highbank: Let the A9/A15MPcore create/wire the CPU cores
>    hw/arm/vexpress: Let the A9/A15MPcore create/wire the CPU cores
>    hw/arm/xilinx_zynq: Let the A9MPcore create/wire the CPU cores
>    hw/arm/npcm7xx: Let the A9MPcore create/wire the CPU cores
>    hw/cpu/a9mpcore: Remove legacy code
>    hw/cpu/arm: Remove 'num-cpu' property alias
>    hw/cpu/arm: Remove use of qemu_get_cpu() in A7/A15 realize()
> 
>   MAINTAINERS                    |   3 +-
>   include/hw/arm/aspeed_soc.h    |   5 +-
>   include/hw/arm/boot.h          |   4 +-
>   include/hw/arm/exynos4210.h    |   6 +-
>   include/hw/arm/fsl-imx6.h      |   6 +-
>   include/hw/arm/fsl-imx6ul.h    |   8 +-
>   include/hw/arm/fsl-imx7.h      |   8 +-
>   include/hw/arm/npcm7xx.h       |   3 +-
>   include/hw/cpu/a15mpcore.h     |  44 -------
>   include/hw/cpu/a9mpcore.h      |  39 -------
>   include/hw/cpu/cortex_mpcore.h | 135 ++++++++++++++++++++++
>   hw/arm/aspeed_ast2600.c        |  61 ++++------
>   hw/arm/boot.c                  |  11 +-
>   hw/arm/exynos4210.c            |  60 ++++------
>   hw/arm/exynos4_boards.c        |   6 +-
>   hw/arm/fsl-imx6.c              |  84 ++++----------
>   hw/arm/fsl-imx6ul.c            |  65 ++++-------
>   hw/arm/fsl-imx7.c              | 103 +++++------------
>   hw/arm/highbank.c              |  56 ++-------
>   hw/arm/mcimx6ul-evk.c          |   3 +-
>   hw/arm/mcimx7d-sabre.c         |   3 +-
>   hw/arm/npcm7xx.c               |  48 ++------
>   hw/arm/realview.c              |   4 +-
>   hw/arm/sabrelite.c             |   4 +-
>   hw/arm/vexpress.c              |  60 +++-------
>   hw/arm/virt.c                  |   2 +-
>   hw/arm/xilinx_zynq.c           |  30 ++---
>   hw/cpu/a15mpcore.c             | 179 +++++++++++++----------------
>   hw/cpu/a9mpcore.c              | 138 +++++++++-------------
>   hw/cpu/arm11mpcore.c           |  23 ++--
>   hw/cpu/cortex_mpcore.c         | 202 +++++++++++++++++++++++++++++++++
>   hw/cpu/realview_mpcore.c       |  30 ++---
>   hw/arm/Kconfig                 |   8 +-
>   hw/cpu/Kconfig                 |   8 --
>   hw/cpu/meson.build             |   1 +
>   35 files changed, 689 insertions(+), 761 deletions(-)
>   delete mode 100644 include/hw/cpu/a15mpcore.h
>   delete mode 100644 include/hw/cpu/a9mpcore.h
>   create mode 100644 include/hw/cpu/cortex_mpcore.h
>   create mode 100644 hw/cpu/cortex_mpcore.c
>   delete mode 100644 hw/cpu/Kconfig
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

Hi Cédric,

On 2/1/24 15:55, Cédric Le Goater wrote:
> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>> Hi,
>>
>> When a MPCore cluster is used, the Cortex-A cores belong the the
>> cluster container, not to the board/soc layer. This series move
>> the creation of vCPUs to the MPCore private container.
>>
>> Doing so we consolidate the QOM model, moving common code in a
>> central place (abstract MPCore parent).
> 
> Changing the QOM hierarchy has an impact on the state of the machine
> and some fixups are then required to maintain migration compatibility.
> This can become a real headache for KVM machines like virt for which
> migration compatibility is a feature, less for emulated ones.

All changes are either moving properties (which are not migrated)
or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
is still migrated elsewhere). So I don't see any obvious migration
problem, but I might be missing something, so I Cc'ed Juan :>

> 
> I don't have a good solution to propose to overcome this problem :/
> 
> C.
> 
> 
>>
>> This eventually allow removing one qemu_get_cpu() use, which we
>> want to remove in heterogeneous machines (machines using MPCore
>> are candidate for heterogeneous emulation).


>>   include/hw/arm/aspeed_soc.h    |   5 +-
>>   include/hw/arm/boot.h          |   4 +-
>>   include/hw/arm/exynos4210.h    |   6 +-
>>   include/hw/arm/fsl-imx6.h      |   6 +-
>>   include/hw/arm/fsl-imx6ul.h    |   8 +-
>>   include/hw/arm/fsl-imx7.h      |   8 +-
>>   include/hw/arm/npcm7xx.h       |   3 +-
>>   include/hw/cpu/a15mpcore.h     |  44 -------
>>   include/hw/cpu/a9mpcore.h      |  39 -------
>>   include/hw/cpu/cortex_mpcore.h | 135 ++++++++++++++++++++++

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
> Hi Cédric,
> 
> On 2/1/24 15:55, Cédric Le Goater wrote:
>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>> Hi,
>>>
>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>> cluster container, not to the board/soc layer. This series move
>>> the creation of vCPUs to the MPCore private container.
>>>
>>> Doing so we consolidate the QOM model, moving common code in a
>>> central place (abstract MPCore parent).
>>
>> Changing the QOM hierarchy has an impact on the state of the machine
>> and some fixups are then required to maintain migration compatibility.
>> This can become a real headache for KVM machines like virt for which
>> migration compatibility is a feature, less for emulated ones.
> 
> All changes are either moving properties (which are not migrated)
> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
> is still migrated elsewhere). So I don't see any obvious migration
> problem, but I might be missing something, so I Cc'ed Juan :>
We broke migration compatibility by moving the IC object in the QOM
hierarchy of the pseries machines in the past. These changes might
be different. Here is the QOM tree of the ast2600 SoC.

before :

   /soc (ast2600-a3)
     /a7mpcore (a15mpcore_priv)
       /a15mp-priv-container[0] (memory-region)
       /gic (arm_gic)
         /gic_cpu[0] (memory-region)
         /gic_cpu[1] (memory-region)
         /gic_cpu[2] (memory-region)
         /gic_dist[0] (memory-region)
         /gic_vcpu[0] (memory-region)
         /gic_viface[0] (memory-region)
         /gic_viface[1] (memory-region)
         /gic_viface[2] (memory-region)
     /cpu[0] (cortex-a7-arm-cpu)
     /cpu[1] (cortex-a7-arm-cpu)

after :

   /soc (ast2600-a3)
     /a7mpcore (a7mpcore_priv)
       /cpu[0] (cortex-a7-arm-cpu)
       /cpu[1] (cortex-a7-arm-cpu)
       /gic (arm_gic)
         /gic_cpu[0] (memory-region)
         /gic_cpu[1] (memory-region)
         /gic_cpu[2] (memory-region)
         /gic_dist[0] (memory-region)
       /mpcore-priv-container[0] (memory-region)


Thanks,

C.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

+Peter/Fabiano

On 2/1/24 17:41, Cédric Le Goater wrote:
> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>> Hi Cédric,
>>
>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>> Hi,
>>>>
>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>> cluster container, not to the board/soc layer. This series move
>>>> the creation of vCPUs to the MPCore private container.
>>>>
>>>> Doing so we consolidate the QOM model, moving common code in a
>>>> central place (abstract MPCore parent).
>>>
>>> Changing the QOM hierarchy has an impact on the state of the machine
>>> and some fixups are then required to maintain migration compatibility.
>>> This can become a real headache for KVM machines like virt for which
>>> migration compatibility is a feature, less for emulated ones.
>>
>> All changes are either moving properties (which are not migrated)
>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>> is still migrated elsewhere). So I don't see any obvious migration
>> problem, but I might be missing something, so I Cc'ed Juan :>
> We broke migration compatibility by moving the IC object in the QOM
> hierarchy of the pseries machines in the past. These changes might
> be different. Here is the QOM tree of the ast2600 SoC.
> 
> before :
> 
>    /soc (ast2600-a3)
>      /a7mpcore (a15mpcore_priv)
>        /a15mp-priv-container[0] (memory-region)
>        /gic (arm_gic)
>          /gic_cpu[0] (memory-region)
>          /gic_cpu[1] (memory-region)
>          /gic_cpu[2] (memory-region)
>          /gic_dist[0] (memory-region)
>          /gic_vcpu[0] (memory-region)
>          /gic_viface[0] (memory-region)
>          /gic_viface[1] (memory-region)
>          /gic_viface[2] (memory-region)
>      /cpu[0] (cortex-a7-arm-cpu)
>      /cpu[1] (cortex-a7-arm-cpu)
> 
> after :
> 
>    /soc (ast2600-a3)
>      /a7mpcore (a7mpcore_priv)
>        /cpu[0] (cortex-a7-arm-cpu)
>        /cpu[1] (cortex-a7-arm-cpu)
>        /gic (arm_gic)
>          /gic_cpu[0] (memory-region)
>          /gic_cpu[1] (memory-region)
>          /gic_cpu[2] (memory-region)
>          /gic_dist[0] (memory-region)
>        /mpcore-priv-container[0] (memory-region)
> 
> 
> Thanks,
> 
> C.
> 
> 
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Fabiano Rosas]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> +Peter/Fabiano
>
> On 2/1/24 17:41, Cédric Le Goater wrote:
>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>> Hi Cédric,
>>>
>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>> Hi,
>>>>>
>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>> cluster container, not to the board/soc layer. This series move
>>>>> the creation of vCPUs to the MPCore private container.
>>>>>
>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>> central place (abstract MPCore parent).
>>>>
>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>> and some fixups are then required to maintain migration compatibility.
>>>> This can become a real headache for KVM machines like virt for which
>>>> migration compatibility is a feature, less for emulated ones.
>>>
>>> All changes are either moving properties (which are not migrated)
>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>> is still migrated elsewhere). So I don't see any obvious migration
>>> problem, but I might be missing something, so I Cc'ed Juan :>

FWIW, I didn't spot anything problematic either.

I've ran this through my migration compatibility series [1] and it
doesn't regress aarch64 migration from/to 8.2. The tests use '-M
virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
think we even support migration of anything non-KVM on arm.

1- https://gitlab.com/farosas/qemu/-/jobs/5853599533

>> We broke migration compatibility by moving the IC object in the QOM
>> hierarchy of the pseries machines in the past. These changes might
>> be different. Here is the QOM tree of the ast2600 SoC.
>> 
>> before :
>> 
>>    /soc (ast2600-a3)
>>      /a7mpcore (a15mpcore_priv)
>>        /a15mp-priv-container[0] (memory-region)
>>        /gic (arm_gic)
>>          /gic_cpu[0] (memory-region)
>>          /gic_cpu[1] (memory-region)
>>          /gic_cpu[2] (memory-region)
>>          /gic_dist[0] (memory-region)
>>          /gic_vcpu[0] (memory-region)
>>          /gic_viface[0] (memory-region)
>>          /gic_viface[1] (memory-region)
>>          /gic_viface[2] (memory-region)
>>      /cpu[0] (cortex-a7-arm-cpu)
>>      /cpu[1] (cortex-a7-arm-cpu)
>> 
>> after :
>> 
>>    /soc (ast2600-a3)
>>      /a7mpcore (a7mpcore_priv)
>>        /cpu[0] (cortex-a7-arm-cpu)
>>        /cpu[1] (cortex-a7-arm-cpu)
>>        /gic (arm_gic)
>>          /gic_cpu[0] (memory-region)
>>          /gic_cpu[1] (memory-region)
>>          /gic_cpu[2] (memory-region)
>>          /gic_dist[0] (memory-region)
>>        /mpcore-priv-container[0] (memory-region)
>> 
>> 
>> Thanks,
>> 
>> C.
>> 
>> 
>> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/3/24 20:53, Fabiano Rosas wrote:
> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> 
>> +Peter/Fabiano
>>
>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>> Hi Cédric,
>>>>
>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>> Hi,
>>>>>>
>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>
>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>> central place (abstract MPCore parent).
>>>>>
>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>> and some fixups are then required to maintain migration compatibility.
>>>>> This can become a real headache for KVM machines like virt for which
>>>>> migration compatibility is a feature, less for emulated ones.
>>>>
>>>> All changes are either moving properties (which are not migrated)
>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>> problem, but I might be missing something, so I Cc'ed Juan :>
> 
> FWIW, I didn't spot anything problematic either.
> 
> I've ran this through my migration compatibility series [1] and it
> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
> think we even support migration of anything non-KVM on arm.

it happens we do.

> 
> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533

yes it depends on the QOM hierarchy and virt seems immune to the changes.
Good.

However, changing the QOM topology clearly breaks migration compat,
at least on the Aspeed SoC. The question is : do we care ? For Aspeed
probably not, but the series should say so.

Thanks,

C.



  
> 
>>> We broke migration compatibility by moving the IC object in the QOM
>>> hierarchy of the pseries machines in the past. These changes might
>>> be different. Here is the QOM tree of the ast2600 SoC.
>>>
>>> before :
>>>
>>>     /soc (ast2600-a3)
>>>       /a7mpcore (a15mpcore_priv)
>>>         /a15mp-priv-container[0] (memory-region)
>>>         /gic (arm_gic)
>>>           /gic_cpu[0] (memory-region)
>>>           /gic_cpu[1] (memory-region)
>>>           /gic_cpu[2] (memory-region)
>>>           /gic_dist[0] (memory-region)
>>>           /gic_vcpu[0] (memory-region)
>>>           /gic_viface[0] (memory-region)
>>>           /gic_viface[1] (memory-region)
>>>           /gic_viface[2] (memory-region)
>>>       /cpu[0] (cortex-a7-arm-cpu)
>>>       /cpu[1] (cortex-a7-arm-cpu)
>>>
>>> after :
>>>
>>>     /soc (ast2600-a3)
>>>       /a7mpcore (a7mpcore_priv)
>>>         /cpu[0] (cortex-a7-arm-cpu)
>>>         /cpu[1] (cortex-a7-arm-cpu)
>>>         /gic (arm_gic)
>>>           /gic_cpu[0] (memory-region)
>>>           /gic_cpu[1] (memory-region)
>>>           /gic_cpu[2] (memory-region)
>>>           /gic_dist[0] (memory-region)
>>>         /mpcore-priv-container[0] (memory-region)
>>>
>>>
>>> Thanks,
>>>
>>> C.
>>>
>>>
>>>

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Fabiano Rosas]

Cédric Le Goater <clg@kaod.org> writes:

> On 1/3/24 20:53, Fabiano Rosas wrote:
>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>> 
>>> +Peter/Fabiano
>>>
>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>> Hi Cédric,
>>>>>
>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>
>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>> central place (abstract MPCore parent).
>>>>>>
>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>
>>>>> All changes are either moving properties (which are not migrated)
>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>> 
>> FWIW, I didn't spot anything problematic either.
>> 
>> I've ran this through my migration compatibility series [1] and it
>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>> think we even support migration of anything non-KVM on arm.
>
> it happens we do.
>

Oh, sorry, I didn't mean TCG here. Probably meant to say something like
non-KVM-capable cpus, as in 32-bit. Nevermind.

>> 
>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>
> yes it depends on the QOM hierarchy and virt seems immune to the changes.
> Good.
>
> However, changing the QOM topology clearly breaks migration compat,

Well, "clearly" is relative =) You've mentioned pseries and aspeed
already, do you have a pointer to one of those cases were we broke
migration so I could take a look?

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/9/24 18:40, Fabiano Rosas wrote:
> Cédric Le Goater <clg@kaod.org> writes:
> 
>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>
>>>> +Peter/Fabiano
>>>>
>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>> Hi Cédric,
>>>>>>
>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>
>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>> central place (abstract MPCore parent).
>>>>>>>
>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>
>>>>>> All changes are either moving properties (which are not migrated)
>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>
>>> FWIW, I didn't spot anything problematic either.
>>>
>>> I've ran this through my migration compatibility series [1] and it
>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>> think we even support migration of anything non-KVM on arm.
>>
>> it happens we do.
>>
> 
> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
> non-KVM-capable cpus, as in 32-bit. Nevermind.

Theoretically, we should be able to migrate to a TCG guest. Well, this
worked in the past for PPC. When I was doing more KVM related changes,
this was very useful for dev. Also, some machines are partially emulated.
Anyhow I agree this is not a strong requirement and we often break it.
Let's focus on KVM only.

>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>
>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>> Good.
>>
>> However, changing the QOM topology clearly breaks migration compat,
> 
> Well, "clearly" is relative =) You've mentioned pseries and aspeed
> already, do you have a pointer to one of those cases were we broke
> migration 

Regarding pseries, migration compat broke because of 5bc8d26de20c
("spapr: allocate the ICPState object from under sPAPRCPUCore") which
is similar to the changes proposed by this series, it impacts the QOM
hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
("spapr: fix migration of ICPState objects from/to older QEMU") which
is quite an headache and this turned out to raise another problem some
months ago ... :/ That's why I sent [1] to prepare removal of old
machines and workarounds becoming a burden.

Regarding aspeed, this series breaks compat. Not that we care much
but ​this caught my attention because of my past experience on pseries.
Same kind of QOM change which could impact other machines, like virt.
Since you checked that migration compat is preserved on virt, we should
be fine.

Thanks,

C.

[1] https://lore.kernel.org/qemu-devel/20231214181723.1520854-1-clg@kaod.org/

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Fabiano Rosas]

Cédric Le Goater <clg@kaod.org> writes:

> On 1/9/24 18:40, Fabiano Rosas wrote:
>> Cédric Le Goater <clg@kaod.org> writes:
>> 
>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>
>>>>> +Peter/Fabiano
>>>>>
>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>> Hi Cédric,
>>>>>>>
>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>
>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>
>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>
>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>
>>>> FWIW, I didn't spot anything problematic either.
>>>>
>>>> I've ran this through my migration compatibility series [1] and it
>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>> think we even support migration of anything non-KVM on arm.
>>>
>>> it happens we do.
>>>
>> 
>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>
> Theoretically, we should be able to migrate to a TCG guest. Well, this
> worked in the past for PPC. When I was doing more KVM related changes,
> this was very useful for dev. Also, some machines are partially emulated.
> Anyhow I agree this is not a strong requirement and we often break it.
> Let's focus on KVM only.
>
>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>
>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>> Good.
>>>
>>> However, changing the QOM topology clearly breaks migration compat,
>> 
>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>> already, do you have a pointer to one of those cases were we broke
>> migration 
>
> Regarding pseries, migration compat broke because of 5bc8d26de20c
> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
> is similar to the changes proposed by this series, it impacts the QOM
> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
> ("spapr: fix migration of ICPState objects from/to older QEMU") which
> is quite an headache and this turned out to raise another problem some
> months ago ... :/ That's why I sent [1] to prepare removal of old
> machines and workarounds becoming a burden.

This feels like something that could be handled by the vmstate code
somehow. The state is there, just under a different path. No one wants
to be policing QOM hierarchy changes in every single series that shows
up on the list.

Anyway, thanks for the pointers. I'll study that code a bit more, maybe
I can come up with some way to handle these cases.

Hopefully between the analyze-migration test and the compat tests we'll
catch the next bug of this kind before it gets merged.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

Hi Fabiano,

On 9/1/24 21:21, Fabiano Rosas wrote:
> Cédric Le Goater <clg@kaod.org> writes:
> 
>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>> Cédric Le Goater <clg@kaod.org> writes:
>>>
>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>
>>>>>> +Peter/Fabiano
>>>>>>
>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>> Hi Cédric,
>>>>>>>>
>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>
>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>
>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>
>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>
>>>>> FWIW, I didn't spot anything problematic either.
>>>>>
>>>>> I've ran this through my migration compatibility series [1] and it
>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>> think we even support migration of anything non-KVM on arm.
>>>>
>>>> it happens we do.
>>>>
>>>
>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>
>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>> worked in the past for PPC. When I was doing more KVM related changes,
>> this was very useful for dev. Also, some machines are partially emulated.
>> Anyhow I agree this is not a strong requirement and we often break it.
>> Let's focus on KVM only.
>>
>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>
>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>> Good.
>>>>
>>>> However, changing the QOM topology clearly breaks migration compat,
>>>
>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>> already, do you have a pointer to one of those cases were we broke
>>> migration
>>
>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>> is similar to the changes proposed by this series, it impacts the QOM
>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>> is quite an headache and this turned out to raise another problem some
>> months ago ... :/ That's why I sent [1] to prepare removal of old
>> machines and workarounds becoming a burden.
> 
> This feels like something that could be handled by the vmstate code
> somehow. The state is there, just under a different path.

What, the QOM path is used in migration? ...

See recent discussions on "QOM path stability":
https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/

> No one wants
> to be policing QOM hierarchy changes in every single series that shows
> up on the list.
> 
> Anyway, thanks for the pointers. I'll study that code a bit more, maybe
> I can come up with some way to handle these cases.
> 
> Hopefully between the analyze-migration test and the compat tests we'll
> catch the next bug of this kind before it gets merged.
> 
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Peter Xu]

On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
> Hi Fabiano,
> 
> On 9/1/24 21:21, Fabiano Rosas wrote:
> > Cédric Le Goater <clg@kaod.org> writes:
> > 
> > > On 1/9/24 18:40, Fabiano Rosas wrote:
> > > > Cédric Le Goater <clg@kaod.org> writes:
> > > > 
> > > > > On 1/3/24 20:53, Fabiano Rosas wrote:
> > > > > > Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> > > > > > 
> > > > > > > +Peter/Fabiano
> > > > > > > 
> > > > > > > On 2/1/24 17:41, Cédric Le Goater wrote:
> > > > > > > > On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
> > > > > > > > > Hi Cédric,
> > > > > > > > > 
> > > > > > > > > On 2/1/24 15:55, Cédric Le Goater wrote:
> > > > > > > > > > On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> > > > > > > > > > > Hi,
> > > > > > > > > > > 
> > > > > > > > > > > When a MPCore cluster is used, the Cortex-A cores belong the the
> > > > > > > > > > > cluster container, not to the board/soc layer. This series move
> > > > > > > > > > > the creation of vCPUs to the MPCore private container.
> > > > > > > > > > > 
> > > > > > > > > > > Doing so we consolidate the QOM model, moving common code in a
> > > > > > > > > > > central place (abstract MPCore parent).
> > > > > > > > > > 
> > > > > > > > > > Changing the QOM hierarchy has an impact on the state of the machine
> > > > > > > > > > and some fixups are then required to maintain migration compatibility.
> > > > > > > > > > This can become a real headache for KVM machines like virt for which
> > > > > > > > > > migration compatibility is a feature, less for emulated ones.
> > > > > > > > > 
> > > > > > > > > All changes are either moving properties (which are not migrated)
> > > > > > > > > or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
> > > > > > > > > is still migrated elsewhere). So I don't see any obvious migration
> > > > > > > > > problem, but I might be missing something, so I Cc'ed Juan :>
> > > > > > 
> > > > > > FWIW, I didn't spot anything problematic either.
> > > > > > 
> > > > > > I've ran this through my migration compatibility series [1] and it
> > > > > > doesn't regress aarch64 migration from/to 8.2. The tests use '-M
> > > > > > virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
> > > > > > think we even support migration of anything non-KVM on arm.
> > > > > 
> > > > > it happens we do.
> > > > > 
> > > > 
> > > > Oh, sorry, I didn't mean TCG here. Probably meant to say something like
> > > > non-KVM-capable cpus, as in 32-bit. Nevermind.
> > > 
> > > Theoretically, we should be able to migrate to a TCG guest. Well, this
> > > worked in the past for PPC. When I was doing more KVM related changes,
> > > this was very useful for dev. Also, some machines are partially emulated.
> > > Anyhow I agree this is not a strong requirement and we often break it.
> > > Let's focus on KVM only.
> > > 
> > > > > > 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
> > > > > 
> > > > > yes it depends on the QOM hierarchy and virt seems immune to the changes.
> > > > > Good.
> > > > > 
> > > > > However, changing the QOM topology clearly breaks migration compat,
> > > > 
> > > > Well, "clearly" is relative =) You've mentioned pseries and aspeed
> > > > already, do you have a pointer to one of those cases were we broke
> > > > migration
> > > 
> > > Regarding pseries, migration compat broke because of 5bc8d26de20c
> > > ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
> > > is similar to the changes proposed by this series, it impacts the QOM
> > > hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
> > > ("spapr: fix migration of ICPState objects from/to older QEMU") which
> > > is quite an headache and this turned out to raise another problem some
> > > months ago ... :/ That's why I sent [1] to prepare removal of old
> > > machines and workarounds becoming a burden.
> > 
> > This feels like something that could be handled by the vmstate code
> > somehow. The state is there, just under a different path.
> 
> What, the QOM path is used in migration? ...

Hopefully not..

> 
> See recent discussions on "QOM path stability":
> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/

If I read it right, the commit 46f7afa37096 example is pretty special that
the QOM path more or less decided more than the hierachy itself but changes
the existances of objects.

> 
> > No one wants
> > to be policing QOM hierarchy changes in every single series that shows
> > up on the list.
> > 
> > Anyway, thanks for the pointers. I'll study that code a bit more, maybe
> > I can come up with some way to handle these cases.
> > 
> > Hopefully between the analyze-migration test and the compat tests we'll
> > catch the next bug of this kind before it gets merged.

Things like that might be able to be detected via vmstate-static-checker.py.
But I'm not 100% sure, also its coverage is limited.

For example, I don't think it can detect changes to objects that will only
be created dynamically, e.g., I think sometimes we create objects after
some guest behaviors (consider guest enables the device, then QEMU
emulation creates some objects on demand of device setup?), and since the
static checker shouldn't ever start the VM and run any code, they won't
trigger.

-- 
Peter Xu

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>> Hi Fabiano,
>> 
>> On 9/1/24 21:21, Fabiano Rosas wrote:
>> > Cédric Le Goater <clg@kaod.org> writes:
>> > 
>> > > On 1/9/24 18:40, Fabiano Rosas wrote:
>> > > > Cédric Le Goater <clg@kaod.org> writes:
>> > > > 
>> > > > > On 1/3/24 20:53, Fabiano Rosas wrote:
>> > > > > > Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>> > > > > > 
>> > > > > > > +Peter/Fabiano
>> > > > > > > 
>> > > > > > > On 2/1/24 17:41, Cédric Le Goater wrote:
>> > > > > > > > On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>> > > > > > > > > Hi Cédric,
>> > > > > > > > > 
>> > > > > > > > > On 2/1/24 15:55, Cédric Le Goater wrote:
>> > > > > > > > > > On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>> > > > > > > > > > > Hi,
>> > > > > > > > > > > 
>> > > > > > > > > > > When a MPCore cluster is used, the Cortex-A cores belong the the
>> > > > > > > > > > > cluster container, not to the board/soc layer. This series move
>> > > > > > > > > > > the creation of vCPUs to the MPCore private container.
>> > > > > > > > > > > 
>> > > > > > > > > > > Doing so we consolidate the QOM model, moving common code in a
>> > > > > > > > > > > central place (abstract MPCore parent).
>> > > > > > > > > > 
>> > > > > > > > > > Changing the QOM hierarchy has an impact on the state of the machine
>> > > > > > > > > > and some fixups are then required to maintain migration compatibility.
>> > > > > > > > > > This can become a real headache for KVM machines like virt for which
>> > > > > > > > > > migration compatibility is a feature, less for emulated ones.
>> > > > > > > > > 
>> > > > > > > > > All changes are either moving properties (which are not migrated)
>> > > > > > > > > or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>> > > > > > > > > is still migrated elsewhere). So I don't see any obvious migration
>> > > > > > > > > problem, but I might be missing something, so I Cc'ed Juan :>
>> > > > > > 
>> > > > > > FWIW, I didn't spot anything problematic either.
>> > > > > > 
>> > > > > > I've ran this through my migration compatibility series [1] and it
>> > > > > > doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>> > > > > > virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>> > > > > > think we even support migration of anything non-KVM on arm.
>> > > > > 
>> > > > > it happens we do.
>> > > > > 
>> > > > 
>> > > > Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>> > > > non-KVM-capable cpus, as in 32-bit. Nevermind.
>> > > 
>> > > Theoretically, we should be able to migrate to a TCG guest. Well, this
>> > > worked in the past for PPC. When I was doing more KVM related changes,
>> > > this was very useful for dev. Also, some machines are partially emulated.
>> > > Anyhow I agree this is not a strong requirement and we often break it.
>> > > Let's focus on KVM only.
>> > > 
>> > > > > > 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>> > > > > 
>> > > > > yes it depends on the QOM hierarchy and virt seems immune to the changes.
>> > > > > Good.
>> > > > > 
>> > > > > However, changing the QOM topology clearly breaks migration compat,
>> > > > 
>> > > > Well, "clearly" is relative =) You've mentioned pseries and aspeed
>> > > > already, do you have a pointer to one of those cases were we broke
>> > > > migration
>> > > 
>> > > Regarding pseries, migration compat broke because of 5bc8d26de20c
>> > > ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>> > > is similar to the changes proposed by this series, it impacts the QOM
>> > > hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>> > > ("spapr: fix migration of ICPState objects from/to older QEMU") which
>> > > is quite an headache and this turned out to raise another problem some
>> > > months ago ... :/ That's why I sent [1] to prepare removal of old
>> > > machines and workarounds becoming a burden.
>> > 
>> > This feels like something that could be handled by the vmstate code
>> > somehow. The state is there, just under a different path.
>> 
>> What, the QOM path is used in migration? ...
>
> Hopefully not..
>
>> 
>> See recent discussions on "QOM path stability":
>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>
> If I read it right, the commit 46f7afa37096 example is pretty special that
> the QOM path more or less decided more than the hierachy itself but changes
> the existances of objects.

Let's see whether I got this...

We removed some useless objects, moved the useful ones to another home.
The move changed their QOM path.

The problem was the removal of useless objects, because this also
removed their vmstate.

The fix was adding the vmstate back as a dummy.

The QOM patch changes are *not* part of the problem.

Correct?

>> > No one wants
>> > to be policing QOM hierarchy changes in every single series that shows
>> > up on the list.
>> > 
>> > Anyway, thanks for the pointers. I'll study that code a bit more, maybe
>> > I can come up with some way to handle these cases.
>> > 
>> > Hopefully between the analyze-migration test and the compat tests we'll
>> > catch the next bug of this kind before it gets merged.
>
> Things like that might be able to be detected via vmstate-static-checker.py.
> But I'm not 100% sure, also its coverage is limited.
>
> For example, I don't think it can detect changes to objects that will only
> be created dynamically, e.g., I think sometimes we create objects after
> some guest behaviors (consider guest enables the device, then QEMU
> emulation creates some objects on demand of device setup?),

Feels nuts to me.

In real hardware, software enabling a device that is disabled by default
doesn't create the device.  The device is always there, it just happens
to be inactive unless enabled.  We should model the device just like
that.

>                                                             and since the
> static checker shouldn't ever start the VM and run any code, they won't
> trigger.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Peter Xu]

On Wed, Jan 10, 2024 at 07:03:06AM +0100, Markus Armbruster wrote:
> Peter Xu <peterx@redhat.com> writes:
> 
> > On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
> >> Hi Fabiano,
> >> 
> >> On 9/1/24 21:21, Fabiano Rosas wrote:
> >> > Cédric Le Goater <clg@kaod.org> writes:
> >> > 
> >> > > On 1/9/24 18:40, Fabiano Rosas wrote:
> >> > > > Cédric Le Goater <clg@kaod.org> writes:
> >> > > > 
> >> > > > > On 1/3/24 20:53, Fabiano Rosas wrote:
> >> > > > > > Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> >> > > > > > 
> >> > > > > > > +Peter/Fabiano
> >> > > > > > > 
> >> > > > > > > On 2/1/24 17:41, Cédric Le Goater wrote:
> >> > > > > > > > On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
> >> > > > > > > > > Hi Cédric,
> >> > > > > > > > > 
> >> > > > > > > > > On 2/1/24 15:55, Cédric Le Goater wrote:
> >> > > > > > > > > > On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
> >> > > > > > > > > > > Hi,
> >> > > > > > > > > > > 
> >> > > > > > > > > > > When a MPCore cluster is used, the Cortex-A cores belong the the
> >> > > > > > > > > > > cluster container, not to the board/soc layer. This series move
> >> > > > > > > > > > > the creation of vCPUs to the MPCore private container.
> >> > > > > > > > > > > 
> >> > > > > > > > > > > Doing so we consolidate the QOM model, moving common code in a
> >> > > > > > > > > > > central place (abstract MPCore parent).
> >> > > > > > > > > > 
> >> > > > > > > > > > Changing the QOM hierarchy has an impact on the state of the machine
> >> > > > > > > > > > and some fixups are then required to maintain migration compatibility.
> >> > > > > > > > > > This can become a real headache for KVM machines like virt for which
> >> > > > > > > > > > migration compatibility is a feature, less for emulated ones.
> >> > > > > > > > > 
> >> > > > > > > > > All changes are either moving properties (which are not migrated)
> >> > > > > > > > > or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
> >> > > > > > > > > is still migrated elsewhere). So I don't see any obvious migration
> >> > > > > > > > > problem, but I might be missing something, so I Cc'ed Juan :>
> >> > > > > > 
> >> > > > > > FWIW, I didn't spot anything problematic either.
> >> > > > > > 
> >> > > > > > I've ran this through my migration compatibility series [1] and it
> >> > > > > > doesn't regress aarch64 migration from/to 8.2. The tests use '-M
> >> > > > > > virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
> >> > > > > > think we even support migration of anything non-KVM on arm.
> >> > > > > 
> >> > > > > it happens we do.
> >> > > > > 
> >> > > > 
> >> > > > Oh, sorry, I didn't mean TCG here. Probably meant to say something like
> >> > > > non-KVM-capable cpus, as in 32-bit. Nevermind.
> >> > > 
> >> > > Theoretically, we should be able to migrate to a TCG guest. Well, this
> >> > > worked in the past for PPC. When I was doing more KVM related changes,
> >> > > this was very useful for dev. Also, some machines are partially emulated.
> >> > > Anyhow I agree this is not a strong requirement and we often break it.
> >> > > Let's focus on KVM only.
> >> > > 
> >> > > > > > 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
> >> > > > > 
> >> > > > > yes it depends on the QOM hierarchy and virt seems immune to the changes.
> >> > > > > Good.
> >> > > > > 
> >> > > > > However, changing the QOM topology clearly breaks migration compat,
> >> > > > 
> >> > > > Well, "clearly" is relative =) You've mentioned pseries and aspeed
> >> > > > already, do you have a pointer to one of those cases were we broke
> >> > > > migration
> >> > > 
> >> > > Regarding pseries, migration compat broke because of 5bc8d26de20c
> >> > > ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
> >> > > is similar to the changes proposed by this series, it impacts the QOM
> >> > > hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
> >> > > ("spapr: fix migration of ICPState objects from/to older QEMU") which
> >> > > is quite an headache and this turned out to raise another problem some
> >> > > months ago ... :/ That's why I sent [1] to prepare removal of old
> >> > > machines and workarounds becoming a burden.
> >> > 
> >> > This feels like something that could be handled by the vmstate code
> >> > somehow. The state is there, just under a different path.
> >> 
> >> What, the QOM path is used in migration? ...
> >
> > Hopefully not..
> >
> >> 
> >> See recent discussions on "QOM path stability":
> >> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
> >> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
> >> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
> >
> > If I read it right, the commit 46f7afa37096 example is pretty special that
> > the QOM path more or less decided more than the hierachy itself but changes
> > the existances of objects.
> 
> Let's see whether I got this...
> 
> We removed some useless objects, moved the useful ones to another home.
> The move changed their QOM path.
> 
> The problem was the removal of useless objects, because this also
> removed their vmstate.
> 
> The fix was adding the vmstate back as a dummy.
> 
> The QOM patch changes are *not* part of the problem.
> 
> Correct?

[I'd leave this to Cedric]

> 
> >> > No one wants
> >> > to be policing QOM hierarchy changes in every single series that shows
> >> > up on the list.
> >> > 
> >> > Anyway, thanks for the pointers. I'll study that code a bit more, maybe
> >> > I can come up with some way to handle these cases.
> >> > 
> >> > Hopefully between the analyze-migration test and the compat tests we'll
> >> > catch the next bug of this kind before it gets merged.
> >
> > Things like that might be able to be detected via vmstate-static-checker.py.
> > But I'm not 100% sure, also its coverage is limited.
> >
> > For example, I don't think it can detect changes to objects that will only
> > be created dynamically, e.g., I think sometimes we create objects after
> > some guest behaviors (consider guest enables the device, then QEMU
> > emulation creates some objects on demand of device setup?),
> 
> Feels nuts to me.
> 
> In real hardware, software enabling a device that is disabled by default
> doesn't create the device.  The device is always there, it just happens
> to be inactive unless enabled.  We should model the device just like
> that.

It doesn't need to be the device itself to be dynamically created, but some
other sub-objects that do not require to exist until the device is enabled,
or some specific function of that device is enabled.  It is logically doable.

Is the example Cedric provided looks like some case like this?  I am not
sure, that's also why I'm not sure the static checker would work here.  But
logically it seems possible, e.g. with migration VMSD needed() facilities.
Consider a device has a sub-function that requires a sub-object.  It may
not need to migrate that object if that sub-feature is not even enabled.
If that object is very large, it might be wise to do so if possible to not
send chunks of junk during the VM downtime.

But then after a 2nd thought I do agree it's probably not sensible, because
even if the src may know whether the sub-object will be needed, there's
probably no good way for the dest QEMU to know.  It can only know in
something like a post_load() hook, but logically that can happen only after
a full load of that device state, so might already be too late.

Thanks,

-- 
Peter Xu

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Wed, Jan 10, 2024 at 07:03:06AM +0100, Markus Armbruster wrote:
>> Peter Xu <peterx@redhat.com> writes:
>> 
>> > On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>> >> Hi Fabiano,
>> >> 
>> >> On 9/1/24 21:21, Fabiano Rosas wrote:

[...]

>> >> > No one wants
>> >> > to be policing QOM hierarchy changes in every single series that shows
>> >> > up on the list.
>> >> > 
>> >> > Anyway, thanks for the pointers. I'll study that code a bit more, maybe
>> >> > I can come up with some way to handle these cases.
>> >> > 
>> >> > Hopefully between the analyze-migration test and the compat tests we'll
>> >> > catch the next bug of this kind before it gets merged.
>> >
>> > Things like that might be able to be detected via vmstate-static-checker.py.
>> > But I'm not 100% sure, also its coverage is limited.
>> >
>> > For example, I don't think it can detect changes to objects that will only
>> > be created dynamically, e.g., I think sometimes we create objects after
>> > some guest behaviors (consider guest enables the device, then QEMU
>> > emulation creates some objects on demand of device setup?),
>> 
>> Feels nuts to me.
>> 
>> In real hardware, software enabling a device that is disabled by default
>> doesn't create the device.  The device is always there, it just happens
>> to be inactive unless enabled.  We should model the device just like
>> that.
>
> It doesn't need to be the device itself to be dynamically created, but some
> other sub-objects that do not require to exist until the device is enabled,
> or some specific function of that device is enabled.  It is logically doable.
>
> Is the example Cedric provided looks like some case like this?  I am not
> sure, that's also why I'm not sure the static checker would work here.  But
> logically it seems possible, e.g. with migration VMSD needed() facilities.
> Consider a device has a sub-function that requires a sub-object.  It may
> not need to migrate that object if that sub-feature is not even enabled.
> If that object is very large, it might be wise to do so if possible to not
> send chunks of junk during the VM downtime.
>
> But then after a 2nd thought I do agree it's probably not sensible, because
> even if the src may know whether the sub-object will be needed, there's
> probably no good way for the dest QEMU to know.  It can only know in
> something like a post_load() hook, but logically that can happen only after
> a full load of that device state, so might already be too late.
>
> Thanks,

If an object has state that needs to be migrated only sometimes, and
that part of the state is large enough to bother, we can put it in an
optional subsection, can't we?

Destination: if present, take it.  If absent, initialize to default.

Source: send unless (known to be) in default state.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Peter Xu]

On Wed, Jan 10, 2024 at 09:09:51AM +0100, Markus Armbruster wrote:
> If an object has state that needs to be migrated only sometimes, and
> that part of the state is large enough to bother, we can put it in an
> optional subsection, can't we?
> 
> Destination: if present, take it.  If absent, initialize to default.
> 
> Source: send unless (known to be) in default state.

Hmm.. correct. I think I messed up VMSD's needed() hook with field_exists()
of the fields; my apologies.

The trick should be that VMSD's subsections is more flexible, due to the
fact that vmstate_subsection_load() has:

    while (qemu_peek_byte(f, 0) == QEMU_VM_SUBSECTION) {
        ...
        sub_vmsd = vmstate_get_subsection(vmsd->subsections, idstr);
        ...
    }

So it tolerates the case where the subsection doesn't exist, or out of
order subsections.

While field_exists() hook seems not that flexible, as it's implemented as
this in vmstate_load_state():

    while (field->name) {
        ...
        if (vmstate_field_exists(vmsd, field, opaque, version_id)) {
           ...
        }
        ...
        field++;
    }

So that vmstate_field_exists() needs to be known even on dest before the
main vmsd got loaded, and it should always return the same value as the
source.  Also, the field has ordering requirements.

Then yes, subsection should work for dynamic objects.

Thanks,

-- 
Peter Xu

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/10/24 07:26, Peter Xu wrote:
> On Wed, Jan 10, 2024 at 07:03:06AM +0100, Markus Armbruster wrote:
>> Peter Xu <peterx@redhat.com> writes:
>>
>>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>>> Hi Fabiano,
>>>>
>>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>
>>>>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>>
>>>>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>>>>
>>>>>>>>>> +Peter/Fabiano
>>>>>>>>>>
>>>>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>> Hi Cédric,
>>>>>>>>>>>>
>>>>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>>>>
>>>>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>>>>
>>>>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>>>>
>>>>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>>>>
>>>>>>>> it happens we do.
>>>>>>>>
>>>>>>>
>>>>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>>>>
>>>>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>>>> worked in the past for PPC. When I was doing more KVM related changes,
>>>>>> this was very useful for dev. Also, some machines are partially emulated.
>>>>>> Anyhow I agree this is not a strong requirement and we often break it.
>>>>>> Let's focus on KVM only.
>>>>>>
>>>>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>>>>
>>>>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>>>>> Good.
>>>>>>>>
>>>>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>>>>
>>>>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>>>>> already, do you have a pointer to one of those cases were we broke
>>>>>>> migration
>>>>>>
>>>>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>>>> is similar to the changes proposed by this series, it impacts the QOM
>>>>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>>>> is quite an headache and this turned out to raise another problem some
>>>>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>>>>> machines and workarounds becoming a burden.
>>>>>
>>>>> This feels like something that could be handled by the vmstate code
>>>>> somehow. The state is there, just under a different path.
>>>>
>>>> What, the QOM path is used in migration? ...
>>>
>>> Hopefully not..
>>>
>>>>
>>>> See recent discussions on "QOM path stability":
>>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>>
>>> If I read it right, the commit 46f7afa37096 example is pretty special that
>>> the QOM path more or less decided more than the hierachy itself but changes
>>> the existances of objects.
>>
>> Let's see whether I got this...
>>
>> We removed some useless objects, moved the useful ones to another home.
>> The move changed their QOM path.
>>
>> The problem was the removal of useless objects, because this also
>> removed their vmstate.
>>
>> The fix was adding the vmstate back as a dummy.
>>
>> The QOM patch changes are *not* part of the problem.
>>
>> Correct?
> 
> [I'd leave this to Cedric]
> 
>>
>>>>> No one wants
>>>>> to be policing QOM hierarchy changes in every single series that shows
>>>>> up on the list.
>>>>>
>>>>> Anyway, thanks for the pointers. I'll study that code a bit more, maybe
>>>>> I can come up with some way to handle these cases.
>>>>>
>>>>> Hopefully between the analyze-migration test and the compat tests we'll
>>>>> catch the next bug of this kind before it gets merged.
>>>
>>> Things like that might be able to be detected via vmstate-static-checker.py.
>>> But I'm not 100% sure, also its coverage is limited.
>>>
>>> For example, I don't think it can detect changes to objects that will only
>>> be created dynamically, e.g., I think sometimes we create objects after
>>> some guest behaviors (consider guest enables the device, then QEMU
>>> emulation creates some objects on demand of device setup?),
>>
>> Feels nuts to me.
>>
>> In real hardware, software enabling a device that is disabled by default
>> doesn't create the device.  The device is always there, it just happens
>> to be inactive unless enabled.  We should model the device just like
>> that.
> 
> It doesn't need to be the device itself to be dynamically created, but some
> other sub-objects that do not require to exist until the device is enabled,
> or some specific function of that device is enabled.  It is logically doable.
> 
> Is the example Cedric provided looks like some case like this?  I am not
> sure, that's also why I'm not sure the static checker would work here.  But
> logically it seems possible, e.g. with migration VMSD needed() facilities.
> Consider a device has a sub-function that requires a sub-object.  It may
> not need to migrate that object if that sub-feature is not even enabled.
> If that object is very large, it might be wise to do so if possible to not
> send chunks of junk during the VM downtime.
> 
> But then after a 2nd thought I do agree it's probably not sensible, because
> even if the src may know whether the sub-object will be needed, there's
> probably no good way for the dest QEMU to know.  It can only know in
> something like a post_load() hook, but logically that can happen only after
> a full load of that device state, so might already be too late.

If features can be dynamically enabled by the OS, after negotiation
with the platform, the source should prepare for all possible
scenarios and migrate all models supported for a given configuration.
The OS could choose to enable a feature on the target which was not
enabled on the source before migration.

Thanks,

C.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Fabiano Rosas]

Markus Armbruster <armbru@redhat.com> writes:

> Peter Xu <peterx@redhat.com> writes:
>
>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>> Hi Fabiano,
>>> 
>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>> > Cédric Le Goater <clg@kaod.org> writes:
>>> > 
>>> > > On 1/9/24 18:40, Fabiano Rosas wrote:
>>> > > > Cédric Le Goater <clg@kaod.org> writes:
>>> > > > 
>>> > > > > On 1/3/24 20:53, Fabiano Rosas wrote:
>>> > > > > > Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>> > > > > > 
>>> > > > > > > +Peter/Fabiano
>>> > > > > > > 
>>> > > > > > > On 2/1/24 17:41, Cédric Le Goater wrote:
>>> > > > > > > > On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>> > > > > > > > > Hi Cédric,
>>> > > > > > > > > 
>>> > > > > > > > > On 2/1/24 15:55, Cédric Le Goater wrote:
>>> > > > > > > > > > On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>> > > > > > > > > > > Hi,
>>> > > > > > > > > > > 
>>> > > > > > > > > > > When a MPCore cluster is used, the Cortex-A cores belong the the
>>> > > > > > > > > > > cluster container, not to the board/soc layer. This series move
>>> > > > > > > > > > > the creation of vCPUs to the MPCore private container.
>>> > > > > > > > > > > 
>>> > > > > > > > > > > Doing so we consolidate the QOM model, moving common code in a
>>> > > > > > > > > > > central place (abstract MPCore parent).
>>> > > > > > > > > > 
>>> > > > > > > > > > Changing the QOM hierarchy has an impact on the state of the machine
>>> > > > > > > > > > and some fixups are then required to maintain migration compatibility.
>>> > > > > > > > > > This can become a real headache for KVM machines like virt for which
>>> > > > > > > > > > migration compatibility is a feature, less for emulated ones.
>>> > > > > > > > > 
>>> > > > > > > > > All changes are either moving properties (which are not migrated)
>>> > > > > > > > > or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>> > > > > > > > > is still migrated elsewhere). So I don't see any obvious migration
>>> > > > > > > > > problem, but I might be missing something, so I Cc'ed Juan :>
>>> > > > > > 
>>> > > > > > FWIW, I didn't spot anything problematic either.
>>> > > > > > 
>>> > > > > > I've ran this through my migration compatibility series [1] and it
>>> > > > > > doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>> > > > > > virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>> > > > > > think we even support migration of anything non-KVM on arm.
>>> > > > > 
>>> > > > > it happens we do.
>>> > > > > 
>>> > > > 
>>> > > > Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>> > > > non-KVM-capable cpus, as in 32-bit. Nevermind.
>>> > > 
>>> > > Theoretically, we should be able to migrate to a TCG guest. Well, this
>>> > > worked in the past for PPC. When I was doing more KVM related changes,
>>> > > this was very useful for dev. Also, some machines are partially emulated.
>>> > > Anyhow I agree this is not a strong requirement and we often break it.
>>> > > Let's focus on KVM only.
>>> > > 
>>> > > > > > 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>> > > > > 
>>> > > > > yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>> > > > > Good.
>>> > > > > 
>>> > > > > However, changing the QOM topology clearly breaks migration compat,
>>> > > > 
>>> > > > Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>> > > > already, do you have a pointer to one of those cases were we broke
>>> > > > migration
>>> > > 
>>> > > Regarding pseries, migration compat broke because of 5bc8d26de20c
>>> > > ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>> > > is similar to the changes proposed by this series, it impacts the QOM
>>> > > hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>> > > ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>> > > is quite an headache and this turned out to raise another problem some
>>> > > months ago ... :/ That's why I sent [1] to prepare removal of old
>>> > > machines and workarounds becoming a burden.
>>> > 
>>> > This feels like something that could be handled by the vmstate code
>>> > somehow. The state is there, just under a different path.
>>> 
>>> What, the QOM path is used in migration? ...
>>
>> Hopefully not..

Unfortunately the original fix doesn't mention _what_ actually broke
with migration. I assumed the QOM path was needed because otherwise I
don't think the fix makes sense. The thread discussing that patch also
directly mentions the QOM path:

https://www.mail-archive.com/qemu-devel@nongnu.org/msg450912.html

But I probably misunderstood something while reading that thread.

>>
>>> 
>>> See recent discussions on "QOM path stability":
>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>
>> If I read it right, the commit 46f7afa37096 example is pretty special that
>> the QOM path more or less decided more than the hierachy itself but changes
>> the existances of objects.
>
> Let's see whether I got this...
>
> We removed some useless objects, moved the useful ones to another home.
> The move changed their QOM path.
>
> The problem was the removal of useless objects, because this also
> removed their vmstate.

If you checkout at the removal commit (5bc8d26de20c), the vmstate has
been kept untouched.

>
> The fix was adding the vmstate back as a dummy.

Since the vmstate was kept I don't see why would we need a dummy. The
incoming migration stream would still have the state, only at a
different point in the stream. It's surprising to me that that would
cause an issue, but I'm not well versed in that code.

>
> The QOM patch changes are *not* part of the problem.

The only explanation I can come up with is that after the patch
migration has broken after a hotplug or similar operation. In such
situation, the preallocated state would always be present before the
patch, but sometimes not present after the patch in case, say, a
hot-unplug has taken away a cpu + ICPState.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Markus Armbruster]

Fabiano Rosas <farosas@suse.de> writes:

> Markus Armbruster <armbru@redhat.com> writes:
>
>> Peter Xu <peterx@redhat.com> writes:
>>
>>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>>> Hi Fabiano,
>>>> 
>>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>> > Cédric Le Goater <clg@kaod.org> writes:
>>>> > 
>>>> > > On 1/9/24 18:40, Fabiano Rosas wrote:
>>>> > > > Cédric Le Goater <clg@kaod.org> writes:
>>>> > > > 
>>>> > > > > On 1/3/24 20:53, Fabiano Rosas wrote:
>>>> > > > > > Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>> > > > > > 
>>>> > > > > > > +Peter/Fabiano
>>>> > > > > > > 
>>>> > > > > > > On 2/1/24 17:41, Cédric Le Goater wrote:
>>>> > > > > > > > On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>> > > > > > > > > Hi Cédric,
>>>> > > > > > > > > 
>>>> > > > > > > > > On 2/1/24 15:55, Cédric Le Goater wrote:
>>>> > > > > > > > > > On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>> > > > > > > > > > > Hi,
>>>> > > > > > > > > > > 
>>>> > > > > > > > > > > When a MPCore cluster is used, the Cortex-A cores belong the the
>>>> > > > > > > > > > > cluster container, not to the board/soc layer. This series move
>>>> > > > > > > > > > > the creation of vCPUs to the MPCore private container.
>>>> > > > > > > > > > > 
>>>> > > > > > > > > > > Doing so we consolidate the QOM model, moving common code in a
>>>> > > > > > > > > > > central place (abstract MPCore parent).
>>>> > > > > > > > > > 
>>>> > > > > > > > > > Changing the QOM hierarchy has an impact on the state of the machine
>>>> > > > > > > > > > and some fixups are then required to maintain migration compatibility.
>>>> > > > > > > > > > This can become a real headache for KVM machines like virt for which
>>>> > > > > > > > > > migration compatibility is a feature, less for emulated ones.
>>>> > > > > > > > > 
>>>> > > > > > > > > All changes are either moving properties (which are not migrated)
>>>> > > > > > > > > or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>> > > > > > > > > is still migrated elsewhere). So I don't see any obvious migration
>>>> > > > > > > > > problem, but I might be missing something, so I Cc'ed Juan :>
>>>> > > > > > 
>>>> > > > > > FWIW, I didn't spot anything problematic either.
>>>> > > > > > 
>>>> > > > > > I've ran this through my migration compatibility series [1] and it
>>>> > > > > > doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>> > > > > > virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>> > > > > > think we even support migration of anything non-KVM on arm.
>>>> > > > > 
>>>> > > > > it happens we do.
>>>> > > > > 
>>>> > > > 
>>>> > > > Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>> > > > non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>> > > 
>>>> > > Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>> > > worked in the past for PPC. When I was doing more KVM related changes,
>>>> > > this was very useful for dev. Also, some machines are partially emulated.
>>>> > > Anyhow I agree this is not a strong requirement and we often break it.
>>>> > > Let's focus on KVM only.
>>>> > > 
>>>> > > > > > 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>> > > > > 
>>>> > > > > yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>> > > > > Good.
>>>> > > > > 
>>>> > > > > However, changing the QOM topology clearly breaks migration compat,
>>>> > > > 
>>>> > > > Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>> > > > already, do you have a pointer to one of those cases were we broke
>>>> > > > migration
>>>> > > 
>>>> > > Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>> > > ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>> > > is similar to the changes proposed by this series, it impacts the QOM
>>>> > > hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>> > > ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>> > > is quite an headache and this turned out to raise another problem some
>>>> > > months ago ... :/ That's why I sent [1] to prepare removal of old
>>>> > > machines and workarounds becoming a burden.
>>>> > 
>>>> > This feels like something that could be handled by the vmstate code
>>>> > somehow. The state is there, just under a different path.
>>>> 
>>>> What, the QOM path is used in migration? ...
>>>
>>> Hopefully not..
>
> Unfortunately the original fix doesn't mention _what_ actually broke
> with migration. I assumed the QOM path was needed because otherwise I
> don't think the fix makes sense. The thread discussing that patch also
> directly mentions the QOM path:
>
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg450912.html
>
> But I probably misunderstood something while reading that thread.
>
>>>
>>>> 
>>>> See recent discussions on "QOM path stability":
>>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>>
>>> If I read it right, the commit 46f7afa37096 example is pretty special that
>>> the QOM path more or less decided more than the hierachy itself but changes
>>> the existances of objects.
>>
>> Let's see whether I got this...
>>
>> We removed some useless objects, moved the useful ones to another home.
>> The move changed their QOM path.
>>
>> The problem was the removal of useless objects, because this also
>> removed their vmstate.
>
> If you checkout at the removal commit (5bc8d26de20c), the vmstate has
> been kept untouched.
>
>>
>> The fix was adding the vmstate back as a dummy.
>
> Since the vmstate was kept I don't see why would we need a dummy. The
> incoming migration stream would still have the state, only at a
> different point in the stream. It's surprising to me that that would
> cause an issue, but I'm not well versed in that code.

Alright, I understand neither the problem nor the fix :)

>> The QOM patch changes are *not* part of the problem.
>
> The only explanation I can come up with is that after the patch
> migration has broken after a hotplug or similar operation. In such
> situation, the preallocated state would always be present before the
> patch, but sometimes not present after the patch in case, say, a
> hot-unplug has taken away a cpu + ICPState.

My head hurts...  Oh, we're talking migration!  Perfectly normal then.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/10/24 14:19, Fabiano Rosas wrote:
> Markus Armbruster <armbru@redhat.com> writes:
> 
>> Peter Xu <peterx@redhat.com> writes:
>>
>>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>>> Hi Fabiano,
>>>>
>>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>
>>>>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>>
>>>>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>>>>
>>>>>>>>>> +Peter/Fabiano
>>>>>>>>>>
>>>>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>> Hi Cédric,
>>>>>>>>>>>>
>>>>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>>>>
>>>>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>>>>
>>>>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>>>>
>>>>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>>>>
>>>>>>>> it happens we do.
>>>>>>>>
>>>>>>>
>>>>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>>>>
>>>>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>>>> worked in the past for PPC. When I was doing more KVM related changes,
>>>>>> this was very useful for dev. Also, some machines are partially emulated.
>>>>>> Anyhow I agree this is not a strong requirement and we often break it.
>>>>>> Let's focus on KVM only.
>>>>>>
>>>>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>>>>
>>>>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>>>>> Good.
>>>>>>>>
>>>>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>>>>
>>>>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>>>>> already, do you have a pointer to one of those cases were we broke
>>>>>>> migration
>>>>>>
>>>>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>>>> is similar to the changes proposed by this series, it impacts the QOM
>>>>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>>>> is quite an headache and this turned out to raise another problem some
>>>>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>>>>> machines and workarounds becoming a burden.
>>>>>
>>>>> This feels like something that could be handled by the vmstate code
>>>>> somehow. The state is there, just under a different path.
>>>>
>>>> What, the QOM path is used in migration? ...
>>>
>>> Hopefully not..
> 
> Unfortunately the original fix doesn't mention _what_ actually broke
> with migration. I assumed the QOM path was needed because otherwise I
> don't think the fix makes sense. The thread discussing that patch also
> directly mentions the QOM path:
> 
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg450912.html
> 
> But I probably misunderstood something while reading that thread.
> 
>>>
>>>>
>>>> See recent discussions on "QOM path stability":
>>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>>
>>> If I read it right, the commit 46f7afa37096 example is pretty special that
>>> the QOM path more or less decided more than the hierachy itself but changes
>>> the existances of objects.
>>
>> Let's see whether I got this...
>>
>> We removed some useless objects, moved the useful ones to another home.
>> The move changed their QOM path.
>>
>> The problem was the removal of useless objects, because this also
>> removed their vmstate.
> 
> If you checkout at the removal commit (5bc8d26de20c), the vmstate has
> been kept untouched.
> 
>>
>> The fix was adding the vmstate back as a dummy.
> 
> Since the vmstate was kept I don't see why would we need a dummy. The
> incoming migration stream would still have the state, only at a
> different point in the stream. It's surprising to me that that would
> cause an issue, but I'm not well versed in that code.
> 
>>
>> The QOM patch changes are *not* part of the problem.
> 
> The only explanation I can come up with is that after the patch
> migration has broken after a hotplug or similar operation. In such
> situation, the preallocated state would always be present before the
> patch, but sometimes not present after the patch in case, say, a
> hot-unplug has taken away a cpu + ICPState.

May be. It has been a while.

For a better understanding, I tried to reproduce. QEMU 2.9 still
compiles on a f38 (memfd_create needs a fix). However, a
QEMU-2.9/pseries-2.9 machine won't start any recent OS because
something broke the OS/platform negotiation process (CAS) ...

I removed the pre_2_10_has_unused_icps hack and worked around
another migration compat issue in IOMMU ... Here are the last
trace-events for migration :

     QEMU-mainline/pseries-2.9 -> QEMU-2.9/pseries-2.9

1016464@1705053752.106417:vmstate_load spapr, spapr
1016464@1705053752.106419:vmstate_load_state spapr v3
1016464@1705053752.106421:vmstate_load_state_field spapr:unused
1016464@1705053752.106424:vmstate_load_state_field spapr:rtc_offset
1016464@1705053752.106425:vmstate_load_state_field spapr:tb
1016464@1705053752.106427:vmstate_n_elems tb: 1
1016464@1705053752.106429:vmstate_load_state timebase v1
1016464@1705053752.106432:vmstate_load_state_field timebase:guest_timebase
1016464@1705053752.106433:vmstate_n_elems guest_timebase: 1
1016464@1705053752.106435:vmstate_load_state_field timebase:time_of_the_day_ns
1016464@1705053752.106436:vmstate_n_elems time_of_the_day_ns: 1
1016464@1705053752.106438:vmstate_subsection_load timebase
1016464@1705053752.106440:vmstate_subsection_load_bad timebase: spapr_option_vector_ov5_cas/(prefix)
1016464@1705053752.106442:vmstate_load_state_end timebase end/0
1016464@1705053752.106443:vmstate_subsection_load spapr
1016464@1705053752.106445:vmstate_load_state spapr_option_vector_ov5_cas v1
1016464@1705053752.106447:vmstate_load_state_field spapr_option_vector_ov5_cas:ov5_cas
1016464@1705053752.106448:vmstate_n_elems ov5_cas: 1
1016464@1705053752.106450:vmstate_load_state spapr_option_vector v1
1016464@1705053752.106452:vmstate_load_state_field spapr_option_vector:bitmap
1016464@1705053752.106454:vmstate_n_elems bitmap: 1
1016464@1705053752.106456:vmstate_subsection_load spapr_option_vector
1016464@1705053752.106457:vmstate_subsection_load_bad spapr_option_vector: (short)/
1016464@1705053752.106459:vmstate_load_state_end spapr_option_vector end/0
1016464@1705053752.106461:vmstate_subsection_load spapr_option_vector_ov5_cas
1016464@1705053752.106462:vmstate_subsection_load_bad spapr_option_vector_ov5_cas: (short)/
1016464@1705053752.106465:vmstate_load_state_end spapr_option_vector_ov5_cas end/0
1016464@1705053752.106466:vmstate_subsection_load_bad spapr: spapr/cap/htm/(lookup)
qemu-system-ppc64: error while loading state for instance 0x0 of device 'spapr'


Hope it helps,

Thanks,

C.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Fabiano Rosas]

Cédric Le Goater <clg@kaod.org> writes:

> On 1/10/24 14:19, Fabiano Rosas wrote:
>> Markus Armbruster <armbru@redhat.com> writes:
>> 
>>> Peter Xu <peterx@redhat.com> writes:
>>>
>>>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>>>> Hi Fabiano,
>>>>>
>>>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>
>>>>>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>>>
>>>>>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>>>>>
>>>>>>>>>>> +Peter/Fabiano
>>>>>>>>>>>
>>>>>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>> Hi Cédric,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>>>>>
>>>>>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>>>>>
>>>>>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>>>>>
>>>>>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>>>>>
>>>>>>>>> it happens we do.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>>>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>>>>>
>>>>>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>>>>> worked in the past for PPC. When I was doing more KVM related changes,
>>>>>>> this was very useful for dev. Also, some machines are partially emulated.
>>>>>>> Anyhow I agree this is not a strong requirement and we often break it.
>>>>>>> Let's focus on KVM only.
>>>>>>>
>>>>>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>>>>>
>>>>>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>>>>>> Good.
>>>>>>>>>
>>>>>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>>>>>
>>>>>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>>>>>> already, do you have a pointer to one of those cases were we broke
>>>>>>>> migration
>>>>>>>
>>>>>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>>>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>>>>> is similar to the changes proposed by this series, it impacts the QOM
>>>>>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>>>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>>>>> is quite an headache and this turned out to raise another problem some
>>>>>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>>>>>> machines and workarounds becoming a burden.
>>>>>>
>>>>>> This feels like something that could be handled by the vmstate code
>>>>>> somehow. The state is there, just under a different path.
>>>>>
>>>>> What, the QOM path is used in migration? ...
>>>>
>>>> Hopefully not..
>> 
>> Unfortunately the original fix doesn't mention _what_ actually broke
>> with migration. I assumed the QOM path was needed because otherwise I
>> don't think the fix makes sense. The thread discussing that patch also
>> directly mentions the QOM path:
>> 
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg450912.html
>> 
>> But I probably misunderstood something while reading that thread.
>> 
>>>>
>>>>>
>>>>> See recent discussions on "QOM path stability":
>>>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>>>
>>>> If I read it right, the commit 46f7afa37096 example is pretty special that
>>>> the QOM path more or less decided more than the hierachy itself but changes
>>>> the existances of objects.
>>>
>>> Let's see whether I got this...
>>>
>>> We removed some useless objects, moved the useful ones to another home.
>>> The move changed their QOM path.
>>>
>>> The problem was the removal of useless objects, because this also
>>> removed their vmstate.
>> 
>> If you checkout at the removal commit (5bc8d26de20c), the vmstate has
>> been kept untouched.
>> 
>>>
>>> The fix was adding the vmstate back as a dummy.
>> 
>> Since the vmstate was kept I don't see why would we need a dummy. The
>> incoming migration stream would still have the state, only at a
>> different point in the stream. It's surprising to me that that would
>> cause an issue, but I'm not well versed in that code.
>> 
>>>
>>> The QOM patch changes are *not* part of the problem.
>> 
>> The only explanation I can come up with is that after the patch
>> migration has broken after a hotplug or similar operation. In such
>> situation, the preallocated state would always be present before the
>> patch, but sometimes not present after the patch in case, say, a
>> hot-unplug has taken away a cpu + ICPState.
>
> May be. It has been a while.
>
> For a better understanding, I tried to reproduce. QEMU 2.9 still
> compiles on a f38 (memfd_create needs a fix). However, a
> QEMU-2.9/pseries-2.9 machine won't start any recent OS because
> something broke the OS/platform negotiation process (CAS) ...

Thanks for letting us know it still builds. That motivated me to try as
well. Although it took me a while because of python's brokenness.

The issue was indeed the difference in preallocation vs. no
preallocation. I don't remember how to do hotplug in ppc anymore, but
maxcpus was sufficient to reproduce:

src 2.9.0:
$ ./ppc64-softmmu/qemu-system-ppc64 -serial mon:stdio -nographic -vga none
-display none -accel tcg -machine pseries -smp 4,maxcpus=8 -m 256M
-nodefaults

dst 2.9.0 prior to 46f7afa:
$ ./ppc64-softmmu/qemu-system-ppc64 -serial mon:stdio -display none
-accel tcg -machine pseries-2.9 -smp 4,maxcpus=8 -m 256M -nodefaults
-incoming defer

(qemu) migrate_incoming tcp:localhost:1234
(qemu) qemu-system-ppc64: Unknown savevm section or instance 'icp/server' 4
qemu-system-ppc64: load of migration failed: Invalid argument

As you can see, both use 4 cpus, but the src will try to migrate all 8
icp/server instances. After the patch, the same migration works.

So this was not related to the QOM path nor the QOM hierarchy, it was
just about having state created at machine init vs. cpu realize time. It
might be wise to keep an eye on the QOM hierarchy anyway as it could
make these kinds of changes more evident during review.

> I removed the pre_2_10_has_unused_icps hack and worked around
> another migration compat issue in IOMMU ... Here are the last
> trace-events for migration :
>
>      QEMU-mainline/pseries-2.9 -> QEMU-2.9/pseries-2.9
>
> 1016464@1705053752.106417:vmstate_load spapr, spapr
> 1016464@1705053752.106419:vmstate_load_state spapr v3
> 1016464@1705053752.106421:vmstate_load_state_field spapr:unused
> 1016464@1705053752.106424:vmstate_load_state_field spapr:rtc_offset
> 1016464@1705053752.106425:vmstate_load_state_field spapr:tb
> 1016464@1705053752.106427:vmstate_n_elems tb: 1
> 1016464@1705053752.106429:vmstate_load_state timebase v1
> 1016464@1705053752.106432:vmstate_load_state_field timebase:guest_timebase
> 1016464@1705053752.106433:vmstate_n_elems guest_timebase: 1
> 1016464@1705053752.106435:vmstate_load_state_field timebase:time_of_the_day_ns
> 1016464@1705053752.106436:vmstate_n_elems time_of_the_day_ns: 1
> 1016464@1705053752.106438:vmstate_subsection_load timebase
> 1016464@1705053752.106440:vmstate_subsection_load_bad timebase: spapr_option_vector_ov5_cas/(prefix)
> 1016464@1705053752.106442:vmstate_load_state_end timebase end/0
> 1016464@1705053752.106443:vmstate_subsection_load spapr
> 1016464@1705053752.106445:vmstate_load_state spapr_option_vector_ov5_cas v1
> 1016464@1705053752.106447:vmstate_load_state_field spapr_option_vector_ov5_cas:ov5_cas
> 1016464@1705053752.106448:vmstate_n_elems ov5_cas: 1
> 1016464@1705053752.106450:vmstate_load_state spapr_option_vector v1
> 1016464@1705053752.106452:vmstate_load_state_field spapr_option_vector:bitmap
> 1016464@1705053752.106454:vmstate_n_elems bitmap: 1
> 1016464@1705053752.106456:vmstate_subsection_load spapr_option_vector
> 1016464@1705053752.106457:vmstate_subsection_load_bad spapr_option_vector: (short)/
> 1016464@1705053752.106459:vmstate_load_state_end spapr_option_vector end/0
> 1016464@1705053752.106461:vmstate_subsection_load spapr_option_vector_ov5_cas
> 1016464@1705053752.106462:vmstate_subsection_load_bad spapr_option_vector_ov5_cas: (short)/
> 1016464@1705053752.106465:vmstate_load_state_end spapr_option_vector_ov5_cas end/0
> 1016464@1705053752.106466:vmstate_subsection_load_bad spapr: spapr/cap/htm/(lookup)
> qemu-system-ppc64: error while loading state for instance 0x0 of device 'spapr'

This trace this seems to be something else entirely. I'd be amazed if
migrating mainline vs. something that old worked at all, even on x86.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/12/24 20:54, Fabiano Rosas wrote:
> Cédric Le Goater <clg@kaod.org> writes:
> 
>> On 1/10/24 14:19, Fabiano Rosas wrote:
>>> Markus Armbruster <armbru@redhat.com> writes:
>>>
>>>> Peter Xu <peterx@redhat.com> writes:
>>>>
>>>>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>>>>> Hi Fabiano,
>>>>>>
>>>>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>>
>>>>>>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>>>>
>>>>>>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>>>>>>
>>>>>>>>>>>> +Peter/Fabiano
>>>>>>>>>>>>
>>>>>>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>>> Hi Cédric,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>>>>>>
>>>>>>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>>>>>>
>>>>>>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>>>>>>
>>>>>>>>>> it happens we do.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>>>>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>>>>>>
>>>>>>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>>>>>> worked in the past for PPC. When I was doing more KVM related changes,
>>>>>>>> this was very useful for dev. Also, some machines are partially emulated.
>>>>>>>> Anyhow I agree this is not a strong requirement and we often break it.
>>>>>>>> Let's focus on KVM only.
>>>>>>>>
>>>>>>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>>>>>>
>>>>>>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>>>>>>> Good.
>>>>>>>>>>
>>>>>>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>>>>>>
>>>>>>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>>>>>>> already, do you have a pointer to one of those cases were we broke
>>>>>>>>> migration
>>>>>>>>
>>>>>>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>>>>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>>>>>> is similar to the changes proposed by this series, it impacts the QOM
>>>>>>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>>>>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>>>>>> is quite an headache and this turned out to raise another problem some
>>>>>>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>>>>>>> machines and workarounds becoming a burden.
>>>>>>>
>>>>>>> This feels like something that could be handled by the vmstate code
>>>>>>> somehow. The state is there, just under a different path.
>>>>>>
>>>>>> What, the QOM path is used in migration? ...
>>>>>
>>>>> Hopefully not..
>>>
>>> Unfortunately the original fix doesn't mention _what_ actually broke
>>> with migration. I assumed the QOM path was needed because otherwise I
>>> don't think the fix makes sense. The thread discussing that patch also
>>> directly mentions the QOM path:
>>>
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg450912.html
>>>
>>> But I probably misunderstood something while reading that thread.
>>>
>>>>>
>>>>>>
>>>>>> See recent discussions on "QOM path stability":
>>>>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>>>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>>>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>>>>
>>>>> If I read it right, the commit 46f7afa37096 example is pretty special that
>>>>> the QOM path more or less decided more than the hierachy itself but changes
>>>>> the existances of objects.
>>>>
>>>> Let's see whether I got this...
>>>>
>>>> We removed some useless objects, moved the useful ones to another home.
>>>> The move changed their QOM path.
>>>>
>>>> The problem was the removal of useless objects, because this also
>>>> removed their vmstate.
>>>
>>> If you checkout at the removal commit (5bc8d26de20c), the vmstate has
>>> been kept untouched.
>>>
>>>>
>>>> The fix was adding the vmstate back as a dummy.
>>>
>>> Since the vmstate was kept I don't see why would we need a dummy. The
>>> incoming migration stream would still have the state, only at a
>>> different point in the stream. It's surprising to me that that would
>>> cause an issue, but I'm not well versed in that code.
>>>
>>>>
>>>> The QOM patch changes are *not* part of the problem.
>>>
>>> The only explanation I can come up with is that after the patch
>>> migration has broken after a hotplug or similar operation. In such
>>> situation, the preallocated state would always be present before the
>>> patch, but sometimes not present after the patch in case, say, a
>>> hot-unplug has taken away a cpu + ICPState.
>>
>> May be. It has been a while.
>>
>> For a better understanding, I tried to reproduce. QEMU 2.9 still
>> compiles on a f38 (memfd_create needs a fix). However, a
>> QEMU-2.9/pseries-2.9 machine won't start any recent OS because
>> something broke the OS/platform negotiation process (CAS) ...
> 
> Thanks for letting us know it still builds. That motivated me to try as
> well. Although it took me a while because of python's brokenness.
> 
> The issue was indeed the difference in preallocation vs. no
> preallocation. I don't remember how to do hotplug in ppc anymore, but
> maxcpus was sufficient to reproduce:
> 
> src 2.9.0:
> $ ./ppc64-softmmu/qemu-system-ppc64 -serial mon:stdio -nographic -vga none
> -display none -accel tcg -machine pseries -smp 4,maxcpus=8 -m 256M
> -nodefaults
> 
> dst 2.9.0 prior to 46f7afa:
> $ ./ppc64-softmmu/qemu-system-ppc64 -serial mon:stdio -display none
> -accel tcg -machine pseries-2.9 -smp 4,maxcpus=8 -m 256M -nodefaults
> -incoming defer
> 
> (qemu) migrate_incoming tcp:localhost:1234
> (qemu) qemu-system-ppc64: Unknown savevm section or instance 'icp/server' 4
> qemu-system-ppc64: load of migration failed: Invalid argument
> 
> As you can see, both use 4 cpus, but the src will try to migrate all 8
> icp/server instances. After the patch, the same migration works.
> 
> So this was not related to the QOM path nor the QOM hierarchy, it was
> just about having state created at machine init vs. cpu realize time. 

Ah ! of course, the machine is created in an init handler.

Thanks Fabiano, for the diagnostic and the time you took to reproduce
and analyze. I had this bad assumption of constraint between QOM vs.
migration and any change made me uncomfortable. You cleared it.

Good that we are deprecating this hack in a couple of versions.

> It
> might be wise to keep an eye on the QOM hierarchy anyway as it could
> make these kinds of changes more evident during review.

Clearly,


Thanks,

C.



>> I removed the pre_2_10_has_unused_icps hack and worked around
>> another migration compat issue in IOMMU ... Here are the last
>> trace-events for migration :
>>
>>       QEMU-mainline/pseries-2.9 -> QEMU-2.9/pseries-2.9
>>
>> 1016464@1705053752.106417:vmstate_load spapr, spapr
>> 1016464@1705053752.106419:vmstate_load_state spapr v3
>> 1016464@1705053752.106421:vmstate_load_state_field spapr:unused
>> 1016464@1705053752.106424:vmstate_load_state_field spapr:rtc_offset
>> 1016464@1705053752.106425:vmstate_load_state_field spapr:tb
>> 1016464@1705053752.106427:vmstate_n_elems tb: 1
>> 1016464@1705053752.106429:vmstate_load_state timebase v1
>> 1016464@1705053752.106432:vmstate_load_state_field timebase:guest_timebase
>> 1016464@1705053752.106433:vmstate_n_elems guest_timebase: 1
>> 1016464@1705053752.106435:vmstate_load_state_field timebase:time_of_the_day_ns
>> 1016464@1705053752.106436:vmstate_n_elems time_of_the_day_ns: 1
>> 1016464@1705053752.106438:vmstate_subsection_load timebase
>> 1016464@1705053752.106440:vmstate_subsection_load_bad timebase: spapr_option_vector_ov5_cas/(prefix)
>> 1016464@1705053752.106442:vmstate_load_state_end timebase end/0
>> 1016464@1705053752.106443:vmstate_subsection_load spapr
>> 1016464@1705053752.106445:vmstate_load_state spapr_option_vector_ov5_cas v1
>> 1016464@1705053752.106447:vmstate_load_state_field spapr_option_vector_ov5_cas:ov5_cas
>> 1016464@1705053752.106448:vmstate_n_elems ov5_cas: 1
>> 1016464@1705053752.106450:vmstate_load_state spapr_option_vector v1
>> 1016464@1705053752.106452:vmstate_load_state_field spapr_option_vector:bitmap
>> 1016464@1705053752.106454:vmstate_n_elems bitmap: 1
>> 1016464@1705053752.106456:vmstate_subsection_load spapr_option_vector
>> 1016464@1705053752.106457:vmstate_subsection_load_bad spapr_option_vector: (short)/
>> 1016464@1705053752.106459:vmstate_load_state_end spapr_option_vector end/0
>> 1016464@1705053752.106461:vmstate_subsection_load spapr_option_vector_ov5_cas
>> 1016464@1705053752.106462:vmstate_subsection_load_bad spapr_option_vector_ov5_cas: (short)/
>> 1016464@1705053752.106465:vmstate_load_state_end spapr_option_vector_ov5_cas end/0
>> 1016464@1705053752.106466:vmstate_subsection_load_bad spapr: spapr/cap/htm/(lookup)
>> qemu-system-ppc64: error while loading state for instance 0x0 of device 'spapr'
> 
> This trace this seems to be something else entirely. I'd be amazed if
> migrating mainline vs. something that old worked at all, even on x86.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/10/24 07:03, Markus Armbruster wrote:
> Peter Xu <peterx@redhat.com> writes:
> 
>> On Tue, Jan 09, 2024 at 10:22:31PM +0100, Philippe Mathieu-Daudé wrote:
>>> Hi Fabiano,
>>>
>>> On 9/1/24 21:21, Fabiano Rosas wrote:
>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>
>>>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>>>
>>>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>>>
>>>>>>>>> +Peter/Fabiano
>>>>>>>>>
>>>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>> Hi Cédric,
>>>>>>>>>>>
>>>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>>>
>>>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>>>
>>>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>>>
>>>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>>>
>>>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>>>
>>>>>>> it happens we do.
>>>>>>>
>>>>>>
>>>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>>>
>>>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>>>> worked in the past for PPC. When I was doing more KVM related changes,
>>>>> this was very useful for dev. Also, some machines are partially emulated.
>>>>> Anyhow I agree this is not a strong requirement and we often break it.
>>>>> Let's focus on KVM only.
>>>>>
>>>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>>>
>>>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>>>> Good.
>>>>>>>
>>>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>>>
>>>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>>>> already, do you have a pointer to one of those cases were we broke
>>>>>> migration
>>>>>
>>>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>>>> is similar to the changes proposed by this series, it impacts the QOM
>>>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>>>> is quite an headache and this turned out to raise another problem some
>>>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>>>> machines and workarounds becoming a burden.
>>>>
>>>> This feels like something that could be handled by the vmstate code
>>>> somehow. The state is there, just under a different path.
>>>
>>> What, the QOM path is used in migration? ...
>>
>> Hopefully not..
>>
>>>
>>> See recent discussions on "QOM path stability":
>>> https://lore.kernel.org/qemu-devel/ZZfYvlmcxBCiaeWE@redhat.com/
>>> https://lore.kernel.org/qemu-devel/87jzojbxt7.fsf@pond.sub.org/
>>> https://lore.kernel.org/qemu-devel/87v883by34.fsf@pond.sub.org/
>>
>> If I read it right, the commit 46f7afa37096 example is pretty special that
>> the QOM path more or less decided more than the hierachy itself but changes
>> the existances of objects.
> 
> Let's see whether I got this...
> 
> We removed some useless objects, moved the useful ones to another home.
> The move changed their QOM path.

They interrupt controller presenter objects were quite useful :)
 From what I recall, we moved them from an array under the machine
to the CPU object, so the interrupt controller presenter states
previously under the machine were not there anymore and this broke
migration compatibility.

Sorry for the noise if this is not a problem anymore. It was at
the time and we found a way to work around it; I should probably
say we hacked our way around it. Nevertheless, this was kind of
a trauma too because since I never dared touch the QOM hierarchy
of a migratable machine again. Migration is complicated.


> The problem was the removal of useless objects, because this also
> removed their vmstate.
> 
> The fix was adding the vmstate back as a dummy.
> 
> The QOM patch changes are *not* part of the problem.
> 
> Correct?
> 
>>>> No one wants
>>>> to be policing QOM hierarchy changes in every single series that shows
>>>> up on the list.
>>>>
>>>> Anyway, thanks for the pointers. I'll study that code a bit more, maybe
>>>> I can come up with some way to handle these cases.
>>>>
>>>> Hopefully between the analyze-migration test and the compat tests we'll
>>>> catch the next bug of this kind before it gets merged.
>>
>> Things like that might be able to be detected via vmstate-static-checker.py.
>> But I'm not 100% sure, also its coverage is limited.
>>
>> For example, I don't think it can detect changes to objects that will only
>> be created dynamically, e.g., I think sometimes we create objects after
>> some guest behaviors (consider guest enables the device, then QEMU
>> emulation creates some objects on demand of device setup?),
> 
> Feels nuts to me.
> 
> In real hardware, software enabling a device that is disabled by default
> doesn't create the device.  The device is always there, it just happens
> to be inactive unless enabled.  We should model the device just like
> that.

yes. That's how we modeled the two interrupt modes in pseries. The
machine has two interrupt controller devices model always present
and the cpus, two interrupt presenters. SW negotiates with the
platform (QEMU) which mode to activate. This is the only way to
support migration with an OS that can choose such complex features.


For the context, POWER9 introduced a new flavor of HW logic for
interrupts, which scaled better on large system (16s) and guests
with newer OS could dynamically switch the SW interface to choose
the new implementation.

Thanks,

C.


> 
>>                                                              and since the
>> static checker shouldn't ever start the VM and run any code, they won't
>> trigger.
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

Hi Cédric,

On 9/1/24 19:06, Cédric Le Goater wrote:
> On 1/9/24 18:40, Fabiano Rosas wrote:
>> Cédric Le Goater <clg@kaod.org> writes:
>>
>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>
>>>>> +Peter/Fabiano
>>>>>
>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>> Hi Cédric,
>>>>>>>
>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>
>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>
>>>>>>>> Changing the QOM hierarchy has an impact on the state of the 
>>>>>>>> machine
>>>>>>>> and some fixups are then required to maintain migration 
>>>>>>>> compatibility.
>>>>>>>> This can become a real headache for KVM machines like virt for 
>>>>>>>> which
>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>
>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>
>>>> FWIW, I didn't spot anything problematic either.
>>>>
>>>> I've ran this through my migration compatibility series [1] and it
>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I 
>>>> don't
>>>> think we even support migration of anything non-KVM on arm.
>>>
>>> it happens we do.
>>>
>>
>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>> non-KVM-capable cpus, as in 32-bit. Nevermind.
> 
> Theoretically, we should be able to migrate to a TCG guest. Well, this
> worked in the past for PPC. When I was doing more KVM related changes,
> this was very useful for dev. Also, some machines are partially emulated.
> Anyhow I agree this is not a strong requirement and we often break it.
> Let's focus on KVM only.

No no, we want the same for TCG.

>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>
>>> yes it depends on the QOM hierarchy and virt seems immune to the 
>>> changes.
>>> Good.
>>>
>>> However, changing the QOM topology clearly breaks migration compat,
>>
>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>> already, do you have a pointer to one of those cases were we broke
>> migration 
> 
> Regarding pseries, migration compat broke because of 5bc8d26de20c
> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
> is similar to the changes proposed by this series, it impacts the QOM
> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
> ("spapr: fix migration of ICPState objects from/to older QEMU") which
> is quite an headache and this turned out to raise another problem some
> months ago ... :/ That's why I sent [1] to prepare removal of old
> machines and workarounds becoming a burden.
> 
> Regarding aspeed, this series breaks compat.

Can you write down the steps to reproduce please? I'll debug it.
We need to understand this.

> Not that we care much
> but ​this caught my attention because of my past experience on pseries.
> Same kind of QOM change which could impact other machines, like virt.
> Since you checked that migration compat is preserved on virt, we should
> be fine.
> 
> Thanks,
> 
> C.
> 
> [1] 
> https://lore.kernel.org/qemu-devel/20231214181723.1520854-1-clg@kaod.org/
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Philippe Mathieu-Daudé]

On 9/1/24 22:07, Philippe Mathieu-Daudé wrote:
> Hi Cédric,
> 
> On 9/1/24 19:06, Cédric Le Goater wrote:
>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>> Cédric Le Goater <clg@kaod.org> writes:
>>>
>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>
>>>>>> +Peter/Fabiano
>>>>>>
>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>> Hi Cédric,
>>>>>>>>
>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>
>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>
>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the 
>>>>>>>>> machine
>>>>>>>>> and some fixups are then required to maintain migration 
>>>>>>>>> compatibility.
>>>>>>>>> This can become a real headache for KVM machines like virt for 
>>>>>>>>> which
>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>
>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>
>>>>> FWIW, I didn't spot anything problematic either.
>>>>>
>>>>> I've ran this through my migration compatibility series [1] and it
>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I 
>>>>> don't
>>>>> think we even support migration of anything non-KVM on arm.
>>>>
>>>> it happens we do.
>>>>
>>>
>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>
>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>> worked in the past for PPC. When I was doing more KVM related changes,
>> this was very useful for dev. Also, some machines are partially emulated.
>> Anyhow I agree this is not a strong requirement and we often break it.
>> Let's focus on KVM only.
> 
> No no, we want the same for TCG.
> 
>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>
>>>> yes it depends on the QOM hierarchy and virt seems immune to the 
>>>> changes.
>>>> Good.
>>>>
>>>> However, changing the QOM topology clearly breaks migration compat,
>>>
>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>> already, do you have a pointer to one of those cases were we broke
>>> migration 
>>
>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>> is similar to the changes proposed by this series, it impacts the QOM
>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>> is quite an headache and this turned out to raise another problem some
>> months ago ... :/ That's why I sent [1] to prepare removal of old
>> machines and workarounds becoming a burden.
>>
>> Regarding aspeed, this series breaks compat.
> 
> Can you write down the steps to reproduce please? I'll debug it.

Also, have you figured (bisecting) which patch start to break?

> We need to understand this.
> 
>> Not that we care much
>> but ​this caught my attention because of my past experience on pseries.
>> Same kind of QOM change which could impact other machines, like virt.
>> Since you checked that migration compat is preserved on virt, we should
>> be fine.
>>
>> Thanks,
>>
>> C.
>>
>> [1] 
>> https://lore.kernel.org/qemu-devel/20231214181723.1520854-1-clg@kaod.org/
>>
> 

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

On 1/9/24 22:09, Philippe Mathieu-Daudé wrote:
> On 9/1/24 22:07, Philippe Mathieu-Daudé wrote:
>> Hi Cédric,
>>
>> On 9/1/24 19:06, Cédric Le Goater wrote:
>>> On 1/9/24 18:40, Fabiano Rosas wrote:
>>>> Cédric Le Goater <clg@kaod.org> writes:
>>>>
>>>>> On 1/3/24 20:53, Fabiano Rosas wrote:
>>>>>> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
>>>>>>
>>>>>>> +Peter/Fabiano
>>>>>>>
>>>>>>> On 2/1/24 17:41, Cédric Le Goater wrote:
>>>>>>>> On 1/2/24 17:15, Philippe Mathieu-Daudé wrote:
>>>>>>>>> Hi Cédric,
>>>>>>>>>
>>>>>>>>> On 2/1/24 15:55, Cédric Le Goater wrote:
>>>>>>>>>> On 12/12/23 17:29, Philippe Mathieu-Daudé wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> When a MPCore cluster is used, the Cortex-A cores belong the the
>>>>>>>>>>> cluster container, not to the board/soc layer. This series move
>>>>>>>>>>> the creation of vCPUs to the MPCore private container.
>>>>>>>>>>>
>>>>>>>>>>> Doing so we consolidate the QOM model, moving common code in a
>>>>>>>>>>> central place (abstract MPCore parent).
>>>>>>>>>>
>>>>>>>>>> Changing the QOM hierarchy has an impact on the state of the machine
>>>>>>>>>> and some fixups are then required to maintain migration compatibility.
>>>>>>>>>> This can become a real headache for KVM machines like virt for which
>>>>>>>>>> migration compatibility is a feature, less for emulated ones.
>>>>>>>>>
>>>>>>>>> All changes are either moving properties (which are not migrated)
>>>>>>>>> or moving non-migrated QOM members (i.e. pointers of ARMCPU, which
>>>>>>>>> is still migrated elsewhere). So I don't see any obvious migration
>>>>>>>>> problem, but I might be missing something, so I Cc'ed Juan :>
>>>>>>
>>>>>> FWIW, I didn't spot anything problematic either.
>>>>>>
>>>>>> I've ran this through my migration compatibility series [1] and it
>>>>>> doesn't regress aarch64 migration from/to 8.2. The tests use '-M
>>>>>> virt -cpu max', so the cortex-a7 and cortex-a15 are not covered. I don't
>>>>>> think we even support migration of anything non-KVM on arm.
>>>>>
>>>>> it happens we do.
>>>>>
>>>>
>>>> Oh, sorry, I didn't mean TCG here. Probably meant to say something like
>>>> non-KVM-capable cpus, as in 32-bit. Nevermind.
>>>
>>> Theoretically, we should be able to migrate to a TCG guest. Well, this
>>> worked in the past for PPC. When I was doing more KVM related changes,
>>> this was very useful for dev. Also, some machines are partially emulated.
>>> Anyhow I agree this is not a strong requirement and we often break it.
>>> Let's focus on KVM only.
>>
>> No no, we want the same for TCG.
>>
>>>>>> 1- https://gitlab.com/farosas/qemu/-/jobs/5853599533
>>>>>
>>>>> yes it depends on the QOM hierarchy and virt seems immune to the changes.
>>>>> Good.
>>>>>
>>>>> However, changing the QOM topology clearly breaks migration compat,
>>>>
>>>> Well, "clearly" is relative =) You've mentioned pseries and aspeed
>>>> already, do you have a pointer to one of those cases were we broke
>>>> migration 
>>>
>>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>>> is similar to the changes proposed by this series, it impacts the QOM
>>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>>> is quite an headache and this turned out to raise another problem some
>>> months ago ... :/ That's why I sent [1] to prepare removal of old
>>> machines and workarounds becoming a burden.
>>>
>>> Regarding aspeed, this series breaks compat.
>>
>> Can you write down the steps to reproduce please? I'll debug it.
> 
> Also, have you figured (bisecting) which patch start to break?

Sorry I didn't. I wished I had more time for that.

Thanks,

C.

Re: [PATCH 00/33] hw/cpu/arm: Remove one use of qemu_get_cpu() in A7/A15 MPCore priv

[Cédric Le Goater]

>> Regarding pseries, migration compat broke because of 5bc8d26de20c
>> ("spapr: allocate the ICPState object from under sPAPRCPUCore") which
>> is similar to the changes proposed by this series, it impacts the QOM
>> hierarchy. Here is the workaround/fix from Greg : 46f7afa37096
>> ("spapr: fix migration of ICPState objects from/to older QEMU") which
>> is quite an headache and this turned out to raise another problem some
>> months ago ... :/ That's why I sent [1] to prepare removal of old
>> machines and workarounds becoming a burden.
>>
>> Regarding aspeed, this series breaks compat.
> 
> Can you write down the steps to reproduce please? I'll debug it.
> We need to understand this.

Nothing complex,

$ wget https://github.com/legoater/qemu-aspeed-boot/raw/master/images/ast2600-evb/buildroot-2023.11/flash.img

$ qemu-system-arm -M ast2600-evb -net user -drive file=./flash.img,format=raw,if=mtd -nographic -snapshot -serial mon:stdio  -trace vmstate* -trace save* -trace load*

$ qemu-system-arm-patch -M ast2600-evb -net user -drive file=./flash.img,format=raw,if=mtd -nographic -snapshot -serial mon:stdio  -trace vmstate* -trace save* -trace load* -incoming tcp::1234

stop the VM in U-boot before loading the kernel because QEMU does not
support migrating CPU when in secure mode. That's how I understood what
Peter told me.

(qemu) migrate tcp:localhost:1234

...
vmstate_load_state_field cpu:cpreg_vmstate_array_len
vmstate_n_elems cpreg_vmstate_array_len: 1
qemu-system-arm: Invalid value 266 expecting positive value <= 223
qemu-system-arm: Failed to load cpu:cpreg_vmstate_array_len
vmstate_load_field_error field "cpreg_vmstate_array_len" load failed, ret = -22
qemu-system-arm: error while loading state for instance 0x0 of device 'cpu'

Thanks,

C.