[Qemu-devel] [PULL 00/18] ppc-for-2.7 queue 20160617

[Qemu-devel] [PULL 00/18] ppc-for-2.7 queue 20160617

[David Gibson]

The following changes since commit 585fcd4b11070b3220685fc54ecca1991cdeb161:

  Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2016-06-16 17:58:45 +0100)

are available in the git repository at:

  git://github.com/dgibson/qemu.git tags/ppc-for-2.7-20160617

for you to fetch changes up to 2474bfd4603b2d354fdb4001b083b7c72bff627f:

  spapr: implement query-hotpluggable-cpus callback (2016-06-17 16:33:49 +1000)

----------------------------------------------------------------
ppc patch queue for 2016-06-17

Here's the current accumulated set of spapr, ppc and related patches.
  * The big thing in here is CPU hotplug for spapr
    - This includes a number of acked generic changes adding new
      infrastructure for hotplugging cpu cores
  * A number of TCG bug fixes are also included
  * This adds a new testcase to make it harder to accidentally break
    Macintosh (and other openbios) platforms

----------------------------------------------------------------
Bharata B Rao (10):
      cpu: Abstract CPU core type
      xics,xics_kvm: Handle CPU unplug correctly
      spapr_drc: Prevent detach racing against attach for CPU DR
      qom: API to get instance_size of a type
      spapr: Abstract CPU core device and type specific core devices
      spapr: Move spapr_cpu_init() to spapr_cpu_core.c
      spapr: convert boot CPUs into CPU core devices
      spapr: CPU hotplug support
      spapr: CPU hot unplug support
      hmp: Add 'info hotpluggable-cpus' HMP command

Gavin Shan (1):
      vfio: Fix broken EEH

Igor Mammedov (3):
      qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
      QMP: Add query-hotpluggable-cpus
      spapr: implement query-hotpluggable-cpus callback

Jakub Horak (1):
      target-ppc: Bug in BookE wait instruction

Richard Henderson (1):
      target-ppc: Fix rlwimi, rlwinm, rlwnm

Thomas Huth (2):
      hw/ppc/spapr: Silence deprecation message in qtest mode
      ppc / sparc: Add a tester for checking whether OpenBIOS runs successfully

 hmp-commands-info.hx            |  14 ++
 hmp.c                           |  42 +++++
 hmp.h                           |   1 +
 hw/core/hotplug.c               |  11 ++
 hw/core/qdev.c                  |   9 +-
 hw/cpu/Makefile.objs            |   1 +
 hw/cpu/core.c                   |  88 +++++++++
 hw/intc/xics.c                  |  14 ++
 hw/intc/xics_kvm.c              |   8 +-
 hw/ppc/Makefile.objs            |   1 +
 hw/ppc/spapr.c                  | 218 +++++++++++++++------
 hw/ppc/spapr_cpu_core.c         | 405 ++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_drc.c              |  12 ++
 hw/ppc/spapr_events.c           |   3 +
 hw/ppc/spapr_pci_vfio.c         |   5 +-
 hw/ppc/spapr_rtas.c             |  24 +++
 hw/vfio/common.c                |   2 +-
 include/hw/boards.h             |   5 +
 include/hw/cpu/core.h           |  31 +++
 include/hw/hotplug.h            |  14 +-
 include/hw/ppc/spapr.h          |   7 +
 include/hw/ppc/spapr_cpu_core.h |  36 ++++
 include/hw/ppc/spapr_drc.h      |   1 +
 include/hw/ppc/xics.h           |   1 +
 include/qom/object.h            |   8 +-
 monitor.c                       |  13 ++
 qapi-schema.json                |  55 ++++++
 qmp-commands.hx                 |  23 +++
 qom/object.c                    |   8 +
 target-ppc/kvm.c                |  28 +++
 target-ppc/translate.c          |  75 +++++---
 tests/Makefile.include          |   6 +
 tests/prom-env-test.c           |  90 +++++++++
 33 files changed, 1169 insertions(+), 90 deletions(-)
 create mode 100644 hw/cpu/core.c
 create mode 100644 hw/ppc/spapr_cpu_core.c
 create mode 100644 include/hw/cpu/core.h
 create mode 100644 include/hw/ppc/spapr_cpu_core.h
 create mode 100644 tests/prom-env-test.c

[Qemu-devel] [PULL 01/18] hw/ppc/spapr: Silence deprecation message in qtest mode

[David Gibson]

From: Thomas Huth <thuth@redhat.com>

When running "make check", there is currently always an error message
saying "spapr-pci-vfio-host-bridge is deprecated". This happens because
the QOM tests are instantiating all possible devices, and the error
message is currently located in the instance_init() function of the
device. Since it is legal for the tests to instantiate a device without
using it, the error message should be silenced when we're running in
test mode.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr_pci_vfio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c
index cbd3d23..f3cb141 100644
--- a/hw/ppc/spapr_pci_vfio.c
+++ b/hw/ppc/spapr_pci_vfio.c
@@ -27,6 +27,7 @@
 #include "linux/vfio.h"
 #include "hw/vfio/vfio.h"
 #include "qemu/error-report.h"
+#include "sysemu/qtest.h"
 
 #define TYPE_SPAPR_PCI_VFIO_HOST_BRIDGE "spapr-pci-vfio-host-bridge"
 
@@ -48,7 +49,9 @@ static Property spapr_phb_vfio_properties[] = {
 
 static void spapr_phb_vfio_instance_init(Object *obj)
 {
-    error_report("spapr-pci-vfio-host-bridge is deprecated");
+    if (!qtest_enabled()) {
+        error_report("spapr-pci-vfio-host-bridge is deprecated");
+    }
 }
 
 bool spapr_phb_eeh_available(sPAPRPHBState *sphb)
-- 
2.5.5

[Qemu-devel] [PULL 02/18] ppc / sparc: Add a tester for checking whether OpenBIOS runs successfully

[David Gibson]

From: Thomas Huth <thuth@redhat.com>

Since the mac99 and g3beige PowerPC machines recently broke without
being noticed, it would be good to have a tester for "make check"
that detects such issues immediately. A simple way to test the firmware
of these machines is to use the "-prom-env" parameter of QEMU. This
parameter can be used to put some Forth code into the 'boot-command'
firmware variable which then can signal success to the tester by
writing a magic value to a known memory location. And since some of the
Sparc machines are also using OpenBIOS, they are now tested with this
prom-env-tester, too.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
[dwg: Removed sparc64, because it trips a TCG bug on 32-bit hosts]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/Makefile.include |  6 ++++
 tests/prom-env-test.c  | 90 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
 create mode 100644 tests/prom-env-test.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6135875..33fcdcb 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -259,6 +259,11 @@ check-qtest-ppc-y += tests/boot-order-test$(EXESUF)
 check-qtest-ppc64-y += tests/boot-order-test$(EXESUF)
 check-qtest-ppc64-y += tests/spapr-phb-test$(EXESUF)
 gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c
+check-qtest-ppc-y = tests/prom-env-test$(EXESUF)
+check-qtest-ppc64-y = tests/prom-env-test$(EXESUF)
+check-qtest-sparc-y = tests/prom-env-test$(EXESUF)
+#Disabled for now, triggers a TCG bug on 32-bit hosts
+#check-qtest-sparc64-y = tests/prom-env-test$(EXESUF)
 check-qtest-microblazeel-y = $(check-qtest-microblaze-y)
 check-qtest-xtensaeb-y = $(check-qtest-xtensa-y)
 
@@ -550,6 +555,7 @@ tests/rtc-test$(EXESUF): tests/rtc-test.o
 tests/m48t59-test$(EXESUF): tests/m48t59-test.o
 tests/endianness-test$(EXESUF): tests/endianness-test.o
 tests/spapr-phb-test$(EXESUF): tests/spapr-phb-test.o $(libqos-obj-y)
+tests/prom-env-test$(EXESUF): tests/prom-env-test.o $(libqos-obj-y)
 tests/fdc-test$(EXESUF): tests/fdc-test.o
 tests/ide-test$(EXESUF): tests/ide-test.o $(libqos-pc-obj-y)
 tests/ahci-test$(EXESUF): tests/ahci-test.o $(libqos-pc-obj-y)
diff --git a/tests/prom-env-test.c b/tests/prom-env-test.c
new file mode 100644
index 0000000..6df57d2
--- /dev/null
+++ b/tests/prom-env-test.c
@@ -0,0 +1,90 @@
+/*
+ * Test OpenBIOS-based machines.
+ *
+ * Copyright (c) 2016 Red Hat Inc.
+ *
+ * Author:
+ *    Thomas Huth <thuth@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2
+ * or later. See the COPYING file in the top-level directory.
+ *
+ * This test is used to check that some OpenBIOS machines can be started
+ * successfully in TCG mode. To do this, we first put some Forth code into
+ * the "boot-command" Open Firmware environment variable. This Forth code
+ * writes a well-known magic value to a known location in memory. Then we
+ * start the guest so that OpenBIOS can boot and finally run the Forth code.
+ * The testing code here then can finally check whether the value has been
+ * successfully written into the guest memory.
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+
+#define MAGIC   0xcafec0de
+#define ADDRESS 0x4000
+
+static void check_guest_memory(void)
+{
+    uint32_t signature;
+    int i;
+
+    /* Poll until code has run and modified memory. Wait at most 30 seconds */
+    for (i = 0; i < 3000; ++i) {
+        signature = readl(ADDRESS);
+        if (signature == MAGIC) {
+            break;
+        }
+        g_usleep(10000);
+    }
+
+    g_assert_cmphex(signature, ==, MAGIC);
+}
+
+static void test_machine(const void *machine)
+{
+    char *args;
+
+    args = g_strdup_printf("-M %s,accel=tcg -prom-env 'boot-command=%x %x l!'",
+                           (const char *)machine, MAGIC, ADDRESS);
+
+    qtest_start(args);
+    check_guest_memory();
+    qtest_quit(global_qtest);
+
+    g_free(args);
+}
+
+static void add_tests(const char *machines[])
+{
+    int i;
+    char *name;
+
+    for (i = 0; machines[i] != NULL; i++) {
+        name = g_strdup_printf("prom-env/%s", machines[i]);
+        qtest_add_data_func(name, machines[i], test_machine);
+        g_free(name);
+    }
+}
+
+int main(int argc, char *argv[])
+{
+    const char *sparc_machines[] = { "SPARCbook", "Voyager", "SS-20", NULL };
+    const char *sparc64_machines[] = { "sun4u", "sun4v", NULL };
+    const char *mac_machines[] = { "mac99", "g3beige", NULL };
+    const char *arch = qtest_get_arch();
+
+    g_test_init(&argc, &argv, NULL);
+
+    if (!strcmp(arch, "ppc") || !strcmp(arch, "ppc64")) {
+        add_tests(mac_machines);
+    } else if (!strcmp(arch, "sparc")) {
+        add_tests(sparc_machines);
+    } else if (!strcmp(arch, "sparc64")) {
+        add_tests(sparc64_machines);
+    } else {
+        g_assert_not_reached();
+    }
+
+    return g_test_run();
+}
-- 
2.5.5

[Qemu-devel] [PULL 03/18] target-ppc: Bug in BookE wait instruction

[David Gibson]

From: Jakub Horak <thement@ibawizard.net>

Fixed bug in code generation for the PowerPC "wait" instruction. It
doesn't make sense to store a non-initialized register.

Signed-off-by: Jakub Horak <thement@ibawizard.net>
[dwg: revised commit message]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 target-ppc/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target-ppc/translate.c b/target-ppc/translate.c
index b689475..1f401b7 100644
--- a/target-ppc/translate.c
+++ b/target-ppc/translate.c
@@ -3499,7 +3499,7 @@ static void gen_sync(DisasContext *ctx)
 /* wait */
 static void gen_wait(DisasContext *ctx)
 {
-    TCGv_i32 t0 = tcg_temp_new_i32();
+    TCGv_i32 t0 = tcg_const_i32(1);
     tcg_gen_st_i32(t0, cpu_env,
                    -offsetof(PowerPCCPU, env) + offsetof(CPUState, halted));
     tcg_temp_free_i32(t0);
-- 
2.5.5

[Qemu-devel] [PULL 04/18] vfio: Fix broken EEH

[David Gibson]

From: Gavin Shan <gwshan@linux.vnet.ibm.com>

vfio_eeh_container_op() is the backend that communicates with
host kernel to support EEH functionality in QEMU. However, the
functon should return the value from host kernel instead of 0
unconditionally.

dwg: Specifically the problem occurs for the handful of EEH
sub-operations which can return a non-zero, non-error result.

Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Acked-by: Alex Williamson <alex.williamson@redhat.com>
[dwg: clarification to commit message]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/vfio/common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 5ff5e92..1898f1f 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1257,7 +1257,7 @@ static int vfio_eeh_container_op(VFIOContainer *container, uint32_t op)
         return -errno;
     }
 
-    return 0;
+    return ret;
 }
 
 static VFIOContainer *vfio_eeh_as_container(AddressSpace *as)
-- 
2.5.5

[Qemu-devel] [PULL 05/18] target-ppc: Fix rlwimi, rlwinm, rlwnm

[David Gibson]

From: Richard Henderson <rth@twiddle.net>

In 63ae0915f8ec, I arranged to use a 32-bit rotate, without
considering the effect of a mask value that wraps around to
the high bits of the word.

Signed-off-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 target-ppc/translate.c | 73 +++++++++++++++++++++++++++++++++++---------------
 1 file changed, 52 insertions(+), 21 deletions(-)

diff --git a/target-ppc/translate.c b/target-ppc/translate.c
index 1f401b7..30dc76a 100644
--- a/target-ppc/translate.c
+++ b/target-ppc/translate.c
@@ -1636,7 +1636,6 @@ static void gen_rlwimi(DisasContext *ctx)
         tcg_gen_deposit_tl(t_ra, t_ra, t_rs, sh, me - mb + 1);
     } else {
         target_ulong mask;
-        TCGv_i32 t0;
         TCGv t1;
 
 #if defined(TARGET_PPC64)
@@ -1645,12 +1644,21 @@ static void gen_rlwimi(DisasContext *ctx)
 #endif
         mask = MASK(mb, me);
 
-        t0 = tcg_temp_new_i32();
         t1 = tcg_temp_new();
-        tcg_gen_trunc_tl_i32(t0, t_rs);
-        tcg_gen_rotli_i32(t0, t0, sh);
-        tcg_gen_extu_i32_tl(t1, t0);
-        tcg_temp_free_i32(t0);
+        if (mask <= 0xffffffffu) {
+            TCGv_i32 t0 = tcg_temp_new_i32();
+            tcg_gen_trunc_tl_i32(t0, t_rs);
+            tcg_gen_rotli_i32(t0, t0, sh);
+            tcg_gen_extu_i32_tl(t1, t0);
+            tcg_temp_free_i32(t0);
+        } else {
+#if defined(TARGET_PPC64)
+            tcg_gen_deposit_i64(t1, t_rs, t_rs, 32, 32);
+            tcg_gen_rotli_i64(t1, t1, sh);
+#else
+            g_assert_not_reached();
+#endif
+        }
 
         tcg_gen_andi_tl(t1, t1, mask);
         tcg_gen_andi_tl(t_ra, t_ra, ~mask);
@@ -1678,20 +1686,30 @@ static void gen_rlwinm(DisasContext *ctx)
         tcg_gen_ext32u_tl(t_ra, t_rs);
         tcg_gen_shri_tl(t_ra, t_ra, mb);
     } else {
+        target_ulong mask;
 #if defined(TARGET_PPC64)
         mb += 32;
         me += 32;
 #endif
+        mask = MASK(mb, me);
+
         if (sh == 0) {
-            tcg_gen_andi_tl(t_ra, t_rs, MASK(mb, me));
-        } else {
+            tcg_gen_andi_tl(t_ra, t_rs, mask);
+        } else if (mask <= 0xffffffffu) {
             TCGv_i32 t0 = tcg_temp_new_i32();
-
             tcg_gen_trunc_tl_i32(t0, t_rs);
             tcg_gen_rotli_i32(t0, t0, sh);
-            tcg_gen_andi_i32(t0, t0, MASK(mb, me));
+            tcg_gen_andi_i32(t0, t0, mask);
             tcg_gen_extu_i32_tl(t_ra, t0);
             tcg_temp_free_i32(t0);
+        } else {
+#if defined(TARGET_PPC64)
+            tcg_gen_deposit_i64(t_ra, t_rs, t_rs, 32, 32);
+            tcg_gen_rotli_i64(t_ra, t_ra, sh);
+            tcg_gen_andi_i64(t_ra, t_ra, mask);
+#else
+            g_assert_not_reached();
+#endif
         }
     }
     if (unlikely(Rc(ctx->opcode) != 0)) {
@@ -1707,24 +1725,37 @@ static void gen_rlwnm(DisasContext *ctx)
     TCGv t_rb = cpu_gpr[rB(ctx->opcode)];
     uint32_t mb = MB(ctx->opcode);
     uint32_t me = ME(ctx->opcode);
-    TCGv_i32 t0, t1;
+    target_ulong mask;
 
 #if defined(TARGET_PPC64)
     mb += 32;
     me += 32;
 #endif
+    mask = MASK(mb, me);
 
-    t0 = tcg_temp_new_i32();
-    t1 = tcg_temp_new_i32();
-    tcg_gen_trunc_tl_i32(t0, t_rb);
-    tcg_gen_trunc_tl_i32(t1, t_rs);
-    tcg_gen_andi_i32(t0, t0, 0x1f);
-    tcg_gen_rotl_i32(t1, t1, t0);
-    tcg_temp_free_i32(t0);
+    if (mask <= 0xffffffffu) {
+        TCGv_i32 t0 = tcg_temp_new_i32();
+        TCGv_i32 t1 = tcg_temp_new_i32();
+        tcg_gen_trunc_tl_i32(t0, t_rb);
+        tcg_gen_trunc_tl_i32(t1, t_rs);
+        tcg_gen_andi_i32(t0, t0, 0x1f);
+        tcg_gen_rotl_i32(t1, t1, t0);
+        tcg_gen_extu_i32_tl(t_ra, t1);
+        tcg_temp_free_i32(t0);
+        tcg_temp_free_i32(t1);
+    } else {
+#if defined(TARGET_PPC64)
+        TCGv_i64 t0 = tcg_temp_new_i64();
+        tcg_gen_andi_i64(t0, t_rb, 0x1f);
+        tcg_gen_deposit_i64(t_ra, t_rs, t_rs, 32, 32);
+        tcg_gen_rotl_i64(t_ra, t_ra, t0);
+        tcg_temp_free_i64(t0);
+#else
+        g_assert_not_reached();
+#endif
+    }
 
-    tcg_gen_andi_i32(t1, t1, MASK(mb, me));
-    tcg_gen_extu_i32_tl(t_ra, t1);
-    tcg_temp_free_i32(t1);
+    tcg_gen_andi_tl(t_ra, t_ra, mask);
 
     if (unlikely(Rc(ctx->opcode) != 0)) {
         gen_set_Rc0(ctx, t_ra);
-- 
2.5.5

[Qemu-devel] [PULL 06/18] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback

[David Gibson]

From: Igor Mammedov <imammedo@redhat.com>

pre_plug callback is to be called before device.realize() is executed.
This would allow to check/set device's properties from HotplugHandler.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/core/hotplug.c    | 11 +++++++++++
 hw/core/qdev.c       |  9 ++++++++-
 include/hw/hotplug.h | 14 +++++++++++++-
 3 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c
index 645cfca..17ac986 100644
--- a/hw/core/hotplug.c
+++ b/hw/core/hotplug.c
@@ -13,6 +13,17 @@
 #include "hw/hotplug.h"
 #include "qemu/module.h"
 
+void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
+                              DeviceState *plugged_dev,
+                              Error **errp)
+{
+    HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler);
+
+    if (hdc->pre_plug) {
+        hdc->pre_plug(plug_handler, plugged_dev, errp);
+    }
+}
+
 void hotplug_handler_plug(HotplugHandler *plug_handler,
                           DeviceState *plugged_dev,
                           Error **errp)
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index dcc00f8..6680089 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -902,6 +902,14 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
             g_free(name);
         }
 
+        hotplug_ctrl = qdev_get_hotplug_handler(dev);
+        if (hotplug_ctrl) {
+            hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err);
+            if (local_err != NULL) {
+                goto fail;
+            }
+        }
+
         if (dc->realize) {
             dc->realize(dev, &local_err);
         }
@@ -912,7 +920,6 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
 
         DEVICE_LISTENER_CALL(realize, Forward, dev);
 
-        hotplug_ctrl = qdev_get_hotplug_handler(dev);
         if (hotplug_ctrl) {
             hotplug_handler_plug(hotplug_ctrl, dev, &local_err);
         }
diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h
index da1d0e4..c0db869 100644
--- a/include/hw/hotplug.h
+++ b/include/hw/hotplug.h
@@ -45,7 +45,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler,
  * hardware (un)plug functions.
  *
  * @parent: Opaque parent interface.
- * @plug: plug callback.
+ * @pre_plug: pre plug callback called at start of device.realize(true)
+ * @plug: plug callback called at end of device.realize(true).
  * @unplug_request: unplug request callback.
  *                  Used as a means to initiate device unplug for devices that
  *                  require asynchronous unplug handling.
@@ -58,6 +59,7 @@ typedef struct HotplugHandlerClass {
     InterfaceClass parent;
 
     /* <public> */
+    hotplug_fn pre_plug;
     hotplug_fn plug;
     hotplug_fn unplug_request;
     hotplug_fn unplug;
@@ -73,6 +75,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler,
                           Error **errp);
 
 /**
+ * hotplug_handler_pre_plug:
+ *
+ * Call #HotplugHandlerClass.pre_plug callback of @plug_handler.
+ */
+void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
+                              DeviceState *plugged_dev,
+                              Error **errp);
+
+
+/**
  * hotplug_handler_unplug_request:
  *
  * Calls #HotplugHandlerClass.unplug_request callback of @plug_handler.
-- 
2.5.5

[Qemu-devel] [PULL 07/18] cpu: Abstract CPU core type

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Add an abstract CPU core type that could be used by machines that want
to define and hotplug CPUs in core granularity.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
               [Integer core property]
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
[dwg: changed property names to 'core-id' and 'nr-threads']
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/cpu/Makefile.objs  |  1 +
 hw/cpu/core.c         | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++
 include/hw/cpu/core.h | 31 ++++++++++++++++++
 3 files changed, 120 insertions(+)
 create mode 100644 hw/cpu/core.c
 create mode 100644 include/hw/cpu/core.h

diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
index 0954a18..942a4bb 100644
--- a/hw/cpu/Makefile.objs
+++ b/hw/cpu/Makefile.objs
@@ -2,4 +2,5 @@ obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_REALVIEW) += realview_mpcore.o
 obj-$(CONFIG_A9MPCORE) += a9mpcore.o
 obj-$(CONFIG_A15MPCORE) += a15mpcore.o
+obj-y += core.o
 
diff --git a/hw/cpu/core.c b/hw/cpu/core.c
new file mode 100644
index 0000000..eff90c1
--- /dev/null
+++ b/hw/cpu/core.c
@@ -0,0 +1,88 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+#include "qapi/visitor.h"
+#include "qapi/error.h"
+#include "sysemu/cpus.h"
+
+static void core_prop_get_core_id(Object *obj, Visitor *v, const char *name,
+                                  void *opaque, Error **errp)
+{
+    CPUCore *core = CPU_CORE(obj);
+    int64_t value = core->core_id;
+
+    visit_type_int(v, name, &value, errp);
+}
+
+static void core_prop_set_core_id(Object *obj, Visitor *v, const char *name,
+                                  void *opaque, Error **errp)
+{
+    CPUCore *core = CPU_CORE(obj);
+    Error *local_err = NULL;
+    int64_t value;
+
+    visit_type_int(v, name, &value, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    core->core_id = value;
+}
+
+static void core_prop_get_nr_threads(Object *obj, Visitor *v, const char *name,
+                                     void *opaque, Error **errp)
+{
+    CPUCore *core = CPU_CORE(obj);
+    int64_t value = core->nr_threads;
+
+    visit_type_int(v, name, &value, errp);
+}
+
+static void core_prop_set_nr_threads(Object *obj, Visitor *v, const char *name,
+                                     void *opaque, Error **errp)
+{
+    CPUCore *core = CPU_CORE(obj);
+    Error *local_err = NULL;
+    int64_t value;
+
+    visit_type_int(v, name, &value, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    core->nr_threads = value;
+}
+
+static void cpu_core_instance_init(Object *obj)
+{
+    CPUCore *core = CPU_CORE(obj);
+
+    object_property_add(obj, "core-id", "int", core_prop_get_core_id,
+                        core_prop_set_core_id, NULL, NULL, NULL);
+    object_property_add(obj, "nr-threads", "int", core_prop_get_nr_threads,
+                        core_prop_set_nr_threads, NULL, NULL, NULL);
+    core->nr_threads = smp_threads;
+}
+
+static const TypeInfo cpu_core_type_info = {
+    .name = TYPE_CPU_CORE,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .instance_size = sizeof(CPUCore),
+    .instance_init = cpu_core_instance_init,
+};
+
+static void cpu_core_register_types(void)
+{
+    type_register_static(&cpu_core_type_info);
+}
+
+type_init(cpu_core_register_types)
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
new file mode 100644
index 0000000..4540a7d
--- /dev/null
+++ b/include/hw/cpu/core.h
@@ -0,0 +1,31 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_CPU_CORE_H
+#define HW_CPU_CORE_H
+
+#include "qemu/osdep.h"
+#include "hw/qdev.h"
+
+#define TYPE_CPU_CORE "cpu-core"
+
+#define CPU_CORE(obj) \
+    OBJECT_CHECK(CPUCore, (obj), TYPE_CPU_CORE)
+
+typedef struct CPUCore {
+    /*< private >*/
+    DeviceState parent_obj;
+
+    /*< public >*/
+    int core_id;
+    int nr_threads;
+} CPUCore;
+
+#define CPU_CORE_PROP_CORE_ID "core-id"
+
+#endif
-- 
2.5.5

[Qemu-devel] [PULL 08/18] xics, xics_kvm: Handle CPU unplug correctly

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

XICS is setup for each CPU during initialization. Provide a routine
to undo the same when CPU is unplugged. While here, move ss->cs management
into xics from xics_kvm since there is nothing KVM specific in it.
Also ensure xics reset doesn't set irq for CPUs that are already unplugged.

This allows reboot of a VM that has undergone CPU hotplug and unplug
to work correctly.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/intc/xics.c        | 14 ++++++++++++++
 hw/intc/xics_kvm.c    |  8 ++++----
 include/hw/ppc/xics.h |  1 +
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/hw/intc/xics.c b/hw/intc/xics.c
index 8659be0..cce7f3d 100644
--- a/hw/intc/xics.c
+++ b/hw/intc/xics.c
@@ -48,6 +48,18 @@ static int get_cpu_index_by_dt_id(int cpu_dt_id)
     return -1;
 }
 
+void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu)
+{
+    CPUState *cs = CPU(cpu);
+    ICPState *ss = &icp->ss[cs->cpu_index];
+
+    assert(cs->cpu_index < icp->nr_servers);
+    assert(cs == ss->cs);
+
+    ss->output = NULL;
+    ss->cs = NULL;
+}
+
 void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
 {
     CPUState *cs = CPU(cpu);
@@ -57,6 +69,8 @@ void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
 
     assert(cs->cpu_index < icp->nr_servers);
 
+    ss->cs = cs;
+
     if (info->cpu_setup) {
         info->cpu_setup(icp, cpu);
     }
diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c
index 55fd801..b17d6a9 100644
--- a/hw/intc/xics_kvm.c
+++ b/hw/intc/xics_kvm.c
@@ -114,8 +114,10 @@ static void icp_kvm_reset(DeviceState *dev)
     icp->pending_priority = 0xff;
     icp->mfrr = 0xff;
 
-    /* Make all outputs are deasserted */
-    qemu_set_irq(icp->output, 0);
+    /* Make all outputs as deasserted only if the CPU thread is in use */
+    if (icp->output) {
+        qemu_set_irq(icp->output, 0);
+    }
 
     icp_set_kvm_state(icp, 1);
 }
@@ -348,8 +350,6 @@ static void xics_kvm_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
     if (icpkvm->kernel_xics_fd != -1) {
         int ret;
 
-        ss->cs = cs;
-
         ret = kvm_vcpu_enable_cap(cs, KVM_CAP_IRQ_XICS, 0,
                                   icpkvm->kernel_xics_fd, kvm_arch_vcpu_id(cs));
         if (ret < 0) {
diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
index f60b06a..9091054 100644
--- a/include/hw/ppc/xics.h
+++ b/include/hw/ppc/xics.h
@@ -167,5 +167,6 @@ int xics_alloc_block(XICSState *icp, int src, int num, bool lsi, bool align,
 void xics_free(XICSState *icp, int irq, int num);
 
 void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu);
+void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu);
 
 #endif /* __XICS_H__ */
-- 
2.5.5

[Qemu-devel] [PULL 09/18] spapr_drc: Prevent detach racing against attach for CPU DR

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

If a CPU is hot removed while hotplug of the same is still in progress,
the guest crashes. Prevent this by ensuring that detach is done only
after attach has completed.

The existing code already prevents such race for PCI hotplug. However
given that CPU is a logical DR unlike PCI and starts with ISOLATED
state, we need a logic that works for CPU too.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
               [Don't set awaiting_attach for PCI devices]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr_drc.c         | 12 ++++++++++++
 include/hw/ppc/spapr_drc.h |  1 +
 2 files changed, 13 insertions(+)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 94c875d..d276db3a 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -140,6 +140,8 @@ static uint32_t set_allocation_state(sPAPRDRConnector *drc,
             DPRINTFN("finalizing device removal");
             drck->detach(drc, DEVICE(drc->dev), drc->detach_cb,
                          drc->detach_cb_opaque, NULL);
+        } else if (drc->allocation_state == SPAPR_DR_ALLOCATION_STATE_USABLE) {
+            drc->awaiting_allocation = false;
         }
     }
     return RTAS_OUT_SUCCESS;
@@ -373,6 +375,10 @@ static void attach(sPAPRDRConnector *drc, DeviceState *d, void *fdt,
     drc->signalled = (drc->type != SPAPR_DR_CONNECTOR_TYPE_PCI)
                      ? true : coldplug;
 
+    if (drc->type != SPAPR_DR_CONNECTOR_TYPE_PCI) {
+        drc->awaiting_allocation = true;
+    }
+
     object_property_add_link(OBJECT(drc), "device",
                              object_get_typename(OBJECT(drc->dev)),
                              (Object **)(&drc->dev),
@@ -421,6 +427,12 @@ static void detach(sPAPRDRConnector *drc, DeviceState *d,
         return;
     }
 
+    if (drc->awaiting_allocation) {
+        drc->awaiting_release = true;
+        DPRINTFN("awaiting allocation to complete before removal");
+        return;
+    }
+
     drc->indicator_state = SPAPR_DR_INDICATOR_STATE_INACTIVE;
 
     if (drc->detach_cb) {
diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
index fa21ba0..08e8411 100644
--- a/include/hw/ppc/spapr_drc.h
+++ b/include/hw/ppc/spapr_drc.h
@@ -152,6 +152,7 @@ typedef struct sPAPRDRConnector {
 
     bool awaiting_release;
     bool signalled;
+    bool awaiting_allocation;
 
     /* device pointer, via link property */
     DeviceState *dev;
-- 
2.5.5

[Qemu-devel] [PULL 10/18] qom: API to get instance_size of a type

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Add an API object_type_get_size(const char *typename) that returns the
instance_size of the give typename.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 include/qom/object.h | 8 +++++++-
 qom/object.c         | 8 ++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/include/qom/object.h b/include/qom/object.h
index 99de539..2f8ac47 100644
--- a/include/qom/object.h
+++ b/include/qom/object.h
@@ -1607,5 +1607,11 @@ int object_child_foreach_recursive(Object *obj,
  */
 Object *container_get(Object *root, const char *path);
 
-
+/**
+ * object_type_get_instance_size:
+ * @typename: Name of the Type whose instance_size is required
+ *
+ * Returns the instance_size of the given @typename.
+ */
+size_t object_type_get_instance_size(const char *typename);
 #endif
diff --git a/qom/object.c b/qom/object.c
index 3bc8a00..0311414 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -202,6 +202,14 @@ static size_t type_object_get_size(TypeImpl *ti)
     return 0;
 }
 
+size_t object_type_get_instance_size(const char *typename)
+{
+    TypeImpl *type = type_get_by_name(typename);
+
+    g_assert(type != NULL);
+    return type_object_get_size(type);
+}
+
 static bool type_is_ancestor(TypeImpl *type, TypeImpl *target_type)
 {
     assert(target_type);
-- 
2.5.5

[Qemu-devel] [PULL 11/18] spapr: Abstract CPU core device and type specific core devices

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Add sPAPR specific abastract CPU core device that is based on generic
CPU core device. Use this as base type to create sPAPR CPU specific core
devices.

TODO:
- Add core types for other remaining CPU types
- Handle CPU model alias correctly

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/Makefile.objs            |   1 +
 hw/ppc/spapr.c                  |   3 +-
 hw/ppc/spapr_cpu_core.c         | 160 ++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h          |   1 +
 include/hw/ppc/spapr_cpu_core.h |  29 ++++++++
 target-ppc/kvm.c                |  28 +++++++
 6 files changed, 220 insertions(+), 2 deletions(-)
 create mode 100644 hw/ppc/spapr_cpu_core.c
 create mode 100644 include/hw/ppc/spapr_cpu_core.h

diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
index c1ffc77..5cc6608 100644
--- a/hw/ppc/Makefile.objs
+++ b/hw/ppc/Makefile.objs
@@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
 obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
 obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
 obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
+obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
 ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
 obj-y += spapr_pci_vfio.o
 endif
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 9a4a803..48df0e4 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1623,8 +1623,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
     machine->boot_order = g_strdup(boot_device);
 }
 
-static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
-                           Error **errp)
+void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
 {
     CPUPPCState *env = &cpu->env;
 
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
new file mode 100644
index 0000000..719777a
--- /dev/null
+++ b/hw/ppc/spapr_cpu_core.c
@@ -0,0 +1,160 @@
+/*
+ * sPAPR CPU core device, acts as container of CPU thread devices.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+#include "hw/ppc/spapr_cpu_core.h"
+#include "target-ppc/cpu.h"
+#include "hw/ppc/spapr.h"
+#include "hw/boards.h"
+#include "qapi/error.h"
+#include <sysemu/cpus.h>
+#include "target-ppc/kvm_ppc.h"
+
+static int spapr_cpu_core_realize_child(Object *child, void *opaque)
+{
+    Error **errp = opaque;
+    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    CPUState *cs = CPU(child);
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    object_property_set_bool(child, true, "realized", errp);
+    if (*errp) {
+        return 1;
+    }
+
+    spapr_cpu_init(spapr, cpu, errp);
+    if (*errp) {
+        return 1;
+    }
+    return 0;
+}
+
+static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
+{
+    sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
+    CPUCore *cc = CPU_CORE(OBJECT(dev));
+    const char *typename = object_class_get_name(sc->cpu_class);
+    size_t size = object_type_get_instance_size(typename);
+    Error *local_err = NULL;
+    Object *obj;
+    int i;
+
+    sc->threads = g_malloc0(size * cc->nr_threads);
+    for (i = 0; i < cc->nr_threads; i++) {
+        char id[32];
+        void *obj = sc->threads + i * size;
+
+        object_initialize(obj, size, typename);
+        snprintf(id, sizeof(id), "thread[%d]", i);
+        object_property_add_child(OBJECT(sc), id, obj, &local_err);
+        if (local_err) {
+            goto err;
+        }
+    }
+    object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, &local_err);
+    if (local_err) {
+        goto err;
+    } else {
+        return;
+    }
+
+err:
+    while (i >= 0) {
+        obj = sc->threads + i * size;
+        object_unparent(obj);
+        i--;
+    }
+    g_free(sc->threads);
+    error_propagate(errp, local_err);
+}
+
+static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    dc->realize = spapr_cpu_core_realize;
+}
+
+/*
+ * instance_init routines from different flavours of sPAPR CPU cores.
+ * TODO: Add support for 'host' core type.
+ */
+#define SPAPR_CPU_CORE_INITFN(_type, _fname) \
+static void glue(glue(spapr_cpu_core_, _fname), _initfn(Object *obj)) \
+{ \
+    sPAPRCPUCore *core = SPAPR_CPU_CORE(obj); \
+    char *name = g_strdup_printf("%s-" TYPE_POWERPC_CPU, stringify(_type)); \
+    ObjectClass *oc = object_class_by_name(name); \
+    g_assert(oc); \
+    g_free((void *)name); \
+    core->cpu_class = oc; \
+}
+
+SPAPR_CPU_CORE_INITFN(POWER7_v2.3, POWER7);
+SPAPR_CPU_CORE_INITFN(POWER7+_v2.1, POWER7plus);
+SPAPR_CPU_CORE_INITFN(POWER8_v2.0, POWER8);
+SPAPR_CPU_CORE_INITFN(POWER8E_v2.1, POWER8E);
+
+typedef struct SPAPRCoreInfo {
+    const char *name;
+    void (*initfn)(Object *obj);
+} SPAPRCoreInfo;
+
+static const SPAPRCoreInfo spapr_cores[] = {
+    /* POWER7 and aliases */
+    { .name = "POWER7_v2.3", .initfn = spapr_cpu_core_POWER7_initfn },
+    { .name = "POWER7", .initfn = spapr_cpu_core_POWER7_initfn },
+
+    /* POWER7+ and aliases */
+    { .name = "POWER7+_v2.1", .initfn = spapr_cpu_core_POWER7plus_initfn },
+    { .name = "POWER7+", .initfn = spapr_cpu_core_POWER7plus_initfn },
+
+    /* POWER8 and aliases */
+    { .name = "POWER8_v2.0", .initfn = spapr_cpu_core_POWER8_initfn },
+    { .name = "POWER8", .initfn = spapr_cpu_core_POWER8_initfn },
+    { .name = "power8", .initfn = spapr_cpu_core_POWER8_initfn },
+
+    /* POWER8E and aliases */
+    { .name = "POWER8E_v2.1", .initfn = spapr_cpu_core_POWER8E_initfn },
+    { .name = "POWER8E", .initfn = spapr_cpu_core_POWER8E_initfn },
+
+    { .name = NULL }
+};
+
+static void spapr_cpu_core_register(const SPAPRCoreInfo *info)
+{
+    TypeInfo type_info = {
+        .parent = TYPE_SPAPR_CPU_CORE,
+        .instance_size = sizeof(sPAPRCPUCore),
+        .instance_init = info->initfn,
+    };
+
+    type_info.name = g_strdup_printf("%s-" TYPE_SPAPR_CPU_CORE, info->name);
+    type_register(&type_info);
+    g_free((void *)type_info.name);
+}
+
+static const TypeInfo spapr_cpu_core_type_info = {
+    .name = TYPE_SPAPR_CPU_CORE,
+    .parent = TYPE_CPU_CORE,
+    .abstract = true,
+    .instance_size = sizeof(sPAPRCPUCore),
+    .class_init = spapr_cpu_core_class_init,
+};
+
+static void spapr_cpu_core_register_types(void)
+{
+    const SPAPRCoreInfo *info = spapr_cores;
+
+    type_register_static(&spapr_cpu_core_type_info);
+    while (info->name) {
+        spapr_cpu_core_register(info);
+        info++;
+    }
+}
+
+type_init(spapr_cpu_core_register_types)
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 3ac85c0..f597cc2 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -582,6 +582,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
                                        uint32_t count);
 void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
                                           uint32_t count);
+void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
 
 /* rtas-configure-connector state */
 struct sPAPRConfigureConnectorState {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
new file mode 100644
index 0000000..424edec
--- /dev/null
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -0,0 +1,29 @@
+/*
+ * sPAPR CPU core device.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_SPAPR_CPU_CORE_H
+#define HW_SPAPR_CPU_CORE_H
+
+#include "hw/qdev.h"
+#include "hw/cpu/core.h"
+#include "target-ppc/cpu-qom.h"
+
+#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
+#define SPAPR_CPU_CORE(obj) \
+    OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
+
+typedef struct sPAPRCPUCore {
+    /*< private >*/
+    CPUCore parent_obj;
+
+    /*< public >*/
+    void *threads;
+    ObjectClass *cpu_class;
+} sPAPRCPUCore;
+
+#endif
diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c
index 1620864..e14da60 100644
--- a/target-ppc/kvm.c
+++ b/target-ppc/kvm.c
@@ -42,6 +42,9 @@
 #include "exec/memattrs.h"
 #include "sysemu/hostmem.h"
 #include "qemu/cutils.h"
+#if defined(TARGET_PPC64)
+#include "hw/ppc/spapr_cpu_core.h"
+#endif
 
 //#define DEBUG_KVM
 
@@ -2341,6 +2344,19 @@ PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
     return pvr_pcc;
 }
 
+#if defined(TARGET_PPC64)
+static void spapr_cpu_core_host_initfn(Object *obj)
+{
+    sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+    char *name = g_strdup_printf("%s-" TYPE_POWERPC_CPU, "host");
+    ObjectClass *oc = object_class_by_name(name);
+
+    g_assert(oc);
+    g_free((void *)name);
+    core->cpu_class = oc;
+}
+#endif
+
 static int kvm_ppc_register_host_cpu_type(void)
 {
     TypeInfo type_info = {
@@ -2358,6 +2374,18 @@ static int kvm_ppc_register_host_cpu_type(void)
     type_info.parent = object_class_get_name(OBJECT_CLASS(pvr_pcc));
     type_register(&type_info);
 
+#if defined(TARGET_PPC64)
+    type_info.name = g_strdup_printf("%s-"TYPE_SPAPR_CPU_CORE, "host");
+    type_info.parent = TYPE_SPAPR_CPU_CORE,
+    type_info.instance_size = sizeof(sPAPRCPUCore),
+    type_info.instance_init = spapr_cpu_core_host_initfn,
+    type_info.class_init = NULL;
+    type_register(&type_info);
+    g_free((void *)type_info.name);
+    type_info.instance_size = 0;
+    type_info.instance_init = NULL;
+#endif
+
     /* Register generic family CPU class for a family */
     pvr_pcc = ppc_cpu_get_family_class(pvr_pcc);
     dc = DEVICE_CLASS(pvr_pcc);
-- 
2.5.5

[Qemu-devel] [PULL 12/18] spapr: Move spapr_cpu_init() to spapr_cpu_core.c

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Start consolidating CPU init related routines in spapr_cpu_core.c. As
part of this, move spapr_cpu_init() and its dependencies from spapr.c
to spapr_cpu_core.c

No functionality change in this patch.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
[dwg: Rename TIMEBASE_FREQ to SPAPR_TIMEBASE_FREQ, since it's now in a
 public(ish) header]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c          | 50 ++-----------------------------------------------
 hw/ppc/spapr_cpu_core.c | 48 +++++++++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h  |  2 ++
 3 files changed, 52 insertions(+), 48 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 48df0e4..b04a389 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -89,8 +89,6 @@
 
 #define MIN_RMA_SLOF            128UL
 
-#define TIMEBASE_FREQ           512000000ULL
-
 #define PHANDLE_XICP            0x00001111
 
 #define HTAB_SIZE(spapr)        (1ULL << ((spapr)->htab_shift))
@@ -599,7 +597,8 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
     int index = ppc_get_vcpu_dt_id(cpu);
     uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
                        0xffffffff, 0xffffffff};
-    uint32_t tbfreq = kvm_enabled() ? kvmppc_get_tbfreq() : TIMEBASE_FREQ;
+    uint32_t tbfreq = kvm_enabled() ? kvmppc_get_tbfreq()
+        : SPAPR_TIMEBASE_FREQ;
     uint32_t cpufreq = kvm_enabled() ? kvmppc_get_clockfreq() : 1000000000;
     uint32_t page_sizes_prop[64];
     size_t page_sizes_prop_size;
@@ -1198,26 +1197,6 @@ static void ppc_spapr_reset(void)
 
 }
 
-static void spapr_cpu_reset(void *opaque)
-{
-    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
-    PowerPCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-    CPUPPCState *env = &cpu->env;
-
-    cpu_reset(cs);
-
-    /* All CPUs start halted.  CPU0 is unhalted from the machine level
-     * reset code and the rest are explicitly started up by the guest
-     * using an RTAS call */
-    cs->halted = 1;
-
-    env->spr[SPR_HIOR] = 0;
-
-    ppc_hash64_set_external_hpt(cpu, spapr->htab, spapr->htab_shift,
-                                &error_fatal);
-}
-
 static void spapr_create_nvram(sPAPRMachineState *spapr)
 {
     DeviceState *dev = qdev_create(&spapr->vio_bus->bus, "spapr-nvram");
@@ -1623,31 +1602,6 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
     machine->boot_order = g_strdup(boot_device);
 }
 
-void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
-{
-    CPUPPCState *env = &cpu->env;
-
-    /* Set time-base frequency to 512 MHz */
-    cpu_ppc_tb_init(env, TIMEBASE_FREQ);
-
-    /* Enable PAPR mode in TCG or KVM */
-    cpu_ppc_set_papr(cpu);
-
-    if (cpu->max_compat) {
-        Error *local_err = NULL;
-
-        ppc_set_compat(cpu, cpu->max_compat, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
-            return;
-        }
-    }
-
-    xics_cpu_setup(spapr->icp, cpu);
-
-    qemu_register_reset(spapr_cpu_reset, cpu);
-}
-
 /*
  * Reset routine for LMB DR devices.
  *
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 719777a..2e24e39 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -14,6 +14,54 @@
 #include "qapi/error.h"
 #include <sysemu/cpus.h>
 #include "target-ppc/kvm_ppc.h"
+#include "hw/ppc/ppc.h"
+#include "target-ppc/mmu-hash64.h"
+#include <sysemu/numa.h>
+
+static void spapr_cpu_reset(void *opaque)
+{
+    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    PowerPCCPU *cpu = opaque;
+    CPUState *cs = CPU(cpu);
+    CPUPPCState *env = &cpu->env;
+
+    cpu_reset(cs);
+
+    /* All CPUs start halted.  CPU0 is unhalted from the machine level
+     * reset code and the rest are explicitly started up by the guest
+     * using an RTAS call */
+    cs->halted = 1;
+
+    env->spr[SPR_HIOR] = 0;
+
+    ppc_hash64_set_external_hpt(cpu, spapr->htab, spapr->htab_shift,
+                                &error_fatal);
+}
+
+void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
+{
+    CPUPPCState *env = &cpu->env;
+
+    /* Set time-base frequency to 512 MHz */
+    cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ);
+
+    /* Enable PAPR mode in TCG or KVM */
+    cpu_ppc_set_papr(cpu);
+
+    if (cpu->max_compat) {
+        Error *local_err = NULL;
+
+        ppc_set_compat(cpu, cpu->max_compat, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            return;
+        }
+    }
+
+    xics_cpu_setup(spapr->icp, cpu);
+
+    qemu_register_reset(spapr_cpu_reset, cpu);
+}
 
 static int spapr_cpu_core_realize_child(Object *child, void *opaque)
 {
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index f597cc2..3277692 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -16,6 +16,8 @@ typedef struct sPAPREventLogEntry sPAPREventLogEntry;
 #define HPTE64_V_HPTE_DIRTY     0x0000000000000040ULL
 #define SPAPR_ENTRY_POINT       0x100
 
+#define SPAPR_TIMEBASE_FREQ     512000000ULL
+
 typedef struct sPAPRMachineClass sPAPRMachineClass;
 typedef struct sPAPRMachineState sPAPRMachineState;
 
-- 
2.5.5

[Qemu-devel] [PULL 13/18] spapr: convert boot CPUs into CPU core devices

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Introduce sPAPRMachineClass.dr_cpu_enabled to indicate support for
CPU core hotplug. Initialize boot time CPUs as core deivces and prevent
topologies that result in partially filled cores. Both of these are done
only if CPU core hotplug is supported.

Note: An unrelated change in the call to xics_system_init() is done
in this patch as it makes sense to use the local variable smt introduced
in this patch instead of kvmppc_smt_threads() call here.

TODO: We derive sPAPR core type by looking at -cpu <model>. However
we don't take care of "compat=" feature yet for boot time as well
as hotplug CPUs.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c                  | 73 +++++++++++++++++++++++++++++++++++------
 hw/ppc/spapr_cpu_core.c         | 58 ++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h          |  2 ++
 include/hw/ppc/spapr_cpu_core.h |  3 ++
 4 files changed, 126 insertions(+), 10 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index b04a389..52e89af 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -66,6 +66,7 @@
 
 #include "hw/compat.h"
 #include "qemu/cutils.h"
+#include "hw/ppc/spapr_cpu_core.h"
 
 #include <libfdt.h>
 
@@ -1683,7 +1684,6 @@ static void ppc_spapr_init(MachineState *machine)
     const char *kernel_filename = machine->kernel_filename;
     const char *kernel_cmdline = machine->kernel_cmdline;
     const char *initrd_filename = machine->initrd_filename;
-    PowerPCCPU *cpu;
     PCIHostState *phb;
     int i;
     MemoryRegion *sysmem = get_system_memory();
@@ -1697,6 +1697,22 @@ static void ppc_spapr_init(MachineState *machine)
     long load_limit, fw_size;
     bool kernel_le = false;
     char *filename;
+    int smt = kvmppc_smt_threads();
+    int spapr_cores = smp_cpus / smp_threads;
+    int spapr_max_cores = max_cpus / smp_threads;
+
+    if (smc->dr_cpu_enabled) {
+        if (smp_cpus % smp_threads) {
+            error_report("smp_cpus (%u) must be multiple of threads (%u)",
+                         smp_cpus, smp_threads);
+            exit(1);
+        }
+        if (max_cpus % smp_threads) {
+            error_report("max_cpus (%u) must be multiple of threads (%u)",
+                         max_cpus, smp_threads);
+            exit(1);
+        }
+    }
 
     msi_nonbroken = true;
 
@@ -1743,8 +1759,7 @@ static void ppc_spapr_init(MachineState *machine)
 
     /* Set up Interrupt Controller before we create the VCPUs */
     spapr->icp = xics_system_init(machine,
-                                  DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(),
-                                               smp_threads),
+                                  DIV_ROUND_UP(max_cpus * smt, smp_threads),
                                   XICS_IRQS, &error_fatal);
 
     if (smc->dr_lmb_enabled) {
@@ -1755,13 +1770,37 @@ static void ppc_spapr_init(MachineState *machine)
     if (machine->cpu_model == NULL) {
         machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
     }
-    for (i = 0; i < smp_cpus; i++) {
-        cpu = cpu_ppc_init(machine->cpu_model);
-        if (cpu == NULL) {
-            error_report("Unable to find PowerPC CPU definition");
-            exit(1);
+
+    if (smc->dr_cpu_enabled) {
+        char *type = spapr_get_cpu_core_type(machine->cpu_model);
+
+        spapr->cores = g_new0(Object *, spapr_max_cores);
+        for (i = 0; i < spapr_cores; i++) {
+            int core_dt_id = i * smt;
+            Object *core;
+
+            if (!object_class_by_name(type)) {
+                error_report("Unable to find sPAPR CPU Core definition");
+                exit(1);
+            }
+
+            core  = object_new(type);
+            object_property_set_int(core, smp_threads, "nr-threads",
+                                    &error_fatal);
+            object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE_ID,
+                                    &error_fatal);
+            object_property_set_bool(core, true, "realized", &error_fatal);
         }
-        spapr_cpu_init(spapr, cpu, &error_fatal);
+        g_free(type);
+    } else {
+        for (i = 0; i < smp_cpus; i++) {
+            PowerPCCPU *cpu = cpu_ppc_init(machine->cpu_model);
+            if (cpu == NULL) {
+                error_report("Unable to find PowerPC CPU definition");
+                exit(1);
+            }
+            spapr_cpu_init(spapr, cpu, &error_fatal);
+       }
     }
 
     if (kvm_enabled()) {
@@ -2227,10 +2266,19 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
     }
 }
 
+static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
+                                          DeviceState *dev, Error **errp)
+{
+    if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+        spapr_core_pre_plug(hotplug_dev, dev, errp);
+    }
+}
+
 static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
                                              DeviceState *dev)
 {
-    if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
+    if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
+        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
         return HOTPLUG_HANDLER(machine);
     }
     return NULL;
@@ -2269,11 +2317,13 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     mc->has_dynamic_sysbus = true;
     mc->pci_allow_0_address = true;
     mc->get_hotplug_handler = spapr_get_hotpug_handler;
+    hc->pre_plug = spapr_machine_device_pre_plug;
     hc->plug = spapr_machine_device_plug;
     hc->unplug = spapr_machine_device_unplug;
     mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
 
     smc->dr_lmb_enabled = true;
+    smc->dr_cpu_enabled = true;
     fwc->get_dev_path = spapr_get_fw_dev_path;
     nc->nmi_monitor_handler = spapr_nmi;
 }
@@ -2349,7 +2399,10 @@ static void spapr_machine_2_6_instance_options(MachineState *machine)
 
 static void spapr_machine_2_6_class_options(MachineClass *mc)
 {
+    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
+
     spapr_machine_2_7_class_options(mc);
+    smc->dr_cpu_enabled = false;
     SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_6);
 }
 
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 2e24e39..d747c26 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -63,6 +63,64 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
     qemu_register_reset(spapr_cpu_reset, cpu);
 }
 
+/*
+ * Return the sPAPR CPU core type for @model which essentially is the CPU
+ * model specified with -cpu cmdline option.
+ */
+char *spapr_get_cpu_core_type(const char *model)
+{
+    char *core_type;
+    gchar **model_pieces = g_strsplit(model, ",", 2);
+
+    core_type = g_strdup_printf("%s-%s", model_pieces[0], TYPE_SPAPR_CPU_CORE);
+    g_strfreev(model_pieces);
+    return core_type;
+}
+
+void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                         Error **errp)
+{
+    MachineState *machine = MACHINE(OBJECT(hotplug_dev));
+    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
+    int spapr_max_cores = max_cpus / smp_threads;
+    int index;
+    int smt = kvmppc_smt_threads();
+    Error *local_err = NULL;
+    CPUCore *cc = CPU_CORE(dev);
+    char *base_core_type = spapr_get_cpu_core_type(machine->cpu_model);
+    const char *type = object_get_typename(OBJECT(dev));
+
+    if (strcmp(base_core_type, type)) {
+        error_setg(&local_err, "CPU core type should be %s", base_core_type);
+        goto out;
+    }
+
+    if (cc->nr_threads != smp_threads) {
+        error_setg(&local_err, "threads must be %d", smp_threads);
+        goto out;
+    }
+
+    if (cc->core_id % smt) {
+        error_setg(&local_err, "invalid core id %d\n", cc->core_id);
+        goto out;
+    }
+
+    index = cc->core_id / smt;
+    if (index < 0 || index >= spapr_max_cores) {
+        error_setg(&local_err, "core id %d out of range", cc->core_id);
+        goto out;
+    }
+
+    if (spapr->cores[index]) {
+        error_setg(&local_err, "core %d already populated", cc->core_id);
+        goto out;
+    }
+
+out:
+    g_free(base_core_type);
+    error_propagate(errp, local_err);
+}
+
 static int spapr_cpu_core_realize_child(Object *child, void *opaque)
 {
     Error **errp = opaque;
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 3277692..2a892bd 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -38,6 +38,7 @@ struct sPAPRMachineClass {
 
     /*< public >*/
     bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
+    bool dr_cpu_enabled;       /* enable dynamic-reconfig/hotplug of CPUs */
     bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
 };
 
@@ -81,6 +82,7 @@ struct sPAPRMachineState {
     /*< public >*/
     char *kvm_type;
     MemoryHotplugState hotplug_memory;
+    Object **cores;
 };
 
 #define H_SUCCESS         0
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 424edec..401381b 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -26,4 +26,7 @@ typedef struct sPAPRCPUCore {
     ObjectClass *cpu_class;
 } sPAPRCPUCore;
 
+void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                         Error **errp);
+char *spapr_get_cpu_core_type(const char *model);
 #endif
-- 
2.5.5

[Qemu-devel] [PULL 14/18] spapr: CPU hotplug support

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Set up device tree entries for the hotplugged CPU core and use the
exising RTAS event logging infrastructure to send CPU hotplug notification
to the guest.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c                  | 78 +++++++++++++++++++++++++++++++++-------
 hw/ppc/spapr_cpu_core.c         | 80 +++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_events.c           |  3 ++
 hw/ppc/spapr_rtas.c             | 24 +++++++++++++
 include/hw/ppc/spapr.h          |  2 ++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 6 files changed, 176 insertions(+), 13 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 52e89af..c444a86 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -605,6 +605,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
     size_t page_sizes_prop_size;
     uint32_t vcpus_per_socket = smp_threads * smp_cores;
     uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
+    sPAPRDRConnector *drc;
+    sPAPRDRConnectorClass *drck;
+    int drc_index;
+
+    drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
+    if (drc) {
+        drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+        drc_index = drck->get_index(drc);
+        _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
+    }
 
     /* Note: we keep CI large pages off for now because a 64K capable guest
      * provisioned with large pages might otherwise try to map a qemu
@@ -1005,6 +1015,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
         _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
     }
 
+    if (smc->dr_cpu_enabled) {
+        int offset = fdt_path_offset(fdt, "/cpus");
+        ret = spapr_drc_populate_dt(fdt, offset, NULL,
+                                    SPAPR_DR_CONNECTOR_TYPE_CPU);
+        if (ret < 0) {
+            error_report("Couldn't set up CPU DR device tree properties");
+            exit(1);
+        }
+    }
+
     _FDT((fdt_pack(fdt)));
 
     if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
@@ -1775,21 +1795,30 @@ static void ppc_spapr_init(MachineState *machine)
         char *type = spapr_get_cpu_core_type(machine->cpu_model);
 
         spapr->cores = g_new0(Object *, spapr_max_cores);
-        for (i = 0; i < spapr_cores; i++) {
+        for (i = 0; i < spapr_max_cores; i++) {
             int core_dt_id = i * smt;
-            Object *core;
-
-            if (!object_class_by_name(type)) {
-                error_report("Unable to find sPAPR CPU Core definition");
-                exit(1);
+            sPAPRDRConnector *drc =
+                spapr_dr_connector_new(OBJECT(spapr),
+                                       SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
+
+            qemu_register_reset(spapr_drc_reset, drc);
+
+            if (i < spapr_cores) {
+                char *type = spapr_get_cpu_core_type(machine->cpu_model);
+                Object *core;
+
+                if (!object_class_by_name(type)) {
+                    error_report("Unable to find sPAPR CPU Core definition");
+                    exit(1);
+                }
+
+                core  = object_new(type);
+                object_property_set_int(core, smp_threads, "nr-threads",
+                                        &error_fatal);
+                object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE_ID,
+                                        &error_fatal);
+                object_property_set_bool(core, true, "realized", &error_fatal);
             }
-
-            core  = object_new(type);
-            object_property_set_int(core, smp_threads, "nr-threads",
-                                    &error_fatal);
-            object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE_ID,
-                                    &error_fatal);
-            object_property_set_bool(core, true, "realized", &error_fatal);
         }
         g_free(type);
     } else {
@@ -2211,6 +2240,27 @@ out:
     error_propagate(errp, local_err);
 }
 
+void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
+                                    sPAPRMachineState *spapr)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+    DeviceClass *dc = DEVICE_GET_CLASS(cs);
+    int id = ppc_get_vcpu_dt_id(cpu);
+    void *fdt;
+    int offset, fdt_size;
+    char *nodename;
+
+    fdt = create_device_tree(&fdt_size);
+    nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
+    offset = fdt_add_subnode(fdt, 0, nodename);
+
+    spapr_populate_cpu_dt(cs, fdt, offset, spapr);
+    g_free(nodename);
+
+    *fdt_offset = offset;
+    return fdt;
+}
+
 static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -2255,6 +2305,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
         }
 
         spapr_memory_plug(hotplug_dev, dev, node, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+        spapr_core_plug(hotplug_dev, dev, errp);
     }
 }
 
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index d747c26..d5fa4e6 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -41,6 +41,8 @@ static void spapr_cpu_reset(void *opaque)
 void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
 {
     CPUPPCState *env = &cpu->env;
+    CPUState *cs = CPU(cpu);
+    int i;
 
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ);
@@ -58,9 +60,18 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
         }
     }
 
+    /* Set NUMA node for the added CPUs  */
+    for (i = 0; i < nb_numa_nodes; i++) {
+        if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
+            cs->numa_node = i;
+            break;
+        }
+    }
+
     xics_cpu_setup(spapr->icp, cpu);
 
     qemu_register_reset(spapr_cpu_reset, cpu);
+    spapr_cpu_reset(cpu);
 }
 
 /*
@@ -77,10 +88,74 @@ char *spapr_get_cpu_core_type(const char *model)
     return core_type;
 }
 
+void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                     Error **errp)
+{
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(OBJECT(hotplug_dev));
+    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
+    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+    CPUCore *cc = CPU_CORE(dev);
+    CPUState *cs = CPU(core->threads);
+    sPAPRDRConnector *drc;
+    sPAPRDRConnectorClass *drck;
+    Error *local_err = NULL;
+    void *fdt = NULL;
+    int fdt_offset = 0;
+    int index;
+    int smt = kvmppc_smt_threads();
+
+    drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core_id);
+    index = cc->core_id / smt;
+    spapr->cores[index] = OBJECT(dev);
+
+    if (!smc->dr_cpu_enabled) {
+        /*
+         * This is a cold plugged CPU core but the machine doesn't support
+         * DR. So skip the hotplug path ensuring that the core is brought
+         * up online with out an associated DR connector.
+         */
+        return;
+    }
+
+    g_assert(drc);
+
+    /*
+     * Setup CPU DT entries only for hotplugged CPUs. For boot time or
+     * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
+     */
+    if (dev->hotplugged) {
+        fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
+    }
+
+    drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+    drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
+    if (local_err) {
+        g_free(fdt);
+        spapr->cores[index] = NULL;
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    if (dev->hotplugged) {
+        /*
+         * Send hotplug notification interrupt to the guest only in case
+         * of hotplugged CPUs.
+         */
+        spapr_hotplug_req_add_by_index(drc);
+    } else {
+        /*
+         * Set the right DRC states for cold plugged CPU.
+         */
+        drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
+        drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
+    }
+}
+
 void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
                          Error **errp)
 {
     MachineState *machine = MACHINE(OBJECT(hotplug_dev));
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(OBJECT(hotplug_dev));
     sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
     int spapr_max_cores = max_cpus / smp_threads;
     int index;
@@ -95,6 +170,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
         goto out;
     }
 
+    if (!smc->dr_cpu_enabled && dev->hotplugged) {
+        error_setg(&local_err, "CPU hotplug not supported for this machine");
+        goto out;
+    }
+
     if (cc->nr_threads != smp_threads) {
         error_setg(&local_err, "threads must be %d", smp_threads);
         goto out;
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 049fb1b..af80992 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -449,6 +449,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
     case SPAPR_DR_CONNECTOR_TYPE_LMB:
         hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
         break;
+    case SPAPR_DR_CONNECTOR_TYPE_CPU:
+        hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
+        break;
     default:
         /* we shouldn't be signaling hotplug events for resources
          * that don't support them
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 43e2c68..dc058e5 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -36,6 +36,7 @@
 
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/spapr_vio.h"
+#include "hw/ppc/ppc.h"
 #include "qapi-event.h"
 #include "hw/boards.h"
 
@@ -164,6 +165,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
     rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
 }
 
+/*
+ * Set the timebase offset of the CPU to that of first CPU.
+ * This helps hotplugged CPU to have the correct timebase offset.
+ */
+static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
+{
+    PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+
+    cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
+}
+
+static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
+{
+    PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
+
+    if (!pcc->interrupts_big_endian(fcpu)) {
+        cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
+    }
+}
+
 static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
                            uint32_t token, uint32_t nargs,
                            target_ulong args,
@@ -200,6 +222,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
         env->nip = start;
         env->gpr[3] = r3;
         cs->halted = 0;
+        spapr_cpu_set_endianness(cpu);
+        spapr_cpu_update_tb_offset(cpu);
 
         qemu_cpu_kick(cs);
 
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 2a892bd..e1f8274 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -587,6 +587,8 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
 void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
                                           uint32_t count);
 void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
+void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
+                                    sPAPRMachineState *spapr);
 
 /* rtas-configure-connector state */
 struct sPAPRConfigureConnectorState {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 401381b..7cb0515 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -29,4 +29,6 @@ typedef struct sPAPRCPUCore {
 void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
                          Error **errp);
 char *spapr_get_cpu_core_type(const char *model);
+void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                     Error **errp);
 #endif
-- 
2.5.5

[Qemu-devel] [PULL 15/18] spapr: CPU hot unplug support

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Remove the CPU core device by removing the underlying CPU thread devices.
Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
notification to the guest. Release the vCPU object after CPU hot unplug so
that vCPU fd can be parked and reused.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c                  |  8 ++++++
 hw/ppc/spapr_cpu_core.c         | 59 +++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 3 files changed, 69 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index c444a86..1dcb9f6 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2313,8 +2313,16 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
 static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+
     if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
         error_setg(errp, "Memory hot unplug not supported by sPAPR");
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+        if (!smc->dr_cpu_enabled) {
+            error_setg(errp, "CPU hot unplug not supported on this machine");
+            return;
+        }
+        spapr_core_unplug(hotplug_dev, dev, errp);
     }
 }
 
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index d5fa4e6..3a5da09 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -38,6 +38,14 @@ static void spapr_cpu_reset(void *opaque)
                                 &error_fatal);
 }
 
+static void spapr_cpu_destroy(PowerPCCPU *cpu)
+{
+    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+
+    xics_cpu_destroy(spapr->icp, cpu);
+    qemu_unregister_reset(spapr_cpu_reset, cpu);
+}
+
 void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
 {
     CPUPPCState *env = &cpu->env;
@@ -88,6 +96,57 @@ char *spapr_get_cpu_core_type(const char *model)
     return core_type;
 }
 
+static void spapr_core_release(DeviceState *dev, void *opaque)
+{
+    sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
+    const char *typename = object_class_get_name(sc->cpu_class);
+    size_t size = object_type_get_instance_size(typename);
+    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+    CPUCore *cc = CPU_CORE(dev);
+    int smt = kvmppc_smt_threads();
+    int i;
+
+    for (i = 0; i < cc->nr_threads; i++) {
+        void *obj = sc->threads + i * size;
+        DeviceState *dev = DEVICE(obj);
+        CPUState *cs = CPU(dev);
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+        spapr_cpu_destroy(cpu);
+        cpu_remove_sync(cs);
+        object_unparent(obj);
+    }
+
+    spapr->cores[cc->core_id / smt] = NULL;
+
+    g_free(core->threads);
+    object_unparent(OBJECT(dev));
+}
+
+void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                       Error **errp)
+{
+    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+    PowerPCCPU *cpu = POWERPC_CPU(core->threads);
+    int id = ppc_get_vcpu_dt_id(cpu);
+    sPAPRDRConnector *drc =
+        spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
+    sPAPRDRConnectorClass *drck;
+    Error *local_err = NULL;
+
+    g_assert(drc);
+
+    drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+    drck->detach(drc, dev, spapr_core_release, NULL, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    spapr_hotplug_req_remove_by_index(drc);
+}
+
 void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
                      Error **errp)
 {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 7cb0515..1c9b319 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -31,4 +31,6 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
 char *spapr_get_cpu_core_type(const char *model);
 void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
                      Error **errp);
+void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                       Error **errp);
 #endif
-- 
2.5.5

[Qemu-devel] [PULL 16/18] QMP: Add query-hotpluggable-cpus

[David Gibson]

From: Igor Mammedov <imammedo@redhat.com>

It will allow mgmt to query present and hotpluggable CPU objects,
it is required from a target platform that wishes to support command
to implement and set MachineClass.query_hotpluggable_cpus callback,
which will return a list of possible CPU objects with options that
would be needed for hotplugging possible CPU objects.

There are:
'type': 'str' - QOM CPU object type for usage with device_add
'vcpus-count': 'int' - number of logical VCPU threads per
                        CPU object (mgmt needs to know)

and a set of optional fields that are to used for hotplugging a CPU
objects and would allows mgmt tools to know what/where it could be
hotplugged;
[node],[socket],[core],[thread]

For present CPUs there is a 'qom-path' field which would allow mgmt to
inspect whatever object/abstraction the target platform considers
as CPU object.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 include/hw/boards.h |  5 +++++
 monitor.c           | 13 +++++++++++++
 qapi-schema.json    | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 qmp-commands.hx     | 23 ++++++++++++++++++++++
 4 files changed, 96 insertions(+)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index d268bd0..3ed6155 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -81,6 +81,10 @@ typedef struct {
  *    Returns an array of @CPUArchId architecture-dependent CPU IDs
  *    which includes CPU IDs for present and possible to hotplug CPUs.
  *    Caller is responsible for freeing returned list.
+ * @query_hotpluggable_cpus:
+ *    Returns a @HotpluggableCPUList, which describes CPUs objects which
+ *    could be added with -device/device_add.
+ *    Caller is responsible for freeing returned list.
  */
 struct MachineClass {
     /*< private >*/
@@ -124,6 +128,7 @@ struct MachineClass {
                                            DeviceState *dev);
     unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
     CPUArchIdList *(*possible_cpu_arch_ids)(MachineState *machine);
+    HotpluggableCPUList *(*query_hotpluggable_cpus)(MachineState *machine);
 };
 
 /**
diff --git a/monitor.c b/monitor.c
index a27e115..a5d054b 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4273,3 +4273,16 @@ GICCapabilityList *qmp_query_gic_capabilities(Error **errp)
     return NULL;
 }
 #endif
+
+HotpluggableCPUList *qmp_query_hotpluggable_cpus(Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (!mc->query_hotpluggable_cpus) {
+        error_setg(errp, QERR_FEATURE_DISABLED, "query-hotpluggable-cpus");
+        return NULL;
+    }
+
+    return mc->query_hotpluggable_cpus(ms);
+}
diff --git a/qapi-schema.json b/qapi-schema.json
index 40b1db4..0964eec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4253,3 +4253,58 @@
 # Since: 2.6
 ##
 { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
+
+##
+# CpuInstanceProperties
+#
+# List of properties to be used for hotplugging a CPU instance,
+# it should be passed by management with device_add command when
+# a CPU is being hotplugged.
+#
+# Note: currently there are 4 properties that could be present
+# but management should be prepared to pass through other
+# properties with device_add command to allow for future
+# interface extension.
+#
+# @node: #optional NUMA node ID the CPU belongs to
+# @socket: #optional socket number within node/board the CPU belongs to
+# @core: #optional core number within socket the CPU belongs to
+# @thread: #optional thread number within core the CPU belongs to
+#
+# Since: 2.7
+##
+{ 'struct': 'CpuInstanceProperties',
+  'data': { '*node': 'int',
+            '*socket': 'int',
+            '*core': 'int',
+            '*thread': 'int'
+  }
+}
+
+##
+# @HotpluggableCPU
+#
+# @type: CPU object type for usage with device_add command
+# @props: list of properties to be used for hotplugging CPU
+# @vcpus-count: number of logical VCPU threads @HotpluggableCPU provides
+# @qom-path: #optional link to existing CPU object if CPU is present or
+#            omitted if CPU is not present.
+#
+# Since: 2.7
+##
+{ 'struct': 'HotpluggableCPU',
+  'data': { 'type': 'str',
+            'vcpus-count': 'int',
+            'props': 'CpuInstanceProperties',
+            '*qom-path': 'str'
+          }
+}
+
+##
+# @query-hotpluggable-cpus
+#
+# Returns: a list of HotpluggableCPU objects.
+#
+# Since: 2.7
+##
+{ 'command': 'query-hotpluggable-cpus', 'returns': ['HotpluggableCPU'] }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 780e7f2..b444c20 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -4960,3 +4960,26 @@ Example:
                 { "version": 3, "emulated": false, "kernel": true } ] }
 
 EQMP
+
+    {
+        .name       = "query-hotpluggable-cpus",
+        .args_type  = "",
+        .mhandler.cmd_new = qmp_marshal_query_hotpluggable_cpus,
+    },
+
+SQMP
+Show existing/possible CPUs
+---------------------------
+
+Arguments: None.
+
+Example for pseries machine type started with
+-smp 2,cores=2,maxcpus=4 -cpu POWER8:
+
+-> { "execute": "query-hotpluggable-cpus" }
+<- {"return": [
+     { "props": { "core": 8 }, "type": "POWER8-spapr-cpu-core",
+       "vcpus-count": 1 },
+     { "props": { "core": 0 }, "type": "POWER8-spapr-cpu-core",
+       "vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
+   ]}'
-- 
2.5.5

[Qemu-devel] [PULL 17/18] hmp: Add 'info hotpluggable-cpus' HMP command

[David Gibson]

From: Bharata B Rao <bharata@linux.vnet.ibm.com>

This is the HMP equivalent for QMP query-hotpluggable-cpus.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
[dwg: Fixed problem with printf formats on 32-bit host]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hmp-commands-info.hx | 14 ++++++++++++++
 hmp.c                | 42 ++++++++++++++++++++++++++++++++++++++++++
 hmp.h                |  1 +
 3 files changed, 57 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 52539c3..7da9e6c 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -800,6 +800,20 @@ STEXI
 Display the latest dump status.
 ETEXI
 
+    {
+        .name       = "hotpluggable-cpus",
+        .args_type  = "",
+        .params     = "",
+        .help       = "Show information about hotpluggable CPUs",
+        .mhandler.cmd = hmp_hotpluggable_cpus,
+    },
+
+STEXI
+@item info hotpluggable-cpus
+@findex hotpluggable-cpus
+Show information about hotpluggable CPUs
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/hmp.c b/hmp.c
index 30897af..997a768 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2433,3 +2433,45 @@ void hmp_info_dump(Monitor *mon, const QDict *qdict)
 
     qapi_free_DumpQueryResult(result);
 }
+
+void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
+{
+    Error *err = NULL;
+    HotpluggableCPUList *l = qmp_query_hotpluggable_cpus(&err);
+    HotpluggableCPUList *saved = l;
+    CpuInstanceProperties *c;
+
+    if (err != NULL) {
+        hmp_handle_error(mon, &err);
+        return;
+    }
+
+    monitor_printf(mon, "Hotpluggable CPUs:\n");
+    while (l) {
+        monitor_printf(mon, "  type: \"%s\"\n", l->value->type);
+        monitor_printf(mon, "  vcpus_count: \"%" PRIu64 "\"\n",
+                       l->value->vcpus_count);
+        if (l->value->has_qom_path) {
+            monitor_printf(mon, "  qom_path: \"%s\"\n", l->value->qom_path);
+        }
+
+        c = l->value->props;
+        monitor_printf(mon, "  CPUInstance Properties:\n");
+        if (c->has_node) {
+            monitor_printf(mon, "    node: \"%" PRIu64 "\"\n", c->node);
+        }
+        if (c->has_socket) {
+            monitor_printf(mon, "    socket: \"%" PRIu64 "\"\n", c->socket);
+        }
+        if (c->has_core) {
+            monitor_printf(mon, "    core: \"%" PRIu64 "\"\n", c->core);
+        }
+        if (c->has_thread) {
+            monitor_printf(mon, "    thread: \"%" PRIu64 "\"\n", c->thread);
+        }
+
+        l = l->next;
+    }
+
+    qapi_free_HotpluggableCPUList(saved);
+}
diff --git a/hmp.h b/hmp.h
index 093d65f..f5d9749 100644
--- a/hmp.h
+++ b/hmp.h
@@ -132,5 +132,6 @@ void hmp_rocker_ports(Monitor *mon, const QDict *qdict);
 void hmp_rocker_of_dpa_flows(Monitor *mon, const QDict *qdict);
 void hmp_rocker_of_dpa_groups(Monitor *mon, const QDict *qdict);
 void hmp_info_dump(Monitor *mon, const QDict *qdict);
+void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict);
 
 #endif
-- 
2.5.5

[Qemu-devel] [PULL 18/18] spapr: implement query-hotpluggable-cpus callback

[David Gibson]

From: Igor Mammedov <imammedo@redhat.com>

It returns a list of present/possible to hotplug CPU
objects with a list of properties to use with
device_add.

in spapr case returned list would looks like:
-> { "execute": "query-hotpluggable-cpus" }
<- {"return": [
     { "props": { "core": 8 }, "type": "POWER8-spapr-cpu-core",
       "vcpus-count": 2 },
     { "props": { "core": 0 }, "type": "POWER8-spapr-cpu-core",
       "vcpus-count": 2,
       "qom-path": "/machine/unattached/device[0]"}
   ]}'

TODO:
  add 'node' property for core <-> numa node mapping

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 hw/ppc/spapr.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1dcb9f6..778fa25 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -67,6 +67,7 @@
 #include "hw/compat.h"
 #include "qemu/cutils.h"
 #include "hw/ppc/spapr_cpu_core.h"
+#include "qmp-commands.h"
 
 #include <libfdt.h>
 
@@ -2351,6 +2352,38 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
     return cpu_index / smp_threads / smp_cores;
 }
 
+static HotpluggableCPUList *spapr_query_hotpluggable_cpus(MachineState *machine)
+{
+    int i;
+    HotpluggableCPUList *head = NULL;
+    sPAPRMachineState *spapr = SPAPR_MACHINE(machine);
+    int spapr_max_cores = max_cpus / smp_threads;
+    int smt = kvmppc_smt_threads();
+
+    for (i = 0; i < spapr_max_cores; i++) {
+        HotpluggableCPUList *list_item = g_new0(typeof(*list_item), 1);
+        HotpluggableCPU *cpu_item = g_new0(typeof(*cpu_item), 1);
+        CpuInstanceProperties *cpu_props = g_new0(typeof(*cpu_props), 1);
+
+        cpu_item->type = spapr_get_cpu_core_type(machine->cpu_model);
+        cpu_item->vcpus_count = smp_threads;
+        cpu_props->has_core = true;
+        cpu_props->core = i * smt;
+        /* TODO: add 'has_node/node' here to describe
+           to which node core belongs */
+
+        cpu_item->props = cpu_props;
+        if (spapr->cores[i]) {
+            cpu_item->has_qom_path = true;
+            cpu_item->qom_path = object_get_canonical_path(spapr->cores[i]);
+        }
+        list_item->value = cpu_item;
+        list_item->next = head;
+        head = list_item;
+    }
+    return head;
+}
+
 static void spapr_machine_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -2381,6 +2414,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     hc->plug = spapr_machine_device_plug;
     hc->unplug = spapr_machine_device_unplug;
     mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
+    mc->query_hotpluggable_cpus = spapr_query_hotpluggable_cpus;
 
     smc->dr_lmb_enabled = true;
     smc->dr_cpu_enabled = true;
-- 
2.5.5

Re: [Qemu-devel] [PULL 00/18] ppc-for-2.7 queue 20160617

[Peter Maydell]

On 17 June 2016 at 07:36, David Gibson <david@gibson.dropbear.id.au> wrote:
> The following changes since commit 585fcd4b11070b3220685fc54ecca1991cdeb161:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2016-06-16 17:58:45 +0100)
>
> are available in the git repository at:
>
>   git://github.com/dgibson/qemu.git tags/ppc-for-2.7-20160617
>
> for you to fetch changes up to 2474bfd4603b2d354fdb4001b083b7c72bff627f:
>
>   spapr: implement query-hotpluggable-cpus callback (2016-06-17 16:33:49 +1000)
>
> ----------------------------------------------------------------
> ppc patch queue for 2016-06-17
>
> Here's the current accumulated set of spapr, ppc and related patches.
>   * The big thing in here is CPU hotplug for spapr
>     - This includes a number of acked generic changes adding new
>       infrastructure for hotplugging cpu cores
>   * A number of TCG bug fixes are also included
>   * This adds a new testcase to make it harder to accidentally break
>     Macintosh (and other openbios) platforms

Applied, thanks.

-- PMM

Re: [Qemu-devel] [PULL 02/18] ppc / sparc: Add a tester for checking whether OpenBIOS runs successfully

[Thomas Huth]

On 17.06.2016 08:36, David Gibson wrote:
> From: Thomas Huth <thuth@redhat.com>
> 
> Since the mac99 and g3beige PowerPC machines recently broke without
> being noticed, it would be good to have a tester for "make check"
> that detects such issues immediately. A simple way to test the firmware
> of these machines is to use the "-prom-env" parameter of QEMU. This
> parameter can be used to put some Forth code into the 'boot-command'
> firmware variable which then can signal success to the tester by
> writing a magic value to a known memory location. And since some of the
> Sparc machines are also using OpenBIOS, they are now tested with this
> prom-env-tester, too.
> 
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> [dwg: Removed sparc64, because it trips a TCG bug on 32-bit hosts]
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  tests/Makefile.include |  6 ++++
>  tests/prom-env-test.c  | 90 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 96 insertions(+)
>  create mode 100644 tests/prom-env-test.c
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 6135875..33fcdcb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -259,6 +259,11 @@ check-qtest-ppc-y += tests/boot-order-test$(EXESUF)
>  check-qtest-ppc64-y += tests/boot-order-test$(EXESUF)
>  check-qtest-ppc64-y += tests/spapr-phb-test$(EXESUF)
>  gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c
> +check-qtest-ppc-y = tests/prom-env-test$(EXESUF)
> +check-qtest-ppc64-y = tests/prom-env-test$(EXESUF)
> +check-qtest-sparc-y = tests/prom-env-test$(EXESUF)
> +#Disabled for now, triggers a TCG bug on 32-bit hosts
> +#check-qtest-sparc64-y = tests/prom-env-test$(EXESUF)

D'Oh! I just realized that there is a stupid copy-n-paste error in these
changes: It should be '+=' instead of '=', otherwise the endianness-test
for these targets is not run anymore... :-( Sorry!

I'll send a fix immediately...

 Thomas

Re: [Qemu-devel] [PULL 15/18] spapr: CPU hot unplug support

[Igor Mammedov]

On Fri, 17 Jun 2016 16:36:36 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> From: Bharata B Rao <bharata@linux.vnet.ibm.com>
> 
> Remove the CPU core device by removing the underlying CPU thread devices.
> Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
> notification to the guest. Release the vCPU object after CPU hot unplug so
> that vCPU fd can be parked and reused.
> 
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[...]


Bharata,

Here is some notes I've made while auditing spapr cpu hotplug code.
  
spapr_core_release() should be spapr_core_unrealize()
except of machine related
 spapr->cores[cc->core_id / smt] = NULL;
which should go to spapr_core_unplug()

> +static void spapr_core_release(DeviceState *dev, void *opaque)
> +{
> +    sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
> +    const char *typename = object_class_get_name(sc->cpu_class);
> +    size_t size = object_type_get_instance_size(typename);
> +    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> +    CPUCore *cc = CPU_CORE(dev);
> +    int smt = kvmppc_smt_threads();
> +    int i;
> +
> +    for (i = 0; i < cc->nr_threads; i++) {
> +        void *obj = sc->threads + i * size;
> +        DeviceState *dev = DEVICE(obj);
> +        CPUState *cs = CPU(dev);
> +        PowerPCCPU *cpu = POWERPC_CPU(cs);
> +
> +        spapr_cpu_destroy(cpu);
> +        cpu_remove_sync(cs);
> +        object_unparent(obj);
> +    }
> +
> +    spapr->cores[cc->core_id / smt] = NULL;
> +
> +    g_free(core->threads);
> +    object_unparent(OBJECT(dev));
> +}
> +

spapr_core_[un]plug() functions belong to machine code and should
be in hw/ppc/spapr.c

> +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                       Error **errp)
> +{
> +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> +    PowerPCCPU *cpu = POWERPC_CPU(core->threads);
> +    int id = ppc_get_vcpu_dt_id(cpu);
> +    sPAPRDRConnector *drc =
> +        spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> +    sPAPRDRConnectorClass *drck;
> +    Error *local_err = NULL;
> +
> +    g_assert(drc);
> +
> +    drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> +    drck->detach(drc, dev, spapr_core_release, NULL, &local_err);

Could you explain call flow during cpu unplug?

My expectations were that unplug_request() handler asks for CPU removal
and unplug() handler removes CPU.
It's obviously messed up somehow.

> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +
> +    spapr_hotplug_req_remove_by_index(drc);
> +}
> +
>  void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
>                       Error **errp)
>  {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index 7cb0515..1c9b319 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -31,4 +31,6 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
>  char *spapr_get_cpu_core_type(const char *model);
>  void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
>                       Error **errp);
> +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                       Error **errp);
>  #endif

Re: [Qemu-devel] [PULL 15/18] spapr: CPU hot unplug support

[Bharata B Rao]

On Thu, Jan 26, 2017 at 12:32:58PM +0100, Igor Mammedov wrote:
> On Fri, 17 Jun 2016 16:36:36 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > From: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > 
> > Remove the CPU core device by removing the underlying CPU thread devices.
> > Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
> > notification to the guest. Release the vCPU object after CPU hot unplug so
> > that vCPU fd can be parked and reused.
> > 
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> [...]
> 
> 
> Bharata,
> 
> Here is some notes I've made while auditing spapr cpu hotplug code.
>   
> spapr_core_release() should be spapr_core_unrealize()
> except of machine related
>  spapr->cores[cc->core_id / smt] = NULL;
> which should go to spapr_core_unplug()

There were some issues in calling cpu_remove_[sync] from unrealize
path. I know that x86 does that way. let me remember and get back on this.

> 
> > +static void spapr_core_release(DeviceState *dev, void *opaque)
> > +{
> > +    sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
> > +    const char *typename = object_class_get_name(sc->cpu_class);
> > +    size_t size = object_type_get_instance_size(typename);
> > +    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > +    CPUCore *cc = CPU_CORE(dev);
> > +    int smt = kvmppc_smt_threads();
> > +    int i;
> > +
> > +    for (i = 0; i < cc->nr_threads; i++) {
> > +        void *obj = sc->threads + i * size;
> > +        DeviceState *dev = DEVICE(obj);
> > +        CPUState *cs = CPU(dev);
> > +        PowerPCCPU *cpu = POWERPC_CPU(cs);
> > +
> > +        spapr_cpu_destroy(cpu);
> > +        cpu_remove_sync(cs);
> > +        object_unparent(obj);
> > +    }
> > +
> > +    spapr->cores[cc->core_id / smt] = NULL;
> > +
> > +    g_free(core->threads);
> > +    object_unparent(OBJECT(dev));
> > +}
> > +
> 
> spapr_core_[un]plug() functions belong to machine code and should
> be in hw/ppc/spapr.c

That's how the series started, but eventually we consolidated all
core related routines in spapr_cpu_core.c

> 
> > +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > +                       Error **errp)
> > +{
> > +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > +    PowerPCCPU *cpu = POWERPC_CPU(core->threads);
> > +    int id = ppc_get_vcpu_dt_id(cpu);
> > +    sPAPRDRConnector *drc =
> > +        spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> > +    sPAPRDRConnectorClass *drck;
> > +    Error *local_err = NULL;
> > +
> > +    g_assert(drc);
> > +
> > +    drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > +    drck->detach(drc, dev, spapr_core_release, NULL, &local_err);
> 
> Could you explain call flow during cpu unplug?

In response to unplug request, spapr_core_unplug() gets called which
does a detach() on the associated DRC object. The detach() registers
a callback (spapr_core_release)  and signals the guest about the unplug
request.

When the guest is ready to let go of the CPU core, DRC subsystem ends up
calling the callback spapr_core_release. For each of the CPU thread objects
of the core, spapr_core_release will call cpu_remove_sync() and waits
for the CPU to be really removed. cpu_remove will result in CPU unrealize
function being called (ppc_cpu_unrealizefn) for each of the removed
CPU.

After we are done waiting for all the threads' removal, the core object is
ready for removal.

> 
> My expectations were that unplug_request() handler asks for CPU removal
> and unplug() handler removes CPU.
> It's obviously messed up somehow.

When we did CPU unplug, we didn't really implement ->unplug_request() for
sPAPR. It was added later when memory unplug came in.

Regards,
Bharata.

Re: [Qemu-devel] [PULL 15/18] spapr: CPU hot unplug support

[Igor Mammedov]

On Thu, 26 Jan 2017 19:56:35 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:

> On Thu, Jan 26, 2017 at 12:32:58PM +0100, Igor Mammedov wrote:
> > On Fri, 17 Jun 2016 16:36:36 +1000
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> > 
> > > From: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > 
> > > Remove the CPU core device by removing the underlying CPU thread devices.
> > > Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
> > > notification to the guest. Release the vCPU object after CPU hot unplug so
> > > that vCPU fd can be parked and reused.
> > > 
> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > [...]
> > 
> > 
> > Bharata,
> > 
> > Here is some notes I've made while auditing spapr cpu hotplug code.
> >   
> > spapr_core_release() should be spapr_core_unrealize()
> > except of machine related
> >  spapr->cores[cc->core_id / smt] = NULL;
> > which should go to spapr_core_unplug()
> 
> There were some issues in calling cpu_remove_[sync] from unrealize
> path. I know that x86 does that way. let me remember and get back on this.
on the first glance it doesn't look like there should be issues with
making it spapr_core_unrealize(), but since it's way out of scope
numa rework I'd leave 'fixing' it upto you. 

> 
> > 
> > > +static void spapr_core_release(DeviceState *dev, void *opaque)
> > > +{
> > > +    sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
> > > +    const char *typename = object_class_get_name(sc->cpu_class);
> > > +    size_t size = object_type_get_instance_size(typename);
> > > +    sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > +    CPUCore *cc = CPU_CORE(dev);
> > > +    int smt = kvmppc_smt_threads();
> > > +    int i;
> > > +
> > > +    for (i = 0; i < cc->nr_threads; i++) {
> > > +        void *obj = sc->threads + i * size;
> > > +        DeviceState *dev = DEVICE(obj);
> > > +        CPUState *cs = CPU(dev);
> > > +        PowerPCCPU *cpu = POWERPC_CPU(cs);
> > > +
> > > +        spapr_cpu_destroy(cpu);
> > > +        cpu_remove_sync(cs);
> > > +        object_unparent(obj);
> > > +    }
> > > +
> > > +    spapr->cores[cc->core_id / smt] = NULL;
> > > +
> > > +    g_free(core->threads);
> > > +    object_unparent(OBJECT(dev));
> > > +}
> > > +
> > 
> > spapr_core_[un]plug() functions belong to machine code and should
> > be in hw/ppc/spapr.c
> 
> That's how the series started, but eventually we consolidated all
> core related routines in spapr_cpu_core.c

Since spapr_core_[un]plug() manage spapr machine state and not internal
core state, I'd like to move them close to other machine code (spapr.c)
if you don't mind.

> 
> > 
> > > +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > +                       Error **errp)
> > > +{
> > > +    sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > +    PowerPCCPU *cpu = POWERPC_CPU(core->threads);
> > > +    int id = ppc_get_vcpu_dt_id(cpu);
> > > +    sPAPRDRConnector *drc =
> > > +        spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> > > +    sPAPRDRConnectorClass *drck;
> > > +    Error *local_err = NULL;
> > > +
> > > +    g_assert(drc);
> > > +
> > > +    drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > > +    drck->detach(drc, dev, spapr_core_release, NULL, &local_err);
> > 
> > Could you explain call flow during cpu unplug?
> 
> In response to unplug request, spapr_core_unplug() gets called which
> does a detach() on the associated DRC object. The detach() registers
> a callback (spapr_core_release)  and signals the guest about the unplug
> request.
> 
> When the guest is ready to let go of the CPU core, DRC subsystem ends up
> calling the callback spapr_core_release. For each of the CPU thread objects
> of the core, spapr_core_release will call cpu_remove_sync() and waits
> for the CPU to be really removed. cpu_remove will result in CPU unrealize
> function being called (ppc_cpu_unrealizefn) for each of the removed
> CPU.
> 
> After we are done waiting for all the threads' removal, the core object is
> ready for removal.
> 
> > 
> > My expectations were that unplug_request() handler asks for CPU removal
> > and unplug() handler removes CPU.
> > It's obviously messed up somehow.
> 
> When we did CPU unplug, we didn't really implement ->unplug_request() for
> sPAPR. It was added later when memory unplug came in.
It ended up that spapr_core_unplug() is called from both ->unplug_request()
and ->unplug(). Where ->unplug() is dead path that's never called.
I'll send patches to fix hot-unlpug flow to conform to generic hotplug
pattern
   ->unplug_request() -> register callback
and
   callback ->unplug() -> release_core()

> 
> Regards,
> Bharata.
> 
>