[RFC 00/41] qom-topo: Abstract Everything about CPU Topology

[RFC 00/41] qom-topo: Abstract Everything about CPU Topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Hi list,

This series is our latest attempt after the previous RFC [1] about
hybrid topology support, which is based on the commit 4705fc0c8511
("Merge tag 'pull-for-8.2-fixes-231123-1' of https://gitlab.com/
stsquad/qemu into staging") with our previous cleanup (patches link:
https://lore.kernel.org/all/20231127145611.925817-1-zhao1.liu@linux.intel.com/).

In the previous RFC, Daniel suggested [2] us to use the modern QOM
approach to define CPU topology, and based on this way, defining hybrid
topology through cli is natural. (Thanks Daniel!)

About why we chose -device other than -object, please see the chapter.3
"History of QOM Topology".

In fact, S390x already implements heterogeneity at the CPU level with
QOM CPUs, i.e., different CPUs have different entitlements [3]. However,
for more thorough heterogeneity, i.e., heterogeneous cores, clusters,
dies, caches and even more, we still need to go farther in the QOM
direction.

With these background, we propose this series to implement QOM "smp"
topology in QEMU, and it's also the first step towards the heterogeneous
topology (including CPU & cache topology) for virtualization case.

The overall goal is to both use QOM "smp" topology to be compatible with
current -smp behavior, and to take into account different architectural
setups/requirements for CPU topology (even including different designs
for CPU hotplug and possible_cpus[] implementation), and ultimately to
extend QEMU's ability to define CPU topology via -device even without
-smp.

The current remining issue, mainly related to PPC, as it chose to build
the pssible_cpus[] list at core granularity. Please see chapter.5 "Open
Questions" for more thoughts on this issue.

For other architectures that build possible_cpus[] at CPU granularity,
the transition to QOM topology will be similar to what was done for i386
in this patchset (please feel free to point out any issues I've missed).

There's a lot of work, and the devil is in the details.


Welcome your feedbacks!


1. Summary about What We Did?
=============================

This series implements the basics of QOM topology and supports QOM
topology for the i386 as the example:

* Introduce the general topology device and abstract all CPU topology
  levels to topology devices:

    - including "cpu", "cpu-core", "cpu-cluster", "cpu-die",
      "cpu-socket", "cpu-book", "cpu-drawer", and a special topology
      root "cpu-slot" to manage topology tree.

* Allow user to create "smp" CPU topology via "-device", for example:

    The topology with 8 CPUs:

    -accel kvm -cpu host \
    -device cpu-socket,id=sock0 \
    -device cpu-die,id=die0,parent=sock0 \
    -device cpu-core,id=core0,parent=die0,nr-threads=2 \
    -device cpu-core,id=core1,parent=die0,nr-threads=2,plugged-threads=1 \
    -device cpu-core,id=core2,parent=die0,nr-threads=2,plugged-threads=2 \
    -device cpu-core,id=core3,parent=die0,nr-threads=2 \
    -device host-x86_64-cpu,socket-id=0,die-id=0,core-id=1,thread-id=1 \

* Build a topology tree under machine, for example:

    One of the above CPUs:

    {
        "props": {
            "core-id": 0,
            "socket-id": 0,
            "thread-id": 0
        },
        "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/core0/host-x86_64-cpu[0]",
        "type": "host-x86_64-cpu",
        "vcpus-count": 1
    }

* Convert topology information configured via -smp into a QOM "smp"
  topology tree (if the arch supports QOM topology).


2. What's the Problem?
======================

As computer architectures continue to innovate, the need for
heterogeneous topology of virtual machine in QEMU is growing.

On the one hand, there is the need for heterogeneous CPU topology
support. Heterogeneous CPU topology refers to systems that use more than
one kind of processor or cores [4], the typical example is intel hybrid
architecture with 2 core types [5], which will even have heterogeneous
die topology [6]. And we can see that not only Intel, but also ARM and
AMD's heterogeneous platforms will have the ability to support
virtualization.

On the other hand, there is also a growing need for heterogeneous cache
topology in QEMU. Not only will Intel's hybrid platforms introduce
complex heterogeneous cache topology, but more platforms have aslo seen
the strong demand for heterogeneous cache topology definition (e.g. ARM
[7]). Although cache topology is strictly speaking a separate topology
from CPU topology, in terms of the actual processor build process and
the current QEMU's way to define cache topology (e.g., i386 [8] & ARM
[9]), cache topology is and should be dependent on CPU topology to be
defined.

With this background of increasing interest in heterogeneous computing,
we need a flexible way (for accel) to define a wide variety of CPU
topologies in QEMU.

Obviously, -smp is not enough, so here we propose a way to define CPU
topology based on -device, in the hope that this will greatly expand the
flexibility of QEMU's CPU topology definition.


3. History of QOM Topology
==========================

The intent of QOM topology is not to stop at QOM CPUs, but to abstract
more CPU topology levels.

In fact, it's not a new term.

Back in 2014, Fan proposed [10] a hierarchical tree based on QOM node,
QOM sockets, QOM cores and QOM cpu. Andreas also propsed [11] socket/
core/thread QOM model to create hierarchical topology in place. From
the discussion at that time, the hierarchy topology tree representation
was what people (Igor said [12]) wanted. However, this work was not
continued.

Then Bharata abstracted [13] the cpu-core to support core granularity
hotplug in spapr. Until now, the only user of cpu-core is still spapr.

Cpu-cluster was introduced by Luc [14] to organize CPUs in different
containers to support GDB for TCG case. Though this abstraction was
descripted as: "mainly an internal QEMU representation and does not
necessarily match with the notion of clusters on the real hardware",
its name and function actually make it easy to correspond to the
physical cluster/smp cluster (the difference, of course, is that TCG
cluster collects CPUs directly, while the cluster as a CPU topology
level collect cores, but this difference is not the impediment gap to
converting "TCG " cluster to "general" cluster).

As CPU architectures evolve, QEMU has supported more topology levels
(clusters, die, book and drawer) for virtualization case, while the
existing cpu-core and cpu-cluster abstractions become fragmented.

And now, the need for defining hybrid topology has prompted us to
rethink the QOM topology.

Daniel suggested [2] to use -object interfaces to define CPU topology,
and we absorbed his design idea. But in practice we found that -device
looked like a better approach to take advantage of the current QOM CPU
device, cpu-core device and cpu-cluster device.


4. Design Overview
==================

4.1. General Topology Device
============================

We introduce a new topology device as the basic abstraction, then all
levels of the CPU topology are derived from this general topology device
type.

This topology device is the basic unit for building the topology tree.
Children topology devices are inserted into the queue of the parent, and
the child<> property is created between the children and their parent.

As the root of the topology tree, we introduce a special topology device
"cpu-slot". It is created by the machine at machine's initialization and
collects the topology devices created by the user from the cli, and thus
builds the topology tree.

The cpu-slot is also responsible for statistics on global topology
information, and whenever there is a new topology child, the cpu-slot as
root is notified to update topology informantion. In addition, different
architectures have different requirements for topology (e.g., support
for different levels), and such limitations/properties are applied to
cpu-slot, which is checked when adding the new child topology unit in
topology tree.


4.2. Derived QOM Topology Devices
=================================

Based on the new general topology device type, we convert CPU, cpu-core
and cpu-cluster from general devices to topology devices.

And we also abstract cpu-die, cpu-socket, cpu-book and cpu-drawer as
topology devices.


4.3. New Device Category "DEVICE_CATEGORY_CPU_DEF"
==================================================

The topology devices can be divided into two general categories:

* One, as the basic topology components, should be created before board
  initialization, to predefine the basic topology structure for the
  system, and is used to initialize MachineState.possible_cpus at
  machine's initialization. This category doesn't support hotplug, such
  as:

    cpu-core (non-PPC core), cpu-cluster, cpu-die, cpu-socket, cpu-book
    and cpu-drawer.

  Thus, we introduce the new device category "DEVICE_CATEGORY_CPU_DEF"
  to mark these devices and create them from cli before board
  initialization.

* The other are CPU and PPC core, which are the granularity of
  MachineState.possible_cpus.

  They're created from MachineState.possible_cpus in place during
  machine initialization or are plugged into MachineState.possible_cpus
  through hotplug way.

  For these devices, they could be created from cli only after board
  initialization.


4.4. User-child Interface to Build Child<> from Cli
===================================================

Topology device is bus-less device, and needs child<> to build topology
hierarchical relationship like:

/machine/peripheral/cpu-slot/sock*/die*/core*/cpu*

Therefore, we introduce a new user-child interface to insert hooks into
device_add path to get/specify object parent for topology devices.

If a topology device specify "parent" option in -device, it will be add
to the corresponding topology parent with child<> property.

If no "parent" option, the topology device will have the default parent
"cpu-slot". This ensures cpu-slot could collect all topology units to
build complete topology tree.


5. Open Questions
=================

There's a special case, user could define topology via -device without
-smp (that's the future hybrid topology case!).

In the design of current QOM topology, the numbers of maximum CPUs and
pre plugged CPUs could be collected during core devices realize.

For the (non-PPC) architectures which build possible_cpus[] at CPU
granularity, the cores will be created before possible_cpus[]
initialization and then CPU slot could know how many maximum CPUs will
be supported to fill possible_cpus[].

But for PPC, the possible_cpus[] is at core granularity and PPC core
could only be created after possible_cpus[] initialization, so that
CPU slot cannot know the the numbers of maximum CPUs (PPC cores) and pre
plugged CPUs (PPC cores). So for PPC, the "-smp" is necessary and cannot
be omitted.

For PPC this potential impact, i.e., even though QOM topology is
supported in PPC, it is not possible to omit -smp to create the topology
only via -device as for i386, and since PPC does not currently support
heterogeneous topology, this potential impact might be acceptable?


6. Future TODOs
===============

The current QOM topology RFC is only the very first step to introduce
the most basic QOM support, and it tries to be as compatible as possible
with existing SMP facilities.

The ultimate goal is to completely replace the current smp-related
topology structures with cpu-slot.

There are many TODOs:

* Add unit tests.
* Support QOM topology for all architectures.
* Get rid of MachineState.smp and MachineClass.smp_props with cpu-slot.
* Extend QOM topology to hybrid topology.
* Introduce "-device-set" which is derived from Daniel's "-object-set"
  idea [2] to create multiple duplicate devices.
...


7. Patch Summary
================

Patch  1- 3: Create DEVICE_CATEGORY_CPU_DEF devices before board
             initialization.
Patch  4- 7: Support child<> creation from cli.
Ptach  8-12: Introduce general topology device.
Patch 13-26: Abstract all topology levels to topology devices.
Patch 27-34: Introduce cpu-slot to manage the CPU topology of machine.
Patch 35-41: Convert i386's CPU creation & hotplug to be based on QOM
             topology.


8. Reference
============

[1]:  Hybrid topology RFC:
      https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg03205.html
[2]:  Daniel's suggestion about QOM topology:
      https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg03320.html
[3]:  S390x topology document (by Nina):
      https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04842.html
[4]:  Heterogeneous computing:
      https://en.wikipedia.org/wiki/Heterogeneous_computing
[5]:  12th gen’s Intel hybrid technology:
      https://www.intel.com/content/www/us/en/support/articles/000091896/processors.html
[6]:  Intel Meteor Lake (14th gen) architecture overview:
      https://www.intel.com/content/www/us/en/content-details/788851/meteor-lake-architecture-overview.html
[7]:  Need of ARM heterogeneous cache topology (by Yanan):
      https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg05139.html
[8]:  Cache topology implementation for i386:
      https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg08251.html
[9]:  Cluster for ARM to define shared L2 cache and L3 tag (by Yanan):
      https://lore.kernel.org/all/20211228092221.21068-1-wangyanan55@huawei.com/
[10]: [PATCH v1 0/4] prebuild cpu QOM tree /machine/node/socket/core
      ->link-cpu (by Fan):
      https://lore.kernel.org/all/cover.1395217538.git.chen.fan.fnst@cn.fujitsu.com/
[11]: [PATCH RFC 0/4] target-i386: PC socket/core/thread modeling,
      part 1 (by Andreas):
      https://lore.kernel.org/all/1427131923-4670-1-git-send-email-afaerber@suse.de/
[12]: "Now we want to have similar QOM tree for introspection which
      helps express topology as well" (by Igor):
      https://lore.kernel.org/all/20150407170734.51faac90@igors-macbook-pro.local/
[13]: [for-2.7 PATCH v3 06/15] cpu: Abstract CPU core type (by Bharata)
      https://lore.kernel.org/all/1463024905-28401-7-git-send-email-bharata@linux.vnet.ibm.com/
[14]: [PATCH v8 01/16] hw/cpu: introduce CPU clusters (by Luc):
      https://lore.kernel.org/all/20181207090135.7651-2-luc.michel@greensocs.com/

Thanks and Best Regards,
Zhao

---
Zhao Liu (41):
  qdev: Introduce new device category to cover basic topology device
  qdev: Allow qdev_device_add() to add specific category device
  system: Create base category devices from cli before board
    initialization
  qom/object: Introduce helper to resolve path from non-direct parent
  qdev: Set device parent and id after setting properties
  qdev: Introduce user-child interface to collect devices from -device
  qdev: Introduce parent option in -device
  hw/core/topo: Introduce CPU topology device abstraction
  hw/core/topo: Support topology index for topology device
  hw/core/topo: Add virtual method to update topology info for parent
  hw/core/topo: Add virtual method to check topology child
  hw/core/topo: Add helpers to traverse the CPU topology tree
  hw/core/cpu: Convert CPU from general device to topology device
  PPC/ppc-core: Offload core-id to PPC specific core abstarction
  hw/cpu/core: Allow to configure plugged threads for cpu-core
  PPC/ppc-core: Limit plugged-threads and nr-threads to be equal
  hw/cpu/core: Convert cpu-core from general device to topology device
  hw/cpu/cluster: Rename CPUClusterState to CPUCluster
  hw/cpu/cluster: Wrap TCG related ops and props into CONFIG_TCG
  hw/cpu/cluster: Descript cluster is not only used for TCG in comment
  hw/cpu/cluster: Allow cpu-cluster to be created by -device
  hw/cpu/cluster: Convert cpu-cluster from general device to topology
    device
  hw/cpu/die: Abstract cpu-die level as topology device
  hw/cpu/socket: Abstract cpu-socket level as topology device
  hw/cpu/book: Abstract cpu-book level as topology device
  hw/cpu/drawer: Abstract cpu-drawer level as topology device
  hw/core/slot: Introduce CPU slot as the root of CPU topology
  hw/core/slot: Maintain the core queue in CPU slot
  hw/core/slot: Statistics topology information in CPU slot
  hw/core/slot: Check topology child to be added under CPU slot
  hw/machine: Plug cpu-slot into machine to maintain topology tree
  hw/machine: Build smp topology tree from -smp
  hw/machine: Validate smp topology tree without -smp
  hw/core/topo: Implement user-child to collect topology device from cli
  hw/i386: Make x86_cpu_new() private in x86.c
  hw/i386: Allow x86_cpu_new() to specify parent for new CPU
  hw/i386: Allow i386 to create new CPUs from QOM topology
  hw/i386: Wrap apic id and topology sub ids assigning as helpers
  hw/i386: Add the interface to search parent for QOM topology
  hw/i386: Support QOM topology
  hw/i386: Cleanup non-QOM topology support

 MAINTAINERS                        |  16 +
 accel/kvm/kvm-all.c                |   4 +-
 gdbstub/system.c                   |   2 +-
 hw/core/cpu-common.c               |  25 +-
 hw/core/cpu-slot.c                 | 605 +++++++++++++++++++++++++++++
 hw/core/cpu-topo.c                 | 399 +++++++++++++++++++
 hw/core/machine-smp.c              |   9 +
 hw/core/machine.c                  |  10 +
 hw/core/meson.build                |   2 +
 hw/cpu/book.c                      |  46 +++
 hw/cpu/cluster.c                   |  50 ++-
 hw/cpu/core.c                      |  72 ++--
 hw/cpu/die.c                       |  46 +++
 hw/cpu/drawer.c                    |  46 +++
 hw/cpu/meson.build                 |   2 +-
 hw/cpu/socket.c                    |  46 +++
 hw/i386/x86.c                      | 319 ++++++++++-----
 hw/net/virtio-net.c                |   2 +-
 hw/ppc/meson.build                 |   1 +
 hw/ppc/pnv.c                       |   6 +-
 hw/ppc/pnv_core.c                  |  17 +-
 hw/ppc/ppc_core.c                  | 102 +++++
 hw/ppc/spapr.c                     |  28 +-
 hw/ppc/spapr_cpu_core.c            |  19 +-
 hw/usb/xen-usb.c                   |   3 +-
 hw/xen/xen-legacy-backend.c        |   2 +-
 include/hw/arm/armsse.h            |   2 +-
 include/hw/arm/xlnx-versal.h       |   4 +-
 include/hw/arm/xlnx-zynqmp.h       |   4 +-
 include/hw/boards.h                |  13 +
 include/hw/core/cpu-slot.h         | 108 +++++
 include/hw/core/cpu-topo.h         | 111 ++++++
 include/hw/core/cpu.h              |   8 +-
 include/hw/cpu/book.h              |  38 ++
 include/hw/cpu/cluster.h           |  51 ++-
 include/hw/cpu/core.h              |  26 +-
 include/hw/cpu/die.h               |  38 ++
 include/hw/cpu/drawer.h            |  38 ++
 include/hw/cpu/socket.h            |  38 ++
 include/hw/i386/x86.h              |   5 +-
 include/hw/ppc/pnv_core.h          |  11 +-
 include/hw/ppc/ppc_core.h          |  58 +++
 include/hw/ppc/spapr_cpu_core.h    |  12 +-
 include/hw/qdev-core.h             |   1 +
 include/hw/riscv/microchip_pfsoc.h |   4 +-
 include/hw/riscv/sifive_u.h        |   4 +-
 include/monitor/qdev.h             |   7 +-
 include/monitor/user-child.h       |  57 +++
 include/qom/object.h               |  26 ++
 qom/object.c                       |  31 ++
 system/meson.build                 |   1 +
 system/qdev-monitor.c              | 141 ++++++-
 system/user-child.c                |  72 ++++
 system/vl.c                        |  53 ++-
 target/i386/cpu.c                  |   4 +
 tests/unit/meson.build             |   5 +-
 56 files changed, 2607 insertions(+), 243 deletions(-)
 create mode 100644 hw/core/cpu-slot.c
 create mode 100644 hw/core/cpu-topo.c
 create mode 100644 hw/cpu/book.c
 create mode 100644 hw/cpu/die.c
 create mode 100644 hw/cpu/drawer.c
 create mode 100644 hw/cpu/socket.c
 create mode 100644 hw/ppc/ppc_core.c
 create mode 100644 include/hw/core/cpu-slot.h
 create mode 100644 include/hw/core/cpu-topo.h
 create mode 100644 include/hw/cpu/book.h
 create mode 100644 include/hw/cpu/die.h
 create mode 100644 include/hw/cpu/drawer.h
 create mode 100644 include/hw/cpu/socket.h
 create mode 100644 include/hw/ppc/ppc_core.h
 create mode 100644 include/monitor/user-child.h
 create mode 100644 system/user-child.c

-- 
2.34.1

[RFC 01/41] qdev: Introduce new device category to cover basic topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Topology devices are used to define CPUs and need to be created and
realized before the board initialization.

Use this new catogory to identify such special devices.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 include/hw/qdev-core.h | 1 +
 system/qdev-monitor.c  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 151d9682380d..97b7cfd04e35 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -86,6 +86,7 @@ typedef enum DeviceCategory {
     DEVICE_CATEGORY_SOUND,
     DEVICE_CATEGORY_MISC,
     DEVICE_CATEGORY_CPU,
+    DEVICE_CATEGORY_CPU_DEF,
     DEVICE_CATEGORY_WATCHDOG,
     DEVICE_CATEGORY_MAX
 } DeviceCategory;
diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index a13db763e5dd..0f163b2d0310 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -173,6 +173,7 @@ static void qdev_print_devinfos(bool show_no_user)
         [DEVICE_CATEGORY_SOUND]   = "Sound",
         [DEVICE_CATEGORY_MISC]    = "Misc",
         [DEVICE_CATEGORY_CPU]     = "CPU",
+        [DEVICE_CATEGORY_CPU_DEF] = "CPU Definition",
         [DEVICE_CATEGORY_WATCHDOG]= "Watchdog",
         [DEVICE_CATEGORY_MAX]     = "Uncategorized",
     };
-- 
2.34.1

[RFC 02/41] qdev: Allow qdev_device_add() to add specific category device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Topology devices need to be created and realized before board
initialization.

Allow qdev_device_add() to specify category to help create topology
devices early.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/net/virtio-net.c    |  2 +-
 hw/usb/xen-usb.c       |  3 ++-
 include/monitor/qdev.h |  4 ++--
 system/qdev-monitor.c  | 12 ++++++++----
 system/vl.c            |  4 ++--
 5 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 80c56f0cfcf1..fc225049ee30 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -965,7 +965,7 @@ static void failover_add_primary(VirtIONet *n, Error **errp)
         return;
     }
 
-    dev = qdev_device_add_from_qdict(n->primary_opts,
+    dev = qdev_device_add_from_qdict(n->primary_opts, NULL,
                                      n->primary_opts_from_json,
                                      &err);
     if (err) {
diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c
index 09ec326aeae5..bc00d47ff021 100644
--- a/hw/usb/xen-usb.c
+++ b/hw/usb/xen-usb.c
@@ -766,7 +766,8 @@ static void usbback_portid_add(struct usbback_info *usbif, unsigned port,
     qdict_put_str(qdict, "hostport", portname);
     opts = qemu_opts_from_qdict(qemu_find_opts("device"), qdict,
                                 &error_abort);
-    usbif->ports[port - 1].dev = USB_DEVICE(qdev_device_add(opts, &local_err));
+    usbif->ports[port - 1].dev = USB_DEVICE(
+                                     qdev_device_add(opts, NULL, &local_err));
     if (!usbif->ports[port - 1].dev) {
         qobject_unref(qdict);
         xen_pv_printf(&usbif->xendev, 0,
diff --git a/include/monitor/qdev.h b/include/monitor/qdev.h
index 1d57bf657794..f5fd6e6c1ffc 100644
--- a/include/monitor/qdev.h
+++ b/include/monitor/qdev.h
@@ -8,8 +8,8 @@ void hmp_info_qdm(Monitor *mon, const QDict *qdict);
 void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp);
 
 int qdev_device_help(QemuOpts *opts);
-DeviceState *qdev_device_add(QemuOpts *opts, Error **errp);
-DeviceState *qdev_device_add_from_qdict(const QDict *opts,
+DeviceState *qdev_device_add(QemuOpts *opts, long *category, Error **errp);
+DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
                                         bool from_json, Error **errp);
 
 /**
diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index 0f163b2d0310..7ee33a50142a 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -617,7 +617,7 @@ const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
     return prop->name;
 }
 
-DeviceState *qdev_device_add_from_qdict(const QDict *opts,
+DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
                                         bool from_json, Error **errp)
 {
     ERRP_GUARD();
@@ -639,6 +639,10 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
         return NULL;
     }
 
+    if (category && !test_bit(*category, dc->categories)) {
+        return NULL;
+    }
+
     /* find bus */
     path = qdict_get_try_str(opts, "bus");
     if (path != NULL) {
@@ -731,12 +735,12 @@ err_del_dev:
 }
 
 /* Takes ownership of @opts on success */
-DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
+DeviceState *qdev_device_add(QemuOpts *opts, long *category, Error **errp)
 {
     QDict *qdict = qemu_opts_to_qdict(opts, NULL);
     DeviceState *ret;
 
-    ret = qdev_device_add_from_qdict(qdict, false, errp);
+    ret = qdev_device_add_from_qdict(qdict, category, false, errp);
     if (ret) {
         qemu_opts_del(opts);
     }
@@ -858,7 +862,7 @@ void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp)
         qemu_opts_del(opts);
         return;
     }
-    dev = qdev_device_add(opts, errp);
+    dev = qdev_device_add(opts, NULL, errp);
 
     /*
      * Drain all pending RCU callbacks. This is done because
diff --git a/system/vl.c b/system/vl.c
index 2bcd9efb9a64..0be155b530b4 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -1198,7 +1198,7 @@ static int device_init_func(void *opaque, QemuOpts *opts, Error **errp)
 {
     DeviceState *dev;
 
-    dev = qdev_device_add(opts, errp);
+    dev = qdev_device_add(opts, NULL, errp);
     if (!dev && *errp) {
         error_report_err(*errp);
         return -1;
@@ -2646,7 +2646,7 @@ static void qemu_create_cli_devices(void)
          * from the start, so call qdev_device_add_from_qdict() directly for
          * now.
          */
-        dev = qdev_device_add_from_qdict(opt->opts, true, &error_fatal);
+        dev = qdev_device_add_from_qdict(opt->opts, NULL, true, &error_fatal);
         object_unref(OBJECT(dev));
         loc_pop(&opt->loc);
     }
-- 
2.34.1

[RFC 03/41] system: Create base category devices from cli before board initialization

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Topology devices are required to complete CPU topology building before
*_init_cpus() in MachineClass.init().

Add a qemu_create_cli_base_devices() before board initialization to
help create and realize topology devices from cli early.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 system/vl.c | 51 ++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 34 insertions(+), 17 deletions(-)

diff --git a/system/vl.c b/system/vl.c
index 0be155b530b4..65add2fb2460 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -1197,8 +1197,9 @@ static int device_help_func(void *opaque, QemuOpts *opts, Error **errp)
 static int device_init_func(void *opaque, QemuOpts *opts, Error **errp)
 {
     DeviceState *dev;
+    long *category = opaque;
 
-    dev = qdev_device_add(opts, NULL, errp);
+    dev = qdev_device_add(opts, category, errp);
     if (!dev && *errp) {
         error_report_err(*errp);
         return -1;
@@ -2617,25 +2618,13 @@ static void qemu_init_board(void)
     realtime_init();
 }
 
-static void qemu_create_cli_devices(void)
+static void qemu_create_cli_devices(long *category)
 {
     DeviceOption *opt;
 
-    soundhw_init();
-
-    qemu_opts_foreach(qemu_find_opts("fw_cfg"),
-                      parse_fw_cfg, fw_cfg_find(), &error_fatal);
-
-    /* init USB devices */
-    if (machine_usb(current_machine)) {
-        if (foreach_device_config(DEV_USB, usb_parse) < 0)
-            exit(1);
-    }
-
-    /* init generic devices */
     rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);
     qemu_opts_foreach(qemu_find_opts("device"),
-                      device_init_func, NULL, &error_fatal);
+                      device_init_func, category, &error_fatal);
     QTAILQ_FOREACH(opt, &device_opts, next) {
         DeviceState *dev;
         loc_push_restore(&opt->loc);
@@ -2646,13 +2635,40 @@ static void qemu_create_cli_devices(void)
          * from the start, so call qdev_device_add_from_qdict() directly for
          * now.
          */
-        dev = qdev_device_add_from_qdict(opt->opts, NULL, true, &error_fatal);
+        dev = qdev_device_add_from_qdict(opt->opts, category,
+                                         true, &error_fatal);
         object_unref(OBJECT(dev));
         loc_pop(&opt->loc);
     }
     rom_reset_order_override();
 }
 
+static void qemu_create_cli_base_devices(void)
+{
+    long category = DEVICE_CATEGORY_CPU_DEF;
+
+    qemu_opts_foreach(qemu_find_opts("fw_cfg"),
+                      parse_fw_cfg, fw_cfg_find(), &error_fatal);
+
+    /* init CPU topology devices which don't support hotplug. */
+    qemu_create_cli_devices(&category);
+}
+
+static void qemu_create_cli_periphery_devices(void)
+{
+    soundhw_init();
+
+    /* init USB devices */
+    if (machine_usb(current_machine)) {
+        if (foreach_device_config(DEV_USB, usb_parse) < 0) {
+            exit(1);
+        }
+    }
+
+    /* init generic devices */
+    qemu_create_cli_devices(NULL);
+}
+
 static void qemu_machine_creation_done(void)
 {
     MachineState *machine = MACHINE(qdev_get_machine());
@@ -2701,8 +2717,9 @@ void qmp_x_exit_preconfig(Error **errp)
         return;
     }
 
+    qemu_create_cli_base_devices();
     qemu_init_board();
-    qemu_create_cli_devices();
+    qemu_create_cli_periphery_devices();
     qemu_machine_creation_done();
 
     if (loadvm) {
-- 
2.34.1

[RFC 04/41] qom/object: Introduce helper to resolve path from non-direct parent

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When we support child<> property creation from cli, the peripheral
container (/machine/peripheral) may not be the direct parent of the
devices created from cli.

For this case, add a helper to resolve path from non-direct parent.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 include/qom/object.h | 15 +++++++++++++++
 qom/object.c         | 18 ++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/include/qom/object.h b/include/qom/object.h
index afccd24ca7ab..494eef801be3 100644
--- a/include/qom/object.h
+++ b/include/qom/object.h
@@ -1562,6 +1562,21 @@ Object *object_resolve_path_type(const char *path, const char *typename,
  */
 Object *object_resolve_path_at(Object *parent, const char *path);
 
+/**
+ * object_resolve_path_from:
+ * @parent: the object from which to resolve the path
+ * @path: the path to resolve
+ * @ambiguous: returns true if the path resolution failed because of an
+ *   ambiguous match
+ *
+ * This is like object_resolve_path_at(), except @parent may be the
+ * partial parent of @path.
+ *
+ * Returns: The resolved object or NULL on path lookup failure.
+ */
+Object *object_resolve_path_from(Object *parent, const char *path,
+                                 bool *ambiguous);
+
 /**
  * object_resolve_path_component:
  * @parent: the object in which to resolve the path
diff --git a/qom/object.c b/qom/object.c
index 95c0dc8285fe..da29e88816b5 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -2192,6 +2192,24 @@ Object *object_resolve_path_at(Object *parent, const char *path)
     return object_resolve_abs_path(parent, parts, TYPE_OBJECT);
 }
 
+Object *object_resolve_path_from(Object *parent, const char *path,
+                                 bool *ambiguousp)
+{
+    g_auto(GStrv) parts = NULL;
+    bool ambiguous = false;
+    Object *obj;
+
+    parts = g_strsplit(path, "/", 0);
+    assert(parts);
+
+    obj = object_resolve_partial_path(parent, parts, TYPE_OBJECT,
+                                      &ambiguous);
+    if (ambiguousp) {
+        *ambiguousp = ambiguous;
+    }
+    return obj;
+}
+
 typedef struct StringProperty
 {
     char *(*get)(Object *, Error **);
-- 
2.34.1

[RFC 05/41] qdev: Set device parent and id after setting properties

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The properties setting does not conflict with the creation of child<>
property.

Pre-setting the device's properties can help the device's parent
selection. Some topology devices (e.g., CPUs that support hotplug)
usually define topology sub indexes as properties, and the selection of
their parent needs to be based on these proteries.

Move qdev_set_id() after properties setting to help the next user-child
introduction.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 system/qdev-monitor.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index 7ee33a50142a..107411bb50cc 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -700,14 +700,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
         }
     }
 
-    /*
-     * set dev's parent and register its id.
-     * If it fails it means the id is already taken.
-     */
     id = g_strdup(qdict_get_try_str(opts, "id"));
-    if (!qdev_set_id(dev, id, errp)) {
-        goto err_del_dev;
-    }
 
     /* set properties */
     dev->opts = qdict_clone_shallow(opts);
@@ -721,6 +714,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
         goto err_del_dev;
     }
 
+    /*
+     * set dev's parent and register its id.
+     * If it fails it means the id is already taken.
+     */
+    if (!qdev_set_id(dev, id, errp)) {
+        goto err_del_dev;
+    }
+
     if (!qdev_realize(dev, bus, errp)) {
         goto err_del_dev;
     }
-- 
2.34.1

[RFC 06/41] qdev: Introduce user-child interface to collect devices from -device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Topology relationship is based on child<> property, therefore introduce
a new user-child interface to help bus-less devices create child<>
property from cli.

User-child interface works in qdev_set_id(), where the child<> property
is created for cli devices.

With several methods, user-child could provide the specific "parent"
device other than the default peripheral/peripheral-anon container.

The topology root (cpu-slot) could collect topology devices based on
user-child implementation.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                  |  2 +
 include/monitor/user-child.h | 57 +++++++++++++++++++++++
 include/qom/object.h         | 11 +++++
 qom/object.c                 | 13 ++++++
 system/meson.build           |  1 +
 system/qdev-monitor.c        | 89 +++++++++++++++++++++++++++++++++---
 system/user-child.c          | 72 +++++++++++++++++++++++++++++
 7 files changed, 239 insertions(+), 6 deletions(-)
 create mode 100644 include/monitor/user-child.h
 create mode 100644 system/user-child.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 695e0bd34fbb..fdbabaa983cc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3208,12 +3208,14 @@ F: hw/core/bus.c
 F: hw/core/sysbus.c
 F: include/hw/qdev*
 F: include/monitor/qdev.h
+F: include/monitor/user-child.h
 F: include/qom/
 F: qapi/qom.json
 F: qapi/qdev.json
 F: scripts/coccinelle/qom-parent-type.cocci
 F: scripts/qom-cast-macro-clean-cocci-gen.py
 F: system/qdev-monitor.c
+F: system/user-child.c
 F: stubs/qdev.c
 F: qom/
 F: tests/unit/check-qom-interface.c
diff --git a/include/monitor/user-child.h b/include/monitor/user-child.h
new file mode 100644
index 000000000000..e246fcefe40a
--- /dev/null
+++ b/include/monitor/user-child.h
@@ -0,0 +1,57 @@
+/*
+ * Child configurable interface header.
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef USER_CHILD_H
+#define USER_CHILD_H
+
+#include "qom/object.h"
+
+#define TYPE_USER_CHILD "user-child"
+
+typedef struct UserChildClass UserChildClass;
+DECLARE_CLASS_CHECKERS(UserChildClass, USER_CHILD, TYPE_USER_CHILD)
+#define USER_CHILD(obj) INTERFACE_CHECK(UserChild, (obj), TYPE_USER_CHILD)
+
+typedef struct UserChild UserChild;
+
+/**
+ * UserChildClass:
+ * @get_parent: Method to get the default parent if user doesn't specify
+ *     the parent in cli.
+ * @get_child_name: Method to get the default device id for this device
+ *     if user doesn't specify id in cli.
+ * @check_parent: Method to check if the parent specified by user in cli
+ *     is valid.
+ */
+struct UserChildClass {
+    /* <private> */
+    InterfaceClass parent_class;
+
+    /* <public> */
+    Object *(*get_parent)(UserChild *uc, Error **errp);
+    char *(*get_child_name)(UserChild *uc, Object *parent);
+    bool (*check_parent)(UserChild *uc, Object *parent);
+};
+
+Object *uc_provide_default_parent(Object *obj, Error **errp);
+char *uc_name_future_child(Object *obj, Object *parent);
+bool uc_check_user_parent(Object *obj, Object *parent);
+
+#endif /* USER_CHILD_H */
diff --git a/include/qom/object.h b/include/qom/object.h
index 494eef801be3..f725d9452c76 100644
--- a/include/qom/object.h
+++ b/include/qom/object.h
@@ -1484,6 +1484,17 @@ Object *object_get_objects_root(void);
  */
 Object *object_get_internal_root(void);
 
+/**
+ * object_is_child_from:
+ * @child: the object.
+ * @parent: the parent/non-direct parent object.
+ *
+ * Check whether @parent is the parent/non-direct parent of @child.
+ *
+ * Returns: true iff @parent is the parent/non-direct parent of @child.
+ */
+bool object_is_child_from(const Object *child, const Object *parent);
+
 /**
  * object_get_canonical_path_component:
  * @obj: the object
diff --git a/qom/object.c b/qom/object.c
index da29e88816b5..d6f55aa59504 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -2024,6 +2024,19 @@ object_property_add_const_link(Object *obj, const char *name,
                                 NULL, OBJ_PROP_LINK_DIRECT);
 }
 
+bool object_is_child_from(const Object *child, const Object *parent)
+{
+    Object *obj = child->parent;
+
+    while (obj) {
+        if (obj == parent) {
+            return true;
+        }
+        obj = obj->parent;
+    }
+    return false;
+}
+
 const char *object_get_canonical_path_component(const Object *obj)
 {
     ObjectProperty *prop = NULL;
diff --git a/system/meson.build b/system/meson.build
index 3a64dd89de1f..ac682a3ca9e1 100644
--- a/system/meson.build
+++ b/system/meson.build
@@ -24,6 +24,7 @@ system_ss.add(files(
   'runstate-hmp-cmds.c',
   'runstate.c',
   'tpm-hmp-cmds.c',
+  'user-child.c',
   'vl.c',
 ), sdl, libpmem, libdaxctl)
 
diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index 107411bb50cc..0261937b8462 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -22,6 +22,7 @@
 #include "monitor/hmp.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
+#include "monitor/user-child.h"
 #include "sysemu/arch_init.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-qdev.h"
@@ -585,26 +586,93 @@ static BusState *qbus_find(const char *path, Error **errp)
     return bus;
 }
 
+static Object *qdev_find_peripheral_parent(DeviceState *dev,
+                                           Error **errp)
+{
+    Object *parent_obj, *obj = OBJECT(dev);
+
+    parent_obj = uc_provide_default_parent(obj, errp);
+    if (*errp) {
+        return NULL;
+    }
+
+    if (parent_obj) {
+        /*
+         * Non-anonymous parents (under "/peripheral") are allowed to
+         * be accessed to create child<> properties.
+         */
+        if (object_is_child_from(parent_obj, qdev_get_peripheral())) {
+            return parent_obj;
+        }
+    }
+
+    return NULL;
+}
+
+static bool qdev_pre_check_device_id(char *id, Error **errp)
+{
+    bool ambiguous = false;
+    Object *obj;
+
+    if (!id) {
+        return true;
+    }
+
+    obj = object_resolve_path_from(qdev_get_peripheral(), id, &ambiguous);
+    if (obj || ambiguous) {
+        error_setg(errp, "Duplicate device default ID '%s'. "
+                   "Please specify another 'id'", id);
+        return false;
+    }
+    return true;
+}
+
 /* Takes ownership of @id, will be freed when deleting the device */
 const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
 {
+    Object *parent_obj = NULL;
     ObjectProperty *prop;
+    UserChild *uc;
 
     assert(!dev->id && !dev->realized);
 
+    uc = (UserChild *)object_dynamic_cast(OBJECT(dev), TYPE_USER_CHILD);
+
+    if (uc) {
+        parent_obj = qdev_find_peripheral_parent(dev, errp);
+        if (*errp) {
+            goto err;
+        }
+
+        if (!id && parent_obj) {
+            /*
+             * Covert anonymous device with user-child interface to
+             * non-anonymous, then it will be insert under "/peripheral"
+             * path.
+             */
+            id = uc_name_future_child(OBJECT(dev), parent_obj);
+            if (!qdev_pre_check_device_id(id, errp)) {
+                goto err;
+            }
+        }
+    }
+
     /*
      * object_property_[try_]add_child() below will assert the device
      * has no parent
      */
     if (id) {
-        prop = object_property_try_add_child(qdev_get_peripheral(), id,
+        if (!parent_obj) {
+            parent_obj = qdev_get_peripheral();
+        }
+
+        prop = object_property_try_add_child(parent_obj, id,
                                              OBJECT(dev), NULL);
         if (prop) {
             dev->id = id;
         } else {
             error_setg(errp, "Duplicate device ID '%s'", id);
-            g_free(id);
-            return NULL;
+            goto err;
         }
     } else {
         static int anon_count;
@@ -615,6 +683,9 @@ const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
     }
 
     return prop->name;
+err:
+    g_free(id);
+    return NULL;
 }
 
 DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
@@ -885,12 +956,18 @@ void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp)
 
 static DeviceState *find_device_state(const char *id, Error **errp)
 {
-    Object *obj = object_resolve_path_at(qdev_get_peripheral(), id);
+    bool ambiguous = false;
     DeviceState *dev;
+    Object *obj;
 
+    obj = object_resolve_path_from(qdev_get_peripheral(), id, &ambiguous);
     if (!obj) {
-        error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-                  "Device '%s' not found", id);
+        if (ambiguous) {
+            error_setg(errp, "Device ID '%s' is ambiguous", id);
+        } else {
+            error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+                      "Device '%s' not found", id);
+        }
         return NULL;
     }
 
diff --git a/system/user-child.c b/system/user-child.c
new file mode 100644
index 000000000000..1e667f09cc87
--- /dev/null
+++ b/system/user-child.c
@@ -0,0 +1,72 @@
+/*
+ * Child configurable interface.
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "monitor/user-child.h"
+
+Object *uc_provide_default_parent(Object *obj, Error **errp)
+{
+    UserChild *uc = USER_CHILD(obj);
+    UserChildClass *ucc = USER_CHILD_GET_CLASS(uc);
+
+    if (ucc->get_parent) {
+        return ucc->get_parent(uc, errp);
+    }
+
+    return NULL;
+}
+
+char *uc_name_future_child(Object *obj, Object *parent)
+{
+    UserChild *uc = USER_CHILD(obj);
+    UserChildClass *ucc = USER_CHILD_GET_CLASS(uc);
+
+    if (ucc->get_child_name) {
+        return ucc->get_child_name(uc, parent);
+    }
+
+    return NULL;
+}
+
+bool uc_check_user_parent(Object *obj, Object *parent)
+{
+    UserChild *uc = USER_CHILD(obj);
+    UserChildClass *ucc = USER_CHILD_GET_CLASS(uc);
+
+    if (ucc->check_parent) {
+        ucc->check_parent(uc, parent);
+    }
+
+    return true;
+}
+
+static const TypeInfo user_child_interface_info = {
+    .name          = TYPE_USER_CHILD,
+    .parent        = TYPE_INTERFACE,
+    .class_size = sizeof(UserChildClass),
+};
+
+static void user_child_register_types(void)
+{
+    type_register_static(&user_child_interface_info);
+}
+
+type_init(user_child_register_types)
-- 
2.34.1

[RFC 07/41] qdev: Introduce parent option in -device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Currently, the devices added by "-device" are linked via bus, and are
set the parent as peripheral-anon or peripheral containers of the
machine.

But this is not enough for building CPU topology hierarchies as:
1. The relationship between different CPU hierarchies is child<>
   property other than link<> property, and they shouldn't be linked
   using the special bus.
2. The canonical path of device is built from the child<> property, and
   the well defined CPU topology hierarchies ask their canonical path to
   reflect the correct topological relationship.

With these, the child<> property support is needed for QDev interface to
allow user to configure proper parent in "-device".

Introduce the "parent" option in "-device" to create the child<>
property. This option asks for the device id of the parent device.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/xen/xen-legacy-backend.c |  2 +-
 include/monitor/qdev.h      |  3 ++-
 system/qdev-monitor.c       | 50 ++++++++++++++++++++++++++-----------
 3 files changed, 38 insertions(+), 17 deletions(-)

diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index 124dd5f3d687..70ad11c6287e 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -184,7 +184,7 @@ static struct XenLegacyDevice *xen_be_get_xendev(const char *type, int dom,
     object_initialize(&xendev->qdev, ops->size, TYPE_XENBACKEND);
     OBJECT(xendev)->free = g_free;
     qdev_set_id(DEVICE(xendev), g_strdup_printf("xen-%s-%d", type, dev),
-                &error_fatal);
+                NULL, &error_fatal);
     qdev_realize(DEVICE(xendev), xen_sysbus, &error_fatal);
     object_unref(OBJECT(xendev));
 
diff --git a/include/monitor/qdev.h b/include/monitor/qdev.h
index f5fd6e6c1ffc..3d9d06158e5f 100644
--- a/include/monitor/qdev.h
+++ b/include/monitor/qdev.h
@@ -16,6 +16,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
  * qdev_set_id: parent the device and set its id if provided.
  * @dev: device to handle
  * @id: id to be given to the device, or NULL.
+ * @parent: parent to be set for the device, or NULL.
  *
  * Returns: the id of the device in case of success; otherwise NULL.
  *
@@ -34,6 +35,6 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
  * returned string is owned by the corresponding child property and must
  * not be freed by the caller.
  */
-const char *qdev_set_id(DeviceState *dev, char *id, Error **errp);
+const char *qdev_set_id(DeviceState *dev, char *id, char *parent, Error **errp);
 
 #endif
diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index 0261937b8462..8f56113eef65 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -587,22 +587,33 @@ static BusState *qbus_find(const char *path, Error **errp)
 }
 
 static Object *qdev_find_peripheral_parent(DeviceState *dev,
+                                           char *parent_id,
                                            Error **errp)
 {
     Object *parent_obj, *obj = OBJECT(dev);
 
-    parent_obj = uc_provide_default_parent(obj, errp);
-    if (*errp) {
-        return NULL;
-    }
+    if (parent_id) {
+        parent_obj = object_resolve_path_from(qdev_get_peripheral(),
+                                              parent_id, NULL);
+        if (parent_obj) {
+            if (uc_check_user_parent(obj, parent_obj)) {
+                return parent_obj;
+            }
+        }
+    } else {
+        parent_obj = uc_provide_default_parent(obj, errp);
+        if (*errp) {
+            return NULL;
+        }
 
-    if (parent_obj) {
-        /*
-         * Non-anonymous parents (under "/peripheral") are allowed to
-         * be accessed to create child<> properties.
-         */
-        if (object_is_child_from(parent_obj, qdev_get_peripheral())) {
-            return parent_obj;
+        if (parent_obj) {
+            /*
+             * Non-anonymous parents (under "/peripheral") are allowed to
+             * be accessed to create child<> properties.
+             */
+            if (object_is_child_from(parent_obj, qdev_get_peripheral())) {
+                return parent_obj;
+            }
         }
     }
 
@@ -628,7 +639,8 @@ static bool qdev_pre_check_device_id(char *id, Error **errp)
 }
 
 /* Takes ownership of @id, will be freed when deleting the device */
-const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
+const char *qdev_set_id(DeviceState *dev, char *id,
+                        char *parent, Error **errp)
 {
     Object *parent_obj = NULL;
     ObjectProperty *prop;
@@ -639,7 +651,7 @@ const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
     uc = (UserChild *)object_dynamic_cast(OBJECT(dev), TYPE_USER_CHILD);
 
     if (uc) {
-        parent_obj = qdev_find_peripheral_parent(dev, errp);
+        parent_obj = qdev_find_peripheral_parent(dev, parent, errp);
         if (*errp) {
             goto err;
         }
@@ -655,6 +667,11 @@ const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
                 goto err;
             }
         }
+        g_free(parent);
+    } else if (parent) {
+        error_setg(errp, "Only the device implemented user-child "
+                   "interface supports `parent` option.");
+        goto err;
     }
 
     /*
@@ -684,6 +701,7 @@ const char *qdev_set_id(DeviceState *dev, char *id, Error **errp)
 
     return prop->name;
 err:
+    g_free(parent);
     g_free(id);
     return NULL;
 }
@@ -694,7 +712,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
     ERRP_GUARD();
     DeviceClass *dc;
     const char *driver, *path;
-    char *id;
+    char *id, *parent;
     DeviceState *dev = NULL;
     BusState *bus = NULL;
 
@@ -772,12 +790,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
     }
 
     id = g_strdup(qdict_get_try_str(opts, "id"));
+    parent = g_strdup(qdict_get_try_str(opts, "parent"));
 
     /* set properties */
     dev->opts = qdict_clone_shallow(opts);
     qdict_del(dev->opts, "driver");
     qdict_del(dev->opts, "bus");
     qdict_del(dev->opts, "id");
+    qdict_del(dev->opts, "parent");
 
     object_set_properties_from_keyval(&dev->parent_obj, dev->opts, from_json,
                                       errp);
@@ -789,7 +809,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts, long *category,
      * set dev's parent and register its id.
      * If it fails it means the id is already taken.
      */
-    if (!qdev_set_id(dev, id, errp)) {
+    if (!qdev_set_id(dev, id, parent, errp)) {
         goto err_del_dev;
     }
 
-- 
2.34.1

[RFC 08/41] hw/core/topo: Introduce CPU topology device abstraction

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

To create more flexible CPU topologies (both symmetric and
heterogeneous) via the "-device" interface, it is necessary to convert
the current CPU topology hierarchies into the special CPU topology
devices.

The CPU topology will be built as a tree, and the device with the
CPU_TOPO_ROOT level is the only root of this CPU topology tree.

The different levels of CPU topology devices are connected in the
"-device" cli with the child<> property, which in turn will be set the
Object.parent through the qdev interface. And ultimately at the
realize(), CPU topology devices will be linked to their topological
parent based on the Object.parent field, and then be inserted into the
topology tree.

As the first step, introduce the basic CPU topology device abstraction,
as well as the topology tree and topology hierarchy construction based
on the CPU topology devices.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                |   2 +
 hw/core/cpu-topo.c         | 201 +++++++++++++++++++++++++++++++++++++
 hw/core/meson.build        |   1 +
 include/hw/core/cpu-topo.h |  79 +++++++++++++++
 4 files changed, 283 insertions(+)
 create mode 100644 hw/core/cpu-topo.c
 create mode 100644 include/hw/core/cpu-topo.h

diff --git a/MAINTAINERS b/MAINTAINERS
index fdbabaa983cc..564cb776ae80 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1854,6 +1854,7 @@ R: Philippe Mathieu-Daudé <philmd@linaro.org>
 R: Yanan Wang <wangyanan55@huawei.com>
 S: Supported
 F: hw/core/cpu.c
+F: hw/core/cpu-topo.c
 F: hw/core/machine-qmp-cmds.c
 F: hw/core/machine.c
 F: hw/core/machine-smp.c
@@ -1865,6 +1866,7 @@ F: qapi/machine-common.json
 F: qapi/machine-target.json
 F: include/hw/boards.h
 F: include/hw/core/cpu.h
+F: include/hw/core/cpu-topo.h
 F: include/hw/cpu/cluster.h
 F: include/sysemu/numa.h
 F: tests/unit/test-smp-parse.c
diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
new file mode 100644
index 000000000000..4428b979a5dc
--- /dev/null
+++ b/hw/core/cpu-topo.c
@@ -0,0 +1,201 @@
+/*
+ * General CPU topology device abstraction
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-properties.h"
+#include "qapi/error.h"
+
+static const char *cpu_topo_level_to_string(CPUTopoLevel level)
+{
+    switch (level) {
+    case CPU_TOPO_UNKNOWN:
+        return "unknown";
+    case CPU_TOPO_THREAD:
+        return "thread";
+    case CPU_TOPO_CORE:
+        return "core";
+    case CPU_TOPO_CLUSTER:
+        return "cluster";
+    case CPU_TOPO_DIE:
+        return "die";
+    case CPU_TOPO_SOCKET:
+        return "socket";
+    case CPU_TOPO_BOOK:
+        return "book";
+    case CPU_TOPO_DRAWER:
+        return "drawer";
+    case CPU_TOPO_ROOT:
+        return "root";
+    }
+
+    return NULL;
+}
+
+static void cpu_topo_build_hierarchy(CPUTopoState *topo, Error **errp)
+{
+    CPUTopoState *parent = topo->parent;
+    CPUTopoLevel level = CPU_TOPO_LEVEL(topo);
+    g_autofree char *name = NULL;
+
+    if (!parent) {
+        return;
+    }
+
+    if (parent->child_level == CPU_TOPO_UNKNOWN) {
+        parent->child_level = level;
+    } else if (parent->child_level != level) {
+        error_setg(errp, "cpu topo: the parent level %s asks for the "
+                   "%s child, but current level is %s",
+                   cpu_topo_level_to_string(CPU_TOPO_LEVEL(parent)),
+                   cpu_topo_level_to_string(parent->child_level),
+                   cpu_topo_level_to_string(level));
+        return;
+    }
+
+    if (parent->max_children && parent->max_children <= parent->num_children) {
+        error_setg(errp, "cpu topo: the parent limit the (%d) children, "
+                   "currently it has %d children",
+                   parent->max_children,
+                   parent->num_children);
+        return;
+    }
+
+    parent->num_children++;
+    QTAILQ_INSERT_TAIL(&parent->children, topo, sibling);
+}
+
+static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
+{
+    Object *obj = OBJECT(topo);
+    CPUTopoLevel level = CPU_TOPO_LEVEL(topo);
+
+    if (!obj->parent) {
+        return;
+    }
+
+    if (object_dynamic_cast(obj->parent, TYPE_CPU_TOPO)) {
+        CPUTopoState *parent = CPU_TOPO(obj->parent);
+
+        if (level >= CPU_TOPO_LEVEL(parent)) {
+            error_setg(errp, "cpu topo: current level (%s) should be "
+                       "lower than parent (%s) level",
+                       object_get_typename(obj),
+                       object_get_typename(OBJECT(parent)));
+            return;
+        }
+        topo->parent = parent;
+    }
+
+    if (topo->parent) {
+        cpu_topo_build_hierarchy(topo, errp);
+    }
+}
+
+static void cpu_topo_realize(DeviceState *dev, Error **errp)
+{
+    CPUTopoState *topo = CPU_TOPO(dev);
+    CPUTopoClass *tc = CPU_TOPO_GET_CLASS(topo);
+
+    if (tc->level == CPU_TOPO_UNKNOWN) {
+        error_setg(errp, "cpu topo: no level specified"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+        return;
+    }
+
+    cpu_topo_set_parent(topo, errp);
+}
+
+static void cpu_topo_destroy_hierarchy(CPUTopoState *topo)
+{
+    CPUTopoState *parent = topo->parent;
+
+    if (!parent) {
+        return;
+    }
+
+    QTAILQ_REMOVE(&parent->children, topo, sibling);
+    parent->num_children--;
+
+    if (!parent->num_children) {
+        parent->child_level = CPU_TOPO_UNKNOWN;
+    }
+}
+
+static void cpu_topo_unrealize(DeviceState *dev)
+{
+    CPUTopoState *topo = CPU_TOPO(dev);
+
+    /*
+     * The specific unrealize method must consider the bottom-up,
+     * layer-by-layer unrealization implementation.
+     */
+    g_assert(!topo->num_children);
+
+    if (topo->parent) {
+        cpu_topo_destroy_hierarchy(topo);
+    }
+}
+
+static void cpu_topo_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    /* All topology devices belong to CPU property. */
+    set_bit(DEVICE_CATEGORY_CPU, dc->categories);
+    dc->realize = cpu_topo_realize;
+    dc->unrealize = cpu_topo_unrealize;
+
+    /*
+     * The general topo device is not hotpluggable by default.
+     * If any topo device needs hotplug support, this flag must be
+     * overridden under arch-specific topo device code.
+     */
+    dc->hotpluggable = false;
+
+    tc->level = CPU_TOPO_UNKNOWN;
+}
+
+static void cpu_topo_instance_init(Object *obj)
+{
+    CPUTopoState *topo = CPU_TOPO(obj);
+    QTAILQ_INIT(&topo->children);
+
+    topo->child_level = CPU_TOPO_UNKNOWN;
+}
+
+static const TypeInfo cpu_topo_type_info = {
+    .name = TYPE_CPU_TOPO,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .class_size = sizeof(CPUTopoClass),
+    .class_init = cpu_topo_class_init,
+    .instance_size = sizeof(CPUTopoState),
+    .instance_init = cpu_topo_instance_init,
+};
+
+static void cpu_topo_register_types(void)
+{
+    type_register_static(&cpu_topo_type_info);
+}
+
+type_init(cpu_topo_register_types)
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 67dad04de559..501d2529697e 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -23,6 +23,7 @@ else
 endif
 
 common_ss.add(files('cpu-common.c'))
+common_ss.add(files('cpu-topo.c'))
 common_ss.add(files('machine-smp.c'))
 system_ss.add(when: 'CONFIG_FITLOADER', if_true: files('loader-fit.c'))
 system_ss.add(when: 'CONFIG_GENERIC_LOADER', if_true: files('generic-loader.c'))
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
new file mode 100644
index 000000000000..ebcbdd854da5
--- /dev/null
+++ b/include/hw/core/cpu-topo.h
@@ -0,0 +1,79 @@
+/*
+ * General CPU topology device abstraction
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef CPU_TOPO_H
+#define CPU_TOPO_H
+
+#include "hw/qdev-core.h"
+#include "qemu/queue.h"
+
+typedef enum CPUTopoLevel {
+    CPU_TOPO_UNKNOWN,
+    CPU_TOPO_THREAD,
+    CPU_TOPO_CORE,
+    CPU_TOPO_CLUSTER,
+    CPU_TOPO_DIE,
+    CPU_TOPO_SOCKET,
+    CPU_TOPO_BOOK,
+    CPU_TOPO_DRAWER,
+    CPU_TOPO_ROOT,
+} CPUTopoLevel;
+
+#define TYPE_CPU_TOPO "cpu-topo"
+OBJECT_DECLARE_TYPE(CPUTopoState, CPUTopoClass, CPU_TOPO)
+
+/**
+ * CPUTopoClass:
+ * @level: Topology level for this CPUTopoClass.
+ */
+struct CPUTopoClass {
+    /*< private >*/
+    DeviceClass parent_class;
+
+    /*< public >*/
+    CPUTopoLevel level;
+};
+
+/**
+ * CPUTopoState:
+ * @num_children: Number of topology children under this topology device.
+ * @max_children: Maximum number of children allowed to be inserted under
+ *     this topology device.
+ * @child_level: Topology level for children.
+ * @parent: Topology parent of this topology device.
+ * @children: Queue of topology children.
+ * @sibling: Queue node to be inserted in parent's topology queue.
+ */
+struct CPUTopoState {
+    /*< private >*/
+    DeviceState parent_obj;
+
+    /*< public >*/
+    int num_children;
+    int max_children;
+    CPUTopoLevel child_level;
+    struct CPUTopoState *parent;
+    QTAILQ_HEAD(, CPUTopoState) children;
+    QTAILQ_ENTRY(CPUTopoState) sibling;
+};
+
+#define CPU_TOPO_LEVEL(topo)    (CPU_TOPO_GET_CLASS(topo)->level)
+
+#endif /* CPU_TOPO_H */
-- 
2.34.1

[RFC 09/41] hw/core/topo: Support topology index for topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Topology index is used to identify the topology child under the same
parent topology device.

This field corresponds to the topology sub index (e.g., socket-id/
core-id/thread-id) used for addressing.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-topo.c         | 77 ++++++++++++++++++++++++++++++++++++++
 include/hw/core/cpu-topo.h |  6 +++
 2 files changed, 83 insertions(+)

diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index 4428b979a5dc..3e0c183388d8 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -50,6 +50,66 @@ static const char *cpu_topo_level_to_string(CPUTopoLevel level)
     return NULL;
 }
 
+static void cpu_topo_refresh_free_child_index(CPUTopoState *topo)
+{
+    CPUTopoState *child;
+
+    /*
+     * Fast way: Assume that the index grows sequentially and that there
+     * are no "index hole" in the previous children.
+     *
+     * The previous check on num_children ensures that free_child_index + 1
+     * does not hit the max_children limit.
+     */
+    if (topo->free_child_index + 1 == topo->num_children) {
+        topo->free_child_index++;
+        return;
+    }
+
+    /* Slow way: Search the "index hole". The index hole must be found. */
+    for (int index = 0; index < topo->num_children; index++) {
+        bool existed = false;
+
+        QTAILQ_FOREACH(child, &topo->children, sibling) {
+            if (child->index == index) {
+                existed = true;
+                break;
+            }
+        }
+
+        if (!existed) {
+            topo->free_child_index = index;
+            return;
+        }
+    }
+}
+
+static void cpu_topo_validate_index(CPUTopoState *topo, Error **errp)
+{
+    CPUTopoState *parent = topo->parent, *child;
+
+    if (topo->index < 0) {
+        error_setg(errp, "Invalid topology index (%d).",
+                   topo->index);
+        return;
+    }
+
+    if (parent->max_children && topo->index >= parent->max_children) {
+        error_setg(errp, "Invalid topology index (%d): "
+                   "The maximum index is %d.",
+                   topo->index, parent->max_children);
+        return;
+    }
+
+    QTAILQ_FOREACH(child, &topo->children, sibling) {
+        if (child->index == topo->index) {
+            error_setg(errp, "Duplicate topology index (%d)",
+                       topo->index);
+            return;
+        }
+    }
+}
+
 static void cpu_topo_build_hierarchy(CPUTopoState *topo, Error **errp)
 {
     CPUTopoState *parent = topo->parent;
@@ -80,7 +140,18 @@ static void cpu_topo_build_hierarchy(CPUTopoState *topo, Error **errp)
     }
 
     parent->num_children++;
+    if (topo->index == UNASSIGNED_TOPO_INDEX) {
+        topo->index = parent->free_child_index;
+    } else if (topo->index != parent->free_child_index) {
+        /* The index has been set, then we need to validate it. */
+        cpu_topo_validate_index(topo, errp);
+        if (*errp) {
+            return;
+        }
+    }
+
     QTAILQ_INSERT_TAIL(&parent->children, topo, sibling);
+    cpu_topo_refresh_free_child_index(parent);
 }
 
 static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
@@ -135,6 +206,10 @@ static void cpu_topo_destroy_hierarchy(CPUTopoState *topo)
     QTAILQ_REMOVE(&parent->children, topo, sibling);
     parent->num_children--;
 
+    if (topo->index < parent->free_child_index) {
+        parent->free_child_index = topo->index;
+    }
+
     if (!parent->num_children) {
         parent->child_level = CPU_TOPO_UNKNOWN;
     }
@@ -180,6 +255,8 @@ static void cpu_topo_instance_init(Object *obj)
     CPUTopoState *topo = CPU_TOPO(obj);
     QTAILQ_INIT(&topo->children);
 
+    topo->index = UNASSIGNED_TOPO_INDEX;
+    topo->free_child_index = 0;
     topo->child_level = CPU_TOPO_UNKNOWN;
 }
 
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index ebcbdd854da5..c0dfff9dc63b 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -24,6 +24,8 @@
 #include "hw/qdev-core.h"
 #include "qemu/queue.h"
 
+#define UNASSIGNED_TOPO_INDEX -1
+
 typedef enum CPUTopoLevel {
     CPU_TOPO_UNKNOWN,
     CPU_TOPO_THREAD,
@@ -53,6 +55,8 @@ struct CPUTopoClass {
 
 /**
  * CPUTopoState:
+ * @index: Topology index within parent's topology queue.
+ * @free_child_index: Cached free index to be specified for next child.
  * @num_children: Number of topology children under this topology device.
  * @max_children: Maximum number of children allowed to be inserted under
  *     this topology device.
@@ -66,6 +70,8 @@ struct CPUTopoState {
     DeviceState parent_obj;
 
     /*< public >*/
+    int index;
+    int free_child_index;
     int num_children;
     int max_children;
     CPUTopoLevel child_level;
-- 
2.34.1

[RFC 10/41] hw/core/topo: Add virtual method to update topology info for parent

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When a new topology device is inserted into the topology tree,
its'parents (including non-direct parent) need to update topology
information.

Add the virtual method to help parents on topology tree update
topology information statistics.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-topo.c         | 20 ++++++++++++++++++++
 include/hw/core/cpu-topo.h |  4 ++++
 2 files changed, 24 insertions(+)

diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index 3e0c183388d8..e244f0a3564e 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -154,6 +154,20 @@ static void cpu_topo_build_hierarchy(CPUTopoState *topo, Error **errp)
     cpu_topo_refresh_free_child_index(parent);
 }
 
+static void cpu_topo_update_info(CPUTopoState *topo, bool is_realize)
+{
+    CPUTopoState *parent = topo->parent;
+    CPUTopoClass *tc;
+
+    while (parent) {
+        tc = CPU_TOPO_GET_CLASS(parent);
+        if (tc->update_topo_info) {
+            tc->update_topo_info(parent, topo, is_realize);
+        }
+        parent = parent->parent;
+    }
+}
+
 static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
 {
     Object *obj = OBJECT(topo);
@@ -178,6 +192,11 @@ static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
 
     if (topo->parent) {
         cpu_topo_build_hierarchy(topo, errp);
+        if (*errp) {
+            return;
+        }
+
+        cpu_topo_update_info(topo, true);
     }
 }
 
@@ -203,6 +222,7 @@ static void cpu_topo_destroy_hierarchy(CPUTopoState *topo)
         return;
     }
 
+    cpu_topo_update_info(topo, false);
     QTAILQ_REMOVE(&parent->children, topo, sibling);
     parent->num_children--;
 
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index c0dfff9dc63b..79cd8606feca 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -44,6 +44,8 @@ OBJECT_DECLARE_TYPE(CPUTopoState, CPUTopoClass, CPU_TOPO)
 /**
  * CPUTopoClass:
  * @level: Topology level for this CPUTopoClass.
+ * @update_topo_info: Method to update topology information statistics when
+ *     new child (including direct child and non-direct child) is added.
  */
 struct CPUTopoClass {
     /*< private >*/
@@ -51,6 +53,8 @@ struct CPUTopoClass {
 
     /*< public >*/
     CPUTopoLevel level;
+    void (*update_topo_info)(CPUTopoState *parent, CPUTopoState *child,
+                             bool is_realize);
 };
 
 /**
-- 
2.34.1

[RFC 11/41] hw/core/topo: Add virtual method to check topology child

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When a new topology child is to be inserted into the topology tree, its
parents (including non-direct parents) need to check if this child is
supported.

Add the virtual method to allow topology device to check the support for
their topology children.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-topo.c         | 22 ++++++++++++++++++++++
 include/hw/core/cpu-topo.h |  4 ++++
 2 files changed, 26 insertions(+)

diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index e244f0a3564e..cba2dc747e74 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -168,6 +168,23 @@ static void cpu_topo_update_info(CPUTopoState *topo, bool is_realize)
     }
 }
 
+static void cpu_topo_check_support(CPUTopoState *topo, Error **errp)
+{
+    CPUTopoState *parent = topo->parent;
+    CPUTopoClass *tc;
+
+    while (parent) {
+        tc = CPU_TOPO_GET_CLASS(parent);
+        if (tc->check_topo_child) {
+            tc->check_topo_child(parent, topo, errp);
+            if (*errp) {
+                return;
+            }
+        }
+        parent = parent->parent;
+    }
+}
+
 static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
 {
     Object *obj = OBJECT(topo);
@@ -191,6 +208,11 @@ static void cpu_topo_set_parent(CPUTopoState *topo, Error **errp)
     }
 
     if (topo->parent) {
+        cpu_topo_check_support(topo, errp);
+        if (*errp) {
+            return;
+        }
+
         cpu_topo_build_hierarchy(topo, errp);
         if (*errp) {
             return;
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index 79cd8606feca..1ffdb0be6d38 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -46,6 +46,8 @@ OBJECT_DECLARE_TYPE(CPUTopoState, CPUTopoClass, CPU_TOPO)
  * @level: Topology level for this CPUTopoClass.
  * @update_topo_info: Method to update topology information statistics when
  *     new child (including direct child and non-direct child) is added.
+ * @check_topo_child: Method to check the support for new child (including
+ *     direct child and non-direct child) to be added.
  */
 struct CPUTopoClass {
     /*< private >*/
@@ -55,6 +57,8 @@ struct CPUTopoClass {
     CPUTopoLevel level;
     void (*update_topo_info)(CPUTopoState *parent, CPUTopoState *child,
                              bool is_realize);
+    void (*check_topo_child)(CPUTopoState *parent, CPUTopoState *child,
+                             Error **errp);
 };
 
 /**
-- 
2.34.1

[RFC 12/41] hw/core/topo: Add helpers to traverse the CPU topology tree

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The topology devices will be organized as a topology tree. Each topology
device may have many topology children with lower topology level.

Add the helpers to traverse the CPU topology tree.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-topo.c         | 41 ++++++++++++++++++++++++++++++++++++++
 include/hw/core/cpu-topo.h | 13 ++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index cba2dc747e74..687a4cc566ec 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -318,3 +318,44 @@ static void cpu_topo_register_types(void)
 }
 
 type_init(cpu_topo_register_types)
+
+static int do_cpu_topo_child_foreach(CPUTopoState *topo,
+                                     unsigned long *levels,
+                                     topo_fn fn, void *opaque,
+                                     bool recurse)
+{
+    CPUTopoState *child;
+    int ret = TOPO_FOREACH_CONTINUE;
+
+    QTAILQ_FOREACH(child, &topo->children, sibling) {
+        if (!levels || (levels && test_bit(CPU_TOPO_LEVEL(child), levels))) {
+            ret = fn(child, opaque);
+            if (ret == TOPO_FOREACH_END || ret == TOPO_FOREACH_ERR) {
+                break;
+            } else if (ret == TOPO_FOREACH_SIBLING) {
+                continue;
+            }
+        }
+
+        if (recurse) {
+            ret = do_cpu_topo_child_foreach(child, levels, fn, opaque, recurse);
+            if (ret != TOPO_FOREACH_CONTINUE) {
+                break;
+            }
+        }
+    }
+    return ret;
+}
+
+int cpu_topo_child_foreach(CPUTopoState *topo, unsigned long *levels,
+                           topo_fn fn, void *opaque)
+{
+    return do_cpu_topo_child_foreach(topo, levels, fn, opaque, false);
+}
+
+int cpu_topo_child_foreach_recursive(CPUTopoState *topo,
+                                     unsigned long *levels,
+                                     topo_fn fn, void *opaque)
+{
+    return do_cpu_topo_child_foreach(topo, levels, fn, opaque, true);
+}
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index 1ffdb0be6d38..453bacbb558b 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -90,4 +90,17 @@ struct CPUTopoState {
 
 #define CPU_TOPO_LEVEL(topo)    (CPU_TOPO_GET_CLASS(topo)->level)
 
+#define TOPO_FOREACH_SIBLING         2
+#define TOPO_FOREACH_END             1
+#define TOPO_FOREACH_CONTINUE        0
+#define TOPO_FOREACH_ERR             -1
+
+typedef int (*topo_fn)(CPUTopoState *topo, void *opaque);
+
+int cpu_topo_child_foreach(CPUTopoState *topo, unsigned long *levels,
+                           topo_fn fn, void *opaque);
+int cpu_topo_child_foreach_recursive(CPUTopoState *topo,
+                                     unsigned long *levels,
+                                     topo_fn fn, void *opaque);
+
 #endif /* CPU_TOPO_H */
-- 
2.34.1

[RFC 13/41] hw/core/cpu: Convert CPU from general device to topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Convert CPU to topology device then its parent topology devices could
count the number of CPUs when new CPUs are added into topology tree.

Note since CPUs are created from *_init_cpus() in MachineClass.init() or
added from hotplug way, it depends on board initialization. Thus CPU
topology device isn't marked as DEVICE_CATEGORY_CPU_DEF and it will only
be created after board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 accel/kvm/kvm-all.c   |  4 ++--
 hw/core/cpu-common.c  | 25 +++++++++++++++++++++----
 include/hw/core/cpu.h |  8 ++++++--
 3 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index e39a810a4e92..2eee3eb95907 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -3953,7 +3953,7 @@ static void query_stats(StatsResultList **result, StatsTarget target,
         break;
     case STATS_TARGET_VCPU:
         add_stats_entry(result, STATS_PROVIDER_KVM,
-                        cpu->parent_obj.canonical_path,
+                        DEVICE(cpu)->canonical_path,
                         stats_list);
         break;
     default:
@@ -4045,7 +4045,7 @@ static void query_stats_cb(StatsResultList **result, StatsTarget target,
         stats_args.names = names;
         stats_args.errp = errp;
         CPU_FOREACH(cpu) {
-            if (!apply_str_list_filter(cpu->parent_obj.canonical_path, targets)) {
+            if (!apply_str_list_filter(DEVICE(cpu)->canonical_path, targets)) {
                 continue;
             }
             query_stats_vcpu(cpu, &stats_args);
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 82dae51a550b..e9ed84ff5386 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -195,6 +195,16 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     CPUState *cpu = CPU(dev);
     Object *machine = qdev_get_machine();
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+
+    /*
+     * The parent topology realize() must be completed before cpu_resume()
+     * where the CPU starts to run.
+     */
+    cc->parent_realize(dev, errp);
+    if (*errp) {
+        return;
+    }
 
     /* qdev_get_machine() can return something that's not TYPE_MACHINE
      * if this is one of the user-only emulators; in that case there's
@@ -225,6 +235,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 static void cpu_common_unrealizefn(DeviceState *dev)
 {
     CPUState *cpu = CPU(dev);
+    CPUClass *cc = CPU_GET_CLASS(cpu);
 
     /* Call the plugin hook before clearing the cpu is fully unrealized */
     if (tcg_enabled()) {
@@ -233,6 +244,7 @@ static void cpu_common_unrealizefn(DeviceState *dev)
 
     /* NOTE: latest generic point before the cpu is fully unrealized */
     cpu_exec_unrealizefn(cpu);
+    cc->parent_unrealize(dev);
 }
 
 static void cpu_common_initfn(Object *obj)
@@ -275,6 +287,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     ResettableClass *rc = RESETTABLE_CLASS(klass);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(klass);
     CPUClass *k = CPU_CLASS(klass);
 
     k->parse_features = cpu_common_parse_features;
@@ -282,9 +295,6 @@ static void cpu_class_init(ObjectClass *klass, void *data)
     k->has_work = cpu_common_has_work;
     k->gdb_read_register = cpu_common_gdb_read_register;
     k->gdb_write_register = cpu_common_gdb_write_register;
-    set_bit(DEVICE_CATEGORY_CPU, dc->categories);
-    dc->realize = cpu_common_realizefn;
-    dc->unrealize = cpu_common_unrealizefn;
     rc->phases.hold = cpu_common_reset_hold;
     cpu_class_init_props(dc);
     /*
@@ -292,11 +302,18 @@ static void cpu_class_init(ObjectClass *klass, void *data)
      * IRQs, adding reset handlers, halting non-first CPUs, ...
      */
     dc->user_creatable = false;
+    /* CPU is the minimum granularity for hotplug. */
+    dc->hotpluggable = true;
+    device_class_set_parent_realize(dc, cpu_common_realizefn,
+                                    &k->parent_realize);
+    device_class_set_parent_unrealize(dc, cpu_common_unrealizefn,
+                                      &k->parent_unrealize);
+    tc->level = CPU_TOPO_THREAD;
 }
 
 static const TypeInfo cpu_type_info = {
     .name = TYPE_CPU,
-    .parent = TYPE_DEVICE,
+    .parent = TYPE_CPU_TOPO,
     .instance_size = sizeof(CPUState),
     .instance_init = cpu_common_initfn,
     .instance_finalize = cpu_common_finalize,
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index c0c8320413e5..a700f7c39140 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -20,6 +20,7 @@
 #ifndef QEMU_CPU_H
 #define QEMU_CPU_H
 
+#include "hw/core/cpu-topo.h"
 #include "hw/qdev-core.h"
 #include "disas/dis-asm.h"
 #include "exec/cpu-common.h"
@@ -146,7 +147,7 @@ struct SysemuCPUOps;
  */
 struct CPUClass {
     /*< private >*/
-    DeviceClass parent_class;
+    CPUTopoClass parent_class;
     /*< public >*/
 
     ObjectClass *(*class_by_name)(const char *cpu_model);
@@ -191,6 +192,9 @@ struct CPUClass {
     int reset_dump_flags;
     int gdb_num_core_regs;
     bool gdb_stop_before_watchpoint;
+
+    DeviceRealize parent_realize;
+    DeviceUnrealize parent_unrealize;
 };
 
 /*
@@ -456,7 +460,7 @@ struct qemu_work_item;
  */
 struct CPUState {
     /*< private >*/
-    DeviceState parent_obj;
+    CPUTopoState parent_obj;
     /* cache to avoid expensive CPU_GET_CLASS */
     CPUClass *cc;
     /*< public >*/
-- 
2.34.1

[RFC 14/41] PPC/ppc-core: Offload core-id to PPC specific core abstarction

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

PPC (spapr) supports hotplugs at the core granularity (spapr core) and
treats core-id as the global id for all cores.

But other architectures that support hotplugging at CPU granularity,
use core-id as the local id to indicate the core within the parent
topology container instand of the global index.

To remove potential ambiguity and make the core abstraction available to
other architectures, introduce the ppc core abstraction and define the
"global" core-id over the ppc specific core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                     |  2 +
 hw/cpu/core.c                   | 29 ------------
 hw/ppc/meson.build              |  1 +
 hw/ppc/pnv.c                    |  6 +--
 hw/ppc/pnv_core.c               |  5 ++-
 hw/ppc/ppc_core.c               | 79 +++++++++++++++++++++++++++++++++
 hw/ppc/spapr.c                  | 28 +++++++-----
 hw/ppc/spapr_cpu_core.c         |  6 +--
 include/hw/cpu/core.h           |  6 ---
 include/hw/ppc/pnv_core.h       |  8 ++--
 include/hw/ppc/ppc_core.h       | 57 ++++++++++++++++++++++++
 include/hw/ppc/spapr_cpu_core.h |  9 ++--
 12 files changed, 175 insertions(+), 61 deletions(-)
 create mode 100644 hw/ppc/ppc_core.c
 create mode 100644 include/hw/ppc/ppc_core.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 564cb776ae80..89e350866d6a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1501,6 +1501,8 @@ F: include/hw/ppc/fdt.h
 F: hw/ppc/fdt.c
 F: include/hw/ppc/pef.h
 F: hw/ppc/pef.c
+F: include/hw/ppc/ppc_core.h
+F: hw/ppc/ppc_core.c
 F: pc-bios/slof.bin
 F: docs/system/ppc/pseries.rst
 F: docs/specs/ppc-spapr-*
diff --git a/hw/cpu/core.c b/hw/cpu/core.c
index 495a5c30ffe1..7e274d6aebb7 100644
--- a/hw/cpu/core.c
+++ b/hw/cpu/core.c
@@ -14,33 +14,6 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 
-static void core_prop_get_core_id(Object *obj, Visitor *v, const char *name,
-                                  void *opaque, Error **errp)
-{
-    CPUCore *core = CPU_CORE(obj);
-    int64_t value = core->core_id;
-
-    visit_type_int(v, name, &value, errp);
-}
-
-static void core_prop_set_core_id(Object *obj, Visitor *v, const char *name,
-                                  void *opaque, Error **errp)
-{
-    CPUCore *core = CPU_CORE(obj);
-    int64_t value;
-
-    if (!visit_type_int(v, name, &value, errp)) {
-        return;
-    }
-
-    if (value < 0) {
-        error_setg(errp, "Invalid core id %"PRId64, value);
-        return;
-    }
-
-    core->core_id = value;
-}
-
 static void core_prop_get_nr_threads(Object *obj, Visitor *v, const char *name,
                                      void *opaque, Error **errp)
 {
@@ -82,8 +55,6 @@ static void cpu_core_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
 
     set_bit(DEVICE_CATEGORY_CPU, dc->categories);
-    object_class_property_add(oc, "core-id", "int", core_prop_get_core_id,
-                              core_prop_set_core_id, NULL, NULL);
     object_class_property_add(oc, "nr-threads", "int", core_prop_get_nr_threads,
                               core_prop_set_nr_threads, NULL, NULL);
 }
diff --git a/hw/ppc/meson.build b/hw/ppc/meson.build
index ea44856d43b0..2b40a91a4661 100644
--- a/hw/ppc/meson.build
+++ b/hw/ppc/meson.build
@@ -2,6 +2,7 @@ ppc_ss = ss.source_set()
 ppc_ss.add(files(
   'ppc.c',
   'ppc_booke.c',
+  'ppc_core.c',
 ))
 ppc_ss.add(when: 'CONFIG_FDT_PPC', if_true: [files(
   'fdt.c',
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 0297871bdd5d..6e92ff8b4f64 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -1224,7 +1224,7 @@ static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp)
     /* Map the ICP registers for each thread */
     for (i = 0; i < chip->nr_cores; i++) {
         PnvCore *pnv_core = chip->cores[i];
-        int core_hwid = CPU_CORE(pnv_core)->core_id;
+        int core_hwid = POWERPC_CORE(pnv_core)->core_id;
 
         for (j = 0; j < CPU_CORE(pnv_core)->nr_threads; j++) {
             uint32_t pir = pcc->core_pir(chip, core_hwid) + j;
@@ -1443,7 +1443,7 @@ static void pnv_chip_quad_realize_one(PnvChip *chip, PnvQuad *eq,
                                       const char *type)
 {
     char eq_name[32];
-    int core_id = CPU_CORE(pnv_core)->core_id;
+    int core_id = POWERPC_CORE(pnv_core)->core_id;
 
     snprintf(eq_name, sizeof(eq_name), "eq[%d]", core_id);
     object_initialize_child_with_props(OBJECT(chip), eq_name, eq,
@@ -1983,7 +1983,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
         chip->cores[i] = pnv_core;
         object_property_set_int(OBJECT(pnv_core), "nr-threads",
                                 chip->nr_threads, &error_fatal);
-        object_property_set_int(OBJECT(pnv_core), CPU_CORE_PROP_CORE_ID,
+        object_property_set_int(OBJECT(pnv_core), POWERPC_CORE_PROP_CORE_ID,
                                 core_hwid, &error_fatal);
         object_property_set_int(OBJECT(pnv_core), "pir",
                                 pcc->core_pir(chip, core_hwid), &error_fatal);
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 8c7afe037f00..a90d1ec92bd8 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -267,6 +267,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
     PnvCore *pc = PNV_CORE(OBJECT(dev));
     PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
     CPUCore *cc = CPU_CORE(OBJECT(dev));
+    PowerPCCore *ppc = POWERPC_CORE(cc);
     const char *typename = pnv_core_cpu_typename(pc);
     Error *local_err = NULL;
     void *obj;
@@ -299,7 +300,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
         }
     }
 
-    snprintf(name, sizeof(name), "xscom-core.%d", cc->core_id);
+    snprintf(name, sizeof(name), "xscom-core.%d", ppc->core_id);
     pnv_xscom_region_init(&pc->xscom_regs, OBJECT(dev), pcc->xscom_ops,
                           pc, name, pcc->xscom_size);
 
@@ -392,7 +393,7 @@ static void pnv_core_class_init(ObjectClass *oc, void *data)
 static const TypeInfo pnv_core_infos[] = {
     {
         .name           = TYPE_PNV_CORE,
-        .parent         = TYPE_CPU_CORE,
+        .parent         = TYPE_POWERPC_CORE,
         .instance_size  = sizeof(PnvCore),
         .class_size     = sizeof(PnvCoreClass),
         .class_init = pnv_core_class_init,
diff --git a/hw/ppc/ppc_core.c b/hw/ppc/ppc_core.c
new file mode 100644
index 000000000000..4433b54af506
--- /dev/null
+++ b/hw/ppc/ppc_core.c
@@ -0,0 +1,79 @@
+/*
+ * Common PPC CPU core abstraction
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/ppc/ppc_core.h"
+#include "qapi/error.h"
+#include "qapi/visitor.h"
+
+static void powerpc_core_prop_get_core_id(Object *obj, Visitor *v,
+                                          const char *name, void *opaque,
+                                          Error **errp)
+{
+    PowerPCCore *core = POWERPC_CORE(obj);
+    int64_t value = core->core_id;
+
+    visit_type_int(v, name, &value, errp);
+}
+
+static void powerpc_core_prop_set_core_id(Object *obj, Visitor *v,
+                                          const char *name, void *opaque,
+                                          Error **errp)
+{
+    PowerPCCore *core = POWERPC_CORE(obj);
+    int64_t value;
+
+    if (!visit_type_int(v, name, &value, errp)) {
+        return;
+    }
+
+    if (value < 0) {
+        error_setg(errp, "Invalid core id %"PRId64, value);
+        return;
+    }
+
+    core->core_id = value;
+}
+
+static void powerpc_core_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+
+    object_class_property_add(oc, "core-id", "int",
+                              powerpc_core_prop_get_core_id,
+                              powerpc_core_prop_set_core_id,
+                              NULL, NULL);
+}
+
+static const TypeInfo powerpc_core_type_info = {
+    .name = TYPE_POWERPC_CORE,
+    .parent = TYPE_CPU_CORE,
+    .abstract = true,
+    .class_init = powerpc_core_class_init,
+    .instance_size = sizeof(PowerPCCore),
+};
+
+static void powerpc_core_register_types(void)
+{
+    type_register_static(&powerpc_core_type_info);
+}
+
+type_init(powerpc_core_register_types)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index df09aa9d6a00..72e9f49da110 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2713,7 +2713,7 @@ static void spapr_init_cpus(SpaprMachineState *spapr)
 
             object_property_set_int(core, "nr-threads", nr_threads,
                                     &error_fatal);
-            object_property_set_int(core, CPU_CORE_PROP_CORE_ID, core_id,
+            object_property_set_int(core, POWERPC_CORE_PROP_CORE_ID, core_id,
                                     &error_fatal);
             qdev_realize(DEVICE(core), NULL, &error_fatal);
 
@@ -3889,7 +3889,8 @@ static void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev)
     MachineState *ms = MACHINE(hotplug_dev);
     SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(ms);
     CPUCore *cc = CPU_CORE(dev);
-    CPUArchId *core_slot = spapr_find_cpu_slot(ms, cc->core_id, NULL);
+    PowerPCCore *ppc = POWERPC_CORE(cc);
+    CPUArchId *core_slot = spapr_find_cpu_slot(ms, ppc->core_id, NULL);
 
     if (smc->pre_2_10_has_unused_icps) {
         SpaprCpuCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
@@ -3915,10 +3916,11 @@ void spapr_core_unplug_request(HotplugHandler *hotplug_dev, DeviceState *dev,
     int index;
     SpaprDrc *drc;
     CPUCore *cc = CPU_CORE(dev);
+    PowerPCCore *ppc = POWERPC_CORE(cc);
 
-    if (!spapr_find_cpu_slot(MACHINE(hotplug_dev), cc->core_id, &index)) {
+    if (!spapr_find_cpu_slot(MACHINE(hotplug_dev), ppc->core_id, &index)) {
         error_setg(errp, "Unable to find CPU core with core-id: %d",
-                   cc->core_id);
+                   ppc->core_id);
         return;
     }
     if (index == 0) {
@@ -3927,7 +3929,7 @@ void spapr_core_unplug_request(HotplugHandler *hotplug_dev, DeviceState *dev,
     }
 
     drc = spapr_drc_by_id(TYPE_SPAPR_DRC_CPU,
-                          spapr_vcpu_id(spapr, cc->core_id));
+                          spapr_vcpu_id(spapr, ppc->core_id));
     g_assert(drc);
 
     if (!spapr_drc_unplug_requested(drc)) {
@@ -3985,6 +3987,7 @@ static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev)
     SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
     SpaprCpuCore *core = SPAPR_CPU_CORE(OBJECT(dev));
     CPUCore *cc = CPU_CORE(dev);
+    PowerPCCore *ppc = POWERPC_CORE(cc);
     CPUState *cs;
     SpaprDrc *drc;
     CPUArchId *core_slot;
@@ -3992,11 +3995,11 @@ static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev)
     bool hotplugged = spapr_drc_hotplugged(dev);
     int i;
 
-    core_slot = spapr_find_cpu_slot(MACHINE(hotplug_dev), cc->core_id, &index);
+    core_slot = spapr_find_cpu_slot(MACHINE(hotplug_dev), ppc->core_id, &index);
     g_assert(core_slot); /* Already checked in spapr_core_pre_plug() */
 
     drc = spapr_drc_by_id(TYPE_SPAPR_DRC_CPU,
-                          spapr_vcpu_id(spapr, cc->core_id));
+                          spapr_vcpu_id(spapr, ppc->core_id));
 
     g_assert(drc || !mc->has_hotpluggable_cpus);
 
@@ -4047,6 +4050,7 @@ static void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
     MachineState *machine = MACHINE(OBJECT(hotplug_dev));
     MachineClass *mc = MACHINE_GET_CLASS(hotplug_dev);
     CPUCore *cc = CPU_CORE(dev);
+    PowerPCCore *ppc = POWERPC_CORE(cc);
     const char *base_core_type = spapr_get_cpu_core_type(machine->cpu_type);
     const char *type = object_get_typename(OBJECT(dev));
     CPUArchId *core_slot;
@@ -4063,8 +4067,8 @@ static void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
         return;
     }
 
-    if (cc->core_id % smp_threads) {
-        error_setg(errp, "invalid core id %d", cc->core_id);
+    if (ppc->core_id % smp_threads) {
+        error_setg(errp, "invalid core id %d", ppc->core_id);
         return;
     }
 
@@ -4080,14 +4084,14 @@ static void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
         return;
     }
 
-    core_slot = spapr_find_cpu_slot(MACHINE(hotplug_dev), cc->core_id, &index);
+    core_slot = spapr_find_cpu_slot(MACHINE(hotplug_dev), ppc->core_id, &index);
     if (!core_slot) {
-        error_setg(errp, "core id %d out of range", cc->core_id);
+        error_setg(errp, "core id %d out of range", ppc->core_id);
         return;
     }
 
     if (core_slot->cpu) {
-        error_setg(errp, "core %d already populated", cc->core_id);
+        error_setg(errp, "core %d already populated", ppc->core_id);
         return;
     }
 
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 91fae56573ee..7c2ea1424747 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -292,7 +292,7 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
 static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 {
     SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_GET_CLASS(sc);
-    CPUCore *cc = CPU_CORE(sc);
+    PowerPCCore *ppc = POWERPC_CORE(sc);
     g_autoptr(Object) obj = NULL;
     g_autofree char *id = NULL;
     CPUState *cs;
@@ -307,7 +307,7 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
      * and the rest are explicitly started up by the guest using an RTAS call.
      */
     cs->start_powered_off = true;
-    cs->cpu_index = cc->core_id + i;
+    cs->cpu_index = ppc->core_id + i;
     if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
         return NULL;
     }
@@ -381,7 +381,7 @@ static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
 static const TypeInfo spapr_cpu_core_type_infos[] = {
     {
         .name = TYPE_SPAPR_CPU_CORE,
-        .parent = TYPE_CPU_CORE,
+        .parent = TYPE_POWERPC_CORE,
         .abstract = true,
         .instance_size = sizeof(SpaprCpuCore),
         .class_size = sizeof(SpaprCpuCoreClass),
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
index 98ab91647eb2..17f117bd5225 100644
--- a/include/hw/cpu/core.h
+++ b/include/hw/cpu/core.h
@@ -21,13 +21,7 @@ struct CPUCore {
     DeviceState parent_obj;
 
     /*< public >*/
-    int core_id;
     int nr_threads;
 };
 
-/* Note: topology field names need to be kept in sync with
- * 'CpuInstanceProperties' */
-
-#define CPU_CORE_PROP_CORE_ID "core-id"
-
 #endif
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 4db21229a68b..56c3f6b51f2f 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -20,9 +20,9 @@
 #ifndef PPC_PNV_CORE_H
 #define PPC_PNV_CORE_H
 
-#include "hw/cpu/core.h"
 #include "target/ppc/cpu.h"
 #include "hw/ppc/pnv.h"
+#include "hw/ppc/ppc_core.h"
 #include "qom/object.h"
 
 #define TYPE_PNV_CORE "powernv-cpu-core"
@@ -31,7 +31,7 @@ OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
 
 struct PnvCore {
     /*< private >*/
-    CPUCore parent_obj;
+    PowerPCCore parent_obj;
 
     /*< public >*/
     PowerPCCPU **threads;
@@ -43,8 +43,10 @@ struct PnvCore {
 };
 
 struct PnvCoreClass {
-    DeviceClass parent_class;
+    /*< private >*/
+    PowerPCCoreClass parent_class;
 
+    /*< public >*/
     const MemoryRegionOps *xscom_ops;
     uint64_t xscom_size;
 };
diff --git a/include/hw/ppc/ppc_core.h b/include/hw/ppc/ppc_core.h
new file mode 100644
index 000000000000..bcc83e426e3f
--- /dev/null
+++ b/include/hw/ppc/ppc_core.h
@@ -0,0 +1,57 @@
+/*
+ * Common PPC CPU core abstraction header
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_PPC_CORE_H
+#define HW_PPC_CORE_H
+
+#include "hw/cpu/core.h"
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define TYPE_POWERPC_CORE "powerpc-core"
+
+OBJECT_DECLARE_TYPE(PowerPCCore, PowerPCCoreClass, POWERPC_CORE)
+
+struct PowerPCCoreClass {
+    /*< private >*/
+    CPUCoreClass parent_class;
+
+    /*< public >*/
+};
+
+struct PowerPCCore {
+    /*< private >*/
+    CPUCore parent_obj;
+
+    /*< public >*/
+    /*
+     * The system-wide id for core, not the sub core-id within the
+     * parent container (which is above the core level).
+     */
+    int core_id;
+};
+
+/*
+ * Note: topology field names need to be kept in sync with
+ * 'CpuInstanceProperties'
+ */
+#define POWERPC_CORE_PROP_CORE_ID "core-id"
+
+#endif /* HW_PPC_CORE_H */
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 69a52e39b850..db3e515051ca 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -9,7 +9,7 @@
 #ifndef HW_SPAPR_CPU_CORE_H
 #define HW_SPAPR_CPU_CORE_H
 
-#include "hw/cpu/core.h"
+#include "hw/ppc/ppc_core.h"
 #include "hw/qdev-core.h"
 #include "target/ppc/cpu-qom.h"
 #include "target/ppc/cpu.h"
@@ -23,7 +23,7 @@ OBJECT_DECLARE_TYPE(SpaprCpuCore, SpaprCpuCoreClass,
 
 struct SpaprCpuCore {
     /*< private >*/
-    CPUCore parent_obj;
+    PowerPCCore parent_obj;
 
     /*< public >*/
     PowerPCCPU **threads;
@@ -32,7 +32,10 @@ struct SpaprCpuCore {
 };
 
 struct SpaprCpuCoreClass {
-    DeviceClass parent_class;
+    /*< private >*/
+    PowerPCCoreClass parent_class;
+
+    /*< public >*/
     const char *cpu_type;
 };
 
-- 
2.34.1

[RFC 15/41] hw/cpu/core: Allow to configure plugged threads for cpu-core

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When the core abstraction is applied for the architecture support CPU
hotplug, the plugged CPUs and unplugged CPUs are distributed among the
cores created in the topology tree.

Add plugged_threads field to help cpu-core decide to how many CPUs to
create.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/core.c                   | 33 +++++++++++++++++++++++++++++++++
 hw/ppc/pnv_core.c               |  6 +++++-
 hw/ppc/spapr_cpu_core.c         |  6 +++++-
 include/hw/cpu/core.h           |  9 +++++++++
 include/hw/ppc/pnv_core.h       |  2 ++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 6 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/hw/cpu/core.c b/hw/cpu/core.c
index 7e274d6aebb7..15546b5b2339 100644
--- a/hw/cpu/core.c
+++ b/hw/cpu/core.c
@@ -36,6 +36,20 @@ static void core_prop_set_nr_threads(Object *obj, Visitor *v, const char *name,
     core->nr_threads = value;
 }
 
+static void core_prop_set_plugged_threads(Object *obj, Visitor *v,
+                                          const char *name, void *opaque,
+                                          Error **errp)
+{
+    CPUCore *core = CPU_CORE(obj);
+    int64_t value;
+
+    if (!visit_type_int(v, name, &value, errp)) {
+        return;
+    }
+
+    core->plugged_threads = value;
+}
+
 static void cpu_core_instance_init(Object *obj)
 {
     CPUCore *core = CPU_CORE(obj);
@@ -48,6 +62,22 @@ static void cpu_core_instance_init(Object *obj)
     if (current_machine) {
         core->nr_threads = current_machine->smp.threads;
     }
+
+    core->plugged_threads = -1;
+}
+
+static void cpu_core_realize(DeviceState *dev, Error **errp)
+{
+    CPUCore *core = CPU_CORE(dev);
+
+    if (core->plugged_threads > core->nr_threads) {
+        error_setg(errp, "Plugged threads (plugged-threads: %d) must "
+                   "not be more than max threads (nr-threads: %d)",
+                   core->plugged_threads, core->nr_threads);
+        return;
+    } else if (core->plugged_threads == -1) {
+        core->plugged_threads = core->nr_threads;
+    }
 }
 
 static void cpu_core_class_init(ObjectClass *oc, void *data)
@@ -57,6 +87,9 @@ static void cpu_core_class_init(ObjectClass *oc, void *data)
     set_bit(DEVICE_CATEGORY_CPU, dc->categories);
     object_class_property_add(oc, "nr-threads", "int", core_prop_get_nr_threads,
                               core_prop_set_nr_threads, NULL, NULL);
+    object_class_property_add(oc, "plugged-threads", "int", NULL,
+                              core_prop_set_plugged_threads, NULL, NULL);
+    dc->realize = cpu_core_realize;
 }
 
 static const TypeInfo cpu_core_type_info = {
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index a90d1ec92bd8..8b75739697d1 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -276,6 +276,8 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
     assert(pc->chip);
 
+    pcc->parent_realize(dev, errp);
+
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
         PowerPCCPU *cpu;
@@ -376,11 +378,13 @@ static void pnv_core_power10_class_init(ObjectClass *oc, void *data)
 static void pnv_core_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
+    PnvCoreClass *pcc = PNV_CORE_CLASS(oc);
 
-    dc->realize = pnv_core_realize;
     dc->unrealize = pnv_core_unrealize;
     device_class_set_props(dc, pnv_core_properties);
     dc->user_creatable = false;
+    device_class_set_parent_realize(dc, pnv_core_realize,
+                                    &pcc->parent_realize);
 }
 
 #define DEFINE_PNV_CORE_TYPE(family, cpu_model) \
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 7c2ea1424747..5533a386f350 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -331,6 +331,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
         (SpaprMachineState *) object_dynamic_cast(qdev_get_machine(),
                                                   TYPE_SPAPR_MACHINE);
     SpaprCpuCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
+    SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_GET_CLASS(sc);
     CPUCore *cc = CPU_CORE(OBJECT(dev));
     int i;
 
@@ -339,6 +340,8 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    scc->parent_realize(dev, errp);
+
     qemu_register_reset(spapr_cpu_core_reset_handler, sc);
     sc->threads = g_new0(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
@@ -363,11 +366,12 @@ static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
     SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_CLASS(oc);
 
-    dc->realize = spapr_cpu_core_realize;
     dc->unrealize = spapr_cpu_core_unrealize;
     dc->reset = spapr_cpu_core_reset;
     device_class_set_props(dc, spapr_cpu_core_properties);
     scc->cpu_type = data;
+    device_class_set_parent_realize(dc, spapr_cpu_core_realize,
+                                    &scc->parent_realize);
 }
 
 #define DEFINE_SPAPR_CPU_CORE_TYPE(cpu_model) \
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
index 17f117bd5225..87d50151ab01 100644
--- a/include/hw/cpu/core.h
+++ b/include/hw/cpu/core.h
@@ -21,7 +21,16 @@ struct CPUCore {
     DeviceState parent_obj;
 
     /*< public >*/
+    int core_id;
+
+    /* Maximum number of threads contained in this core. */
     int nr_threads;
+
+    /*
+     * How many threads should be plugged in this core via
+     * "-device"/"device_add"?
+     */
+    int plugged_threads;
 };
 
 #endif
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 56c3f6b51f2f..3b9edf69f9fb 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -49,6 +49,8 @@ struct PnvCoreClass {
     /*< public >*/
     const MemoryRegionOps *xscom_ops;
     uint64_t xscom_size;
+
+    DeviceRealize parent_realize;
 };
 
 #define PNV_CORE_TYPE_SUFFIX "-" TYPE_PNV_CORE
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index db3e515051ca..dabdbd4bcbc9 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -37,6 +37,8 @@ struct SpaprCpuCoreClass {
 
     /*< public >*/
     const char *cpu_type;
+
+    DeviceRealize parent_realize;
 };
 
 const char *spapr_get_cpu_core_type(const char *cpu_type);
-- 
2.34.1

[RFC 16/41] PPC/ppc-core: Limit plugged-threads and nr-threads to be equal

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

PPC supports CPU hotplug at core granularity, thus ppc-core only accepts
all CPUs in a core are plugged.

Check if plugged_threads and nr_threads are equal when ppc-core
realizes.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/ppc/ppc_core.c         | 18 ++++++++++++++++++
 include/hw/ppc/ppc_core.h |  1 +
 2 files changed, 19 insertions(+)

diff --git a/hw/ppc/ppc_core.c b/hw/ppc/ppc_core.c
index 4433b54af506..3857f3150052 100644
--- a/hw/ppc/ppc_core.c
+++ b/hw/ppc/ppc_core.c
@@ -53,20 +53,38 @@ static void powerpc_core_prop_set_core_id(Object *obj, Visitor *v,
     core->core_id = value;
 }
 
+static void powerpc_core_realize(DeviceState *dev, Error **errp)
+{
+    CPUCore *core = CPU_CORE(dev);
+    PowerPCCoreClass *ppc_class = POWERPC_CORE_GET_CLASS(dev);
+
+    if (core->plugged_threads != -1 &&
+        core->nr_threads != core->plugged_threads) {
+        error_setg(errp, "nr_threads and plugged-threads must be equal");
+        return;
+    }
+
+    ppc_class->parent_realize(dev, errp);
+}
+
 static void powerpc_core_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
+    PowerPCCoreClass *ppc_class = POWERPC_CORE_CLASS(oc);
 
     object_class_property_add(oc, "core-id", "int",
                               powerpc_core_prop_get_core_id,
                               powerpc_core_prop_set_core_id,
                               NULL, NULL);
+    device_class_set_parent_realize(dc, powerpc_core_realize,
+                                    &ppc_class->parent_realize);
 }
 
 static const TypeInfo powerpc_core_type_info = {
     .name = TYPE_POWERPC_CORE,
     .parent = TYPE_CPU_CORE,
     .abstract = true,
+    .class_size = sizeof(PowerPCCoreClass),
     .class_init = powerpc_core_class_init,
     .instance_size = sizeof(PowerPCCore),
 };
diff --git a/include/hw/ppc/ppc_core.h b/include/hw/ppc/ppc_core.h
index bcc83e426e3f..1207d220872a 100644
--- a/include/hw/ppc/ppc_core.h
+++ b/include/hw/ppc/ppc_core.h
@@ -34,6 +34,7 @@ struct PowerPCCoreClass {
     CPUCoreClass parent_class;
 
     /*< public >*/
+    DeviceRealize parent_realize;
 };
 
 struct PowerPCCore {
-- 
2.34.1

[RFC 17/41] hw/cpu/core: Convert cpu-core from general device to topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Convert cpu-core to topology device then user could create core level
topology from cli and later the cpu-cores could be added into topology
tree.

In addition, mark the common cpu-core as DEVICE_CATEGORY_CPU_DEF
category to indicate it belongs to the basic CPU definition and should
be created from cli before board initialization.

But since PPC supports CPU hotplug at core granularity, ppc-core should
be created after board initialization. Thus, mask the category flag
DEVICE_CATEGORY_CPU_DEF for ppc-core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/core.c                   | 19 ++++++++++++++++---
 hw/ppc/pnv_core.c               |  6 +++++-
 hw/ppc/ppc_core.c               |  5 +++++
 hw/ppc/spapr_cpu_core.c         |  7 ++++++-
 include/hw/cpu/core.h           | 13 +++++++++++--
 include/hw/ppc/pnv_core.h       |  1 +
 include/hw/ppc/spapr_cpu_core.h |  1 +
 7 files changed, 45 insertions(+), 7 deletions(-)

diff --git a/hw/cpu/core.c b/hw/cpu/core.c
index 15546b5b2339..261b15fa8171 100644
--- a/hw/cpu/core.c
+++ b/hw/cpu/core.c
@@ -27,6 +27,7 @@ static void core_prop_set_nr_threads(Object *obj, Visitor *v, const char *name,
                                      void *opaque, Error **errp)
 {
     CPUCore *core = CPU_CORE(obj);
+    CPUTopoState *topo = CPU_TOPO(obj);
     int64_t value;
 
     if (!visit_type_int(v, name, &value, errp)) {
@@ -34,6 +35,7 @@ static void core_prop_set_nr_threads(Object *obj, Visitor *v, const char *name,
     }
 
     core->nr_threads = value;
+    topo->max_children = core->nr_threads;
 }
 
 static void core_prop_set_plugged_threads(Object *obj, Visitor *v,
@@ -53,6 +55,7 @@ static void core_prop_set_plugged_threads(Object *obj, Visitor *v,
 static void cpu_core_instance_init(Object *obj)
 {
     CPUCore *core = CPU_CORE(obj);
+    CPUTopoState *topo = CPU_TOPO(obj);
 
     /*
      * Only '-device something-cpu-core,help' can get us there before
@@ -64,11 +67,14 @@ static void cpu_core_instance_init(Object *obj)
     }
 
     core->plugged_threads = -1;
+    /* Core's child can only be the thread. */
+    topo->child_level = CPU_TOPO_THREAD;
 }
 
 static void cpu_core_realize(DeviceState *dev, Error **errp)
 {
     CPUCore *core = CPU_CORE(dev);
+    CPUCoreClass *cc = CPU_CORE_GET_CLASS(core);
 
     if (core->plugged_threads > core->nr_threads) {
         error_setg(errp, "Plugged threads (plugged-threads: %d) must "
@@ -78,25 +84,32 @@ static void cpu_core_realize(DeviceState *dev, Error **errp)
     } else if (core->plugged_threads == -1) {
         core->plugged_threads = core->nr_threads;
     }
+
+    cc->parent_realize(dev, errp);
 }
 
 static void cpu_core_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+    CPUCoreClass *cc = CPU_CORE_CLASS(oc);
 
-    set_bit(DEVICE_CATEGORY_CPU, dc->categories);
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
     object_class_property_add(oc, "nr-threads", "int", core_prop_get_nr_threads,
                               core_prop_set_nr_threads, NULL, NULL);
     object_class_property_add(oc, "plugged-threads", "int", NULL,
                               core_prop_set_plugged_threads, NULL, NULL);
-    dc->realize = cpu_core_realize;
+    device_class_set_parent_realize(dc, cpu_core_realize, &cc->parent_realize);
+
+    tc->level = CPU_TOPO_CORE;
 }
 
 static const TypeInfo cpu_core_type_info = {
     .name = TYPE_CPU_CORE,
-    .parent = TYPE_DEVICE,
+    .parent = TYPE_CPU_TOPO,
     .abstract = true,
     .class_init = cpu_core_class_init,
+    .class_size = sizeof(CPUCoreClass),
     .instance_size = sizeof(CPUCore),
     .instance_init = cpu_core_instance_init,
 };
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 8b75739697d1..315b823e7d38 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -334,6 +334,7 @@ static void pnv_core_unrealize(DeviceState *dev)
 {
     PnvCore *pc = PNV_CORE(dev);
     CPUCore *cc = CPU_CORE(dev);
+    PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
     int i;
 
     qemu_unregister_reset(pnv_core_reset, pc);
@@ -342,6 +343,8 @@ static void pnv_core_unrealize(DeviceState *dev)
         pnv_core_cpu_unrealize(pc, pc->threads[i]);
     }
     g_free(pc->threads);
+
+    pcc->parent_unrealize(dev);
 }
 
 static Property pnv_core_properties[] = {
@@ -380,11 +383,12 @@ static void pnv_core_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
     PnvCoreClass *pcc = PNV_CORE_CLASS(oc);
 
-    dc->unrealize = pnv_core_unrealize;
     device_class_set_props(dc, pnv_core_properties);
     dc->user_creatable = false;
     device_class_set_parent_realize(dc, pnv_core_realize,
                                     &pcc->parent_realize);
+    device_class_set_parent_unrealize(dc, pnv_core_unrealize,
+                                      &pcc->parent_unrealize);
 }
 
 #define DEFINE_PNV_CORE_TYPE(family, cpu_model) \
diff --git a/hw/ppc/ppc_core.c b/hw/ppc/ppc_core.c
index 3857f3150052..0a700d6a5b42 100644
--- a/hw/ppc/ppc_core.c
+++ b/hw/ppc/ppc_core.c
@@ -72,6 +72,11 @@ static void powerpc_core_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
     PowerPCCoreClass *ppc_class = POWERPC_CORE_CLASS(oc);
 
+    /*
+     * PPC cores support hotplug and must be created after
+     * qemu_init_board().
+     */
+    clear_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
     object_class_property_add(oc, "core-id", "int",
                               powerpc_core_prop_get_core_id,
                               powerpc_core_prop_set_core_id,
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 5533a386f350..c965c213ab14 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -235,6 +235,7 @@ static void spapr_delete_vcpu(PowerPCCPU *cpu)
 static void spapr_cpu_core_unrealize(DeviceState *dev)
 {
     SpaprCpuCore *sc = SPAPR_CPU_CORE(OBJECT(dev));
+    SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_GET_CLASS(sc);
     CPUCore *cc = CPU_CORE(dev);
     int i;
 
@@ -254,6 +255,8 @@ static void spapr_cpu_core_unrealize(DeviceState *dev)
     }
     g_free(sc->threads);
     qemu_unregister_reset(spapr_cpu_core_reset_handler, sc);
+
+    scc->parent_unrealize(dev);
 }
 
 static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
@@ -366,12 +369,14 @@ static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
     SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_CLASS(oc);
 
-    dc->unrealize = spapr_cpu_core_unrealize;
     dc->reset = spapr_cpu_core_reset;
     device_class_set_props(dc, spapr_cpu_core_properties);
+    dc->hotpluggable = true;
     scc->cpu_type = data;
     device_class_set_parent_realize(dc, spapr_cpu_core_realize,
                                     &scc->parent_realize);
+    device_class_set_parent_unrealize(dc, spapr_cpu_core_unrealize,
+                                      &scc->parent_unrealize);
 }
 
 #define DEFINE_SPAPR_CPU_CORE_TYPE(cpu_model) \
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
index 87d50151ab01..591240861efb 100644
--- a/include/hw/cpu/core.h
+++ b/include/hw/cpu/core.h
@@ -10,15 +10,24 @@
 #define HW_CPU_CORE_H
 
 #include "hw/qdev-core.h"
+#include "hw/core/cpu-topo.h"
 #include "qom/object.h"
 
 #define TYPE_CPU_CORE "cpu-core"
 
-OBJECT_DECLARE_SIMPLE_TYPE(CPUCore, CPU_CORE)
+OBJECT_DECLARE_TYPE(CPUCore, CPUCoreClass, CPU_CORE)
+
+struct CPUCoreClass {
+    /*< private >*/
+    CPUTopoClass parent_class;
+
+    /*< public >*/
+    DeviceRealize parent_realize;
+};
 
 struct CPUCore {
     /*< private >*/
-    DeviceState parent_obj;
+    CPUTopoState parent_obj;
 
     /*< public >*/
     int core_id;
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 3b9edf69f9fb..ca04461c8531 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -51,6 +51,7 @@ struct PnvCoreClass {
     uint64_t xscom_size;
 
     DeviceRealize parent_realize;
+    DeviceUnrealize parent_unrealize;
 };
 
 #define PNV_CORE_TYPE_SUFFIX "-" TYPE_PNV_CORE
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index dabdbd4bcbc9..46cf68fc8859 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -39,6 +39,7 @@ struct SpaprCpuCoreClass {
     const char *cpu_type;
 
     DeviceRealize parent_realize;
+    DeviceUnrealize parent_unrealize;
 };
 
 const char *spapr_get_cpu_core_type(const char *cpu_type);
-- 
2.34.1

[RFC 18/41] hw/cpu/cluster: Rename CPUClusterState to CPUCluster

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

To keep the same naming style as cpu-core, rename CPUClusterState to
CPUCluster.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 gdbstub/system.c                   | 2 +-
 hw/cpu/cluster.c                   | 8 ++++----
 include/hw/arm/armsse.h            | 2 +-
 include/hw/arm/xlnx-versal.h       | 4 ++--
 include/hw/arm/xlnx-zynqmp.h       | 4 ++--
 include/hw/cpu/cluster.h           | 6 +++---
 include/hw/riscv/microchip_pfsoc.h | 4 ++--
 include/hw/riscv/sifive_u.h        | 4 ++--
 8 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/gdbstub/system.c b/gdbstub/system.c
index 783ac140b982..1c0b55d3ebe7 100644
--- a/gdbstub/system.c
+++ b/gdbstub/system.c
@@ -277,7 +277,7 @@ static int find_cpu_clusters(Object *child, void *opaque)
 {
     if (object_dynamic_cast(child, TYPE_CPU_CLUSTER)) {
         GDBState *s = (GDBState *) opaque;
-        CPUClusterState *cluster = CPU_CLUSTER(child);
+        CPUCluster *cluster = CPU_CLUSTER(child);
         GDBProcess *process;
 
         s->processes = g_renew(GDBProcess, s->processes, ++s->process_num);
diff --git a/hw/cpu/cluster.c b/hw/cpu/cluster.c
index 61289a840d46..fd978a543e40 100644
--- a/hw/cpu/cluster.c
+++ b/hw/cpu/cluster.c
@@ -26,12 +26,12 @@
 #include "qapi/error.h"
 
 static Property cpu_cluster_properties[] = {
-    DEFINE_PROP_UINT32("cluster-id", CPUClusterState, cluster_id, 0),
+    DEFINE_PROP_UINT32("cluster-id", CPUCluster, cluster_id, 0),
     DEFINE_PROP_END_OF_LIST()
 };
 
 typedef struct CallbackData {
-    CPUClusterState *cluster;
+    CPUCluster *cluster;
     int cpu_count;
 } CallbackData;
 
@@ -50,7 +50,7 @@ static int add_cpu_to_cluster(Object *obj, void *opaque)
 static void cpu_cluster_realize(DeviceState *dev, Error **errp)
 {
     /* Iterate through all our CPU children and set their cluster_index */
-    CPUClusterState *cluster = CPU_CLUSTER(dev);
+    CPUCluster *cluster = CPU_CLUSTER(dev);
     Object *cluster_obj = OBJECT(dev);
     CallbackData cbdata = {
         .cluster = cluster,
@@ -87,7 +87,7 @@ static void cpu_cluster_class_init(ObjectClass *klass, void *data)
 static const TypeInfo cpu_cluster_type_info = {
     .name = TYPE_CPU_CLUSTER,
     .parent = TYPE_DEVICE,
-    .instance_size = sizeof(CPUClusterState),
+    .instance_size = sizeof(CPUCluster),
     .class_init = cpu_cluster_class_init,
 };
 
diff --git a/include/hw/arm/armsse.h b/include/hw/arm/armsse.h
index 88b3b759c5a8..886586a3bed4 100644
--- a/include/hw/arm/armsse.h
+++ b/include/hw/arm/armsse.h
@@ -153,7 +153,7 @@ struct ARMSSE {
 
     /*< public >*/
     ARMv7MState armv7m[SSE_MAX_CPUS];
-    CPUClusterState cluster[SSE_MAX_CPUS];
+    CPUCluster cluster[SSE_MAX_CPUS];
     IoTKitSecCtl secctl;
     TZPPC apb_ppc[NUM_INTERNAL_PPCS];
     TZMPC mpc[IOTS_NUM_MPC];
diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h
index b24fa64557fd..61bde52b6af5 100644
--- a/include/hw/arm/xlnx-versal.h
+++ b/include/hw/arm/xlnx-versal.h
@@ -58,7 +58,7 @@ struct Versal {
     struct {
         struct {
             MemoryRegion mr;
-            CPUClusterState cluster;
+            CPUCluster cluster;
             ARMCPU cpu[XLNX_VERSAL_NR_ACPUS];
             GICv3State gic;
         } apu;
@@ -88,7 +88,7 @@ struct Versal {
             MemoryRegion mr;
             MemoryRegion mr_ps_alias;
 
-            CPUClusterState cluster;
+            CPUCluster cluster;
             ARMCPU cpu[XLNX_VERSAL_NR_RCPUS];
         } rpu;
 
diff --git a/include/hw/arm/xlnx-zynqmp.h b/include/hw/arm/xlnx-zynqmp.h
index 96358d51ebb7..5eea765ea76c 100644
--- a/include/hw/arm/xlnx-zynqmp.h
+++ b/include/hw/arm/xlnx-zynqmp.h
@@ -98,8 +98,8 @@ struct XlnxZynqMPState {
     DeviceState parent_obj;
 
     /*< public >*/
-    CPUClusterState apu_cluster;
-    CPUClusterState rpu_cluster;
+    CPUCluster apu_cluster;
+    CPUCluster rpu_cluster;
     ARMCPU apu_cpu[XLNX_ZYNQMP_NUM_APU_CPUS];
     ARMCPU rpu_cpu[XLNX_ZYNQMP_NUM_RPU_CPUS];
     GICState gic;
diff --git a/include/hw/cpu/cluster.h b/include/hw/cpu/cluster.h
index 53fbf36af542..644b87350268 100644
--- a/include/hw/cpu/cluster.h
+++ b/include/hw/cpu/cluster.h
@@ -55,7 +55,7 @@
  */
 
 #define TYPE_CPU_CLUSTER "cpu-cluster"
-OBJECT_DECLARE_SIMPLE_TYPE(CPUClusterState, CPU_CLUSTER)
+OBJECT_DECLARE_SIMPLE_TYPE(CPUCluster, CPU_CLUSTER)
 
 /*
  * This limit is imposed by TCG, which puts the cluster ID into an
@@ -64,13 +64,13 @@ OBJECT_DECLARE_SIMPLE_TYPE(CPUClusterState, CPU_CLUSTER)
 #define MAX_CLUSTERS 255
 
 /**
- * CPUClusterState:
+ * CPUCluster:
  * @cluster_id: The cluster ID. This value is for internal use only and should
  *   not be exposed directly to the user or to the guest.
  *
  * State of a CPU cluster.
  */
-struct CPUClusterState {
+struct CPUCluster {
     /*< private >*/
     DeviceState parent_obj;
 
diff --git a/include/hw/riscv/microchip_pfsoc.h b/include/hw/riscv/microchip_pfsoc.h
index daef086da602..c9ac14e35625 100644
--- a/include/hw/riscv/microchip_pfsoc.h
+++ b/include/hw/riscv/microchip_pfsoc.h
@@ -38,8 +38,8 @@ typedef struct MicrochipPFSoCState {
     DeviceState parent_obj;
 
     /*< public >*/
-    CPUClusterState e_cluster;
-    CPUClusterState u_cluster;
+    CPUCluster e_cluster;
+    CPUCluster u_cluster;
     RISCVHartArrayState e_cpus;
     RISCVHartArrayState u_cpus;
     DeviceState *plic;
diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
index 0696f8594277..fda4a708e960 100644
--- a/include/hw/riscv/sifive_u.h
+++ b/include/hw/riscv/sifive_u.h
@@ -40,8 +40,8 @@ typedef struct SiFiveUSoCState {
     DeviceState parent_obj;
 
     /*< public >*/
-    CPUClusterState e_cluster;
-    CPUClusterState u_cluster;
+    CPUCluster e_cluster;
+    CPUCluster u_cluster;
     RISCVHartArrayState e_cpus;
     RISCVHartArrayState u_cpus;
     DeviceState *plic;
-- 
2.34.1

[RFC 19/41] hw/cpu/cluster: Wrap TCG related ops and props into CONFIG_TCG

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Currenltly cpu-cluster is used in TCG case to organize CPUs with the
same type.

Wrap 2 things into TCG specific areas:
1. cluster-id:

   The cluster-id in TCG case is global, since no higher topology
   container above cluster. To simplify the logic of cluster topology
   in virtualization, move the cluster-id into CONFIG_TCG, then it
   won't be exposed in cli.

2. CPU collection in realize():

   In TCG case, the CPUs are added into cluster directly via child<>
   property. But in virtualization case, the CPU topology will be built
   via topology tree. Thus, wrap CPU collection as the TCG operation.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/cluster.c         | 30 +++++++++++++++++++++++++-----
 include/hw/cpu/cluster.h | 22 ++++++++++++++++++++--
 2 files changed, 45 insertions(+), 7 deletions(-)

diff --git a/hw/cpu/cluster.c b/hw/cpu/cluster.c
index fd978a543e40..340cfad9f8f1 100644
--- a/hw/cpu/cluster.c
+++ b/hw/cpu/cluster.c
@@ -26,7 +26,9 @@
 #include "qapi/error.h"
 
 static Property cpu_cluster_properties[] = {
+#ifdef CONFIG_TCG
     DEFINE_PROP_UINT32("cluster-id", CPUCluster, cluster_id, 0),
+#endif
     DEFINE_PROP_END_OF_LIST()
 };
 
@@ -47,18 +49,17 @@ static int add_cpu_to_cluster(Object *obj, void *opaque)
     return 0;
 }
 
-static void cpu_cluster_realize(DeviceState *dev, Error **errp)
+static void cpu_cluster_common_collect_cpus(CPUCluster *cluster, Error **errp)
 {
     /* Iterate through all our CPU children and set their cluster_index */
-    CPUCluster *cluster = CPU_CLUSTER(dev);
-    Object *cluster_obj = OBJECT(dev);
+    Object *cluster_obj = OBJECT(cluster);
     CallbackData cbdata = {
         .cluster = cluster,
         .cpu_count = 0,
     };
 
-    if (cluster->cluster_id >= MAX_CLUSTERS) {
-        error_setg(errp, "cluster-id must be less than %d", MAX_CLUSTERS);
+    if (cluster->cluster_id >= MAX_TCG_CLUSTERS) {
+        error_setg(errp, "cluster-id must be less than %d", MAX_TCG_CLUSTERS);
         return;
     }
 
@@ -73,15 +74,34 @@ static void cpu_cluster_realize(DeviceState *dev, Error **errp)
     assert(cbdata.cpu_count > 0);
 }
 
+static const struct TCGClusterOps common_cluster_tcg_ops = {
+    .collect_cpus = cpu_cluster_common_collect_cpus,
+};
+
+static void cpu_cluster_realize(DeviceState *dev, Error **errp)
+{
+    CPUCluster *cluster = CPU_CLUSTER(dev);
+    CPUClusterClass *cc = CPU_CLUSTER_GET_CLASS(dev);
+
+    if (cc->tcg_clu_ops->collect_cpus) {
+        cc->tcg_clu_ops->collect_cpus(cluster, errp);
+    }
+}
+
 static void cpu_cluster_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
+    CPUClusterClass *cc = CPU_CLUSTER_CLASS(klass);
 
     device_class_set_props(dc, cpu_cluster_properties);
     dc->realize = cpu_cluster_realize;
 
     /* This is not directly for users, CPU children must be attached by code */
     dc->user_creatable = false;
+
+#ifdef CONFIG_TCG
+    cc->tcg_clu_ops = &common_cluster_tcg_ops;
+#endif
 }
 
 static const TypeInfo cpu_cluster_type_info = {
diff --git a/include/hw/cpu/cluster.h b/include/hw/cpu/cluster.h
index 644b87350268..c038f05ddc9f 100644
--- a/include/hw/cpu/cluster.h
+++ b/include/hw/cpu/cluster.h
@@ -55,13 +55,31 @@
  */
 
 #define TYPE_CPU_CLUSTER "cpu-cluster"
-OBJECT_DECLARE_SIMPLE_TYPE(CPUCluster, CPU_CLUSTER)
+OBJECT_DECLARE_TYPE(CPUCluster, CPUClusterClass, CPU_CLUSTER)
 
 /*
  * This limit is imposed by TCG, which puts the cluster ID into an
  * 8 bit field (and uses all-1s for the default "not in any cluster").
  */
-#define MAX_CLUSTERS 255
+#define MAX_TCG_CLUSTERS 255
+
+struct TCGClusterOps {
+    /**
+     * @collect_cpus: Iterate children CPUs and set cluser_index.
+     *
+     * Called when the cluster is realized.
+     */
+    void (*collect_cpus)(CPUCluster *cluster, Error **errp);
+};
+
+struct CPUClusterClass {
+    /*< private >*/
+    DeviceClass parent_class;
+
+    /*< public >*/
+    /* when TCG is not available, this pointer is NULL */
+    const struct TCGClusterOps *tcg_clu_ops;
+};
 
 /**
  * CPUCluster:
-- 
2.34.1

[RFC 20/41] hw/cpu/cluster: Descript cluster is not only used for TCG in comment

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Update the comment to make the cpu-cluster description more general for
both TCG and accel cases.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/cluster.c         |  2 +-
 include/hw/cpu/cluster.h | 20 +++++++++++++++-----
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/hw/cpu/cluster.c b/hw/cpu/cluster.c
index 340cfad9f8f1..27ab9e25a265 100644
--- a/hw/cpu/cluster.c
+++ b/hw/cpu/cluster.c
@@ -1,5 +1,5 @@
 /*
- * QEMU CPU cluster
+ * CPU cluster abstract device
  *
  * Copyright (c) 2018 GreenSocs SAS
  *
diff --git a/include/hw/cpu/cluster.h b/include/hw/cpu/cluster.h
index c038f05ddc9f..b3185e2f2566 100644
--- a/include/hw/cpu/cluster.h
+++ b/include/hw/cpu/cluster.h
@@ -1,5 +1,5 @@
 /*
- * QEMU CPU cluster
+ * CPU cluster abstract device
  *
  * Copyright (c) 2018 GreenSocs SAS
  *
@@ -24,17 +24,27 @@
 #include "qom/object.h"
 
 /*
- * CPU Cluster type
+ * # CPU Cluster
  *
- * A cluster is a group of CPUs which are all identical and have the same view
- * of the rest of the system. It is mainly an internal QEMU representation and
- * does not necessarily match with the notion of clusters on the real hardware.
+ * A cluster is a group of CPUs, that is, a level above the CPU (or Core).
+ *
+ * - For accel case, it's a CPU topology level concept above cores, in which
+ * the cores may share some resources (L2 cache or some others like L3
+ * cache tags, depending on the Archs). It is used to emulate the physical
+ * CPU cluster/module.
+ *
+ * - For TCG, cluster is used to organize CPUs directly without core. In one
+ * cluster, CPUs are all identical and have the same view of the rest of the
+ * system. It is mainly an internal QEMU representation and may not necessarily
+ * match with the notion of clusters on the real hardware.
  *
  * If CPUs are not identical (for example, Cortex-A53 and Cortex-A57 CPUs in an
  * Arm big.LITTLE system) they should be in different clusters. If the CPUs do
  * not have the same view of memory (for example the main CPU and a management
  * controller processor) they should be in different clusters.
  *
+ * # Use case for cluster in TCG
+ *
  * A cluster is created by creating an object of TYPE_CPU_CLUSTER, and then
  * adding the CPUs to it as QOM child objects (e.g. using the
  * object_initialize_child() or object_property_add_child() functions).
-- 
2.34.1

[RFC 21/41] hw/cpu/cluster: Allow cpu-cluster to be created by -device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since we will extend cpu-cluster to build CPU topology in virtualization
case via -device, remove the original limitition.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/cluster.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/hw/cpu/cluster.c b/hw/cpu/cluster.c
index 27ab9e25a265..8a666c27d151 100644
--- a/hw/cpu/cluster.c
+++ b/hw/cpu/cluster.c
@@ -96,9 +96,6 @@ static void cpu_cluster_class_init(ObjectClass *klass, void *data)
     device_class_set_props(dc, cpu_cluster_properties);
     dc->realize = cpu_cluster_realize;
 
-    /* This is not directly for users, CPU children must be attached by code */
-    dc->user_creatable = false;
-
 #ifdef CONFIG_TCG
     cc->tcg_clu_ops = &common_cluster_tcg_ops;
 #endif
-- 
2.34.1

[RFC 22/41] hw/cpu/cluster: Convert cpu-cluster from general device to topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Convert cpu-cluster to topology device then user could create cluster
level topology from cli and later the cpu-clusters could be added into
topology tree.

In addition, mark the cpu-cluster as DEVICE_CATEGORY_CPU_DEF category to
indicate it belongs to the basic CPU definition and should be created
from cli before board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/cluster.c         | 11 +++++++++--
 include/hw/cpu/cluster.h |  7 +++++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/hw/cpu/cluster.c b/hw/cpu/cluster.c
index 8a666c27d151..adf0ef23e8d4 100644
--- a/hw/cpu/cluster.c
+++ b/hw/cpu/cluster.c
@@ -86,15 +86,21 @@ static void cpu_cluster_realize(DeviceState *dev, Error **errp)
     if (cc->tcg_clu_ops->collect_cpus) {
         cc->tcg_clu_ops->collect_cpus(cluster, errp);
     }
+
+    cc->parent_realize(dev, errp);
 }
 
 static void cpu_cluster_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(klass);
     CPUClusterClass *cc = CPU_CLUSTER_CLASS(klass);
 
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
     device_class_set_props(dc, cpu_cluster_properties);
-    dc->realize = cpu_cluster_realize;
+    device_class_set_parent_realize(dc, cpu_cluster_realize,
+                                    &cc->parent_realize);
+    tc->level = CPU_TOPO_CLUSTER;
 
 #ifdef CONFIG_TCG
     cc->tcg_clu_ops = &common_cluster_tcg_ops;
@@ -103,8 +109,9 @@ static void cpu_cluster_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo cpu_cluster_type_info = {
     .name = TYPE_CPU_CLUSTER,
-    .parent = TYPE_DEVICE,
+    .parent = TYPE_CPU_TOPO,
     .instance_size = sizeof(CPUCluster),
+    .class_size = sizeof(CPUClusterClass),
     .class_init = cpu_cluster_class_init,
 };
 
diff --git a/include/hw/cpu/cluster.h b/include/hw/cpu/cluster.h
index b3185e2f2566..888993c36da4 100644
--- a/include/hw/cpu/cluster.h
+++ b/include/hw/cpu/cluster.h
@@ -20,6 +20,7 @@
 #ifndef HW_CPU_CLUSTER_H
 #define HW_CPU_CLUSTER_H
 
+#include "hw/core/cpu-topo.h"
 #include "hw/qdev-core.h"
 #include "qom/object.h"
 
@@ -84,11 +85,13 @@ struct TCGClusterOps {
 
 struct CPUClusterClass {
     /*< private >*/
-    DeviceClass parent_class;
+    CPUTopoClass parent_class;
 
     /*< public >*/
     /* when TCG is not available, this pointer is NULL */
     const struct TCGClusterOps *tcg_clu_ops;
+
+    DeviceRealize parent_realize;
 };
 
 /**
@@ -100,7 +103,7 @@ struct CPUClusterClass {
  */
 struct CPUCluster {
     /*< private >*/
-    DeviceState parent_obj;
+    CPUTopoState parent_obj;
 
     /*< public >*/
     uint32_t cluster_id;
-- 
2.34.1

[RFC 23/41] hw/cpu/die: Abstract cpu-die level as topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Abstract die level as a topology device "cpu-die" to allow user to
create die level topology from cli and later the cpu-dies could be added
into topology tree.

In addition, mark the cpu-die as DEVICE_CATEGORY_CPU_DEF category to
indicate it belongs to the basic CPU definition and should be created
from cli before board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS          |  2 ++
 hw/cpu/die.c         | 46 ++++++++++++++++++++++++++++++++++++++++++++
 hw/cpu/meson.build   |  2 +-
 include/hw/cpu/die.h | 38 ++++++++++++++++++++++++++++++++++++
 4 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 hw/cpu/die.c
 create mode 100644 include/hw/cpu/die.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 89e350866d6a..91d0936edb32 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1863,6 +1863,7 @@ F: hw/core/machine-smp.c
 F: hw/core/null-machine.c
 F: hw/core/numa.c
 F: hw/cpu/cluster.c
+F: hw/cpu/die.c
 F: qapi/machine.json
 F: qapi/machine-common.json
 F: qapi/machine-target.json
@@ -1870,6 +1871,7 @@ F: include/hw/boards.h
 F: include/hw/core/cpu.h
 F: include/hw/core/cpu-topo.h
 F: include/hw/cpu/cluster.h
+F: include/hw/cpu/die.h
 F: include/sysemu/numa.h
 F: tests/unit/test-smp-parse.c
 T: git https://gitlab.com/ehabkost/qemu.git machine-next
diff --git a/hw/cpu/die.c b/hw/cpu/die.c
new file mode 100644
index 000000000000..06c4f7cc8fa2
--- /dev/null
+++ b/hw/cpu/die.c
@@ -0,0 +1,46 @@
+/*
+ * CPU die abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cpu/die.h"
+
+static void cpu_die_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
+
+    tc->level = CPU_TOPO_DIE;
+}
+
+static const TypeInfo cpu_die_type_info = {
+    .name = TYPE_CPU_DIE,
+    .parent = TYPE_CPU_TOPO,
+    .class_init = cpu_die_class_init,
+    .instance_size = sizeof(CPUDie),
+};
+
+static void cpu_die_register_types(void)
+{
+    type_register_static(&cpu_die_type_info);
+}
+
+type_init(cpu_die_register_types)
diff --git a/hw/cpu/meson.build b/hw/cpu/meson.build
index 6d319947ca0b..e685fe1c7d8a 100644
--- a/hw/cpu/meson.build
+++ b/hw/cpu/meson.build
@@ -1,4 +1,4 @@
-system_ss.add(files('core.c', 'cluster.c'))
+system_ss.add(files('core.c', 'cluster.c', 'die.c'))
 
 system_ss.add(when: 'CONFIG_ARM11MPCORE', if_true: files('arm11mpcore.c'))
 system_ss.add(when: 'CONFIG_REALVIEW', if_true: files('realview_mpcore.c'))
diff --git a/include/hw/cpu/die.h b/include/hw/cpu/die.h
new file mode 100644
index 000000000000..4c68786b5f2f
--- /dev/null
+++ b/include/hw/cpu/die.h
@@ -0,0 +1,38 @@
+/*
+ * CPU die abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_CPU_DIE_H
+#define HW_CPU_DIE_H
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_CPU_DIE "cpu-die"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CPUDie, CPU_DIE)
+
+struct CPUDie {
+    /*< private >*/
+    CPUTopoState obj;
+
+    /*< public >*/
+};
+
+#endif /* HW_CPU_DIE_H */
-- 
2.34.1

[RFC 24/41] hw/cpu/socket: Abstract cpu-socket level as topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Abstract socket level as a topology device "cpu-socket" to allow user to
create socket level topology from cli and later the cpu-sockets could be
added into topology tree.

In addition, mark the cpu-socket as DEVICE_CATEGORY_CPU_DEF category to
indicate it belongs to the basic CPU definition and should be created
from cli before board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS             |  2 ++
 hw/cpu/meson.build      |  2 +-
 hw/cpu/socket.c         | 46 +++++++++++++++++++++++++++++++++++++++++
 include/hw/cpu/socket.h | 38 ++++++++++++++++++++++++++++++++++
 4 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 hw/cpu/socket.c
 create mode 100644 include/hw/cpu/socket.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 91d0936edb32..6a9fa0aeed0c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1864,6 +1864,7 @@ F: hw/core/null-machine.c
 F: hw/core/numa.c
 F: hw/cpu/cluster.c
 F: hw/cpu/die.c
+F: hw/cpu/socket.c
 F: qapi/machine.json
 F: qapi/machine-common.json
 F: qapi/machine-target.json
@@ -1872,6 +1873,7 @@ F: include/hw/core/cpu.h
 F: include/hw/core/cpu-topo.h
 F: include/hw/cpu/cluster.h
 F: include/hw/cpu/die.h
+F: include/hw/cpu/socket.h
 F: include/sysemu/numa.h
 F: tests/unit/test-smp-parse.c
 T: git https://gitlab.com/ehabkost/qemu.git machine-next
diff --git a/hw/cpu/meson.build b/hw/cpu/meson.build
index e685fe1c7d8a..251724fea86c 100644
--- a/hw/cpu/meson.build
+++ b/hw/cpu/meson.build
@@ -1,4 +1,4 @@
-system_ss.add(files('core.c', 'cluster.c', 'die.c'))
+system_ss.add(files('core.c', 'cluster.c', 'die.c', 'socket.c'))
 
 system_ss.add(when: 'CONFIG_ARM11MPCORE', if_true: files('arm11mpcore.c'))
 system_ss.add(when: 'CONFIG_REALVIEW', if_true: files('realview_mpcore.c'))
diff --git a/hw/cpu/socket.c b/hw/cpu/socket.c
new file mode 100644
index 000000000000..afd29f8a91c1
--- /dev/null
+++ b/hw/cpu/socket.c
@@ -0,0 +1,46 @@
+/*
+ * CPU socket abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cpu/socket.h"
+
+static void cpu_socket_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
+
+    tc->level = CPU_TOPO_SOCKET;
+}
+
+static const TypeInfo cpu_socket_type_info = {
+    .name = TYPE_CPU_SOCKET,
+    .parent = TYPE_CPU_TOPO,
+    .class_init = cpu_socket_class_init,
+    .instance_size = sizeof(CPUSocket),
+};
+
+static void cpu_socket_register_types(void)
+{
+    type_register_static(&cpu_socket_type_info);
+}
+
+type_init(cpu_socket_register_types)
diff --git a/include/hw/cpu/socket.h b/include/hw/cpu/socket.h
new file mode 100644
index 000000000000..897852903cd2
--- /dev/null
+++ b/include/hw/cpu/socket.h
@@ -0,0 +1,38 @@
+/*
+ * CPU socket abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_CPU_SOCKET_H
+#define HW_CPU_SOCKET_H
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_CPU_SOCKET "cpu-socket"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CPUSocket, CPU_SOCKET)
+
+struct CPUSocket {
+    /*< private >*/
+    CPUTopoState parent_obj;
+
+    /*< public >*/
+};
+
+#endif /* HW_CPU_SOCKET_H */
-- 
2.34.1

[RFC 25/41] hw/cpu/book: Abstract cpu-book level as topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Abstract book level as a topology device "cpu-book" to allow user to
create book level topology from cli and later the cpu-books could be
added into topology tree.

In addition, mark the cpu-book as DEVICE_CATEGORY_CPU_DEF category to
indicate it belongs to the basic CPU definition and should be created
from cli before board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS           |  2 ++
 hw/cpu/book.c         | 46 +++++++++++++++++++++++++++++++++++++++++++
 hw/cpu/meson.build    |  2 +-
 include/hw/cpu/book.h | 38 +++++++++++++++++++++++++++++++++++
 4 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 hw/cpu/book.c
 create mode 100644 include/hw/cpu/book.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 6a9fa0aeed0c..dd5adfda64cc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1862,6 +1862,7 @@ F: hw/core/machine.c
 F: hw/core/machine-smp.c
 F: hw/core/null-machine.c
 F: hw/core/numa.c
+F: hw/cpu/book.c
 F: hw/cpu/cluster.c
 F: hw/cpu/die.c
 F: hw/cpu/socket.c
@@ -1871,6 +1872,7 @@ F: qapi/machine-target.json
 F: include/hw/boards.h
 F: include/hw/core/cpu.h
 F: include/hw/core/cpu-topo.h
+F: include/hw/cpu/book.h
 F: include/hw/cpu/cluster.h
 F: include/hw/cpu/die.h
 F: include/hw/cpu/socket.h
diff --git a/hw/cpu/book.c b/hw/cpu/book.c
new file mode 100644
index 000000000000..4b16267b10eb
--- /dev/null
+++ b/hw/cpu/book.c
@@ -0,0 +1,46 @@
+/*
+ * CPU book abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cpu/book.h"
+
+static void cpu_book_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
+
+    tc->level = CPU_TOPO_BOOK;
+}
+
+static const TypeInfo cpu_book_type_info = {
+    .name = TYPE_CPU_BOOK,
+    .parent = TYPE_CPU_TOPO,
+    .class_init = cpu_book_class_init,
+    .instance_size = sizeof(CPUBook),
+};
+
+static void cpu_book_register_types(void)
+{
+    type_register_static(&cpu_book_type_info);
+}
+
+type_init(cpu_book_register_types)
diff --git a/hw/cpu/meson.build b/hw/cpu/meson.build
index 251724fea86c..c44b54c5abb0 100644
--- a/hw/cpu/meson.build
+++ b/hw/cpu/meson.build
@@ -1,4 +1,4 @@
-system_ss.add(files('core.c', 'cluster.c', 'die.c', 'socket.c'))
+system_ss.add(files('core.c', 'cluster.c', 'die.c', 'socket.c', 'book.c'))
 
 system_ss.add(when: 'CONFIG_ARM11MPCORE', if_true: files('arm11mpcore.c'))
 system_ss.add(when: 'CONFIG_REALVIEW', if_true: files('realview_mpcore.c'))
diff --git a/include/hw/cpu/book.h b/include/hw/cpu/book.h
new file mode 100644
index 000000000000..b91bd553bea6
--- /dev/null
+++ b/include/hw/cpu/book.h
@@ -0,0 +1,38 @@
+/*
+ * CPU book abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_CPU_BOOK_H
+#define HW_CPU_BOOK_H
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_CPU_BOOK "cpu-book"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CPUBook, CPU_BOOK)
+
+struct CPUBook {
+    /*< private >*/
+    CPUTopoState obj;
+
+    /*< public >*/
+};
+
+#endif /* HW_CPU_BOOK_H */
-- 
2.34.1

[RFC 26/41] hw/cpu/drawer: Abstract cpu-drawer level as topology device

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Abstract drawer level as a topology device "cpu-drawer" to allow user to
create drawer level topology from cli and later the cpu-drawers could be
added into topology tree.

In addition, mark the cpu-drawer as DEVICE_CATEGORY_CPU_DEF category to
indicate it belongs to the basic CPU definition and should be created
from cli before board initialization.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS             |  2 ++
 hw/cpu/drawer.c         | 46 +++++++++++++++++++++++++++++++++++++++++
 hw/cpu/meson.build      |  2 +-
 include/hw/cpu/drawer.h | 38 ++++++++++++++++++++++++++++++++++
 4 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 hw/cpu/drawer.c
 create mode 100644 include/hw/cpu/drawer.h

diff --git a/MAINTAINERS b/MAINTAINERS
index dd5adfda64cc..4b373ff46ce3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1865,6 +1865,7 @@ F: hw/core/numa.c
 F: hw/cpu/book.c
 F: hw/cpu/cluster.c
 F: hw/cpu/die.c
+F: hw/cpu/drawer.c
 F: hw/cpu/socket.c
 F: qapi/machine.json
 F: qapi/machine-common.json
@@ -1875,6 +1876,7 @@ F: include/hw/core/cpu-topo.h
 F: include/hw/cpu/book.h
 F: include/hw/cpu/cluster.h
 F: include/hw/cpu/die.h
+F: include/hw/cpu/drawer.h
 F: include/hw/cpu/socket.h
 F: include/sysemu/numa.h
 F: tests/unit/test-smp-parse.c
diff --git a/hw/cpu/drawer.c b/hw/cpu/drawer.c
new file mode 100644
index 000000000000..f1ccfd153284
--- /dev/null
+++ b/hw/cpu/drawer.c
@@ -0,0 +1,46 @@
+/*
+ * CPU drawer abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cpu/drawer.h"
+
+static void cpu_drawer_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
+
+    tc->level = CPU_TOPO_DRAWER;
+}
+
+static const TypeInfo cpu_drawer_type_info = {
+    .name = TYPE_CPU_DRAWER,
+    .parent = TYPE_CPU_TOPO,
+    .class_init = cpu_drawer_class_init,
+    .instance_size = sizeof(CPUDrawer),
+};
+
+static void cpu_drawer_register_types(void)
+{
+    type_register_static(&cpu_drawer_type_info);
+}
+
+type_init(cpu_drawer_register_types)
diff --git a/hw/cpu/meson.build b/hw/cpu/meson.build
index c44b54c5abb0..0dea39364b98 100644
--- a/hw/cpu/meson.build
+++ b/hw/cpu/meson.build
@@ -1,4 +1,4 @@
-system_ss.add(files('core.c', 'cluster.c', 'die.c', 'socket.c', 'book.c'))
+system_ss.add(files('core.c', 'cluster.c', 'die.c', 'socket.c', 'book.c', 'drawer.c'))
 
 system_ss.add(when: 'CONFIG_ARM11MPCORE', if_true: files('arm11mpcore.c'))
 system_ss.add(when: 'CONFIG_REALVIEW', if_true: files('realview_mpcore.c'))
diff --git a/include/hw/cpu/drawer.h b/include/hw/cpu/drawer.h
new file mode 100644
index 000000000000..34ae089d33bf
--- /dev/null
+++ b/include/hw/cpu/drawer.h
@@ -0,0 +1,38 @@
+/*
+ * CPU drawer abstract device
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_CPU_DRAWER_H
+#define HW_CPU_DRAWER_H
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_CPU_DRAWER "cpu-drawer"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CPUDrawer, CPU_DRAWER)
+
+struct CPUDrawer {
+    /*< private >*/
+    CPUTopoState obj;
+
+    /*< public >*/
+};
+
+#endif /* HW_CPU_DRAWER_H */
-- 
2.34.1

[RFC 27/41] hw/core/slot: Introduce CPU slot as the root of CPU topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Abstract the root of topology tree as a special topology device
"cpu-slot".

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                |  2 ++
 hw/core/cpu-slot.c         | 48 ++++++++++++++++++++++++++++++++++++++
 hw/core/meson.build        |  1 +
 include/hw/core/cpu-slot.h | 38 ++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+)
 create mode 100644 hw/core/cpu-slot.c
 create mode 100644 include/hw/core/cpu-slot.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 4b373ff46ce3..ac08b5a8c4e0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1856,6 +1856,7 @@ R: Philippe Mathieu-Daudé <philmd@linaro.org>
 R: Yanan Wang <wangyanan55@huawei.com>
 S: Supported
 F: hw/core/cpu.c
+F: hw/core/cpu-slot.c
 F: hw/core/cpu-topo.c
 F: hw/core/machine-qmp-cmds.c
 F: hw/core/machine.c
@@ -1872,6 +1873,7 @@ F: qapi/machine-common.json
 F: qapi/machine-target.json
 F: include/hw/boards.h
 F: include/hw/core/cpu.h
+F: include/hw/core/cpu-slot.h
 F: include/hw/core/cpu-topo.h
 F: include/hw/cpu/book.h
 F: include/hw/cpu/cluster.h
diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
new file mode 100644
index 000000000000..5aef5b0189c2
--- /dev/null
+++ b/hw/core/cpu-slot.c
@@ -0,0 +1,48 @@
+/*
+ * CPU slot device abstraction
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/core/cpu-slot.h"
+
+static void cpu_slot_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+
+    set_bit(DEVICE_CATEGORY_CPU_DEF, dc->categories);
+    dc->user_creatable = false;
+
+    tc->level = CPU_TOPO_ROOT;
+}
+
+static const TypeInfo cpu_slot_type_info = {
+    .name = TYPE_CPU_SLOT,
+    .parent = TYPE_CPU_TOPO,
+    .class_init = cpu_slot_class_init,
+    .instance_size = sizeof(CPUSlot),
+};
+
+static void cpu_slot_register_types(void)
+{
+    type_register_static(&cpu_slot_type_info);
+}
+
+type_init(cpu_slot_register_types)
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 501d2529697e..3347c054e162 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -23,6 +23,7 @@ else
 endif
 
 common_ss.add(files('cpu-common.c'))
+common_ss.add(files('cpu-slot.c'))
 common_ss.add(files('cpu-topo.c'))
 common_ss.add(files('machine-smp.c'))
 system_ss.add(when: 'CONFIG_FITLOADER', if_true: files('loader-fit.c'))
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
new file mode 100644
index 000000000000..718c8ecaa751
--- /dev/null
+++ b/include/hw/core/cpu-slot.h
@@ -0,0 +1,38 @@
+/*
+ * CPU slot device abstraction
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef CPU_SLOT_H
+#define CPU_SLOT_H
+
+#include "hw/core/cpu-topo.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_CPU_SLOT "cpu-slot"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
+
+struct CPUSlot {
+    /*< private >*/
+    CPUTopoState parent_obj;
+
+    /*< public >*/
+};
+
+#endif /* CPU_SLOT_H */
-- 
2.34.1

[RFC 28/41] hw/core/slot: Maintain the core queue in CPU slot

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Maintain the cores queue at cpu-slot to facilitate direct traversal
of all cores.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 43 ++++++++++++++++++++++++++++++++++++++
 include/hw/core/cpu-slot.h |  9 ++++++++
 include/hw/cpu/core.h      |  2 ++
 3 files changed, 54 insertions(+)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index 5aef5b0189c2..a6b7d98dea18 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -22,6 +22,40 @@
 
 #include "hw/core/cpu-slot.h"
 
+static void cpu_slot_add_topo_info(CPUTopoState *root, CPUTopoState *child)
+{
+    CPUSlot *slot = CPU_SLOT(root);
+    CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+
+    if (level == CPU_TOPO_CORE) {
+        QTAILQ_INSERT_TAIL(&slot->cores, CPU_CORE(child), node);
+    }
+    return;
+}
+
+static void cpu_slot_del_topo_info(CPUTopoState *root, CPUTopoState *child)
+{
+    CPUSlot *slot = CPU_SLOT(root);
+    CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+
+    if (level == CPU_TOPO_CORE) {
+        QTAILQ_REMOVE(&slot->cores, CPU_CORE(child), node);
+    }
+    return;
+}
+
+static void cpu_slot_update_topo_info(CPUTopoState *root, CPUTopoState *child,
+                                      bool is_realize)
+{
+    g_assert(child->parent);
+
+    if (is_realize) {
+        cpu_slot_add_topo_info(root, child);
+    } else {
+        cpu_slot_del_topo_info(root, child);
+    }
+}
+
 static void cpu_slot_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
@@ -31,12 +65,21 @@ static void cpu_slot_class_init(ObjectClass *oc, void *data)
     dc->user_creatable = false;
 
     tc->level = CPU_TOPO_ROOT;
+    tc->update_topo_info = cpu_slot_update_topo_info;
+}
+
+static void cpu_slot_instance_init(Object *obj)
+{
+    CPUSlot *slot = CPU_SLOT(obj);
+
+    QTAILQ_INIT(&slot->cores);
 }
 
 static const TypeInfo cpu_slot_type_info = {
     .name = TYPE_CPU_SLOT,
     .parent = TYPE_CPU_TOPO,
     .class_init = cpu_slot_class_init,
+    .instance_init = cpu_slot_instance_init,
     .instance_size = sizeof(CPUSlot),
 };
 
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index 718c8ecaa751..d2a1160562be 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -22,17 +22,26 @@
 #define CPU_SLOT_H
 
 #include "hw/core/cpu-topo.h"
+#include "hw/cpu/core.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_CPU_SLOT "cpu-slot"
 
 OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
 
+/**
+ * CPUSlot:
+ * @cores: Queue consisting of all the cores in the topology tree
+ *     where the cpu-slot is the root. cpu-slot can maintain similar
+ *     queues for other topology levels to facilitate traversal
+ *     when necessary.
+ */
 struct CPUSlot {
     /*< private >*/
     CPUTopoState parent_obj;
 
     /*< public >*/
+    QTAILQ_HEAD(, CPUCore) cores;
 };
 
 #endif /* CPU_SLOT_H */
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
index 591240861efb..65dc10931190 100644
--- a/include/hw/cpu/core.h
+++ b/include/hw/cpu/core.h
@@ -40,6 +40,8 @@ struct CPUCore {
      * "-device"/"device_add"?
      */
     int plugged_threads;
+
+    QTAILQ_ENTRY(CPUCore) node;
 };
 
 #endif
-- 
2.34.1

[RFC 29/41] hw/core/slot: Statistics topology information in CPU slot

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The CPU slot, as the root of the topology tree, is responsible for
global topology information collection and updates.

When a new topology device is added to/deleted from the topology tree,
update the corresponding information in the slot.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 41 +++++++++++++++++++++++++++++++++++-
 include/hw/core/cpu-slot.h | 43 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 83 insertions(+), 1 deletion(-)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index a6b7d98dea18..e8e6f4d25532 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -22,14 +22,44 @@
 
 #include "hw/core/cpu-slot.h"
 
+static inline
+CPUTopoStatEntry *get_topo_stat_entry(CPUTopoStat *stat,
+                                      CPUTopoLevel level)
+{
+    assert(level != CPU_TOPO_UNKNOWN);
+
+    return &stat->entries[TOPO_STAT_ENTRY_IDX(level)];
+}
+
 static void cpu_slot_add_topo_info(CPUTopoState *root, CPUTopoState *child)
 {
     CPUSlot *slot = CPU_SLOT(root);
     CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+    CPUTopoStatEntry *entry;
 
     if (level == CPU_TOPO_CORE) {
-        QTAILQ_INSERT_TAIL(&slot->cores, CPU_CORE(child), node);
+        CPUCore *core = CPU_CORE(child);
+        CPUTopoStatEntry *thread_entry;
+
+        QTAILQ_INSERT_TAIL(&slot->cores, core, node);
+
+        /* Max CPUs per core is pre-configured by "nr-threads". */
+        slot->stat.max_cpus += core->nr_threads;
+        slot->stat.pre_plugged_cpus += core->plugged_threads;
+
+        thread_entry = get_topo_stat_entry(&slot->stat, CPU_TOPO_THREAD);
+        if (child->max_children > thread_entry->max_units) {
+            thread_entry->max_units = child->max_children;
+        }
     }
+
+    entry = get_topo_stat_entry(&slot->stat, level);
+    entry->total_units++;
+    if (child->parent->num_children > entry->max_units) {
+        entry->max_units = child->parent->num_children;
+    }
+
+    set_bit(level, slot->stat.curr_levels);
     return;
 }
 
@@ -37,10 +67,18 @@ static void cpu_slot_del_topo_info(CPUTopoState *root, CPUTopoState *child)
 {
     CPUSlot *slot = CPU_SLOT(root);
     CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+    CPUTopoStatEntry *entry;
+
+    assert(level != CPU_TOPO_UNKNOWN);
 
     if (level == CPU_TOPO_CORE) {
         QTAILQ_REMOVE(&slot->cores, CPU_CORE(child), node);
     }
+
+    entry = get_topo_stat_entry(&slot->stat, level);
+    entry->total_units--;
+
+    /* No need to update entries[*].max_units and curr_levels. */
     return;
 }
 
@@ -73,6 +111,7 @@ static void cpu_slot_instance_init(Object *obj)
     CPUSlot *slot = CPU_SLOT(obj);
 
     QTAILQ_INIT(&slot->cores);
+    set_bit(CPU_TOPO_ROOT, slot->stat.curr_levels);
 }
 
 static const TypeInfo cpu_slot_type_info = {
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index d2a1160562be..fa2bd4af247d 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -25,6 +25,47 @@
 #include "hw/cpu/core.h"
 #include "hw/qdev-core.h"
 
+/**
+ * @USER_AVAIL_LEVEL_NUM: the number of total topology levels in topology
+ *                        bitmap, which includes CPU_TOPO_UNKNOWN.
+ */
+#define USER_AVAIL_LEVEL_NUM (CPU_TOPO_ROOT + 1)
+
+/**
+ * @VALID_LEVEL_NUM: the number of valid topology levels, which excludes
+ *                   CPU_TOPO_UNKNOWN and CPU_TOPO_ROOT.
+ */
+#define VALID_LEVEL_NUM (CPU_TOPO_ROOT - 1)
+
+#define TOPO_STAT_ENTRY_IDX(level) ((level) - 1)
+
+/**
+ * CPUTopoStatEntry:
+ * @total: Total number of topological units at the same level that are
+ *         currently inserted in CPU slot
+ * @max: Maximum number of topological units at the same level under the
+ *       parent topolofical container
+ */
+typedef struct CPUTopoStatEntry {
+    unsigned int total_units;
+    unsigned int max_units;
+} CPUTopoStatEntry;
+
+/**
+ * CPUTopoStat:
+ * @max_cpus: Maximum number of CPUs in CPU slot.
+ * @pre_plugged_cpus: Number of pre-plugged CPUs in CPU slot.
+ * @entries: Detail count information for valid topology levels under
+ *           CPU slot
+ * @curr_levels: Current CPU topology levels inserted in CPU slot
+ */
+typedef struct CPUTopoStat {
+    unsigned int max_cpus;
+    unsigned int pre_plugged_cpus;
+    CPUTopoStatEntry entries[VALID_LEVEL_NUM];
+    DECLARE_BITMAP(curr_levels, USER_AVAIL_LEVEL_NUM);
+} CPUTopoStat;
+
 #define TYPE_CPU_SLOT "cpu-slot"
 
 OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
@@ -35,6 +76,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
  *     where the cpu-slot is the root. cpu-slot can maintain similar
  *     queues for other topology levels to facilitate traversal
  *     when necessary.
+ * @stat: Statistical topology information for topology tree.
  */
 struct CPUSlot {
     /*< private >*/
@@ -42,6 +84,7 @@ struct CPUSlot {
 
     /*< public >*/
     QTAILQ_HEAD(, CPUCore) cores;
+    CPUTopoStat stat;
 };
 
 #endif /* CPU_SLOT_H */
-- 
2.34.1

[RFC 30/41] hw/core/slot: Check topology child to be added under CPU slot

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Implement CPUTopoClass.check_topo_child() in cpu-slot to be compatible
with the limitations of the current smp topology.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 37 +++++++++++++++++++++++++++++++++++++
 hw/core/cpu-topo.c         |  2 +-
 include/hw/core/cpu-slot.h |  2 ++
 include/hw/core/cpu-topo.h |  1 +
 4 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index e8e6f4d25532..2a796ad5b6e7 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -21,6 +21,7 @@
 #include "qemu/osdep.h"
 
 #include "hw/core/cpu-slot.h"
+#include "qapi/error.h"
 
 static inline
 CPUTopoStatEntry *get_topo_stat_entry(CPUTopoStat *stat,
@@ -94,6 +95,37 @@ static void cpu_slot_update_topo_info(CPUTopoState *root, CPUTopoState *child,
     }
 }
 
+static void cpu_slot_check_topo_support(CPUTopoState *root, CPUTopoState *child,
+                                        Error **errp)
+{
+    CPUSlot *slot = CPU_SLOT(root);
+    CPUTopoLevel child_level = CPU_TOPO_LEVEL(child);
+
+    if (!test_bit(child_level, slot->supported_levels)) {
+        error_setg(errp, "cpu topo: the level %s is not supported",
+                   cpu_topo_level_to_string(child_level));
+        return;
+    }
+
+    /*
+     * Currently we doesn't support hybrid topology. For SMP topology,
+     * each child under the same parent are same type.
+     */
+    if (child->parent->num_children) {
+        CPUTopoState *sibling = QTAILQ_FIRST(&child->parent->children);
+        const char *sibling_type = object_get_typename(OBJECT(sibling));
+        const char *child_type = object_get_typename(OBJECT(child));
+
+        if (strcmp(sibling_type, child_type)) {
+            error_setg(errp, "Invalid smp topology: different CPU "
+                       "topology types (%s child vs %s sibling) "
+                       "under the same parent (%s).",
+                       child_type, sibling_type,
+                       object_get_typename(OBJECT(child->parent)));
+        }
+    }
+}
+
 static void cpu_slot_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
@@ -104,6 +136,7 @@ static void cpu_slot_class_init(ObjectClass *oc, void *data)
 
     tc->level = CPU_TOPO_ROOT;
     tc->update_topo_info = cpu_slot_update_topo_info;
+    tc->check_topo_child = cpu_slot_check_topo_support;
 }
 
 static void cpu_slot_instance_init(Object *obj)
@@ -112,6 +145,10 @@ static void cpu_slot_instance_init(Object *obj)
 
     QTAILQ_INIT(&slot->cores);
     set_bit(CPU_TOPO_ROOT, slot->stat.curr_levels);
+
+    /* Set all levels by default. */
+    bitmap_fill(slot->supported_levels, USER_AVAIL_LEVEL_NUM);
+    clear_bit(CPU_TOPO_UNKNOWN, slot->supported_levels);
 }
 
 static const TypeInfo cpu_slot_type_info = {
diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index 687a4cc566ec..351112ca7a73 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -24,7 +24,7 @@
 #include "hw/qdev-properties.h"
 #include "qapi/error.h"
 
-static const char *cpu_topo_level_to_string(CPUTopoLevel level)
+const char *cpu_topo_level_to_string(CPUTopoLevel level)
 {
     switch (level) {
     case CPU_TOPO_UNKNOWN:
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index fa2bd4af247d..7bf51988afb3 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -77,6 +77,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
  *     queues for other topology levels to facilitate traversal
  *     when necessary.
  * @stat: Statistical topology information for topology tree.
+ * @supported_levels: Supported topology levels for topology tree.
  */
 struct CPUSlot {
     /*< private >*/
@@ -85,6 +86,7 @@ struct CPUSlot {
     /*< public >*/
     QTAILQ_HEAD(, CPUCore) cores;
     CPUTopoStat stat;
+    DECLARE_BITMAP(supported_levels, USER_AVAIL_LEVEL_NUM);
 };
 
 #endif /* CPU_SLOT_H */
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index 453bacbb558b..d27da0335c42 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -102,5 +102,6 @@ int cpu_topo_child_foreach(CPUTopoState *topo, unsigned long *levels,
 int cpu_topo_child_foreach_recursive(CPUTopoState *topo,
                                      unsigned long *levels,
                                      topo_fn fn, void *opaque);
+const char *cpu_topo_level_to_string(CPUTopoLevel level);
 
 #endif /* CPU_TOPO_H */
-- 
2.34.1

[RFC 31/41] hw/machine: Plug cpu-slot into machine to maintain topology tree

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Add a cpu-slot in machine as the root of topology tree to maintain the
QOM topology.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 31 +++++++++++++++++++++++++++++++
 include/hw/boards.h        |  2 ++
 include/hw/core/cpu-slot.h |  7 +++++++
 system/vl.c                |  2 ++
 4 files changed, 42 insertions(+)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index 2a796ad5b6e7..4b148440ed3d 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -20,6 +20,7 @@
 
 #include "qemu/osdep.h"
 
+#include "hw/boards.h"
 #include "hw/core/cpu-slot.h"
 #include "qapi/error.h"
 
@@ -165,3 +166,33 @@ static void cpu_slot_register_types(void)
 }
 
 type_init(cpu_slot_register_types)
+
+void machine_plug_cpu_slot(MachineState *ms)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    ms->topo = CPU_SLOT(qdev_new(TYPE_CPU_SLOT));
+
+    object_property_add_child(container_get(OBJECT(ms), "/peripheral"),
+                              "cpu-slot", OBJECT(ms->topo));
+    DEVICE(ms->topo)->id = g_strdup_printf("%s", "cpu-slot");
+
+    qdev_realize_and_unref(DEVICE(ms->topo), NULL, &error_abort);
+    ms->topo->ms = ms;
+
+    if (!mc->smp_props.clusters_supported) {
+        clear_bit(CPU_TOPO_CLUSTER, ms->topo->supported_levels);
+    }
+
+    if (!mc->smp_props.dies_supported) {
+        clear_bit(CPU_TOPO_DIE, ms->topo->supported_levels);
+    }
+
+    if (!mc->smp_props.books_supported) {
+        clear_bit(CPU_TOPO_BOOK, ms->topo->supported_levels);
+    }
+
+    if (!mc->smp_props.drawers_supported) {
+        clear_bit(CPU_TOPO_DRAWER, ms->topo->supported_levels);
+    }
+}
diff --git a/include/hw/boards.h b/include/hw/boards.h
index da85f86efb91..81a7b04ece86 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -10,6 +10,7 @@
 #include "qemu/module.h"
 #include "qom/object.h"
 #include "hw/core/cpu.h"
+#include "hw/core/cpu-slot.h"
 
 #define TYPE_MACHINE_SUFFIX "-machine"
 
@@ -398,6 +399,7 @@ struct MachineState {
     AccelState *accelerator;
     CPUArchIdList *possible_cpus;
     CpuTopology smp;
+    CPUSlot *topo;
     struct NVDIMMState *nvdimms_state;
     struct NumaState *numa_state;
 };
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index 7bf51988afb3..1361af4ccfc0 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -78,6 +78,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
  *     when necessary.
  * @stat: Statistical topology information for topology tree.
  * @supported_levels: Supported topology levels for topology tree.
+ * @ms: Machine in which this cpu-slot is plugged.
  */
 struct CPUSlot {
     /*< private >*/
@@ -87,6 +88,12 @@ struct CPUSlot {
     QTAILQ_HEAD(, CPUCore) cores;
     CPUTopoStat stat;
     DECLARE_BITMAP(supported_levels, USER_AVAIL_LEVEL_NUM);
+    MachineState *ms;
 };
 
+#define MACHINE_CORE_FOREACH(ms, core) \
+    QTAILQ_FOREACH((core), &(ms)->topo->cores, node)
+
+void machine_plug_cpu_slot(MachineState *ms);
+
 #endif /* CPU_SLOT_H */
diff --git a/system/vl.c b/system/vl.c
index 65add2fb2460..637f708d2265 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -2128,6 +2128,8 @@ static void qemu_create_machine(QDict *qdict)
                                           false, &error_abort);
         qobject_unref(default_opts);
     }
+
+    machine_plug_cpu_slot(current_machine);
 }
 
 static int global_init_func(void *opaque, QemuOpts *opts, Error **errp)
-- 
2.34.1

[RFC 32/41] hw/machine: Build smp topology tree from -smp

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For the architecture supports QOM topology (the field
MachineClass.possible_cpus_qom_granu is set), implement smp QOM topology
tree from MachineState.smp.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 217 +++++++++++++++++++++++++++++++++++++
 hw/core/machine-smp.c      |   9 ++
 hw/cpu/core.c              |   1 -
 include/hw/boards.h        |  11 ++
 include/hw/core/cpu-slot.h |   5 +
 tests/unit/meson.build     |   5 +-
 6 files changed, 246 insertions(+), 2 deletions(-)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index 4b148440ed3d..ade155baf60b 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -22,6 +22,11 @@
 
 #include "hw/boards.h"
 #include "hw/core/cpu-slot.h"
+#include "hw/cpu/book.h"
+#include "hw/cpu/cluster.h"
+#include "hw/cpu/die.h"
+#include "hw/cpu/drawer.h"
+#include "hw/cpu/socket.h"
 #include "qapi/error.h"
 
 static inline
@@ -196,3 +201,215 @@ void machine_plug_cpu_slot(MachineState *ms)
         clear_bit(CPU_TOPO_DRAWER, ms->topo->supported_levels);
     }
 }
+
+static unsigned int *get_smp_info_by_level(CpuTopology *smp_info,
+                                           CPUTopoLevel child_level)
+{
+    switch (child_level) {
+    case CPU_TOPO_THREAD:
+        return &smp_info->threads;
+    case CPU_TOPO_CORE:
+        return &smp_info->cores;
+    case CPU_TOPO_CLUSTER:
+        return &smp_info->clusters;
+    case CPU_TOPO_DIE:
+        return &smp_info->dies;
+    case CPU_TOPO_SOCKET:
+        return &smp_info->sockets;
+    case CPU_TOPO_BOOK:
+        return &smp_info->books;
+    case CPU_TOPO_DRAWER:
+        return &smp_info->drawers;
+    default:
+        /* No need to consider CPU_TOPO_UNKNOWN, and CPU_TOPO_ROOT. */
+        g_assert_not_reached();
+    }
+
+    return NULL;
+}
+
+static const char *get_topo_typename_by_level(CPUTopoLevel level)
+{
+    switch (level) {
+    case CPU_TOPO_CORE:
+        return TYPE_CPU_CORE;
+    case CPU_TOPO_CLUSTER:
+        return TYPE_CPU_CLUSTER;
+    case CPU_TOPO_DIE:
+        return TYPE_CPU_DIE;
+    case CPU_TOPO_SOCKET:
+        return TYPE_CPU_SOCKET;
+    case CPU_TOPO_BOOK:
+        return TYPE_CPU_BOOK;
+    case CPU_TOPO_DRAWER:
+        return TYPE_CPU_DRAWER;
+    default:
+        /*
+         * No need to consider CPU_TOPO_UNKNOWN, CPU_TOPO_THREAD
+         * and CPU_TOPO_ROOT.
+         */
+        g_assert_not_reached();
+    }
+
+    return NULL;
+}
+
+static char *get_topo_global_name(CPUTopoStat *stat,
+                                  CPUTopoLevel level)
+{
+    const char *type = NULL;
+    CPUTopoStatEntry *entry;
+
+    type = cpu_topo_level_to_string(level);
+    entry = get_topo_stat_entry(stat, level);
+    return g_strdup_printf("%s[%d]", type, entry->total_units);
+}
+
+typedef struct SMPBuildCbData {
+    unsigned long *supported_levels;
+    unsigned int plugged_cpus;
+    CpuTopology *smp_info;
+    CPUTopoStat *stat;
+    Error **errp;
+} SMPBuildCbData;
+
+static int smp_core_set_threads(Object *core, unsigned int max_threads,
+                                unsigned int *plugged_cpus, Error **errp)
+{
+    if (!object_property_set_int(core, "nr-threads", max_threads, errp)) {
+        object_unref(core);
+        return TOPO_FOREACH_ERR;
+    }
+
+    if (*plugged_cpus > max_threads) {
+        if (!object_property_set_int(core, "plugged-threads",
+                                     max_threads, errp)) {
+            object_unref(core);
+            return TOPO_FOREACH_ERR;
+        }
+        *plugged_cpus -= max_threads;
+    } else{
+        if (!object_property_set_int(core, "plugged-threads",
+                                     *plugged_cpus, errp)) {
+            object_unref(core);
+            return TOPO_FOREACH_ERR;
+        }
+        *plugged_cpus = 0;
+    }
+
+    return TOPO_FOREACH_CONTINUE;
+}
+
+static int add_smp_topo_child(CPUTopoState *topo, void *opaque)
+{
+    CPUTopoLevel level, child_level;
+    Object *parent = OBJECT(topo);
+    SMPBuildCbData *cb = opaque;
+    unsigned int *nr_children;
+    Error **errp = cb->errp;
+
+    level = CPU_TOPO_LEVEL(topo);
+    child_level = find_last_bit(cb->supported_levels, level);
+
+    /*
+     * child_level equals to level, that means no child needs to create.
+     * This shouldn't happen.
+     */
+    g_assert(child_level != level);
+
+    nr_children = get_smp_info_by_level(cb->smp_info, child_level);
+    topo->max_children = *nr_children;
+
+    for (int i = 0; i < topo->max_children; i++) {
+        ObjectProperty *prop;
+        Object *child;
+        gchar *name;
+
+        child = object_new(get_topo_typename_by_level(child_level));
+        name = get_topo_global_name(cb->stat, child_level);
+
+        prop = object_property_try_add_child(parent, name, child, errp);
+        g_free(name);
+        if (!prop) {
+            return TOPO_FOREACH_ERR;
+        }
+
+        if (child_level == CPU_TOPO_CORE) {
+            int ret = smp_core_set_threads(child, cb->smp_info->threads,
+                                           &cb->plugged_cpus, errp);
+
+            if (ret == TOPO_FOREACH_ERR) {
+                return ret;
+            }
+        }
+
+        qdev_realize(DEVICE(child), NULL, errp);
+        if (*errp) {
+            return TOPO_FOREACH_ERR;
+        }
+    }
+
+    return TOPO_FOREACH_CONTINUE;
+}
+
+void machine_create_smp_topo_tree(MachineState *ms, Error **errp)
+{
+    DECLARE_BITMAP(foreach_bitmap, USER_AVAIL_LEVEL_NUM);
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    CPUSlot *slot = ms->topo;
+    SMPBuildCbData cb;
+
+    if (!slot) {
+        error_setg(errp, "Invalid machine: "
+                   "the cpu-slot of machine is not initialized.");
+        return;
+    }
+
+    if (mc->smp_props.possible_cpus_qom_granu != CPU_TOPO_CORE &&
+        mc->smp_props.possible_cpus_qom_granu != CPU_TOPO_THREAD) {
+        error_setg(errp, "Invalid machine: Only support building "
+                   "qom smp topology with core/thread granularity. "
+                   "Not support %s granularity.",
+                   cpu_topo_level_to_string(
+                       mc->smp_props.possible_cpus_qom_granu));
+        return;
+    }
+
+    cb.supported_levels = slot->supported_levels;
+    cb.plugged_cpus = ms->smp.cpus;
+    cb.smp_info = &ms->smp;
+    cb.stat = &slot->stat;
+    cb.errp = errp;
+
+    if (add_smp_topo_child(CPU_TOPO(slot), &cb) < 0) {
+        return;
+    }
+
+    bitmap_copy(foreach_bitmap, slot->supported_levels, USER_AVAIL_LEVEL_NUM);
+
+    /*
+     * Don't create threads from -smp, and just record threads
+     * number in core.
+     */
+    clear_bit(CPU_TOPO_CORE, foreach_bitmap);
+    clear_bit(CPU_TOPO_THREAD, foreach_bitmap);
+
+    /*
+     * If the core level is inserted by hotplug way, don't create cores
+     * from -smp ethier.
+     */
+    if (mc->smp_props.possible_cpus_qom_granu == CPU_TOPO_CORE) {
+        CPUTopoLevel next_level;
+
+        next_level = find_next_bit(foreach_bitmap, USER_AVAIL_LEVEL_NUM,
+                                   CPU_TOPO_CORE + 1);
+        clear_bit(next_level, foreach_bitmap);
+    }
+
+    cpu_topo_child_foreach_recursive(CPU_TOPO(slot), foreach_bitmap,
+                                     add_smp_topo_child, &cb);
+    if (*errp) {
+        return;
+    }
+    slot->smp_parsed = true;
+}
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index 25019c91ee36..a0d091b23b97 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -19,6 +19,7 @@
 
 #include "qemu/osdep.h"
 #include "hw/boards.h"
+#include "hw/core/cpu-slot.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 
@@ -230,6 +231,14 @@ void machine_parse_smp_config(MachineState *ms,
                    mc->name, mc->max_cpus);
         return;
     }
+
+    /*
+     * TODO: drop this check and convert "smp" to QOM topology tree by
+     * default when all arches support QOM topology.
+     */
+    if (mc->smp_props.possible_cpus_qom_granu) {
+        machine_create_smp_topo_tree(ms, errp);
+    }
 }
 
 unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
diff --git a/hw/cpu/core.c b/hw/cpu/core.c
index 261b15fa8171..07100754e9d1 100644
--- a/hw/cpu/core.c
+++ b/hw/cpu/core.c
@@ -107,7 +107,6 @@ static void cpu_core_class_init(ObjectClass *oc, void *data)
 static const TypeInfo cpu_core_type_info = {
     .name = TYPE_CPU_CORE,
     .parent = TYPE_CPU_TOPO,
-    .abstract = true,
     .class_init = cpu_core_class_init,
     .class_size = sizeof(CPUCoreClass),
     .instance_size = sizeof(CPUCore),
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 81a7b04ece86..88de08d98f9f 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -138,6 +138,11 @@ typedef struct {
  *                 provided SMP configuration
  * @books_supported - whether books are supported by the machine
  * @drawers_supported - whether drawers are supported by the machine
+ * @possible_cpus_qom_granu - the topology granularity for possible_cpus[]
+ *                            based on QOM CPU topology to plug CPUs.
+ *                            Note this flag indicates the support for QOM CPU
+ *                            topology. If QOM CPU topology is not supported,
+ *                            must set this field as CPU_TOPO_UNKNOWN.
  */
 typedef struct {
     bool prefer_sockets;
@@ -146,6 +151,7 @@ typedef struct {
     bool has_clusters;
     bool books_supported;
     bool drawers_supported;
+    CPUTopoLevel possible_cpus_qom_granu;
 } SMPCompatProps;
 
 /**
@@ -187,6 +193,9 @@ typedef struct {
  *    specifies default CPU_TYPE, which will be used for parsing target
  *    specific features and for creating CPUs if CPU name wasn't provided
  *    explicitly at CLI
+ * @default_core_tyep:
+ *    specifies default CORE_TYPE, which will be used to create default core
+ *    topology device for smp topology tree from -smp
  * @minimum_page_bits:
  *    If non-zero, the board promises never to create a CPU with a page size
  *    smaller than this, so QEMU can use a more efficient larger page
@@ -267,6 +276,7 @@ struct MachineClass {
     const char *hw_version;
     ram_addr_t default_ram_size;
     const char *default_cpu_type;
+    const char *default_core_type;
     bool default_kernel_irqchip_split;
     bool option_rom_has_mr;
     bool rom_file_has_mr;
@@ -398,6 +408,7 @@ struct MachineState {
     const char *cpu_type;
     AccelState *accelerator;
     CPUArchIdList *possible_cpus;
+    /* TODO: get rid of "smp" when all arches support QOM topology. */
     CpuTopology smp;
     CPUSlot *topo;
     struct NVDIMMState *nvdimms_state;
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index 1361af4ccfc0..de3d08fcb2ac 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -79,6 +79,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(CPUSlot, CPU_SLOT)
  * @stat: Statistical topology information for topology tree.
  * @supported_levels: Supported topology levels for topology tree.
  * @ms: Machine in which this cpu-slot is plugged.
+ * @smp_parsed: Flag indicates if topology tree is derived from "-smp".
+ *      If not, MachineState.smp needs to be initialized based on
+ *      topology tree.
  */
 struct CPUSlot {
     /*< private >*/
@@ -89,11 +92,13 @@ struct CPUSlot {
     CPUTopoStat stat;
     DECLARE_BITMAP(supported_levels, USER_AVAIL_LEVEL_NUM);
     MachineState *ms;
+    bool smp_parsed;
 };
 
 #define MACHINE_CORE_FOREACH(ms, core) \
     QTAILQ_FOREACH((core), &(ms)->topo->cores, node)
 
 void machine_plug_cpu_slot(MachineState *ms);
+void machine_create_smp_topo_tree(MachineState *ms, Error **errp);
 
 #endif /* CPU_SLOT_H */
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index a05d47109040..5806dc5d813c 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -138,7 +138,10 @@ if have_system
     'test-util-sockets': ['socket-helpers.c'],
     'test-base64': [],
     'test-bufferiszero': [],
-    'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c'],
+    'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c',
+                       meson.project_source_root() / 'hw/core/cpu-slot.c',
+                       meson.project_source_root() / 'hw/core/cpu-topo.c',
+                       hwcore],
     'test-vmstate': [migration, io],
     'test-yank': ['socket-helpers.c', qom, io, chardev]
   }
-- 
2.34.1

[RFC 33/41] hw/machine: Validate smp topology tree without -smp

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

QOM topology allows user to create topology tree from cli without -smp,
in this case, validate the topology tree to meet the smp requirement.

Currently, for compatibility with MachineState.smp, initialize
MachineState.smp from topology tree in this case.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 146 +++++++++++++++++++++++++++++++++++++
 hw/core/machine.c          |  10 +++
 include/hw/core/cpu-slot.h |   1 +
 3 files changed, 157 insertions(+)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index ade155baf60b..45b6aef0750a 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -413,3 +413,149 @@ void machine_create_smp_topo_tree(MachineState *ms, Error **errp)
     }
     slot->smp_parsed = true;
 }
+
+static void set_smp_child_topo_info(CpuTopology *smp_info,
+                                    CPUTopoStat *stat,
+                                    CPUTopoLevel child_level)
+{
+    unsigned int *smp_count;
+    CPUTopoStatEntry *entry;
+
+    smp_count = get_smp_info_by_level(smp_info, child_level);
+    entry = get_topo_stat_entry(stat, child_level);
+    *smp_count = entry->max_units ? entry->max_units : 1;
+
+    return;
+}
+
+typedef struct ValidateCbData {
+    CPUTopoStat *stat;
+    CpuTopology *smp_info;
+    Error **errp;
+} ValidateCbData;
+
+static int validate_topo_children(CPUTopoState *topo, void *opaque)
+{
+    CPUTopoLevel level = CPU_TOPO_LEVEL(topo), next_level;
+    ValidateCbData *cb = opaque;
+    unsigned int max_children;
+    CPUTopoStatEntry *entry;
+    Error **errp = cb->errp;
+
+    if (level != CPU_TOPO_THREAD && !topo->num_children &&
+        !topo->max_children) {
+        error_setg(errp, "Invalid topology: the CPU topology "
+                   "(level: %s, index: %d) isn't completed.",
+                   cpu_topo_level_to_string(level), topo->index);
+        return TOPO_FOREACH_ERR;
+    }
+
+    if (level == CPU_TOPO_UNKNOWN) {
+        error_setg(errp, "Invalid CPU topology: unknown topology level.");
+        return TOPO_FOREACH_ERR;
+    }
+
+    /*
+     * Only CPU_TOPO_THREAD level's child_level could be CPU_TOPO_UNKNOWN,
+     * but machine_validate_cpu_topology() is before CPU creation.
+     */
+    if (topo->child_level == CPU_TOPO_UNKNOWN) {
+        error_setg(errp, "Invalid CPU topology: incomplete topology "
+                   "(level: %s, index: %d), no child?.",
+                   cpu_topo_level_to_string(level), topo->index);
+        return TOPO_FOREACH_ERR;
+    }
+
+    /*
+     * Currently hybrid topology isn't supported, so only SMP topology
+     * is allowed.
+     */
+
+    entry = get_topo_stat_entry(cb->stat, topo->child_level);
+
+    /* Max threads per core is pre-configured by "nr-threads". */
+    max_children = topo->child_level != CPU_TOPO_THREAD ?
+                   topo->num_children : topo->max_children;
+
+    if (entry->max_units != max_children) {
+        error_setg(errp, "Invalid SMP topology: "
+                   "The %s topology is asymmetric.",
+                   cpu_topo_level_to_string(level));
+        return TOPO_FOREACH_ERR;
+    }
+
+    next_level = find_next_bit(cb->stat->curr_levels, USER_AVAIL_LEVEL_NUM,
+                               topo->child_level + 1);
+
+    if (next_level != level) {
+        error_setg(errp, "Invalid smp topology: "
+                   "asymmetric CPU topology depth.");
+        return TOPO_FOREACH_ERR;
+    }
+
+    set_smp_child_topo_info(cb->smp_info, cb->stat, topo->child_level);
+
+    return TOPO_FOREACH_CONTINUE;
+}
+
+/*
+ * Only check the case user configures CPU topology via -device
+ * without -smp. In this case, MachineState.smp also needs to be
+ * initialized based on topology tree.
+ */
+bool machine_validate_cpu_topology(MachineState *ms, Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    CPUTopoState *slot_topo = CPU_TOPO(ms->topo);
+    CPUTopoStat *stat = &ms->topo->stat;
+    CpuTopology *smp_info = &ms->smp;
+    unsigned int total_cpus;
+    ValidateCbData cb;
+
+    if (ms->topo->smp_parsed) {
+        return true;
+    } else if (!slot_topo->num_children) {
+        /*
+         * If there's no -smp nor -device to add topology children,
+         * then create the default topology.
+         */
+        machine_create_smp_topo_tree(ms, errp);
+        if (*errp) {
+            return false;
+        }
+        return true;
+    }
+
+    if (test_bit(CPU_TOPO_CLUSTER, stat->curr_levels)) {
+        mc->smp_props.has_clusters = true;
+    }
+
+    /*
+     * The next cpu_topo_child_foreach_recursive() doesn't cover the
+     * parent topology unit. Update information for root here.
+     */
+    set_smp_child_topo_info(smp_info, stat, slot_topo->child_level);
+
+    cb.stat = stat;
+    cb.smp_info = smp_info;
+    cb.errp = errp;
+
+    cpu_topo_child_foreach_recursive(slot_topo, NULL,
+                                     validate_topo_children, &cb);
+    if (*errp) {
+        return false;
+    }
+
+    ms->smp.cpus = stat->pre_plugged_cpus ?
+                   stat->pre_plugged_cpus : 1;
+    ms->smp.max_cpus = stat->max_cpus ?
+                       stat->max_cpus : 1;
+
+    total_cpus = ms->smp.drawers * ms->smp.books *
+                 ms->smp.sockets * ms->smp.dies *
+                 ms->smp.clusters * ms->smp.cores *
+                 ms->smp.threads;
+    g_assert(total_cpus == ms->smp.max_cpus);
+
+    return true;
+}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 0c1739814124..5fa0c826f35e 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1505,6 +1505,16 @@ void machine_run_board_init(MachineState *machine, const char *mem_path, Error *
                                    "on", false);
     }
 
+    /*
+     * TODO: drop this check and validate topology tree by default
+     * when all arches support QOM topology.
+     */
+    if (machine_class->smp_props.possible_cpus_qom_granu) {
+        if (!machine_validate_cpu_topology(machine, errp)) {
+            return;
+        }
+    }
+
     accel_init_interfaces(ACCEL_GET_CLASS(machine->accelerator));
     machine_class->init(machine);
     phase_advance(PHASE_MACHINE_INITIALIZED);
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index de3d08fcb2ac..9e1c6d6b7ff2 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -100,5 +100,6 @@ struct CPUSlot {
 
 void machine_plug_cpu_slot(MachineState *ms);
 void machine_create_smp_topo_tree(MachineState *ms, Error **errp);
+bool machine_validate_cpu_topology(MachineState *ms, Error **errp);
 
 #endif /* CPU_SLOT_H */
-- 
2.34.1

[RFC 34/41] hw/core/topo: Implement user-child to collect topology device from cli

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Support user-child for topology devices.

This will affect these 2 aspects:
1. For the basic topology device (with DEVICE_CATEGORY_CPU_DEF
   category), user could specify "parent" to build the topology
   relationship from cli. And cpu-slot will collect all topology
   devices.

2. For the hotplug topology devices (ppc-core or CPUs of other arches),
   user-child could help to search the correct topology parent in
   topology tree with the index properties. This is compatible with
   the original behavior of inserting CPU/core from cli. And this
   requires arch to support QOM topology with a few arch-specific
   modifications, before this support, hotplugged CPUs/cores are
   inserted into cpu-slot by default.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-slot.c         | 44 ++++++++++++++++++++++++++++++++++++++
 hw/core/cpu-topo.c         | 38 ++++++++++++++++++++++++++++++++
 include/hw/core/cpu-slot.h |  3 +++
 include/hw/core/cpu-topo.h |  4 ++++
 4 files changed, 89 insertions(+)

diff --git a/hw/core/cpu-slot.c b/hw/core/cpu-slot.c
index 45b6aef0750a..413daa66aaad 100644
--- a/hw/core/cpu-slot.c
+++ b/hw/core/cpu-slot.c
@@ -559,3 +559,47 @@ bool machine_validate_cpu_topology(MachineState *ms, Error **errp)
 
     return true;
 }
+
+Object *cpu_slot_get_free_parent(CPUTopoState *child, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+    CPUSlot *slot = ms->topo;
+
+    if (!slot) {
+        return NULL;
+    }
+
+    /*
+     * For CPUs and cores that support hotplug, the behavior is to specify
+     * some topology sub ids. This requires special handling.
+     */
+    if (level == mc->smp_props.possible_cpus_qom_granu) {
+        CPUTopoClass *child_tc = CPU_TOPO_GET_CLASS(child);
+
+        if (child_tc->search_parent_pre_plug) {
+            return child_tc->search_parent_pre_plug(child,
+                       CPU_TOPO(slot), errp);
+        }
+    }
+
+    /*
+     * For other topology devices, just collect them into CPU slot.
+     * The realize() of CPUTopoClass will check no "parent" option case.
+     */
+    return OBJECT(slot);
+}
+
+char *cpu_slot_name_future_child(CPUTopoState *child)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+    CPUTopoLevel level = CPU_TOPO_LEVEL(child);
+    CPUSlot *slot = ms->topo;
+
+    if (!slot) {
+        return NULL;
+    }
+
+    return get_topo_global_name(&slot->stat, level);
+}
diff --git a/hw/core/cpu-topo.c b/hw/core/cpu-topo.c
index 351112ca7a73..143b0a148b17 100644
--- a/hw/core/cpu-topo.c
+++ b/hw/core/cpu-topo.c
@@ -20,8 +20,10 @@
 
 #include "qemu/osdep.h"
 
+#include "hw/core/cpu-slot.h"
 #include "hw/core/cpu-topo.h"
 #include "hw/qdev-properties.h"
+#include "monitor/user-child.h"
 #include "qapi/error.h"
 
 const char *cpu_topo_level_to_string(CPUTopoLevel level)
@@ -272,10 +274,38 @@ static void cpu_topo_unrealize(DeviceState *dev)
     }
 }
 
+/*
+ * Prefer to insert topology device into topology tree where the CPU
+ * slot is the root.
+ */
+static Object *cpu_topo_get_parent(UserChild *uc, Error **errp)
+{
+    return cpu_slot_get_free_parent(CPU_TOPO(uc), errp);
+}
+
+static char *cpu_topo_get_child_name(UserChild *uc, Object *parent)
+{
+    return cpu_slot_name_future_child(CPU_TOPO(uc));
+}
+
+/* Only check type. Other topology details with be checked at realize(). */
+static bool cpu_topo_check_user_parent(UserChild *uc, Object *parent)
+{
+    CPUTopoState *topo;
+
+    topo = (CPUTopoState *)object_dynamic_cast(parent, TYPE_CPU_TOPO);
+    if (!topo) {
+        return false;
+    }
+
+    return true;
+}
+
 static void cpu_topo_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     CPUTopoClass *tc = CPU_TOPO_CLASS(oc);
+    UserChildClass *ucc = USER_CHILD_CLASS(oc);
 
     /* All topology devices belong to CPU property. */
     set_bit(DEVICE_CATEGORY_CPU, dc->categories);
@@ -290,6 +320,10 @@ static void cpu_topo_class_init(ObjectClass *oc, void *data)
     dc->hotpluggable = false;
 
     tc->level = CPU_TOPO_UNKNOWN;
+
+    ucc->get_parent = cpu_topo_get_parent;
+    ucc->get_child_name = cpu_topo_get_child_name;
+    ucc->check_parent = cpu_topo_check_user_parent;
 }
 
 static void cpu_topo_instance_init(Object *obj)
@@ -310,6 +344,10 @@ static const TypeInfo cpu_topo_type_info = {
     .class_init = cpu_topo_class_init,
     .instance_size = sizeof(CPUTopoState),
     .instance_init = cpu_topo_instance_init,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CHILD },
+        { }
+    }
 };
 
 static void cpu_topo_register_types(void)
diff --git a/include/hw/core/cpu-slot.h b/include/hw/core/cpu-slot.h
index 9e1c6d6b7ff2..1a63f55f52c3 100644
--- a/include/hw/core/cpu-slot.h
+++ b/include/hw/core/cpu-slot.h
@@ -102,4 +102,7 @@ void machine_plug_cpu_slot(MachineState *ms);
 void machine_create_smp_topo_tree(MachineState *ms, Error **errp);
 bool machine_validate_cpu_topology(MachineState *ms, Error **errp);
 
+Object *cpu_slot_get_free_parent(CPUTopoState *child, Error **errp);
+char *cpu_slot_name_future_child(CPUTopoState *child);
+
 #endif /* CPU_SLOT_H */
diff --git a/include/hw/core/cpu-topo.h b/include/hw/core/cpu-topo.h
index d27da0335c42..6cef26cce0b7 100644
--- a/include/hw/core/cpu-topo.h
+++ b/include/hw/core/cpu-topo.h
@@ -48,6 +48,8 @@ OBJECT_DECLARE_TYPE(CPUTopoState, CPUTopoClass, CPU_TOPO)
  *     new child (including direct child and non-direct child) is added.
  * @check_topo_child: Method to check the support for new child (including
  *     direct child and non-direct child) to be added.
+ * @search_parent_pre_plug: Method to search proper topology parent of @child
+ *     from @root.
  */
 struct CPUTopoClass {
     /*< private >*/
@@ -59,6 +61,8 @@ struct CPUTopoClass {
                              bool is_realize);
     void (*check_topo_child)(CPUTopoState *parent, CPUTopoState *child,
                              Error **errp);
+    Object *(*search_parent_pre_plug)(CPUTopoState *child, CPUTopoState *root,
+                                      Error **errp);
 };
 
 /**
-- 
2.34.1

[RFC 35/41] hw/i386: Make x86_cpu_new() private in x86.c

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

x86_cpu_new() is only invoked in x86.c. Declear it as static in x86.c.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c         | 3 ++-
 include/hw/i386/x86.h | 2 --
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index b3d054889bba..d9293846db64 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -95,7 +95,8 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
 }
 
 
-void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
+static void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id,
+                        Error **errp)
 {
     Object *cpu = object_new(MACHINE(x86ms)->cpu_type);
 
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index da19ae15463a..19e9f93fe286 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -97,8 +97,6 @@ OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
 
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *pcms,
                                     unsigned int cpu_index);
-
-void x86_cpu_new(X86MachineState *pcms, int64_t apic_id, Error **errp);
 void x86_cpus_init(X86MachineState *pcms, int default_cpu_version);
 CpuInstanceProperties x86_cpu_index_to_props(MachineState *ms,
                                              unsigned cpu_index);
-- 
2.34.1

[RFC 36/41] hw/i386: Allow x86_cpu_new() to specify parent for new CPU

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For QOM topology, CPU should be inserted under its parent core.

Extend x86_cpu_new() to allow caller to specify topology parent.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index d9293846db64..3c99f4c3ab51 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -94,15 +94,22 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
     return x86_apicid_from_cpu_idx(&topo_info, cpu_index);
 }
 
-
 static void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id,
-                        Error **errp)
+                        Object *parent, int index, Error **errp)
 {
-    Object *cpu = object_new(MACHINE(x86ms)->cpu_type);
+    const char *cpu_type = MACHINE(x86ms)->cpu_type;
+    Object *cpu = object_new(cpu_type);
 
     if (!object_property_set_uint(cpu, "apic-id", apic_id, errp)) {
         goto out;
     }
+
+    if (parent) {
+        char *name = g_strdup_printf("%s[%d]", cpu_type, index);
+        object_property_add_child(parent, name, cpu);
+        g_free(name);
+    }
+
     qdev_realize(DEVICE(cpu), NULL, errp);
 
 out:
@@ -146,7 +153,8 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
 
     possible_cpus = mc->possible_cpu_arch_ids(ms);
     for (i = 0; i < ms->smp.cpus; i++) {
-        x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id, &error_fatal);
+        x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id,
+                    NULL, i, &error_fatal);
     }
 }
 
-- 
2.34.1

[RFC 37/41] hw/i386: Allow i386 to create new CPUs from QOM topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For QOM topology, maximum number of CPUs and the number of plugged CPUs
are configured in core level.

Iterate through all the cpu-cores to determine how many CPUs should be
created in each cpu-core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 3c99f4c3ab51..febffed92a83 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -152,9 +152,35 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
     }
 
     possible_cpus = mc->possible_cpu_arch_ids(ms);
-    for (i = 0; i < ms->smp.cpus; i++) {
-        x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id,
-                    NULL, i, &error_fatal);
+
+    /*
+     * possible_cpus_qom_granu means the QOM topology support.
+     *
+     * TODO: Drop the "!mc->smp_props.possible_cpus_qom_granu" case when
+     * i386 completes QOM topology support.
+     */
+    if (mc->smp_props.possible_cpus_qom_granu) {
+        CPUCore *core;
+        int cpu_index = 0;
+        int core_idx = 0;
+
+        MACHINE_CORE_FOREACH(ms, core) {
+            for (i = 0; i < core->plugged_threads; i++) {
+                x86_cpu_new(x86ms, possible_cpus->cpus[cpu_index].arch_id,
+                            OBJECT(core), cpu_index, &error_fatal);
+                cpu_index++;
+            }
+
+            if (core->plugged_threads < core->nr_threads) {
+                cpu_index += core->nr_threads - core->plugged_threads;
+            }
+            core_idx++;
+        }
+    } else {
+        for (i = 0; i < ms->smp.cpus; i++) {
+            x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id,
+                        NULL, i, &error_fatal);
+        }
     }
 }
 
-- 
2.34.1

[RFC 38/41] hw/i386: Wrap apic id and topology sub ids assigning as helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For QOM topology, these 2 helpers are needed for hotplugged CPU to
verify its topology sub indexes and then search its parent core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 173 ++++++++++++++++++++++++++++----------------------
 1 file changed, 96 insertions(+), 77 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index febffed92a83..04edd6de6aeb 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -306,6 +306,98 @@ void x86_cpu_unplug_cb(HotplugHandler *hotplug_dev,
     error_propagate(errp, local_err);
 }
 
+static void x86_cpu_assign_apic_id(MachineState *ms, X86CPU *cpu,
+                                   X86CPUTopoIDs *topo_ids,
+                                   X86CPUTopoInfo *topo_info,
+                                   Error **errp)
+{
+    int max_socket = (ms->smp.max_cpus - 1) /
+                     ms->smp.threads / ms->smp.cores / ms->smp.dies;
+
+    /*
+     * die-id was optional in QEMU 4.0 and older, so keep it optional
+     * if there's only one die per socket.
+     */
+    if (cpu->die_id < 0 && ms->smp.dies == 1) {
+        cpu->die_id = 0;
+    }
+
+    if (cpu->socket_id < 0) {
+        error_setg(errp, "CPU socket-id is not set");
+        return;
+    } else if (cpu->socket_id > max_socket) {
+        error_setg(errp, "Invalid CPU socket-id: %u must be in range 0:%u",
+                   cpu->socket_id, max_socket);
+        return;
+    }
+    if (cpu->die_id < 0) {
+        error_setg(errp, "CPU die-id is not set");
+        return;
+    } else if (cpu->die_id > ms->smp.dies - 1) {
+        error_setg(errp, "Invalid CPU die-id: %u must be in range 0:%u",
+                   cpu->die_id, ms->smp.dies - 1);
+        return;
+    }
+    if (cpu->core_id < 0) {
+        error_setg(errp, "CPU core-id is not set");
+        return;
+    } else if (cpu->core_id > (ms->smp.cores - 1)) {
+        error_setg(errp, "Invalid CPU core-id: %u must be in range 0:%u",
+                   cpu->core_id, ms->smp.cores - 1);
+        return;
+    }
+    if (cpu->thread_id < 0) {
+        error_setg(errp, "CPU thread-id is not set");
+        return;
+    } else if (cpu->thread_id > (ms->smp.threads - 1)) {
+        error_setg(errp, "Invalid CPU thread-id: %u must be in range 0:%u",
+                   cpu->thread_id, ms->smp.threads - 1);
+        return;
+    }
+
+    topo_ids->pkg_id = cpu->socket_id;
+    topo_ids->die_id = cpu->die_id;
+    topo_ids->core_id = cpu->core_id;
+    topo_ids->smt_id = cpu->thread_id;
+    cpu->apic_id = x86_apicid_from_topo_ids(topo_info, topo_ids);
+}
+
+static void x86_cpu_assign_topo_id(X86CPU *cpu,
+                                   X86CPUTopoIDs *topo_ids,
+                                   Error **errp)
+{
+    if (cpu->socket_id != -1 && cpu->socket_id != topo_ids->pkg_id) {
+        error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
+            " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id,
+            topo_ids->pkg_id);
+        return;
+    }
+    cpu->socket_id = topo_ids->pkg_id;
+
+    if (cpu->die_id != -1 && cpu->die_id != topo_ids->die_id) {
+        error_setg(errp, "property die-id: %u doesn't match set apic-id:"
+            " 0x%x (die-id: %u)", cpu->die_id, cpu->apic_id, topo_ids->die_id);
+        return;
+    }
+    cpu->die_id = topo_ids->die_id;
+
+    if (cpu->core_id != -1 && cpu->core_id != topo_ids->core_id) {
+        error_setg(errp, "property core-id: %u doesn't match set apic-id:"
+            " 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id,
+            topo_ids->core_id);
+        return;
+    }
+    cpu->core_id = topo_ids->core_id;
+
+    if (cpu->thread_id != -1 && cpu->thread_id != topo_ids->smt_id) {
+        error_setg(errp, "property thread-id: %u doesn't match set apic-id:"
+            " 0x%x (thread-id: %u)", cpu->thread_id, cpu->apic_id,
+            topo_ids->smt_id);
+        return;
+    }
+    cpu->thread_id = topo_ids->smt_id;
+}
+
 void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
                       DeviceState *dev, Error **errp)
 {
@@ -317,8 +409,6 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     CPUX86State *env = &cpu->env;
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
-    unsigned int smp_cores = ms->smp.cores;
-    unsigned int smp_threads = ms->smp.threads;
     X86CPUTopoInfo topo_info;
 
     if (!object_dynamic_cast(OBJECT(cpu), ms->cpu_type)) {
@@ -347,55 +437,10 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
      * set it based on socket/die/core/thread properties.
      */
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
-        int max_socket = (ms->smp.max_cpus - 1) /
-                                smp_threads / smp_cores / ms->smp.dies;
-
-        /*
-         * die-id was optional in QEMU 4.0 and older, so keep it optional
-         * if there's only one die per socket.
-         */
-        if (cpu->die_id < 0 && ms->smp.dies == 1) {
-            cpu->die_id = 0;
-        }
-
-        if (cpu->socket_id < 0) {
-            error_setg(errp, "CPU socket-id is not set");
-            return;
-        } else if (cpu->socket_id > max_socket) {
-            error_setg(errp, "Invalid CPU socket-id: %u must be in range 0:%u",
-                       cpu->socket_id, max_socket);
+        x86_cpu_assign_apic_id(ms, cpu, &topo_ids, &topo_info, errp);
+        if (*errp) {
             return;
         }
-        if (cpu->die_id < 0) {
-            error_setg(errp, "CPU die-id is not set");
-            return;
-        } else if (cpu->die_id > ms->smp.dies - 1) {
-            error_setg(errp, "Invalid CPU die-id: %u must be in range 0:%u",
-                       cpu->die_id, ms->smp.dies - 1);
-            return;
-        }
-        if (cpu->core_id < 0) {
-            error_setg(errp, "CPU core-id is not set");
-            return;
-        } else if (cpu->core_id > (smp_cores - 1)) {
-            error_setg(errp, "Invalid CPU core-id: %u must be in range 0:%u",
-                       cpu->core_id, smp_cores - 1);
-            return;
-        }
-        if (cpu->thread_id < 0) {
-            error_setg(errp, "CPU thread-id is not set");
-            return;
-        } else if (cpu->thread_id > (smp_threads - 1)) {
-            error_setg(errp, "Invalid CPU thread-id: %u must be in range 0:%u",
-                       cpu->thread_id, smp_threads - 1);
-            return;
-        }
-
-        topo_ids.pkg_id = cpu->socket_id;
-        topo_ids.die_id = cpu->die_id;
-        topo_ids.core_id = cpu->core_id;
-        topo_ids.smt_id = cpu->thread_id;
-        cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
     }
 
     cpu_slot = x86_find_cpu_slot(MACHINE(x86ms), cpu->apic_id, &idx);
@@ -422,36 +467,10 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
      * once -smp refactoring is complete and there will be CPU private
      * CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
     x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
-    if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
-        error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
-            " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id,
-            topo_ids.pkg_id);
-        return;
-    }
-    cpu->socket_id = topo_ids.pkg_id;
-
-    if (cpu->die_id != -1 && cpu->die_id != topo_ids.die_id) {
-        error_setg(errp, "property die-id: %u doesn't match set apic-id:"
-            " 0x%x (die-id: %u)", cpu->die_id, cpu->apic_id, topo_ids.die_id);
-        return;
-    }
-    cpu->die_id = topo_ids.die_id;
-
-    if (cpu->core_id != -1 && cpu->core_id != topo_ids.core_id) {
-        error_setg(errp, "property core-id: %u doesn't match set apic-id:"
-            " 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id,
-            topo_ids.core_id);
-        return;
-    }
-    cpu->core_id = topo_ids.core_id;
-
-    if (cpu->thread_id != -1 && cpu->thread_id != topo_ids.smt_id) {
-        error_setg(errp, "property thread-id: %u doesn't match set apic-id:"
-            " 0x%x (thread-id: %u)", cpu->thread_id, cpu->apic_id,
-            topo_ids.smt_id);
+    x86_cpu_assign_topo_id(cpu, &topo_ids, errp);
+    if (*errp) {
         return;
     }
-    cpu->thread_id = topo_ids.smt_id;
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
         kvm_enabled() && !kvm_hv_vpindex_settable()) {
-- 
2.34.1

[RFC 39/41] hw/i386: Add the interface to search parent for QOM topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

QOM topology needs to search parent cpu-core for hotplugged CPU to
create topology child<> property in qdev_set_id().

This process is before x86_cpu_pre_plug(), thus place 2 helpers
x86_cpu_assign_apic_id() and x86_cpu_assign_topo_id() in
x86_cpu_search_parent_pre_plug() to help get the correct topology sub
indexes. Then x86_cpu_search_parent_pre_plug() searches the parent
cpu-core with these sub indexes.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c         | 128 +++++++++++++++++++++++++++++++++++++++---
 include/hw/i386/x86.h |   3 +
 2 files changed, 122 insertions(+), 9 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 04edd6de6aeb..595d4365fdd1 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -460,16 +460,18 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         return;
     }
 
-    /* if 'address' properties socket-id/core-id/thread-id are not set, set them
-     * so that machine_query_hotpluggable_cpus would show correct values
+    /*
+     * possible_cpus_qom_granu means the QOM topology support.
+     *
+     * TODO: Drop the "!mc->smp_props.possible_cpus_qom_granu" case when
+     * i386 completes QOM topology support.
      */
-    /* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
-     * once -smp refactoring is complete and there will be CPU private
-     * CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
-    x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
-    x86_cpu_assign_topo_id(cpu, &topo_ids, errp);
-    if (*errp) {
-        return;
+    if (!mc->smp_props.possible_cpus_qom_granu) {
+        x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+        x86_cpu_assign_topo_id(cpu, &topo_ids, errp);
+        if (*errp) {
+            return;
+        }
     }
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
@@ -484,6 +486,114 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     numa_cpu_pre_plug(cpu_slot, dev, errp);
 }
 
+static int x86_cpu_get_topo_id_by_level(X86CPU *cpu,
+                                        CPUTopoLevel level)
+{
+    switch (level) {
+    case CPU_TOPO_THREAD:
+        return cpu->thread_id;
+    case CPU_TOPO_CORE:
+        return cpu->core_id;
+    case CPU_TOPO_DIE:
+        return cpu->die_id;
+    case CPU_TOPO_SOCKET:
+        return cpu->socket_id;
+    default:
+        g_assert_not_reached();
+    }
+
+    return -1;
+}
+
+typedef struct SearchCoreCb {
+    X86CPU *cpu;
+    CPUTopoState *parent;
+    int id;
+} SearchCoreCb;
+
+static int x86_cpu_search_parent_core(CPUTopoState *topo,
+                                      void *opaque)
+{
+    SearchCoreCb *cb = opaque;
+    CPUTopoLevel level = CPU_TOPO_LEVEL(topo);
+
+    cb->parent = topo;
+    cb->id = x86_cpu_get_topo_id_by_level(cb->cpu, level);
+
+    if (cb->id == topo->index) {
+        if (level == CPU_TOPO_CORE) {
+            return TOPO_FOREACH_END;
+        }
+        return TOPO_FOREACH_CONTINUE;
+    }
+    return TOPO_FOREACH_SIBLING;
+}
+
+Object *x86_cpu_search_parent_pre_plug(CPUTopoState *topo,
+                                       CPUTopoState *root,
+                                       Error **errp)
+{
+    int ret;
+    SearchCoreCb cb;
+    X86CPUTopoIDs topo_ids;
+    X86CPUTopoInfo topo_info;
+    X86CPU *cpu = X86_CPU(topo);
+    CPUSlot *slot = CPU_SLOT(root);
+    MachineState *ms = slot->ms;
+    DECLARE_BITMAP(foreach_bitmap, USER_AVAIL_LEVEL_NUM);
+
+    topo_info.dies_per_pkg = ms->smp.dies;
+    topo_info.cores_per_die = ms->smp.cores;
+    topo_info.threads_per_core = ms->smp.threads;
+
+    if (cpu->apic_id == UNASSIGNED_APIC_ID) {
+        x86_cpu_assign_apic_id(ms, cpu, &topo_ids, &topo_info, errp);
+        if (*errp) {
+            return NULL;
+        }
+    } else {
+        /*
+         * if 'address' properties socket-id/core-id/thread-id are not set,
+         * set them so that machine_query_hotpluggable_cpus would show
+         * correct values.
+         *
+         * TODO: move socket_id/core_id/thread_id checks into
+         * x86_cpu_realizefn() once -smp refactoring is complete and there
+         * will be CPU private CPUState::nr_cores and CPUState::nr_threads
+         * fields instead of globals.
+         */
+        x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+    }
+
+    x86_cpu_assign_topo_id(cpu, &topo_ids, errp);
+    if (*errp) {
+        return NULL;
+    }
+
+    cb.cpu = cpu;
+    cb.parent = NULL;
+    cb.id = -1;
+    bitmap_fill(foreach_bitmap, USER_AVAIL_LEVEL_NUM);
+    clear_bit(CPU_TOPO_UNKNOWN, foreach_bitmap);
+    clear_bit(CPU_TOPO_THREAD, foreach_bitmap);
+
+    ret = cpu_topo_child_foreach_recursive(root, foreach_bitmap,
+                                           x86_cpu_search_parent_core, &cb);
+    if (ret != TOPO_FOREACH_END) {
+        g_autofree char *search_info = NULL;
+
+        search_info = !cb.parent ? g_strdup("") :
+            g_strdup_printf(" for %s level with id: %d",
+            cpu_topo_level_to_string(CPU_TOPO_LEVEL(cb.parent)), cb.id);
+        error_setg(errp, "Can't find parent%s", search_info);
+        return NULL;
+    }
+
+    /* Keep the index of CPU topology device the same as the thread_id. */
+    topo->index = cpu->thread_id;
+    return OBJECT(cb.parent);
+}
+
 CpuInstanceProperties
 x86_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
 {
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 19e9f93fe286..e8c9ddc36359 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -104,6 +104,9 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx);
 const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms);
 CPUArchId *x86_find_cpu_slot(MachineState *ms, uint32_t id, int *idx);
 void x86_rtc_set_cpus_count(ISADevice *rtc, uint16_t cpus_count);
+Object *x86_cpu_search_parent_pre_plug(CPUTopoState *topo,
+                                       CPUTopoState *root,
+                                       Error **errp);
 void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
                       DeviceState *dev, Error **errp);
 void x86_cpu_plug(HotplugHandler *hotplug_dev,
-- 
2.34.1

[RFC 40/41] hw/i386: Support QOM topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Set MachineClass.smp_props.possible_cpus_qom_granu and
TopoClass.search_parent_pre_plug for i386.

So far, the i386 topology is based on the QOM topology.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c     | 1 +
 target/i386/cpu.c | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 595d4365fdd1..99f6c502de43 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1565,6 +1565,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
     mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
     mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
+    mc->smp_props.possible_cpus_qom_granu = CPU_TOPO_THREAD;
     x86mc->save_tsc_khz = true;
     x86mc->fwcfg_dma_enabled = true;
     nc->nmi_monitor_handler = x86_nmi;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index cd16cb893daf..1de5726691e1 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -41,6 +41,7 @@
 #include "exec/address-spaces.h"
 #include "hw/boards.h"
 #include "hw/i386/sgx-epc.h"
+#include "hw/i386/x86.h"
 #endif
 
 #include "disas/capstone.h"
@@ -8009,6 +8010,9 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 #if !defined(CONFIG_USER_ONLY)
     object_class_property_add(oc, "crash-information", "GuestPanicInformation",
                               x86_cpu_get_crash_info_qom, NULL, NULL, NULL);
+
+    CPU_TOPO_CLASS(oc)->search_parent_pre_plug =
+        x86_cpu_search_parent_pre_plug;
 #endif
 
     for (w = 0; w < FEATURE_WORDS; w++) {
-- 
2.34.1

[RFC 41/41] hw/i386: Cleanup non-QOM topology support

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

After i386 supports QOM topology, drop original topology logic.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 52 +++++++++++----------------------------------------
 1 file changed, 11 insertions(+), 41 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 99f6c502de43..cba8b806cdb6 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -118,7 +118,8 @@ out:
 
 void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
 {
-    int i;
+    CPUCore *core;
+    int i, cpu_index = 0, core_idx = 0;
     const CPUArchIdList *possible_cpus;
     MachineState *ms = MACHINE(x86ms);
     MachineClass *mc = MACHINE_GET_CLASS(x86ms);
@@ -153,34 +154,17 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
 
     possible_cpus = mc->possible_cpu_arch_ids(ms);
 
-    /*
-     * possible_cpus_qom_granu means the QOM topology support.
-     *
-     * TODO: Drop the "!mc->smp_props.possible_cpus_qom_granu" case when
-     * i386 completes QOM topology support.
-     */
-    if (mc->smp_props.possible_cpus_qom_granu) {
-        CPUCore *core;
-        int cpu_index = 0;
-        int core_idx = 0;
-
-        MACHINE_CORE_FOREACH(ms, core) {
-            for (i = 0; i < core->plugged_threads; i++) {
-                x86_cpu_new(x86ms, possible_cpus->cpus[cpu_index].arch_id,
-                            OBJECT(core), cpu_index, &error_fatal);
-                cpu_index++;
-            }
-
-            if (core->plugged_threads < core->nr_threads) {
-                cpu_index += core->nr_threads - core->plugged_threads;
-            }
-            core_idx++;
+    MACHINE_CORE_FOREACH(ms, core) {
+        for (i = 0; i < core->plugged_threads; i++) {
+            x86_cpu_new(x86ms, possible_cpus->cpus[cpu_index].arch_id,
+                        OBJECT(core), cpu_index, &error_fatal);
+            cpu_index++;
         }
-    } else {
-        for (i = 0; i < ms->smp.cpus; i++) {
-            x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id,
-                        NULL, i, &error_fatal);
+
+        if (core->plugged_threads < core->nr_threads) {
+            cpu_index += core->nr_threads - core->plugged_threads;
         }
+        core_idx++;
     }
 }
 
@@ -460,20 +444,6 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         return;
     }
 
-    /*
-     * possible_cpus_qom_granu means the QOM topology support.
-     *
-     * TODO: Drop the "!mc->smp_props.possible_cpus_qom_granu" case when
-     * i386 completes QOM topology support.
-     */
-    if (!mc->smp_props.possible_cpus_qom_granu) {
-        x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
-        x86_cpu_assign_topo_id(cpu, &topo_ids, errp);
-        if (*errp) {
-            return;
-        }
-    }
-
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
         kvm_enabled() && !kvm_hv_vpindex_settable()) {
         error_setg(errp, "kernel doesn't allow setting HyperV VP_INDEX");
-- 
2.34.1

Re: [RFC 00/41] qom-topo: Abstract Everything about CPU Topology

[Zhao Liu]

Hi maintainers,

Just a ping. Welcome your feedbacks!

We wonder if the current RFC is an appropriate attempt towards the final
hybrid topology.

Thanks,
Zhao

On Thu, Nov 30, 2023 at 10:41:22PM +0800, Zhao Liu wrote:
> Date: Thu, 30 Nov 2023 22:41:22 +0800
> From: Zhao Liu <zhao1.liu@linux.intel.com>
> Subject: [RFC 00/41] qom-topo: Abstract Everything about CPU Topology
> X-Mailer: git-send-email 2.34.1
> 
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Hi list,
> 
> This series is our latest attempt after the previous RFC [1] about
> hybrid topology support, which is based on the commit 4705fc0c8511
> ("Merge tag 'pull-for-8.2-fixes-231123-1' of https://gitlab.com/
> stsquad/qemu into staging") with our previous cleanup (patches link:
> https://lore.kernel.org/all/20231127145611.925817-1-zhao1.liu@linux.intel.com/).
> 
> In the previous RFC, Daniel suggested [2] us to use the modern QOM
> approach to define CPU topology, and based on this way, defining hybrid
> topology through cli is natural. (Thanks Daniel!)
> 
> About why we chose -device other than -object, please see the chapter.3
> "History of QOM Topology".
> 
> In fact, S390x already implements heterogeneity at the CPU level with
> QOM CPUs, i.e., different CPUs have different entitlements [3]. However,
> for more thorough heterogeneity, i.e., heterogeneous cores, clusters,
> dies, caches and even more, we still need to go farther in the QOM
> direction.
> 
> With these background, we propose this series to implement QOM "smp"
> topology in QEMU, and it's also the first step towards the heterogeneous
> topology (including CPU & cache topology) for virtualization case.
> 
> The overall goal is to both use QOM "smp" topology to be compatible with
> current -smp behavior, and to take into account different architectural
> setups/requirements for CPU topology (even including different designs
> for CPU hotplug and possible_cpus[] implementation), and ultimately to
> extend QEMU's ability to define CPU topology via -device even without
> -smp.
> 
> The current remining issue, mainly related to PPC, as it chose to build
> the pssible_cpus[] list at core granularity. Please see chapter.5 "Open
> Questions" for more thoughts on this issue.
> 
> For other architectures that build possible_cpus[] at CPU granularity,
> the transition to QOM topology will be similar to what was done for i386
> in this patchset (please feel free to point out any issues I've missed).
> 
> There's a lot of work, and the devil is in the details.
> 
> 
> Welcome your feedbacks!
> 
> 
> 1. Summary about What We Did?
> =============================
> 
> This series implements the basics of QOM topology and supports QOM
> topology for the i386 as the example:
> 
> * Introduce the general topology device and abstract all CPU topology
>   levels to topology devices:
> 
>     - including "cpu", "cpu-core", "cpu-cluster", "cpu-die",
>       "cpu-socket", "cpu-book", "cpu-drawer", and a special topology
>       root "cpu-slot" to manage topology tree.
> 
> * Allow user to create "smp" CPU topology via "-device", for example:
> 
>     The topology with 8 CPUs:
> 
>     -accel kvm -cpu host \
>     -device cpu-socket,id=sock0 \
>     -device cpu-die,id=die0,parent=sock0 \
>     -device cpu-core,id=core0,parent=die0,nr-threads=2 \
>     -device cpu-core,id=core1,parent=die0,nr-threads=2,plugged-threads=1 \
>     -device cpu-core,id=core2,parent=die0,nr-threads=2,plugged-threads=2 \
>     -device cpu-core,id=core3,parent=die0,nr-threads=2 \
>     -device host-x86_64-cpu,socket-id=0,die-id=0,core-id=1,thread-id=1 \
> 
> * Build a topology tree under machine, for example:
> 
>     One of the above CPUs:
> 
>     {
>         "props": {
>             "core-id": 0,
>             "socket-id": 0,
>             "thread-id": 0
>         },
>         "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/core0/host-x86_64-cpu[0]",
>         "type": "host-x86_64-cpu",
>         "vcpus-count": 1
>     }
> 
> * Convert topology information configured via -smp into a QOM "smp"
>   topology tree (if the arch supports QOM topology).
> 
> 
> 2. What's the Problem?
> ======================
> 
> As computer architectures continue to innovate, the need for
> heterogeneous topology of virtual machine in QEMU is growing.
> 
> On the one hand, there is the need for heterogeneous CPU topology
> support. Heterogeneous CPU topology refers to systems that use more than
> one kind of processor or cores [4], the typical example is intel hybrid
> architecture with 2 core types [5], which will even have heterogeneous
> die topology [6]. And we can see that not only Intel, but also ARM and
> AMD's heterogeneous platforms will have the ability to support
> virtualization.
> 
> On the other hand, there is also a growing need for heterogeneous cache
> topology in QEMU. Not only will Intel's hybrid platforms introduce
> complex heterogeneous cache topology, but more platforms have aslo seen
> the strong demand for heterogeneous cache topology definition (e.g. ARM
> [7]). Although cache topology is strictly speaking a separate topology
> from CPU topology, in terms of the actual processor build process and
> the current QEMU's way to define cache topology (e.g., i386 [8] & ARM
> [9]), cache topology is and should be dependent on CPU topology to be
> defined.
> 
> With this background of increasing interest in heterogeneous computing,
> we need a flexible way (for accel) to define a wide variety of CPU
> topologies in QEMU.
> 
> Obviously, -smp is not enough, so here we propose a way to define CPU
> topology based on -device, in the hope that this will greatly expand the
> flexibility of QEMU's CPU topology definition.
> 
> 
> 3. History of QOM Topology
> ==========================
> 
> The intent of QOM topology is not to stop at QOM CPUs, but to abstract
> more CPU topology levels.
> 
> In fact, it's not a new term.
> 
> Back in 2014, Fan proposed [10] a hierarchical tree based on QOM node,
> QOM sockets, QOM cores and QOM cpu. Andreas also propsed [11] socket/
> core/thread QOM model to create hierarchical topology in place. From
> the discussion at that time, the hierarchy topology tree representation
> was what people (Igor said [12]) wanted. However, this work was not
> continued.
> 
> Then Bharata abstracted [13] the cpu-core to support core granularity
> hotplug in spapr. Until now, the only user of cpu-core is still spapr.
> 
> Cpu-cluster was introduced by Luc [14] to organize CPUs in different
> containers to support GDB for TCG case. Though this abstraction was
> descripted as: "mainly an internal QEMU representation and does not
> necessarily match with the notion of clusters on the real hardware",
> its name and function actually make it easy to correspond to the
> physical cluster/smp cluster (the difference, of course, is that TCG
> cluster collects CPUs directly, while the cluster as a CPU topology
> level collect cores, but this difference is not the impediment gap to
> converting "TCG " cluster to "general" cluster).
> 
> As CPU architectures evolve, QEMU has supported more topology levels
> (clusters, die, book and drawer) for virtualization case, while the
> existing cpu-core and cpu-cluster abstractions become fragmented.
> 
> And now, the need for defining hybrid topology has prompted us to
> rethink the QOM topology.
> 
> Daniel suggested [2] to use -object interfaces to define CPU topology,
> and we absorbed his design idea. But in practice we found that -device
> looked like a better approach to take advantage of the current QOM CPU
> device, cpu-core device and cpu-cluster device.
> 
> 
> 4. Design Overview
> ==================
> 
> 4.1. General Topology Device
> ============================
> 
> We introduce a new topology device as the basic abstraction, then all
> levels of the CPU topology are derived from this general topology device
> type.
> 
> This topology device is the basic unit for building the topology tree.
> Children topology devices are inserted into the queue of the parent, and
> the child<> property is created between the children and their parent.
> 
> As the root of the topology tree, we introduce a special topology device
> "cpu-slot". It is created by the machine at machine's initialization and
> collects the topology devices created by the user from the cli, and thus
> builds the topology tree.
> 
> The cpu-slot is also responsible for statistics on global topology
> information, and whenever there is a new topology child, the cpu-slot as
> root is notified to update topology informantion. In addition, different
> architectures have different requirements for topology (e.g., support
> for different levels), and such limitations/properties are applied to
> cpu-slot, which is checked when adding the new child topology unit in
> topology tree.
> 
> 
> 4.2. Derived QOM Topology Devices
> =================================
> 
> Based on the new general topology device type, we convert CPU, cpu-core
> and cpu-cluster from general devices to topology devices.
> 
> And we also abstract cpu-die, cpu-socket, cpu-book and cpu-drawer as
> topology devices.
> 
> 
> 4.3. New Device Category "DEVICE_CATEGORY_CPU_DEF"
> ==================================================
> 
> The topology devices can be divided into two general categories:
> 
> * One, as the basic topology components, should be created before board
>   initialization, to predefine the basic topology structure for the
>   system, and is used to initialize MachineState.possible_cpus at
>   machine's initialization. This category doesn't support hotplug, such
>   as:
> 
>     cpu-core (non-PPC core), cpu-cluster, cpu-die, cpu-socket, cpu-book
>     and cpu-drawer.
> 
>   Thus, we introduce the new device category "DEVICE_CATEGORY_CPU_DEF"
>   to mark these devices and create them from cli before board
>   initialization.
> 
> * The other are CPU and PPC core, which are the granularity of
>   MachineState.possible_cpus.
> 
>   They're created from MachineState.possible_cpus in place during
>   machine initialization or are plugged into MachineState.possible_cpus
>   through hotplug way.
> 
>   For these devices, they could be created from cli only after board
>   initialization.
> 
> 
> 4.4. User-child Interface to Build Child<> from Cli
> ===================================================
> 
> Topology device is bus-less device, and needs child<> to build topology
> hierarchical relationship like:
> 
> /machine/peripheral/cpu-slot/sock*/die*/core*/cpu*
> 
> Therefore, we introduce a new user-child interface to insert hooks into
> device_add path to get/specify object parent for topology devices.
> 
> If a topology device specify "parent" option in -device, it will be add
> to the corresponding topology parent with child<> property.
> 
> If no "parent" option, the topology device will have the default parent
> "cpu-slot". This ensures cpu-slot could collect all topology units to
> build complete topology tree.
> 
> 
> 5. Open Questions
> =================
> 
> There's a special case, user could define topology via -device without
> -smp (that's the future hybrid topology case!).
> 
> In the design of current QOM topology, the numbers of maximum CPUs and
> pre plugged CPUs could be collected during core devices realize.
> 
> For the (non-PPC) architectures which build possible_cpus[] at CPU
> granularity, the cores will be created before possible_cpus[]
> initialization and then CPU slot could know how many maximum CPUs will
> be supported to fill possible_cpus[].
> 
> But for PPC, the possible_cpus[] is at core granularity and PPC core
> could only be created after possible_cpus[] initialization, so that
> CPU slot cannot know the the numbers of maximum CPUs (PPC cores) and pre
> plugged CPUs (PPC cores). So for PPC, the "-smp" is necessary and cannot
> be omitted.
> 
> For PPC this potential impact, i.e., even though QOM topology is
> supported in PPC, it is not possible to omit -smp to create the topology
> only via -device as for i386, and since PPC does not currently support
> heterogeneous topology, this potential impact might be acceptable?
> 
> 
> 6. Future TODOs
> ===============
> 
> The current QOM topology RFC is only the very first step to introduce
> the most basic QOM support, and it tries to be as compatible as possible
> with existing SMP facilities.
> 
> The ultimate goal is to completely replace the current smp-related
> topology structures with cpu-slot.
> 
> There are many TODOs:
> 
> * Add unit tests.
> * Support QOM topology for all architectures.
> * Get rid of MachineState.smp and MachineClass.smp_props with cpu-slot.
> * Extend QOM topology to hybrid topology.
> * Introduce "-device-set" which is derived from Daniel's "-object-set"
>   idea [2] to create multiple duplicate devices.
> ...
> 
> 
> 7. Patch Summary
> ================
> 
> Patch  1- 3: Create DEVICE_CATEGORY_CPU_DEF devices before board
>              initialization.
> Patch  4- 7: Support child<> creation from cli.
> Ptach  8-12: Introduce general topology device.
> Patch 13-26: Abstract all topology levels to topology devices.
> Patch 27-34: Introduce cpu-slot to manage the CPU topology of machine.
> Patch 35-41: Convert i386's CPU creation & hotplug to be based on QOM
>              topology.
> 
> 
> 8. Reference
> ============
> 
> [1]:  Hybrid topology RFC:
>       https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg03205.html
> [2]:  Daniel's suggestion about QOM topology:
>       https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg03320.html
> [3]:  S390x topology document (by Nina):
>       https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04842.html
> [4]:  Heterogeneous computing:
>       https://en.wikipedia.org/wiki/Heterogeneous_computing
> [5]:  12th gen’s Intel hybrid technology:
>       https://www.intel.com/content/www/us/en/support/articles/000091896/processors.html
> [6]:  Intel Meteor Lake (14th gen) architecture overview:
>       https://www.intel.com/content/www/us/en/content-details/788851/meteor-lake-architecture-overview.html
> [7]:  Need of ARM heterogeneous cache topology (by Yanan):
>       https://mail.gnu.org/archive/html/qemu-devel/2023-02/msg05139.html
> [8]:  Cache topology implementation for i386:
>       https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg08251.html
> [9]:  Cluster for ARM to define shared L2 cache and L3 tag (by Yanan):
>       https://lore.kernel.org/all/20211228092221.21068-1-wangyanan55@huawei.com/
> [10]: [PATCH v1 0/4] prebuild cpu QOM tree /machine/node/socket/core
>       ->link-cpu (by Fan):
>       https://lore.kernel.org/all/cover.1395217538.git.chen.fan.fnst@cn.fujitsu.com/
> [11]: [PATCH RFC 0/4] target-i386: PC socket/core/thread modeling,
>       part 1 (by Andreas):
>       https://lore.kernel.org/all/1427131923-4670-1-git-send-email-afaerber@suse.de/
> [12]: "Now we want to have similar QOM tree for introspection which
>       helps express topology as well" (by Igor):
>       https://lore.kernel.org/all/20150407170734.51faac90@igors-macbook-pro.local/
> [13]: [for-2.7 PATCH v3 06/15] cpu: Abstract CPU core type (by Bharata)
>       https://lore.kernel.org/all/1463024905-28401-7-git-send-email-bharata@linux.vnet.ibm.com/
> [14]: [PATCH v8 01/16] hw/cpu: introduce CPU clusters (by Luc):
>       https://lore.kernel.org/all/20181207090135.7651-2-luc.michel@greensocs.com/
> 
> Thanks and Best Regards,
> Zhao
> 
> ---
> Zhao Liu (41):
>   qdev: Introduce new device category to cover basic topology device
>   qdev: Allow qdev_device_add() to add specific category device
>   system: Create base category devices from cli before board
>     initialization
>   qom/object: Introduce helper to resolve path from non-direct parent
>   qdev: Set device parent and id after setting properties
>   qdev: Introduce user-child interface to collect devices from -device
>   qdev: Introduce parent option in -device
>   hw/core/topo: Introduce CPU topology device abstraction
>   hw/core/topo: Support topology index for topology device
>   hw/core/topo: Add virtual method to update topology info for parent
>   hw/core/topo: Add virtual method to check topology child
>   hw/core/topo: Add helpers to traverse the CPU topology tree
>   hw/core/cpu: Convert CPU from general device to topology device
>   PPC/ppc-core: Offload core-id to PPC specific core abstarction
>   hw/cpu/core: Allow to configure plugged threads for cpu-core
>   PPC/ppc-core: Limit plugged-threads and nr-threads to be equal
>   hw/cpu/core: Convert cpu-core from general device to topology device
>   hw/cpu/cluster: Rename CPUClusterState to CPUCluster
>   hw/cpu/cluster: Wrap TCG related ops and props into CONFIG_TCG
>   hw/cpu/cluster: Descript cluster is not only used for TCG in comment
>   hw/cpu/cluster: Allow cpu-cluster to be created by -device
>   hw/cpu/cluster: Convert cpu-cluster from general device to topology
>     device
>   hw/cpu/die: Abstract cpu-die level as topology device
>   hw/cpu/socket: Abstract cpu-socket level as topology device
>   hw/cpu/book: Abstract cpu-book level as topology device
>   hw/cpu/drawer: Abstract cpu-drawer level as topology device
>   hw/core/slot: Introduce CPU slot as the root of CPU topology
>   hw/core/slot: Maintain the core queue in CPU slot
>   hw/core/slot: Statistics topology information in CPU slot
>   hw/core/slot: Check topology child to be added under CPU slot
>   hw/machine: Plug cpu-slot into machine to maintain topology tree
>   hw/machine: Build smp topology tree from -smp
>   hw/machine: Validate smp topology tree without -smp
>   hw/core/topo: Implement user-child to collect topology device from cli
>   hw/i386: Make x86_cpu_new() private in x86.c
>   hw/i386: Allow x86_cpu_new() to specify parent for new CPU
>   hw/i386: Allow i386 to create new CPUs from QOM topology
>   hw/i386: Wrap apic id and topology sub ids assigning as helpers
>   hw/i386: Add the interface to search parent for QOM topology
>   hw/i386: Support QOM topology
>   hw/i386: Cleanup non-QOM topology support
> 
>  MAINTAINERS                        |  16 +
>  accel/kvm/kvm-all.c                |   4 +-
>  gdbstub/system.c                   |   2 +-
>  hw/core/cpu-common.c               |  25 +-
>  hw/core/cpu-slot.c                 | 605 +++++++++++++++++++++++++++++
>  hw/core/cpu-topo.c                 | 399 +++++++++++++++++++
>  hw/core/machine-smp.c              |   9 +
>  hw/core/machine.c                  |  10 +
>  hw/core/meson.build                |   2 +
>  hw/cpu/book.c                      |  46 +++
>  hw/cpu/cluster.c                   |  50 ++-
>  hw/cpu/core.c                      |  72 ++--
>  hw/cpu/die.c                       |  46 +++
>  hw/cpu/drawer.c                    |  46 +++
>  hw/cpu/meson.build                 |   2 +-
>  hw/cpu/socket.c                    |  46 +++
>  hw/i386/x86.c                      | 319 ++++++++++-----
>  hw/net/virtio-net.c                |   2 +-
>  hw/ppc/meson.build                 |   1 +
>  hw/ppc/pnv.c                       |   6 +-
>  hw/ppc/pnv_core.c                  |  17 +-
>  hw/ppc/ppc_core.c                  | 102 +++++
>  hw/ppc/spapr.c                     |  28 +-
>  hw/ppc/spapr_cpu_core.c            |  19 +-
>  hw/usb/xen-usb.c                   |   3 +-
>  hw/xen/xen-legacy-backend.c        |   2 +-
>  include/hw/arm/armsse.h            |   2 +-
>  include/hw/arm/xlnx-versal.h       |   4 +-
>  include/hw/arm/xlnx-zynqmp.h       |   4 +-
>  include/hw/boards.h                |  13 +
>  include/hw/core/cpu-slot.h         | 108 +++++
>  include/hw/core/cpu-topo.h         | 111 ++++++
>  include/hw/core/cpu.h              |   8 +-
>  include/hw/cpu/book.h              |  38 ++
>  include/hw/cpu/cluster.h           |  51 ++-
>  include/hw/cpu/core.h              |  26 +-
>  include/hw/cpu/die.h               |  38 ++
>  include/hw/cpu/drawer.h            |  38 ++
>  include/hw/cpu/socket.h            |  38 ++
>  include/hw/i386/x86.h              |   5 +-
>  include/hw/ppc/pnv_core.h          |  11 +-
>  include/hw/ppc/ppc_core.h          |  58 +++
>  include/hw/ppc/spapr_cpu_core.h    |  12 +-
>  include/hw/qdev-core.h             |   1 +
>  include/hw/riscv/microchip_pfsoc.h |   4 +-
>  include/hw/riscv/sifive_u.h        |   4 +-
>  include/monitor/qdev.h             |   7 +-
>  include/monitor/user-child.h       |  57 +++
>  include/qom/object.h               |  26 ++
>  qom/object.c                       |  31 ++
>  system/meson.build                 |   1 +
>  system/qdev-monitor.c              | 141 ++++++-
>  system/user-child.c                |  72 ++++
>  system/vl.c                        |  53 ++-
>  target/i386/cpu.c                  |   4 +
>  tests/unit/meson.build             |   5 +-
>  56 files changed, 2607 insertions(+), 243 deletions(-)
>  create mode 100644 hw/core/cpu-slot.c
>  create mode 100644 hw/core/cpu-topo.c
>  create mode 100644 hw/cpu/book.c
>  create mode 100644 hw/cpu/die.c
>  create mode 100644 hw/cpu/drawer.c
>  create mode 100644 hw/cpu/socket.c
>  create mode 100644 hw/ppc/ppc_core.c
>  create mode 100644 include/hw/core/cpu-slot.h
>  create mode 100644 include/hw/core/cpu-topo.h
>  create mode 100644 include/hw/cpu/book.h
>  create mode 100644 include/hw/cpu/die.h
>  create mode 100644 include/hw/cpu/drawer.h
>  create mode 100644 include/hw/cpu/socket.h
>  create mode 100644 include/hw/ppc/ppc_core.h
>  create mode 100644 include/monitor/user-child.h
>  create mode 100644 system/user-child.c
> 
> -- 
> 2.34.1
> 
>