Re: [RFC 00/52] Introduce hybrid CPU topology

["Alex Bennée" <alex.bennee@linaro.org>]

Zhao Liu <zhao1.liu@linux.intel.com> writes:

> From: Zhao Liu <zhao1.liu@intel.com>
>
> Hi list,
>
> This RFC series is to introduce hybrid topology in QEMU, and is based
> on our previous patches about cluster support for x86 [1] ([PATCH RESEND
> 00/18] Support smp.clusters for x86).
>
> Though our cluster support patches [1] has not yet accepted, we are
> still posting the hybrid topology support patches in advance, hoping to
> get discussion and feedbacks.
>
> In this RFC series, we mainly have the following work:
> * Introduce the new cpu topology under heterogeneous architecture, which
>   we called "hybrid" cpu topology.
>       - since the name "hmp" is already used in QEMU and "hybrid" is
>         also a synonym for heterogeneous.
> * Reorganized the relevant topology information in qemu to adapt to the
>   hybrid topology.
> * Introduce a new command "hybrid", which allows user to set a custom
>   hybrid topology.
>       - currently limited to custom cluster and custom core
> * Support hybrid topology in i386 PC machine.
> * Expose basic topology information (topology type and core type) in
>   query_cpus_fast.
>
> We hope that the hybrid topology can be general enough to be compatible
> with the needs of hybrids with various architectures in the future.
>
> Welcome your feedbacks!
>
>
> # 1. Background
>
> Heterogeneous computing refers to systems that use more than one kind of
> processor or cores [2]. Now there are more and more heterogeneous
> computing architectures, typically including arm's big.LITTLE [3] and
> intel hybrid architecture [4].
>
> The development of heterogeneous computing brings new challenges to
> virtualization. The first problem we face is how to support the vCPU
> topology under the heterogeneous architecture.
>
> Here, to distinguish it from "smp" cpu topology, we call it "hybrid" cpu
> topology.
>
>
> # 2. Motivation
>
> In a typical hybrid system, different types of core may have different
> capabilities. This difference may be reflected in different capacities,
> performance or power efficiencies, and even different instruction-per-
> cycle (IPC) capabilities. The direct consequence of these differences is
> to affect the performance of the workload.

This elides a use case for heterogeneous computing under emulation where
different cores can have totally different ISA's. We already support
emulation models where we have varying classes of the same ISA, for
example xlnx-zyncmp supports 4 A profile and 2 R profile Arm chips.

I've glanced over the patch set and I don't think there is any direct
clash with that as this is mainly about representing the topology for
x86 non-emulation accelerators but I just want ensure we don't end up
stepping on each others toes.

> For a virtual machines, vCPUs are just threads, although we can bind
> vCPUs to different cores to benefit from the different advantages of
> these physical cores, it's obviously not enough since CPU topology (and
> cache topology) will still have a significant impact on scheduling
> performance.
>
<snip>
> ## 3.3. "-hybrid" command
>
> For hybrid cpu topology configuration, the original "-smp" lack of
> flexibility to expand, and unables to customize different cores.
>
> So we introduce "-hybrid" command and implement it as the multi-
> line command. The multi-line command format is more complex than the
> single-line one, but it can bring stronger scalability and
> intuitiveness. In the future, it can also be easily extended to more
> heterogeneous topology levels.
>
> "-hybrid" command is as follows:
>
> -hybrid socket,sockets=n
> -hybrid die,dies=n
> -hybrid cluster,clusters=n
> -hybrid core,cores=n,type=core_type[,threads=threads]
>         [,clusterid=cluster]
>
> Here, we first define the corresponding qapi options for these 4
> topology levels: core, cluster, die and socket.
>
> We doesn't need a thread level since thread doesn't have different
> type.
>
> For example:
>
> -hybrid socket,sockets=1
> -hybrid die,dies=1
> -hybrid cluster,clusters=4
> -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
> -hybrid core,cores=4,coretype="atom",threads=1
>
> Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
> socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
> core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
> cores with 2 threads.
>
> Please note this example is not an actual machine topology, but it shows
> the powerful flexibility of "hybrid" command.

Is there any intention to apply this command to emulation or will it
only be for hardware virtualisation? For example the x86 translator
should be capable of emulating two different classes of x86 with
different subsets of instructions. It requires ensuring each CPU is
assigned to the correct TYPE_CPU_CLUSTER so we don't re-use TBs from
the "wrong" CPU.

<snip>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro