* [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-04 7:03 ` Cédric Le Goater
2023-04-18 8:53 ` Nina Schoetterl-Glausch
2023-04-03 16:28 ` [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
` (19 subsequent siblings)
20 siblings, 2 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
S390 adds two new SMP levels, drawers and books to the CPU
topology.
The S390 CPU have specific topology features like dedication
and entitlement to give to the guest indications on the host
vCPUs scheduling and help the guest take the best decisions
on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 5 ++++
qapi/machine-common.json | 22 ++++++++++++++
qapi/machine-target.json | 12 ++++++++
qapi/machine.json | 17 +++++++++--
include/hw/boards.h | 10 ++++++-
include/hw/s390x/cpu-topology.h | 15 ++++++++++
target/s390x/cpu.h | 5 ++++
hw/core/machine-smp.c | 53 ++++++++++++++++++++++++++++-----
hw/core/machine.c | 4 +++
hw/s390x/s390-virtio-ccw.c | 2 ++
softmmu/vl.c | 6 ++++
target/s390x/cpu.c | 7 +++++
qapi/meson.build | 1 +
qemu-options.hx | 7 +++--
14 files changed, 152 insertions(+), 14 deletions(-)
create mode 100644 qapi/machine-common.json
create mode 100644 include/hw/s390x/cpu-topology.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 5340de0515..9b1f80739e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1654,6 +1654,11 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s390x@nongnu.org
+S390 CPU topology
+M: Pierre Morel <pmorel@linux.ibm.com>
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+
X86 Machines
------------
PC
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 0000000000..73ea38d976
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,22 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of cpu entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.1
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
+
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 2e267fa458..42a6a40333 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -342,3 +342,15 @@
'TARGET_S390X',
'TARGET_MIPS',
'TARGET_LOONGARCH64' ] } }
+
+##
+# @CpuS390Polarization:
+#
+# An enumeration of cpu polarization that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.1
+##
+{ 'enum': 'CpuS390Polarization',
+ 'prefix': 'S390_CPU_POLARIZATION',
+ 'data': [ 'horizontal', 'vertical' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index 604b686e59..1cdd83f3fd 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -70,7 +71,7 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
+# @props: properties describing to which node/drawer/book/socket/core/thread
# virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
@@ -902,13 +903,15 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to (since 8.1)
+# @book-id: book number within drawer/node/board the CPU belongs to (since 8.1)
+# @socket-id: socket number within book/node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @cluster-id: cluster number within die the CPU belongs to (since 7.1)
# @core-id: core number within cluster the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present
+# Note: currently there are 8 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -918,6 +921,8 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*drawer-id': 'int',
+ '*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1467,6 +1472,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.1)
+#
+# @books: number of books in the CPU topology (since 8.1)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1483,6 +1492,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6fbbfd56c8..9ef0bb76cf 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -131,12 +131,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool has_clusters;
+ bool books_supported;
+ bool drawers_supported;
} SMPCompatProps;
/**
@@ -301,7 +305,9 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
- * @sockets: the number of sockets on the machine
+ * @drawers: the number of drawers on the machine
+ * @books: the number of books in one drawer
+ * @sockets: the number of sockets in one book
* @dies: the number of dies in one socket
* @clusters: the number of clusters in one die
* @cores: the number of cores in one cluster
@@ -310,6 +316,8 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int drawers;
+ unsigned int books;
unsigned int sockets;
unsigned int dies;
unsigned int clusters;
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 0000000000..83f31604cc
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,15 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 7d6d01325b..f2b2a38fe7 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -131,6 +131,11 @@ struct CPUArchState {
#if !defined(CONFIG_USER_ONLY)
uint32_t core_id; /* PoP "CPU address", same as cpu_index */
+ int32_t socket_id;
+ int32_t book_id;
+ int32_t drawer_id;
+ bool dedicated;
+ uint8_t entitlement; /* Used only for vertical polarization */
uint64_t cpuid;
#endif
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index c3dab007da..6f5622a024 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -30,8 +30,19 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ const char *multiply = " * ", *prefix = "";
- g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
+ if (mc->smp_props.drawers_supported) {
+ g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
+ prefix = multiply;
+ }
+
+ if (mc->smp_props.books_supported) {
+ g_string_append_printf(s, "%sbooks (%u)", prefix, ms->smp.books);
+ prefix = multiply;
+ }
+
+ g_string_append_printf(s, "%ssockets (%u)", prefix, ms->smp.sockets);
if (mc->smp_props.dies_supported) {
g_string_append_printf(s, " * dies (%u)", ms->smp.dies);
@@ -73,6 +84,8 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned drawers = config->has_drawers ? config->drawers : 0;
+ unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
unsigned clusters = config->has_clusters ? config->clusters : 0;
@@ -85,6 +98,8 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_drawers && config->drawers == 0) ||
+ (config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
(config->has_clusters && config->clusters == 0) ||
@@ -111,6 +126,19 @@ void machine_parse_smp_config(MachineState *ms,
dies = dies > 0 ? dies : 1;
clusters = clusters > 0 ? clusters : 1;
+ if (!mc->smp_props.books_supported && books > 1) {
+ error_setg(errp, "books not supported by this machine's CPU topology");
+ return;
+ }
+ books = books > 0 ? books : 1;
+
+ if (!mc->smp_props.drawers_supported && drawers > 1) {
+ error_setg(errp,
+ "drawers not supported by this machine's CPU topology");
+ return;
+ }
+ drawers = drawers > 0 ? drawers : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -124,33 +152,41 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (sockets * dies * clusters * cores);
+ threads = maxcpus /
+ (drawers * books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
+ maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
+ clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.drawers = drawers;
+ ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
ms->smp.clusters = clusters;
@@ -161,7 +197,8 @@ void machine_parse_smp_config(MachineState *ms,
mc->smp_props.has_clusters = config->has_clusters;
/* sanity-check of the computed topology */
- if (sockets * dies * clusters * cores * threads != maxcpus) {
+ if (drawers * books * sockets * dies * clusters * cores * threads !=
+ maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 1cf6822e06..41c7ba7027 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -831,6 +831,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_drawers = true, .drawers = ms->smp.drawers,
+ .has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
.has_clusters = true, .clusters = ms->smp.clusters,
@@ -1096,6 +1098,8 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.drawers = 1;
+ ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
ms->smp.clusters = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 503f212a31..1a9bcda8b6 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -736,6 +736,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
mc->no_sdcard = 1;
mc->max_cpus = S390_MAX_CPUS;
mc->has_hotpluggable_cpus = true;
+ mc->smp_props.books_supported = true;
+ mc->smp_props.drawers_supported = true;
assert(!mc->get_hotplug_handler);
mc->get_hotplug_handler = s390_get_hotplug_handler;
mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 3340f63c37..bc293f8100 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -724,6 +724,12 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "drawers",
+ .type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "books",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "sockets",
.type = QEMU_OPT_NUMBER,
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index b10a8541ff..57165fa3a0 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -37,6 +37,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC2000000UL;
@@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
static Property s390x_cpu_properties[] = {
#if !defined(CONFIG_USER_ONLY)
DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
+ DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
+ DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
+ DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
+ DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
+ DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
+ S390_CPU_ENTITLEMENT__MAX),
#endif
DEFINE_PROP_END_OF_LIST()
};
diff --git a/qapi/meson.build b/qapi/meson.build
index fbdb442fdf..b5b01fb7b5 100644
--- a/qapi/meson.build
+++ b/qapi/meson.build
@@ -35,6 +35,7 @@ qapi_all_modules = [
'error',
'introspect',
'job',
+ 'machine-common',
'machine',
'machine-target',
'migration',
diff --git a/qemu-options.hx b/qemu-options.hx
index d42f60fb91..4c79e153fb 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -250,11 +250,14 @@ SRST
ERST
DEF("smp", HAS_ARG, QEMU_OPTION_smp,
- "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
+ "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"
+ " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
" set the number of initial CPUs to 'n' [default=1]\n"
" maxcpus= maximum number of total CPUs, including\n"
" offline CPUs for hotplug, etc\n"
- " sockets= number of sockets on the machine board\n"
+ " drawers= number of drawers on the machine board\n"
+ " books= number of books in one drawer\n"
+ " sockets= number of sockets in one book\n"
" dies= number of dies in one socket\n"
" clusters= number of clusters in one die\n"
" cores= number of cores in one cluster\n"
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-03 16:28 ` [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology Pierre Morel
@ 2023-04-04 7:03 ` Cédric Le Goater
2023-04-04 12:26 ` Pierre Morel
2023-04-18 8:53 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 7:03 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> The S390 CPU have specific topology features like dedication
> and entitlement to give to the guest indications on the host
> vCPUs scheduling and help the guest take the best decisions
> on the scheduling of threads on the vCPUs.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and entitlement,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
Some minor comments below,
> ---
> MAINTAINERS | 5 ++++
> qapi/machine-common.json | 22 ++++++++++++++
> qapi/machine-target.json | 12 ++++++++
> qapi/machine.json | 17 +++++++++--
> include/hw/boards.h | 10 ++++++-
> include/hw/s390x/cpu-topology.h | 15 ++++++++++
> target/s390x/cpu.h | 5 ++++
> hw/core/machine-smp.c | 53 ++++++++++++++++++++++++++++-----
> hw/core/machine.c | 4 +++
> hw/s390x/s390-virtio-ccw.c | 2 ++
> softmmu/vl.c | 6 ++++
> target/s390x/cpu.c | 7 +++++
> qapi/meson.build | 1 +
> qemu-options.hx | 7 +++--
> 14 files changed, 152 insertions(+), 14 deletions(-)
> create mode 100644 qapi/machine-common.json
> create mode 100644 include/hw/s390x/cpu-topology.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 5340de0515..9b1f80739e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1654,6 +1654,11 @@ F: hw/s390x/event-facility.c
> F: hw/s390x/sclp*.c
> L: qemu-s390x@nongnu.org
>
> +S390 CPU topology
> +M: Pierre Morel <pmorel@linux.ibm.com>
> +S: Supported
> +F: include/hw/s390x/cpu-topology.h
> +
> X86 Machines
> ------------
> PC
> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> new file mode 100644
> index 0000000000..73ea38d976
> --- /dev/null
> +++ b/qapi/machine-common.json
> @@ -0,0 +1,22 @@
> +# -*- Mode: Python -*-
> +# vim: filetype=python
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = Machines S390 data types
> +##
> +
> +##
> +# @CpuS390Entitlement:
> +#
> +# An enumeration of cpu entitlements that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.1
> +##
> +{ 'enum': 'CpuS390Entitlement',
> + 'prefix': 'S390_CPU_ENTITLEMENT',
> + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
> +
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index 2e267fa458..42a6a40333 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -342,3 +342,15 @@
> 'TARGET_S390X',
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64' ] } }
> +
> +##
> +# @CpuS390Polarization:
> +#
> +# An enumeration of cpu polarization that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.1
> +##
> +{ 'enum': 'CpuS390Polarization',
> + 'prefix': 'S390_CPU_POLARIZATION',
> + 'data': [ 'horizontal', 'vertical' ] }
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 604b686e59..1cdd83f3fd 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -9,6 +9,7 @@
> ##
>
> { 'include': 'common.json' }
> +{ 'include': 'machine-common.json' }
>
> ##
> # @SysEmuTarget:
> @@ -70,7 +71,7 @@
> #
> # @thread-id: ID of the underlying host thread
> #
> -# @props: properties describing to which node/socket/core/thread
> +# @props: properties describing to which node/drawer/book/socket/core/thread
> # virtual CPU belongs to, provided if supported by board
> #
> # @target: the QEMU system emulation target, which determines which
> @@ -902,13 +903,15 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> -# @socket-id: socket number within node/board the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to (since 8.1)
> +# @book-id: book number within drawer/node/board the CPU belongs to (since 8.1)
> +# @socket-id: socket number within book/node/board the CPU belongs to
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @cluster-id: cluster number within die the CPU belongs to (since 7.1)
> # @core-id: core number within cluster the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present
> +# Note: currently there are 8 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -918,6 +921,8 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> + '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> '*cluster-id': 'int',
> @@ -1467,6 +1472,10 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @drawers: number of drawers in the CPU topology (since 8.1)
> +#
> +# @books: number of books in the CPU topology (since 8.1)
> +#
> # @sockets: number of sockets in the CPU topology
> #
> # @dies: number of dies per socket in the CPU topology
> @@ -1483,6 +1492,8 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*drawers': 'int',
> + '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> '*clusters': 'int',
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 6fbbfd56c8..9ef0bb76cf 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -131,12 +131,16 @@ typedef struct {
> * @clusters_supported - whether clusters are supported by the machine
> * @has_clusters - whether clusters are explicitly specified in the user
> * provided SMP configuration
> + * @books_supported - whether books are supported by the machine
> + * @drawers_supported - whether drawers are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> bool has_clusters;
> + bool books_supported;
> + bool drawers_supported;
> } SMPCompatProps;
>
> /**
> @@ -301,7 +305,9 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> - * @sockets: the number of sockets on the machine
> + * @drawers: the number of drawers on the machine
> + * @books: the number of books in one drawer
> + * @sockets: the number of sockets in one book
> * @dies: the number of dies in one socket
> * @clusters: the number of clusters in one die
> * @cores: the number of cores in one cluster
> @@ -310,6 +316,8 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int drawers;
> + unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> unsigned int clusters;
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> new file mode 100644
> index 0000000000..83f31604cc
> --- /dev/null
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -0,0 +1,15 @@
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022
Shouldn't we have some range : 2022-2023 ?
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
QEMU uses a SPDX tag like the kernel now :
/* SPDX-License-Identifier: GPL-2.0-or-later */
> +#ifndef HW_S390X_CPU_TOPOLOGY_H
> +#define HW_S390X_CPU_TOPOLOGY_H
> +
> +#define S390_TOPOLOGY_CPU_IFL 0x03
This definition is only used in patch 3. May be introduce it then,
it would cleaner.
> +
> +#endif
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index 7d6d01325b..f2b2a38fe7 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -131,6 +131,11 @@ struct CPUArchState {
>
> #if !defined(CONFIG_USER_ONLY)
> uint32_t core_id; /* PoP "CPU address", same as cpu_index */
> + int32_t socket_id;
> + int32_t book_id;
> + int32_t drawer_id;
> + bool dedicated;
> + uint8_t entitlement; /* Used only for vertical polarization */
> uint64_t cpuid;
> #endif
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index c3dab007da..6f5622a024 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -30,8 +30,19 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
> + const char *multiply = " * ", *prefix = "";
>
> - g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
> + prefix = multiply;
indent issue.
> + }
> +
> + if (mc->smp_props.books_supported) {
> + g_string_append_printf(s, "%sbooks (%u)", prefix, ms->smp.books);
> + prefix = multiply;
ditto.
> + }
> +
> + g_string_append_printf(s, "%ssockets (%u)", prefix, ms->smp.sockets);
>
> if (mc->smp_props.dies_supported) {
> g_string_append_printf(s, " * dies (%u)", ms->smp.dies);
> @@ -73,6 +84,8 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> + unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> unsigned clusters = config->has_clusters ? config->clusters : 0;
> @@ -85,6 +98,8 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> + (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> (config->has_clusters && config->clusters == 0) ||
> @@ -111,6 +126,19 @@ void machine_parse_smp_config(MachineState *ms,
> dies = dies > 0 ? dies : 1;
> clusters = clusters > 0 ? clusters : 1;
>
> + if (!mc->smp_props.books_supported && books > 1) {
> + error_setg(errp, "books not supported by this machine's CPU topology");
> + return;
> + }
> + books = books > 0 ? books : 1;
> +
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp,
> + "drawers not supported by this machine's CPU topology");
> + return;
> + }
> + drawers = drawers > 0 ? drawers : 1;
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -124,33 +152,41 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> + clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> + ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> ms->smp.clusters = clusters;
> @@ -161,7 +197,8 @@ void machine_parse_smp_config(MachineState *ms,
> mc->smp_props.has_clusters = config->has_clusters;
>
> /* sanity-check of the computed topology */
> - if (sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads !=
> + maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 1cf6822e06..41c7ba7027 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -831,6 +831,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> + .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> .has_clusters = true, .clusters = ms->smp.clusters,
> @@ -1096,6 +1098,8 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> + ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> ms->smp.clusters = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 503f212a31..1a9bcda8b6 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -736,6 +736,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> mc->no_sdcard = 1;
> mc->max_cpus = S390_MAX_CPUS;
> mc->has_hotpluggable_cpus = true;
> + mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> assert(!mc->get_hotplug_handler);
> mc->get_hotplug_handler = s390_get_hotplug_handler;
> mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 3340f63c37..bc293f8100 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -724,6 +724,12 @@ static QemuOptsList qemu_smp_opts = {
> {
> .name = "cpus",
> .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "drawers",
> + .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "books",
> + .type = QEMU_OPT_NUMBER,
> }, {
> .name = "sockets",
> .type = QEMU_OPT_NUMBER,
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index b10a8541ff..57165fa3a0 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -37,6 +37,7 @@
> #ifndef CONFIG_USER_ONLY
> #include "sysemu/reset.h"
> #endif
> +#include "hw/s390x/cpu-topology.h"
>
> #define CR0_RESET 0xE0UL
> #define CR14_RESET 0xC2000000UL;
> @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
> static Property s390x_cpu_properties[] = {
> #if !defined(CONFIG_USER_ONLY)
> DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
> + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
> + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> + S390_CPU_ENTITLEMENT__MAX),
> #endif
> DEFINE_PROP_END_OF_LIST()
> };
> diff --git a/qapi/meson.build b/qapi/meson.build
> index fbdb442fdf..b5b01fb7b5 100644
> --- a/qapi/meson.build
> +++ b/qapi/meson.build
> @@ -35,6 +35,7 @@ qapi_all_modules = [
> 'error',
> 'introspect',
> 'job',
> + 'machine-common',
> 'machine',
> 'machine-target',
> 'migration',
> diff --git a/qemu-options.hx b/qemu-options.hx
> index d42f60fb91..4c79e153fb 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -250,11 +250,14 @@ SRST
> ERST
>
> DEF("smp", HAS_ARG, QEMU_OPTION_smp,
> - "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
> + "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"
> + " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
> " set the number of initial CPUs to 'n' [default=1]\n"
> " maxcpus= maximum number of total CPUs, including\n"
> " offline CPUs for hotplug, etc\n"
> - " sockets= number of sockets on the machine board\n"
> + " drawers= number of drawers on the machine board\n"
> + " books= number of books in one drawer\n"
> + " sockets= number of sockets in one book\n"
> " dies= number of dies in one socket\n"
> " clusters= number of clusters in one die\n"
> " cores= number of cores in one cluster\n"
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-04 7:03 ` Cédric Le Goater
@ 2023-04-04 12:26 ` Pierre Morel
2023-04-04 12:35 ` Cédric Le Goater
2023-04-18 15:57 ` Daniel P. Berrangé
0 siblings, 2 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 12:26 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 09:03, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> S390 adds two new SMP levels, drawers and books to the CPU
>> topology.
>> The S390 CPU have specific topology features like dedication
>> and entitlement to give to the guest indications on the host
>> vCPUs scheduling and help the guest take the best decisions
>> on the scheduling of threads on the vCPUs.
>>
>> Let us provide the SMP properties with books and drawers levels
>> and S390 CPU with dedication and entitlement,
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>
> Some minor comments below,
>
>> ---
>> MAINTAINERS | 5 ++++
>> qapi/machine-common.json | 22 ++++++++++++++
>> qapi/machine-target.json | 12 ++++++++
>> qapi/machine.json | 17 +++++++++--
>> include/hw/boards.h | 10 ++++++-
>> include/hw/s390x/cpu-topology.h | 15 ++++++++++
>> target/s390x/cpu.h | 5 ++++
>> hw/core/machine-smp.c | 53 ++++++++++++++++++++++++++++-----
>> hw/core/machine.c | 4 +++
>> hw/s390x/s390-virtio-ccw.c | 2 ++
>> softmmu/vl.c | 6 ++++
>> target/s390x/cpu.c | 7 +++++
>> qapi/meson.build | 1 +
>> qemu-options.hx | 7 +++--
>> 14 files changed, 152 insertions(+), 14 deletions(-)
>> create mode 100644 qapi/machine-common.json
>> create mode 100644 include/hw/s390x/cpu-topology.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 5340de0515..9b1f80739e 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1654,6 +1654,11 @@ F: hw/s390x/event-facility.c
>> F: hw/s390x/sclp*.c
>> L: qemu-s390x@nongnu.org
>> +S390 CPU topology
>> +M: Pierre Morel <pmorel@linux.ibm.com>
>> +S: Supported
>> +F: include/hw/s390x/cpu-topology.h
>> +
>> X86 Machines
>> ------------
>> PC
>> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
>> new file mode 100644
>> index 0000000000..73ea38d976
>> --- /dev/null
>> +++ b/qapi/machine-common.json
>> @@ -0,0 +1,22 @@
>> +# -*- Mode: Python -*-
>> +# vim: filetype=python
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> later.
>> +# See the COPYING file in the top-level directory.
>> +
>> +##
>> +# = Machines S390 data types
>> +##
>> +
>> +##
>> +# @CpuS390Entitlement:
>> +#
>> +# An enumeration of cpu entitlements that can be assumed by a virtual
>> +# S390 CPU
>> +#
>> +# Since: 8.1
>> +##
>> +{ 'enum': 'CpuS390Entitlement',
>> + 'prefix': 'S390_CPU_ENTITLEMENT',
>> + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
>> +
>> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
>> index 2e267fa458..42a6a40333 100644
>> --- a/qapi/machine-target.json
>> +++ b/qapi/machine-target.json
>> @@ -342,3 +342,15 @@
>> 'TARGET_S390X',
>> 'TARGET_MIPS',
>> 'TARGET_LOONGARCH64' ] } }
>> +
>> +##
>> +# @CpuS390Polarization:
>> +#
>> +# An enumeration of cpu polarization that can be assumed by a virtual
>> +# S390 CPU
>> +#
>> +# Since: 8.1
>> +##
>> +{ 'enum': 'CpuS390Polarization',
>> + 'prefix': 'S390_CPU_POLARIZATION',
>> + 'data': [ 'horizontal', 'vertical' ] }
>> diff --git a/qapi/machine.json b/qapi/machine.json
>> index 604b686e59..1cdd83f3fd 100644
>> --- a/qapi/machine.json
>> +++ b/qapi/machine.json
>> @@ -9,6 +9,7 @@
>> ##
>> { 'include': 'common.json' }
>> +{ 'include': 'machine-common.json' }
>> ##
>> # @SysEmuTarget:
>> @@ -70,7 +71,7 @@
>> #
>> # @thread-id: ID of the underlying host thread
>> #
>> -# @props: properties describing to which node/socket/core/thread
>> +# @props: properties describing to which
>> node/drawer/book/socket/core/thread
>> # virtual CPU belongs to, provided if supported by board
>> #
>> # @target: the QEMU system emulation target, which determines which
>> @@ -902,13 +903,15 @@
>> # a CPU is being hotplugged.
>> #
>> # @node-id: NUMA node ID the CPU belongs to
>> -# @socket-id: socket number within node/board the CPU belongs to
>> +# @drawer-id: drawer number within node/board the CPU belongs to
>> (since 8.1)
>> +# @book-id: book number within drawer/node/board the CPU belongs to
>> (since 8.1)
>> +# @socket-id: socket number within book/node/board the CPU belongs to
>> # @die-id: die number within socket the CPU belongs to (since 4.1)
>> # @cluster-id: cluster number within die the CPU belongs to (since
>> 7.1)
>> # @core-id: core number within cluster the CPU belongs to
>> # @thread-id: thread number within core the CPU belongs to
>> #
>> -# Note: currently there are 6 properties that could be present
>> +# Note: currently there are 8 properties that could be present
>> # but management should be prepared to pass through other
>> # properties with device_add command to allow for future
>> # interface extension. This also requires the filed names to
>> be kept in
>> @@ -918,6 +921,8 @@
>> ##
>> { 'struct': 'CpuInstanceProperties',
>> 'data': { '*node-id': 'int',
>> + '*drawer-id': 'int',
>> + '*book-id': 'int',
>> '*socket-id': 'int',
>> '*die-id': 'int',
>> '*cluster-id': 'int',
>> @@ -1467,6 +1472,10 @@
>> #
>> # @cpus: number of virtual CPUs in the virtual machine
>> #
>> +# @drawers: number of drawers in the CPU topology (since 8.1)
>> +#
>> +# @books: number of books in the CPU topology (since 8.1)
>> +#
>> # @sockets: number of sockets in the CPU topology
>> #
>> # @dies: number of dies per socket in the CPU topology
>> @@ -1483,6 +1492,8 @@
>> ##
>> { 'struct': 'SMPConfiguration', 'data': {
>> '*cpus': 'int',
>> + '*drawers': 'int',
>> + '*books': 'int',
>> '*sockets': 'int',
>> '*dies': 'int',
>> '*clusters': 'int',
>> diff --git a/include/hw/boards.h b/include/hw/boards.h
>> index 6fbbfd56c8..9ef0bb76cf 100644
>> --- a/include/hw/boards.h
>> +++ b/include/hw/boards.h
>> @@ -131,12 +131,16 @@ typedef struct {
>> * @clusters_supported - whether clusters are supported by the machine
>> * @has_clusters - whether clusters are explicitly specified in the
>> user
>> * provided SMP configuration
>> + * @books_supported - whether books are supported by the machine
>> + * @drawers_supported - whether drawers are supported by the machine
>> */
>> typedef struct {
>> bool prefer_sockets;
>> bool dies_supported;
>> bool clusters_supported;
>> bool has_clusters;
>> + bool books_supported;
>> + bool drawers_supported;
>> } SMPCompatProps;
>> /**
>> @@ -301,7 +305,9 @@ typedef struct DeviceMemoryState {
>> /**
>> * CpuTopology:
>> * @cpus: the number of present logical processors on the machine
>> - * @sockets: the number of sockets on the machine
>> + * @drawers: the number of drawers on the machine
>> + * @books: the number of books in one drawer
>> + * @sockets: the number of sockets in one book
>> * @dies: the number of dies in one socket
>> * @clusters: the number of clusters in one die
>> * @cores: the number of cores in one cluster
>> @@ -310,6 +316,8 @@ typedef struct DeviceMemoryState {
>> */
>> typedef struct CpuTopology {
>> unsigned int cpus;
>> + unsigned int drawers;
>> + unsigned int books;
>> unsigned int sockets;
>> unsigned int dies;
>> unsigned int clusters;
>> diff --git a/include/hw/s390x/cpu-topology.h
>> b/include/hw/s390x/cpu-topology.h
>> new file mode 100644
>> index 0000000000..83f31604cc
>> --- /dev/null
>> +++ b/include/hw/s390x/cpu-topology.h
>> @@ -0,0 +1,15 @@
>> +/*
>> + * CPU Topology
>> + *
>> + * Copyright IBM Corp. 2022
>
> Shouldn't we have some range : 2022-2023 ?
There was a discussion on this in the first spins, I think to remember
that Nina wanted 22 and Thomas 23,
now we have a third opinion :) .
I must say that all three have their reasons and I take what the
majority wants.
A vote?
>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2
>> or (at
>> + * your option) any later version. See the COPYING file in the
>> top-level
>> + * directory.
>> + */
>
> QEMU uses a SPDX tag like the kernel now :
>
> /* SPDX-License-Identifier: GPL-2.0-or-later */
OK, so I will add it on all .c and .h new files
>
>> +#ifndef HW_S390X_CPU_TOPOLOGY_H
>> +#define HW_S390X_CPU_TOPOLOGY_H
>> +
>> +#define S390_TOPOLOGY_CPU_IFL 0x03
>
> This definition is only used in patch 3. May be introduce it then,
> it would cleaner.
yes.
>
>> +
>> +#endif
>> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
>> index 7d6d01325b..f2b2a38fe7 100644
>> --- a/target/s390x/cpu.h
>> +++ b/target/s390x/cpu.h
>> @@ -131,6 +131,11 @@ struct CPUArchState {
>> #if !defined(CONFIG_USER_ONLY)
>> uint32_t core_id; /* PoP "CPU address", same as cpu_index */
>> + int32_t socket_id;
>> + int32_t book_id;
>> + int32_t drawer_id;
>> + bool dedicated;
>> + uint8_t entitlement; /* Used only for vertical
>> polarization */
>> uint64_t cpuid;
>> #endif
>> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
>> index c3dab007da..6f5622a024 100644
>> --- a/hw/core/machine-smp.c
>> +++ b/hw/core/machine-smp.c
>> @@ -30,8 +30,19 @@ static char *cpu_hierarchy_to_string(MachineState
>> *ms)
>> {
>> MachineClass *mc = MACHINE_GET_CLASS(ms);
>> GString *s = g_string_new(NULL);
>> + const char *multiply = " * ", *prefix = "";
>> - g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>> + if (mc->smp_props.drawers_supported) {
>> + g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
>> + prefix = multiply;
>
> indent issue.
right, seems I forgot to update the patch set after the checkpatch.
>
>> + }
>> +
>> + if (mc->smp_props.books_supported) {
>> + g_string_append_printf(s, "%sbooks (%u)", prefix,
>> ms->smp.books);
>> + prefix = multiply;
>
> ditto.
>
>> + }
>> +
[...]
Thanks
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-04 12:26 ` Pierre Morel
@ 2023-04-04 12:35 ` Cédric Le Goater
2023-04-04 14:04 ` Pierre Morel
2023-04-18 15:57 ` Daniel P. Berrangé
1 sibling, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 12:35 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
>>> @@ -0,0 +1,15 @@
>>> +/*
>>> + * CPU Topology
>>> + *
>>> + * Copyright IBM Corp. 2022
>>
>> Shouldn't we have some range : 2022-2023 ?
>
> There was a discussion on this in the first spins, I think to remember that Nina wanted 22 and Thomas 23,
>
> now we have a third opinion :) .
>
> I must say that all three have their reasons and I take what the majority wants.
There is an internal IBM document describing the copyright tags. If I recall
well, first date is the first year the code was officially published, second
year is the last year it was modified (so last commit of the year). Or
something like that and it's theory, because we tend to forget.
For an example, see the OPAL FW https://github.com/open-power/skiboot/,
and run :
"grep Copyright.*IBM" in the OPAL FW
[ ...]
>>> @@ -30,8 +30,19 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
>>> {
>>> MachineClass *mc = MACHINE_GET_CLASS(ms);
>>> GString *s = g_string_new(NULL);
>>> + const char *multiply = " * ", *prefix = "";
>>> - g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>>> + if (mc->smp_props.drawers_supported) {
>>> + g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
>>> + prefix = multiply;
>>
>> indent issue.
>
> right, seems I forgot to update the patch set after the checkpatch.
nope, you didn't. checkpatch doesn't report it. It's not perfect :/
C.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-04 12:35 ` Cédric Le Goater
@ 2023-04-04 14:04 ` Pierre Morel
2023-04-11 12:27 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 14:04 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 14:35, Cédric Le Goater wrote:
>>>> @@ -0,0 +1,15 @@
>>>> +/*
>>>> + * CPU Topology
>>>> + *
>>>> + * Copyright IBM Corp. 2022
>>>
>>> Shouldn't we have some range : 2022-2023 ?
>>
>> There was a discussion on this in the first spins, I think to
>> remember that Nina wanted 22 and Thomas 23,
>>
>> now we have a third opinion :) .
>>
>> I must say that all three have their reasons and I take what the
>> majority wants.
>
> There is an internal IBM document describing the copyright tags. If I
> recall
> well, first date is the first year the code was officially published,
> second
> year is the last year it was modified (so last commit of the year). Or
> something like that and it's theory, because we tend to forget.
>
> For an example, see the OPAL FW https://github.com/open-power/skiboot/,
> and run :
>
> "grep Copyright.*IBM" in the OPAL FW
OK for me, it looks logical, and all three of you are right then.
So I will use
Copyright IBM Corp. 2022-2023
in the next spin if nobody is against.
Thanks,
Pierre
> [ ...]
>
>>>> @@ -30,8 +30,19 @@ static char
>>>> *cpu_hierarchy_to_string(MachineState *ms)
>>>> {
>>>> MachineClass *mc = MACHINE_GET_CLASS(ms);
>>>> GString *s = g_string_new(NULL);
>>>> + const char *multiply = " * ", *prefix = "";
>>>> - g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>>>> + if (mc->smp_props.drawers_supported) {
>>>> + g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
>>>> + prefix = multiply;
>>>
>>> indent issue.
>>
>> right, seems I forgot to update the patch set after the checkpatch.
>
> nope, you didn't. checkpatch doesn't report it. It's not perfect :/
>
> C.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-04 14:04 ` Pierre Morel
@ 2023-04-11 12:27 ` Nina Schoetterl-Glausch
2023-04-17 9:15 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-11 12:27 UTC (permalink / raw)
To: Pierre Morel, Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange
On Tue, 2023-04-04 at 16:04 +0200, Pierre Morel wrote:
> On 4/4/23 14:35, Cédric Le Goater wrote:
> > > > > @@ -0,0 +1,15 @@
> > > > > +/*
> > > > > + * CPU Topology
> > > > > + *
> > > > > + * Copyright IBM Corp. 2022
> > > >
> > > > Shouldn't we have some range : 2022-2023 ?
> > >
> > > There was a discussion on this in the first spins, I think to
> > > remember that Nina wanted 22 and Thomas 23,
> > >
> > > now we have a third opinion :) .
> > >
> > > I must say that all three have their reasons and I take what the
> > > majority wants.
> >
> > There is an internal IBM document describing the copyright tags. If I
> > recall
> > well, first date is the first year the code was officially published,
> > second
> > year is the last year it was modified (so last commit of the year). Or
> > something like that and it's theory, because we tend to forget.
> >
> > For an example, see the OPAL FW https://github.com/open-power/skiboot/,
> > and run :
> >
> > "grep Copyright.*IBM" in the OPAL FW
>
>
> OK for me, it looks logical, and all three of you are right then.
>
> So I will use
>
> Copyright IBM Corp. 2022-2023
You should use a comma instead of a hyphen as per IBM policy.
I.e. 2022, 2023
>
> in the next spin if nobody is against.
>
> Thanks,
>
> Pierre
>
>
> > [ ...]
> >
> > > > > @@ -30,8 +30,19 @@ static char
> > > > > *cpu_hierarchy_to_string(MachineState *ms)
> > > > > {
> > > > > MachineClass *mc = MACHINE_GET_CLASS(ms);
> > > > > GString *s = g_string_new(NULL);
> > > > > + const char *multiply = " * ", *prefix = "";
> > > > > - g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
> > > > > + if (mc->smp_props.drawers_supported) {
> > > > > + g_string_append_printf(s, "drawers (%u)", ms->smp.drawers);
> > > > > + prefix = multiply;
> > > >
> > > > indent issue.
> > >
> > > right, seems I forgot to update the patch set after the checkpatch.
> >
> > nope, you didn't. checkpatch doesn't report it. It's not perfect :/
> >
> > C.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-11 12:27 ` Nina Schoetterl-Glausch
@ 2023-04-17 9:15 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-17 9:15 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange
On 4/11/23 14:27, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-04-04 at 16:04 +0200, Pierre Morel wrote:
>> On 4/4/23 14:35, Cédric Le Goater wrote:
>>>>>> @@ -0,0 +1,15 @@
>>>>>> +/*
>>>>>> + * CPU Topology
>>>>>> + *
>>>>>> + * Copyright IBM Corp. 2022
>>>>> Shouldn't we have some range : 2022-2023 ?
>>>> There was a discussion on this in the first spins, I think to
>>>> remember that Nina wanted 22 and Thomas 23,
>>>>
>>>> now we have a third opinion :) .
>>>>
>>>> I must say that all three have their reasons and I take what the
>>>> majority wants.
>>> There is an internal IBM document describing the copyright tags. If I
>>> recall
>>> well, first date is the first year the code was officially published,
>>> second
>>> year is the last year it was modified (so last commit of the year). Or
>>> something like that and it's theory, because we tend to forget.
>>>
>>> For an example, see the OPAL FW https://github.com/open-power/skiboot/,
>>> and run :
>>>
>>> "grep Copyright.*IBM" in the OPAL FW
>>
>> OK for me, it looks logical, and all three of you are right then.
>>
>> So I will use
>>
>> Copyright IBM Corp. 2022-2023
> You should use a comma instead of a hyphen as per IBM policy.
> I.e. 2022, 2023
OK, thanks.
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-04 12:26 ` Pierre Morel
2023-04-04 12:35 ` Cédric Le Goater
@ 2023-04-18 15:57 ` Daniel P. Berrangé
2023-04-19 9:46 ` Pierre Morel
1 sibling, 1 reply; 57+ messages in thread
From: Daniel P. Berrangé @ 2023-04-18 15:57 UTC (permalink / raw)
To: Pierre Morel
Cc: Cédric Le Goater, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, armbru, seiden, nrb, nsg,
frankja
On Tue, Apr 04, 2023 at 02:26:05PM +0200, Pierre Morel wrote:
>
> On 4/4/23 09:03, Cédric Le Goater wrote:
> > On 4/3/23 18:28, Pierre Morel wrote:
> > > diff --git a/include/hw/s390x/cpu-topology.h
> > > b/include/hw/s390x/cpu-topology.h
> > > new file mode 100644
> > > index 0000000000..83f31604cc
> > > --- /dev/null
> > > +++ b/include/hw/s390x/cpu-topology.h
> > > @@ -0,0 +1,15 @@
> > > +/*
> > > + * CPU Topology
> > > + *
> > > + * Copyright IBM Corp. 2022
> >
> > Shouldn't we have some range : 2022-2023 ?
>
> There was a discussion on this in the first spins, I think to remember that
> Nina wanted 22 and Thomas 23,
>
> now we have a third opinion :) .
>
> I must say that all three have their reasons and I take what the majority
> wants.
>
> A vote?
Whether or not to include a single year, or range of years in
the copyright statement is ultimately a policy decision for the
copyright holder to take (IBM in this case I presume), and not
subject to community vote/preferences.
I will note that some (possibly even many) organizations consider
the year to be largely redundant and devoid of legal benefit, so
are happy with basically any usage of dates (first year, most recent
year, a range of years, or none at all). With this in mind, QEMU is
willing to accept any usage wrt dates in the copyright statement.
It is possible that IBM have a specific policy their employees are
expected to follow. If so, follow that.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 15:57 ` Daniel P. Berrangé
@ 2023-04-19 9:46 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-19 9:46 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Cédric Le Goater, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, armbru, seiden, nrb, nsg,
frankja
On 4/18/23 17:57, Daniel P. Berrangé wrote:
> On Tue, Apr 04, 2023 at 02:26:05PM +0200, Pierre Morel wrote:
>> On 4/4/23 09:03, Cédric Le Goater wrote:
>>> On 4/3/23 18:28, Pierre Morel wrote:
>>>> diff --git a/include/hw/s390x/cpu-topology.h
>>>> b/include/hw/s390x/cpu-topology.h
>>>> new file mode 100644
>>>> index 0000000000..83f31604cc
>>>> --- /dev/null
>>>> +++ b/include/hw/s390x/cpu-topology.h
>>>> @@ -0,0 +1,15 @@
>>>> +/*
>>>> + * CPU Topology
>>>> + *
>>>> + * Copyright IBM Corp. 2022
>>> Shouldn't we have some range : 2022-2023 ?
>> There was a discussion on this in the first spins, I think to remember that
>> Nina wanted 22 and Thomas 23,
>>
>> now we have a third opinion :) .
>>
>> I must say that all three have their reasons and I take what the majority
>> wants.
>>
>> A vote?
> Whether or not to include a single year, or range of years in
> the copyright statement is ultimately a policy decision for the
> copyright holder to take (IBM in this case I presume), and not
> subject to community vote/preferences.
>
> I will note that some (possibly even many) organizations consider
> the year to be largely redundant and devoid of legal benefit, so
> are happy with basically any usage of dates (first year, most recent
> year, a range of years, or none at all). With this in mind, QEMU is
> willing to accept any usage wrt dates in the copyright statement.
>
> It is possible that IBM have a specific policy their employees are
> expected to follow. If so, follow that.
>
> With regards,
> Daniel
OK, thanks,
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-03 16:28 ` [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology Pierre Morel
2023-04-04 7:03 ` Cédric Le Goater
@ 2023-04-18 8:53 ` Nina Schoetterl-Glausch
2023-04-18 10:01 ` Pierre Morel
1 sibling, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-18 8:53 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> The S390 CPU have specific topology features like dedication
> and entitlement to give to the guest indications on the host
> vCPUs scheduling and help the guest take the best decisions
> on the scheduling of threads on the vCPUs.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and entitlement,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 5 ++++
> qapi/machine-common.json | 22 ++++++++++++++
> qapi/machine-target.json | 12 ++++++++
> qapi/machine.json | 17 +++++++++--
> include/hw/boards.h | 10 ++++++-
> include/hw/s390x/cpu-topology.h | 15 ++++++++++
Is hw/s390x the right path for cpu-topology?
I haven't understood the difference between hw/s390x and target/s390x
but target/s390x feels more correct, I could be mistaken though.
> target/s390x/cpu.h | 5 ++++
> hw/core/machine-smp.c | 53 ++++++++++++++++++++++++++++-----
> hw/core/machine.c | 4 +++
> hw/s390x/s390-virtio-ccw.c | 2 ++
> softmmu/vl.c | 6 ++++
> target/s390x/cpu.c | 7 +++++
> qapi/meson.build | 1 +
> qemu-options.hx | 7 +++--
> 14 files changed, 152 insertions(+), 14 deletions(-)
> create mode 100644 qapi/machine-common.json
> create mode 100644 include/hw/s390x/cpu-topology.h
>
[...]
> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> new file mode 100644
> index 0000000000..73ea38d976
> --- /dev/null
> +++ b/qapi/machine-common.json
> @@ -0,0 +1,22 @@
> +# -*- Mode: Python -*-
> +# vim: filetype=python
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = Machines S390 data types
> +##
> +
> +##
> +# @CpuS390Entitlement:
> +#
> +# An enumeration of cpu entitlements that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.1
> +##
> +{ 'enum': 'CpuS390Entitlement',
> + 'prefix': 'S390_CPU_ENTITLEMENT',
> + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
You can get rid of the horizontal value now that the entitlement is ignored if the
polarization is vertical.
[...]
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index b10a8541ff..57165fa3a0 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -37,6 +37,7 @@
> #ifndef CONFIG_USER_ONLY
> #include "sysemu/reset.h"
> #endif
> +#include "hw/s390x/cpu-topology.h"
>
> #define CR0_RESET 0xE0UL
> #define CR14_RESET 0xC2000000UL;
> @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
> static Property s390x_cpu_properties[] = {
> #if !defined(CONFIG_USER_ONLY)
> DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
> + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
> + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> + S390_CPU_ENTITLEMENT__MAX),
I would define an entitlement PropertyInfo in qdev-properties-system.[ch],
then one can use e.g.
-device z14-s390x-cpu,core-id=11,entitlement=high
on the command line and cpu hotplug.
I think setting the default entitlement to medium here should be fine.
[...]
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 8:53 ` Nina Schoetterl-Glausch
@ 2023-04-18 10:01 ` Pierre Morel
2023-04-18 10:15 ` Thomas Huth
2023-04-18 12:38 ` Nina Schoetterl-Glausch
0 siblings, 2 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-18 10:01 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
>> S390 adds two new SMP levels, drawers and books to the CPU
>> topology.
>> The S390 CPU have specific topology features like dedication
>> and entitlement to give to the guest indications on the host
>> vCPUs scheduling and help the guest take the best decisions
>> on the scheduling of threads on the vCPUs.
>>
>> Let us provide the SMP properties with books and drawers levels
>> and S390 CPU with dedication and entitlement,
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>> ---
>> MAINTAINERS | 5 ++++
>> qapi/machine-common.json | 22 ++++++++++++++
>> qapi/machine-target.json | 12 ++++++++
>> qapi/machine.json | 17 +++++++++--
>> include/hw/boards.h | 10 ++++++-
>> include/hw/s390x/cpu-topology.h | 15 ++++++++++
> Is hw/s390x the right path for cpu-topology?
> I haven't understood the difference between hw/s390x and target/s390x
> but target/s390x feels more correct, I could be mistaken though.
AFAIK target/s390 is for CPU emulation code while hw/s390 is for other
emulation.
So it depends how we classify the CPU topology, it is related to CPU but
it is no emulation.
Since Thomas approved this layout I would like to keep it like this.
>
>> target/s390x/cpu.h | 5 ++++
>> hw/core/machine-smp.c | 53 ++++++++++++++++++++++++++++-----
>> hw/core/machine.c | 4 +++
>> hw/s390x/s390-virtio-ccw.c | 2 ++
>> softmmu/vl.c | 6 ++++
>> target/s390x/cpu.c | 7 +++++
>> qapi/meson.build | 1 +
>> qemu-options.hx | 7 +++--
>> 14 files changed, 152 insertions(+), 14 deletions(-)
>> create mode 100644 qapi/machine-common.json
>> create mode 100644 include/hw/s390x/cpu-topology.h
>>
> [...]
>
>> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
>> new file mode 100644
>> index 0000000000..73ea38d976
>> --- /dev/null
>> +++ b/qapi/machine-common.json
>> @@ -0,0 +1,22 @@
>> +# -*- Mode: Python -*-
>> +# vim: filetype=python
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +
>> +##
>> +# = Machines S390 data types
>> +##
>> +
>> +##
>> +# @CpuS390Entitlement:
>> +#
>> +# An enumeration of cpu entitlements that can be assumed by a virtual
>> +# S390 CPU
>> +#
>> +# Since: 8.1
>> +##
>> +{ 'enum': 'CpuS390Entitlement',
>> + 'prefix': 'S390_CPU_ENTITLEMENT',
>> + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
> You can get rid of the horizontal value now that the entitlement is ignored if the
> polarization is vertical.
Right, horizontal is not used, but what would you like?
- replace horizontal with 'none' ?
- add or substract 1 when we do the conversion between enum string and
value ?
frankly I prefer to keep horizontal here which is exactly what is given
in the documentation for entitlement = 0
>
> [...]
>
>> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
>> index b10a8541ff..57165fa3a0 100644
>> --- a/target/s390x/cpu.c
>> +++ b/target/s390x/cpu.c
>> @@ -37,6 +37,7 @@
>> #ifndef CONFIG_USER_ONLY
>> #include "sysemu/reset.h"
>> #endif
>> +#include "hw/s390x/cpu-topology.h"
>>
>> #define CR0_RESET 0xE0UL
>> #define CR14_RESET 0xC2000000UL;
>> @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
>> static Property s390x_cpu_properties[] = {
>> #if !defined(CONFIG_USER_ONLY)
>> DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
>> + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
>> + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
>> + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
>> + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
>> + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
>> + S390_CPU_ENTITLEMENT__MAX),
> I would define an entitlement PropertyInfo in qdev-properties-system.[ch],
> then one can use e.g.
>
> -device z14-s390x-cpu,core-id=11,entitlement=high
Don't you think it is an enhancement we can do later?
>
> on the command line and cpu hotplug.
>
> I think setting the default entitlement to medium here should be fine.
>
> [...]
right, I had medium before and should not have change it.
Anyway what ever the default is, it must be changed later depending on
dedication.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 10:01 ` Pierre Morel
@ 2023-04-18 10:15 ` Thomas Huth
2023-04-18 12:28 ` Pierre Morel
2023-04-18 12:38 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 57+ messages in thread
From: Thomas Huth @ 2023-04-18 10:15 UTC (permalink / raw)
To: Pierre Morel, Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, frankja, berrange, clg
On 18/04/2023 12.01, Pierre Morel wrote:
>
> On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
>> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
>>> S390 adds two new SMP levels, drawers and books to the CPU
>>> topology.
>>> The S390 CPU have specific topology features like dedication
>>> and entitlement to give to the guest indications on the host
>>> vCPUs scheduling and help the guest take the best decisions
>>> on the scheduling of threads on the vCPUs.
>>>
>>> Let us provide the SMP properties with books and drawers levels
>>> and S390 CPU with dedication and entitlement,
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> MAINTAINERS | 5 ++++
>>> qapi/machine-common.json | 22 ++++++++++++++
>>> qapi/machine-target.json | 12 ++++++++
>>> qapi/machine.json | 17 +++++++++--
>>> include/hw/boards.h | 10 ++++++-
>>> include/hw/s390x/cpu-topology.h | 15 ++++++++++
>> Is hw/s390x the right path for cpu-topology?
>> I haven't understood the difference between hw/s390x and target/s390x
>> but target/s390x feels more correct, I could be mistaken though.
>
> AFAIK target/s390 is for CPU emulation code while hw/s390 is for other
> emulation.
>
> So it depends how we classify the CPU topology, it is related to CPU but it
> is no emulation.
Normally I'd say target/ is for everything what happens within a CPU chip,
and hw/ is for everything that happens outside of a CPU chip, i.e. machine
definitions and other devices.
Now CPU topology is borderline - drawers and books are rather a concept of
the machine and not of the CPU, but things like dies and threads rather
happen within a CPU chip.
So I don't mind too much either way, but I think it's certainly ok to keep
it in hw/s390x/ if you prefer that.
Thomas
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 10:15 ` Thomas Huth
@ 2023-04-18 12:28 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-18 12:28 UTC (permalink / raw)
To: Thomas Huth, Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, frankja, berrange, clg
On 4/18/23 12:15, Thomas Huth wrote:
> On 18/04/2023 12.01, Pierre Morel wrote:
>>
>> On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
>>> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
>>>> S390 adds two new SMP levels, drawers and books to the CPU
>>>> topology.
>>>> The S390 CPU have specific topology features like dedication
>>>> and entitlement to give to the guest indications on the host
>>>> vCPUs scheduling and help the guest take the best decisions
>>>> on the scheduling of threads on the vCPUs.
>>>>
>>>> Let us provide the SMP properties with books and drawers levels
>>>> and S390 CPU with dedication and entitlement,
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>> MAINTAINERS | 5 ++++
>>>> qapi/machine-common.json | 22 ++++++++++++++
>>>> qapi/machine-target.json | 12 ++++++++
>>>> qapi/machine.json | 17 +++++++++--
>>>> include/hw/boards.h | 10 ++++++-
>>>> include/hw/s390x/cpu-topology.h | 15 ++++++++++
>>> Is hw/s390x the right path for cpu-topology?
>>> I haven't understood the difference between hw/s390x and target/s390x
>>> but target/s390x feels more correct, I could be mistaken though.
>>
>> AFAIK target/s390 is for CPU emulation code while hw/s390 is for
>> other emulation.
>>
>> So it depends how we classify the CPU topology, it is related to CPU
>> but it is no emulation.
>
> Normally I'd say target/ is for everything what happens within a CPU
> chip, and hw/ is for everything that happens outside of a CPU chip,
> i.e. machine definitions and other devices.
> Now CPU topology is borderline - drawers and books are rather a
> concept of the machine and not of the CPU, but things like dies and
> threads rather happen within a CPU chip.
> So I don't mind too much either way, but I think it's certainly ok to
> keep it in hw/s390x/ if you prefer that.
>
> Thomas
>
Thanks for the clarification Thomas.
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 10:01 ` Pierre Morel
2023-04-18 10:15 ` Thomas Huth
@ 2023-04-18 12:38 ` Nina Schoetterl-Glausch
2023-04-18 13:52 ` Pierre Morel
1 sibling, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-18 12:38 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Tue, 2023-04-18 at 12:01 +0200, Pierre Morel wrote:
> On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
> > On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> > > S390 adds two new SMP levels, drawers and books to the CPU
> > > topology.
> > > The S390 CPU have specific topology features like dedication
> > > and entitlement to give to the guest indications on the host
> > > vCPUs scheduling and help the guest take the best decisions
> > > on the scheduling of threads on the vCPUs.
> > >
> > > Let us provide the SMP properties with books and drawers levels
> > > and S390 CPU with dedication and entitlement,
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > Reviewed-by: Thomas Huth <thuth@redhat.com>
> > > ---
[...]
> >
> > > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > > new file mode 100644
> > > index 0000000000..73ea38d976
> > > --- /dev/null
> > > +++ b/qapi/machine-common.json
> > > @@ -0,0 +1,22 @@
> > > +# -*- Mode: Python -*-
> > > +# vim: filetype=python
> > > +#
> > > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > +# See the COPYING file in the top-level directory.
> > > +
> > > +##
> > > +# = Machines S390 data types
> > > +##
> > > +
> > > +##
> > > +# @CpuS390Entitlement:
> > > +#
> > > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > > +# S390 CPU
> > > +#
> > > +# Since: 8.1
> > > +##
> > > +{ 'enum': 'CpuS390Entitlement',
> > > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > > + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
> > You can get rid of the horizontal value now that the entitlement is ignored if the
> > polarization is vertical.
>
>
> Right, horizontal is not used, but what would you like?
>
> - replace horizontal with 'none' ?
>
> - add or substract 1 when we do the conversion between enum string and
> value ?
Yeah, I would completely drop it because it is a meaningless value
and adjust the conversion to the cpu value accordingly.
>
> frankly I prefer to keep horizontal here which is exactly what is given
> in the documentation for entitlement = 0
Not sure what you mean with this.
>
>
>
> >
> > [...]
> >
> > > diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> > > index b10a8541ff..57165fa3a0 100644
> > > --- a/target/s390x/cpu.c
> > > +++ b/target/s390x/cpu.c
> > > @@ -37,6 +37,7 @@
> > > #ifndef CONFIG_USER_ONLY
> > > #include "sysemu/reset.h"
> > > #endif
> > > +#include "hw/s390x/cpu-topology.h"
> > >
> > > #define CR0_RESET 0xE0UL
> > > #define CR14_RESET 0xC2000000UL;
> > > @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
> > > static Property s390x_cpu_properties[] = {
> > > #if !defined(CONFIG_USER_ONLY)
> > > DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
> > > + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
> > > + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> > > + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> > > + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> > > + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> > > + S390_CPU_ENTITLEMENT__MAX),
> > I would define an entitlement PropertyInfo in qdev-properties-system.[ch],
> > then one can use e.g.
> >
> > -device z14-s390x-cpu,core-id=11,entitlement=high
>
>
> Don't you think it is an enhancement we can do later?
It's a user visible change, so no.
But it's not complicated, should be just:
const PropertyInfo qdev_prop_cpus390entitlement = {
.name = "CpuS390Entitlement",
.enum_table = &CpuS390Entitlement_lookup,
.get = qdev_propinfo_get_enum,
.set = qdev_propinfo_set_enum,
.set_default_value = qdev_propinfo_set_default_value_enum,
};
Plus a comment & build bug in qdev-properties-system.c
and
extern const PropertyInfo qdev_prop_cpus390entitlement;
#define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f, _d) \
DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_cpus390entitlement, \
CpuS390Entitlement)
in qdev-properties-system.h
You need to change the type of env.entitlement and set the default to 1 for medium
and that should be it.
>
>
> >
> > on the command line and cpu hotplug.
> >
> > I think setting the default entitlement to medium here should be fine.
> >
> > [...]
>
> right, I had medium before and should not have change it.
>
> Anyway what ever the default is, it must be changed later depending on
> dedication.
No, you can just set it to medium and get rid of the adjustment code.
s390_topology_check will reject invalid changes and the default above
is fine since dedication is false.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 12:38 ` Nina Schoetterl-Glausch
@ 2023-04-18 13:52 ` Pierre Morel
2023-04-18 14:58 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-18 13:52 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 4/18/23 14:38, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-04-18 at 12:01 +0200, Pierre Morel wrote:
>> On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
>>> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
>>>> S390 adds two new SMP levels, drawers and books to the CPU
>>>> topology.
>>>> The S390 CPU have specific topology features like dedication
>>>> and entitlement to give to the guest indications on the host
>>>> vCPUs scheduling and help the guest take the best decisions
>>>> on the scheduling of threads on the vCPUs.
>>>>
>>>> Let us provide the SMP properties with books and drawers levels
>>>> and S390 CPU with dedication and entitlement,
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>>> ---
> [...]
>>>> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
>>>> new file mode 100644
>>>> index 0000000000..73ea38d976
>>>> --- /dev/null
>>>> +++ b/qapi/machine-common.json
>>>> @@ -0,0 +1,22 @@
>>>> +# -*- Mode: Python -*-
>>>> +# vim: filetype=python
>>>> +#
>>>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>>>> +# See the COPYING file in the top-level directory.
>>>> +
>>>> +##
>>>> +# = Machines S390 data types
>>>> +##
>>>> +
>>>> +##
>>>> +# @CpuS390Entitlement:
>>>> +#
>>>> +# An enumeration of cpu entitlements that can be assumed by a virtual
>>>> +# S390 CPU
>>>> +#
>>>> +# Since: 8.1
>>>> +##
>>>> +{ 'enum': 'CpuS390Entitlement',
>>>> + 'prefix': 'S390_CPU_ENTITLEMENT',
>>>> + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
>>> You can get rid of the horizontal value now that the entitlement is ignored if the
>>> polarization is vertical.
>>
>> Right, horizontal is not used, but what would you like?
>>
>> - replace horizontal with 'none' ?
>>
>> - add or substract 1 when we do the conversion between enum string and
>> value ?
> Yeah, I would completely drop it because it is a meaningless value
> and adjust the conversion to the cpu value accordingly.
>> frankly I prefer to keep horizontal here which is exactly what is given
>> in the documentation for entitlement = 0
> Not sure what you mean with this.
I mean: Extract from the PoP:
----
The following values are used:
PP Meaning
0 The one or more CPUs represented by the TLE are
horizontally polarized.
1 The one or more CPUs represented by the TLE are
vertically polarized. Entitlement is low.
2 The one or more CPUs represented by the TLE are
vertically polarized. Entitlement is medium.
3 The one or more CPUs represented by the TLE are
vertically polarized. Entitlement is high.
----
Also I find that using an enum to systematically add/subtract a value is
for me weird.
so I really prefer to keep "horizontal", "low", "medium", "high" event
"horizontal" will never appear.
A mater of taste, it does not change anything to the functionality or
the API.
>>
>>
>>> [...]
>>>
>>>> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
>>>> index b10a8541ff..57165fa3a0 100644
>>>> --- a/target/s390x/cpu.c
>>>> +++ b/target/s390x/cpu.c
>>>> @@ -37,6 +37,7 @@
>>>> #ifndef CONFIG_USER_ONLY
>>>> #include "sysemu/reset.h"
>>>> #endif
>>>> +#include "hw/s390x/cpu-topology.h"
>>>>
>>>> #define CR0_RESET 0xE0UL
>>>> #define CR14_RESET 0xC2000000UL;
>>>> @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
>>>> static Property s390x_cpu_properties[] = {
>>>> #if !defined(CONFIG_USER_ONLY)
>>>> DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
>>>> + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
>>>> + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
>>>> + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
>>>> + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
>>>> + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
>>>> + S390_CPU_ENTITLEMENT__MAX),
>>> I would define an entitlement PropertyInfo in qdev-properties-system.[ch],
>>> then one can use e.g.
>>>
>>> -device z14-s390x-cpu,core-id=11,entitlement=high
>>
>> Don't you think it is an enhancement we can do later?
> It's a user visible change, so no.
We could have kept both string and integer.
> But it's not complicated, should be just:
>
> const PropertyInfo qdev_prop_cpus390entitlement = {
> .name = "CpuS390Entitlement",
> .enum_table = &CpuS390Entitlement_lookup,
> .get = qdev_propinfo_get_enum,
> .set = qdev_propinfo_set_enum,
> .set_default_value = qdev_propinfo_set_default_value_enum,
> };
>
> Plus a comment & build bug in qdev-properties-system.c
>
> and
>
> extern const PropertyInfo qdev_prop_cpus390entitlement;
> #define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f, _d) \
> DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_cpus390entitlement, \
> CpuS390Entitlement)
>
> in qdev-properties-system.h
>
> You need to change the type of env.entitlement and set the default to 1 for medium
> and that should be it.
OK, it does not change anything to the functionality but is a little bit
more pretty.
>>
>>> on the command line and cpu hotplug.
>>>
>>> I think setting the default entitlement to medium here should be fine.
>>>
>>> [...]
>> right, I had medium before and should not have change it.
>>
>> Anyway what ever the default is, it must be changed later depending on
>> dedication.
> No, you can just set it to medium and get rid of the adjustment code.
> s390_topology_check will reject invalid changes and the default above
> is fine since dedication is false.
I do not want a default specification for the entitlement to depend on
the polarization.
If we do as you propose, by horizontal polarization a default
entitlement with dedication will be accepted but will be refused after
the guest switched for vertical polarization.
So we need adjustment before the check in both cases.
I find it easier and more logical if there is no default value than to
have a default we need to overwrite.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology
2023-04-18 13:52 ` Pierre Morel
@ 2023-04-18 14:58 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-18 14:58 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
> On 4/18/23 14:38, Nina Schoetterl-Glausch wrote:
> > On Tue, 2023-04-18 at 12:01 +0200, Pierre Morel wrote:
> > > On 4/18/23 10:53, Nina Schoetterl-Glausch wrote:
> > > > On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> > > > > S390 adds two new SMP levels, drawers and books to the CPU
> > > > > topology.
> > > > > The S390 CPU have specific topology features like dedication
> > > > > and entitlement to give to the guest indications on the host
> > > > > vCPUs scheduling and help the guest take the best decisions
> > > > > on the scheduling of threads on the vCPUs.
> > > > >
> > > > > Let us provide the SMP properties with books and drawers levels
> > > > > and S390 CPU with dedication and entitlement,
> > > > >
> > > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > > Reviewed-by: Thomas Huth <thuth@redhat.com>
> > > > > ---
> > [...]
> > > > > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > > > > new file mode 100644
> > > > > index 0000000000..73ea38d976
> > > > > --- /dev/null
> > > > > +++ b/qapi/machine-common.json
> > > > > @@ -0,0 +1,22 @@
> > > > > +# -*- Mode: Python -*-
> > > > > +# vim: filetype=python
> > > > > +#
> > > > > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > > +# See the COPYING file in the top-level directory.
> > > > > +
> > > > > +##
> > > > > +# = Machines S390 data types
> > > > > +##
> > > > > +
> > > > > +##
> > > > > +# @CpuS390Entitlement:
> > > > > +#
> > > > > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > > > > +# S390 CPU
> > > > > +#
> > > > > +# Since: 8.1
> > > > > +##
> > > > > +{ 'enum': 'CpuS390Entitlement',
> > > > > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > > > > + 'data': [ 'horizontal', 'low', 'medium', 'high' ] }
> > > > You can get rid of the horizontal value now that the entitlement is ignored if the
> > > > polarization is vertical.
> > >
> > > Right, horizontal is not used, but what would you like?
> > >
> > > - replace horizontal with 'none' ?
> > >
> > > - add or substract 1 when we do the conversion between enum string and
> > > value ?
> > Yeah, I would completely drop it because it is a meaningless value
> > and adjust the conversion to the cpu value accordingly.
> > > frankly I prefer to keep horizontal here which is exactly what is given
> > > in the documentation for entitlement = 0
> > Not sure what you mean with this.
>
> I mean: Extract from the PoP:
>
> ----
>
> The following values are used:
> PP Meaning
> 0 The one or more CPUs represented by the TLE are
> horizontally polarized.
> 1 The one or more CPUs represented by the TLE are
> vertically polarized. Entitlement is low.
> 2 The one or more CPUs represented by the TLE are
> vertically polarized. Entitlement is medium.
> 3 The one or more CPUs represented by the TLE are
> vertically polarized. Entitlement is high.
>
> ----
>
> Also I find that using an enum to systematically add/subtract a value is
> for me weird.
It is, I'd do:
+static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
+{
+ struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
+ s390_topology_id topology_id = {0};
+
+ topology_id.drawer = cpu->env.drawer_id;
+ topology_id.book = cpu->env.book_id;
+ topology_id.socket = cpu->env.socket_id;
+ topology_id.origin = cpu->env.core_id / 64;
+ topology_id.type = S390_TOPOLOGY_CPU_IFL;
+ topology_id.dedicated = cpu->env.dedicated;
+
+ if (s390ms->vertical_polarization) {
+ uint8_t to_polarization[] = {
+ [S390_CPU_ENTITLEMENT_LOW] = 1,
+ [S390_CPU_ENTITLEMENT_MEDIUM] = 2,
+ [S390_CPU_ENTITLEMENT_HIGH] = 3,
+ };
+ topology_id.entitlement = to_polarization[cpu->env.entitlement];
+ }
+
+ return topology_id;
+}
You can also use a switch of course.
I'd also rename s390_topology_id.entitlement to polarization.
>
> so I really prefer to keep "horizontal", "low", "medium", "high" event
> "horizontal" will never appear.
>
> A mater of taste, it does not change anything to the functionality or
> the API.
Well, it does change the API a bit, namely which values mean what,
currently there is a value 0 that you're not supposed to use, that would go away.
It also shows up in some meta command to print qapi interfaces.
And dropping it simplifies the implementation IMO --- you don't need
to think about and prevent usage of a nonexistent state.
>
>
> > >
> > >
> > > > [...]
> > > >
> > > > > diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> > > > > index b10a8541ff..57165fa3a0 100644
> > > > > --- a/target/s390x/cpu.c
> > > > > +++ b/target/s390x/cpu.c
> > > > > @@ -37,6 +37,7 @@
> > > > > #ifndef CONFIG_USER_ONLY
> > > > > #include "sysemu/reset.h"
> > > > > #endif
> > > > > +#include "hw/s390x/cpu-topology.h"
> > > > >
> > > > > #define CR0_RESET 0xE0UL
> > > > > #define CR14_RESET 0xC2000000UL;
> > > > > @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
> > > > > static Property s390x_cpu_properties[] = {
> > > > > #if !defined(CONFIG_USER_ONLY)
> > > > > DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
> > > > > + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
> > > > > + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> > > > > + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> > > > > + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> > > > > + DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> > > > > + S390_CPU_ENTITLEMENT__MAX),
> > > > I would define an entitlement PropertyInfo in qdev-properties-system.[ch],
> > > > then one can use e.g.
> > > >
> > > > -device z14-s390x-cpu,core-id=11,entitlement=high
> > >
> > > Don't you think it is an enhancement we can do later?
> > It's a user visible change, so no.
>
>
> We could have kept both string and integer.
That sounds harder to do, I guess you'd have to reimplement the PropertyInfo
getters and setters to do that.
>
>
> > But it's not complicated, should be just:
> >
> > const PropertyInfo qdev_prop_cpus390entitlement = {
> > .name = "CpuS390Entitlement",
> > .enum_table = &CpuS390Entitlement_lookup,
> > .get = qdev_propinfo_get_enum,
> > .set = qdev_propinfo_set_enum,
> > .set_default_value = qdev_propinfo_set_default_value_enum,
> > };
> >
> > Plus a comment & build bug in qdev-properties-system.c
> >
> > and
> >
> > extern const PropertyInfo qdev_prop_cpus390entitlement;
> > #define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f, _d) \
> > DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_cpus390entitlement, \
> > CpuS390Entitlement)
> >
> > in qdev-properties-system.h
> >
> > You need to change the type of env.entitlement and set the default to 1 for medium
> > and that should be it.
>
>
> OK, it does not change anything to the functionality but is a little bit
> more pretty.
>
>
> > >
> > > > on the command line and cpu hotplug.
> > > >
> > > > I think setting the default entitlement to medium here should be fine.
> > > >
> > > > [...]
> > > right, I had medium before and should not have change it.
> > >
> > > Anyway what ever the default is, it must be changed later depending on
> > > dedication.
> > No, you can just set it to medium and get rid of the adjustment code.
> > s390_topology_check will reject invalid changes and the default above
> > is fine since dedication is false.
>
>
> I do not want a default specification for the entitlement to depend on
> the polarization.
I don't see why we cannot just set it to medium.
>
> If we do as you propose, by horizontal polarization a default
> entitlement with dedication will be accepted but will be refused after
> the guest switched for vertical polarization.
No, your check function doesn't look the polarization at all (and shouldn't):
+static void s390_topology_check(uint16_t socket_id, uint16_t book_id,
+ uint16_t drawer_id, uint16_t entitlement,
+ bool dedicated, Error **errp)
+{
[...]
+ if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
+ entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
+ error_setg(errp, "A dedicated cpu implies high entitlement");
+ return;
+ }
+}
>
> So we need adjustment before the check in both cases.
I don't see why, just always reject it.
>
> I find it easier and more logical if there is no default value than to
> have a default we need to overwrite.
>
>
>
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
2023-04-03 16:28 ` [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-04 7:31 ` Cédric Le Goater
2023-04-20 8:59 ` Nina Schoetterl-Glausch
2023-04-03 16:28 ` [PATCH v19 03/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
` (18 subsequent siblings)
20 siblings, 2 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
MAINTAINERS | 1 +
include/hw/s390x/cpu-topology.h | 44 +++++
include/hw/s390x/s390-virtio-ccw.h | 1 +
hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build | 1 +
6 files changed, 349 insertions(+), 2 deletions(-)
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 9b1f80739e..bb7b34d0d8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1658,6 +1658,7 @@ S390 CPU topology
M: Pierre Morel <pmorel@linux.ibm.com>
S: Supported
F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
X86 Machines
------------
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 83f31604cc..6326cfeff8 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -10,6 +10,50 @@
#ifndef HW_S390X_CPU_TOPOLOGY_H
#define HW_S390X_CPU_TOPOLOGY_H
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
#define S390_TOPOLOGY_CPU_IFL 0x03
+typedef struct S390Topology {
+ uint8_t *cores_per_socket;
+ CpuTopology *smp;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+int s390_socket_nb(S390CPU *cpu);
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+ return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+ return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+ return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
#endif
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..ea10a6c6e1 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -28,6 +28,7 @@ struct S390CcwMachineState {
bool dea_key_wrap;
bool pv;
uint8_t loadparm[8];
+ bool vertical_polarization;
};
struct S390CcwMachineClass {
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 0000000000..96da67bd7e
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,282 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ * per socket.
+ * .smp: keeps track of the machine topology.
+ *
+ */
+S390Topology s390_topology = {
+ /* will be initialized after the cpu model is realized */
+ .cores_per_socket = NULL,
+ .smp = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a cpu.
+ */
+int s390_socket_nb(S390CPU *cpu)
+{
+ return (cpu->env.drawer_id * s390_topology.smp->books + cpu->env.book_id) *
+ s390_topology.smp->sockets + cpu->env.socket_id;
+}
+
+/**
+ * s390_has_topology:
+ *
+ * Return value: if the topology is supported by the machine.
+ */
+bool s390_has_topology(void)
+{
+ return false;
+}
+
+/**
+ * s390_topology_init:
+ * @ms: the machine state where the machine topology is defined
+ *
+ * Keep track of the machine topology.
+ *
+ * Allocate an array to keep the count of cores per socket.
+ * The index of the array starts at socket 0 from book 0 and
+ * drawer 0 up to the maximum allowed by the machine topology.
+ */
+static void s390_topology_init(MachineState *ms)
+{
+ CpuTopology *smp = &ms->smp;
+
+ s390_topology.smp = smp;
+ s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
+ smp->books * smp->drawers);
+}
+
+/**
+ * s390_topology_cpu_default:
+ * @cpu: pointer to a S390CPU
+ * @errp: Error pointer
+ *
+ * Setup the default topology if no attributes are already set.
+ * Passing a CPU with some, but not all, attributes set is considered
+ * an error.
+ *
+ * The function calculates the (drawer_id, book_id, socket_id)
+ * topology by filling the cores starting from the first socket
+ * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
+ *
+ * CPU type and dedication have defaults values set in the
+ * s390x_cpu_properties, entitlement must be adjust depending on the
+ * dedication.
+ */
+static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
+{
+ CpuTopology *smp = s390_topology.smp;
+ CPUS390XState *env = &cpu->env;
+
+ /* All geometry topology attributes must be set or all unset */
+ if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
+ (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
+ error_setg(errp,
+ "Please define all or none of the topology geometry attributes");
+ return;
+ }
+
+ /* Check if one of the geometry topology is unset */
+ if (env->socket_id < 0) {
+ /* Calculate default geometry topology attributes */
+ env->socket_id = s390_std_socket(env->core_id, smp);
+ env->book_id = s390_std_book(env->core_id, smp);
+ env->drawer_id = s390_std_drawer(env->core_id, smp);
+ }
+
+ /*
+ * Even the user can specify the entitlement as horizontal on the
+ * command line, qemu will only use env->entitlement during vertical
+ * polarization.
+ * Medium entitlement is chosen as the default entitlement when the CPU
+ * is not dedicated.
+ * A dedicated CPU always receives a high entitlement.
+ */
+ if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
+ env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
+ if (env->dedicated) {
+ env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
+ }
+ }
+}
+
+/**
+ * s390_topology_check:
+ * @socket_id: socket to check
+ * @book_id: book to check
+ * @drawer_id: drawer to check
+ * @entitlement: entitlement to check
+ * @dedicated: dedication to check
+ * @errp: Error pointer
+ *
+ * The function checks if the topology
+ * attributes fits inside the system topology.
+ */
+static void s390_topology_check(uint16_t socket_id, uint16_t book_id,
+ uint16_t drawer_id, uint16_t entitlement,
+ bool dedicated, Error **errp)
+{
+ CpuTopology *smp = s390_topology.smp;
+ ERRP_GUARD();
+
+ if (socket_id >= smp->sockets) {
+ error_setg(errp, "Unavailable socket: %d", socket_id);
+ return;
+ }
+ if (book_id >= smp->books) {
+ error_setg(errp, "Unavailable book: %d", book_id);
+ return;
+ }
+ if (drawer_id >= smp->drawers) {
+ error_setg(errp, "Unavailable drawer: %d", drawer_id);
+ return;
+ }
+ if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
+ error_setg(errp, "Unknown entitlement: %d", entitlement);
+ return;
+ }
+ if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
+ entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
+ error_setg(errp, "A dedicated cpu implies high entitlement");
+ return;
+ }
+}
+
+/**
+ * s390_topology_add_core_to_socket:
+ * @cpu: the new S390CPU to insert in the topology structure
+ * @drawer_id: new drawer_id
+ * @book_id: new book_id
+ * @socket_id: new socket_id
+ * @creation: if is true the CPU is a new CPU and there is no old socket
+ * to handle.
+ * if is false, this is a moving the CPU and old socket count
+ * must be decremented.
+ * @errp: the error pointer
+ *
+ */
+static void s390_topology_add_core_to_socket(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ bool creation, Error **errp)
+{
+ int old_socket_entry = s390_socket_nb(cpu);
+ int new_socket_entry;
+
+ if (creation) {
+ new_socket_entry = old_socket_entry;
+ } else {
+ new_socket_entry = (drawer_id * s390_topology.smp->books + book_id) *
+ s390_topology.smp->sockets + socket_id;
+ }
+
+ /* Check for space on new socket */
+ if ((new_socket_entry != old_socket_entry) &&
+ (s390_topology.cores_per_socket[new_socket_entry] >=
+ s390_topology.smp->cores)) {
+ error_setg(errp, "No more space on this socket");
+ return;
+ }
+
+ /* Update the count of cores in sockets */
+ s390_topology.cores_per_socket[new_socket_entry] += 1;
+ if (!creation) {
+ s390_topology.cores_per_socket[old_socket_entry] -= 1;
+ }
+}
+
+/**
+ * s390_update_cpu_props:
+ * @ms: the machine state
+ * @cpu: the CPU for which to update the properties from the environment.
+ *
+ */
+static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
+{
+ CpuInstanceProperties *props;
+
+ props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
+
+ props->socket_id = cpu->env.socket_id;
+ props->book_id = cpu->env.book_id;
+ props->drawer_id = cpu->env.drawer_id;
+}
+
+/**
+ * s390_topology_setup_cpu:
+ * @ms: MachineState used to initialize the topology structure on
+ * first call.
+ * @cpu: the new S390CPU to insert in the topology structure
+ * @errp: the error pointer
+ *
+ * Called from CPU Hotplug to check and setup the CPU attributes
+ * before to insert the CPU in the topology.
+ * There is no use to update the MTCR explicitely here because it
+ * will be updated by KVM on creation of the new vCPU.
+ */
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
+{
+ ERRP_GUARD();
+
+ /*
+ * We do not want to initialize the topology if the cpu model
+ * does not support topology, consequently, we have to wait for
+ * the first CPU to be realized, which realizes the CPU model
+ * to initialize the topology structures.
+ *
+ * s390_topology_setup_cpu() is called from the cpu hotplug.
+ */
+ if (!s390_topology.cores_per_socket) {
+ s390_topology_init(ms);
+ }
+
+ s390_topology_cpu_default(cpu, errp);
+ if (*errp) {
+ return;
+ }
+
+ s390_topology_check(cpu->env.socket_id, cpu->env.book_id,
+ cpu->env.drawer_id, cpu->env.entitlement,
+ cpu->env.dedicated, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* Set the CPU inside the socket */
+ s390_topology_add_core_to_socket(cpu, 0, 0, 0, true, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+}
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 1a9bcda8b6..9df60ac447 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -45,6 +45,7 @@
#include "hw/s390x/pv.h"
#include "migration/blocker.h"
#include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
static Error *pv_mig_blocker;
@@ -311,10 +312,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
{
MachineState *ms = MACHINE(hotplug_dev);
S390CPU *cpu = S390_CPU(dev);
+ ERRP_GUARD();
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ if (s390_has_topology()) {
+ s390_topology_setup_cpu(ms, cpu, errp);
+ if (*errp) {
+ return;
+ }
+ }
+
if (dev->hotplugged) {
raise_irq_cpu_hotplug();
}
@@ -554,11 +563,20 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
+ CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
+
ms->possible_cpus->cpus[i].type = ms->cpu_type;
ms->possible_cpus->cpus[i].vcpus_count = 1;
ms->possible_cpus->cpus[i].arch_id = i;
- ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = i;
+
+ props->has_core_id = true;
+ props->core_id = i;
+ props->has_socket_id = true;
+ props->socket_id = s390_std_socket(i, &ms->smp);
+ props->has_book_id = true;
+ props->book_id = s390_std_book(i, &ms->smp);
+ props->has_drawer_id = true;
+ props->drawer_id = s390_std_drawer(i, &ms->smp);
}
return ms->possible_cpus;
diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
index f291016fee..58dfbdff4f 100644
--- a/hw/s390x/meson.build
+++ b/hw/s390x/meson.build
@@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
's390-stattrib-kvm.c',
'pv.c',
's390-pci-kvm.c',
+ 'cpu-topology.c',
))
s390x_ss.add(when: 'CONFIG_TCG', if_true: files(
'tod-tcg.c',
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-03 16:28 ` [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
@ 2023-04-04 7:31 ` Cédric Le Goater
2023-04-04 11:39 ` Pierre Morel
2023-04-19 17:15 ` Nina Schoetterl-Glausch
2023-04-20 8:59 ` Nina Schoetterl-Glausch
1 sibling, 2 replies; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 7:31 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> The topology information are attributes of the CPU and are
> specified during the CPU device creation.
>
> On hot plug we:
> - calculate the default values for the topology for drawers,
> books and sockets in the case they are not specified.
> - verify the CPU attributes
> - check that we have still room on the desired socket
>
> The possibility to insert a CPU in a mask is dependent on the
> number of cores allowed in a socket, a book or a drawer, the
> checking is done during the hot plug of the CPU to have an
> immediate answer.
>
> If the complete topology is not specified, the core is added
> in the physical topology based on its core ID and it gets
> defaults values for the modifier attributes.
>
> This way, starting QEMU without specifying the topology can
> still get some advantage of the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> include/hw/s390x/cpu-topology.h | 44 +++++
> include/hw/s390x/s390-virtio-ccw.h | 1 +
> hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
> hw/s390x/s390-virtio-ccw.c | 22 ++-
> hw/s390x/meson.build | 1 +
> 6 files changed, 349 insertions(+), 2 deletions(-)
> create mode 100644 hw/s390x/cpu-topology.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9b1f80739e..bb7b34d0d8 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1658,6 +1658,7 @@ S390 CPU topology
> M: Pierre Morel <pmorel@linux.ibm.com>
> S: Supported
> F: include/hw/s390x/cpu-topology.h
> +F: hw/s390x/cpu-topology.c
>
> X86 Machines
> ------------
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> index 83f31604cc..6326cfeff8 100644
> --- a/include/hw/s390x/cpu-topology.h
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -10,6 +10,50 @@
> #ifndef HW_S390X_CPU_TOPOLOGY_H
> #define HW_S390X_CPU_TOPOLOGY_H
>
> +#ifndef CONFIG_USER_ONLY
> +
> +#include "qemu/queue.h"
> +#include "hw/boards.h"
> +#include "qapi/qapi-types-machine-target.h"
> +
> #define S390_TOPOLOGY_CPU_IFL 0x03
>
> +typedef struct S390Topology {
> + uint8_t *cores_per_socket;
> + CpuTopology *smp;
> +} S390Topology;
> +
> +#ifdef CONFIG_KVM
> +bool s390_has_topology(void);
> +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
> +#else
> +static inline bool s390_has_topology(void)
> +{
> + return false;
> +}
> +static inline void s390_topology_setup_cpu(MachineState *ms,
> + S390CPU *cpu,
> + Error **errp) {}
> +#endif
> +
> +extern S390Topology s390_topology;
> +int s390_socket_nb(S390CPU *cpu);
> +
> +static inline int s390_std_socket(int n, CpuTopology *smp)
> +{
> + return (n / smp->cores) % smp->sockets;
> +}
> +
> +static inline int s390_std_book(int n, CpuTopology *smp)
> +{
> + return (n / (smp->cores * smp->sockets)) % smp->books;
> +}
> +
> +static inline int s390_std_drawer(int n, CpuTopology *smp)
> +{
> + return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
> +}
> +
> +#endif /* CONFIG_USER_ONLY */
> +
> #endif
> diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
> index 9bba21a916..ea10a6c6e1 100644
> --- a/include/hw/s390x/s390-virtio-ccw.h
> +++ b/include/hw/s390x/s390-virtio-ccw.h
> @@ -28,6 +28,7 @@ struct S390CcwMachineState {
> bool dea_key_wrap;
> bool pv;
> uint8_t loadparm[8];
> + bool vertical_polarization;
> };
>
> struct S390CcwMachineClass {
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> new file mode 100644
> index 0000000000..96da67bd7e
> --- /dev/null
> +++ b/hw/s390x/cpu-topology.c
> @@ -0,0 +1,282 @@
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022
> + * Author(s): Pierre Morel <pmorel@linux.ibm.com>
> +
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qemu/error-report.h"
> +#include "hw/qdev-properties.h"
> +#include "hw/boards.h"
> +#include "qemu/typedefs.h"
> +#include "target/s390x/cpu.h"
> +#include "hw/s390x/s390-virtio-ccw.h"
> +#include "hw/s390x/cpu-topology.h"
> +
> +/*
> + * s390_topology is used to keep the topology information.
> + * .cores_per_socket: tracks information on the count of cores
> + * per socket.
> + * .smp: keeps track of the machine topology.
> + *
> + */
> +S390Topology s390_topology = {
> + /* will be initialized after the cpu model is realized */
> + .cores_per_socket = NULL,
> + .smp = NULL,
> +};
> +
> +/**
> + * s390_socket_nb:
> + * @cpu: s390x CPU
> + *
> + * Returns the socket number used inside the cores_per_socket array
> + * for a cpu.
> + */
> +int s390_socket_nb(S390CPU *cpu)
s390_socket_nb() doesn't seem to be used anywhere else than in
hw/s390x/cpu-topology.c. It should be static.
> +{
> + return (cpu->env.drawer_id * s390_topology.smp->books + cpu->env.book_id) *
> + s390_topology.smp->sockets + cpu->env.socket_id;
> +}
> +
> +/**
> + * s390_has_topology:
> + *
> + * Return value: if the topology is supported by the machine.
> + */
> +bool s390_has_topology(void)
> +{
> + return false;
> +}
> +
> +/**
> + * s390_topology_init:
> + * @ms: the machine state where the machine topology is defined
> + *
> + * Keep track of the machine topology.
> + *
> + * Allocate an array to keep the count of cores per socket.
> + * The index of the array starts at socket 0 from book 0 and
> + * drawer 0 up to the maximum allowed by the machine topology.
> + */
> +static void s390_topology_init(MachineState *ms)
> +{
> + CpuTopology *smp = &ms->smp;
> +
> + s390_topology.smp = smp;
> + s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
> + smp->books * smp->drawers);
> +}
> +
> +/**
> + * s390_topology_cpu_default:
> + * @cpu: pointer to a S390CPU
> + * @errp: Error pointer
> + *
> + * Setup the default topology if no attributes are already set.
> + * Passing a CPU with some, but not all, attributes set is considered
> + * an error.
> + *
> + * The function calculates the (drawer_id, book_id, socket_id)
> + * topology by filling the cores starting from the first socket
> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
> + *
> + * CPU type and dedication have defaults values set in the
> + * s390x_cpu_properties, entitlement must be adjust depending on the
> + * dedication.
> + */
> +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> +{
> + CpuTopology *smp = s390_topology.smp;
> + CPUS390XState *env = &cpu->env;
> +
> + /* All geometry topology attributes must be set or all unset */
> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
> + error_setg(errp,
> + "Please define all or none of the topology geometry attributes");
> + return;
> + }
> +
> + /* Check if one of the geometry topology is unset */
> + if (env->socket_id < 0) {
> + /* Calculate default geometry topology attributes */
> + env->socket_id = s390_std_socket(env->core_id, smp);
> + env->book_id = s390_std_book(env->core_id, smp);
> + env->drawer_id = s390_std_drawer(env->core_id, smp);
> + }
> +
> + /*
> + * Even the user can specify the entitlement as horizontal on the
> + * command line, qemu will only use env->entitlement during vertical
> + * polarization.
> + * Medium entitlement is chosen as the default entitlement when the CPU
> + * is not dedicated.
> + * A dedicated CPU always receives a high entitlement.
> + */
> + if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
> + env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
> + if (env->dedicated) {
> + env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> + } else {
> + env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
> + }
> + }
> +}
> +
> +/**
> + * s390_topology_check:
> + * @socket_id: socket to check
> + * @book_id: book to check
> + * @drawer_id: drawer to check
> + * @entitlement: entitlement to check
> + * @dedicated: dedication to check
> + * @errp: Error pointer
> + *
> + * The function checks if the topology
> + * attributes fits inside the system topology.
> + */
> +static void s390_topology_check(uint16_t socket_id, uint16_t book_id,
> + uint16_t drawer_id, uint16_t entitlement,
> + bool dedicated, Error **errp)
> +{
> + CpuTopology *smp = s390_topology.smp;
> + ERRP_GUARD();
> +
> + if (socket_id >= smp->sockets) {
> + error_setg(errp, "Unavailable socket: %d", socket_id);
> + return;
> + }
> + if (book_id >= smp->books) {
> + error_setg(errp, "Unavailable book: %d", book_id);
> + return;
> + }
> + if (drawer_id >= smp->drawers) {
> + error_setg(errp, "Unavailable drawer: %d", drawer_id);
> + return;
> + }
> + if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
> + error_setg(errp, "Unknown entitlement: %d", entitlement);
> + return;
> + }
> + if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
> + entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
> + error_setg(errp, "A dedicated cpu implies high entitlement");
> + return;
> + }
> +}
> +
> +/**
> + * s390_topology_add_core_to_socket:
> + * @cpu: the new S390CPU to insert in the topology structure
> + * @drawer_id: new drawer_id
> + * @book_id: new book_id
> + * @socket_id: new socket_id
> + * @creation: if is true the CPU is a new CPU and there is no old socket
> + * to handle.
> + * if is false, this is a moving the CPU and old socket count
> + * must be decremented.
> + * @errp: the error pointer
> + *
> + */
> +static void s390_topology_add_core_to_socket(S390CPU *cpu, int drawer_id,
> + int book_id, int socket_id,
> + bool creation, Error **errp)
> +{
Since this routine is called twice, in s390_topology_setup_cpu() for
creation, and in s390_change_topology() for socket migration, we could
duplicate the code in two distinct routines.
I think this would simplify a bit each code path and avoid the 'creation'
parameter which is confusing.
> + int old_socket_entry = s390_socket_nb(cpu);
> + int new_socket_entry;
> +
> + if (creation) {
> + new_socket_entry = old_socket_entry;
> + } else {
> + new_socket_entry = (drawer_id * s390_topology.smp->books + book_id) *
> + s390_topology.smp->sockets + socket_id;
A helper common routine that s390_socket_nb() could use also would be a plus.
> + }
> +
> + /* Check for space on new socket */
> + if ((new_socket_entry != old_socket_entry) &&
> + (s390_topology.cores_per_socket[new_socket_entry] >=
> + s390_topology.smp->cores)) {
> + error_setg(errp, "No more space on this socket");
> + return;
> + }
> +
> + /* Update the count of cores in sockets */
> + s390_topology.cores_per_socket[new_socket_entry] += 1;
> + if (!creation) {
> + s390_topology.cores_per_socket[old_socket_entry] -= 1;
> + }
> +}
> +
> +/**
> + * s390_update_cpu_props:
> + * @ms: the machine state
> + * @cpu: the CPU for which to update the properties from the environment.
> + *
> + */
> +static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
> +{
> + CpuInstanceProperties *props;
> +
> + props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
> +
> + props->socket_id = cpu->env.socket_id;
> + props->book_id = cpu->env.book_id;
> + props->drawer_id = cpu->env.drawer_id;
> +}
> +
> +/**
> + * s390_topology_setup_cpu:
> + * @ms: MachineState used to initialize the topology structure on
> + * first call.
> + * @cpu: the new S390CPU to insert in the topology structure
> + * @errp: the error pointer
> + *
> + * Called from CPU Hotplug to check and setup the CPU attributes
> + * before to insert the CPU in the topology.
... before the CPU is inserted in the topology.
> + * There is no use to update the MTCR explicitely here because it
... is no need ... sounds better.
> + * will be updated by KVM on creation of the new vCPU.
"CPU" is used everywhere else.
> + */
> +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> +{
> + ERRP_GUARD();
> +
> + /*
> + * We do not want to initialize the topology if the cpu model
> + * does not support topology, consequently, we have to wait for
> + * the first CPU to be realized, which realizes the CPU model
> + * to initialize the topology structures.
> + *
> + * s390_topology_setup_cpu() is called from the cpu hotplug.
> + */
> + if (!s390_topology.cores_per_socket) {
> + s390_topology_init(ms);
> + }
> +
> + s390_topology_cpu_default(cpu, errp);
> + if (*errp) {
May be having s390_topology_cpu_default() return a bool would be cleaner.
Same comment for the routines below. This is minor.
> + return;
> + }
> +
> + s390_topology_check(cpu->env.socket_id, cpu->env.book_id,
> + cpu->env.drawer_id, cpu->env.entitlement,
> + cpu->env.dedicated, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* Set the CPU inside the socket */
> + s390_topology_add_core_to_socket(cpu, 0, 0, 0, true, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +}
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 1a9bcda8b6..9df60ac447 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -45,6 +45,7 @@
> #include "hw/s390x/pv.h"
> #include "migration/blocker.h"
> #include "qapi/visitor.h"
> +#include "hw/s390x/cpu-topology.h"
>
> static Error *pv_mig_blocker;
>
> @@ -311,10 +312,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
> {
> MachineState *ms = MACHINE(hotplug_dev);
> S390CPU *cpu = S390_CPU(dev);
> + ERRP_GUARD();
>
> g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
> ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
>
> + if (s390_has_topology()) {
> + s390_topology_setup_cpu(ms, cpu, errp);
> + if (*errp) {
> + return;
> + }
> + }
> +
> if (dev->hotplugged) {
> raise_irq_cpu_hotplug();
> }
> @@ -554,11 +563,20 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
> sizeof(CPUArchId) * max_cpus);
> ms->possible_cpus->len = max_cpus;
> for (i = 0; i < ms->possible_cpus->len; i++) {
> + CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
> +
> ms->possible_cpus->cpus[i].type = ms->cpu_type;
> ms->possible_cpus->cpus[i].vcpus_count = 1;
> ms->possible_cpus->cpus[i].arch_id = i;
> - ms->possible_cpus->cpus[i].props.has_core_id = true;
> - ms->possible_cpus->cpus[i].props.core_id = i;
> +
> + props->has_core_id = true;
> + props->core_id = i;
> + props->has_socket_id = true;
> + props->socket_id = s390_std_socket(i, &ms->smp);
> + props->has_book_id = true;
> + props->book_id = s390_std_book(i, &ms->smp);
> + props->has_drawer_id = true;
> + props->drawer_id = s390_std_drawer(i, &ms->smp);
> }
>
> return ms->possible_cpus;
> diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
> index f291016fee..58dfbdff4f 100644
> --- a/hw/s390x/meson.build
> +++ b/hw/s390x/meson.build
> @@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
> 's390-stattrib-kvm.c',
> 'pv.c',
> 's390-pci-kvm.c',
> + 'cpu-topology.c',
> ))
> s390x_ss.add(when: 'CONFIG_TCG', if_true: files(
> 'tod-tcg.c',
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-04 7:31 ` Cédric Le Goater
@ 2023-04-04 11:39 ` Pierre Morel
2023-04-19 17:15 ` Nina Schoetterl-Glausch
1 sibling, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 11:39 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 09:31, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>>
[...]
>> +
>> +/**
>> + * s390_socket_nb:
>> + * @cpu: s390x CPU
>> + *
>> + * Returns the socket number used inside the cores_per_socket array
>> + * for a cpu.
>> + */
>> +int s390_socket_nb(S390CPU *cpu)
>
> s390_socket_nb() doesn't seem to be used anywhere else than in
> hw/s390x/cpu-topology.c. It should be static.
right
[...]
>> +/**
>> + * s390_topology_add_core_to_socket:
>> + * @cpu: the new S390CPU to insert in the topology structure
>> + * @drawer_id: new drawer_id
>> + * @book_id: new book_id
>> + * @socket_id: new socket_id
>> + * @creation: if is true the CPU is a new CPU and there is no old
>> socket
>> + * to handle.
>> + * if is false, this is a moving the CPU and old socket
>> count
>> + * must be decremented.
>> + * @errp: the error pointer
>> + *
>> + */
>> +static void s390_topology_add_core_to_socket(S390CPU *cpu, int
>> drawer_id,
>> + int book_id, int
>> socket_id,
>> + bool creation, Error
>> **errp)
>> +{
>
> Since this routine is called twice, in s390_topology_setup_cpu() for
> creation, and in s390_change_topology() for socket migration, we could
> duplicate the code in two distinct routines.
>
> I think this would simplify a bit each code path and avoid the 'creation'
> parameter which is confusing.
right
Thanks.
>
>
>> + int old_socket_entry = s390_socket_nb(cpu);
>> + int new_socket_entry;
>> +
>> + if (creation) {
>> + new_socket_entry = old_socket_entry;
>> + } else {
>> + new_socket_entry = (drawer_id * s390_topology.smp->books +
>> book_id) *
>> + s390_topology.smp->sockets + socket_id;
>
> A helper common routine that s390_socket_nb() could use also would be
> a plus.
Yes, thanks
>
>> + }
>> +
>> + /* Check for space on new socket */
>> + if ((new_socket_entry != old_socket_entry) &&
>> + (s390_topology.cores_per_socket[new_socket_entry] >=
>> + s390_topology.smp->cores)) {
>> + error_setg(errp, "No more space on this socket");
>> + return;
>> + }
>> +
>> + /* Update the count of cores in sockets */
>> + s390_topology.cores_per_socket[new_socket_entry] += 1;
>> + if (!creation) {
>> + s390_topology.cores_per_socket[old_socket_entry] -= 1;
>> + }
>> +}
>> +
>> +/**
>> + * s390_update_cpu_props:
>> + * @ms: the machine state
>> + * @cpu: the CPU for which to update the properties from the
>> environment.
>> + *
>> + */
>> +static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
>> +{
>> + CpuInstanceProperties *props;
>> +
>> + props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
>> +
>> + props->socket_id = cpu->env.socket_id;
>> + props->book_id = cpu->env.book_id;
>> + props->drawer_id = cpu->env.drawer_id;
>> +}
>> +
>> +/**
>> + * s390_topology_setup_cpu:
>> + * @ms: MachineState used to initialize the topology structure on
>> + * first call.
>> + * @cpu: the new S390CPU to insert in the topology structure
>> + * @errp: the error pointer
>> + *
>> + * Called from CPU Hotplug to check and setup the CPU attributes
>> + * before to insert the CPU in the topology.
>
> ... before the CPU is inserted in the topology.
OK
>
>> + * There is no use to update the MTCR explicitely here because it
>
> ... is no need ... sounds better.
OK
>
>> + * will be updated by KVM on creation of the new vCPU.
>
> "CPU" is used everywhere else.
OK
>
>> + */
>> +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error
>> **errp)
>> +{
>> + ERRP_GUARD();
>> +
>> + /*
>> + * We do not want to initialize the topology if the cpu model
>> + * does not support topology, consequently, we have to wait for
>> + * the first CPU to be realized, which realizes the CPU model
>> + * to initialize the topology structures.
>> + *
>> + * s390_topology_setup_cpu() is called from the cpu hotplug.
>> + */
>> + if (!s390_topology.cores_per_socket) {
>> + s390_topology_init(ms);
>> + }
>> +
>> + s390_topology_cpu_default(cpu, errp);
>> + if (*errp) {
>
> May be having s390_topology_cpu_default() return a bool would be cleaner.
> Same comment for the routines below. This is minor.
Yes and it is more readable. I do it.
Thanks for the comments.
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-04 7:31 ` Cédric Le Goater
2023-04-04 11:39 ` Pierre Morel
@ 2023-04-19 17:15 ` Nina Schoetterl-Glausch
1 sibling, 0 replies; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-19 17:15 UTC (permalink / raw)
To: Cédric Le Goater, Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange
On Tue, 2023-04-04 at 09:31 +0200, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
> > The topology information are attributes of the CPU and are
> > specified during the CPU device creation.
> >
> > On hot plug we:
> > - calculate the default values for the topology for drawers,
> > books and sockets in the case they are not specified.
> > - verify the CPU attributes
> > - check that we have still room on the desired socket
> >
> > The possibility to insert a CPU in a mask is dependent on the
> > number of cores allowed in a socket, a book or a drawer, the
> > checking is done during the hot plug of the CPU to have an
> > immediate answer.
> >
> > If the complete topology is not specified, the core is added
> > in the physical topology based on its core ID and it gets
> > defaults values for the modifier attributes.
> >
> > This way, starting QEMU without specifying the topology can
> > still get some advantage of the CPU topology.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > ---
> > MAINTAINERS | 1 +
> > include/hw/s390x/cpu-topology.h | 44 +++++
> > include/hw/s390x/s390-virtio-ccw.h | 1 +
> > hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
> > hw/s390x/s390-virtio-ccw.c | 22 ++-
> > hw/s390x/meson.build | 1 +
> > 6 files changed, 349 insertions(+), 2 deletions(-)
> > create mode 100644 hw/s390x/cpu-topology.c
[...]
> > +
> > +/**
> > + * s390_socket_nb:
> > + * @cpu: s390x CPU
> > + *
> > + * Returns the socket number used inside the cores_per_socket array
> > + * for a cpu.
> > + */
> > +int s390_socket_nb(S390CPU *cpu)
>
> s390_socket_nb() doesn't seem to be used anywhere else than in
> hw/s390x/cpu-topology.c. It should be static.
>
>
> > +{
> > + return (cpu->env.drawer_id * s390_topology.smp->books + cpu->env.book_id) *
> > + s390_topology.smp->sockets + cpu->env.socket_id;
> > +}
[...]
> > +/**
> > + * s390_topology_add_core_to_socket:
> > + * @cpu: the new S390CPU to insert in the topology structure
> > + * @drawer_id: new drawer_id
> > + * @book_id: new book_id
> > + * @socket_id: new socket_id
> > + * @creation: if is true the CPU is a new CPU and there is no old socket
> > + * to handle.
> > + * if is false, this is a moving the CPU and old socket count
> > + * must be decremented.
> > + * @errp: the error pointer
> > + *
> > + */
> > +static void s390_topology_add_core_to_socket(S390CPU *cpu, int drawer_id,
> > + int book_id, int socket_id,
> > + bool creation, Error **errp)
> > +{
>
> Since this routine is called twice, in s390_topology_setup_cpu() for
> creation, and in s390_change_topology() for socket migration, we could
> duplicate the code in two distinct routines.
>
> I think this would simplify a bit each code path and avoid the 'creation'
> parameter which is confusing.
>
>
> > + int old_socket_entry = s390_socket_nb(cpu);
> > + int new_socket_entry;
> > +
> > + if (creation) {
> > + new_socket_entry = old_socket_entry;
> > + } else {
> > + new_socket_entry = (drawer_id * s390_topology.smp->books + book_id) *
> > + s390_topology.smp->sockets + socket_id;
>
> A helper common routine that s390_socket_nb() could use also would be a plus.
An alternative to consider would be to define a
struct topology_pos {
int socket;
int book;
int drawer;
};
or similar so you can do
old_socket_entry = s390_socket_nb(cpu->env.topology_pos, smp);
struct topology_pos topology_pos = { socket_id, book_id, drawer_id };
new_socket_entry = s390_socket_nb(topology_pos, smp);
It might also make sense to pass a topology_pos around instead of three ids,
since that is quite common.
>
> > + }
> > +
> > + /* Check for space on new socket */
> > + if ((new_socket_entry != old_socket_entry) &&
> > + (s390_topology.cores_per_socket[new_socket_entry] >=
> > + s390_topology.smp->cores)) {
> > + error_setg(errp, "No more space on this socket");
> > + return;
> > + }
> > +
> > + /* Update the count of cores in sockets */
> > + s390_topology.cores_per_socket[new_socket_entry] += 1;
> > + if (!creation) {
> > + s390_topology.cores_per_socket[old_socket_entry] -= 1;
> > + }
> > +}
[...]
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-03 16:28 ` [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
2023-04-04 7:31 ` Cédric Le Goater
@ 2023-04-20 8:59 ` Nina Schoetterl-Glausch
2023-04-21 10:20 ` Pierre Morel
1 sibling, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-20 8:59 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> The topology information are attributes of the CPU and are
> specified during the CPU device creation.
>
> On hot plug we:
> - calculate the default values for the topology for drawers,
> books and sockets in the case they are not specified.
> - verify the CPU attributes
> - check that we have still room on the desired socket
>
> The possibility to insert a CPU in a mask is dependent on the
> number of cores allowed in a socket, a book or a drawer, the
> checking is done during the hot plug of the CPU to have an
> immediate answer.
>
> If the complete topology is not specified, the core is added
> in the physical topology based on its core ID and it gets
> defaults values for the modifier attributes.
>
> This way, starting QEMU without specifying the topology can
> still get some advantage of the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> include/hw/s390x/cpu-topology.h | 44 +++++
> include/hw/s390x/s390-virtio-ccw.h | 1 +
> hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
> hw/s390x/s390-virtio-ccw.c | 22 ++-
> hw/s390x/meson.build | 1 +
> 6 files changed, 349 insertions(+), 2 deletions(-)
> create mode 100644 hw/s390x/cpu-topology.c
[...]
> #endif
> diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
> index 9bba21a916..ea10a6c6e1 100644
> --- a/include/hw/s390x/s390-virtio-ccw.h
> +++ b/include/hw/s390x/s390-virtio-ccw.h
> @@ -28,6 +28,7 @@ struct S390CcwMachineState {
> bool dea_key_wrap;
> bool pv;
> uint8_t loadparm[8];
> + bool vertical_polarization;
Why is this here and not in s390_topology?
This splits up the topology state somewhat.
I don't quite recall, did you use to have s390_topology in S390CcwMachineState at some point?
I think putting everything in S390CcwMachineState might make sense too.
> };
>
> struct S390CcwMachineClass {
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> new file mode 100644
> index 0000000000..96da67bd7e
> --- /dev/null
> +++ b/hw/s390x/cpu-topology.c
[...]
> +/**
> + * s390_topology_cpu_default:
> + * @cpu: pointer to a S390CPU
> + * @errp: Error pointer
> + *
> + * Setup the default topology if no attributes are already set.
> + * Passing a CPU with some, but not all, attributes set is considered
> + * an error.
> + *
> + * The function calculates the (drawer_id, book_id, socket_id)
> + * topology by filling the cores starting from the first socket
> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
> + *
> + * CPU type and dedication have defaults values set in the
> + * s390x_cpu_properties, entitlement must be adjust depending on the
> + * dedication.
> + */
> +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> +{
> + CpuTopology *smp = s390_topology.smp;
> + CPUS390XState *env = &cpu->env;
> +
> + /* All geometry topology attributes must be set or all unset */
> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
> + error_setg(errp,
> + "Please define all or none of the topology geometry attributes");
> + return;
> + }
> +
> + /* Check if one of the geometry topology is unset */
> + if (env->socket_id < 0) {
> + /* Calculate default geometry topology attributes */
> + env->socket_id = s390_std_socket(env->core_id, smp);
> + env->book_id = s390_std_book(env->core_id, smp);
> + env->drawer_id = s390_std_drawer(env->core_id, smp);
> + }
> +
> + /*
> + * Even the user can specify the entitlement as horizontal on the
> + * command line, qemu will only use env->entitlement during vertical
> + * polarization.
> + * Medium entitlement is chosen as the default entitlement when the CPU
> + * is not dedicated.
> + * A dedicated CPU always receives a high entitlement.
> + */
> + if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
> + env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
> + if (env->dedicated) {
> + env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> + } else {
> + env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
> + }
> + }
As you know, in my opinion there should be not possibility for the user
to set the entitlement to horizontal and dedicated && != HIGH should be
rejected as an error.
If you do this, you can delete this.
> +}
> +
> +/**
> + * s390_topology_check:
> + * @socket_id: socket to check
> + * @book_id: book to check
> + * @drawer_id: drawer to check
> + * @entitlement: entitlement to check
> + * @dedicated: dedication to check
> + * @errp: Error pointer
> + *
> + * The function checks if the topology
> + * attributes fits inside the system topology.
fitS
The function checks the validity of the provided topology arguments,
namely that they're in bounds and non contradictory.
> + */
> +static void s390_topology_check(uint16_t socket_id, uint16_t book_id,
I'd prefer if you stick to one id type. There defined as int32_t in env,
here you use uint16_t and below int.
In env, you want a signed type with sufficient range, int16_t should suffice there,
but bigger is also fine.
You don't want the user to pass a negative id, so by using an unsigned type you
can avoid this without extra code.
But IMO there should be one point where a type conversion occurs.
> + uint16_t drawer_id, uint16_t entitlement,
> + bool dedicated, Error **errp)
> +{
> + CpuTopology *smp = s390_topology.smp;
> + ERRP_GUARD();
> +
> + if (socket_id >= smp->sockets) {
> + error_setg(errp, "Unavailable socket: %d", socket_id);
> + return;
> + }
> + if (book_id >= smp->books) {
> + error_setg(errp, "Unavailable book: %d", book_id);
> + return;
> + }
> + if (drawer_id >= smp->drawers) {
> + error_setg(errp, "Unavailable drawer: %d", drawer_id);
> + return;
> + }
> + if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
> + error_setg(errp, "Unknown entitlement: %d", entitlement);
> + return;
> + }
I think you can delete this, too, there is no way that entitlement is > MAX.
> + if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
> + entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
Without HORIZONTAL you can do != HIGH and save one line, but that is
cosmetic only.
> + error_setg(errp, "A dedicated cpu implies high entitlement");
> + return;
> + }
> +}
[...]
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-20 8:59 ` Nina Schoetterl-Glausch
@ 2023-04-21 10:20 ` Pierre Morel
2023-04-24 15:32 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-21 10:20 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 4/20/23 10:59, Nina Schoetterl-Glausch wrote:
> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
>> The topology information are attributes of the CPU and are
>> specified during the CPU device creation.
>>
>> On hot plug we:
>> - calculate the default values for the topology for drawers,
>> books and sockets in the case they are not specified.
>> - verify the CPU attributes
>> - check that we have still room on the desired socket
>>
>> The possibility to insert a CPU in a mask is dependent on the
>> number of cores allowed in a socket, a book or a drawer, the
>> checking is done during the hot plug of the CPU to have an
>> immediate answer.
>>
>> If the complete topology is not specified, the core is added
>> in the physical topology based on its core ID and it gets
>> defaults values for the modifier attributes.
>>
>> This way, starting QEMU without specifying the topology can
>> still get some advantage of the CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> MAINTAINERS | 1 +
>> include/hw/s390x/cpu-topology.h | 44 +++++
>> include/hw/s390x/s390-virtio-ccw.h | 1 +
>> hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
>> hw/s390x/s390-virtio-ccw.c | 22 ++-
>> hw/s390x/meson.build | 1 +
>> 6 files changed, 349 insertions(+), 2 deletions(-)
>> create mode 100644 hw/s390x/cpu-topology.c
> [...]
>
>> #endif
>> diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
>> index 9bba21a916..ea10a6c6e1 100644
>> --- a/include/hw/s390x/s390-virtio-ccw.h
>> +++ b/include/hw/s390x/s390-virtio-ccw.h
>> @@ -28,6 +28,7 @@ struct S390CcwMachineState {
>> bool dea_key_wrap;
>> bool pv;
>> uint8_t loadparm[8];
>> + bool vertical_polarization;
> Why is this here and not in s390_topology?
> This splits up the topology state somewhat.
> I don't quite recall, did you use to have s390_topology in S390CcwMachineState at some point?
> I think putting everything in S390CcwMachineState might make sense too.
Hum.
This is a left over from an abandoned try.
I put it back where it was, in s390_topology.
>
>> };
>>
>> struct S390CcwMachineClass {
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> new file mode 100644
>> index 0000000000..96da67bd7e
>> --- /dev/null
>> +++ b/hw/s390x/cpu-topology.c
> [...]
>
>> +/**
>> + * s390_topology_cpu_default:
>> + * @cpu: pointer to a S390CPU
>> + * @errp: Error pointer
>> + *
>> + * Setup the default topology if no attributes are already set.
>> + * Passing a CPU with some, but not all, attributes set is considered
>> + * an error.
>> + *
>> + * The function calculates the (drawer_id, book_id, socket_id)
>> + * topology by filling the cores starting from the first socket
>> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
>> + *
>> + * CPU type and dedication have defaults values set in the
>> + * s390x_cpu_properties, entitlement must be adjust depending on the
>> + * dedication.
>> + */
>> +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
>> +{
>> + CpuTopology *smp = s390_topology.smp;
>> + CPUS390XState *env = &cpu->env;
>> +
>> + /* All geometry topology attributes must be set or all unset */
>> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
>> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
>> + error_setg(errp,
>> + "Please define all or none of the topology geometry attributes");
>> + return;
>> + }
>> +
>> + /* Check if one of the geometry topology is unset */
>> + if (env->socket_id < 0) {
>> + /* Calculate default geometry topology attributes */
>> + env->socket_id = s390_std_socket(env->core_id, smp);
>> + env->book_id = s390_std_book(env->core_id, smp);
>> + env->drawer_id = s390_std_drawer(env->core_id, smp);
>> + }
>> +
>> + /*
>> + * Even the user can specify the entitlement as horizontal on the
>> + * command line, qemu will only use env->entitlement during vertical
>> + * polarization.
>> + * Medium entitlement is chosen as the default entitlement when the CPU
>> + * is not dedicated.
>> + * A dedicated CPU always receives a high entitlement.
>> + */
>> + if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
>> + env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
>> + if (env->dedicated) {
>> + env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
>> + } else {
>> + env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
>> + }
>> + }
> As you know, in my opinion there should be not possibility for the user
> to set the entitlement to horizontal and dedicated && != HIGH should be
> rejected as an error.
> If you do this, you can delete this.
In the next version with entitlement being an enum it is right.
However, deleting this means that the default value for entitlement
depends on dedication.
If we have only low, medium, high and default for entitlement is medium.
If the user specifies the dedication true without specifying entitlement
we could force entitlement to high.
But we can not distinguish this from the user specifying dedication true
with a medium entitlement, which is wrong.
So three solution:
1) We ignore what the user say if dedication is specified as true
2) We specify that both dedication and entitlement must be specified if
dedication is true
3) We set an impossible default to distinguish default from medium
entitlement
For me the solution 3 is the best one, it is more flexible for the user.
Solution 1 is obviously bad.
Solution 2 forces the user to specify entitlement high and only high if
it specifies dedication true.
AFAIU, you prefer the solution 2, forcing user to specify both
dedication and entitlement to suppress a default value in the enum.
Why is it bad to have a default value in the enum that we do not use to
specify that the value must be calculated?
>
>> +}
>> +
>> +/**
>> + * s390_topology_check:
>> + * @socket_id: socket to check
>> + * @book_id: book to check
>> + * @drawer_id: drawer to check
>> + * @entitlement: entitlement to check
>> + * @dedicated: dedication to check
>> + * @errp: Error pointer
>> + *
>> + * The function checks if the topology
>> + * attributes fits inside the system topology.
> fitS
>
> The function checks the validity of the provided topology arguments,
> namely that they're in bounds and non contradictory.
OK, thanks.
>
>> + */
>> +static void s390_topology_check(uint16_t socket_id, uint16_t book_id,
> I'd prefer if you stick to one id type. There defined as int32_t in env,
> here you use uint16_t and below int.
>
> In env, you want a signed type with sufficient range, int16_t should suffice there,
> but bigger is also fine.
> You don't want the user to pass a negative id, so by using an unsigned type you
> can avoid this without extra code.
> But IMO there should be one point where a type conversion occurs.
OK
>
>> + uint16_t drawer_id, uint16_t entitlement,
>> + bool dedicated, Error **errp)
>> +{
>> + CpuTopology *smp = s390_topology.smp;
>> + ERRP_GUARD();
>> +
>> + if (socket_id >= smp->sockets) {
>> + error_setg(errp, "Unavailable socket: %d", socket_id);
>> + return;
>> + }
>> + if (book_id >= smp->books) {
>> + error_setg(errp, "Unavailable book: %d", book_id);
>> + return;
>> + }
>> + if (drawer_id >= smp->drawers) {
>> + error_setg(errp, "Unavailable drawer: %d", drawer_id);
>> + return;
>> + }
>> + if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
>> + error_setg(errp, "Unknown entitlement: %d", entitlement);
>> + return;
>> + }
> I think you can delete this, too, there is no way that entitlement is > MAX.
With entitlement being an enum in CPU properties yes.
>
>> + if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
>> + entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
> Without HORIZONTAL you can do != HIGH and save one line, but that is
> cosmetic only.
Right, HORIZONTAL is eliminated during s390_topology_cpu_default()
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-21 10:20 ` Pierre Morel
@ 2023-04-24 15:32 ` Nina Schoetterl-Glausch
2023-04-25 8:45 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-24 15:32 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Fri, 2023-04-21 at 12:20 +0200, Pierre Morel wrote:
> > On 4/20/23 10:59, Nina Schoetterl-Glausch wrote:
> > > > On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> > > > > > The topology information are attributes of the CPU and are
> > > > > > specified during the CPU device creation.
> > > > > >
> > > > > > On hot plug we:
> > > > > > - calculate the default values for the topology for drawers,
> > > > > > books and sockets in the case they are not specified.
> > > > > > - verify the CPU attributes
> > > > > > - check that we have still room on the desired socket
> > > > > >
> > > > > > The possibility to insert a CPU in a mask is dependent on the
> > > > > > number of cores allowed in a socket, a book or a drawer, the
> > > > > > checking is done during the hot plug of the CPU to have an
> > > > > > immediate answer.
> > > > > >
> > > > > > If the complete topology is not specified, the core is added
> > > > > > in the physical topology based on its core ID and it gets
> > > > > > defaults values for the modifier attributes.
> > > > > >
> > > > > > This way, starting QEMU without specifying the topology can
> > > > > > still get some advantage of the CPU topology.
> > > > > >
> > > > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > > > ---
> > > > > > MAINTAINERS | 1 +
> > > > > > include/hw/s390x/cpu-topology.h | 44 +++++
> > > > > > include/hw/s390x/s390-virtio-ccw.h | 1 +
> > > > > > hw/s390x/cpu-topology.c | 282 +++++++++++++++++++++++++++++
> > > > > > hw/s390x/s390-virtio-ccw.c | 22 ++-
> > > > > > hw/s390x/meson.build | 1 +
> > > > > > 6 files changed, 349 insertions(+), 2 deletions(-)
> > > > > > create mode 100644 hw/s390x/cpu-topology.c
> > > > > > };
[...]
> > > > > > struct S390CcwMachineClass {
> > > > > > diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> > > > > > new file mode 100644
> > > > > > index 0000000000..96da67bd7e
> > > > > > --- /dev/null
> > > > > > +++ b/hw/s390x/cpu-topology.c
> > > > [...]
> > > >
> > > > > > +/**
> > > > > > + * s390_topology_cpu_default:
> > > > > > + * @cpu: pointer to a S390CPU
> > > > > > + * @errp: Error pointer
> > > > > > + *
> > > > > > + * Setup the default topology if no attributes are already set.
> > > > > > + * Passing a CPU with some, but not all, attributes set is considered
> > > > > > + * an error.
> > > > > > + *
> > > > > > + * The function calculates the (drawer_id, book_id, socket_id)
> > > > > > + * topology by filling the cores starting from the first socket
> > > > > > + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
> > > > > > + *
> > > > > > + * CPU type and dedication have defaults values set in the
> > > > > > + * s390x_cpu_properties, entitlement must be adjust depending on the
> > > > > > + * dedication.
> > > > > > + */
> > > > > > +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> > > > > > +{
> > > > > > + CpuTopology *smp = s390_topology.smp;
> > > > > > + CPUS390XState *env = &cpu->env;
> > > > > > +
> > > > > > + /* All geometry topology attributes must be set or all unset */
> > > > > > + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
> > > > > > + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
> > > > > > + error_setg(errp,
> > > > > > + "Please define all or none of the topology geometry attributes");
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + /* Check if one of the geometry topology is unset */
> > > > > > + if (env->socket_id < 0) {
> > > > > > + /* Calculate default geometry topology attributes */
> > > > > > + env->socket_id = s390_std_socket(env->core_id, smp);
> > > > > > + env->book_id = s390_std_book(env->core_id, smp);
> > > > > > + env->drawer_id = s390_std_drawer(env->core_id, smp);
> > > > > > + }
> > > > > > +
> > > > > > + /*
> > > > > > + * Even the user can specify the entitlement as horizontal on the
> > > > > > + * command line, qemu will only use env->entitlement during vertical
> > > > > > + * polarization.
> > > > > > + * Medium entitlement is chosen as the default entitlement when the CPU
> > > > > > + * is not dedicated.
> > > > > > + * A dedicated CPU always receives a high entitlement.
> > > > > > + */
> > > > > > + if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
> > > > > > + env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
> > > > > > + if (env->dedicated) {
> > > > > > + env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> > > > > > + } else {
> > > > > > + env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
> > > > > > + }
> > > > > > + }
> > > > As you know, in my opinion there should be not possibility for the user
> > > > to set the entitlement to horizontal and dedicated && != HIGH should be
> > > > rejected as an error.
> > > > If you do this, you can delete this.
> >
> > In the next version with entitlement being an enum it is right.
> >
> > However, deleting this means that the default value for entitlement
> > depends on dedication.
> >
> > If we have only low, medium, high and default for entitlement is medium.
> >
> > If the user specifies the dedication true without specifying entitlement
> > we could force entitlement to high.
> >
> > But we can not distinguish this from the user specifying dedication true
> > with a medium entitlement, which is wrong.
> >
> > So three solution:
> >
> > 1) We ignore what the user say if dedication is specified as true
> >
> > 2) We specify that both dedication and entitlement must be specified if
> > dedication is true
> >
> > 3) We set an impossible default to distinguish default from medium
> > entitlement
> >
> >
> > For me the solution 3 is the best one, it is more flexible for the user.
> >
> > Solution 1 is obviously bad.
> >
> > Solution 2 forces the user to specify entitlement high and only high if
> > it specifies dedication true.
> >
> > AFAIU, you prefer the solution 2, forcing user to specify both
> > dedication and entitlement to suppress a default value in the enum.
> > Why is it bad to have a default value in the enum that we do not use to
> > specify that the value must be calculated?
Yes, I'd prefer solution 2. I don't like adapting the internal state where only
the three values make sense for the user interface.
It also keeps things simple and requires less code.
I also don't think it's a bad thing for the user, as it's not a thing done manually often.
I'm also not a fan of a value being implicitly being changed even though it doesn't look
like it from the command.
However, what I really don't like is the additional state and naming it "horizontal",
not so much the adjustment if dedication is switched to true without an entitlement given.
For the monitor command there is no problem, you currently have:
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement, uint16_t entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
[...]
+ /*
+ * Even user can specify the entitlement as horizontal on the command line,
+ * qemu will only use entitlement during vertical polarization.
+ * Medium entitlement is chosen as the default entitlement when the CPU
+ * is not dedicated.
+ * A dedicated CPU always receives a high entitlement.
+ */
+ if (!has_entitlement || (entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL)) {
+ if (dedicated) {
+ entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
+ }
+ }
So you can just set it if (!has_entitlement).
There is also ways to set the value for cpus defined on the command line, e.g.:
diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properties-system.h
index 0ac327ae60..41a605c5a7 100644
--- a/include/hw/qdev-properties-system.h
+++ b/include/hw/qdev-properties-system.h
@@ -22,6 +22,7 @@ extern const PropertyInfo qdev_prop_audiodev;
extern const PropertyInfo qdev_prop_off_auto_pcibar;
extern const PropertyInfo qdev_prop_pcie_link_speed;
extern const PropertyInfo qdev_prop_pcie_link_width;
+extern const PropertyInfo qdev_prop_cpus390entitlement;
#define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
@@ -73,5 +74,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
#define DEFINE_PROP_UUID_NODEFAULT(_name, _state, _field) \
DEFINE_PROP(_name, _state, _field, qdev_prop_uuid, QemuUUID)
+#define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_cpus390entitlement, int)
+
#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 54541d2230..01308e0b94 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -135,7 +135,7 @@ struct CPUArchState {
int32_t book_id;
int32_t drawer_id;
bool dedicated;
- uint8_t entitlement; /* Used only for vertical polarization */
+ int entitlement; /* Used only for vertical polarization */
uint64_t cpuid;
#endif
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index d42493f630..db5c3d4fe6 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -1143,3 +1143,14 @@ const PropertyInfo qdev_prop_uuid = {
.set = set_uuid,
.set_default_value = set_default_uuid_auto,
};
+
+/* --- s390x cpu topology entitlement --- */
+
+QEMU_BUILD_BUG_ON(sizeof(CpuS390Entitlement) != sizeof(int));
+
+const PropertyInfo qdev_prop_cpus390entitlement = {
+ .name = "CpuS390Entitlement",
+ .enum_table = &CpuS390Entitlement_lookup,
+ .get = qdev_propinfo_get_enum,
+ .set = qdev_propinfo_set_enum,
+};
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index b8a292340c..1b3f5c61ae 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -199,8 +199,7 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
* is not dedicated.
* A dedicated CPU always receives a high entitlement.
*/
- if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
- env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
+ if (env->entitlement < 0) {
if (env->dedicated) {
env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
} else {
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 57165fa3a0..dea50a3e06 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -31,6 +31,7 @@
#include "qapi/qapi-types-machine.h"
#include "sysemu/hw_accel.h"
#include "hw/qdev-properties.h"
+#include "hw/qdev-properties-system.h"
#include "fpu/softfloat-helpers.h"
#include "disas/capstone.h"
#include "sysemu/tcg.h"
@@ -248,6 +249,7 @@ static void s390_cpu_initfn(Object *obj)
cs->exception_index = EXCP_HLT;
#if !defined(CONFIG_USER_ONLY)
+ cpu->env.entitlement = -1;
s390_cpu_init_sysemu(obj);
#endif
}
@@ -264,8 +266,7 @@ static Property s390x_cpu_properties[] = {
DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
- DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
- S390_CPU_ENTITLEMENT__MAX),
+ DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement),
#endif
DEFINE_PROP_END_OF_LIST()
};
There are other ways to achieve the same, you could also
implement get, set and set_default_value so that there is an additional
"auto"/"uninitialized" value that is not in the enum.
If you insist on having an additional state in the enum, name it "auto".
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-24 15:32 ` Nina Schoetterl-Glausch
@ 2023-04-25 8:45 ` Pierre Morel
2023-04-25 9:27 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-25 8:45 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 4/24/23 17:32, Nina Schoetterl-Glausch wrote:
> On Fri, 2023-04-21 at 12:20 +0200, Pierre Morel wrote:
>>> On 4/20/23 10:59, Nina Schoetterl-Glausch wrote:
>>>>> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
[..]
>>> In the next version with entitlement being an enum it is right.
>>>
>>> However, deleting this means that the default value for entitlement
>>> depends on dedication.
>>>
>>> If we have only low, medium, high and default for entitlement is medium.
>>>
>>> If the user specifies the dedication true without specifying entitlement
>>> we could force entitlement to high.
>>>
>>> But we can not distinguish this from the user specifying dedication true
>>> with a medium entitlement, which is wrong.
>>>
>>> So three solution:
>>>
>>> 1) We ignore what the user say if dedication is specified as true
>>>
>>> 2) We specify that both dedication and entitlement must be specified if
>>> dedication is true
>>>
>>> 3) We set an impossible default to distinguish default from medium
>>> entitlement
>>>
>>>
>>> For me the solution 3 is the best one, it is more flexible for the user.
>>>
>>> Solution 1 is obviously bad.
>>>
>>> Solution 2 forces the user to specify entitlement high and only high if
>>> it specifies dedication true.
>>>
>>> AFAIU, you prefer the solution 2, forcing user to specify both
>>> dedication and entitlement to suppress a default value in the enum.
>>> Why is it bad to have a default value in the enum that we do not use to
>>> specify that the value must be calculated?
> Yes, I'd prefer solution 2. I don't like adapting the internal state where only
> the three values make sense for the user interface.
> It also keeps things simple and requires less code.
> I also don't think it's a bad thing for the user, as it's not a thing done manually often.
> I'm also not a fan of a value being implicitly being changed even though it doesn't look
> like it from the command.
>
> However, what I really don't like is the additional state and naming it "horizontal",
No problem to use another name like "auto" as you propose later.
> not so much the adjustment if dedication is switched to true without an entitlement given.
> For the monitor command there is no problem, you currently have:
That is clear, the has_xxx does the job.
[..]
> So you can just set it if (!has_entitlement).
> There is also ways to set the value for cpus defined on the command line, e.g.:
Yes, thanks, I already said I find your proposition to use a
DEFINE_PROP_CPUS390ENTITLEMENT good and will use it.
>
> diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properties-system.h
> index 0ac327ae60..41a605c5a7 100644
> --- a/include/hw/qdev-properties-system.h
> +++ b/include/hw/qdev-properties-system.h
> @@ -22,6 +22,7 @@ extern const PropertyInfo qdev_prop_audiodev;
> extern const PropertyInfo qdev_prop_off_auto_pcibar;
> extern const PropertyInfo qdev_prop_pcie_link_speed;
> extern const PropertyInfo qdev_prop_pcie_link_width;
> +extern const PropertyInfo qdev_prop_cpus390entitlement;
>
> #define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
> DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
> @@ -73,5 +74,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
> #define DEFINE_PROP_UUID_NODEFAULT(_name, _state, _field) \
> DEFINE_PROP(_name, _state, _field, qdev_prop_uuid, QemuUUID)
>
> +#define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f) \
> + DEFINE_PROP(_n, _s, _f, qdev_prop_cpus390entitlement, int)
> +
>
> #endif
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index 54541d2230..01308e0b94 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -135,7 +135,7 @@ struct CPUArchState {
> int32_t book_id;
> int32_t drawer_id;
> bool dedicated;
> - uint8_t entitlement; /* Used only for vertical polarization */
> + int entitlement; /* Used only for vertical polarization */
Isn't it better to use:
+ CpuS390Entitlement entitlement; /* Used only for vertical
polarization */
> uint64_t cpuid;
> #endif
>
> diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
> index d42493f630..db5c3d4fe6 100644
> --- a/hw/core/qdev-properties-system.c
> +++ b/hw/core/qdev-properties-system.c
> @@ -1143,3 +1143,14 @@ const PropertyInfo qdev_prop_uuid = {
> .set = set_uuid,
> .set_default_value = set_default_uuid_auto,
> };
> +
> +/* --- s390x cpu topology entitlement --- */
> +
> +QEMU_BUILD_BUG_ON(sizeof(CpuS390Entitlement) != sizeof(int));
> +
> +const PropertyInfo qdev_prop_cpus390entitlement = {
> + .name = "CpuS390Entitlement",
> + .enum_table = &CpuS390Entitlement_lookup,
> + .get = qdev_propinfo_get_enum,
> + .set = qdev_propinfo_set_enum,
> +};
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index b8a292340c..1b3f5c61ae 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -199,8 +199,7 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> * is not dedicated.
> * A dedicated CPU always receives a high entitlement.
> */
> - if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
> - env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
> + if (env->entitlement < 0) {
Here we can have:
+ if (env->entitlement == S390_CPU_ENTITLEMENT_AUTO) {
...
> if (env->dedicated) {
> env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> } else {
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index 57165fa3a0..dea50a3e06 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -31,6 +31,7 @@
> #include "qapi/qapi-types-machine.h"
> #include "sysemu/hw_accel.h"
> #include "hw/qdev-properties.h"
> +#include "hw/qdev-properties-system.h"
> #include "fpu/softfloat-helpers.h"
> #include "disas/capstone.h"
> #include "sysemu/tcg.h"
> @@ -248,6 +249,7 @@ static void s390_cpu_initfn(Object *obj)
> cs->exception_index = EXCP_HLT;
>
> #if !defined(CONFIG_USER_ONLY)
> + cpu->env.entitlement = -1;
Then we do not need this initialization if here under we define
DEFINE_PROP_CPUS390ENTITLEMENT differently
> s390_cpu_init_sysemu(obj);
> #endif
> }
> @@ -264,8 +266,7 @@ static Property s390x_cpu_properties[] = {
> DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> - DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> - S390_CPU_ENTITLEMENT__MAX),
> + DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement),
+ DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement,
S390_CPU_ENTITLEMENT_AUTO),
> #endif
> DEFINE_PROP_END_OF_LIST()
> };
>
> There are other ways to achieve the same, you could also
> implement get, set and set_default_value so that there is an additional
> "auto"/"uninitialized" value that is not in the enum.
> If you insist on having an additional state in the enum, name it "auto".
Yes, I think it is a better name.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-25 8:45 ` Pierre Morel
@ 2023-04-25 9:27 ` Nina Schoetterl-Glausch
2023-04-25 11:24 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-04-25 9:27 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Tue, 2023-04-25 at 10:45 +0200, Pierre Morel wrote:
> On 4/24/23 17:32, Nina Schoetterl-Glausch wrote:
> > On Fri, 2023-04-21 at 12:20 +0200, Pierre Morel wrote:
> > > > On 4/20/23 10:59, Nina Schoetterl-Glausch wrote:
> > > > > > On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
> [..]
> > > > In the next version with entitlement being an enum it is right.
> > > >
> > > > However, deleting this means that the default value for entitlement
> > > > depends on dedication.
> > > >
> > > > If we have only low, medium, high and default for entitlement is medium.
> > > >
> > > > If the user specifies the dedication true without specifying entitlement
> > > > we could force entitlement to high.
> > > >
> > > > But we can not distinguish this from the user specifying dedication true
> > > > with a medium entitlement, which is wrong.
> > > >
> > > > So three solution:
> > > >
> > > > 1) We ignore what the user say if dedication is specified as true
> > > >
> > > > 2) We specify that both dedication and entitlement must be specified if
> > > > dedication is true
> > > >
> > > > 3) We set an impossible default to distinguish default from medium
> > > > entitlement
> > > >
> > > >
> > > > For me the solution 3 is the best one, it is more flexible for the user.
> > > >
> > > > Solution 1 is obviously bad.
> > > >
> > > > Solution 2 forces the user to specify entitlement high and only high if
> > > > it specifies dedication true.
> > > >
> > > > AFAIU, you prefer the solution 2, forcing user to specify both
> > > > dedication and entitlement to suppress a default value in the enum.
> > > > Why is it bad to have a default value in the enum that we do not use to
> > > > specify that the value must be calculated?
> > Yes, I'd prefer solution 2. I don't like adapting the internal state where only
> > the three values make sense for the user interface.
> > It also keeps things simple and requires less code.
> > I also don't think it's a bad thing for the user, as it's not a thing done manually often.
> > I'm also not a fan of a value being implicitly being changed even though it doesn't look
> > like it from the command.
> >
> > However, what I really don't like is the additional state and naming it "horizontal",
>
>
> No problem to use another name like "auto" as you propose later.
>
>
> > not so much the adjustment if dedication is switched to true without an entitlement given.
> > For the monitor command there is no problem, you currently have:
>
>
> That is clear, the has_xxx does the job.
>
> [..]
>
>
> > So you can just set it if (!has_entitlement).
> > There is also ways to set the value for cpus defined on the command line, e.g.:
>
>
> Yes, thanks, I already said I find your proposition to use a
> DEFINE_PROP_CPUS390ENTITLEMENT good and will use it.
>
>
> >
> > diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properties-system.h
> > index 0ac327ae60..41a605c5a7 100644
> > --- a/include/hw/qdev-properties-system.h
> > +++ b/include/hw/qdev-properties-system.h
> > @@ -22,6 +22,7 @@ extern const PropertyInfo qdev_prop_audiodev;
> > extern const PropertyInfo qdev_prop_off_auto_pcibar;
> > extern const PropertyInfo qdev_prop_pcie_link_speed;
> > extern const PropertyInfo qdev_prop_pcie_link_width;
> > +extern const PropertyInfo qdev_prop_cpus390entitlement;
> >
> > #define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
> > DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
> > @@ -73,5 +74,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
> > #define DEFINE_PROP_UUID_NODEFAULT(_name, _state, _field) \
> > DEFINE_PROP(_name, _state, _field, qdev_prop_uuid, QemuUUID)
> >
> > +#define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f) \
> > + DEFINE_PROP(_n, _s, _f, qdev_prop_cpus390entitlement, int)
> > +
> >
> > #endif
> > diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> > index 54541d2230..01308e0b94 100644
> > --- a/target/s390x/cpu.h
> > +++ b/target/s390x/cpu.h
> > @@ -135,7 +135,7 @@ struct CPUArchState {
> > int32_t book_id;
> > int32_t drawer_id;
> > bool dedicated;
> > - uint8_t entitlement; /* Used only for vertical polarization */
> > + int entitlement; /* Used only for vertical polarization */
>
>
> Isn't it better to use:
>
> + CpuS390Entitlement entitlement; /* Used only for vertical
> polarization */
>
>
> > uint64_t cpuid;
> > #endif
> >
> > diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
> > index d42493f630..db5c3d4fe6 100644
> > --- a/hw/core/qdev-properties-system.c
> > +++ b/hw/core/qdev-properties-system.c
> > @@ -1143,3 +1143,14 @@ const PropertyInfo qdev_prop_uuid = {
> > .set = set_uuid,
> > .set_default_value = set_default_uuid_auto,
> > };
> > +
> > +/* --- s390x cpu topology entitlement --- */
> > +
> > +QEMU_BUILD_BUG_ON(sizeof(CpuS390Entitlement) != sizeof(int));
> > +
> > +const PropertyInfo qdev_prop_cpus390entitlement = {
> > + .name = "CpuS390Entitlement",
> > + .enum_table = &CpuS390Entitlement_lookup,
> > + .get = qdev_propinfo_get_enum,
> > + .set = qdev_propinfo_set_enum,
> > +};
> > diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> > index b8a292340c..1b3f5c61ae 100644
> > --- a/hw/s390x/cpu-topology.c
> > +++ b/hw/s390x/cpu-topology.c
> > @@ -199,8 +199,7 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> > * is not dedicated.
> > * A dedicated CPU always receives a high entitlement.
> > */
> > - if (env->entitlement >= S390_CPU_ENTITLEMENT__MAX ||
> > - env->entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL) {
> > + if (env->entitlement < 0) {
>
>
> Here we can have:
>
> + if (env->entitlement == S390_CPU_ENTITLEMENT_AUTO) {
> ...
>
> > if (env->dedicated) {
> > env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> > } else {
> > diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> > index 57165fa3a0..dea50a3e06 100644
> > --- a/target/s390x/cpu.c
> > +++ b/target/s390x/cpu.c
> > @@ -31,6 +31,7 @@
> > #include "qapi/qapi-types-machine.h"
> > #include "sysemu/hw_accel.h"
> > #include "hw/qdev-properties.h"
> > +#include "hw/qdev-properties-system.h"
> > #include "fpu/softfloat-helpers.h"
> > #include "disas/capstone.h"
> > #include "sysemu/tcg.h"
> > @@ -248,6 +249,7 @@ static void s390_cpu_initfn(Object *obj)
> > cs->exception_index = EXCP_HLT;
> >
> > #if !defined(CONFIG_USER_ONLY)
> > + cpu->env.entitlement = -1;
>
>
> Then we do not need this initialization if here under we define
> DEFINE_PROP_CPUS390ENTITLEMENT differently
>
>
> > s390_cpu_init_sysemu(obj);
> > #endif
> > }
> > @@ -264,8 +266,7 @@ static Property s390x_cpu_properties[] = {
> > DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
> > DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
> > DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
> > - DEFINE_PROP_UINT8("entitlement", S390CPU, env.entitlement,
> > - S390_CPU_ENTITLEMENT__MAX),
> > + DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement),
>
>
> + DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement,
> S390_CPU_ENTITLEMENT_AUTO),
>
> > #endif
> > DEFINE_PROP_END_OF_LIST()
> > };
> >
> > There are other ways to achieve the same, you could also
> > implement get, set and set_default_value so that there is an additional
> > "auto"/"uninitialized" value that is not in the enum.
> > If you insist on having an additional state in the enum, name it "auto".
>
> Yes, I think it is a better name.
IMO using entitlement=auto doesn't make too much sense with the set-cpu-topology command,
because you can just leave if off or specify the entitlement you want directly.
So there is no actual need to have a user visible auto value and no need to have it in the enum.
Then the only problem is adjusting the entitlement when doing dedicated=on on the command line.
(If you want that)
So with my proposal there are only the low, medium and high values in the enum.
In order to set the entitlement automatically when using the command line I initialize
the entitlement to -1, so we later know if it has been set via the command line or not.
But you cannot set -1 via the property because qdev_propinfo_get_enum expects a string,
which is why I do it in s390_cpu_initfn.
I'm not sure if you can define entitlement as CpuS390Entitlement.
I think I changed it to int when I was exploring different solutions
and had to change it because of a type check. But what I proposed above doesn't cause the same issue.
DEFINE_PROP_CPUS390ENTITLEMENT could then also use CpuS390Entitlement.
So there are three possible solutions now:
1. My proposal above, which as automatic adjustment, but only the three required values in the enum.
2. Don't do automatic adjustment, three enum values.
3. Automatic adjustment with auto value in the enum.
I still favor 2. but the other ones aren't terrible.
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug
2023-04-25 9:27 ` Nina Schoetterl-Glausch
@ 2023-04-25 11:24 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-25 11:24 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 4/25/23 11:27, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-04-25 at 10:45 +0200, Pierre Morel wrote:
>> On 4/24/23 17:32, Nina Schoetterl-Glausch wrote:
>>> On Fri, 2023-04-21 at 12:20 +0200, Pierre Morel wrote:
>>>>> On 4/20/23 10:59, Nina Schoetterl-Glausch wrote:
>>>>>>> On Mon, 2023-04-03 at 18:28 +0200, Pierre Morel wrote:
[..]
>>> #endif
>>> DEFINE_PROP_END_OF_LIST()
>>> };
>>>
>>> There are other ways to achieve the same, you could also
>>> implement get, set and set_default_value so that there is an additional
>>> "auto"/"uninitialized" value that is not in the enum.
>>> If you insist on having an additional state in the enum, name it "auto".
>> Yes, I think it is a better name.
> IMO using entitlement=auto doesn't make too much sense with the set-cpu-topology command,
> because you can just leave if off or specify the entitlement you want directly.
> So there is no actual need to have a user visible auto value and no need to have it in the enum.
This value is only usable but not required on input and is never
displayed by the qapi.
> Then the only problem is adjusting the entitlement when doing dedicated=on on the command line.
> (If you want that)
Even the exact usage of dedication depends on the administration entity
it will give the guest the
knowledge of something like: "A real CPU is dedicated to this vCPU".
The people using the qapi interface or using QEMU hotplug can easily
understand
the concept without going deeper with entitlement which once implemented
will be,
quite more complex to deal with.
> So with my proposal there are only the low, medium and high values in the enum.
> In order to set the entitlement automatically when using the command line I initialize
> the entitlement to -1, so we later know if it has been set via the command line or not.
> But you cannot set -1 via the property because qdev_propinfo_get_enum expects a string,
> which is why I do it in s390_cpu_initfn.
>
> I'm not sure if you can define entitlement as CpuS390Entitlement.
> I think I changed it to int when I was exploring different solutions
> and had to change it because of a type check. But what I proposed above doesn't cause the same issue.
> DEFINE_PROP_CPUS390ENTITLEMENT could then also use CpuS390Entitlement.
I did not have the problem by using CpuS390Entitlement.
>
> So there are three possible solutions now:
> 1. My proposal above, which as automatic adjustment, but only the three required values in the enum.
> 2. Don't do automatic adjustment, three enum values.
> 3. Automatic adjustment with auto value in the enum.
>
> I still favor 2. but the other ones aren't terrible.
Thank you to accept the solution 3, I understand your objections, but,
I really think that there is no need to bother the user with entitlement
while it will not get used in QEMU/KVM until the Linux scheduler is modified
for the host and for the guest to handle it.
Regards,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 03/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
2023-04-03 16:28 ` [PATCH v19 01/21] s390x/cpu topology: add s390 specifics to CPU topology Pierre Morel
2023-04-03 16:28 ` [PATCH v19 02/21] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 04/21] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
` (17 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
MAINTAINERS | 1 +
include/hw/s390x/cpu-topology.h | 21 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 72 ++++++++
hw/s390x/cpu-topology.c | 12 ++
target/s390x/kvm/cpu_topology.c | 311 ++++++++++++++++++++++++++++++++
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/meson.build | 3 +-
8 files changed, 424 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/cpu_topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index bb7b34d0d8..de9052f753 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1659,6 +1659,7 @@ M: Pierre Morel <pmorel@linux.ibm.com>
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
+F: target/s390x/kvm/cpu_topology.c
X86 Machines
------------
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 6326cfeff8..7da9277a9a 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -18,8 +18,29 @@
#define S390_TOPOLOGY_CPU_IFL 0x03
+typedef union s390_topology_id {
+ uint64_t id;
+ struct {
+ uint8_t sentinel;
+ uint8_t drawer;
+ uint8_t book;
+ uint8_t socket;
+ uint8_t dedicated;
+ uint8_t entitlement;
+ uint8_t type;
+ uint8_t origin;
+ };
+} s390_topology_id;
+
+typedef struct S390TopologyEntry {
+ QTAILQ_ENTRY(S390TopologyEntry) next;
+ s390_topology_id id;
+ uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+ QTAILQ_HEAD(, S390TopologyEntry) list;
CpuTopology *smp;
} S390Topology;
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index d3ade40a5a..712fd68123 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST 2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index f2b2a38fe7..d5b9b79a3f 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -560,6 +560,25 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+ uint8_t reserved0[2];
+ uint16_t length;
+ uint8_t mag[S390_TOPOLOGY_MAG];
+ uint8_t reserved1;
+ uint8_t mnest;
+ uint32_t reserved2;
+ char tle[];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -567,9 +586,62 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+ SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
+} SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+ uint8_t nl;
+ uint8_t reserved0[3];
+#define SYSIB_TLE_POLARITY_MASK 0x03
+#define SYSIB_TLE_DEDICATED 0x04
+ uint8_t flags;
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
+} SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
#define ASCE_SUBSPACE 0x200 /* subspace group control */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 96da67bd7e..1c6d23eda6 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,12 +24,16 @@
* .cores_per_socket: tracks information on the count of cores
* per socket.
* .smp: keeps track of the machine topology.
+ * .list: queue the topology entries inside which
+ * we keep the information on the CPU topology.
+ * .polarization: the current subsystem polarization
*
*/
S390Topology s390_topology = {
/* will be initialized after the cpu model is realized */
.cores_per_socket = NULL,
.smp = NULL,
+ .list = QTAILQ_HEAD_INITIALIZER(s390_topology.list),
};
/**
@@ -64,14 +68,22 @@ bool s390_has_topology(void)
* Allocate an array to keep the count of cores per socket.
* The index of the array starts at socket 0 from book 0 and
* drawer 0 up to the maximum allowed by the machine topology.
+ *
+ * Insert a sentinel entry with a non null value.
+ * This entry will never be free.
*/
static void s390_topology_init(MachineState *ms)
{
CpuTopology *smp = &ms->smp;
+ S390TopologyEntry *entry;
s390_topology.smp = smp;
s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
smp->books * smp->drawers);
+
+ entry = g_malloc0(sizeof(S390TopologyEntry));
+ entry->id.sentinel = 0xff;
+ QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
/**
diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c
new file mode 100644
index 0000000000..0936affbc1
--- /dev/null
+++ b/target/s390x/kvm/cpu_topology.c
@@ -0,0 +1,311 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/sclp.h"
+#include "hw/s390x/cpu-topology.h"
+
+/**
+ * fill_container:
+ * @p: The address of the container TLE to fill
+ * @level: The level of nesting for this container
+ * @id: The container receives a uniq ID inside its own container
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_container(char *p, int level, int id)
+{
+ SysIBTl_container *tle = (SysIBTl_container *)p;
+
+ tle->nl = level;
+ tle->id = id;
+ return p + sizeof(*tle);
+}
+
+/**
+ * fill_tle_cpu:
+ * @p: The address of the CPU TLE to fill
+ * @entry: a pointer to the S390TopologyEntry defining this
+ * CPU container.
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
+{
+ SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+ s390_topology_id topology_id = entry->id;
+
+ tle->nl = 0;
+ if (topology_id.dedicated) {
+ tle->flags = SYSIB_TLE_DEDICATED;
+ }
+ tle->flags |= topology_id.entitlement;
+ tle->type = topology_id.type;
+ tle->origin = cpu_to_be16(topology_id.origin * 64);
+ tle->mask = cpu_to_be64(entry->mask);
+ return p + sizeof(*tle);
+}
+
+/*
+ * Macro to check that the size of data after increment
+ * will not get bigger than the size of the SysIB.
+ */
+#define SYSIB_GUARD(data, x) do { \
+ data += x; \
+ if (data > sizeof(SysIB)) { \
+ return 0; \
+ } \
+ } while (0)
+
+/**
+ * stsi_topology_fill_sysib:
+ * @p: A pointer to the position of the first TLE
+ * @level: The nested level wanted by the guest
+ *
+ * Fill the SYSIB with the topology information as described in
+ * the PoP, nesting containers as appropriate, with the maximum
+ * nesting limited by @level.
+ *
+ * Return value:
+ * On success: the size of the SysIB_15x after being filled with TLE.
+ * On error: 0 in the case we would overrun the end of the SysIB.
+ */
+static int stsi_topology_fill_sysib(char *p, int level)
+{
+ S390TopologyEntry *entry;
+ int last_drawer = -1;
+ int last_book = -1;
+ int last_socket = -1;
+ int drawer_id = 0;
+ int book_id = 0;
+ int socket_id = 0;
+ int n = sizeof(SysIB_151x);
+
+ QTAILQ_FOREACH(entry, &s390_topology.list, next) {
+ bool drawer_change = last_drawer != entry->id.drawer;
+ bool book_change = drawer_change || last_book != entry->id.book;
+ bool socket_change = book_change || last_socket != entry->id.socket;
+
+ /* If we reach the sentinel get out */
+ if (entry->id.sentinel) {
+ break;
+ }
+
+ if (level > 3 && drawer_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 3, drawer_id++);
+ book_id = 0;
+ }
+ if (level > 2 && book_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 2, book_id++);
+ socket_id = 0;
+ }
+ if (socket_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 1, socket_id++);
+ }
+
+ SYSIB_GUARD(n, sizeof(SysIBTl_cpu));
+ p = fill_tle_cpu(p, entry);
+ last_drawer = entry->id.drawer;
+ last_book = entry->id.book;
+ last_socket = entry->id.socket;
+ }
+
+ return n;
+}
+
+/**
+ * setup_stsi:
+ * sysib: pointer to a SysIB to be filled with SysIB_151x data
+ * level: Nested level specified by the guest
+ *
+ * Setup the SYSIB for STSI 15.1, the header as well as the description
+ * of the topology.
+ */
+static int setup_stsi(SysIB_151x *sysib, int level)
+{
+ sysib->mnest = level;
+ switch (level) {
+ case 4:
+ sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 3:
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
+ current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 2:
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
+ current_machine->smp.books *
+ current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ }
+
+ return stsi_topology_fill_sysib(sysib->tle, level);
+}
+
+/**
+ * s390_topology_add_cpu_to_entry:
+ * @entry: Topology entry to setup
+ * @cpu: the S390CPU to add
+ *
+ * Set the core bit inside the topology mask and
+ * increments the number of cores for the socket.
+ */
+static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
+ S390CPU *cpu)
+{
+ set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
+}
+
+/**
+ * s390_topology_from_cpu:
+ * @cpu: The S390CPU
+ *
+ * Initialize the topology id from the CPU environment.
+ */
+static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
+{
+ struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
+ s390_topology_id topology_id = {0};
+
+ topology_id.drawer = cpu->env.drawer_id;
+ topology_id.book = cpu->env.book_id;
+ topology_id.socket = cpu->env.socket_id;
+ topology_id.origin = cpu->env.core_id / 64;
+ topology_id.type = S390_TOPOLOGY_CPU_IFL;
+ topology_id.dedicated = cpu->env.dedicated;
+
+ if (s390ms->vertical_polarization) {
+ /*
+ * Vertical polarization with dedicated CPU implies
+ * vertical high entitlement.
+ */
+ if (topology_id.dedicated) {
+ topology_id.entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ topology_id.entitlement = cpu->env.entitlement;
+ }
+ }
+
+ return topology_id;
+}
+
+/**
+ * s390_topology_insert:
+ * @cpu: s390CPU insert.
+ *
+ * Parse the topology list to find if the entry already
+ * exist and add the core in it.
+ * If it does not exist, allocate a new entry and insert
+ * it in the queue from lower id to greater id.
+ */
+static void s390_topology_insert(S390CPU *cpu)
+{
+ s390_topology_id id = s390_topology_from_cpu(cpu);
+ S390TopologyEntry *entry = NULL;
+ S390TopologyEntry *tmp = NULL;
+
+ QTAILQ_FOREACH(tmp, &s390_topology.list, next) {
+ if (id.id == tmp->id.id) {
+ s390_topology_add_cpu_to_entry(tmp, cpu);
+ return;
+ } else if (id.id < tmp->id.id) {
+ entry = g_malloc0(sizeof(S390TopologyEntry));
+ entry->id.id = id.id;
+ s390_topology_add_cpu_to_entry(entry, cpu);
+ QTAILQ_INSERT_BEFORE(tmp, entry, next);
+ return;
+ }
+ }
+}
+
+/**
+ * s390_topology_fill_list_sorted:
+ *
+ * Loop over all CPU and insert it at the right place
+ * inside the TLE entry list.
+ * Fill the S390Topology list with entries according to the order
+ * specified by the PoP.
+ */
+static void s390_topology_fill_list_sorted(void)
+{
+ CPUState *cs;
+
+ CPU_FOREACH(cs) {
+ s390_topology_insert(S390_CPU(cs));
+ }
+}
+
+/**
+ * s390_topology_empty_list:
+ *
+ * Clear all entries in the S390Topology list except the sentinel.
+ */
+static void s390_topology_empty_list(void)
+{
+ S390TopologyEntry *entry = NULL;
+ S390TopologyEntry *tmp = NULL;
+
+ QTAILQ_FOREACH_SAFE(entry, &s390_topology.list, next, tmp) {
+ if (!entry->id.sentinel) {
+ QTAILQ_REMOVE(&s390_topology.list, entry, next);
+ g_free(entry);
+ }
+ }
+}
+
+/**
+ * insert_stsi_15_1_x:
+ * cpu: the CPU doing the call for which we set CC
+ * sel2: the selector 2, containing the nested level
+ * addr: Guest logical address of the guest SysIB
+ * ar: the access register number
+ *
+ * Emulate STSI 15.1.x, that is, perform all necessary checks and
+ * fill the SYSIB.
+ * In case the topology description is too long to fit into the SYSIB,
+ * set CC=3 and abort without writing the SYSIB.
+ */
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar)
+{
+ SysIB sysib = {0};
+ int length;
+
+ if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ s390_topology_fill_list_sorted();
+
+ length = setup_stsi(&sysib.sysib_151x, sel2);
+
+ if (!length) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ sysib.sysib_151x.length = cpu_to_be16(length);
+ s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length);
+ setcc(cpu, 0);
+
+ s390_topology_empty_list();
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 3ac7ec9acf..5ea358cbb0 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu)
if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
return 0;
}
- /* Only sysib 3.2.2 needs post-handling for now. */
insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+ run->s390_stsi.ar);
+ return 0;
default:
return 0;
}
diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build
index aef52b6686..5daa5c6033 100644
--- a/target/s390x/kvm/meson.build
+++ b/target/s390x/kvm/meson.build
@@ -1,6 +1,7 @@
s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
- 'kvm.c'
+ 'kvm.c',
+ 'cpu_topology.c'
), if_false: files(
'stubs.c'
))
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 04/21] s390x/sclp: reporting the maximum nested topology entries
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (2 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 03/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 05/21] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
` (16 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 712fd68123..902252b319 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST 2
+#define SCLP_READ_SCP_INFO_MNEST 4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
- uint8_t _reserved1[16 - 11]; /* 11-15 */
+ uint8_t _reserved1[15 - 11]; /* 11-14 */
+ uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu; /* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+ if (s390_has_topology()) {
+ read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+ }
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, &cpu_count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 05/21] s390x/cpu topology: resetting the Topology-Change-Report
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (3 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 04/21] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction Pierre Morel
` (15 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h | 1 +
hw/s390x/cpu-topology.c | 11 +++++++++++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +++++++++++++
target/s390x/kvm/kvm.c | 17 +++++++++++++++++
7 files changed, 47 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 7da9277a9a..9e30744295 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -59,6 +59,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,
extern S390Topology s390_topology;
int s390_socket_nb(S390CPU *cpu);
+void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index d5b9b79a3f..54541d2230 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 1c6d23eda6..1d672d4d81 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -86,6 +86,17 @@ static void s390_topology_init(MachineState *ms)
QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * to reset the kernel Modified Topology Change Record.
+ */
+void s390_topology_reset(void)
+{
+ s390_cpu_topology_set_changed(false);
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 9df60ac447..7e6d88e23c 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -123,6 +123,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+ if (s390_has_topology()) {
+ s390_topology_reset();
+ }
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 948e4bd3e0..9d41acfe1a 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_set_changed(bool changed)
+{
+ int ret;
+
+ if (kvm_enabled()) {
+ ret = kvm_s390_topology_set_mtcr(changed);
+ if (ret) {
+ error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+ }
+ }
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 5ea358cbb0..bc953151ce 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+ struct kvm_device_attr attribute = {
+ .group = KVM_S390_VM_CPU_TOPOLOGY,
+ .attr = attr,
+ };
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ return 0;
+ }
+ if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+ return -ENOTSUP;
+ }
+
+ return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute);
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (4 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 05/21] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-04 7:41 ` Cédric Le Goater
2023-04-03 16:28 ` [PATCH v19 07/21] target/s390x/cpu topology: activate CPU topology Pierre Morel
` (14 subsequent siblings)
20 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervisor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
include/hw/s390x/s390-virtio-ccw.h | 6 ++++
hw/s390x/cpu-topology.c | 56 ++++++++++++++++++++++++++++--
target/s390x/kvm/kvm.c | 11 ++++++
3 files changed, 71 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index ea10a6c6e1..9aa9f48bd0 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -31,6 +31,12 @@ struct S390CcwMachineState {
bool vertical_polarization;
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 1d672d4d81..eec6c9a896 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -26,8 +26,6 @@
* .smp: keeps track of the machine topology.
* .list: queue the topology entries inside which
* we keep the information on the CPU topology.
- * .polarization: the current subsystem polarization
- *
*/
S390Topology s390_topology = {
/* will be initialized after the cpu model is realized */
@@ -86,6 +84,57 @@ static void s390_topology_init(MachineState *ms)
QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+ struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ int fc = reg & S390_TOPO_FC_MASK;
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ s390_program_interrupt(env, PGM_OPERATION, ra);
+ return;
+ }
+
+ if (env->psw.mask & PSW_MASK_PSTATE) {
+ s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+ return;
+ }
+
+ if (reg & ~S390_TOPO_FC_MASK) {
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ return;
+ }
+
+ switch (fc) {
+ case S390_CPU_POLARIZATION_VERTICAL:
+ case S390_CPU_POLARIZATION_HORIZONTAL:
+ if (s390ms->vertical_polarization == !!fc) {
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ setcc(cpu, 2);
+ } else {
+ s390ms->vertical_polarization = !!fc;
+ s390_cpu_topology_set_changed(true);
+ setcc(cpu, 0);
+ }
+ break;
+ default:
+ /* Note that fc == 2 is interpreted by the SIE */
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ }
+}
+
/**
* s390_topology_reset:
*
@@ -94,7 +143,10 @@ static void s390_topology_init(MachineState *ms)
*/
void s390_topology_reset(void)
{
+ struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
+
s390_cpu_topology_set_changed(false);
+ s390ms->vertical_polarization = false;
}
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index bc953151ce..fb63be41b7 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -96,6 +96,7 @@
#define PRIV_B9_EQBS 0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+ uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+ s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+ case PRIV_B9_PTF:
+ kvm_handle_ptf(cpu, run);
+ break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction
2023-04-03 16:28 ` [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction Pierre Morel
@ 2023-04-04 7:41 ` Cédric Le Goater
2023-04-04 9:07 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 7:41 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> When the host supports the CPU topology facility, the PTF
> instruction with function code 2 is interpreted by the SIE,
> provided that the userland hypervisor activates the interpretation
> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>
> The PTF instructions with function code 0 and 1 are intercepted
> and must be emulated by the userland hypervisor.
>
> During RESET all CPU of the configuration are placed in
> horizontal polarity.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> include/hw/s390x/s390-virtio-ccw.h | 6 ++++
> hw/s390x/cpu-topology.c | 56 ++++++++++++++++++++++++++++--
> target/s390x/kvm/kvm.c | 11 ++++++
> 3 files changed, 71 insertions(+), 2 deletions(-)
>
> diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
> index ea10a6c6e1..9aa9f48bd0 100644
> --- a/include/hw/s390x/s390-virtio-ccw.h
> +++ b/include/hw/s390x/s390-virtio-ccw.h
> @@ -31,6 +31,12 @@ struct S390CcwMachineState {
> bool vertical_polarization;
> };
>
> +#define S390_PTF_REASON_NONE (0x00 << 8)
> +#define S390_PTF_REASON_DONE (0x01 << 8)
> +#define S390_PTF_REASON_BUSY (0x02 << 8)
> +#define S390_TOPO_FC_MASK 0xffUL
> +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
> +
> struct S390CcwMachineClass {
> /*< private >*/
> MachineClass parent_class;
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 1d672d4d81..eec6c9a896 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -26,8 +26,6 @@
> * .smp: keeps track of the machine topology.
> * .list: queue the topology entries inside which
> * we keep the information on the CPU topology.
> - * .polarization: the current subsystem polarization
> - *
Please remove from patch 3 instead.
> */
> S390Topology s390_topology = {
> /* will be initialized after the cpu model is realized */
> @@ -86,6 +84,57 @@ static void s390_topology_init(MachineState *ms)
> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> }
>
> +/*
> + * s390_handle_ptf:
> + *
> + * @register 1: contains the function code
> + *
> + * Function codes 0 (horizontal) and 1 (vertical) define the CPU
> + * polarization requested by the guest.
> + *
> + * Function code 2 is handling topology changes and is interpreted
> + * by the SIE.
> + */
> +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
> +{
> + struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
> + CPUS390XState *env = &cpu->env;
> + uint64_t reg = env->regs[r1];
> + int fc = reg & S390_TOPO_FC_MASK;
> +
> + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
> + s390_program_interrupt(env, PGM_OPERATION, ra);
> + return;
> + }
> +
> + if (env->psw.mask & PSW_MASK_PSTATE) {
> + s390_program_interrupt(env, PGM_PRIVILEGED, ra);
> + return;
> + }
> +
> + if (reg & ~S390_TOPO_FC_MASK) {
> + s390_program_interrupt(env, PGM_SPECIFICATION, ra);
> + return;
> + }
> +
> + switch (fc) {
> + case S390_CPU_POLARIZATION_VERTICAL:
> + case S390_CPU_POLARIZATION_HORIZONTAL:
> + if (s390ms->vertical_polarization == !!fc) {
> + env->regs[r1] |= S390_PTF_REASON_DONE;
> + setcc(cpu, 2);
> + } else {
> + s390ms->vertical_polarization = !!fc;
> + s390_cpu_topology_set_changed(true);
> + setcc(cpu, 0);
> + }
> + break;
> + default:
> + /* Note that fc == 2 is interpreted by the SIE */
> + s390_program_interrupt(env, PGM_SPECIFICATION, ra);
> + }
> +}
> +
> /**
> * s390_topology_reset:
> *
> @@ -94,7 +143,10 @@ static void s390_topology_init(MachineState *ms)
> */
> void s390_topology_reset(void)
> {
> + struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
> +
> s390_cpu_topology_set_changed(false);
> + s390ms->vertical_polarization = false;
> }
>
> /**
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index bc953151ce..fb63be41b7 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -96,6 +96,7 @@
>
> #define PRIV_B9_EQBS 0x9c
> #define PRIV_B9_CLP 0xa0
> +#define PRIV_B9_PTF 0xa2
> #define PRIV_B9_PCISTG 0xd0
> #define PRIV_B9_PCILG 0xd2
> #define PRIV_B9_RPCIT 0xd3
> @@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
> }
> }
>
> +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
> +{
> + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
> +
> + s390_handle_ptf(cpu, r1, RA_IGNORED);
> +}
> +
> static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
> {
> int r = 0;
> @@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
> case PRIV_B9_RPCIT:
> r = kvm_rpcit_service_call(cpu, run);
> break;
> + case PRIV_B9_PTF:
> + kvm_handle_ptf(cpu, run);
> + break;
> case PRIV_B9_EQBS:
> /* just inject exception */
> r = -1;
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction
2023-04-04 7:41 ` Cédric Le Goater
@ 2023-04-04 9:07 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 9:07 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 09:41, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> When the host supports the CPU topology facility, the PTF
>> instruction with function code 2 is interpreted by the SIE,
>> provided that the userland hypervisor activates the interpretation
>> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>>
>> The PTF instructions with function code 0 and 1 are intercepted
>> and must be emulated by the userland hypervisor.
>>
>> During RESET all CPU of the configuration are placed in
>> horizontal polarity.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> include/hw/s390x/s390-virtio-ccw.h | 6 ++++
>> hw/s390x/cpu-topology.c | 56 ++++++++++++++++++++++++++++--
>> target/s390x/kvm/kvm.c | 11 ++++++
>> 3 files changed, 71 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/hw/s390x/s390-virtio-ccw.h
>> b/include/hw/s390x/s390-virtio-ccw.h
>> index ea10a6c6e1..9aa9f48bd0 100644
>> --- a/include/hw/s390x/s390-virtio-ccw.h
>> +++ b/include/hw/s390x/s390-virtio-ccw.h
>> @@ -31,6 +31,12 @@ struct S390CcwMachineState {
>> bool vertical_polarization;
>> };
>> +#define S390_PTF_REASON_NONE (0x00 << 8)
>> +#define S390_PTF_REASON_DONE (0x01 << 8)
>> +#define S390_PTF_REASON_BUSY (0x02 << 8)
>> +#define S390_TOPO_FC_MASK 0xffUL
>> +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
>> +
>> struct S390CcwMachineClass {
>> /*< private >*/
>> MachineClass parent_class;
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index 1d672d4d81..eec6c9a896 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -26,8 +26,6 @@
>> * .smp: keeps track of the machine topology.
>> * .list: queue the topology entries inside which
>> * we keep the information on the CPU topology.
>> - * .polarization: the current subsystem polarization
>> - *
>
> Please remove from patch 3 instead.
Yes of course
Thanks,
Pierre
>
>> */
>> S390Topology s390_topology = {
>> /* will be initialized after the cpu model is realized */
>> @@ -86,6 +84,57 @@ static void s390_topology_init(MachineState *ms)
>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>> }
>> +/*
>> + * s390_handle_ptf:
>> + *
>> + * @register 1: contains the function code
>> + *
>> + * Function codes 0 (horizontal) and 1 (vertical) define the CPU
>> + * polarization requested by the guest.
>> + *
>> + * Function code 2 is handling topology changes and is interpreted
>> + * by the SIE.
>> + */
>> +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
>> +{
>> + struct S390CcwMachineState *s390ms =
>> S390_CCW_MACHINE(current_machine);
>> + CPUS390XState *env = &cpu->env;
>> + uint64_t reg = env->regs[r1];
>> + int fc = reg & S390_TOPO_FC_MASK;
>> +
>> + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
>> + s390_program_interrupt(env, PGM_OPERATION, ra);
>> + return;
>> + }
>> +
>> + if (env->psw.mask & PSW_MASK_PSTATE) {
>> + s390_program_interrupt(env, PGM_PRIVILEGED, ra);
>> + return;
>> + }
>> +
>> + if (reg & ~S390_TOPO_FC_MASK) {
>> + s390_program_interrupt(env, PGM_SPECIFICATION, ra);
>> + return;
>> + }
>> +
>> + switch (fc) {
>> + case S390_CPU_POLARIZATION_VERTICAL:
>> + case S390_CPU_POLARIZATION_HORIZONTAL:
>> + if (s390ms->vertical_polarization == !!fc) {
>> + env->regs[r1] |= S390_PTF_REASON_DONE;
>> + setcc(cpu, 2);
>> + } else {
>> + s390ms->vertical_polarization = !!fc;
>> + s390_cpu_topology_set_changed(true);
>> + setcc(cpu, 0);
>> + }
>> + break;
>> + default:
>> + /* Note that fc == 2 is interpreted by the SIE */
>> + s390_program_interrupt(env, PGM_SPECIFICATION, ra);
>> + }
>> +}
>> +
>> /**
>> * s390_topology_reset:
>> *
>> @@ -94,7 +143,10 @@ static void s390_topology_init(MachineState *ms)
>> */
>> void s390_topology_reset(void)
>> {
>> + struct S390CcwMachineState *s390ms =
>> S390_CCW_MACHINE(current_machine);
>> +
>> s390_cpu_topology_set_changed(false);
>> + s390ms->vertical_polarization = false;
>> }
>> /**
>> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
>> index bc953151ce..fb63be41b7 100644
>> --- a/target/s390x/kvm/kvm.c
>> +++ b/target/s390x/kvm/kvm.c
>> @@ -96,6 +96,7 @@
>> #define PRIV_B9_EQBS 0x9c
>> #define PRIV_B9_CLP 0xa0
>> +#define PRIV_B9_PTF 0xa2
>> #define PRIV_B9_PCISTG 0xd0
>> #define PRIV_B9_PCILG 0xd2
>> #define PRIV_B9_RPCIT 0xd3
>> @@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU
>> *cpu, struct kvm_run *run)
>> }
>> }
>> +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
>> +{
>> + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
>> +
>> + s390_handle_ptf(cpu, r1, RA_IGNORED);
>> +}
>> +
>> static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
>> {
>> int r = 0;
>> @@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct
>> kvm_run *run, uint8_t ipa1)
>> case PRIV_B9_RPCIT:
>> r = kvm_rpcit_service_call(cpu, run);
>> break;
>> + case PRIV_B9_PTF:
>> + kvm_handle_ptf(cpu, run);
>> + break;
>> case PRIV_B9_EQBS:
>> /* just inject exception */
>> r = -1;
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 07/21] target/s390x/cpu topology: activate CPU topology
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (5 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 06/21] s390x/cpu topology: interception of PTF instruction Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 08/21] qapi/s390x/cpu topology: set-cpu-topology qmp command Pierre Morel
` (13 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 9 +++++++++
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index eec6c9a896..35426e23f0 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -54,7 +54,7 @@ int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
- return false;
+ return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 065ec6d66c..aca2c5c96b 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -254,6 +254,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+ case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index fb63be41b7..e6f5b65dbe 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -371,6 +371,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+ kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -2470,6 +2471,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+ /*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+ if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+ set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+ }
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 08/21] qapi/s390x/cpu topology: set-cpu-topology qmp command
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (6 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 07/21] target/s390x/cpu topology: activate CPU topology Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 09/21] machine: adding s390 topology to query-cpu-fast Pierre Morel
` (12 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine-target.json | 37 +++++++++++
hw/s390x/cpu-topology.c | 129 +++++++++++++++++++++++++++++++++++++++
2 files changed, 166 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 42a6a40333..3b7a0b77f4 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -354,3 +356,38 @@
{ 'enum': 'CpuS390Polarization',
'prefix': 'S390_CPU_POLARIZATION',
'data': [ 'horizontal', 'vertical' ] }
+
+##
+# @set-cpu-topology:
+#
+# @core-id: the vCPU ID to be moved
+# @socket-id: optional destination socket where to move the vCPU
+# @book-id: optional destination book where to move the vCPU
+# @drawer-id: optional destination drawer where to move the vCPU
+# @entitlement: optional entitlement
+# @dedicated: optional, if the vCPU is dedicated to a real CPU
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+# Default value for optional parameter is the current value
+# used by the CPU.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: 8.1
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 35426e23f0..3c7e4e07e6 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -18,6 +18,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -289,6 +290,27 @@ static void s390_topology_add_core_to_socket(S390CPU *cpu, int drawer_id,
}
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static int s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+ return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -355,3 +377,110 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement, uint16_t entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+ MachineState *ms = current_machine;
+ S390CPU *cpu;
+ int report_needed;
+ ERRP_GUARD();
+
+ if (core_id >= ms->smp.max_cpus) {
+ error_setg(errp, "Core-id %d out of range!", core_id);
+ return;
+ }
+
+ cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
+ if (!cpu) {
+ error_setg(errp, "Core-id %d does not exist!", core_id);
+ return;
+ }
+
+ /* Get attributes not provided from cpu and verify the new topology */
+ if (!has_socket_id) {
+ socket_id = cpu->env.socket_id;
+ }
+ if (!has_book_id) {
+ book_id = cpu->env.book_id;
+ }
+ if (!has_drawer_id) {
+ drawer_id = cpu->env.drawer_id;
+ }
+ if (!has_dedicated) {
+ dedicated = cpu->env.dedicated;
+ }
+
+ /*
+ * Even user can specify the entitlement as horizontal on the command line,
+ * qemu will only use entitlement during vertical polarization.
+ * Medium entitlement is chosen as the default entitlement when the CPU
+ * is not dedicated.
+ * A dedicated CPU always receives a high entitlement.
+ */
+ if (!has_entitlement || (entitlement == S390_CPU_ENTITLEMENT_HORIZONTAL)) {
+ if (dedicated) {
+ entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
+ }
+ }
+
+ s390_topology_check(socket_id, book_id, drawer_id,
+ entitlement, dedicated, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* Move the CPU into its new socket */
+ s390_topology_add_core_to_socket(cpu, drawer_id, book_id,
+ socket_id, false, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* Check if we need to report the modified topology */
+ report_needed = s390_topology_need_report(cpu, drawer_id, book_id,
+ socket_id, entitlement,
+ dedicated);
+
+ /* All checks done, report new topology into the vCPU */
+ cpu->env.drawer_id = drawer_id;
+ cpu->env.book_id = book_id;
+ cpu->env.socket_id = socket_id;
+ cpu->env.dedicated = dedicated;
+ cpu->env.entitlement = entitlement;
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+
+ /* Advertise the topology change */
+ if (report_needed) {
+ s390_cpu_topology_set_changed(true);
+ }
+}
+
+void qmp_set_cpu_topology(uint16_t core,
+ bool has_socket, uint16_t socket,
+ bool has_book, uint16_t book,
+ bool has_drawer, uint16_t drawer,
+ bool has_entitlement, CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+ ERRP_GUARD();
+
+ if (!s390_has_topology()) {
+ error_setg(errp, "This machine doesn't support topology");
+ return;
+ }
+
+ s390_change_topology(core, has_socket, socket, has_book, book,
+ has_drawer, drawer, has_entitlement, entitlement,
+ has_dedicated, dedicated, errp);
+}
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 09/21] machine: adding s390 topology to query-cpu-fast
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (7 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 08/21] qapi/s390x/cpu topology: set-cpu-topology qmp command Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 10/21] machine: adding s390 topology to info hotpluggable-cpus Pierre Morel
` (11 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
qapi/machine.json | 9 ++++++++-
hw/core/machine-qmp-cmds.c | 2 ++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 1cdd83f3fd..c6a12044e0 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -55,10 +55,17 @@
# Additional information about a virtual S390 CPU
#
# @cpu-state: the virtual CPU's state
+# @dedicated: the virtual CPU's dedication (since 8.1)
+# @entitlement: the virtual CPU's entitlement (since 8.1)
#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+ 'dedicated': 'bool',
+ 'entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index b98ff15089..3f35ed83a6 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -35,6 +35,8 @@ static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu)
CPUS390XState *env = &s390_cpu->env;
info->cpu_state = env->cpu_state;
+ info->dedicated = env->dedicated;
+ info->entitlement = env->entitlement;
#else
abort();
#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 10/21] machine: adding s390 topology to info hotpluggable-cpus
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (8 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 09/21] machine: adding s390 topology to query-cpu-fast Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 11/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event Pierre Morel
` (10 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine-hmp-cmds.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..971212242d 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, " node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+ if (c->has_drawer_id) {
+ monitor_printf(mon, " drawer_id: \"%" PRIu64 "\"\n", c->drawer_id);
+ }
+ if (c->has_book_id) {
+ monitor_printf(mon, " book_id: \"%" PRIu64 "\"\n", c->book_id);
+ }
if (c->has_socket_id) {
monitor_printf(mon, " socket-id: \"%" PRIu64 "\"\n", c->socket_id);
}
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 11/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (9 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 10/21] machine: adding s390 topology to info hotpluggable-cpus Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 12/21] qapi/s390x/cpu topology: query-cpu-polarization qmp command Pierre Morel
` (9 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine-target.json | 33 +++++++++++++++++++++++++++++++++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 35 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 3b7a0b77f4..ffde2e9cbd 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -391,3 +391,36 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# Since: 8.1
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": 0 },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 3c7e4e07e6..97706a1daf 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -19,6 +19,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -127,6 +128,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390ms->vertical_polarization = !!fc;
s390_cpu_topology_set_changed(true);
+ qapi_event_send_cpu_polarization_change(fc);
setcc(cpu, 0);
}
break;
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 12/21] qapi/s390x/cpu topology: query-cpu-polarization qmp command
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (10 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 11/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 16:28 ` [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
` (8 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The query-cpu-polarization qmp command returns the current
CPU polarization of the machine.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
hw/s390x/cpu-topology.c | 16 ++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index ffde2e9cbd..8eb05755cd 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'common.json' }
{ 'include': 'machine-common.json' }
##
@@ -424,3 +425,32 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CpuPolarizationInfo:
+#
+# The result of a cpu polarization
+#
+# @polarization: the CPU polarization
+#
+# Since: 2.8
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
+
+##
+# @query-cpu-polarization:
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Returns: the machine polarization
+#
+# Since: 8.1
+##
+{ 'command': 'query-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 97706a1daf..b8a292340c 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -20,6 +20,7 @@
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
#include "qapi/qapi-events-machine-target.h"
+#include "qapi/type-helpers.h"
/*
* s390_topology is used to keep the topology information.
@@ -486,3 +487,18 @@ void qmp_set_cpu_topology(uint16_t core,
has_drawer, drawer, has_entitlement, entitlement,
has_dedicated, dedicated, errp);
}
+
+CpuPolarizationInfo *qmp_query_cpu_polarization(Error **errp)
+{
+ struct S390CcwMachineState *s390ms = S390_CCW_MACHINE(current_machine);
+ CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
+
+ if (s390ms->vertical_polarization) {
+ info->polarization = S390_CPU_POLARIZATION_VERTICAL;
+ } else {
+ info->polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+ }
+
+ return info;
+}
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (11 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 12/21] qapi/s390x/cpu topology: query-cpu-polarization qmp command Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-03 17:00 ` Cédric Le Goater
2023-04-03 16:28 ` [PATCH v19 14/21] tests/avocado: s390x cpu topology core Pierre Morel
` (7 subsequent siblings)
20 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
MAINTAINERS | 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 161 +++++++++++++++++++
docs/system/s390x/cpu-topology.rst | 238 +++++++++++++++++++++++++++++
docs/system/target-s390x.rst | 1 +
5 files changed, 403 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index de9052f753..fe5638e31d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1660,6 +1660,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/cpu_topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
------------
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 0000000000..0b7bb42079
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,161 @@
+QAPI interface for S390 CPU topology
+====================================
+
+Let's start QEMU with the following command:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+ -enable-kvm \
+ -cpu z14,ctop=on \
+ -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+ \
+ -device z14-s390x-cpu,core-id=19,polarization=3 \
+ -device z14-s390x-cpu,core-id=11,polarization=1 \
+ -device z14-s390x-cpu,core-id=112,polarization=3 \
+ ...
+
+and see the result when using the QAPI interface.
+
+Addons to query-cpus-fast
+-------------------------
+
+The command query-cpus-fast allows to query the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+ {
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+ "core-id": 0,
+ "socket-id": 0,
+ "drawer-id": 0,
+ "book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+ "core-id": 19,
+ "socket-id": 1,
+ "drawer-id": 0,
+ "book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+ "core-id": 11,
+ "socket-id": 1,
+ "drawer-id": 0,
+ "book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+ },
+ {
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+ "core-id": 112,
+ "socket-id": 0,
+ "drawer-id": 3,
+ "book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+ }
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+------------------------------
+
+The command set-cpu-topology allows to modify the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+ { "execute": "set-cpu-topology",
+ "arguments": {
+ "core-id": 11,
+ "socket-id": 0,
+ "book-id": 0,
+ "drawer-id": 0,
+ "entitlement": "low",
+ "dedicated": false
+ }
+ }
+ {"return": {}}
+
+The core-id parameter is the only non optional parameter and every
+unspecified parameter keeps its previous value.
+
+QAPI event CPU_POLARIZATION_CHANGE
+----------------------------------
+
+When a guest is requests a modification of the polarization,
+QEMU sends a CPU_POLARIZATION_CHANGE event.
+
+When requesting the change, the guest only specifies horizontal or
+vertical polarization.
+It is the job of the upper layer to set the dedication and fine grained
+vertical entitlement in response to this event.
+
+Note that a vertical polarized dedicated vCPU can only have a high
+entitlement, this gives 6 possibilities for vCPU polarization:
+
+- Horizontal
+- Horizontal dedicated
+- Vertical low
+- Vertical medium
+- Vertical high
+- Vertical high dedicated
+
+Example of the event received when the guest issues the CPU instruction
+Perform Topology Function PTF(0) to request an horizontal polarization:
+
+.. code-block:: QMP
+
+ { "event": "CPU_POLARIZATION_CHANGE",
+ "data": { "polarization": 0 },
+ "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+
+QAPI query command: query-cpu-polarization
+------------------------------
+
+The query command query-cpu-polarization returns the current
+CPU polarization of the machine.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpu-polarization" }
+ {
+ "return": {
+ "polarization": "vertical"
+ }
+ }
diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
new file mode 100644
index 0000000000..c1fe3da51c
--- /dev/null
+++ b/docs/system/s390x/cpu-topology.rst
@@ -0,0 +1,238 @@
+CPU topology on s390x
+=====================
+
+Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of
+topology containers: drawers, books, sockets, defining a tree shaped
+hierarchy.
+
+The socket container contains one or more CPU entries.
+Each of these CPU entries consists of a bitmap and three CPU attributes:
+
+- CPU type
+- polarization entitlement
+- dedication
+
+Each bit set in the bitmap correspond to the core-id of a vCPU with
+matching the three attribute.
+
+This documentation provide general information on S390 CPU topology,
+how to enable it and on the new CPU attributes.
+For information on how to modify the S390 CPU topology and on how to
+monitor the polarization change see ``Developer Information``.
+
+Prerequisites
+-------------
+
+To use the CPU topology, you need to run with KVM on a s390x host that
+uses the Linux kernel v6.0 or newer (which provide the so-called
+``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the
+CPU topology facility via the so-called STFLE bit 11 to the VM).
+
+Enabling CPU topology
+---------------------
+
+Currently, CPU topology is only enabled in the host model by default.
+
+Enabling CPU topology in a CPU model is done by setting the CPU flag
+``ctop`` to ``on`` like in:
+
+.. code-block:: bash
+
+ -cpu gen16b,ctop=on
+
+Having the topology disabled by default allows migration between
+old and new QEMU without adding new flags.
+
+Default topology usage
+----------------------
+
+The CPU topology can be specified on the QEMU command line
+with the ``-smp`` or the ``-device`` QEMU command arguments.
+
+Note also that since 7.2 threads are no longer supported in the topology
+and the ``-smp`` command line argument accepts only ``threads=1``.
+
+If none of the containers attributes (drawers, books, sockets) are
+specified for the ``-smp`` flag, the number of these containers
+is ``1`` .
+
+.. code-block:: bash
+
+ -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32
+
+or
+
+.. code-block:: bash
+
+ -smp cpus=5,sockets=8,cores=4,maxcpus=32
+
+When a CPU is defined by the ``-smp`` command argument, its position
+inside the topology is calculated by adding the CPUs to the topology
+based on the core-id starting with core-0 at position 0 of socket-0,
+book-0, drawer-0 and filling all CPUs of socket-0 before to fill socket-1
+of book-0 and so on up to the last socket of the last book of the last
+drawer.
+
+When a CPU is defined by the ``-device`` command argument, the
+tree topology attributes must be all defined or all not defined.
+
+.. code-block:: bash
+
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
+
+or
+
+.. code-block:: bash
+
+ -device gen16b-s390x-cpu,core-id=1,dedication=true
+
+If none of the tree attributes (drawer, book, sockets), are specified
+for the ``-device`` argument, as for all CPUs defined with the ``-smp``
+command argument the topology tree attributes will be set by simply
+adding the CPUs to the topology based on the core-id starting with
+core-0 at position 0 of socket-0, book-0, drawer-0.
+
+QEMU will not try to solve collisions and will report an error if the
+CPU topology, explicitly or implicitly defined on a ``-device``
+argument collides with the definition of a CPU implicitely defined
+on the ``-smp`` argument.
+
+When the topology modifier attributes are not defined for the
+``-device`` command argument they takes following default values:
+
+- dedication: ``false``
+- entitlement: ``medium``
+
+
+Hot plug
+++++++++
+
+New CPUs can be plugged using the device_add hmp command as in:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
+The same placement of the CPU is derived from the core-id as described above.
+
+The topology can of course be fully defined:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
+
+
+Examples
+++++++++
+
+In the following machine we define 8 sockets with 4 cores each.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
+ -device host-s390x-cpu,core-id=14 \
+
+A new CPUs can be plugged using the device_add hmp command as before:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
+The core-id defines the placement of the core in the topology by
+starting with core 0 in socket 0 up to maxcpus.
+
+In the example above:
+
+* There are 5 CPUs provided to the guest with the ``-smp`` command line
+ They will take the core-ids 0,1,2,3,4
+ As we have 4 cores in a socket, we have 4 CPUs provided
+ to the guest in socket 0, with core-ids 0,1,2,3.
+ The last cpu, with core-id 4, will be on socket 1.
+
+* the core with ID 14 provided by the ``-device`` command line will
+ be placed in socket 3, with core-id 14
+
+* the core with ID 9 provided by the ``device_add`` qmp command will
+ be placed in socket 2, with core-id 9
+
+
+Polarization, entitlement and dedication
+----------------------------------------
+
+Polarization
+++++++++++++
+
+The polarization is an indication given by the ``guest`` to the host
+that it is able to make use of CPU provisioning information.
+The guest indicates the polarization by using the PTF instruction.
+
+Polarization is define two models of CPU provisioning: horizontal
+and vertical.
+
+The horizontal polarization is the default model on boot and after
+subsystem reset in which the guest considers all vCPUs being having
+an equal provisioning of CPUs by the host.
+
+In the vertical polarization model the guest can make use of the
+vCPU entitlement information provided by the host to optimize
+kernel thread scheduling.
+
+A subsystem reset puts all vCPU of the configuration into the
+horizontal polarization.
+
+Entitlement
++++++++++++
+
+The vertical polarization specifies that the guest's vCPU can get
+different real CPU provisions:
+
+- a vCPU with vertical high entitlement specifies that this
+ vCPU gets 100% of the real CPU provisioning.
+
+- a vCPU with vertical medium entitlement specifies that this
+ vCPU shares the real CPU with other vCPUs.
+
+- a vCPU with vertical low entitlement specifies that this
+ vCPU only gets real CPU provisioning when no other vCPUs needs it.
+
+In the case a vCPU with vertical high entitlement does not use
+the real CPU, the unused "slack" can be dispatched to other vCPU
+with medium or low entitlement.
+
+The upper level specifies a vCPU as ``dedicated`` when the vCPU is
+fully dedicated to a single real CPU.
+
+The dedicated bit is an indication of affinity of a vCPU for a real CPU
+while the entitlement indicates the sharing or exclusivity of use.
+
+Defining the topology on command line
+-------------------------------------
+
+The topology can entirely be defined using -device cpu statements,
+with the exception of CPU 0 which must be defined with the -smp
+argument.
+
+For example, here we set the position of the cores 1,2,3 to
+drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0,
+book 0, socket 0 with all horizontal polarization and not dedicated.
+The core 4, will be set on its default position on socket 1
+(since we have 4 core per socket) and we define it with dedication and
+vertical high entitlement.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
+ \
+ -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarization=3 \
+
diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
index f6f11433c7..94c981e732 100644
--- a/docs/system/target-s390x.rst
+++ b/docs/system/target-s390x.rst
@@ -34,3 +34,4 @@ Architectural features
.. toctree::
s390x/bootdevices
s390x/protvirt
+ s390x/cpu-topology
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology
2023-04-03 16:28 ` [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
@ 2023-04-03 17:00 ` Cédric Le Goater
2023-04-03 17:21 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-03 17:00 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> Add some basic examples for the definition of cpu topology
> in s390x.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> MAINTAINERS | 2 +
> docs/devel/index-internals.rst | 1 +
> docs/devel/s390-cpu-topology.rst | 161 +++++++++++++++++++
> docs/system/s390x/cpu-topology.rst | 238 +++++++++++++++++++++++++++++
> docs/system/target-s390x.rst | 1 +
> 5 files changed, 403 insertions(+)
> create mode 100644 docs/devel/s390-cpu-topology.rst
> create mode 100644 docs/system/s390x/cpu-topology.rst
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index de9052f753..fe5638e31d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1660,6 +1660,8 @@ S: Supported
> F: include/hw/s390x/cpu-topology.h
> F: hw/s390x/cpu-topology.c
> F: target/s390x/kvm/cpu_topology.c
> +F: docs/devel/s390-cpu-topology.rst
> +F: docs/system/s390x/cpu-topology.rst
>
> X86 Machines
> ------------
> diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
> index e1a93df263..6f81df92bc 100644
> --- a/docs/devel/index-internals.rst
> +++ b/docs/devel/index-internals.rst
> @@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add features to them.
> migration
> multi-process
> reset
> + s390-cpu-topology
> s390-dasd-ipl
> tracing
> vfio-migration
> diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
> new file mode 100644
> index 0000000000..0b7bb42079
> --- /dev/null
> +++ b/docs/devel/s390-cpu-topology.rst
> @@ -0,0 +1,161 @@
> +QAPI interface for S390 CPU topology
> +====================================
> +
> +Let's start QEMU with the following command:
> +
> +.. code-block:: bash
> +
> + qemu-system-s390x \
> + -enable-kvm \
> + -cpu z14,ctop=on \
> + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
> + \
> + -device z14-s390x-cpu,core-id=19,polarization=3 \
> + -device z14-s390x-cpu,core-id=11,polarization=1 \
> + -device z14-s390x-cpu,core-id=112,polarization=3 \
> + ...
> +
> +and see the result when using the QAPI interface.
> +
> +Addons to query-cpus-fast
> +-------------------------
> +
> +The command query-cpus-fast allows to query the topology tree and
> +modifiers for all configured vCPUs.
> +
> +.. code-block:: QMP
> +
> + { "execute": "query-cpus-fast" }
> + {
> + "return": [
> + {
> + "dedicated": false,
> + "thread-id": 536993,
> + "props": {
> + "core-id": 0,
> + "socket-id": 0,
> + "drawer-id": 0,
> + "book-id": 0
> + },
> + "cpu-state": "operating",
> + "entitlement": "medium",
> + "qom-path": "/machine/unattached/device[0]",
> + "cpu-index": 0,
> + "target": "s390x"
> + },
> + {
> + "dedicated": false,
> + "thread-id": 537003,
> + "props": {
> + "core-id": 19,
> + "socket-id": 1,
> + "drawer-id": 0,
> + "book-id": 2
> + },
> + "cpu-state": "operating",
> + "entitlement": "high",
> + "qom-path": "/machine/peripheral-anon/device[0]",
> + "cpu-index": 19,
> + "target": "s390x"
> + },
> + {
> + "dedicated": false,
> + "thread-id": 537004,
> + "props": {
> + "core-id": 11,
> + "socket-id": 1,
> + "drawer-id": 0,
> + "book-id": 1
> + },
> + "cpu-state": "operating",
> + "entitlement": "low",
> + "qom-path": "/machine/peripheral-anon/device[1]",
> + "cpu-index": 11,
> + "target": "s390x"
> + },
> + {
> + "dedicated": true,
> + "thread-id": 537005,
> + "props": {
> + "core-id": 112,
> + "socket-id": 0,
> + "drawer-id": 3,
> + "book-id": 2
> + },
> + "cpu-state": "operating",
> + "entitlement": "high",
> + "qom-path": "/machine/peripheral-anon/device[2]",
> + "cpu-index": 112,
> + "target": "s390x"
> + }
> + ]
> + }
> +
> +
> +QAPI command: set-cpu-topology
> +------------------------------
> +
> +The command set-cpu-topology allows to modify the topology tree
> +or the topology modifiers of a vCPU in the configuration.
> +
> +.. code-block:: QMP
> +
> + { "execute": "set-cpu-topology",
> + "arguments": {
> + "core-id": 11,
> + "socket-id": 0,
> + "book-id": 0,
> + "drawer-id": 0,
> + "entitlement": "low",
> + "dedicated": false
> + }
> + }
> + {"return": {}}
> +
> +The core-id parameter is the only non optional parameter and every
> +unspecified parameter keeps its previous value.
> +
> +QAPI event CPU_POLARIZATION_CHANGE
> +----------------------------------
> +
> +When a guest is requests a modification of the polarization,
> +QEMU sends a CPU_POLARIZATION_CHANGE event.
> +
> +When requesting the change, the guest only specifies horizontal or
> +vertical polarization.
> +It is the job of the upper layer to set the dedication and fine grained
> +vertical entitlement in response to this event.
> +
> +Note that a vertical polarized dedicated vCPU can only have a high
> +entitlement, this gives 6 possibilities for vCPU polarization:
> +
> +- Horizontal
> +- Horizontal dedicated
> +- Vertical low
> +- Vertical medium
> +- Vertical high
> +- Vertical high dedicated
> +
> +Example of the event received when the guest issues the CPU instruction
> +Perform Topology Function PTF(0) to request an horizontal polarization:
> +
> +.. code-block:: QMP
> +
> + { "event": "CPU_POLARIZATION_CHANGE",
> + "data": { "polarization": 0 },
> + "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> +
> +QAPI query command: query-cpu-polarization
> +------------------------------
Some dashes are missing from this line. No need to resend, it's easy to fix.
Thanks,
C.
> +
> +The query command query-cpu-polarization returns the current
> +CPU polarization of the machine.
> +
> +.. code-block:: QMP
> +
> + { "execute": "query-cpu-polarization" }
> + {
> + "return": {
> + "polarization": "vertical"
> + }
> + }
> diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
> new file mode 100644
> index 0000000000..c1fe3da51c
> --- /dev/null
> +++ b/docs/system/s390x/cpu-topology.rst
> @@ -0,0 +1,238 @@
> +CPU topology on s390x
> +=====================
> +
> +Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of
> +topology containers: drawers, books, sockets, defining a tree shaped
> +hierarchy.
> +
> +The socket container contains one or more CPU entries.
> +Each of these CPU entries consists of a bitmap and three CPU attributes:
> +
> +- CPU type
> +- polarization entitlement
> +- dedication
> +
> +Each bit set in the bitmap correspond to the core-id of a vCPU with
> +matching the three attribute.
> +
> +This documentation provide general information on S390 CPU topology,
> +how to enable it and on the new CPU attributes.
> +For information on how to modify the S390 CPU topology and on how to
> +monitor the polarization change see ``Developer Information``.
> +
> +Prerequisites
> +-------------
> +
> +To use the CPU topology, you need to run with KVM on a s390x host that
> +uses the Linux kernel v6.0 or newer (which provide the so-called
> +``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the
> +CPU topology facility via the so-called STFLE bit 11 to the VM).
> +
> +Enabling CPU topology
> +---------------------
> +
> +Currently, CPU topology is only enabled in the host model by default.
> +
> +Enabling CPU topology in a CPU model is done by setting the CPU flag
> +``ctop`` to ``on`` like in:
> +
> +.. code-block:: bash
> +
> + -cpu gen16b,ctop=on
> +
> +Having the topology disabled by default allows migration between
> +old and new QEMU without adding new flags.
> +
> +Default topology usage
> +----------------------
> +
> +The CPU topology can be specified on the QEMU command line
> +with the ``-smp`` or the ``-device`` QEMU command arguments.
> +
> +Note also that since 7.2 threads are no longer supported in the topology
> +and the ``-smp`` command line argument accepts only ``threads=1``.
> +
> +If none of the containers attributes (drawers, books, sockets) are
> +specified for the ``-smp`` flag, the number of these containers
> +is ``1`` .
> +
> +.. code-block:: bash
> +
> + -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32
> +
> +or
> +
> +.. code-block:: bash
> +
> + -smp cpus=5,sockets=8,cores=4,maxcpus=32
> +
> +When a CPU is defined by the ``-smp`` command argument, its position
> +inside the topology is calculated by adding the CPUs to the topology
> +based on the core-id starting with core-0 at position 0 of socket-0,
> +book-0, drawer-0 and filling all CPUs of socket-0 before to fill socket-1
> +of book-0 and so on up to the last socket of the last book of the last
> +drawer.
> +
> +When a CPU is defined by the ``-device`` command argument, the
> +tree topology attributes must be all defined or all not defined.
> +
> +.. code-block:: bash
> +
> + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
> +
> +or
> +
> +.. code-block:: bash
> +
> + -device gen16b-s390x-cpu,core-id=1,dedication=true
> +
> +If none of the tree attributes (drawer, book, sockets), are specified
> +for the ``-device`` argument, as for all CPUs defined with the ``-smp``
> +command argument the topology tree attributes will be set by simply
> +adding the CPUs to the topology based on the core-id starting with
> +core-0 at position 0 of socket-0, book-0, drawer-0.
> +
> +QEMU will not try to solve collisions and will report an error if the
> +CPU topology, explicitly or implicitly defined on a ``-device``
> +argument collides with the definition of a CPU implicitely defined
> +on the ``-smp`` argument.
> +
> +When the topology modifier attributes are not defined for the
> +``-device`` command argument they takes following default values:
> +
> +- dedication: ``false``
> +- entitlement: ``medium``
> +
> +
> +Hot plug
> +++++++++
> +
> +New CPUs can be plugged using the device_add hmp command as in:
> +
> +.. code-block:: bash
> +
> + (qemu) device_add gen16b-s390x-cpu,core-id=9
> +
> +The same placement of the CPU is derived from the core-id as described above.
> +
> +The topology can of course be fully defined:
> +
> +.. code-block:: bash
> +
> + (qemu) device_add gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
> +
> +
> +Examples
> +++++++++
> +
> +In the following machine we define 8 sockets with 4 cores each.
> +
> +.. code-block:: bash
> +
> + $ qemu-system-s390x -m 2G \
> + -cpu gen16b,ctop=on \
> + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
> + -device host-s390x-cpu,core-id=14 \
> +
> +A new CPUs can be plugged using the device_add hmp command as before:
> +
> +.. code-block:: bash
> +
> + (qemu) device_add gen16b-s390x-cpu,core-id=9
> +
> +The core-id defines the placement of the core in the topology by
> +starting with core 0 in socket 0 up to maxcpus.
> +
> +In the example above:
> +
> +* There are 5 CPUs provided to the guest with the ``-smp`` command line
> + They will take the core-ids 0,1,2,3,4
> + As we have 4 cores in a socket, we have 4 CPUs provided
> + to the guest in socket 0, with core-ids 0,1,2,3.
> + The last cpu, with core-id 4, will be on socket 1.
> +
> +* the core with ID 14 provided by the ``-device`` command line will
> + be placed in socket 3, with core-id 14
> +
> +* the core with ID 9 provided by the ``device_add`` qmp command will
> + be placed in socket 2, with core-id 9
> +
> +
> +Polarization, entitlement and dedication
> +----------------------------------------
> +
> +Polarization
> +++++++++++++
> +
> +The polarization is an indication given by the ``guest`` to the host
> +that it is able to make use of CPU provisioning information.
> +The guest indicates the polarization by using the PTF instruction.
> +
> +Polarization is define two models of CPU provisioning: horizontal
> +and vertical.
> +
> +The horizontal polarization is the default model on boot and after
> +subsystem reset in which the guest considers all vCPUs being having
> +an equal provisioning of CPUs by the host.
> +
> +In the vertical polarization model the guest can make use of the
> +vCPU entitlement information provided by the host to optimize
> +kernel thread scheduling.
> +
> +A subsystem reset puts all vCPU of the configuration into the
> +horizontal polarization.
> +
> +Entitlement
> ++++++++++++
> +
> +The vertical polarization specifies that the guest's vCPU can get
> +different real CPU provisions:
> +
> +- a vCPU with vertical high entitlement specifies that this
> + vCPU gets 100% of the real CPU provisioning.
> +
> +- a vCPU with vertical medium entitlement specifies that this
> + vCPU shares the real CPU with other vCPUs.
> +
> +- a vCPU with vertical low entitlement specifies that this
> + vCPU only gets real CPU provisioning when no other vCPUs needs it.
> +
> +In the case a vCPU with vertical high entitlement does not use
> +the real CPU, the unused "slack" can be dispatched to other vCPU
> +with medium or low entitlement.
> +
> +The upper level specifies a vCPU as ``dedicated`` when the vCPU is
> +fully dedicated to a single real CPU.
> +
> +The dedicated bit is an indication of affinity of a vCPU for a real CPU
> +while the entitlement indicates the sharing or exclusivity of use.
> +
> +Defining the topology on command line
> +-------------------------------------
> +
> +The topology can entirely be defined using -device cpu statements,
> +with the exception of CPU 0 which must be defined with the -smp
> +argument.
> +
> +For example, here we set the position of the cores 1,2,3 to
> +drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0,
> +book 0, socket 0 with all horizontal polarization and not dedicated.
> +The core 4, will be set on its default position on socket 1
> +(since we have 4 core per socket) and we define it with dedication and
> +vertical high entitlement.
> +
> +.. code-block:: bash
> +
> + $ qemu-system-s390x -m 2G \
> + -cpu gen16b,ctop=on \
> + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
> + \
> + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
> + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
> + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
> + \
> + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
> + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
> + \
> + -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarization=3 \
> +
> diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
> index f6f11433c7..94c981e732 100644
> --- a/docs/system/target-s390x.rst
> +++ b/docs/system/target-s390x.rst
> @@ -34,3 +34,4 @@ Architectural features
> .. toctree::
> s390x/bootdevices
> s390x/protvirt
> + s390x/cpu-topology
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology
2023-04-03 17:00 ` Cédric Le Goater
@ 2023-04-03 17:21 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 17:21 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 19:00, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> Add some basic examples for the definition of cpu topology
>> in s390x.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> MAINTAINERS | 2 +
>> docs/devel/index-internals.rst | 1 +
>> docs/devel/s390-cpu-topology.rst | 161 +++++++++++++++++++
>> docs/system/s390x/cpu-topology.rst | 238 +++++++++++++++++++++++++++++
>> docs/system/target-s390x.rst | 1 +
>> 5 files changed, 403 insertions(+)
>> create mode 100644 docs/devel/s390-cpu-topology.rst
>> create mode 100644 docs/system/s390x/cpu-topology.rst
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index de9052f753..fe5638e31d 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1660,6 +1660,8 @@ S: Supported
>> F: include/hw/s390x/cpu-topology.h
>> F: hw/s390x/cpu-topology.c
>> F: target/s390x/kvm/cpu_topology.c
>> +F: docs/devel/s390-cpu-topology.rst
>> +F: docs/system/s390x/cpu-topology.rst
>> X86 Machines
>> ------------
>> diff --git a/docs/devel/index-internals.rst
>> b/docs/devel/index-internals.rst
>> index e1a93df263..6f81df92bc 100644
>> --- a/docs/devel/index-internals.rst
>> +++ b/docs/devel/index-internals.rst
>> @@ -14,6 +14,7 @@ Details about QEMU's various subsystems including
>> how to add features to them.
>> migration
>> multi-process
>> reset
>> + s390-cpu-topology
>> s390-dasd-ipl
>> tracing
>> vfio-migration
>> diff --git a/docs/devel/s390-cpu-topology.rst
>> b/docs/devel/s390-cpu-topology.rst
>> new file mode 100644
>> index 0000000000..0b7bb42079
>> --- /dev/null
>> +++ b/docs/devel/s390-cpu-topology.rst
>> @@ -0,0 +1,161 @@
>> +QAPI interface for S390 CPU topology
>> +====================================
>> +
>> +Let's start QEMU with the following command:
>> +
>> +.. code-block:: bash
>> +
>> + qemu-system-s390x \
>> + -enable-kvm \
>> + -cpu z14,ctop=on \
>> + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
>> + \
>> + -device z14-s390x-cpu,core-id=19,polarization=3 \
>> + -device z14-s390x-cpu,core-id=11,polarization=1 \
>> + -device z14-s390x-cpu,core-id=112,polarization=3 \
>> + ...
>> +
>> +and see the result when using the QAPI interface.
>> +
>> +Addons to query-cpus-fast
>> +-------------------------
>> +
>> +The command query-cpus-fast allows to query the topology tree and
>> +modifiers for all configured vCPUs.
>> +
>> +.. code-block:: QMP
>> +
>> + { "execute": "query-cpus-fast" }
>> + {
>> + "return": [
>> + {
>> + "dedicated": false,
>> + "thread-id": 536993,
>> + "props": {
>> + "core-id": 0,
>> + "socket-id": 0,
>> + "drawer-id": 0,
>> + "book-id": 0
>> + },
>> + "cpu-state": "operating",
>> + "entitlement": "medium",
>> + "qom-path": "/machine/unattached/device[0]",
>> + "cpu-index": 0,
>> + "target": "s390x"
>> + },
>> + {
>> + "dedicated": false,
>> + "thread-id": 537003,
>> + "props": {
>> + "core-id": 19,
>> + "socket-id": 1,
>> + "drawer-id": 0,
>> + "book-id": 2
>> + },
>> + "cpu-state": "operating",
>> + "entitlement": "high",
>> + "qom-path": "/machine/peripheral-anon/device[0]",
>> + "cpu-index": 19,
>> + "target": "s390x"
>> + },
>> + {
>> + "dedicated": false,
>> + "thread-id": 537004,
>> + "props": {
>> + "core-id": 11,
>> + "socket-id": 1,
>> + "drawer-id": 0,
>> + "book-id": 1
>> + },
>> + "cpu-state": "operating",
>> + "entitlement": "low",
>> + "qom-path": "/machine/peripheral-anon/device[1]",
>> + "cpu-index": 11,
>> + "target": "s390x"
>> + },
>> + {
>> + "dedicated": true,
>> + "thread-id": 537005,
>> + "props": {
>> + "core-id": 112,
>> + "socket-id": 0,
>> + "drawer-id": 3,
>> + "book-id": 2
>> + },
>> + "cpu-state": "operating",
>> + "entitlement": "high",
>> + "qom-path": "/machine/peripheral-anon/device[2]",
>> + "cpu-index": 112,
>> + "target": "s390x"
>> + }
>> + ]
>> + }
>> +
>> +
>> +QAPI command: set-cpu-topology
>> +------------------------------
>> +
>> +The command set-cpu-topology allows to modify the topology tree
>> +or the topology modifiers of a vCPU in the configuration.
>> +
>> +.. code-block:: QMP
>> +
>> + { "execute": "set-cpu-topology",
>> + "arguments": {
>> + "core-id": 11,
>> + "socket-id": 0,
>> + "book-id": 0,
>> + "drawer-id": 0,
>> + "entitlement": "low",
>> + "dedicated": false
>> + }
>> + }
>> + {"return": {}}
>> +
>> +The core-id parameter is the only non optional parameter and every
>> +unspecified parameter keeps its previous value.
>> +
>> +QAPI event CPU_POLARIZATION_CHANGE
>> +----------------------------------
>> +
>> +When a guest is requests a modification of the polarization,
>> +QEMU sends a CPU_POLARIZATION_CHANGE event.
>> +
>> +When requesting the change, the guest only specifies horizontal or
>> +vertical polarization.
>> +It is the job of the upper layer to set the dedication and fine grained
>> +vertical entitlement in response to this event.
>> +
>> +Note that a vertical polarized dedicated vCPU can only have a high
>> +entitlement, this gives 6 possibilities for vCPU polarization:
>> +
>> +- Horizontal
>> +- Horizontal dedicated
>> +- Vertical low
>> +- Vertical medium
>> +- Vertical high
>> +- Vertical high dedicated
>> +
>> +Example of the event received when the guest issues the CPU instruction
>> +Perform Topology Function PTF(0) to request an horizontal polarization:
>> +
>> +.. code-block:: QMP
>> +
>> + { "event": "CPU_POLARIZATION_CHANGE",
>> + "data": { "polarization": 0 },
>> + "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
>> +
>> +QAPI query command: query-cpu-polarization
>> +------------------------------
>
> Some dashes are missing from this line. No need to resend, it's easy
> to fix.
>
> Thanks,
>
> C.
grrr stupido.
Thanks Cédric
Pierre
>
>> +
>> +The query command query-cpu-polarization returns the current
>> +CPU polarization of the machine.
>> +
>> +.. code-block:: QMP
>> +
>> + { "execute": "query-cpu-polarization" }
>> + {
>> + "return": {
>> + "polarization": "vertical"
>> + }
>> + }
>> diff --git a/docs/system/s390x/cpu-topology.rst
>> b/docs/system/s390x/cpu-topology.rst
>> new file mode 100644
>> index 0000000000..c1fe3da51c
>> --- /dev/null
>> +++ b/docs/system/s390x/cpu-topology.rst
>> @@ -0,0 +1,238 @@
>> +CPU topology on s390x
>> +=====================
>> +
>> +Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of
>> +topology containers: drawers, books, sockets, defining a tree shaped
>> +hierarchy.
>> +
>> +The socket container contains one or more CPU entries.
>> +Each of these CPU entries consists of a bitmap and three CPU
>> attributes:
>> +
>> +- CPU type
>> +- polarization entitlement
>> +- dedication
>> +
>> +Each bit set in the bitmap correspond to the core-id of a vCPU with
>> +matching the three attribute.
>> +
>> +This documentation provide general information on S390 CPU topology,
>> +how to enable it and on the new CPU attributes.
>> +For information on how to modify the S390 CPU topology and on how to
>> +monitor the polarization change see ``Developer Information``.
>> +
>> +Prerequisites
>> +-------------
>> +
>> +To use the CPU topology, you need to run with KVM on a s390x host that
>> +uses the Linux kernel v6.0 or newer (which provide the so-called
>> +``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the
>> +CPU topology facility via the so-called STFLE bit 11 to the VM).
>> +
>> +Enabling CPU topology
>> +---------------------
>> +
>> +Currently, CPU topology is only enabled in the host model by default.
>> +
>> +Enabling CPU topology in a CPU model is done by setting the CPU flag
>> +``ctop`` to ``on`` like in:
>> +
>> +.. code-block:: bash
>> +
>> + -cpu gen16b,ctop=on
>> +
>> +Having the topology disabled by default allows migration between
>> +old and new QEMU without adding new flags.
>> +
>> +Default topology usage
>> +----------------------
>> +
>> +The CPU topology can be specified on the QEMU command line
>> +with the ``-smp`` or the ``-device`` QEMU command arguments.
>> +
>> +Note also that since 7.2 threads are no longer supported in the
>> topology
>> +and the ``-smp`` command line argument accepts only ``threads=1``.
>> +
>> +If none of the containers attributes (drawers, books, sockets) are
>> +specified for the ``-smp`` flag, the number of these containers
>> +is ``1`` .
>> +
>> +.. code-block:: bash
>> +
>> + -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32
>> +
>> +or
>> +
>> +.. code-block:: bash
>> +
>> + -smp cpus=5,sockets=8,cores=4,maxcpus=32
>> +
>> +When a CPU is defined by the ``-smp`` command argument, its position
>> +inside the topology is calculated by adding the CPUs to the topology
>> +based on the core-id starting with core-0 at position 0 of socket-0,
>> +book-0, drawer-0 and filling all CPUs of socket-0 before to fill
>> socket-1
>> +of book-0 and so on up to the last socket of the last book of the last
>> +drawer.
>> +
>> +When a CPU is defined by the ``-device`` command argument, the
>> +tree topology attributes must be all defined or all not defined.
>> +
>> +.. code-block:: bash
>> +
>> + -device
>> gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
>> +
>> +or
>> +
>> +.. code-block:: bash
>> +
>> + -device gen16b-s390x-cpu,core-id=1,dedication=true
>> +
>> +If none of the tree attributes (drawer, book, sockets), are specified
>> +for the ``-device`` argument, as for all CPUs defined with the ``-smp``
>> +command argument the topology tree attributes will be set by simply
>> +adding the CPUs to the topology based on the core-id starting with
>> +core-0 at position 0 of socket-0, book-0, drawer-0.
>> +
>> +QEMU will not try to solve collisions and will report an error if the
>> +CPU topology, explicitly or implicitly defined on a ``-device``
>> +argument collides with the definition of a CPU implicitely defined
>> +on the ``-smp`` argument.
>> +
>> +When the topology modifier attributes are not defined for the
>> +``-device`` command argument they takes following default values:
>> +
>> +- dedication: ``false``
>> +- entitlement: ``medium``
>> +
>> +
>> +Hot plug
>> +++++++++
>> +
>> +New CPUs can be plugged using the device_add hmp command as in:
>> +
>> +.. code-block:: bash
>> +
>> + (qemu) device_add gen16b-s390x-cpu,core-id=9
>> +
>> +The same placement of the CPU is derived from the core-id as
>> described above.
>> +
>> +The topology can of course be fully defined:
>> +
>> +.. code-block:: bash
>> +
>> + (qemu) device_add
>> gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
>> +
>> +
>> +Examples
>> +++++++++
>> +
>> +In the following machine we define 8 sockets with 4 cores each.
>> +
>> +.. code-block:: bash
>> +
>> + $ qemu-system-s390x -m 2G \
>> + -cpu gen16b,ctop=on \
>> + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
>> + -device host-s390x-cpu,core-id=14 \
>> +
>> +A new CPUs can be plugged using the device_add hmp command as before:
>> +
>> +.. code-block:: bash
>> +
>> + (qemu) device_add gen16b-s390x-cpu,core-id=9
>> +
>> +The core-id defines the placement of the core in the topology by
>> +starting with core 0 in socket 0 up to maxcpus.
>> +
>> +In the example above:
>> +
>> +* There are 5 CPUs provided to the guest with the ``-smp`` command line
>> + They will take the core-ids 0,1,2,3,4
>> + As we have 4 cores in a socket, we have 4 CPUs provided
>> + to the guest in socket 0, with core-ids 0,1,2,3.
>> + The last cpu, with core-id 4, will be on socket 1.
>> +
>> +* the core with ID 14 provided by the ``-device`` command line will
>> + be placed in socket 3, with core-id 14
>> +
>> +* the core with ID 9 provided by the ``device_add`` qmp command will
>> + be placed in socket 2, with core-id 9
>> +
>> +
>> +Polarization, entitlement and dedication
>> +----------------------------------------
>> +
>> +Polarization
>> +++++++++++++
>> +
>> +The polarization is an indication given by the ``guest`` to the host
>> +that it is able to make use of CPU provisioning information.
>> +The guest indicates the polarization by using the PTF instruction.
>> +
>> +Polarization is define two models of CPU provisioning: horizontal
>> +and vertical.
>> +
>> +The horizontal polarization is the default model on boot and after
>> +subsystem reset in which the guest considers all vCPUs being having
>> +an equal provisioning of CPUs by the host.
>> +
>> +In the vertical polarization model the guest can make use of the
>> +vCPU entitlement information provided by the host to optimize
>> +kernel thread scheduling.
>> +
>> +A subsystem reset puts all vCPU of the configuration into the
>> +horizontal polarization.
>> +
>> +Entitlement
>> ++++++++++++
>> +
>> +The vertical polarization specifies that the guest's vCPU can get
>> +different real CPU provisions:
>> +
>> +- a vCPU with vertical high entitlement specifies that this
>> + vCPU gets 100% of the real CPU provisioning.
>> +
>> +- a vCPU with vertical medium entitlement specifies that this
>> + vCPU shares the real CPU with other vCPUs.
>> +
>> +- a vCPU with vertical low entitlement specifies that this
>> + vCPU only gets real CPU provisioning when no other vCPUs needs it.
>> +
>> +In the case a vCPU with vertical high entitlement does not use
>> +the real CPU, the unused "slack" can be dispatched to other vCPU
>> +with medium or low entitlement.
>> +
>> +The upper level specifies a vCPU as ``dedicated`` when the vCPU is
>> +fully dedicated to a single real CPU.
>> +
>> +The dedicated bit is an indication of affinity of a vCPU for a real CPU
>> +while the entitlement indicates the sharing or exclusivity of use.
>> +
>> +Defining the topology on command line
>> +-------------------------------------
>> +
>> +The topology can entirely be defined using -device cpu statements,
>> +with the exception of CPU 0 which must be defined with the -smp
>> +argument.
>> +
>> +For example, here we set the position of the cores 1,2,3 to
>> +drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0,
>> +book 0, socket 0 with all horizontal polarization and not dedicated.
>> +The core 4, will be set on its default position on socket 1
>> +(since we have 4 core per socket) and we define it with dedication and
>> +vertical high entitlement.
>> +
>> +.. code-block:: bash
>> +
>> + $ qemu-system-s390x -m 2G \
>> + -cpu gen16b,ctop=on \
>> + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
>> + \
>> + -device
>> gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
>> + -device
>> gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
>> + -device
>> gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
>> + \
>> + -device
>> gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
>> + -device
>> gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
>> + \
>> + -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarization=3 \
>> +
>> diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
>> index f6f11433c7..94c981e732 100644
>> --- a/docs/system/target-s390x.rst
>> +++ b/docs/system/target-s390x.rst
>> @@ -34,3 +34,4 @@ Architectural features
>> .. toctree::
>> s390x/bootdevices
>> s390x/protvirt
>> + s390x/cpu-topology
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 14/21] tests/avocado: s390x cpu topology core
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (12 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 13/21] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-04 9:21 ` Cédric Le Goater
2023-04-03 16:28 ` [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation Pierre Morel
` (6 subsequent siblings)
20 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Introduction of the s390x cpu topology core functions and
basic tests.
We test the corelation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
MAINTAINERS | 1 +
tests/avocado/s390_topology.py | 196 +++++++++++++++++++++++++++++++++
2 files changed, 197 insertions(+)
create mode 100644 tests/avocado/s390_topology.py
diff --git a/MAINTAINERS b/MAINTAINERS
index fe5638e31d..41419840b0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1662,6 +1662,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/cpu_topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
+F: tests/avocado/s390_topology.py
X86 Machines
------------
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
new file mode 100644
index 0000000000..38e9cc4f16
--- /dev/null
+++ b/tests/avocado/s390_topology.py
@@ -0,0 +1,196 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright (c) 2023 IBM Corp.
+#
+# Author:
+# Pierre Morel <pmorel@linux.ibm.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import shutil
+import time
+
+from avocado import skip
+from avocado import skipUnless
+from avocado import skipIf
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import exec_command
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class LinuxKernelTest(QemuSystemTest):
+ KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+
+ def wait_for_console_pattern(self, success_message, vm=None):
+ wait_for_console_pattern(self, success_message,
+ failure_message='Kernel panic - not syncing',
+ vm=vm)
+
+
+class S390CPUTopology(LinuxKernelTest):
+ """
+ S390x CPU topology consist of 4 topology layers, from bottom to top,
+ the cores, sockets, books and drawers and 2 modifiers attributes,
+ the entitlement and the dedication.
+ See: docs/system/s390x/cpu-topology.rst.
+
+ S390x CPU topology is setup in different ways:
+ - implicitely from the '-smp' argument by completing each topology
+ level one after the other begining with drawer 0, book 0 and socket 0.
+ - explicitely from the '-device' argument on the QEMU command line
+ - explicitely by hotplug of a new CPU using QMP or HMP
+ - it is modified by using QMP 'set-cpu-topology'
+
+ The S390x modifier attribute entitlement depends on the machine
+ polarization, which can be horizontal or vertical.
+ The polarization is changed on a request from the guest.
+ """
+ timeout = 90
+
+
+ def check_topology(self, c, s, b, d, e, t):
+ res = self.vm.qmp('query-cpus-fast')
+ line = res['return']
+ for x in line:
+ core = x['props']['core-id']
+ socket = x['props']['socket-id']
+ book = x['props']['book-id']
+ drawer = x['props']['drawer-id']
+ entitlement = x['entitlement']
+ dedicated = x['dedicated']
+ if core == c:
+ self.assertEqual(drawer, d)
+ self.assertEqual(book, b)
+ self.assertEqual(socket, s)
+ self.assertEqual(entitlement, e)
+ self.assertEqual(dedicated, t)
+
+ def kernel_init(self):
+ """
+ We need a kernel supporting the CPU topology.
+ We need a minimal root filesystem with a shell.
+ """
+ kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
+ kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
+ kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
+ asset_hash=kernel_hash)
+
+ initrd_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img')
+ initrd_hash = 'a122057d95725ac030e2ec51df46e172'
+ initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
+ asset_hash=initrd_hash)
+ initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
+ archive.lzma_uncompress(initrd_path_xz, initrd_path)
+
+ self.vm.set_console()
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+ 'root=/dev/ram '
+ 'selinux=0 '
+ 'rdinit=/bin/sh')
+ self.vm.add_args('-nographic',
+ '-enable-kvm',
+ '-cpu', 'z14,ctop=on',
+ '-m', '512',
+ '-name', 'Some Guest Name',
+ '-uuid', '30de4fd9-b4d5-409e-86a5-09b387f70bfa',
+ '-kernel', kernel_path,
+ '-initrd', initrd_path,
+ '-append', kernel_command_line)
+
+ def test_single(self):
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ def test_default(self):
+ """
+ This test checks the implicite topology.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+ self.check_topology(1, 0, 0, 0, 'medium', False)
+ self.check_topology(2, 1, 0, 0, 'medium', False)
+ self.check_topology(3, 1, 0, 0, 'medium', False)
+ self.check_topology(4, 2, 0, 0, 'medium', False)
+ self.check_topology(5, 2, 0, 0, 'medium', False)
+ self.check_topology(6, 0, 1, 0, 'medium', False)
+ self.check_topology(7, 0, 1, 0, 'medium', False)
+ self.check_topology(8, 1, 1, 0, 'medium', False)
+ self.check_topology(9, 1, 1, 0, 'medium', False)
+ self.check_topology(10, 2, 1, 0, 'medium', False)
+ self.check_topology(11, 2, 1, 0, 'medium', False)
+ self.check_topology(12, 0, 0, 1, 'medium', False)
+
+ def test_move(self):
+ """
+ This test checks the topology modification by moving a CPU
+ to another socket: CPU 0 is moved from socket 0 to socket 2.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'socket-id': 2, 'entitlement': 'low'})
+ self.assertEqual(res['return'], {})
+ self.check_topology(0, 2, 0, 0, 'low', False)
+
+ def test_hotplug(self):
+ """
+ This test verifies that a CPU defined with '-device' command line
+ argument finds its right place inside the topology.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=10')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.check_topology(10, 2, 1, 0, 'medium', False)
+
+ def test_hotplug_full(self):
+ """
+ This test verifies that a hotplugged fully defined with '-device'
+ command line argument finds its right place inside the topology.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.add_args('-device',
+ 'z14-s390x-cpu,'
+ 'core-id=1,socket-id=1,book-id=1,drawer-id=1')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+ self.check_topology(1, 1, 1, 1, 'medium', False)
+
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 14/21] tests/avocado: s390x cpu topology core
2023-04-03 16:28 ` [PATCH v19 14/21] tests/avocado: s390x cpu topology core Pierre Morel
@ 2023-04-04 9:21 ` Cédric Le Goater
2023-04-04 12:17 ` Cédric Le Goater
2023-04-25 15:03 ` Pierre Morel
0 siblings, 2 replies; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 9:21 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> Introduction of the s390x cpu topology core functions and
> basic tests.
>
> We test the corelation between the command line and
> the QMP results in query-cpus-fast for various CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
I gave the tests a run on a z LPAR. Nice job !
I hope we can maintain the avocado framework to the level of
expectations. I find it very useful for such test cases.
Thanks,
C.
> ---
> MAINTAINERS | 1 +
> tests/avocado/s390_topology.py | 196 +++++++++++++++++++++++++++++++++
> 2 files changed, 197 insertions(+)
> create mode 100644 tests/avocado/s390_topology.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index fe5638e31d..41419840b0 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1662,6 +1662,7 @@ F: hw/s390x/cpu-topology.c
> F: target/s390x/kvm/cpu_topology.c
> F: docs/devel/s390-cpu-topology.rst
> F: docs/system/s390x/cpu-topology.rst
> +F: tests/avocado/s390_topology.py
>
> X86 Machines
> ------------
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> new file mode 100644
> index 0000000000..38e9cc4f16
> --- /dev/null
> +++ b/tests/avocado/s390_topology.py
> @@ -0,0 +1,196 @@
> +# Functional test that boots a Linux kernel and checks the console
> +#
> +# Copyright (c) 2023 IBM Corp.
> +#
> +# Author:
> +# Pierre Morel <pmorel@linux.ibm.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import os
> +import shutil
> +import time
> +
> +from avocado import skip
> +from avocado import skipUnless
> +from avocado import skipIf
> +from avocado_qemu import QemuSystemTest
> +from avocado_qemu import exec_command
> +from avocado_qemu import exec_command_and_wait_for_pattern
> +from avocado_qemu import interrupt_interactive_console_until_pattern
> +from avocado_qemu import wait_for_console_pattern
> +from avocado.utils import process
> +from avocado.utils import archive
> +
> +
> +class LinuxKernelTest(QemuSystemTest):
> + KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> +
> + def wait_for_console_pattern(self, success_message, vm=None):
> + wait_for_console_pattern(self, success_message,
> + failure_message='Kernel panic - not syncing',
> + vm=vm)
> +
> +
> +class S390CPUTopology(LinuxKernelTest):
> + """
> + S390x CPU topology consist of 4 topology layers, from bottom to top,
> + the cores, sockets, books and drawers and 2 modifiers attributes,
> + the entitlement and the dedication.
> + See: docs/system/s390x/cpu-topology.rst.
> +
> + S390x CPU topology is setup in different ways:
> + - implicitely from the '-smp' argument by completing each topology
> + level one after the other begining with drawer 0, book 0 and socket 0.
> + - explicitely from the '-device' argument on the QEMU command line
> + - explicitely by hotplug of a new CPU using QMP or HMP
> + - it is modified by using QMP 'set-cpu-topology'
> +
> + The S390x modifier attribute entitlement depends on the machine
> + polarization, which can be horizontal or vertical.
> + The polarization is changed on a request from the guest.
> + """
> + timeout = 90
> +
> +
> + def check_topology(self, c, s, b, d, e, t):
> + res = self.vm.qmp('query-cpus-fast')
> + line = res['return']
> + for x in line:
> + core = x['props']['core-id']
> + socket = x['props']['socket-id']
> + book = x['props']['book-id']
> + drawer = x['props']['drawer-id']
> + entitlement = x['entitlement']
> + dedicated = x['dedicated']
> + if core == c:
> + self.assertEqual(drawer, d)
> + self.assertEqual(book, b)
> + self.assertEqual(socket, s)
> + self.assertEqual(entitlement, e)
> + self.assertEqual(dedicated, t)
> +
> + def kernel_init(self):
> + """
> + We need a kernel supporting the CPU topology.
> + We need a minimal root filesystem with a shell.
> + """
> + kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> + '/fedora-secondary/releases/35/Server/s390x/os'
> + '/images/kernel.img')
> + kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
> + kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
> + asset_hash=kernel_hash)
> +
> + initrd_url = ('https://archives.fedoraproject.org/pub/archive'
> + '/fedora-secondary/releases/35/Server/s390x/os'
> + '/images/initrd.img')
> + initrd_hash = 'a122057d95725ac030e2ec51df46e172'
> + initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
> + asset_hash=initrd_hash)
> + initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
> + archive.lzma_uncompress(initrd_path_xz, initrd_path)
> +
> + self.vm.set_console()
> + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> + 'root=/dev/ram '
> + 'selinux=0 '
> + 'rdinit=/bin/sh')
> + self.vm.add_args('-nographic',
> + '-enable-kvm',
> + '-cpu', 'z14,ctop=on',
> + '-m', '512',
> + '-name', 'Some Guest Name',
> + '-uuid', '30de4fd9-b4d5-409e-86a5-09b387f70bfa',
> + '-kernel', kernel_path,
> + '-initrd', initrd_path,
> + '-append', kernel_command_line)
> +
> + def test_single(self):
> + self.kernel_init()
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + def test_default(self):
> + """
> + This test checks the implicite topology.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> + self.check_topology(1, 0, 0, 0, 'medium', False)
> + self.check_topology(2, 1, 0, 0, 'medium', False)
> + self.check_topology(3, 1, 0, 0, 'medium', False)
> + self.check_topology(4, 2, 0, 0, 'medium', False)
> + self.check_topology(5, 2, 0, 0, 'medium', False)
> + self.check_topology(6, 0, 1, 0, 'medium', False)
> + self.check_topology(7, 0, 1, 0, 'medium', False)
> + self.check_topology(8, 1, 1, 0, 'medium', False)
> + self.check_topology(9, 1, 1, 0, 'medium', False)
> + self.check_topology(10, 2, 1, 0, 'medium', False)
> + self.check_topology(11, 2, 1, 0, 'medium', False)
> + self.check_topology(12, 0, 0, 1, 'medium', False)
> +
> + def test_move(self):
> + """
> + This test checks the topology modification by moving a CPU
> + to another socket: CPU 0 is moved from socket 0 to socket 2.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> +
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> + res = self.vm.qmp('set-cpu-topology',
> + {'core-id': 0, 'socket-id': 2, 'entitlement': 'low'})
> + self.assertEqual(res['return'], {})
> + self.check_topology(0, 2, 0, 0, 'low', False)
> +
> + def test_hotplug(self):
> + """
> + This test verifies that a CPU defined with '-device' command line
> + argument finds its right place inside the topology.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.add_args('-device', 'z14-s390x-cpu,core-id=10')
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> +
> + self.check_topology(10, 2, 1, 0, 'medium', False)
> +
> + def test_hotplug_full(self):
> + """
> + This test verifies that a hotplugged fully defined with '-device'
> + command line argument finds its right place inside the topology.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.add_args('-device',
> + 'z14-s390x-cpu,'
> + 'core-id=1,socket-id=1,book-id=1,drawer-id=1')
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> + self.check_topology(1, 1, 1, 1, 'medium', False)
> +
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 14/21] tests/avocado: s390x cpu topology core
2023-04-04 9:21 ` Cédric Le Goater
@ 2023-04-04 12:17 ` Cédric Le Goater
2023-04-25 15:03 ` Pierre Morel
1 sibling, 0 replies; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 12:17 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 11:21, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> Introduction of the s390x cpu topology core functions and
>> basic tests.
>>
>> We test the corelation between the command line and
>> the QMP results in query-cpus-fast for various CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>
> I gave the tests a run on a z LPAR. Nice job !
I forgot to add some timing output :
$ build/tests/venv/bin/avocado --show=app run build/tests/avocado/s390_topology.py
(01/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_single: PASS (4.78 s)
(02/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_default: PASS (3.90 s)
(03/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_move: PASS (3.82 s)
(04/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_hotplug: PASS (3.84 s)
(05/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_hotplug_full: PASS (3.94 s)
(06/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_polarisation: PASS (4.59 s)
(07/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_entitlement: PASS (4.65 s)
(08/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_dedicated: PASS (4.65 s)
(09/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_socket_full: PASS (4.25 s)
(10/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_dedicated_error: PASS (4.46 s)
(11/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_move_error: PASS (4.22 s)
(12/12) build/tests/avocado/s390_topology.py:S390CPUTopology.test_query_polarization: PASS (4.63 s)
C.
^ permalink raw reply [flat|nested] 57+ messages in thread
* Re: [PATCH v19 14/21] tests/avocado: s390x cpu topology core
2023-04-04 9:21 ` Cédric Le Goater
2023-04-04 12:17 ` Cédric Le Goater
@ 2023-04-25 15:03 ` Pierre Morel
1 sibling, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-25 15:03 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 11:21, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> Introduction of the s390x cpu topology core functions and
>> basic tests.
>>
>> We test the corelation between the command line and
>> the QMP results in query-cpus-fast for various CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>
> I gave the tests a run on a z LPAR. Nice job !
:) Thanks
Pierre
>
> I hope we can maintain the avocado framework to the level of
> expectations. I find it very useful for such test cases.
>
> Thanks,
>
> C.
>
>
>> ---
>> MAINTAINERS | 1 +
>> tests/avocado/s390_topology.py | 196 +++++++++++++++++++++++++++++++++
>> 2 files changed, 197 insertions(+)
>> create mode 100644 tests/avocado/s390_topology.py
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index fe5638e31d..41419840b0 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1662,6 +1662,7 @@ F: hw/s390x/cpu-topology.c
>> F: target/s390x/kvm/cpu_topology.c
>> F: docs/devel/s390-cpu-topology.rst
>> F: docs/system/s390x/cpu-topology.rst
>> +F: tests/avocado/s390_topology.py
>> X86 Machines
>> ------------
>> diff --git a/tests/avocado/s390_topology.py
>> b/tests/avocado/s390_topology.py
>> new file mode 100644
>> index 0000000000..38e9cc4f16
>> --- /dev/null
>> +++ b/tests/avocado/s390_topology.py
>> @@ -0,0 +1,196 @@
>> +# Functional test that boots a Linux kernel and checks the console
>> +#
>> +# Copyright (c) 2023 IBM Corp.
>> +#
>> +# Author:
>> +# Pierre Morel <pmorel@linux.ibm.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later. See the COPYING file in the top-level directory.
>> +
>> +import os
>> +import shutil
>> +import time
>> +
>> +from avocado import skip
>> +from avocado import skipUnless
>> +from avocado import skipIf
>> +from avocado_qemu import QemuSystemTest
>> +from avocado_qemu import exec_command
>> +from avocado_qemu import exec_command_and_wait_for_pattern
>> +from avocado_qemu import interrupt_interactive_console_until_pattern
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import process
>> +from avocado.utils import archive
>> +
>> +
>> +class LinuxKernelTest(QemuSystemTest):
>> + KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>> +
>> + def wait_for_console_pattern(self, success_message, vm=None):
>> + wait_for_console_pattern(self, success_message,
>> + failure_message='Kernel panic - not
>> syncing',
>> + vm=vm)
>> +
>> +
>> +class S390CPUTopology(LinuxKernelTest):
>> + """
>> + S390x CPU topology consist of 4 topology layers, from bottom to
>> top,
>> + the cores, sockets, books and drawers and 2 modifiers attributes,
>> + the entitlement and the dedication.
>> + See: docs/system/s390x/cpu-topology.rst.
>> +
>> + S390x CPU topology is setup in different ways:
>> + - implicitely from the '-smp' argument by completing each topology
>> + level one after the other begining with drawer 0, book 0 and
>> socket 0.
>> + - explicitely from the '-device' argument on the QEMU command line
>> + - explicitely by hotplug of a new CPU using QMP or HMP
>> + - it is modified by using QMP 'set-cpu-topology'
>> +
>> + The S390x modifier attribute entitlement depends on the machine
>> + polarization, which can be horizontal or vertical.
>> + The polarization is changed on a request from the guest.
>> + """
>> + timeout = 90
>> +
>> +
>> + def check_topology(self, c, s, b, d, e, t):
>> + res = self.vm.qmp('query-cpus-fast')
>> + line = res['return']
>> + for x in line:
>> + core = x['props']['core-id']
>> + socket = x['props']['socket-id']
>> + book = x['props']['book-id']
>> + drawer = x['props']['drawer-id']
>> + entitlement = x['entitlement']
>> + dedicated = x['dedicated']
>> + if core == c:
>> + self.assertEqual(drawer, d)
>> + self.assertEqual(book, b)
>> + self.assertEqual(socket, s)
>> + self.assertEqual(entitlement, e)
>> + self.assertEqual(dedicated, t)
>> +
>> + def kernel_init(self):
>> + """
>> + We need a kernel supporting the CPU topology.
>> + We need a minimal root filesystem with a shell.
>> + """
>> + kernel_url = ('https://archives.fedoraproject.org/pub/archive'
>> + '/fedora-secondary/releases/35/Server/s390x/os'
>> + '/images/kernel.img')
>> + kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
>> + kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
>> + asset_hash=kernel_hash)
>> +
>> + initrd_url = ('https://archives.fedoraproject.org/pub/archive'
>> + '/fedora-secondary/releases/35/Server/s390x/os'
>> + '/images/initrd.img')
>> + initrd_hash = 'a122057d95725ac030e2ec51df46e172'
>> + initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
>> + asset_hash=initrd_hash)
>> + initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
>> + archive.lzma_uncompress(initrd_path_xz, initrd_path)
>> +
>> + self.vm.set_console()
>> + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> + 'root=/dev/ram '
>> + 'selinux=0 '
>> + 'rdinit=/bin/sh')
>> + self.vm.add_args('-nographic',
>> + '-enable-kvm',
>> + '-cpu', 'z14,ctop=on',
>> + '-m', '512',
>> + '-name', 'Some Guest Name',
>> + '-uuid',
>> '30de4fd9-b4d5-409e-86a5-09b387f70bfa',
>> + '-kernel', kernel_path,
>> + '-initrd', initrd_path,
>> + '-append', kernel_command_line)
>> +
>> + def test_single(self):
>> + self.kernel_init()
>> + self.vm.launch()
>> + self.wait_for_console_pattern('no job control')
>> + self.check_topology(0, 0, 0, 0, 'medium', False)
>> +
>> + def test_default(self):
>> + """
>> + This test checks the implicite topology.
>> +
>> + :avocado: tags=arch:s390x
>> + :avocado: tags=machine:s390-ccw-virtio
>> + """
>> + self.kernel_init()
>> + self.vm.add_args('-smp',
>> + '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
>> + self.vm.launch()
>> + self.wait_for_console_pattern('no job control')
>> + self.check_topology(0, 0, 0, 0, 'medium', False)
>> + self.check_topology(1, 0, 0, 0, 'medium', False)
>> + self.check_topology(2, 1, 0, 0, 'medium', False)
>> + self.check_topology(3, 1, 0, 0, 'medium', False)
>> + self.check_topology(4, 2, 0, 0, 'medium', False)
>> + self.check_topology(5, 2, 0, 0, 'medium', False)
>> + self.check_topology(6, 0, 1, 0, 'medium', False)
>> + self.check_topology(7, 0, 1, 0, 'medium', False)
>> + self.check_topology(8, 1, 1, 0, 'medium', False)
>> + self.check_topology(9, 1, 1, 0, 'medium', False)
>> + self.check_topology(10, 2, 1, 0, 'medium', False)
>> + self.check_topology(11, 2, 1, 0, 'medium', False)
>> + self.check_topology(12, 0, 0, 1, 'medium', False)
>> +
>> + def test_move(self):
>> + """
>> + This test checks the topology modification by moving a CPU
>> + to another socket: CPU 0 is moved from socket 0 to socket 2.
>> +
>> + :avocado: tags=arch:s390x
>> + :avocado: tags=machine:s390-ccw-virtio
>> + """
>> + self.kernel_init()
>> + self.vm.add_args('-smp',
>> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
>> + self.vm.launch()
>> + self.wait_for_console_pattern('no job control')
>> +
>> + self.check_topology(0, 0, 0, 0, 'medium', False)
>> + res = self.vm.qmp('set-cpu-topology',
>> + {'core-id': 0, 'socket-id': 2,
>> 'entitlement': 'low'})
>> + self.assertEqual(res['return'], {})
>> + self.check_topology(0, 2, 0, 0, 'low', False)
>> +
>> + def test_hotplug(self):
>> + """
>> + This test verifies that a CPU defined with '-device' command
>> line
>> + argument finds its right place inside the topology.
>> +
>> + :avocado: tags=arch:s390x
>> + :avocado: tags=machine:s390-ccw-virtio
>> + """
>> + self.kernel_init()
>> + self.vm.add_args('-smp',
>> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
>> + self.vm.add_args('-device', 'z14-s390x-cpu,core-id=10')
>> + self.vm.launch()
>> + self.wait_for_console_pattern('no job control')
>> +
>> + self.check_topology(10, 2, 1, 0, 'medium', False)
>> +
>> + def test_hotplug_full(self):
>> + """
>> + This test verifies that a hotplugged fully defined with
>> '-device'
>> + command line argument finds its right place inside the
>> topology.
>> +
>> + :avocado: tags=arch:s390x
>> + :avocado: tags=machine:s390-ccw-virtio
>> + """
>> + self.kernel_init()
>> + self.vm.add_args('-smp',
>> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
>> + self.vm.add_args('-device',
>> + 'z14-s390x-cpu,'
>> + 'core-id=1,socket-id=1,book-id=1,drawer-id=1')
>> + self.vm.launch()
>> + self.wait_for_console_pattern('no job control')
>> + self.check_topology(1, 1, 1, 1, 'medium', False)
>> +
>
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (13 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 14/21] tests/avocado: s390x cpu topology core Pierre Morel
@ 2023-04-03 16:28 ` Pierre Morel
2023-04-04 9:22 ` Cédric Le Goater
2023-04-03 16:29 ` [PATCH v19 16/21] tests/avocado: s390x cpu topology entitlement tests Pierre Morel
` (5 subsequent siblings)
20 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:28 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 38 +++++++++++++++++++++++++++++++++-
1 file changed, 37 insertions(+), 1 deletion(-)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 38e9cc4f16..a4bbbc2cb6 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -107,6 +107,15 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+ def system_init(self):
+ self.log.info("System init")
+ exec_command(self, 'mount proc -t proc /proc')
+ time.sleep(0.2)
+ exec_command(self, 'mount sys -t sysfs /sys')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
+
def test_single(self):
self.kernel_init()
self.vm.launch()
@@ -116,7 +125,6 @@ def test_single(self):
def test_default(self):
"""
This test checks the implicite topology.
-
:avocado: tags=arch:s390x
:avocado: tags=machine:s390-ccw-virtio
"""
@@ -194,3 +202,31 @@ def test_hotplug_full(self):
self.wait_for_console_pattern('no job control')
self.check_topology(1, 1, 1, 1, 'medium', False)
+ def test_polarisation(self):
+ """
+ This test verifies that QEMU modifies the entitlement change after
+ several guest polarization change requests.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '1')
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation
2023-04-03 16:28 ` [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation Pierre Morel
@ 2023-04-04 9:22 ` Cédric Le Goater
2023-04-04 12:26 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 9:22 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:28, Pierre Morel wrote:
> Polarization is changed on a request from the guest.
> Let's verify the polarization is accordingly set by QEMU.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 38 +++++++++++++++++++++++++++++++++-
> 1 file changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> index 38e9cc4f16..a4bbbc2cb6 100644
> --- a/tests/avocado/s390_topology.py
> +++ b/tests/avocado/s390_topology.py
> @@ -107,6 +107,15 @@ def kernel_init(self):
> '-initrd', initrd_path,
> '-append', kernel_command_line)
>
> + def system_init(self):
> + self.log.info("System init")
> + exec_command(self, 'mount proc -t proc /proc')
> + time.sleep(0.2)
> + exec_command(self, 'mount sys -t sysfs /sys')
> + time.sleep(0.2)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', '0')
> +
> def test_single(self):
> self.kernel_init()
> self.vm.launch()
> @@ -116,7 +125,6 @@ def test_single(self):
> def test_default(self):
> """
> This test checks the implicite topology.
> -
unwanted change I guess ?
> :avocado: tags=arch:s390x
> :avocado: tags=machine:s390-ccw-virtio
> """
> @@ -194,3 +202,31 @@ def test_hotplug_full(self):
> self.wait_for_console_pattern('no job control')
> self.check_topology(1, 1, 1, 1, 'medium', False)
>
> + def test_polarisation(self):
> + """
> + This test verifies that QEMU modifies the entitlement change after
> + several guest polarization change requests.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> +
> + self.system_init()
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
> + time.sleep(0.2)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', '1')
> +
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
> + time.sleep(0.2)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', '0')
> +
> + self.check_topology(0, 0, 0, 0, 'medium', False)
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation
2023-04-04 9:22 ` Cédric Le Goater
@ 2023-04-04 12:26 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 12:26 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 11:22, Cédric Le Goater wrote:
> On 4/3/23 18:28, Pierre Morel wrote:
>> Polarization is changed on a request from the guest.
>> Let's verify the polarization is accordingly set by QEMU.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> tests/avocado/s390_topology.py | 38 +++++++++++++++++++++++++++++++++-
>> 1 file changed, 37 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/avocado/s390_topology.py
>> b/tests/avocado/s390_topology.py
>> index 38e9cc4f16..a4bbbc2cb6 100644
>> --- a/tests/avocado/s390_topology.py
>> +++ b/tests/avocado/s390_topology.py
>> @@ -107,6 +107,15 @@ def kernel_init(self):
>> '-initrd', initrd_path,
>> '-append', kernel_command_line)
>> + def system_init(self):
>> + self.log.info("System init")
>> + exec_command(self, 'mount proc -t proc /proc')
>> + time.sleep(0.2)
>> + exec_command(self, 'mount sys -t sysfs /sys')
>> + time.sleep(0.2)
>> + exec_command_and_wait_for_pattern(self,
>> + '/bin/cat /sys/devices/system/cpu/dispatching', '0')
>> +
>> def test_single(self):
>> self.kernel_init()
>> self.vm.launch()
>> @@ -116,7 +125,6 @@ def test_single(self):
>> def test_default(self):
>> """
>> This test checks the implicite topology.
>> -
>
> unwanted change I guess ?
oh right
thanks
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 16/21] tests/avocado: s390x cpu topology entitlement tests
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (14 preceding siblings ...)
2023-04-03 16:28 ` [PATCH v19 15/21] tests/avocado: s390x cpu topology polarisation Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
2023-04-03 16:29 ` [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU Pierre Morel
` (4 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
This test takes care to check the changes on different entitlements
when the guest requests a polarization change.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 56 ++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index a4bbbc2cb6..f12f0ae148 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -230,3 +230,59 @@ def test_polarisation(self):
'/bin/cat /sys/devices/system/cpu/dispatching', '0')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ def test_set_cpu_topology_entitlement(self):
+ """
+ This test verifies that QEMU modifies the polarization
+ after a guest request.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=1')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=2')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=3')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+ self.assertEqual(res['return'], {})
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 1, 'entitlement': 'medium'})
+ self.assertEqual(res['return'], {})
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'entitlement': 'high'})
+ self.assertEqual(res['return'], {})
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 3, 'entitlement': 'high'})
+ self.assertEqual(res['return'], {})
+ self.check_topology(0, 0, 0, 0, 'low', False)
+ self.check_topology(1, 0, 0, 0, 'medium', False)
+ self.check_topology(2, 1, 0, 0, 'high', False)
+ self.check_topology(3, 1, 0, 0, 'high', False)
+
+ exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '1')
+
+ self.check_topology(0, 0, 0, 0, 'low', False)
+ self.check_topology(1, 0, 0, 0, 'medium', False)
+ self.check_topology(2, 1, 0, 0, 'high', False)
+ self.check_topology(3, 1, 0, 0, 'high', False)
+
+ exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
+
+ self.check_topology(0, 0, 0, 0, 'low', False)
+ self.check_topology(1, 0, 0, 0, 'medium', False)
+ self.check_topology(2, 1, 0, 0, 'high', False)
+ self.check_topology(3, 1, 0, 0, 'high', False)
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (15 preceding siblings ...)
2023-04-03 16:29 ` [PATCH v19 16/21] tests/avocado: s390x cpu topology entitlement tests Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
2023-04-04 9:19 ` Cédric Le Goater
2023-04-03 16:29 ` [PATCH v19 18/21] tests/avocado: s390x cpu topology test socket full Pierre Morel
` (3 subsequent siblings)
20 siblings, 1 reply; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 43 +++++++++++++++++++++++++++++++++-
1 file changed, 42 insertions(+), 1 deletion(-)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index f12f0ae148..6a41f08897 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -52,6 +52,7 @@ class S390CPUTopology(LinuxKernelTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+ skip_basis = False
def check_topology(self, c, s, b, d, e, t):
@@ -116,12 +117,14 @@ def system_init(self):
exec_command_and_wait_for_pattern(self,
'/bin/cat /sys/devices/system/cpu/dispatching', '0')
+ @skipIf(skip_basis, 'skipping basis tests')
def test_single(self):
self.kernel_init()
self.vm.launch()
self.wait_for_console_pattern('no job control')
self.check_topology(0, 0, 0, 0, 'medium', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_default(self):
"""
This test checks the implicite topology.
@@ -147,6 +150,7 @@ def test_default(self):
self.check_topology(11, 2, 1, 0, 'medium', False)
self.check_topology(12, 0, 0, 1, 'medium', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_move(self):
"""
This test checks the topology modification by moving a CPU
@@ -167,6 +171,7 @@ def test_move(self):
self.assertEqual(res['return'], {})
self.check_topology(0, 2, 0, 0, 'low', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_hotplug(self):
"""
This test verifies that a CPU defined with '-device' command line
@@ -184,6 +189,7 @@ def test_hotplug(self):
self.check_topology(10, 2, 1, 0, 'medium', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_hotplug_full(self):
"""
This test verifies that a hotplugged fully defined with '-device'
@@ -202,6 +208,7 @@ def test_hotplug_full(self):
self.wait_for_console_pattern('no job control')
self.check_topology(1, 1, 1, 1, 'medium', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_polarisation(self):
"""
This test verifies that QEMU modifies the entitlement change after
@@ -231,7 +238,7 @@ def test_polarisation(self):
self.check_topology(0, 0, 0, 0, 'medium', False)
- def test_set_cpu_topology_entitlement(self):
+ def test_entitlement(self):
"""
This test verifies that QEMU modifies the polarization
after a guest request.
@@ -286,3 +293,37 @@ def test_set_cpu_topology_entitlement(self):
self.check_topology(1, 0, 0, 0, 'medium', False)
self.check_topology(2, 1, 0, 0, 'high', False)
self.check_topology(3, 1, 0, 0, 'high', False)
+
+ def test_dedicated(self):
+ """
+ This test verifies that QEMU modifies the entitlement change correctly
+ for a dedicated CPU after several guest polarization change requests.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+ self.assertEqual(res['return'], {})
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '1')
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* Re: [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU
2023-04-03 16:29 ` [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU Pierre Morel
@ 2023-04-04 9:19 ` Cédric Le Goater
2023-04-04 12:02 ` Pierre Morel
0 siblings, 1 reply; 57+ messages in thread
From: Cédric Le Goater @ 2023-04-04 9:19 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/3/23 18:29, Pierre Morel wrote:
> A dedicated CPU in vertical polarization can only have
> a high entitlement.
> Let's check this.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 43 +++++++++++++++++++++++++++++++++-
> 1 file changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> index f12f0ae148..6a41f08897 100644
> --- a/tests/avocado/s390_topology.py
> +++ b/tests/avocado/s390_topology.py
> @@ -52,6 +52,7 @@ class S390CPUTopology(LinuxKernelTest):
> The polarization is changed on a request from the guest.
> """
> timeout = 90
> + skip_basis = False
>
This should come through its own patch.
> def check_topology(self, c, s, b, d, e, t):
> @@ -116,12 +117,14 @@ def system_init(self):
> exec_command_and_wait_for_pattern(self,
> '/bin/cat /sys/devices/system/cpu/dispatching', '0')
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_single(self):
> self.kernel_init()
> self.vm.launch()
> self.wait_for_console_pattern('no job control')
> self.check_topology(0, 0, 0, 0, 'medium', False)
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_default(self):
> """
> This test checks the implicite topology.
> @@ -147,6 +150,7 @@ def test_default(self):
> self.check_topology(11, 2, 1, 0, 'medium', False)
> self.check_topology(12, 0, 0, 1, 'medium', False)
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_move(self):
> """
> This test checks the topology modification by moving a CPU
> @@ -167,6 +171,7 @@ def test_move(self):
> self.assertEqual(res['return'], {})
> self.check_topology(0, 2, 0, 0, 'low', False)
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_hotplug(self):
> """
> This test verifies that a CPU defined with '-device' command line
> @@ -184,6 +189,7 @@ def test_hotplug(self):
>
> self.check_topology(10, 2, 1, 0, 'medium', False)
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_hotplug_full(self):
> """
> This test verifies that a hotplugged fully defined with '-device'
> @@ -202,6 +208,7 @@ def test_hotplug_full(self):
> self.wait_for_console_pattern('no job control')
> self.check_topology(1, 1, 1, 1, 'medium', False)
>
> + @skipIf(skip_basis, 'skipping basis tests')
> def test_polarisation(self):
> """
> This test verifies that QEMU modifies the entitlement change after
> @@ -231,7 +238,7 @@ def test_polarisation(self):
>
> self.check_topology(0, 0, 0, 0, 'medium', False)
>
> - def test_set_cpu_topology_entitlement(self):
> + def test_entitlement(self):
May be introduce the correct name in the first patch.
> """
> This test verifies that QEMU modifies the polarization
> after a guest request.
> @@ -286,3 +293,37 @@ def test_set_cpu_topology_entitlement(self):
> self.check_topology(1, 0, 0, 0, 'medium', False)
> self.check_topology(2, 1, 0, 0, 'high', False)
> self.check_topology(3, 1, 0, 0, 'high', False)
> +
> + def test_dedicated(self):
> + """
> + This test verifies that QEMU modifies the entitlement change correctly
> + for a dedicated CPU after several guest polarization change requests.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.launch()
> + self.wait_for_console_pattern('no job control')
> +
> + self.system_init()
> +
> + res = self.vm.qmp('set-cpu-topology',
> + {'core-id': 0, 'dedicated': True})
> + self.assertEqual(res['return'], {})
> +
> + self.check_topology(0, 0, 0, 0, 'high', True)
> +
> + exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
> + time.sleep(0.2)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', '1')
> +
> + self.check_topology(0, 0, 0, 0, 'high', True)
> +
> + exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
> + time.sleep(0.2)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', '0')
> +
> + self.check_topology(0, 0, 0, 0, 'high', True)
^ permalink raw reply [flat|nested] 57+ messages in thread* Re: [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU
2023-04-04 9:19 ` Cédric Le Goater
@ 2023-04-04 12:02 ` Pierre Morel
0 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-04 12:02 UTC (permalink / raw)
To: Cédric Le Goater, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange
On 4/4/23 11:19, Cédric Le Goater wrote:
> On 4/3/23 18:29, Pierre Morel wrote:
>> A dedicated CPU in vertical polarization can only have
>> a high entitlement.
>> Let's check this.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> tests/avocado/s390_topology.py | 43 +++++++++++++++++++++++++++++++++-
>> 1 file changed, 42 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/avocado/s390_topology.py
>> b/tests/avocado/s390_topology.py
>> index f12f0ae148..6a41f08897 100644
>> --- a/tests/avocado/s390_topology.py
>> +++ b/tests/avocado/s390_topology.py
>> @@ -52,6 +52,7 @@ class S390CPUTopology(LinuxKernelTest):
>> The polarization is changed on a request from the guest.
>> """
>> timeout = 90
>> + skip_basis = False
>
> This should come through its own patch.
OK, this is more a debug help, I wonder if I should not just remove it
>
>
>> def check_topology(self, c, s, b, d, e, t):
>> @@ -116,12 +117,14 @@ def system_init(self):
>> exec_command_and_wait_for_pattern(self,
>> '/bin/cat /sys/devices/system/cpu/dispatching', '0')
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_single(self):
>> self.kernel_init()
>> self.vm.launch()
>> self.wait_for_console_pattern('no job control')
>> self.check_topology(0, 0, 0, 0, 'medium', False)
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_default(self):
>> """
>> This test checks the implicite topology.
>> @@ -147,6 +150,7 @@ def test_default(self):
>> self.check_topology(11, 2, 1, 0, 'medium', False)
>> self.check_topology(12, 0, 0, 1, 'medium', False)
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_move(self):
>> """
>> This test checks the topology modification by moving a CPU
>> @@ -167,6 +171,7 @@ def test_move(self):
>> self.assertEqual(res['return'], {})
>> self.check_topology(0, 2, 0, 0, 'low', False)
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_hotplug(self):
>> """
>> This test verifies that a CPU defined with '-device'
>> command line
>> @@ -184,6 +189,7 @@ def test_hotplug(self):
>> self.check_topology(10, 2, 1, 0, 'medium', False)
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_hotplug_full(self):
>> """
>> This test verifies that a hotplugged fully defined with
>> '-device'
>> @@ -202,6 +208,7 @@ def test_hotplug_full(self):
>> self.wait_for_console_pattern('no job control')
>> self.check_topology(1, 1, 1, 1, 'medium', False)
>> + @skipIf(skip_basis, 'skipping basis tests')
>> def test_polarisation(self):
>> """
>> This test verifies that QEMU modifies the entitlement
>> change after
>> @@ -231,7 +238,7 @@ def test_polarisation(self):
>> self.check_topology(0, 0, 0, 0, 'medium', False)
>> - def test_set_cpu_topology_entitlement(self):
>> + def test_entitlement(self):
>
> May be introduce the correct name in the first patch.
right.
Thanks,
Pierre
^ permalink raw reply [flat|nested] 57+ messages in thread
* [PATCH v19 18/21] tests/avocado: s390x cpu topology test socket full
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (16 preceding siblings ...)
2023-04-03 16:29 ` [PATCH v19 17/21] tests/avocado: s390x cpu topology test dedicated CPU Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
2023-04-03 16:29 ` [PATCH v19 19/21] tests/avocado: s390x cpu topology dedicated errors Pierre Morel
` (2 subsequent siblings)
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
This test verifies that QMP set-cpu-topology does not accept
to overload a socket.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 6a41f08897..702b3a8443 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -327,3 +327,30 @@ def test_dedicated(self):
'/bin/cat /sys/devices/system/cpu/dispatching', '0')
self.check_topology(0, 0, 0, 0, 'high', True)
+
+ def test_socket_full(self):
+ """
+ This test verifies that QEMU does not accept to overload a socket.
+ The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+ not accept any new CPU while socket-id 0 on book-id 1 is free.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=1')
+ self.vm.add_args('-device', 'z14-s390x-cpu,core-id=2')
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1})
+ self.assertEqual(res['return'], {})
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 19/21] tests/avocado: s390x cpu topology dedicated errors
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (17 preceding siblings ...)
2023-04-03 16:29 ` [PATCH v19 18/21] tests/avocado: s390x cpu topology test socket full Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
2023-04-03 16:29 ` [PATCH v19 20/21] tests/avocado: s390x cpu topology bad move Pierre Morel
2023-04-03 16:29 ` [PATCH v19 21/21] tests/avocado: s390x cpu topology query-cpu-polarization Pierre Morel
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Let's test that QEMU refuses to setup a dedicated CPU with
low or medium entitlement.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 54 ++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 702b3a8443..02c6a8cbe8 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -238,6 +238,7 @@ def test_polarisation(self):
self.check_topology(0, 0, 0, 0, 'medium', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_entitlement(self):
"""
This test verifies that QEMU modifies the polarization
@@ -294,6 +295,7 @@ def test_entitlement(self):
self.check_topology(2, 1, 0, 0, 'high', False)
self.check_topology(3, 1, 0, 0, 'high', False)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_dedicated(self):
"""
This test verifies that QEMU modifies the entitlement change correctly
@@ -328,6 +330,7 @@ def test_dedicated(self):
self.check_topology(0, 0, 0, 0, 'high', True)
+ @skipIf(skip_basis, 'skipping basis tests')
def test_socket_full(self):
"""
This test verifies that QEMU does not accept to overload a socket.
@@ -354,3 +357,54 @@ def test_socket_full(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 2, 'socket-id': 0, 'book-id': 1})
self.assertEqual(res['return'], {})
+
+ def test_dedicated_error(self):
+ """
+ This test verifies that QEMU refuses to lower the entitlement
+ of a dedicated CPU
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+ self.assertEqual(res['return'], {})
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '1')
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated': True})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated': True})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated': False})
+ self.assertEqual(res['return'], {})
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated': False})
+ self.assertEqual(res['return'], {})
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 20/21] tests/avocado: s390x cpu topology bad move
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (18 preceding siblings ...)
2023-04-03 16:29 ` [PATCH v19 19/21] tests/avocado: s390x cpu topology dedicated errors Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
2023-04-03 16:29 ` [PATCH v19 21/21] tests/avocado: s390x cpu topology query-cpu-polarization Pierre Morel
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
This test verifies that QEMU refuses to move a CPU to an
unexistant location.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 02c6a8cbe8..665876affe 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -408,3 +408,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated': False})
self.assertEqual(res['return'], {})
+
+ def test_move_error(self):
+ """
+ This test verifies that QEMU refuses to move a CPU to an
+ unexistant location
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread* [PATCH v19 21/21] tests/avocado: s390x cpu topology query-cpu-polarization
2023-04-03 16:28 [PATCH v19 00/21] s390x: CPU Topology Pierre Morel
` (19 preceding siblings ...)
2023-04-03 16:29 ` [PATCH v19 20/21] tests/avocado: s390x cpu topology bad move Pierre Morel
@ 2023-04-03 16:29 ` Pierre Morel
20 siblings, 0 replies; 57+ messages in thread
From: Pierre Morel @ 2023-04-03 16:29 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
This test verifies that the qmp query query-cpu-polarization
reports correctly the polarization when the guest requests
a polarization change.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
tests/avocado/s390_topology.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 665876affe..88af48ba71 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -433,3 +433,36 @@ def test_move_error(self):
self.assertEqual(res['error']['class'], 'GenericError')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ def test_query_polarization(self):
+ """
+ This test verifies that query-cpu-polarization gives the right
+ answer
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_for_console_pattern('no job control')
+
+ self.system_init()
+
+ res = self.vm.qmp('query-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
+
+ exec_command(self, 'echo 1 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '1')
+
+ res = self.vm.qmp('query-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'vertical')
+
+ exec_command(self, 'echo 0 > /sys/devices/system/cpu/dispatching')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
+
+ res = self.vm.qmp('query-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
--
2.31.1
^ permalink raw reply related [flat|nested] 57+ messages in thread