* [PATCH v22 00/20] s390x: CPU Topology
@ 2023-09-01 15:57 Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 01/20] CPU topology: extend with s390 specifics Nina Schoetterl-Glausch
` (19 more replies)
0 siblings, 20 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal
Changes since v21 (range-diff below):
* fix ordering of entries in topology list
* don't leak topology list on error condition in insert_stsi_15_1_x
* make entitlement, dedication optional in query-cpu-info-fast
* rename query-cpu-polarization to query-s390x-cpu-polarization
* documentation changes
* add tests of guest view to avocado tests
* require KVM in avocado tests
* increase timeout in tests
* bump version in comments documenting availability to 8.2
* rename some symbols
* picked up R-b's (thanks Thomas)
* minor stuff, typos
Implementation discussions
==========================
CPU models
----------
Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already
in the CPU model for old QEMU we could not activate it as usual from
KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
Checking and enabling this capability enables facility 11,
S390_FEAT_CONFIGURATION_TOPOLOGY.
It is the responsibility of the admin to ensure the same CPU
model for source and target host in a migration.
Migration
---------
When the target guest is started, the Multi-processor Topology Change
Report (MTCR) bit is set during the creation of the vCPU by KVM.
We do not need to migrate its state, in the worst case, the target
guest will see the MTCR and actualize its view of the topology
without necessity, but this will be done only one time.
Reset
-----
Reseting the topology is done during subsystem reset, the
polarization is reset to horizontal polarization.
Topology attributes
-------------------
The topology attributes are carried by the CPU object and defined
on object creation.
In the case the new attributes, socket, book, drawer, dedicated,
entitlement are not provided QEMU provides defaults values.
- Geometry defaults
The geometry default are based on the core-id of the core to
fill the geometry in a monotone way starting with drawer 0,
book 0, and filling socket 0 with the number of cores per socket,
then filling socket 1, socket 2 ... etc until the book is complete
and all books until the first drawer is complete before starting with
the next drawer.
This allows to keep existing start scripts and Libvirt existing
interface until it is extended.
- Modifiers defaults
Default entitlement is medium
Default dedication is not dedicated.
- Machine polarization default to horizontal
Dynamic topology modification
-----------------------------
QAPI interface is extended with:
- a command: 'set-cpu-topology'
- a query: 'query-cpu-polarization'
- a query: extension of qmp 'query-cpus-fast'
- a query: extension of hmp 'hotpluggable-cpus'
- an event: 'CPU_POLARITY_CHANGE'
New command and interface are specified as unstable.
The admin may use query-cpus-fast to verify the topology provided
to the guest and set-cpu-topology to modify it.
The event CPU_POLARITY_CHANGE is sent when the guest successfuly
uses the PTF(2) instruction to request a polarization change.
In that case, the admin is supposed to modify the CPU provisioning
accordingly.
Testing
=======
To use the QEMU patches, you will need Linux V6-rc1 or newer,
or use the following Linux mainline patches:
f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report
24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function
0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..
Currently this code is for KVM only, I have no idea if it is interesting
to provide a TCG patch. If ever it will be done in another series.
This series provide 12 avocado tests using Fedora-35 kernel and initrd
image.
Documentation
=============
To have a better understanding of the S390x CPU Topology and its
implementation in QEMU you can have a look at the documentation in the
last patch of this series.
The admin will want to match the host and the guest topology, taking
into account that the guest does not recognize multithreading.
Consequently, two vCPU assigned to threads of the same real CPU should
preferably be assigned to the same socket of the guest machine.
Previous changes:
Since v20:
- changed name of target/s390/kvm/cpu_topology to
target/s390/kvm/stsi-topology
(Thomas, Cedric)
- moved the TLE list head from a global to a local in
insert_stsi_15_1_x()
(Nina)
- cleaning and merging some of the avocado tests
(Nina)
- Several cleanings
(Cedric, Thomas, Nina)
- moved setting of entitlement and dedicated from disapeared
cpustate_to_cpuinfo_s390() to new s390_query_cpu_fast()
- small changes to documentation to reflect last changes using
enum instead of int for polarization.
0 -> horizontal and 1 -> vertical
Since v19:
- use enum to specify the entitlement
(Nina)
- Change default entitlement to "auto"
- suppress skip_basis in avocado tests
(after comment from Cedric)
- Correction of the documentation
(Cedric)
- better code organization for s390_topology_add_core_to_socket
and s390_socket_nb
(Cedric)
- Changed Copyright to respect IBM policy
(Nina)
- set vertical_polarization back into s390_topology
(Nina)
Since v18:
- Changed default entitlement to S390_CPU_ENTITLEMENT__MAX
because no default can be correct.
- added polarization entry to the S390CcwMachineState
which allow to suppress shadow entitlement
- Suppress shadow entitlement
(Nina)
- Added query-cpu-polarization
- Added more avocado tests
- modified cpu_hierarchy_to_string to look better
(Nina)
Since v17:
- bug correction in handling PTF
- added avocado tests
(Thomas)
- Change comments on QEMU machine to 8.1
Since v16:
- documentation, split, bug correction and rephrasing
(Nina, Thomas)
- create machine-common.json
- use of entitlement_shadow to keep track of the entitlement
- adding drawers and books to query-hotpluggable-cpus
- keep hmp interface for set-cpu-topology for a future series
Since v15:
- Use Enum for polarity and entitlement
(Nina)
- move kvm_vm_enable_cap(KVM_CAP_S390_CPU_TOPOLOGY) to
kvm_arch_init()
(Thomas)
- Make all CPU attributes optional for set-cpu-topology monitor
command
(Thomas, Nina)
- Change use of the prefix "x-" to the use of feature unstable
to declare set-cpu-topology as unstable.
(Nina)
- Make CPU_POLARITY_CHANGE even as unstable
(Nina)
- Documentation update
(Thomas, Nina)
Since v14:
- move the ordering of TLE to just before filling the SYSIB,
optimize TLE ordering to be done on need only.
(Cedric in previous series)
- remove 'query-topology' and instead extend 'query-cpus-fast'
(Daniel)
- rename POLARITY_CHANGE to CPU_POLARITY_CHANGE
(Thomas)
- Divers bugs correction and doc changes
(Thomas, Nina)
- Separate topology and entitlement, simplify pft handling
(Nina)
- add the resetting of all CPU to horizontal polarity
once implementing PTF interpretation
Since v13:
- Suppress the topology device to simplify the code
(Cedric)
- moved reset of MTCR from device reset into subsystem
reset and removed previous reviewed-by from Nico and
Janis
- No need for Migration
- No need for machine dependencies
(Christian, Thomas)
- Adding all features, drawer/book and dynamic
(Cedric)
- since v12
- suppress new CPU flag "disable-topology" just use ctop
- no use of special fields in CCW machine or in CPU
- modifications in documentation
- insert documentation in tree
(Cedric)
- moved cpu-topology.c from target/s390 to target/s390/kvm
to compile smoothly (without topology) for TCG
(Cedric)
- since v11
- new CPU flag "disable-topology"
I would have take "topology" if I was able to have
it false on default.
(Christian, Thomas)
- Build the topology during the interception of the
STSI instruction.
(Cedric)
- return CC3 in case the calculated SYSIB length is
greater than 4096.
(Janis)
- minor corections on documentation
- since v10
- change machine attribute "topology-disable" to "topology"
(Cedric)
- Add preliminary patch for machine properties
(Cedric)
- Use next machine as 7.2
(Cedric / Connie)
- Remove unecessary mutex
(Thomas)
- use ENOTSUP return value for kvm_s390_topology_set_mtcr()
(Cedric)
- Add explanation on container and cpu TLEs
(Thomas)
- use again cpu and socket count in topology structure
(Cedric)
- Suppress the S390TopoTLE structure and integrate
the TLE masks to the socket structure.
(-)
- the STSI instruction now finds the topology from the machine
(Cedric)
- since v9
- remove books and drawers
- remove thread denying and replace with a merge
of cores * threads to specify the CPUs available
to the guest
- add a class option to avoid topology on older
machines
(Cedric)
- Allocate a SYSIB buffer of the maximal length to
avoid overflow.
(Nico, Janis)
- suppress redundancy of smp parameters in topology
and use directly the machine smp structure
- Early check for topology support
(Cedric)
- since v8
- Linux patches are now mainline
- simplification of the implementation
(Janis)
- Migration, new machine definition
(Thomas)
- Documentation
- since v7
- Coherence with the Linux patch series changes for MTCR get
(Pierre)
- check return values during new CPU creation
(Thomas)
- Improving codding style and argument usages
(Thomas)
- since v6
- Changes on smp args in qemu-options
(Daniel)
- changed comments in machine.jason
(Daniel)
- Added reset
(Janosch)
- since v5
- rebasing on newer QEMU version
- reworked most lines above 80 characters.
- since v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
Pierre Morel (20):
CPU topology: extend with s390 specifics
s390x/cpu topology: add topology entries on CPU hotplug
target/s390x/cpu topology: handle STSI(15) and build the SYSIB
s390x/sclp: reporting the maximum nested topology entries
s390x/cpu topology: resetting the Topology-Change-Report
s390x/cpu topology: interception of PTF instruction
target/s390x/cpu topology: activate CPU topology
qapi/s390x/cpu topology: set-cpu-topology qmp command
machine: adding s390 topology to query-cpu-fast
machine: adding s390 topology to info hotpluggable-cpus
qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
qapi/s390x/cpu topology: query-cpu-polarization qmp command
docs/s390x/cpu topology: document s390x cpu topology
tests/avocado: s390x cpu topology core
tests/avocado: s390x cpu topology polarization
tests/avocado: s390x cpu topology entitlement tests
tests/avocado: s390x cpu topology test dedicated CPU
tests/avocado: s390x cpu topology test socket full
tests/avocado: s390x cpu topology dedicated errors
tests/avocado: s390x cpu topology bad move
MAINTAINERS | 10 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 171 ++++++++++
docs/system/s390x/cpu-topology.rst | 242 ++++++++++++++
docs/system/target-s390x.rst | 1 +
qapi/machine-common.json | 21 ++
qapi/machine-target.json | 113 +++++++
qapi/machine.json | 30 +-
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +
include/hw/s390x/cpu-topology.h | 80 +++++
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
target/s390x/cpu.h | 83 +++++
target/s390x/kvm/kvm_s390x.h | 1 +
hw/core/machine-hmp-cmds.c | 6 +
hw/core/machine-smp.c | 48 ++-
hw/core/machine.c | 4 +
hw/core/qdev-properties-system.c | 13 +
hw/s390x/cpu-topology.c | 475 ++++++++++++++++++++++++++++
hw/s390x/s390-virtio-ccw.c | 27 +-
hw/s390x/sclp.c | 5 +
softmmu/vl.c | 6 +
target/s390x/cpu-sysemu.c | 13 +
target/s390x/cpu.c | 16 +
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 42 ++-
target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++
hw/s390x/meson.build | 1 +
qapi/meson.build | 1 +
qemu-options.hx | 7 +-
target/s390x/kvm/meson.build | 3 +-
tests/avocado/s390_topology.py | 432 +++++++++++++++++++++++++
33 files changed, 2153 insertions(+), 20 deletions(-)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
create mode 100644 qapi/machine-common.json
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 target/s390x/kvm/stsi-topology.c
create mode 100644 tests/avocado/s390_topology.py
Range-diff against v21:
1: 890004859f ! 1: c26d3df9f9 s390x/cpu topology: add s390 specifics to CPU topology
@@ Metadata
Author: Pierre Morel <pmorel@linux.ibm.com>
## Commit message ##
- s390x/cpu topology: add s390 specifics to CPU topology
+ CPU topology: extend with s390 specifics
S390 adds two new SMP levels, drawers and books to the CPU
topology.
- The S390 CPU have specific topology features like dedication
- and entitlement to give to the guest indications on the host
- vCPUs scheduling and help the guest take the best decisions
- on the scheduling of threads on the vCPUs.
+ S390 CPUs have specific topology features like dedication and
+ entitlement. These indicate to the guest information on host
+ vCPU scheduling and help the guest make better scheduling decisions.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## qapi/machine-common.json (new) ##
@@
@@ qapi/machine-common.json (new)
+# An enumeration of cpu entitlements that can be assumed by a virtual
+# S390 CPU
+#
-+# Since: 8.1
++# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
-+
## qapi/machine.json ##
@@
@@ qapi/machine.json
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
+-# virtual CPU belongs to, provided if supported by board
+# @props: properties describing to which node/drawer/book/socket/core/thread
- # virtual CPU belongs to, provided if supported by board
++# virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
+ # additional fields will be listed (since 3.0)
@@
- # a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
+ #
-# @socket-id: socket number within node/board the CPU belongs to
-+# @drawer-id: drawer number within node/board the CPU belongs to (since 8.1)
-+# @book-id: book number within drawer/node/board the CPU belongs to (since 8.1)
++# @drawer-id: drawer number within node/board the CPU belongs to (since 8.2)
++#
++# @book-id: book number within drawer/node/board the CPU belongs to (since 8.2)
++#
+# @socket-id: socket number within book/node/board the CPU belongs to
+ #
# @die-id: die number within socket the CPU belongs to (since 4.1)
- # @cluster-id: cluster number within die the CPU belongs to (since 7.1)
- # @core-id: core number within cluster the CPU belongs to
+ #
+@@
+ #
# @thread-id: thread number within core the CPU belongs to
#
--# Note: currently there are 6 properties that could be present
-+# Note: currently there are 8 properties that could be present
- # but management should be prepared to pass through other
- # properties with device_add command to allow for future
- # interface extension. This also requires the filed names to be kept in
+-# Note: currently there are 6 properties that could be present but
++# Note: currently there are 8 properties that could be present but
+ # management should be prepared to pass through other properties
+ # with device_add command to allow for future interface extension.
+ # This also requires the filed names to be kept in sync with the
@@
##
{ 'struct': 'CpuInstanceProperties',
@@ qapi/machine.json
#
# @cpus: number of virtual CPUs in the virtual machine
#
-+# @drawers: number of drawers in the CPU topology (since 8.1)
++# @drawers: number of drawers in the CPU topology (since 8.2)
+#
-+# @books: number of books in the CPU topology (since 8.1)
++# @books: number of books in the CPU topology (since 8.2)
+#
# @sockets: number of sockets in the CPU topology
#
@@ include/hw/qdev-properties-system.h: extern const PropertyInfo qdev_prop_pcie_li
## target/s390x/cpu.h ##
@@
- #include "cpu_models.h"
#include "exec/cpu-defs.h"
#include "qemu/cpu-float.h"
+ #include "tcg/tcg_s390x.h"
+#include "qapi/qapi-types-machine-common.h"
#define ELF_MACHINE_UNAME "S390X"
2: 282bd17cab ! 2: 6d32f1e3d4 s390x/cpu topology: add topology entries on CPU hotplug
@@ Commit message
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## MAINTAINERS ##
@@ MAINTAINERS: F: hw/s390x/event-facility.c
@@ MAINTAINERS: F: hw/s390x/event-facility.c
L: qemu-s390x@nongnu.org
+S390 CPU topology
-+M: Pierre Morel <pmorel@linux.ibm.com>
++M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
@@ include/hw/s390x/cpu-topology.h (new)
+/*
+ * CPU Topology
+ *
-+ * Copyright IBM Corp. 2022,2023
++ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ */
@@ hw/s390x/cpu-topology.c (new)
+/*
+ * CPU Topology
+ *
-+ * Copyright IBM Corp. 2022,2023
++ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * S390 topology handling can be divided in two parts:
@@ hw/s390x/cpu-topology.c (new)
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ * per socket.
-+ * .smp: keeps track of the machine topology.
-+ *
+ */
+S390Topology s390_topology = {
-+ /* will be initialized after the cpu model is realized */
++ /* will be initialized after the CPU model is realized */
+ .cores_per_socket = NULL,
+};
+
@@ hw/s390x/cpu-topology.c (new)
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
-+static int __s390_socket_nb(int drawer_id, int book_id, int socket_id)
++static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+ return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
@@ hw/s390x/cpu-topology.c (new)
+ */
+static int s390_socket_nb(S390CPU *cpu)
+{
-+ return __s390_socket_nb(cpu->env.drawer_id, cpu->env.book_id,
-+ cpu->env.socket_id);
++ return s390_socket_nb_from_ids(cpu->env.drawer_id, cpu->env.book_id,
++ cpu->env.socket_id);
+}
+
+/**
+ * s390_has_topology:
+ *
-+ * Return value: if the topology is supported by the machine.
++ * Return: true if the topology is supported by the machine.
+ */
+bool s390_has_topology(void)
+{
@@ hw/s390x/cpu-topology.c (new)
+
+ /*
+ * When the user specifies the entitlement as 'auto' on the command line,
-+ * qemu will set the entitlement as:
++ * QEMU will set the entitlement as:
+ * Medium when the CPU is not dedicated.
+ * High when dedicated is true.
+ */
@@ hw/s390x/cpu-topology.c (new)
+ }
+ if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
+ entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
-+ error_setg(errp, "A dedicated cpu implies high entitlement");
++ error_setg(errp, "A dedicated CPU implies high entitlement");
+ return false;
+ }
+ return true;
@@ hw/s390x/cpu-topology.c (new)
+ int entry;
+
+ /*
-+ * We do not want to initialize the topology if the cpu model
++ * We do not want to initialize the topology if the CPU model
+ * does not support topology, consequently, we have to wait for
+ * the first CPU to be realized, which realizes the CPU model
+ * to initialize the topology structures.
+ *
-+ * s390_topology_setup_cpu() is called from the cpu hotplug.
++ * s390_topology_setup_cpu() is called from the CPU hotplug.
+ */
+ if (!s390_topology.cores_per_socket) {
+ s390_topology_init(ms);
@@ hw/s390x/cpu-topology.c (new)
## hw/s390x/s390-virtio-ccw.c ##
@@
- #include "hw/s390x/pv.h"
+ #include "target/s390x/kvm/pv.h"
#include "migration/blocker.h"
#include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
@@ hw/s390x/s390-virtio-ccw.c: static const CPUArchIdList *s390_possible_cpu_arch_i
## hw/s390x/meson.build ##
@@ hw/s390x/meson.build: s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
+ 's390-skeys-kvm.c',
's390-stattrib-kvm.c',
- 'pv.c',
's390-pci-kvm.c',
+ 'cpu-topology.c',
))
3: 1952150e75 ! 3: ab54bb7881 target/s390x/cpu topology: handle STSI(15) and build the SYSIB
@@ Commit message
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## MAINTAINERS ##
-@@ MAINTAINERS: M: Pierre Morel <pmorel@linux.ibm.com>
+@@ MAINTAINERS: M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
@@ MAINTAINERS: M: Pierre Morel <pmorel@linux.ibm.com>
X86 Machines
------------
+ ## qapi/machine-target.json ##
+@@
+ 'TARGET_MIPS',
+ 'TARGET_LOONGARCH64',
+ 'TARGET_RISCV' ] } }
++
++##
++# @CpuS390Polarization:
++#
++# An enumeration of cpu polarization that can be assumed by a virtual
++# S390 CPU
++#
++# Since: 8.2
++##
++{ 'enum': 'CpuS390Polarization',
++ 'prefix': 'S390_CPU_POLARIZATION',
++ 'data': [ 'horizontal', 'vertical' ],
++ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
++}
+
## include/hw/s390x/cpu-topology.h ##
@@
#include "hw/boards.h"
@@ include/hw/s390x/cpu-topology.h
+typedef union s390_topology_id {
+ uint64_t id;
+ struct {
-+ uint8_t sentinel;
++ uint8_t _reserved0;
+ uint8_t drawer;
+ uint8_t book;
+ uint8_t socket;
-+ uint8_t dedicated;
-+ uint8_t entitlement;
+ uint8_t type;
++ uint8_t inv_polarization;
++ uint8_t not_dedicated;
+ uint8_t origin;
+ };
+} s390_topology_id;
@@ include/hw/s390x/cpu-topology.h
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
-+ bool vertical_polarization;
++ CpuS390Polarization polarization;
} S390Topology;
+typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
@@ target/s390x/cpu.h: typedef union SysIB {
+ *
+ */
+/* Container type Topology List Entry */
-+typedef struct SysIBTl_container {
++typedef struct SYSIBContainerListEntry {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
-+} SysIBTl_container;
-+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
++} SYSIBContainerListEntry;
++QEMU_BUILD_BUG_ON(sizeof(SYSIBContainerListEntry) != 8);
+
+/* CPU type Topology List Entry */
-+typedef struct SysIBTl_cpu {
++typedef struct SysIBCPUListEntry {
+ uint8_t nl;
+ uint8_t reserved0[3];
+#define SYSIB_TLE_POLARITY_MASK 0x03
@@ target/s390x/cpu.h: typedef union SysIB {
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
-+} SysIBTl_cpu;
-+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
++} SysIBCPUListEntry;
++QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
+
-+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar);
++void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
@@ target/s390x/cpu.h: typedef union SysIB {
## hw/s390x/cpu-topology.c ##
@@
+ * s390_topology is used to keep the topology information.
* .cores_per_socket: tracks information on the count of cores
* per socket.
- * .smp: keeps track of the machine topology.
-- *
-+ * .list: queue the topology entries inside which
-+ * we keep the information on the CPU topology.
++ * .polarization: tracks machine polarization.
*/
S390Topology s390_topology = {
- /* will be initialized after the cpu model is realized */
+ /* will be initialized after the CPU model is realized */
.cores_per_socket = NULL,
-+ .vertical_polarization = false,
++ .polarization = S390_CPU_POLARIZATION_HORIZONTAL,
};
/**
@@ target/s390x/kvm/kvm.c: static int handle_stsi(S390CPU *cpu)
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
-+ run->s390_stsi.ar);
++ run->s390_stsi.ar, RA_IGNORED);
+ return 0;
default:
return 0;
@@ target/s390x/kvm/stsi-topology.c (new)
+/*
+ * QEMU S390x CPU Topology
+ *
-+ * Copyright IBM Corp. 2022,2023
++ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
-+#include "hw/s390x/pv.h"
-+#include "hw/sysbus.h"
+#include "hw/s390x/sclp.h"
+#include "hw/s390x/cpu-topology.h"
+
++QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_LOW != 1);
++QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_MEDIUM != 2);
++QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_HIGH != 3);
++
+/**
+ * fill_container:
+ * @p: The address of the container TLE to fill
@@ target/s390x/kvm/stsi-topology.c (new)
+ */
+static char *fill_container(char *p, int level, int id)
+{
-+ SysIBTl_container *tle = (SysIBTl_container *)p;
++ SYSIBContainerListEntry *tle = (SYSIBContainerListEntry *)p;
+
+ tle->nl = level;
+ tle->id = id;
@@ target/s390x/kvm/stsi-topology.c (new)
+ */
+static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
+{
-+ SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
++ SysIBCPUListEntry *tle = (SysIBCPUListEntry *)p;
+ s390_topology_id topology_id = entry->id;
+
+ tle->nl = 0;
-+ if (topology_id.dedicated) {
-+ tle->flags = SYSIB_TLE_DEDICATED;
++ tle->flags = 3 - topology_id.inv_polarization;
++ if (!topology_id.not_dedicated) {
++ tle->flags |= SYSIB_TLE_DEDICATED;
+ }
-+ tle->flags |= topology_id.entitlement;
+ tle->type = topology_id.type;
+ tle->origin = cpu_to_be16(topology_id.origin * 64);
+ tle->mask = cpu_to_be64(entry->mask);
@@ target/s390x/kvm/stsi-topology.c (new)
+ */
+#define SYSIB_GUARD(data, x) do { \
+ data += x; \
-+ if (data > sizeof(SysIB)) { \
++ if (data > sizeof(SysIB)) { \
+ return 0; \
+ } \
+ } while (0)
@@ target/s390x/kvm/stsi-topology.c (new)
+ bool book_change = drawer_change || last_book != entry->id.book;
+ bool socket_change = book_change || last_socket != entry->id.socket;
+
-+ /* If we reach the sentinel get out */
-+ if (entry->id.sentinel) {
-+ break;
-+ }
-+
+ if (level > 3 && drawer_change) {
-+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
++ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 3, drawer_id++);
+ book_id = 0;
+ }
+ if (level > 2 && book_change) {
-+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
++ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 2, book_id++);
+ socket_id = 0;
+ }
+ if (socket_change) {
-+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
++ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 1, socket_id++);
+ }
+
-+ SYSIB_GUARD(n, sizeof(SysIBTl_cpu));
++ SYSIB_GUARD(n, sizeof(SysIBCPUListEntry));
+ p = fill_tle_cpu(p, entry);
+ last_drawer = entry->id.drawer;
+ last_book = entry->id.book;
@@ target/s390x/kvm/stsi-topology.c (new)
+ * @entry: Topology entry to setup
+ * @cpu: the S390CPU to add
+ *
-+ * Set the core bit inside the topology mask and
-+ * increments the number of cores for the socket.
++ * Set the core bit inside the topology mask.
+ */
+static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
+ S390CPU *cpu)
@@ target/s390x/kvm/stsi-topology.c (new)
+
+/**
+ * s390_topology_from_cpu:
-+ * @cpu: The S390CPU
++ * @cpu: S390CPU to calculate the topology id
+ *
+ * Initialize the topology id from the CPU environment.
+ */
@@ target/s390x/kvm/stsi-topology.c (new)
+ topology_id.drawer = cpu->env.drawer_id;
+ topology_id.book = cpu->env.book_id;
+ topology_id.socket = cpu->env.socket_id;
-+ topology_id.origin = cpu->env.core_id / 64;
+ topology_id.type = S390_TOPOLOGY_CPU_IFL;
-+ topology_id.dedicated = cpu->env.dedicated;
++ topology_id.not_dedicated = !cpu->env.dedicated;
+
-+ if (s390_topology.vertical_polarization) {
-+ topology_id.entitlement = cpu->env.entitlement;
++ topology_id.inv_polarization = 3;
++ if (s390_topology.polarization == S390_CPU_POLARIZATION_VERTICAL) {
++ topology_id.inv_polarization -= cpu->env.entitlement;
+ }
+
-+ return topology_id;
-+}
-+
-+/**
-+ * s390_topology_insert:
-+ * @cpu: s390CPU insert.
-+ *
-+ * Parse the topology list to find if the entry already
-+ * exist and add the core in it.
-+ * If it does not exist, allocate a new entry and insert
-+ * it in the queue from lower id to greater id.
-+ */
-+static void s390_topology_insert(S390TopologyList *topology_list, S390CPU *cpu)
-+{
-+ s390_topology_id id = s390_topology_from_cpu(cpu);
-+ S390TopologyEntry *entry = NULL;
-+ S390TopologyEntry *tmp = NULL;
++ topology_id.origin = cpu->env.core_id / 64;
+
-+ QTAILQ_FOREACH(tmp, topology_list, next) {
-+ if (id.id == tmp->id.id) {
-+ s390_topology_add_cpu_to_entry(tmp, cpu);
-+ return;
-+ } else if (id.id < tmp->id.id) {
-+ entry = g_malloc0(sizeof(S390TopologyEntry));
-+ entry->id.id = id.id;
-+ s390_topology_add_cpu_to_entry(entry, cpu);
-+ QTAILQ_INSERT_BEFORE(tmp, entry, next);
-+ return;
-+ }
-+ }
++ return topology_id;
+}
+
+/**
@@ target/s390x/kvm/stsi-topology.c (new)
+static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
+{
+ CPUState *cs;
++ S390TopologyEntry sentinel;
++
++ QTAILQ_INIT(topology_list);
++
++ sentinel.id.id = cpu_to_be64(UINT64_MAX);
++ QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
+
+ CPU_FOREACH(cs) {
-+ s390_topology_insert(topology_list, S390_CPU(cs));
++ s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
++ S390TopologyEntry *entry, *tmp;
++
++ QTAILQ_FOREACH(tmp, topology_list, next) {
++ if (id.id == tmp->id.id) {
++ entry = tmp;
++ break;
++ } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
++ entry = g_malloc0(sizeof(*entry));
++ entry->id.id = id.id;
++ QTAILQ_INSERT_BEFORE(tmp, entry, next);
++ break;
++ }
++ }
++ s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
+ }
++
++ QTAILQ_REMOVE(topology_list, &sentinel, next);
+}
+
+/**
+ * s390_topology_empty_list:
+ *
-+ * Clear all entries in the S390Topology list except the sentinel.
++ * Clear all entries in the S390Topology list.
+ */
+static void s390_topology_empty_list(S390TopologyList *topology_list)
+{
@@ target/s390x/kvm/stsi-topology.c (new)
+ * addr: Guest logical address of the guest SysIB
+ * ar: the access register number
+ *
-+ * Create a list head for the Topology entries and initialize it.
-+ * Insert the first entry as a sentinelle.
-+ *
+ * Emulate STSI 15.1.x, that is, perform all necessary checks and
+ * fill the SYSIB.
+ * In case the topology description is too long to fit into the SYSIB,
+ * set CC=3 and abort without writing the SYSIB.
+ */
-+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar)
++void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra)
+{
+ S390TopologyList topology_list;
-+ S390TopologyEntry *entry;
+ SysIB sysib = {0};
+ int length;
+
@@ target/s390x/kvm/stsi-topology.c (new)
+ return;
+ }
+
-+ QTAILQ_INIT(&topology_list);
-+ entry = g_malloc0(sizeof(S390TopologyEntry));
-+ entry->id.sentinel = 0xff;
-+ QTAILQ_INSERT_HEAD(&topology_list, entry, next);
-+
+ s390_topology_fill_list_sorted(&topology_list);
+
+ length = setup_stsi(&topology_list, &sysib.sysib_151x, sel2);
+
+ if (!length) {
++ s390_topology_empty_list(&topology_list);
+ setcc(cpu, 3);
+ return;
+ }
+
+ sysib.sysib_151x.length = cpu_to_be16(length);
-+ s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length);
-+ setcc(cpu, 0);
++ if (!s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length)) {
++ setcc(cpu, 0);
++ } else {
++ s390_cpu_virt_mem_handle_exc(cpu, ra);
++ }
+
+ s390_topology_empty_list(&topology_list);
+}
@@ target/s390x/kvm/meson.build
@@
s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
+ 'pv.c',
- 'kvm.c'
+ 'kvm.c',
+ 'stsi-topology.c'
4: 8864107e23 = 4: 67278aa3a0 s390x/sclp: reporting the maximum nested topology entries
5: 35aac0c335 ! 5: ce87494fd1 s390x/cpu topology: resetting the Topology-Change-Report
@@ Commit message
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
+ Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## include/hw/s390x/cpu-topology.h ##
@@ include/hw/s390x/cpu-topology.h: static inline void s390_topology_setup_cpu(MachineState *ms,
@@ include/hw/s390x/cpu-topology.h: static inline void s390_topology_setup_cpu(Mach
{
## target/s390x/cpu.h ##
-@@ target/s390x/cpu.h: typedef struct SysIBTl_cpu {
- QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+@@ target/s390x/cpu.h: typedef struct SysIBCPUListEntry {
+ QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
- void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar);
+ void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
6: b64ec041ba ! 6: c56178005a s390x/cpu topology: interception of PTF instruction
@@ Commit message
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
-
- ## qapi/machine-target.json ##
-@@
- 'TARGET_S390X',
- 'TARGET_MIPS',
- 'TARGET_LOONGARCH64' ] } }
-+
-+##
-+# @CpuS390Polarization:
-+#
-+# An enumeration of cpu polarization that can be assumed by a virtual
-+# S390 CPU
-+#
-+# Since: 8.1
-+##
-+{ 'enum': 'CpuS390Polarization',
-+ 'prefix': 'S390_CPU_POLARIZATION',
-+ 'data': [ 'horizontal', 'vertical' ],
-+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
-+}
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## include/hw/s390x/s390-virtio-ccw.h ##
@@ include/hw/s390x/s390-virtio-ccw.h: struct S390CcwMachineState {
@@ hw/s390x/cpu-topology.c: static void s390_topology_init(MachineState *ms)
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
-+ CpuS390Polarization polarization = S390_CPU_POLARIZATION_HORIZONTAL;
++ CpuS390Polarization polarization;
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ int fc = reg & S390_TOPO_FC_MASK;
@@ hw/s390x/cpu-topology.c: static void s390_topology_init(MachineState *ms)
+ return;
+ }
+
++ polarization = S390_CPU_POLARIZATION_VERTICAL;
+ switch (fc) {
-+ case S390_CPU_POLARIZATION_VERTICAL:
-+ polarization = S390_CPU_POLARIZATION_VERTICAL;
++ case 0:
++ polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+ /* fallthrough */
-+ case S390_CPU_POLARIZATION_HORIZONTAL:
-+ if (s390_topology.vertical_polarization == polarization) {
++ case 1:
++ if (s390_topology.polarization == polarization) {
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ setcc(cpu, 2);
+ } else {
-+ s390_topology.vertical_polarization = polarization;
++ s390_topology.polarization = polarization;
+ s390_cpu_topology_set_changed(true);
+ setcc(cpu, 0);
+ }
@@ hw/s390x/cpu-topology.c: static void s390_topology_init(MachineState *ms)
void s390_topology_reset(void)
{
s390_cpu_topology_set_changed(false);
-+ s390_topology.vertical_polarization = false;
++ s390_topology.polarization = S390_CPU_POLARIZATION_HORIZONTAL;
}
/**
7: bf42e59614 = 7: e56149e34f target/s390x/cpu topology: activate CPU topology
8: f584523eff ! 8: c94943deef qapi/s390x/cpu topology: set-cpu-topology qmp command
@@ Commit message
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## qapi/machine-target.json ##
@@
@@ qapi/machine-target.json
+#
+# Returns: Nothing on success, the reason on failure.
+#
-+# Since: 8.1
++# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
@@ hw/s390x/cpu-topology.c: void s390_topology_setup_cpu(MachineState *ms, S390CPU
+ }
+
+ if (!s390_topology_check(socket_id, book_id, drawer_id,
-+ entitlement, dedicated, errp))
++ entitlement, dedicated, errp)) {
+ return;
++ }
+
+ /* Check for space on new socket */
+ old_socket_entry = s390_socket_nb(cpu);
-+ new_socket_entry = __s390_socket_nb(drawer_id, book_id, socket_id);
++ new_socket_entry = s390_socket_nb_from_ids(drawer_id, book_id, socket_id);
+
+ if (new_socket_entry != old_socket_entry) {
+ if (s390_topology.cores_per_socket[new_socket_entry] >=
9: 11da1041a7 ! 9: 516a727ef5 machine: adding s390 topology to query-cpu-fast
@@ Commit message
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
- Reviewed-by: Thomas Huth <thuth@redhat.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## qapi/machine.json ##
@@
# Additional information about a virtual S390 CPU
#
# @cpu-state: the virtual CPU's state
-+# @dedicated: the virtual CPU's dedication (since 8.1)
-+# @entitlement: the virtual CPU's entitlement (since 8.1)
++# @dedicated: the virtual CPU's dedication (since 8.2)
++# @entitlement: the virtual CPU's entitlement (since 8.2)
#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
-+ 'dedicated': 'bool',
-+ 'entitlement': 'CpuS390Entitlement'
++ '*dedicated': 'bool',
++ '*entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
- ## hw/core/machine-qmp-cmds.c ##
-@@ hw/core/machine-qmp-cmds.c: static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu)
- CPUS390XState *env = &s390_cpu->env;
+ ## target/s390x/cpu.c ##
+@@
+ #ifndef CONFIG_USER_ONLY
+ #include "sysemu/reset.h"
+ #endif
++#include "hw/s390x/cpu-topology.h"
+
+ #define CR0_RESET 0xE0UL
+ #define CR14_RESET 0xC2000000UL;
+@@ target/s390x/cpu.c: static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast *value)
+ S390CPU *s390_cpu = S390_CPU(cpu);
- info->cpu_state = env->cpu_state;
+ value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
-+ info->dedicated = env->dedicated;
-+ info->entitlement = env->entitlement;
-+#endif /* !defined(CONFIG_USER_ONLY) */
- #else
- abort();
- #endif
++ if (s390_has_topology()) {
++ value->u.s390x.has_dedicated = true;
++ value->u.s390x.dedicated = s390_cpu->env.dedicated;
++ value->u.s390x.has_entitlement = true;
++ value->u.s390x.entitlement = s390_cpu->env.entitlement;
++ }
++#endif
+ }
+
+ /* S390CPUClass::reset() */
10: 46e2545116 ! 10: 4b8bfddea9 machine: adding s390 topology to info hotpluggable-cpus
@@ hw/core/machine-hmp-cmds.c: void hmp_hotpluggable_cpus(Monitor *mon, const QDict
+ monitor_printf(mon, " drawer-id: \"%" PRIu64 "\"\n", c->drawer_id);
+ }
+ if (c->has_book_id) {
-+ monitor_printf(mon, " book-id: \"%" PRIu64 "\"\n", c->book_id);
++ monitor_printf(mon, " book-id: \"%" PRIu64 "\"\n", c->book_id);
+ }
if (c->has_socket_id) {
monitor_printf(mon, " socket-id: \"%" PRIu64 "\"\n", c->socket_id);
11: f893efcf54 ! 11: 6584a655e9 qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
@@ Commit message
CPU provisioning.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
+ Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## qapi/machine-target.json ##
@@
@@ qapi/machine-target.json
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
-+# Since: 8.1
++# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
-+# "data": { "polarization": 0 },
++# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
@@ hw/s390x/cpu-topology.c
* s390_topology is used to keep the topology information.
@@ hw/s390x/cpu-topology.c: void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
- s390_topology.vertical_polarization = polarization;
+ s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
-+ qapi_event_send_cpu_polarization_change(fc);
++ qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
12: 02b4452443 ! 12: 7e18ccdbcc qapi/s390x/cpu topology: query-cpu-polarization qmp command
@@ Commit message
CPU polarization of the machine.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## qapi/machine-target.json ##
@@
@@ qapi/machine-target.json
+#
+# @polarization: the CPU polarization
+#
-+# Since: 2.8
++# Since: 8.2
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
@@ qapi/machine-target.json
+}
+
+##
-+# @query-cpu-polarization:
++# @query-s390x-cpu-polarization:
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Returns: the machine polarization
+#
-+# Since: 8.1
++# Since: 8.2
+##
-+{ 'command': 'query-cpu-polarization', 'returns': 'CpuPolarizationInfo',
++{ 'command': 'query-s390x-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
@@ hw/s390x/cpu-topology.c: void qmp_set_cpu_topology(uint16_t core,
has_dedicated, dedicated, errp);
}
+
-+CpuPolarizationInfo *qmp_query_cpu_polarization(Error **errp)
++CpuPolarizationInfo *qmp_query_s390x_cpu_polarization(Error **errp)
+{
+ CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
-+ if (s390_topology.vertical_polarization) {
-+ info->polarization = S390_CPU_POLARIZATION_VERTICAL;
-+ } else {
-+ info->polarization = S390_CPU_POLARIZATION_HORIZONTAL;
-+ }
-+
++ info->polarization = s390_topology.polarization;
+ return info;
+}
13: 743fd027f0 ! 13: 810118478c docs/s390x/cpu topology: document s390x cpu topology
@@ Commit message
in s390x.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## MAINTAINERS ##
@@ MAINTAINERS: S: Supported
@@ docs/devel/s390-cpu-topology.rst (new)
+QAPI interface for S390 CPU topology
+====================================
+
-+Let's start QEMU with the following command defining 4 CPUs,
-+CPU[0] defined by the -smp argument will have default values:
++The following sections will explain the QAPI interface for S390 CPU topology
++with the help of exemplary output.
++For this, let's assume that QEMU has been started with the following
++command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
++have default values:
+
+.. code-block:: bash
+
@@ docs/devel/s390-cpu-topology.rst (new)
+ -device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
-+and see the result when using the QAPI interface.
++Additions to query-cpus-fast
++----------------------------
+
-+Addons to query-cpus-fast
-+-------------------------
-+
-+The command query-cpus-fast allows to query the topology tree and
++The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
@@ docs/devel/s390-cpu-topology.rst (new)
+QAPI command: set-cpu-topology
+------------------------------
+
-+The command set-cpu-topology allows to modify the topology tree
++The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
@@ docs/devel/s390-cpu-topology.rst (new)
+ }
+ {"return": {}}
+
-+The core-id parameter is the only non optional parameter and every
++The core-id parameter is the only mandatory parameter and every
+unspecified parameter keeps its previous value.
+
+QAPI event CPU_POLARIZATION_CHANGE
+----------------------------------
+
-+When a guest is requests a modification of the polarization,
++When a guest requests a modification of the polarization,
+QEMU sends a CPU_POLARIZATION_CHANGE event.
+
+When requesting the change, the guest only specifies horizontal or
+vertical polarization.
-+It is the job of the upper layer to set the dedication and fine grained
-+vertical entitlement in response to this event.
++It is the job of the entity administrating QEMU to set the dedication and fine
++grained vertical entitlement in response to this event.
+
+Note that a vertical polarized dedicated vCPU can only have a high
-+entitlement, this gives 6 possibilities for vCPU entitlement:
++entitlement, giving 6 possibilities for vCPU polarization:
+
+- Horizontal
+- Horizontal dedicated
@@ docs/devel/s390-cpu-topology.rst (new)
+ }
+ }
+
-+QAPI query command: query-cpu-polarization
-+------------------------------------------
++QAPI query command: query-s390x-cpu-polarization
++------------------------------------------------
+
-+The query command query-cpu-polarization returns the current
++The query command query-s390x-cpu-polarization returns the current
+CPU polarization of the machine.
-+In the case the guest issued a PTF(1) to request a vertical polarization:
++In this case the guest previously issued a PTF(1) to request vertical polarization:
+
+.. code-block:: QMP
+
-+ { "execute": "query-cpu-polarization" }
++ { "execute": "query-s390x-cpu-polarization" }
+ {
+ "return": {
+ "polarization": "vertical"
@@ docs/system/s390x/cpu-topology.rst (new)
+CPU topology on s390x
+=====================
+
-+Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of
-+topology containers: drawers, books, sockets, defining a tree shaped
-+hierarchy.
++Since QEMU 8.2, CPU topology on s390x provides up to 3 levels of
++topology containers: drawers, books and sockets. They define a
++tree-shaped hierarchy.
+
-+The socket container contains one or more CPU entries.
++The socket container has one or more CPU entries.
+Each of these CPU entries consists of a bitmap and three CPU attributes:
+
+- CPU type
+- entitlement
+- dedication
+
-+Each bit set in the bitmap correspond to the core-id of a vCPU with
-+matching the three attribute.
++Each bit set in the bitmap correspond to a core-id of a vCPU with matching
++attributes.
+
-+This documentation provide general information on S390 CPU topology,
-+how to enable it and on the new CPU attributes.
-+For information on how to modify the S390 CPU topology and on how to
-+monitor the polarization change see ``Developer Information``.
++This documentation provides general information on S390 CPU topology,
++how to enable it and explains the new CPU attributes.
++For information on how to modify the S390 CPU topology and how to
++monitor polarization changes, see ``docs/devel/s390-cpu-topology.rst``.
+
+Prerequisites
+-------------
@@ docs/system/s390x/cpu-topology.rst (new)
+Currently, CPU topology is only enabled in the host model by default.
+
+Enabling CPU topology in a CPU model is done by setting the CPU flag
-+``ctop`` to ``on`` like in:
++``ctop`` to ``on`` as in:
+
+.. code-block:: bash
+
@@ docs/system/s390x/cpu-topology.rst (new)
+
+If none of the containers attributes (drawers, books, sockets) are
+specified for the ``-smp`` flag, the number of these containers
-+is ``1`` .
++is 1.
++
++Thus the following two options will result in the same topology:
+
+.. code-block:: bash
+
+ -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32
+
-+or
++and
+
+.. code-block:: bash
+
@@ docs/system/s390x/cpu-topology.rst (new)
+When a CPU is defined by the ``-smp`` command argument, its position
+inside the topology is calculated by adding the CPUs to the topology
+based on the core-id starting with core-0 at position 0 of socket-0,
-+book-0, drawer-0 and filling all CPUs of socket-0 before to fill socket-1
++book-0, drawer-0 and filling all CPUs of socket-0 before filling socket-1
+of book-0 and so on up to the last socket of the last book of the last
+drawer.
+
+When a CPU is defined by the ``-device`` command argument, the
-+tree topology attributes must be all defined or all not defined.
++tree topology attributes must all be defined or all not defined.
+
+.. code-block:: bash
+
@@ docs/system/s390x/cpu-topology.rst (new)
+ -device gen16b-s390x-cpu,core-id=1,dedicated=true
+
+If none of the tree attributes (drawer, book, sockets), are specified
-+for the ``-device`` argument, as for all CPUs defined with the ``-smp``
++for the ``-device`` argument, like for all CPUs defined with the ``-smp``
+command argument the topology tree attributes will be set by simply
-+adding the CPUs to the topology based on the core-id starting with
-+core-0 at position 0 of socket-0, book-0, drawer-0.
++adding the CPUs to the topology based on the core-id.
+
-+QEMU will not try to solve collisions and will report an error if the
-+CPU topology, explicitly or implicitly defined on a ``-device``
-+argument collides with the definition of a CPU implicitely defined
++QEMU will not try to resolve collisions and will report an error if the
++CPU topology defined explicitly or implicitly on a ``-device``
++argument collides with the definition of a CPU implicitly defined
+on the ``-smp`` argument.
+
+When the topology modifier attributes are not defined for the
@@ docs/system/s390x/cpu-topology.rst (new)
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
-+The same placement of the CPU is derived from the core-id as described above.
++The placement of the CPU is derived from the core-id as described above.
+
-+The topology can of course be fully defined:
++The topology can of course also be fully defined:
+
+.. code-block:: bash
+
@@ docs/system/s390x/cpu-topology.rst (new)
+Polarization
+++++++++++++
+
-+The polarization is an indication given by the ``guest`` to the host
-+that it is able to make use of CPU provisioning information.
-+The guest indicates the polarization by using the PTF instruction.
++The polarization affects how the CPUs of a shared host are utilized/distributed
++among guests.
++The guest determines the polarization by using the PTF instruction.
+
-+Polarization is define two models of CPU provisioning: horizontal
++Polarization defines two models of CPU provisioning: horizontal
+and vertical.
+
+The horizontal polarization is the default model on boot and after
-+subsystem reset in which the guest considers all vCPUs being having
-+an equal provisioning of CPUs by the host.
++subsystem reset. When horizontal polarization is in effect all vCPUs should
++have about equal resource provisioning.
+
-+In the vertical polarization model the guest can make use of the
-+vCPU entitlement information provided by the host to optimize
-+kernel thread scheduling.
++In the vertical polarization model vCPUs are unequal, but overall more resources
++might be available.
++The guest can make use of the vCPU entitlement information provided by the host
++to optimize kernel thread scheduling.
+
+A subsystem reset puts all vCPU of the configuration into the
+horizontal polarization.
@@ docs/system/s390x/cpu-topology.rst (new)
++++++++++++
+
+The vertical polarization specifies that the guest's vCPU can get
-+different real CPU provisions:
++different real CPU provisioning:
+
+- a vCPU with vertical high entitlement specifies that this
+ vCPU gets 100% of the real CPU provisioning.
@@ docs/system/s390x/cpu-topology.rst (new)
+the real CPU, the unused "slack" can be dispatched to other vCPU
+with medium or low entitlement.
+
-+The upper level specifies a vCPU as ``dedicated`` when the vCPU is
-+fully dedicated to a single real CPU.
++A vCPU can be "dedicated" in which case the vCPU is fully dedicated to a single
++real CPU.
+
+The dedicated bit is an indication of affinity of a vCPU for a real CPU
+while the entitlement indicates the sharing or exclusivity of use.
+
-+Defining the topology on command line
-+-------------------------------------
++Defining the topology on the command line
++-----------------------------------------
+
+The topology can entirely be defined using -device cpu statements,
+with the exception of CPU 0 which must be defined with the -smp
@@ docs/system/s390x/cpu-topology.rst (new)
+For example, here we set the position of the cores 1,2,3 to
+drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0,
+book 0, socket 0 without defining entitlement or dedication.
-+The core 4, will be set on its default position on socket 1
++Core 4 will be set on its default position on socket 1
+(since we have 4 core per socket) and we define it as dedicated and
+with vertical high entitlement.
+
@@ docs/system/s390x/cpu-topology.rst (new)
+ \
+ -device gen16b-s390x-cpu,core-id=4,dedicated=on,entitlement=high
+
-+The entitlement defined for the CPU 4, will only be used after the guest
++The entitlement defined for the CPU 4 will only be used after the guest
+successfully enables vertical polarization by using the PTF instruction.
## docs/system/target-s390x.rst ##
14: d24ddf00ce ! 14: 717f901bb0 tests/avocado: s390x cpu topology core
@@ Commit message
Introduction of the s390x cpu topology core functions and
basic tests.
- We test the corelation between the command line and
+ We test the correlation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
@@ tests/avocado/s390_topology.py (new)
+
+class S390CPUTopology(QemuSystemTest):
+ """
-+ S390x CPU topology consist of 4 topology layers, from bottom to top,
++ S390x CPU topology consists of 4 topology layers, from bottom to top,
+ the cores, sockets, books and drawers and 2 modifiers attributes,
+ the entitlement and the dedication.
+ See: docs/system/s390x/cpu-topology.rst.
+
+ S390x CPU topology is setup in different ways:
-+ - implicitely from the '-smp' argument by completing each topology
-+ level one after the other begining with drawer 0, book 0 and socket 0.
-+ - explicitely from the '-device' argument on the QEMU command line
-+ - explicitely by hotplug of a new CPU using QMP or HMP
++ - implicitly from the '-smp' argument by completing each topology
++ level one after the other beginning with drawer 0, book 0 and
++ socket 0.
++ - explicitly from the '-device' argument on the QEMU command line
++ - explicitly by hotplug of a new CPU using QMP or HMP
+ - it is modified by using QMP 'set-cpu-topology'
+
+ The S390x modifier attribute entitlement depends on the machine
@@ tests/avocado/s390_topology.py (new)
+ socket = cpu['props']['socket-id']
+ book = cpu['props']['book-id']
+ drawer = cpu['props']['drawer-id']
-+ entitlement = cpu['entitlement']
-+ dedicated = cpu['dedicated']
++ entitlement = cpu.get('entitlement')
++ dedicated = cpu.get('dedicated')
+ if core == c:
+ self.assertEqual(drawer, d)
+ self.assertEqual(book, b)
@@ tests/avocado/s390_topology.py (new)
+
+ def kernel_init(self):
+ """
++ We need a VM that supports CPU topology,
++ currently this only the case when using KVM, not TCG.
+ We need a kernel supporting the CPU topology.
+ We need a minimal root filesystem with a shell.
+ """
++ self.require_accelerator("kvm")
+ kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
15: 4bee9fa1af ! 15: 7c5793cf9b tests/avocado: s390x cpu topology polarisation
@@ Metadata
Author: Pierre Morel <pmorel@linux.ibm.com>
## Commit message ##
- tests/avocado: s390x cpu topology polarisation
+ tests/avocado: s390x cpu topology polarization
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## tests/avocado/s390_topology.py ##
@@ tests/avocado/s390_topology.py: class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
-+ event_timeout = 1
++ event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ tests/avocado/s390_topology.py: def kernel_init(self):
+ def system_init(self):
+ self.log.info("System init")
-+ exec_command(self, 'mount proc -t proc /proc')
-+ time.sleep(0.2)
-+ exec_command(self, 'mount sys -t sysfs /sys')
-+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self,
-+ '/bin/cat /sys/devices/system/cpu/dispatching', '0')
++ """ mount proc -t proc /proc;
++ mount sys -t sysfs /sys;
++ /bin/cat /sys/devices/system/cpu/dispatching """,
++ '0')
+
def test_single(self):
"""
@@ tests/avocado/s390_topology.py: def test_hotplug_full(self):
+ '/bin/cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
-+ def test_polarisation(self):
++ def test_polarization(self):
+ """
+ This test verifies that QEMU modifies the entitlement change after
+ several guest polarization change requests.
@@ tests/avocado/s390_topology.py: def test_hotplug_full(self):
+ self.wait_until_booted()
+
+ self.system_init()
-+ res = self.vm.qmp('query-cpu-polarization')
++ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ self.guest_set_dispatching('1');
-+ res = self.vm.qmp('query-cpu-polarization')
++ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'vertical')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ self.guest_set_dispatching('0');
-+ res = self.vm.qmp('query-cpu-polarization')
++ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
16: dfadecd4eb ! 16: f1aa5e3d91 tests/avocado: s390x cpu topology entitlement tests
@@ Metadata
## Commit message ##
tests/avocado: s390x cpu topology entitlement tests
- This test takes care to check the changes on different entitlements
- when the guest requests a polarization change.
+ Test changes in the entitlement from both a guest and a host point of
+ view, depending on the polarization.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## tests/avocado/s390_topology.py ##
-@@ tests/avocado/s390_topology.py: def test_polarisation(self):
- res = self.vm.qmp('query-cpu-polarization')
+@@ tests/avocado/s390_topology.py: def test_polarization(self):
+ res = self.vm.qmp('query-s390x-cpu-polarization')
self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
++
++ def check_polarization(self, polarization):
++ """
++ We need the polarization change to get stabilized in kernel
++ before we check it through the sysfs.
++ """
++ exec_command_and_wait_for_pattern(self,
++ """ sleep 1 ;
++ /bin/cat /sys/devices/system/cpu/cpu0/polarization """,
++ polarization)
++
++
+ def test_entitlement(self):
+ """
-+ This test verifies that QEMU modifies the polarization
-+ after a guest request.
++ This test verifies that QEMU modifies the entitlement
++ after a guest request and that the guest sees the change.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
-+ self.vm.add_args('-smp',
-+ '4,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
++ self.check_polarization('horizontal')
++ self.check_topology(0, 0, 0, 0, 'medium', False)
++
++ self.guest_set_dispatching('1')
++ self.check_polarization('vertical:medium')
++ self.check_topology(0, 0, 0, 0, 'medium', False)
++
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+ self.assertEqual(res['return'], {})
++ self.check_polarization('vertical:low')
++ self.check_topology(0, 0, 0, 0, 'low', False)
++
+ res = self.vm.qmp('set-cpu-topology',
-+ {'core-id': 1, 'entitlement': 'medium'})
-+ self.assertEqual(res['return'], {})
-+ res = self.vm.qmp('set-cpu-topology',
-+ {'core-id': 2, 'entitlement': 'high'})
++ {'core-id': 0, 'entitlement': 'medium'})
+ self.assertEqual(res['return'], {})
++ self.check_polarization('vertical:medium')
++ self.check_topology(0, 0, 0, 0, 'medium', False)
++
+ res = self.vm.qmp('set-cpu-topology',
-+ {'core-id': 3, 'entitlement': 'high'})
++ {'core-id': 0, 'entitlement': 'high'})
+ self.assertEqual(res['return'], {})
-+ self.check_topology(0, 0, 0, 0, 'low', False)
-+ self.check_topology(1, 0, 0, 0, 'medium', False)
-+ self.check_topology(2, 1, 0, 0, 'high', False)
-+ self.check_topology(3, 1, 0, 0, 'high', False)
-+
-+ self.guest_set_dispatching('1');
-+
-+ self.check_topology(0, 0, 0, 0, 'low', False)
-+ self.check_topology(1, 0, 0, 0, 'medium', False)
-+ self.check_topology(2, 1, 0, 0, 'high', False)
-+ self.check_topology(3, 1, 0, 0, 'high', False)
++ self.check_polarization('vertical:high')
++ self.check_topology(0, 0, 0, 0, 'high', False)
+
+ self.guest_set_dispatching('0');
-+
-+ self.check_topology(0, 0, 0, 0, 'low', False)
-+ self.check_topology(1, 0, 0, 0, 'medium', False)
-+ self.check_topology(2, 1, 0, 0, 'high', False)
-+ self.check_topology(3, 1, 0, 0, 'high', False)
++ self.check_polarization("horizontal")
++ self.check_topology(0, 0, 0, 0, 'high', False)
17: 59457e4e40 ! 17: 820481f18c tests/avocado: s390x cpu topology test dedicated CPU
@@ Commit message
A dedicated CPU in vertical polarization can only have
a high entitlement.
- Let's check this.
+ Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+ Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
## tests/avocado/s390_topology.py ##
@@ tests/avocado/s390_topology.py: def test_entitlement(self):
- self.check_topology(1, 0, 0, 0, 'medium', False)
- self.check_topology(2, 1, 0, 0, 'high', False)
- self.check_topology(3, 1, 0, 0, 'high', False)
+ self.guest_set_dispatching('0');
+ self.check_polarization("horizontal")
+ self.check_topology(0, 0, 0, 0, 'high', False)
++
+
+ def test_dedicated(self):
+ """
-+ This test verifies that QEMU modifies the entitlement change correctly
-+ for a dedicated CPU after several guest polarization change requests.
++ This test verifies that QEMU adjusts the entitlement correctly when a
++ CPU is made dedicated.
++ QEMU retains the entitlement value when horizontal polarization is in effect.
++ For the guest, the field shows the effective value of the entitlement.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
@@ tests/avocado/s390_topology.py: def test_entitlement(self):
+
+ self.system_init()
+
++ self.check_polarization("horizontal")
++
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+ self.assertEqual(res['return'], {})
-+
+ self.check_topology(0, 0, 0, 0, 'high', True)
++ self.check_polarization("horizontal")
+
+ self.guest_set_dispatching('1');
-+
+ self.check_topology(0, 0, 0, 0, 'high', True)
++ self.check_polarization("vertical:high")
+
+ self.guest_set_dispatching('0');
-+
+ self.check_topology(0, 0, 0, 0, 'high', True)
++ self.check_polarization("horizontal")
18: a2ebb1be4c ! 18: d1efe180f3 tests/avocado: s390x cpu topology test socket full
@@ Commit message
to overload a socket.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
## tests/avocado/s390_topology.py ##
@@ tests/avocado/s390_topology.py: def test_dedicated(self):
self.guest_set_dispatching('0');
-
self.check_topology(0, 0, 0, 0, 'high', True)
+ self.check_polarization("horizontal")
++
+
+ def test_socket_full(self):
+ """
19: aa692304a4 ! 19: 568de9fef9 tests/avocado: s390x cpu topology dedicated errors
@@ Commit message
low or medium entitlement.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
## tests/avocado/s390_topology.py ##
@@ tests/avocado/s390_topology.py: def test_socket_full(self):
20: 59580e0a42 ! 20: 9f9672adc2 tests/avocado: s390x cpu topology bad move
@@ Commit message
tests/avocado: s390x cpu topology bad move
This test verifies that QEMU refuses to move a CPU to an
- unexistant location.
+ nonexistent location.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
+ Reviewed-by: Thomas Huth <thuth@redhat.com>
## tests/avocado/s390_topology.py ##
@@ tests/avocado/s390_topology.py: def test_dedicated_error(self):
@@ tests/avocado/s390_topology.py: def test_dedicated_error(self):
+ def test_move_error(self):
+ """
+ This test verifies that QEMU refuses to move a CPU to an
-+ unexistant location
++ nonexistent location
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
base-commit: 17780edd81d27fcfdb7a802efc870a99788bd2fc
--
2.39.2
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v22 01/20] CPU topology: extend with s390 specifics
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-05 9:01 ` Thomas Huth
2023-09-01 15:57 ` [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug Nina Schoetterl-Glausch
` (18 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
S390 adds two new SMP levels, drawers and books to the CPU
topology.
S390 CPUs have specific topology features like dedication and
entitlement. These indicate to the guest information on host
vCPU scheduling and help the guest make better scheduling decisions.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
qapi/machine-common.json | 21 +++++++++++++
qapi/machine.json | 21 ++++++++++---
include/hw/boards.h | 10 +++++-
include/hw/qdev-properties-system.h | 4 +++
target/s390x/cpu.h | 6 ++++
hw/core/machine-smp.c | 48 ++++++++++++++++++++++++-----
hw/core/machine.c | 4 +++
hw/core/qdev-properties-system.c | 13 ++++++++
hw/s390x/s390-virtio-ccw.c | 2 ++
softmmu/vl.c | 6 ++++
target/s390x/cpu.c | 7 +++++
qapi/meson.build | 1 +
qemu-options.hx | 7 +++--
13 files changed, 136 insertions(+), 14 deletions(-)
create mode 100644 qapi/machine-common.json
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 0000000000..e40421bb37
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,21 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of cpu entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..54f99f4ac1 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -71,8 +72,8 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
-# virtual CPU belongs to, provided if supported by board
+# @props: properties describing to which node/drawer/book/socket/core/thread
+# virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
# additional fields will be listed (since 3.0)
@@ -901,7 +902,11 @@
#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to (since 8.2)
+#
+# @book-id: book number within drawer/node/board the CPU belongs to (since 8.2)
+#
+# @socket-id: socket number within book/node/board the CPU belongs to
#
# @die-id: die number within socket the CPU belongs to (since 4.1)
#
@@ -912,7 +917,7 @@
#
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present but
+# Note: currently there are 8 properties that could be present but
# management should be prepared to pass through other properties
# with device_add command to allow for future interface extension.
# This also requires the filed names to be kept in sync with the
@@ -922,6 +927,8 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*drawer-id': 'int',
+ '*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1480,6 +1487,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.2)
+#
+# @books: number of books in the CPU topology (since 8.2)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1498,6 +1509,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 3b541ffd24..5d2cef73ee 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -134,12 +134,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool has_clusters;
+ bool books_supported;
+ bool drawers_supported;
} SMPCompatProps;
/**
@@ -310,7 +314,9 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
- * @sockets: the number of sockets on the machine
+ * @drawers: the number of drawers on the machine
+ * @books: the number of books in one drawer
+ * @sockets: the number of sockets in one book
* @dies: the number of dies in one socket
* @clusters: the number of clusters in one die
* @cores: the number of cores in one cluster
@@ -319,6 +325,8 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int drawers;
+ unsigned int books;
unsigned int sockets;
unsigned int dies;
unsigned int clusters;
diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properties-system.h
index 0ac327ae60..e4f8a13afc 100644
--- a/include/hw/qdev-properties-system.h
+++ b/include/hw/qdev-properties-system.h
@@ -22,6 +22,7 @@ extern const PropertyInfo qdev_prop_audiodev;
extern const PropertyInfo qdev_prop_off_auto_pcibar;
extern const PropertyInfo qdev_prop_pcie_link_speed;
extern const PropertyInfo qdev_prop_pcie_link_width;
+extern const PropertyInfo qdev_prop_cpus390entitlement;
#define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
@@ -73,5 +74,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
#define DEFINE_PROP_UUID_NODEFAULT(_name, _state, _field) \
DEFINE_PROP(_name, _state, _field, qdev_prop_uuid, QemuUUID)
+#define DEFINE_PROP_CPUS390ENTITLEMENT(_n, _s, _f, _d) \
+ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_cpus390entitlement, \
+ CpuS390Entitlement)
#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index eb5b65b7d3..7ebd5e05b6 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -30,6 +30,7 @@
#include "exec/cpu-defs.h"
#include "qemu/cpu-float.h"
#include "tcg/tcg_s390x.h"
+#include "qapi/qapi-types-machine-common.h"
#define ELF_MACHINE_UNAME "S390X"
@@ -130,6 +131,11 @@ struct CPUArchState {
#if !defined(CONFIG_USER_ONLY)
uint32_t core_id; /* PoP "CPU address", same as cpu_index */
+ int32_t socket_id;
+ int32_t book_id;
+ int32_t drawer_id;
+ bool dedicated;
+ CpuS390Entitlement entitlement; /* Used only for vertical polarization */
uint64_t cpuid;
#endif
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index 0f4d9b6f7a..25019c91ee 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -33,6 +33,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.drawers_supported) {
+ g_string_append_printf(s, "drawers (%u) * ", ms->smp.drawers);
+ }
+
+ if (mc->smp_props.books_supported) {
+ g_string_append_printf(s, "books (%u) * ", ms->smp.books);
+ }
+
g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
if (mc->smp_props.dies_supported) {
@@ -75,6 +83,8 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned drawers = config->has_drawers ? config->drawers : 0;
+ unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
unsigned clusters = config->has_clusters ? config->clusters : 0;
@@ -87,6 +97,8 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_drawers && config->drawers == 0) ||
+ (config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
(config->has_clusters && config->clusters == 0) ||
@@ -113,6 +125,19 @@ void machine_parse_smp_config(MachineState *ms,
dies = dies > 0 ? dies : 1;
clusters = clusters > 0 ? clusters : 1;
+ if (!mc->smp_props.books_supported && books > 1) {
+ error_setg(errp, "books not supported by this machine's CPU topology");
+ return;
+ }
+ books = books > 0 ? books : 1;
+
+ if (!mc->smp_props.drawers_supported && drawers > 1) {
+ error_setg(errp,
+ "drawers not supported by this machine's CPU topology");
+ return;
+ }
+ drawers = drawers > 0 ? drawers : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -126,33 +151,41 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (sockets * dies * clusters * cores);
+ threads = maxcpus /
+ (drawers * books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
+ maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
+ clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.drawers = drawers;
+ ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
ms->smp.clusters = clusters;
@@ -163,7 +196,8 @@ void machine_parse_smp_config(MachineState *ms,
mc->smp_props.has_clusters = config->has_clusters;
/* sanity-check of the computed topology */
- if (sockets * dies * clusters * cores * threads != maxcpus) {
+ if (drawers * books * sockets * dies * clusters * cores * threads !=
+ maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index da699cf4e1..363ddafc89 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -843,6 +843,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_drawers = true, .drawers = ms->smp.drawers,
+ .has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
.has_clusters = true, .clusters = ms->smp.clusters,
@@ -1108,6 +1110,8 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.drawers = 1;
+ ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
ms->smp.clusters = 1;
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index 6d5d43eda2..7f50ce9746 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -1147,3 +1147,16 @@ const PropertyInfo qdev_prop_uuid = {
.set = set_uuid,
.set_default_value = set_default_uuid_auto,
};
+
+/* --- s390 cpu entitlement policy --- */
+
+QEMU_BUILD_BUG_ON(sizeof(CpuS390Entitlement) != sizeof(int));
+
+const PropertyInfo qdev_prop_cpus390entitlement = {
+ .name = "CpuS390Entitlement",
+ .description = "low/medium (default)/high",
+ .enum_table = &CpuS390Entitlement_lookup,
+ .get = qdev_propinfo_get_enum,
+ .set = qdev_propinfo_set_enum,
+ .set_default_value = qdev_propinfo_set_default_value_enum,
+};
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index bfcf64d007..3b34aa0e35 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -733,6 +733,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
mc->no_sdcard = 1;
mc->max_cpus = S390_MAX_CPUS;
mc->has_hotpluggable_cpus = true;
+ mc->smp_props.books_supported = true;
+ mc->smp_props.drawers_supported = true;
assert(!mc->get_hotplug_handler);
mc->get_hotplug_handler = s390_get_hotplug_handler;
mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index b0b96f67fa..802c3c0c22 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -725,6 +725,12 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "drawers",
+ .type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "books",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "sockets",
.type = QEMU_OPT_NUMBER,
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index df167493c3..74405beb51 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -31,6 +31,7 @@
#include "qapi/qapi-types-machine.h"
#include "sysemu/hw_accel.h"
#include "hw/qdev-properties.h"
+#include "hw/qdev-properties-system.h"
#include "fpu/softfloat-helpers.h"
#include "disas/capstone.h"
#include "sysemu/tcg.h"
@@ -292,6 +293,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
static Property s390x_cpu_properties[] = {
#if !defined(CONFIG_USER_ONLY)
DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
+ DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
+ DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
+ DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
+ DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
+ DEFINE_PROP_CPUS390ENTITLEMENT("entitlement", S390CPU, env.entitlement,
+ S390_CPU_ENTITLEMENT_AUTO),
#endif
DEFINE_PROP_END_OF_LIST()
};
diff --git a/qapi/meson.build b/qapi/meson.build
index 60a668b343..f81a37565c 100644
--- a/qapi/meson.build
+++ b/qapi/meson.build
@@ -36,6 +36,7 @@ qapi_all_modules = [
'error',
'introspect',
'job',
+ 'machine-common',
'machine',
'machine-target',
'migration',
diff --git a/qemu-options.hx b/qemu-options.hx
index 29b98c3d4c..2fb0cc9e29 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -257,11 +257,14 @@ SRST
ERST
DEF("smp", HAS_ARG, QEMU_OPTION_smp,
- "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
+ "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"
+ " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
" set the number of initial CPUs to 'n' [default=1]\n"
" maxcpus= maximum number of total CPUs, including\n"
" offline CPUs for hotplug, etc\n"
- " sockets= number of sockets on the machine board\n"
+ " drawers= number of drawers on the machine board\n"
+ " books= number of books in one drawer\n"
+ " sockets= number of sockets in one book\n"
" dies= number of dies in one socket\n"
" clusters= number of clusters in one die\n"
" cores= number of cores in one cluster\n"
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 01/20] CPU topology: extend with s390 specifics Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-05 12:28 ` Thomas Huth
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
` (17 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
MAINTAINERS | 6 +
include/hw/s390x/cpu-topology.h | 54 +++++++
hw/s390x/cpu-topology.c | 262 ++++++++++++++++++++++++++++++++
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build | 1 +
5 files changed, 343 insertions(+), 2 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 6111b6b4d9..b10b83583f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1695,6 +1695,12 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s390x@nongnu.org
+S390 CPU topology
+M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
+
X86 Machines
------------
PC
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 0000000000..97b0af2795
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
+typedef struct S390Topology {
+ uint8_t *cores_per_socket;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+ return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+ return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+ return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
+#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 0000000000..06b60ebce4
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,262 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * S390 topology handling can be divided in two parts:
+ *
+ * - The first part in this file is taking care of all common functions
+ * used by KVM and TCG to create and modify the topology.
+ *
+ * - The second part, building the topology information data for the
+ * guest with CPU and KVM specificity will be implemented inside
+ * the target/s390/kvm sub tree.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ * per socket.
+ */
+S390Topology s390_topology = {
+ /* will be initialized after the CPU model is realized */
+ .cores_per_socket = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
+static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+ return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
+}
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a cpu.
+ */
+static int s390_socket_nb(S390CPU *cpu)
+{
+ return s390_socket_nb_from_ids(cpu->env.drawer_id, cpu->env.book_id,
+ cpu->env.socket_id);
+}
+
+/**
+ * s390_has_topology:
+ *
+ * Return: true if the topology is supported by the machine.
+ */
+bool s390_has_topology(void)
+{
+ return false;
+}
+
+/**
+ * s390_topology_init:
+ * @ms: the machine state where the machine topology is defined
+ *
+ * Keep track of the machine topology.
+ *
+ * Allocate an array to keep the count of cores per socket.
+ * The index of the array starts at socket 0 from book 0 and
+ * drawer 0 up to the maximum allowed by the machine topology.
+ */
+static void s390_topology_init(MachineState *ms)
+{
+ CpuTopology *smp = &ms->smp;
+
+ s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
+ smp->books * smp->drawers);
+}
+
+/**
+ * s390_topology_cpu_default:
+ * @cpu: pointer to a S390CPU
+ * @errp: Error pointer
+ *
+ * Setup the default topology if no attributes are already set.
+ * Passing a CPU with some, but not all, attributes set is considered
+ * an error.
+ *
+ * The function calculates the (drawer_id, book_id, socket_id)
+ * topology by filling the cores starting from the first socket
+ * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
+ *
+ * CPU type and dedication have defaults values set in the
+ * s390x_cpu_properties, entitlement must be adjust depending on the
+ * dedication.
+ *
+ * Returns false if it is impossible to setup a default topology
+ * true otherwise.
+ */
+static bool s390_topology_cpu_default(S390CPU *cpu, Error **errp)
+{
+ CpuTopology *smp = ¤t_machine->smp;
+ CPUS390XState *env = &cpu->env;
+
+ /* All geometry topology attributes must be set or all unset */
+ if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
+ (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
+ error_setg(errp,
+ "Please define all or none of the topology geometry attributes");
+ return false;
+ }
+
+ /* Check if one of the geometry topology is unset */
+ if (env->socket_id < 0) {
+ /* Calculate default geometry topology attributes */
+ env->socket_id = s390_std_socket(env->core_id, smp);
+ env->book_id = s390_std_book(env->core_id, smp);
+ env->drawer_id = s390_std_drawer(env->core_id, smp);
+ }
+
+ /*
+ * When the user specifies the entitlement as 'auto' on the command line,
+ * QEMU will set the entitlement as:
+ * Medium when the CPU is not dedicated.
+ * High when dedicated is true.
+ */
+ if (env->entitlement == S390_CPU_ENTITLEMENT_AUTO) {
+ if (env->dedicated) {
+ env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
+ }
+ }
+ return true;
+}
+
+/**
+ * s390_topology_check:
+ * @socket_id: socket to check
+ * @book_id: book to check
+ * @drawer_id: drawer to check
+ * @entitlement: entitlement to check
+ * @dedicated: dedication to check
+ * @errp: Error pointer
+ *
+ * The function checks if the topology
+ * attributes fits inside the system topology.
+ *
+ * Returns false if the specified topology does not match with
+ * the machine topology.
+ */
+static bool s390_topology_check(uint16_t socket_id, uint16_t book_id,
+ uint16_t drawer_id, uint16_t entitlement,
+ bool dedicated, Error **errp)
+{
+ CpuTopology *smp = ¤t_machine->smp;
+ ERRP_GUARD();
+
+ if (socket_id >= smp->sockets) {
+ error_setg(errp, "Unavailable socket: %d", socket_id);
+ return false;
+ }
+ if (book_id >= smp->books) {
+ error_setg(errp, "Unavailable book: %d", book_id);
+ return false;
+ }
+ if (drawer_id >= smp->drawers) {
+ error_setg(errp, "Unavailable drawer: %d", drawer_id);
+ return false;
+ }
+ if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
+ error_setg(errp, "Unknown entitlement: %d", entitlement);
+ return false;
+ }
+ if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
+ entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
+ error_setg(errp, "A dedicated CPU implies high entitlement");
+ return false;
+ }
+ return true;
+}
+
+/**
+ * s390_update_cpu_props:
+ * @ms: the machine state
+ * @cpu: the CPU for which to update the properties from the environment.
+ *
+ */
+static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
+{
+ CpuInstanceProperties *props;
+
+ props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
+
+ props->socket_id = cpu->env.socket_id;
+ props->book_id = cpu->env.book_id;
+ props->drawer_id = cpu->env.drawer_id;
+}
+
+/**
+ * s390_topology_setup_cpu:
+ * @ms: MachineState used to initialize the topology structure on
+ * first call.
+ * @cpu: the new S390CPU to insert in the topology structure
+ * @errp: the error pointer
+ *
+ * Called from CPU hotplug to check and setup the CPU attributes
+ * before the CPU is inserted in the topology.
+ * There is no need to update the MTCR explicitly here because it
+ * will be updated by KVM on creation of the new CPU.
+ */
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
+{
+ ERRP_GUARD();
+ int entry;
+
+ /*
+ * We do not want to initialize the topology if the CPU model
+ * does not support topology, consequently, we have to wait for
+ * the first CPU to be realized, which realizes the CPU model
+ * to initialize the topology structures.
+ *
+ * s390_topology_setup_cpu() is called from the CPU hotplug.
+ */
+ if (!s390_topology.cores_per_socket) {
+ s390_topology_init(ms);
+ }
+
+ if (!s390_topology_cpu_default(cpu, errp)) {
+ return;
+ }
+
+ if (!s390_topology_check(cpu->env.socket_id, cpu->env.book_id,
+ cpu->env.drawer_id, cpu->env.entitlement,
+ cpu->env.dedicated, errp)) {
+ return;
+ }
+
+ /* Do we still have space in the socket */
+ entry = s390_socket_nb(cpu);
+ if (s390_topology.cores_per_socket[entry] >= current_machine->smp.cores) {
+ error_setg(errp, "No more space on this socket");
+ return;
+ }
+
+ /* Update the count of cores in sockets */
+ s390_topology.cores_per_socket[entry] += 1;
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+}
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 3b34aa0e35..4f9ec18723 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -45,6 +45,7 @@
#include "target/s390x/kvm/pv.h"
#include "migration/blocker.h"
#include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
static Error *pv_mig_blocker;
@@ -308,10 +309,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
{
MachineState *ms = MACHINE(hotplug_dev);
S390CPU *cpu = S390_CPU(dev);
+ ERRP_GUARD();
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ if (s390_has_topology()) {
+ s390_topology_setup_cpu(ms, cpu, errp);
+ if (*errp) {
+ return;
+ }
+ }
+
if (dev->hotplugged) {
raise_irq_cpu_hotplug();
}
@@ -551,11 +560,20 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
+ CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
+
ms->possible_cpus->cpus[i].type = ms->cpu_type;
ms->possible_cpus->cpus[i].vcpus_count = 1;
ms->possible_cpus->cpus[i].arch_id = i;
- ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = i;
+
+ props->has_core_id = true;
+ props->core_id = i;
+ props->has_socket_id = true;
+ props->socket_id = s390_std_socket(i, &ms->smp);
+ props->has_book_id = true;
+ props->book_id = s390_std_book(i, &ms->smp);
+ props->has_drawer_id = true;
+ props->drawer_id = s390_std_drawer(i, &ms->smp);
}
return ms->possible_cpus;
diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
index 6fd096813a..482fd13420 100644
--- a/hw/s390x/meson.build
+++ b/hw/s390x/meson.build
@@ -23,6 +23,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
's390-skeys-kvm.c',
's390-stattrib-kvm.c',
's390-pci-kvm.c',
+ 'cpu-topology.c',
))
s390x_ss.add(when: 'CONFIG_TCG', if_true: files(
'tod-tcg.c',
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 01/20] CPU topology: extend with s390 specifics Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-04 18:23 ` Nina Schoetterl-Glausch
` (2 more replies)
2023-09-01 15:57 ` [PATCH v22 04/20] s390x/sclp: reporting the maximum nested topology entries Nina Schoetterl-Glausch
` (16 subsequent siblings)
19 siblings, 3 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
MAINTAINERS | 1 +
qapi/machine-target.json | 14 ++
include/hw/s390x/cpu-topology.h | 25 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 76 ++++++++
hw/s390x/cpu-topology.c | 2 +
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++++++++++++++++
target/s390x/kvm/meson.build | 3 +-
9 files changed, 421 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/stsi-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index b10b83583f..692ce9f121 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1700,6 +1700,7 @@ M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
+F: target/s390x/kvm/stsi-topology.c
X86 Machines
------------
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index f0a6b72414..275234a20f 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -361,3 +361,17 @@
'TARGET_MIPS',
'TARGET_LOONGARCH64',
'TARGET_RISCV' ] } }
+
+##
+# @CpuS390Polarization:
+#
+# An enumeration of cpu polarization that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Polarization',
+ 'prefix': 'S390_CPU_POLARIZATION',
+ 'data': [ 'horizontal', 'vertical' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 97b0af2795..fc15acf297 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -15,10 +15,35 @@
#include "hw/boards.h"
#include "qapi/qapi-types-machine-target.h"
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+typedef union s390_topology_id {
+ uint64_t id;
+ struct {
+ uint8_t _reserved0;
+ uint8_t drawer;
+ uint8_t book;
+ uint8_t socket;
+ uint8_t type;
+ uint8_t inv_polarization;
+ uint8_t not_dedicated;
+ uint8_t origin;
+ };
+} s390_topology_id;
+
+typedef struct S390TopologyEntry {
+ QTAILQ_ENTRY(S390TopologyEntry) next;
+ s390_topology_id id;
+ uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+ CpuS390Polarization polarization;
} S390Topology;
+typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
+
#ifdef CONFIG_KVM
bool s390_has_topology(void);
void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index cf1f2efae2..c49051e17e 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST 2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 7ebd5e05b6..b8a0c02714 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -569,6 +569,29 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+ uint8_t reserved0[2];
+ uint16_t length;
+ uint8_t mag[S390_TOPOLOGY_MAG];
+ uint8_t reserved1;
+ uint8_t mnest;
+ uint32_t reserved2;
+ char tle[];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -576,9 +599,62 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+ SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SYSIBContainerListEntry {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
+} SYSIBContainerListEntry;
+QEMU_BUILD_BUG_ON(sizeof(SYSIBContainerListEntry) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBCPUListEntry {
+ uint8_t nl;
+ uint8_t reserved0[3];
+#define SYSIB_TLE_POLARITY_MASK 0x03
+#define SYSIB_TLE_DEDICATED 0x04
+ uint8_t flags;
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
+} SysIBCPUListEntry;
+QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
#define ASCE_SUBSPACE 0x200 /* subspace group control */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 06b60ebce4..5971804680 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -28,10 +28,12 @@
* s390_topology is used to keep the topology information.
* .cores_per_socket: tracks information on the count of cores
* per socket.
+ * .polarization: tracks machine polarization.
*/
S390Topology s390_topology = {
/* will be initialized after the CPU model is realized */
.cores_per_socket = NULL,
+ .polarization = S390_CPU_POLARIZATION_HORIZONTAL,
};
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 852fbd0df7..56b31b8aae 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -1911,9 +1911,12 @@ static int handle_stsi(S390CPU *cpu)
if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
return 0;
}
- /* Only sysib 3.2.2 needs post-handling for now. */
insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+ run->s390_stsi.ar, RA_IGNORED);
+ return 0;
default:
return 0;
}
diff --git a/target/s390x/kvm/stsi-topology.c b/target/s390x/kvm/stsi-topology.c
new file mode 100644
index 0000000000..cb78040ea5
--- /dev/null
+++ b/target/s390x/kvm/stsi-topology.c
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/sclp.h"
+#include "hw/s390x/cpu-topology.h"
+
+QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_LOW != 1);
+QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_MEDIUM != 2);
+QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_HIGH != 3);
+
+/**
+ * fill_container:
+ * @p: The address of the container TLE to fill
+ * @level: The level of nesting for this container
+ * @id: The container receives a unique ID inside its own container
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_container(char *p, int level, int id)
+{
+ SYSIBContainerListEntry *tle = (SYSIBContainerListEntry *)p;
+
+ tle->nl = level;
+ tle->id = id;
+ return p + sizeof(*tle);
+}
+
+/**
+ * fill_tle_cpu:
+ * @p: The address of the CPU TLE to fill
+ * @entry: a pointer to the S390TopologyEntry defining this
+ * CPU container.
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
+{
+ SysIBCPUListEntry *tle = (SysIBCPUListEntry *)p;
+ s390_topology_id topology_id = entry->id;
+
+ tle->nl = 0;
+ tle->flags = 3 - topology_id.inv_polarization;
+ if (!topology_id.not_dedicated) {
+ tle->flags |= SYSIB_TLE_DEDICATED;
+ }
+ tle->type = topology_id.type;
+ tle->origin = cpu_to_be16(topology_id.origin * 64);
+ tle->mask = cpu_to_be64(entry->mask);
+ return p + sizeof(*tle);
+}
+
+/*
+ * Macro to check that the size of data after increment
+ * will not get bigger than the size of the SysIB.
+ */
+#define SYSIB_GUARD(data, x) do { \
+ data += x; \
+ if (data > sizeof(SysIB)) { \
+ return 0; \
+ } \
+ } while (0)
+
+/**
+ * stsi_topology_fill_sysib:
+ * @p: A pointer to the position of the first TLE
+ * @level: The nested level wanted by the guest
+ *
+ * Fill the SYSIB with the topology information as described in
+ * the PoP, nesting containers as appropriate, with the maximum
+ * nesting limited by @level.
+ *
+ * Return value:
+ * On success: the size of the SysIB_15x after being filled with TLE.
+ * On error: 0 in the case we would overrun the end of the SysIB.
+ */
+static int stsi_topology_fill_sysib(S390TopologyList *topology_list,
+ char *p, int level)
+{
+ S390TopologyEntry *entry;
+ int last_drawer = -1;
+ int last_book = -1;
+ int last_socket = -1;
+ int drawer_id = 0;
+ int book_id = 0;
+ int socket_id = 0;
+ int n = sizeof(SysIB_151x);
+
+ QTAILQ_FOREACH(entry, topology_list, next) {
+ bool drawer_change = last_drawer != entry->id.drawer;
+ bool book_change = drawer_change || last_book != entry->id.book;
+ bool socket_change = book_change || last_socket != entry->id.socket;
+
+ if (level > 3 && drawer_change) {
+ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 3, drawer_id++);
+ book_id = 0;
+ }
+ if (level > 2 && book_change) {
+ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 2, book_id++);
+ socket_id = 0;
+ }
+ if (socket_change) {
+ SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
+ p = fill_container(p, 1, socket_id++);
+ }
+
+ SYSIB_GUARD(n, sizeof(SysIBCPUListEntry));
+ p = fill_tle_cpu(p, entry);
+ last_drawer = entry->id.drawer;
+ last_book = entry->id.book;
+ last_socket = entry->id.socket;
+ }
+
+ return n;
+}
+
+/**
+ * setup_stsi:
+ * sysib: pointer to a SysIB to be filled with SysIB_151x data
+ * level: Nested level specified by the guest
+ *
+ * Setup the SYSIB for STSI 15.1, the header as well as the description
+ * of the topology.
+ */
+static int setup_stsi(S390TopologyList *topology_list, SysIB_151x *sysib,
+ int level)
+{
+ sysib->mnest = level;
+ switch (level) {
+ case 4:
+ sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 3:
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
+ current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 2:
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
+ current_machine->smp.books *
+ current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ }
+
+ return stsi_topology_fill_sysib(topology_list, sysib->tle, level);
+}
+
+/**
+ * s390_topology_add_cpu_to_entry:
+ * @entry: Topology entry to setup
+ * @cpu: the S390CPU to add
+ *
+ * Set the core bit inside the topology mask.
+ */
+static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
+ S390CPU *cpu)
+{
+ set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
+}
+
+/**
+ * s390_topology_from_cpu:
+ * @cpu: S390CPU to calculate the topology id
+ *
+ * Initialize the topology id from the CPU environment.
+ */
+static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
+{
+ s390_topology_id topology_id = {0};
+
+ topology_id.drawer = cpu->env.drawer_id;
+ topology_id.book = cpu->env.book_id;
+ topology_id.socket = cpu->env.socket_id;
+ topology_id.type = S390_TOPOLOGY_CPU_IFL;
+ topology_id.not_dedicated = !cpu->env.dedicated;
+
+ topology_id.inv_polarization = 3;
+ if (s390_topology.polarization == S390_CPU_POLARIZATION_VERTICAL) {
+ topology_id.inv_polarization -= cpu->env.entitlement;
+ }
+
+ topology_id.origin = cpu->env.core_id / 64;
+
+ return topology_id;
+}
+
+/**
+ * s390_topology_fill_list_sorted:
+ *
+ * Loop over all CPU and insert it at the right place
+ * inside the TLE entry list.
+ * Fill the S390Topology list with entries according to the order
+ * specified by the PoP.
+ */
+static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
+{
+ CPUState *cs;
+ S390TopologyEntry sentinel;
+
+ QTAILQ_INIT(topology_list);
+
+ sentinel.id.id = cpu_to_be64(UINT64_MAX);
+ QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
+
+ CPU_FOREACH(cs) {
+ s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
+ S390TopologyEntry *entry, *tmp;
+
+ QTAILQ_FOREACH(tmp, topology_list, next) {
+ if (id.id == tmp->id.id) {
+ entry = tmp;
+ break;
+ } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
+ entry = g_malloc0(sizeof(*entry));
+ entry->id.id = id.id;
+ QTAILQ_INSERT_BEFORE(tmp, entry, next);
+ break;
+ }
+ }
+ s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
+ }
+
+ QTAILQ_REMOVE(topology_list, &sentinel, next);
+}
+
+/**
+ * s390_topology_empty_list:
+ *
+ * Clear all entries in the S390Topology list.
+ */
+static void s390_topology_empty_list(S390TopologyList *topology_list)
+{
+ S390TopologyEntry *entry = NULL;
+ S390TopologyEntry *tmp = NULL;
+
+ QTAILQ_FOREACH_SAFE(entry, topology_list, next, tmp) {
+ QTAILQ_REMOVE(topology_list, entry, next);
+ g_free(entry);
+ }
+}
+
+/**
+ * insert_stsi_15_1_x:
+ * cpu: the CPU doing the call for which we set CC
+ * sel2: the selector 2, containing the nested level
+ * addr: Guest logical address of the guest SysIB
+ * ar: the access register number
+ *
+ * Emulate STSI 15.1.x, that is, perform all necessary checks and
+ * fill the SYSIB.
+ * In case the topology description is too long to fit into the SYSIB,
+ * set CC=3 and abort without writing the SYSIB.
+ */
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra)
+{
+ S390TopologyList topology_list;
+ SysIB sysib = {0};
+ int length;
+
+ if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ s390_topology_fill_list_sorted(&topology_list);
+
+ length = setup_stsi(&topology_list, &sysib.sysib_151x, sel2);
+
+ if (!length) {
+ s390_topology_empty_list(&topology_list);
+ setcc(cpu, 3);
+ return;
+ }
+
+ sysib.sysib_151x.length = cpu_to_be16(length);
+ if (!s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length)) {
+ setcc(cpu, 0);
+ } else {
+ s390_cpu_virt_mem_handle_exc(cpu, ra);
+ }
+
+ s390_topology_empty_list(&topology_list);
+}
diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build
index d6aca590ae..588a9aa737 100644
--- a/target/s390x/kvm/meson.build
+++ b/target/s390x/kvm/meson.build
@@ -1,7 +1,8 @@
s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
'pv.c',
- 'kvm.c'
+ 'kvm.c',
+ 'stsi-topology.c'
), if_false: files(
'stubs.c'
))
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 04/20] s390x/sclp: reporting the maximum nested topology entries
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (2 preceding siblings ...)
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 05/20] s390x/cpu topology: resetting the Topology-Change-Report Nina Schoetterl-Glausch
` (15 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index c49051e17e..9aef6d9370 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST 2
+#define SCLP_READ_SCP_INFO_MNEST 4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
- uint8_t _reserved1[16 - 11]; /* 11-15 */
+ uint8_t _reserved1[15 - 11]; /* 11-14 */
+ uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu; /* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+ if (s390_has_topology()) {
+ read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+ }
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, &cpu_count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 05/20] s390x/cpu topology: resetting the Topology-Change-Report
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (3 preceding siblings ...)
2023-09-01 15:57 ` [PATCH v22 04/20] s390x/sclp: reporting the maximum nested topology entries Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 06/20] s390x/cpu topology: interception of PTF instruction Nina Schoetterl-Glausch
` (14 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h | 1 +
hw/s390x/cpu-topology.c | 11 +++++++++++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +++++++++++++
target/s390x/kvm/kvm.c | 17 +++++++++++++++++
7 files changed, 47 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index fc15acf297..ad8110fbc7 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -58,6 +58,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,
#endif
extern S390Topology s390_topology;
+void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index b8a0c02714..b737fd975d 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -654,6 +654,7 @@ typedef struct SysIBCPUListEntry {
QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 5971804680..3a9fc14a90 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,17 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * to reset the kernel Modified Topology Change Record.
+ */
+void s390_topology_reset(void)
+{
+ s390_cpu_topology_set_changed(false);
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 4f9ec18723..6eebd8a054 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -123,6 +123,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+ if (s390_has_topology()) {
+ s390_topology_reset();
+ }
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 8112561e5e..1cd30c1d84 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -307,3 +307,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_set_changed(bool changed)
+{
+ int ret;
+
+ if (kvm_enabled()) {
+ ret = kvm_s390_topology_set_mtcr(changed);
+ if (ret) {
+ error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+ }
+ }
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 56b31b8aae..ee6345ba27 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2584,6 +2584,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+ struct kvm_device_attr attribute = {
+ .group = KVM_S390_VM_CPU_TOPOLOGY,
+ .attr = attr,
+ };
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ return 0;
+ }
+ if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+ return -ENOTSUP;
+ }
+
+ return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute);
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 06/20] s390x/cpu topology: interception of PTF instruction
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (4 preceding siblings ...)
2023-09-01 15:57 ` [PATCH v22 05/20] s390x/cpu topology: resetting the Topology-Change-Report Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 07/20] target/s390x/cpu topology: activate CPU topology Nina Schoetterl-Glausch
` (13 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervisor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
include/hw/s390x/s390-virtio-ccw.h | 6 ++++
hw/s390x/cpu-topology.c | 55 ++++++++++++++++++++++++++++++
target/s390x/kvm/kvm.c | 11 ++++++
3 files changed, 72 insertions(+)
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..c1d46e78af 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 3a9fc14a90..b86d582653 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,60 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+ CpuS390Polarization polarization;
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ int fc = reg & S390_TOPO_FC_MASK;
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ s390_program_interrupt(env, PGM_OPERATION, ra);
+ return;
+ }
+
+ if (env->psw.mask & PSW_MASK_PSTATE) {
+ s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+ return;
+ }
+
+ if (reg & ~S390_TOPO_FC_MASK) {
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ return;
+ }
+
+ polarization = S390_CPU_POLARIZATION_VERTICAL;
+ switch (fc) {
+ case 0:
+ polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+ /* fallthrough */
+ case 1:
+ if (s390_topology.polarization == polarization) {
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ setcc(cpu, 2);
+ } else {
+ s390_topology.polarization = polarization;
+ s390_cpu_topology_set_changed(true);
+ setcc(cpu, 0);
+ }
+ break;
+ default:
+ /* Note that fc == 2 is interpreted by the SIE */
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ }
+}
+
/**
* s390_topology_reset:
*
@@ -99,6 +153,7 @@ static void s390_topology_init(MachineState *ms)
void s390_topology_reset(void)
{
s390_cpu_topology_set_changed(false);
+ s390_topology.polarization = S390_CPU_POLARIZATION_HORIZONTAL;
}
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index ee6345ba27..10d66c2b65 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -86,6 +86,7 @@
#define PRIV_B9_EQBS 0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1457,6 +1458,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+ uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+ s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1474,6 +1482,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+ case PRIV_B9_PTF:
+ kvm_handle_ptf(cpu, run);
+ break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 07/20] target/s390x/cpu topology: activate CPU topology
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (5 preceding siblings ...)
2023-09-01 15:57 ` [PATCH v22 06/20] s390x/cpu topology: interception of PTF instruction Nina Schoetterl-Glausch
@ 2023-09-01 15:57 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command Nina Schoetterl-Glausch
` (12 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:57 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 9 +++++++++
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index b86d582653..52bfbe12a9 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -69,7 +69,7 @@ static int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
- return false;
+ return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 91ce896491..c67b7eeb84 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -255,6 +255,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+ case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 10d66c2b65..42795d3027 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -366,6 +366,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+ kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -2462,6 +2463,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+ /*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+ if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+ set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+ }
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (6 preceding siblings ...)
2023-09-01 15:57 ` [PATCH v22 07/20] target/s390x/cpu topology: activate CPU topology Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-06 8:56 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast Nina Schoetterl-Glausch
` (11 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
qapi/machine-target.json | 37 +++++++++++
hw/s390x/cpu-topology.c | 135 +++++++++++++++++++++++++++++++++++++++
2 files changed, 172 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 275234a20f..1e7f512477 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -375,3 +377,38 @@
'data': [ 'horizontal', 'vertical' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @set-cpu-topology:
+#
+# @core-id: the vCPU ID to be moved
+# @socket-id: optional destination socket where to move the vCPU
+# @book-id: optional destination book where to move the vCPU
+# @drawer-id: optional destination drawer where to move the vCPU
+# @entitlement: optional entitlement
+# @dedicated: optional, if the vCPU is dedicated to a real CPU
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+# Default value for optional parameter is the current value
+# used by the CPU.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 52bfbe12a9..c828538f2a 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -23,6 +23,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -259,6 +260,29 @@ static bool s390_topology_check(uint16_t socket_id, uint16_t book_id,
return true;
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+ return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -328,3 +352,114 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement,
+ CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+ MachineState *ms = current_machine;
+ int old_socket_entry;
+ int new_socket_entry;
+ bool report_needed;
+ S390CPU *cpu;
+ ERRP_GUARD();
+
+ cpu = s390_cpu_addr2state(core_id);
+ if (!cpu) {
+ error_setg(errp, "Core-id %d does not exist!", core_id);
+ return;
+ }
+
+ /* Get attributes not provided from cpu and verify the new topology */
+ if (!has_socket_id) {
+ socket_id = cpu->env.socket_id;
+ }
+ if (!has_book_id) {
+ book_id = cpu->env.book_id;
+ }
+ if (!has_drawer_id) {
+ drawer_id = cpu->env.drawer_id;
+ }
+ if (!has_dedicated) {
+ dedicated = cpu->env.dedicated;
+ }
+
+ /*
+ * When the user specifies the entitlement as 'auto' on the command line,
+ * qemu will set the entitlement as:
+ * Medium when the CPU is not dedicated.
+ * High when dedicated is true.
+ */
+ if (!has_entitlement || (entitlement == S390_CPU_ENTITLEMENT_AUTO)) {
+ if (dedicated) {
+ entitlement = S390_CPU_ENTITLEMENT_HIGH;
+ } else {
+ entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
+ }
+ }
+
+ if (!s390_topology_check(socket_id, book_id, drawer_id,
+ entitlement, dedicated, errp)) {
+ return;
+ }
+
+ /* Check for space on new socket */
+ old_socket_entry = s390_socket_nb(cpu);
+ new_socket_entry = s390_socket_nb_from_ids(drawer_id, book_id, socket_id);
+
+ if (new_socket_entry != old_socket_entry) {
+ if (s390_topology.cores_per_socket[new_socket_entry] >=
+ ms->smp.cores) {
+ error_setg(errp, "No more space on this socket");
+ return;
+ }
+ /* Update the count of cores in sockets */
+ s390_topology.cores_per_socket[new_socket_entry] += 1;
+ s390_topology.cores_per_socket[old_socket_entry] -= 1;
+ }
+
+ /* Check if we will need to report the modified topology */
+ report_needed = s390_topology_need_report(cpu, drawer_id, book_id,
+ socket_id, entitlement,
+ dedicated);
+
+ /* All checks done, report new topology into the vCPU */
+ cpu->env.drawer_id = drawer_id;
+ cpu->env.book_id = book_id;
+ cpu->env.socket_id = socket_id;
+ cpu->env.dedicated = dedicated;
+ cpu->env.entitlement = entitlement;
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+
+ /* Advertise the topology change */
+ if (report_needed) {
+ s390_cpu_topology_set_changed(true);
+ }
+}
+
+void qmp_set_cpu_topology(uint16_t core,
+ bool has_socket, uint16_t socket,
+ bool has_book, uint16_t book,
+ bool has_drawer, uint16_t drawer,
+ bool has_entitlement, CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+ ERRP_GUARD();
+
+ if (!s390_has_topology()) {
+ error_setg(errp, "This machine doesn't support topology");
+ return;
+ }
+
+ s390_change_topology(core, has_socket, socket, has_book, book,
+ has_drawer, drawer, has_entitlement, entitlement,
+ has_dedicated, dedicated, errp);
+}
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (7 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-06 9:10 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus Nina Schoetterl-Glausch
` (10 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
qapi/machine.json | 9 ++++++++-
target/s390x/cpu.c | 9 +++++++++
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 54f99f4ac1..eba1f5c363 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -56,10 +56,17 @@
# Additional information about a virtual S390 CPU
#
# @cpu-state: the virtual CPU's state
+# @dedicated: the virtual CPU's dedication (since 8.2)
+# @entitlement: the virtual CPU's entitlement (since 8.2)
#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+ '*dedicated': 'bool',
+ '*entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 74405beb51..5967e34a85 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -38,6 +38,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC2000000UL;
@@ -146,6 +147,14 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast *value)
S390CPU *s390_cpu = S390_CPU(cpu);
value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
+ if (s390_has_topology()) {
+ value->u.s390x.has_dedicated = true;
+ value->u.s390x.dedicated = s390_cpu->env.dedicated;
+ value->u.s390x.has_entitlement = true;
+ value->u.s390x.entitlement = s390_cpu->env.entitlement;
+ }
+#endif
}
/* S390CPUClass::reset() */
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (8 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-06 9:14 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event Nina Schoetterl-Glausch
` (9 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
hw/core/machine-hmp-cmds.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..9a4b59c6f2 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, " node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+ if (c->has_drawer_id) {
+ monitor_printf(mon, " drawer-id: \"%" PRIu64 "\"\n", c->drawer_id);
+ }
+ if (c->has_book_id) {
+ monitor_printf(mon, " book-id: \"%" PRIu64 "\"\n", c->book_id);
+ }
if (c->has_socket_id) {
monitor_printf(mon, " socket-id: \"%" PRIu64 "\"\n", c->socket_id);
}
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (9 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command Nina Schoetterl-Glausch
` (8 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
qapi/machine-target.json | 33 +++++++++++++++++++++++++++++++++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 35 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 1e7f512477..5c7a34c25f 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -412,3 +412,36 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index c828538f2a..92513cb027 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,6 +24,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -136,6 +137,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
+ qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (10 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology Nina Schoetterl-Glausch
` (7 subsequent siblings)
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
The query-cpu-polarization qmp command returns the current
CPU polarization of the machine.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
qapi/machine-target.json | 29 +++++++++++++++++++++++++++++
hw/s390x/cpu-topology.c | 8 ++++++++
2 files changed, 37 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 5c7a34c25f..898e4458d7 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -445,3 +445,32 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CpuPolarizationInfo:
+#
+# The result of a cpu polarization
+#
+# @polarization: the CPU polarization
+#
+# Since: 8.2
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
+
+##
+# @query-s390x-cpu-polarization:
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Returns: the machine polarization
+#
+# Since: 8.2
+##
+{ 'command': 'query-s390x-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 92513cb027..e1e6853e17 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -465,3 +465,11 @@ void qmp_set_cpu_topology(uint16_t core,
has_drawer, drawer, has_entitlement, entitlement,
has_dedicated, dedicated, errp);
}
+
+CpuPolarizationInfo *qmp_query_s390x_cpu_polarization(Error **errp)
+{
+ CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
+ info->polarization = s390_topology.polarization;
+ return info;
+}
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (11 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-07 8:15 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 14/20] tests/avocado: s390x cpu topology core Nina Schoetterl-Glausch
` (6 subsequent siblings)
19 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
MAINTAINERS | 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 171 ++++++++++++++++++++
docs/system/s390x/cpu-topology.rst | 242 +++++++++++++++++++++++++++++
docs/system/target-s390x.rst | 1 +
5 files changed, 417 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 692ce9f121..7f2937b0f2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1701,6 +1701,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
------------
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 0000000000..70633f30cb
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,171 @@
+QAPI interface for S390 CPU topology
+====================================
+
+The following sections will explain the QAPI interface for S390 CPU topology
+with the help of exemplary output.
+For this, let's assume that QEMU has been started with the following
+command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
+have default values:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+ -enable-kvm \
+ -cpu z14,ctop=on \
+ -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+ \
+ -device z14-s390x-cpu,core-id=19,entitlement=high \
+ -device z14-s390x-cpu,core-id=11,entitlement=low \
+ -device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
+Additions to query-cpus-fast
+----------------------------
+
+The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+ {
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+ "core-id": 0,
+ "socket-id": 0,
+ "drawer-id": 0,
+ "book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+ "core-id": 19,
+ "socket-id": 1,
+ "drawer-id": 0,
+ "book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+ "core-id": 11,
+ "socket-id": 1,
+ "drawer-id": 0,
+ "book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+ },
+ {
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+ "core-id": 112,
+ "socket-id": 0,
+ "drawer-id": 3,
+ "book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+ }
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+------------------------------
+
+The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+ { "execute": "set-cpu-topology",
+ "arguments": {
+ "core-id": 11,
+ "socket-id": 0,
+ "book-id": 0,
+ "drawer-id": 0,
+ "entitlement": "low",
+ "dedicated": false
+ }
+ }
+ {"return": {}}
+
+The core-id parameter is the only mandatory parameter and every
+unspecified parameter keeps its previous value.
+
+QAPI event CPU_POLARIZATION_CHANGE
+----------------------------------
+
+When a guest requests a modification of the polarization,
+QEMU sends a CPU_POLARIZATION_CHANGE event.
+
+When requesting the change, the guest only specifies horizontal or
+vertical polarization.
+It is the job of the entity administrating QEMU to set the dedication and fine
+grained vertical entitlement in response to this event.
+
+Note that a vertical polarized dedicated vCPU can only have a high
+entitlement, giving 6 possibilities for vCPU polarization:
+
+- Horizontal
+- Horizontal dedicated
+- Vertical low
+- Vertical medium
+- Vertical high
+- Vertical high dedicated
+
+Example of the event received when the guest issues the CPU instruction
+Perform Topology Function PTF(0) to request an horizontal polarization:
+
+.. code-block:: QMP
+
+ {
+ "timestamp": {
+ "seconds": 1687870305,
+ "microseconds": 566299
+ },
+ "event": "CPU_POLARIZATION_CHANGE",
+ "data": {
+ "polarization": "horizontal"
+ }
+ }
+
+QAPI query command: query-s390x-cpu-polarization
+------------------------------------------------
+
+The query command query-s390x-cpu-polarization returns the current
+CPU polarization of the machine.
+In this case the guest previously issued a PTF(1) to request vertical polarization:
+
+.. code-block:: QMP
+
+ { "execute": "query-s390x-cpu-polarization" }
+ {
+ "return": {
+ "polarization": "vertical"
+ }
+ }
diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
new file mode 100644
index 0000000000..78b98f978a
--- /dev/null
+++ b/docs/system/s390x/cpu-topology.rst
@@ -0,0 +1,242 @@
+CPU topology on s390x
+=====================
+
+Since QEMU 8.2, CPU topology on s390x provides up to 3 levels of
+topology containers: drawers, books and sockets. They define a
+tree-shaped hierarchy.
+
+The socket container has one or more CPU entries.
+Each of these CPU entries consists of a bitmap and three CPU attributes:
+
+- CPU type
+- entitlement
+- dedication
+
+Each bit set in the bitmap correspond to a core-id of a vCPU with matching
+attributes.
+
+This documentation provides general information on S390 CPU topology,
+how to enable it and explains the new CPU attributes.
+For information on how to modify the S390 CPU topology and how to
+monitor polarization changes, see ``docs/devel/s390-cpu-topology.rst``.
+
+Prerequisites
+-------------
+
+To use the CPU topology, you need to run with KVM on a s390x host that
+uses the Linux kernel v6.0 or newer (which provide the so-called
+``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the
+CPU topology facility via the so-called STFLE bit 11 to the VM).
+
+Enabling CPU topology
+---------------------
+
+Currently, CPU topology is only enabled in the host model by default.
+
+Enabling CPU topology in a CPU model is done by setting the CPU flag
+``ctop`` to ``on`` as in:
+
+.. code-block:: bash
+
+ -cpu gen16b,ctop=on
+
+Having the topology disabled by default allows migration between
+old and new QEMU without adding new flags.
+
+Default topology usage
+----------------------
+
+The CPU topology can be specified on the QEMU command line
+with the ``-smp`` or the ``-device`` QEMU command arguments.
+
+Note also that since 7.2 threads are no longer supported in the topology
+and the ``-smp`` command line argument accepts only ``threads=1``.
+
+If none of the containers attributes (drawers, books, sockets) are
+specified for the ``-smp`` flag, the number of these containers
+is 1.
+
+Thus the following two options will result in the same topology:
+
+.. code-block:: bash
+
+ -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32
+
+and
+
+.. code-block:: bash
+
+ -smp cpus=5,sockets=8,cores=4,maxcpus=32
+
+When a CPU is defined by the ``-smp`` command argument, its position
+inside the topology is calculated by adding the CPUs to the topology
+based on the core-id starting with core-0 at position 0 of socket-0,
+book-0, drawer-0 and filling all CPUs of socket-0 before filling socket-1
+of book-0 and so on up to the last socket of the last book of the last
+drawer.
+
+When a CPU is defined by the ``-device`` command argument, the
+tree topology attributes must all be defined or all not defined.
+
+.. code-block:: bash
+
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
+
+or
+
+.. code-block:: bash
+
+ -device gen16b-s390x-cpu,core-id=1,dedicated=true
+
+If none of the tree attributes (drawer, book, sockets), are specified
+for the ``-device`` argument, like for all CPUs defined with the ``-smp``
+command argument the topology tree attributes will be set by simply
+adding the CPUs to the topology based on the core-id.
+
+QEMU will not try to resolve collisions and will report an error if the
+CPU topology defined explicitly or implicitly on a ``-device``
+argument collides with the definition of a CPU implicitly defined
+on the ``-smp`` argument.
+
+When the topology modifier attributes are not defined for the
+``-device`` command argument they takes following default values:
+
+- dedicated: ``false``
+- entitlement: ``medium``
+
+
+Hot plug
+++++++++
+
+New CPUs can be plugged using the device_add hmp command as in:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
+The placement of the CPU is derived from the core-id as described above.
+
+The topology can of course also be fully defined:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
+
+
+Examples
+++++++++
+
+In the following machine we define 8 sockets with 4 cores each.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
+ -device host-s390x-cpu,core-id=14 \
+
+A new CPUs can be plugged using the device_add hmp command as before:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
+The core-id defines the placement of the core in the topology by
+starting with core 0 in socket 0 up to maxcpus.
+
+In the example above:
+
+* There are 5 CPUs provided to the guest with the ``-smp`` command line
+ They will take the core-ids 0,1,2,3,4
+ As we have 4 cores in a socket, we have 4 CPUs provided
+ to the guest in socket 0, with core-ids 0,1,2,3.
+ The last cpu, with core-id 4, will be on socket 1.
+
+* the core with ID 14 provided by the ``-device`` command line will
+ be placed in socket 3, with core-id 14
+
+* the core with ID 9 provided by the ``device_add`` qmp command will
+ be placed in socket 2, with core-id 9
+
+
+Polarization, entitlement and dedication
+----------------------------------------
+
+Polarization
+++++++++++++
+
+The polarization affects how the CPUs of a shared host are utilized/distributed
+among guests.
+The guest determines the polarization by using the PTF instruction.
+
+Polarization defines two models of CPU provisioning: horizontal
+and vertical.
+
+The horizontal polarization is the default model on boot and after
+subsystem reset. When horizontal polarization is in effect all vCPUs should
+have about equal resource provisioning.
+
+In the vertical polarization model vCPUs are unequal, but overall more resources
+might be available.
+The guest can make use of the vCPU entitlement information provided by the host
+to optimize kernel thread scheduling.
+
+A subsystem reset puts all vCPU of the configuration into the
+horizontal polarization.
+
+Entitlement
++++++++++++
+
+The vertical polarization specifies that the guest's vCPU can get
+different real CPU provisioning:
+
+- a vCPU with vertical high entitlement specifies that this
+ vCPU gets 100% of the real CPU provisioning.
+
+- a vCPU with vertical medium entitlement specifies that this
+ vCPU shares the real CPU with other vCPUs.
+
+- a vCPU with vertical low entitlement specifies that this
+ vCPU only gets real CPU provisioning when no other vCPUs needs it.
+
+In the case a vCPU with vertical high entitlement does not use
+the real CPU, the unused "slack" can be dispatched to other vCPU
+with medium or low entitlement.
+
+A vCPU can be "dedicated" in which case the vCPU is fully dedicated to a single
+real CPU.
+
+The dedicated bit is an indication of affinity of a vCPU for a real CPU
+while the entitlement indicates the sharing or exclusivity of use.
+
+Defining the topology on the command line
+-----------------------------------------
+
+The topology can entirely be defined using -device cpu statements,
+with the exception of CPU 0 which must be defined with the -smp
+argument.
+
+For example, here we set the position of the cores 1,2,3 to
+drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0,
+book 0, socket 0 without defining entitlement or dedication.
+Core 4 will be set on its default position on socket 1
+(since we have 4 core per socket) and we define it as dedicated and
+with vertical high entitlement.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
+ \
+ -device gen16b-s390x-cpu,core-id=4,dedicated=on,entitlement=high
+
+The entitlement defined for the CPU 4 will only be used after the guest
+successfully enables vertical polarization by using the PTF instruction.
diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
index f6f11433c7..94c981e732 100644
--- a/docs/system/target-s390x.rst
+++ b/docs/system/target-s390x.rst
@@ -34,3 +34,4 @@ Architectural features
.. toctree::
s390x/bootdevices
s390x/protvirt
+ s390x/cpu-topology
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 14/20] tests/avocado: s390x cpu topology core
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (12 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-04 18:44 ` Nina Schoetterl-Glausch
2023-09-07 8:27 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization Nina Schoetterl-Glausch
` (5 subsequent siblings)
19 siblings, 2 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
Introduction of the s390x cpu topology core functions and
basic tests.
We test the correlation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
MAINTAINERS | 1 +
tests/avocado/s390_topology.py | 200 +++++++++++++++++++++++++++++++++
2 files changed, 201 insertions(+)
create mode 100644 tests/avocado/s390_topology.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 7f2937b0f2..d1b8676666 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1703,6 +1703,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
+F: tests/avocado/s390_topology.py
X86 Machines
------------
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
new file mode 100644
index 0000000000..3a54fe2e00
--- /dev/null
+++ b/tests/avocado/s390_topology.py
@@ -0,0 +1,200 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright IBM Corp. 2023
+#
+# Author:
+# Pierre Morel <pmorel@linux.ibm.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import shutil
+import time
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import exec_command
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class S390CPUTopology(QemuSystemTest):
+ """
+ S390x CPU topology consists of 4 topology layers, from bottom to top,
+ the cores, sockets, books and drawers and 2 modifiers attributes,
+ the entitlement and the dedication.
+ See: docs/system/s390x/cpu-topology.rst.
+
+ S390x CPU topology is setup in different ways:
+ - implicitly from the '-smp' argument by completing each topology
+ level one after the other beginning with drawer 0, book 0 and
+ socket 0.
+ - explicitly from the '-device' argument on the QEMU command line
+ - explicitly by hotplug of a new CPU using QMP or HMP
+ - it is modified by using QMP 'set-cpu-topology'
+
+ The S390x modifier attribute entitlement depends on the machine
+ polarization, which can be horizontal or vertical.
+ The polarization is changed on a request from the guest.
+ """
+ timeout = 90
+
+ KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
+ 'root=/dev/ram '
+ 'selinux=0 '
+ 'rdinit=/bin/sh')
+
+ def wait_until_booted(self):
+ wait_for_console_pattern(self, 'no job control',
+ failure_message='Kernel panic - not syncing',
+ vm=None)
+
+ def check_topology(self, c, s, b, d, e, t):
+ res = self.vm.qmp('query-cpus-fast')
+ cpus = res['return']
+ for cpu in cpus:
+ core = cpu['props']['core-id']
+ socket = cpu['props']['socket-id']
+ book = cpu['props']['book-id']
+ drawer = cpu['props']['drawer-id']
+ entitlement = cpu.get('entitlement')
+ dedicated = cpu.get('dedicated')
+ if core == c:
+ self.assertEqual(drawer, d)
+ self.assertEqual(book, b)
+ self.assertEqual(socket, s)
+ self.assertEqual(entitlement, e)
+ self.assertEqual(dedicated, t)
+
+ def kernel_init(self):
+ """
+ We need a VM that supports CPU topology,
+ currently this only the case when using KVM, not TCG.
+ We need a kernel supporting the CPU topology.
+ We need a minimal root filesystem with a shell.
+ """
+ self.require_accelerator("kvm")
+ kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
+ kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
+ kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
+ asset_hash=kernel_hash)
+
+ initrd_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img')
+ initrd_hash = 'a122057d95725ac030e2ec51df46e172'
+ initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
+ asset_hash=initrd_hash)
+ initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
+ archive.lzma_uncompress(initrd_path_xz, initrd_path)
+
+ self.vm.set_console()
+ kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
+ self.vm.add_args('-nographic',
+ '-enable-kvm',
+ '-cpu', 'max,ctop=on',
+ '-m', '512',
+ '-kernel', kernel_path,
+ '-initrd', initrd_path,
+ '-append', kernel_command_line)
+
+ def test_single(self):
+ """
+ This test checks the simplest topology with a single CPU.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ def test_default(self):
+ """
+ This test checks the implicit topology.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_until_booted()
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+ self.check_topology(1, 0, 0, 0, 'medium', False)
+ self.check_topology(2, 1, 0, 0, 'medium', False)
+ self.check_topology(3, 1, 0, 0, 'medium', False)
+ self.check_topology(4, 2, 0, 0, 'medium', False)
+ self.check_topology(5, 2, 0, 0, 'medium', False)
+ self.check_topology(6, 0, 1, 0, 'medium', False)
+ self.check_topology(7, 0, 1, 0, 'medium', False)
+ self.check_topology(8, 1, 1, 0, 'medium', False)
+ self.check_topology(9, 1, 1, 0, 'medium', False)
+ self.check_topology(10, 2, 1, 0, 'medium', False)
+ self.check_topology(11, 2, 1, 0, 'medium', False)
+ self.check_topology(12, 0, 0, 1, 'medium', False)
+
+ def test_move(self):
+ """
+ This test checks the topology modification by moving a CPU
+ to another socket: CPU 0 is moved from socket 0 to socket 2.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'socket-id': 2, 'entitlement': 'low'})
+ self.assertEqual(res['return'], {})
+ self.check_topology(0, 2, 0, 0, 'low', False)
+
+ def test_hotplug_full(self):
+ """
+ This test verifies that a hotplugged defined with '-device'
+ command line argument finds its right place inside the topology.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.add_args('-device', 'max-s390x-cpu,core-id=10')
+ self.vm.add_args('-device',
+ 'max-s390x-cpu,'
+ 'core-id=1,socket-id=0,book-id=1,drawer-id=1,entitlement=low')
+ self.vm.add_args('-device',
+ 'max-s390x-cpu,'
+ 'core-id=2,socket-id=0,book-id=1,drawer-id=1,entitlement=medium')
+ self.vm.add_args('-device',
+ 'max-s390x-cpu,'
+ 'core-id=3,socket-id=1,book-id=1,drawer-id=1,entitlement=high')
+ self.vm.add_args('-device',
+ 'max-s390x-cpu,'
+ 'core-id=4,socket-id=1,book-id=1,drawer-id=1')
+ self.vm.add_args('-device',
+ 'max-s390x-cpu,'
+ 'core-id=5,socket-id=2,book-id=1,drawer-id=1,dedicated=true')
+
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.check_topology(10, 2, 1, 0, 'medium', False)
+ self.check_topology(1, 0, 1, 1, 'low', False)
+ self.check_topology(2, 0, 1, 1, 'medium', False)
+ self.check_topology(3, 1, 1, 1, 'high', False)
+ self.check_topology(4, 1, 1, 1, 'medium', False)
+ self.check_topology(5, 2, 1, 1, 'high', True)
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (13 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 14/20] tests/avocado: s390x cpu topology core Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-04 19:43 ` Nina Schoetterl-Glausch
2023-09-07 9:02 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests Nina Schoetterl-Glausch
` (4 subsequent siblings)
19 siblings, 2 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
tests/avocado/s390_topology.py | 45 ++++++++++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 3a54fe2e00..d7ee36cb89 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+ event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ -103,6 +104,14 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+ def system_init(self):
+ self.log.info("System init")
+ exec_command_and_wait_for_pattern(self,
+ """ mount proc -t proc /proc;
+ mount sys -t sysfs /sys;
+ /bin/cat /sys/devices/system/cpu/dispatching """,
+ '0')
+
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
@@ -198,3 +207,39 @@ def test_hotplug_full(self):
self.check_topology(3, 1, 1, 1, 'high', False)
self.check_topology(4, 1, 1, 1, 'medium', False)
self.check_topology(5, 2, 1, 1, 'high', True)
+
+
+ def guest_set_dispatching(self, dispatching):
+ exec_command(self,
+ f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
+ self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
+ exec_command_and_wait_for_pattern(self,
+ '/bin/cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
+ def test_polarization(self):
+ """
+ This test verifies that QEMU modifies the entitlement change after
+ several guest polarization change requests.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ self.guest_set_dispatching('1');
+ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'vertical')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ self.guest_set_dispatching('0');
+ res = self.vm.qmp('query-s390x-cpu-polarization')
+ self.assertEqual(res['return']['polarization'], 'horizontal')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (14 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-04 19:51 ` Nina Schoetterl-Glausch
2023-09-07 9:05 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU Nina Schoetterl-Glausch
` (3 subsequent siblings)
19 siblings, 2 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
Test changes in the entitlement from both a guest and a host point of
view, depending on the polarization.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
tests/avocado/s390_topology.py | 55 ++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d7ee36cb89..733093723a 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -243,3 +243,58 @@ def test_polarization(self):
res = self.vm.qmp('query-s390x-cpu-polarization')
self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+
+ def check_polarization(self, polarization):
+ """
+ We need the polarization change to get stabilized in kernel
+ before we check it through the sysfs.
+ """
+ exec_command_and_wait_for_pattern(self,
+ """ sleep 1 ;
+ /bin/cat /sys/devices/system/cpu/cpu0/polarization """,
+ polarization)
+
+
+ def test_entitlement(self):
+ """
+ This test verifies that QEMU modifies the entitlement
+ after a guest request and that the guest sees the change.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
+ self.check_polarization('horizontal')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ self.guest_set_dispatching('1')
+ self.check_polarization('vertical:medium')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+ self.assertEqual(res['return'], {})
+ self.check_polarization('vertical:low')
+ self.check_topology(0, 0, 0, 0, 'low', False)
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+ self.assertEqual(res['return'], {})
+ self.check_polarization('vertical:medium')
+ self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'high'})
+ self.assertEqual(res['return'], {})
+ self.check_polarization('vertical:high')
+ self.check_topology(0, 0, 0, 0, 'high', False)
+
+ self.guest_set_dispatching('0');
+ self.check_polarization("horizontal")
+ self.check_topology(0, 0, 0, 0, 'high', False)
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (15 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-04 19:54 ` Nina Schoetterl-Glausch
2023-09-07 10:08 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 18/20] tests/avocado: s390x cpu topology test socket full Nina Schoetterl-Glausch
` (2 subsequent siblings)
19 siblings, 2 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
tests/avocado/s390_topology.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 733093723a..3b3902d646 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -298,3 +298,36 @@ def test_entitlement(self):
self.guest_set_dispatching('0');
self.check_polarization("horizontal")
self.check_topology(0, 0, 0, 0, 'high', False)
+
+
+ def test_dedicated(self):
+ """
+ This test verifies that QEMU adjusts the entitlement correctly when a
+ CPU is made dedicated.
+ QEMU retains the entitlement value when horizontal polarization is in effect.
+ For the guest, the field shows the effective value of the entitlement.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
+ self.check_polarization("horizontal")
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+ self.assertEqual(res['return'], {})
+ self.check_topology(0, 0, 0, 0, 'high', True)
+ self.check_polarization("horizontal")
+
+ self.guest_set_dispatching('1');
+ self.check_topology(0, 0, 0, 0, 'high', True)
+ self.check_polarization("vertical:high")
+
+ self.guest_set_dispatching('0');
+ self.check_topology(0, 0, 0, 0, 'high', True)
+ self.check_polarization("horizontal")
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 18/20] tests/avocado: s390x cpu topology test socket full
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (16 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 19/20] tests/avocado: s390x cpu topology dedicated errors Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 20/20] tests/avocado: s390x cpu topology bad move Nina Schoetterl-Glausch
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
This test verifies that QMP set-cpu-topology does not accept
to overload a socket.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/s390_topology.py | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 3b3902d646..d1d7a91276 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -331,3 +331,29 @@ def test_dedicated(self):
self.guest_set_dispatching('0');
self.check_topology(0, 0, 0, 0, 'high', True)
self.check_polarization("horizontal")
+
+
+ def test_socket_full(self):
+ """
+ This test verifies that QEMU does not accept to overload a socket.
+ The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+ not accept any new CPU while socket-id 0 on book-id 1 is free.
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.add_args('-smp',
+ '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1})
+ self.assertEqual(res['return'], {})
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 19/20] tests/avocado: s390x cpu topology dedicated errors
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (17 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 18/20] tests/avocado: s390x cpu topology test socket full Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 20/20] tests/avocado: s390x cpu topology bad move Nina Schoetterl-Glausch
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
Let's test that QEMU refuses to setup a dedicated CPU with
low or medium entitlement.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/s390_topology.py | 48 ++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d1d7a91276..2c47e38dab 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -357,3 +357,51 @@ def test_socket_full(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 2, 'socket-id': 0, 'book-id': 1})
self.assertEqual(res['return'], {})
+
+ def test_dedicated_error(self):
+ """
+ This test verifies that QEMU refuses to lower the entitlement
+ of a dedicated CPU
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+ self.assertEqual(res['return'], {})
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ self.guest_set_dispatching('1');
+
+ self.check_topology(0, 0, 0, 0, 'high', True)
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated': True})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated': True})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated': False})
+ self.assertEqual(res['return'], {})
+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated': False})
+ self.assertEqual(res['return'], {})
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v22 20/20] tests/avocado: s390x cpu topology bad move
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
` (18 preceding siblings ...)
2023-09-01 15:58 ` [PATCH v22 19/20] tests/avocado: s390x cpu topology dedicated errors Nina Schoetterl-Glausch
@ 2023-09-01 15:58 ` Nina Schoetterl-Glausch
19 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-01 15:58 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth, Nina Schoetterl-Glausch
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
From: Pierre Morel <pmorel@linux.ibm.com>
This test verifies that QEMU refuses to move a CPU to an
nonexistent location.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 2c47e38dab..abc599befd 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -405,3 +405,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated': False})
self.assertEqual(res['return'], {})
+
+ def test_move_error(self):
+ """
+ This test verifies that QEMU refuses to move a CPU to an
+ nonexistent location
+
+ :avocado: tags=arch:s390x
+ :avocado: tags=machine:s390-ccw-virtio
+ """
+ self.kernel_init()
+ self.vm.launch()
+ self.wait_until_booted()
+
+ self.system_init()
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+ self.assertEqual(res['error']['class'], 'GenericError')
+
+ self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
@ 2023-09-04 18:23 ` Nina Schoetterl-Glausch
2023-09-05 13:26 ` Thomas Huth
2023-09-06 14:53 ` Cédric Le Goater
2 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-04 18:23 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Fri, 2023-09-01 at 17:57 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> qapi/machine-target.json | 14 ++
> include/hw/s390x/cpu-topology.h | 25 +++
> include/hw/s390x/sclp.h | 1 +
> target/s390x/cpu.h | 76 ++++++++
> hw/s390x/cpu-topology.c | 2 +
> target/s390x/kvm/kvm.c | 5 +-
> target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++++++++++++++++
> target/s390x/kvm/meson.build | 3 +-
> 9 files changed, 421 insertions(+), 2 deletions(-)
> create mode 100644 target/s390x/kvm/stsi-topology.c
[...]
> +/*
> + * CPU Topology List provided by STSI with fc=15 provides a list
> + * of two different Topology List Entries (TLE) types to specify
> + * the topology hierarchy.
> + *
> + * - Container Topology List Entry
> + * Defines a container to contain other Topology List Entries
> + * of any type, nested containers or CPU.
> + * - CPU Topology List Entry
> + * Specifies the CPUs position, type, entitlement and polarization
> + * of the CPUs contained in the last Container TLE.
> + *
> + * There can be theoretically up to five levels of containers, QEMU
> + * uses only three levels, the drawer's, book's and socket's level.
> + *
> + * A container with a nesting level (NL) greater than 1 can only
> + * contain another container of nesting level NL-1.
> + *
> + * A container of nesting level 1 (socket), contains as many CPU TLE
> + * as needed to describe the position and qualities of all CPUs inside
> + * the container.
> + * The qualities of a CPU are polarization, entitlement and type.
> + *
> + * The CPU TLE defines the position of the CPUs of identical qualities
> + * using a 64bits mask which first bit has its offset defined by
> + * the CPU address orgin field of the CPU TLE like in:
s/orgin/origin/
> + * CPU address = origin * 64 + bit position within the mask
> + *
> + */
[...]
> diff --git a/target/s390x/kvm/stsi-topology.c b/target/s390x/kvm/stsi-topology.c
> new file mode 100644
> index 0000000000..cb78040ea5
> --- /dev/null
> +++ b/target/s390x/kvm/stsi-topology.c
[...]
> +/**
> + * setup_stsi:
> + * sysib: pointer to a SysIB to be filled with SysIB_151x data
> + * level: Nested level specified by the guest
No @ in front of the arguments here.
> + *
> + * Setup the SYSIB for STSI 15.1, the header as well as the description
> + * of the topology.
> + */
> +static int setup_stsi(S390TopologyList *topology_list, SysIB_151x *sysib,
> + int level)
[...]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 14/20] tests/avocado: s390x cpu topology core
2023-09-01 15:58 ` [PATCH v22 14/20] tests/avocado: s390x cpu topology core Nina Schoetterl-Glausch
@ 2023-09-04 18:44 ` Nina Schoetterl-Glausch
2023-09-07 8:27 ` Thomas Huth
1 sibling, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-04 18:44 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Fri, 2023-09-01 at 17:58 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Introduction of the s390x cpu topology core functions and
> basic tests.
>
> We test the correlation between the command line and
> the QMP results in query-cpus-fast for various CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization
2023-09-01 15:58 ` [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization Nina Schoetterl-Glausch
@ 2023-09-04 19:43 ` Nina Schoetterl-Glausch
2023-09-07 9:02 ` Thomas Huth
1 sibling, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-04 19:43 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Fri, 2023-09-01 at 17:58 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Polarization is changed on a request from the guest.
> Let's verify the polarization is accordingly set by QEMU.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests
2023-09-01 15:58 ` [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests Nina Schoetterl-Glausch
@ 2023-09-04 19:51 ` Nina Schoetterl-Glausch
2023-09-07 9:05 ` Thomas Huth
1 sibling, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-04 19:51 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Fri, 2023-09-01 at 17:58 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Test changes in the entitlement from both a guest and a host point of
> view, depending on the polarization.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 55 ++++++++++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
>
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> index d7ee36cb89..733093723a 100644
> --- a/tests/avocado/s390_topology.py
> +++ b/tests/avocado/s390_topology.py
> @@ -243,3 +243,58 @@ def test_polarization(self):
> res = self.vm.qmp('query-s390x-cpu-polarization')
> self.assertEqual(res['return']['polarization'], 'horizontal')
> self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> +
> + def check_polarization(self, polarization):
> + """
> + We need the polarization change to get stabilized in kernel
> + before we check it through the sysfs.
> + """
IMO this should be a comment, not a function doc.
#We need to wait for the change to have been propagated to the kernel
> + exec_command_and_wait_for_pattern(self,
> + """ sleep 1 ;
> + /bin/cat /sys/devices/system/cpu/cpu0/polarization """,
> + polarization)
[...]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU
2023-09-01 15:58 ` [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU Nina Schoetterl-Glausch
@ 2023-09-04 19:54 ` Nina Schoetterl-Glausch
2023-09-07 10:08 ` Thomas Huth
1 sibling, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-04 19:54 UTC (permalink / raw)
To: qemu-devel, qemu-s390x, Eduardo Habkost, Marcel Apfelbaum,
Paolo Bonzini, Thomas Huth, Halil Pasic, Christian Borntraeger,
Eric Farman, Richard Henderson, David Hildenbrand, Eric Blake,
Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Fri, 2023-09-01 at 17:58 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> A dedicated CPU in vertical polarization can only have
> a high entitlement.
> Let's check this from both host and guest point of view.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 01/20] CPU topology: extend with s390 specifics
2023-09-01 15:57 ` [PATCH v22 01/20] CPU topology: extend with s390 specifics Nina Schoetterl-Glausch
@ 2023-09-05 9:01 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-05 9:01 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Paolo Bonzini, Halil Pasic, Christian Borntraeger, Eric Farman,
David Hildenbrand, Eric Blake, Markus Armbruster
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Beraldo Leal
On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> S390 CPUs have specific topology features like dedication and
> entitlement. These indicate to the guest information on host
> vCPU scheduling and help the guest make better scheduling decisions.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and entitlement,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> qapi/machine-common.json | 21 +++++++++++++
> qapi/machine.json | 21 ++++++++++---
> include/hw/boards.h | 10 +++++-
> include/hw/qdev-properties-system.h | 4 +++
> target/s390x/cpu.h | 6 ++++
> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++-----
> hw/core/machine.c | 4 +++
> hw/core/qdev-properties-system.c | 13 ++++++++
> hw/s390x/s390-virtio-ccw.c | 2 ++
> softmmu/vl.c | 6 ++++
> target/s390x/cpu.c | 7 +++++
> qapi/meson.build | 1 +
> qemu-options.hx | 7 +++--
> 13 files changed, 136 insertions(+), 14 deletions(-)
> create mode 100644 qapi/machine-common.json
>
> diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> new file mode 100644
> index 0000000000..e40421bb37
> --- /dev/null
> +++ b/qapi/machine-common.json
> @@ -0,0 +1,21 @@
> +# -*- Mode: Python -*-
> +# vim: filetype=python
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = Machines S390 data types
> +##
> +
> +##
> +# @CpuS390Entitlement:
> +#
> +# An enumeration of cpu entitlements that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.2
> +##
> +{ 'enum': 'CpuS390Entitlement',
> + 'prefix': 'S390_CPU_ENTITLEMENT',
> + 'data': [ 'auto', 'low', 'medium', 'high' ] }
> diff --git a/qapi/machine.json b/qapi/machine.json
> index a08b6576ca..54f99f4ac1 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -9,6 +9,7 @@
> ##
>
> { 'include': 'common.json' }
> +{ 'include': 'machine-common.json' }
>
> ##
> # @SysEmuTarget:
> @@ -71,8 +72,8 @@
> #
> # @thread-id: ID of the underlying host thread
> #
> -# @props: properties describing to which node/socket/core/thread
> -# virtual CPU belongs to, provided if supported by board
> +# @props: properties describing to which node/drawer/book/socket/core/thread
> +# virtual CPU belongs to, provided if supported by board
The QAPI docs recently switched to 4 spaces indentation, see commit
08349786c8430 ... so please do not re-indent the second line here.
> #
> # @target: the QEMU system emulation target, which determines which
> # additional fields will be listed (since 3.0)
> @@ -901,7 +902,11 @@
> #
> # @node-id: NUMA node ID the CPU belongs to
> #
> -# @socket-id: socket number within node/board the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to (since 8.2)
> +#
> +# @book-id: book number within drawer/node/board the CPU belongs to (since 8.2)
> +#
> +# @socket-id: socket number within book/node/board the CPU belongs to
> #
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> #
> @@ -912,7 +917,7 @@
> #
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present but
> +# Note: currently there are 8 properties that could be present but
> # management should be prepared to pass through other properties
> # with device_add command to allow for future interface extension.
> # This also requires the filed names to be kept in sync with the
> @@ -922,6 +927,8 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> + '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> '*cluster-id': 'int',
> @@ -1480,6 +1487,10 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @drawers: number of drawers in the CPU topology (since 8.2)
> +#
> +# @books: number of books in the CPU topology (since 8.2)
> +#
> # @sockets: number of sockets in the CPU topology
> #
> # @dies: number of dies per socket in the CPU topology
> @@ -1498,6 +1509,8 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*drawers': 'int',
> + '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> '*clusters': 'int',
...
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index 0f4d9b6f7a..25019c91ee 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -33,6 +33,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, "drawers (%u) * ", ms->smp.drawers);
> + }
> +
> + if (mc->smp_props.books_supported) {
> + g_string_append_printf(s, "books (%u) * ", ms->smp.books);
> + }
> +
> g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>
> if (mc->smp_props.dies_supported) {
> @@ -75,6 +83,8 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> + unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> unsigned clusters = config->has_clusters ? config->clusters : 0;
> @@ -87,6 +97,8 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> + (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> (config->has_clusters && config->clusters == 0) ||
> @@ -113,6 +125,19 @@ void machine_parse_smp_config(MachineState *ms,
> dies = dies > 0 ? dies : 1;
> clusters = clusters > 0 ? clusters : 1;
>
> + if (!mc->smp_props.books_supported && books > 1) {
> + error_setg(errp, "books not supported by this machine's CPU topology");
> + return;
> + }
> + books = books > 0 ? books : 1;
> +
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp,
> + "drawers not supported by this machine's CPU topology");
> + return;
> + }
> + drawers = drawers > 0 ? drawers : 1;
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -126,33 +151,41 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
Assuming that we forbid the drawers and books for older machine types, I
think you could drop the above two changes. (But I also don't mind if you
keep them)
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> + clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> + ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> ms->smp.clusters = clusters;
> @@ -163,7 +196,8 @@ void machine_parse_smp_config(MachineState *ms,
> mc->smp_props.has_clusters = config->has_clusters;
>
> /* sanity-check of the computed topology */
> - if (sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads !=
> + maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index da699cf4e1..363ddafc89 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -843,6 +843,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> + .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> .has_clusters = true, .clusters = ms->smp.clusters,
> @@ -1108,6 +1110,8 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> + ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> ms->smp.clusters = 1;
...
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index bfcf64d007..3b34aa0e35 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -733,6 +733,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> mc->no_sdcard = 1;
> mc->max_cpus = S390_MAX_CPUS;
> mc->has_hotpluggable_cpus = true;
> + mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> assert(!mc->get_hotplug_handler);
> mc->get_hotplug_handler = s390_get_hotplug_handler;
> mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
I think we should forbid the books and drawers in older machine types to
avoid problems with migrating such guests between older and newer QEMU versions.
So please add books_supported = false and drawers_supported = false to the
ccw_machine_8_1_class_options() function.
Thomas
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug
2023-09-01 15:57 ` [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug Nina Schoetterl-Glausch
@ 2023-09-05 12:28 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-05 12:28 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> The topology information are attributes of the CPU and are
> specified during the CPU device creation.
>
> On hot plug we:
> - calculate the default values for the topology for drawers,
> books and sockets in the case they are not specified.
> - verify the CPU attributes
> - check that we have still room on the desired socket
>
> The possibility to insert a CPU in a mask is dependent on the
> number of cores allowed in a socket, a book or a drawer, the
> checking is done during the hot plug of the CPU to have an
> immediate answer.
>
> If the complete topology is not specified, the core is added
> in the physical topology based on its core ID and it gets
> defaults values for the modifier attributes.
>
> This way, starting QEMU without specifying the topology can
> still get some advantage of the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
...
> +/**
> + * s390_topology_cpu_default:
> + * @cpu: pointer to a S390CPU
> + * @errp: Error pointer
> + *
> + * Setup the default topology if no attributes are already set.
> + * Passing a CPU with some, but not all, attributes set is considered
> + * an error.
> + *
> + * The function calculates the (drawer_id, book_id, socket_id)
> + * topology by filling the cores starting from the first socket
> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
> + *
> + * CPU type and dedication have defaults values set in the
> + * s390x_cpu_properties, entitlement must be adjust depending on the
> + * dedication.
> + *
> + * Returns false if it is impossible to setup a default topology
> + * true otherwise.
> + */
> +static bool s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> +{
> + CpuTopology *smp = ¤t_machine->smp;
> + CPUS390XState *env = &cpu->env;
> +
> + /* All geometry topology attributes must be set or all unset */
> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
> + error_setg(errp,
> + "Please define all or none of the topology geometry attributes");
> + return false;
> + }
> +
> + /* Check if one of the geometry topology is unset */
The comment isn't too helpful - maybe rather something like this:
"If one value of the topology is still unset, this means that now that all
of them are unset, so calculate the default attributes now." (and then
remove also the comment within the curly braces below)
> + if (env->socket_id < 0) {
> + /* Calculate default geometry topology attributes */
> + env->socket_id = s390_std_socket(env->core_id, smp);
> + env->book_id = s390_std_book(env->core_id, smp);
> + env->drawer_id = s390_std_drawer(env->core_id, smp);
> + }
> +
> + /*
> + * When the user specifies the entitlement as 'auto' on the command line,
> + * QEMU will set the entitlement as:
> + * Medium when the CPU is not dedicated.
> + * High when dedicated is true.
> + */
> + if (env->entitlement == S390_CPU_ENTITLEMENT_AUTO) {
> + if (env->dedicated) {
> + env->entitlement = S390_CPU_ENTITLEMENT_HIGH;
> + } else {
> + env->entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
> + }
> + }
> + return true;
> +}
> +
> +/**
> + * s390_topology_check:
> + * @socket_id: socket to check
> + * @book_id: book to check
> + * @drawer_id: drawer to check
> + * @entitlement: entitlement to check
> + * @dedicated: dedication to check
> + * @errp: Error pointer
> + *
> + * The function checks if the topology
> + * attributes fits inside the system topology.
> + *
> + * Returns false if the specified topology does not match with
> + * the machine topology.
> + */
> +static bool s390_topology_check(uint16_t socket_id, uint16_t book_id,
> + uint16_t drawer_id, uint16_t entitlement,
> + bool dedicated, Error **errp)
> +{
> + CpuTopology *smp = ¤t_machine->smp;
> + ERRP_GUARD();
No need for ERRP_GUARD() here, I think.
> + if (socket_id >= smp->sockets) {
> + error_setg(errp, "Unavailable socket: %d", socket_id);
> + return false;
> + }
> + if (book_id >= smp->books) {
> + error_setg(errp, "Unavailable book: %d", book_id);
> + return false;
> + }
> + if (drawer_id >= smp->drawers) {
> + error_setg(errp, "Unavailable drawer: %d", drawer_id);
> + return false;
> + }
> + if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
> + error_setg(errp, "Unknown entitlement: %d", entitlement);
> + return false;
> + }
> + if (dedicated && (entitlement == S390_CPU_ENTITLEMENT_LOW ||
> + entitlement == S390_CPU_ENTITLEMENT_MEDIUM)) {
> + error_setg(errp, "A dedicated CPU implies high entitlement");
> + return false;
> + }
> + return true;
> +}
> +
> +/**
> + * s390_update_cpu_props:
> + * @ms: the machine state
> + * @cpu: the CPU for which to update the properties from the environment.
> + *
> + */
> +static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
> +{
> + CpuInstanceProperties *props;
> +
> + props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
> +
> + props->socket_id = cpu->env.socket_id;
> + props->book_id = cpu->env.book_id;
> + props->drawer_id = cpu->env.drawer_id;
> +}
> +
> +/**
> + * s390_topology_setup_cpu:
> + * @ms: MachineState used to initialize the topology structure on
> + * first call.
> + * @cpu: the new S390CPU to insert in the topology structure
> + * @errp: the error pointer
> + *
> + * Called from CPU hotplug to check and setup the CPU attributes
> + * before the CPU is inserted in the topology.
> + * There is no need to update the MTCR explicitly here because it
> + * will be updated by KVM on creation of the new CPU.
> + */
> +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> +{
> + ERRP_GUARD();
I think ERRP_GUARD is also not necessary here?
> + int entry;
> +
> + /*
> + * We do not want to initialize the topology if the CPU model
> + * does not support topology, consequently, we have to wait for
> + * the first CPU to be realized, which realizes the CPU model
> + * to initialize the topology structures.
> + *
> + * s390_topology_setup_cpu() is called from the CPU hotplug.
> + */
> + if (!s390_topology.cores_per_socket) {
> + s390_topology_init(ms);
> + }
> +
> + if (!s390_topology_cpu_default(cpu, errp)) {
> + return;
> + }
> +
> + if (!s390_topology_check(cpu->env.socket_id, cpu->env.book_id,
> + cpu->env.drawer_id, cpu->env.entitlement,
> + cpu->env.dedicated, errp)) {
> + return;
> + }
> +
> + /* Do we still have space in the socket */
> + entry = s390_socket_nb(cpu);
> + if (s390_topology.cores_per_socket[entry] >= current_machine->smp.cores) {
> + error_setg(errp, "No more space on this socket");
> + return;
> + }
> +
> + /* Update the count of cores in sockets */
> + s390_topology.cores_per_socket[entry] += 1;
> +
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +}
Anyway, with or without the nits fixed, this looks fine to me, so:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
2023-09-04 18:23 ` Nina Schoetterl-Glausch
@ 2023-09-05 13:26 ` Thomas Huth
2023-09-05 15:25 ` Nina Schoetterl-Glausch
2023-09-06 14:53 ` Cédric Le Goater
2 siblings, 1 reply; 44+ messages in thread
From: Thomas Huth @ 2023-09-05 13:26 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> qapi/machine-target.json | 14 ++
> include/hw/s390x/cpu-topology.h | 25 +++
> include/hw/s390x/sclp.h | 1 +
> target/s390x/cpu.h | 76 ++++++++
> hw/s390x/cpu-topology.c | 2 +
> target/s390x/kvm/kvm.c | 5 +-
> target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++++++++++++++++
> target/s390x/kvm/meson.build | 3 +-
> 9 files changed, 421 insertions(+), 2 deletions(-)
> create mode 100644 target/s390x/kvm/stsi-topology.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b10b83583f..692ce9f121 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1700,6 +1700,7 @@ M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> S: Supported
> F: include/hw/s390x/cpu-topology.h
> F: hw/s390x/cpu-topology.c
> +F: target/s390x/kvm/stsi-topology.c
>
> X86 Machines
> ------------
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index f0a6b72414..275234a20f 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -361,3 +361,17 @@
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64',
> 'TARGET_RISCV' ] } }
> +
> +##
> +# @CpuS390Polarization:
> +#
> +# An enumeration of cpu polarization that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.2
> +##
> +{ 'enum': 'CpuS390Polarization',
> + 'prefix': 'S390_CPU_POLARIZATION',
> + 'data': [ 'horizontal', 'vertical' ],
> + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
> +}
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> index 97b0af2795..fc15acf297 100644
> --- a/include/hw/s390x/cpu-topology.h
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -15,10 +15,35 @@
> #include "hw/boards.h"
> #include "qapi/qapi-types-machine-target.h"
>
> +#define S390_TOPOLOGY_CPU_IFL 0x03
> +
> +typedef union s390_topology_id {
> + uint64_t id;
> + struct {
> + uint8_t _reserved0;
> + uint8_t drawer;
> + uint8_t book;
> + uint8_t socket;
> + uint8_t type;
> + uint8_t inv_polarization;
What sense does it make to store the polarization in an inverted way? ... I
don't get that ... could you please at least add a comment somewhere for the
rationale?
> + uint8_t not_dedicated;
> + uint8_t origin;
> + };
> +} s390_topology_id;
> +
> +typedef struct S390TopologyEntry {
> + QTAILQ_ENTRY(S390TopologyEntry) next;
> + s390_topology_id id;
> + uint64_t mask;
> +} S390TopologyEntry;
> +
> typedef struct S390Topology {
> uint8_t *cores_per_socket;
> + CpuS390Polarization polarization;
> } S390Topology;
>
> +typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
> +
> #ifdef CONFIG_KVM
> bool s390_has_topology(void);
> void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
> diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
> index cf1f2efae2..c49051e17e 100644
> --- a/include/hw/s390x/sclp.h
> +++ b/include/hw/s390x/sclp.h
> @@ -112,6 +112,7 @@ typedef struct CPUEntry {
> } QEMU_PACKED CPUEntry;
>
> #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
> +#define SCLP_READ_SCP_INFO_MNEST 2
> typedef struct ReadInfo {
> SCCBHeader h;
> uint16_t rnmax;
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index 7ebd5e05b6..b8a0c02714 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -569,6 +569,29 @@ typedef struct SysIB_322 {
> } SysIB_322;
> QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
>
> +/*
> + * Topology Magnitude fields (MAG) indicates the maximum number of
> + * topology list entries (TLE) at the corresponding nesting level.
> + */
> +#define S390_TOPOLOGY_MAG 6
> +#define S390_TOPOLOGY_MAG6 0
> +#define S390_TOPOLOGY_MAG5 1
> +#define S390_TOPOLOGY_MAG4 2
> +#define S390_TOPOLOGY_MAG3 3
> +#define S390_TOPOLOGY_MAG2 4
> +#define S390_TOPOLOGY_MAG1 5
> +/* Configuration topology */
> +typedef struct SysIB_151x {
> + uint8_t reserved0[2];
> + uint16_t length;
> + uint8_t mag[S390_TOPOLOGY_MAG];
> + uint8_t reserved1;
> + uint8_t mnest;
> + uint32_t reserved2;
> + char tle[];
> +} SysIB_151x;
> +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
> +
> typedef union SysIB {
> SysIB_111 sysib_111;
> SysIB_121 sysib_121;
> @@ -576,9 +599,62 @@ typedef union SysIB {
> SysIB_221 sysib_221;
> SysIB_222 sysib_222;
> SysIB_322 sysib_322;
> + SysIB_151x sysib_151x;
> } SysIB;
> QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
>
> +/*
> + * CPU Topology List provided by STSI with fc=15 provides a list
> + * of two different Topology List Entries (TLE) types to specify
> + * the topology hierarchy.
> + *
> + * - Container Topology List Entry
> + * Defines a container to contain other Topology List Entries
> + * of any type, nested containers or CPU.
> + * - CPU Topology List Entry
> + * Specifies the CPUs position, type, entitlement and polarization
> + * of the CPUs contained in the last Container TLE.
> + *
> + * There can be theoretically up to five levels of containers, QEMU
> + * uses only three levels, the drawer's, book's and socket's level.
> + *
> + * A container with a nesting level (NL) greater than 1 can only
> + * contain another container of nesting level NL-1.
> + *
> + * A container of nesting level 1 (socket), contains as many CPU TLE
> + * as needed to describe the position and qualities of all CPUs inside
> + * the container.
> + * The qualities of a CPU are polarization, entitlement and type.
> + *
> + * The CPU TLE defines the position of the CPUs of identical qualities
> + * using a 64bits mask which first bit has its offset defined by
> + * the CPU address orgin field of the CPU TLE like in:
> + * CPU address = origin * 64 + bit position within the mask
> + *
Remove the empty line at the end?
> + */
> +/* Container type Topology List Entry */
> +typedef struct SYSIBContainerListEntry {
> + uint8_t nl;
> + uint8_t reserved[6];
> + uint8_t id;
> +} SYSIBContainerListEntry;
> +QEMU_BUILD_BUG_ON(sizeof(SYSIBContainerListEntry) != 8);
> +
> +/* CPU type Topology List Entry */
> +typedef struct SysIBCPUListEntry {
> + uint8_t nl;
> + uint8_t reserved0[3];
> +#define SYSIB_TLE_POLARITY_MASK 0x03
> +#define SYSIB_TLE_DEDICATED 0x04
> + uint8_t flags;
> + uint8_t type;
> + uint16_t origin;
> + uint64_t mask;
> +} SysIBCPUListEntry;
> +QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
> +
> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
> +
> /* MMU defines */
> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
> #define ASCE_SUBSPACE 0x200 /* subspace group control */
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 06b60ebce4..5971804680 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -28,10 +28,12 @@
> * s390_topology is used to keep the topology information.
> * .cores_per_socket: tracks information on the count of cores
> * per socket.
> + * .polarization: tracks machine polarization.
> */
> S390Topology s390_topology = {
> /* will be initialized after the CPU model is realized */
> .cores_per_socket = NULL,
> + .polarization = S390_CPU_POLARIZATION_HORIZONTAL,
> };
>
> /**
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index 852fbd0df7..56b31b8aae 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -1911,9 +1911,12 @@ static int handle_stsi(S390CPU *cpu)
> if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
> return 0;
> }
> - /* Only sysib 3.2.2 needs post-handling for now. */
> insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
> return 0;
> + case 15:
> + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
> + run->s390_stsi.ar, RA_IGNORED);
> + return 0;
> default:
> return 0;
> }
> diff --git a/target/s390x/kvm/stsi-topology.c b/target/s390x/kvm/stsi-topology.c
> new file mode 100644
> index 0000000000..cb78040ea5
> --- /dev/null
> +++ b/target/s390x/kvm/stsi-topology.c
> @@ -0,0 +1,296 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * QEMU S390x CPU Topology
> + *
> + * Copyright IBM Corp. 2022, 2023
> + * Author(s): Pierre Morel <pmorel@linux.ibm.com>
> + *
> + */
> +#include "qemu/osdep.h"
> +#include "cpu.h"
> +#include "hw/s390x/sclp.h"
> +#include "hw/s390x/cpu-topology.h"
> +
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_LOW != 1);
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_MEDIUM != 2);
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_HIGH != 3);
> +
> +/**
> + * fill_container:
> + * @p: The address of the container TLE to fill
> + * @level: The level of nesting for this container
> + * @id: The container receives a unique ID inside its own container
> + *
> + * Returns the next free TLE entry.
> + */
> +static char *fill_container(char *p, int level, int id)
> +{
> + SYSIBContainerListEntry *tle = (SYSIBContainerListEntry *)p;
> +
> + tle->nl = level;
> + tle->id = id;
> + return p + sizeof(*tle);
> +}
> +
> +/**
> + * fill_tle_cpu:
> + * @p: The address of the CPU TLE to fill
> + * @entry: a pointer to the S390TopologyEntry defining this
> + * CPU container.
> + *
> + * Returns the next free TLE entry.
> + */
> +static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
> +{
> + SysIBCPUListEntry *tle = (SysIBCPUListEntry *)p;
> + s390_topology_id topology_id = entry->id;
> +
> + tle->nl = 0;
> + tle->flags = 3 - topology_id.inv_polarization;
Can you avoid the magic number 3 here?
Also, you seem to simply revert the "3 - entitlement" from
s390_topology_from_cpu() ... so that raises again the question: Why doing
the inversion at all?
> + if (!topology_id.not_dedicated) {
> + tle->flags |= SYSIB_TLE_DEDICATED;
> + }
> + tle->type = topology_id.type;
> + tle->origin = cpu_to_be16(topology_id.origin * 64);
> + tle->mask = cpu_to_be64(entry->mask);
> + return p + sizeof(*tle);
> +}
> +
> +/*
> + * Macro to check that the size of data after increment
> + * will not get bigger than the size of the SysIB.
> + */
> +#define SYSIB_GUARD(data, x) do { \
> + data += x; \
> + if (data > sizeof(SysIB)) { \
> + return 0; \
> + } \
> + } while (0)
> +
> +/**
> + * stsi_topology_fill_sysib:
> + * @p: A pointer to the position of the first TLE
> + * @level: The nested level wanted by the guest
> + *
> + * Fill the SYSIB with the topology information as described in
> + * the PoP, nesting containers as appropriate, with the maximum
> + * nesting limited by @level.
> + *
> + * Return value:
> + * On success: the size of the SysIB_15x after being filled with TLE.
> + * On error: 0 in the case we would overrun the end of the SysIB.
> + */
> +static int stsi_topology_fill_sysib(S390TopologyList *topology_list,
> + char *p, int level)
> +{
> + S390TopologyEntry *entry;
> + int last_drawer = -1;
> + int last_book = -1;
> + int last_socket = -1;
> + int drawer_id = 0;
> + int book_id = 0;
> + int socket_id = 0;
> + int n = sizeof(SysIB_151x);
> +
> + QTAILQ_FOREACH(entry, topology_list, next) {
> + bool drawer_change = last_drawer != entry->id.drawer;
> + bool book_change = drawer_change || last_book != entry->id.book;
> + bool socket_change = book_change || last_socket != entry->id.socket;
> +
> + if (level > 3 && drawer_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 3, drawer_id++);
> + book_id = 0;
> + }
> + if (level > 2 && book_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 2, book_id++);
> + socket_id = 0;
> + }
> + if (socket_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 1, socket_id++);
> + }
> +
> + SYSIB_GUARD(n, sizeof(SysIBCPUListEntry));
> + p = fill_tle_cpu(p, entry);
> + last_drawer = entry->id.drawer;
> + last_book = entry->id.book;
> + last_socket = entry->id.socket;
> + }
> +
> + return n;
> +}
> +
> +/**
> + * setup_stsi:
> + * sysib: pointer to a SysIB to be filled with SysIB_151x data
> + * level: Nested level specified by the guest
> + *
> + * Setup the SYSIB for STSI 15.1, the header as well as the description
> + * of the topology.
> + */
> +static int setup_stsi(S390TopologyList *topology_list, SysIB_151x *sysib,
> + int level)
> +{
> + sysib->mnest = level;
> + switch (level) {
> + case 4:
> + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + case 3:
> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
> + current_machine->smp.books;
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + case 2:
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
> + current_machine->smp.books *
> + current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + }
> +
> + return stsi_topology_fill_sysib(topology_list, sysib->tle, level);
> +}
> +
> +/**
> + * s390_topology_add_cpu_to_entry:
> + * @entry: Topology entry to setup
> + * @cpu: the S390CPU to add
> + *
> + * Set the core bit inside the topology mask.
> + */
> +static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
> + S390CPU *cpu)
> +{
> + set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
> +}
> +
> +/**
> + * s390_topology_from_cpu:
> + * @cpu: S390CPU to calculate the topology id
> + *
> + * Initialize the topology id from the CPU environment.
> + */
> +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
> +{
> + s390_topology_id topology_id = {0};
> +
> + topology_id.drawer = cpu->env.drawer_id;
> + topology_id.book = cpu->env.book_id;
> + topology_id.socket = cpu->env.socket_id;
> + topology_id.type = S390_TOPOLOGY_CPU_IFL;
> + topology_id.not_dedicated = !cpu->env.dedicated;
> +
> + topology_id.inv_polarization = 3;
Magic number 3 again.
> + if (s390_topology.polarization == S390_CPU_POLARIZATION_VERTICAL) {
> + topology_id.inv_polarization -= cpu->env.entitlement;
> + }
> +
> + topology_id.origin = cpu->env.core_id / 64;
> +
> + return topology_id;
> +}
> +
> +/**
> + * s390_topology_fill_list_sorted:
> + *
> + * Loop over all CPU and insert it at the right place
> + * inside the TLE entry list.
> + * Fill the S390Topology list with entries according to the order
> + * specified by the PoP.
> + */
> +static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
> +{
> + CPUState *cs;
> + S390TopologyEntry sentinel;
> +
> + QTAILQ_INIT(topology_list);
> +
> + sentinel.id.id = cpu_to_be64(UINT64_MAX);
> + QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
> +
> + CPU_FOREACH(cs) {
> + s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
> + S390TopologyEntry *entry, *tmp;
> +
> + QTAILQ_FOREACH(tmp, topology_list, next) {
> + if (id.id == tmp->id.id) {
> + entry = tmp;
> + break;
> + } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
> + entry = g_malloc0(sizeof(*entry));
Maybe nicer to use g_new0 here instead?
> + entry->id.id = id.id;
Should this get a cpu_to_be64() ?
> + QTAILQ_INSERT_BEFORE(tmp, entry, next);
> + break;
> + }
> + }
> + s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
> + }
> +
> + QTAILQ_REMOVE(topology_list, &sentinel, next);
> +}
Thomas
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-05 13:26 ` Thomas Huth
@ 2023-09-05 15:25 ` Nina Schoetterl-Glausch
2023-09-06 8:21 ` Thomas Huth
0 siblings, 1 reply; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-05 15:25 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Tue, 2023-09-05 at 15:26 +0200, Thomas Huth wrote:
> On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
> > From: Pierre Morel <pmorel@linux.ibm.com>
> >
> > On interception of STSI(15.1.x) the System Information Block
> > (SYSIB) is built from the list of pre-ordered topology entries.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > ---
> > MAINTAINERS | 1 +
> > qapi/machine-target.json | 14 ++
> > include/hw/s390x/cpu-topology.h | 25 +++
> > include/hw/s390x/sclp.h | 1 +
> > target/s390x/cpu.h | 76 ++++++++
> > hw/s390x/cpu-topology.c | 2 +
> > target/s390x/kvm/kvm.c | 5 +-
> > target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++++++++++++++++
> > target/s390x/kvm/meson.build | 3 +-
> > 9 files changed, 421 insertions(+), 2 deletions(-)
> > create mode 100644 target/s390x/kvm/stsi-topology.c
[...]
> > diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> > index 97b0af2795..fc15acf297 100644
> > --- a/include/hw/s390x/cpu-topology.h
> > +++ b/include/hw/s390x/cpu-topology.h
> > @@ -15,10 +15,35 @@
> > #include "hw/boards.h"
> > #include "qapi/qapi-types-machine-target.h"
> >
> > +#define S390_TOPOLOGY_CPU_IFL 0x03
> > +
> > +typedef union s390_topology_id {
> > + uint64_t id;
> > + struct {
> > + uint8_t _reserved0;
> > + uint8_t drawer;
> > + uint8_t book;
> > + uint8_t socket;
> > + uint8_t type;
> > + uint8_t inv_polarization;
>
> What sense does it make to store the polarization in an inverted way? ... I
> don't get that ... could you please at least add a comment somewhere for the
> rationale?
>
It inverts the ordering with regards to polarization, as required by
the PoP. The dedication is inverted for the same reason, dedicated
CPUs show up before non dedicated ones, so the id must have a lower
value.
I will add a comment.
> > + uint8_t not_dedicated;
> > + uint8_t origin;
> > + };
> > +} s390_topology_id;
[...]
> > + * fill_tle_cpu:
> > + * @p: The address of the CPU TLE to fill
> > + * @entry: a pointer to the S390TopologyEntry defining this
> > + * CPU container.
> > + *
> > + * Returns the next free TLE entry.
> > + */
> > +static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
> > +{
> > + SysIBCPUListEntry *tle = (SysIBCPUListEntry *)p;
> > + s390_topology_id topology_id = entry->id;
> > +
> > + tle->nl = 0;
> > + tle->flags = 3 - topology_id.inv_polarization;
>
> Can you avoid the magic number 3 here?
Hmm, any number larger than 2 will do.
I could also use a int8_t and just negate, but relying on the
reinterpretation of two's complement is also magical.
I guess S390_CPU_ENTITLEMENT_HIGH makes the most sense.
[...]
> > +/**
> > + * s390_topology_fill_list_sorted:
> > + *
> > + * Loop over all CPU and insert it at the right place
> > + * inside the TLE entry list.
> > + * Fill the S390Topology list with entries according to the order
> > + * specified by the PoP.
> > + */
> > +static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
> > +{
> > + CPUState *cs;
> > + S390TopologyEntry sentinel;
> > +
> > + QTAILQ_INIT(topology_list);
> > +
> > + sentinel.id.id = cpu_to_be64(UINT64_MAX);
> > + QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
> > +
> > + CPU_FOREACH(cs) {
> > + s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
> > + S390TopologyEntry *entry, *tmp;
> > +
> > + QTAILQ_FOREACH(tmp, topology_list, next) {
> > + if (id.id == tmp->id.id) {
> > + entry = tmp;
> > + break;
I think I'll add a comment here.
/*
* Earlier bytes have higher order -> big endian.
* E.g. an entry with higher drawer number should be later in the list,
* no matter the later fields (book, socket, etc)
*/
> > + } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
> > + entry = g_malloc0(sizeof(*entry));
>
> Maybe nicer to use g_new0 here instead?
I don't think it makes much of a difference.
>
> > + entry->id.id = id.id;
>
> Should this get a cpu_to_be64() ?
No, there is no interpretation of the value here, just a copy.
>
> > + QTAILQ_INSERT_BEFORE(tmp, entry, next);
> > + break;
> > + }
> > + }
> > + s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
> > + }
> > +
> > + QTAILQ_REMOVE(topology_list, &sentinel, next);
> > +}
>
> Thomas
>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-05 15:25 ` Nina Schoetterl-Glausch
@ 2023-09-06 8:21 ` Thomas Huth
2023-09-06 12:12 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 44+ messages in thread
From: Thomas Huth @ 2023-09-06 8:21 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 05/09/2023 17.25, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-09-05 at 15:26 +0200, Thomas Huth wrote:
>> On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
>>> From: Pierre Morel <pmorel@linux.ibm.com>
>>>
>>> On interception of STSI(15.1.x) the System Information Block
>>> (SYSIB) is built from the list of pre-ordered topology entries.
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
>>> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
>>> ---
...
>>> +/**
>>> + * s390_topology_fill_list_sorted:
>>> + *
>>> + * Loop over all CPU and insert it at the right place
>>> + * inside the TLE entry list.
>>> + * Fill the S390Topology list with entries according to the order
>>> + * specified by the PoP.
>>> + */
>>> +static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
>>> +{
>>> + CPUState *cs;
>>> + S390TopologyEntry sentinel;
>>> +
>>> + QTAILQ_INIT(topology_list);
>>> +
>>> + sentinel.id.id = cpu_to_be64(UINT64_MAX);
Since you don't do swapping for entry->id.id below, why do you do it here?
>>> + QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
>>> +
>>> + CPU_FOREACH(cs) {
>>> + s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
>>> + S390TopologyEntry *entry, *tmp;
>>> +
>>> + QTAILQ_FOREACH(tmp, topology_list, next) {
>>> + if (id.id == tmp->id.id) {
>>> + entry = tmp;
>>> + break;
>
> I think I'll add a comment here.
>
> /*
> * Earlier bytes have higher order -> big endian.
> * E.g. an entry with higher drawer number should be later in the list,
> * no matter the later fields (book, socket, etc)
> */
Ugh, so this swapping is not due to real endianness issues, but just due to
ordering? ... that's very ugly! I'd prefer to be more verbose and compare
book by book, drawer by drawer, etc. instread. Or is this function that
performance critical that we must save every possible CPU cycle here?
Thomas
>
>>> + } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
>>> + entry = g_malloc0(sizeof(*entry));
>>
>> Maybe nicer to use g_new0 here instead?
>
> I don't think it makes much of a difference.
>
>>
>>> + entry->id.id = id.id;
>>
>> Should this get a cpu_to_be64() ?
>
> No, there is no interpretation of the value here, just a copy.
>>
>>> + QTAILQ_INSERT_BEFORE(tmp, entry, next);
>>> + break;
>>> + }
>>> + }
>>> + s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
>>> + }
>>> +
>>> + QTAILQ_REMOVE(topology_list, &sentinel, next);
>>> +}
>>
>> Thomas
>>
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
2023-09-01 15:58 ` [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command Nina Schoetterl-Glausch
@ 2023-09-06 8:56 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-06 8:56 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> The modification of the CPU attributes are done through a monitor
> command.
>
> It allows to move the core inside the topology tree to optimize
> the cache usage in the case the host's hypervisor previously
> moved the CPU.
>
> The same command allows to modify the CPU attributes modifiers
> like polarization entitlement and the dedicated attribute to notify
> the guest if the host admin modified scheduling or dedication of a vCPU.
>
> With this knowledge the guest has the possibility to optimize the
> usage of the vCPUs.
>
> The command has a feature unstable for the moment.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
...
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 52bfbe12a9..c828538f2a 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -23,6 +23,7 @@
> #include "target/s390x/cpu.h"
> #include "hw/s390x/s390-virtio-ccw.h"
> #include "hw/s390x/cpu-topology.h"
> +#include "qapi/qapi-commands-machine-target.h"
>
> /*
> * s390_topology is used to keep the topology information.
> @@ -259,6 +260,29 @@ static bool s390_topology_check(uint16_t socket_id, uint16_t book_id,
> return true;
> }
>
> +/**
> + * s390_topology_need_report
> + * @cpu: Current cpu
> + * @drawer_id: future drawer ID
> + * @book_id: future book ID
> + * @socket_id: future socket ID
> + * @entitlement: future entitlement
> + * @dedicated: future dedicated
> + *
> + * A modified topology change report is needed if the topology
> + * tree or the topology attributes change.
> + */
> +static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
> + int book_id, int socket_id,
> + uint16_t entitlement, bool dedicated)
Nit: Indentation should match the "(" ?
> +{
> + return cpu->env.drawer_id != drawer_id ||
> + cpu->env.book_id != book_id ||
> + cpu->env.socket_id != socket_id ||
> + cpu->env.entitlement != entitlement ||
> + cpu->env.dedicated != dedicated;
> +}
> +
> /**
> * s390_update_cpu_props:
> * @ms: the machine state
> @@ -328,3 +352,114 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> /* topology tree is reflected in props */
> s390_update_cpu_props(ms, cpu);
> }
> +
> +static void s390_change_topology(uint16_t core_id,
> + bool has_socket_id, uint16_t socket_id,
> + bool has_book_id, uint16_t book_id,
> + bool has_drawer_id, uint16_t drawer_id,
> + bool has_entitlement,
> + CpuS390Entitlement entitlement,
> + bool has_dedicated, bool dedicated,
> + Error **errp)
> +{
> + MachineState *ms = current_machine;
> + int old_socket_entry;
> + int new_socket_entry;
> + bool report_needed;
> + S390CPU *cpu;
> + ERRP_GUARD();
> +
> + cpu = s390_cpu_addr2state(core_id);
> + if (!cpu) {
> + error_setg(errp, "Core-id %d does not exist!", core_id);
> + return;
> + }
> +
> + /* Get attributes not provided from cpu and verify the new topology */
> + if (!has_socket_id) {
> + socket_id = cpu->env.socket_id;
> + }
> + if (!has_book_id) {
> + book_id = cpu->env.book_id;
> + }
> + if (!has_drawer_id) {
> + drawer_id = cpu->env.drawer_id;
> + }
> + if (!has_dedicated) {
> + dedicated = cpu->env.dedicated;
> + }
> +
> + /*
> + * When the user specifies the entitlement as 'auto' on the command line,
> + * qemu will set the entitlement as:
s/qemu/QEMU/
> + * Medium when the CPU is not dedicated.
> + * High when dedicated is true.
> + */
> + if (!has_entitlement || (entitlement == S390_CPU_ENTITLEMENT_AUTO)) {
nit: Remove the parentheses around "entitlement == S390_CPU_ENTITLEMENT_AUTO".
> + if (dedicated) {
> + entitlement = S390_CPU_ENTITLEMENT_HIGH;
> + } else {
> + entitlement = S390_CPU_ENTITLEMENT_MEDIUM;
> + }
> + }
> +
> + if (!s390_topology_check(socket_id, book_id, drawer_id,
> + entitlement, dedicated, errp)) {
> + return;
> + }
> +
> + /* Check for space on new socket */
> + old_socket_entry = s390_socket_nb(cpu);
> + new_socket_entry = s390_socket_nb_from_ids(drawer_id, book_id, socket_id);
> +
> + if (new_socket_entry != old_socket_entry) {
> + if (s390_topology.cores_per_socket[new_socket_entry] >=
> + ms->smp.cores) {
> + error_setg(errp, "No more space on this socket");
> + return;
> + }
> + /* Update the count of cores in sockets */
> + s390_topology.cores_per_socket[new_socket_entry] += 1;
> + s390_topology.cores_per_socket[old_socket_entry] -= 1;
> + }
> +
> + /* Check if we will need to report the modified topology */
> + report_needed = s390_topology_need_report(cpu, drawer_id, book_id,
> + socket_id, entitlement,
> + dedicated);
> +
> + /* All checks done, report new topology into the vCPU */
> + cpu->env.drawer_id = drawer_id;
> + cpu->env.book_id = book_id;
> + cpu->env.socket_id = socket_id;
> + cpu->env.dedicated = dedicated;
> + cpu->env.entitlement = entitlement;
> +
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +
> + /* Advertise the topology change */
> + if (report_needed) {
> + s390_cpu_topology_set_changed(true);
> + }
> +}
> +
> +void qmp_set_cpu_topology(uint16_t core,
> + bool has_socket, uint16_t socket,
> + bool has_book, uint16_t book,
> + bool has_drawer, uint16_t drawer,
> + bool has_entitlement, CpuS390Entitlement entitlement,
> + bool has_dedicated, bool dedicated,
> + Error **errp)
> +{
> + ERRP_GUARD();
No need for ERRP_GUARD here, I think.
> + if (!s390_has_topology()) {
> + error_setg(errp, "This machine doesn't support topology");
> + return;
> + }
> +
> + s390_change_topology(core, has_socket, socket, has_book, book,
> + has_drawer, drawer, has_entitlement, entitlement,
> + has_dedicated, dedicated, errp);
> +}
Thomas
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast
2023-09-01 15:58 ` [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast Nina Schoetterl-Glausch
@ 2023-09-06 9:10 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-06 9:10 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> S390x provides two more topology attributes, entitlement and dedication.
>
> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus
2023-09-01 15:58 ` [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus Nina Schoetterl-Glausch
@ 2023-09-06 9:14 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-06 9:14 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> S390 topology adds books and drawers topology containers.
> Let's add these to the HMP information for hotpluggable cpus.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> hw/core/machine-hmp-cmds.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
> index c3e55ef9e9..9a4b59c6f2 100644
> --- a/hw/core/machine-hmp-cmds.c
> +++ b/hw/core/machine-hmp-cmds.c
> @@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
> if (c->has_node_id) {
> monitor_printf(mon, " node-id: \"%" PRIu64 "\"\n", c->node_id);
> }
> + if (c->has_drawer_id) {
> + monitor_printf(mon, " drawer-id: \"%" PRIu64 "\"\n", c->drawer_id);
> + }
> + if (c->has_book_id) {
> + monitor_printf(mon, " book-id: \"%" PRIu64 "\"\n", c->book_id);
> + }
> if (c->has_socket_id) {
> monitor_printf(mon, " socket-id: \"%" PRIu64 "\"\n", c->socket_id);
> }
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-06 8:21 ` Thomas Huth
@ 2023-09-06 12:12 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-06 12:12 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Wed, 2023-09-06 at 10:21 +0200, Thomas Huth wrote:
> On 05/09/2023 17.25, Nina Schoetterl-Glausch wrote:
> > On Tue, 2023-09-05 at 15:26 +0200, Thomas Huth wrote:
> > > On 01/09/2023 17.57, Nina Schoetterl-Glausch wrote:
> > > > From: Pierre Morel <pmorel@linux.ibm.com>
> > > >
> > > > On interception of STSI(15.1.x) the System Information Block
> > > > (SYSIB) is built from the list of pre-ordered topology entries.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > > > Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > > > ---
> ...
> > > > +/**
> > > > + * s390_topology_fill_list_sorted:
> > > > + *
> > > > + * Loop over all CPU and insert it at the right place
> > > > + * inside the TLE entry list.
> > > > + * Fill the S390Topology list with entries according to the order
> > > > + * specified by the PoP.
> > > > + */
> > > > +static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
> > > > +{
> > > > + CPUState *cs;
> > > > + S390TopologyEntry sentinel;
> > > > +
> > > > + QTAILQ_INIT(topology_list);
> > > > +
> > > > + sentinel.id.id = cpu_to_be64(UINT64_MAX);
>
> Since you don't do swapping for entry->id.id below, why do you do it here?
Because an integer in cpu endianess is converted to the big endian
storage format. So then there is a cpu -> big -> cpu round trip with
the comparison below and the value is the max.
Of course this is entirely cosmetic, since UINT64_MAX is all ones.
> > > > + QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
> > > > +
> > > > + CPU_FOREACH(cs) {
> > > > + s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
> > > > + S390TopologyEntry *entry, *tmp;
> > > > +
> > > > + QTAILQ_FOREACH(tmp, topology_list, next) {
> > > > + if (id.id == tmp->id.id) {
> > > > + entry = tmp;
> > > > + break;
> >
> > I think I'll add a comment here.
> >
> > /*
> > * Earlier bytes have higher order -> big endian.
> > * E.g. an entry with higher drawer number should be later in the list,
> > * no matter the later fields (book, socket, etc)
> > */
>
> Ugh, so this swapping is not due to real endianness issues, but just due to
Yeah.
> ordering? ... that's very ugly! I'd prefer to be more verbose and compare
I kinda didn't like the verbosity of it, since I then need to copy
paste the whole thing because I also need an equality check.
I considered implementing <=, then a == b as a <= b && b <= a, which
seems fine on second thought, so I'll do that.
And maybe help the compiler by putting __attribute__((pure)) on there.
> book by book, drawer by drawer, etc. instread. Or is this function that
> performance critical that we must save every possible CPU cycle here?
No.
>
> Thomas
>
>
> >
> > > > + } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
> > > > + entry = g_malloc0(sizeof(*entry));
> > >
> > > Maybe nicer to use g_new0 here instead?
> >
> > I don't think it makes much of a difference.
> >
> > >
> > > > + entry->id.id = id.id;
> > >
> > > Should this get a cpu_to_be64() ?
> >
> > No, there is no interpretation of the value here, just a copy.
> > >
> > > > + QTAILQ_INSERT_BEFORE(tmp, entry, next);
> > > > + break;
> > > > + }
> > > > + }
> > > > + s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
> > > > + }
> > > > +
> > > > + QTAILQ_REMOVE(topology_list, &sentinel, next);
> > > > +}
> > >
> > > Thomas
> > >
> > >
> >
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
2023-09-04 18:23 ` Nina Schoetterl-Glausch
2023-09-05 13:26 ` Thomas Huth
@ 2023-09-06 14:53 ` Cédric Le Goater
2 siblings, 0 replies; 44+ messages in thread
From: Cédric Le Goater @ 2023-09-06 14:53 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Thomas Huth, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
Hello Nina,
On 9/1/23 17:57, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> qapi/machine-target.json | 14 ++
> include/hw/s390x/cpu-topology.h | 25 +++
> include/hw/s390x/sclp.h | 1 +
> target/s390x/cpu.h | 76 ++++++++
> hw/s390x/cpu-topology.c | 2 +
> target/s390x/kvm/kvm.c | 5 +-
> target/s390x/kvm/stsi-topology.c | 296 +++++++++++++++++++++++++++++++
> target/s390x/kvm/meson.build | 3 +-
> 9 files changed, 421 insertions(+), 2 deletions(-)
> create mode 100644 target/s390x/kvm/stsi-topology.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b10b83583f..692ce9f121 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1700,6 +1700,7 @@ M: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> S: Supported
> F: include/hw/s390x/cpu-topology.h
> F: hw/s390x/cpu-topology.c
> +F: target/s390x/kvm/stsi-topology.c
>
> X86 Machines
> ------------
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index f0a6b72414..275234a20f 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -361,3 +361,17 @@
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64',
> 'TARGET_RISCV' ] } }
> +
> +##
> +# @CpuS390Polarization:
> +#
> +# An enumeration of cpu polarization that can be assumed by a virtual
> +# S390 CPU
> +#
> +# Since: 8.2
> +##
> +{ 'enum': 'CpuS390Polarization',
> + 'prefix': 'S390_CPU_POLARIZATION',
> + 'data': [ 'horizontal', 'vertical' ],
Since :
> + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
> +}
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> index 97b0af2795..fc15acf297 100644
> --- a/include/hw/s390x/cpu-topology.h
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -15,10 +15,35 @@
> #include "hw/boards.h"
> #include "qapi/qapi-types-machine-target.h"
>
> +#define S390_TOPOLOGY_CPU_IFL 0x03
> +
> +typedef union s390_topology_id {
> + uint64_t id;
> + struct {
> + uint8_t _reserved0;
> + uint8_t drawer;
> + uint8_t book;
> + uint8_t socket;
> + uint8_t type;
> + uint8_t inv_polarization;
> + uint8_t not_dedicated;
> + uint8_t origin;
> + };
> +} s390_topology_id;
> +
> +typedef struct S390TopologyEntry {
> + QTAILQ_ENTRY(S390TopologyEntry) next;
> + s390_topology_id id;
> + uint64_t mask;
> +} S390TopologyEntry;
> +
> typedef struct S390Topology {
> uint8_t *cores_per_socket;
> + CpuS390Polarization polarization;
This won't compile for non s390x.
> } S390Topology;
>
> +typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
> +
> #ifdef CONFIG_KVM
> bool s390_has_topology(void);
> void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
> diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
> index cf1f2efae2..c49051e17e 100644
> --- a/include/hw/s390x/sclp.h
> +++ b/include/hw/s390x/sclp.h
> @@ -112,6 +112,7 @@ typedef struct CPUEntry {
> } QEMU_PACKED CPUEntry;
>
> #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
> +#define SCLP_READ_SCP_INFO_MNEST 2
> typedef struct ReadInfo {
> SCCBHeader h;
> uint16_t rnmax;
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index 7ebd5e05b6..b8a0c02714 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -569,6 +569,29 @@ typedef struct SysIB_322 {
> } SysIB_322;
> QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
>
> +/*
> + * Topology Magnitude fields (MAG) indicates the maximum number of
> + * topology list entries (TLE) at the corresponding nesting level.
> + */
> +#define S390_TOPOLOGY_MAG 6
> +#define S390_TOPOLOGY_MAG6 0
> +#define S390_TOPOLOGY_MAG5 1
> +#define S390_TOPOLOGY_MAG4 2
> +#define S390_TOPOLOGY_MAG3 3
> +#define S390_TOPOLOGY_MAG2 4
> +#define S390_TOPOLOGY_MAG1 5
> +/* Configuration topology */
> +typedef struct SysIB_151x {
> + uint8_t reserved0[2];
> + uint16_t length;
> + uint8_t mag[S390_TOPOLOGY_MAG];
> + uint8_t reserved1;
> + uint8_t mnest;
> + uint32_t reserved2;
> + char tle[];
> +} SysIB_151x;
> +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
> +
> typedef union SysIB {
> SysIB_111 sysib_111;
> SysIB_121 sysib_121;
> @@ -576,9 +599,62 @@ typedef union SysIB {
> SysIB_221 sysib_221;
> SysIB_222 sysib_222;
> SysIB_322 sysib_322;
> + SysIB_151x sysib_151x;
> } SysIB;
> QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
>
> +/*
> + * CPU Topology List provided by STSI with fc=15 provides a list
> + * of two different Topology List Entries (TLE) types to specify
> + * the topology hierarchy.
> + *
> + * - Container Topology List Entry
> + * Defines a container to contain other Topology List Entries
> + * of any type, nested containers or CPU.
> + * - CPU Topology List Entry
> + * Specifies the CPUs position, type, entitlement and polarization
> + * of the CPUs contained in the last Container TLE.
> + *
> + * There can be theoretically up to five levels of containers, QEMU
> + * uses only three levels, the drawer's, book's and socket's level.
> + *
> + * A container with a nesting level (NL) greater than 1 can only
> + * contain another container of nesting level NL-1.
> + *
> + * A container of nesting level 1 (socket), contains as many CPU TLE
> + * as needed to describe the position and qualities of all CPUs inside
> + * the container.
> + * The qualities of a CPU are polarization, entitlement and type.
> + *
> + * The CPU TLE defines the position of the CPUs of identical qualities
> + * using a 64bits mask which first bit has its offset defined by
> + * the CPU address orgin field of the CPU TLE like in:
> + * CPU address = origin * 64 + bit position within the mask
> + *
> + */
> +/* Container type Topology List Entry */
> +typedef struct SYSIBContainerListEntry {
> + uint8_t nl;
> + uint8_t reserved[6];
> + uint8_t id;
> +} SYSIBContainerListEntry;
> +QEMU_BUILD_BUG_ON(sizeof(SYSIBContainerListEntry) != 8);
> +
> +/* CPU type Topology List Entry */
> +typedef struct SysIBCPUListEntry {
> + uint8_t nl;
> + uint8_t reserved0[3];
> +#define SYSIB_TLE_POLARITY_MASK 0x03
> +#define SYSIB_TLE_DEDICATED 0x04
> + uint8_t flags;
> + uint8_t type;
> + uint16_t origin;
> + uint64_t mask;
> +} SysIBCPUListEntry;
> +QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
> +
> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra);
> +
> /* MMU defines */
> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
> #define ASCE_SUBSPACE 0x200 /* subspace group control */
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 06b60ebce4..5971804680 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -28,10 +28,12 @@
> * s390_topology is used to keep the topology information.
> * .cores_per_socket: tracks information on the count of cores
> * per socket.
> + * .polarization: tracks machine polarization.
> */
> S390Topology s390_topology = {
> /* will be initialized after the CPU model is realized */
> .cores_per_socket = NULL,
> + .polarization = S390_CPU_POLARIZATION_HORIZONTAL,
> };
>
> /**
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index 852fbd0df7..56b31b8aae 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -1911,9 +1911,12 @@ static int handle_stsi(S390CPU *cpu)
> if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
> return 0;
> }
> - /* Only sysib 3.2.2 needs post-handling for now. */
> insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
> return 0;
> + case 15:
> + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
> + run->s390_stsi.ar, RA_IGNORED);
> + return 0;
> default:
> return 0;
> }
> diff --git a/target/s390x/kvm/stsi-topology.c b/target/s390x/kvm/stsi-topology.c
> new file mode 100644
> index 0000000000..cb78040ea5
> --- /dev/null
> +++ b/target/s390x/kvm/stsi-topology.c
> @@ -0,0 +1,296 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * QEMU S390x CPU Topology
> + *
> + * Copyright IBM Corp. 2022, 2023
> + * Author(s): Pierre Morel <pmorel@linux.ibm.com>
> + *
> + */
> +#include "qemu/osdep.h"
> +#include "cpu.h"
> +#include "hw/s390x/sclp.h"
> +#include "hw/s390x/cpu-topology.h"
> +
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_LOW != 1);
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_MEDIUM != 2);
> +QEMU_BUILD_BUG_ON(S390_CPU_ENTITLEMENT_HIGH != 3);
> +
> +/**
> + * fill_container:
> + * @p: The address of the container TLE to fill
> + * @level: The level of nesting for this container
> + * @id: The container receives a unique ID inside its own container
> + *
> + * Returns the next free TLE entry.
> + */
> +static char *fill_container(char *p, int level, int id)
> +{
> + SYSIBContainerListEntry *tle = (SYSIBContainerListEntry *)p;
> +
> + tle->nl = level;
> + tle->id = id;
> + return p + sizeof(*tle);
> +}
> +
> +/**
> + * fill_tle_cpu:
> + * @p: The address of the CPU TLE to fill
> + * @entry: a pointer to the S390TopologyEntry defining this
> + * CPU container.
> + *
> + * Returns the next free TLE entry.
> + */
> +static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
> +{
> + SysIBCPUListEntry *tle = (SysIBCPUListEntry *)p;
> + s390_topology_id topology_id = entry->id;
> +
> + tle->nl = 0;
> + tle->flags = 3 - topology_id.inv_polarization;
> + if (!topology_id.not_dedicated) {
> + tle->flags |= SYSIB_TLE_DEDICATED;
> + }
> + tle->type = topology_id.type;
> + tle->origin = cpu_to_be16(topology_id.origin * 64);
> + tle->mask = cpu_to_be64(entry->mask);
> + return p + sizeof(*tle);
> +}
> +
> +/*
> + * Macro to check that the size of data after increment
> + * will not get bigger than the size of the SysIB.
> + */
> +#define SYSIB_GUARD(data, x) do { \
> + data += x; \
> + if (data > sizeof(SysIB)) { \
> + return 0; \
> + } \
> + } while (0)
> +
> +/**
> + * stsi_topology_fill_sysib:
> + * @p: A pointer to the position of the first TLE
> + * @level: The nested level wanted by the guest
> + *
> + * Fill the SYSIB with the topology information as described in
> + * the PoP, nesting containers as appropriate, with the maximum
> + * nesting limited by @level.
> + *
> + * Return value:
> + * On success: the size of the SysIB_15x after being filled with TLE.
> + * On error: 0 in the case we would overrun the end of the SysIB.
> + */
> +static int stsi_topology_fill_sysib(S390TopologyList *topology_list,
> + char *p, int level)
> +{
> + S390TopologyEntry *entry;
> + int last_drawer = -1;
> + int last_book = -1;
> + int last_socket = -1;
> + int drawer_id = 0;
> + int book_id = 0;
> + int socket_id = 0;
> + int n = sizeof(SysIB_151x);
> +
> + QTAILQ_FOREACH(entry, topology_list, next) {
> + bool drawer_change = last_drawer != entry->id.drawer;
> + bool book_change = drawer_change || last_book != entry->id.book;
> + bool socket_change = book_change || last_socket != entry->id.socket;
> +
> + if (level > 3 && drawer_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 3, drawer_id++);
> + book_id = 0;
> + }
> + if (level > 2 && book_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 2, book_id++);
> + socket_id = 0;
> + }
> + if (socket_change) {
> + SYSIB_GUARD(n, sizeof(SYSIBContainerListEntry));
> + p = fill_container(p, 1, socket_id++);
> + }
> +
> + SYSIB_GUARD(n, sizeof(SysIBCPUListEntry));
> + p = fill_tle_cpu(p, entry);
> + last_drawer = entry->id.drawer;
> + last_book = entry->id.book;
> + last_socket = entry->id.socket;
> + }
> +
> + return n;
> +}
> +
> +/**
> + * setup_stsi:
> + * sysib: pointer to a SysIB to be filled with SysIB_151x data
> + * level: Nested level specified by the guest
> + *
> + * Setup the SYSIB for STSI 15.1, the header as well as the description
> + * of the topology.
> + */
> +static int setup_stsi(S390TopologyList *topology_list, SysIB_151x *sysib,
> + int level)
> +{
> + sysib->mnest = level;
> + switch (level) {
> + case 4:
> + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + case 3:
> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
> + current_machine->smp.books;
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + case 2:
> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
> + current_machine->smp.books *
> + current_machine->smp.sockets;
> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> + break;
> + }
> +
> + return stsi_topology_fill_sysib(topology_list, sysib->tle, level);
> +}
> +
> +/**
> + * s390_topology_add_cpu_to_entry:
> + * @entry: Topology entry to setup
> + * @cpu: the S390CPU to add
> + *
> + * Set the core bit inside the topology mask.
> + */
> +static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
> + S390CPU *cpu)
> +{
> + set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
> +}
> +
> +/**
> + * s390_topology_from_cpu:
> + * @cpu: S390CPU to calculate the topology id
> + *
> + * Initialize the topology id from the CPU environment.
> + */
> +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
> +{
> + s390_topology_id topology_id = {0};
> +
> + topology_id.drawer = cpu->env.drawer_id;
> + topology_id.book = cpu->env.book_id;
> + topology_id.socket = cpu->env.socket_id;
> + topology_id.type = S390_TOPOLOGY_CPU_IFL;
> + topology_id.not_dedicated = !cpu->env.dedicated;
> +
> + topology_id.inv_polarization = 3;
> + if (s390_topology.polarization == S390_CPU_POLARIZATION_VERTICAL) {
> + topology_id.inv_polarization -= cpu->env.entitlement;
> + }
> +
> + topology_id.origin = cpu->env.core_id / 64;
> +
> + return topology_id;
> +}
> +
> +/**
> + * s390_topology_fill_list_sorted:
> + *
> + * Loop over all CPU and insert it at the right place
> + * inside the TLE entry list.
> + * Fill the S390Topology list with entries according to the order
> + * specified by the PoP.
> + */
> +static void s390_topology_fill_list_sorted(S390TopologyList *topology_list)
> +{
> + CPUState *cs;
> + S390TopologyEntry sentinel;
> +
> + QTAILQ_INIT(topology_list);
> +
> + sentinel.id.id = cpu_to_be64(UINT64_MAX);
> + QTAILQ_INSERT_HEAD(topology_list, &sentinel, next);
> +
> + CPU_FOREACH(cs) {
> + s390_topology_id id = s390_topology_from_cpu(S390_CPU(cs));
> + S390TopologyEntry *entry, *tmp;
Please add :
S390TopologyEntry *entry = NULL, *tmp;
It fixes a compile breakage in :
../target/s390x/kvm/stsi-topology.c: In function ‘insert_stsi_15_1_x’:
../include/qemu/bitops.h:41:9: error: ‘entry’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
*p |= mask;
~~~~^~~~~~~
../target/s390x/kvm/stsi-topology.c:220:28: note: ‘entry’ was declared here
S390TopologyEntry *entry, *tmp;
^~~~~
cc1: all warnings being treated as errors
Thanks,
C.
> +
> + QTAILQ_FOREACH(tmp, topology_list, next) {
> + if (id.id == tmp->id.id) {
> + entry = tmp;
> + break;
> + } else if (be64_to_cpu(id.id) < be64_to_cpu(tmp->id.id)) {
> + entry = g_malloc0(sizeof(*entry));
> + entry->id.id = id.id;
> + QTAILQ_INSERT_BEFORE(tmp, entry, next);
> + break;
> + }
> + }
> + s390_topology_add_cpu_to_entry(entry, S390_CPU(cs));
> + }
> +
> + QTAILQ_REMOVE(topology_list, &sentinel, next);
> +}
> +
> +/**
> + * s390_topology_empty_list:
> + *
> + * Clear all entries in the S390Topology list.
> + */
> +static void s390_topology_empty_list(S390TopologyList *topology_list)
> +{
> + S390TopologyEntry *entry = NULL;
> + S390TopologyEntry *tmp = NULL;
> +
> + QTAILQ_FOREACH_SAFE(entry, topology_list, next, tmp) {
> + QTAILQ_REMOVE(topology_list, entry, next);
> + g_free(entry);
> + }
> +}
> +
> +/**
> + * insert_stsi_15_1_x:
> + * cpu: the CPU doing the call for which we set CC
> + * sel2: the selector 2, containing the nested level
> + * addr: Guest logical address of the guest SysIB
> + * ar: the access register number
> + *
> + * Emulate STSI 15.1.x, that is, perform all necessary checks and
> + * fill the SYSIB.
> + * In case the topology description is too long to fit into the SYSIB,
> + * set CC=3 and abort without writing the SYSIB.
> + */
> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar, uintptr_t ra)
> +{
> + S390TopologyList topology_list;
> + SysIB sysib = {0};
> + int length;
> +
> + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
> + setcc(cpu, 3);
> + return;
> + }
> +
> + s390_topology_fill_list_sorted(&topology_list);
> +
> + length = setup_stsi(&topology_list, &sysib.sysib_151x, sel2);
> +
> + if (!length) {
> + s390_topology_empty_list(&topology_list);
> + setcc(cpu, 3);
> + return;
> + }
> +
> + sysib.sysib_151x.length = cpu_to_be16(length);
> + if (!s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length)) {
> + setcc(cpu, 0);
> + } else {
> + s390_cpu_virt_mem_handle_exc(cpu, ra);
> + }
> +
> + s390_topology_empty_list(&topology_list);
> +}
> diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build
> index d6aca590ae..588a9aa737 100644
> --- a/target/s390x/kvm/meson.build
> +++ b/target/s390x/kvm/meson.build
> @@ -1,7 +1,8 @@
>
> s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
> 'pv.c',
> - 'kvm.c'
> + 'kvm.c',
> + 'stsi-topology.c'
> ), if_false: files(
> 'stubs.c'
> ))
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology
2023-09-01 15:58 ` [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology Nina Schoetterl-Glausch
@ 2023-09-07 8:15 ` Thomas Huth
0 siblings, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-07 8:15 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Add some basic examples for the definition of cpu topology
> in s390x.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
...
> diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
> new file mode 100644
> index 0000000000..70633f30cb
> --- /dev/null
> +++ b/docs/devel/s390-cpu-topology.rst
> @@ -0,0 +1,171 @@
> +QAPI interface for S390 CPU topology
> +====================================
> +
> +The following sections will explain the QAPI interface for S390 CPU topology
> +with the help of exemplary output.
> +For this, let's assume that QEMU has been started with the following
> +command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
> +have default values:
> +
> +.. code-block:: bash
> +
> + qemu-system-s390x \
> + -enable-kvm \
> + -cpu z14,ctop=on \
> + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
> + \
This empty line first confused me, but after reading the other examples
later, I now think it is done on purpose, right? I'd maybe rather remove it
in this example here anyway, it's not that big that you need a visual
separation here.
> + -device z14-s390x-cpu,core-id=19,entitlement=high \
> + -device z14-s390x-cpu,core-id=11,entitlement=low \
> + -device z14-s390x-cpu,core-id=112,entitlement=high \
> + ...
...
> diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
> new file mode 100644
> index 0000000000..78b98f978a
> --- /dev/null
> +++ b/docs/system/s390x/cpu-topology.rst
...
> +Examples
> +++++++++
> +
> +In the following machine we define 8 sockets with 4 cores each.
> +
> +.. code-block:: bash
> +
> + $ qemu-system-s390x -m 2G \
> + -cpu gen16b,ctop=on \
> + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
> + -device host-s390x-cpu,core-id=14 \
> +
> +A new CPUs can be plugged using the device_add hmp command as before:
> +
> +.. code-block:: bash
> +
> + (qemu) device_add gen16b-s390x-cpu,core-id=9
> +
> +The core-id defines the placement of the core in the topology by
> +starting with core 0 in socket 0 up to maxcpus.
> +
> +In the example above:
> +
> +* There are 5 CPUs provided to the guest with the ``-smp`` command line
> + They will take the core-ids 0,1,2,3,4
> + As we have 4 cores in a socket, we have 4 CPUs provided
> + to the guest in socket 0, with core-ids 0,1,2,3.
> + The last cpu, with core-id 4, will be on socket 1.
I'd suggest to write "cpu" with capital letters for consistency.
Anyway:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 14/20] tests/avocado: s390x cpu topology core
2023-09-01 15:58 ` [PATCH v22 14/20] tests/avocado: s390x cpu topology core Nina Schoetterl-Glausch
2023-09-04 18:44 ` Nina Schoetterl-Glausch
@ 2023-09-07 8:27 ` Thomas Huth
2023-09-08 14:27 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 44+ messages in thread
From: Thomas Huth @ 2023-09-07 8:27 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Introduction of the s390x cpu topology core functions and
> basic tests.
>
> We test the correlation between the command line and
> the QMP results in query-cpus-fast for various CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> MAINTAINERS | 1 +
> tests/avocado/s390_topology.py | 200 +++++++++++++++++++++++++++++++++
> 2 files changed, 201 insertions(+)
> create mode 100644 tests/avocado/s390_topology.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 7f2937b0f2..d1b8676666 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1703,6 +1703,7 @@ F: hw/s390x/cpu-topology.c
> F: target/s390x/kvm/stsi-topology.c
> F: docs/devel/s390-cpu-topology.rst
> F: docs/system/s390x/cpu-topology.rst
> +F: tests/avocado/s390_topology.py
>
> X86 Machines
> ------------
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> new file mode 100644
> index 0000000000..3a54fe2e00
> --- /dev/null
> +++ b/tests/avocado/s390_topology.py
> @@ -0,0 +1,200 @@
> +# Functional test that boots a Linux kernel and checks the console
> +#
> +# Copyright IBM Corp. 2023
> +#
> +# Author:
> +# Pierre Morel <pmorel@linux.ibm.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import os
> +import shutil
> +import time
> +
> +from avocado_qemu import QemuSystemTest
> +from avocado_qemu import exec_command
> +from avocado_qemu import exec_command_and_wait_for_pattern
> +from avocado_qemu import interrupt_interactive_console_until_pattern
> +from avocado_qemu import wait_for_console_pattern
> +from avocado.utils import process
> +from avocado.utils import archive
> +
> +
> +class S390CPUTopology(QemuSystemTest):
> + """
> + S390x CPU topology consists of 4 topology layers, from bottom to top,
> + the cores, sockets, books and drawers and 2 modifiers attributes,
> + the entitlement and the dedication.
> + See: docs/system/s390x/cpu-topology.rst.
> +
> + S390x CPU topology is setup in different ways:
> + - implicitly from the '-smp' argument by completing each topology
> + level one after the other beginning with drawer 0, book 0 and
> + socket 0.
> + - explicitly from the '-device' argument on the QEMU command line
> + - explicitly by hotplug of a new CPU using QMP or HMP
> + - it is modified by using QMP 'set-cpu-topology'
> +
> + The S390x modifier attribute entitlement depends on the machine
> + polarization, which can be horizontal or vertical.
> + The polarization is changed on a request from the guest.
> + """
> + timeout = 90
> +
> + KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
> + 'root=/dev/ram '
> + 'selinux=0 '
> + 'rdinit=/bin/sh')
> +
> + def wait_until_booted(self):
> + wait_for_console_pattern(self, 'no job control',
> + failure_message='Kernel panic - not syncing',
> + vm=None)
> +
> + def check_topology(self, c, s, b, d, e, t):
> + res = self.vm.qmp('query-cpus-fast')
> + cpus = res['return']
> + for cpu in cpus:
> + core = cpu['props']['core-id']
> + socket = cpu['props']['socket-id']
> + book = cpu['props']['book-id']
> + drawer = cpu['props']['drawer-id']
> + entitlement = cpu.get('entitlement')
> + dedicated = cpu.get('dedicated')
> + if core == c:
> + self.assertEqual(drawer, d)
> + self.assertEqual(book, b)
> + self.assertEqual(socket, s)
> + self.assertEqual(entitlement, e)
> + self.assertEqual(dedicated, t)
> +
> + def kernel_init(self):
> + """
> + We need a VM that supports CPU topology,
> + currently this only the case when using KVM, not TCG.
> + We need a kernel supporting the CPU topology.
> + We need a minimal root filesystem with a shell.
> + """
> + self.require_accelerator("kvm")
> + kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> + '/fedora-secondary/releases/35/Server/s390x/os'
> + '/images/kernel.img')
> + kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
> + kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
> + asset_hash=kernel_hash)
> +
> + initrd_url = ('https://archives.fedoraproject.org/pub/archive'
> + '/fedora-secondary/releases/35/Server/s390x/os'
> + '/images/initrd.img')
> + initrd_hash = 'a122057d95725ac030e2ec51df46e172'
> + initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
> + asset_hash=initrd_hash)
> + initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
> + archive.lzma_uncompress(initrd_path_xz, initrd_path)
> +
> + self.vm.set_console()
> + kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
> + self.vm.add_args('-nographic',
> + '-enable-kvm',
> + '-cpu', 'max,ctop=on',
> + '-m', '512',
> + '-kernel', kernel_path,
> + '-initrd', initrd_path,
> + '-append', kernel_command_line)
> +
> + def test_single(self):
> + """
> + This test checks the simplest topology with a single CPU.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.launch()
> + self.wait_until_booted()
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + def test_default(self):
> + """
> + This test checks the implicit topology.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.launch()
> + self.wait_until_booted()
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> + self.check_topology(1, 0, 0, 0, 'medium', False)
> + self.check_topology(2, 1, 0, 0, 'medium', False)
> + self.check_topology(3, 1, 0, 0, 'medium', False)
> + self.check_topology(4, 2, 0, 0, 'medium', False)
> + self.check_topology(5, 2, 0, 0, 'medium', False)
> + self.check_topology(6, 0, 1, 0, 'medium', False)
> + self.check_topology(7, 0, 1, 0, 'medium', False)
> + self.check_topology(8, 1, 1, 0, 'medium', False)
> + self.check_topology(9, 1, 1, 0, 'medium', False)
> + self.check_topology(10, 2, 1, 0, 'medium', False)
> + self.check_topology(11, 2, 1, 0, 'medium', False)
> + self.check_topology(12, 0, 0, 1, 'medium', False)
> +
> + def test_move(self):
> + """
> + This test checks the topology modification by moving a CPU
> + to another socket: CPU 0 is moved from socket 0 to socket 2.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.launch()
> + self.wait_until_booted()
> +
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> + res = self.vm.qmp('set-cpu-topology',
> + {'core-id': 0, 'socket-id': 2, 'entitlement': 'low'})
> + self.assertEqual(res['return'], {})
> + self.check_topology(0, 2, 0, 0, 'low', False)
Booting a Linux kernel in the guest always takes some precious seconds of
testing time ... I wonder whether it maybe makes sense to merge the above
two tests into one?
> + def test_hotplug_full(self):
> + """
> + This test verifies that a hotplugged defined with '-device'
> + command line argument finds its right place inside the topology.
"hotplug" is normally rather a "device_add" via HMP/QMP during run-time, so
I'd rather avoid this term here. Maybe call it "test_device_option" or so?
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.add_args('-smp',
> + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> + self.vm.add_args('-device', 'max-s390x-cpu,core-id=10')
> + self.vm.add_args('-device',
> + 'max-s390x-cpu,'
> + 'core-id=1,socket-id=0,book-id=1,drawer-id=1,entitlement=low')
> + self.vm.add_args('-device',
> + 'max-s390x-cpu,'
> + 'core-id=2,socket-id=0,book-id=1,drawer-id=1,entitlement=medium')
> + self.vm.add_args('-device',
> + 'max-s390x-cpu,'
> + 'core-id=3,socket-id=1,book-id=1,drawer-id=1,entitlement=high')
> + self.vm.add_args('-device',
> + 'max-s390x-cpu,'
> + 'core-id=4,socket-id=1,book-id=1,drawer-id=1')
> + self.vm.add_args('-device',
> + 'max-s390x-cpu,'
> + 'core-id=5,socket-id=2,book-id=1,drawer-id=1,dedicated=true')
> +
> + self.vm.launch()
> + self.wait_until_booted()
> +
> + self.check_topology(10, 2, 1, 0, 'medium', False)
> + self.check_topology(1, 0, 1, 1, 'low', False)
> + self.check_topology(2, 0, 1, 1, 'medium', False)
> + self.check_topology(3, 1, 1, 1, 'high', False)
> + self.check_topology(4, 1, 1, 1, 'medium', False)
> + self.check_topology(5, 2, 1, 1, 'high', True)
Thomas
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization
2023-09-01 15:58 ` [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization Nina Schoetterl-Glausch
2023-09-04 19:43 ` Nina Schoetterl-Glausch
@ 2023-09-07 9:02 ` Thomas Huth
2023-09-08 14:47 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 44+ messages in thread
From: Thomas Huth @ 2023-09-07 9:02 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Polarization is changed on a request from the guest.
> Let's verify the polarization is accordingly set by QEMU.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 45 ++++++++++++++++++++++++++++++++++
> 1 file changed, 45 insertions(+)
>
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> index 3a54fe2e00..d7ee36cb89 100644
> --- a/tests/avocado/s390_topology.py
> +++ b/tests/avocado/s390_topology.py
> @@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
> The polarization is changed on a request from the guest.
> """
> timeout = 90
> + event_timeout = 10
>
> KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
> 'root=/dev/ram '
> @@ -103,6 +104,14 @@ def kernel_init(self):
> '-initrd', initrd_path,
> '-append', kernel_command_line)
>
> + def system_init(self):
> + self.log.info("System init")
> + exec_command_and_wait_for_pattern(self,
> + """ mount proc -t proc /proc;
> + mount sys -t sysfs /sys;
> + /bin/cat /sys/devices/system/cpu/dispatching """,
Is the /bin/ prefix really required here?
> + '0')
> +
> def test_single(self):
> """
> This test checks the simplest topology with a single CPU.
> @@ -198,3 +207,39 @@ def test_hotplug_full(self):
> self.check_topology(3, 1, 1, 1, 'high', False)
> self.check_topology(4, 1, 1, 1, 'medium', False)
> self.check_topology(5, 2, 1, 1, 'high', True)
> +
> +
> + def guest_set_dispatching(self, dispatching):
> + exec_command(self,
> + f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
> + self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
> + exec_command_and_wait_for_pattern(self,
> + '/bin/cat /sys/devices/system/cpu/dispatching', dispatching)
dito?
> +
> + def test_polarization(self):
> + """
> + This test verifies that QEMU modifies the entitlement change after
> + several guest polarization change requests.
> +
> + :avocado: tags=arch:s390x
> + :avocado: tags=machine:s390-ccw-virtio
> + """
> + self.kernel_init()
> + self.vm.launch()
> + self.wait_until_booted()
> +
> + self.system_init()
> + res = self.vm.qmp('query-s390x-cpu-polarization')
> + self.assertEqual(res['return']['polarization'], 'horizontal')
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + self.guest_set_dispatching('1');
> + res = self.vm.qmp('query-s390x-cpu-polarization')
> + self.assertEqual(res['return']['polarization'], 'vertical')
> + self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> + self.guest_set_dispatching('0');
> + res = self.vm.qmp('query-s390x-cpu-polarization')
> + self.assertEqual(res['return']['polarization'], 'horizontal')
> + self.check_topology(0, 0, 0, 0, 'medium', False)
Anyway:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests
2023-09-01 15:58 ` [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests Nina Schoetterl-Glausch
2023-09-04 19:51 ` Nina Schoetterl-Glausch
@ 2023-09-07 9:05 ` Thomas Huth
2023-09-08 18:30 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 44+ messages in thread
From: Thomas Huth @ 2023-09-07 9:05 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> Test changes in the entitlement from both a guest and a host point of
> view, depending on the polarization.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 55 ++++++++++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
>
> diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> index d7ee36cb89..733093723a 100644
> --- a/tests/avocado/s390_topology.py
> +++ b/tests/avocado/s390_topology.py
> @@ -243,3 +243,58 @@ def test_polarization(self):
> res = self.vm.qmp('query-s390x-cpu-polarization')
> self.assertEqual(res['return']['polarization'], 'horizontal')
> self.check_topology(0, 0, 0, 0, 'medium', False)
> +
> +
> + def check_polarization(self, polarization):
> + """
> + We need the polarization change to get stabilized in kernel
> + before we check it through the sysfs.
> + """
> + exec_command_and_wait_for_pattern(self,
> + """ sleep 1 ;
> + /bin/cat /sys/devices/system/cpu/cpu0/polarization """,
> + polarization)
Using "sleep" in a function that is called multiple times will blow up the
testing time quite a bit ... could this be done in a different way instead,
e.g. by polling the sysfs file for the right value, or maybe there is
something in the "dmesg" output that could be used as a reference?
Thomas
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU
2023-09-01 15:58 ` [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU Nina Schoetterl-Glausch
2023-09-04 19:54 ` Nina Schoetterl-Glausch
@ 2023-09-07 10:08 ` Thomas Huth
1 sibling, 0 replies; 44+ messages in thread
From: Thomas Huth @ 2023-09-07 10:08 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel <pmorel@linux.ibm.com>
>
> A dedicated CPU in vertical polarization can only have
> a high entitlement.
> Let's check this from both host and guest point of view.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> tests/avocado/s390_topology.py | 33 +++++++++++++++++++++++++++++++++
> 1 file changed, 33 insertions(+)
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 14/20] tests/avocado: s390x cpu topology core
2023-09-07 8:27 ` Thomas Huth
@ 2023-09-08 14:27 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-08 14:27 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Thu, 2023-09-07 at 10:27 +0200, Thomas Huth wrote:
> On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> > From: Pierre Morel <pmorel@linux.ibm.com>
> >
> > Introduction of the s390x cpu topology core functions and
> > basic tests.
> >
> > We test the correlation between the command line and
> > the QMP results in query-cpus-fast for various CPU topology.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > ---
> > MAINTAINERS | 1 +
> > tests/avocado/s390_topology.py | 200 +++++++++++++++++++++++++++++++++
> > 2 files changed, 201 insertions(+)
> > create mode 100644 tests/avocado/s390_topology.py
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 7f2937b0f2..d1b8676666 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1703,6 +1703,7 @@ F: hw/s390x/cpu-topology.c
> > F: target/s390x/kvm/stsi-topology.c
> > F: docs/devel/s390-cpu-topology.rst
> > F: docs/system/s390x/cpu-topology.rst
> > +F: tests/avocado/s390_topology.py
> >
> > X86 Machines
> > ------------
> > diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> > new file mode 100644
> > index 0000000000..3a54fe2e00
> > --- /dev/null
> > +++ b/tests/avocado/s390_topology.py
> > @@ -0,0 +1,200 @@
[...]
> > + def test_default(self):
> > + """
> > + This test checks the implicit topology.
> > +
> > + :avocado: tags=arch:s390x
> > + :avocado: tags=machine:s390-ccw-virtio
> > + """
> > + self.kernel_init()
> > + self.vm.add_args('-smp',
> > + '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> > + self.vm.launch()
> > + self.wait_until_booted()
> > + self.check_topology(0, 0, 0, 0, 'medium', False)
> > + self.check_topology(1, 0, 0, 0, 'medium', False)
> > + self.check_topology(2, 1, 0, 0, 'medium', False)
> > + self.check_topology(3, 1, 0, 0, 'medium', False)
> > + self.check_topology(4, 2, 0, 0, 'medium', False)
> > + self.check_topology(5, 2, 0, 0, 'medium', False)
> > + self.check_topology(6, 0, 1, 0, 'medium', False)
> > + self.check_topology(7, 0, 1, 0, 'medium', False)
> > + self.check_topology(8, 1, 1, 0, 'medium', False)
> > + self.check_topology(9, 1, 1, 0, 'medium', False)
> > + self.check_topology(10, 2, 1, 0, 'medium', False)
> > + self.check_topology(11, 2, 1, 0, 'medium', False)
> > + self.check_topology(12, 0, 0, 1, 'medium', False)
> > +
> > + def test_move(self):
> > + """
> > + This test checks the topology modification by moving a CPU
> > + to another socket: CPU 0 is moved from socket 0 to socket 2.
> > +
> > + :avocado: tags=arch:s390x
> > + :avocado: tags=machine:s390-ccw-virtio
> > + """
> > + self.kernel_init()
> > + self.vm.add_args('-smp',
> > + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> > + self.vm.launch()
> > + self.wait_until_booted()
> > +
> > + self.check_topology(0, 0, 0, 0, 'medium', False)
> > + res = self.vm.qmp('set-cpu-topology',
> > + {'core-id': 0, 'socket-id': 2, 'entitlement': 'low'})
> > + self.assertEqual(res['return'], {})
> > + self.check_topology(0, 2, 0, 0, 'low', False)
>
> Booting a Linux kernel in the guest always takes some precious seconds of
> testing time ... I wonder whether it maybe makes sense to merge the above
> two tests into one?
IMO it's cleaner to keep them separated.
> > + def test_hotplug_full(self):
> > + """
> > + This test verifies that a hotplugged defined with '-device'
> > + command line argument finds its right place inside the topology.
>
> "hotplug" is normally rather a "device_add" via HMP/QMP during run-time, so
> I'd rather avoid this term here. Maybe call it "test_device_option" or so?
I'll go with test_dash_device.
>
> > + :avocado: tags=arch:s390x
> > + :avocado: tags=machine:s390-ccw-virtio
> > + """
> > + self.kernel_init()
> > + self.vm.add_args('-smp',
> > + '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
> > + self.vm.add_args('-device', 'max-s390x-cpu,core-id=10')
> > + self.vm.add_args('-device',
> > + 'max-s390x-cpu,'
> > + 'core-id=1,socket-id=0,book-id=1,drawer-id=1,entitlement=low')
> > + self.vm.add_args('-device',
> > + 'max-s390x-cpu,'
> > + 'core-id=2,socket-id=0,book-id=1,drawer-id=1,entitlement=medium')
> > + self.vm.add_args('-device',
> > + 'max-s390x-cpu,'
> > + 'core-id=3,socket-id=1,book-id=1,drawer-id=1,entitlement=high')
> > + self.vm.add_args('-device',
> > + 'max-s390x-cpu,'
> > + 'core-id=4,socket-id=1,book-id=1,drawer-id=1')
> > + self.vm.add_args('-device',
> > + 'max-s390x-cpu,'
> > + 'core-id=5,socket-id=2,book-id=1,drawer-id=1,dedicated=true')
> > +
> > + self.vm.launch()
> > + self.wait_until_booted()
> > +
> > + self.check_topology(10, 2, 1, 0, 'medium', False)
> > + self.check_topology(1, 0, 1, 1, 'low', False)
> > + self.check_topology(2, 0, 1, 1, 'medium', False)
> > + self.check_topology(3, 1, 1, 1, 'high', False)
> > + self.check_topology(4, 1, 1, 1, 'medium', False)
> > + self.check_topology(5, 2, 1, 1, 'high', True)
>
> Thomas
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization
2023-09-07 9:02 ` Thomas Huth
@ 2023-09-08 14:47 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-08 14:47 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Thu, 2023-09-07 at 11:02 +0200, Thomas Huth wrote:
> On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> > From: Pierre Morel <pmorel@linux.ibm.com>
> >
> > Polarization is changed on a request from the guest.
> > Let's verify the polarization is accordingly set by QEMU.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > ---
> > tests/avocado/s390_topology.py | 45 ++++++++++++++++++++++++++++++++++
> > 1 file changed, 45 insertions(+)
> >
> > diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> > index 3a54fe2e00..d7ee36cb89 100644
> > --- a/tests/avocado/s390_topology.py
> > +++ b/tests/avocado/s390_topology.py
> > @@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
> > The polarization is changed on a request from the guest.
> > """
> > timeout = 90
> > + event_timeout = 10
> >
> > KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
> > 'root=/dev/ram '
> > @@ -103,6 +104,14 @@ def kernel_init(self):
> > '-initrd', initrd_path,
> > '-append', kernel_command_line)
> >
> > + def system_init(self):
> > + self.log.info("System init")
> > + exec_command_and_wait_for_pattern(self,
> > + """ mount proc -t proc /proc;
> > + mount sys -t sysfs /sys;
> > + /bin/cat /sys/devices/system/cpu/dispatching """,
>
> Is the /bin/ prefix really required here?
No, /bin is a symlink to /usr/bin and that's in PATH.
Not sure why Pierre did it this way, might be a remnant of some other
vm image/initrd.
I'll remove it.
>
> > + '0')
> > +
> > def test_single(self):
> > """
> > This test checks the simplest topology with a single CPU.
> > @@ -198,3 +207,39 @@ def test_hotplug_full(self):
> > self.check_topology(3, 1, 1, 1, 'high', False)
> > self.check_topology(4, 1, 1, 1, 'medium', False)
> > self.check_topology(5, 2, 1, 1, 'high', True)
> > +
> > +
> > + def guest_set_dispatching(self, dispatching):
> > + exec_command(self,
> > + f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
> > + self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
> > + exec_command_and_wait_for_pattern(self,
> > + '/bin/cat /sys/devices/system/cpu/dispatching', dispatching)
>
> dito?
>
> > +
> > + def test_polarization(self):
> > + """
> > + This test verifies that QEMU modifies the entitlement change after
> > + several guest polarization change requests.
> > +
> > + :avocado: tags=arch:s390x
> > + :avocado: tags=machine:s390-ccw-virtio
> > + """
> > + self.kernel_init()
> > + self.vm.launch()
> > + self.wait_until_booted()
> > +
> > + self.system_init()
> > + res = self.vm.qmp('query-s390x-cpu-polarization')
> > + self.assertEqual(res['return']['polarization'], 'horizontal')
> > + self.check_topology(0, 0, 0, 0, 'medium', False)
> > +
> > + self.guest_set_dispatching('1');
> > + res = self.vm.qmp('query-s390x-cpu-polarization')
> > + self.assertEqual(res['return']['polarization'], 'vertical')
> > + self.check_topology(0, 0, 0, 0, 'medium', False)
> > +
> > + self.guest_set_dispatching('0');
> > + res = self.vm.qmp('query-s390x-cpu-polarization')
> > + self.assertEqual(res['return']['polarization'], 'horizontal')
> > + self.check_topology(0, 0, 0, 0, 'medium', False)
>
> Anyway:
> Reviewed-by: Thomas Huth <thuth@redhat.com>
Thanks!
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests
2023-09-07 9:05 ` Thomas Huth
@ 2023-09-08 18:30 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 44+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-09-08 18:30 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, qemu-s390x, Eduardo Habkost,
Marcel Apfelbaum, Paolo Bonzini, Halil Pasic,
Christian Borntraeger, Eric Farman, Richard Henderson,
David Hildenbrand, Eric Blake, Markus Armbruster, Michael Roth
Cc: Philippe Mathieu-Daudé, Yanan Wang, Daniel P. Berrangé,
Ilya Leoshkevich, Cleber Rosa, Wainer dos Santos Moschetta,
Beraldo Leal, Pierre Morel
On Thu, 2023-09-07 at 11:05 +0200, Thomas Huth wrote:
> On 01/09/2023 17.58, Nina Schoetterl-Glausch wrote:
> > From: Pierre Morel <pmorel@linux.ibm.com>
> >
> > Test changes in the entitlement from both a guest and a host point of
> > view, depending on the polarization.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> > ---
> > tests/avocado/s390_topology.py | 55 ++++++++++++++++++++++++++++++++++
> > 1 file changed, 55 insertions(+)
> >
> > diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
> > index d7ee36cb89..733093723a 100644
> > --- a/tests/avocado/s390_topology.py
> > +++ b/tests/avocado/s390_topology.py
> > @@ -243,3 +243,58 @@ def test_polarization(self):
> > res = self.vm.qmp('query-s390x-cpu-polarization')
> > self.assertEqual(res['return']['polarization'], 'horizontal')
> > self.check_topology(0, 0, 0, 0, 'medium', False)
> > +
> > +
> > + def check_polarization(self, polarization):
> > + """
> > + We need the polarization change to get stabilized in kernel
> > + before we check it through the sysfs.
> > + """
> > + exec_command_and_wait_for_pattern(self,
> > + """ sleep 1 ;
> > + /bin/cat /sys/devices/system/cpu/cpu0/polarization """,
> > + polarization)
>
> Using "sleep" in a function that is called multiple times will blow up the
> testing time quite a bit ... could this be done in a different way instead,
> e.g. by polling the sysfs file for the right value, or maybe there is
> something in the "dmesg" output that could be used as a reference?
Nothing in dmesg.
This works:
def check_polarization(self, polarization):
#We need to wait for the change to have been propagated to the kernel
exec_command_and_wait_for_pattern(self,
"\n".join([
"timeout 1 sh -c 'while true",
'do',
' syspath="/sys/devices/system/cpu/cpu0/polarization"',
' polarization="$(cat "$syspath")" || exit',
f' if [ "$polarization" = "{polarization}" ]; then',
' exit 0',
' fi',
' sleep 0.01',
#searched for strings mustn't show up in command, '' to obfuscate
"done' && echo succ''ess || echo fail''ure",
]),
"success", "failure")
>
> Thomas
>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2023-09-08 18:33 UTC | newest]
Thread overview: 44+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-01 15:57 [PATCH v22 00/20] s390x: CPU Topology Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 01/20] CPU topology: extend with s390 specifics Nina Schoetterl-Glausch
2023-09-05 9:01 ` Thomas Huth
2023-09-01 15:57 ` [PATCH v22 02/20] s390x/cpu topology: add topology entries on CPU hotplug Nina Schoetterl-Glausch
2023-09-05 12:28 ` Thomas Huth
2023-09-01 15:57 ` [PATCH v22 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Nina Schoetterl-Glausch
2023-09-04 18:23 ` Nina Schoetterl-Glausch
2023-09-05 13:26 ` Thomas Huth
2023-09-05 15:25 ` Nina Schoetterl-Glausch
2023-09-06 8:21 ` Thomas Huth
2023-09-06 12:12 ` Nina Schoetterl-Glausch
2023-09-06 14:53 ` Cédric Le Goater
2023-09-01 15:57 ` [PATCH v22 04/20] s390x/sclp: reporting the maximum nested topology entries Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 05/20] s390x/cpu topology: resetting the Topology-Change-Report Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 06/20] s390x/cpu topology: interception of PTF instruction Nina Schoetterl-Glausch
2023-09-01 15:57 ` [PATCH v22 07/20] target/s390x/cpu topology: activate CPU topology Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command Nina Schoetterl-Glausch
2023-09-06 8:56 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 09/20] machine: adding s390 topology to query-cpu-fast Nina Schoetterl-Glausch
2023-09-06 9:10 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 10/20] machine: adding s390 topology to info hotpluggable-cpus Nina Schoetterl-Glausch
2023-09-06 9:14 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 13/20] docs/s390x/cpu topology: document s390x cpu topology Nina Schoetterl-Glausch
2023-09-07 8:15 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 14/20] tests/avocado: s390x cpu topology core Nina Schoetterl-Glausch
2023-09-04 18:44 ` Nina Schoetterl-Glausch
2023-09-07 8:27 ` Thomas Huth
2023-09-08 14:27 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 15/20] tests/avocado: s390x cpu topology polarization Nina Schoetterl-Glausch
2023-09-04 19:43 ` Nina Schoetterl-Glausch
2023-09-07 9:02 ` Thomas Huth
2023-09-08 14:47 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 16/20] tests/avocado: s390x cpu topology entitlement tests Nina Schoetterl-Glausch
2023-09-04 19:51 ` Nina Schoetterl-Glausch
2023-09-07 9:05 ` Thomas Huth
2023-09-08 18:30 ` Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 17/20] tests/avocado: s390x cpu topology test dedicated CPU Nina Schoetterl-Glausch
2023-09-04 19:54 ` Nina Schoetterl-Glausch
2023-09-07 10:08 ` Thomas Huth
2023-09-01 15:58 ` [PATCH v22 18/20] tests/avocado: s390x cpu topology test socket full Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 19/20] tests/avocado: s390x cpu topology dedicated errors Nina Schoetterl-Glausch
2023-09-01 15:58 ` [PATCH v22 20/20] tests/avocado: s390x cpu topology bad move Nina Schoetterl-Glausch
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).