Re: [PATCH v16 00/11] s390x: CPU Topology

[Pierre Morel <pmorel@linux.ibm.com>]

On 2/27/23 15:00, Thomas Huth wrote:
> On 22/02/2023 15.20, Pierre Morel wrote:
>> Hi,
>>
>> No big changes here, some bug corrections and comments modifications
>> following Thomas and Nina comments and Daniel and Markus reommandations.
>>
>> Implementation discussions
>> ==========================
>>
>> CPU models
>> ----------
>>
>> Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already
>> in the CPU model for old QEMU we could not activate it as usual from
>> KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
>> Checking and enabling this capability enables facility 11,
>> S390_FEAT_CONFIGURATION_TOPOLOGY.
>>
>> It is the responsibility of the admin to ensure the same CPU
>> model for source and target host in a migration.
>>
>> Migration
>> ---------
>>
>> When the target guest is started, the Multi-processor Topology Change
>> Report (MTCR) bit is set during the creation of the vCPU by KVM.
>> We do not need to migrate its state, in the worst case, the target
>> guest will see the MTCR and actualize its view of the topology
>> without necessity, but this will be done only one time.
>>
>> Reset
>> -----
>>
>> Reseting the topology is done during subsystem reset, the
>> polarization is reset to horizontal polarization.
>>
>> Topology attributes
>> -------------------
>>
>> The topology attributes are carried by the CPU object and defined
>> on object creation.
>> In the case the new attributes, socket, book, drawer, dedicated,
>> polarity are not provided QEMU provides defaults values.
>>
>> - Geometry defaults
>>    The geometry default are based on the core-id of the core to
>>    fill the geometry in a monotone way starting with drawer 0,
>>    book 0, and filling socket 0 with the number of cores per socket,
>>    then filling socket 1, socket 2 ... etc until the book is complete
>>    and all books until the first drawer is complete before starting with
>>    the next drawer.
>>
>>    This allows to keep existing start scripts and Libvirt existing
>>    interface until it is extended.
>>
>> - Modifiers defaults
>>    Default polarization is horizontal
>>    Default dedication is not dedicated.
>>
>> Dynamic topology modification
>> -----------------------------
>>
>> QAPI interface is extended with:
>> - a command: 'x-set-cpu-topology'
>> - a query: extension of 'query-cpus-fast'
>> - an event: 'CPU_POLARITY_CHANGE'
>>
>> The admin may use query-cpus-fast to verify the topology provided
>> to the guest and x-set-cpu-topology to modify it.
>>
>> The event CPU_POLARITY_CHANGE is sent when the guest successfuly
>> uses the PTF(2) instruction to request a polarization change.
>> In that case, the admin is supposed to modify the CPU provisioning
>> accordingly.
>>
>> Testing
>> =======
>>
>> To use the QEMU patches, you will need Linux V6-rc1 or newer,
>> or use the following Linux mainline patches:
>>
>> f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report
>> 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function
>> 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF 
>> fac..
>>
>> Currently this code is for KVM only, I have no idea if it is interesting
>> to provide a TCG patch. If ever it will be done in another series.
>>
>> Documentation
>> =============
>>
>> To have a better understanding of the S390x CPU Topology and its
>> implementation in QEMU you can have a look at the documentation in the
>> last patch of this series.
>>
>> The admin will want to match the host and the guest topology, taking
>> into account that the guest does not recognize multithreading.
>> Consequently, two vCPU assigned to threads of the same real CPU should
>> preferably be assigned to the same socket of the guest machine.
>>
>>
>> Regards,
>> Pierre
>>
>> Pierre Morel (11):
>>    s390x/cpu topology: add s390 specifics to CPU topology
>>    s390x/cpu topology: add topology entries on CPU hotplug
>>    target/s390x/cpu topology: handle STSI(15) and build the SYSIB
>>    s390x/sclp: reporting the maximum nested topology entries
>>    s390x/cpu topology: resetting the Topology-Change-Report
>>    s390x/cpu topology: interception of PTF instruction
>>    target/s390x/cpu topology: activating CPU topology
>>    qapi/s390x/cpu topology: set-cpu-topology monitor command
>>    machine: adding s390 topology to query-cpu-fast
>>    qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
>>    docs/s390x/cpu topology: document s390x cpu topology
>>
>>   docs/system/s390x/cpu-topology.rst | 378 ++++++++++++++++++++
>>   docs/system/target-s390x.rst       |   1 +
>>   qapi/machine-target.json           |  81 +++++
>>   qapi/machine.json                  |  37 +-
>>   include/hw/boards.h                |  10 +-
>>   include/hw/s390x/cpu-topology.h    |  78 +++++
>>   include/hw/s390x/s390-virtio-ccw.h |   6 +
>>   include/hw/s390x/sclp.h            |   4 +-
>>   include/monitor/hmp.h              |   1 +
>>   target/s390x/cpu.h                 |  78 +++++
>>   target/s390x/kvm/kvm_s390x.h       |   1 +
>>   hw/core/machine-qmp-cmds.c         |   2 +
>>   hw/core/machine-smp.c              |  48 ++-
>>   hw/core/machine.c                  |   4 +
>>   hw/s390x/cpu-topology.c            | 534 +++++++++++++++++++++++++++++
>>   hw/s390x/s390-virtio-ccw.c         |  27 +-
>>   hw/s390x/sclp.c                    |   5 +
>>   softmmu/vl.c                       |   6 +
>>   target/s390x/cpu-sysemu.c          |  13 +
>>   target/s390x/cpu.c                 |   7 +
>>   target/s390x/cpu_models.c          |   1 +
>>   target/s390x/kvm/cpu_topology.c    | 312 +++++++++++++++++
>>   target/s390x/kvm/kvm.c             |  42 ++-
>>   hmp-commands.hx                    |  17 +
>>   hw/s390x/meson.build               |   1 +
>>   qemu-options.hx                    |   7 +-
>>   target/s390x/kvm/meson.build       |   3 +-
>>   27 files changed, 1685 insertions(+), 19 deletions(-)
>>   create mode 100644 docs/system/s390x/cpu-topology.rst
>>   create mode 100644 include/hw/s390x/cpu-topology.h
>>   create mode 100644 hw/s390x/cpu-topology.c
>>   create mode 100644 target/s390x/kvm/cpu_topology.c
>
> Any chance that you could also add some qtests for checking that the 
> topology works as expected? I.e. set some topology via the command 
> line, then use QMP to check whether all CPUs got the right settings?
>
>  Thomas
>
Yes I intend to develop some tests but it will take some time to develop 
it I need to do change of polarization from inside the guest and qmp 
from the host.

Regards,

Pierre