Re: [PATCH v9 10/10] docs/s390x: document s390x cpu topology

[Janis Schoetterl-Glausch <scgl@linux.ibm.com>]

On Fri, 2022-09-02 at 09:55 +0200, Pierre Morel wrote:
> Add some basic examples for the definition of cpu topology
> in s390x.
> 
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
>  docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++
>  1 file changed, 88 insertions(+)
>  create mode 100644 docs/system/s390x/cpu_topology.rst
> 
> diff --git a/docs/system/s390x/cpu_topology.rst b/docs/system/s390x/cpu_topology.rst
> new file mode 100644
> index 0000000000..00977d4319
> --- /dev/null
> +++ b/docs/system/s390x/cpu_topology.rst
> @@ -0,0 +1,88 @@
> +CPU Topology on s390x
> +=====================
> +
> +CPU Topology on S390x provides up to 4 levels of topology containers:
> +drawers, books, sockets and CPUs.
> +While the three higher level containers, Containers Topology List Entries,
> +(Containers TLE) define a tree hierarchy, the lowest level of topology
> +definition, the CPU Topology List Entry (CPU TLE), provides the placement
> +of the CPUs inside the last container.

inside the parent container
> +
> +Prerequisites
> +-------------
> +
> +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility
> +(STFLE bit 11) is required.
> +
> +However, since this facility has been enabled by default in an early version,
> +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM
> +that QEMU support CPU Topology.

I don't understand this paragraph. Early version of what?
> +
> +Indicating the CPU topology to the Virtual Machine
> +--------------------------------------------------
> +
> +The CPU Topology, number of drawers, number of books per drawers, number of
> +sockets per book and number of cores per sockets is specified with the
> +``-smp`` qemu command arguments.
> +
> +Like in :
> +
> +.. code-block:: sh
> +    -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192
> +
> +If drawers or books are not specified, their default to 1.
> +
> +New CPUs can be plugged using the device_add hmp command like in:
> +
> +.. code-block:: sh
> +   (qemu) device_add host-s390x-cpu,core-id=9
> +
> +The core-id defines the placement of the core in the topology by
> +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum
> +core number of the last socket of the last book in the last drawer.
> +
> +In the example above:
> +
> +* the core with ID 9 will be placed in container (0,0,1), as core 9
> +  of CPU TLE 0 of socket 1 in book 0 from drawer 0.
> +* the core ID 0 is defined by the -smp cpus=1 command and will be
> +  placed as core 0 in CPU TLE 0 of container (0,0,0)
> +
> +Note that the core ID is machine wide and the CPU TLE masks provided
> +by the STSI instruction will be:
> +
> +* in socket 0: 0x80000000 (core id 0)
> +* in socket 1: 0x00400000 (core id 9)
> +
> +Indicating the CPU topology to the Guest
> +----------------------------------------
> +
> +The guest can query for topology changes using the PTF instruction.
> +In case of a topology change it can request the new topology by issuing
> +STSI instructions specifying the level of detail required, drawer with
> +STSI(15.1.4) or books STSI(15.1.3).
> +
> +The virtual machine will fill the provided buffer with the count of
> +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and
> +cores per socket (MAG1).
> +
> +Note that the STSI(15.1.2) is special in two ways:
> +
> +* When the firmware detect a change in the values calculated for STSI(15.1.2)
> +  it will trigger the report of the topology change for the PTF instruction.

I don't know if we need this section, after all documenting this is the
job of the principles of operation. You could just refer to the
relevant sections.
> +
> +Migration
> +---------
> +
> +For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is
> +by default disabled.
> +
> +CPU Topoogy is by default enabled for s390-virtio-ccw-7.2 and newer machines.
> +
> +Enabling the CPU topology on older Machine is done by setting the global
> +option ''topology-disable'' to false before enabling cpu topology with the
> +cpu feature "ctop" like in:
> +
> +.. code-block:: sh
> +   -machine s390-ccw-virtio-3.0,accel=kvm,topology-disable=false
> +   -cpu z14,ctop=on