Re: [PATCH 0/2] KVM: PPC: Book3S HV: Dynamic micro-threading/split-core

[Laurent Vivier <lvivier@redhat.com>]

[I resend my message because MLs have refused the first one in HTML]

On 28/05/2015 07:17, Paul Mackerras wrote:
> This patch series provides a way to use more of the capacity of each
> processor core when running guests configured with threads=1, 2 or 4
> on a POWER8 host with HV KVM, without having to change the static
> micro-threading (the official name for split-core) mode for the whole
> machine.  The problem with setting the machine to static 2-way or
> 4-way micro-threading mode is that (a) then you can't run guests with
> threads=8 and (b) selecting the right mode can be tricky and requires
> knowledge of what guests you will be running.
>
> Instead, with these two patches, we can now run more than one virtual
> core (vcore) on a given physical core if possible, and if that means
> we need to switch the core to 2-way or 4-way micro-threading mode,
> then we do that on entry to the guests and switch back to whole-core
> mode on exit (and we only switch the one core, not the whole machine).
> The core mode switching is only done if the machine is in static
> whole-core mode.
>
> All of this only comes into effect when a core is over-committed.
> When the machine is lightly loaded everything operates the same with
> these patches as without.  Only when some core has a vcore that is
> able to run while there is also another vcore that was wanting to run
> on that core but got preempted does the logic kick in to try to run
> both vcores at once.
>
> Paul.
> ---
>
>  arch/powerpc/include/asm/kvm_book3s_asm.h |  20 +
>  arch/powerpc/include/asm/kvm_host.h       |  22 +-
>  arch/powerpc/kernel/asm-offsets.c         |   9 +
>  arch/powerpc/kvm/book3s_hv.c              | 648 ++++++++++++++++++++++++++----
>  arch/powerpc/kvm/book3s_hv_builtin.c      |  32 +-
>  arch/powerpc/kvm/book3s_hv_rm_xics.c      |   4 +-
>  arch/powerpc/kvm/book3s_hv_rmhandlers.S   | 111 ++++-
>  7 files changed, 740 insertions(+), 106 deletions(-)

Tested-by: Laurent Vivier <lvivier@redhat.com>

Performance is better, but Paul could you explain why it is better if I disable dynamic micro-threading ?
Did I miss something ?

My test system is an IBM Power S822L.

I run two guests with 8 vCPUs (-smp 8,sockets=8,cores=1,threads=1) both
attached on the same core (with pinning option of virt-manager). Then, I
measure the time needed to compile a kernel in parallel in both guests
with "make -j 16".

My kernel without micro-threading:

real    37m23.424s                 real    37m24.959s
user    167m31.474s                user    165m44.142s
sys     113m26.195s                sys     113m45.072s

With micro-threading patches (PATCH 1+2):

target_smt_mode 0 [in fact It was 8 here, but it should behave like 0, as it is > max threads/sub-core]
dynamic_mt_modes 6

real    32m13.338s                 real  32m26.652s
user    139m21.181s                user  140m20.994s
sys     77m35.339s                 sys   78m16.599s

It's better, but if I disable dynamic micro-threading (but PATCH 1+2):

target_smt_mode 0
dynamic_mt_modes 0

real    30m49.100s                 real 30m48.161s
user    144m22.989s                user 142m53.886s
sys     65m4.942s                  sys  66m8.159s

it's even better.

without dynamic micro-threading patch (with PATCH1 but not PATCH2):

target_smt_mode 0

real    33m57.279s                 real 34m19.524s
user    158m43.064s                user 156m19.863s
sys     74m25.442s                 sys  76m42.994s


Laurent