Re: [PATCH RFC 0/2] kvm: Improving undercommit,overcommit scenarios in PLE handler

[Chegu Vinod <chegu_vinod@hp.com>]

On 9/21/2012 4:59 AM, Raghavendra K T wrote:
> In some special scenarios like #vcpu <= #pcpu, PLE handler may
> prove very costly,

Yes.
>   because there is no need to iterate over vcpus
> and do unsuccessful yield_to burning CPU.
>
> An idea to solve this is:
> 1) As Avi had proposed we can modify hardware ple_window
> dynamically to avoid frequent PL-exit.

Yes. We had to do this to get around some scaling issues for large 
(>20way) guests (with no overcommitment)

As part of some experimentation we even tried "switching off"  PLE too :(



> (IMHO, it is difficult to
> decide when we have mixed type of VMs).

Agree.

Not sure if the following alternatives have also been looked at :

- Could the  behavior  associated with the "ple_window" be modified to 
be a function of some [new] per-guest attribute (which can be conveyed 
to the host as part of the guest launch sequence). The user can choose 
to set this [new] attribute for a given guest. This would help avoid the 
frequent exits due to PLE (as Avi had mentioned earlier) ?

- Can the PLE feature ( in VT) be "enhanced" to be made a per guest 
attribute ?


IMHO, the approach of not taking a frequent exit is better than taking 
an exit and returning back from the handler etc.

Thanks
Vinod




>
> Another idea, proposed in the first patch, is to identify
> non-overcommit case and just return from the PLE handler.
>
> There are are many ways to identify non-overcommit scenario.
> 1) Using loadavg etc (get_avenrun/calc_global_load
>   /this_cpu_load)
>
> 2) Explicitly check nr_running()/num_online_cpus()
>
> 3) Check source vcpu runqueue length.
>
> Not sure how can we make use of (1) effectively/how to use it.
> (2) has significant overhead since it iterates all cpus.
> so this patch uses third method. (I feel it is uglier to export
> runqueue length, but expecting suggestion on this).
>
> In second patch, when we have large number of small guests, it is
> possible that a spinning vcpu fails to yield_to any vcpu of same
> VM and go back and spin. This is also not effective when we are
> over-committed. Instead, we do a schedule() so that we give chance
> to other VMs to run.
>
> Raghavendra K T(2):
>   Handle undercommitted guest case in PLE handler
>   Be courteous to other VMs in overcommitted scenario in PLE handler
>
> Results:
> base = 3.6.0-rc5 + ple handler optimization patches from kvm tree.
> patched = base + patch1 + patch2
> machine: x240 with 16 core with HT enabled (32 cpu thread).
> 32 vcpu guest with 8GB RAM.
>
> +-----------+-----------+-----------+------------+-----------+
>           ebizzy (record/sec higher is better)
> +-----------+-----------+-----------+------------+-----------+
>     base        stddev       patched    stdev        %improve
> +-----------+-----------+-----------+------------+-----------+
>   11293.3750   624.4378	 18209.6250   371.7061	  61.24166
>    3641.8750   468.9400	  3725.5000   253.7823	   2.29621
> +-----------+-----------+-----------+------------+-----------+
>
> +-----------+-----------+-----------+------------+-----------+
>          kernbench (time in sec lower is better)
> +-----------+-----------+-----------+------------+-----------+
>     base        stddev       patched    stdev        %improve
> +-----------+-----------+-----------+------------+-----------+
>      30.6020     1.3018	    30.8287     1.1517	  -0.74080
>      64.0825     2.3764	    63.4721     5.0191	   0.95252
>      95.8638     8.7030	    94.5988     8.3832	   1.31958
> +-----------+-----------+-----------+------------+-----------+
>
> Note:
> on mx3850x5 machine with 32 cores HT disabled I got around
> ebizzy      209%
> kernbench   6%
> improvement for 1x scenario.
>
> Thanks Srikar for his active partipation in discussing ideas and
> reviewing the patch.
>
> Please let me know your suggestions and comments.
> ---
>   include/linux/sched.h |    1 +
>   kernel/sched/core.c   |    6 ++++++
>   virt/kvm/kvm_main.c   |    7 +++++++
>   3 files changed, 14 insertions(+), 0 deletions(-)
>
> .
>