Virt overehead with HT [was: Re: Xen 4.5 development update]

[Dario Faggioli <dario.faggioli@citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2796 bytes --]

[Sorry to the ones that will receive this mail twice, but I managed to
drop xen-devel when replying the first time :-(]

On mar, 2014-07-01 at 12:43 -0400, konrad.wilk@oracle.com wrote:
> == x86 == 

> *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
>    kernbench demonstrated it
>    looking and tracing it
>   -  Dario Faggioli
> 
I spent a few time running kernbench on different boxes and with
different configurations. After all this, here's what I found.

So, on a non-NUMA, HT and EPT capable box, both BAREMETAL and HVM case
were using 8G RAM and 8 CPUs/VCPUs. HT was enabled in BIOS:

Elapsed(stddev)   BAREMETAL             HVM
kernbench -j4     31.604 (0.0963328)    34.078 (0.168582)
kernbench -j8     26.586 (0.145705)     26.672 (0.0432435)
kernbench -j      27.358 (0.440307)     27.49 (0.364897)

With HT disabled in BIOS (which means only 4 CPUs for both):
Elapsed(stddev)   BAREMETAL             HVM
kernbench -j4     57.754 (0.0642651)    56.46 (0.0578792)
kernbench -j8     31.228 (0.0775887)    31.362 (0.210998)
kernbench -j      32.316 (0.0270185)    33.084 (0.600442)

So, first of all, no much difference, in terms of performance
degradation when going from baremetal to guest, between the HT and no-HT
cases.

In the HT enabled case, there is a slight degradation in perf on the
least loaded case, but certainly not of the nature Stefano saw, and all
goes pretty well when load increases.

I guess I can investigate a bit more about what happens with '-j4'. What
I suspect is that the scheduler may make a few non-optimal decisions wrt
HT, when there are more PCPUs than busy guest VCPUs. This may be due to
the fact that Dom0 (or another guest VCPU doing other stuff than
kernbench) may be already running on PCPUs that are on different cores
than the guest's one (i.e., the guest VCPUs that wants to run
kernbench), and that may force two guest's vCPUs to execute on two HTs
some of the time (which of course is something that does not happen on
baremetal!).

However, that is, I think, a separate issue, and it looks to me that
the original HT perf regression we were chasing, the one this item is
about, may actually have been disappear, or it was caused to something
different than HT. :-P

Thoughts? Do we think this is enough to kill the "disable hyperthreading
hint" from the performance tuning page on the Wiki?
http://wiki.xenproject.org/wiki/Tuning_Xen_for_Performance#Hyperthreading_in_Xen_4.3_and_4.4

Regards,
Dario


-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel