Re: [PATCH 07 of 10 [RFC]] sched_credit: Let the scheduler know about `node affinity`

[Dario Faggioli <raistlin@linux.it>]

[-- Attachment #1.1: Type: text/plain, Size: 2798 bytes --]

On Wed, 2012-04-11 at 15:17 +0200, Dario Faggioli wrote: 
> Basic idea is: cpu-affinity tells where a vcpu can run,
> while node-affinity tells where (in terms of on what NUMA nodes,
> but that of course imply a set of CPUs) all the vcpus of the
> domain should/prefer to run... Problems starts when you have
> both of them speaking at the same time!
>
> <snip>
> 
> XXX I'm benchmarking this just right now, and peeking at the results
>     they don't seem too bad. Numbers are mostly close to the case where
>     the VM is already pinned to a node when created. I'll post the
>     results as soon as I can, and I'll be happy to investigate any issue,
>     if you feel like the approach could be the right one.
> 
Some of them are ready, and here they come:

  http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/index.html

Look, for example, at this one here:

 http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/kernbench_avgstd.png

Where the red line is the default Xen behavior (no pinning, no affinity,
nothing!), the dark blue line is the best case (domain pinned at
creation ==> all memory accesses are local) and the light blue line is
this one patch in action, i.e., domain _with_affinity_ to node #0 and no
pinning at all.

It's very nice (at least for me! :-D) to see it almost always does
better than default, and it manage in getting quite close to the best
case scenario when load increases.
Still on the "nice things" side, there doesn't appear to be
significative regressions introduced by this patch, if the new
node-affinity feature is not stressed, as it shows by comparing the
"before" and "after" the patch graphs (looking at matching curves):

(before) http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005/kernbench_avgstd.png
 (after)  http://xenbits.xen.org/people/dariof/benchmarks/specjbb2005-numa/kernbench_avgstd.png

Unfortunately, there seems to be at least some issues with the cases
where load is not so high. In fact, given VMs have 4 vcpus and host has
16 pcpus, there are less vcpus than pcpus up to the run involving 4 VMs,
which is actually where things starts to work well, while before they're
near or even worst than default! :-O

Any thoughts?

Again, if you think the approach taken here could be a sensible one,
I'll start digging what's going on (by some tracing etc.), and I'm sure
I can work out the low-load cases and "persuade" them to behave
better. :-D

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel