Re: Xen credit scheduler question

[George Dunlap <george.dunlap@eu.citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3586 bytes --]

On 15/11/12 15:43, Michael Palmeter wrote:
>
> Hi all (and Mr. Dunlap in particular),
>

Haha -- please don't call me "Mr"; I prefer "George", but if you want a 
title, use "Dr" (since I have  PhD). :-)

> Example scenario:
>
>   * Server hardware: 2 sockets, 8-cores per socket, 2 hardware threads
>     per core (total of 32 hardware threads)
>   * Test VM: a single virtual machine with a single vCPU, weight=256
>     and cap=100%
>
> In this scenario, from what I understand, I should be able to load the 
> Test VM with traffic to a maximum of approximately 1/32 of the 
> aggregate compute capacity of the server.  The total CPU utilization 
> of the server hardware should be approximately 3.4%, plus the overhead 
> of dom0 (say 1-2).  The credits available to any vCPU capped at 100% 
> should be equal to 1/32 of the aggregate compute available for the 
> whole server, correct?
>

I think to really be precise, you should say, "1/32nd of the logical cpu 
time available", where "logical cpu time" simply means, "time processing 
on one logical CPU".  At the moment, that is all that either the credit1 
or credit2 schedulers look at.

As I'm sure you're aware, not all "logical cpu time" is equal.  If one 
thread of a hyperthread pair is running but the other idle, it will get 
significantly higher performance than if the other thread is busy.  How 
much is highly unpredictable, and depends very much on exactly what 
units are shared with the other hyperthread, and the workload running on 
each unit.  But even when both threads are busy, it should (in theory) 
be rare for both threads to get a throughput of 50%; the whole idea of 
HT is that threads typically get 70-80% of the full performance of the 
core (so the overall throughput is increased).

But if course, while this is particularly extreme in the case of 
hyperthreads, it's also true on a smaller scale even without that -- 
cores share caches, NUMA nodes share memory bandwidth, and so on. No 
attempt is made to compensate VMs for cache misses or extra memory 
latency due to sharing either. :-)

> Put simply, is there a way to constrain a VM with 1 vCPU to consume no 
> more than 0.5 of a physical core (hyper-threaded) on the server 
> hardware mentioned below? Does the cap help in that respect?
>

You can use "cap" to make the VM in question get 50% of logical vcpu 
time, which on an idle system will give it 0.5 of the capacity of a 
physical core (if we don't consider Intel's Turbo Boost technology).  
But if the system becomes busy, it will get less than 0.5 of the 
processing capacity of a physical core.

> I have been struggling to understand how the scheduler can deal with 
> the uncertainty that hyperthreading introduces, however.  I know this 
> is an issue that you are tackling in the credit2 scheduler, but I 
> would like to know what your thoughts are on this problem (if you are 
> able to share).  Any insight or assistance you could offer would be 
> greatly appreciated.
>

At the moment it does not attempt to; the only thing it does is try not 
to schedule two hyperthreads that share a core if there is an idle 
core.  But if there are more active vcpus than cores, then some will 
share; and the ones that share a core with another vcpu will be charged 
the same as the ones that have the core all to themselves.

Could you explain why you your question is important to you -- i.e,. 
what are you trying to accomplish?  It sounds a bit like you're more 
concerned with accuracy in reporting, and control of resources, rather 
than fairness, for instance.

  -George

[-- Attachment #1.2: Type: text/html, Size: 8275 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel