Re: Xen credit scheduler question

[George Dunlap <george.dunlap@eu.citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4963 bytes --]

On 15/11/12 19:03, Michael Palmeter wrote:
>
> Thank you for your answer, George.
>
> The origin of my question is more of a business concern than a 
> technical one.  Many software products are licensed based on a cost 
> per processor core.  It is desirable to sometimes allow customers to 
> pay a fraction of software license costs in exchange for running that 
> software using only a commensurate fraction of available compute power 
> (capacity sub-licensing).  If the cap is a means of making a vCPU 
> more-or-less deterministic (in terms of its effective computational 
> capacity) then that would be useful as a programmatic means of 
> enabling capacity sub-licensing.  My example below was based on a case 
> where I have a customer that would like to use ‘cap’ to constrain 
> their single vCPU VM to only ½ of a core worth of compute capacity 
> (logically 1/32 of the compute power) in exchange for only paying 1/32 
> of the license cost for the physical server.
>

Right -- I've seen the "limit cpu power for licensing purposes" thing 
before, but I think that only went down to cores, not sub-core.

> Below you answered:
>
> “You can use ‘cap’ to make the VM in question get 50% of logical vcpu 
> time, which on an idle system will give it 0.5 of the capacity of a 
> physical core (if we don't consider Intel's Turbo Boost technology).  
> But if the system becomes busy, it will get less than 0.5 of the 
> processing capacity of a physical core.”
>
> Are you saying that cap would be able to CONSTRAIN a vCPU to an 
> effective compute capacity equal to 50% of a physical core, but it 
> does not GUARANTEE effective compute capacity equal to 50% of a 
> physical core?
>

Theoretically, a cap at 50 will give your single-vcpu VM 50% of the time 
of one hyperthread.

So if C is "typicall throughput of a single non-hyperthreaded core 
running at standard requency", and we factor out Turbo Boost, then there 
are two cases to consider:

* Other thread is idle.  In that case, the VM will get 0.5C.
* The other thread is busy.  In this case, assuming a 0.7 factor, the VM 
will get 0.5 * (0.7 * C), or about 0.35C

So the total computing power available to the VM should be <= 0.5C 
(satisfying the licensing requirements), but on a busy system it may be 
significantly less than 0.5C (perhaps not so satisfying to the owner of 
the VM).

I don't think it should be terribly difficult to put a simple "shared 
hyperthread" multiplier on the credit burned -- if someone at Oracle 
wanted to help implement this, we'd be happy to point you in the right 
direction. :-)

If you have Turbo Boost, then (as I understand it) the CPU can raise the 
clock speed of the processor when threads or cores are idle; the 
wikipedia article seems to think some processors can increase the clock 
speed up to 1.6x over the baseline frequency.  That would throw a bit of 
a wrench in the works, as you might end up with 0.5 * 1.6 * C = 0.8 C > 
0.5 C; however, looking at Intel's website, it looks like only 2- and 
4-core processors have TurboBoost, so maybe on 8-core processors we can 
punt on that thorny issue for a little while yet. :-)

> Can you offer any guidance regarding real-world scheduler overhead 
> (when cap>0 is used) and precision (how variable is actual compute 
> power for a vCPU with a cap of 100%, for example)?
>

I have not done extensive testing with the cap; I mainly know the 
mechanism by which it works.  There is no extra accounting done in the 
scheduler for having a cap: all vcpus are assigned credit every 30ms 
according to their weight and cap.  The difference is that if a 
non-capped vcpu uses up its credits, it is allowed to go negative; 
whereas a capped vcpu will be paused until it receives more credits.  So 
there should be no extra hypervisor overhead from using a cap.

The cap fundamentally works by locking out a vcpu for very small amounts 
of time within the 30ms accounting window.  But this same effect might 
happen just by having other VMs competing for the cpu; so in theory 
shouldn't be any riskier than virtualizing in the first place.

Executive summary: Factoring out Turbo Boost, "cap" should be able to 
set a sub-core upper-bound on processing power.  But on a busy system, 
it may result in the VM getting less than its upper-bound in processing 
power.

However, scheduling is a very complex and dynamic system, and like 
economics, very simple changes can have unpredictable results.  So it's 
probably a good idea to do some testing before recommending it to 
customers. :-)

BTW, are you familiar with Xen's cpupool functionality?  The guys at 
Fujitsu wrote it so that a customer could rent a fixed number of cores 
to a customer, who could then run as many VMs on those cores as they 
wanted.  I think licensing restrictions had something to do with that as 
well.  More about that here, if you're interested:
  http://blog.xen.org/index.php/2012/04/23/xen-4-2-cpupools/

  -George

[-- Attachment #1.2: Type: text/html, Size: 10862 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel