Re: [RFC] Scheduler work, part 1: High-level goals and interface.

[Jeremy Fitzhardinge <jeremy@goop.org>]

Dan Magenheimer wrote:
>> In any case, it's a bit like asking, "Why would I buy
>> a machine with two hyperthreads instead of two cores?"
>>     
>
> Yes.  In a physical machine, the OS takes advantage of all
> resources available.  So it doesn't matter if some of the
> "processors" are cores and some are hyperthreads.  You
> are using ALL of the CPU resources you paid for.
>
> But in a virtualized environment, each VM gets a fraction
> of the resources and if grabbing some fixed number of
> "processors" sometimes gets hyperthreads and sometimes
> gets cores, this will cause interesting issues for some
> workloads.
>
> Think about a cloud where one pays for resources used.
> You likely would demand to pay less for a hyperpair than
> a non-vht pair.
>
> As a result, I think it will be a requirement that
> a system administrator be able to specify "I want two
> FULL cores" vs "I am willing to accept two hyperthreads".
> And once you get beyond hyperpairs, this is going to
> get very messy.
>   

I think you're over-complicating it.  At very worst, it will be no worse 
than the current situation where Xen will place the vcpus on 
threads/cores in more or less arbitrary ways.

I think George's proposal can already accommodate the user needs you're 
talking about:

If the scheduler accounts for time spent executing on a contended HT 
thread (ie, the threads are not paired, so the other thread could be 
idle or running any other code) at a lesser rate than a full 
core/uncontended thread, then the charging works out.

If the user has a requirement that domain X's vcpus must be running at 
full speed, then they can set their reservation to 100%.  If we say that 
a contended HT thread is only worth, say, 70% of a "real" core, then 
that not only factors into the charging, but also means that any domain 
with a reservation > 70% is ineligible to run on a contended HT thread.  
(I think in practise this means that any domain with high reservations 
will end up running on gang scheduled thread pairs, just to guarantee 
that the other thread is idle, so the uncontended HT thread can run at 
100%.)

(Another way to look at it is that HT contention is a bit like having 
your vcpu being preempted by Xen, but rather than going from 100% 
running to 0% running, your vcpu drops to 70%.)

    J