Re: [RFC][PATCH] scheduler: credit scheduler for client virtualization

[NISHIGUCHI Naoki <nisiguti@jp.fujitsu.com>]

George Dunlap wrote:
> For basic workload tests, a couple are pretty handy.  vConsolidate is
> a good test, but pretty hard to set up; I should be able to manage it
> with our infrastructure here, though.  Other tests include:
> * kernel-build (i.e., time how long it takes to build the Linux
> kernel) and or ddk-build (Windows equivalent)
> * specjbb (a cpu-intensive workload)
> * netperf (for networks)
> 
> For testing its effect on network, the paper I mentioned has three
> workloads that it combines with different ways:
> * cpu (just busy spinning)
> * sustained network (netbench): throughput
> * network ping: latency.

Thanks! I'll try to prepare.

> I think I understand you to mean: If we set the timer for 10ms, and in
> the mean time another vcpu wakes up and is set at BOOST, then it won't
> get a chance to run for another 10 ms.  And you're suggesting that we
> run the scheduler at 2ms if there are any vcpus that *may* wake up and
> be at BOOST, just in case; and you don't think this situation will
> happen very often.  Is that correct?

Almost that is correct.
I had thought that we run the scheduler at 2ms only if there are vcpus 
that have boost credit and are already at BOOST. But I don't think so now.

> Unfortunately, in consolidated server workloads you're pretty likely
> to have more vcpus than physical cpus, so I think this case would come
> up pretty often.  Furthermore, 2ms is really too short a scheduling
> quantum for normal use, especially for HVM domains, which have to take
> a vmexit/vmenter cycle to handle every interrupt.  (I did some tests
> back when we were using the SEDF scheduler, and the scheduling alone
> was a 4-5% overhead for HVM domains.)

I see.

> But I don't think we actually have a problem here: if a vcpu wakes up
> and is promoted to BOOST, won't it "tickle" the runqueues to find
> somewhere for it to run?  At very least the current cpu should be able
> to run it, or if it's already running one at BOOST, it can set its own
> timer to 2ms.  In any case, I think handling this corner case with
> some extra code is preferrable to running a 2ms timer any time it
> *might* happen.

OK.
I implemented as follows:
- If next running vcpu is at BOOST and first vcpu on run-queue is at 
BOOST, set the timer to 2ms.
- If next running vcpu is at BOOST and first vcpu on run-queue is not at 
BOOST, set the timer to 10ms.
- If next running vcpu is not at BOOST, set the timer to 30ms.
- When a vcpu wakes up, if the vcpu has boost credit then send scheduler 
interrupts to at least one CPU.

In my test environment, it works well.
I'll post last patch today.

Thanks,
Naoki