Questions regarding Xen Credit Scheduler

[Gaurav Dhiman <dimanuec@gmail.com>]

Hi All,

I am using Xen 3.3.2 for some of my experiments, and have been
consistently observing some sub-optimal results in our experiments
with latency sensitive I/O intensive and compute bound VMs running
together on a physical machine. We are observing latency issues in
cases with enough CPU resources available for both the VMs to co-exist
well, even if we give much higher weight to the latency sensitive VM.
I suspect it is due to the way the Xen credit scheduler works. In this
context, I have some questions regarding the scheduler:

1) In the sched_acct function, the credit cap is set to 300, enough to
survive one time slice. But if some VCPU crosses that cap, it is set
to 0, and marked inactive. Why is there no concept of a ceiling (like
that of a floor for the VCPUs going over the credit line), i.e. why is
it not set to 300? Is there some fundamental reason for setting it to
0? I believe this is resulting in a lot of times when our latency
sensitive VCPUs have to wait for maybe a time slice, when they can
immediately run. This might happen if they run with BOOST priority and
get interrupted by a timer tick, which takes that priority away.

2) Why is the runq sorted by just priority (which is very coarse
grained: BOOST, UNDER and OVER), and not the credit? This can result
in VCPUs with higher credit getting starved for CPU if we have batch
and latency sensitive VCPUs in the system.

3) Is there some patch, which makes the current credit scheduler
fairer to the latency sensitive VCPUs? I see that the sched_credit2
scheduler addresses these issues, but right now it has just one global
runq and no load balancing features.

Any advice/inputs here will be extremely valuable!

Thanks in advance,
-Gaurav