Re: RE: [RFC][PATCH 0/4] Modification of credit scheduler rev2

[NISHIGUCHI Naoki <nisiguti@jp.fujitsu.com>]

Tian, Kevin wrote:
>>>>> 	4. issues left:
>>>>> 		a. Abrupt glitches are still generated when the 
>>>> QEMU emulated mouse being used and moving mouse quickly in 
>>>> guest A. Passing-through USB mouse/keyboard to guest A, then 
>>>> no glitches.
>>>>
>>>> I also noticed that. Though I don't know the precise cause, I 
>>>> found that 
>>>> dom0 and guest A would consume largely CPU time (hundreds of 
>>>> milliseconds) in such situation. In this case, the priority of 
>>>> dom0 and 
>>>> guest A falls rapidly, then guest B runs until the priority of 
>>>> dom0 and 
>>>> guest A becomes BOOST. In worst case, it will take about 120ms.
>>> I remember that Disheng once told me that BOOST only happens
>>> when vcpu is waken up and its current priority is UNDER. In your
>>> case guest A should be in OVER after running hundreds of ms, 
>>> and then it waits enough long time to become UNDER and then 
>>> BOOST. If this is the case, your enhancement on BOOST level
>>> seems only solving part of the latency issue. Here either assigning
>>> a static priority, or adding more BOOST source (like event, intr,
>>> etc) seems more complete solution.
>> In my case, though the vcpu should be switched to other vcpu in time 
>> slice, the cpu running the vcpu doesn't schedule during 
>> hundreds of ms. 
>> I don't know why this happens.
> 
> What's running within your guest B? Unless full cpu intensive workload
> happens within guest B, there's chance for guest B to issue block 
> hypercall once it enters idle loop, and then once it's blocked, Xen 
> credit scheduler can pick dom0 or guest A anyway. So 1st thing you
> could figure out the activity within guest B.
> 
> If guest B does be always busy, then you may need to check the 30ms
> credit allocation algorithm in credit scheduler. It looks like some sequence
> that guest A may be always granted as OVER priority due to its earlier
> overrun, until guestB also overruns a similar length. Then in this punish
> period, guest A has no chance to be boosted with all cycles granted to
> guest B instead. if it's intended for fairness p.o.v, it may not suit for rt
> usage.

Sorry, I didn't explain well.
I mean that softirq for scheduling (SCHEDULE_SOFTIRQ) might not occur 
during hundreds of ms. I found similar issue when connecting vncviewer 
to guest B. Guest B runs nothing. But I don't use Disheng's configuration.
I assumed that this issue (Disheng said) is the same issue as mine.

>> In credit scheduler, credit consumed by the vcpu must be subtracted. 
>> Therefore I think it is correct that dom0 and guest A are OVER because 
>> my approach is to boost the vcpu within the range of weight.
>>
>> I think assigning a static priority is one solution. However, I think 
>> that it affects credit accounting because we don't know how long the 
>> domain with the static priority (probably highest priority) is run.
> 
> It could be one configurable option for some client usages, where 
> a coarse-level static priority could better ensure the deterministic
> to satisfy specific rt requirement.

I see.

>> About adding more BOOST source, could you explain more to me?
> 
> Current the only source for boost is the wakeup event on a vcpu
> with UNDER priority to catch up which is simply from fairness p.o.v
> But for vcpu with RT requirement, more boost sources can be added.
> E.g. when audio interrupt (either emulated, or passthrough), boost 
> target vcpu and trigger a reschedule softirq immediately to reduce 
> uncertainty of schedule latency. We need such a manual boost 
> interface which is then inserted into some critical event paths where
> we believe immediate schedule is necessary. Disheng is working on
> this area now, I think. :-)

Thanks.

Regards,
Naoki