Re: Notes on stubdoms and latency on ARM

[Dario Faggioli <dario.faggioli@citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3006 bytes --]

On Mon, 2017-07-17 at 10:25 +0100, George Dunlap wrote:
> On 07/12/2017 07:14 AM, Dario Faggioli wrote:
> > 
> > That being said, I personally have never liked rate-limiting, it
> > always
> > looked to me like the wrong solution.
> 
> In fact, I *think* the only reason it may have been introduced is
> that
> there was a bug in the credit2 code at the time such that it always
> had
> a single runqueue no matter what your actual pcpu topology was.
> 
It has been introduced because SpecVirt perf were bad because, during
interrupt storms, the context-switch rate was really really high.

It was all about Credit1... Work on Credit2 was stalled at the time,
and there has been, AFAICR, no evaluation of Credit2 was involved:

https://wiki.xen.org/wiki/Credit_Scheduler#Context-Switch_Rate_Limiting
https://lists.xenproject.org/archives/html/xen-devel/2011-12/msg00897.html%7C

(And in fact, it was not implemented in Credit2, until something like
last year, Anshul wrote the code for that.)

SpecVirt performance were judged to be important enough (e.g., because
we've been told people was using that for comparing us with other virt.
solutions), that this was set to on by default. 

I don't know if that is still the case, as I've run many benchmarks,
but never had the chance to try SpecVirt first hand myself. Fact is
that Credit1 does not have any measure in place for limit/control
context-switch rate, and it has boosting, which means that rate-
limiting (as much as I may hate it :-P) is actually useful.

Whether we should have it disabled by default, and tell people (in
documentation) to enable it if they think they're seeing the system
going into trashing because of context switching, or the vice-versa,
it's one of those things which is rather hard to tell. Let's see...

In Credit2, we do have CSCHED2_MIN_TIMER (which is not equivalent to
ratelimiting, of course, but it at least is something that goes in the
direction of trying to avoid too frequent interruptions), and (much
more important, IMO) we don't have boosting... So, I think it would be
interesting to try figuring out the role that rate-limiting plays, when
Credit2 is in use (and then, maybe, if we find that there are
differences, find a way to have, as default, it enabled on Credit1 and
disabled on Credit2).

> > I'll think about it, and see if I'll be able to run some benchmarks
> > with it on and off.
> 
> Thanks.  FYI the main benchmark that was used to justify its
> inclusion
> (and on by default) was specvirt (I think).
> 
Yeah, I know. I'm not sure I will have the chance to run that soon,
though. I'll try a bunch of other workloads, and we'll see what I will
find. :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel