[PATCH] credit: Change default timeslice to 5ms

[PATCH] credit: Change default timeslice to 5ms

[George Dunlap]

The 30ms timeslice was chosen nearly a decade ago now, with cpu
"burning" workloads in mind.  In the mean time, processors have gotten
faster and VMEXITs have gotten faster.  A timeslice of 30ms has a
major cost when running latency-sensitive workloads like network or
audio streaming: getting caught behind just one or two other VMs can
introduce a processing delay of up to 60ms, and the "round-robin"
nature of the credit scheduler means this delay may be introduced
every time the VM yields for periods of time.

The XenServer performance team at Citrix have done extensive testing
with various timeslices, including 30ms, 10ms, 5ms, and 2ms.  None of
the workloads exhibited any performance degradation with a 5ms
timeslice.

Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
CC: Keir Fraser <keir@xen.org>
CC: Dario Faggioli <dario.faggioli@citrix.com>
CC: Marcus Granado <Marcus.Granado@eu.citrix.com>
---
 xen/common/sched_credit.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index db5512e..61553d9 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -29,9 +29,9 @@
  * Basic constants
  */
 #define CSCHED_DEFAULT_WEIGHT       256
-#define CSCHED_TICKS_PER_TSLICE     3
-/* Default timeslice: 30ms */
-#define CSCHED_DEFAULT_TSLICE_MS    30
+#define CSCHED_TICKS_PER_TSLICE     1
+/* Default timeslice: 5ms */
+#define CSCHED_DEFAULT_TSLICE_MS    5
 #define CSCHED_CREDITS_PER_MSEC     10
 
 
-- 
1.7.9.5

Re: [PATCH] credit: Change default timeslice to 5ms

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1457 bytes --]

On mer, 2014-03-05 at 16:29 +0000, George Dunlap wrote:
> The 30ms timeslice was chosen nearly a decade ago now, with cpu
> "burning" workloads in mind.  In the mean time, processors have gotten
> faster and VMEXITs have gotten faster.  A timeslice of 30ms has a
> major cost when running latency-sensitive workloads like network or
> audio streaming: getting caught behind just one or two other VMs can
> introduce a processing delay of up to 60ms, and the "round-robin"
> nature of the credit scheduler means this delay may be introduced
> every time the VM yields for periods of time.
> 
> The XenServer performance team at Citrix have done extensive testing
> with various timeslices, including 30ms, 10ms, 5ms, and 2ms.  None of
> the workloads exhibited any performance degradation with a 5ms
> timeslice.
> 
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Tim Deegan <tim@xen.org>
> CC: Keir Fraser <keir@xen.org>
> CC: Dario Faggioli <dario.faggioli@citrix.com>
> CC: Marcus Granado <Marcus.Granado@eu.citrix.com>
> ---
>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] credit: Change default timeslice to 5ms

[David Vrabel]

On 05/03/14 16:29, George Dunlap wrote:
> The 30ms timeslice was chosen nearly a decade ago now, with cpu
> "burning" workloads in mind.  In the mean time, processors have gotten
> faster and VMEXITs have gotten faster.  A timeslice of 30ms has a
> major cost when running latency-sensitive workloads like network or
> audio streaming: getting caught behind just one or two other VMs can
> introduce a processing delay of up to 60ms, and the "round-robin"
> nature of the credit scheduler means this delay may be introduced
> every time the VM yields for periods of time.
> 
> The XenServer performance team at Citrix have done extensive testing
> with various timeslices, including 30ms, 10ms, 5ms, and 2ms.  None of
> the workloads exhibited any performance degradation with a 5ms
> timeslice.
[...]
> --- a/xen/common/sched_credit.c
> +++ b/xen/common/sched_credit.c
> @@ -29,9 +29,9 @@
>   * Basic constants
>   */
>  #define CSCHED_DEFAULT_WEIGHT       256
> -#define CSCHED_TICKS_PER_TSLICE     3
> -/* Default timeslice: 30ms */
> -#define CSCHED_DEFAULT_TSLICE_MS    30
> +#define CSCHED_TICKS_PER_TSLICE     1

The TICKS_PER_TSLICE change doubles the tick rate.  Is this intentional?
 It's not mentioned in the commit message.

> +/* Default timeslice: 5ms */
> +#define CSCHED_DEFAULT_TSLICE_MS    5
>  #define CSCHED_CREDITS_PER_MSEC     10

David

Re: [PATCH] credit: Change default timeslice to 5ms

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1159 bytes --]

On mer, 2014-03-05 at 17:18 +0000, David Vrabel wrote:
> On 05/03/14 16:29, George Dunlap wrote:

> > --- a/xen/common/sched_credit.c
> > +++ b/xen/common/sched_credit.c
> > @@ -29,9 +29,9 @@
> >   * Basic constants
> >   */
> >  #define CSCHED_DEFAULT_WEIGHT       256
> > -#define CSCHED_TICKS_PER_TSLICE     3
> > -/* Default timeslice: 30ms */
> > -#define CSCHED_DEFAULT_TSLICE_MS    30
> > +#define CSCHED_TICKS_PER_TSLICE     1
> 
> The TICKS_PER_TSLICE change doubles the tick rate.  Is this intentional?
>  It's not mentioned in the commit message.
> 
Well, I guess higher tick rate is exactly what you expect from reducing
the timeslice... otherwise, you could not be able to enforce the
timeslice boundaries precisely, could you?

It's unavoidable, I think: if you stick with a tick every 30ms/3=10ms,
how can you enforce a timeslice of 5ms?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] credit: Change default timeslice to 5ms

[George Dunlap]

On 03/05/2014 05:18 PM, David Vrabel wrote:
> On 05/03/14 16:29, George Dunlap wrote:
>> The 30ms timeslice was chosen nearly a decade ago now, with cpu
>> "burning" workloads in mind.  In the mean time, processors have gotten
>> faster and VMEXITs have gotten faster.  A timeslice of 30ms has a
>> major cost when running latency-sensitive workloads like network or
>> audio streaming: getting caught behind just one or two other VMs can
>> introduce a processing delay of up to 60ms, and the "round-robin"
>> nature of the credit scheduler means this delay may be introduced
>> every time the VM yields for periods of time.
>>
>> The XenServer performance team at Citrix have done extensive testing
>> with various timeslices, including 30ms, 10ms, 5ms, and 2ms.  None of
>> the workloads exhibited any performance degradation with a 5ms
>> timeslice.
> [...]
>> --- a/xen/common/sched_credit.c
>> +++ b/xen/common/sched_credit.c
>> @@ -29,9 +29,9 @@
>>    * Basic constants
>>    */
>>   #define CSCHED_DEFAULT_WEIGHT       256
>> -#define CSCHED_TICKS_PER_TSLICE     3
>> -/* Default timeslice: 30ms */
>> -#define CSCHED_DEFAULT_TSLICE_MS    30
>> +#define CSCHED_TICKS_PER_TSLICE     1
> The TICKS_PER_TSLICE change doubles the tick rate.  Is this intentional?
>   It's not mentioned in the commit message.

Hmm -- actually, I just realized that Marcus' test was done with 3 ticks 
per timeslice, so "5ms / 1 tick" has *not* been validated. And this is 
actually important, because the main purpose of the ticks is to give the 
scheduler an opportunity to switch VMs out of "BOOST" priority and into 
"UNDER" priority.  Reducing the ticks per timeslice changes that 
dynamic, and would need to be tested separately.

Also, I just discovered a rather pathological case in general that seems 
to be in the scheduler, so for the time being let me retract this while 
I figure that out.

  -George

Re: [PATCH] credit: Change default timeslice to 5ms

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1688 bytes --]

On gio, 2014-03-06 at 12:04 +0000, George Dunlap wrote:
> On 03/05/2014 05:18 PM, David Vrabel wrote:
> > On 05/03/14 16:29, George Dunlap wrote:
> >> --- a/xen/common/sched_credit.c
> >> +++ b/xen/common/sched_credit.c
> >> @@ -29,9 +29,9 @@
> >>    * Basic constants
> >>    */
> >>   #define CSCHED_DEFAULT_WEIGHT       256
> >> -#define CSCHED_TICKS_PER_TSLICE     3
> >> -/* Default timeslice: 30ms */
> >> -#define CSCHED_DEFAULT_TSLICE_MS    30
> >> +#define CSCHED_TICKS_PER_TSLICE     1
> > The TICKS_PER_TSLICE change doubles the tick rate.  Is this intentional?
> >   It's not mentioned in the commit message.
> 
> Hmm -- actually, I just realized that Marcus' test was done with 3 ticks 
> per timeslice, so "5ms / 1 tick" has *not* been validated. And this is 
> actually important, because the main purpose of the ticks is to give the 
> scheduler an opportunity to switch VMs out of "BOOST" priority and into 
> "UNDER" priority.  Reducing the ticks per timeslice changes that 
> dynamic, and would need to be tested separately.
> 
Which is even higher tick frequency, isn't it? Anyway, you're right, we
should go for the tested solution.

> Also, I just discovered a rather pathological case in general that seems 
> to be in the scheduler, so for the time being let me retract this while 
> I figure that out.
> 
Care to share some more details about this?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] credit: Change default timeslice to 5ms

[Tim Deegan]

At 16:29 +0000 on 05 Mar (1394033376), George Dunlap wrote:
> The 30ms timeslice was chosen nearly a decade ago now, with cpu
> "burning" workloads in mind.  In the mean time, processors have gotten
> faster and VMEXITs have gotten faster.  A timeslice of 30ms has a
> major cost when running latency-sensitive workloads like network or
> audio streaming: getting caught behind just one or two other VMs can
> introduce a processing delay of up to 60ms, and the "round-robin"
> nature of the credit scheduler means this delay may be introduced
> every time the VM yields for periods of time.
> 
> The XenServer performance team at Citrix have done extensive testing
> with various timeslices, including 30ms, 10ms, 5ms, and 2ms.  None of
> the workloads exhibited any performance degradation with a 5ms
> timeslice.
> 
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Tim Deegan <tim@xen.org>

Acked-by: Tim Deegan <tim@xen.org>

> CC: Keir Fraser <keir@xen.org>
> CC: Dario Faggioli <dario.faggioli@citrix.com>
> CC: Marcus Granado <Marcus.Granado@eu.citrix.com>
> ---
>  xen/common/sched_credit.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
> index db5512e..61553d9 100644
> --- a/xen/common/sched_credit.c
> +++ b/xen/common/sched_credit.c
> @@ -29,9 +29,9 @@
>   * Basic constants
>   */
>  #define CSCHED_DEFAULT_WEIGHT       256
> -#define CSCHED_TICKS_PER_TSLICE     3
> -/* Default timeslice: 30ms */
> -#define CSCHED_DEFAULT_TSLICE_MS    30
> +#define CSCHED_TICKS_PER_TSLICE     1
> +/* Default timeslice: 5ms */
> +#define CSCHED_DEFAULT_TSLICE_MS    5
>  #define CSCHED_CREDITS_PER_MSEC     10
>  
>  
> -- 
> 1.7.9.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel