[PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[George Dunlap]

Suggested-by: Massimo Canonico <mex@di.unipmn.it>
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Massimo Canonico <mex@di.unipmn.it>
---
 docs/man/xl.cfg.pod.5 |   13 +++++++++++++
 docs/man/xl.pod.1     |   13 +++++++++++++
 docs/man/xm.pod.1     |   13 +++++++++++++
 3 files changed, 39 insertions(+)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index b7d64a6..069b73f 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -153,6 +153,19 @@ The cap is expressed in percentage of one physical CPU:
 The default, 0, means there is no upper cap.
 Honoured by the credit and credit2 schedulers.
 
+NB: Many systems have features that will scale down the computing
+power of a cpu that is not 100% utilized.  This can be in the
+operating system, but can also sometimes be below the operating system
+in the BIOS.  If you set a cap such that individual cores are running
+at less than 100%, this may have an impact on the performance of your
+workload over and above the impact of the cap. For example, if your
+processor runs at 2GHz, and you cap a vm at 50%, the power management
+system may also reduce the clock speed to 1GHz; the effect will be
+that your VM gets 25% of the available power (50% of 1GHz) rather than
+50% (50% of 2GHz).  If you are not getting the performance you expect,
+look at performance and cpufreq options in your operating system and
+your BIOS.
+
 =item B<period=NANOSECONDS>
 
 The normal EDF scheduling usage in nanoseconds. This means every period
diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index 57c6a79..0e2fe65 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -848,6 +848,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
 50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
 no upper cap.
 
+NB: Many systems have features that will scale down the computing
+power of a cpu that is not 100% utilized.  This can be in the
+operating system, but can also sometimes be below the operating system
+in the BIOS.  If you set a cap such that individual cores are running
+at less than 100%, this may have an impact on the performance of your
+workload over and above the impact of the cap. For example, if your
+processor runs at 2GHz, and you cap a vm at 50%, the power management
+system may also reduce the clock speed to 1GHz; the effect will be
+that your VM gets 25% of the available power (50% of 1GHz) rather than
+50% (50% of 2GHz).  If you are not getting the performance you expect,
+look at performance and cpufreq options in your operating system and
+your BIOS.
+
 =item B<-p CPUPOOL>, B<--cpupool=CPUPOOL>
 
 Restrict output to domains in the specified cpupool.
diff --git a/docs/man/xm.pod.1 b/docs/man/xm.pod.1
index 7c4ef85..4d47388 100644
--- a/docs/man/xm.pod.1
+++ b/docs/man/xm.pod.1
@@ -767,6 +767,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
 50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
 no upper cap.
 
+NB: Many systems have features that will scale down the computing
+power of a cpu that is not 100% utilized.  This can be in the
+operating system, but can also sometimes be below the operating system
+in the BIOS.  If you set a cap such that individual cores are running
+at less than 100%, this may have an impact on the performance of your
+workload over and above the impact of the cap. For example, if your
+processor runs at 2GHz, and you cap a vm at 50%, the power management
+system may also reduce the clock speed to 1GHz; the effect will be
+that your VM gets 25% of the available power (50% of 1GHz) rather than
+50% (50% of 2GHz).  If you are not getting the performance you expect,
+look at performance and cpufreq options in your operating system and
+your BIOS.
+
 =back
 
 =item B<sched-sedf> I<period> I<slice> I<latency-hint> I<extratime> I<weight>
-- 
1.7.9.5

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Massimo Canonico]

Thanks.
I was expecting that this warning appears somewhere here:
http://wiki.xen.org/wiki/Credit_Scheduler#Cap

but I did not find it.

M
On 06/11/2013 02:31 PM, George Dunlap wrote:
> Suggested-by: Massimo Canonico <mex@di.unipmn.it>
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Ian Jackson <ian.jackson@citrix.com>
> CC: Massimo Canonico <mex@di.unipmn.it>
> ---
>   docs/man/xl.cfg.pod.5 |   13 +++++++++++++
>   docs/man/xl.pod.1     |   13 +++++++++++++
>   docs/man/xm.pod.1     |   13 +++++++++++++
>   3 files changed, 39 insertions(+)
>
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index b7d64a6..069b73f 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -153,6 +153,19 @@ The cap is expressed in percentage of one physical CPU:
>   The default, 0, means there is no upper cap.
>   Honoured by the credit and credit2 schedulers.
>   
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>   =item B<period=NANOSECONDS>
>   
>   The normal EDF scheduling usage in nanoseconds. This means every period
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index 57c6a79..0e2fe65 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -848,6 +848,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
>   50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
>   no upper cap.
>   
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>   =item B<-p CPUPOOL>, B<--cpupool=CPUPOOL>
>   
>   Restrict output to domains in the specified cpupool.
> diff --git a/docs/man/xm.pod.1 b/docs/man/xm.pod.1
> index 7c4ef85..4d47388 100644
> --- a/docs/man/xm.pod.1
> +++ b/docs/man/xm.pod.1
> @@ -767,6 +767,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
>   50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
>   no upper cap.
>   
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>   =back
>   
>   =item B<sched-sedf> I<period> I<slice> I<latency-hint> I<extratime> I<weight>

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 526 bytes --]

On mar, 2013-06-11 at 17:32 +0200, Massimo Canonico wrote:
> Thanks.
> I was expecting that this warning appears somewhere here:
> http://wiki.xen.org/wiki/Credit_Scheduler#Cap
> 
> but I did not find it.
> 
I just added it there too.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2089 bytes --]

On mar, 2013-06-11 at 13:31 +0100, George Dunlap wrote:
> Suggested-by: Massimo Canonico <mex@di.unipmn.it>
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Ian Jackson <ian.jackson@citrix.com>
> CC: Massimo Canonico <mex@di.unipmn.it>
>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>

> ---
>  docs/man/xl.cfg.pod.5 |   13 +++++++++++++
>  docs/man/xl.pod.1     |   13 +++++++++++++
>  docs/man/xm.pod.1     |   13 +++++++++++++
>  3 files changed, 39 insertions(+)
> 
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index b7d64a6..069b73f 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -153,6 +153,19 @@ The cap is expressed in percentage of one physical CPU:
>  The default, 0, means there is no upper cap.
>  Honoured by the credit and credit2 schedulers.
>  
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>
I'd have been less 'politically correct' toward BIOSes, but I guess
that's me! :-P

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Ian Campbell]

On Tue, 2013-06-11 at 13:31 +0100, George Dunlap wrote:
> Suggested-by: Massimo Canonico <mex@di.unipmn.it>
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Ian Jackson <ian.jackson@citrix.com>
> CC: Massimo Canonico <mex@di.unipmn.it>

Seems a bit repetitive but Acked + applied (with Dario's reviewed-by
too)

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1051 bytes --]

On mer, 2013-06-12 at 10:44 +0100, Ian Campbell wrote:
> On Tue, 2013-06-11 at 13:31 +0100, George Dunlap wrote:
> > Suggested-by: Massimo Canonico <mex@di.unipmn.it>
> > Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> > CC: Ian Campbell <ian.campbell@citrix.com>
> > CC: Ian Jackson <ian.jackson@citrix.com>
> > CC: Massimo Canonico <mex@di.unipmn.it>
> 
> Seems a bit repetitive but Acked + applied (with Dario's reviewed-by
> too)
> 
I suppose it is but, given how weird the BIOSes might behave with
respect to this, and looking at how much time it took to track down an
issue that this was causing (compared to what the issue really was!)...
Well, better state it a couple of times than leave _any_ room for people
not noticing it! :-P

Thanks,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Konrad Rzeszutek Wilk]

On Tue, Jun 11, 2013 at 01:31:38PM +0100, George Dunlap wrote:
> Suggested-by: Massimo Canonico <mex@di.unipmn.it>
> Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Ian Jackson <ian.jackson@citrix.com>
> CC: Massimo Canonico <mex@di.unipmn.it>
> ---
>  docs/man/xl.cfg.pod.5 |   13 +++++++++++++
>  docs/man/xl.pod.1     |   13 +++++++++++++
>  docs/man/xm.pod.1     |   13 +++++++++++++
>  3 files changed, 39 insertions(+)
> 
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index b7d64a6..069b73f 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -153,6 +153,19 @@ The cap is expressed in percentage of one physical CPU:
>  The default, 0, means there is no upper cap.
>  Honoured by the credit and credit2 schedulers.
>  
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.

Or .. use 'cpufreq=xen:performance' ?

That should set it to the highest P state.

> +
>  =item B<period=NANOSECONDS>
>  
>  The normal EDF scheduling usage in nanoseconds. This means every period
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index 57c6a79..0e2fe65 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -848,6 +848,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
>  50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
>  no upper cap.
>  
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>  =item B<-p CPUPOOL>, B<--cpupool=CPUPOOL>
>  
>  Restrict output to domains in the specified cpupool.
> diff --git a/docs/man/xm.pod.1 b/docs/man/xm.pod.1
> index 7c4ef85..4d47388 100644
> --- a/docs/man/xm.pod.1
> +++ b/docs/man/xm.pod.1
> @@ -767,6 +767,19 @@ is expressed in percentage of one physical CPU: 100 is 1 physical CPU,
>  50 is half a CPU, 400 is 4 CPUs, etc. The default, 0, means there is
>  no upper cap.
>  
> +NB: Many systems have features that will scale down the computing
> +power of a cpu that is not 100% utilized.  This can be in the
> +operating system, but can also sometimes be below the operating system
> +in the BIOS.  If you set a cap such that individual cores are running
> +at less than 100%, this may have an impact on the performance of your
> +workload over and above the impact of the cap. For example, if your
> +processor runs at 2GHz, and you cap a vm at 50%, the power management
> +system may also reduce the clock speed to 1GHz; the effect will be
> +that your VM gets 25% of the available power (50% of 1GHz) rather than
> +50% (50% of 2GHz).  If you are not getting the performance you expect,
> +look at performance and cpufreq options in your operating system and
> +your BIOS.
> +
>  =back
>  
>  =item B<sched-sedf> I<period> I<slice> I<latency-hint> I<extratime> I<weight>
> -- 
> 1.7.9.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Ian Campbell]

On Wed, 2013-06-12 at 09:57 -0400, Konrad Rzeszutek Wilk wrote:

> > +NB: Many systems have features that will scale down the computing
> > +power of a cpu that is not 100% utilized.  This can be in the
> > +operating system, but can also sometimes be below the operating system
> > +in the BIOS.  If you set a cap such that individual cores are running
> > +at less than 100%, this may have an impact on the performance of your
> > +workload over and above the impact of the cap. For example, if your
> > +processor runs at 2GHz, and you cap a vm at 50%, the power management
> > +system may also reduce the clock speed to 1GHz; the effect will be
> > +that your VM gets 25% of the available power (50% of 1GHz) rather than
> > +50% (50% of 2GHz).  If you are not getting the performance you expect,
> > +look at performance and cpufreq options in your operating system and
> > +your BIOS.
> 
> Or .. use 'cpufreq=xen:performance' ?
> 
> That should set it to the highest P state.

I committed this already. Assuming this is a good suggestion can we get
an incremental patch please.

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[George Dunlap]

On 12/06/13 15:59, Ian Campbell wrote:
> On Wed, 2013-06-12 at 09:57 -0400, Konrad Rzeszutek Wilk wrote:
>
>>> +NB: Many systems have features that will scale down the computing
>>> +power of a cpu that is not 100% utilized.  This can be in the
>>> +operating system, but can also sometimes be below the operating system
>>> +in the BIOS.  If you set a cap such that individual cores are running
>>> +at less than 100%, this may have an impact on the performance of your
>>> +workload over and above the impact of the cap. For example, if your
>>> +processor runs at 2GHz, and you cap a vm at 50%, the power management
>>> +system may also reduce the clock speed to 1GHz; the effect will be
>>> +that your VM gets 25% of the available power (50% of 1GHz) rather than
>>> +50% (50% of 2GHz).  If you are not getting the performance you expect,
>>> +look at performance and cpufreq options in your operating system and
>>> +your BIOS.
>> Or .. use 'cpufreq=xen:performance' ?
>>
>> That should set it to the highest P state.
> I committed this already. Assuming this is a good suggestion can we get
> an incremental patch please.

Might that kind of thing be better on the wiki page?

  -George

Re: [PATCH] docs: Make note for the scheduler "cap" option warning about power management effects

[Konrad Rzeszutek Wilk]

On Wed, Jun 12, 2013 at 04:01:12PM +0100, George Dunlap wrote:
> On 12/06/13 15:59, Ian Campbell wrote:
> >On Wed, 2013-06-12 at 09:57 -0400, Konrad Rzeszutek Wilk wrote:
> >
> >>>+NB: Many systems have features that will scale down the computing
> >>>+power of a cpu that is not 100% utilized.  This can be in the
> >>>+operating system, but can also sometimes be below the operating system
> >>>+in the BIOS.  If you set a cap such that individual cores are running
> >>>+at less than 100%, this may have an impact on the performance of your
> >>>+workload over and above the impact of the cap. For example, if your
> >>>+processor runs at 2GHz, and you cap a vm at 50%, the power management
> >>>+system may also reduce the clock speed to 1GHz; the effect will be
> >>>+that your VM gets 25% of the available power (50% of 1GHz) rather than
> >>>+50% (50% of 2GHz).  If you are not getting the performance you expect,
> >>>+look at performance and cpufreq options in your operating system and
> >>>+your BIOS.
> >>Or .. use 'cpufreq=xen:performance' ?
> >>
> >>That should set it to the highest P state.
> >I committed this already. Assuming this is a good suggestion can we get
> >an incremental patch please.
> 
> Might that kind of thing be better on the wiki page?

Perhaps. But since the docs talk about 'your operating system
and your BIOS' I figured it should also mention how to configure Xen
to pretty much ignore any P-states and just sit at P0 all the time.