AW: How to use Px/Cx for power saving?

AW: How to use Px/Cx for power saving?

[Carsten Schiers]

Hi,

cpufreq support can be either handled by xen or dom0-kernel. You have to specify cpufreq=xen or
cpufreq=dom0-kernel as parameters when starting the hypervisor.

Kevin Tian wrote yesterday:

  > It's intentional since Xen itself supports cpufreq now. To avoid 
  > dependency on dom0's CONFIG_CPU_FREQ and also intrusive 
  > checks on whether external control is enabled in every linux
  > cpufreq drivers, CPU_FREQ in dom0 is silented by default now.
  > It's not a neat approach, but makes sense since xen controlled
  > cpufreq is sure a better model. You can enable CPU_FREQ in
  > dom0 manually though, by removing below dependency.

    > (drivers/cpufreq/Kconfig)
    > config CPU_FREQ
    >        bool "CPU Frequency scaling"
    > ---->  depends on !PROCESSOR_EXTERNAL_CONTROL

  > However once that dependency is removed, there's no automatic
  > check to avoid confliction when dom0 and xen both want to
  > control freq change, and then you should keep an eye yourself.


For AMD, pre-Family 10 CPUs you are supposed to use dom0-kernel, where you can encounter problems
with TSC drifts. For other CPUs, I have no knowledge. Family 10 CPUs will have P-State invariant
TSCs and will be should be supported by xen cpufreq. C-State handling is switched on by cpuidle as
parameter.
Note: xenpm is used to handle xen cpufreq support. In dom0-kernel, you use the dom0 tools.

BR,
Carsten.

----- Originalnachricht -----
Von: Akio Takebe <takebe_akio@jp.fujitsu.com>
Gesendet: Mit, 14.1.2009 15:32
An: xen-devel <xen-devel@lists.xensource.com> ; "Tian, Kevin" <kevin.tian@intel.com> ; Yu Ke <ke.yu@intel.com>
Betreff: [Xen-devel] How to use Px/Cx for power saving?

Hi,

If we want to use ACPI power saving(Cx/Px), how do we use them?
I tried to turn CONFIG_xxxx of cpufreq  on with linux-2.6.18-xen.hg,
but I couldn't turn them on.
Also xenpm said the following error.

# xenpm get-cpuidle-states
Xen cpuidle is not enabled!
# xenpm get-cpufreq-states
Xen cpufreq is not enabled!

What should I do?
If there are some good documents, please tell me.

Best Regards,

Akio Takebe

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: AW: How to use Px/Cx for power saving?

[Akio Takebe]

Hi, Carsten

Thank you for your infomation.
I didn't check the mail, sorry.
It's helpful. Is FREQCTL_none the default value?
If we use FREQCTL_dom0_kernel, xen/guests don't use cpufreq?

Best Regards,

Akio Takebe

Carsten Schiers wrote:
> Hi,
> 
> cpufreq support can be either handled by xen or dom0-kernel. You have to specify cpufreq=xen or
> cpufreq=dom0-kernel as parameters when starting the hypervisor.
> 
> Kevin Tian wrote yesterday:
> 
>   > It's intentional since Xen itself supports cpufreq now. To avoid 
>   > dependency on dom0's CONFIG_CPU_FREQ and also intrusive 
>   > checks on whether external control is enabled in every linux
>   > cpufreq drivers, CPU_FREQ in dom0 is silented by default now.
>   > It's not a neat approach, but makes sense since xen controlled
>   > cpufreq is sure a better model. You can enable CPU_FREQ in
>   > dom0 manually though, by removing below dependency.
> 
>     > (drivers/cpufreq/Kconfig)
>     > config CPU_FREQ
>     >        bool "CPU Frequency scaling"
>     > ---->  depends on !PROCESSOR_EXTERNAL_CONTROL
> 
>   > However once that dependency is removed, there's no automatic
>   > check to avoid confliction when dom0 and xen both want to
>   > control freq change, and then you should keep an eye yourself.
> 
> 
> For AMD, pre-Family 10 CPUs you are supposed to use dom0-kernel, where you can encounter problems
> with TSC drifts. For other CPUs, I have no knowledge. Family 10 CPUs will have P-State invariant
> TSCs and will be should be supported by xen cpufreq. C-State handling is switched on by cpuidle as
> parameter.
> Note: xenpm is used to handle xen cpufreq support. In dom0-kernel, you use the dom0 tools.
> 
> BR,
> Carsten.
> 
> ----- Originalnachricht -----
> Von: Akio Takebe <takebe_akio@jp.fujitsu.com>
> Gesendet: Mit, 14.1.2009 15:32
> An: xen-devel <xen-devel@lists.xensource.com> ; "Tian, Kevin" <kevin.tian@intel.com> ; Yu Ke <ke.yu@intel.com>
> Betreff: [Xen-devel] How to use Px/Cx for power saving?
> 
> Hi,
> 
> If we want to use ACPI power saving(Cx/Px), how do we use them?
> I tried to turn CONFIG_xxxx of cpufreq  on with linux-2.6.18-xen.hg,
> but I couldn't turn them on.
> Also xenpm said the following error.
> 
> # xenpm get-cpuidle-states
> Xen cpuidle is not enabled!
> # xenpm get-cpufreq-states
> Xen cpufreq is not enabled!
> 
> What should I do?
> If there are some good documents, please tell me.
> 
> Best Regards,
> 
> Akio Takebe
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

RE: AW: How to use Px/Cx for power saving?

[Tian, Kevin]

Which version are you using? In latest Xen unstable, Xen controlled 
cpufreq is always enabled with userspace governor as the default. 
In such case, you could use xenpm to manually change frequency 
(xenpm set-scaling-speed). Then "xenpm set-scaling-governor"
can be used to choose a new governor like ondemand in the fly.
Before 18950, cpufreq is by default disabled, and you have to add
'cpufreq=xen' in grub to activate xen controlled cpufreq logic. In 
either case, CONFIG_CPU_FREQ in dom0 is not required.

'cpufreq=dom0-kernel' can be always used to allow dom0 control 
freq directly. In such case, Xen itself exits the game, and then you
have to follow below trick to enable CONFIG_CPU_FREQ in dom0
kernel.

For cpuidle, you need a similar boot option 'cpuidle' in xen cmdline
which is off by default. We have plan to allow in-the-fly on/off from
xenpm too.

One of our engineer is working on document side, including all
relevant options and xenpm usage. We tempt to place it on Xen
wiki first, and will let you know for comments. :-)

Thanks,
Kevin

>From: Akio Takebe [mailto:takebe_akio@jp.fujitsu.com] 
>Sent: Thursday, January 15, 2009 8:02 AM
>
>Hi, Carsten
>
>Thank you for your infomation.
>I didn't check the mail, sorry.
>It's helpful. Is FREQCTL_none the default value?
>If we use FREQCTL_dom0_kernel, xen/guests don't use cpufreq?
>
>Best Regards,
>
>Akio Takebe
>
>Carsten Schiers wrote:
>> Hi,
>> 
>> cpufreq support can be either handled by xen or dom0-kernel. 
>You have to specify cpufreq=xen or
>> cpufreq=dom0-kernel as parameters when starting the hypervisor.
>> 
>> Kevin Tian wrote yesterday:
>> 
>>   > It's intentional since Xen itself supports cpufreq now. To avoid 
>>   > dependency on dom0's CONFIG_CPU_FREQ and also intrusive 
>>   > checks on whether external control is enabled in every linux
>>   > cpufreq drivers, CPU_FREQ in dom0 is silented by default now.
>>   > It's not a neat approach, but makes sense since xen controlled
>>   > cpufreq is sure a better model. You can enable CPU_FREQ in
>>   > dom0 manually though, by removing below dependency.
>> 
>>     > (drivers/cpufreq/Kconfig)
>>     > config CPU_FREQ
>>     >        bool "CPU Frequency scaling"
>>     > ---->  depends on !PROCESSOR_EXTERNAL_CONTROL
>> 
>>   > However once that dependency is removed, there's no automatic
>>   > check to avoid confliction when dom0 and xen both want to
>>   > control freq change, and then you should keep an eye yourself.
>> 
>> 
>> For AMD, pre-Family 10 CPUs you are supposed to use 
>dom0-kernel, where you can encounter problems
>> with TSC drifts. For other CPUs, I have no knowledge. Family 
>10 CPUs will have P-State invariant
>> TSCs and will be should be supported by xen cpufreq. C-State 
>handling is switched on by cpuidle as
>> parameter.
>> Note: xenpm is used to handle xen cpufreq support. In 
>dom0-kernel, you use the dom0 tools.
>> 
>> BR,
>> Carsten.
>> 
>> ----- Originalnachricht -----
>> Von: Akio Takebe <takebe_akio@jp.fujitsu.com>
>> Gesendet: Mit, 14.1.2009 15:32
>> An: xen-devel <xen-devel@lists.xensource.com> ; "Tian, 
>Kevin" <kevin.tian@intel.com> ; Yu Ke <ke.yu@intel.com>
>> Betreff: [Xen-devel] How to use Px/Cx for power saving?
>> 
>> Hi,
>> 
>> If we want to use ACPI power saving(Cx/Px), how do we use them?
>> I tried to turn CONFIG_xxxx of cpufreq  on with linux-2.6.18-xen.hg,
>> but I couldn't turn them on.
>> Also xenpm said the following error.
>> 
>> # xenpm get-cpuidle-states
>> Xen cpuidle is not enabled!
>> # xenpm get-cpufreq-states
>> Xen cpufreq is not enabled!
>> 
>> What should I do?
>> If there are some good documents, please tell me.
>> 
>> Best Regards,
>> 
>> Akio Takebe
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>> 
>

Re: AW: How to use Px/Cx for power saving?

[Akio Takebe]

Hi, Kevin

Thank you for your infomation.

Tian, Kevin wrote:
> Which version are you using? In latest Xen unstable, Xen controlled 
> cpufreq is always enabled with userspace governor as the default. 
> In such case, you could use xenpm to manually change frequency 
> (xenpm set-scaling-speed). Then "xenpm set-scaling-governor"
> can be used to choose a new governor like ondemand in the fly.
> Before 18950, cpufreq is by default disabled, and you have to add
> 'cpufreq=xen' in grub to activate xen controlled cpufreq logic. In 
> either case, CONFIG_CPU_FREQ in dom0 is not required.
> 
Oh, I used cset:18942.

> 'cpufreq=dom0-kernel' can be always used to allow dom0 control 
> freq directly. In such case, Xen itself exits the game, and then you
> have to follow below trick to enable CONFIG_CPU_FREQ in dom0
> kernel.
>
If dom0_max_vcpus < total_phys_cpus, dom0 control only his cpus.
So other cpus don't enter px/cx state, right?
And you recommend cpufreq=xen?

> For cpuidle, you need a similar boot option 'cpuidle' in xen cmdline
> which is off by default. We have plan to allow in-the-fly on/off from
> xenpm too.
> 
Thanks.

> One of our engineer is working on document side, including all
> relevant options and xenpm usage. We tempt to place it on Xen
> wiki first, and will let you know for comments. :-)
> 
Great! It will be very helpful.

Best Regards,

Akio Takebe

RE: AW: How to use Px/Cx for power saving?

[Tian, Kevin]

>From: Akio Takebe [mailto:takebe_akio@jp.fujitsu.com] 
>Sent: Thursday, January 15, 2009 10:05 AM
>
>> 'cpufreq=dom0-kernel' can be always used to allow dom0 control 
>> freq directly. In such case, Xen itself exits the game, and then you
>> have to follow below trick to enable CONFIG_CPU_FREQ in dom0
>> kernel.
>>
>If dom0_max_vcpus < total_phys_cpus, dom0 control only his cpus.
>So other cpus don't enter px/cx state, right?
>And you recommend cpufreq=xen?

Yes, I'm recommending cpufreq=xen which is more efficient and
clearer method. For cpufreq=dom0-kernel, such implication exists
to have dom0 vcpus pinned to corresponding physical cpus with
same number. Unless your platform couldn't work with acpi-cpufreq
driver on native linux, you should always spin on cpufreq=xen. 

Thanks,
Kevin

AW: AW: How to use Px/Cx for power saving?

[Carsten Schiers]

Kevin, did I understand you right, that I have to tare care when using
cpufreq=dom-kernel to pin vcpus to physical cpus? I didn't do anythinh
specific, but xm vcpu-list says:

> Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
> Domain-0                             0     0     0   r--     303.0 0
> Domain-0                             0     1     1   -b-     225.2 1

which looks ok. What would be the implication, if there is a fault? Could
this be the reason for my TSC drifts mentioned elsewhere here on the list?

Also, is acpi_cpufreq an alternative to e.g. powernow-k8 or speedstep-centrino? 
Whereas powernow-k8 works on my system, acpi_cpufreq is reporting a FATAL: 
No such device. It's a quite new board, so I am not sure, whether this means the 
BIOS doesn't support it (my asumption is, acpi_cpufreq uses the BIOS instead of 
a CPU-based driver, like powernow-k8).

Thanks for helping me with all that stuff, it's quite complicated if you're new
to it.

BR,
Carsten.

----- Originalnachricht -----
Von: "Tian, Kevin" <kevin.tian@intel.com>
Gesendet: Don, 15.1.2009 03:40
An: 'Akio Takebe' <takebe_akio@jp.fujitsu.com>
Cc: Carsten Schiers <carsten@schiers.de> ; xen-devel <xen-devel@lists.xensource.com> ; "Yu, Ke" <ke.yu@intel.com>
Betreff: RE: AW: [Xen-devel] How to use Px/Cx for power saving?

>From: Akio Takebe [mailto:takebe_akio@jp.fujitsu.com] 
>Sent: Thursday, January 15, 2009 10:05 AM
>
>> 'cpufreq=dom0-kernel' can be always used to allow dom0 control 
>> freq directly. In such case, Xen itself exits the game, and then you
>> have to follow below trick to enable CONFIG_CPU_FREQ in dom0
>> kernel.
>>
>If dom0_max_vcpus < total_phys_cpus, dom0 control only his cpus.
>So other cpus don't enter px/cx state, right?
>And you recommend cpufreq=xen?

Yes, I'm recommending cpufreq=xen which is more efficient and
clearer method. For cpufreq=dom0-kernel, such implication exists
to have dom0 vcpus pinned to corresponding physical cpus with
same number. Unless your platform couldn't work with acpi-cpufreq
driver on native linux, you should always spin on cpufreq=xen. 

Thanks,
Kevin

RE: AW: How to use Px/Cx for power saving?

[Tian, Kevin]

>From: Carsten Schiers [mailto:carsten@schiers.de] 
>Sent: Thursday, January 15, 2009 6:54 PM
>
>Kevin, did I understand you right, that I have to tare care when using
>cpufreq=dom-kernel to pin vcpus to physical cpus? I didn't do anythinh
>specific, but xm vcpu-list says:
>
>> Name                                ID  VCPU   CPU State   
>Time(s) CPU Affinity
>> Domain-0                             0     0     0   r--     303.0 0
>> Domain-0                             0     1     1   -b-     225.2 1
>
>which looks ok. What would be the implication, if there is a 
>fault? Could
>this be the reason for my TSC drifts mentioned elsewhere here 
>on the list?

It's implicitly enforced within Xen once you have dom0-kernel to
control cpufreq.

>
>Also, is acpi_cpufreq an alternative to e.g. powernow-k8 or 
>speedstep-centrino? 

acpi_cpufreq is a more generic driver as long as corresponding
ACPI tables are reported, which is the default one now in native
Linux kernel. If underlying BIOS doesn't conform to ACPI spec,
then you need try specific cpufreq driver like speedstep-centrino
then.

although xen controls cpufreq itself now, xen still depends on
dom0's ACPI parser to retrieve necessary Px information. That's
why I said that xen controlled cpufreq should work if you could
use acpi_cpufreq driver on native Linux which indicates a ACPI
compliant system.

>Whereas powernow-k8 works on my system, acpi_cpufreq is 
>reporting a FATAL: 
>No such device. It's a quite new board, so I am not sure, 
>whether this means the 
>BIOS doesn't support it (my asumption is, acpi_cpufreq uses 
>the BIOS instead of 
>a CPU-based driver, like powernow-k8).

I don't know AMD platform. But on most Intel platforms 
acpi_cpufreq should work well on native and so does Xen side.

Thanks,
Kevin

RE: AW: How to use Px/Cx for power saving?

[Langsdorf, Mark]

> Also, is acpi_cpufreq an alternative to e.g. powernow-k8 or 
> speedstep-centrino? 

It is for Intel parts, but AMD power management does not
quite follow the ACPI specification and acpi-cpufreq does
not support it.

For AMD parts, use powernow-k8.

-Mark Langsdorf
Operating System Research Center
AMD