Disable sleep states on P7+

Disable sleep states on P7+

[Steven Pratt]

I am looking for info on when and how we are able to disable power saving features of current (P7, P7+) chips in order to reduce latency. This is often done in latency sensitive applications when power consumption is not an issue. On Intel boxes we can disable P-state frequency changes as well as disabling C-State or sleep state changes. In fact we can control how deep a sleep the processor can go into.  I know we have control Dynamic Processor Scaling and Idle Power Savings, but what states do these really affect?  Can I really disable Nap mode of a processor? If so how?  Can I disable even the lightest winkle mode?  Looking for current information (read RHEL 6 and SLES11), future changes are interesting.

Steve

Re: Disable sleep states on P7+

[Preeti U Murthy]

Hi Steven,

On 01/14/2014 08:06 PM, Steven Pratt wrote:
> I am looking for info on when and how we are able to disable power saving features of current (P7, P7+) chips in order to reduce latency. This is often done in latency sensitive applications when power consumption is not an issue. On Intel boxes we can disable P-state frequency changes as well as disabling C-State or sleep state changes. In fact we can control how deep a sleep the processor can go into.  I know we have control Dynamic Processor Scaling and Idle Power Savings, but what states do these really affect?  Can I really disable Nap mode of a processor? If so how?  Can I disable even the lightest winkle mode?  Looking for current information (read RHEL 6 and SLES11), future changes are interesting.
> 
> Steve

I can answer this question with respect to cpuidle on PowerNV platforms.

1. In order to disable cpuidle states management altogether, one can
pass the powersave=off kernel cmd line parameter during boot up of the
kernel. This will ensure that each time a CPU has nothing to do, it can
enter low thread priority which could lower power consumption to some
extent but is not expected to hit latency of applications noticeably.

2. In order to exactly control the cpuidle states into which idle CPUs
can enter into during runtime, one can make use of the sysfs files under:
/sys/devices/system/cpu/cpux/cpuidle/statex/disable option to
selectively disable any state.

However if one is using the menu cpuidle governor, disabling an idle
state does not disable the idle states which are deeper than it. They
continue to remain active unless they are specifically disabled. What
this means is that one cannot control the depth of the idle states
available for a CPU, although we can control the exact idle states
available for a processor.

But if the ladder governor is used, one can control the depth of the
idle states that a CPU can enter into. The governor can be chosen by
echoing either menu/ladder to
/sys/devices/system/cpu/cpuidle/current_governor_ro. The cpuidle
governor takes decisions about the idle state for a cpu to enter into
depending on its idle history. The popular governor used by most archs
is the menu governor.

Hence nap/sleep/winkle any of these states can be disabled. The code
which enables the above mentioned functionalities on powernv is yet to
go upstream although the same is already upstream and can be used for
the pseries platform to disable/enable the idle states on it.

Today on powernv the default idle state nap is entered into all the
time. One can disable it by echoing 0 to powersave_nap under
/proc/sys/kernel/powersave_nap, in which case the cpu enters low thread
priority.

Thanks

Regards
Preeti U Murthy

> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
> 

Re: Disable sleep states on P7+

[Steven Pratt]

On 01/14/2014 10:10 AM, Preeti U Murthy wrote:
> Hi Steven,
>
> On 01/14/2014 08:06 PM, Steven Pratt wrote:
>> I am looking for info on when and how we are able to disable power saving features of current (P7, P7+) chips in order to reduce latency. This is often done in latency sensitive applications when power consumption is not an issue. On Intel boxes we can disable P-state frequency changes as well as disabling C-State or sleep state changes. In fact we can control how deep a sleep the processor can go into.  I know we have control Dynamic Processor Scaling and Idle Power Savings, but what states do these really affect?  Can I really disable Nap mode of a processor? If so how?  Can I disable even the lightest winkle mode?  Looking for current information (read RHEL 6 and SLES11), future changes are interesting.
>>
>> Steve
> I can answer this question with respect to cpuidle on PowerNV platforms.
>
> 1. In order to disable cpuidle states management altogether, one can
> pass the powersave=off kernel cmd line parameter during boot up of the
> kernel. This will ensure that each time a CPU has nothing to do, it can
> enter low thread priority which could lower power consumption to some
> extent but is not expected to hit latency of applications noticeably.
>
> 2. In order to exactly control the cpuidle states into which idle CPUs
> can enter into during runtime, one can make use of the sysfs files under:
> /sys/devices/system/cpu/cpux/cpuidle/statex/disable option to
> selectively disable any state.
>
> However if one is using the menu cpuidle governor, disabling an idle
> state does not disable the idle states which are deeper than it. They
> continue to remain active unless they are specifically disabled. What
> this means is that one cannot control the depth of the idle states
> available for a CPU, although we can control the exact idle states
> available for a processor.
>
> But if the ladder governor is used, one can control the depth of the
> idle states that a CPU can enter into. The governor can be chosen by
> echoing either menu/ladder to
> /sys/devices/system/cpu/cpuidle/current_governor_ro. The cpuidle
> governor takes decisions about the idle state for a cpu to enter into
> depending on its idle history. The popular governor used by most archs
> is the menu governor.
>
> Hence nap/sleep/winkle any of these states can be disabled. The code
> which enables the above mentioned functionalities on powernv is yet to
> go upstream although the same is already upstream and can be used for
> the pseries platform to disable/enable the idle states on it.
>
> Today on powernv the default idle state nap is entered into all the
> time. One can disable it by echoing 0 to powersave_nap under
> /proc/sys/kernel/powersave_nap, in which case the cpu enters low thread

Thanks, that is great information going forward, now I just need info on what works today in PowerVM.

Steve

> priority.
>
> Thanks
>
> Regards
> Preeti U Murthy
>
>> _______________________________________________
>> Linuxppc-dev mailing list
>> Linuxppc-dev@lists.ozlabs.org
>> https://lists.ozlabs.org/listinfo/linuxppc-dev
>>

Re: Disable sleep states on P7+

[Deepthi Dharwar]

Hi Steven,

On 01/14/2014 08:06 PM, Steven Pratt wrote:
> I am looking for info on when and how we are able to disable power saving features of current (P7, P7+) chips in order to reduce latency. This is often done in latency sensitive applications when

 power consumption is not an issue. On Intel boxes we can disable
P-state frequency changes as well as disabling C-State or sleep state
changes. In fact we can control how deep a sleep the processor can go
into.

I know we have control Dynamic Processor Scaling and Idle Power Savings,
but what states do these really affect?  Can I really disable Nap mode
of a processor? If so how?  Can I disable even the lightest winkle mode?

Looking for current information (read RHEL 6 and SLES11), future changes
are interesting.
> 

On POWERVM platforms idle states currently supported are:
Snooze - reducing thread priority.
Nap
Sleep.

Snooze and Nap can be controlled through in-band kernel mechanisms and
Sleep state through AEM.

Currently you can turn off Idle Power Savings mode and run in dynamic
processor scaling mode. By doing so you will disable entry into Sleep
state on all CPUS.

If you further want to disable NAP, then you could just boot the kernel
with powersave=off. This will disable the entry into any of the idle
state like nap and just reduce the priority of the thread when there is
no work to be done. This is part of cpuidle framework which is available
on SLES 11 SP3 and RHEL7.

In the newer kernels cpuidle framework is adopted for POWERVM platform
but in case if you are using RHEL 6 or SLES 11 SP1/2, then you could use
the ppc64_cpu util and set a high smt-snooze-delay value say 1000.first

#> ppc64_cpu --smt-snooze-delay=1000.

smt-snooze-delay variable potentially delays entry to NAP state.
So if the idle time predicted on a cpu = 1000us and smt-snooze-delay is
set to 100 (which is default value), then on RHEL 6 and SLES 11 SP1/2
kernels cpus would reduce the thread priority and spin for  first 100us
and if the cpu continues to be idle further then automatically go to NAP
state for remaining (1000-100us) time.

By setting a very high value, one would always be looping  This could
potentially delay ure entry to NAP state and effectively disable entry
into NAP state most of the time.

Please let me know if you have any queries around it.

Regards,
Deepthi






> Steve
> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
>