[PATCH] mechanism to disable built in cpu frequency drivers/governors. (v1)

[PATCH] mechanism to disable built in cpu frequency drivers/governors. (v1)

[Konrad Rzeszutek Wilk]

The purpose of these patches is two-fold:
 1) Remove weird performance issues encountered in the field where the
    Linux kernel cpufreq scaling drivers would change the P-states while the
    Xen hypervisor would do the same.
 2) Provide a runtime knob from within the kernel to disable the cpu frequency
    API (akin to the behavior of "#CONFIG_CPU_CPUFREQ is not set").

The patches are based on the mechanism that Len Brown came up for the cpuidle
drivers - where there is a disable_cpuidle() call. His patches also provided
a module parameter - but I wasn't sure whether that is something you would want.
(for reference, Len's patch is d91ee5863b71e8c90eaf6035bff3078a85e2e7b5)

The other way of disabling the cpufreq scaling drivers from being invoked
is by thwarting the cpufreq drivers from loading by influencing their startup
checks.  For powernow-k8 that is easy - disallow the RDMSR (and that is currently
happening). For acpi-cpufreq I am not sure what could be done - the calls it
makes are to the ACPI libraries which we need for other uses.

Another solution I thought off was to provide a cpufreq governor that would
be a nop, and hence end up not calling the cpufreq scaling drivers. But that
just seems heavy-handed and I am not sure how the decision of when to load it
(and set it) would be done.

 arch/x86/xen/setup.c      |    1 +
 drivers/cpufreq/cpufreq.c |   24 ++++++++++++++++++++++++
 include/linux/cpufreq.h   |    2 ++
 3 files changed, 27 insertions(+), 0 deletions(-)

[PATCH 1/2] [CPUFREQ]: provide disable_cpufreq() function to disable the API.

[Konrad Rzeszutek Wilk]

useful for disabling cpufreq altogether. The cpu frequency
scaling drivers and cpu frequency governors will fail to register.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/cpufreq/cpufreq.c |   24 ++++++++++++++++++++++++
 include/linux/cpufreq.h   |    2 ++
 2 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 622013f..7f2f149 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -126,6 +126,15 @@ static int __init init_cpufreq_transition_notifier_list(void)
 }
 pure_initcall(init_cpufreq_transition_notifier_list);
 
+static int off __read_mostly;
+int cpufreq_disabled(void)
+{
+	return off;
+}
+void disable_cpufreq(void)
+{
+	off = 1;
+}
 static LIST_HEAD(cpufreq_governor_list);
 static DEFINE_MUTEX(cpufreq_governor_mutex);
 
@@ -1441,6 +1450,9 @@ int __cpufreq_driver_target(struct cpufreq_policy *policy,
 {
 	int retval = -EINVAL;
 
+	if (cpufreq_disabled())
+		return -ENODEV;
+
 	pr_debug("target for CPU %u: %u kHz, relation %u\n", policy->cpu,
 		target_freq, relation);
 	if (cpu_online(policy->cpu) && cpufreq_driver->target)
@@ -1549,6 +1561,9 @@ int cpufreq_register_governor(struct cpufreq_governor *governor)
 	if (!governor)
 		return -EINVAL;
 
+	if (cpufreq_disabled())
+		return -ENODEV;
+
 	mutex_lock(&cpufreq_governor_mutex);
 
 	err = -EBUSY;
@@ -1572,6 +1587,9 @@ void cpufreq_unregister_governor(struct cpufreq_governor *governor)
 	if (!governor)
 		return;
 
+	if (cpufreq_disabled())
+		return;
+
 #ifdef CONFIG_HOTPLUG_CPU
 	for_each_present_cpu(cpu) {
 		if (cpu_online(cpu))
@@ -1814,6 +1832,9 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data)
 	unsigned long flags;
 	int ret;
 
+	if (cpufreq_disabled())
+		return -ENODEV;
+
 	if (!driver_data || !driver_data->verify || !driver_data->init ||
 	    ((!driver_data->setpolicy) && (!driver_data->target)))
 		return -EINVAL;
@@ -1901,6 +1922,9 @@ static int __init cpufreq_core_init(void)
 {
 	int cpu;
 
+	if (cpufreq_disabled())
+		return -ENODEV;
+
 	for_each_possible_cpu(cpu) {
 		per_cpu(cpufreq_policy_cpu, cpu) = -1;
 		init_rwsem(&per_cpu(cpu_policy_rwsem, cpu));
diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h
index 6216115..8ff4427 100644
--- a/include/linux/cpufreq.h
+++ b/include/linux/cpufreq.h
@@ -35,6 +35,7 @@
 #ifdef CONFIG_CPU_FREQ
 int cpufreq_register_notifier(struct notifier_block *nb, unsigned int list);
 int cpufreq_unregister_notifier(struct notifier_block *nb, unsigned int list);
+extern void disable_cpufreq(void);
 #else		/* CONFIG_CPU_FREQ */
 static inline int cpufreq_register_notifier(struct notifier_block *nb,
 						unsigned int list)
@@ -46,6 +47,7 @@ static inline int cpufreq_unregister_notifier(struct notifier_block *nb,
 {
 	return 0;
 }
+static inline void disable_cpufreq(void) { }
 #endif		/* CONFIG_CPU_FREQ */
 
 /* if (cpufreq_driver->target) exists, the ->governor decides what frequency
-- 
1.7.9.48.g85da4d

[PATCH 2/2] xen/cpufreq: Disable the cpu frequency scaling drivers from loading.

[Konrad Rzeszutek Wilk]

By using the functionality provided by "[CPUFREQ]: provide
disable_cpuidle() function to disable the API."

Under the Xen hypervisor we do not want the initial domain to exercise
the cpufreq scaling drivers. This is b/c the Xen hypervisor is
in charge of doing this as well and we can end up with both the
Linux kernel and the hypervisor trying to change the P-states
leading to weird performance issues.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/xen/setup.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index e03c636..4461e0a 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -420,6 +420,7 @@ void __init xen_arch_setup(void)
 	boot_cpu_data.hlt_works_ok = 1;
 #endif
 	disable_cpuidle();
+	disable_cpufreq();
 	boot_option_idle_override = IDLE_HALT;
 	WARN_ON(set_pm_idle_to_default());
 	fiddle_vdso();
-- 
1.7.9.48.g85da4d

Re: [PATCH] mechanism to disable built in cpu frequency drivers/governors. (v1)

[Jan Beulich]

>>> On 14.03.12 at 00:18, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> The purpose of these patches is two-fold:
>  1) Remove weird performance issues encountered in the field where the
>     Linux kernel cpufreq scaling drivers would change the P-states while the
>     Xen hypervisor would do the same.
>  2) Provide a runtime knob from within the kernel to disable the cpu 
> frequency
>     API (akin to the behavior of "#CONFIG_CPU_CPUFREQ is not set").

Acked-by: Jan Beulich <jbeulich@suse.com>

(Albeit I'd recommend using 'bool' for the variable and function return
types in the first patch.)

> The patches are based on the mechanism that Len Brown came up for the 
> cpuidle
> drivers - where there is a disable_cpuidle() call. His patches also provided
> a module parameter - but I wasn't sure whether that is something you would 
> want.
> (for reference, Len's patch is d91ee5863b71e8c90eaf6035bff3078a85e2e7b5)
> 
> The other way of disabling the cpufreq scaling drivers from being invoked
> is by thwarting the cpufreq drivers from loading by influencing their 
> startup
> checks.  For powernow-k8 that is easy - disallow the RDMSR (and that is 
> currently
> happening). For acpi-cpufreq I am not sure what could be done - the calls it
> makes are to the ACPI libraries which we need for other uses.
> 
> Another solution I thought off was to provide a cpufreq governor that would
> be a nop, and hence end up not calling the cpufreq scaling drivers. But that
> just seems heavy-handed and I am not sure how the decision of when to load it
> (and set it) would be done.
> 
>  arch/x86/xen/setup.c      |    1 +
>  drivers/cpufreq/cpufreq.c |   24 ++++++++++++++++++++++++
>  include/linux/cpufreq.h   |    2 ++
>  3 files changed, 27 insertions(+), 0 deletions(-)

Re: [PATCH 1/2] [CPUFREQ]: provide disable_cpufreq() function to disable the API.

[Dave Jones]

On Tue, Mar 13, 2012 at 07:18:39PM -0400, Konrad Rzeszutek Wilk wrote:
 > useful for disabling cpufreq altogether. The cpu frequency
 > scaling drivers and cpu frequency governors will fail to register.

looks good to me. Are you ok with me taking 1/2 through cpufreq.git, and
then you pushing the xen bit yourself ?
(or I could take them both if it makes things easier)

	Dave

Re: [PATCH 1/2] [CPUFREQ]: provide disable_cpufreq() function to disable the API.

[Konrad Rzeszutek Wilk]

On Wed, Mar 14, 2012 at 10:25:43AM -0400, Dave Jones wrote:
> On Tue, Mar 13, 2012 at 07:18:39PM -0400, Konrad Rzeszutek Wilk wrote:
>  > useful for disabling cpufreq altogether. The cpu frequency
>  > scaling drivers and cpu frequency governors will fail to register.
> 
> looks good to me. Are you ok with me taking 1/2 through cpufreq.git, and
> then you pushing the xen bit yourself ?

No trouble! Thanks!
> (or I could take them both if it makes things easier)
> 
> 	Dave