[PATCH] cpufreq: Initialize cpufreq-based invariance before subsys

[PATCH] cpufreq: Initialize cpufreq-based invariance before subsys

[Christian Loehle]

commit 2a6c72738706 ("cpufreq: Initialize cpufreq-based
frequency-invariance later") postponed the frequency invariance
initialization to avoid disabling it in the error case.
This isn't locking safe, instead move the initialization up before
the subsys interface is registered (which will rebuild the
sched_domains) and add the corresponding disable on the error path.

Observed lockdep without this patch:
[    0.989686] ======================================================
[    0.989688] WARNING: possible circular locking dependency detected
[    0.989690] 6.17.0-rc4-cix-build+ #31 Tainted: G S
[    0.989691] ------------------------------------------------------
[    0.989692] swapper/0/1 is trying to acquire lock:
[    0.989693] ffff800082ada7f8 (sched_energy_mutex){+.+.}-{4:4}, at: rebuild_sched_domains_energy+0x30/0x58
[    0.989705]
               but task is already holding lock:
[    0.989706] ffff000088c89bc8 (&policy->rwsem){+.+.}-{4:4}, at: cpufreq_online+0x7f8/0xbe0
[    0.989713]
               which lock already depends on the new lock.

Fixes: 2a6c72738706 ("cpufreq: Initialize cpufreq-based frequency-invariance later")
Signed-off-by: Christian Loehle <christian.loehle@arm.com>
---
 drivers/cpufreq/cpufreq.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index fc7eace8b65b..58e3839a2140 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -2953,6 +2953,15 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data)
 			goto err_null_driver;
 	}
 
+	/*
+	 * Mark support for the scheduler's frequency invariance engine for
+	 * drivers that implement target(), target_index() or fast_switch().
+	 */
+	if (!cpufreq_driver->setpolicy) {
+		static_branch_enable_cpuslocked(&cpufreq_freq_invariance);
+		pr_debug("cpufreq: supports frequency invariance\n");
+	}
+
 	ret = subsys_interface_register(&cpufreq_interface);
 	if (ret)
 		goto err_boost_unreg;
@@ -2974,21 +2983,14 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data)
 	hp_online = ret;
 	ret = 0;
 
-	/*
-	 * Mark support for the scheduler's frequency invariance engine for
-	 * drivers that implement target(), target_index() or fast_switch().
-	 */
-	if (!cpufreq_driver->setpolicy) {
-		static_branch_enable_cpuslocked(&cpufreq_freq_invariance);
-		pr_debug("supports frequency invariance");
-	}
-
 	pr_debug("driver %s up and running\n", driver_data->name);
 	goto out;
 
 err_if_unreg:
 	subsys_interface_unregister(&cpufreq_interface);
 err_boost_unreg:
+	if (!cpufreq_driver->setpolicy)
+		static_branch_disable_cpuslocked(&cpufreq_freq_invariance);
 	remove_boost_sysfs_file();
 err_null_driver:
 	write_lock_irqsave(&cpufreq_driver_lock, flags);
-- 
2.34.1

Re: [PATCH] cpufreq: Initialize cpufreq-based invariance before subsys

[Rafael J. Wysocki]

On Thu, Sep 18, 2025 at 12:15 PM Christian Loehle
<christian.loehle@arm.com> wrote:
>
> commit 2a6c72738706 ("cpufreq: Initialize cpufreq-based
> frequency-invariance later") postponed the frequency invariance
> initialization to avoid disabling it in the error case.
> This isn't locking safe, instead move the initialization up before
> the subsys interface is registered (which will rebuild the
> sched_domains) and add the corresponding disable on the error path.
>
> Observed lockdep without this patch:
> [    0.989686] ======================================================
> [    0.989688] WARNING: possible circular locking dependency detected
> [    0.989690] 6.17.0-rc4-cix-build+ #31 Tainted: G S
> [    0.989691] ------------------------------------------------------
> [    0.989692] swapper/0/1 is trying to acquire lock:
> [    0.989693] ffff800082ada7f8 (sched_energy_mutex){+.+.}-{4:4}, at: rebuild_sched_domains_energy+0x30/0x58
> [    0.989705]
>                but task is already holding lock:
> [    0.989706] ffff000088c89bc8 (&policy->rwsem){+.+.}-{4:4}, at: cpufreq_online+0x7f8/0xbe0
> [    0.989713]
>                which lock already depends on the new lock.
>
> Fixes: 2a6c72738706 ("cpufreq: Initialize cpufreq-based frequency-invariance later")
> Signed-off-by: Christian Loehle <christian.loehle@arm.com>
> ---
>  drivers/cpufreq/cpufreq.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index fc7eace8b65b..58e3839a2140 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -2953,6 +2953,15 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data)
>                         goto err_null_driver;
>         }
>
> +       /*
> +        * Mark support for the scheduler's frequency invariance engine for
> +        * drivers that implement target(), target_index() or fast_switch().
> +        */
> +       if (!cpufreq_driver->setpolicy) {
> +               static_branch_enable_cpuslocked(&cpufreq_freq_invariance);
> +               pr_debug("cpufreq: supports frequency invariance\n");
> +       }
> +
>         ret = subsys_interface_register(&cpufreq_interface);
>         if (ret)
>                 goto err_boost_unreg;
> @@ -2974,21 +2983,14 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data)
>         hp_online = ret;
>         ret = 0;
>
> -       /*
> -        * Mark support for the scheduler's frequency invariance engine for
> -        * drivers that implement target(), target_index() or fast_switch().
> -        */
> -       if (!cpufreq_driver->setpolicy) {
> -               static_branch_enable_cpuslocked(&cpufreq_freq_invariance);
> -               pr_debug("supports frequency invariance");
> -       }
> -
>         pr_debug("driver %s up and running\n", driver_data->name);
>         goto out;
>
>  err_if_unreg:
>         subsys_interface_unregister(&cpufreq_interface);
>  err_boost_unreg:
> +       if (!cpufreq_driver->setpolicy)
> +               static_branch_disable_cpuslocked(&cpufreq_freq_invariance);
>         remove_boost_sysfs_file();
>  err_null_driver:
>         write_lock_irqsave(&cpufreq_driver_lock, flags);
> --

Applied as 6.17-rc material, thanks!