[PATCH v4 0/1] drm/amd/pm: restore the forced performance level after a reset

[PATCH v4 0/1] drm/amd/pm: restore the forced performance level after a reset

[Priya Hosur]

After a GPU reset the level reported through
power_dpm_force_performance_level no longer matches the hardware. The reset
reinitialises PMFW, which comes back on its default curve, while
smu_dpm.dpm_level still holds the level the user forced. On an SMU 14.0.1
APU with the level set to high, gfxclk drops from 2.8GHz to 600MHz while
sysfs still reports high.

amdgpu_dpm_reapply_umd_profile() already replays the retained level after
reset and resume, but only for the PROFILE_* levels.

v4 follows Lijo's review of v3 and restores every level through that
common path, renamed to amdgpu_dpm_reapply_performance_level(). The
PROFILE_* levels keep their UMD pstate handling, other levels only replay
the level itself. smu_reapply_power_state_dynamic() now calls
smu_adjust_power_state_dynamic(smu, level, true, true).

Tested on an SMU 14.0.1 APU (PMFW 100.113.0) on amd-staging-drm-next,
triggering resets through debugfs/amdgpu_gpu_recover and reading the level
and gfxclk before and after each reset.

Changing smu_reapply_power_state_dynamic() alone does not restore high, as
the reapply path returns early for it:

  level              before    reset 1   reset 2   reset 3
  high               2448MHz    630MHz    600MHz    600MHz

With this patch:

  level              before    reset 1   reset 2   reset 3
  high               2444MHz   2794MHz   2796MHz   2796MHz
  low                1403MHz    604MHz    600MHz    600MHz
  manual              600MHz    600MHz    600MHz    600MHz
  profile_peak       1848MHz   2237MHz   2019MHz   2318MHz
  profile_standard   1265MHz    703MHz    700MHz    700MHz
  auto                637MHz    600MHz    600MHz    600MHz

The level is retained across every reset. The "before" sample is taken one
second after the level is written. 18 MODE2 resets over the run, with no
new messages in the kernel log.

As the reapply path also runs on resume, s2idle was checked as well: high
2668MHz before suspend and 2799MHz after resume, profile_peak 2508MHz
before and 2266MHz after, with the level retained in both cases.

Notes:
  - auto is not replayed, as PMFW comes back in auto after a reset, as it
    does at probe. This also keeps the probe path, which runs the same
    function, unchanged.
  - Replaying manual does not program any clocks. The user clock masks set
    in manual are restored by smu_restore_dpm_user_profile(), which only
    runs on resume; restoring them after a reset is not part of this patch.
  - Only the swSMU path was tested. The powerplay path takes the same
    route through pp_dpm_reapply_performance_level() but I have no
    hardware to test it on, nor S3 or runtime PM resume.

v4:
  - Restore the level through the common reapply path instead of
    smu_handle_task(), so the level is restored in one place (Lijo)
  - Rename amdgpu_dpm_reapply_umd_profile() to
    amdgpu_dpm_reapply_performance_level() and replay non-profile levels
    directly (Lijo)
  - Replace the body of smu_reapply_power_state_dynamic() with
    smu_adjust_power_state_dynamic(smu, level, true, true) (Lijo)
  - Add the Fixes tag (Kevin)
  - Add the version changelog to the commit message
  - Kevin's Reviewed-by on v3 is not carried over, as it was given for
    his suggested change in smu_handle_task(), which v4 does not use
  - Add test results for the smu_reapply_power_state_dynamic() change alone
  - Link: https://lore.kernel.org/amd-gfx/cover.1793689935.git.Priya.Hosur@amd.com/

v3:
  - Use the existing force_performance_level argument of
    smu_adjust_power_state_dynamic() for AMD_PP_TASK_COMPLETE_INIT during
    reset recovery, instead of switching the cached level to AUTO first
  - Rebased on amd-staging-drm-next, which now includes Kevin's
    "drm/amd/pm: reapply umd profile state after recovery"
  - Explain in the commit message why that path does not cover high,
    low and manual (Kevin's question on v2)

v2:
  - Restore the forced level after the reset instead of reporting the
    default, as the driver is expected to keep the level the user asked
    for and the reset often goes unnoticed (Lijo)
  - Link: https://lore.kernel.org/amd-gfx/20261008104026.4102222-1-Priya.Hosur@amd.com/

v1:
  - Reset the cached level to AUTO after a reset so that sysfs reports the
    level PMFW is actually running at
  - Link: https://lore.kernel.org/amd-gfx/20261007122103.4019578-1-Priya.Hosur@amd.com/

Priya Hosur (1):
  drm/amd/pm: restore the forced performance level after a reset

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
 drivers/gpu/drm/amd/pm/amdgpu_dpm.c        | 21 +++++++++---
 drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h    |  2 +-
 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c  | 39 ++++------------------
 4 files changed, 26 insertions(+), 40 deletions(-)

-- 
2.43.0

[PATCH v4 1/1] drm/amd/pm: restore the forced performance level after a reset

[Priya Hosur]

After a GPU reset PMFW is reinitialised and comes back on its default
curve, while smu_dpm.dpm_level still holds the level the user forced. On
an SMU 14.0.1 APU with the level set to high, gfxclk drops from 2.8GHz to
600MHz while sysfs still reports high. Writing the same level again does
not help, as amdgpu_dpm_force_performance_level() returns early when the
requested level matches the cached one.

smu_late_init() does not re-program the level either, as the
AMD_PP_TASK_COMPLETE_INIT path compares the cached level against itself.

amdgpu_dpm_reapply_umd_profile() replays the retained level after reset
and resume, but only for the PROFILE_* levels, and returns early for any
other level.

Restore every level through this common path and rename it to
amdgpu_dpm_reapply_performance_level(). The PROFILE_* levels keep their
UMD pstate handling, other levels only replay the level itself. auto is
skipped, as PMFW comes back in auto, as it does at probe. If replaying a
non-profile level fails, the failure is logged and the level the user
asked for is kept.

smu_reapply_power_state_dynamic() duplicates
smu_adjust_power_state_dynamic() with the display settings skipped and
the level forced, so call that instead.

v2: restore the forced level instead of reporting the default after the
    reset (Lijo).

v3: use the force_performance_level argument of
    smu_adjust_power_state_dynamic() instead of switching the cached level
    to auto first.

v4: restore the level through the common reapply path instead of
    smu_handle_task(), rename amdgpu_dpm_reapply_umd_profile() to
    amdgpu_dpm_reapply_performance_level(), replay non-profile levels
    directly and replace the body of smu_reapply_power_state_dynamic()
    (Lijo). Add the Fixes tag (Kevin).

Fixes: 580ad7cbd4b7 ("drm/amdgpu/swsmu: Only force workload setup on init")
Suggested-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Priya Hosur <Priya.Hosur@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
 drivers/gpu/drm/amd/pm/amdgpu_dpm.c        | 21 +++++++++---
 drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h    |  2 +-
 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c  | 39 ++++------------------
 4 files changed, 26 insertions(+), 40 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 5069042b5279..2f7ef4bb2a65 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -2774,9 +2774,9 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
 	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
 	amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
 
-	r = amdgpu_dpm_reapply_umd_profile(adev);
+	r = amdgpu_dpm_reapply_performance_level(adev);
 	if (r) {
-		dev_err(adev->dev, "Failed to restore UMD profile state: %d\n", r);
+		dev_err(adev->dev, "Failed to restore performance level: %d\n", r);
 		return r;
 	}
 
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
index 75d8bb4868e8..ea698fe48e07 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
@@ -1083,14 +1083,16 @@ static void amdgpu_dpm_exit_umd_state(struct amdgpu_device *adev)
 }
 
 /**
- * amdgpu_dpm_reapply_umd_profile - Restore the active UMD profile
+ * amdgpu_dpm_reapply_performance_level - Restore the retained DPM level
  * @adev: amdgpu device
  *
  * Resume and reset retain the selected DPM level in software, but the
- * underlying GFX and PMFW state is lost. Reapply a retained UMD profile
- * after the common resume path has restored its default CG and PG state.
+ * underlying GFX and PMFW state is lost. Reapply the retained level after
+ * the common resume path has restored its default CG and PG state. UMD
+ * profile levels also restore their GFX state, other levels only replay
+ * the level itself.
  */
-int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
+int amdgpu_dpm_reapply_performance_level(struct amdgpu_device *adev)
 {
 	const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
 	enum amd_dpm_forced_level level;
@@ -1120,7 +1122,18 @@ int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
 	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
 	case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
 		break;
+	case AMD_DPM_FORCED_LEVEL_AUTO:
+		/* PMFW comes back in auto, as it does at probe. */
+		ret = 0;
+		goto unlock;
 	default:
+		/* No UMD pstate state to restore, only the level itself. */
+		ret = pp_funcs->reapply_performance_level(adev->powerplay.pp_handle,
+							     level);
+		if (ret)
+			dev_warn(adev->dev,
+				 "Failed to restore performance level %d: %d\n",
+				 level, ret);
 		ret = 0;
 		goto unlock;
 	}
diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
index ba8624d28f28..8ae3fcc016f8 100644
--- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
+++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
@@ -484,7 +484,7 @@ void amdgpu_dpm_set_power_state(struct amdgpu_device *adev,
 enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device *adev);
 int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
 				       enum amd_dpm_forced_level level);
-int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev);
+int amdgpu_dpm_reapply_performance_level(struct amdgpu_device *adev);
 int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev,
 				 struct pp_states_info *states);
 int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index a901f8535598..e45aeb2c565b 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -2507,33 +2507,7 @@ static int
 smu_reapply_power_state_dynamic(struct smu_context *smu,
 				enum amd_dpm_forced_level level)
 {
-	struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
-	int ret;
-
-	ret = smu_apply_clocks_adjust_rules(smu);
-	if (ret) {
-		dev_err(smu->adev->dev, "Failed to apply clocks adjust rules!");
-		return ret;
-	}
-
-	ret = smu_asic_set_performance_level(smu, level);
-	if (ret) {
-		if (ret == -EOPNOTSUPP)
-			dev_info(smu->adev->dev, "set performance level %d not supported",
-				 level);
-		else
-			dev_err(smu->adev->dev, "Failed to set performance level %d",
-				level);
-		return ret;
-	}
-
-	smu_dpm_ctx->dpm_level = level;
-
-	if (smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL ||
-	    smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM)
-		return 0;
-
-	return smu_bump_power_profile_mode(smu, NULL, 0);
+	return smu_adjust_power_state_dynamic(smu, level, true, true);
 }
 
 static int smu_handle_task(struct smu_context *smu,
@@ -2698,21 +2672,20 @@ smu_reapply_performance_level(void *handle,
 	if (!smu->is_apu && !smu_dpm_ctx->dpm_context)
 		return -EINVAL;
 
+	if (smu_dpm_ctx->dpm_level != level)
+		return -EINVAL;
+
 	switch (level) {
 	case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
 	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
 	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
 	case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+		smu_enter_umd_pstate(smu);
 		break;
 	default:
-		return -EINVAL;
+		break;
 	}
 
-	if (smu_dpm_ctx->dpm_level != level)
-		return -EINVAL;
-
-	smu_enter_umd_pstate(smu);
-
 	return smu_reapply_power_state_dynamic(smu, level);
 }
 
-- 
2.43.0

Re: [PATCH v4 1/1] drm/amd/pm: restore the forced performance level after a reset

[Lazar, Lijo]

On 04-Nov-26 3:52 PM, Priya Hosur wrote:
> After a GPU reset PMFW is reinitialised and comes back on its default
> curve, while smu_dpm.dpm_level still holds the level the user forced. On
> an SMU 14.0.1 APU with the level set to high, gfxclk drops from 2.8GHz to
> 600MHz while sysfs still reports high. Writing the same level again does
> not help, as amdgpu_dpm_force_performance_level() returns early when the
> requested level matches the cached one.
> 
> smu_late_init() does not re-program the level either, as the
> AMD_PP_TASK_COMPLETE_INIT path compares the cached level against itself.
> 
> amdgpu_dpm_reapply_umd_profile() replays the retained level after reset
> and resume, but only for the PROFILE_* levels, and returns early for any
> other level.
> 
> Restore every level through this common path and rename it to
> amdgpu_dpm_reapply_performance_level(). The PROFILE_* levels keep their
> UMD pstate handling, other levels only replay the level itself. auto is
> skipped, as PMFW comes back in auto, as it does at probe. If replaying a
> non-profile level fails, the failure is logged and the level the user
> asked for is kept.
> 
> smu_reapply_power_state_dynamic() duplicates
> smu_adjust_power_state_dynamic() with the display settings skipped and
> the level forced, so call that instead.
> 
> v2: restore the forced level instead of reporting the default after the
>      reset (Lijo).
> 
> v3: use the force_performance_level argument of
>      smu_adjust_power_state_dynamic() instead of switching the cached level
>      to auto first.
> 
> v4: restore the level through the common reapply path instead of
>      smu_handle_task(), rename amdgpu_dpm_reapply_umd_profile() to
>      amdgpu_dpm_reapply_performance_level(), replay non-profile levels
>      directly and replace the body of smu_reapply_power_state_dynamic()
>      (Lijo). Add the Fixes tag (Kevin).
> 
> Fixes: 580ad7cbd4b7 ("drm/amdgpu/swsmu: Only force workload setup on init")
> Suggested-by: Lijo Lazar <lijo.lazar@amd.com>
> Signed-off-by: Priya Hosur <Priya.Hosur@amd.com>

Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>

Thanks,
Lijo

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
>   drivers/gpu/drm/amd/pm/amdgpu_dpm.c        | 21 +++++++++---
>   drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h    |  2 +-
>   drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c  | 39 ++++------------------
>   4 files changed, 26 insertions(+), 40 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 5069042b5279..2f7ef4bb2a65 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -2774,9 +2774,9 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
>   	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
>   	amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
>   
> -	r = amdgpu_dpm_reapply_umd_profile(adev);
> +	r = amdgpu_dpm_reapply_performance_level(adev);
>   	if (r) {
> -		dev_err(adev->dev, "Failed to restore UMD profile state: %d\n", r);
> +		dev_err(adev->dev, "Failed to restore performance level: %d\n", r);
>   		return r;
>   	}
>   
> diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
> index 75d8bb4868e8..ea698fe48e07 100644
> --- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
> +++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
> @@ -1083,14 +1083,16 @@ static void amdgpu_dpm_exit_umd_state(struct amdgpu_device *adev)
>   }
>   
>   /**
> - * amdgpu_dpm_reapply_umd_profile - Restore the active UMD profile
> + * amdgpu_dpm_reapply_performance_level - Restore the retained DPM level
>    * @adev: amdgpu device
>    *
>    * Resume and reset retain the selected DPM level in software, but the
> - * underlying GFX and PMFW state is lost. Reapply a retained UMD profile
> - * after the common resume path has restored its default CG and PG state.
> + * underlying GFX and PMFW state is lost. Reapply the retained level after
> + * the common resume path has restored its default CG and PG state. UMD
> + * profile levels also restore their GFX state, other levels only replay
> + * the level itself.
>    */
> -int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
> +int amdgpu_dpm_reapply_performance_level(struct amdgpu_device *adev)
>   {
>   	const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
>   	enum amd_dpm_forced_level level;
> @@ -1120,7 +1122,18 @@ int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
>   		break;
> +	case AMD_DPM_FORCED_LEVEL_AUTO:
> +		/* PMFW comes back in auto, as it does at probe. */
> +		ret = 0;
> +		goto unlock;
>   	default:
> +		/* No UMD pstate state to restore, only the level itself. */
> +		ret = pp_funcs->reapply_performance_level(adev->powerplay.pp_handle,
> +							     level);
> +		if (ret)
> +			dev_warn(adev->dev,
> +				 "Failed to restore performance level %d: %d\n",
> +				 level, ret);
>   		ret = 0;
>   		goto unlock;
>   	}
> diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
> index ba8624d28f28..8ae3fcc016f8 100644
> --- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
> +++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
> @@ -484,7 +484,7 @@ void amdgpu_dpm_set_power_state(struct amdgpu_device *adev,
>   enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device *adev);
>   int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
>   				       enum amd_dpm_forced_level level);
> -int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev);
> +int amdgpu_dpm_reapply_performance_level(struct amdgpu_device *adev);
>   int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev,
>   				 struct pp_states_info *states);
>   int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev,
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> index a901f8535598..e45aeb2c565b 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> @@ -2507,33 +2507,7 @@ static int
>   smu_reapply_power_state_dynamic(struct smu_context *smu,
>   				enum amd_dpm_forced_level level)
>   {
> -	struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
> -	int ret;
> -
> -	ret = smu_apply_clocks_adjust_rules(smu);
> -	if (ret) {
> -		dev_err(smu->adev->dev, "Failed to apply clocks adjust rules!");
> -		return ret;
> -	}
> -
> -	ret = smu_asic_set_performance_level(smu, level);
> -	if (ret) {
> -		if (ret == -EOPNOTSUPP)
> -			dev_info(smu->adev->dev, "set performance level %d not supported",
> -				 level);
> -		else
> -			dev_err(smu->adev->dev, "Failed to set performance level %d",
> -				level);
> -		return ret;
> -	}
> -
> -	smu_dpm_ctx->dpm_level = level;
> -
> -	if (smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL ||
> -	    smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM)
> -		return 0;
> -
> -	return smu_bump_power_profile_mode(smu, NULL, 0);
> +	return smu_adjust_power_state_dynamic(smu, level, true, true);
>   }
>   
>   static int smu_handle_task(struct smu_context *smu,
> @@ -2698,21 +2672,20 @@ smu_reapply_performance_level(void *handle,
>   	if (!smu->is_apu && !smu_dpm_ctx->dpm_context)
>   		return -EINVAL;
>   
> +	if (smu_dpm_ctx->dpm_level != level)
> +		return -EINVAL;
> +
>   	switch (level) {
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
>   	case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
> +		smu_enter_umd_pstate(smu);
>   		break;
>   	default:
> -		return -EINVAL;
> +		break;
>   	}
>   
> -	if (smu_dpm_ctx->dpm_level != level)
> -		return -EINVAL;
> -
> -	smu_enter_umd_pstate(smu);
> -
>   	return smu_reapply_power_state_dynamic(smu, level);
>   }
>