[PATCH v3 0/2] send message to pmfw when SMT changes

[PATCH v3 0/2] send message to pmfw when SMT changes

[Wenyou Yang]

When the CPU SMT changes on the fly, send the message to pmfw
to notify the SMT status changed.

Changes in v3
1./ Because it is only required for Vangogh, move registering notifier
to vangogh_ppt.c, then remove the patch 2, and the number of patches
decreased to 2.

Changes in v2:
1/. Embed the smt notifer callback into "struct smu_context" structure.
2/. Correct the PPSMC_Message_Count value.
3/. Improve several code styles and others.

Wenyou Yang (2):
  cpu/smt: add a notifier to notify the SMT changes
  drm/amd/pm: vangogh: send the SMT enable message to pmfw

 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
 drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
 .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
 drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
 .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43 +++++++++++++++++++
 include/linux/cpu.h                           |  5 +++
 kernel/cpu.c                                  | 10 ++++-
 7 files changed, 73 insertions(+), 3 deletions(-)

-- 
2.39.2

[PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Wenyou Yang]

Add the notifier chain to notify the cpu SMT status changes

Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
---
 include/linux/cpu.h |  5 +++++
 kernel/cpu.c        | 10 +++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 314802f98b9d..9a842317fe2d 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -213,6 +213,11 @@ enum cpuhp_smt_control {
 	CPU_SMT_NOT_IMPLEMENTED,
 };
 
+enum cpuhp_smt_status {
+	SMT_ENABLED,
+	SMT_DISABLED,
+};
+
 #if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
 extern enum cpuhp_smt_control cpu_smt_control;
 extern void cpu_smt_disable(bool force);
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 6c0a92ca6bb5..1af66a3ffd99 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -89,6 +89,9 @@ static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state) = {
 cpumask_t cpus_booted_once_mask;
 #endif
 
+RAW_NOTIFIER_HEAD(smt_notifier_head);
+EXPORT_SYMBOL(smt_notifier_head);
+
 #if defined(CONFIG_LOCKDEP) && defined(CONFIG_SMP)
 static struct lockdep_map cpuhp_state_up_map =
 	STATIC_LOCKDEP_MAP_INIT("cpuhp_state-up", &cpuhp_state_up_map);
@@ -2281,8 +2284,10 @@ int cpuhp_smt_disable(enum cpuhp_smt_control ctrlval)
 		 */
 		cpuhp_offline_cpu_device(cpu);
 	}
-	if (!ret)
+	if (!ret) {
 		cpu_smt_control = ctrlval;
+		raw_notifier_call_chain(&smt_notifier_head, SMT_DISABLED, NULL);
+	}
 	cpu_maps_update_done();
 	return ret;
 }
@@ -2303,6 +2308,9 @@ int cpuhp_smt_enable(void)
 		/* See comment in cpuhp_smt_disable() */
 		cpuhp_online_cpu_device(cpu);
 	}
+	if (!ret)
+		raw_notifier_call_chain(&smt_notifier_head, SMT_ENABLED, NULL);
+
 	cpu_maps_update_done();
 	return ret;
 }
-- 
2.39.2

Re: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Peter Zijlstra]

On Wed, Mar 29, 2023 at 09:51:48AM +0800, Wenyou Yang wrote:
> Add the notifier chain to notify the cpu SMT status changes
> 

Why!?!? What's the purpose of all this? IIRC this doesn't trigger if you
manually disable all the siblings. And because you didn't tell us why
you need this I can't tell you if that matters or not :/

RE: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Yang, WenYou]

[AMD Official Use Only - General]

Hi Peter,

Thank you for your review.

The purpose of the patch set is to improve the performance when playing game for some AMD APUs with SMT enabled/disabled.

When change the SMT state on the fly through " echo on/off > /sys/devices/system/cpu/smt/control", the kernel needs to send a message to notify PMFW to adjust a variable's value, which impacts the performance.

Best Regards,
Wenyou

> -----Original Message-----
> From: Peter Zijlstra <peterz@infradead.org>
> Sent: Wednesday, March 29, 2023 3:10 PM
> To: Yang, WenYou <WenYou.Yang@amd.com>
> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan, Evan
> <Evan.Quan@amd.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> bp@suse.de; jpoimboe@kernel.org; Phillips, Kim <kim.phillips@amd.com>;
> tglx@linutronix.de; Yuan, Perry <Perry.Yuan@amd.com>; Liang, Richard qi
> <Richardqi.Liang@amd.com>; Li, Ying <YING.LI@amd.com>; Liu, Kun
> <Kun.Liu2@amd.com>; gpiccoli@igalia.com; amd-gfx@lists.freedesktop.org;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes
> 
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
> 
> 
> On Wed, Mar 29, 2023 at 09:51:48AM +0800, Wenyou Yang wrote:
> > Add the notifier chain to notify the cpu SMT status changes
> >
> 
> Why!?!? What's the purpose of all this? IIRC this doesn't trigger if you manually
> disable all the siblings. And because you didn't tell us why you need this I can't
> tell you if that matters or not :/

Re: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Peter Zijlstra]

On Wed, Mar 29, 2023 at 07:23:29AM +0000, Yang, WenYou wrote:
> [AMD Official Use Only - General]

^^^ that has no business being in a public email.

> Hi Peter,
> 
> Thank you for your review.
> 
> The purpose of the patch set is to improve the performance when playing game for some AMD APUs with SMT enabled/disabled.
> 
> When change the SMT state on the fly through " echo on/off > /sys/devices/system/cpu/smt/control", the kernel needs to send a message to notify PMFW to adjust a variable's value, which impacts the performance.

When top posting I normally ignore the email. When not wrapping email I
typically get cranky. You 'win' *3* 'I cannot use email' trophies in a
singly try. Surely AMD has a HOWTO somewhere you can read?

So what do you want to have happen when someone goes and manually
offlines all the SMT siblings using /sys/devices/system/cpu/cpu*/online
?

I'm thinking that wants the same PMFW (whatever the heck that is)
notification change done, right?

If the answer is "yes", then your patch does not meet the goals and is
inadequate.

RE: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Yang, WenYou]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Peter Zijlstra <peterz@infradead.org>
> Sent: Wednesday, March 29, 2023 4:50 PM
> To: Yang, WenYou <WenYou.Yang@amd.com>
> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan, Evan
> <Evan.Quan@amd.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> bp@alien8.de; jpoimboe@kernel.org; Phillips, Kim <kim.phillips@amd.com>;
> tglx@linutronix.de; Yuan, Perry <Perry.Yuan@amd.com>; Liang, Richard qi
> <Richardqi.Liang@amd.com>; Li, Ying <YING.LI@amd.com>; Liu, Kun
> <Kun.Liu2@amd.com>; gpiccoli@igalia.com; amd-gfx@lists.freedesktop.org;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes
> 
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
> 
> 
> On Wed, Mar 29, 2023 at 07:23:29AM +0000, Yang, WenYou wrote:
> > [AMD Official Use Only - General]
> 
> ^^^ that has no business being in a public email.
> 
> > Hi Peter,
> >
> > Thank you for your review.
> >
> > The purpose of the patch set is to improve the performance when playing
> game for some AMD APUs with SMT enabled/disabled.
> >
> > When change the SMT state on the fly through " echo on/off >
> /sys/devices/system/cpu/smt/control", the kernel needs to send a message to
> notify PMFW to adjust a variable's value, which impacts the performance.
> 
> When top posting I normally ignore the email. When not wrapping email I
> typically get cranky. You 'win' *3* 'I cannot use email' trophies in a singly try.
> Surely AMD has a HOWTO somewhere you can read?

Yes. It is my fault. Sorry.

> 
> So what do you want to have happen when someone goes and manually offlines
> all the SMT siblings using /sys/devices/system/cpu/cpu*/online
> ?

I don't consider this situation.  Any suggestions will be deeply appreciated.

> 
> I'm thinking that wants the same PMFW (whatever the heck that is) notification
> change done, right?
> 
> If the answer is "yes", then your patch does not meet the goals and is
> inadequate.

Yes.  Need to improve.

Thank you.
Wenyou

RE: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Yang, WenYou]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Yang, WenYou
> Sent: Wednesday, March 29, 2023 5:43 PM
> To: Peter Zijlstra <peterz@infradead.org>
> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan, Evan
> <Evan.Quan@amd.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> bp@alien8.de; jpoimboe@kernel.org; Phillips, Kim <kim.phillips@amd.com>;
> tglx@linutronix.de; Yuan, Perry <Perry.Yuan@amd.com>; Liang, Richard qi
> <Richardqi.Liang@amd.com>; Li, Ying <YING.LI@amd.com>; Liu, Kun
> <Kun.Liu2@amd.com>; gpiccoli@igalia.com; amd-gfx@lists.freedesktop.org;
> linux-kernel@vger.kernel.org
> Subject: RE: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes
> 
> [AMD Official Use Only - General]
> 
> 
> 
> > -----Original Message-----
> > From: Peter Zijlstra <peterz@infradead.org>
> > Sent: Wednesday, March 29, 2023 4:50 PM
> > To: Yang, WenYou <WenYou.Yang@amd.com>
> > Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> > <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan,
> > Evan <Evan.Quan@amd.com>; Limonciello, Mario
> > <Mario.Limonciello@amd.com>; bp@alien8.de; jpoimboe@kernel.org;
> > Phillips, Kim <kim.phillips@amd.com>; tglx@linutronix.de; Yuan, Perry
> > <Perry.Yuan@amd.com>; Liang, Richard qi <Richardqi.Liang@amd.com>; Li,
> > Ying <YING.LI@amd.com>; Liu, Kun <Kun.Liu2@amd.com>;
> > gpiccoli@igalia.com; amd-gfx@lists.freedesktop.org;
> > linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT
> > changes
> >
> > Caution: This message originated from an External Source. Use proper
> > caution when opening attachments, clicking links, or responding.
> >
> >
> > On Wed, Mar 29, 2023 at 07:23:29AM +0000, Yang, WenYou wrote:
> > > [AMD Official Use Only - General]
> >
> > ^^^ that has no business being in a public email.
> >
> > > Hi Peter,
> > >
> > > Thank you for your review.
> > >
> > > The purpose of the patch set is to improve the performance when
> > > playing
> > game for some AMD APUs with SMT enabled/disabled.
> > >
> > > When change the SMT state on the fly through " echo on/off >
> > /sys/devices/system/cpu/smt/control", the kernel needs to send a
> > message to notify PMFW to adjust a variable's value, which impacts the
> performance.
> >
> > When top posting I normally ignore the email. When not wrapping email
> > I typically get cranky. You 'win' *3* 'I cannot use email' trophies in a singly try.
> > Surely AMD has a HOWTO somewhere you can read?
> 
> Yes. It is my fault. Sorry.
> 
> >
> > So what do you want to have happen when someone goes and manually
> > offlines all the SMT siblings using
> > /sys/devices/system/cpu/cpu*/online
> > ?
> 
> I don't consider this situation.  Any suggestions will be deeply appreciated.

Hi Peter,

I don't find a good method to handle this situation.
Yes, manually offlining all the SMT sibling will get the same result of SMT disabling on the fly.

Actually, the normal way to enable/disable SMT on the fly is to echo on/off > /sys/device/system/cpu/smt/control

What are your opinions?

Best Regards,
Wenyou

> 
> >
> > I'm thinking that wants the same PMFW (whatever the heck that is)
> > notification change done, right?
> >
> > If the answer is "yes", then your patch does not meet the goals and is
> > inadequate.
> 
> Yes.  Need to improve.
> 
> Thank you.
> Wenyou

RE: [PATCH v3 1/2] cpu/smt: add a notifier to notify the SMT changes

[Thomas Gleixner]

On Fri, Mar 31 2023 at 05:49, WenYou Yang wrote:

<SNIP>
        Removing pointlessly copied mail headers. Please fix your email
        client
</SNIP>

>> >
>> > So what do you want to have happen when someone goes and manually
>> > offlines all the SMT siblings using
>> > /sys/devices/system/cpu/cpu*/online
>> > ?
>> 
>> I don't consider this situation.  Any suggestions will be deeply appreciated.
>
> Hi Peter,
>
> I don't find a good method to handle this situation.
> Yes, manually offlining all the SMT sibling will get the same result of SMT disabling on the fly.
>
> Actually, the normal way to enable/disable SMT on the fly is to echo on/off > /sys/device/system/cpu/smt/control

That's the most convenient way, right.

But why do we need a kernel notifier for this, if you can do the same
with a sysfs knob for your driver?

Then user space can fiddle with SMT control in sysfs and afterwards tell
the driver that it should reconfigure.

That makes a ton more sense than this random notifier.

Thanks,

        tglx

[PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable message to pmfw

[Wenyou Yang]

When the CPU SMT status is changed in the fly, sent the SMT enable
message to pmfw to notify it that the SMT status changed.

Add the support to send PPSMC_MSG_SetCClkSMTEnable(0x58) message
to pmfw for vangogh.

Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
---
 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
 drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
 .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
 drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
 .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43 +++++++++++++++++++
 5 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index b5d64749990e..d53d2acc9b46 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -69,6 +69,8 @@ static int smu_set_fan_speed_rpm(void *handle, uint32_t speed);
 static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
 static int smu_set_mp1_state(void *handle, enum pp_mp1_state mp1_state);
 
+extern struct raw_notifier_head smt_notifier_head;
+
 static int smu_sys_get_pp_feature_mask(void *handle,
 				       char *buf)
 {
@@ -1122,6 +1124,9 @@ static int smu_sw_fini(void *handle)
 
 	smu_fini_microcode(smu);
 
+	if (smu->nb.notifier_call != NULL)
+		raw_notifier_chain_unregister(&smt_notifier_head, &smu->nb);
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
index 09469c750a96..4d51ac5ec8ba 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
@@ -566,6 +566,8 @@ struct smu_context
 
 	struct firmware pptable_firmware;
 
+	struct notifier_block nb;
+
 	u32 param_reg;
 	u32 msg_reg;
 	u32 resp_reg;
@@ -1354,6 +1356,11 @@ struct pptable_funcs {
 	 * @init_pptable_microcode: Prepare the pptable microcode to upload via PSP
 	 */
 	int (*init_pptable_microcode)(struct smu_context *smu);
+
+	/**
+	 * @set_cpu_smt_enable: Set the CPU SMT status.
+	 */
+	int (*set_cpu_smt_enable)(struct smu_context *smu, bool smt_enable);
 };
 
 typedef enum {
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
index 7471e2df2828..a6bfa1912c42 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
@@ -111,7 +111,8 @@
 #define PPSMC_MSG_GetGfxOffStatus		       0x50
 #define PPSMC_MSG_GetGfxOffEntryCount		       0x51
 #define PPSMC_MSG_LogGfxOffResidency		       0x52
-#define PPSMC_Message_Count                            0x53
+#define PPSMC_MSG_SetCClkSMTEnable		       0x58
+#define PPSMC_Message_Count                            0x59
 
 //Argument for PPSMC_MSG_GfxDeviceDriverReset
 enum {
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
index 297b70b9388f..820812d910bf 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
@@ -245,7 +245,8 @@
 	__SMU_DUMMY_MAP(AllowGpo),	\
 	__SMU_DUMMY_MAP(Mode2Reset),	\
 	__SMU_DUMMY_MAP(RequestI2cTransaction), \
-	__SMU_DUMMY_MAP(GetMetricsTable),
+	__SMU_DUMMY_MAP(GetMetricsTable), \
+	__SMU_DUMMY_MAP(SetCClkSMTEnable),
 
 #undef __SMU_DUMMY_MAP
 #define __SMU_DUMMY_MAP(type)	SMU_MSG_##type
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
index 7433dcaa16e0..07f8822f2eb0 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
@@ -35,6 +35,7 @@
 #include "asic_reg/gc/gc_10_3_0_offset.h"
 #include "asic_reg/gc/gc_10_3_0_sh_mask.h"
 #include <asm/processor.h>
+#include <linux/cpu.h>
 
 /*
  * DO NOT use these for err/warn/info/debug messages.
@@ -70,6 +71,8 @@
 	FEATURE_MASK(FEATURE_DCFCLK_DPM_BIT)| \
 	FEATURE_MASK(FEATURE_GFX_DPM_BIT))
 
+extern struct raw_notifier_head smt_notifier_head;
+
 static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
 	MSG_MAP(TestMessage,                    PPSMC_MSG_TestMessage,			0),
 	MSG_MAP(GetSmuVersion,                  PPSMC_MSG_GetSmuVersion,		0),
@@ -141,6 +144,7 @@ static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
 	MSG_MAP(GetGfxOffStatus,		    PPSMC_MSG_GetGfxOffStatus,						0),
 	MSG_MAP(GetGfxOffEntryCount,		    PPSMC_MSG_GetGfxOffEntryCount,					0),
 	MSG_MAP(LogGfxOffResidency,		    PPSMC_MSG_LogGfxOffResidency,					0),
+	MSG_MAP(SetCClkSMTEnable,		    PPSMC_MSG_SetCClkSMTEnable,						0),
 };
 
 static struct cmn2asic_mapping vangogh_feature_mask_map[SMU_FEATURE_COUNT] = {
@@ -221,6 +225,9 @@ static const uint8_t vangogh_throttler_map[] = {
 	[THROTTLER_STATUS_BIT_TDC_CVIP]	= (SMU_THROTTLER_TDC_CVIP_BIT),
 };
 
+static int smt_notifier_callback(struct notifier_block *nb,
+				 unsigned long action, void *data);
+
 static int vangogh_tables_init(struct smu_context *smu)
 {
 	struct smu_table_context *smu_table = &smu->smu_table;
@@ -477,6 +484,9 @@ static int vangogh_init_smc_tables(struct smu_context *smu)
 	smu->cpu_core_num = 4;
 #endif
 
+	smu->nb.notifier_call = smt_notifier_callback;
+	raw_notifier_chain_register(&smt_notifier_head, &smu->nb);
+
 	return smu_v11_0_init_smc_tables(smu);
 }
 
@@ -2428,6 +2438,12 @@ static u32 vangogh_get_gfxoff_entrycount(struct smu_context *smu, uint64_t *entr
 	return ret;
 }
 
+static int vangogh_set_cpu_smt_enable(struct smu_context *smu, bool enable)
+{
+	return smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_SetCClkSMTEnable,
+					       enable ? 1 : 0, NULL);
+}
+
 static const struct pptable_funcs vangogh_ppt_funcs = {
 
 	.check_fw_status = smu_v11_0_check_fw_status,
@@ -2474,6 +2490,7 @@ static const struct pptable_funcs vangogh_ppt_funcs = {
 	.get_power_limit = vangogh_get_power_limit,
 	.set_power_limit = vangogh_set_power_limit,
 	.get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
+	.set_cpu_smt_enable = vangogh_set_cpu_smt_enable,
 };
 
 void vangogh_set_ppt_funcs(struct smu_context *smu)
@@ -2486,3 +2503,29 @@ void vangogh_set_ppt_funcs(struct smu_context *smu)
 	smu->is_apu = true;
 	smu_v11_0_set_smu_mailbox_registers(smu);
 }
+
+static int smu_set_cpu_smt_enable(struct smu_context *smu, bool enable)
+{
+	int ret = -EINVAL;
+
+	if (smu->ppt_funcs && smu->ppt_funcs->set_cpu_smt_enable)
+		ret = smu->ppt_funcs->set_cpu_smt_enable(smu, enable);
+
+	return ret;
+}
+
+static int smt_notifier_callback(struct notifier_block *nb,
+				 unsigned long action, void *data)
+{
+	struct smu_context *smu = container_of(nb, struct smu_context, nb);
+	int ret;
+
+	smu = container_of(nb, struct smu_context, nb);
+
+	ret = smu_set_cpu_smt_enable(smu, action == SMT_ENABLED);
+
+	dev_dbg(smu->adev->dev, "failed to set cclk_pd_limit for SMT %sabled: %d\n",
+		action == SMT_ENABLED ?	"en" : "dis", ret);
+
+	return ret ? NOTIFY_BAD : NOTIFY_OK;
+}
-- 
2.39.2

Re: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable message to pmfw

[Mario Limonciello]

On 3/28/23 20:51, Wenyou Yang wrote:
> When the CPU SMT status is changed in the fly, sent the SMT enable
> message to pmfw to notify it that the SMT status changed.
>
> Add the support to send PPSMC_MSG_SetCClkSMTEnable(0x58) message
> to pmfw for vangogh.
>
> Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
> ---
>   drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
>   drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
>   .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
>   drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
>   .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43 +++++++++++++++++++
>   5 files changed, 59 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> index b5d64749990e..d53d2acc9b46 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> @@ -69,6 +69,8 @@ static int smu_set_fan_speed_rpm(void *handle, uint32_t speed);
>   static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
>   static int smu_set_mp1_state(void *handle, enum pp_mp1_state mp1_state);
>   
> +extern struct raw_notifier_head smt_notifier_head;
> +
>   static int smu_sys_get_pp_feature_mask(void *handle,
>   				       char *buf)
>   {
> @@ -1122,6 +1124,9 @@ static int smu_sw_fini(void *handle)
>   
>   	smu_fini_microcode(smu);
>   
> +	if (smu->nb.notifier_call != NULL)
> +		raw_notifier_chain_unregister(&smt_notifier_head, &smu->nb);
> +
>   	return 0;
>   }
>   
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> index 09469c750a96..4d51ac5ec8ba 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> @@ -566,6 +566,8 @@ struct smu_context
>   
>   	struct firmware pptable_firmware;
>   
> +	struct notifier_block nb;
> +
>   	u32 param_reg;
>   	u32 msg_reg;
>   	u32 resp_reg;
> @@ -1354,6 +1356,11 @@ struct pptable_funcs {
>   	 * @init_pptable_microcode: Prepare the pptable microcode to upload via PSP
>   	 */
>   	int (*init_pptable_microcode)(struct smu_context *smu);
> +
> +	/**
> +	 * @set_cpu_smt_enable: Set the CPU SMT status.
> +	 */
> +	int (*set_cpu_smt_enable)(struct smu_context *smu, bool smt_enable);
>   };
>   
>   typedef enum {
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> index 7471e2df2828..a6bfa1912c42 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> @@ -111,7 +111,8 @@
>   #define PPSMC_MSG_GetGfxOffStatus		       0x50
>   #define PPSMC_MSG_GetGfxOffEntryCount		       0x51
>   #define PPSMC_MSG_LogGfxOffResidency		       0x52
> -#define PPSMC_Message_Count                            0x53
> +#define PPSMC_MSG_SetCClkSMTEnable		       0x58
> +#define PPSMC_Message_Count                            0x59
>   
>   //Argument for PPSMC_MSG_GfxDeviceDriverReset
>   enum {
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> index 297b70b9388f..820812d910bf 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> @@ -245,7 +245,8 @@
>   	__SMU_DUMMY_MAP(AllowGpo),	\
>   	__SMU_DUMMY_MAP(Mode2Reset),	\
>   	__SMU_DUMMY_MAP(RequestI2cTransaction), \
> -	__SMU_DUMMY_MAP(GetMetricsTable),
> +	__SMU_DUMMY_MAP(GetMetricsTable), \
> +	__SMU_DUMMY_MAP(SetCClkSMTEnable),
>   
>   #undef __SMU_DUMMY_MAP
>   #define __SMU_DUMMY_MAP(type)	SMU_MSG_##type
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> index 7433dcaa16e0..07f8822f2eb0 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> @@ -35,6 +35,7 @@
>   #include "asic_reg/gc/gc_10_3_0_offset.h"
>   #include "asic_reg/gc/gc_10_3_0_sh_mask.h"
>   #include <asm/processor.h>
> +#include <linux/cpu.h>
>   
>   /*
>    * DO NOT use these for err/warn/info/debug messages.
> @@ -70,6 +71,8 @@
>   	FEATURE_MASK(FEATURE_DCFCLK_DPM_BIT)| \
>   	FEATURE_MASK(FEATURE_GFX_DPM_BIT))
>   
> +extern struct raw_notifier_head smt_notifier_head;
> +
>   static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
>   	MSG_MAP(TestMessage,                    PPSMC_MSG_TestMessage,			0),
>   	MSG_MAP(GetSmuVersion,                  PPSMC_MSG_GetSmuVersion,		0),
> @@ -141,6 +144,7 @@ static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
>   	MSG_MAP(GetGfxOffStatus,		    PPSMC_MSG_GetGfxOffStatus,						0),
>   	MSG_MAP(GetGfxOffEntryCount,		    PPSMC_MSG_GetGfxOffEntryCount,					0),
>   	MSG_MAP(LogGfxOffResidency,		    PPSMC_MSG_LogGfxOffResidency,					0),
> +	MSG_MAP(SetCClkSMTEnable,		    PPSMC_MSG_SetCClkSMTEnable,						0),
>   };
>   
>   static struct cmn2asic_mapping vangogh_feature_mask_map[SMU_FEATURE_COUNT] = {
> @@ -221,6 +225,9 @@ static const uint8_t vangogh_throttler_map[] = {
>   	[THROTTLER_STATUS_BIT_TDC_CVIP]	= (SMU_THROTTLER_TDC_CVIP_BIT),
>   };
>   
> +static int smt_notifier_callback(struct notifier_block *nb,
> +				 unsigned long action, void *data);
> +
>   static int vangogh_tables_init(struct smu_context *smu)
>   {
>   	struct smu_table_context *smu_table = &smu->smu_table;
> @@ -477,6 +484,9 @@ static int vangogh_init_smc_tables(struct smu_context *smu)
>   	smu->cpu_core_num = 4;
>   #endif
>   
> +	smu->nb.notifier_call = smt_notifier_callback;
> +	raw_notifier_chain_register(&smt_notifier_head, &smu->nb);
> +
>   	return smu_v11_0_init_smc_tables(smu);
>   }
>   
> @@ -2428,6 +2438,12 @@ static u32 vangogh_get_gfxoff_entrycount(struct smu_context *smu, uint64_t *entr
>   	return ret;
>   }
>   
> +static int vangogh_set_cpu_smt_enable(struct smu_context *smu, bool enable)
> +{
> +	return smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_SetCClkSMTEnable,
> +					       enable ? 1 : 0, NULL);
> +}
> +
>   static const struct pptable_funcs vangogh_ppt_funcs = {
>   
>   	.check_fw_status = smu_v11_0_check_fw_status,
> @@ -2474,6 +2490,7 @@ static const struct pptable_funcs vangogh_ppt_funcs = {
>   	.get_power_limit = vangogh_get_power_limit,
>   	.set_power_limit = vangogh_set_power_limit,
>   	.get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
> +	.set_cpu_smt_enable = vangogh_set_cpu_smt_enable,
>   };
>   
>   void vangogh_set_ppt_funcs(struct smu_context *smu)
> @@ -2486,3 +2503,29 @@ void vangogh_set_ppt_funcs(struct smu_context *smu)
>   	smu->is_apu = true;
>   	smu_v11_0_set_smu_mailbox_registers(smu);
>   }
> +
> +static int smu_set_cpu_smt_enable(struct smu_context *smu, bool enable)
> +{
> +	int ret = -EINVAL;
> +
> +	if (smu->ppt_funcs && smu->ppt_funcs->set_cpu_smt_enable)
> +		ret = smu->ppt_funcs->set_cpu_smt_enable(smu, enable);
> +
> +	return ret;
> +}
> +
> +static int smt_notifier_callback(struct notifier_block *nb,
> +				 unsigned long action, void *data)
> +{
> +	struct smu_context *smu = container_of(nb, struct smu_context, nb);
> +	int ret;
> +
> +	smu = container_of(nb, struct smu_context, nb);
> +
> +	ret = smu_set_cpu_smt_enable(smu, action == SMT_ENABLED);
> +
> +	dev_dbg(smu->adev->dev, "failed to set cclk_pd_limit for SMT %sabled: %d\n",
> +		action == SMT_ENABLED ?	"en" : "dis", ret);
> +
> +	return ret ? NOTIFY_BAD : NOTIFY_OK;
> +}

Is there a particular PMFW version requirement for this message to work?

Perhaps you should only register/unregister for this notifier when that
minimum PMFW is present.

Otherwise this is going to lead to smu_set_cpu_smt_enable always failing
on older PMFW and the notifier callback will return NOTIFY_BAD every time.

RE: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable message to pmfw

[Yang, WenYou]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Limonciello, Mario <Mario.Limonciello@amd.com>
> Sent: Wednesday, March 29, 2023 12:18 PM
> To: Yang, WenYou <WenYou.Yang@amd.com>; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan, Evan
> <Evan.Quan@amd.com>; bp@suse.de; peterz@infradead.org;
> jpoimboe@kernel.org; Phillips, Kim <kim.phillips@amd.com>; tglx@linutronix.de
> Cc: Yuan, Perry <Perry.Yuan@amd.com>; Liang, Richard qi
> <Richardqi.Liang@amd.com>; Li, Ying <YING.LI@amd.com>; Liu, Kun
> <Kun.Liu2@amd.com>; gpiccoli@igalia.com; amd-gfx@lists.freedesktop.org;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable
> message to pmfw
> 
> 
> On 3/28/23 20:51, Wenyou Yang wrote:
> > When the CPU SMT status is changed in the fly, sent the SMT enable
> > message to pmfw to notify it that the SMT status changed.
> >
> > Add the support to send PPSMC_MSG_SetCClkSMTEnable(0x58) message to
> > pmfw for vangogh.
> >
> > Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
> > ---
> >   drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
> >   drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
> >   .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
> >   drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
> >   .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43
> +++++++++++++++++++
> >   5 files changed, 59 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > index b5d64749990e..d53d2acc9b46 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > @@ -69,6 +69,8 @@ static int smu_set_fan_speed_rpm(void *handle,
> uint32_t speed);
> >   static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
> >   static int smu_set_mp1_state(void *handle, enum pp_mp1_state
> > mp1_state);
> >
> > +extern struct raw_notifier_head smt_notifier_head;
> > +
> >   static int smu_sys_get_pp_feature_mask(void *handle,
> >   				       char *buf)
> >   {
> > @@ -1122,6 +1124,9 @@ static int smu_sw_fini(void *handle)
> >
> >   	smu_fini_microcode(smu);
> >
> > +	if (smu->nb.notifier_call != NULL)
> > +		raw_notifier_chain_unregister(&smt_notifier_head, &smu->nb);
> > +
> >   	return 0;
> >   }
> >
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > index 09469c750a96..4d51ac5ec8ba 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > @@ -566,6 +566,8 @@ struct smu_context
> >
> >   	struct firmware pptable_firmware;
> >
> > +	struct notifier_block nb;
> > +
> >   	u32 param_reg;
> >   	u32 msg_reg;
> >   	u32 resp_reg;
> > @@ -1354,6 +1356,11 @@ struct pptable_funcs {
> >   	 * @init_pptable_microcode: Prepare the pptable microcode to upload
> via PSP
> >   	 */
> >   	int (*init_pptable_microcode)(struct smu_context *smu);
> > +
> > +	/**
> > +	 * @set_cpu_smt_enable: Set the CPU SMT status.
> > +	 */
> > +	int (*set_cpu_smt_enable)(struct smu_context *smu, bool smt_enable);
> >   };
> >
> >   typedef enum {
> > diff --git
> > a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > index 7471e2df2828..a6bfa1912c42 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > @@ -111,7 +111,8 @@
> >   #define PPSMC_MSG_GetGfxOffStatus		       0x50
> >   #define PPSMC_MSG_GetGfxOffEntryCount		       0x51
> >   #define PPSMC_MSG_LogGfxOffResidency		       0x52
> > -#define PPSMC_Message_Count                            0x53
> > +#define PPSMC_MSG_SetCClkSMTEnable		       0x58
> > +#define PPSMC_Message_Count                            0x59
> >
> >   //Argument for PPSMC_MSG_GfxDeviceDriverReset
> >   enum {
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > index 297b70b9388f..820812d910bf 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > @@ -245,7 +245,8 @@
> >   	__SMU_DUMMY_MAP(AllowGpo),	\
> >   	__SMU_DUMMY_MAP(Mode2Reset),	\
> >   	__SMU_DUMMY_MAP(RequestI2cTransaction), \
> > -	__SMU_DUMMY_MAP(GetMetricsTable),
> > +	__SMU_DUMMY_MAP(GetMetricsTable), \
> > +	__SMU_DUMMY_MAP(SetCClkSMTEnable),
> >
> >   #undef __SMU_DUMMY_MAP
> >   #define __SMU_DUMMY_MAP(type)	SMU_MSG_##type
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > index 7433dcaa16e0..07f8822f2eb0 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > @@ -35,6 +35,7 @@
> >   #include "asic_reg/gc/gc_10_3_0_offset.h"
> >   #include "asic_reg/gc/gc_10_3_0_sh_mask.h"
> >   #include <asm/processor.h>
> > +#include <linux/cpu.h>
> >
> >   /*
> >    * DO NOT use these for err/warn/info/debug messages.
> > @@ -70,6 +71,8 @@
> >   	FEATURE_MASK(FEATURE_DCFCLK_DPM_BIT)| \
> >   	FEATURE_MASK(FEATURE_GFX_DPM_BIT))
> >
> > +extern struct raw_notifier_head smt_notifier_head;
> > +
> >   static struct cmn2asic_msg_mapping
> vangogh_message_map[SMU_MSG_MAX_COUNT] = {
> >   	MSG_MAP(TestMessage,                    PPSMC_MSG_TestMessage,
> 		0),
> >   	MSG_MAP(GetSmuVersion,                  PPSMC_MSG_GetSmuVersion,
> 		0),
> > @@ -141,6 +144,7 @@ static struct cmn2asic_msg_mapping
> vangogh_message_map[SMU_MSG_MAX_COUNT] = {
> >   	MSG_MAP(GetGfxOffStatus,
> PPSMC_MSG_GetGfxOffStatus,						0),
> >   	MSG_MAP(GetGfxOffEntryCount,
> PPSMC_MSG_GetGfxOffEntryCount,					0),
> >   	MSG_MAP(LogGfxOffResidency,
> PPSMC_MSG_LogGfxOffResidency,					0),
> > +	MSG_MAP(SetCClkSMTEnable,
> PPSMC_MSG_SetCClkSMTEnable,
> 	0),
> >   };
> >
> >   static struct cmn2asic_mapping
> > vangogh_feature_mask_map[SMU_FEATURE_COUNT] = { @@ -221,6 +225,9
> @@ static const uint8_t vangogh_throttler_map[] = {
> >   	[THROTTLER_STATUS_BIT_TDC_CVIP]	=
> (SMU_THROTTLER_TDC_CVIP_BIT),
> >   };
> >
> > +static int smt_notifier_callback(struct notifier_block *nb,
> > +				 unsigned long action, void *data);
> > +
> >   static int vangogh_tables_init(struct smu_context *smu)
> >   {
> >   	struct smu_table_context *smu_table = &smu->smu_table; @@ -477,6
> > +484,9 @@ static int vangogh_init_smc_tables(struct smu_context *smu)
> >   	smu->cpu_core_num = 4;
> >   #endif
> >
> > +	smu->nb.notifier_call = smt_notifier_callback;
> > +	raw_notifier_chain_register(&smt_notifier_head, &smu->nb);
> > +
> >   	return smu_v11_0_init_smc_tables(smu);
> >   }
> >
> > @@ -2428,6 +2438,12 @@ static u32 vangogh_get_gfxoff_entrycount(struct
> smu_context *smu, uint64_t *entr
> >   	return ret;
> >   }
> >
> > +static int vangogh_set_cpu_smt_enable(struct smu_context *smu, bool
> > +enable) {
> > +	return smu_cmn_send_smc_msg_with_param(smu,
> SMU_MSG_SetCClkSMTEnable,
> > +					       enable ? 1 : 0, NULL);
> > +}
> > +
> >   static const struct pptable_funcs vangogh_ppt_funcs = {
> >
> >   	.check_fw_status = smu_v11_0_check_fw_status, @@ -2474,6 +2490,7
> @@
> > static const struct pptable_funcs vangogh_ppt_funcs = {
> >   	.get_power_limit = vangogh_get_power_limit,
> >   	.set_power_limit = vangogh_set_power_limit,
> >   	.get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
> > +	.set_cpu_smt_enable = vangogh_set_cpu_smt_enable,
> >   };
> >
> >   void vangogh_set_ppt_funcs(struct smu_context *smu) @@ -2486,3
> > +2503,29 @@ void vangogh_set_ppt_funcs(struct smu_context *smu)
> >   	smu->is_apu = true;
> >   	smu_v11_0_set_smu_mailbox_registers(smu);
> >   }
> > +
> > +static int smu_set_cpu_smt_enable(struct smu_context *smu, bool
> > +enable) {
> > +	int ret = -EINVAL;
> > +
> > +	if (smu->ppt_funcs && smu->ppt_funcs->set_cpu_smt_enable)
> > +		ret = smu->ppt_funcs->set_cpu_smt_enable(smu, enable);
> > +
> > +	return ret;
> > +}
> > +
> > +static int smt_notifier_callback(struct notifier_block *nb,
> > +				 unsigned long action, void *data) {
> > +	struct smu_context *smu = container_of(nb, struct smu_context, nb);
> > +	int ret;
> > +
> > +	smu = container_of(nb, struct smu_context, nb);
> > +
> > +	ret = smu_set_cpu_smt_enable(smu, action == SMT_ENABLED);
> > +
> > +	dev_dbg(smu->adev->dev, "failed to set cclk_pd_limit for
> SMT %sabled: %d\n",
> > +		action == SMT_ENABLED ?	"en" : "dis", ret);
> > +
> > +	return ret ? NOTIFY_BAD : NOTIFY_OK; }
> 
> Is there a particular PMFW version requirement for this message to work?
> 
> Perhaps you should only register/unregister for this notifier when that minimum
> PMFW is present.
> 
> Otherwise this is going to lead to smu_set_cpu_smt_enable always failing on
> older PMFW and the notifier callback will return NOTIFY_BAD every time.

Thank you for your suggestion, Will add it.

Best Regards,
Wenyou

Re: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable message to pmfw

[Lazar, Lijo]

On 3/29/2023 7:21 AM, Wenyou Yang wrote:
> When the CPU SMT status is changed in the fly, sent the SMT enable
> message to pmfw to notify it that the SMT status changed.
> 
> Add the support to send PPSMC_MSG_SetCClkSMTEnable(0x58) message
> to pmfw for vangogh.
> 
> Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
> ---
>   drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
>   drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
>   .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
>   drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
>   .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43 +++++++++++++++++++
>   5 files changed, 59 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> index b5d64749990e..d53d2acc9b46 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> @@ -69,6 +69,8 @@ static int smu_set_fan_speed_rpm(void *handle, uint32_t speed);
>   static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
>   static int smu_set_mp1_state(void *handle, enum pp_mp1_state mp1_state);
>   
> +extern struct raw_notifier_head smt_notifier_head;
> +
>   static int smu_sys_get_pp_feature_mask(void *handle,
>   				       char *buf)
>   {
> @@ -1122,6 +1124,9 @@ static int smu_sw_fini(void *handle)
>   
>   	smu_fini_microcode(smu);
>   
> +	if (smu->nb.notifier_call != NULL)
> +		raw_notifier_chain_unregister(&smt_notifier_head, &smu->nb);
> +
>   	return 0;
>   }
>   
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> index 09469c750a96..4d51ac5ec8ba 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> @@ -566,6 +566,8 @@ struct smu_context
>   
>   	struct firmware pptable_firmware;
>   
> +	struct notifier_block nb;
> +
>   	u32 param_reg;
>   	u32 msg_reg;
>   	u32 resp_reg;
> @@ -1354,6 +1356,11 @@ struct pptable_funcs {
>   	 * @init_pptable_microcode: Prepare the pptable microcode to upload via PSP
>   	 */
>   	int (*init_pptable_microcode)(struct smu_context *smu);
> +
> +	/**
> +	 * @set_cpu_smt_enable: Set the CPU SMT status.
> +	 */
> +	int (*set_cpu_smt_enable)(struct smu_context *smu, bool smt_enable);
>   };
>   
>   typedef enum {
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> index 7471e2df2828..a6bfa1912c42 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> @@ -111,7 +111,8 @@
>   #define PPSMC_MSG_GetGfxOffStatus		       0x50
>   #define PPSMC_MSG_GetGfxOffEntryCount		       0x51
>   #define PPSMC_MSG_LogGfxOffResidency		       0x52
> -#define PPSMC_Message_Count                            0x53
> +#define PPSMC_MSG_SetCClkSMTEnable		       0x58
> +#define PPSMC_Message_Count                            0x59
>   
>   //Argument for PPSMC_MSG_GfxDeviceDriverReset
>   enum {
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> index 297b70b9388f..820812d910bf 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> @@ -245,7 +245,8 @@
>   	__SMU_DUMMY_MAP(AllowGpo),	\
>   	__SMU_DUMMY_MAP(Mode2Reset),	\
>   	__SMU_DUMMY_MAP(RequestI2cTransaction), \
> -	__SMU_DUMMY_MAP(GetMetricsTable),
> +	__SMU_DUMMY_MAP(GetMetricsTable), \
> +	__SMU_DUMMY_MAP(SetCClkSMTEnable),
>   
>   #undef __SMU_DUMMY_MAP
>   #define __SMU_DUMMY_MAP(type)	SMU_MSG_##type
> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> index 7433dcaa16e0..07f8822f2eb0 100644
> --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> @@ -35,6 +35,7 @@
>   #include "asic_reg/gc/gc_10_3_0_offset.h"
>   #include "asic_reg/gc/gc_10_3_0_sh_mask.h"
>   #include <asm/processor.h>
> +#include <linux/cpu.h>
>   
>   /*
>    * DO NOT use these for err/warn/info/debug messages.
> @@ -70,6 +71,8 @@
>   	FEATURE_MASK(FEATURE_DCFCLK_DPM_BIT)| \
>   	FEATURE_MASK(FEATURE_GFX_DPM_BIT))
>   
> +extern struct raw_notifier_head smt_notifier_head;
> +
>   static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
>   	MSG_MAP(TestMessage,                    PPSMC_MSG_TestMessage,			0),
>   	MSG_MAP(GetSmuVersion,                  PPSMC_MSG_GetSmuVersion,		0),
> @@ -141,6 +144,7 @@ static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = {
>   	MSG_MAP(GetGfxOffStatus,		    PPSMC_MSG_GetGfxOffStatus,						0),
>   	MSG_MAP(GetGfxOffEntryCount,		    PPSMC_MSG_GetGfxOffEntryCount,					0),
>   	MSG_MAP(LogGfxOffResidency,		    PPSMC_MSG_LogGfxOffResidency,					0),
> +	MSG_MAP(SetCClkSMTEnable,		    PPSMC_MSG_SetCClkSMTEnable,						0),
>   };
>   
>   static struct cmn2asic_mapping vangogh_feature_mask_map[SMU_FEATURE_COUNT] = {
> @@ -221,6 +225,9 @@ static const uint8_t vangogh_throttler_map[] = {
>   	[THROTTLER_STATUS_BIT_TDC_CVIP]	= (SMU_THROTTLER_TDC_CVIP_BIT),
>   };
>   
> +static int smt_notifier_callback(struct notifier_block *nb,
> +				 unsigned long action, void *data);
> +
>   static int vangogh_tables_init(struct smu_context *smu)
>   {
>   	struct smu_table_context *smu_table = &smu->smu_table;
> @@ -477,6 +484,9 @@ static int vangogh_init_smc_tables(struct smu_context *smu)
>   	smu->cpu_core_num = 4;
>   #endif
>   
> +	smu->nb.notifier_call = smt_notifier_callback;
> +	raw_notifier_chain_register(&smt_notifier_head, &smu->nb);
> +
>   	return smu_v11_0_init_smc_tables(smu);
>   }
>   
> @@ -2428,6 +2438,12 @@ static u32 vangogh_get_gfxoff_entrycount(struct smu_context *smu, uint64_t *entr
>   	return ret;
>   }
>   
> +static int vangogh_set_cpu_smt_enable(struct smu_context *smu, bool enable)
> +{
> +	return smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_SetCClkSMTEnable,
> +					       enable ? 1 : 0, NULL);
> +}
> +
>   static const struct pptable_funcs vangogh_ppt_funcs = {
>   
>   	.check_fw_status = smu_v11_0_check_fw_status,
> @@ -2474,6 +2490,7 @@ static const struct pptable_funcs vangogh_ppt_funcs = {
>   	.get_power_limit = vangogh_get_power_limit,
>   	.set_power_limit = vangogh_set_power_limit,
>   	.get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
> +	.set_cpu_smt_enable = vangogh_set_cpu_smt_enable,
>   };
>   
>   void vangogh_set_ppt_funcs(struct smu_context *smu)
> @@ -2486,3 +2503,29 @@ void vangogh_set_ppt_funcs(struct smu_context *smu)
>   	smu->is_apu = true;
>   	smu_v11_0_set_smu_mailbox_registers(smu);
>   }
> +
> +static int smu_set_cpu_smt_enable(struct smu_context *smu, bool enable)
> +{
> +	int ret = -EINVAL;
> +
> +	if (smu->ppt_funcs && smu->ppt_funcs->set_cpu_smt_enable)
> +		ret = smu->ppt_funcs->set_cpu_smt_enable(smu, enable);
> +
> +	return ret;
> +}
> +
> +static int smt_notifier_callback(struct notifier_block *nb,
> +				 unsigned long action, void *data)
> +{

This and the one above can be moved to smu_cmn.c as they are generic ones.

Thanks,
Lijo

> +	struct smu_context *smu = container_of(nb, struct smu_context, nb);
> +	int ret;
> +
> +	smu = container_of(nb, struct smu_context, nb);
> +
> +	ret = smu_set_cpu_smt_enable(smu, action == SMT_ENABLED);
> +
> +	dev_dbg(smu->adev->dev, "failed to set cclk_pd_limit for SMT %sabled: %d\n",
> +		action == SMT_ENABLED ?	"en" : "dis", ret);
> +
> +	return ret ? NOTIFY_BAD : NOTIFY_OK;
> +}

RE: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable message to pmfw

[Yang, WenYou]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Lazar, Lijo <Lijo.Lazar@amd.com>
> Sent: Wednesday, March 29, 2023 2:15 PM
> To: Yang, WenYou <WenYou.Yang@amd.com>; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Pan, Xinhui <Xinhui.Pan@amd.com>; Quan, Evan
> <Evan.Quan@amd.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> bp@suse.de; peterz@infradead.org; jpoimboe@kernel.org; Phillips, Kim
> <kim.phillips@amd.com>; tglx@linutronix.de
> Cc: Yuan, Perry <Perry.Yuan@amd.com>; Li, Ying <YING.LI@amd.com>; linux-
> kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; gpiccoli@igalia.com;
> Liu, Kun <Kun.Liu2@amd.com>; Liang, Richard qi <Richardqi.Liang@amd.com>
> Subject: Re: [PATCH v3 2/2] drm/amd/pm: vangogh: send the SMT enable
> message to pmfw
> 
> 
> 
> On 3/29/2023 7:21 AM, Wenyou Yang wrote:
> > When the CPU SMT status is changed in the fly, sent the SMT enable
> > message to pmfw to notify it that the SMT status changed.
> >
> > Add the support to send PPSMC_MSG_SetCClkSMTEnable(0x58) message to
> > pmfw for vangogh.
> >
> > Signed-off-by: Wenyou Yang <WenYou.Yang@amd.com>
> > ---
> >   drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  5 +++
> >   drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h |  7 +++
> >   .../pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h    |  3 +-
> >   drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h  |  3 +-
> >   .../gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c  | 43
> +++++++++++++++++++
> >   5 files changed, 59 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > index b5d64749990e..d53d2acc9b46 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
> > @@ -69,6 +69,8 @@ static int smu_set_fan_speed_rpm(void *handle,
> uint32_t speed);
> >   static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
> >   static int smu_set_mp1_state(void *handle, enum pp_mp1_state
> > mp1_state);
> >
> > +extern struct raw_notifier_head smt_notifier_head;
> > +
> >   static int smu_sys_get_pp_feature_mask(void *handle,
> >   				       char *buf)
> >   {
> > @@ -1122,6 +1124,9 @@ static int smu_sw_fini(void *handle)
> >
> >   	smu_fini_microcode(smu);
> >
> > +	if (smu->nb.notifier_call != NULL)
> > +		raw_notifier_chain_unregister(&smt_notifier_head, &smu->nb);
> > +
> >   	return 0;
> >   }
> >
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > index 09469c750a96..4d51ac5ec8ba 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
> > @@ -566,6 +566,8 @@ struct smu_context
> >
> >   	struct firmware pptable_firmware;
> >
> > +	struct notifier_block nb;
> > +
> >   	u32 param_reg;
> >   	u32 msg_reg;
> >   	u32 resp_reg;
> > @@ -1354,6 +1356,11 @@ struct pptable_funcs {
> >   	 * @init_pptable_microcode: Prepare the pptable microcode to upload
> via PSP
> >   	 */
> >   	int (*init_pptable_microcode)(struct smu_context *smu);
> > +
> > +	/**
> > +	 * @set_cpu_smt_enable: Set the CPU SMT status.
> > +	 */
> > +	int (*set_cpu_smt_enable)(struct smu_context *smu, bool smt_enable);
> >   };
> >
> >   typedef enum {
> > diff --git
> > a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > index 7471e2df2828..a6bfa1912c42 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v11_5_ppsmc.h
> > @@ -111,7 +111,8 @@
> >   #define PPSMC_MSG_GetGfxOffStatus		       0x50
> >   #define PPSMC_MSG_GetGfxOffEntryCount		       0x51
> >   #define PPSMC_MSG_LogGfxOffResidency		       0x52
> > -#define PPSMC_Message_Count                            0x53
> > +#define PPSMC_MSG_SetCClkSMTEnable		       0x58
> > +#define PPSMC_Message_Count                            0x59
> >
> >   //Argument for PPSMC_MSG_GfxDeviceDriverReset
> >   enum {
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > index 297b70b9388f..820812d910bf 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
> > @@ -245,7 +245,8 @@
> >   	__SMU_DUMMY_MAP(AllowGpo),	\
> >   	__SMU_DUMMY_MAP(Mode2Reset),	\
> >   	__SMU_DUMMY_MAP(RequestI2cTransaction), \
> > -	__SMU_DUMMY_MAP(GetMetricsTable),
> > +	__SMU_DUMMY_MAP(GetMetricsTable), \
> > +	__SMU_DUMMY_MAP(SetCClkSMTEnable),
> >
> >   #undef __SMU_DUMMY_MAP
> >   #define __SMU_DUMMY_MAP(type)	SMU_MSG_##type
> > diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > index 7433dcaa16e0..07f8822f2eb0 100644
> > --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
> > @@ -35,6 +35,7 @@
> >   #include "asic_reg/gc/gc_10_3_0_offset.h"
> >   #include "asic_reg/gc/gc_10_3_0_sh_mask.h"
> >   #include <asm/processor.h>
> > +#include <linux/cpu.h>
> >
> >   /*
> >    * DO NOT use these for err/warn/info/debug messages.
> > @@ -70,6 +71,8 @@
> >   	FEATURE_MASK(FEATURE_DCFCLK_DPM_BIT)| \
> >   	FEATURE_MASK(FEATURE_GFX_DPM_BIT))
> >
> > +extern struct raw_notifier_head smt_notifier_head;
> > +
> >   static struct cmn2asic_msg_mapping
> vangogh_message_map[SMU_MSG_MAX_COUNT] = {
> >   	MSG_MAP(TestMessage,                    PPSMC_MSG_TestMessage,
> 		0),
> >   	MSG_MAP(GetSmuVersion,                  PPSMC_MSG_GetSmuVersion,
> 		0),
> > @@ -141,6 +144,7 @@ static struct cmn2asic_msg_mapping
> vangogh_message_map[SMU_MSG_MAX_COUNT] = {
> >   	MSG_MAP(GetGfxOffStatus,
> PPSMC_MSG_GetGfxOffStatus,						0),
> >   	MSG_MAP(GetGfxOffEntryCount,
> PPSMC_MSG_GetGfxOffEntryCount,					0),
> >   	MSG_MAP(LogGfxOffResidency,
> PPSMC_MSG_LogGfxOffResidency,					0),
> > +	MSG_MAP(SetCClkSMTEnable,
> PPSMC_MSG_SetCClkSMTEnable,
> 	0),
> >   };
> >
> >   static struct cmn2asic_mapping
> > vangogh_feature_mask_map[SMU_FEATURE_COUNT] = { @@ -221,6 +225,9
> @@ static const uint8_t vangogh_throttler_map[] = {
> >   	[THROTTLER_STATUS_BIT_TDC_CVIP]	=
> (SMU_THROTTLER_TDC_CVIP_BIT),
> >   };
> >
> > +static int smt_notifier_callback(struct notifier_block *nb,
> > +				 unsigned long action, void *data);
> > +
> >   static int vangogh_tables_init(struct smu_context *smu)
> >   {
> >   	struct smu_table_context *smu_table = &smu->smu_table; @@ -477,6
> > +484,9 @@ static int vangogh_init_smc_tables(struct smu_context *smu)
> >   	smu->cpu_core_num = 4;
> >   #endif
> >
> > +	smu->nb.notifier_call = smt_notifier_callback;
> > +	raw_notifier_chain_register(&smt_notifier_head, &smu->nb);
> > +
> >   	return smu_v11_0_init_smc_tables(smu);
> >   }
> >
> > @@ -2428,6 +2438,12 @@ static u32 vangogh_get_gfxoff_entrycount(struct
> smu_context *smu, uint64_t *entr
> >   	return ret;
> >   }
> >
> > +static int vangogh_set_cpu_smt_enable(struct smu_context *smu, bool
> > +enable) {
> > +	return smu_cmn_send_smc_msg_with_param(smu,
> SMU_MSG_SetCClkSMTEnable,
> > +					       enable ? 1 : 0, NULL);
> > +}
> > +
> >   static const struct pptable_funcs vangogh_ppt_funcs = {
> >
> >   	.check_fw_status = smu_v11_0_check_fw_status, @@ -2474,6 +2490,7
> @@
> > static const struct pptable_funcs vangogh_ppt_funcs = {
> >   	.get_power_limit = vangogh_get_power_limit,
> >   	.set_power_limit = vangogh_set_power_limit,
> >   	.get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
> > +	.set_cpu_smt_enable = vangogh_set_cpu_smt_enable,
> >   };
> >
> >   void vangogh_set_ppt_funcs(struct smu_context *smu) @@ -2486,3
> > +2503,29 @@ void vangogh_set_ppt_funcs(struct smu_context *smu)
> >   	smu->is_apu = true;
> >   	smu_v11_0_set_smu_mailbox_registers(smu);
> >   }
> > +
> > +static int smu_set_cpu_smt_enable(struct smu_context *smu, bool
> > +enable) {
> > +	int ret = -EINVAL;
> > +
> > +	if (smu->ppt_funcs && smu->ppt_funcs->set_cpu_smt_enable)
> > +		ret = smu->ppt_funcs->set_cpu_smt_enable(smu, enable);
> > +
> > +	return ret;
> > +}
> > +
> > +static int smt_notifier_callback(struct notifier_block *nb,
> > +				 unsigned long action, void *data) {
> 
> This and the one above can be moved to smu_cmn.c as they are generic ones.
> 
Thank you for your review. Will do.

Best Regards,
Wenyou

> Thanks,
> Lijo
> 
> > +	struct smu_context *smu = container_of(nb, struct smu_context, nb);
> > +	int ret;
> > +
> > +	smu = container_of(nb, struct smu_context, nb);
> > +
> > +	ret = smu_set_cpu_smt_enable(smu, action == SMT_ENABLED);
> > +
> > +	dev_dbg(smu->adev->dev, "failed to set cclk_pd_limit for
> SMT %sabled: %d\n",
> > +		action == SMT_ENABLED ?	"en" : "dis", ret);
> > +
> > +	return ret ? NOTIFY_BAD : NOTIFY_OK; }