[PATCH v2 0/2] cpufreq: amd-pstate, platform/x86/amd/hfi: Enable hybrid capacity scaling & SMT ITMT ordering on Zen 5 / Zen 5c

[PATCH v2 0/2] cpufreq: amd-pstate, platform/x86/amd/hfi: Enable hybrid capacity scaling & SMT ITMT ordering on Zen 5 / Zen 5c

[Aiden Hecker]

Hardware Observation on AMD Ryzen AI 7 350 ("Krackan Point", Family 1Ah Model 60h)
=================================================================================
On AMD Ryzen AI 7 350 (Fedora 45, Linux 7.3.3-300.fc45.x86_64 and linux-pm 7.4-rc2),
the silicon interleaves 4 Classic Zen 5 physical cores (5.09 GHz max, CPUID
0x80000026 CoreType=0 / TOPO_CPU_TYPE_PERFORMANCE, ACPI CPPC prefcore_ranking
208/202/196) and 4 Dense Zen 5c physical cores (3.51 GHz max, CoreType=1 /
TOPO_CPU_TYPE_EFFICIENCY, prefcore_ranking 135) inside a single 16 MiB L3 CCX:

  * Even CPUs (0, 2, 4, 6; SMT 8, 10, 12, 14): Classic Zen 5 (5090.9 MHz)
  * Odd CPUs  (1, 3, 5, 7; SMT 9, 11, 13, 15): Dense Zen 5c  (3506.5 MHz)

Observed Issues in Mainline Linux
---------------------------------
1. When X86_FEATURE_AMD_WORKLOAD_CLASS and CONFIG_AMD_HFI are enabled,
   amd_pstate_init_prefcore() disables amd_pstate_prefcore and delegates ITMT
   priorities to amd_hfi (`AMDI0104:00`). However, neither amd-pstate nor amd_hfi
   calls arch_enable_hybrid_capacity_scale(), arch_set_cpu_capacity(), or
   topology_set_cpu_scale(), leaving /sys/devices/system/cpu/cpu*/cpu_capacity
   and arch_scale_cpu_capacity() at a uniform 1024 across all 16 threads.
2. In drivers/platform/x86/amd/hfi/hfi.c, amd_set_hfi_ipcc_score() passes the
   class 0 IPCC score identically for primary threads and secondary SMT siblings,
   without distinguishing SMT secondaries via topology_sibling_cpumask().

Measured Impact on Real Silicon (Ryzen AI 7 350)
------------------------------------------------
- High-entropy 64 KiB pointer-chase + branch workload (450M instructions):
    * CPU  4 (Zen 5 Golden, CPPC 208):  66.69 ms
    * CPU 12 (Zen 5 Golden SMT)      :  66.09 ms
    * CPU  1 (Zen 5c Dense, CPPC 135):  95.18 ms (+42.7% latency penalty)
- 16-thread mixed desktop/audio contention benchmark (2 critical + 6 worker
  + 8 background cruncher threads):
    * Unfenced equal-capacity placement: 75.77 ms avg | 109.22 ms worst-tail
    * Hybrid core partitioning:          34.66 ms avg |  39.42 ms worst-tail

Changes in v2
-------------
- Rebased cleanly onto rafael/linux-pm.git (v7.4-rc2) and generated with git
  format-patch; full `bzImage` + modules + `W=1 KCFLAGS="-Werror"` build-tested
  and verified `0 errors, 0 warnings, 0 checks` with `scripts/checkpatch.pl --strict`.
- Patch 1/2 (`drivers/cpufreq/amd-pstate.c`):
  * Uses x86 `arch_enable_hybrid_capacity_scale()`, `arch_set_cpu_capacity()`,
    `topology_set_cpu_scale()`, and workqueue-deferred `rebuild_sched_domains()`
    rather than writing generic `cpu_scale` directly.
  * Synchronizes SMT sibling capacities via `topology_sibling_cpumask()` so
    `SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY` warnings never trigger in
    `kernel/sched/topology.c`.
  * Removes hardcoded CPU index checks and boot-order `num_online_cpus()` races.
- Patch 2/2 (`drivers/platform/x86/amd/hfi/hfi.c`):
  * Scales secondary SMT sibling ITMT priorities (7/8 of primary IPCC score) via
    `topology_sibling_cpumask()` when `X86_FEATURE_AMD_HTR_CORES` is set.
  * Removes unreferenced module-parameter stubs and raw per-CPU `cpuid_count()`
    loops.
- Updated recipient list using `scripts/get_maintainer.pl`.

Aiden Hecker (2):
  cpufreq: amd-pstate: Enable hybrid CPU capacity scaling on
    heterogeneous Zen CPUs
  platform/x86/amd/hfi: Rank SMT siblings below primary cores for ITMT
    packing

 drivers/cpufreq/amd-pstate.c       | 121 +++++++++++++++++++++++++++++
 drivers/platform/x86/amd/hfi/hfi.c |  18 ++++-
 2 files changed, 138 insertions(+), 1 deletion(-)

-- 
2.56.1

[PATCH v2 1/2] cpufreq: amd-pstate: Enable hybrid CPU capacity scaling on heterogeneous Zen CPUs

[Aiden Hecker]

On heterogeneous AMD Zen 5 / Zen 5c processors (X86_FEATURE_AMD_HTR_CORES,
such as Ryzen AI 7 350 Family 1Ah Model 60h), physical Zen 5 cores
(5.09 GHz, CPPC highest_perf 196..208) and dense Zen 5c cores (3.51 GHz,
CPPC highest_perf 135) alternate inside a single shared 16 MiB L3 CCX.

Currently amd-pstate records prefcore_ranking and nominal_perf per CPU,
and disables amd_pstate_prefcore when amd_hfi is active, but neither
driver initializes x86 hybrid capacity scaling. As a result, all logical
CPUs remain at uniform capacity 1024 across the shared L3 domain.

Record per-CPU highest_perf and nominal_perf when X86_FEATURE_AMD_HTR_CORES
is present and program arch_enable_hybrid_capacity_scale(),
arch_set_cpu_capacity(), and topology_set_cpu_scale() once asymmetry is
observed across online CPUs. SMT siblings are explicitly synchronized to
the same capacity value so SD_SHARE_CPUCAPACITY never triggers asymmetric
SMT warnings, while rebuild_sched_domains() is deferred to a workqueue.

Developed with coding assistance from Antigravity (Google DeepMind).

Tested-by: Aiden Hecker <heckeraiden@gmail.com>
Signed-off-by: Aiden Hecker <heckeraiden@gmail.com>
---
 drivers/cpufreq/amd-pstate.c | 121 +++++++++++++++++++++++++++++++++++
 1 file changed, 121 insertions(+)

diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 4bedfbfc1..3f4b6a2b0 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -36,9 +36,13 @@
 #include <linux/io.h>
 #include <linux/delay.h>
 #include <linux/uaccess.h>
+#include <linux/cleanup.h>
+#include <linux/cpuset.h>
+#include <linux/mutex.h>
 #include <linux/power_supply.h>
 #include <linux/static_call.h>
 #include <linux/topology.h>
+#include <linux/workqueue.h>
 
 #include <acpi/processor.h>
 #include <acpi/cppc_acpi.h>
@@ -88,6 +92,23 @@ static int cppc_state = AMD_PSTATE_UNDEFINED;
 static bool amd_pstate_prefcore = true;
 static struct quirk_entry *quirks;
 
+struct amd_hybrid_cpu_cap {
+	u32 highest_perf;
+	u32 nominal_perf;
+};
+
+static DEFINE_PER_CPU(struct amd_hybrid_cpu_cap, amd_hybrid_caps);
+static DEFINE_MUTEX(amd_hybrid_cap_lock);
+static u32 amd_hybrid_max_perf;
+static bool amd_hybrid_cap_enabled;
+
+static void amd_hybrid_cap_workfn(struct work_struct *work)
+{
+	rebuild_sched_domains();
+}
+
+static DECLARE_WORK(amd_hybrid_cap_work, amd_hybrid_cap_workfn);
+
 /*
  * The numeric EPP value programmed for each named preference. First dimension
  * is CPU type (TOPO_CPU_TYPE_ANY for non-hybrid, TOPO_CPU_TYPE_PERFORMANCE/
@@ -969,6 +990,99 @@ static void amd_pstate_init_prefcore(struct amd_cpudata *cpudata)
 	sched_set_itmt_core_prio((int)READ_ONCE(cpudata->prefcore_ranking), cpudata->cpu);
 }
 
+static void amd_pstate_record_hybrid_cap(struct amd_cpudata *cpudata)
+{
+	struct amd_hybrid_cpu_cap *cap;
+	union perf_cached perf;
+	u32 highest_perf;
+
+	if (!cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES))
+		return;
+
+	perf = READ_ONCE(cpudata->perf);
+	highest_perf = READ_ONCE(cpudata->prefcore_ranking);
+	if (!highest_perf)
+		highest_perf = perf.highest_perf;
+
+	guard(mutex)(&amd_hybrid_cap_lock);
+	cap = per_cpu_ptr(&amd_hybrid_caps, cpudata->cpu);
+	cap->highest_perf = highest_perf;
+	cap->nominal_perf = perf.nominal_perf;
+}
+
+static void amd_pstate_sync_hybrid_capacity(void)
+{
+	u32 max_perf = 0, min_perf = U32_MAX;
+	bool updated = false;
+	int cpu;
+
+	if (!cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES))
+		return;
+
+	guard(mutex)(&amd_hybrid_cap_lock);
+
+	for_each_online_cpu(cpu) {
+		const struct cpumask *sib_mask = topology_sibling_cpumask(cpu);
+		u32 sib_max = 0;
+		int sib;
+
+		/*
+		 * SMT siblings share SD_SHARE_CPUCAPACITY and must expose identical
+		 * arch_scale_cpu_capacity() values to avoid scheduler warnings.
+		 */
+		for_each_cpu(sib, sib_mask) {
+			u32 p = per_cpu(amd_hybrid_caps, sib).highest_perf;
+
+			if (p > sib_max)
+				sib_max = p;
+		}
+		if (!sib_max)
+			continue;
+
+		for_each_cpu(sib, sib_mask)
+			per_cpu(amd_hybrid_caps, sib).highest_perf = sib_max;
+
+		if (sib_max > max_perf)
+			max_perf = sib_max;
+		if (sib_max < min_perf)
+			min_perf = sib_max;
+	}
+
+	if (!max_perf || min_perf >= max_perf)
+		return;
+
+	if (!amd_hybrid_cap_enabled) {
+		if (!arch_enable_hybrid_capacity_scale())
+			return;
+		amd_hybrid_cap_enabled = true;
+		updated = true;
+	}
+
+	if (max_perf != amd_hybrid_max_perf) {
+		amd_hybrid_max_perf = max_perf;
+		updated = true;
+	}
+
+	for_each_online_cpu(cpu) {
+		struct amd_hybrid_cpu_cap info = per_cpu(amd_hybrid_caps, cpu);
+		unsigned long prev_cap;
+
+		if (!info.highest_perf || !info.nominal_perf)
+			continue;
+
+		prev_cap = arch_scale_cpu_capacity(cpu);
+		arch_set_cpu_capacity(cpu, info.highest_perf, max_perf,
+				      info.highest_perf, info.nominal_perf);
+		topology_set_cpu_scale(cpu, arch_scale_cpu_capacity(cpu));
+
+		if (arch_scale_cpu_capacity(cpu) != prev_cap)
+			updated = true;
+	}
+
+	if (updated)
+		schedule_work(&amd_hybrid_cap_work);
+}
+
 static void amd_pstate_update_limits(struct cpufreq_policy *policy)
 {
 	struct amd_cpudata *cpudata;
@@ -1130,6 +1244,7 @@ static int amd_pstate_cpu_init(struct cpufreq_policy *policy)
 	if (ret)
 		goto free_cpudata1;
 
+	amd_pstate_record_hybrid_cap(cpudata);
 	amd_pstate_init_prefcore(cpudata);
 
 	ret = amd_pstate_init_freq(cpudata);
@@ -1769,11 +1884,14 @@ static int amd_pstate_register_driver(int mode)
 	if (amd_pstate_prefcore)
 		sched_set_itmt_support();
 
+	amd_pstate_sync_hybrid_capacity();
+
 	return 0;
 }
 
 static int amd_pstate_unregister_driver(int dummy)
 {
+	cancel_work_sync(&amd_hybrid_cap_work);
 	cpufreq_unregister_driver(current_pstate_driver);
 	amd_pstate_driver_cleanup();
 	return 0;
@@ -1985,6 +2103,7 @@ static int amd_pstate_epp_cpu_init(struct cpufreq_policy *policy)
 	if (ret)
 		goto free_cpudata1;
 
+	amd_pstate_record_hybrid_cap(cpudata);
 	amd_pstate_init_prefcore(cpudata);
 
 	ret = amd_pstate_init_freq(cpudata);
@@ -2149,6 +2268,8 @@ static int amd_pstate_cpu_online(struct cpufreq_policy *policy)
 		return ret;
 
 	cached_floor_perf = freq_to_perf(perf, cpudata->nominal_freq, cpudata->floor_freq);
+	amd_pstate_record_hybrid_cap(cpudata);
+	amd_pstate_sync_hybrid_capacity();
 	return amd_pstate_set_floor_perf(policy, cached_floor_perf);
 }
 
-- 
2.56.1

[PATCH v2 2/2] platform/x86/amd/hfi: Rank SMT siblings below primary cores for ITMT packing

[Aiden Hecker]

On heterogeneous AMD processors (X86_FEATURE_AMD_HTR_CORES) with 2-way
SMT enabled on both Zen 5 and Zen 5c cores, amd_set_hfi_ipcc_score()
assigns identical ITMT priorities to primary threads and their SMT
siblings.

Scale secondary SMT sibling ITMT priorities slightly below their primary
thread sibling (7/8 of the primary IPCC score) using
topology_sibling_cpumask(). This preserves the SMT > dense-core advantage
of classic Zen 5 SMT siblings ahead of Zen 5c cores while guiding
SD_ASYM_PACKING to prefer idle physical cores first.

Developed with coding assistance from Antigravity (Google DeepMind).

Tested-by: Aiden Hecker <heckeraiden@gmail.com>
Signed-off-by: Aiden Hecker <heckeraiden@gmail.com>
---
 drivers/platform/x86/amd/hfi/hfi.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/drivers/platform/x86/amd/hfi/hfi.c b/drivers/platform/x86/amd/hfi/hfi.c
index 86a7f4332..d7e35a2bf 100644
--- a/drivers/platform/x86/amd/hfi/hfi.c
+++ b/drivers/platform/x86/amd/hfi/hfi.c
@@ -12,6 +12,7 @@
 
 #include <linux/acpi.h>
 #include <linux/cpu.h>
+#include <linux/cpumask.h>
 #include <linux/debugfs.h>
 #include <linux/gfp.h>
 #include <linux/init.h>
@@ -261,11 +262,26 @@ static void amd_hfi_remove(struct platform_device *pdev)
 
 static int amd_set_hfi_ipcc_score(struct amd_hfi_cpuinfo *hfi_cpuinfo, int cpu)
 {
+	const struct cpumask *sib_mask;
+	int prio;
+
 	for (int i = 0; i < hfi_cpuinfo->nr_class; i++)
 		WRITE_ONCE(hfi_cpuinfo->ipcc_scores[i],
 			   hfi_cpuinfo->amd_hfi_classes[i].perf);
 
-	sched_set_itmt_core_prio(hfi_cpuinfo->ipcc_scores[0], cpu);
+	/*
+	 * Rank secondary SMT siblings slightly below primary threads so
+	 * SD_ASYM_PACKING fills idle physical cores first while still keeping
+	 * classic Zen 5 SMT siblings ordered ahead of dense Zen 5c cores.
+	 */
+	prio = hfi_cpuinfo->ipcc_scores[0];
+	if (cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES)) {
+		sib_mask = topology_sibling_cpumask(cpu);
+		if (!cpumask_empty(sib_mask) && cpumask_first(sib_mask) != cpu)
+			prio = (prio * 7) / 8;
+	}
+
+	sched_set_itmt_core_prio(prio, cpu);
 
 	return 0;
 }
-- 
2.56.1