[RFC PATCH 0/4] cpufreq/amd-pstate: Per-core EPP boost for recently-busy CPUs

[David Vernet <void@manifault.com>]

In active (EPP) mode the platform autonomously picks the operating point
between min_perf and max_perf, biased by the EPP hint, and the kernel
only rewrites the CPPC request on policy or limit changes. A workload
dominated by one mostly-busy thread that takes frequent short sleeps
(common in gaming workloads, for example) can fare poorly under this
strategy. Each sleep decays the hardware's performance signal, causing
post-wakeup bursts to start at a low operating point and inflating tail
latency even though the CPU is essentially fully busy while work is
available.

As mentioned above, the motivating case here is gaming. A game's main or
render thread typically blocks briefly on a futex or a GPU fence every
frame, and the resulting frequency droop shows up directly as stale
frames and inflated frame-time percentiles.

Solving this at the cpufreq layer requires walking a fairly narrow
path. Globally forcing EPP=performance fixes the tail but burns power
on every core for every workload, which matters on handhelds where the
CPU and GPU share a power budget. Raising min_perf on the busy core
seems like the obvious surgical fix, but it pins the core at or above
nominal even during micro-idle and vsync waits. On Van Gogh (Steam
Deck) experiments that I ran, that perturbs the SMU's shared CPU/GPU
boost management enough to regress the frame-time tail relative to doing
nothing (numbers below).

This series instead adds an opt-in, per-core EPP boost. When the
epp_boost module parameter is enabled, an update-util hook samples each
core's C0 residency (delta MPERF over delta TSC) at most once every
10 ms. If a sample shows the core at least 50% busy, the EPP field of
its MSR_AMD_CPPC_REQ is set to performance (0) and held there until
300 ms pass without another busy sample, at which point the hook
restores the request that policy management last stored in
cppc_req_cached. Both writes happen only on the busy and idle edges, so
the CPPC_REQ write rate matches that of a global EPP=performance
setting. min, max and desired perf are never touched. The mechanism is
only available in active mode on MSR (X86_FEATURE_CPPC) systems, since
the hook does local MSR accesses from scheduler context which the
shared memory interface cannot do. It composes with dynamic_epp, which
selects the policy EPP from the platform profile and power source.
epp_boost temporarily overrides whatever policy EPP is installed and
restores it when the core goes idle.

Precedents
==========

The closest precedent is intel_pstate's hwp_boost. It has the same
overall shape as this feature. It is an opt-in update-util hook that
temporarily rewrites the HWP request from scheduler context on a boost
edge and restores the unboosted request after a hardcoded hold time
(hwp_boost_hold_time_ns). It differs in two ways, both deliberate:

1. Trigger. hwp_boost activates on SCHED_CPUFREQ_IOWAIT. The waits
   that matter here are futex waits and amdgpu fence waits, which do
   not set the iowait flag, so a C0 residency trigger is used instead.
   Residency also naturally covers the "mostly busy with short gaps"
   pattern rather than only the wakeup instant.

2. Knob being boosted. hwp_boost raises the HWP min. As described
   above, a min_perf floor measurably regressed the tail on Van Gogh,
   so this feature biases only the EPP hint and leaves the platform
   free to drop the operating point during the idle portions of the
   frame.

On the thresholds themselves, the sample period, busy threshold and
decay window are hardcoded rather than exposed as tunables. I'm not sure
if this is appropriate or not, but it seemed like it followed existing
contours.

hwp_boost_hold_time_ns for example is a hardcoded 3 ms, and schedutil's
iowait boost decay is tied to TICK_NSEC, with no knobs for either. The
300 ms decay is sized so that a render thread which is only 50-80% busy
from periodic vsync and GPU-fence waits holds the boost across its whole
busy period at a couple of CPPC_REQ writes total, while an idle core
sheds the boost well before it can matter. The energy exposure of a wide
window is small because EPP only influences behavior in C0 and an idle
core sits in CC6 regardless. If folks want me to make these tunable I am
happy to expose them.

Testing methodology
===================

All numbers are from a Steam Deck LCD (Van Gogh APU) running in active
mode at EPP=balance_performance, using the Civilization VI graphics
benchmark as a single-thread CPU-bound workload with a repeatable
built-in benchmark pass.

Comparisons were run as interleaved A/B tests with 6 iterations per
configuration. For each run I collected per-frame frame times and the
busy core's frequency, and derived average fps, 1%-low fps and the p99
and p999 frame-time percentiles. Deltas were evaluated with Welch's
t-test, and I report the p-values alongside the deltas below.

Results:

Default settings
----------------
The busy core's median frequency sat at 2.43 GHz despite 98%
utilization, which is the frequency droop described above.

Global EPP=performance
----------------------
Globally forcing EPP=performance lifts the median to 3.5 GHz, cuts
frame-time p999 by ~40% and raises 1%-low fps by ~16%, but does so on
every core and for every workload.

Raising min_perf to nominal on the busy core
--------------------------------------------
A variant of this patch that instead raised min_perf to nominal on the
busy core reached the same 3.5 GHz median yet regressed p999 by 13-21%
by perturbing the SMU boost management as described above.

epp_boost enabled
-----------------
With epp_boost enabled, the busy core running the Civ VI main thread
runs at a 3.5 GHz median and the benchmark gains 31.8% in 1%-low fps
(p=0.014) and 4.1% in frame-time p99 (p=0.015), with p999 and average
fps unchanged.

David Vernet (4):
  cpufreq/amd-pstate: Document missing kernel-doc members
  cpufreq/amd-pstate: Update cppc_req_cached before writing the MSR
  cpufreq/amd-pstate: Add per-core EPP boost for recently-busy CPUs
  Documentation: amd-pstate: Document the epp_boost parameter

 Documentation/admin-guide/pm/amd-pstate.rst |  16 ++
 drivers/cpufreq/amd-pstate.c                | 230 ++++++++++++++++++++++++++--
 drivers/cpufreq/amd-pstate.h                |  20 +++
 3 files changed, 255 insertions(+), 11 deletions(-)


base-commit: 92bf086d086f7cfe0d6f807dfd6b8d11bc61b626
-- 
2.53.0