Re: [RFC PATCH 0/4] cpufreq/amd-pstate: Per-core EPP boost for recently-busy CPUs

[David Vernet <void@manifault.com>]

[-- Attachment #1: Type: text/plain, Size: 12459 bytes --]

On Wed, Jul 29, 2026 at 08:42:45AM +0530, K Prateek Nayak wrote:

Hi Prateek!

[...]

> >>> This series instead adds an opt-in, per-core EPP boost. When the
> >>> epp_boost module parameter is enabled, an update-util hook samples each
> >>> core's C0 residency (delta MPERF over delta TSC) at most once every
> >>> 10 ms. If a sample shows the core at least 50% busy, the EPP field of
> >>> its MSR_AMD_CPPC_REQ is set to performance (0) and held there until
> >>> 300 ms pass without another busy sample, at which point the hook
> >>> restores the request that policy management last stored in
> >>> cppc_req_cached. Both writes happen only on the busy and idle edges, so
> >>> the CPPC_REQ write rate matches that of a global EPP=performance
> >>> setting. min, max and desired perf are never touched. The mechanism is
> >>> only available in active mode on MSR (X86_FEATURE_CPPC) systems, since
> >>> the hook does local MSR accesses from scheduler context which the
> >>> shared memory interface cannot do. It composes with dynamic_epp, which
> >>> selects the policy EPP from the platform profile and power source.
> >>> epp_boost temporarily overrides whatever policy EPP is installed and
> >>> restores it when the core goes idle.
> 
> I would also like to know Rafael's thoughts on kernel modifying the
> EPP to something other than what user has programmed.
> 
> There are a ton of userspace tools that have currently profiled a bunch
> of systems and use the EPP interface to program a good default and their
> expectations may diverge if kernel starts modifying EPP underneath them.

Yeah that makes sense. I figured EPP was the right abstraction to tune
if we're going to use scheduler input to provide hints to firmware but
yes it is indeed modifying EPP under what was explicitly configured. My
thinking was that this is an opt-in parameter but I agree that it's
maybe not desirable to change it underneath what was explicitly set by
the user.

> This almost feels like putting the ondemand governor into the driver.
> Perhaps, the active mode drivers can be made to support the dynamic
> governors as opposed to just the static ones? that fits the overall
> cpufreq subsystem better but I'll wait on Rafael's thoughts on that.

Good question. I'll defer to you folks on whether that direction makes
sense. If it does and would help facilitate this, I'm happy to help
explore it.

> > Have you seen the series from Prateek that reworks how dynamic EPP
> > works [1]?  It becomes an energy_performance_preference that userspace
> > can opt into.
> 
> I feel like that will come short of the expectations here because we
> would like to do this per-CPU based on utilization metrics as opposed
> to system profile / power status both of which are global and can lead
> to faster battery drain.

Yep, +1, I do think we need per-CPU tuning here. That said, I am happy
to test dynamic EPP and see how it fares. I can try to find time to do
that this weekend.

> > I did have aspirations to hook into more changes than just platform
> > profile and power source eventually, so it sounds like we're at least
> > thinking in the same area.
> 
> Ack! We'll at least need more input to the dynamic mode to cater to
> this specific case but it is going into the governor territory at that
> point if we want to use CPU specific signals.
> 
> > 
> > [1] https://lore.kernel.org/linux-pm/20260727072056.1248-1-kprateek.nayak@amd.com/
> > 
> > Can you rework your series on top of that and see how the mechanics work out?
> > 
> >>>
> >>> Precedents
> >>> ==========
> >>>
> >>> The closest precedent is intel_pstate's hwp_boost. It has the same
> >>> overall shape as this feature. It is an opt-in update-util hook that
> >>> temporarily rewrites the HWP request from scheduler context on a boost
> >>> edge and restores the unboosted request after a hardcoded hold time
> >>> (hwp_boost_hold_time_ns). It differs in two ways, both deliberate:
> >>>
> >>> 1. Trigger. hwp_boost activates on SCHED_CPUFREQ_IOWAIT. The waits
> >>>     that matter here are futex waits and amdgpu fence waits, which do
> >>>     not set the iowait flag, so a C0 residency trigger is used instead.
> >>>     Residency also naturally covers the "mostly busy with short gaps"
> >>>     pattern rather than only the wakeup instant.
> >>>
> >>> 2. Knob being boosted. hwp_boost raises the HWP min.
> 
> I think that is why it is acceptable to be used with FW based scaling
> as it leaves the user programmed EPP alone.
> 
> >>>                                                       As described
> >>>     above, a min_perf floor measurably regressed the tail on Van Gogh,
> >>>     so this feature biases only the EPP hint and leaves the platform
> >>>     free to drop the operating point during the idle portions of the
> >>>     frame.
> >>>
> >>> On the thresholds themselves, the sample period, busy threshold and
> >>> decay window are hardcoded rather than exposed as tunables. I'm not sure
> >>> if this is appropriate or not, but it seemed like it followed existing
> >>> contours.
> >>>
> >>> hwp_boost_hold_time_ns for example is a hardcoded 3 ms, and schedutil's
> >>> iowait boost decay is tied to TICK_NSEC, with no knobs for either. The
> >>> 300 ms decay is sized so that a render thread which is only 50-80% busy
> >>> from periodic vsync and GPU-fence waits holds the boost across its whole
> >>> busy period at a couple of CPPC_REQ writes total, while an idle core
> >>> sheds the boost well before it can matter. The energy exposure of a wide
> >>> window is small because EPP only influences behavior in C0 and an idle
> >>> core sits in CC6 regardless. If folks want me to make these tunable I am
> >>> happy to expose them.
> > 
> > In a lot of ways I feel like you're fighting with the hardware (and
> > active mode) by doing it this way.  Did you look into using passive
> > or guided mode instead?  They might be better suited for what you're
> > trying to do.
> 
> Ack! Seems like UCLAMP_MIN + natural accumulation of PELT signals when
> running with passive mode + schedutil can achieve the desired results
> here but ...

Actually, the first thing I tried in general with this effort was to
raise min_perf to nominal on busy cores, with an idle-path restore
(answering your question below). It reached the same 3.5 GHz median as
the EPP approach but surprisngly regressed frame-time p999 by 13-21%
which I was of course not expecting. I'm not 100% sure why this happens
but I _think_ what's going on is that on this APU, the CPU and GPU share
one SMU-managed power budget, and holding a frequency floor through the
micro-idle portions of each frame (vsync and fence waits) takes away the
headroom the SMU needs to keep the GPU fed.

I'm not sure this is what's happening but in the traces I was looking at
where this occurs, I didn't see CPU pressure so I'm assuming it was GPU
hitches. I'll validate this again over the weekend. I also haven't tried
using UCLAMP_MIN, so I'll put that on my TODO list as well.

With all of that said, something I did realize while looking into this
is that Proton doesn't actually implement the functions to tag the
render thread [0] (Changwoo please correct me if I'm misunderstanding
something here).

[0]: https://github.com/ValveSoftware/wine/blob/proton_11.0/dlls/avrt/main.c#L57

If we were to leverage this and be able to configure these high priority
tasks with the correct UCLAMP settings, etc, I think it's likely that it
would give us as good or better results, yes. Though, I guess that's
still assuming that the SMU will do the right thing w/ the CPU/GPU power
budgeting.

> > 
> >>>
> >>> Testing methodology
> >>> ===================
> >>>
> >>> All numbers are from a Steam Deck LCD (Van Gogh APU) running in active
> >>> mode at EPP=balance_performance, using the Civilization VI graphics
> >>> benchmark as a single-thread CPU-bound workload with a repeatable
> >>> built-in benchmark pass.
> >>>
> >>> Comparisons were run as interleaved A/B tests with 6 iterations per
> >>> configuration. For each run I collected per-frame frame times and the
> >>> busy core's frequency, and derived average fps, 1%-low fps and the p99
> >>> and p999 frame-time percentiles. Deltas were evaluated with Welch's
> >>> t-test, and I report the p-values alongside the deltas below.
> >>>
> >>> Results:
> >>>
> >>> Default settings
> >>> ----------------
> >>> The busy core's median frequency sat at 2.43 GHz despite 98%
> >>> utilization, which is the frequency droop described above.
> >>
> >> Two things to confirm for my understanding:
> >> -Median frequency is the 50th percentile when weighing the OPPs by
> >> residency, right?
> >> - You're also using busy_pct = delta_MPERF * 100 / delta_TSC as
> >> utilization, right? (Not util_avg or anything like that)
> >>
> >> Your observation looks like a firmware or SMU issue.
> >> Can we confirm the (short) sleeps reset the demand-estimation, i.e.
> >> by tracing cpu_idle/sched_switch and sample APERF/MPERF alongside it?
> >> Also rt-app might be helpful to get a feel of how this behaves (and
> >> create a similar pathological case like the single-threaded game).
> >>
> >>>
> >>> Global EPP=performance
> >>> ----------------------
> >>> Globally forcing EPP=performance lifts the median to 3.5 GHz, cuts
> >>> frame-time p999 by ~40% and raises 1%-low fps by ~16%, but does so on
> >>> every core and for every workload.
> >>>
> >>> Raising min_perf to nominal on the busy core
> >>> --------------------------------------------
> >>> A variant of this patch that instead raised min_perf to nominal on the
> >>> busy core reached the same 3.5 GHz median yet regressed p999 by 13-21%
> >>> by perturbing the SMU boost management as described above.
> 
> ... this is interesting. Isn't Steam Deck a Zen2 machine? Does that
> have the MSR interface for CPPC?

Yep. Van Gogh sets X86_FEATURE_CPPC and amd-pstate runs the MSR path on
it. Also, FYI I just realized that in the cover letter I said that I was
using the LCD Steam Deck but I double checked and I actually have the
newer OLED one. So all of the data in the cover letter is for an OLED
Steam Deck.

> Also when you say you set min_perf on a busy_core, do you restore it
> back on the idle path?

Yes, I did the same thing there as the posted series (meaning raise it
on the busy edge and restore after a decay window with no busy samples).

> I'm assuming what happens with min_perf method is that SMU is more
> aggressive at delivering nominal_perf and beyond, compared to the EPP
> performance route which perhaps starts off lower but has a steeper
> rampup curve compared to EPP balanced.

That reading is consistent with everything I have measured so far. The
trace data that informed my reply to Christian [1] shows that under
balance_performance, any block longer than ~0.2 ms resets the busy core
to a sticky ~2.4 GHz plateau, independent of which C states are
available. So the EPP hint seems to govern how the core recovers after a
sleep rather than preventing the reset, while the min floor prevents the
drop across the sleep entirely, which is seemingly what perturbs the
shared CPU/GPU budget here (again I'm not sure about this but I think
that's what's happening because I didn't see CPU contention in the
traces where this was happening). The synthetic reproducer I'm putting
together this weekend should let me measure the two rampup curves
directly, and I'll post those numbers in this thread.

[1]: https://lore.kernel.org/all/amrZ08BnDP4OriHa@maniforge/

> Could you get some turbostat traces for both case with Civ VI to see
> how the frequency scaling behaves with the EPP route vs min_perf
> method?

Yes, I will do this too. It's been a crazy oncall week at work so I've been
pretty tied up, but planning to put some more time into this over the weekend.

> I'll try to compare your method vs min_perf = nominal_perf method on
> my Zen3 laptop over the coming weekend to see if I'm able to repro
> this SMU behavior.

Nice, yeah that would be great. I'm not sure if it will be fully apples
to apples given the whole shared power budget thing, but would
definitely be good to see if you can reproduce on your end.

Thanks,
David

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]