[Bug 221909] amd-pstate: Cezanne (Ryzen 7 5800U) data fabric sync flood on DC gated by CPPC max_perf

[bugzilla-daemon@kernel.org]

https://bugzilla.kernel.org/show_bug.cgi?id=221909

--- Comment #5 from David Smith (smithd98@gmail.com) ---
UPDATE 2026-08-27. ONE CLEAN NEW RESULT, AND TWO MECHANISMS EXCLUDED -
INCLUDING THE ONE I WAS ABOUT TO PROPOSE.

This follows comment #4 and does not repeat it. Comment #4's withdrawals all
still stand. What is new here is a single-variable A/B pair that isolates the
frequency cap, and which also falsifies the voltage mechanism I had been
building toward. I am reporting the exclusions because a falsified mechanism
seems more useful than the confounded correlation I would otherwise have
published.

=====================================================
1. THE ONE CLEAN RESULT: A SINGLE-VARIABLE PAIR
=====================================================
Everything in comment #0, and most of comment #4, is confounded in some way.
This is not. It is the only experiment in this investigation in which exactly
one variable moves.

Both arms: same boot, same machine, same battery, same workload (llama.cpp
128-token prefill, ~6.7 GB model, verified resident, CPU only), 27 minutes
apart. Both with amdgpu power_dpm_force_performance_level=high, which pins
FCLK at its top DPM state. CPU VDD plane sampled at 492 Hz.

                              LETHAL              SURVIVOR
  scaling_max_freq            4508086             3800000
  FCLK                        1333 MHz, 100%      1333 MHz, 100%
  FCLK transitions            0 in 23864 samples  0 in 66913 samples
  VDDNB (SoC rail)            818 mV 84.5%        818 mV 94.0%
  CPU VDD plane, peak         1250 mV             1281 mV
  CPU VDD plane, % >=1300 mV  0.00%               0.00%
  CPU VDD plane, % 1100-1300  25.6%               33.0%
  exposure                    died at 20.9 s      survived 318 s, 9/9 steps
  outcome                     0x08000800          clean

scaling_max_freq is the only thing that differs. Prior cap-efficacy evidence
in this bug was confounded: the compared runs differed in the cap AND in the
power source. This pair is not.

Note how harsh the survivor's configuration is. FCLK pinned at 1333 MHz is the
modifier that kills this machine FASTEST at stock max_perf - 11.6 s in one
arm, 21 s in another. Under the cap it ran 318 s clean.

=====================================================
2. EXCLUDED: CPU-PLANE VOLTAGE IS NOT THE MECHANISM
=====================================================
I had built a voltage story and was about to publish it: on DC at low thread
counts the plane sits at 1.37-1.39 V for ~99% of the time, and I believed the
cap protects by removing that operating point.

The pair above kills it, in the awkward direction:

  the run that SURVIVED sat at a HIGHER peak plane voltage (1281 vs 1250 mV)
  and spent MORE time in the 1100-1300 mV band (33.0% vs 25.6%), for 15x
  longer exposure - and lived.

So CPU-plane voltage is not the discriminator: not as a peak, not as
residency, not as dose. Withdrawn before publication rather than after.

PROVENANCE CORRECTION, which anyone reading my hwmon numbers needs: the
"vddgfx" column in comment #0's gating table is NOT a GPU rail. On this part
it is the shared VDDCR_VDD plane and it tracks the CPU V/F point. Verified
directly: 1 thread -> cores 4574 MHz, plane 1393 mV; 16 threads -> cores
2761 MHz, plane 931 mV, i.e. the plane follows the CPU down while the iGPU
clock also falls. Any reading of that column as GPU behaviour, mine included,
was wrong.

SECOND INSTRUMENTATION CAVEAT: I sampled that plane at 4 Hz for most of this
investigation, while it updates about every 2.7 ms - roughly 1 value in 92.
Any voltage conclusion in comment #0 or comment #4 was drawn from aliased
data. The 492 Hz figures above are the first honest measurements I have of
that rail.

=====================================================
3. EXCLUDED: FABRIC DPM TRANSITIONS ARE NOT NECESSARY
=====================================================
Since the failure is a fabric sync flood, the obvious question is whether the
fabric's own DPM transitions are involved. They are not:

  FCLK pinned at 1333 MHz, 100% residency, ZERO transitions across 23864
  samples at 492 Hz - and the machine sync-flooded anyway, 21 s in.

I had also observed FCLK thrash (400<->1200, 13 transitions in 888 ms) in an
earlier death captured at 50 Hz, and wondered whether it was causal. In this
death the thrash is not merely downstream - it is entirely absent.

Combined with section 2, and with VDDNB identical at 818 mV on both sides of
the pair, that leaves:

  EXCLUDED:  fabric DPM transitions, SoC-rail level, CPU-plane voltage
  REMAINING: peak core frequency itself, or core current / di-dt at the top
             P-states

i.e. the fault lives in the top ~700 MHz, between 3.8 and 4.5 GHz, and is not
reachable through the plane voltage, the fabric clock, or the SoC rail.

I cannot narrow it further from here. There is no writable PPT interface on
this part - amdgpu hwmon exposes only power1_input, read-only - so I cannot
hold frequency at stock and lower the power budget independently.

The band 3.83-4.45 GHz is entirely untested. My coverage is <=3.80 GHz (0
deaths, 52 steps), 4.48 GHz (1/1 dead), and 4.51 GHz stock (dies reliably). A
cap at ~4.1 GHz bisects it. I will run that bisection if it is useful to
anyone.

=====================================================
4. INSTRUMENTATION WARNING: journald UNDER-REPORTS THE MOMENT OF DEATH
=====================================================
Relevant to anyone else triaging these resets from logs. In the death I
captured at 492 Hz:

  journald's last write      09:42:52
  my sampler's last write    09:43:00.907   (fsync'd every 0.25 s)

journald stops being flushed 8.0 seconds before the part actually resets. So
the last line in your journal is NOT the moment of death, and "the machine was
idle when it died, because the last log line is idle" is an unsafe inference.

=====================================================
5. CORRECTIONS TO COMMENT #4
=====================================================
(a) TIMING CLAIM WITHDRAWN. Comment #4 section 7 says the compute load
    collapses to idle "0.3-2.8 s BEFORE" the reset, and treats that as cutting
    against a simple di/dt story. Withdraw that. Those gaps were computed
    against journald's last write, which section 4 above shows is up to 8.0 s
    early, so they are not trustworthy. In the one death measured against a
    reliable clock the collapse precedes the reset by under 0.4 s: last busy
    sample 09:43:00.510 at 3065 MHz mean, first collapsed sample .758 at
    498 MHz, reset at .907 - bounded between 0.15 and 0.40 s by the 250 ms
    sampler. I am no longer claiming a long quiet interval before the flood.

(b) UPDATED TALLIES.
      At stock max_perf on DC:   14 deaths in 15 valid arms
      At smax <= 3800000 on DC:  52 ladder steps + a 35-min mixed soak
                                 + a 20-min integrity soak, 0 deaths
    The 52 now includes the 9 steps from section 1, run with FCLK pinned at
    1333 MHz. The ledger since 2026-08-21 now records 23 data-fabric sync
    floods plus one hard hang.

(c) The userspace-fault control in comment #4 section 7 becomes: two faults in
    ~15 valid stock-max_perf arms, zero across 52 capped ladder steps and two
    capped soaks.

=====================================================
6. WHERE IT STANDS
=====================================================
Still mitigated but not root-caused, and the mitigation is unchanged in form:
cap scaling_max_freq to 3800000 on DC only.

What is now closed: BIOS (three versions), C6 (both directions), EPP,
governor, min_perf pinning, fabric clock pinning, three kernels, and - new
here - CPU-plane voltage and SoC-rail level as mechanisms.

What remains is peak core frequency itself, or core current / di-dt at the top
P-states in the 3.8-4.5 GHz band, plus the DC power path, which I cannot close
without a replacement pack I am not able to test. I cannot separate those two
from userspace: there is no writable PPT interface here, and every voltage and
power number available to me from hwmon is an SMU self-report, while the one
independent instrument - the battery EC - refreshes about every 9 s against an
event lasting under a second.

ADDITIONAL QUESTIONS
====================
These are in addition to the four in comment #4.

5. Is there any Cezanne rail telemetry that is NOT an SMU self-report? If the
   SMU is the suspect and also the only witness, I cannot distinguish "the SMU
   commanded a bad V/F point" from "the rail sagged and the SMU did not see
   it".

6. Is there a debug interface to set a PPT limit on this part? amdgpu exposes
   only power1_input read-only here, so I cannot separate frequency from power
   budget - which is exactly the separation section 3 now needs.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.