Re: the stuttering regression in 7.0: should I have done something different

[Thomas Gleixner <tglx@kernel.org>]

On Mon, May 11 2026 at 22:03, Tony Rodriguez wrote:
>> Can you apply the debug patch below, which will disable tracing once it
>> hits the hung task detector and then retrieve the trace? 
> As requested, I applied your debug patch to v7.1‑rc3 and captured the
> trace output.  On the SPARC64 S7‑2 system the machine becomes
> unresponsive and produces many thousands of lines of trace data that
> do not appear to terminate.

Yes, it takes a while to spill out over serial.

> Posting the full output inline or as an attachment may be impractical,
> so I’ve included the key sections below.

Kinda.

> If you prefer the complete trace, please let me know the best way to 
> provide it. Guessing the kernel mailing isn't best to attach that?

Correct.

> [  249.004209] [<0000000000000000>] 0x0
> [  249.019116] Dumping ftrace buffer:
> [  249.025666] ---------------------------------
> [  249.034534]   <idle>-0         0d.... 1836659us : 
> clockevents_program_event: Successfully programmed 4000000 4000000
> [  249.055418]   <idle>-0         0d.h.. 1845926us : timer_interrupt: 

So this is the interesting part, but that's starting at 1.836659s
while the actual problem happens ~120 seconds later and the detection
takes another 120 seconds.

Assuming that one of the CPUs does not get timer interrupts anymore, the
trace of that CPU should end around the time the last programming
happened. So the interesting part is at the end of the output. The
default buffer size per CPU is 1408k, which holds about 150k entries, so
we can just shorten the buffers to make this less painful.

Can you add 'trace_buf_size=50k' to the kernel command line, which
limits the buffer size to about 640 entries. Assuming 115200 Baud this
should then take about 4 seconds per CPU to dump, which still is a bunch
on a large machine, but definitely way more workable than the default.

IIRC, SPARC64 S7‑2 has 128 threads total, so the resulting uncompressed
output should be around 7-8M. That's highly compressable text, so the
resulting dump.xz should be suitable to be stored in github. If github
does not allow you, let me know and we work something out.

Thanks,

        tglx