From: Lance Yang <lance.yang@linux.dev>
To: atomlin@atomlin.com
Cc: akpm@linux-foundation.org, lance.yang@linux.dev,
mhiramat@kernel.org, pmladek@suse.com,
linux-kernel@vger.kernel.org, david.laight.linux@gmail.com,
neelx@suse.com, sean@ashe.io, chjohnst@gmail.com, steve@abita.co,
mproche@gmail.com, nick.lange@gmail.com
Subject: Re: [PATCH v2] hung_task: Add per-round stack trace deduplication
Date: Sat, 20 Jun 2026 11:37:15 +0800 [thread overview]
Message-ID: <20260620033715.71108-1-lance.yang@linux.dev> (raw)
In-Reply-To: <20260620013559.1537893-1-atomlin@atomlin.com>
Hi Aaron,
On Fri, Jun 19, 2026 at 09:35:59PM -0400, Aaron Tomlin wrote:
>Currently, when multiple tasks hang in the exact same location (e.g.,
>such as severe contention for a mutex), khungtaskd indiscriminately
>reports every single instance. This wastes ring buffer space with
>identical stack traces up to the defined warning limit (i.e.,
>kernel.hung_task_warnings), obscuring the root cause without providing
>any additional diagnostic value.
>
>Introduce a lightweight, hash-based stack trace deduplicator for
>khungtaskd to ensure only unique stack traces are reported during
>a single detection interval.
>
>Technical details of the implementation:
> - Uses a 12-bit hash table (4096 slots), consuming just 16 KB of
> static memory to prevent cache thrashing during massive hangs.
>
> - Operates purely serially within the single khungtaskd thread,
> requiring zero atomic operations or concurrent locking overhead.
>
> - Flushes the lossy cache via memset() at the beginning of each
> detection round. This ensures the immediate "thundering herd" of
> duplicates is suppressed, but guarantees the system will not
> permanently suppress identical hangs that occur in future rounds.
>
> - Introduces a new sysctl, kernel.hung_task_dedup, which defaults to 1
> (enabled). The sysctl is locally cached at the outset of each
> interval to prevent tearing caused by concurrent userspace toggling.
>
Thanks for working on this, but ... guess I'll be the bad guy here, not
convinced this should go in ...
When khungtaskd fires, somthing is already wrong, no? I don't see why it
should grow a new sysctl, a stack hash table, and extra filtering logic
just ot hide part of the report ...
Emm ... do you have real cases where duplicate hung-task stacks caused
serious pain?
If many tasks hang at once, usually one root cause, not a bunch of
different bugs. At least from what I've seen, any one of those stacks is
enough to start debugging ...
We already have hung_task_detect_count and trace_sched_process_hang() for
basic counting/observability. Even if hung_task_warnings is finite and
the warning budget runs out, we still don't lose detections: counter gets
bumped and tracepoint fires before printk output is gated :)
If someone wants stack grouping, I'd rather leave that to a tool than add
another policy knob to khungtaskd. Once it lands, maintainers have to
carry it forever. Not every nice-to-have feature is worth that cost, IMHO
And if someone really wants more hung-task stacks in the log, we already
have hung_task_warnings for that. Raise it, or set it to -1.
Also, looking at the v1 thread, I don't think the concerns there have
really settled yet ... If nobody replies, maybe give it a week before
sending a new version.
Thanks, Lance
next prev parent reply other threads:[~2026-06-20 3:37 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-20 1:35 [PATCH v2] hung_task: Add per-round stack trace deduplication Aaron Tomlin
2026-06-20 3:37 ` Lance Yang [this message]
2026-06-20 17:54 ` Aaron Tomlin
2026-06-21 5:17 ` Lance Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260620033715.71108-1-lance.yang@linux.dev \
--to=lance.yang@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=atomlin@atomlin.com \
--cc=chjohnst@gmail.com \
--cc=david.laight.linux@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mproche@gmail.com \
--cc=neelx@suse.com \
--cc=nick.lange@gmail.com \
--cc=pmladek@suse.com \
--cc=sean@ashe.io \
--cc=steve@abita.co \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox