From: Lance Yang <lance.yang@linux.dev>
To: atomlin@atomlin.com
Cc: akpm@linux-foundation.org, lance.yang@linux.dev,
mhiramat@kernel.org, pmladek@suse.com,
linux-kernel@vger.kernel.org, david.laight.linux@gmail.com,
neelx@suse.com, sean@ashe.io, chjohnst@gmail.com, steve@abita.co,
mproche@gmail.com, nick.lange@gmail.com
Subject: Re: [PATCH v2] hung_task: Add per-round stack trace deduplication
Date: Sat, 20 Jun 2026 11:37:15 +0800 [thread overview]
Message-ID: <20260620033715.71108-1-lance.yang@linux.dev> (raw)
In-Reply-To: <20260620013559.1537893-1-atomlin@atomlin.com>
Hi Aaron,
On Fri, Jun 19, 2026 at 09:35:59PM -0400, Aaron Tomlin wrote:
>Currently, when multiple tasks hang in the exact same location (e.g.,
>such as severe contention for a mutex), khungtaskd indiscriminately
>reports every single instance. This wastes ring buffer space with
>identical stack traces up to the defined warning limit (i.e.,
>kernel.hung_task_warnings), obscuring the root cause without providing
>any additional diagnostic value.
>
>Introduce a lightweight, hash-based stack trace deduplicator for
>khungtaskd to ensure only unique stack traces are reported during
>a single detection interval.
>
>Technical details of the implementation:
> - Uses a 12-bit hash table (4096 slots), consuming just 16 KB of
> static memory to prevent cache thrashing during massive hangs.
>
> - Operates purely serially within the single khungtaskd thread,
> requiring zero atomic operations or concurrent locking overhead.
>
> - Flushes the lossy cache via memset() at the beginning of each
> detection round. This ensures the immediate "thundering herd" of
> duplicates is suppressed, but guarantees the system will not
> permanently suppress identical hangs that occur in future rounds.
>
> - Introduces a new sysctl, kernel.hung_task_dedup, which defaults to 1
> (enabled). The sysctl is locally cached at the outset of each
> interval to prevent tearing caused by concurrent userspace toggling.
>
Thanks for working on this, but ... guess I'll be the bad guy here, not
convinced this should go in ...
When khungtaskd fires, somthing is already wrong, no? I don't see why it
should grow a new sysctl, a stack hash table, and extra filtering logic
just ot hide part of the report ...
Emm ... do you have real cases where duplicate hung-task stacks caused
serious pain?
If many tasks hang at once, usually one root cause, not a bunch of
different bugs. At least from what I've seen, any one of those stacks is
enough to start debugging ...
We already have hung_task_detect_count and trace_sched_process_hang() for
basic counting/observability. Even if hung_task_warnings is finite and
the warning budget runs out, we still don't lose detections: counter gets
bumped and tracepoint fires before printk output is gated :)
If someone wants stack grouping, I'd rather leave that to a tool than add
another policy knob to khungtaskd. Once it lands, maintainers have to
carry it forever. Not every nice-to-have feature is worth that cost, IMHO
And if someone really wants more hung-task stacks in the log, we already
have hung_task_warnings for that. Raise it, or set it to -1.
Also, looking at the v1 thread, I don't think the concerns there have
really settled yet ... If nobody replies, maybe give it a week before
sending a new version.
Thanks, Lance
next prev parent reply other threads:[~2026-06-20 3:37 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-20 1:35 [PATCH v2] hung_task: Add per-round stack trace deduplication Aaron Tomlin
2026-06-20 3:37 ` Lance Yang [this message]
2026-06-20 17:54 ` Aaron Tomlin
2026-06-21 5:17 ` Lance Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260620033715.71108-1-lance.yang@linux.dev \
--to=lance.yang@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=atomlin@atomlin.com \
--cc=chjohnst@gmail.com \
--cc=david.laight.linux@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mproche@gmail.com \
--cc=neelx@suse.com \
--cc=nick.lange@gmail.com \
--cc=pmladek@suse.com \
--cc=sean@ashe.io \
--cc=steve@abita.co \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.