The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Lance Yang <lance.yang@linux.dev>
To: atomlin@atomlin.com
Cc: akpm@linux-foundation.org, lance.yang@linux.dev,
	mhiramat@kernel.org, pmladek@suse.com,
	linux-kernel@vger.kernel.org, david.laight.linux@gmail.com,
	neelx@suse.com, sean@ashe.io, chjohnst@gmail.com, steve@abita.co,
	mproche@gmail.com, nick.lange@gmail.com
Subject: Re: [PATCH v2] hung_task: Add per-round stack trace deduplication
Date: Sat, 20 Jun 2026 11:37:15 +0800	[thread overview]
Message-ID: <20260620033715.71108-1-lance.yang@linux.dev> (raw)
In-Reply-To: <20260620013559.1537893-1-atomlin@atomlin.com>

Hi Aaron,

On Fri, Jun 19, 2026 at 09:35:59PM -0400, Aaron Tomlin wrote:
>Currently, when multiple tasks hang in the exact same location (e.g.,
>such as severe contention for a mutex), khungtaskd indiscriminately
>reports every single instance. This wastes ring buffer space with
>identical stack traces up to the defined warning limit (i.e.,
>kernel.hung_task_warnings), obscuring the root cause without providing
>any additional diagnostic value.
>
>Introduce a lightweight, hash-based stack trace deduplicator for
>khungtaskd to ensure only unique stack traces are reported during
>a single detection interval.
>
>Technical details of the implementation:
> - Uses a 12-bit hash table (4096 slots), consuming just 16 KB of
>   static memory to prevent cache thrashing during massive hangs.
>
> - Operates purely serially within the single khungtaskd thread,
>   requiring zero atomic operations or concurrent locking overhead.
>
> - Flushes the lossy cache via memset() at the beginning of each
>   detection round. This ensures the immediate "thundering herd" of
>   duplicates is suppressed, but guarantees the system will not
>   permanently suppress identical hangs that occur in future rounds.
>
> - Introduces a new sysctl, kernel.hung_task_dedup, which defaults to 1
>   (enabled). The sysctl is locally cached at the outset of each
>   interval to prevent tearing caused by concurrent userspace toggling.
>

Thanks for working on this, but ... guess I'll be the bad guy here, not
convinced this should go in ...

When khungtaskd fires, somthing is already wrong, no? I don't see why it
should grow a new sysctl, a stack hash table, and extra filtering logic
just ot hide part of the report ...

Emm ... do you have real cases where duplicate hung-task stacks caused
serious pain?

If many tasks hang at once, usually one root cause, not a bunch of
different bugs. At least from what I've seen, any one of those stacks is
enough to start debugging ...

We already have hung_task_detect_count and trace_sched_process_hang() for
basic counting/observability. Even if hung_task_warnings is finite and
the warning budget runs out, we still don't lose detections: counter gets
bumped and tracepoint fires before printk output is gated :)

If someone wants stack grouping, I'd rather leave that to a tool than add
another policy knob to khungtaskd. Once it lands, maintainers have to
carry it forever. Not every nice-to-have feature is worth that cost, IMHO

And if someone really wants more hung-task stacks in the log, we already
have hung_task_warnings for that. Raise it, or set it to -1.

Also, looking at the v1 thread, I don't think the concerns there have
really settled yet ... If nobody replies, maybe give it a week before
sending a new version.

Thanks, Lance

  reply	other threads:[~2026-06-20  3:37 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-20  1:35 [PATCH v2] hung_task: Add per-round stack trace deduplication Aaron Tomlin
2026-06-20  3:37 ` Lance Yang [this message]
2026-06-20 17:54   ` Aaron Tomlin
2026-06-21  5:17     ` Lance Yang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260620033715.71108-1-lance.yang@linux.dev \
    --to=lance.yang@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=atomlin@atomlin.com \
    --cc=chjohnst@gmail.com \
    --cc=david.laight.linux@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mhiramat@kernel.org \
    --cc=mproche@gmail.com \
    --cc=neelx@suse.com \
    --cc=nick.lange@gmail.com \
    --cc=pmladek@suse.com \
    --cc=sean@ashe.io \
    --cc=steve@abita.co \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox