From: "Li,Rongqing" <lirongqing@baidu.com>
To: "paulmck@kernel.org" <paulmck@kernel.org>,
Lance Yang <lance.yang@linux.dev>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
"arnd@arndb.de" <arnd@arndb.de>,
"feng.tang@linux.alibaba.com" <feng.tang@linux.alibaba.com>,
"joel.granados@kernel.org" <joel.granados@kernel.org>,
"kees@kernel.org" <kees@kernel.org>,
"rostedt@goodmis.org" <rostedt@goodmis.org>,
"pauld@redhat.com" <pauld@redhat.com>,
"pawan.kumar.gupta@linux.intel.com"
<pawan.kumar.gupta@linux.intel.com>,
"mhiramat@kernel.org" <mhiramat@kernel.org>,
"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
"corbet@lwn.net" <corbet@lwn.net>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"mingo@kernel.org" <mingo@kernel.org>
Subject: RE: [????] Re: [PATCH] hung_task: Panic after fixed number of hung tasks
Date: Sun, 28 Sep 2025 01:51:52 +0000 [thread overview]
Message-ID: <8828a890f37048da8b9846b08c321c2b@baidu.com> (raw)
In-Reply-To: <81514e1d-4a10-4466-8a87-2d4b0927195b@paulmck-laptop>
>
> > On 2025/9/25 14:06, lirongqing wrote:
> > > From: Li RongQing <lirongqing@baidu.com>
> > >
> > > Currently, when hung_task_panic is enabled, kernel will panic
> > > immediately upon detecting the first hung task. However, some hung
> > > tasks are transient and the system can recover fully, while others
> > > are unrecoverable and trigger consecutive hung task reports, and a panic is
> expected.
> >
> > The new hung_task_count_to_panic relies on an absolute count, but I
> > assume the real indicator you're trying to capture is the trend or
> > rate of increase over a time window (e.g., "panic if count increases
> > by 5 in 10 minutes").
> >
> > IMHO, this kind of time-windowed, trend-based logic seems much more
> > flexible and better suited for a userspace monitoring agent :)
> >
> > In other words, why is this the right place for this feature?
>
> A possibly related question is "why are RCU CPU stall warnings implemented in
> the kernel instead of in userspace?" One reason is that by the time that
> things get bad enough to trigger an RCU CPU stall warning, userspace might
> not be capable of doing much of anything. Thus, there is an uncomfortably
> high probability that orchestrating RCU CPU stall warnings from userspace
> would cause these warnings to be lost entirely.
>
Thank you, I think so too.
-Li
> Similar reasoning might (or might not) apply to the hung-task mechanism.
>
> Thanx, Paul
>
> > Please sell it to us ;)
> > Lance
> >
> > >
> > > This commit adds a new sysctl parameter hung_task_count_to_panic to
> > > allows specifying the number of consecutive hung tasks that must be
> > > detected before triggering a kernel panic. This provides finer
> > > control for environments where transient hangs maybe happen but
> > > persistent hangs should still be fatal.
> > >
> > > Signed-off-by: Li RongQing <lirongqing@baidu.com>
> > > ---
> > > Documentation/admin-guide/sysctl/kernel.rst | 6 ++++++
> > > kernel/hung_task.c | 14 +++++++++++++-
> > > 2 files changed, 19 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/sysctl/kernel.rst
> > > b/Documentation/admin-guide/sysctl/kernel.rst
> > > index 8b49eab..4240e7b 100644
> > > --- a/Documentation/admin-guide/sysctl/kernel.rst
> > > +++ b/Documentation/admin-guide/sysctl/kernel.rst
> > > @@ -405,6 +405,12 @@ This file shows up if
> ``CONFIG_DETECT_HUNG_TASK`` is enabled.
> > > 1 Panic immediately.
> > > = =================================================
> > > +hung_task_count_to_panic
> > > +=====================
> > > +
> > > +When set to a non-zero value, after the number of consecutive hung
> > > +task occur, the kernel will triggers a panic
> > > +
> > > hung_task_check_count
> > > =====================
> > > diff --git a/kernel/hung_task.c b/kernel/hung_task.c index
> > > 8708a12..87a6421 100644
> > > --- a/kernel/hung_task.c
> > > +++ b/kernel/hung_task.c
> > > @@ -83,6 +83,8 @@ static unsigned int __read_mostly
> sysctl_hung_task_all_cpu_backtrace;
> > > static unsigned int __read_mostly sysctl_hung_task_panic =
> > > IS_ENABLED(CONFIG_BOOTPARAM_HUNG_TASK_PANIC);
> > > +static unsigned int __read_mostly sysctl_hung_task_count_to_panic;
> > > +
> > > static int
> > > hung_task_panic(struct notifier_block *this, unsigned long event, void
> *ptr)
> > > {
> > > @@ -219,7 +221,9 @@ static void check_hung_task(struct task_struct *t,
> unsigned long timeout)
> > > trace_sched_process_hang(t);
> > > - if (sysctl_hung_task_panic) {
> > > + if (sysctl_hung_task_panic ||
> > > + (sysctl_hung_task_count_to_panic &&
> > > + (sysctl_hung_task_detect_count >=
> > > +sysctl_hung_task_count_to_panic))) {
> > > console_verbose();
> > > hung_task_show_lock = true;
> > > hung_task_call_panic = true;
> > > @@ -388,6 +392,14 @@ static const struct ctl_table hung_task_sysctls[] =
> {
> > > .extra2 = SYSCTL_ONE,
> > > },
> > > {
> > > + .procname = "hung_task_count_to_panic",
> > > + .data = &sysctl_hung_task_count_to_panic,
> > > + .maxlen = sizeof(int),
> > > + .mode = 0644,
> > > + .proc_handler = proc_dointvec_minmax,
> > > + .extra1 = SYSCTL_ZERO,
> > > + },
> > > + {
> > > .procname = "hung_task_check_count",
> > > .data = &sysctl_hung_task_check_count,
> > > .maxlen = sizeof(int),
> >
next prev parent reply other threads:[~2025-09-28 1:53 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-25 6:06 [PATCH] hung_task: Panic after fixed number of hung tasks lirongqing
2025-09-25 7:37 ` Hillf Danton
2025-09-25 8:03 ` [外部邮件] " Li,Rongqing
2025-09-25 10:26 ` Lance Yang
2025-09-26 18:02 ` Paul E. McKenney
2025-09-27 2:18 ` Lance Yang
2025-09-28 1:51 ` Li,Rongqing [this message]
2025-09-27 2:39 ` Lance Yang
2025-09-28 1:54 ` [外部邮件] " Li,Rongqing
2025-09-28 3:19 ` Li,Rongqing
2025-09-28 3:29 ` Lance Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8828a890f37048da8b9846b08c321c2b@baidu.com \
--to=lirongqing@baidu.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=feng.tang@linux.alibaba.com \
--cc=joel.granados@kernel.org \
--cc=kees@kernel.org \
--cc=lance.yang@linux.dev \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mingo@kernel.org \
--cc=pauld@redhat.com \
--cc=paulmck@kernel.org \
--cc=pawan.kumar.gupta@linux.intel.com \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox