public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: "Li,Rongqing" <lirongqing@baidu.com>
To: "paulmck@kernel.org" <paulmck@kernel.org>,
	Lance Yang <lance.yang@linux.dev>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"feng.tang@linux.alibaba.com" <feng.tang@linux.alibaba.com>,
	"joel.granados@kernel.org" <joel.granados@kernel.org>,
	"kees@kernel.org" <kees@kernel.org>,
	"rostedt@goodmis.org" <rostedt@goodmis.org>,
	"pauld@redhat.com" <pauld@redhat.com>,
	"pawan.kumar.gupta@linux.intel.com"
	<pawan.kumar.gupta@linux.intel.com>,
	"mhiramat@kernel.org" <mhiramat@kernel.org>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"mingo@kernel.org" <mingo@kernel.org>
Subject: RE: [????] Re: [PATCH] hung_task: Panic after fixed number of hung tasks
Date: Sun, 28 Sep 2025 01:51:52 +0000	[thread overview]
Message-ID: <8828a890f37048da8b9846b08c321c2b@baidu.com> (raw)
In-Reply-To: <81514e1d-4a10-4466-8a87-2d4b0927195b@paulmck-laptop>



> 
> > On 2025/9/25 14:06, lirongqing wrote:
> > > From: Li RongQing <lirongqing@baidu.com>
> > >
> > > Currently, when hung_task_panic is enabled, kernel will panic
> > > immediately upon detecting the first hung task. However, some hung
> > > tasks are transient and the system can recover fully, while others
> > > are unrecoverable and trigger consecutive hung task reports, and a panic is
> expected.
> >
> > The new hung_task_count_to_panic relies on an absolute count, but I
> > assume the real indicator you're trying to capture is the trend or
> > rate of increase over a time window (e.g., "panic if count increases
> > by 5 in 10 minutes").
> >
> > IMHO, this kind of time-windowed, trend-based logic seems much more
> > flexible and better suited for a userspace monitoring agent :)
> >
> > In other words, why is this the right place for this feature?
> 
> A possibly related question is "why are RCU CPU stall warnings implemented in
> the kernel instead of in userspace?"  One reason is that by the time that
> things get bad enough to trigger an RCU CPU stall warning, userspace might
> not be capable of doing much of anything.  Thus, there is an uncomfortably
> high probability that orchestrating RCU CPU stall warnings from userspace
> would cause these warnings to be lost entirely.
> 


Thank you, I think so too.

-Li


> Similar reasoning might (or might not) apply to the hung-task mechanism.
> 
> 							Thanx, Paul
> 
> > Please sell it to us ;)
> > Lance
> >
> > >
> > > This commit adds a new sysctl parameter hung_task_count_to_panic to
> > > allows specifying the number of consecutive hung tasks that must be
> > > detected before triggering a kernel panic. This provides finer
> > > control for environments where transient hangs maybe happen but
> > > persistent hangs should still be fatal.
> > >
> > > Signed-off-by: Li RongQing <lirongqing@baidu.com>
> > > ---
> > >   Documentation/admin-guide/sysctl/kernel.rst |  6 ++++++
> > >   kernel/hung_task.c                          | 14 +++++++++++++-
> > >   2 files changed, 19 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/sysctl/kernel.rst
> > > b/Documentation/admin-guide/sysctl/kernel.rst
> > > index 8b49eab..4240e7b 100644
> > > --- a/Documentation/admin-guide/sysctl/kernel.rst
> > > +++ b/Documentation/admin-guide/sysctl/kernel.rst
> > > @@ -405,6 +405,12 @@ This file shows up if
> ``CONFIG_DETECT_HUNG_TASK`` is enabled.
> > >   1 Panic immediately.
> > >   = =================================================
> > > +hung_task_count_to_panic
> > > +=====================
> > > +
> > > +When set to a non-zero value, after the number of consecutive hung
> > > +task occur, the kernel will triggers a panic
> > > +
> > >   hung_task_check_count
> > >   =====================
> > > diff --git a/kernel/hung_task.c b/kernel/hung_task.c index
> > > 8708a12..87a6421 100644
> > > --- a/kernel/hung_task.c
> > > +++ b/kernel/hung_task.c
> > > @@ -83,6 +83,8 @@ static unsigned int __read_mostly
> sysctl_hung_task_all_cpu_backtrace;
> > >   static unsigned int __read_mostly sysctl_hung_task_panic =
> > >   	IS_ENABLED(CONFIG_BOOTPARAM_HUNG_TASK_PANIC);
> > > +static unsigned int __read_mostly sysctl_hung_task_count_to_panic;
> > > +
> > >   static int
> > >   hung_task_panic(struct notifier_block *this, unsigned long event, void
> *ptr)
> > >   {
> > > @@ -219,7 +221,9 @@ static void check_hung_task(struct task_struct *t,
> unsigned long timeout)
> > >   	trace_sched_process_hang(t);
> > > -	if (sysctl_hung_task_panic) {
> > > +	if (sysctl_hung_task_panic ||
> > > +	    (sysctl_hung_task_count_to_panic &&
> > > +	     (sysctl_hung_task_detect_count >=
> > > +sysctl_hung_task_count_to_panic))) {
> > >   		console_verbose();
> > >   		hung_task_show_lock = true;
> > >   		hung_task_call_panic = true;
> > > @@ -388,6 +392,14 @@ static const struct ctl_table hung_task_sysctls[] =
> {
> > >   		.extra2		= SYSCTL_ONE,
> > >   	},
> > >   	{
> > > +		.procname	= "hung_task_count_to_panic",
> > > +		.data		= &sysctl_hung_task_count_to_panic,
> > > +		.maxlen		= sizeof(int),
> > > +		.mode		= 0644,
> > > +		.proc_handler	= proc_dointvec_minmax,
> > > +		.extra1		= SYSCTL_ZERO,
> > > +	},
> > > +	{
> > >   		.procname	= "hung_task_check_count",
> > >   		.data		= &sysctl_hung_task_check_count,
> > >   		.maxlen		= sizeof(int),
> >

  parent reply	other threads:[~2025-09-28  1:53 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-25  6:06 [PATCH] hung_task: Panic after fixed number of hung tasks lirongqing
2025-09-25  7:37 ` Hillf Danton
2025-09-25  8:03   ` [外部邮件] " Li,Rongqing
2025-09-25 10:26 ` Lance Yang
2025-09-26 18:02   ` Paul E. McKenney
2025-09-27  2:18     ` Lance Yang
2025-09-28  1:51     ` Li,Rongqing [this message]
2025-09-27  2:39 ` Lance Yang
2025-09-28  1:54   ` [外部邮件] " Li,Rongqing
2025-09-28  3:19   ` Li,Rongqing
2025-09-28  3:29     ` Lance Yang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8828a890f37048da8b9846b08c321c2b@baidu.com \
    --to=lirongqing@baidu.com \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=feng.tang@linux.alibaba.com \
    --cc=joel.granados@kernel.org \
    --cc=kees@kernel.org \
    --cc=lance.yang@linux.dev \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mhiramat@kernel.org \
    --cc=mingo@kernel.org \
    --cc=pauld@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=pawan.kumar.gupta@linux.intel.com \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox