RE: [????] Re: [PATCH] hung_task: Panic after fixed number of hung tasks

Li,Rongqing
2025-09-28T01:53:20Z

>
> > On 2025/9/25 14:06, lirongqing wrote:
> > > From: Li RongQing <lirongqing@baidu.com>
> > >
> > > Currently, when hung_task_panic is enabled, kernel will panic
> > > immediately upon detecting the first hung task. However, some hung
> > > tasks are transient and the system can recover fully, while others
> > > are unrecoverable and trigger consecutive hung task reports, and a panic is
> expected.
> >
> > The new hung_task_count_to_panic relies on an absolute count, but I
> > assume the real indicator you're trying to capture is the trend or
> > rate of increase over a time window (e.g., "panic if count increases
> > by 5 in 10 minutes").
> >
> > IMHO, this kind of time-windowed, trend-based logic seems much more
> > flexible and better suited for a userspace monitoring agent :)
> >
> > In other words, why is this the right place for this feature?
>
> A possibly related question is "why are RCU CPU stall warnings implemented in
> the kernel instead of in userspace?" One reason is that by the time that
> things get bad enough to trigger an RCU CPU stall warning, userspace might
> not be capable of doing much of anything. Thus, there is an uncomfortably
> high probability that orchestrating RCU CPU stall warnings from userspace
> would cause these warnings to be lost entirely.
>

Thank you, I think so too.

-Li

> Similar reasoning might (or might not) apply to the hung-task mechanism.
>
> Thanx, Paul
>
> > Please sell it to us ;)
> > Lance
> >
> > >
> > > This commit adds a new sysctl parameter hung_task_count_to_panic to
> > > allows specifying the number of consecutive hung tasks that must be
> > > detected before triggering a kernel panic. This provides finer
> > > control for environments where transient hangs maybe happen but
> > > persistent hangs should still be fatal.
> > >
> > > Signed-off-by: Li RongQing <lirongqing@baidu.com>
> > > ---
> > > Documentation/admin-guide/sysctl/kernel.rst | 6 ++++++
> > > kernel/hung_task.c | 14 +++++++++++++-
> > > 2 files changed, 19 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/sysctl/kernel.rst
> > > b/Documentation/admin-guide/sysctl/kernel.rst
> > > index 8b49eab..4240e7b 100644
> > > --- a/Documentation/admin-guide/sysctl/kernel.rst
> > > +++ b/Documentation/admin-guide/sysctl/kernel.rst
> > > @@ -405,6 +405,12 @@ This file shows up if
> ``CONFIG_DETECT_HUNG_TASK`` is enabled.
> > > 1 Panic immediately.
> > > = =================================================
> > > +hung_task_count_to_panic
> > > +=====================
> > > +
> > > +When set to a non-zero value, after the number of consecutive hung
> > > +task occur, the kernel will triggers a panic
> > > +
> > > hung_task_check_count
> > > =====================
> > > diff --git a/kernel/hung_task.c b/kernel/hung_task.c index
> > > 8708a12..87a6421 100644
> > > --- a/kernel/hung_task.c
> > > +++ b/kernel/hung_task.c
> > > @@ -83,6 +83,8 @@ static unsigned int __read_mostly
> sysctl_hung_task_all_cpu_backtrace;
> > > static unsigned int __read_mostly sysctl_hung_task_panic =
> > > IS_ENABLED(CONFIG_BOOTPARAM_HUNG_TASK_PANIC);
> > > +static unsigned int __read_mostly sysctl_hung_task_count_to_panic;
> > > +
> > > static int
> > > hung_task_panic(struct notifier_block *this, unsigned long event, void
> *ptr)
> > > {
> > > @@ -219,7 +221,9 @@ static void check_hung_task(struct task_struct *t,
> unsigned long timeout)
> > > trace_sched_process_hang(t);
> > > - if (sysctl_hung_task_panic) {
> > > + if (sysctl_hung_task_panic ||
> > > + (sysctl_hung_task_count_to_panic &&
> > > + (sysctl_hung_task_detect_count >=
> > > +sysctl_hung_task_count_to_panic))) {
> > > console_verbose();
> > > hung_task_show_lock = true;
> > > hung_task_call_panic = true;
> > > @@ -388,6 +392,14 @@ static const struct ctl_table hung_task_sysctls[] =
> {
> > > .extra2 = SYSCTL_ONE,
> > > },
> > > {
> > > + .procname = "hung_task_count_to_panic",
> > > + .data = &sysctl_hung_task_count_to_panic,
> > > + .maxlen = sizeof(int),
> > > + .mode = 0644,
> > > + .proc_handler = proc_dointvec_minmax,
> > > + .extra1 = SYSCTL_ZERO,
> > > + },
> > > + {
> > > .procname = "hung_task_check_count",
> > > .data = &sysctl_hung_task_check_count,
> > > .maxlen = sizeof(int),
> >