Re: [PATCH] hung_task: Panic after fixed number of hung tasks

Lance Yang
2025-09-25T10:26:11Z

Thanks for the patch!

On 2025/9/25 14:06, lirongqing wrote:
> From: Li RongQing <lirongqing@baidu.com>
>
> Currently, when hung_task_panic is enabled, kernel will panic immediately
> upon detecting the first hung task. However, some hung tasks are transient
> and the system can recover fully, while others are unrecoverable and
> trigger consecutive hung task reports, and a panic is expected.

The new hung_task_count_to_panic relies on an absolute count, but I
assume the real indicator you're trying to capture is the trend or
rate of increase over a time window (e.g., "panic if count increases
by 5 in 10 minutes").

IMHO, this kind of time-windowed, trend-based logic seems much more
flexible and better suited for a userspace monitoring agent :)

In other words, why is this the right place for this feature?

Please sell it to us ;)
Lance

>
> This commit adds a new sysctl parameter hung_task_count_to_panic to allows
> specifying the number of consecutive hung tasks that must be detected
> before triggering a kernel panic. This provides finer control for
> environments where transient hangs maybe happen but persistent hangs should
> still be fatal.
>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>
> ---
> Documentation/admin-guide/sysctl/kernel.rst | 6 ++++++
> kernel/hung_task.c | 14 +++++++++++++-
> 2 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
> index 8b49eab..4240e7b 100644
> --- a/Documentation/admin-guide/sysctl/kernel.rst
> +++ b/Documentation/admin-guide/sysctl/kernel.rst
> @@ -405,6 +405,12 @@ This file shows up if ``CONFIG_DETECT_HUNG_TASK`` is enabled.
> 1 Panic immediately.
> = =================================================
>
> +hung_task_count_to_panic
> +=====================
> +
> +When set to a non-zero value, after the number of consecutive hung task
> +occur, the kernel will triggers a panic
> +
>
> hung_task_check_count
> =====================
> diff --git a/kernel/hung_task.c b/kernel/hung_task.c
> index 8708a12..87a6421 100644
> --- a/kernel/hung_task.c
> +++ b/kernel/hung_task.c
> @@ -83,6 +83,8 @@ static unsigned int __read_mostly sysctl_hung_task_all_cpu_backtrace;
> static unsigned int __read_mostly sysctl_hung_task_panic =
> IS_ENABLED(CONFIG_BOOTPARAM_HUNG_TASK_PANIC);
>
> +static unsigned int __read_mostly sysctl_hung_task_count_to_panic;
> +
> static int
> hung_task_panic(struct notifier_block *this, unsigned long event, void *ptr)
> {
> @@ -219,7 +221,9 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout)
>
> trace_sched_process_hang(t);
>
> - if (sysctl_hung_task_panic) {
> + if (sysctl_hung_task_panic ||
> + (sysctl_hung_task_count_to_panic &&
> + (sysctl_hung_task_detect_count >= sysctl_hung_task_count_to_panic))) {
> console_verbose();
> hung_task_show_lock = true;
> hung_task_call_panic = true;
> @@ -388,6 +392,14 @@ static const struct ctl_table hung_task_sysctls[] = {
> .extra2 = SYSCTL_ONE,
> },
> {
> + .procname = "hung_task_count_to_panic",
> + .data = &sysctl_hung_task_count_to_panic,
> + .maxlen = sizeof(int),
> + .mode = 0644,
> + .proc_handler = proc_dointvec_minmax,
> + .extra1 = SYSCTL_ZERO,
> + },
> + {
> .procname = "hung_task_check_count",
> .data = &sysctl_hung_task_check_count,
> .maxlen = sizeof(int),