[PATCH v8 0/2] hung_task: Improve warning budget handling and task reporting

[Aaron Tomlin <atomlin@atomlin.com>]

The hung_task watchdog detects tasks stuck in TASK_UNINTERRUPTIBLE (D)
state for longer than CONFIG_DEFAULT_HUNG_TASK_TIMEOUT seconds. To prevent
log spam during system spikes, sysctl_hung_task_warnings enforces a budget
on the number of logged warnings.

However, the current implementation has two major limitations:

    1. Permanent exhaustion of warning budget

       sysctl_hung_task_warnings is decremented directly when printing
       warnings. Once this budget hits zero, no further warnings are
       reported until an administrator manually updates the sysctl value or
       reboots the system. Consequently, a single temporary hang episode
       permanently blinds the kernel watchdog to any subsequent hung tasks
       after system recovery.

    2. Total log suppression when budget is exhausted

       Once the warning budget reaches zero, hung_task_info() completely
       suppresses all output, including the basic single-line alert. While
       suppressing verbose stack dumps and lock debugging is desirable to
       prevent dmesg flooding, hiding basic task alerts leaves
       administrators entirely unaware that tasks are hanging.

This patch series resolves both limitations by decoupling the configured
warning budget from the runtime warning counter, automatically resetting
the budget when the system recovers, and keeping basic single-line hung
task alerts visible.

Patch 1 decouples the user-configured limit sysctl_hung_task_warnings from
the runtime counter hung_task_warnings_printed, automatically resetting
hung_task_warnings_printed back to the configured limit whenever a check
interval passes with zero hung tasks detected. The runtime counter is also
kept synchronised whenever the sysctl parameter is modified, and the
corresponding sysctl documentation
(i.e., Documentation/admin-guide/sysctl/kernel.rst) is updated to reflect
this auto-reset behaviour.

Patch 2 ensures that the basic single-line hung task messsage is always
logged regardless of warning budget exhaustion, while restricting warning
budget enforcement solely to verbose diagnostics such as process stack
dumps, taint and release information, and lock blocker details.
Additionally, it updates the warning exhaustion log message to clarify to
administrators that future reports will only omit detailed process dumps
rather than being completely suppressed.

Changes since v7:

 - Consolidated the commit message of each patch (Lance Yang)

 - Linked to v7: https://lore.kernel.org/lkml/20260804155406.254810-1-atomlin@atomlin.com/

Changes since v6:

 - Restructured the series in a new direction. Introduced an internal
   counter (hung_task_warnings_printed) decoupled from
   sysctl_hung_task_warnings. The warning budget automatically resets to
   the configured limit once a check interval completes with zero hung
   tasks detected, or when modified via sysctl (Petr Mladek)

 - Ensured the single-line blocked hung task message is always printed even
   after the warning budget reaches zero. Now budget enforcement is
   restricted solely to suppressing verbose diagnostics (Petr Mladek)

 - Refined the description of sysctl hung_task_warnings (Lance Yang)

 - Removed field hung_task_reported from struct task_struct

 - Removed the CONFIG_DETECT_HUNG_TASK_BLOCKER integration and
   hung_task_blockers[] used to track memory addresses of blocker locks
   across check intervals

 - Removed the skip_show_task logic and dmesg log suppression messages

 - Removed tracking variables (warnings_decremented and
   hung_task_has_active) and the dmesg recovery notice printed when
   clearing the blocker array

 - Linked to v6: https://lore.kernel.org/lkml/20260719161305.428947-1-atomlin@atomlin.com/

Changes since v5:

 - Skipped hung_task_info() and sys_info() for tasks already reported in
   previous rounds to avoid log spam

 - Linked to v5: https://lore.kernel.org/lkml/20260712202100.123934-1-atomlin@atomlin.com/

Changes since v4:

 - Replaced the stack-local hashmap implementation with a persistent
   array (Petr Mladek)

 - Persistently track blocker addresses across scan intervals. Suppress
   warning reports and keep the sysctl_hung_task_warnings budget intact
   if the blocker is already tracked in the array (Petr Mladek)

 - Reset the warnings budget and clear the blocker array when the hang
   resolves (Petr Mladek)

 - Output a recovery message to the kernel ring buffer upon hang
   resolution

 - Linked to v4: https://lore.kernel.org/lkml/20260627205733.90983-1-atomlin@atomlin.com/

Changes since v3:

 - Deduct from the global budget if printing a full stack trace

 - Pivoted from heuristic Wait Channel hashing to deterministic
   blocker address hashing via CONFIG_DETECT_HUNG_TASK_BLOCKER

 - Replaced the hung_task_reported bit-field with a standalone u8 byte.
   Move hung_task_reported into an existing structural alignment hole
   within task_struct following blocked_lock, resulting in zero overall
   memory footprint increase and optimal cacheline grouping

 - Linked to v3: https://lore.kernel.org/lkml/20260621213756.43225-1-atomlin@atomlin.com/

Changes since v2:

 - Replaced the per-round cache flush with a task_struct bit-field for
   persistent cross-scan tracking, mitigating delayed budget exhaustion

 - Abandoned exact-stack hashing in favour of Wait Channel hashing

 - Transitioned from jhash() to hash_long() to optimise single-pointer
   hashing, and relocated the hash map to the local stack

 - Linked to v2: https://lore.kernel.org/lkml/20260620013559.1537893-1-atomlin@atomlin.com/

Changes since v1:

 - Preserve "INFO:" headers for all hung tasks; suppress only the stack
   dumps for duplicates (Masami Hiramatsu)

 - Print a clear notification when a trace is explicitly suppressed

 - Add #ifdef CONFIG_STACKTRACE guards to prevent Kconfig build errors

 - Optimise overhead by unwinding the stack only if a warning is
   actually going to be printed

 - Linked to v1: https://lore.kernel.org/lkml/20260617184841.1447955-1-atomlin@atomlin.com/

Aaron Tomlin (2):
  hung_task: Reset warning budget when problem gets resolved
  hung_task: Always print basic hung task info header

 Documentation/admin-guide/sysctl/kernel.rst |  5 ++-
 kernel/hung_task.c                          | 46 +++++++++++++++------
 2 files changed, 37 insertions(+), 14 deletions(-)

-- 
2.55.0