From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 890C4346FA8 for ; Tue, 18 Nov 2025 17:57:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763488669; cv=none; b=iR3xz3xLOZ/9dbXRQF2SWx/QLUMGxjZCqJwCCBEIOdM2Kd47o59Gy4d9ZFUl8XCKyE0hOlQ9lNcLQd0SDj0WYt4dfZrPG0CgRgO57YdpgjkFyQQ1erdNINrhr91gxkk7J556jac7ARKMm4vDuwXdtmP7cBq7TYStgkLZLwKCii8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763488669; c=relaxed/simple; bh=bz8WODHooHSMMGzYKWwaCTqq6iVLgfJYLmAvN/5gxJg=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=nAOeAefoi260ejDLnfEMABvE7UCfpog82vMZfIktfrlfIFxXqi9CTaod8igWpjTMVzyGkWqWIzJQ0u0hFEnj0//t5un0oE7NAPA7iPKVZ3+SK397JyI4K2pyDef0ZgCrUrSW0N+JXmc/o+LjBu0sLGXxOf0EvHP5H4KAlrEEHEs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=eKLHoKUS; arc=none smtp.client-ip=91.218.175.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="eKLHoKUS" Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1763488663; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nx9RmuIxsCHy8L9JS7g30Cf5lv/ZHtv3WsT3WPzu2KA=; b=eKLHoKUS4kCHNaCZ+pfBUkOVUWvYMqGA2nsVlrKXHi1qgM86UJQvKzieQoIVp92lQYLYbz 9FHqhHx6zOFnXU0trLCheARDlNhSBzgBJsfaDGo+oNZDmELroX5/BeXjVtN0e4mLrHSvFH GCFsodXIuX5CZPgp/uzdEUwT2FesxBc= Date: Wed, 19 Nov 2025 01:57:36 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Subject: Re: [PATCH v2 2/4] hung_task: Add hung_task_sys_info sysctl to dump sys info on task-hung Content-Language: en-US To: Petr Mladek , Andrew Morton Cc: Feng Tang , Steven Rostedt , Lance Yang , linux-kernel@vger.kernel.org, Jonathan Corbet , paulmck@kernel.org, lirongqing@baidu.com, leonylgao@tencent.com References: <20251113111039.22701-1-feng.tang@linux.alibaba.com> <20251113111039.22701-3-feng.tang@linux.alibaba.com> <20251117095352.8dfb46ec468ba5a69a829031@linux-foundation.org> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Lance Yang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT On 2025/11/18 23:20, Petr Mladek wrote: > On Mon 2025-11-17 09:53:52, Andrew Morton wrote: >> On Sun, 16 Nov 2025 22:13:58 +0800 Feng Tang wrote: >> >>>>> if (need_warning || hung_task_call_panic) { >>>>> si_mask |= SYS_INFO_LOCKS; >>>> >>>> Looks good to me now! I assume v3 would be expected, can you >>>> post a new version? >>> >>> Andrew has taken the patchset to -mm tree. >>> >>> Andrew, which way do you prefer? I send a v3 patch for hung-task or you >>> pickup the fixup patch and squash it into the orginal 0002 patch? >>> >>> Anyway, I make a squshed version v3 patch below. >> >> I prefer little fixup patches, generally. So people can see what >> changed and don't feel they should re-review everything. >> >> I queued the below, thanks. >> >> From: Feng Tang >> Subject: hung_task-add-hung_task_sys_info-sysctl-to-dump-sys-info-on-task-hung-fix >> Date: Wed, 5 Nov 2025 19:30:36 +0800 >> >> maintain consistecy established behavior, per Lance and Petr >> >> Link: https://lkml.kernel.org/r/aRncJo1mA5Zk77Hr@U-2FWC9VHC-2323.local >> Suggested-by: Petr Mladek >> Signed-off-by: Feng Tang >> Cc: Jonathan Corbet >> Cc: Lance Yang >> Cc: "Paul E . McKenney" >> Cc: Steven Rostedt >> Signed-off-by: Andrew Morton > > Thanks a lot for catching and fixing the regression caused > by this patchset. The patch looks good. > > See a comment below. > >> --- a/kernel/hung_task.c~hung_task-add-hung_task_sys_info-sysctl-to-dump-sys-info-on-task-hung-fix >> +++ a/kernel/hung_task.c >> @@ -223,8 +223,11 @@ static inline void debug_show_blocker(st >> } >> #endif >> >> -static void check_hung_task(struct task_struct *t, unsigned long timeout) >> +static void check_hung_task(struct task_struct *t, unsigned long timeout, >> + unsigned long prev_detect_count) >> { >> + unsigned long total_hung_task; >> + >> if (!task_is_hung(t, timeout)) >> return; >> >> @@ -234,13 +237,19 @@ static void check_hung_task(struct task_ >> */ >> sysctl_hung_task_detect_count++; >> >> + total_hung_task = sysctl_hung_task_detect_count - prev_detect_count; >> trace_sched_process_hang(t); >> >> + if (sysctl_hung_task_panic && total_hung_task >= sysctl_hung_task_panic) { >> + console_verbose(); >> + hung_task_call_panic = true; >> + } >> + >> /* >> * Ok, the task did not get scheduled for more than 2 minutes, >> * complain: >> */ >> - if (sysctl_hung_task_warnings) { >> + if (sysctl_hung_task_warnings || hung_task_call_panic) { >> if (sysctl_hung_task_warnings > 0) >> sysctl_hung_task_warnings--; >> pr_err("INFO: task %s:%d blocked for more than %ld seconds.\n", > > This restores the behavior after the commit 9544f9e6947f6508 > ("hung_task: panic when there are more than N hung tasks at > the same time"). It is better than nothing. > > Well, the behavior is still not ideal. It would be better when > we printed backtraces from _all_ "hung" tasks before panicking. > But it prints the backtraces only when sysctl_hung_task_panic > limit is reached. > > I mean, for example, let's have: > > + sysctl_hung_task_warnings = 2; > + sysctl_hung_task_panic = 5; > + and detect 6 hung tasks. > > The code will report 1st and 2nd hung tasks. It will skip 3rd and 4th > because sysctl_hung_task_warnings reached 0. It will report 5th and > 6th tasks because (total_hung_task >= 5). > > It is better than nothing. But it might be confusing. Right, I can see how it might be confusing. IMHO, sysctl_hung_task_warnings is a user-configured limit on verbosity. It makes sense that reports are suppressed after the limit is exhausted, except when the sysctl_hung_task_panic threshold is reached ;) > > I am not sure how to fix it. A minimalist solution would be to print > a warning. Something like: > > if (sysctl_hung_task_panic > 1 && > (total_hung_task == sysctl_hung_task_panic) && > !sysctl_hung_task_warnings) { > pr_err("INFO: %d blocked tasks might have been skipped because reached hung_task_warnings limit\n", > sysctl_hung_task_panic - 1); > > Or we could print the "total_hung_task" counter somewhere, for > example, > > pr_err("INFO[%lu]: task %s:%d blocked for more than %ld seconds.\n", > total_hung_task, ... > > Or we could restart the for_each_process_thread() cycle and make sure > that all hung tasks will get reported. > > Or we could ignore it until anyone complains. It looks like we already inform the user when that happens. When sysctl_hung_task_warnings is finally decremented to zero, the code prints: ``` if (!sysctl_hung_task_warnings) pr_info("Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings\n"); ``` Given that this explicit warning is already in place, perhaps the current behavior is sufficient and clear enough? Thanks, Lance