From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8245F38E5E3 for ; Tue, 3 Feb 2026 09:03:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770109439; cv=none; b=Mtl9Gt9YEP4JNNxuswMGF2v/bMKnBTd2NBxTrnNupH4M7/KOPoQ2z7/AzxFU8rC5p8OfoHVSbOa4TZHsQASD3Wrpn0HcKi6hUDCEFldqdpYl1eQxq73fJWTtsdnTnoJn6vnTOO46bm68LtGxrMePS1RkmLjUIpqQIY7ObSN4dkI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770109439; c=relaxed/simple; bh=9N0fy9+0Jv7QLOFEfdW9KFq1hQ3HALadbrMmj9kndfY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=p5jPKb4/IRgZFKflpVDRfrp7ztM7ZRjVLRy2d96hLIujKwH9jCUJ9F4LH/U5L9c0lsuMSJK1UCFrugKiKXnQwGs6VnjhWgBTVNvu4SQ3At/UWbMOcrKQJeDrZPzfj765suFmT4T4Mj4f3vyQCWUuY1UGzcT2HOlgvPc2Kf77sAE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b=gYuKUT9F; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b="gYuKUT9F" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-4806f3fc50bso53821885e9.0 for ; Tue, 03 Feb 2026 01:03:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1770109436; x=1770714236; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=F5mHP1xDsJS0vOeX1NnBoIb1xnCY+AxYy8aLLTyTZsU=; b=gYuKUT9FjC0Df/S3FaB15alm90JggLJxcZ80/UnkVQFveXhzrINkwBcKZ0vJ1ZvhQH wcxf8IJeF1B/osTTvIecuJwSYMvFs2hVQdIoATvz/tN3lGBZrLL9jxHxiC1QAlAAwD+N 1VUq9zIT2V1IqynHeo3sPIITNPWwSlER//qe+QYO5vAAS3iD0LjhPYWNfXTGUKD0bAPZ 8kM06cmmXL8ic2FLwKD8JGh9P1KTzEByHb6nHCs+D2VDQWAbGR8mA80rUyNs+b4Rdt+9 qPB6oSXYJMcUuqhIIQxi7U/1UoqbUw+eZkonTwHHpsHjDgeQb6Czu7ml4t2g4OG0XzQ1 zmjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770109436; x=1770714236; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F5mHP1xDsJS0vOeX1NnBoIb1xnCY+AxYy8aLLTyTZsU=; b=rV+tcEyn4ZfACV440IiG5oR0VmQ2Trv6jDejx2DCfSOS9FPgWaCMmClMkCYKSHAC4/ TqckGvMRhkm2hyArLnwgYyYuXiTkUPJvns9cE3STxIvEHDrtSheeKdBsmYKEO6Jayw6j UaRj206tmpwMN82z0WKSlLgHVtv1SWlFWknyTY11F9gRhAiux/e9irfLaY5k/FWMp11m 2vM7jlv+YESmc5ln7C/Tk2XfYAi4mESdvF4zphDdKiiobXazhWv9o07NS5FSQ1bt774w 7D7eVVrBRIqQ2OWqjSfSLXlIDo/HJ0o1f+zfjk0j7KIK2R0YhFkhRzYKhcj4PC2ukvDl WylA== X-Forwarded-Encrypted: i=1; AJvYcCUluDtrx2N3J/2b9BMZgvQxaT6uQwY6FwMv6SRzUDEgl/Pk5OIp/A1qfE/+7jtbvB8ScvfI7mDhXL/z/ag=@vger.kernel.org X-Gm-Message-State: AOJu0Ywd/julax/gLPVW3Q0gdKnvDZadT9/Ue/HFLhf72Mm4H98LG4Bq 1m/7A1J/qxnAEsahg36VaKxFQRSi+XxeSk+erHkVha+ndhIHXs343o7PPA/Jx9NsUTI= X-Gm-Gg: AZuq6aLLYOH37MtQKty09uTgXKw90L0no1RcrGfCkuHJoQOYEyXy14jIg3zOzFvRTmB DzP81QD9ec1AKfWXlTa5bL1E9gCt/6hhGotfFSS60d2uoQbarDyijZIYUMHiMB9DOxuCp7K9pva wDsY+ADPkovFjqlnM4aU4VUZpNAc87JIqilU4KiQebcJumci7r4TKk7TdODMVZ1Sk2WtNnmusil qAFTxJFSMLjLpoaRqQ4dkvG/I28eMMByz3WKMdYbKtDAsBMRUdQ28hxgoXbxy6QMiyt0r9bopGP oRkd7Rwh+FXXacei92Ic7S3d0o/KI3w7ceEGW7DuTnD9gbbt4njnKqOktVp7j56SulFtsr+sWkK xxrAisM0ZzXdj4M8NqvnHWeBA0BiF8y2XdrHenv2n1qedONQovLAhdO4GXk+cVAzAlR7ubMyGQn 8gy6oLdo94Ca6O4g== X-Received: by 2002:a05:600c:4591:b0:480:4a4f:c363 with SMTP id 5b1f17b1804b1-482db45fc65mr172291295e9.9.1770109435609; Tue, 03 Feb 2026 01:03:55 -0800 (PST) Received: from pathway.suse.cz ([176.114.240.130]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4830513352csm49654645e9.10.2026.02.03.01.03.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Feb 2026 01:03:54 -0800 (PST) Date: Tue, 3 Feb 2026 10:03:52 +0100 From: Petr Mladek To: Lance Yang Cc: Aaron Tomlin , neelx@suse.com, sean@ashe.io, akpm@linux-foundation.org, mproche@gmail.com, chjohnst@gmail.com, nick.lange@gmail.com, linux-kernel@vger.kernel.org, mhiramat@kernel.org, joel.granados@kernel.org, gregkh@linuxfoundation.org Subject: Re: [v7 PATCH 1/2] hung_task: Refactor detection logic and atomicise detection count Message-ID: References: <20260125135848.3356585-1-atomlin@atomlin.com> <20260125135848.3356585-2-atomlin@atomlin.com> <45c17b93-a6f1-45e0-8b25-20665a281949@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <45c17b93-a6f1-45e0-8b25-20665a281949@linux.dev> On Tue 2026-02-03 11:08:33, Lance Yang wrote: > On 2026/2/3 11:05, Lance Yang wrote: > > On 2026/1/25 21:58, Aaron Tomlin wrote: > > > The check_hung_task() function currently conflates two distinct > > > responsibilities: validating whether a task is hung and handling the > > > subsequent reporting (printing warnings, triggering panics, or > > > tracepoints). > > > > > > This patch refactors the logic by introducing hung_task_info(), a > > > function dedicated solely to reporting. The actual detection check, > > > task_is_hung(), is hoisted into the primary loop within > > > check_hung_uninterruptible_tasks(). This separation clearly decouples > > > the mechanism of detection from the policy of reporting. > > > > > > Furthermore, to facilitate future support for concurrent hung task > > > detection, the global sysctl_hung_task_detect_count variable is > > > converted from unsigned long to atomic_long_t. Consequently, the > > > counting logic is updated to accumulate the number of hung tasks locally > > > (this_round_count) during the iteration. The global counter is then > > > updated atomically via atomic_long_cmpxchg_relaxed() once the loop > > > concludes, rather than incrementally during the scan. > > > > > > These changes are strictly preparatory and introduce no functional > > > change to the system's runtime behaviour. > > > > > > Signed-off-by: Aaron Tomlin > > > --- > > >   kernel/hung_task.c | 58 ++++++++++++++++++++++++++-------------------- > > >   1 file changed, 33 insertions(+), 25 deletions(-) > > > > > > diff --git a/kernel/hung_task.c b/kernel/hung_task.c > > > index d2254c91450b..df10830ed9ef 100644 > > > --- a/kernel/hung_task.c > > > +++ b/kernel/hung_task.c > > > @@ -36,7 +36,7 @@ static int __read_mostly > > > sysctl_hung_task_check_count = PID_MAX_LIMIT; > > >   /* > > >    * Total number of tasks detected as hung since boot: > > >    */ > > > -static unsigned long __read_mostly sysctl_hung_task_detect_count; > > > +static atomic_long_t sysctl_hung_task_detect_count = > > > ATOMIC_LONG_INIT(0); > > >   /* > > >    * Limit number of tasks checked in a batch. > > > @@ -223,31 +223,29 @@ static inline void debug_show_blocker(struct > > > task_struct *task, unsigned long ti > > >   } > > >   #endif > > > -static void check_hung_task(struct task_struct *t, unsigned long > > > timeout, > > > -        unsigned long prev_detect_count) > > > +/** > > > + * hung_task_info - Print diagnostic details for a hung task > > > + * @t: Pointer to the detected hung task. > > > + * @timeout: Timeout threshold for detecting hung tasks > > > + * @this_round_count: Count of hung tasks detected in the current > > > iteration > > > + * > > > + * Print structured information about the specified hung task, if > > > warnings > > > + * are enabled or if the panic batch threshold is exceeded. > > > + */ > > > +static void hung_task_info(struct task_struct *t, unsigned long timeout, > > > +               unsigned long this_round_count) > > >   { > > > -    unsigned long total_hung_task; > > > - > > > -    if (!task_is_hung(t, timeout)) > > > -        return; > > > - > > > -    /* > > > -     * This counter tracks the total number of tasks detected as hung > > > -     * since boot. > > > -     */ > > > -    sysctl_hung_task_detect_count++; > > > > Previously, the global detect count updated immediately when a hung task > > was found. BUT now, it only updates after the full scan finishes ... > > > > Ideally, the count should update as soon as possible, so that userspace > > can react in time :) > > > > For example, by migrating critical containers away from the node before > > the situation gets worse - something we already do. > > Sorry, I should have said that earlier - just realized it ... Better late then sorry ;-) That said, is the delay really critical? I guess that the userspace checks the counter in regular intervals (seconds or tens of seconds). Or is there any way to get a notification immediately? Anyway, I thought how the counting and barriers might work when we update the global counter immediately. And I came up with the following: diff --git a/kernel/hung_task.c b/kernel/hung_task.c index 350093de0535..8bc043fbe89c 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -302,15 +302,10 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout) int max_count = sysctl_hung_task_check_count; unsigned long last_break = jiffies; struct task_struct *g, *t; - unsigned long total_count, this_round_count; + unsigned long this_round_count; int need_warning = sysctl_hung_task_warnings; unsigned long si_mask = hung_task_si_mask; - /* - * The counter might get reset. Remember the initial value. - * Acquire prevents reordering task checks before this point. - */ - total_count = atomic_long_read_acquire(&sysctl_hung_task_detect_count); /* * If the system crashed already then all bets are off, * do not report extra hung tasks: @@ -330,6 +325,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout) } if (task_is_hung(t, timeout)) { + /* + * Increment the global counter so that userspace could + * start migrating tasks ASAP. But count the current + * round separately because userspace could reset + * the global counter at any time. + */ + atomic_long_inc(&sysctl_hung_task_detect_count); this_round_count++; hung_task_info(t, timeout, this_round_count); } @@ -340,15 +342,6 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout) if (!this_round_count) return; - /* - * Do not count this round when the global counter has been reset - * during this check. Release ensures we see all hang details - * recorded during the scan. - */ - atomic_long_cmpxchg_release(&sysctl_hung_task_detect_count, - total_count, total_count + - this_round_count); - if (need_warning || hung_task_call_panic) { si_mask |= SYS_INFO_LOCKS; I am not sure of the comment above the increment is needed. Well, it might help anyone to understand the motivation without digging in the git log history. Best Regards, Petr