* Re: [PATCH v3 1/3] mm/kmemleak: avoid soft lockup when scanning task stacks
2026-06-15 17:49 ` [PATCH v3 1/3] " Breno Leitao
@ 2026-06-15 18:24 ` Catalin Marinas
2026-06-15 18:46 ` Davidlohr Bueso
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Catalin Marinas @ 2026-06-15 18:24 UTC (permalink / raw)
To: Breno Leitao
Cc: Andrew Morton, lance.yang, Davidlohr Bueso, Oleg Nesterov,
Qian Cai, sj, linux-mm, linux-kernel, kernel-team, stable
On Mon, Jun 15, 2026 at 10:49:06AM -0700, Breno Leitao wrote:
> kmemleak_scan() walks every thread and scans its kernel stack under a
> single rcu_read_lock() with no reschedule point. On a host with very
> many threads -- amplified by KASAN/lockdep in debug builds -- this loop
> can hog a CPU long enough to trip the soft lockup watchdog:
>
> watchdog: BUG: soft lockup - CPU#35 stuck for 22s! [kmemleak:537]
> scan_block
> kmemleak_scan
> kmemleak_scan_thread
> kthread
>
> A cond_resched() cannot be added directly: the loop runs inside an RCU
> read-side critical section.
>
> Walk the tasks one PID at a time with find_ge_pid(), taking the RCU read
> lock only to look up and pin each task. The stack is then scanned with no
> lock held, so cond_resched() runs between tasks and the scan stops early
> on scan_should_stop(). This follows the next_tgid()/task_seq_get_next()
> iteration pattern and keeps each RCU critical section short.
>
> Fixes: c4b28963fd79 ("mm/kmemleak: rely on rcu for task stack scanning")
> Cc: stable@vger.kernel.org
> Signed-off-by: Breno Leitao <leitao@debian.org>
I think the Fixes is just a marker to tell how far back to go. Before
the above commit, we used a read_lock(&tasklist_lock) which probably had
similar issues.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Thanks.
--
Catalin
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3 1/3] mm/kmemleak: avoid soft lockup when scanning task stacks
2026-06-15 17:49 ` [PATCH v3 1/3] " Breno Leitao
2026-06-15 18:24 ` Catalin Marinas
@ 2026-06-15 18:46 ` Davidlohr Bueso
2026-06-16 2:31 ` Lance Yang
2026-06-16 9:48 ` Oleg Nesterov
3 siblings, 0 replies; 6+ messages in thread
From: Davidlohr Bueso @ 2026-06-15 18:46 UTC (permalink / raw)
To: Breno Leitao
Cc: Catalin Marinas, Andrew Morton, lance.yang, Oleg Nesterov,
Qian Cai, sj, linux-mm, linux-kernel, kernel-team, stable
On Mon, 15 Jun 2026, Breno Leitao wrote:
>kmemleak_scan() walks every thread and scans its kernel stack under a
>single rcu_read_lock() with no reschedule point. On a host with very
>many threads -- amplified by KASAN/lockdep in debug builds -- this loop
>can hog a CPU long enough to trip the soft lockup watchdog:
>
> watchdog: BUG: soft lockup - CPU#35 stuck for 22s! [kmemleak:537]
> scan_block
> kmemleak_scan
> kmemleak_scan_thread
> kthread
>
>A cond_resched() cannot be added directly: the loop runs inside an RCU
>read-side critical section.
>
>Walk the tasks one PID at a time with find_ge_pid(), taking the RCU read
>lock only to look up and pin each task. The stack is then scanned with no
>lock held, so cond_resched() runs between tasks and the scan stops early
>on scan_should_stop(). This follows the next_tgid()/task_seq_get_next()
>iteration pattern and keeps each RCU critical section short.
>
>Fixes: c4b28963fd79 ("mm/kmemleak: rely on rcu for task stack scanning")
>Cc: stable@vger.kernel.org
>Signed-off-by: Breno Leitao <leitao@debian.org>
LGTM
Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3 1/3] mm/kmemleak: avoid soft lockup when scanning task stacks
2026-06-15 17:49 ` [PATCH v3 1/3] " Breno Leitao
2026-06-15 18:24 ` Catalin Marinas
2026-06-15 18:46 ` Davidlohr Bueso
@ 2026-06-16 2:31 ` Lance Yang
2026-06-16 9:48 ` Oleg Nesterov
3 siblings, 0 replies; 6+ messages in thread
From: Lance Yang @ 2026-06-16 2:31 UTC (permalink / raw)
To: leitao
Cc: catalin.marinas, akpm, lance.yang, dave, oleg, cai, sj, linux-mm,
linux-kernel, kernel-team, stable
On Mon, Jun 15, 2026 at 10:49:06AM -0700, Breno Leitao wrote:
>kmemleak_scan() walks every thread and scans its kernel stack under a
>single rcu_read_lock() with no reschedule point. On a host with very
>many threads -- amplified by KASAN/lockdep in debug builds -- this loop
>can hog a CPU long enough to trip the soft lockup watchdog:
>
> watchdog: BUG: soft lockup - CPU#35 stuck for 22s! [kmemleak:537]
> scan_block
> kmemleak_scan
> kmemleak_scan_thread
> kthread
>
>A cond_resched() cannot be added directly: the loop runs inside an RCU
>read-side critical section.
>
>Walk the tasks one PID at a time with find_ge_pid(), taking the RCU read
>lock only to look up and pin each task. The stack is then scanned with no
>lock held, so cond_resched() runs between tasks and the scan stops early
>on scan_should_stop(). This follows the next_tgid()/task_seq_get_next()
>iteration pattern and keeps each RCU critical section short.
>
>Fixes: c4b28963fd79 ("mm/kmemleak: rely on rcu for task stack scanning")
>Cc: stable@vger.kernel.org
>Signed-off-by: Breno Leitao <leitao@debian.org>
>---
> mm/kmemleak.c | 51 ++++++++++++++++++++++++++++++++++++++-------------
> 1 file changed, 38 insertions(+), 13 deletions(-)
>
>diff --git a/mm/kmemleak.c b/mm/kmemleak.c
>index 7c7ba17ce7af0..a7786b6bc174e 100644
>--- a/mm/kmemleak.c
>+++ b/mm/kmemleak.c
>@@ -1695,6 +1695,42 @@ static void kmemleak_cond_resched(struct kmemleak_object *object)
> put_object(object);
> }
>
>+/*
>+ * Scan all task kernel stacks, rescheduling between tasks. Each task is looked
>+ * up and pinned within its own RCU read-side section, so no lock is held across
>+ * the scan and the walk cannot trip the soft lockup watchdog.
>+ */
>+static void kmemleak_scan_task_stacks(void)
>+{
>+ struct pid *pid;
>+ int nr = 1;
>+
>+ do {
>+ struct task_struct *p = NULL;
>+
>+ rcu_read_lock();
>+ pid = find_ge_pid(nr, &init_pid_ns);
I wasn't aware of find_ge_pid() before. It walks the pid IDR, not every
possible pid number :) LGTM.
Reviewed-by: Lance Yang <lance.yang@linux.dev>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3 1/3] mm/kmemleak: avoid soft lockup when scanning task stacks
2026-06-15 17:49 ` [PATCH v3 1/3] " Breno Leitao
` (2 preceding siblings ...)
2026-06-16 2:31 ` Lance Yang
@ 2026-06-16 9:48 ` Oleg Nesterov
3 siblings, 0 replies; 6+ messages in thread
From: Oleg Nesterov @ 2026-06-16 9:48 UTC (permalink / raw)
To: Breno Leitao
Cc: Catalin Marinas, Andrew Morton, lance.yang, Davidlohr Bueso,
Qian Cai, sj, linux-mm, linux-kernel, kernel-team, stable
On 06/15, Breno Leitao wrote:
>
> Walk the tasks one PID at a time with find_ge_pid(), taking the RCU read
> lock only to look up and pin each task. The stack is then scanned with no
> lock held, so cond_resched() runs between tasks and the scan stops early
> on scan_should_stop(). This follows the next_tgid()/task_seq_get_next()
> iteration pattern and keeps each RCU critical section short.
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
^ permalink raw reply [flat|nested] 6+ messages in thread