Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Lance Yang <lance.yang@linux.dev>
To: Breno Leitao <leitao@debian.org>
Cc: catalin.marinas@arm.com, akpm@linux-foundation.org,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	kernel-team@meta.com, sj@kernel.org
Subject: Re: [PATCH RFC] mm/kmemleak: avoid soft lockup when scanning task stacks
Date: Fri, 12 Jun 2026 17:57:12 +0800	[thread overview]
Message-ID: <6bc446b7-8e25-4add-9d72-a3c9e9191533@linux.dev> (raw)
In-Reply-To: <aivJ5Dv0hAQ1sJu1@gmail.com>



On 2026/6/12 17:09, Breno Leitao wrote:
> Hello Lance,
> 
> First of all, thanks for ther review, really awesome!

Cool!

> On Fri, Jun 12, 2026 at 11:16:05AM +0800, Lance Yang wrote:
>> On Thu, Jun 11, 2026 at 05:45:00AM -0700, Breno Leitao wrote:
>>> kmemleak_scan() walks every thread and scans its kernel stack under a
>>> single rcu_read_lock() with no reschedule point. On a host with very
>>> many threads -- amplified by KASAN/lockdep in debug builds -- this loop
>>> can hog a CPU long enough to trip the soft lockup watchdog:
>>>
>>>   watchdog: BUG: soft lockup - CPU#35 stuck for 22s! [kmemleak:537]
>>>    scan_block
>>>    kmemleak_scan
>>>    kmemleak_scan_thread
>>>    kthread
>>
>> Neat, good catch!
>>
>>> A cond_resched() cannot be added directly: the loop runs inside an RCU
>>> read-side critical section.
>>>
>>> Split the scan in two parts:
>>>
>>> 1) get the list of tasks (with RCU read lock) in an array
>>> 2) run scan_block() for the tasks (with cond_reschd()).
>>>
>>> Is it a sane approach?
>>
>> Why not use the kernel/hung_task.c pattern here? Seems simpler, with no
>> extra task-array allocation ;)
> 
> I've looked at it, but I am not sure we want to break the loop mid-air,
> that seems to increase the false positives, given we did a half-baked
> scan, right?
> 
>> Could break RCU only when resched is needed. Pin the current cursors,
>> drop RCU, cond_resched(), take RCU again, and continue only if the
>> cursors are still alive ;)
>>
>> If either cursor died while RCU was droped, stopping this scan round
>> should be fine, IMHO.
> 
> I am not sure, this is not the same as the existing kmemleak_cond_resched()
> raciness in the object_list loops. Those iterate the marked set, where a miss
> only means "this object isn't reported until the next scan" -- under-reporting,
> self-healing, and the in-tree comment says exactly that.
> 
> Dropping a *root* mid-scan is the opposite: it makes *other* objects get
> falsely reported. So the "it's already racy, bailing is fine" reasoning doesn't
> carry over from the object loop to the stack loop.
> 
> If we go this route, the aborted round has to suppress reporting, reusing
> kmemleak's existing "scan was interrupted -> don't report" path:
> 
>        if (need_resched() && !kmemleak_stack_scan_break(g, p)) {
>                aborted = true;
>                goto unlock;
>        }

I'd expect the normal case to just drop RCU, cond_resched(), take RCU
again, see both cursors still alive, and keep walking :)

>        ...
>        if (scan_should_stop() || aborted)
>                return;

And yeah, you're right. If we do lost a cursor, bailing out and
skipping reporting fot that incomplete root scan should be the
right thing, I guess :D

> Then an abort means "this round reports nothing; the next full scan
> reports the real leaks" instead of a false-positive flood.
> 
> On boxes with very many threads, where the stack walk is long and
> need_resched() fires constantly, so the break helper runs a lot -- which makes
> aborts (and thus fully-suppressed, non-reporting rounds) plausibly more than
> "rare".
> 
> Since each round restarts from the head, the tail of the thread list is the
> most likely to be perpetually skipped, on exactly the workload this is meant to
> fix.
> 
> The snapshot avoids that by scanning a complete, similar to what we have today.
> 
> Anyway, I would love to get rid of the array, but, I am not convinced that
> dropping the scan mid-air will not cause false positives.
> 
> Thanks for the review,

Cheers!



  reply	other threads:[~2026-06-12  9:57 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-11 12:45 [PATCH RFC] mm/kmemleak: avoid soft lockup when scanning task stacks Breno Leitao
2026-06-12  1:10 ` SeongJae Park
2026-06-12  9:42   ` Breno Leitao
2026-06-12  3:16 ` Lance Yang
2026-06-12  4:27   ` Lance Yang
2026-06-12  9:09   ` Breno Leitao
2026-06-12  9:57     ` Lance Yang [this message]
2026-06-12 10:39       ` Breno Leitao
2026-06-12 11:22         ` Lance Yang
2026-06-12 11:57           ` Breno Leitao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6bc446b7-8e25-4add-9d72-a3c9e9191533@linux.dev \
    --to=lance.yang@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=catalin.marinas@arm.com \
    --cc=kernel-team@meta.com \
    --cc=leitao@debian.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=sj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox