All of lore.kernel.org
 help / color / mirror / Atom feed
From: Breno Leitao <leitao@debian.org>
To: "Paul E. McKenney" <paulmck@kernel.org>,
	rostedt@goodmis.org,  pjaroszynski@nvidia.com
Cc: Andrew Morton <akpm@linux-foundation.org>,
	 Catalin Marinas <catalin.marinas@arm.com>,
	puranjay@kernel.org, linux-mm@kvack.org,
	 linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	kernel-team@meta.com, dcostantino@meta.com
Subject: Re: [PATCH] mm/kmemleak: report RCU-tasks quiescent states during the scan
Date: Thu, 30 Jul 2026 07:06:44 -0700	[thread overview]
Message-ID: <amtZNwxZ3gpOsPER@gmail.com> (raw)
In-Reply-To: <amdWVTs0WKOxguxP@gmail.com>

On Mon, Jul 27, 2026 at 06:04:37AM -0700, Breno Leitao wrote:
> I found 3 different cases on Meta fleet, where rcu task stalls show up:
...
> 3) Nvidia driver
>   * stuck in nv_procfs_read_lock_params

I've investigated this issue. The nvidia driver contains the following
locking pattern:

        static inline int nv_down_read_interruptible(struct rw_semaphore *lock)
        {
                while (!down_read_trylock(lock))
                {
                        if (signal_pending(current))
                        return -EINTR;
                        cond_resched();
                }
                return 0;
        }

Code at: https://github.com/NVIDIA/open-gpu-kernel-modules/blob/452cec62d827034798072827d3866d1881662b77/kernel-open/common/inc/nv-lock.h#L54

The lock is never acquired, and cond_resched() provides insufficient
scheduling relief.

This triggers a more severe issue with ftrace, which holds ftrace_lock
across RCU synchronization:

        mutex_lock(&ftrace_lock);
                ftrace_shutdown(struct ftrace_ops *ops, int command) {
                        synchronize_rcu_tasks_rude();
                        synchronize_rcu_tasks();
                }
        mutex_unlock(&ftrace_lock);

Since nv_down_read_interruptible() spins indefinitely, ftrace_lock
now becomes completely wedged.

I suppose we want two things:

1) change cond_resched() with cond_resched_tasks_rcu_qs()
2) Avoiding holding the ftrace lock while calling
   synchronize_rcu_tasks()? It can take up to 10 minutes on a healthy
   system to be releasd, right?

--breno


  parent reply	other threads:[~2026-07-30 14:07 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-20 13:23 [PATCH] mm/kmemleak: report RCU-tasks quiescent states during the scan Breno Leitao
2026-07-20 13:37 ` sashiko-bot
2026-07-20 15:25 ` Paul E. McKenney
2026-07-20 22:39 ` Andrew Morton
2026-07-21  4:21   ` Paul E. McKenney
2026-07-23 13:26     ` Breno Leitao
2026-07-23 15:56       ` Paul E. McKenney
2026-07-24  9:58         ` Breno Leitao
2026-07-24 17:29           ` Paul E. McKenney
2026-07-27 13:04             ` Breno Leitao
2026-07-27 16:02               ` Breno Leitao
2026-07-30 14:06               ` Breno Leitao [this message]
2026-07-30 14:57                 ` Paul E. McKenney
2026-07-21  0:00 ` SJ Park

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=amtZNwxZ3gpOsPER@gmail.com \
    --to=leitao@debian.org \
    --cc=akpm@linux-foundation.org \
    --cc=bpf@vger.kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=dcostantino@meta.com \
    --cc=kernel-team@meta.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=paulmck@kernel.org \
    --cc=pjaroszynski@nvidia.com \
    --cc=puranjay@kernel.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.