All of lore.kernel.org
 help / color / mirror / Atom feed
From: Leon Romanovsky <leon@kernel.org>
To: Palla Raghunath <raghunathpalla.0209@gmail.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
	linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
	Shuah Khan <shuah@kernel.org>,
	Brigham Campbell <me@brighamcampbell.com>,
	linux-kernel-mentees@lists.linux.dev,
	syzbot+ae549381b4daac2895b1@syzkaller.appspotmail.com
Subject: Re: [PATCH] RDMA/cma: wait for addr_handler() to finish in rdma_destroy_id()
Date: Sun, 11 Oct 2026 13:34:33 +0300	[thread overview]
Message-ID: <20261011103433.GB27033@unreal> (raw)
In-Reply-To: <20261004112613.18171-1-raghunathpalla.0209@gmail.com>

On Sun, Oct 04, 2026 at 12:26:13PM +0100, Palla Raghunath wrote:
> syzbot hit a use-after-free of id_priv in addr_handler(). The bad read
> is in debug_mutex_unlock(), from the last mutex_unlock() of
> handler_mutex, and the memory was freed by ucma_close() ->
> rdma_destroy_id().
> 
> addr_handler() moves the state from RDMA_CM_ADDR_QUERY to
> RDMA_CM_ADDR_RESOLVED (or RDMA_CM_ADDR_BOUND on error) under
> handler_mutex. If rdma_destroy_id() runs at that point, it waits on
> handler_mutex. When addr_handler() unlocks, the destroying task can take
> the mutex before mutex_unlock() has returned. It then sees a state other
> than RDMA_CM_ADDR_QUERY, so cma_cancel_operation() skips
> rdma_addr_cancel(), and _destroy_id() frees id_priv. mutex_unlock() in
> the work then touches the freed lock:
> 
>   ib_addr work                         close()
>   addr_handler()
>     mutex_lock(handler_mutex)
>     ADDR_QUERY -> ADDR_RESOLVED
>     ...                                rdma_destroy_id()
>                                          mutex_lock(handler_mutex)
>     mutex_unlock(handler_mutex)
>       owner cleared                      gets the mutex
>                                          state != ADDR_QUERY, no cancel
>                                          kfree(id_priv)
>       debug_mutex_unlock(lock)  <- use-after-free
> 
> Documentation/locking/mutex-design.rst says mutex_unlock() may still
> touch the mutex after another task has acquired it, so handler_mutex
> can't be what keeps id_priv alive here. The comment in
> cma_cancel_operation() assumes it can. Before commit 722c7b2bfead
> ("RDMA/{cma, core}: Avoid callback on rdma_addr_cancel()"),
> addr_handler() held a reference on id_priv until after mutex_unlock(),
> which covered this.
> 
> So in rdma_destroy_id(), call rdma_addr_cancel() before taking
> handler_mutex if a resolve was ever started on this id. The req stays on
> req_list until the callback returns, so rdma_addr_cancel() finds it and
> cancel_delayed_work_sync() waits until addr_handler() is really done.
> This doesn't deadlock with the work itself. When addr_handler() destroys
> the id because the event handler returned non-zero, it uses
> destroy_id_handler_unlock(), not rdma_destroy_id(). Event handlers also
> run with handler_mutex held, so they can't call rdma_destroy_id() on
> their own id anyway.
> 
> There's no reproducer from syzbot, so I made the window bigger with a
> debug-only mdelay(1000) after the mutex_unlock() in addr_handler(),
> followed by a read of handler_mutex.magic. A small test program creates
> an id, resolves an address on an rxe device and closes the fd while the
> work sits in that delay. Without this patch KASAN reports the same
> slab-use-after-free as syzbot (1048 bytes into a kmalloc-2k object,
> freed by ucma_close()). With it the test runs clean, and close() just
> waits for the work to finish.
> 
> Fixes: 722c7b2bfead ("RDMA/{cma, core}: Avoid callback on rdma_addr_cancel()")
> Reported-by: syzbot+ae549381b4daac2895b1@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=ae549381b4daac2895b1
> Signed-off-by: Palla Raghunath <raghunathpalla.0209@gmail.com>
> ---
>  drivers/infiniband/core/cma.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 337a49d1acf7..24235eb0c49b 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -1968,10 +1968,10 @@ static void cma_cancel_operation(struct rdma_id_private *id_priv,
>  		/*
>  		 * We can avoid doing the rdma_addr_cancel() based on state,
>  		 * only RDMA_CM_ADDR_QUERY has a work that could still execute.
> -		 * Notice that the addr_handler work could still be exiting
> -		 * outside this state, however due to the interaction with the
> -		 * handler_mutex the work is guaranteed not to touch id_priv
> -		 * during exit.
> +		 * The addr_handler work can still be finishing its
> +		 * mutex_unlock() after it has left this state.
> +		 * rdma_destroy_id() waits for that before it takes
> +		 * handler_mutex.
>  		 */
>  		rdma_addr_cancel(&id_priv->id.route.addr.dev_addr);
>  		break;
> @@ -2121,6 +2121,17 @@ void rdma_destroy_id(struct rdma_cm_id *id)
>  	struct rdma_id_private *id_priv =
>  		container_of(id, struct rdma_id_private, id);
>  
> +	/*
> +	 * addr_handler() can still be in mutex_unlock(&handler_mutex) after
> +	 * it has moved the state on from RDMA_CM_ADDR_QUERY, and
> +	 * mutex_unlock() may touch the mutex even after we have taken it.
> +	 * Wait for the work to finish before we free id_priv. The req stays
> +	 * on req_list until the callback returns, so rdma_addr_cancel() will
> +	 * find it.
> +	 */
> +	if (id_priv->used_resolve_ip)
> +		rdma_addr_cancel(&id->route.addr.dev_addr);
> +

It may call `rdma_addr_cancel()` twice. This is probably harmless, but still
undesirable.

Thanks

>  	mutex_lock(&id_priv->handler_mutex);
>  	destroy_id_handler_unlock(id_priv);
>  }
> -- 
> 2.34.1
> 

      parent reply	other threads:[~2026-10-11 10:34 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-04 11:26 [PATCH] RDMA/cma: wait for addr_handler() to finish in rdma_destroy_id() Palla Raghunath
2026-10-04 11:40 ` sashiko-bot
2026-10-11 10:34 ` Leon Romanovsky [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261011103433.GB27033@unreal \
    --to=leon@kernel.org \
    --cc=jgg@ziepe.ca \
    --cc=linux-kernel-mentees@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=me@brighamcampbell.com \
    --cc=raghunathpalla.0209@gmail.com \
    --cc=shuah@kernel.org \
    --cc=syzbot+ae549381b4daac2895b1@syzkaller.appspotmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.