Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Anuj Gupta" <anuj20.g@samsung.com>
Cc: linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org
Subject: Re: [RFC 4/4] nvme-rdma: support dma-buf backed requests
Date: Wed, 18 Nov 2026 11:57:15 +0000	[thread overview]
Message-ID: <sashiko-outbox-220866@kernel.org> (raw)
In-Reply-To: <20261118103327.1788667-5-anuj20.g@samsung.com>

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] nvme-rdma: use-after-free on ctrl->device in
  nvme_rdma_remove_one()
- [Medium] nvme-rdma: nvme_rdma_remove_one() fails to wait for
  nvme_delete_ctrl_sync()
--

commit 7ad9fcf127f13cdfc7ce401ed605b13cf6ca3225
Author: Nitesh Shetty <nj.shetty@samsung.com>

nvme-rdma: support dma-buf backed requests

This commit implements ->init_dma_buf_io_ctx for nvme-rdma so that memory
exported through dma-buf can be registered with io_uring and used for I/O
to an nvme-rdma namespace.

> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 32189fdfc0f51..e7c26bd4f77d0 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c

[ ... ]

> @@ -2413,7 +2573,7 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
>  {
>  	struct nvme_rdma_ctrl *ctrl;
>  	struct nvme_rdma_device *ndev;
> -	bool found = false;
> +	bool found = false, need_flush;
>  
>  	mutex_lock(&device_list_mutex);
>  	list_for_each_entry(ndev, &device_list, entry) {
> @@ -2423,20 +2583,24 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
>  		}
>  	}
>  	mutex_unlock(&device_list_mutex);
> +	need_flush = found;
>  
> -	if (!found)
> -		return;
> -
> -	/* Delete all controllers using this device */
> +	/*
> +	 * Delete controllers using this device, and controllers that
> +	 * reconnected elsewhere but still have dma-bufs attached to it.
> +	 */
>  	mutex_lock(&nvme_rdma_ctrl_mutex);
>  	list_for_each_entry(ctrl, &nvme_rdma_ctrl_list, list) {
> -		if (ctrl->device->dev != ib_device)
> +		if (!(found && ctrl->device->dev == ib_device) &&
> +		    !nvme_dmabufs_use_owner(&ctrl->dmabufs, ib_device))

[Severity: High]
Can dereferencing ctrl->device->dev here cause a use-after-free on
ctrl->device?

ctrl->device is assigned from ctrl->queues[0].device without its own
nvme_rdma_device reference, so when nvme_rdma_destroy_admin_queue() frees
ctrl->queues[0], nvme_rdma_dev_put() can free ctrl->device while leaving
ctrl->device non-NULL.  With this patch, __nvme_init_dma_buf_io_ctx_locked()
takes an nvme_get_ctrl() reference that is only dropped in
nvme_dma_buf_io_release(), keeping deleted controllers on
nvme_rdma_ctrl_list after their admin queue and ctrl->device have been
freed (and reconnecting controllers on nvme_rdma_ctrl_list also have no
admin queue).

When found is true for ib_device, this loop visits every controller on
nvme_rdma_ctrl_list and reads ctrl->device->dev without holding
ctrl->dmabufs.lock or checking !ctrl->dmabufs.dead.  Even for a controller
using ib_device, its admin queue can be torn down after device_list_mutex
is dropped:

CPU0                                  CPU1
nvme_rdma_remove_one(ib_device)
  mutex_lock(&device_list_mutex)
  found = true
  mutex_unlock(&device_list_mutex)
                                      nvme_rdma_error_recovery_work()
                                        nvme_rdma_destroy_admin_queue()
                                          nvme_rdma_free_queue(&queues[0])
                                            nvme_rdma_dev_put(device)
                                              kfree(device)
  mutex_lock(&nvme_rdma_ctrl_mutex)
  ctrl->device->dev == ib_device  <-- use-after-free

Checking ctrl->device->dev under ctrl->dmabufs.lock only when
!ctrl->dmabufs.dead would ensure the admin queue and ctrl->device are still
live.

>  			continue;
>  		nvme_delete_ctrl(&ctrl->ctrl);
> +		need_flush = true;
>  	}
>  	mutex_unlock(&nvme_rdma_ctrl_mutex);
>  
> -	flush_workqueue(nvme_delete_wq);
> +	if (need_flush)
> +		flush_workqueue(nvme_delete_wq);
>  }

[Severity: Medium]
Can nvme_rdma_remove_one() return before nvme_remove_dmabufs() detaches
contexts from ib_device when racing with nvme_delete_ctrl_sync()?

CPU0                                  CPU1
nvme_sysfs_delete()
  nvme_delete_ctrl_sync()
    nvme_change_ctrl_state(...,
      NVME_CTRL_DELETING)
    nvme_do_delete_ctrl()
                                      nvme_rdma_remove_one(ib_device)
                                        nvme_dmabufs_use_owner(...) == true
                                        nvme_delete_ctrl(&ctrl->ctrl)
                                          /* returns -EBUSY, already DELETING */
                                        flush_workqueue(nvme_delete_wq)
                                        /* returns to ib_unregister_device() */
      nvme_rdma_delete_ctrl()
        nvme_remove_dmabufs()  <-- detaches after remove_one() returned

Because nvme_delete_ctrl_sync() runs nvme_do_delete_ctrl() directly in task
context rather than on nvme_delete_wq, nvme_delete_ctrl() fails to queue
delete_work and flush_workqueue(nvme_delete_wq) does not wait for the
in-progress deletion to finish detaching the dma-bufs from ib_device.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261118103327.1788667-1-anuj20.g@samsung.com?part=4

      reply	other threads:[~2026-11-18 11:57 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20261118103434epcas5p4ddef3806fafdb60f309fe882e354881c@epcas5p4.samsung.com>
2026-11-18 10:33 ` [RFC 0/4] nvme-rdma: dma-buf backed I/O Anuj Gupta
2026-11-18 10:33   ` [RFC 1/4] nvme: move dma-buf I/O context handling to nvme-core Anuj Gupta
2026-11-18 11:36     ` sashiko-bot
2026-11-18 10:33   ` [RFC 2/4] nvme: let transports provide the dma-buf attachment Anuj Gupta
2026-11-18 11:37     ` sashiko-bot
2026-11-18 10:33   ` [RFC 3/4] RDMA/core: Add ib_dma_buf_dynamic_attach() Anuj Gupta
2026-11-18 11:36     ` sashiko-bot
2026-11-18 10:33   ` [RFC 4/4] nvme-rdma: support dma-buf backed requests Anuj Gupta
2026-11-18 11:57     ` sashiko-bot [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-220866@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=anuj20.g@samsung.com \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox