Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Edward Srouji" <edwards@nvidia.com>
Cc: linux-rdma@vger.kernel.org
Subject: Re: [PATCH rdma-next 05/11] RDMA/mlx5: Cache the MR core device and resolve the MW's via its PD
Date: Wed, 18 Nov 2026 14:04:16 +0000	[thread overview]
Message-ID: <sashiko-outbox-221134@kernel.org> (raw)
In-Reply-To: <20261117-mlx5-sd-prep-v1-5-8efd124c7895@nvidia.com>

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] RDMA/mlx5: unlocked state check in mlx5r_umr_post_send() misses
  error flush [1]
--

commit 6252ceba1b31eba36693680458665fa14b71227b
Author: Akiva Goldberger <agoldberger@nvidia.com>

RDMA/mlx5: Cache the MR core device and resolve the MW's via its PD

This commit caches the mlx5_core_dev pointer in struct mlx5_ib_mr and adds
mr_to_mdev() and mw_to_mdev() helpers so MR and MW operations resolve the
core device from the resource instead of dev->mdev.

> diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
> index f030885505df5..f9485591130d3 100644
> --- a/drivers/infiniband/hw/mlx5/umr.c
> +++ b/drivers/infiniband/hw/mlx5/umr.c

[ ... ]

> @@ -243,8 +243,7 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
>  		with_data ? sizeof(struct mlx5r_umr_wqe) :
>  			    sizeof(struct mlx5r_umr_wqe) -
>  				    sizeof(struct mlx5_wqe_data_seg);
> -	struct mlx5_ib_dev *dev = to_dev(ibqp->device);
> -	struct mlx5_core_dev *mdev = dev->mdev;
> +	struct mlx5_core_dev *mdev = qp_to_mdev(ibqp);
>  	struct mlx5_ib_qp *qp = to_mqp(ibqp);
>  	struct mlx5_wqe_ctrl_seg *ctrl;
>  	union {

[Severity: High]
This problem wasn't introduced by this patch, but can checking mdev->state
outside qp->sq.lock in mlx5r_umr_post_send() [1] cause
mlx5r_umr_post_send_wait() to hang forever in wait_for_completion()?

mlx5_ib_handle_internal_error() walks ibdev->qp_list under mqp->sq.lock and
arms send_mcq for software flushing only when mqp->sq.tail != mqp->sq.head.
Because mlx5r_umr_post_send() checks mdev->state before taking qp->sq.lock,
it can queue a WQE after the error handler has already checked the SQ:

CPU0                                  CPU1
mlx5r_umr_post_send_wait()
  down(&umrc->sem)
  mlx5r_umr_post_send()
    if (mdev->state ==
        MLX5_DEVICE_STATE_INTERNAL_ERROR)
                                      enter_error_state()
                                        mdev->state =
                                          MLX5_DEVICE_STATE_INTERNAL_ERROR
                                        mlx5_ib_handle_internal_error()
                                          spin_lock(&mqp->sq.lock)
                                          if (mqp->sq.tail != mqp->sq.head)
                                            /* arm send_mcq for SW flush */
                                          spin_unlock(&mqp->sq.lock)
    spin_lock_irqsave(&qp->sq.lock, flags)
    mlx5r_finish_wqe(...)
    mlx5r_ring_db(...)
    spin_unlock_irqrestore(&qp->sq.lock, flags)
  wait_for_completion(&umr_context.done)  <-- hangs forever holding umrc->sem

With hardware completions stopped and software flushing already past this QP,
umr_context.done is never completed, blocking subsequent UMR operations and
device teardown on umrc->sem.  Checking mdev->state under qp->sq.lock (as
mlx5_ib_post_send() does) avoids this race.

[1] https://sashiko.dev/#/bug/linux-d43f2378-f2de-4246-8c9d-53b5f5f256a1

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-mlx5-sd-prep-v1-0-8efd124c7895@nvidia.com?part=5

  reply	other threads:[~2026-11-18 14:04 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 14:01 [PATCH rdma-next 00/11] RDMA/mlx5: Preparation for SD support Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 01/11] RDMA/mlx5: Enforce dev/mdev naming for device pointers Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 02/11] RDMA/mlx5: Rename to_mdev() helpers that return an IB device Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 03/11] RDMA/mlx5: Select the PD and SRQ core device through the PD Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 04/11] RDMA/mlx5: Select the core device through PD for QP commands Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 05/11] RDMA/mlx5: Cache the MR core device and resolve the MW's via its PD Edward Srouji
2026-11-18 14:04   ` sashiko-bot [this message]
2026-11-17 14:01 ` [PATCH rdma-next 06/11] RDMA/mlx5: Select the AH core device through their PD Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 07/11] RDMA/mlx5: Select the QP core device through its PD Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 08/11] RDMA/mlx5: Resolve core device from ucontext for DEVX resources Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 09/11] RDMA/mlx5: Cache the CQ core device and resolve it via cq_to_mdev() Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 10/11] RDMA/mlx5: Resolve the bfreg UAR-index core device from the ucontext Edward Srouji
2026-11-18 14:04   ` sashiko-bot
2026-11-17 14:01 ` [PATCH rdma-next 11/11] RDMA/mlx5: Resolve UAR and mmap core devices via " Edward Srouji
2026-11-18 14:04   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-221134@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=edwards@nvidia.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox