Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: Selvin Xavier <selvin.xavier@broadcom.com>
To: jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com,
	Kashyap Desai <kashyap.desai@broadcom.com>,
	Selvin Xavier <selvin.xavier@broadcom.com>
Subject: [PATCH for-next 02/17] RDMA/bnxt_re: Avoid calling wake_up threads from spin_lock context
Date: Thu,  8 Jun 2023 03:24:53 -0700	[thread overview]
Message-ID: <1686219908-11181-3-git-send-email-selvin.xavier@broadcom.com> (raw)
In-Reply-To: <1686219908-11181-1-git-send-email-selvin.xavier@broadcom.com>

[-- Attachment #1: Type: text/plain, Size: 3161 bytes --]

From: Kashyap Desai <kashyap.desai@broadcom.com>

bnxt_qplib_service_creq can be called from interrupt or tasklet or
process context. So the function take irq variant  of spin_lock.
But when wake_up is invoked with the lock held, it is putting the
calling context to sleep.

[exception RIP: __wake_up_common+190]
RIP: ffffffffb7539d7e  RSP: ffffa73300207ad8  RFLAGS: 00000083
RAX: 0000000000000001  RBX: ffff91fa295f69b8  RCX: dead000000000200
RDX: ffffa733344af940  RSI: ffffa73336527940  RDI: ffffa73336527940
RBP: 000000000000001c   R8: 0000000000000002   R9: 00000000000299c0
R10: 0000017230de82c5  R11: 0000000000000002  R12: ffffa73300207b28
R13: 0000000000000000  R14: ffffa733341bf928  R15: 0000000000000000
ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

Call the wakeup after releasing the lock.

Fixes: 1ac5a4047975 ("RDMA/bnxt_re: Add bnxt_re RoCE driver")
Signed-off-by: Kashyap Desai <kashyap.desai@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
index d4ce82b..c11b8e7 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
@@ -295,7 +295,8 @@ static int bnxt_qplib_process_func_event(struct bnxt_qplib_rcfw *rcfw,
 }
 
 static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw,
-				       struct creq_qp_event *qp_event)
+				       struct creq_qp_event *qp_event,
+				       u32 *num_wait)
 {
 	struct creq_qp_error_notification *err_event;
 	struct bnxt_qplib_hwq *hwq = &rcfw->cmdq.hwq;
@@ -304,6 +305,7 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw,
 	u16 cbit, blocked = 0;
 	struct pci_dev *pdev;
 	unsigned long flags;
+	u32 wait_cmds = 0;
 	__le16  mcookie;
 	u16 cookie;
 	int rc = 0;
@@ -363,9 +365,10 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw,
 		crsqe->req_size = 0;
 
 		if (!blocked)
-			wake_up(&rcfw->cmdq.waitq);
+			wait_cmds++;
 		spin_unlock_irqrestore(&hwq->lock, flags);
 	}
+	*num_wait += wait_cmds;
 	return rc;
 }
 
@@ -379,6 +382,7 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t)
 	struct creq_base *creqe;
 	u32 sw_cons, raw_cons;
 	unsigned long flags;
+	u32 num_wakeup = 0;
 
 	/* Service the CREQ until budget is over */
 	spin_lock_irqsave(&hwq->lock, flags);
@@ -397,7 +401,8 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t)
 		switch (type) {
 		case CREQ_BASE_TYPE_QP_EVENT:
 			bnxt_qplib_process_qp_event
-				(rcfw, (struct creq_qp_event *)creqe);
+				(rcfw, (struct creq_qp_event *)creqe,
+				 &num_wakeup);
 			creq->stats.creq_qp_event_processed++;
 			break;
 		case CREQ_BASE_TYPE_FUNC_EVENT:
@@ -425,6 +430,8 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t)
 				      rcfw->res->cctx, true);
 	}
 	spin_unlock_irqrestore(&hwq->lock, flags);
+	if (num_wakeup)
+		wake_up_nr(&rcfw->cmdq.waitq, num_wakeup);
 }
 
 static irqreturn_t bnxt_qplib_creq_irq(int irq, void *dev_instance)
-- 
2.5.5


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4224 bytes --]

  parent reply	other threads:[~2023-06-08 10:37 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-08 10:24 [PATCH for-next 00/17] RDMA/bnxt_re: Control path updates Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 01/17] RDMA/bnxt_re: wraparound mbox producer index Selvin Xavier
2023-06-08 10:24 ` Selvin Xavier [this message]
2023-06-08 10:24 ` [PATCH for-next 03/17] RDMA/bnxt_re: remove virt_func check while creating RoCE FW channel Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 04/17] RDMA/bnxt_re: set fixed command queue depth Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 05/17] RDMA/bnxt_re: Enhance the existing functions that wait for FW responses Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 06/17] RDMA/bnxt_re: Avoid the command wait if firmware is inactive Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 07/17] RDMA/bnxt_re: use shadow qd while posting non blocking rcfw command Selvin Xavier
2023-06-08 10:24 ` [PATCH for-next 08/17] RDMA/bnxt_re: Simplify the function that sends the FW commands Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 09/17] RDMA/bnxt_re: add helper function __poll_for_resp Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 10/17] RDMA/bnxt_re: handle command completions after driver detect a timedout Selvin Xavier
2023-06-08 12:53   ` kernel test robot
2023-06-08 10:25 ` [PATCH for-next 11/17] RDMA/bnxt_re: Add firmware stall check detection Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 12/17] RDMA/bnxt_re: post destroy_ah for delayed completion of AH creation Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 13/17] RDMA/bnxt_re: consider timeout of destroy ah as success Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 14/17] RDMA/bnxt_re: cancel all control path command waiters upon error Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 15/17] RDMA/bnxt_re: use firmware provided max request timeout Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 16/17] RDMA/bnxt_re: remove redundant cmdq_bitmap Selvin Xavier
2023-06-08 10:25 ` [PATCH for-next 17/17] RDMA/bnxt_re: optimize the parameters passed to helper functions Selvin Xavier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1686219908-11181-3-git-send-email-selvin.xavier@broadcom.com \
    --to=selvin.xavier@broadcom.com \
    --cc=andrew.gospodarek@broadcom.com \
    --cc=jgg@ziepe.ca \
    --cc=kashyap.desai@broadcom.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox