[for-next 00/12] RDMA/irdma: A few fixes for irdma

[for-next 00/12] RDMA/irdma: A few fixes for irdma

[Tatyana Nikolova]

This series includes a few irdma fixes:

 - Change request_done type to atomic
 - Change ah_valid type to atomic
 - Clean up unnecessary dereference of event->cm_node
 - Initialize free_qp completion before using it
 - Harden SQ/RQ depth calculation functions
 - Update ibqp state to error if QP is already in error state
 - Fix deadlock during netdev reset with active connections
 - Return EINVAL for invalid arp index error
 - Remove a NOP wait_event() in irdma_modify_qp_roce()
 - Remove reset check from irdma_modify_qp_to_err() to ensure disconnect

The series also include the following additions:

 - Add support for GEN4 hardware
 - Provide scratch buffers to firmware for internal use

Anil Samal (1):
  RDMA/irdma: Fix deadlock during netdev reset with active connections

Ivan Barrera (1):
  RDMA/irdma: Clean up unnecessary dereference of event->cm_node

Jacob Moroni (2):
  RDMA/irdma: Initialize free_qp completion before using it
  RDMA/irdma: Add support for GEN4 hardware

Jay Bhat (1):
  RDMA/irdma: Provide scratch buffers to firmware for internal use

Krzysztof Czurylo (2):
  RDMA/irdma: Fix data race on cqp_request->request_done
  RDMA/irdma: Change ah_valid type to atomic

Shiraz Saleem (1):
  RDMA/irdma: Harden depth calculation functions

Tatyana Nikolova (4):
  RDMA/irdma: Update ibqp state to error if QP is already in error state
  RDMA/irdma: Remove a NOP wait_event() in irdma_modify_qp_roce()
  RDMA/irdma: Remove reset check from irdma_modify_qp_to_err()
  RDMA/irdma: Return EINVAL for invalid arp index error

 drivers/infiniband/hw/irdma/cm.c         | 31 +++++++++--------
 drivers/infiniband/hw/irdma/ctrl.c       | 44 +++++++++++++++++++++++-
 drivers/infiniband/hw/irdma/defs.h       |  4 +++
 drivers/infiniband/hw/irdma/hw.c         | 27 ++++++++++++---
 drivers/infiniband/hw/irdma/ig3rdma_hw.c |  1 -
 drivers/infiniband/hw/irdma/irdma.h      |  1 +
 drivers/infiniband/hw/irdma/main.h       |  2 +-
 drivers/infiniband/hw/irdma/puda.c       |  2 +-
 drivers/infiniband/hw/irdma/type.h       |  2 ++
 drivers/infiniband/hw/irdma/uda.h        |  2 +-
 drivers/infiniband/hw/irdma/uk.c         | 39 ++++++++++++---------
 drivers/infiniband/hw/irdma/user.h       |  4 +--
 drivers/infiniband/hw/irdma/utils.c      | 24 ++++++-------
 drivers/infiniband/hw/irdma/verbs.c      | 16 +++++----
 14 files changed, 137 insertions(+), 62 deletions(-)

-- 
2.31.1

[for-next 01/12] RDMA/irdma: Initialize free_qp completion before using it

[Tatyana Nikolova]

From: Jacob Moroni <jmoroni@google.com>

In irdma_create_qp, if ib_copy_to_udata fails, it will call
irdma_destroy_qp to clean up which will attempt to wait on
the free_qp completion, which is not initialized yet. Fix this
by initializing the completion before the ib_copy_to_udata call.

Fixes: b48c24c2d710 ("RDMA/irdma: Implement device supported verb APIs")
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/verbs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 1d0c2d8453a8..38bc0e656ecf 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -1105,6 +1105,7 @@ static int irdma_create_qp(struct ib_qp *ibqp,
 	spin_lock_init(&iwqp->sc_qp.pfpdu.lock);
 	iwqp->sig_all = init_attr->sq_sig_type == IB_SIGNAL_ALL_WR;
 	rf->qp_table[qp_num] = iwqp;
+	init_completion(&iwqp->free_qp);
 
 	if (udata) {
 		/* GEN_1 legacy support with libi40iw does not have expanded uresp struct */
@@ -1129,7 +1130,6 @@ static int irdma_create_qp(struct ib_qp *ibqp,
 		}
 	}
 
-	init_completion(&iwqp->free_qp);
 	return 0;
 
 error:
-- 
2.31.1

[for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Tatyana Nikolova]

From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>

Changes type of request_done flag from bool to atomic_t to fix
data race in irdma_complete_cqp_request / irdma_wait_event
reported by KCSAN:

BUG: KCSAN: data-race in irdma_complete_cqp_request [irdma] / irdma_wait_event [irdma]

write (marked) to 0xffffa0bef390ad5c of 1 bytes by task 7761 on cpu 0:
 irdma_complete_cqp_request+0x19/0x90 [irdma]
 irdma_cqp_ce_handler+0x22d/0x290 [irdma]
 cqp_compl_worker+0x1f/0x30 [irdma]
 process_one_work+0x3dc/0x7c0
 worker_thread+0xa6/0x6c0
 kthread+0x17f/0x1c0
 ret_from_fork+0x2c/0x50

read to 0xffffa0bef390ad5c of 1 bytes by task 28566 on cpu 3:
 irdma_wait_event+0x242/0x390 [irdma]
 irdma_handle_cqp_op+0x93/0x210 [irdma]
 irdma_hw_modify_qp+0xe3/0x2f0 [irdma]
 irdma_modify_qp_roce+0x13df/0x1630 [irdma]
 ib_security_modify_qp+0x34a/0x640 [ib_core]
 _ib_modify_qp+0x16b/0x620 [ib_core]
 ib_modify_qp_with_udata+0x3c/0x50 [ib_core]
 modify_qp+0x6e1/0x920 [ib_uverbs]
 ib_uverbs_ex_modify_qp+0xa6/0xf0 [ib_uverbs]
 ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0x16c/0x200 [ib_uverbs]
 ib_uverbs_run_method+0x342/0x380 [ib_uverbs]
 ib_uverbs_cmd_verbs+0x365/0x440 [ib_uverbs]
 ib_uverbs_ioctl+0x111/0x190 [ib_uverbs]
 __x64_sys_ioctl+0xc3/0x100
 do_syscall_64+0x3f/0x90
 entry_SYSCALL_64_after_hwframe+0x72/0xdc

value changed: 0x00 -> 0x01

Fixes: f0842bb3d388 ("RDMA/irdma: Fix data race on CQP request done")
Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/hw.c    | 2 +-
 drivers/infiniband/hw/irdma/main.h  | 2 +-
 drivers/infiniband/hw/irdma/utils.c | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
index 6e0466ce83d1..3ba4809bc1ef 100644
--- a/drivers/infiniband/hw/irdma/hw.c
+++ b/drivers/infiniband/hw/irdma/hw.c
@@ -235,7 +235,7 @@ static void irdma_complete_cqp_request(struct irdma_cqp *cqp,
 				       struct irdma_cqp_request *cqp_request)
 {
 	if (cqp_request->waiting) {
-		WRITE_ONCE(cqp_request->request_done, true);
+		atomic_set(&cqp_request->request_done, true);
 		wake_up(&cqp_request->waitq);
 	} else if (cqp_request->callback_fcn) {
 		cqp_request->callback_fcn(cqp_request);
diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h
index 3d49bd57bae7..e22160e2ba33 100644
--- a/drivers/infiniband/hw/irdma/main.h
+++ b/drivers/infiniband/hw/irdma/main.h
@@ -167,7 +167,7 @@ struct irdma_cqp_request {
 	void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
 	void *param;
 	struct irdma_cqp_compl_info compl_info;
-	bool request_done; /* READ/WRITE_ONCE macros operate on it */
+	atomic_t request_done;
 	bool waiting:1;
 	bool dynamic:1;
 	bool pending:1;
diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
index ab8c5284d4be..f9c99c216a2c 100644
--- a/drivers/infiniband/hw/irdma/utils.c
+++ b/drivers/infiniband/hw/irdma/utils.c
@@ -480,7 +480,7 @@ void irdma_free_cqp_request(struct irdma_cqp *cqp,
 	if (cqp_request->dynamic) {
 		kfree(cqp_request);
 	} else {
-		WRITE_ONCE(cqp_request->request_done, false);
+		atomic_set(&cqp_request->request_done, false);
 		cqp_request->callback_fcn = NULL;
 		cqp_request->waiting = false;
 		cqp_request->pending = false;
@@ -515,7 +515,7 @@ irdma_free_pending_cqp_request(struct irdma_cqp *cqp,
 {
 	if (cqp_request->waiting) {
 		cqp_request->compl_info.error = true;
-		WRITE_ONCE(cqp_request->request_done, true);
+		atomic_set(&cqp_request->request_done, true);
 		wake_up(&cqp_request->waitq);
 	}
 	wait_event_timeout(cqp->remove_wq,
@@ -610,7 +610,7 @@ static int irdma_wait_event(struct irdma_pci_f *rf,
 	do {
 		irdma_cqp_ce_handler(rf, &rf->ccq.sc_cq);
 		if (wait_event_timeout(cqp_request->waitq,
-				       READ_ONCE(cqp_request->request_done),
+				       atomic_read(&cqp_request->request_done),
 				       msecs_to_jiffies(CQP_COMPL_WAIT_TIME_MS)))
 			break;
 
-- 
2.31.1

[for-next 03/12] RDMA/irdma: Change ah_valid type to atomic

[Tatyana Nikolova]

From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>

Converts ah_valid flag to atomic to protect it against data-race.

[  809.561229] BUG: KCSAN: data-race in irdma_create_hw_ah [irdma] / irdma_gsi_ud_qp_ah_cb [irdma]

[  809.565182] write to 0xffff8d911c8eee73 of 1 bytes by task 38949 on cpu 10:
[  809.567113]  irdma_gsi_ud_qp_ah_cb+0x41/0x60 [irdma]
[  809.567343]  irdma_complete_cqp_request+0x44/0xb0 [irdma]
[  809.567572]  irdma_cqp_ce_handler+0x242/0x290 [irdma]
[  809.567810]  irdma_wait_event+0xf2/0x3c0 [irdma]

[  809.570951] read to 0xffff8d911c8eee73 of 1 bytes by task 38952 on cpu 7:
[  809.573037]  irdma_create_hw_ah+0x1e7/0x370 [irdma]
[  809.573265]  irdma_create_ah+0x61/0x70 [irdma]
[  809.573491]  _rdma_create_ah+0x262/0x290 [ib_core]
[  809.573831]  rdma_create_ah+0xa3/0x140 [ib_core]

[  809.576933] value changed: 0x06 -> 0x07
[  809.581184] Reported by Kernel Concurrency Sanitizer

Fixes: dd90451fac23 ("RDMA/irdma: Add RoCEv2 UD OP support")
Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/cm.c    |  2 +-
 drivers/infiniband/hw/irdma/puda.c  |  2 +-
 drivers/infiniband/hw/irdma/uda.h   |  2 +-
 drivers/infiniband/hw/irdma/utils.c | 16 +++++++++-------
 drivers/infiniband/hw/irdma/verbs.c |  7 ++++---
 5 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
index 3d084d4ff577..947fd93f5fb0 100644
--- a/drivers/infiniband/hw/irdma/cm.c
+++ b/drivers/infiniband/hw/irdma/cm.c
@@ -313,7 +313,7 @@ static struct irdma_puda_buf *irdma_form_ah_cm_frame(struct irdma_cm_node *cm_no
 	u32 pd_len = 0;
 	u32 hdr_len = 0;
 
-	if (!cm_node->ah || !cm_node->ah->ah_info.ah_valid) {
+	if (!cm_node->ah || !atomic_read(&cm_node->ah->ah_info.ah_valid)) {
 		ibdev_dbg(&cm_node->iwdev->ibdev, "CM: AH invalid\n");
 		return NULL;
 	}
diff --git a/drivers/infiniband/hw/irdma/puda.c b/drivers/infiniband/hw/irdma/puda.c
index 4f1a8c97faf1..3410be38f602 100644
--- a/drivers/infiniband/hw/irdma/puda.c
+++ b/drivers/infiniband/hw/irdma/puda.c
@@ -1652,7 +1652,7 @@ static void irdma_ieq_handle_exception(struct irdma_puda_rsrc *ieq,
 	}
 	if (hw_rev == IRDMA_GEN_1)
 		irdma_ieq_process_fpdus(qp, ieq);
-	else if (pfpdu->ah && pfpdu->ah->ah_info.ah_valid)
+	else if (pfpdu->ah && atomic_read(&pfpdu->ah->ah_info.ah_valid))
 		irdma_ieq_process_fpdus(qp, ieq);
 exit:
 	spin_unlock_irqrestore(&pfpdu->lock, flags);
diff --git a/drivers/infiniband/hw/irdma/uda.h b/drivers/infiniband/hw/irdma/uda.h
index 27b8701cf21b..773c33ce62a2 100644
--- a/drivers/infiniband/hw/irdma/uda.h
+++ b/drivers/infiniband/hw/irdma/uda.h
@@ -22,7 +22,7 @@ struct irdma_ah_info {
 	u8 tc_tos;
 	u8 hop_ttl;
 	u8 mac_addr[ETH_ALEN];
-	bool ah_valid:1;
+	atomic_t ah_valid;
 	bool ipv4_valid:1;
 	bool do_lpbk:1;
 };
diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
index f9c99c216a2c..be2dc92d008f 100644
--- a/drivers/infiniband/hw/irdma/utils.c
+++ b/drivers/infiniband/hw/irdma/utils.c
@@ -1967,7 +1967,8 @@ int irdma_ah_cqp_op(struct irdma_pci_f *rf, struct irdma_sc_ah *sc_ah, u8 cmd,
 		return -ENOMEM;
 
 	if (wait)
-		sc_ah->ah_info.ah_valid = (cmd == IRDMA_OP_AH_CREATE);
+		atomic_set(&sc_ah->ah_info.ah_valid,
+			   (cmd == IRDMA_OP_AH_CREATE));
 
 	return 0;
 }
@@ -1984,10 +1985,10 @@ static void irdma_ieq_ah_cb(struct irdma_cqp_request *cqp_request)
 
 	spin_lock_irqsave(&qp->pfpdu.lock, flags);
 	if (!cqp_request->compl_info.op_ret_val) {
-		sc_ah->ah_info.ah_valid = true;
+		atomic_set(&sc_ah->ah_info.ah_valid, true);
 		irdma_ieq_process_fpdus(qp, qp->vsi->ieq);
 	} else {
-		sc_ah->ah_info.ah_valid = false;
+		atomic_set(&sc_ah->ah_info.ah_valid, false);
 		irdma_ieq_cleanup_qp(qp->vsi->ieq, qp);
 	}
 	spin_unlock_irqrestore(&qp->pfpdu.lock, flags);
@@ -2002,7 +2003,8 @@ static void irdma_ilq_ah_cb(struct irdma_cqp_request *cqp_request)
 	struct irdma_cm_node *cm_node = cqp_request->param;
 	struct irdma_sc_ah *sc_ah = cm_node->ah;
 
-	sc_ah->ah_info.ah_valid = !cqp_request->compl_info.op_ret_val;
+	atomic_set(&sc_ah->ah_info.ah_valid,
+		   !cqp_request->compl_info.op_ret_val);
 	irdma_add_conn_est_qh(cm_node);
 }
 
@@ -2069,7 +2071,7 @@ void irdma_puda_free_ah(struct irdma_sc_dev *dev, struct irdma_sc_ah *ah)
 	if (!ah)
 		return;
 
-	if (ah->ah_info.ah_valid) {
+	if (atomic_read(&ah->ah_info.ah_valid)) {
 		irdma_ah_cqp_op(rf, ah, IRDMA_OP_AH_DESTROY, false, NULL, NULL);
 		irdma_free_rsrc(rf, rf->allocated_ahs, ah->ah_info.ah_idx);
 	}
@@ -2086,9 +2088,9 @@ void irdma_gsi_ud_qp_ah_cb(struct irdma_cqp_request *cqp_request)
 	struct irdma_sc_ah *sc_ah = cqp_request->param;
 
 	if (!cqp_request->compl_info.op_ret_val)
-		sc_ah->ah_info.ah_valid = true;
+		atomic_set(&sc_ah->ah_info.ah_valid, true);
 	else
-		sc_ah->ah_info.ah_valid = false;
+		atomic_set(&sc_ah->ah_info.ah_valid, false);
 }
 
 /**
diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 38bc0e656ecf..9cfcdf7b053e 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -5116,8 +5116,8 @@ static int irdma_create_hw_ah(struct irdma_device *iwdev, struct irdma_ah *ah, b
 
 		if (poll_timeout_us_atomic(irdma_cqp_ce_handler(rf,
 								&rf->ccq.sc_cq),
-					   ah->sc_ah.ah_info.ah_valid, 1,
-					   tmout_ms * USEC_PER_MSEC, false)) {
+					   atomic_read(&ah->sc_ah.ah_info.ah_valid),
+					   1, tmout_ms * USEC_PER_MSEC, false)) {
 			ibdev_dbg(&iwdev->ibdev,
 				  "VERBS: CQP create AH timed out");
 			err = -ETIMEDOUT;
@@ -5236,7 +5236,8 @@ static bool irdma_ah_exists(struct irdma_device *iwdev,
 	hash_for_each_possible(iwdev->rf->ah_hash_tbl, ah, list, key) {
 		/* Set ah_valid and ah_id the same so memcmp can work */
 		new_ah->sc_ah.ah_info.ah_idx = ah->sc_ah.ah_info.ah_idx;
-		new_ah->sc_ah.ah_info.ah_valid = ah->sc_ah.ah_info.ah_valid;
+		atomic_set(&new_ah->sc_ah.ah_info.ah_valid,
+			   atomic_read(&ah->sc_ah.ah_info.ah_valid));
 		if (!memcmp(&ah->sc_ah.ah_info, &new_ah->sc_ah.ah_info,
 			    sizeof(ah->sc_ah.ah_info))) {
 			refcount_inc(&ah->refcnt);
-- 
2.31.1

[for-next 04/12] RDMA/irdma: Update ibqp state to error if QP is already in error state

[Tatyana Nikolova]

In irdma_modify_qp() update ibqp state to error if the irdma QP is already
in error state, otherwise the ibqp state which is visible to the consumer
app remains stale.

Fixes: b48c24c2d710 ("RDMA/irdma: Implement device supported verb APIs")
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/verbs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 9cfcdf7b053e..8de09cda7884 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -1540,6 +1540,7 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 		case IB_QPS_ERR:
 		case IB_QPS_RESET:
 			if (iwqp->iwarp_state == IRDMA_QP_STATE_ERROR) {
+				iwqp->ibqp_state = attr->qp_state;
 				spin_unlock_irqrestore(&iwqp->lock, flags);
 				if (udata && udata->inlen) {
 					if (ib_copy_from_udata(&ureq, udata,
@@ -1745,6 +1746,7 @@ int irdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask,
 		case IB_QPS_ERR:
 		case IB_QPS_RESET:
 			if (iwqp->iwarp_state == IRDMA_QP_STATE_ERROR) {
+				iwqp->ibqp_state = attr->qp_state;
 				spin_unlock_irqrestore(&iwqp->lock, flags);
 				if (udata && udata->inlen) {
 					if (ib_copy_from_udata(&ureq, udata,
-- 
2.31.1

[for-next 05/12] RDMA/irdma: Remove a NOP wait_event() in irdma_modify_qp_roce()

[Tatyana Nikolova]

Remove a NOP wait_event() in irdma_modify_qp_roce() which is relevant
for iWARP and likely a copy and paste artifact for RoCEv2. The wait event
is for sending a reset on a TCP connection, after the reset has been
requested in irdma_modify_qp(), which occurs only in iWarp mode.

Fixes: b48c24c2d710 ("RDMA/irdma: Implement device supported verb APIs")
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/verbs.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 8de09cda7884..41c75f79500d 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -1462,8 +1462,6 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 				ctx_info->remote_atomics_en = true;
 	}
 
-	wait_event(iwqp->mod_qp_waitq, !atomic_read(&iwqp->hw_mod_qp_pend));
-
 	ibdev_dbg(&iwdev->ibdev,
 		  "VERBS: caller: %pS qp_id=%d to_ibqpstate=%d ibqpstate=%d irdma_qpstate=%d attr_mask=0x%x\n",
 		  __builtin_return_address(0), ibqp->qp_num, attr->qp_state,
-- 
2.31.1

[for-next 06/12] RDMA/irdma: Clean up unnecessary dereference of event->cm_node

[Tatyana Nikolova]

From: Ivan Barrera <ivan.d.barrera@intel.com>

The cm_node is available and the usage of cm_node and event->cm_node
seems arbitrary. Clean up unnecessary dereference of event->cm_node.

Fixes: 146b9756f14c ("RDMA/irdma: Add connection manager")
Signed-off-by: Ivan Barrera <ivan.d.barrera@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/cm.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
index 947fd93f5fb0..6557e1299a47 100644
--- a/drivers/infiniband/hw/irdma/cm.c
+++ b/drivers/infiniband/hw/irdma/cm.c
@@ -4239,21 +4239,21 @@ static void irdma_cm_event_handler(struct work_struct *work)
 		irdma_cm_event_reset(event);
 		break;
 	case IRDMA_CM_EVENT_CONNECTED:
-		if (!event->cm_node->cm_id ||
-		    event->cm_node->state != IRDMA_CM_STATE_OFFLOADED)
+		if (!cm_node->cm_id ||
+		    cm_node->state != IRDMA_CM_STATE_OFFLOADED)
 			break;
 		irdma_cm_event_connected(event);
 		break;
 	case IRDMA_CM_EVENT_MPA_REJECT:
-		if (!event->cm_node->cm_id ||
+		if (!cm_node->cm_id ||
 		    cm_node->state == IRDMA_CM_STATE_OFFLOADED)
 			break;
 		irdma_send_cm_event(cm_node, cm_node->cm_id,
 				    IW_CM_EVENT_CONNECT_REPLY, -ECONNREFUSED);
 		break;
 	case IRDMA_CM_EVENT_ABORTED:
-		if (!event->cm_node->cm_id ||
-		    event->cm_node->state == IRDMA_CM_STATE_OFFLOADED)
+		if (!cm_node->cm_id ||
+		    cm_node->state == IRDMA_CM_STATE_OFFLOADED)
 			break;
 		irdma_event_connect_error(event);
 		break;
@@ -4263,7 +4263,7 @@ static void irdma_cm_event_handler(struct work_struct *work)
 		break;
 	}
 
-	irdma_rem_ref_cm_node(event->cm_node);
+	irdma_rem_ref_cm_node(cm_node);
 	kfree(event);
 }
 
-- 
2.31.1

[for-next 07/12] RDMA/irdma: Remove reset check from irdma_modify_qp_to_err()

[Tatyana Nikolova]

During reset, irdma_modify_qp() to error should be called to disconnect
the QP. Without this fix, if not preceded by irdma_modify_qp() to error, the
API call irdma_destroy_qp() gets stuck waiting for the QP refcount to go
to zero, because the cm_node associated with this QP isn't disconnected.

Fixes 915cc7ac0f8e ("RDMA/irdma: Add miscellaneous utility definitions")
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/utils.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
index be2dc92d008f..18d0f29100f3 100644
--- a/drivers/infiniband/hw/irdma/utils.c
+++ b/drivers/infiniband/hw/irdma/utils.c
@@ -2324,8 +2324,6 @@ void irdma_modify_qp_to_err(struct irdma_sc_qp *sc_qp)
 	struct irdma_qp *qp = sc_qp->qp_uk.back_qp;
 	struct ib_qp_attr attr;
 
-	if (qp->iwdev->rf->reset)
-		return;
 	attr.qp_state = IB_QPS_ERR;
 
 	if (rdma_protocol_roce(qp->ibqp.device, 1))
-- 
2.31.1

[for-next 08/12] RDMA/irdma: Fix deadlock during netdev reset with active connections

[Tatyana Nikolova]

From: Anil Samal <anil.samal@intel.com>

Resolve deadlock that occurs when user executes netdev reset while RDMA
applications (e.g., rping) are active. The netdev reset causes ice
driver to remove irdma auxiliary driver, triggering device_delete and
subsequent client removal. During client removal, uverbs_client waits
for QP reference count to reach zero while cma_client holds the final
reference, creating circular dependency and indefinite wait in iWARP
mode. Skip QP reference count wait during device reset to prevent
deadlock.

Fixes: c8f304d75f6c ("RDMA/irdma: Prevent QP use after free")
Signed-off-by: Anil Samal <anil.samal@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/verbs.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 41c75f79500d..b6da26aa5cd7 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -558,7 +558,8 @@ static int irdma_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata)
 	}
 
 	irdma_qp_rem_ref(&iwqp->ibqp);
-	wait_for_completion(&iwqp->free_qp);
+	if (!iwdev->rf->reset)
+		wait_for_completion(&iwqp->free_qp);
 	irdma_free_lsmm_rsrc(iwqp);
 	irdma_cqp_qp_destroy_cmd(&iwdev->rf->sc_dev, &iwqp->sc_qp);
 
-- 
2.31.1

[for-next 09/12] RDMA/irdma: Return EINVAL for invalid arp index error

[Tatyana Nikolova]

When rdma_connect() fails due to an invalid arp index, user space rdma core
reports ENOMEM which is confusing. Modify irdma_make_cm_node() to return the
correct error code.

Fixes: 146b9756f14c ("RDMA/irdma: Add connection manager")
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/cm.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
index 6557e1299a47..c952b735e187 100644
--- a/drivers/infiniband/hw/irdma/cm.c
+++ b/drivers/infiniband/hw/irdma/cm.c
@@ -2241,11 +2241,12 @@ irdma_make_cm_node(struct irdma_cm_core *cm_core, struct irdma_device *iwdev,
 	int oldarpindex;
 	int arpindex;
 	struct net_device *netdev = iwdev->netdev;
+	int ret;
 
 	/* create an hte and cm_node for this instance */
 	cm_node = kzalloc_obj(*cm_node, GFP_ATOMIC);
 	if (!cm_node)
-		return NULL;
+		return ERR_PTR(-ENOMEM);
 
 	/* set our node specific transport info */
 	cm_node->ipv4 = cm_info->ipv4;
@@ -2348,8 +2349,10 @@ irdma_make_cm_node(struct irdma_cm_core *cm_core, struct irdma_device *iwdev,
 			arpindex = -EINVAL;
 	}
 
-	if (arpindex < 0)
+	if (arpindex < 0) {
+		ret = -EINVAL;
 		goto err;
+	}
 
 	ether_addr_copy(cm_node->rem_mac,
 			iwdev->rf->arp_table[arpindex].mac_addr);
@@ -2360,7 +2363,7 @@ irdma_make_cm_node(struct irdma_cm_core *cm_core, struct irdma_device *iwdev,
 err:
 	kfree(cm_node);
 
-	return NULL;
+	return ERR_PTR(ret);
 }
 
 static void irdma_destroy_connection(struct irdma_cm_node *cm_node)
@@ -3021,8 +3024,8 @@ static int irdma_create_cm_node(struct irdma_cm_core *cm_core,
 
 	/* create a CM connection node */
 	cm_node = irdma_make_cm_node(cm_core, iwdev, cm_info, NULL);
-	if (!cm_node)
-		return -ENOMEM;
+	if (IS_ERR(cm_node))
+		return PTR_ERR(cm_node);
 
 	/* set our node side to client (active) side */
 	cm_node->tcp_cntxt.client = 1;
@@ -3219,9 +3222,9 @@ void irdma_receive_ilq(struct irdma_sc_vsi *vsi, struct irdma_puda_buf *rbuf)
 		cm_info.cm_id = listener->cm_id;
 		cm_node = irdma_make_cm_node(cm_core, iwdev, &cm_info,
 					     listener);
-		if (!cm_node) {
+		if (IS_ERR(cm_node)) {
 			ibdev_dbg(&cm_core->iwdev->ibdev,
-				  "CM: allocate node failed\n");
+				  "CM: allocate node failed ret=%ld\n", PTR_ERR(cm_node));
 			refcount_dec(&listener->refcnt);
 			return;
 		}
-- 
2.31.1

[for-next 10/12] RDMA/irdma: Harden depth calculation functions

[Tatyana Nikolova]

From: Shiraz Saleem <shiraz.saleem@intel.com>

An issue was exposed where OS can pass in U32_MAX for SQ/RQ/SRQ size.
This can cause integer overflow and truncation of SQ/RQ/SRQ depth
returning a success when it should have failed.

Harden the functions to do all depth calculations and boundary
checking in u64 sizes.

Fixes: 563e1feb5f6e ("RDMA/irdma: Add SRQ support")
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/uk.c | 39 ++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 17 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/uk.c b/drivers/infiniband/hw/irdma/uk.c
index ac3721a5747a..4718acf6c6fd 100644
--- a/drivers/infiniband/hw/irdma/uk.c
+++ b/drivers/infiniband/hw/irdma/uk.c
@@ -1438,7 +1438,7 @@ int irdma_uk_cq_poll_cmpl(struct irdma_cq_uk *cq,
  * irdma_round_up_wq - return round up qp wq depth
  * @wqdepth: wq depth in quanta to round up
  */
-static int irdma_round_up_wq(u32 wqdepth)
+static u64 irdma_round_up_wq(u64 wqdepth)
 {
 	int scount = 1;
 
@@ -1491,15 +1491,16 @@ void irdma_get_wqe_shift(struct irdma_uk_attrs *uk_attrs, u32 sge,
 int irdma_get_sqdepth(struct irdma_uk_attrs *uk_attrs, u32 sq_size, u8 shift,
 		      u32 *sqdepth)
 {
-	u32 min_size = (u32)uk_attrs->min_hw_wq_size << shift;
+	u32 min_hw_quanta = (u32)uk_attrs->min_hw_wq_size << shift;
+	u64 hw_quanta =
+		irdma_round_up_wq(((u64)sq_size << shift) + IRDMA_SQ_RSVD);
 
-	*sqdepth = irdma_round_up_wq((sq_size << shift) + IRDMA_SQ_RSVD);
-
-	if (*sqdepth < min_size)
-		*sqdepth = min_size;
-	else if (*sqdepth > uk_attrs->max_hw_wq_quanta)
+	if (hw_quanta < min_hw_quanta)
+		hw_quanta = min_hw_quanta;
+	else if (hw_quanta > uk_attrs->max_hw_wq_quanta)
 		return -EINVAL;
 
+	*sqdepth = hw_quanta;
 	return 0;
 }
 
@@ -1513,15 +1514,16 @@ int irdma_get_sqdepth(struct irdma_uk_attrs *uk_attrs, u32 sq_size, u8 shift,
 int irdma_get_rqdepth(struct irdma_uk_attrs *uk_attrs, u32 rq_size, u8 shift,
 		      u32 *rqdepth)
 {
-	u32 min_size = (u32)uk_attrs->min_hw_wq_size << shift;
-
-	*rqdepth = irdma_round_up_wq((rq_size << shift) + IRDMA_RQ_RSVD);
+	u32 min_hw_quanta = (u32)uk_attrs->min_hw_wq_size << shift;
+	u64 hw_quanta =
+		irdma_round_up_wq(((u64)rq_size << shift) + IRDMA_RQ_RSVD);
 
-	if (*rqdepth < min_size)
-		*rqdepth = min_size;
-	else if (*rqdepth > uk_attrs->max_hw_rq_quanta)
+	if (hw_quanta < min_hw_quanta)
+		hw_quanta = min_hw_quanta;
+	else if (hw_quanta > uk_attrs->max_hw_rq_quanta)
 		return -EINVAL;
 
+	*rqdepth = hw_quanta;
 	return 0;
 }
 
@@ -1535,13 +1537,16 @@ int irdma_get_rqdepth(struct irdma_uk_attrs *uk_attrs, u32 rq_size, u8 shift,
 int irdma_get_srqdepth(struct irdma_uk_attrs *uk_attrs, u32 srq_size, u8 shift,
 		       u32 *srqdepth)
 {
-	*srqdepth = irdma_round_up_wq((srq_size << shift) + IRDMA_RQ_RSVD);
+	u32 min_hw_quanta = (u32)uk_attrs->min_hw_wq_size << shift;
+	u64 hw_quanta =
+		irdma_round_up_wq(((u64)srq_size << shift) + IRDMA_RQ_RSVD);
 
-	if (*srqdepth < ((u32)uk_attrs->min_hw_wq_size << shift))
-		*srqdepth = uk_attrs->min_hw_wq_size << shift;
-	else if (*srqdepth > uk_attrs->max_hw_srq_quanta)
+	if (hw_quanta < min_hw_quanta)
+		hw_quanta = min_hw_quanta;
+	else if (hw_quanta > uk_attrs->max_hw_srq_quanta)
 		return -EINVAL;
 
+	*srqdepth = hw_quanta;
 	return 0;
 }
 
-- 
2.31.1

[for-next 11/12] RDMA/irdma: Provide scratch buffers to firmware for internal use

[Tatyana Nikolova]

From: Jay Bhat <jay.bhat@intel.com>

For GEN3 and higher, FW requires scratch buffers for bookkeeping
during cleanup, specifically during QP and MR destroy ops.

Signed-off-by: Jay Bhat <jay.bhat@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/ctrl.c | 43 +++++++++++++++++++++++++++++-
 drivers/infiniband/hw/irdma/defs.h |  4 +++
 drivers/infiniband/hw/irdma/hw.c   | 11 ++++++++
 drivers/infiniband/hw/irdma/type.h |  2 ++
 drivers/infiniband/hw/irdma/user.h |  4 +--
 5 files changed, 61 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/ctrl.c b/drivers/infiniband/hw/irdma/ctrl.c
index 45c7433c96f3..13820f1a48a4 100644
--- a/drivers/infiniband/hw/irdma/ctrl.c
+++ b/drivers/infiniband/hw/irdma/ctrl.c
@@ -3570,6 +3570,41 @@ static int irdma_sc_parse_fpm_query_buf(struct irdma_sc_dev *dev, __le64 *buf,
 		hmc_fpm_misc->loc_mem_pages = (u32)FIELD_GET(IRDMA_QUERY_FPM_LOC_MEM_PAGES, temp);
 		if (!hmc_fpm_misc->loc_mem_pages)
 			return -EINVAL;
+
+		get_64bit_val(buf, 184, &temp);
+		if (temp) {
+			hmc_fpm_misc->fw_scratch_buf0.size = temp;
+			hmc_fpm_misc->fw_scratch_buf0.va =
+				dma_alloc_coherent(dev->hw->device,
+						   hmc_fpm_misc->fw_scratch_buf0.size,
+						   &hmc_fpm_misc->fw_scratch_buf0.pa,
+						   GFP_KERNEL);
+
+			if (!hmc_fpm_misc->fw_scratch_buf0.va) {
+				hmc_fpm_misc->fw_scratch_buf0.size = 0;
+				return -ENOMEM;
+			}
+		}
+		get_64bit_val(buf, 192, &temp);
+		if (temp) {
+			hmc_fpm_misc->fw_scratch_buf1.size = temp;
+			hmc_fpm_misc->fw_scratch_buf1.va =
+				dma_alloc_coherent(dev->hw->device,
+						   hmc_fpm_misc->fw_scratch_buf1.size,
+						   &hmc_fpm_misc->fw_scratch_buf1.pa,
+						   GFP_KERNEL);
+
+			if (!hmc_fpm_misc->fw_scratch_buf1.va) {
+				hmc_fpm_misc->fw_scratch_buf1.size = 0;
+				dma_free_coherent(dev->hw->device,
+						  hmc_fpm_misc->fw_scratch_buf0.size,
+						  hmc_fpm_misc->fw_scratch_buf0.va,
+						  hmc_fpm_misc->fw_scratch_buf0.pa);
+				hmc_fpm_misc->fw_scratch_buf0.va = NULL;
+				hmc_fpm_misc->fw_scratch_buf0.size = 0;
+				return -ENOMEM;
+			}
+		}
 	}
 
 	return 0;
@@ -4187,6 +4222,8 @@ static int irdma_sc_commit_fpm_val(struct irdma_sc_cqp *cqp, u64 scratch,
 
 	hdr = FIELD_PREP(IRDMA_CQPSQ_BUFSIZE, IRDMA_COMMIT_FPM_BUF_SIZE) |
 	      FIELD_PREP(IRDMA_CQPSQ_OPCODE, IRDMA_CQP_OP_COMMIT_FPM_VAL) |
+	      FIELD_PREP(IRDMA_CQPSQ_CFPM_FW_SCRATCH_BUF_PRESENT,
+			 cqp->dev->hmc_fpm_misc.fw_scratch_buf0.va != NULL) |
 	      FIELD_PREP(IRDMA_CQPSQ_WQEVALID, cqp->polarity);
 
 	dma_wmb(); /* make sure WQE is written before valid bit is set */
@@ -5034,7 +5071,9 @@ static void irdma_set_loc_mem(__le64 *buf)
 
 	for (offset = 0; offset < IRDMA_COMMIT_FPM_BUF_SIZE;
 	     offset += sizeof(__le64)) {
-		if (offset == IRDMA_PBLE_COMMIT_OFFSET)
+		if (offset == IRDMA_PBLE_COMMIT_OFFSET ||
+		    offset == IRDMA_SCRATCH_BUF0_COMMIT_OFFSET ||
+		    offset == IRDMA_SCRATCH_BUF1_COMMIT_OFFSET)
 			continue;
 		get_64bit_val(buf, offset, &temp);
 		if (temp)
@@ -5090,6 +5129,8 @@ static int irdma_sc_cfg_iw_fpm(struct irdma_sc_dev *dev, u8 hmc_fn_id)
 		      (u64)obj_info[IRDMA_HMC_IW_OOISC].cnt);
 	set_64bit_val(buf, 168,
 		      (u64)obj_info[IRDMA_HMC_IW_OOISCFFL].cnt);
+	set_64bit_val(buf, 192, dev->hmc_fpm_misc.fw_scratch_buf0.pa);
+	set_64bit_val(buf, 200, dev->hmc_fpm_misc.fw_scratch_buf1.pa);
 	if (dev->hw_attrs.uk_attrs.hw_rev >= IRDMA_GEN_3 &&
 	    dev->hmc_fpm_misc.loc_mem_pages)
 		irdma_set_loc_mem(buf);
diff --git a/drivers/infiniband/hw/irdma/defs.h b/drivers/infiniband/hw/irdma/defs.h
index 983b22d7ae23..d6a3152959dd 100644
--- a/drivers/infiniband/hw/irdma/defs.h
+++ b/drivers/infiniband/hw/irdma/defs.h
@@ -133,6 +133,8 @@ enum irdma_protocol_used {
 #define MAX_MR_PER_SD			0x8000
 #define MAX_MR_SD_PER_FCN		0x80
 #define IRDMA_PBLE_COMMIT_OFFSET	112
+#define IRDMA_SCRATCH_BUF0_COMMIT_OFFSET 192
+#define IRDMA_SCRATCH_BUF1_COMMIT_OFFSET 200
 #define IRDMA_MAX_QUANTA_PER_WR	8
 
 #define IRDMA_QP_SW_MAX_WQ_QUANTA	32768
@@ -658,6 +660,8 @@ enum irdma_cqp_op_type {
 #define IRDMA_COMMIT_FPM_QPCNT GENMASK_ULL(20, 0)
 #define IRDMA_COMMIT_FPM_BASE_S 32
 #define IRDMA_CQPSQ_CFPM_HMCFNID GENMASK_ULL(15, 0)
+#define IRDMA_CQPSQ_CFPM_FW_SCRATCH_BUF_PRESENT_S 38
+#define IRDMA_CQPSQ_CFPM_FW_SCRATCH_BUF_PRESENT BIT_ULL(38)
 
 #define IRDMA_CQPSQ_FWQE_AECODE GENMASK_ULL(15, 0)
 #define IRDMA_CQPSQ_FWQE_AESOURCE GENMASK_ULL(19, 16)
diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
index 3ba4809bc1ef..10eb21213cf9 100644
--- a/drivers/infiniband/hw/irdma/hw.c
+++ b/drivers/infiniband/hw/irdma/hw.c
@@ -1693,6 +1693,8 @@ static int irdma_hmc_setup(struct irdma_pci_f *rf)
 static void irdma_del_init_mem(struct irdma_pci_f *rf)
 {
 	struct irdma_sc_dev *dev = &rf->sc_dev;
+	struct irdma_dma_mem *fw_scratch_buf0;
+	struct irdma_dma_mem *fw_scratch_buf1;
 
 	if (!rf->sc_dev.privileged)
 		irdma_vchnl_req_put_hmc_fcn(&rf->sc_dev);
@@ -1713,6 +1715,15 @@ static void irdma_del_init_mem(struct irdma_pci_f *rf)
 	rf->iw_msixtbl = NULL;
 	kfree(rf->hmc_info_mem);
 	rf->hmc_info_mem = NULL;
+
+	fw_scratch_buf0 = &dev->hmc_fpm_misc.fw_scratch_buf0;
+	fw_scratch_buf1 = &dev->hmc_fpm_misc.fw_scratch_buf1;
+	if (fw_scratch_buf0->va)
+		dma_free_coherent(dev->hw->device, fw_scratch_buf0->size,
+				  fw_scratch_buf0->va, fw_scratch_buf0->pa);
+	if (fw_scratch_buf1->va)
+		dma_free_coherent(dev->hw->device, fw_scratch_buf1->size,
+				  fw_scratch_buf1->va, fw_scratch_buf1->pa);
 }
 
 /**
diff --git a/drivers/infiniband/hw/irdma/type.h b/drivers/infiniband/hw/irdma/type.h
index da8c54d1f035..5557d9338796 100644
--- a/drivers/infiniband/hw/irdma/type.h
+++ b/drivers/infiniband/hw/irdma/type.h
@@ -622,6 +622,8 @@ struct irdma_hmc_fpm_misc {
 	u32 timer_bucket;
 	u32 rrf_block_size;
 	u32 ooiscf_block_size;
+	struct irdma_dma_mem fw_scratch_buf0;
+	struct irdma_dma_mem fw_scratch_buf1;
 };
 
 #define IRDMA_VCHNL_MAX_MSG_SIZE 512
diff --git a/drivers/infiniband/hw/irdma/user.h b/drivers/infiniband/hw/irdma/user.h
index 9eb7fd0b1cbf..008af1acc928 100644
--- a/drivers/infiniband/hw/irdma/user.h
+++ b/drivers/infiniband/hw/irdma/user.h
@@ -159,8 +159,8 @@ enum irdma_device_caps_const {
 	IRDMA_CEQE_SIZE =			1,
 	IRDMA_CQP_CTX_SIZE =			8,
 	IRDMA_SHADOW_AREA_SIZE =		8,
-	IRDMA_QUERY_FPM_BUF_SIZE =		192,
-	IRDMA_COMMIT_FPM_BUF_SIZE =		192,
+	IRDMA_QUERY_FPM_BUF_SIZE =		200,
+	IRDMA_COMMIT_FPM_BUF_SIZE =		208,
 	IRDMA_GATHER_STATS_BUF_SIZE =		1024,
 	IRDMA_MIN_IW_QP_ID =			0,
 	IRDMA_MAX_IW_QP_ID =			262143,
-- 
2.31.1

[for-next 12/12] RDMA/irdma: Add support for GEN4 hardware

[Tatyana Nikolova]

From: Jacob Moroni <jmoroni@google.com>

GEN4 hardware is similar to GEN3 and requires only a few special cases.

Signed-off-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 drivers/infiniband/hw/irdma/ctrl.c       |  1 +
 drivers/infiniband/hw/irdma/hw.c         | 14 ++++++++++----
 drivers/infiniband/hw/irdma/ig3rdma_hw.c |  1 -
 drivers/infiniband/hw/irdma/irdma.h      |  1 +
 4 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/hw/irdma/ctrl.c b/drivers/infiniband/hw/irdma/ctrl.c
index 13820f1a48a4..335ae3c82e17 100644
--- a/drivers/infiniband/hw/irdma/ctrl.c
+++ b/drivers/infiniband/hw/irdma/ctrl.c
@@ -6465,6 +6465,7 @@ static inline void irdma_sc_init_hw(struct irdma_sc_dev *dev)
 		icrdma_init_hw(dev);
 		break;
 	case IRDMA_GEN_3:
+	case IRDMA_GEN_4:
 		ig3rdma_init_hw(dev);
 		break;
 	}
diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
index 10eb21213cf9..c587872a430d 100644
--- a/drivers/infiniband/hw/irdma/hw.c
+++ b/drivers/infiniband/hw/irdma/hw.c
@@ -1082,6 +1082,7 @@ static int irdma_create_cqp(struct irdma_pci_f *rf)
 		cqp_init_info.hw_maj_ver = IRDMA_CQPHC_HW_MAJVER_GEN_2;
 		break;
 	case IRDMA_GEN_3:
+	case IRDMA_GEN_4:
 		cqp_init_info.hw_maj_ver = IRDMA_CQPHC_HW_MAJVER_GEN_3;
 		cqp_init_info.ts_override = 1;
 		break;
@@ -1508,7 +1509,7 @@ static int irdma_create_aeq(struct irdma_pci_f *rf)
 		   hmc_info->hmc_obj[IRDMA_HMC_IW_CQ].cnt;
 	aeq_size = min(aeq_size, dev->hw_attrs.max_hw_aeq_size);
 	/* GEN_3 does not support virtual AEQ. Cap at max Kernel alloc size */
-	if (rf->rdma_ver == IRDMA_GEN_3)
+	if (rf->rdma_ver >= IRDMA_GEN_3)
 		aeq_size = min(aeq_size, (u32)((PAGE_SIZE << MAX_PAGE_ORDER) /
 			       sizeof(struct irdma_sc_aeqe)));
 	aeq->mem.size = ALIGN(sizeof(struct irdma_sc_aeqe) * aeq_size,
@@ -1518,7 +1519,7 @@ static int irdma_create_aeq(struct irdma_pci_f *rf)
 					 GFP_KERNEL | __GFP_NOWARN);
 	if (aeq->mem.va)
 		goto skip_virt_aeq;
-	else if (rf->rdma_ver == IRDMA_GEN_3)
+	else if (rf->rdma_ver >= IRDMA_GEN_3)
 		return -ENOMEM;
 
 	/* physically mapped aeq failed. setup virtual aeq */
@@ -2192,8 +2193,13 @@ u32 irdma_initialize_hw_rsrc(struct irdma_pci_f *rf)
 	set_bit(2, rf->allocated_pds);
 
 	INIT_LIST_HEAD(&rf->mc_qht_list.list);
-	/* stag index mask has a minimum of 14 bits */
-	mrdrvbits = 24 - max(get_count_order(rf->max_mr), 14);
+
+	if (rf->rdma_ver >= IRDMA_GEN_4)
+		mrdrvbits = 24 - max(get_count_order(rf->max_mr), 16);
+	else
+		/* stag index mask has a minimum of 14 bits */
+		mrdrvbits = 24 - max(get_count_order(rf->max_mr), 14);
+
 	rf->mr_stagmask = ~(((1 << mrdrvbits) - 1) << (32 - mrdrvbits));
 
 	return 0;
diff --git a/drivers/infiniband/hw/irdma/ig3rdma_hw.c b/drivers/infiniband/hw/irdma/ig3rdma_hw.c
index 2e8bb475e22a..f0361675c2de 100644
--- a/drivers/infiniband/hw/irdma/ig3rdma_hw.c
+++ b/drivers/infiniband/hw/irdma/ig3rdma_hw.c
@@ -113,7 +113,6 @@ void ig3rdma_init_hw(struct irdma_sc_dev *dev)
 	dev->irq_ops = &ig3rdma_irq_ops;
 	dev->hw_stats_map = ig3rdma_hw_stat_map;
 
-	dev->hw_attrs.uk_attrs.hw_rev = IRDMA_GEN_3;
 	dev->hw_attrs.uk_attrs.max_hw_wq_frags = IG3RDMA_MAX_WQ_FRAGMENT_COUNT;
 	dev->hw_attrs.uk_attrs.max_hw_read_sges = IG3RDMA_MAX_SGE_RD;
 	dev->hw_attrs.uk_attrs.max_hw_sq_chunk = IRDMA_MAX_QUANTA_PER_WR;
diff --git a/drivers/infiniband/hw/irdma/irdma.h b/drivers/infiniband/hw/irdma/irdma.h
index ff938a01d70c..b5ce515f4ee8 100644
--- a/drivers/infiniband/hw/irdma/irdma.h
+++ b/drivers/infiniband/hw/irdma/irdma.h
@@ -119,6 +119,7 @@ enum irdma_vers {
 	IRDMA_GEN_1,
 	IRDMA_GEN_2,
 	IRDMA_GEN_3,
+	IRDMA_GEN_4,
 	IRDMA_GEN_NEXT,
 	IRDMA_GEN_MAX = IRDMA_GEN_NEXT-1
 };
-- 
2.31.1

Re: [for-next 03/12] RDMA/irdma: Change ah_valid type to atomic

[Leon Romanovsky]

On Mon, Mar 16, 2026 at 01:39:40PM -0500, Tatyana Nikolova wrote:
> From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> 
> Converts ah_valid flag to atomic to protect it against data-race.

Atomic operations don't prevent data races; they only guarantee that a
read or write happens atomically. Use the "xxx yyy:1" bitfield construct
only for fields that cannot be modified concurrently.

Thanks

> 
> [  809.561229] BUG: KCSAN: data-race in irdma_create_hw_ah [irdma] / irdma_gsi_ud_qp_ah_cb [irdma]
> 
> [  809.565182] write to 0xffff8d911c8eee73 of 1 bytes by task 38949 on cpu 10:
> [  809.567113]  irdma_gsi_ud_qp_ah_cb+0x41/0x60 [irdma]
> [  809.567343]  irdma_complete_cqp_request+0x44/0xb0 [irdma]
> [  809.567572]  irdma_cqp_ce_handler+0x242/0x290 [irdma]
> [  809.567810]  irdma_wait_event+0xf2/0x3c0 [irdma]
> 
> [  809.570951] read to 0xffff8d911c8eee73 of 1 bytes by task 38952 on cpu 7:
> [  809.573037]  irdma_create_hw_ah+0x1e7/0x370 [irdma]
> [  809.573265]  irdma_create_ah+0x61/0x70 [irdma]
> [  809.573491]  _rdma_create_ah+0x262/0x290 [ib_core]
> [  809.573831]  rdma_create_ah+0xa3/0x140 [ib_core]
> 
> [  809.576933] value changed: 0x06 -> 0x07
> [  809.581184] Reported by Kernel Concurrency Sanitizer
> 
> Fixes: dd90451fac23 ("RDMA/irdma: Add RoCEv2 UD OP support")
> Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> ---
>  drivers/infiniband/hw/irdma/cm.c    |  2 +-
>  drivers/infiniband/hw/irdma/puda.c  |  2 +-
>  drivers/infiniband/hw/irdma/uda.h   |  2 +-
>  drivers/infiniband/hw/irdma/utils.c | 16 +++++++++-------
>  drivers/infiniband/hw/irdma/verbs.c |  7 ++++---
>  5 files changed, 16 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
> index 3d084d4ff577..947fd93f5fb0 100644
> --- a/drivers/infiniband/hw/irdma/cm.c
> +++ b/drivers/infiniband/hw/irdma/cm.c
> @@ -313,7 +313,7 @@ static struct irdma_puda_buf *irdma_form_ah_cm_frame(struct irdma_cm_node *cm_no
>  	u32 pd_len = 0;
>  	u32 hdr_len = 0;
>  
> -	if (!cm_node->ah || !cm_node->ah->ah_info.ah_valid) {
> +	if (!cm_node->ah || !atomic_read(&cm_node->ah->ah_info.ah_valid)) {
>  		ibdev_dbg(&cm_node->iwdev->ibdev, "CM: AH invalid\n");
>  		return NULL;
>  	}
> diff --git a/drivers/infiniband/hw/irdma/puda.c b/drivers/infiniband/hw/irdma/puda.c
> index 4f1a8c97faf1..3410be38f602 100644
> --- a/drivers/infiniband/hw/irdma/puda.c
> +++ b/drivers/infiniband/hw/irdma/puda.c
> @@ -1652,7 +1652,7 @@ static void irdma_ieq_handle_exception(struct irdma_puda_rsrc *ieq,
>  	}
>  	if (hw_rev == IRDMA_GEN_1)
>  		irdma_ieq_process_fpdus(qp, ieq);
> -	else if (pfpdu->ah && pfpdu->ah->ah_info.ah_valid)
> +	else if (pfpdu->ah && atomic_read(&pfpdu->ah->ah_info.ah_valid))
>  		irdma_ieq_process_fpdus(qp, ieq);
>  exit:
>  	spin_unlock_irqrestore(&pfpdu->lock, flags);
> diff --git a/drivers/infiniband/hw/irdma/uda.h b/drivers/infiniband/hw/irdma/uda.h
> index 27b8701cf21b..773c33ce62a2 100644
> --- a/drivers/infiniband/hw/irdma/uda.h
> +++ b/drivers/infiniband/hw/irdma/uda.h
> @@ -22,7 +22,7 @@ struct irdma_ah_info {
>  	u8 tc_tos;
>  	u8 hop_ttl;
>  	u8 mac_addr[ETH_ALEN];
> -	bool ah_valid:1;
> +	atomic_t ah_valid;
>  	bool ipv4_valid:1;
>  	bool do_lpbk:1;
>  };
> diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
> index f9c99c216a2c..be2dc92d008f 100644
> --- a/drivers/infiniband/hw/irdma/utils.c
> +++ b/drivers/infiniband/hw/irdma/utils.c
> @@ -1967,7 +1967,8 @@ int irdma_ah_cqp_op(struct irdma_pci_f *rf, struct irdma_sc_ah *sc_ah, u8 cmd,
>  		return -ENOMEM;
>  
>  	if (wait)
> -		sc_ah->ah_info.ah_valid = (cmd == IRDMA_OP_AH_CREATE);
> +		atomic_set(&sc_ah->ah_info.ah_valid,
> +			   (cmd == IRDMA_OP_AH_CREATE));
>  
>  	return 0;
>  }
> @@ -1984,10 +1985,10 @@ static void irdma_ieq_ah_cb(struct irdma_cqp_request *cqp_request)
>  
>  	spin_lock_irqsave(&qp->pfpdu.lock, flags);
>  	if (!cqp_request->compl_info.op_ret_val) {
> -		sc_ah->ah_info.ah_valid = true;
> +		atomic_set(&sc_ah->ah_info.ah_valid, true);
>  		irdma_ieq_process_fpdus(qp, qp->vsi->ieq);
>  	} else {
> -		sc_ah->ah_info.ah_valid = false;
> +		atomic_set(&sc_ah->ah_info.ah_valid, false);
>  		irdma_ieq_cleanup_qp(qp->vsi->ieq, qp);
>  	}
>  	spin_unlock_irqrestore(&qp->pfpdu.lock, flags);
> @@ -2002,7 +2003,8 @@ static void irdma_ilq_ah_cb(struct irdma_cqp_request *cqp_request)
>  	struct irdma_cm_node *cm_node = cqp_request->param;
>  	struct irdma_sc_ah *sc_ah = cm_node->ah;
>  
> -	sc_ah->ah_info.ah_valid = !cqp_request->compl_info.op_ret_val;
> +	atomic_set(&sc_ah->ah_info.ah_valid,
> +		   !cqp_request->compl_info.op_ret_val);
>  	irdma_add_conn_est_qh(cm_node);
>  }
>  
> @@ -2069,7 +2071,7 @@ void irdma_puda_free_ah(struct irdma_sc_dev *dev, struct irdma_sc_ah *ah)
>  	if (!ah)
>  		return;
>  
> -	if (ah->ah_info.ah_valid) {
> +	if (atomic_read(&ah->ah_info.ah_valid)) {
>  		irdma_ah_cqp_op(rf, ah, IRDMA_OP_AH_DESTROY, false, NULL, NULL);
>  		irdma_free_rsrc(rf, rf->allocated_ahs, ah->ah_info.ah_idx);
>  	}
> @@ -2086,9 +2088,9 @@ void irdma_gsi_ud_qp_ah_cb(struct irdma_cqp_request *cqp_request)
>  	struct irdma_sc_ah *sc_ah = cqp_request->param;
>  
>  	if (!cqp_request->compl_info.op_ret_val)
> -		sc_ah->ah_info.ah_valid = true;
> +		atomic_set(&sc_ah->ah_info.ah_valid, true);
>  	else
> -		sc_ah->ah_info.ah_valid = false;
> +		atomic_set(&sc_ah->ah_info.ah_valid, false);
>  }
>  
>  /**
> diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
> index 38bc0e656ecf..9cfcdf7b053e 100644
> --- a/drivers/infiniband/hw/irdma/verbs.c
> +++ b/drivers/infiniband/hw/irdma/verbs.c
> @@ -5116,8 +5116,8 @@ static int irdma_create_hw_ah(struct irdma_device *iwdev, struct irdma_ah *ah, b
>  
>  		if (poll_timeout_us_atomic(irdma_cqp_ce_handler(rf,
>  								&rf->ccq.sc_cq),
> -					   ah->sc_ah.ah_info.ah_valid, 1,
> -					   tmout_ms * USEC_PER_MSEC, false)) {
> +					   atomic_read(&ah->sc_ah.ah_info.ah_valid),
> +					   1, tmout_ms * USEC_PER_MSEC, false)) {
>  			ibdev_dbg(&iwdev->ibdev,
>  				  "VERBS: CQP create AH timed out");
>  			err = -ETIMEDOUT;
> @@ -5236,7 +5236,8 @@ static bool irdma_ah_exists(struct irdma_device *iwdev,
>  	hash_for_each_possible(iwdev->rf->ah_hash_tbl, ah, list, key) {
>  		/* Set ah_valid and ah_id the same so memcmp can work */
>  		new_ah->sc_ah.ah_info.ah_idx = ah->sc_ah.ah_info.ah_idx;
> -		new_ah->sc_ah.ah_info.ah_valid = ah->sc_ah.ah_info.ah_valid;
> +		atomic_set(&new_ah->sc_ah.ah_info.ah_valid,
> +			   atomic_read(&ah->sc_ah.ah_info.ah_valid));
>  		if (!memcmp(&ah->sc_ah.ah_info, &new_ah->sc_ah.ah_info,
>  			    sizeof(ah->sc_ah.ah_info))) {
>  			refcount_inc(&ah->refcnt);
> -- 
> 2.31.1
> 

Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Leon Romanovsky]

On Mon, Mar 16, 2026 at 01:39:39PM -0500, Tatyana Nikolova wrote:
> From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> 
> Changes type of request_done flag from bool to atomic_t to fix
> data race in irdma_complete_cqp_request / irdma_wait_event
> reported by KCSAN:

Again, this fix is wrong too.

Thanks

> 
> BUG: KCSAN: data-race in irdma_complete_cqp_request [irdma] / irdma_wait_event [irdma]
> 
> write (marked) to 0xffffa0bef390ad5c of 1 bytes by task 7761 on cpu 0:
>  irdma_complete_cqp_request+0x19/0x90 [irdma]
>  irdma_cqp_ce_handler+0x22d/0x290 [irdma]
>  cqp_compl_worker+0x1f/0x30 [irdma]
>  process_one_work+0x3dc/0x7c0
>  worker_thread+0xa6/0x6c0
>  kthread+0x17f/0x1c0
>  ret_from_fork+0x2c/0x50
> 
> read to 0xffffa0bef390ad5c of 1 bytes by task 28566 on cpu 3:
>  irdma_wait_event+0x242/0x390 [irdma]
>  irdma_handle_cqp_op+0x93/0x210 [irdma]
>  irdma_hw_modify_qp+0xe3/0x2f0 [irdma]
>  irdma_modify_qp_roce+0x13df/0x1630 [irdma]
>  ib_security_modify_qp+0x34a/0x640 [ib_core]
>  _ib_modify_qp+0x16b/0x620 [ib_core]
>  ib_modify_qp_with_udata+0x3c/0x50 [ib_core]
>  modify_qp+0x6e1/0x920 [ib_uverbs]
>  ib_uverbs_ex_modify_qp+0xa6/0xf0 [ib_uverbs]
>  ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0x16c/0x200 [ib_uverbs]
>  ib_uverbs_run_method+0x342/0x380 [ib_uverbs]
>  ib_uverbs_cmd_verbs+0x365/0x440 [ib_uverbs]
>  ib_uverbs_ioctl+0x111/0x190 [ib_uverbs]
>  __x64_sys_ioctl+0xc3/0x100
>  do_syscall_64+0x3f/0x90
>  entry_SYSCALL_64_after_hwframe+0x72/0xdc
> 
> value changed: 0x00 -> 0x01
> 
> Fixes: f0842bb3d388 ("RDMA/irdma: Fix data race on CQP request done")
> Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> ---
>  drivers/infiniband/hw/irdma/hw.c    | 2 +-
>  drivers/infiniband/hw/irdma/main.h  | 2 +-
>  drivers/infiniband/hw/irdma/utils.c | 6 +++---
>  3 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
> index 6e0466ce83d1..3ba4809bc1ef 100644
> --- a/drivers/infiniband/hw/irdma/hw.c
> +++ b/drivers/infiniband/hw/irdma/hw.c
> @@ -235,7 +235,7 @@ static void irdma_complete_cqp_request(struct irdma_cqp *cqp,
>  				       struct irdma_cqp_request *cqp_request)
>  {
>  	if (cqp_request->waiting) {
> -		WRITE_ONCE(cqp_request->request_done, true);
> +		atomic_set(&cqp_request->request_done, true);
>  		wake_up(&cqp_request->waitq);
>  	} else if (cqp_request->callback_fcn) {
>  		cqp_request->callback_fcn(cqp_request);
> diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h
> index 3d49bd57bae7..e22160e2ba33 100644
> --- a/drivers/infiniband/hw/irdma/main.h
> +++ b/drivers/infiniband/hw/irdma/main.h
> @@ -167,7 +167,7 @@ struct irdma_cqp_request {
>  	void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
>  	void *param;
>  	struct irdma_cqp_compl_info compl_info;
> -	bool request_done; /* READ/WRITE_ONCE macros operate on it */
> +	atomic_t request_done;
>  	bool waiting:1;
>  	bool dynamic:1;
>  	bool pending:1;
> diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
> index ab8c5284d4be..f9c99c216a2c 100644
> --- a/drivers/infiniband/hw/irdma/utils.c
> +++ b/drivers/infiniband/hw/irdma/utils.c
> @@ -480,7 +480,7 @@ void irdma_free_cqp_request(struct irdma_cqp *cqp,
>  	if (cqp_request->dynamic) {
>  		kfree(cqp_request);
>  	} else {
> -		WRITE_ONCE(cqp_request->request_done, false);
> +		atomic_set(&cqp_request->request_done, false);
>  		cqp_request->callback_fcn = NULL;
>  		cqp_request->waiting = false;
>  		cqp_request->pending = false;
> @@ -515,7 +515,7 @@ irdma_free_pending_cqp_request(struct irdma_cqp *cqp,
>  {
>  	if (cqp_request->waiting) {
>  		cqp_request->compl_info.error = true;
> -		WRITE_ONCE(cqp_request->request_done, true);
> +		atomic_set(&cqp_request->request_done, true);
>  		wake_up(&cqp_request->waitq);
>  	}
>  	wait_event_timeout(cqp->remove_wq,
> @@ -610,7 +610,7 @@ static int irdma_wait_event(struct irdma_pci_f *rf,
>  	do {
>  		irdma_cqp_ce_handler(rf, &rf->ccq.sc_cq);
>  		if (wait_event_timeout(cqp_request->waitq,
> -				       READ_ONCE(cqp_request->request_done),
> +				       atomic_read(&cqp_request->request_done),
>  				       msecs_to_jiffies(CQP_COMPL_WAIT_TIME_MS)))
>  			break;
>  
> -- 
> 2.31.1
> 

RE: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Czurylo, Krzysztof]

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: 17 March, 2026 12:13
> To: Nikolova, Tatyana E <tatyana.e.nikolova@intel.com>
> Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Czurylo, Krzysztof
> <krzysztof.czurylo@intel.com>
> Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> >request_done
> 
> On Mon, Mar 16, 2026 at 01:39:39PM -0500, Tatyana Nikolova wrote:
> > From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> >
> > Changes type of request_done flag from bool to atomic_t to fix
> > data race in irdma_complete_cqp_request / irdma_wait_event
> > reported by KCSAN:
> 
> Again, this fix is wrong too.

Could you please elaborate on what is wrong with this fix?
And/or suggest how to fix it properly?

Please note 'request_done' is _not_ a bitfield and we only do simple
load/store operations on it.  There is no RMW.
Despite this, KCSAN still reports a data race on it.

Honestly, the original idea was just to change the type from
'bool' to 'u8'.  This is enough to silence KCSAN, but it is
not clear why.  Perhaps it indicates a bug in KCSAN?
I mean, maybe the report below is a false positive?

Thanks

> 
> Thanks
> 
> >
> > BUG: KCSAN: data-race in irdma_complete_cqp_request [irdma] /
> irdma_wait_event [irdma]
> >
> > write (marked) to 0xffffa0bef390ad5c of 1 bytes by task 7761 on cpu 0:
> >  irdma_complete_cqp_request+0x19/0x90 [irdma]
> >  irdma_cqp_ce_handler+0x22d/0x290 [irdma]
> >  cqp_compl_worker+0x1f/0x30 [irdma]
> >  process_one_work+0x3dc/0x7c0
> >  worker_thread+0xa6/0x6c0
> >  kthread+0x17f/0x1c0
> >  ret_from_fork+0x2c/0x50
> >
> > read to 0xffffa0bef390ad5c of 1 bytes by task 28566 on cpu 3:
> >  irdma_wait_event+0x242/0x390 [irdma]
> >  irdma_handle_cqp_op+0x93/0x210 [irdma]
> >  irdma_hw_modify_qp+0xe3/0x2f0 [irdma]
> >  irdma_modify_qp_roce+0x13df/0x1630 [irdma]
> >  ib_security_modify_qp+0x34a/0x640 [ib_core]
> >  _ib_modify_qp+0x16b/0x620 [ib_core]
> >  ib_modify_qp_with_udata+0x3c/0x50 [ib_core]
> >  modify_qp+0x6e1/0x920 [ib_uverbs]
> >  ib_uverbs_ex_modify_qp+0xa6/0xf0 [ib_uverbs]
> >  ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0x16c/0x200 [ib_uverbs]
> >  ib_uverbs_run_method+0x342/0x380 [ib_uverbs]
> >  ib_uverbs_cmd_verbs+0x365/0x440 [ib_uverbs]
> >  ib_uverbs_ioctl+0x111/0x190 [ib_uverbs]
> >  __x64_sys_ioctl+0xc3/0x100
> >  do_syscall_64+0x3f/0x90
> >  entry_SYSCALL_64_after_hwframe+0x72/0xdc
> >
> > value changed: 0x00 -> 0x01
> >
> > Fixes: f0842bb3d388 ("RDMA/irdma: Fix data race on CQP request done")
> > Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> > ---
> >  drivers/infiniband/hw/irdma/hw.c    | 2 +-
> >  drivers/infiniband/hw/irdma/main.h  | 2 +-
> >  drivers/infiniband/hw/irdma/utils.c | 6 +++---
> >  3 files changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/irdma/hw.c
> b/drivers/infiniband/hw/irdma/hw.c
> > index 6e0466ce83d1..3ba4809bc1ef 100644
> > --- a/drivers/infiniband/hw/irdma/hw.c
> > +++ b/drivers/infiniband/hw/irdma/hw.c
> > @@ -235,7 +235,7 @@ static void irdma_complete_cqp_request(struct
> irdma_cqp *cqp,
> >  				       struct irdma_cqp_request *cqp_request)
> >  {
> >  	if (cqp_request->waiting) {
> > -		WRITE_ONCE(cqp_request->request_done, true);
> > +		atomic_set(&cqp_request->request_done, true);
> >  		wake_up(&cqp_request->waitq);
> >  	} else if (cqp_request->callback_fcn) {
> >  		cqp_request->callback_fcn(cqp_request);
> > diff --git a/drivers/infiniband/hw/irdma/main.h
> b/drivers/infiniband/hw/irdma/main.h
> > index 3d49bd57bae7..e22160e2ba33 100644
> > --- a/drivers/infiniband/hw/irdma/main.h
> > +++ b/drivers/infiniband/hw/irdma/main.h
> > @@ -167,7 +167,7 @@ struct irdma_cqp_request {
> >  	void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
> >  	void *param;
> >  	struct irdma_cqp_compl_info compl_info;
> > -	bool request_done; /* READ/WRITE_ONCE macros operate on it */
> > +	atomic_t request_done;
> >  	bool waiting:1;
> >  	bool dynamic:1;
> >  	bool pending:1;
> > diff --git a/drivers/infiniband/hw/irdma/utils.c
> b/drivers/infiniband/hw/irdma/utils.c
> > index ab8c5284d4be..f9c99c216a2c 100644
> > --- a/drivers/infiniband/hw/irdma/utils.c
> > +++ b/drivers/infiniband/hw/irdma/utils.c
> > @@ -480,7 +480,7 @@ void irdma_free_cqp_request(struct irdma_cqp *cqp,
> >  	if (cqp_request->dynamic) {
> >  		kfree(cqp_request);
> >  	} else {
> > -		WRITE_ONCE(cqp_request->request_done, false);
> > +		atomic_set(&cqp_request->request_done, false);
> >  		cqp_request->callback_fcn = NULL;
> >  		cqp_request->waiting = false;
> >  		cqp_request->pending = false;
> > @@ -515,7 +515,7 @@ irdma_free_pending_cqp_request(struct irdma_cqp
> *cqp,
> >  {
> >  	if (cqp_request->waiting) {
> >  		cqp_request->compl_info.error = true;
> > -		WRITE_ONCE(cqp_request->request_done, true);
> > +		atomic_set(&cqp_request->request_done, true);
> >  		wake_up(&cqp_request->waitq);
> >  	}
> >  	wait_event_timeout(cqp->remove_wq,
> > @@ -610,7 +610,7 @@ static int irdma_wait_event(struct irdma_pci_f *rf,
> >  	do {
> >  		irdma_cqp_ce_handler(rf, &rf->ccq.sc_cq);
> >  		if (wait_event_timeout(cqp_request->waitq,
> > -				       READ_ONCE(cqp_request->request_done),
> > +				       atomic_read(&cqp_request->request_done),
> >  				       msecs_to_jiffies(CQP_COMPL_WAIT_TIME_MS)))
> >  			break;
> >
> > --
> > 2.31.1
> >

Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Leon Romanovsky]

On Tue, Mar 17, 2026 at 12:14:21PM +0000, Czurylo, Krzysztof wrote:
> 
> 
> > -----Original Message-----
> > From: Leon Romanovsky <leon@kernel.org>
> > Sent: 17 March, 2026 12:13
> > To: Nikolova, Tatyana E <tatyana.e.nikolova@intel.com>
> > Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Czurylo, Krzysztof
> > <krzysztof.czurylo@intel.com>
> > Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> > >request_done
> > 
> > On Mon, Mar 16, 2026 at 01:39:39PM -0500, Tatyana Nikolova wrote:
> > > From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > >
> > > Changes type of request_done flag from bool to atomic_t to fix
> > > data race in irdma_complete_cqp_request / irdma_wait_event
> > > reported by KCSAN:
> > 
> > Again, this fix is wrong too.
> 
> Could you please elaborate on what is wrong with this fix?
> And/or suggest how to fix it properly?
> 
> Please note 'request_done' is _not_ a bitfield and we only do simple
> load/store operations on it.  There is no RMW.
> Despite this, KCSAN still reports a data race on it.
> 
> Honestly, the original idea was just to change the type from
> 'bool' to 'u8'.  This is enough to silence KCSAN, but it is
> not clear why.  Perhaps it indicates a bug in KCSAN?

Yes, both u8 and atomic_t behave the same, they can't be interrupted
during read/write. This is why KCSAN doesn't warn you.

> I mean, maybe the report below is a false positive?

Sounds like that.

> 
> Thanks
> 
> > 
> > Thanks
> > 
> > >
> > > BUG: KCSAN: data-race in irdma_complete_cqp_request [irdma] /
> > irdma_wait_event [irdma]
> > >
> > > write (marked) to 0xffffa0bef390ad5c of 1 bytes by task 7761 on cpu 0:
> > >  irdma_complete_cqp_request+0x19/0x90 [irdma]
> > >  irdma_cqp_ce_handler+0x22d/0x290 [irdma]
> > >  cqp_compl_worker+0x1f/0x30 [irdma]
> > >  process_one_work+0x3dc/0x7c0
> > >  worker_thread+0xa6/0x6c0
> > >  kthread+0x17f/0x1c0
> > >  ret_from_fork+0x2c/0x50
> > >
> > > read to 0xffffa0bef390ad5c of 1 bytes by task 28566 on cpu 3:
> > >  irdma_wait_event+0x242/0x390 [irdma]
> > >  irdma_handle_cqp_op+0x93/0x210 [irdma]
> > >  irdma_hw_modify_qp+0xe3/0x2f0 [irdma]
> > >  irdma_modify_qp_roce+0x13df/0x1630 [irdma]
> > >  ib_security_modify_qp+0x34a/0x640 [ib_core]
> > >  _ib_modify_qp+0x16b/0x620 [ib_core]
> > >  ib_modify_qp_with_udata+0x3c/0x50 [ib_core]
> > >  modify_qp+0x6e1/0x920 [ib_uverbs]
> > >  ib_uverbs_ex_modify_qp+0xa6/0xf0 [ib_uverbs]
> > >  ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0x16c/0x200 [ib_uverbs]
> > >  ib_uverbs_run_method+0x342/0x380 [ib_uverbs]
> > >  ib_uverbs_cmd_verbs+0x365/0x440 [ib_uverbs]
> > >  ib_uverbs_ioctl+0x111/0x190 [ib_uverbs]
> > >  __x64_sys_ioctl+0xc3/0x100
> > >  do_syscall_64+0x3f/0x90
> > >  entry_SYSCALL_64_after_hwframe+0x72/0xdc
> > >
> > > value changed: 0x00 -> 0x01
> > >
> > > Fixes: f0842bb3d388 ("RDMA/irdma: Fix data race on CQP request done")
> > > Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > > Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> > > ---
> > >  drivers/infiniband/hw/irdma/hw.c    | 2 +-
> > >  drivers/infiniband/hw/irdma/main.h  | 2 +-
> > >  drivers/infiniband/hw/irdma/utils.c | 6 +++---
> > >  3 files changed, 5 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/infiniband/hw/irdma/hw.c
> > b/drivers/infiniband/hw/irdma/hw.c
> > > index 6e0466ce83d1..3ba4809bc1ef 100644
> > > --- a/drivers/infiniband/hw/irdma/hw.c
> > > +++ b/drivers/infiniband/hw/irdma/hw.c
> > > @@ -235,7 +235,7 @@ static void irdma_complete_cqp_request(struct
> > irdma_cqp *cqp,
> > >  				       struct irdma_cqp_request *cqp_request)
> > >  {
> > >  	if (cqp_request->waiting) {
> > > -		WRITE_ONCE(cqp_request->request_done, true);
> > > +		atomic_set(&cqp_request->request_done, true);
> > >  		wake_up(&cqp_request->waitq);
> > >  	} else if (cqp_request->callback_fcn) {
> > >  		cqp_request->callback_fcn(cqp_request);
> > > diff --git a/drivers/infiniband/hw/irdma/main.h
> > b/drivers/infiniband/hw/irdma/main.h
> > > index 3d49bd57bae7..e22160e2ba33 100644
> > > --- a/drivers/infiniband/hw/irdma/main.h
> > > +++ b/drivers/infiniband/hw/irdma/main.h
> > > @@ -167,7 +167,7 @@ struct irdma_cqp_request {
> > >  	void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
> > >  	void *param;
> > >  	struct irdma_cqp_compl_info compl_info;
> > > -	bool request_done; /* READ/WRITE_ONCE macros operate on it */
> > > +	atomic_t request_done;
> > >  	bool waiting:1;
> > >  	bool dynamic:1;
> > >  	bool pending:1;
> > > diff --git a/drivers/infiniband/hw/irdma/utils.c
> > b/drivers/infiniband/hw/irdma/utils.c
> > > index ab8c5284d4be..f9c99c216a2c 100644
> > > --- a/drivers/infiniband/hw/irdma/utils.c
> > > +++ b/drivers/infiniband/hw/irdma/utils.c
> > > @@ -480,7 +480,7 @@ void irdma_free_cqp_request(struct irdma_cqp *cqp,
> > >  	if (cqp_request->dynamic) {
> > >  		kfree(cqp_request);
> > >  	} else {
> > > -		WRITE_ONCE(cqp_request->request_done, false);
> > > +		atomic_set(&cqp_request->request_done, false);
> > >  		cqp_request->callback_fcn = NULL;
> > >  		cqp_request->waiting = false;
> > >  		cqp_request->pending = false;
> > > @@ -515,7 +515,7 @@ irdma_free_pending_cqp_request(struct irdma_cqp
> > *cqp,
> > >  {
> > >  	if (cqp_request->waiting) {
> > >  		cqp_request->compl_info.error = true;
> > > -		WRITE_ONCE(cqp_request->request_done, true);
> > > +		atomic_set(&cqp_request->request_done, true);
> > >  		wake_up(&cqp_request->waitq);
> > >  	}
> > >  	wait_event_timeout(cqp->remove_wq,
> > > @@ -610,7 +610,7 @@ static int irdma_wait_event(struct irdma_pci_f *rf,
> > >  	do {
> > >  		irdma_cqp_ce_handler(rf, &rf->ccq.sc_cq);
> > >  		if (wait_event_timeout(cqp_request->waitq,
> > > -				       READ_ONCE(cqp_request->request_done),
> > > +				       atomic_read(&cqp_request->request_done),
> > >  				       msecs_to_jiffies(CQP_COMPL_WAIT_TIME_MS)))
> > >  			break;
> > >
> > > --
> > > 2.31.1
> > >
> 

RE: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Nikolova, Tatyana E]

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Tuesday, March 17, 2026 8:23 AM
> To: Czurylo, Krzysztof <krzysztof.czurylo@intel.com>
> Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Nikolova, Tatyana E
> <tatyana.e.nikolova@intel.com>
> Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> >request_done
> 
> On Tue, Mar 17, 2026 at 12:14:21PM +0000, Czurylo, Krzysztof wrote:
> >
> >
> > > -----Original Message-----
> > > From: Leon Romanovsky <leon@kernel.org>
> > > Sent: 17 March, 2026 12:13
> > > To: Nikolova, Tatyana E <tatyana.e.nikolova@intel.com>
> > > Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Czurylo, Krzysztof
> > > <krzysztof.czurylo@intel.com>
> > > Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> > > >request_done
> > >
> > > On Mon, Mar 16, 2026 at 01:39:39PM -0500, Tatyana Nikolova wrote:
> > > > From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > > >
> > > > Changes type of request_done flag from bool to atomic_t to fix
> > > > data race in irdma_complete_cqp_request / irdma_wait_event
> > > > reported by KCSAN:
> > >
> > > Again, this fix is wrong too.
> >
> > Could you please elaborate on what is wrong with this fix?
> > And/or suggest how to fix it properly?
> >
> > Please note 'request_done' is _not_ a bitfield and we only do simple
> > load/store operations on it.  There is no RMW.
> > Despite this, KCSAN still reports a data race on it.
> >
> > Honestly, the original idea was just to change the type from
> > 'bool' to 'u8'.  This is enough to silence KCSAN, but it is
> > not clear why.  Perhaps it indicates a bug in KCSAN?
> 
> Yes, both u8 and atomic_t behave the same, they can't be interrupted
> during read/write. This is why KCSAN doesn't warn you.
> 
> > I mean, maybe the report below is a false positive?
> 
> Sounds like that.

Leon,

Are you okay taking the rest of the patches in the series (they apply without the two KCSAN related patches) or you prefer that I resubmit the series?

For this specific patch, are you okay with dropping all atomic changes, but making request_done u8 (currently bool) to silence the KCSAN warning?

Thank you,
Tatyana

> 
> >
> > Thanks
> >
> > >
> > > Thanks
> > >
> > > >
> > > > BUG: KCSAN: data-race in irdma_complete_cqp_request [irdma] /
> > > irdma_wait_event [irdma]
> > > >
> > > > write (marked) to 0xffffa0bef390ad5c of 1 bytes by task 7761 on cpu 0:
> > > >  irdma_complete_cqp_request+0x19/0x90 [irdma]
> > > >  irdma_cqp_ce_handler+0x22d/0x290 [irdma]
> > > >  cqp_compl_worker+0x1f/0x30 [irdma]
> > > >  process_one_work+0x3dc/0x7c0
> > > >  worker_thread+0xa6/0x6c0
> > > >  kthread+0x17f/0x1c0
> > > >  ret_from_fork+0x2c/0x50
> > > >
> > > > read to 0xffffa0bef390ad5c of 1 bytes by task 28566 on cpu 3:
> > > >  irdma_wait_event+0x242/0x390 [irdma]
> > > >  irdma_handle_cqp_op+0x93/0x210 [irdma]
> > > >  irdma_hw_modify_qp+0xe3/0x2f0 [irdma]
> > > >  irdma_modify_qp_roce+0x13df/0x1630 [irdma]
> > > >  ib_security_modify_qp+0x34a/0x640 [ib_core]
> > > >  _ib_modify_qp+0x16b/0x620 [ib_core]
> > > >  ib_modify_qp_with_udata+0x3c/0x50 [ib_core]
> > > >  modify_qp+0x6e1/0x920 [ib_uverbs]
> > > >  ib_uverbs_ex_modify_qp+0xa6/0xf0 [ib_uverbs]
> > > >  ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0x16c/0x200
> [ib_uverbs]
> > > >  ib_uverbs_run_method+0x342/0x380 [ib_uverbs]
> > > >  ib_uverbs_cmd_verbs+0x365/0x440 [ib_uverbs]
> > > >  ib_uverbs_ioctl+0x111/0x190 [ib_uverbs]
> > > >  __x64_sys_ioctl+0xc3/0x100
> > > >  do_syscall_64+0x3f/0x90
> > > >  entry_SYSCALL_64_after_hwframe+0x72/0xdc
> > > >
> > > > value changed: 0x00 -> 0x01
> > > >
> > > > Fixes: f0842bb3d388 ("RDMA/irdma: Fix data race on CQP request
> done")
> > > > Signed-off-by: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > > > Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> > > > ---
> > > >  drivers/infiniband/hw/irdma/hw.c    | 2 +-
> > > >  drivers/infiniband/hw/irdma/main.h  | 2 +-
> > > >  drivers/infiniband/hw/irdma/utils.c | 6 +++---
> > > >  3 files changed, 5 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/drivers/infiniband/hw/irdma/hw.c
> > > b/drivers/infiniband/hw/irdma/hw.c
> > > > index 6e0466ce83d1..3ba4809bc1ef 100644
> > > > --- a/drivers/infiniband/hw/irdma/hw.c
> > > > +++ b/drivers/infiniband/hw/irdma/hw.c
> > > > @@ -235,7 +235,7 @@ static void irdma_complete_cqp_request(struct
> > > irdma_cqp *cqp,
> > > >  				       struct irdma_cqp_request *cqp_request)
> > > >  {
> > > >  	if (cqp_request->waiting) {
> > > > -		WRITE_ONCE(cqp_request->request_done, true);
> > > > +		atomic_set(&cqp_request->request_done, true);
> > > >  		wake_up(&cqp_request->waitq);
> > > >  	} else if (cqp_request->callback_fcn) {
> > > >  		cqp_request->callback_fcn(cqp_request);
> > > > diff --git a/drivers/infiniband/hw/irdma/main.h
> > > b/drivers/infiniband/hw/irdma/main.h
> > > > index 3d49bd57bae7..e22160e2ba33 100644
> > > > --- a/drivers/infiniband/hw/irdma/main.h
> > > > +++ b/drivers/infiniband/hw/irdma/main.h
> > > > @@ -167,7 +167,7 @@ struct irdma_cqp_request {
> > > >  	void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
> > > >  	void *param;
> > > >  	struct irdma_cqp_compl_info compl_info;
> > > > -	bool request_done; /* READ/WRITE_ONCE macros operate on it */
> > > > +	atomic_t request_done;
> > > >  	bool waiting:1;
> > > >  	bool dynamic:1;
> > > >  	bool pending:1;
> > > > diff --git a/drivers/infiniband/hw/irdma/utils.c
> > > b/drivers/infiniband/hw/irdma/utils.c
> > > > index ab8c5284d4be..f9c99c216a2c 100644
> > > > --- a/drivers/infiniband/hw/irdma/utils.c
> > > > +++ b/drivers/infiniband/hw/irdma/utils.c
> > > > @@ -480,7 +480,7 @@ void irdma_free_cqp_request(struct irdma_cqp
> *cqp,
> > > >  	if (cqp_request->dynamic) {
> > > >  		kfree(cqp_request);
> > > >  	} else {
> > > > -		WRITE_ONCE(cqp_request->request_done, false);
> > > > +		atomic_set(&cqp_request->request_done, false);
> > > >  		cqp_request->callback_fcn = NULL;
> > > >  		cqp_request->waiting = false;
> > > >  		cqp_request->pending = false;
> > > > @@ -515,7 +515,7 @@ irdma_free_pending_cqp_request(struct
> irdma_cqp
> > > *cqp,
> > > >  {
> > > >  	if (cqp_request->waiting) {
> > > >  		cqp_request->compl_info.error = true;
> > > > -		WRITE_ONCE(cqp_request->request_done, true);
> > > > +		atomic_set(&cqp_request->request_done, true);
> > > >  		wake_up(&cqp_request->waitq);
> > > >  	}
> > > >  	wait_event_timeout(cqp->remove_wq,
> > > > @@ -610,7 +610,7 @@ static int irdma_wait_event(struct irdma_pci_f
> *rf,
> > > >  	do {
> > > >  		irdma_cqp_ce_handler(rf, &rf->ccq.sc_cq);
> > > >  		if (wait_event_timeout(cqp_request->waitq,
> > > > -				       READ_ONCE(cqp_request-
> >request_done),
> > > > +				       atomic_read(&cqp_request-
> >request_done),
> > > >
> msecs_to_jiffies(CQP_COMPL_WAIT_TIME_MS)))
> > > >  			break;
> > > >
> > > > --
> > > > 2.31.1
> > > >
> >

Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request->request_done

[Leon Romanovsky]

On Tue, Mar 17, 2026 at 07:27:47PM +0000, Nikolova, Tatyana E wrote:
> 
> 
> > -----Original Message-----
> > From: Leon Romanovsky <leon@kernel.org>
> > Sent: Tuesday, March 17, 2026 8:23 AM
> > To: Czurylo, Krzysztof <krzysztof.czurylo@intel.com>
> > Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Nikolova, Tatyana E
> > <tatyana.e.nikolova@intel.com>
> > Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> > >request_done
> > 
> > On Tue, Mar 17, 2026 at 12:14:21PM +0000, Czurylo, Krzysztof wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Leon Romanovsky <leon@kernel.org>
> > > > Sent: 17 March, 2026 12:13
> > > > To: Nikolova, Tatyana E <tatyana.e.nikolova@intel.com>
> > > > Cc: jgg@nvidia.com; linux-rdma@vger.kernel.org; Czurylo, Krzysztof
> > > > <krzysztof.czurylo@intel.com>
> > > > Subject: Re: [for-next 02/12] RDMA/irdma: Fix data race on cqp_request-
> > > > >request_done
> > > >
> > > > On Mon, Mar 16, 2026 at 01:39:39PM -0500, Tatyana Nikolova wrote:
> > > > > From: Krzysztof Czurylo <krzysztof.czurylo@intel.com>
> > > > >
> > > > > Changes type of request_done flag from bool to atomic_t to fix
> > > > > data race in irdma_complete_cqp_request / irdma_wait_event
> > > > > reported by KCSAN:
> > > >
> > > > Again, this fix is wrong too.
> > >
> > > Could you please elaborate on what is wrong with this fix?
> > > And/or suggest how to fix it properly?
> > >
> > > Please note 'request_done' is _not_ a bitfield and we only do simple
> > > load/store operations on it.  There is no RMW.
> > > Despite this, KCSAN still reports a data race on it.
> > >
> > > Honestly, the original idea was just to change the type from
> > > 'bool' to 'u8'.  This is enough to silence KCSAN, but it is
> > > not clear why.  Perhaps it indicates a bug in KCSAN?
> > 
> > Yes, both u8 and atomic_t behave the same, they can't be interrupted
> > during read/write. This is why KCSAN doesn't warn you.
> > 
> > > I mean, maybe the report below is a false positive?
> > 
> > Sounds like that.
> 
> Leon,
> 
> Are you okay taking the rest of the patches in the series (they apply without the two KCSAN related patches) or you prefer that I resubmit the series?

Sure, let's me to pick them now.

> 
> For this specific patch, are you okay with dropping all atomic changes, but making request_done u8 (currently bool) to silence the KCSAN warning?

In general, I don't like changing code just to silence a tool, but in this
case it is justified. Switching to u8 not only quiets the warning, it also
reduces the size of struct irdma_cqp_request, just run pahole on it.

Thanks

Re: (subset) [for-next 00/12] RDMA/irdma: A few fixes for irdma

[Leon Romanovsky]

On Mon, 16 Mar 2026 13:39:37 -0500, Tatyana Nikolova wrote:
> This series includes a few irdma fixes:
> 
>  - Change request_done type to atomic
>  - Change ah_valid type to atomic
>  - Clean up unnecessary dereference of event->cm_node
>  - Initialize free_qp completion before using it
>  - Harden SQ/RQ depth calculation functions
>  - Update ibqp state to error if QP is already in error state
>  - Fix deadlock during netdev reset with active connections
>  - Return EINVAL for invalid arp index error
>  - Remove a NOP wait_event() in irdma_modify_qp_roce()
>  - Remove reset check from irdma_modify_qp_to_err() to ensure disconnect
> 
> [...]

Applied, thanks!

[01/12] RDMA/irdma: Initialize free_qp completion before using it
        (no commit info)
[04/12] RDMA/irdma: Update ibqp state to error if QP is already in error state
        (no commit info)
[05/12] RDMA/irdma: Remove a NOP wait_event() in irdma_modify_qp_roce()
        (no commit info)
[06/12] RDMA/irdma: Clean up unnecessary dereference of event->cm_node
        (no commit info)
[07/12] RDMA/irdma: Remove reset check from irdma_modify_qp_to_err()
        (no commit info)
[08/12] RDMA/irdma: Fix deadlock during netdev reset with active connections
        (no commit info)
[09/12] RDMA/irdma: Return EINVAL for invalid arp index error
        (no commit info)
[10/12] RDMA/irdma: Harden depth calculation functions
        (no commit info)
[11/12] RDMA/irdma: Provide scratch buffers to firmware for internal use
        (no commit info)
[12/12] RDMA/irdma: Add support for GEN4 hardware
        (no commit info)

Best regards,
-- 
Leon Romanovsky <leon@kernel.org>

Re: (subset) [for-next 00/12] RDMA/irdma: A few fixes for irdma

[Leon Romanovsky]

On Wed, Mar 18, 2026 at 06:24:02AM -0400, Leon Romanovsky wrote:
> 
> On Mon, 16 Mar 2026 13:39:37 -0500, Tatyana Nikolova wrote:
> > This series includes a few irdma fixes:
> > 
> >  - Change request_done type to atomic
> >  - Change ah_valid type to atomic
> >  - Clean up unnecessary dereference of event->cm_node
> >  - Initialize free_qp completion before using it
> >  - Harden SQ/RQ depth calculation functions
> >  - Update ibqp state to error if QP is already in error state
> >  - Fix deadlock during netdev reset with active connections
> >  - Return EINVAL for invalid arp index error
> >  - Remove a NOP wait_event() in irdma_modify_qp_roce()
> >  - Remove reset check from irdma_modify_qp_to_err() to ensure disconnect
> > 
> > [...]
> 
> Applied, thanks!
> 
> [01/12] RDMA/irdma: Initialize free_qp completion before using it
>         (no commit info)
> [04/12] RDMA/irdma: Update ibqp state to error if QP is already in error state
>         (no commit info)
> [05/12] RDMA/irdma: Remove a NOP wait_event() in irdma_modify_qp_roce()
>         (no commit info)
> [06/12] RDMA/irdma: Clean up unnecessary dereference of event->cm_node
>         (no commit info)
> [07/12] RDMA/irdma: Remove reset check from irdma_modify_qp_to_err()
>         (no commit info)
> [08/12] RDMA/irdma: Fix deadlock during netdev reset with active connections
>         (no commit info)
> [09/12] RDMA/irdma: Return EINVAL for invalid arp index error
>         (no commit info)
> [10/12] RDMA/irdma: Harden depth calculation functions
>         (no commit info)

Everything above were applied to wip/leon-for-rc

> [11/12] RDMA/irdma: Provide scratch buffers to firmware for internal use
>         (no commit info)
> [12/12] RDMA/irdma: Add support for GEN4 hardware
>         (no commit info)

These were applied to wip/leon-for-next

Thanks

> 
> Best regards,
> -- 
> Leon Romanovsky <leon@kernel.org>
> 
>