[PATCH] RDMA/irdma: Bound in-flight CQP ops across deferred completions

[PATCH] RDMA/irdma: Bound in-flight CQP ops across deferred completions

[Sunny Tiwari]

On IRDMA_GEN_3+ devices, CQP operations can complete in two stages:
Stage-1 on the CCQ acknowledges receipt and moves an entry from
cqp->ooo_avail to cqp->ooo_pnd, while Stage-2 signals final completion
via an AEQ event or a subsequent CCQ CQE.

In irdma_sc_ccq_get_cqe_info(), Stage-1 processing advances
cqp->sq_ring.tail so the SQ descriptor slot can be reused once consumed
by hardware, whereas cqp->completed_ops is only incremented on Stage-2.
Because irdma_cqp_ring_full() only checks cqp->sq_ring, a burst of
backlogged commands in dev->cqp_cmd_head allows irdma_process_bh() to
submit more concurrent in-flight CQP commands than the maximum supported
limit (cqp->sq_size - 1) while earlier commands await Stage-2. This
causes CQP completions to be lost and synchronous waiters to time out
with -ETIMEDOUT (-110).

Fix this by:
  - Updating irdma_cqp_ring_full() to bound total in-flight CQP
    operations (cqp->requested_ops - cqp->completed_ops) to
    (cqp->sq_size - 1).
  - Calling irdma_process_bh() at the end of irdma_process_ae_def_cmpl()
    so backlogged commands are drained after Stage-2 AEQ completions
    increment cqp->completed_ops and free in-flight capacity.

Fixes: c7db0abe5f2b ("RDMA/irdma: Add GEN3 CQP support with deferred completions")
Assisted-by: Gemini
Signed-off-by: Sunny Tiwari <sunnytiwari@google.com>
---
 drivers/infiniband/hw/irdma/ctrl.c | 6 +++++-
 drivers/infiniband/hw/irdma/hw.c   | 1 +
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/irdma/ctrl.c b/drivers/infiniband/hw/irdma/ctrl.c
index 335ae3c82e17..99fc9bf4face 100644
--- a/drivers/infiniband/hw/irdma/ctrl.c
+++ b/drivers/infiniband/hw/irdma/ctrl.c
@@ -5331,7 +5331,11 @@ int irdma_sc_static_hmc_pages_allocated(struct irdma_sc_cqp *cqp, u64 scratch,
  */
 static bool irdma_cqp_ring_full(struct irdma_sc_cqp *cqp)
 {
-	return IRDMA_RING_FULL_ERR(cqp->sq_ring);
+	u64 in_flight = cqp->requested_ops -
+			atomic64_read(&cqp->completed_ops);
+
+	return IRDMA_RING_FULL_ERR(cqp->sq_ring) ||
+	       in_flight >= (cqp->sq_size - 1);
 }
 
 /**
diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
index c345cc654256..f678ef8c1d24 100644
--- a/drivers/infiniband/hw/irdma/hw.c
+++ b/drivers/infiniband/hw/irdma/hw.c
@@ -265,6 +265,7 @@ static void irdma_process_ae_def_cmpl(struct irdma_pci_f *rf,
 		irdma_sc_cqp_def_cmpl_ae_handler(&rf->sc_dev, info, false,
 						 &scratch, &sw_def_info);
 	}
+	irdma_process_bh(&rf->sc_dev);
 }
 
 /**
-- 
2.56.0.176.g3575e65682-goog

Re: [PATCH] RDMA/irdma: Bound in-flight CQP ops across deferred completions

[sashiko-bot]

> On IRDMA_GEN_3+ devices, CQP operations can complete in two stages:
> Stage-1 on the CCQ acknowledges receipt and moves an entry from
> cqp->ooo_avail to cqp->ooo_pnd, while Stage-2 signals final completion
> via an AEQ event or a subsequent CCQ CQE.
> 
> In irdma_sc_ccq_get_cqe_info(), Stage-1 processing advances
> cqp->sq_ring.tail so the SQ descriptor slot can be reused once consumed
> by hardware, whereas cqp->completed_ops is only incremented on Stage-2.
> Because irdma_cqp_ring_full() only checks cqp->sq_ring, a burst of
> backlogged commands in dev->cqp_cmd_head allows irdma_process_bh() to
> submit more concurrent in-flight CQP commands than the maximum supported
> limit (cqp->sq_size - 1) while earlier commands await Stage-2. This
> causes CQP completions to be lost and synchronous waiters to time out
> with -ETIMEDOUT (-110).
> 
> Fix this by:
>   - Updating irdma_cqp_ring_full() to bound total in-flight CQP
>     operations (cqp->requested_ops - cqp->completed_ops) to
>     (cqp->sq_size - 1).
>   - Calling irdma_process_bh() at the end of irdma_process_ae_def_cmpl()
>     so backlogged commands are drained after Stage-2 AEQ completions
>     increment cqp->completed_ops and free in-flight capacity.
> 
> Fixes: c7db0abe5f2b ("RDMA/irdma: Add GEN3 CQP support with deferred completions")
> Assisted-by: Gemini
> Signed-off-by: Sunny Tiwari <sunnytiwari@google.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261017052856.3570063-1-sunnytiwari@google.com?part=1