Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
* [PATCH for-rc] RDMA/efa: Fix race between admin command timeout and completion
@ 2026-09-01 11:30 Yonatan Nachum
  2026-09-02  7:58 ` Gal Pressman
  0 siblings, 1 reply; 2+ messages in thread
From: Yonatan Nachum @ 2026-09-01 11:30 UTC (permalink / raw)
  To: jgg, leon, linux-rdma
  Cc: mrgolin, sleybo, matua, gal.pressman, Yonatan Nachum, Tom Sela

When an admin command times out, there is a potential race between the
interrupt handler processing the completion and the timeout path
releasing the completion context. The interrupt handler may observe the
command as still submitted and attempt to complete it while the timeout
path is concurrently freeing the context.

Fix this by introducing a timed-out state in the completion context. On
timeout, both the polling and interrupt wait paths set the status to
timed-out under the CQ lock before returning. The completion handler
only process commands in submitted state, eliminating the race.

Fixes: 0420e542569b ("RDMA/efa: Implement functions that submit and complete admin commands")
Reviewed-by: Tom Sela <tomsela@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
---
 drivers/infiniband/hw/efa/efa_com.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/efa/efa_com.c b/drivers/infiniband/hw/efa/efa_com.c
index 583b1cf0d721..a91de9283b6b 100644
--- a/drivers/infiniband/hw/efa/efa_com.c
+++ b/drivers/infiniband/hw/efa/efa_com.c
@@ -41,6 +41,7 @@ enum efa_cmd_status {
 	EFA_CMD_ALLOCATED,
 	EFA_CMD_SUBMITTED,
 	EFA_CMD_COMPLETED,
+	EFA_CMD_TIMED_OUT,
 };
 
 struct efa_comp_ctx {
@@ -605,6 +606,10 @@ static int efa_com_wait_and_process_admin_cq_polling(struct efa_comp_ctx *comp_c
 			break;
 
 		if (time_is_before_jiffies(timeout)) {
+			spin_lock_irqsave(&aq->cq.lock, flags);
+			comp_ctx->status = EFA_CMD_TIMED_OUT;
+			spin_unlock_irqrestore(&aq->cq.lock, flags);
+
 			ibdev_err_ratelimited(
 				aq->efa_dev,
 				"Wait for completion (polling) timeout\n");
@@ -639,7 +644,6 @@ static int efa_com_wait_and_process_admin_cq_interrupts(struct efa_comp_ctx *com
 	if (comp_ctx->status == EFA_CMD_SUBMITTED) {
 		spin_lock_irqsave(&aq->cq.lock, flags);
 		efa_com_handle_admin_completion(aq);
-		spin_unlock_irqrestore(&aq->cq.lock, flags);
 
 		atomic64_inc(&aq->stats.no_completion);
 
@@ -660,6 +664,9 @@ static int efa_com_wait_and_process_admin_cq_interrupts(struct efa_comp_ctx *com
 				comp_ctx->cmd_id, aq->sq.pc, aq->sq.cc,
 				aq->cq.cc);
 
+		comp_ctx->status = EFA_CMD_TIMED_OUT;
+		spin_unlock_irqrestore(&aq->cq.lock, flags);
+
 		clear_bit(EFA_AQ_STATE_RUNNING_BIT, &aq->state);
 		return -ETIME;
 	}
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH for-rc] RDMA/efa: Fix race between admin command timeout and completion
  2026-09-01 11:30 [PATCH for-rc] RDMA/efa: Fix race between admin command timeout and completion Yonatan Nachum
@ 2026-09-02  7:58 ` Gal Pressman
  0 siblings, 0 replies; 2+ messages in thread
From: Gal Pressman @ 2026-09-02  7:58 UTC (permalink / raw)
  To: Yonatan Nachum, jgg, leon, linux-rdma; +Cc: mrgolin, sleybo, matua, Tom Sela

On 01/09/2026 14:30, Yonatan Nachum wrote:
> When an admin command times out, there is a potential race between the
> interrupt handler processing the completion and the timeout path
> releasing the completion context. The interrupt handler may observe the
> command as still submitted and attempt to complete it while the timeout
> path is concurrently freeing the context.
> 
> Fix this by introducing a timed-out state in the completion context. On
> timeout, both the polling and interrupt wait paths set the status to
> timed-out under the CQ lock before returning. The completion handler
> only process commands in submitted state, eliminating the race.
> 
> Fixes: 0420e542569b ("RDMA/efa: Implement functions that submit and complete admin commands")
> Reviewed-by: Tom Sela <tomsela@amazon.com>
> Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
> ---
>  drivers/infiniband/hw/efa/efa_com.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/efa/efa_com.c b/drivers/infiniband/hw/efa/efa_com.c
> index 583b1cf0d721..a91de9283b6b 100644
> --- a/drivers/infiniband/hw/efa/efa_com.c
> +++ b/drivers/infiniband/hw/efa/efa_com.c
> @@ -41,6 +41,7 @@ enum efa_cmd_status {
>  	EFA_CMD_ALLOCATED,
>  	EFA_CMD_SUBMITTED,
>  	EFA_CMD_COMPLETED,
> +	EFA_CMD_TIMED_OUT,
>  };
>  
>  struct efa_comp_ctx {
> @@ -605,6 +606,10 @@ static int efa_com_wait_and_process_admin_cq_polling(struct efa_comp_ctx *comp_c
>  			break;
>  
>  		if (time_is_before_jiffies(timeout)) {
> +			spin_lock_irqsave(&aq->cq.lock, flags);
> +			comp_ctx->status = EFA_CMD_TIMED_OUT;
> +			spin_unlock_irqrestore(&aq->cq.lock, flags);

This smells bad, a few lines above 'comp_ctx->status' is being read
without a lock. Can't it change during this window?

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-09-02  7:58 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-01 11:30 [PATCH for-rc] RDMA/efa: Fix race between admin command timeout and completion Yonatan Nachum
2026-09-02  7:58 ` Gal Pressman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox