Re: [PATCH for-rc] RDMA/rxe: don't re-execute the current packet after the QP moved to error state

[Zhu Yanjun <yanjun.zhu@linux.dev>]

在 2026/7/11 9:54, Allison Henderson 写道:
> When do_complete() finds the QP in the error state it returns
> RESPST_CHK_RESOURCE.  Before commit 49dc9c1f0c7e ("RDMA/rxe: Cleanup
> reset state handling in rxe_resp.c") this was the flush loop:
> check_resource() had an error-state branch that fetched each remaining
> recv WQE and completed it with IB_WC_WR_FLUSH_ERR, without touching
> the current packet.  That commit removed the error-state branch from
> check_resource() (draining is now done at rxe_receiver() entry) but
> kept the do_complete() error-state return.
> 
> As a result, when a QP moves to the error state while a packet is
> being completed - e.g. an rdma_cm disconnect racing with receive
> processing - the responder state machine loops back into the request
> processing chain with the already-completed packet still in hand:
> check_resource() fetches a fresh recv WQE, execute()/send_data_in()
> copies the same packet payload again, do_complete() posts another
> IB_WC_SUCCESS CQE (qp->resp.status is still 0), and control returns
> to the error-state check.  The loop re-executes the same packet once
> per posted recv WQE (observed: ~1000 duplicate IB_WC_SUCCESS
> completions of one SEND, one per ~8us, matching the RQ occupancy)
> until the RQ is exhausted, after which qp->resp.wqe is NULL and
> send_data_in() dereferences it:
> 
>    BUG: kernel NULL pointer dereference, address: 0000000000000014
>    Workqueue: rxe_wq do_work
>    RIP: copy_data+0x29/0x1f0
>    Call Trace:
>     send_data_in+0x25/0x50
>     rxe_receiver+0xf36/0x1dd0
> 
> The duplicate completions are indistinguishable from real receives to
> the ULP.  During an rds stress test, the message was accepted as new and
> delivered the same datagram to user space hundreds of times, corrupting
> the stream; any ULP that relies on RC exactly-once delivery is affected.
> 
> A live packet reaching the error-state check in do_complete() has
> been executed and completed exactly once and must be consumed, not
> re-processed.  Return RESPST_CLEANUP for it (dequeue and free); keep
> returning RESPST_CHK_RESOURCE for the pkt == NULL case.
> 
> Fixes: 49dc9c1f0c7e ("RDMA/rxe: Cleanup reset state handling in rxe_resp.c")
> Assisted-by: Claude-Code:claude-fable-5
> Signed-off-by: Allison Henderson <achender@kernel.org>

The fix is correct and strictly adheres to the InfiniBand RC service
guarantees. When a QP transitions to the error state while processing
a live packet, returning RESPST_CLEANUP ensures the executed packet
is properly consumed rather than improperly re-injected into the request
chain. This accurately prevents both data stream corruption from
duplicate completions and the kernel panic caused by RQ exhaustion.

Thanks a lot.
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>

Zhu Yanjun

> ---
>   drivers/infiniband/sw/rxe/rxe_resp.c | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c
> index d8cbdfa70cdbd..02b16e2b49b8f 100644
> --- a/drivers/infiniband/sw/rxe/rxe_resp.c
> +++ b/drivers/infiniband/sw/rxe/rxe_resp.c
> @@ -1217,7 +1217,14 @@ static enum resp_states do_complete(struct rxe_qp *qp,
>   	spin_lock_irqsave(&qp->state_lock, flags);
>   	if (unlikely(qp_state(qp) == IB_QPS_ERR)) {
>   		spin_unlock_irqrestore(&qp->state_lock, flags);
> -		return RESPST_CHK_RESOURCE;
> +		/* The packet was executed and completed before the QP
> +		 * moved to ERROR; it must be consumed exactly once.
> +		 * Re-entering the request chain with the stale packet
> +		 * would copy it into every remaining recv WQE as a new
> +		 * completion.  Remaining WQEs are flushed by the drain
> +		 * path at rxe_receiver() entry.
> +		 */
> +		return pkt ? RESPST_CLEANUP : RESPST_CHK_RESOURCE;
>   	}
>   	spin_unlock_irqrestore(&qp->state_lock, flags);
>