CX314A WCE error: WR_FLUSH_ERR

Liu, Changcheng
2019-08-21T12:10:37Z

Hi all,
In one system, it always frequently hit "IBV_WC_WR_FLUSH_ERR" in the WCE(work completion element) polled from completion queue bound with RQ(Receive Queue).
Does anyone has some idea to debug "IBV_WC_WR_FLUSH_ERR" problem?

With CX314A/40Gb NIC, I hit this error when using RC transport type with only Send Operation(IBV_WR_SEND) WR(work request) on SQ(Send Queue).
Every WR only has one SGE(scatter/gather element) and all the SGE on RQ has the same size. The SGE size in SQ WR is not greater than the SGE size in RQ WR.

There’s one explanation about IBV_WC_WR_FLUSH_ERR on page 114 in the "RDMA Aware Networks Programming User Manual" http://www.mellanox.com/related-docs/prod_software/RDMA_Aware_Programming_user_manual.pdf
But I still didn't understand it well. How to trigger this error with a short demo program?
"
IBV_WC_WR_FLUSH_ERR
This event is generated when an invalid remote error is thrown when the responder detects an
invalid request. It may be that the operation is not supported by the request queue or there is
insufficient buffer space to receive the request.
"

B.R.
Changcheng