public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@nvidia.com>
To: Daisuke Matsuda <dskmtsd@gmail.com>
Cc: linux-rdma@vger.kernel.org, leon@kernel.org,
	zyjzyj2000@gmail.com, yanjun.zhu@linux.dev,
	philipp.reisner@linbit.com
Subject: Re: [PATCH for-rc v1] RDMA/rxe: Avoid CQ polling hang triggered by CQ resize
Date: Mon, 25 Aug 2025 15:10:53 -0300	[thread overview]
Message-ID: <20250825181053.GA2085854@nvidia.com> (raw)
In-Reply-To: <20250817123752.153735-1-dskmtsd@gmail.com>

On Sun, Aug 17, 2025 at 12:37:52PM +0000, Daisuke Matsuda wrote:
> When running the test_resize_cq testcase from rdma-core, polling a
> completion queue from userspace may occasionally hang and eventually fail
> with a timeout:
> =====
> ERROR: test_resize_cq (tests.test_cq.CQTest.test_resize_cq)
> Test resize CQ, start with specific value and then increase and decrease
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>     File "/root/deb/rdma-core/tests/test_cq.py", line 135, in test_resize_cq
>       u.poll_cq(self.client.cq)
>     File "/root/deb/rdma-core/tests/utils.py", line 687, in poll_cq
>       wcs = _poll_cq(cq, count, data)
>             ^^^^^^^^^^^^^^^^^^^^^^^^^
>     File "/root/deb/rdma-core/tests/utils.py", line 669, in _poll_cq
>       raise PyverbsError(f'Got timeout on polling ({count} CQEs remaining)')
> pyverbs.pyverbs_error.PyverbsError: Got timeout on polling (1 CQEs
> remaining)
> =====
> 
> The issue is caused when rxe_cq_post() fails to post a CQE due to the queue
> being temporarily full, and the CQE is effectively lost. To mitigate this,
> add a bounded busy-wait with fallback rescheduling so that CQE does not get
> lost.

Nothing should spin like this, that is not right.

The CQE queue is intended to be properly sized for the workload and I
seem to remember overflowing a CQE can just ERR the QP.

Alternatively you can drop the packet and do nothing.

But not spin hoping something emptys it.

Jason

  parent reply	other threads:[~2025-08-25 18:10 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-08-17 12:37 [PATCH for-rc v1] RDMA/rxe: Avoid CQ polling hang triggered by CQ resize Daisuke Matsuda
2025-08-18  4:44 ` Zhu Yanjun
2025-08-19 15:15   ` Daisuke Matsuda
2025-08-21  3:12     ` Zhu Yanjun
2025-08-23  4:19       ` Daisuke Matsuda
2025-08-23  5:22         ` Zhu Yanjun
2025-08-25 18:10 ` Jason Gunthorpe [this message]
2025-08-27 11:14   ` Daisuke Matsuda
2025-08-27 12:04     ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250825181053.GA2085854@nvidia.com \
    --to=jgg@nvidia.com \
    --cc=dskmtsd@gmail.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=philipp.reisner@linbit.com \
    --cc=yanjun.zhu@linux.dev \
    --cc=zyjzyj2000@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox