From: Peter Xu <peterx@redhat.com>
To: Yanfei Xu <yanfei.xu@bytedance.com>
Cc: qemu-devel@nongnu.org, farosas@suse.de, lizhijian@fujitsu.com,
Jinpu Wang <jinpu.wang@cloud.ionos.com>
Subject: Re: [PATCH 3/3] migration/rdma: Retry control sends on full queue
Date: Wed, 19 Aug 2026 15:18:31 -0400 [thread overview]
Message-ID: <aoYBhxpSL5w5gQcX@x1.local> (raw)
In-Reply-To: <20260817105117.3072040-4-yanfei.xu@bytedance.com>
On Mon, Aug 17, 2026 at 06:51:17PM +0800, Yanfei Xu wrote:
> RAM writes and control messages share the send queue. If
> outstanding writes fill it, RDMA writes drain a completion and retry,
> but control sends fail the migration.
>
> Drain one outstanding write and retry the control send on ENOMEM.
>
> Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
> ---
> migration/rdma.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/migration/rdma.c b/migration/rdma.c
> index 6e8436ccc1..d1f44a5f55 100644
> --- a/migration/rdma.c
> +++ b/migration/rdma.c
> @@ -1572,9 +1572,19 @@ static int qemu_rdma_post_send_control(RDMAContext *rdma, uint8_t *buf,
> memcpy(wr->control + sizeof(RDMAControlHeader), buf, head->len);
> }
>
> -
> +retry:
> ret = ibv_post_send(rdma->qp, &send_wr, &bad_wr);
>
> + if (ret == ENOMEM && rdma->nb_sent) {
> + ret = qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL);
> + if (ret < 0) {
> + error_setg(errp, "rdma migration: failed to make room for "
> + "control send");
> + return -1;
> + }
> + goto retry;
> + }
Looks also correct, but two questions:
- Should we provide a helper instead of duplicating the WRITE op handling?
I believe only WRITE wrids can be on the fly.
- Could ENOMEM be returned when nb_sent==0? If that check applies to WRITE
path too?
Thanks,
> +
> if (ret > 0) {
> error_setg(errp, "Failed to use post IB SEND for control");
> return -1;
> --
> 2.20.1
>
--
Peter Xu
prev parent reply other threads:[~2026-08-19 19:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-17 10:51 [PATCH 0/3] migration/rdma: Misc RDMA migration fixes Yanfei Xu
2026-08-17 10:51 ` [PATCH 1/3] migration/rdma: Fix write-side shutdown Yanfei Xu
2026-08-19 14:58 ` Fabiano Rosas
2026-08-19 19:19 ` Peter Xu
2026-08-17 10:51 ` [PATCH 2/3] migration/rdma: Post initial receive before accepting Yanfei Xu
2026-08-19 19:16 ` Peter Xu
2026-08-17 10:51 ` [PATCH 3/3] migration/rdma: Retry control sends on full queue Yanfei Xu
2026-08-19 19:18 ` Peter Xu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aoYBhxpSL5w5gQcX@x1.local \
--to=peterx@redhat.com \
--cc=farosas@suse.de \
--cc=jinpu.wang@cloud.ionos.com \
--cc=lizhijian@fujitsu.com \
--cc=qemu-devel@nongnu.org \
--cc=yanfei.xu@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.