[PATCH v3 0/4] migration/rdma: Allow multiple writes per chunk

[PATCH v3 0/4] migration/rdma: Allow multiple writes per chunk

[Yanfei Xu]

This series improves RDMA migration throughput during iterative precopy
and the iterable stop-copy phase by allowing multiple writes to be in
flight for the same registration chunk.

RDMA tracks RAM in registration chunks.  During the final iterable
transfer, workloads can leave many scattered 4 KiB dirty pages, with
multiple pages falling into the same chunk.  The current code waits for
an earlier write to a chunk to complete before posting the next one.
These serial completion waits can leave send queue capacity unused and
limit RDMA bandwidth, increasing downtime.

A ram_save_iterate() call can stop due to rate limiting before the RAM
scanner completes a full round.  Therefore, it is not the correct
boundary for ordering page versions.

Add a return-valued notifier at the point where the RAM scanner wraps to
the first RAMBlock.  Convert multifd's existing per-round
synchronization to use this notifier, and register an RDMA notifier that
flushes buffered writes and drains outstanding completions at the same
boundary.

This provides the explicit completion barrier before a newer version of
a page can be sent.  The per-chunk wait is therefore unnecessary and is
removed, together with the now-unused transit bitmap.

The test configuration is:

  RDMA chunk size:       32 MiB
  Guest:                 32 vCPUs, 128 GiB RAM
  Test runs:             10
  Maximum HCA bandwidth: 100 Gbps
  Workload:              idle


Average result(pin-all=true)    Before       After
  pin-all is true
  Downtime                      396.7 ms     345.7 ms        ~12.9% improve
  Final iterable bandwidth      9952.0 MiB/s 11639.10 MiB/s  ~17.0% improve


Average results(pin-all=false)  Before       After
  pin-all is false
  Downtime                      248.7 ms     191.1 ms        ~23.2% improve
  Final iterable bandwidth      5605.1 MiB/s 9826.0 MiB/s    ~75.3% improve

"Final iterable bandwidth" is the bandwidth measured during
qemu_savevm_state_complete_precopy_iterable().  Non-iterable data is
not transferred using RDMA Write.


Changes in v3:

  - Optimze comments about RAMRoundNotifyData in patch1
  - Collect Reviewed-by tags

Changes in v2 (Suggested by Peter):

  - Add a generic RAM round synchronization notifier.
  - Drain RDMA writes at the RAM scan-round boundary.
  - Remove the per-chunk wait and the unused transit bitmap.

Yanfei Xu (4):
  migration/ram: Add RAM round synchronization notifiers
  migration/rdma: Drain writes at RAM round boundaries
  migration/rdma: Allow multiple in-flight writes per chunk
  migration/rdma: Remove unused transit bitmap

 migration/multifd-nocomp.c | 24 ++++++++++
 migration/ram.c            | 45 +++++++++++++++---
 migration/ram.h            | 12 +++++
 migration/rdma.c           | 97 ++++++++++++++++++++------------------
 migration/trace-events     |  5 +-
 5 files changed, 128 insertions(+), 55 deletions(-)

-- 
2.20.1

[PATCH v3 1/4] migration/ram: Add RAM round synchronization notifiers

[Yanfei Xu]

find_dirty_block() directly invokes multifd's per-round synchronization
when the RAM scan wraps.  Other transports need the same boundary to
order page versions.

Add a return-valued notifier for RAM round completion and register
multifd's existing synchronization through it.  This keeps the current
behavior while allowing other transports to synchronize at the same
boundary.

Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 migration/multifd-nocomp.c | 24 ++++++++++++++++++++
 migration/ram.c            | 45 +++++++++++++++++++++++++++++++++-----
 migration/ram.h            | 12 ++++++++++
 3 files changed, 75 insertions(+), 6 deletions(-)

diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c
index 9f7a792fa7..3067c19cc6 100644
--- a/migration/multifd-nocomp.c
+++ b/migration/multifd-nocomp.c
@@ -19,6 +19,7 @@
 #include "multifd-colo.h"
 #include "options.h"
 #include "migration.h"
+#include "ram.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
 #include "qemu/error-report.h"
@@ -27,6 +28,28 @@
 
 static MultiFDSendData *multifd_ram_send;
 
+static int multifd_ram_round_notify(NotifierWithReturn *n G_GNUC_UNUSED,
+                                    void *data, Error **errp)
+{
+    RAMRoundNotifyData *round_data = data;
+    QEMUFile *f = round_data->file;
+    int ret;
+
+    if (!multifd_ram_sync_per_round()) {
+        return 0;
+    }
+
+    ret = multifd_ram_flush_and_sync(f);
+    if (ret < 0) {
+        error_setg(errp, "multifd RAM round synchronization failed");
+    }
+    return ret;
+}
+
+static NotifierWithReturn multifd_ram_round_notifier = {
+    .notify = multifd_ram_round_notify,
+};
+
 void multifd_ram_payload_alloc(MultiFDPages_t *pages)
 {
     pages->offset = g_new0(ram_addr_t, multifd_ram_page_count());
@@ -471,6 +494,7 @@ static const MultiFDMethods multifd_nocomp_ops = {
 static void multifd_nocomp_register(void)
 {
     multifd_register_ops(MULTIFD_COMPRESSION_NONE, &multifd_nocomp_ops);
+    ram_round_add_notifier(&multifd_ram_round_notifier);
 }
 
 migration_init(multifd_nocomp_register);
diff --git a/migration/ram.c b/migration/ram.c
index 8918b2f03b..2dd898b305 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -440,6 +440,8 @@ typedef struct RAMState RAMState;
 static RAMState *ram_state;
 
 static NotifierWithReturnList precopy_notifier_list;
+static NotifierWithReturnList ram_round_notifier_list =
+    NOTIFIER_WITH_RETURN_LIST_INITIALIZER(ram_round_notifier_list);
 
 /* Whether postcopy has queued requests? */
 static bool postcopy_has_request(RAMState *rs)
@@ -470,6 +472,38 @@ int precopy_notify(PrecopyNotifyReason reason, Error **errp)
     return notifier_with_return_list_notify(&precopy_notifier_list, &pnd, errp);
 }
 
+void ram_round_add_notifier(NotifierWithReturn *n)
+{
+    notifier_with_return_list_add(&ram_round_notifier_list, n);
+}
+
+void ram_round_remove_notifier(NotifierWithReturn *n)
+{
+    notifier_with_return_remove(n);
+}
+
+static int ram_round_notify(QEMUFile *f)
+{
+    RAMRoundNotifyData data = {
+        .file = f,
+    };
+    Error *local_err = NULL;
+    int ret;
+
+    ret = notifier_with_return_list_notify(&ram_round_notifier_list, &data,
+                                           &local_err);
+    if (ret) {
+        if (local_err) {
+            error_report_err(local_err);
+        } else {
+            error_report("RAM round notifier failed: %d", ret);
+        }
+        return -1;
+    }
+
+    return 0;
+}
+
 uint64_t ram_bytes_remaining(void)
 {
     return ram_state ? (ram_state->migration_dirty_pages * TARGET_PAGE_SIZE) :
@@ -1380,12 +1414,11 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss)
         pss->page = 0;
         pss->block = QLIST_NEXT_RCU(pss->block, next);
         if (!pss->block) {
-            if (multifd_ram_sync_per_round()) {
-                QEMUFile *f = rs->pss[RAM_CHANNEL_PRECOPY].pss_channel;
-                int ret = multifd_ram_flush_and_sync(f);
-                if (ret < 0) {
-                    return ret;
-                }
+            QEMUFile *f = rs->pss[RAM_CHANNEL_PRECOPY].pss_channel;
+            int ret = ram_round_notify(f);
+
+            if (ret < 0) {
+                return ret;
             }
 
             /* Hit the end of the list */
diff --git a/migration/ram.h b/migration/ram.h
index 41697a7599..84f10087e8 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -31,6 +31,7 @@
 
 #include "qapi/qapi-types-migration.h"
 #include "exec/cpu-common.h"
+#include "qemu/notify.h"
 #include "system/ram_addr.h"
 #include "io/channel.h"
 
@@ -95,6 +96,17 @@ void ram_handle_zero(void *host, uint64_t size);
 void ram_transferred_add(uint64_t bytes);
 void ram_release_page(const char *rbname, uint64_t offset);
 
+/*
+ * The RAM round notifier is invoked whenever the migration RAM core
+ * completes one full scan of guest memory.
+ */
+typedef struct RAMRoundNotifyData {
+    QEMUFile *file;
+} RAMRoundNotifyData;
+
+void ram_round_add_notifier(NotifierWithReturn *n);
+void ram_round_remove_notifier(NotifierWithReturn *n);
+
 int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr);
 bool ramblock_recv_bitmap_test_byte_offset(RAMBlock *rb, uint64_t byte_offset);
 void ramblock_recv_bitmap_set(RAMBlock *rb, void *host_addr);
-- 
2.20.1

[PATCH v3 2/4] migration/rdma: Drain writes at RAM round boundaries

[Yanfei Xu]

rdma_registration_stop() runs after every ram_save_iterate() call, which
can stop before a complete RAM scan because of rate limiting.  Draining
there ties the completion barrier to a SaveVM section rather than the
point where the scan wraps and a newer page version can be sent.

Register an RDMA RAM round notifier that drains all outstanding writes.
Keep posting the buffered tail in rdma_registration_stop() so it remains
ordered before REGISTER_FINISHED.

Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 migration/rdma.c | 58 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 47 insertions(+), 11 deletions(-)

diff --git a/migration/rdma.c b/migration/rdma.c
index b8a0e39d2a..16e9c6c30e 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2627,21 +2627,16 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc,
     return done;
 }
 
-/*
- * Block until all the outstanding chunks have been delivered by the hardware.
- */
-static int qemu_rdma_drain_cq(RDMAContext *rdma)
+/* Block until all outstanding writes have been delivered by the hardware. */
+static int qemu_rdma_drain_cq(RDMAContext *rdma, Error **errp)
 {
-    Error *err = NULL;
-
-    if (qemu_rdma_write_flush(rdma, &err) < 0) {
-        error_report_err(err);
+    if (qemu_rdma_write_flush(rdma, errp) < 0) {
         return -1;
     }
 
     while (rdma->nb_sent) {
         if (qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL) < 0) {
-            error_report("rdma migration: complete polling error!");
+            error_setg(errp, "rdma migration: completion polling error");
             return -1;
         }
     }
@@ -2649,6 +2644,38 @@ static int qemu_rdma_drain_cq(RDMAContext *rdma)
     return 0;
 }
 
+static int rdma_ram_round_notify(NotifierWithReturn *n G_GNUC_UNUSED,
+                                 void *data, Error **errp)
+{
+    RAMRoundNotifyData *round_data = data;
+    QEMUFile *f;
+    QIOChannelRDMA *rioc;
+    RDMAContext *rdma;
+
+    if (!migrate_rdma()) {
+        return 0;
+    }
+
+    f = round_data->file;
+    RCU_READ_LOCK_GUARD();
+    rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
+    rdma = qatomic_rcu_read(&rioc->rdmaout);
+    if (!rdma) {
+        error_setg(errp, "RDMA output context is not set");
+        return -1;
+    }
+
+    if (rdma_errored(rdma)) {
+        error_setg(errp, "RDMA is in an error state");
+        return -1;
+    }
+
+    return qemu_rdma_drain_cq(rdma, errp);
+}
+
+static NotifierWithReturn rdma_ram_round_notifier = {
+    .notify = rdma_ram_round_notify,
+};
 
 static int qio_channel_rdma_set_blocking(QIOChannel *ioc,
                                          bool blocking,
@@ -3549,7 +3576,8 @@ int rdma_registration_start(QEMUFile *f, uint64_t flags)
 
 /*
  * Inform dest that dynamic registrations are done for now.
- * First, flush writes, if any.
+ * Post any buffered write before sending the control marker. Outstanding
+ * writes are drained at the RAM scan round boundary.
  */
 int rdma_registration_stop(QEMUFile *f, uint64_t flags)
 {
@@ -3575,9 +3603,10 @@ int rdma_registration_stop(QEMUFile *f, uint64_t flags)
     }
 
     qemu_fflush(f);
-    ret = qemu_rdma_drain_cq(rdma);
+    ret = qemu_rdma_write_flush(rdma, &err);
 
     if (ret < 0) {
+        error_report_err(err);
         goto err;
     }
 
@@ -3711,6 +3740,13 @@ static void qio_channel_rdma_register_types(void)
 
 type_init(qio_channel_rdma_register_types);
 
+static void rdma_register_migration_notifiers(void)
+{
+    ram_round_add_notifier(&rdma_ram_round_notifier);
+}
+
+migration_init(rdma_register_migration_notifiers);
+
 static QIOChannel *rdma_new_input(RDMAContext *rdma)
 {
     QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(object_new(TYPE_QIO_CHANNEL_RDMA));
-- 
2.20.1

[PATCH v3 3/4] migration/rdma: Allow multiple in-flight writes per chunk

[Yanfei Xu]

qemu_rdma_write_one() waits for an earlier write that shares the same
registration chunk, even when the dirty ranges do not overlap.  This
serializes sparse writes and leaves send queue capacity unused.

RAM round notifiers now provide the completion barrier before a newer
page version can be sent, and the obsolete unregister path no longer
requires per-chunk serialization.  Remove the wait.  The global nb_sent
count continues to manage CQ and send queue resources.

Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 migration/rdma.c       | 19 +------------------
 migration/trace-events |  1 -
 2 files changed, 1 insertion(+), 19 deletions(-)

diff --git a/migration/rdma.c b/migration/rdma.c
index 16e9c6c30e..6f9b474e29 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -1740,7 +1740,7 @@ static int qemu_rdma_write_one(RDMAContext *rdma,
     struct ibv_sge sge;
     struct ibv_send_wr send_wr = { 0 };
     struct ibv_send_wr *bad_wr;
-    int reg_result_idx, ret, count = 0;
+    int reg_result_idx, ret;
     uint64_t chunk, chunks;
     uint64_t chunk_size = migrate_rdma_chunk_size();
     uint8_t *chunk_start, *chunk_end;
@@ -1772,23 +1772,6 @@ retry:
 
     chunk_end = ram_chunk_end(block, chunk + chunks);
 
-
-    while (test_bit(chunk, block->transit_bitmap)) {
-        (void)count;
-        trace_rdma_write_one_block(count++, current_index, chunk,
-                sge.addr, length, rdma->nb_sent, block->nb_chunks);
-
-        ret = qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL);
-
-        if (ret < 0) {
-            error_setg(errp, "Failed to Wait for previous write to complete "
-                    "block %d chunk %" PRIu64
-                    " current %" PRIu64 " len %" PRIu64 " %d",
-                    current_index, chunk, sge.addr, length, rdma->nb_sent);
-            return -1;
-        }
-    }
-
     if (!rdma->pin_all) {
         if (!block->remote_keys[chunk]) {
             /*
diff --git a/migration/trace-events b/migration/trace-events
index a9b02b3113..70d083f3fe 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -243,7 +243,6 @@ rdma_register_odp_mr(const char *name) "Try to register On-Demand Paging memory
 rdma_advise_mr(const char *name, uint32_t len, uint64_t addr, const char *res) "Try to advise block %s prefetch at %" PRIu32 "@0x%" PRIx64 ": %s"
 rdma_resolve_host_trying(const char *host, const char *ip) "Trying %s => %s"
 rdma_write_flush(int sent) "sent total: %d"
-rdma_write_one_block(int count, int block, uint64_t chunk, uint64_t current, uint64_t len, int nb_sent, int nb_chunks) "(%d) Not clobbering: block: %d chunk %" PRIu64 " current %" PRIu64 " len %" PRIu64 " %d %d"
 rdma_write_one_post(uint64_t chunk, long addr, long remote, uint32_t len) "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, bytes %" PRIu32
 rdma_write_one_queue_full(void) ""
 rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "Received registration result: my key: 0x%x their key 0x%x, chunk %" PRIu64
-- 
2.20.1

[PATCH v3 4/4] migration/rdma: Remove unused transit bitmap

[Yanfei Xu]

After the per-chunk write wait is removed, transit_bitmap is no longer
read.  The remaining set and clear operations only maintain unused
state.

Remove the bitmap and simplify the block lifecycle tracepoints.

Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 migration/rdma.c       | 20 +++-----------------
 migration/trace-events |  4 ++--
 2 files changed, 5 insertions(+), 19 deletions(-)

diff --git a/migration/rdma.c b/migration/rdma.c
index 6f9b474e29..7a884ecb6e 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -30,7 +30,6 @@
 #include "qemu/module.h"
 #include "qemu/rcu.h"
 #include "qemu/sockets.h"
-#include "qemu/bitmap.h"
 #include "qemu/coroutine.h"
 #include "system/memory.h"
 #include <sys/socket.h>
@@ -197,7 +196,6 @@ typedef struct RDMALocalBlock {
     int            index;           /* which block are we */
     unsigned int   src_index;       /* (Only used on dest) */
     int            nb_chunks;
-    unsigned long *transit_bitmap;
 } RDMALocalBlock;
 
 /*
@@ -571,8 +569,6 @@ static void rdma_add_block(RDMAContext *rdma, const char *block_name,
     block->index = local->nb_blocks;
     block->src_index = ~0U; /* Filled in by the receipt of the block list */
     block->nb_chunks = ram_chunk_index(host_addr, host_addr + length) + 1UL;
-    block->transit_bitmap = bitmap_new(block->nb_chunks);
-    bitmap_clear(block->transit_bitmap, 0, block->nb_chunks);
     block->remote_keys = g_new0(uint32_t, block->nb_chunks);
 
     if (rdma->blockmap) {
@@ -583,8 +579,6 @@ static void rdma_add_block(RDMAContext *rdma, const char *block_name,
                          (uintptr_t) block->local_host_addr,
                          block->offset, block->length,
                          (uintptr_t) (block->local_host_addr + block->length),
-                         BITS_TO_LONGS(block->nb_chunks) *
-                             sizeof(unsigned long) * 8,
                          block->nb_chunks);
 
     local->nb_blocks++;
@@ -654,9 +648,6 @@ static void rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block)
         block->mr = NULL;
     }
 
-    g_free(block->transit_bitmap);
-    block->transit_bitmap = NULL;
-
     g_free(block->remote_keys);
     block->remote_keys = NULL;
 
@@ -693,9 +684,8 @@ static void rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block)
 
     trace_rdma_delete_block(block, (uintptr_t)block->local_host_addr,
                            block->offset, block->length,
-                            (uintptr_t)(block->local_host_addr + block->length),
-                           BITS_TO_LONGS(block->nb_chunks) *
-                               sizeof(unsigned long) * 8, block->nb_chunks);
+                           (uintptr_t)(block->local_host_addr + block->length),
+                           block->nb_chunks);
 
     g_free(old);
 
@@ -1221,8 +1211,6 @@ static int qemu_rdma_poll(RDMAContext *rdma, struct ibv_cq *cq,
                                    index, chunk, block->local_host_addr,
                                    (void *)(uintptr_t)block->remote_host_addr);
 
-        clear_bit(chunk, block->transit_bitmap);
-
         if (rdma->nb_sent > 0) {
             rdma->nb_sent--;
         }
@@ -1877,8 +1865,7 @@ retry:
     /*
      * Encode the ram block index and chunk within this wrid.
      * We will use this information at the time of completion
-     * to figure out which bitmap to check against and then which
-     * chunk in the bitmap to look for.
+     * to identify the completed write in trace output.
      */
     send_wr.wr_id = qemu_rdma_make_wrid(RDMA_WRID_RDMA_WRITE,
                                         current_index, chunk);
@@ -1916,7 +1903,6 @@ retry:
         return -1;
     }
 
-    set_bit(chunk, block->transit_bitmap);
     qatomic_add(&mig_stats.normal_pages, sge.length / qemu_target_page_size());
     /*
      * We are adding to transferred the amount of data written, but no
diff --git a/migration/trace-events b/migration/trace-events
index 70d083f3fe..4b3239636c 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -249,9 +249,9 @@ rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "Received reg
 rdma_write_one_sendreg(uint64_t chunk, int len, int index, int64_t offset) "Sending registration request chunk %" PRIu64 " for %d bytes, index: %d, offset: %" PRId64
 rdma_write_one_top(uint64_t chunks, uint64_t size) "Writing %" PRIu64 " chunks, (%" PRIu64 " MB)"
 rdma_write_one_zero(uint64_t chunk, int len, int index, int64_t offset) "Entire chunk is zero, sending compress: %" PRIu64 " for %d bytes, index: %d, offset: %" PRId64
-rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Added Block: '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d"
+rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, int chunks) "Added Block: '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " chunks %d"
 rdma_block_notification_handle(const char *name, int index) "%s at %d"
-rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Deleted Block: %p, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d"
+rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, int chunks) "Deleted Block: %p, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " chunks %d"
 rdma_registration_handle_compress(int64_t length, int index, int64_t offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64
 rdma_registration_handle_finished(void) ""
 rdma_registration_handle_ram_blocks(void) ""
-- 
2.20.1

Re: [PATCH v3 0/4] migration/rdma: Allow multiple writes per chunk

[Jinpu Wang]

Hi Yanfei,

FYI, I ran a quick migration test on our test machines and didn't
notice any issues with the v2 patchset. We saw a 10–12% downtime
reduction on our setup (100 Gb/s dual-port IB fabric, 100 GB VM, idle
load).

As v3 don't change the implementation.
Tested-by: Jack Wang <jinpu.wang@cloud.ionos.com>

Best regards,
Jinpu Wang

Re: [PATCH v3 0/4] migration/rdma: Allow multiple writes per chunk

[Yanfei Xu]

On 2026/9/14 13:04, Jinpu Wang wrote:
> Hi Yanfei,
> 
> FYI, I ran a quick migration test on our test machines and didn't
> notice any issues with the v2 patchset. We saw a 10–12% downtime
> reduction on our setup (100 Gb/s dual-port IB fabric, 100 GB VM, idle
> load).
> 
> As v3 don't change the implementation.
> Tested-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Thanks Jinpu!

Yanfei

> 
> Best regards,
> Jinpu Wang

Re: [PATCH v3 0/4] migration/rdma: Allow multiple writes per chunk

[Fabiano Rosas]

"Yanfei Xu" <yanfei.xu@bytedance.com> writes:

> This series improves RDMA migration throughput during iterative precopy
> and the iterable stop-copy phase by allowing multiple writes to be in
> flight for the same registration chunk.
>
> RDMA tracks RAM in registration chunks.  During the final iterable
> transfer, workloads can leave many scattered 4 KiB dirty pages, with
> multiple pages falling into the same chunk.  The current code waits for
> an earlier write to a chunk to complete before posting the next one.
> These serial completion waits can leave send queue capacity unused and
> limit RDMA bandwidth, increasing downtime.
>
> A ram_save_iterate() call can stop due to rate limiting before the RAM
> scanner completes a full round.  Therefore, it is not the correct
> boundary for ordering page versions.
>
> Add a return-valued notifier at the point where the RAM scanner wraps to
> the first RAMBlock.  Convert multifd's existing per-round
> synchronization to use this notifier, and register an RDMA notifier that
> flushes buffered writes and drains outstanding completions at the same
> boundary.
>
> This provides the explicit completion barrier before a newer version of
> a page can be sent.  The per-chunk wait is therefore unnecessary and is
> removed, together with the now-unused transit bitmap.
>
> The test configuration is:
>
>   RDMA chunk size:       32 MiB
>   Guest:                 32 vCPUs, 128 GiB RAM
>   Test runs:             10
>   Maximum HCA bandwidth: 100 Gbps
>   Workload:              idle
>
>
> Average result(pin-all=true)    Before       After
>   pin-all is true
>   Downtime                      396.7 ms     345.7 ms        ~12.9% improve
>   Final iterable bandwidth      9952.0 MiB/s 11639.10 MiB/s  ~17.0% improve
>
>
> Average results(pin-all=false)  Before       After
>   pin-all is false
>   Downtime                      248.7 ms     191.1 ms        ~23.2% improve
>   Final iterable bandwidth      5605.1 MiB/s 9826.0 MiB/s    ~75.3% improve
>
> "Final iterable bandwidth" is the bandwidth measured during
> qemu_savevm_state_complete_precopy_iterable().  Non-iterable data is
> not transferred using RDMA Write.
>
>
> Changes in v3:
>
>   - Optimze comments about RAMRoundNotifyData in patch1
>   - Collect Reviewed-by tags
>
> Changes in v2 (Suggested by Peter):
>
>   - Add a generic RAM round synchronization notifier.
>   - Drain RDMA writes at the RAM scan-round boundary.
>   - Remove the per-chunk wait and the unused transit bitmap.
>
> Yanfei Xu (4):
>   migration/ram: Add RAM round synchronization notifiers
>   migration/rdma: Drain writes at RAM round boundaries
>   migration/rdma: Allow multiple in-flight writes per chunk
>   migration/rdma: Remove unused transit bitmap
>
>  migration/multifd-nocomp.c | 24 ++++++++++
>  migration/ram.c            | 45 +++++++++++++++---
>  migration/ram.h            | 12 +++++
>  migration/rdma.c           | 97 ++++++++++++++++++++------------------
>  migration/trace-events     |  5 +-
>  5 files changed, 128 insertions(+), 55 deletions(-)

Queued, thanks!