[PULL 0/8] Migration 20230428 patches

[PULL 0/8] Migration 20230428 patches

[Juan Quintela]

The following changes since commit 4ebc33f3f3b656ebf62112daca6aa0f8019b4891:

  Merge tag 'pull-tcg-20230502-2' of https://gitlab.com/rth7680/qemu into staging (2023-05-02 21:18:45 +0100)

are available in the Git repository at:

  https://gitlab.com/juan.quintela/qemu.git tags/migration-20230428-pull-request

for you to fetch changes up to 0deb7e9b6cfdb39d89a629e705d6176c511fa83e:

  migration: Drop unused parameter for migration_tls_client_create() (2023-05-03 11:24:20 +0200)

----------------------------------------------------------------
Migraiton Pull request (20230428 take 2)

Hi

Dropped the compression cleanups to see if we find what is going on.

Please apply.

Later, Juan.

----------------------------------------------------------------

Juan Quintela (8):
  multifd: We already account for this packet on the multifd thread
  migration: Move ram_stats to its own file migration-stats.[ch]
  migration: Rename ram_counters to mig_stats
  migration: Rename RAMStats to MigrationAtomicStats
  migration/rdma: Split the zero page case from acct_update_position
  migration/rdma: Unfold last user of acct_update_position()
  migration: Drop unused parameter for migration_tls_get_creds()
  migration: Drop unused parameter for migration_tls_client_create()

 migration/meson.build       |  1 +
 migration/migration-stats.c | 17 +++++++++++
 migration/migration-stats.h | 41 +++++++++++++++++++++++++
 migration/migration.c       | 33 ++++++++++----------
 migration/multifd.c         | 12 ++++----
 migration/postcopy-ram.c    |  2 +-
 migration/ram.c             | 60 ++++++++++++++-----------------------
 migration/ram.h             | 24 ---------------
 migration/rdma.c            |  9 ++++--
 migration/savevm.c          |  3 +-
 migration/tls.c             | 15 ++++------
 migration/tls.h             |  3 +-
 12 files changed, 120 insertions(+), 100 deletions(-)
 create mode 100644 migration/migration-stats.c
 create mode 100644 migration/migration-stats.h

-- 
2.40.0

[PULL 1/8] multifd: We already account for this packet on the multifd thread

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
---
 migration/multifd.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 6a59c03dd2..6053012ad9 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -626,10 +626,7 @@ int multifd_send_sync_main(QEMUFile *f)
         p->packet_num = multifd_send_state->packet_num++;
         p->flags |= MULTIFD_FLAG_SYNC;
         p->pending_job++;
-        qemu_file_acct_rate_limit(f, p->packet_len);
         qemu_mutex_unlock(&p->mutex);
-        stat64_add(&ram_counters.transferred, p->packet_len);
-        stat64_add(&ram_counters.multifd_bytes, p->packet_len);
         qemu_sem_post(&p->sem);
     }
     for (i = 0; i < migrate_multifd_channels(); i++) {
-- 
2.40.0

[PULL 2/8] migration: Move ram_stats to its own file migration-stats.[ch]

[Juan Quintela]

There is already include/qemu/stats.h, so stats.h was a bad idea.
We want this file to not depend on anything else, we will move all the
migration counters/stats to this struct.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
---
 migration/meson.build       |  1 +
 migration/migration-stats.c | 17 +++++++++++++++
 migration/migration-stats.h | 41 +++++++++++++++++++++++++++++++++++++
 migration/migration.c       |  1 +
 migration/multifd.c         |  1 +
 migration/ram.c             |  3 +--
 migration/ram.h             | 23 ---------------------
 migration/savevm.c          |  1 +
 8 files changed, 63 insertions(+), 25 deletions(-)
 create mode 100644 migration/migration-stats.c
 create mode 100644 migration/migration-stats.h

diff --git a/migration/meson.build b/migration/meson.build
index 480ff6854a..da1897fadf 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -19,6 +19,7 @@ softmmu_ss.add(files(
   'fd.c',
   'global_state.c',
   'migration-hmp-cmds.c',
+  'migration-stats.c',
   'migration.c',
   'multifd.c',
   'multifd-zlib.c',
diff --git a/migration/migration-stats.c b/migration/migration-stats.c
new file mode 100644
index 0000000000..b0eb5ae73c
--- /dev/null
+++ b/migration/migration-stats.c
@@ -0,0 +1,17 @@
+/*
+ * Migration stats
+ *
+ * Copyright (c) 2012-2023 Red Hat Inc
+ *
+ * Authors:
+ *  Juan Quintela <quintela@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/stats64.h"
+#include "migration-stats.h"
+
+RAMStats ram_counters;
diff --git a/migration/migration-stats.h b/migration/migration-stats.h
new file mode 100644
index 0000000000..2edea0c779
--- /dev/null
+++ b/migration/migration-stats.h
@@ -0,0 +1,41 @@
+/*
+ * Migration stats
+ *
+ * Copyright (c) 2012-2023 Red Hat Inc
+ *
+ * Authors:
+ *  Juan Quintela <quintela@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_MIGRATION_STATS_H
+#define QEMU_MIGRATION_STATS_H
+
+#include "qemu/stats64.h"
+
+/*
+ * These are the ram migration statistic counters.  It is loosely
+ * based on MigrationStats.  We change to Stat64 any counter that
+ * needs to be updated using atomic ops (can be accessed by more than
+ * one thread).
+ */
+typedef struct {
+    Stat64 dirty_bytes_last_sync;
+    Stat64 dirty_pages_rate;
+    Stat64 dirty_sync_count;
+    Stat64 dirty_sync_missed_zero_copy;
+    Stat64 downtime_bytes;
+    Stat64 zero_pages;
+    Stat64 multifd_bytes;
+    Stat64 normal_pages;
+    Stat64 postcopy_bytes;
+    Stat64 postcopy_requests;
+    Stat64 precopy_bytes;
+    Stat64 transferred;
+} RAMStats;
+
+extern RAMStats ram_counters;
+
+#endif
diff --git a/migration/migration.c b/migration/migration.c
index abcadbb619..5ecf3dc381 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -29,6 +29,7 @@
 #include "migration/global_state.h"
 #include "migration/misc.h"
 #include "migration.h"
+#include "migration-stats.h"
 #include "savevm.h"
 #include "qemu-file.h"
 #include "channel.h"
diff --git a/migration/multifd.c b/migration/multifd.c
index 6053012ad9..347999f84a 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -19,6 +19,7 @@
 #include "qapi/error.h"
 #include "ram.h"
 #include "migration.h"
+#include "migration-stats.h"
 #include "socket.h"
 #include "tls.h"
 #include "qemu-file.h"
diff --git a/migration/ram.c b/migration/ram.c
index 89be3e3320..a6d5478ef8 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -36,6 +36,7 @@
 #include "xbzrle.h"
 #include "ram.h"
 #include "migration.h"
+#include "migration-stats.h"
 #include "migration/register.h"
 #include "migration/misc.h"
 #include "qemu-file.h"
@@ -460,8 +461,6 @@ uint64_t ram_bytes_remaining(void)
                        0;
 }
 
-RAMStats ram_counters;
-
 void ram_transferred_add(uint64_t bytes)
 {
     if (runstate_is_running()) {
diff --git a/migration/ram.h b/migration/ram.h
index 04b05e1b2c..8692de6ba0 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -32,30 +32,7 @@
 #include "qapi/qapi-types-migration.h"
 #include "exec/cpu-common.h"
 #include "io/channel.h"
-#include "qemu/stats64.h"
 
-/*
- * These are the ram migration statistic counters.  It is loosely
- * based on MigrationStats.  We change to Stat64 any counter that
- * needs to be updated using atomic ops (can be accessed by more than
- * one thread).
- */
-typedef struct {
-    Stat64 dirty_bytes_last_sync;
-    Stat64 dirty_pages_rate;
-    Stat64 dirty_sync_count;
-    Stat64 dirty_sync_missed_zero_copy;
-    Stat64 downtime_bytes;
-    Stat64 zero_pages;
-    Stat64 multifd_bytes;
-    Stat64 normal_pages;
-    Stat64 postcopy_bytes;
-    Stat64 postcopy_requests;
-    Stat64 precopy_bytes;
-    Stat64 transferred;
-} RAMStats;
-
-extern RAMStats ram_counters;
 extern XBZRLECacheStats xbzrle_counters;
 extern CompressionStats compression_counters;
 
diff --git a/migration/savevm.c b/migration/savevm.c
index a9181b444b..8e2efb1a19 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -31,6 +31,7 @@
 #include "net/net.h"
 #include "migration.h"
 #include "migration/snapshot.h"
+#include "migration-stats.h"
 #include "migration/vmstate.h"
 #include "migration/misc.h"
 #include "migration/register.h"
-- 
2.40.0

[PULL 3/8] migration: Rename ram_counters to mig_stats

[Juan Quintela]

migration_stats is just too long, and it is going to have more than
ram counters in the near future.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
---
 migration/migration-stats.c |  2 +-
 migration/migration-stats.h |  2 +-
 migration/migration.c       | 32 ++++++++++++-------------
 migration/multifd.c         |  6 ++---
 migration/ram.c             | 48 ++++++++++++++++++-------------------
 migration/savevm.c          |  2 +-
 6 files changed, 46 insertions(+), 46 deletions(-)

diff --git a/migration/migration-stats.c b/migration/migration-stats.c
index b0eb5ae73c..8c0af9b80a 100644
--- a/migration/migration-stats.c
+++ b/migration/migration-stats.c
@@ -14,4 +14,4 @@
 #include "qemu/stats64.h"
 #include "migration-stats.h"
 
-RAMStats ram_counters;
+RAMStats mig_stats;
diff --git a/migration/migration-stats.h b/migration/migration-stats.h
index 2edea0c779..197374b4f6 100644
--- a/migration/migration-stats.h
+++ b/migration/migration-stats.h
@@ -36,6 +36,6 @@ typedef struct {
     Stat64 transferred;
 } RAMStats;
 
-extern RAMStats ram_counters;
+extern RAMStats mig_stats;
 
 #endif
diff --git a/migration/migration.c b/migration/migration.c
index 5ecf3dc381..feb5ab7493 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -909,26 +909,26 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s)
     size_t page_size = qemu_target_page_size();
 
     info->ram = g_malloc0(sizeof(*info->ram));
-    info->ram->transferred = stat64_get(&ram_counters.transferred);
+    info->ram->transferred = stat64_get(&mig_stats.transferred);
     info->ram->total = ram_bytes_total();
-    info->ram->duplicate = stat64_get(&ram_counters.zero_pages);
+    info->ram->duplicate = stat64_get(&mig_stats.zero_pages);
     /* legacy value.  It is not used anymore */
     info->ram->skipped = 0;
-    info->ram->normal = stat64_get(&ram_counters.normal_pages);
+    info->ram->normal = stat64_get(&mig_stats.normal_pages);
     info->ram->normal_bytes = info->ram->normal * page_size;
     info->ram->mbps = s->mbps;
     info->ram->dirty_sync_count =
-        stat64_get(&ram_counters.dirty_sync_count);
+        stat64_get(&mig_stats.dirty_sync_count);
     info->ram->dirty_sync_missed_zero_copy =
-        stat64_get(&ram_counters.dirty_sync_missed_zero_copy);
+        stat64_get(&mig_stats.dirty_sync_missed_zero_copy);
     info->ram->postcopy_requests =
-        stat64_get(&ram_counters.postcopy_requests);
+        stat64_get(&mig_stats.postcopy_requests);
     info->ram->page_size = page_size;
-    info->ram->multifd_bytes = stat64_get(&ram_counters.multifd_bytes);
+    info->ram->multifd_bytes = stat64_get(&mig_stats.multifd_bytes);
     info->ram->pages_per_second = s->pages_per_second;
-    info->ram->precopy_bytes = stat64_get(&ram_counters.precopy_bytes);
-    info->ram->downtime_bytes = stat64_get(&ram_counters.downtime_bytes);
-    info->ram->postcopy_bytes = stat64_get(&ram_counters.postcopy_bytes);
+    info->ram->precopy_bytes = stat64_get(&mig_stats.precopy_bytes);
+    info->ram->downtime_bytes = stat64_get(&mig_stats.downtime_bytes);
+    info->ram->postcopy_bytes = stat64_get(&mig_stats.postcopy_bytes);
 
     if (migrate_xbzrle()) {
         info->xbzrle_cache = g_malloc0(sizeof(*info->xbzrle_cache));
@@ -960,7 +960,7 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s)
     if (s->state != MIGRATION_STATUS_COMPLETED) {
         info->ram->remaining = ram_bytes_remaining();
         info->ram->dirty_pages_rate =
-           stat64_get(&ram_counters.dirty_pages_rate);
+           stat64_get(&mig_stats.dirty_pages_rate);
     }
 }
 
@@ -1613,10 +1613,10 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc,
 
     migrate_init(s);
     /*
-     * set ram_counters compression_counters memory to zero for a
+     * set mig_stats compression_counters memory to zero for a
      * new migration
      */
-    memset(&ram_counters, 0, sizeof(ram_counters));
+    memset(&mig_stats, 0, sizeof(mig_stats));
     memset(&compression_counters, 0, sizeof(compression_counters));
 
     return true;
@@ -2627,7 +2627,7 @@ static MigThrError migration_detect_error(MigrationState *s)
 static uint64_t migration_total_bytes(MigrationState *s)
 {
     return qemu_file_total_transferred(s->to_dst_file) +
-        stat64_get(&ram_counters.multifd_bytes);
+        stat64_get(&mig_stats.multifd_bytes);
 }
 
 static void migration_calculate_complete(MigrationState *s)
@@ -2691,10 +2691,10 @@ static void migration_update_counters(MigrationState *s,
      * if we haven't sent anything, we don't want to
      * recalculate. 10000 is a small enough number for our purposes
      */
-    if (stat64_get(&ram_counters.dirty_pages_rate) &&
+    if (stat64_get(&mig_stats.dirty_pages_rate) &&
         transferred > 10000) {
         s->expected_downtime =
-            stat64_get(&ram_counters.dirty_bytes_last_sync) / bandwidth;
+            stat64_get(&mig_stats.dirty_bytes_last_sync) / bandwidth;
     }
 
     qemu_file_reset_rate_limit(s->to_dst_file);
diff --git a/migration/multifd.c b/migration/multifd.c
index 347999f84a..4a2e1a47ce 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -434,8 +434,8 @@ static int multifd_send_pages(QEMUFile *f)
     transferred = ((uint64_t) pages->num) * p->page_size + p->packet_len;
     qemu_file_acct_rate_limit(f, transferred);
     qemu_mutex_unlock(&p->mutex);
-    stat64_add(&ram_counters.transferred, transferred);
-    stat64_add(&ram_counters.multifd_bytes, transferred);
+    stat64_add(&mig_stats.transferred, transferred);
+    stat64_add(&mig_stats.multifd_bytes, transferred);
     qemu_sem_post(&p->sem);
 
     return 1;
@@ -577,7 +577,7 @@ static int multifd_zero_copy_flush(QIOChannel *c)
         return -1;
     }
     if (ret == 1) {
-        stat64_add(&ram_counters.dirty_sync_missed_zero_copy, 1);
+        stat64_add(&mig_stats.dirty_sync_missed_zero_copy, 1);
     }
 
     return ret;
diff --git a/migration/ram.c b/migration/ram.c
index a6d5478ef8..c3981f64e4 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -464,13 +464,13 @@ uint64_t ram_bytes_remaining(void)
 void ram_transferred_add(uint64_t bytes)
 {
     if (runstate_is_running()) {
-        stat64_add(&ram_counters.precopy_bytes, bytes);
+        stat64_add(&mig_stats.precopy_bytes, bytes);
     } else if (migration_in_postcopy()) {
-        stat64_add(&ram_counters.postcopy_bytes, bytes);
+        stat64_add(&mig_stats.postcopy_bytes, bytes);
     } else {
-        stat64_add(&ram_counters.downtime_bytes, bytes);
+        stat64_add(&mig_stats.downtime_bytes, bytes);
     }
-    stat64_add(&ram_counters.transferred, bytes);
+    stat64_add(&mig_stats.transferred, bytes);
 }
 
 struct MigrationOps {
@@ -744,7 +744,7 @@ void mig_throttle_counter_reset(void)
 
     rs->time_last_bitmap_sync = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
     rs->num_dirty_pages_period = 0;
-    rs->bytes_xfer_prev = stat64_get(&ram_counters.transferred);
+    rs->bytes_xfer_prev = stat64_get(&mig_stats.transferred);
 }
 
 /**
@@ -764,7 +764,7 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_addr_t current_addr)
     /* We don't care if this fails to allocate a new cache page
      * as long as it updated an old one */
     cache_insert(XBZRLE.cache, current_addr, XBZRLE.zero_target_page,
-                 stat64_get(&ram_counters.dirty_sync_count));
+                 stat64_get(&mig_stats.dirty_sync_count));
 }
 
 #define ENCODING_FLAG_XBZRLE 0x1
@@ -790,7 +790,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss,
     int encoded_len = 0, bytes_xbzrle;
     uint8_t *prev_cached_page;
     QEMUFile *file = pss->pss_channel;
-    uint64_t generation = stat64_get(&ram_counters.dirty_sync_count);
+    uint64_t generation = stat64_get(&mig_stats.dirty_sync_count);
 
     if (!cache_is_cached(XBZRLE.cache, current_addr, generation)) {
         xbzrle_counters.cache_miss++;
@@ -1118,8 +1118,8 @@ uint64_t ram_pagesize_summary(void)
 
 uint64_t ram_get_total_transferred_pages(void)
 {
-    return stat64_get(&ram_counters.normal_pages) +
-        stat64_get(&ram_counters.zero_pages) +
+    return stat64_get(&mig_stats.normal_pages) +
+        stat64_get(&mig_stats.zero_pages) +
         compression_counters.pages + xbzrle_counters.pages;
 }
 
@@ -1129,7 +1129,7 @@ static void migration_update_rates(RAMState *rs, int64_t end_time)
     double compressed_size;
 
     /* calculate period counters */
-    stat64_set(&ram_counters.dirty_pages_rate,
+    stat64_set(&mig_stats.dirty_pages_rate,
                rs->num_dirty_pages_period * 1000 /
                (end_time - rs->time_last_bitmap_sync));
 
@@ -1180,7 +1180,7 @@ static void migration_trigger_throttle(RAMState *rs)
 {
     uint64_t threshold = migrate_throttle_trigger_threshold();
     uint64_t bytes_xfer_period =
-        stat64_get(&ram_counters.transferred) - rs->bytes_xfer_prev;
+        stat64_get(&mig_stats.transferred) - rs->bytes_xfer_prev;
     uint64_t bytes_dirty_period = rs->num_dirty_pages_period * TARGET_PAGE_SIZE;
     uint64_t bytes_dirty_threshold = bytes_xfer_period * threshold / 100;
 
@@ -1209,7 +1209,7 @@ static void migration_bitmap_sync(RAMState *rs)
     RAMBlock *block;
     int64_t end_time;
 
-    stat64_add(&ram_counters.dirty_sync_count, 1);
+    stat64_add(&mig_stats.dirty_sync_count, 1);
 
     if (!rs->time_last_bitmap_sync) {
         rs->time_last_bitmap_sync = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
@@ -1223,7 +1223,7 @@ static void migration_bitmap_sync(RAMState *rs)
         RAMBLOCK_FOREACH_NOT_IGNORED(block) {
             ramblock_sync_dirty_bitmap(rs, block);
         }
-        stat64_set(&ram_counters.dirty_bytes_last_sync, ram_bytes_remaining());
+        stat64_set(&mig_stats.dirty_bytes_last_sync, ram_bytes_remaining());
     }
     qemu_mutex_unlock(&rs->bitmap_mutex);
 
@@ -1243,10 +1243,10 @@ static void migration_bitmap_sync(RAMState *rs)
         /* reset period counters */
         rs->time_last_bitmap_sync = end_time;
         rs->num_dirty_pages_period = 0;
-        rs->bytes_xfer_prev = stat64_get(&ram_counters.transferred);
+        rs->bytes_xfer_prev = stat64_get(&mig_stats.transferred);
     }
     if (migrate_events()) {
-        uint64_t generation = stat64_get(&ram_counters.dirty_sync_count);
+        uint64_t generation = stat64_get(&mig_stats.dirty_sync_count);
         qapi_event_send_migration_pass(generation);
     }
 }
@@ -1320,7 +1320,7 @@ static int save_zero_page(PageSearchStatus *pss, QEMUFile *f, RAMBlock *block,
     int len = save_zero_page_to_file(pss, f, block, offset);
 
     if (len) {
-        stat64_add(&ram_counters.zero_pages, 1);
+        stat64_add(&mig_stats.zero_pages, 1);
         ram_transferred_add(len);
         return 1;
     }
@@ -1357,9 +1357,9 @@ static bool control_save_page(PageSearchStatus *pss, RAMBlock *block,
     }
 
     if (bytes_xmit > 0) {
-        stat64_add(&ram_counters.normal_pages, 1);
+        stat64_add(&mig_stats.normal_pages, 1);
     } else if (bytes_xmit == 0) {
-        stat64_add(&ram_counters.zero_pages, 1);
+        stat64_add(&mig_stats.zero_pages, 1);
     }
 
     return true;
@@ -1391,7 +1391,7 @@ static int save_normal_page(PageSearchStatus *pss, RAMBlock *block,
         qemu_put_buffer(file, buf, TARGET_PAGE_SIZE);
     }
     ram_transferred_add(TARGET_PAGE_SIZE);
-    stat64_add(&ram_counters.normal_pages, 1);
+    stat64_add(&mig_stats.normal_pages, 1);
     return 1;
 }
 
@@ -1447,7 +1447,7 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block,
     if (multifd_queue_page(file, block, offset) < 0) {
         return -1;
     }
-    stat64_add(&ram_counters.normal_pages, 1);
+    stat64_add(&mig_stats.normal_pages, 1);
 
     return 1;
 }
@@ -1486,7 +1486,7 @@ update_compress_thread_counts(const CompressParam *param, int bytes_xmit)
     ram_transferred_add(bytes_xmit);
 
     if (param->zero_page) {
-        stat64_add(&ram_counters.zero_pages, 1);
+        stat64_add(&mig_stats.zero_pages, 1);
         return;
     }
 
@@ -2179,7 +2179,7 @@ int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len)
     RAMBlock *ramblock;
     RAMState *rs = ram_state;
 
-    stat64_add(&ram_counters.postcopy_requests, 1);
+    stat64_add(&mig_stats.postcopy_requests, 1);
     RCU_READ_LOCK_GUARD();
 
     if (!rbname) {
@@ -2634,9 +2634,9 @@ void acct_update_position(QEMUFile *f, size_t size, bool zero)
     uint64_t pages = size / TARGET_PAGE_SIZE;
 
     if (zero) {
-        stat64_add(&ram_counters.zero_pages, pages);
+        stat64_add(&mig_stats.zero_pages, pages);
     } else {
-        stat64_add(&ram_counters.normal_pages, pages);
+        stat64_add(&mig_stats.normal_pages, pages);
         ram_transferred_add(size);
         qemu_file_credit_transfer(f, size);
     }
diff --git a/migration/savevm.c b/migration/savevm.c
index 8e2efb1a19..a9d0a88e62 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1622,7 +1622,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp)
     }
 
     migrate_init(ms);
-    memset(&ram_counters, 0, sizeof(ram_counters));
+    memset(&mig_stats, 0, sizeof(mig_stats));
     memset(&compression_counters, 0, sizeof(compression_counters));
     ms->to_dst_file = f;
 
-- 
2.40.0

[PULL 4/8] migration: Rename RAMStats to MigrationAtomicStats

[Juan Quintela]

It is lousely based on MigrationStats, but that name is taken, so this
is the best one that I came with.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>

---

If you have any good suggestion for the name, I am all ears.
---
 migration/migration-stats.c | 2 +-
 migration/migration-stats.h | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/migration/migration-stats.c b/migration/migration-stats.c
index 8c0af9b80a..2f2cea965c 100644
--- a/migration/migration-stats.c
+++ b/migration/migration-stats.c
@@ -14,4 +14,4 @@
 #include "qemu/stats64.h"
 #include "migration-stats.h"
 
-RAMStats mig_stats;
+MigrationAtomicStats mig_stats;
diff --git a/migration/migration-stats.h b/migration/migration-stats.h
index 197374b4f6..149af932d7 100644
--- a/migration/migration-stats.h
+++ b/migration/migration-stats.h
@@ -34,8 +34,8 @@ typedef struct {
     Stat64 postcopy_requests;
     Stat64 precopy_bytes;
     Stat64 transferred;
-} RAMStats;
+} MigrationAtomicStats;
 
-extern RAMStats mig_stats;
+extern MigrationAtomicStats mig_stats;
 
 #endif
-- 
2.40.0

[PULL 5/8] migration/rdma: Split the zero page case from acct_update_position

[Juan Quintela]

Now that we have atomic counters, we can do it on the place that we
need it, no need to do it inside ram.c.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
---
 migration/ram.c  | 12 ++++--------
 migration/ram.h  |  2 +-
 migration/rdma.c |  7 +++++--
 3 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index c3981f64e4..c249a1f468 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2629,17 +2629,13 @@ static int ram_find_and_save_block(RAMState *rs)
     return pages;
 }
 
-void acct_update_position(QEMUFile *f, size_t size, bool zero)
+void acct_update_position(QEMUFile *f, size_t size)
 {
     uint64_t pages = size / TARGET_PAGE_SIZE;
 
-    if (zero) {
-        stat64_add(&mig_stats.zero_pages, pages);
-    } else {
-        stat64_add(&mig_stats.normal_pages, pages);
-        ram_transferred_add(size);
-        qemu_file_credit_transfer(f, size);
-    }
+    stat64_add(&mig_stats.normal_pages, pages);
+    ram_transferred_add(size);
+    qemu_file_credit_transfer(f, size);
 }
 
 static uint64_t ram_bytes_total_with_ignored(void)
diff --git a/migration/ram.h b/migration/ram.h
index 8692de6ba0..3804753ca3 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -53,7 +53,7 @@ void mig_throttle_counter_reset(void);
 
 uint64_t ram_pagesize_summary(void);
 int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len);
-void acct_update_position(QEMUFile *f, size_t size, bool zero);
+void acct_update_position(QEMUFile *f, size_t size);
 void ram_postcopy_migrated_memory_release(MigrationState *ms);
 /* For outgoing discard bitmap */
 void ram_postcopy_send_discard_bitmap(MigrationState *ms);
diff --git a/migration/rdma.c b/migration/rdma.c
index 0af5e944f0..7a9b284c3f 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -17,8 +17,10 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
+#include "exec/target_page.h"
 #include "rdma.h"
 #include "migration.h"
+#include "migration-stats.h"
 #include "qemu-file.h"
 #include "ram.h"
 #include "qemu/error-report.h"
@@ -2120,7 +2122,8 @@ retry:
                     return -EIO;
                 }
 
-                acct_update_position(f, sge.length, true);
+                stat64_add(&mig_stats.zero_pages,
+                           sge.length / qemu_target_page_size());
 
                 return 1;
             }
@@ -2228,7 +2231,7 @@ retry:
     }
 
     set_bit(chunk, block->transit_bitmap);
-    acct_update_position(f, sge.length, false);
+    acct_update_position(f, sge.length);
     rdma->total_writes++;
 
     return 0;
-- 
2.40.0

[PULL 6/8] migration/rdma: Unfold last user of acct_update_position()

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
---
 migration/ram.c  | 9 ---------
 migration/ram.h  | 1 -
 migration/rdma.c | 4 +++-
 3 files changed, 3 insertions(+), 11 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index c249a1f468..7d81c4a39e 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2629,15 +2629,6 @@ static int ram_find_and_save_block(RAMState *rs)
     return pages;
 }
 
-void acct_update_position(QEMUFile *f, size_t size)
-{
-    uint64_t pages = size / TARGET_PAGE_SIZE;
-
-    stat64_add(&mig_stats.normal_pages, pages);
-    ram_transferred_add(size);
-    qemu_file_credit_transfer(f, size);
-}
-
 static uint64_t ram_bytes_total_with_ignored(void)
 {
     RAMBlock *block;
diff --git a/migration/ram.h b/migration/ram.h
index 3804753ca3..6fffbeb5f1 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -53,7 +53,6 @@ void mig_throttle_counter_reset(void);
 
 uint64_t ram_pagesize_summary(void);
 int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len);
-void acct_update_position(QEMUFile *f, size_t size);
 void ram_postcopy_migrated_memory_release(MigrationState *ms);
 /* For outgoing discard bitmap */
 void ram_postcopy_send_discard_bitmap(MigrationState *ms);
diff --git a/migration/rdma.c b/migration/rdma.c
index 7a9b284c3f..7e747b2595 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2231,7 +2231,9 @@ retry:
     }
 
     set_bit(chunk, block->transit_bitmap);
-    acct_update_position(f, sge.length);
+    stat64_add(&mig_stats.normal_pages, sge.length / qemu_target_page_size());
+    ram_transferred_add(sge.length);
+    qemu_file_credit_transfer(f, sge.length);
     rdma->total_writes++;
 
     return 0;
-- 
2.40.0

[PULL 7/8] migration: Drop unused parameter for migration_tls_get_creds()

[Juan Quintela]

It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 migration/tls.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/migration/tls.c b/migration/tls.c
index cd29177957..d4a76cf590 100644
--- a/migration/tls.c
+++ b/migration/tls.c
@@ -29,9 +29,7 @@
 #include "trace.h"
 
 static QCryptoTLSCreds *
-migration_tls_get_creds(MigrationState *s,
-                        QCryptoTLSCredsEndpoint endpoint,
-                        Error **errp)
+migration_tls_get_creds(QCryptoTLSCredsEndpoint endpoint, Error **errp)
 {
     Object *creds;
     const char *tls_creds = migrate_tls_creds();
@@ -80,8 +78,7 @@ void migration_tls_channel_process_incoming(MigrationState *s,
     QCryptoTLSCreds *creds;
     QIOChannelTLS *tioc;
 
-    creds = migration_tls_get_creds(
-        s, QCRYPTO_TLS_CREDS_ENDPOINT_SERVER, errp);
+    creds = migration_tls_get_creds(QCRYPTO_TLS_CREDS_ENDPOINT_SERVER, errp);
     if (!creds) {
         return;
     }
@@ -124,8 +121,7 @@ QIOChannelTLS *migration_tls_client_create(MigrationState *s,
 {
     QCryptoTLSCreds *creds;
 
-    creds = migration_tls_get_creds(
-        s, QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT, errp);
+    creds = migration_tls_get_creds(QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT, errp);
     if (!creds) {
         return NULL;
     }
-- 
2.40.0

[PULL 8/8] migration: Drop unused parameter for migration_tls_client_create()

[Juan Quintela]

It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 migration/multifd.c      | 2 +-
 migration/postcopy-ram.c | 2 +-
 migration/tls.c          | 5 ++---
 migration/tls.h          | 3 +--
 4 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 4a2e1a47ce..4e71c19292 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -821,7 +821,7 @@ static void multifd_tls_channel_connect(MultiFDSendParams *p,
     const char *hostname = s->hostname;
     QIOChannelTLS *tioc;
 
-    tioc = migration_tls_client_create(s, ioc, hostname, errp);
+    tioc = migration_tls_client_create(ioc, hostname, errp);
     if (!tioc) {
         return;
     }
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 75aa276bb1..5615ec29eb 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -1632,7 +1632,7 @@ postcopy_preempt_send_channel_new(QIOTask *task, gpointer opaque)
     }
 
     if (migrate_channel_requires_tls_upgrade(ioc)) {
-        tioc = migration_tls_client_create(s, ioc, s->hostname, &local_err);
+        tioc = migration_tls_client_create(ioc, s->hostname, &local_err);
         if (!tioc) {
             goto out;
         }
diff --git a/migration/tls.c b/migration/tls.c
index d4a76cf590..fa03d9136c 100644
--- a/migration/tls.c
+++ b/migration/tls.c
@@ -114,8 +114,7 @@ static void migration_tls_outgoing_handshake(QIOTask *task,
     object_unref(OBJECT(ioc));
 }
 
-QIOChannelTLS *migration_tls_client_create(MigrationState *s,
-                                           QIOChannel *ioc,
+QIOChannelTLS *migration_tls_client_create(QIOChannel *ioc,
                                            const char *hostname,
                                            Error **errp)
 {
@@ -141,7 +140,7 @@ void migration_tls_channel_connect(MigrationState *s,
 {
     QIOChannelTLS *tioc;
 
-    tioc = migration_tls_client_create(s, ioc, hostname, errp);
+    tioc = migration_tls_client_create(ioc, hostname, errp);
     if (!tioc) {
         return;
     }
diff --git a/migration/tls.h b/migration/tls.h
index 98e23c9b0e..5797d153cb 100644
--- a/migration/tls.h
+++ b/migration/tls.h
@@ -28,8 +28,7 @@ void migration_tls_channel_process_incoming(MigrationState *s,
                                             QIOChannel *ioc,
                                             Error **errp);
 
-QIOChannelTLS *migration_tls_client_create(MigrationState *s,
-                                           QIOChannel *ioc,
+QIOChannelTLS *migration_tls_client_create(QIOChannel *ioc,
                                            const char *hostname,
                                            Error **errp);
 
-- 
2.40.0

Re: [PULL 0/8] Migration 20230428 patches

[Richard Henderson]

On 5/3/23 10:25, Juan Quintela wrote:
> The following changes since commit 4ebc33f3f3b656ebf62112daca6aa0f8019b4891:
> 
>    Merge tag 'pull-tcg-20230502-2' ofhttps://gitlab.com/rth7680/qemu  into staging (2023-05-02 21:18:45 +0100)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/juan.quintela/qemu.git  tags/migration-20230428-pull-request
> 
> for you to fetch changes up to 0deb7e9b6cfdb39d89a629e705d6176c511fa83e:
> 
>    migration: Drop unused parameter for migration_tls_client_create() (2023-05-03 11:24:20 +0200)
> 
> ----------------------------------------------------------------
> Migraiton Pull request (20230428 take 2)
> 
> Hi
> 
> Dropped the compression cleanups to see if we find what is going on.
> 
> Please apply.
> 
> Later, Juan.

Applied, thanks.  Please update https://wiki.qemu.org/ChangeLog/8.1 as appropriate.


r~