qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PULL 00/11] Migration 20231018 patches
@ 2023-10-18 10:06 Juan Quintela
  2023-10-18 10:06 ` [PULL 01/11] migration: RDMA is not compatible with anything else Juan Quintela
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas

The following changes since commit ec6f9f135d5e5596ab0258da2ddd048f1fd8c359:

  Merge tag 'migration-20231017-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-10-17 10:06:21 -0400)

are available in the Git repository at:

  https://gitlab.com/juan.quintela/qemu.git tags/migration-20231018-pull-request

for you to fetch changes up to e8e4e7acd8e2113d900fe6b9a9870daa7ecf1042:

  migration: save_zero_page() can take block through pss (2023-10-18 10:39:03 +0200)

----------------------------------------------------------------
Migration Pull request (20231018)

In this pull request:
- RDMA cleanups
- compression cleanups

CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1040780020

Please apply.

PD.  I tried to get the deprecated bits integrated, but I broke
     qemu-iotests duer to blk warning.  Will resend it.

----------------------------------------------------------------

Juan Quintela (11):
  migration: RDMA is not compatible with anything else
  migration: Move compression_counters cleanup ram-compress.c
  migration: Create populate_compress()
  migration: Create ram_compressed_pages()
  migration: Move update_compress_threads_counts() to ram-compress.c
  migration: Simplify decompress_data_with_multi_threads()
  migration: Use "i" as an for index in ram-compress.c
  migration: Print block status when needed
  migration: save_compress_page() can take block through pss
  migration: control_save_page() can take block through pss
  migration: save_zero_page() can take block through pss

 migration/ram-compress.h       |   5 ++
 migration/migration-hmp-cmds.c |   5 +-
 migration/migration.c          |  27 +++++----
 migration/ram-compress.c       | 105 +++++++++++++++++++++------------
 migration/ram.c                |  45 +++++---------
 5 files changed, 104 insertions(+), 83 deletions(-)

-- 
2.41.0



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PULL 01/11] migration: RDMA is not compatible with anything else
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 02/11] migration: Move compression_counters cleanup ram-compress.c Juan Quintela
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

So give an error instead of just ignoring the other methods.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-4-quintela@redhat.com>
---
 migration/migration.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/migration/migration.c b/migration/migration.c
index 6ba5e145ac..81862fcaa9 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -447,6 +447,18 @@ static void qemu_start_incoming_migration(const char *uri, Error **errp)
         socket_start_incoming_migration(p ? p : uri, errp);
 #ifdef CONFIG_RDMA
     } else if (strstart(uri, "rdma:", &p)) {
+        if (migrate_compress()) {
+            error_setg(errp, "RDMA and compression can't be used together");
+            return;
+        }
+        if (migrate_xbzrle()) {
+            error_setg(errp, "RDMA and XBZRLE can't be used together");
+            return;
+        }
+        if (migrate_multifd()) {
+            error_setg(errp, "RDMA and multifd can't be used together");
+            return;
+        }
         rdma_start_incoming_migration(p, errp);
 #endif
     } else if (strstart(uri, "exec:", &p)) {
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 02/11] migration: Move compression_counters cleanup ram-compress.c
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
  2023-10-18 10:06 ` [PULL 01/11] migration: RDMA is not compatible with anything else Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 03/11] migration: Create populate_compress() Juan Quintela
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-6-quintela@redhat.com>
---
 migration/migration.c    | 4 +---
 migration/ram-compress.c | 5 +++++
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 81862fcaa9..a808abd294 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1466,11 +1466,9 @@ int migrate_init(MigrationState *s, Error **errp)
     s->switchover_acked = false;
     s->rdma_migration = false;
     /*
-     * set mig_stats compression_counters memory to zero for a
-     * new migration
+     * set mig_stats memory to zero for a new migration
      */
     memset(&mig_stats, 0, sizeof(mig_stats));
-    memset(&compression_counters, 0, sizeof(compression_counters));
     migration_reset_vfio_bytes_transferred();
 
     return 0;
diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index 06254d8c69..a0f6a56470 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -430,6 +430,11 @@ int compress_threads_load_setup(QEMUFile *f)
         return 0;
     }
 
+    /*
+     * set compression_counters memory to zero for a new migration
+     */
+    memset(&compression_counters, 0, sizeof(compression_counters));
+
     thread_count = migrate_decompress_threads();
     decompress_threads = g_new0(QemuThread, thread_count);
     decomp_param = g_new0(DecompressParam, thread_count);
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 03/11] migration: Create populate_compress()
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
  2023-10-18 10:06 ` [PULL 01/11] migration: RDMA is not compatible with anything else Juan Quintela
  2023-10-18 10:06 ` [PULL 02/11] migration: Move compression_counters cleanup ram-compress.c Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 04/11] migration: Create ram_compressed_pages() Juan Quintela
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

So we don't have to access compression_counters from outside
ram-compress.c.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-7-quintela@redhat.com>
---
 migration/ram-compress.h |  3 +++
 migration/migration.c    | 11 +----------
 migration/ram-compress.c | 13 +++++++++++++
 3 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/migration/ram-compress.h b/migration/ram-compress.h
index 6f7fe2f472..43ff44c0ba 100644
--- a/migration/ram-compress.h
+++ b/migration/ram-compress.h
@@ -30,6 +30,7 @@
 #define QEMU_MIGRATION_COMPRESS_H
 
 #include "qemu-file.h"
+#include "qapi/qapi-types-migration.h"
 
 enum CompressResult {
     RES_NONE = 0,
@@ -67,4 +68,6 @@ void compress_threads_load_cleanup(void);
 int compress_threads_load_setup(QEMUFile *f);
 void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len);
 
+void populate_compress(MigrationInfo *info);
+
 #endif
diff --git a/migration/migration.c b/migration/migration.c
index a808abd294..05c0b801ba 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -974,16 +974,7 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s)
         info->xbzrle_cache->overflow = xbzrle_counters.overflow;
     }
 
-    if (migrate_compress()) {
-        info->compression = g_malloc0(sizeof(*info->compression));
-        info->compression->pages = compression_counters.pages;
-        info->compression->busy = compression_counters.busy;
-        info->compression->busy_rate = compression_counters.busy_rate;
-        info->compression->compressed_size =
-                                    compression_counters.compressed_size;
-        info->compression->compression_rate =
-                                    compression_counters.compression_rate;
-    }
+    populate_compress(info);
 
     if (cpu_throttle_active()) {
         info->has_cpu_throttle_percentage = true;
diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index a0f6a56470..bbf78a02d4 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -488,3 +488,16 @@ void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len)
         }
     }
 }
+
+void populate_compress(MigrationInfo *info)
+{
+    if (!migrate_compress()) {
+        return;
+    }
+    info->compression = g_malloc0(sizeof(*info->compression));
+    info->compression->pages = compression_counters.pages;
+    info->compression->busy = compression_counters.busy;
+    info->compression->busy_rate = compression_counters.busy_rate;
+    info->compression->compressed_size = compression_counters.compressed_size;
+    info->compression->compression_rate = compression_counters.compression_rate;
+}
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 04/11] migration: Create ram_compressed_pages()
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (2 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 03/11] migration: Create populate_compress() Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 05/11] migration: Move update_compress_threads_counts() to ram-compress.c Juan Quintela
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-8-quintela@redhat.com>
---
 migration/ram-compress.h | 1 +
 migration/ram-compress.c | 6 ++++++
 migration/ram.c          | 2 +-
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/migration/ram-compress.h b/migration/ram-compress.h
index 43ff44c0ba..77465dad4b 100644
--- a/migration/ram-compress.h
+++ b/migration/ram-compress.h
@@ -69,5 +69,6 @@ int compress_threads_load_setup(QEMUFile *f);
 void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len);
 
 void populate_compress(MigrationInfo *info);
+uint64_t ram_compressed_pages(void);
 
 #endif
diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index bbf78a02d4..d54345d0c1 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -501,3 +501,9 @@ void populate_compress(MigrationInfo *info)
     info->compression->compressed_size = compression_counters.compressed_size;
     info->compression->compression_rate = compression_counters.compression_rate;
 }
+
+uint64_t ram_compressed_pages(void)
+{
+    return compression_counters.pages;
+}
+
diff --git a/migration/ram.c b/migration/ram.c
index c844151ee9..2a9c899cb0 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -939,7 +939,7 @@ uint64_t ram_get_total_transferred_pages(void)
 {
     return stat64_get(&mig_stats.normal_pages) +
         stat64_get(&mig_stats.zero_pages) +
-        compression_counters.pages + xbzrle_counters.pages;
+        ram_compressed_pages() + xbzrle_counters.pages;
 }
 
 static void migration_update_rates(RAMState *rs, int64_t end_time)
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 05/11] migration: Move update_compress_threads_counts() to ram-compress.c
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (3 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 04/11] migration: Create ram_compressed_pages() Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 06/11] migration: Simplify decompress_data_with_multi_threads() Juan Quintela
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-9-quintela@redhat.com>
---
 migration/ram-compress.h |  1 +
 migration/ram-compress.c | 17 +++++++++++++++++
 migration/ram.c          | 15 ---------------
 3 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/migration/ram-compress.h b/migration/ram-compress.h
index 77465dad4b..e55d3b50bd 100644
--- a/migration/ram-compress.h
+++ b/migration/ram-compress.h
@@ -70,5 +70,6 @@ void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len);
 
 void populate_compress(MigrationInfo *info);
 uint64_t ram_compressed_pages(void);
+void update_compress_thread_counts(const CompressParam *param, int bytes_xmit);
 
 #endif
diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index d54345d0c1..3ad5975d48 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -32,11 +32,14 @@
 #include "ram-compress.h"
 
 #include "qemu/error-report.h"
+#include "qemu/stats64.h"
 #include "migration.h"
 #include "options.h"
 #include "io/channel-null.h"
 #include "exec/target_page.h"
 #include "exec/ramblock.h"
+#include "ram.h"
+#include "migration-stats.h"
 
 CompressionStats compression_counters;
 
@@ -507,3 +510,17 @@ uint64_t ram_compressed_pages(void)
     return compression_counters.pages;
 }
 
+void update_compress_thread_counts(const CompressParam *param, int bytes_xmit)
+{
+    ram_transferred_add(bytes_xmit);
+
+    if (param->result == RES_ZEROPAGE) {
+        stat64_add(&mig_stats.zero_pages, 1);
+        return;
+    }
+
+    /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */
+    compression_counters.compressed_size += bytes_xmit - 8;
+    compression_counters.pages++;
+}
+
diff --git a/migration/ram.c b/migration/ram.c
index 2a9c899cb0..5595379fa6 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1292,21 +1292,6 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block,
     return 1;
 }
 
-static void
-update_compress_thread_counts(const CompressParam *param, int bytes_xmit)
-{
-    ram_transferred_add(bytes_xmit);
-
-    if (param->result == RES_ZEROPAGE) {
-        stat64_add(&mig_stats.zero_pages, 1);
-        return;
-    }
-
-    /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */
-    compression_counters.compressed_size += bytes_xmit - 8;
-    compression_counters.pages++;
-}
-
 static bool save_page_use_compression(RAMState *rs);
 
 static int send_queued_data(CompressParam *param)
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 06/11] migration: Simplify decompress_data_with_multi_threads()
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (4 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 05/11] migration: Move update_compress_threads_counts() to ram-compress.c Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 07/11] migration: Use "i" as an for index in ram-compress.c Juan Quintela
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Doing a break to do another break is just confused.  Just call return
when we know we want to return.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-14-quintela@redhat.com>
---
 migration/ram-compress.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index 3ad5975d48..95a8d6b906 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -481,14 +481,10 @@ void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len)
                 decomp_param[idx].len = len;
                 qemu_cond_signal(&decomp_param[idx].cond);
                 qemu_mutex_unlock(&decomp_param[idx].mutex);
-                break;
+                return;
             }
         }
-        if (idx < thread_count) {
-            break;
-        } else {
-            qemu_cond_wait(&decomp_done_cond, &decomp_done_lock);
-        }
+        qemu_cond_wait(&decomp_done_cond, &decomp_done_lock);
     }
 }
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 07/11] migration: Use "i" as an for index in ram-compress.c
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (5 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 06/11] migration: Simplify decompress_data_with_multi_threads() Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 08/11] migration: Print block status when needed Juan Quintela
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

It is used everywhere else in C.  Once there, make sure that we don't
use the index outside of the for declaring the variable there.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Message-ID: <20230613145757.10131-15-quintela@redhat.com>
---
 migration/ram-compress.c | 56 ++++++++++++++++++----------------------
 1 file changed, 25 insertions(+), 31 deletions(-)

diff --git a/migration/ram-compress.c b/migration/ram-compress.c
index 95a8d6b906..d037dfe6cf 100644
--- a/migration/ram-compress.c
+++ b/migration/ram-compress.c
@@ -230,27 +230,25 @@ static inline void compress_reset_result(CompressParam *param)
 
 void flush_compressed_data(int (send_queued_data(CompressParam *)))
 {
-    int idx, thread_count;
-
-    thread_count = migrate_compress_threads();
+    int thread_count = migrate_compress_threads();
 
     qemu_mutex_lock(&comp_done_lock);
-    for (idx = 0; idx < thread_count; idx++) {
-        while (!comp_param[idx].done) {
+    for (int i = 0; i < thread_count; i++) {
+        while (!comp_param[i].done) {
             qemu_cond_wait(&comp_done_cond, &comp_done_lock);
         }
     }
     qemu_mutex_unlock(&comp_done_lock);
 
-    for (idx = 0; idx < thread_count; idx++) {
-        qemu_mutex_lock(&comp_param[idx].mutex);
-        if (!comp_param[idx].quit) {
-            CompressParam *param = &comp_param[idx];
+    for (int i = 0; i < thread_count; i++) {
+        qemu_mutex_lock(&comp_param[i].mutex);
+        if (!comp_param[i].quit) {
+            CompressParam *param = &comp_param[i];
             send_queued_data(param);
             assert(qemu_file_buffer_empty(param->file));
             compress_reset_result(param);
         }
-        qemu_mutex_unlock(&comp_param[idx].mutex);
+        qemu_mutex_unlock(&comp_param[i].mutex);
     }
 }
 
@@ -265,15 +263,15 @@ static inline void set_compress_params(CompressParam *param, RAMBlock *block,
 int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset,
                                 int (send_queued_data(CompressParam *)))
 {
-    int idx, thread_count, pages = -1;
+    int  thread_count, pages = -1;
     bool wait = migrate_compress_wait_thread();
 
     thread_count = migrate_compress_threads();
     qemu_mutex_lock(&comp_done_lock);
 retry:
-    for (idx = 0; idx < thread_count; idx++) {
-        if (comp_param[idx].done) {
-            CompressParam *param = &comp_param[idx];
+    for (int i = 0; i < thread_count; i++) {
+        if (comp_param[i].done) {
+            CompressParam *param = &comp_param[i];
             qemu_mutex_lock(&param->mutex);
             param->done = false;
             send_queued_data(param);
@@ -367,16 +365,14 @@ static void *do_data_decompress(void *opaque)
 
 int wait_for_decompress_done(void)
 {
-    int idx, thread_count;
-
     if (!migrate_compress()) {
         return 0;
     }
 
-    thread_count = migrate_decompress_threads();
+    int thread_count = migrate_decompress_threads();
     qemu_mutex_lock(&decomp_done_lock);
-    for (idx = 0; idx < thread_count; idx++) {
-        while (!decomp_param[idx].done) {
+    for (int i = 0; i < thread_count; i++) {
+        while (!decomp_param[i].done) {
             qemu_cond_wait(&decomp_done_cond, &decomp_done_lock);
         }
     }
@@ -467,20 +463,18 @@ exit:
 
 void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len)
 {
-    int idx, thread_count;
-
-    thread_count = migrate_decompress_threads();
+    int thread_count = migrate_decompress_threads();
     QEMU_LOCK_GUARD(&decomp_done_lock);
     while (true) {
-        for (idx = 0; idx < thread_count; idx++) {
-            if (decomp_param[idx].done) {
-                decomp_param[idx].done = false;
-                qemu_mutex_lock(&decomp_param[idx].mutex);
-                qemu_get_buffer(f, decomp_param[idx].compbuf, len);
-                decomp_param[idx].des = host;
-                decomp_param[idx].len = len;
-                qemu_cond_signal(&decomp_param[idx].cond);
-                qemu_mutex_unlock(&decomp_param[idx].mutex);
+        for (int i = 0; i < thread_count; i++) {
+            if (decomp_param[i].done) {
+                decomp_param[i].done = false;
+                qemu_mutex_lock(&decomp_param[i].mutex);
+                qemu_get_buffer(f, decomp_param[i].compbuf, len);
+                decomp_param[i].des = host;
+                decomp_param[i].len = len;
+                qemu_cond_signal(&decomp_param[i].cond);
+                qemu_mutex_unlock(&decomp_param[i].mutex);
                 return;
             }
         }
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 08/11] migration: Print block status when needed
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (6 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 07/11] migration: Use "i" as an for index in ram-compress.c Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 09/11] migration: save_compress_page() can take block through pss Juan Quintela
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas

The new line was only printed when command options were used.  When we
used migration parameters and capabilities, it wasn't.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231017172307.22858-2-quintela@redhat.com>
---
 migration/migration-hmp-cmds.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c
index d206700a43..a82597f18e 100644
--- a/migration/migration-hmp-cmds.c
+++ b/migration/migration-hmp-cmds.c
@@ -30,6 +30,7 @@
 #include "sysemu/runstate.h"
 #include "ui/qemu-spice.h"
 #include "sysemu/sysemu.h"
+#include "options.h"
 #include "migration.h"
 
 static void migration_global_dump(Monitor *mon)
@@ -696,7 +697,6 @@ void hmp_x_colo_lost_heartbeat(Monitor *mon, const QDict *qdict)
 typedef struct HMPMigrationStatus {
     QEMUTimer *timer;
     Monitor *mon;
-    bool is_block_migration;
 } HMPMigrationStatus;
 
 static void hmp_migrate_status_cb(void *opaque)
@@ -722,7 +722,7 @@ static void hmp_migrate_status_cb(void *opaque)
 
         timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 1000);
     } else {
-        if (status->is_block_migration) {
+        if (migrate_block()) {
             monitor_printf(status->mon, "\n");
         }
         if (info->error_desc) {
@@ -762,7 +762,6 @@ void hmp_migrate(Monitor *mon, const QDict *qdict)
 
         status = g_malloc0(sizeof(*status));
         status->mon = mon;
-        status->is_block_migration = blk || inc;
         status->timer = timer_new_ms(QEMU_CLOCK_REALTIME, hmp_migrate_status_cb,
                                           status);
         timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME));
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 09/11] migration: save_compress_page() can take block through pss
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (7 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 08/11] migration: Print block status when needed Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 10/11] migration: control_save_page() " Juan Quintela
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 5595379fa6..378d310fbd 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2067,7 +2067,7 @@ static bool save_page_use_compression(RAMState *rs)
  * paths to handle it
  */
 static bool save_compress_page(RAMState *rs, PageSearchStatus *pss,
-                               RAMBlock *block, ram_addr_t offset)
+                               ram_addr_t offset)
 {
     if (!save_page_use_compression(rs)) {
         return false;
@@ -2083,12 +2083,13 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss,
      * We post the fist page as normal page as compression will take
      * much CPU resource.
      */
-    if (block != pss->last_sent_block) {
+    if (pss->block != pss->last_sent_block) {
         ram_flush_compressed_data(rs);
         return false;
     }
 
-    if (compress_page_with_multi_thread(block, offset, send_queued_data) > 0) {
+    if (compress_page_with_multi_thread(pss->block, offset,
+                                        send_queued_data) > 0) {
         return true;
     }
 
@@ -2114,7 +2115,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return res;
     }
 
-    if (save_compress_page(rs, pss, block, offset)) {
+    if (save_compress_page(rs, pss, offset)) {
         return 1;
     }
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 10/11] migration: control_save_page() can take block through pss
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (8 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 09/11] migration: save_compress_page() can take block through pss Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-18 10:06 ` [PULL 11/11] migration: save_zero_page() " Juan Quintela
  2023-10-19 18:51 ` [PULL 00/11] Migration 20231018 patches Stefan Hajnoczi
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 378d310fbd..7f18c1808d 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1186,12 +1186,12 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
  *
  * Return true if the pages has been saved, otherwise false is returned.
  */
-static bool control_save_page(PageSearchStatus *pss, RAMBlock *block,
+static bool control_save_page(PageSearchStatus *pss,
                               ram_addr_t offset, int *pages)
 {
     int ret;
 
-    ret = rdma_control_save_page(pss->pss_channel, block->offset, offset,
+    ret = rdma_control_save_page(pss->pss_channel, pss->block->offset, offset,
                                  TARGET_PAGE_SIZE);
     if (ret == RAM_SAVE_CONTROL_NOT_SUPP) {
         return false;
@@ -2111,7 +2111,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
     ram_addr_t offset = ((ram_addr_t)pss->page) << TARGET_PAGE_BITS;
     int res;
 
-    if (control_save_page(pss, block, offset, &res)) {
+    if (control_save_page(pss, offset, &res)) {
         return res;
     }
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PULL 11/11] migration: save_zero_page() can take block through pss
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (9 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 10/11] migration: control_save_page() " Juan Quintela
@ 2023-10-18 10:06 ` Juan Quintela
  2023-10-19 18:51 ` [PULL 00/11] Migration 20231018 patches Stefan Hajnoczi
  11 siblings, 0 replies; 13+ messages in thread
From: Juan Quintela @ 2023-10-18 10:06 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas,
	Lukas Straub

Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 7f18c1808d..16c30a9d7a 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1144,13 +1144,12 @@ void ram_release_page(const char *rbname, uint64_t offset)
  *
  * @rs: current RAM state
  * @pss: current PSS channel
- * @block: block that contains the page we want to send
  * @offset: offset inside the block for the page
  */
-static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
+static int save_zero_page(RAMState *rs, PageSearchStatus *pss,
                           ram_addr_t offset)
 {
-    uint8_t *p = block->host + offset;
+    uint8_t *p = pss->block->host + offset;
     QEMUFile *file = pss->pss_channel;
     int len = 0;
 
@@ -1158,10 +1157,10 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
         return 0;
     }
 
-    len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
+    len += save_page_header(pss, file, pss->block, offset | RAM_SAVE_FLAG_ZERO);
     qemu_put_byte(file, 0);
     len += 1;
-    ram_release_page(block->idstr, offset);
+    ram_release_page(pss->block->idstr, offset);
 
     stat64_add(&mig_stats.zero_pages, 1);
     ram_transferred_add(len);
@@ -1172,7 +1171,7 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
      */
     if (rs->xbzrle_started) {
         XBZRLE_cache_lock();
-        xbzrle_cache_zero_page(block->offset + offset);
+        xbzrle_cache_zero_page(pss->block->offset + offset);
         XBZRLE_cache_unlock();
     }
 
@@ -2119,7 +2118,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    if (save_zero_page(rs, pss, block, offset)) {
+    if (save_zero_page(rs, pss, offset)) {
         return 1;
     }
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PULL 00/11] Migration 20231018 patches
  2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
                   ` (10 preceding siblings ...)
  2023-10-18 10:06 ` [PULL 11/11] migration: save_zero_page() " Juan Quintela
@ 2023-10-19 18:51 ` Stefan Hajnoczi
  11 siblings, 0 replies; 13+ messages in thread
From: Stefan Hajnoczi @ 2023-10-19 18:51 UTC (permalink / raw)
  To: Juan Quintela
  Cc: qemu-devel, Peter Xu, Leonardo Bras, Juan Quintela, Fabiano Rosas

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2023-10-20 16:07 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-18 10:06 [PULL 00/11] Migration 20231018 patches Juan Quintela
2023-10-18 10:06 ` [PULL 01/11] migration: RDMA is not compatible with anything else Juan Quintela
2023-10-18 10:06 ` [PULL 02/11] migration: Move compression_counters cleanup ram-compress.c Juan Quintela
2023-10-18 10:06 ` [PULL 03/11] migration: Create populate_compress() Juan Quintela
2023-10-18 10:06 ` [PULL 04/11] migration: Create ram_compressed_pages() Juan Quintela
2023-10-18 10:06 ` [PULL 05/11] migration: Move update_compress_threads_counts() to ram-compress.c Juan Quintela
2023-10-18 10:06 ` [PULL 06/11] migration: Simplify decompress_data_with_multi_threads() Juan Quintela
2023-10-18 10:06 ` [PULL 07/11] migration: Use "i" as an for index in ram-compress.c Juan Quintela
2023-10-18 10:06 ` [PULL 08/11] migration: Print block status when needed Juan Quintela
2023-10-18 10:06 ` [PULL 09/11] migration: save_compress_page() can take block through pss Juan Quintela
2023-10-18 10:06 ` [PULL 10/11] migration: control_save_page() " Juan Quintela
2023-10-18 10:06 ` [PULL 11/11] migration: save_zero_page() " Juan Quintela
2023-10-19 18:51 ` [PULL 00/11] Migration 20231018 patches Stefan Hajnoczi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).