[Qemu-devel] [PULL 0/5] Migration PULL request

[Qemu-devel] [PULL 0/5] Migration PULL request

[Juan Quintela]

Hi

In this pull requset:
- I included the v1 of peter return path, fix it
- fix the compliation with older compilers (new compliers don't
   complain about a repeated typedef, so I didn't noticed)
- Add test for disabled features
- Last bit of misc cleanup, remove of not needed includes

Please, apply.

Thanks, Juan.

The following changes since commit 3f0602927b120a480b35dcf58cf6f95435b3ae91:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170613' into staging (2017-06-13 15:49:07 +0100)

are available in the git repository at:

  git://github.com/juanquintela/qemu.git tags/migration/20170614

for you to fetch changes up to 3416ab5bb452f1b6cea58aed8983ffb9a455b7c4:

  migration: Don't create decompression threads if not enabled (2017-06-14 11:11:06 +0200)

----------------------------------------------------------------
migration/next for 20170614

----------------------------------------------------------------
Juan Quintela (4):
      migration: Fix compilation with older compilers
      migration: Remove unneeded includes
      migration: Test for disabled features on reception
      migration: Don't create decompression threads if not enabled

Peter Xu (1):
      migration: fix incorrect enable return path

 include/migration/colo.h     |  3 ---
 include/migration/misc.h     |  2 ++
 include/migration/register.h |  3 ---
 include/migration/vmstate.h  |  1 -
 include/qemu/typedefs.h      |  2 ++
 migration/block.c            |  6 ------
 migration/colo-failover.c    |  2 ++
 migration/colo.c             |  2 --
 migration/exec.c             |  2 --
 migration/fd.c               |  2 --
 migration/global_state.c     |  1 -
 migration/migration.c        | 12 +++++-------
 migration/migration.h        |  2 --
 migration/postcopy-ram.c     |  2 --
 migration/qemu-file.c        |  2 --
 migration/ram.c              | 24 +++++++++++++++++++-----
 migration/savevm.c           |  6 ------
 17 files changed, 30 insertions(+), 44 deletions(-)

[Qemu-devel] [PULL 1/5] migration: Fix compilation with older compilers

[Juan Quintela]

That typedefs are needed on both files.  New compilers (F25 where I
work) don't complain about repeating a typedef.  But older ones
complain.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 include/migration/register.h | 3 ---
 include/migration/vmstate.h  | 1 -
 include/qemu/typedefs.h      | 2 ++
 3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/include/migration/register.h b/include/migration/register.h
index 717c617..d9498d9 100644
--- a/include/migration/register.h
+++ b/include/migration/register.h
@@ -14,9 +14,6 @@
 #ifndef MIGRATION_REGISTER_H
 #define MIGRATION_REGISTER_H
 
-typedef void SaveStateHandler(QEMUFile *f, void *opaque);
-typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
-
 typedef struct SaveVMHandlers {
     /* This runs inside the iothread lock.  */
     SaveStateHandler *save_state;
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index f3f3c2a..e85fbd8 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -29,7 +29,6 @@
 
 #include "migration/qjson.h"
 
-typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
 typedef struct VMStateInfo VMStateInfo;
 typedef struct VMStateDescription VMStateDescription;
 typedef struct VMStateField VMStateField;
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 51958bf..f745d5f 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -96,5 +96,7 @@ typedef struct uWireSlave uWireSlave;
 typedef struct VirtIODevice VirtIODevice;
 typedef struct Visitor Visitor;
 typedef struct node_info NodeInfo;
+typedef void SaveStateHandler(QEMUFile *f, void *opaque);
+typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
 
 #endif /* QEMU_TYPEDEFS_H */
-- 
2.9.4

[Qemu-devel] [PULL 2/5] migration: fix incorrect enable return path

[Juan Quintela]

From: Peter Xu <peterx@redhat.com>

0425dc9 is actually v1 of that patch, but it was accidentally
merged (while there was a v2). That will cause problem when we try to
migrate to some old QEMUs when return path is not really there. Let's
fix it, then squashing this patch with 0425dc9 will be exactly patch
content of v2.

Fixes: 0425dc9 ("migration: isolate return path on src")
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/migration.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 8ef6d6c..f9053b0 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1814,7 +1814,11 @@ static void *migration_thread(void *opaque)
 
     qemu_savevm_state_header(s->to_dst_file);
 
-    if (s->to_dst_file) {
+    /*
+     * If we opened the return path, we need to make sure dst has it
+     * opened as well.
+     */
+    if (s->rp_state.from_dst_file) {
         /* Now tell the dest that it should open its end so it can reply */
         qemu_savevm_send_open_return_path(s->to_dst_file);
 
-- 
2.9.4

[Qemu-devel] [PULL 3/5] migration: Remove unneeded includes

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 include/migration/colo.h  | 3 ---
 include/migration/misc.h  | 2 ++
 migration/block.c         | 6 ------
 migration/colo-failover.c | 2 ++
 migration/colo.c          | 2 --
 migration/exec.c          | 2 --
 migration/fd.c            | 2 --
 migration/global_state.c  | 1 -
 migration/migration.c     | 6 ------
 migration/migration.h     | 2 --
 migration/postcopy-ram.c  | 2 --
 migration/qemu-file.c     | 2 --
 migration/ram.c           | 4 ----
 migration/savevm.c        | 6 ------
 14 files changed, 4 insertions(+), 38 deletions(-)

diff --git a/include/migration/colo.h b/include/migration/colo.h
index ba0bb6e..be6beba 100644
--- a/include/migration/colo.h
+++ b/include/migration/colo.h
@@ -14,9 +14,6 @@
 #define QEMU_COLO_H
 
 #include "qemu-common.h"
-#include "qemu/coroutine_int.h"
-#include "qemu/thread.h"
-#include "qemu/main-loop.h"
 
 bool colo_supported(void);
 void colo_info_init(void);
diff --git a/include/migration/misc.h b/include/migration/misc.h
index d5a433a..65c7070 100644
--- a/include/migration/misc.h
+++ b/include/migration/misc.h
@@ -14,6 +14,8 @@
 #ifndef MIGRATION_MISC_H
 #define MIGRATION_MISC_H
 
+#include "qemu/notify.h"
+
 /* migration/ram.c */
 
 void ram_mig_init(void);
diff --git a/migration/block.c b/migration/block.c
index ae06975..3aae5a3 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -15,19 +15,13 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
-#include "qemu-common.h"
-#include "block/block.h"
 #include "qemu/error-report.h"
-#include "qemu/main-loop.h"
-#include "hw/hw.h"
 #include "qemu/cutils.h"
 #include "qemu/queue.h"
-#include "qemu/timer.h"
 #include "block.h"
 #include "migration/misc.h"
 #include "migration.h"
 #include "migration/register.h"
-#include "sysemu/blockdev.h"
 #include "qemu-file.h"
 #include "migration/vmstate.h"
 #include "sysemu/block-backend.h"
diff --git a/migration/colo-failover.c b/migration/colo-failover.c
index cc229f5..f991486 100644
--- a/migration/colo-failover.c
+++ b/migration/colo-failover.c
@@ -13,6 +13,8 @@
 #include "qemu/osdep.h"
 #include "migration/colo.h"
 #include "migration/failover.h"
+#include "qemu/main-loop.h"
+#include "migration.h"
 #include "qmp-commands.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/error-report.h"
diff --git a/migration/colo.c b/migration/colo.c
index 45e9b46..c436d63 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -11,7 +11,6 @@
  */
 
 #include "qemu/osdep.h"
-#include "qemu/timer.h"
 #include "sysemu/sysemu.h"
 #include "qemu-file-channel.h"
 #include "migration.h"
@@ -22,7 +21,6 @@
 #include "io/channel-buffer.h"
 #include "trace.h"
 #include "qemu/error-report.h"
-#include "qapi/error.h"
 #include "migration/failover.h"
 #include "replication.h"
 #include "qmp-commands.h"
diff --git a/migration/exec.c b/migration/exec.c
index b1de445..08b599e 100644
--- a/migration/exec.c
+++ b/migration/exec.c
@@ -19,10 +19,8 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
-#include "qemu-common.h"
 #include "channel.h"
 #include "exec.h"
-#include "migration.h"
 #include "io/channel-command.h"
 #include "trace.h"
 
diff --git a/migration/fd.c b/migration/fd.c
index b2384bf..30f5258 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -16,10 +16,8 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
-#include "qemu-common.h"
 #include "channel.h"
 #include "fd.h"
-#include "migration.h"
 #include "monitor/monitor.h"
 #include "io/channel-util.h"
 #include "trace.h"
diff --git a/migration/global_state.c b/migration/global_state.c
index 16ac63f..f792cf5 100644
--- a/migration/global_state.c
+++ b/migration/global_state.c
@@ -17,7 +17,6 @@
 #include "qapi/util.h"
 #include "migration/global_state.h"
 #include "migration/vmstate.h"
-#include "sysemu/sysemu.h"
 #include "trace.h"
 
 typedef struct {
diff --git a/migration/migration.c b/migration/migration.c
index f9053b0..b9d8798 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -16,7 +16,6 @@
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
 #include "qemu/error-report.h"
-#include "qemu/main-loop.h"
 #include "migration/blocker.h"
 #include "exec.h"
 #include "fd.h"
@@ -30,11 +29,9 @@
 #include "qemu-file-channel.h"
 #include "qemu-file.h"
 #include "migration/vmstate.h"
-#include "sysemu/sysemu.h"
 #include "block/block.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/util.h"
-#include "qemu/sockets.h"
 #include "qemu/rcu.h"
 #include "block.h"
 #include "postcopy-ram.h"
@@ -42,9 +39,6 @@
 #include "qmp-commands.h"
 #include "trace.h"
 #include "qapi-event.h"
-#include "qom/cpu.h"
-#include "exec/memory.h"
-#include "exec/address-spaces.h"
 #include "exec/target_page.h"
 #include "io/channel-buffer.h"
 #include "migration/colo.h"
diff --git a/migration/migration.h b/migration/migration.h
index 601e4ab..d9a268a 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -14,10 +14,8 @@
 #ifndef QEMU_MIGRATION_H
 #define QEMU_MIGRATION_H
 
-#include "qapi/qmp/qdict.h"
 #include "qemu-common.h"
 #include "qemu/thread.h"
-#include "qemu/notify.h"
 #include "qapi-types.h"
 #include "exec/cpu-common.h"
 #include "qemu/coroutine_int.h"
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index c8c4500..7e21e6f 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -17,8 +17,6 @@
  */
 
 #include "qemu/osdep.h"
-
-#include "qemu-common.h"
 #include "exec/target_page.h"
 #include "migration.h"
 #include "qemu-file.h"
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index e65c373..2ab2bf3 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -26,8 +26,6 @@
 #include "qemu-common.h"
 #include "qemu/error-report.h"
 #include "qemu/iov.h"
-#include "qemu/sockets.h"
-#include "qemu/coroutine.h"
 #include "migration.h"
 #include "qemu-file.h"
 #include "trace.h"
diff --git a/migration/ram.c b/migration/ram.c
index c1b4f4a..7f687f5 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -26,14 +26,12 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
-#include "qemu-common.h"
 #include "cpu.h"
 #include <zlib.h>
 #include "qapi-event.h"
 #include "qemu/cutils.h"
 #include "qemu/bitops.h"
 #include "qemu/bitmap.h"
-#include "qemu/timer.h"
 #include "qemu/main-loop.h"
 #include "xbzrle.h"
 #include "ram.h"
@@ -41,9 +39,7 @@
 #include "migration/register.h"
 #include "migration/misc.h"
 #include "qemu-file.h"
-#include "migration/vmstate.h"
 #include "postcopy-ram.h"
-#include "exec/address-spaces.h"
 #include "migration/page_cache.h"
 #include "qemu/error-report.h"
 #include "trace.h"
diff --git a/migration/savevm.c b/migration/savevm.c
index 30cda2c..f32a82d 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -28,12 +28,8 @@
 
 #include "qemu/osdep.h"
 #include "hw/boards.h"
-#include "hw/hw.h"
-#include "hw/qdev.h"
 #include "hw/xen/xen.h"
 #include "net/net.h"
-#include "sysemu/sysemu.h"
-#include "qemu/timer.h"
 #include "migration.h"
 #include "migration/snapshot.h"
 #include "migration/misc.h"
@@ -46,13 +42,11 @@
 #include "postcopy-ram.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/error-report.h"
-#include "qemu/queue.h"
 #include "sysemu/cpus.h"
 #include "exec/memory.h"
 #include "exec/target_page.h"
 #include "qmp-commands.h"
 #include "trace.h"
-#include "qemu/bitops.h"
 #include "qemu/iov.h"
 #include "block/snapshot.h"
 #include "qemu/cutils.h"
-- 
2.9.4

[Qemu-devel] [PULL 4/5] migration: Test for disabled features on reception

[Juan Quintela]

Right now, if we receive a compressed page while this features are
disabled, Bad Things (TM) can happen.  Just add a test for them.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

--

I had XBZRLE here also, but it don't need extra resources on
destination, only on source.  Additionally libvirt don't enable it on
destination, so don't put it here.

- initialize invalid_flags at declaration time.
- remove extra space (peter)
---
 migration/ram.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index 7f687f5..09b483c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2455,7 +2455,7 @@ static int ram_load_postcopy(QEMUFile *f)
 
 static int ram_load(QEMUFile *f, void *opaque, int version_id)
 {
-    int flags = 0, ret = 0;
+    int flags = 0, ret = 0, invalid_flags = 0;
     static uint64_t seq_iter;
     int len = 0;
     /*
@@ -2472,6 +2472,9 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
         ret = -EINVAL;
     }
 
+    if (!migrate_use_compression()) {
+        invalid_flags |= RAM_SAVE_FLAG_COMPRESS_PAGE;
+    }
     /* This RCU critical section can be very long running.
      * When RCU reclaims in the code start to become numerous,
      * it will be necessary to reduce the granularity of this
@@ -2492,6 +2495,15 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
         flags = addr & ~TARGET_PAGE_MASK;
         addr &= TARGET_PAGE_MASK;
 
+        if (flags & invalid_flags) {
+            if (flags & invalid_flags & RAM_SAVE_FLAG_COMPRESS_PAGE) {
+                error_report("Received an unexpected compressed page");
+            }
+
+            ret = -EINVAL;
+            break;
+        }
+
         if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE |
                      RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) {
             RAMBlock *block = ram_block_from_stream(f, flags);
-- 
2.9.4

[Qemu-devel] [PULL 5/5] migration: Don't create decompression threads if not enabled

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

--

I removed the [HACK] part because previous patch just check that
compression pages are not received.
---
 migration/ram.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 09b483c..0baa1e0 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2234,6 +2234,9 @@ void migrate_decompress_threads_create(void)
 {
     int i, thread_count;
 
+    if (!migrate_use_compression()) {
+        return;
+    }
     thread_count = migrate_decompress_threads();
     decompress_threads = g_new0(QemuThread, thread_count);
     decomp_param = g_new0(DecompressParam, thread_count);
@@ -2255,6 +2258,9 @@ void migrate_decompress_threads_join(void)
 {
     int i, thread_count;
 
+    if (!migrate_use_compression()) {
+        return;
+    }
     thread_count = migrate_decompress_threads();
     for (i = 0; i < thread_count; i++) {
         qemu_mutex_lock(&decomp_param[i].mutex);
-- 
2.9.4

Re: [Qemu-devel] [PULL 0/5] Migration PULL request

[Peter Maydell]

On 14 June 2017 at 12:51, Juan Quintela <quintela@redhat.com> wrote:
> Hi
>
> In this pull requset:
> - I included the v1 of peter return path, fix it
> - fix the compliation with older compilers (new compliers don't
>    complain about a repeated typedef, so I didn't noticed)
> - Add test for disabled features
> - Last bit of misc cleanup, remove of not needed includes
>
> Please, apply.
>
> Thanks, Juan.
>
> The following changes since commit 3f0602927b120a480b35dcf58cf6f95435b3ae91:
>
>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170613' into staging (2017-06-13 15:49:07 +0100)
>
> are available in the git repository at:
>
>   git://github.com/juanquintela/qemu.git tags/migration/20170614
>
> for you to fetch changes up to 3416ab5bb452f1b6cea58aed8983ffb9a455b7c4:
>
>   migration: Don't create decompression threads if not enabled (2017-06-14 11:11:06 +0200)
>
> ----------------------------------------------------------------
> migration/next for 20170614
>

Applied, thanks.

-- PMM