[Qemu-devel] [PATCH v6 0/5] Introduce a new --only-migratable option

[Qemu-devel] [PATCH v6 0/5] Introduce a new --only-migratable option

[Ashijeet Acharya]

Previously posted series patches:
http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg02324.html
http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg01277.html
http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg00320.html
http://lists.nongnu.org/archive/html/qemu-devel/2016-12/msg02391.html
http://lists.nongnu.org/archive/html/qemu-devel/2016-12/msg02062.html

This series adds a new command line option "--only-migratable" which will only
allow addition of those devices to a QEMU instance which are migratable and do
not abruptly fail QEMU after migration.

Patch 1 is a preparatory patch to remove an undesirable comment and make patchew
happy.

Patch 2 adds the new option "-only-migratable".

Patch 3 adds compatibility for various "device adding" options for both via
command line and hotplug methods.

Patch 4 helps to fail the migration blocker if the migration is already in
progress and thus cannot be blocked.
Note: This patch was originally written by John Snow and I have only made few
changes.

Patch 5 handles the special case of devices which become unmigratable
dynamically by making call to "migrate_add_blocker". Here we fail the
migration blocker if --only-migratable was specified.
Eg: 9pfs fails to mount the filesystem.

Note: I have not been able to test and compile the ARM drivers for KVM. They
are:
hw/intc/arm_gic_kvm.c
hw/intc/arm_gicv3_its_kvm.c
hw/intc/arm_gicv3_kvm.c

Changes in v6:
-make Error *local_err = NULL at some places
-free reason at call sites
-rebase
-pass error_copy(reason) in error_propagate()

Changes in v5:
-drop =0 for global variable (Dave)
-print error message if klass=NULL (Dave)
-deal with migration blocker before pdu_marshal (Greg)
-call migrate_add_blocker before kvm_create_device (Dave)
-fix typo in subject line for 4/4 (Eric)
-fix error_free in migrate_add_blocker (Dave)
-make migrate_add_blocker to deal with error messages itself (Peter, Greg, Dave)

Changes in v4:
- drop diff in 9pfs for patch 4/4
- call clunk_fid() after freeing migration_blocker
- drop ret and use err directly

Changes in v3:
- set s->root_fid after migrate_add_blocker
- free migration_blocker inside v9fs_attach()
- change back ret<0 to just ret
- free local_err

Changes in v2:
- change the documentation for the new option
- add a NULL check for ObjectClass
- break patch 3 into patch 3 and 4
- use error_append_hint
- return -EACCES for only-migratable
- fix the error messages

Ashijeet Acharya (5):
  block/vvfat: Remove the undesirable comment
  migration: Add a new option to enable only-migratable
  migration: Allow "device add" options to only add migratable devices
  migration: disallow migrate_add_blocker during migration
  migration: Fail migration blocker for --only-migratable

 block/qcow.c                  |  8 +++++++-
 block/vdi.c                   |  8 +++++++-
 block/vhdx.c                  | 17 +++++++++++------
 block/vmdk.c                  |  9 ++++++++-
 block/vpc.c                   | 11 ++++++++---
 block/vvfat.c                 | 20 ++++++++++++--------
 hw/9pfs/9p.c                  | 33 +++++++++++++++++++++-----------
 hw/display/virtio-gpu.c       | 32 ++++++++++++++++++-------------
 hw/intc/arm_gic_kvm.c         | 17 +++++++++++------
 hw/intc/arm_gicv3_its_kvm.c   | 20 +++++++++++++-------
 hw/intc/arm_gicv3_kvm.c       | 19 ++++++++++++-------
 hw/misc/ivshmem.c             | 14 ++++++++++----
 hw/scsi/vhost-scsi.c          | 25 ++++++++++++++++++------
 hw/usb/bus.c                  | 19 +++++++++++++++++++
 hw/virtio/vhost.c             |  8 +++++++-
 include/migration/migration.h | 10 +++++++++-
 migration/migration.c         | 44 +++++++++++++++++++++++++++++++++++++++++--
 qdev-monitor.c                |  9 +++++++++
 qemu-options.hx               |  9 +++++++++
 stubs/migr-blocker.c          |  3 ++-
 target/i386/kvm.c             | 16 +++++++++++++---
 vl.c                          |  4 ++++
 22 files changed, 273 insertions(+), 82 deletions(-)

-- 
2.6.2

[Qemu-devel] [PATCH v6 1/5] block/vvfat: Remove the undesirable comment

[Ashijeet Acharya]

Remove the "// assert(is_consistent(s))" comment in block/vvfat.c

Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
---
 block/vvfat.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/block/vvfat.c b/block/vvfat.c
index ded2109..7b706dc 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -1189,7 +1189,6 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags,
         init_mbr(s, cyls, heads, secs);
     }
 
-    //    assert(is_consistent(s));
     qemu_co_mutex_init(&s->lock);
 
     /* Disable migration when vvfat is used rw */
-- 
2.6.2

Re: [Qemu-devel] [PATCH v6 1/5] block/vvfat: Remove the undesirable comment

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> Remove the "// assert(is_consistent(s))" comment in block/vvfat.c
> 
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

It looks like it's been commented out for about 11 years now - it can go.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  block/vvfat.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/block/vvfat.c b/block/vvfat.c
> index ded2109..7b706dc 100644
> --- a/block/vvfat.c
> +++ b/block/vvfat.c
> @@ -1189,7 +1189,6 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags,
>          init_mbr(s, cyls, heads, secs);
>      }
>  
> -    //    assert(is_consistent(s));
>      qemu_co_mutex_init(&s->lock);
>  
>      /* Disable migration when vvfat is used rw */
> -- 
> 2.6.2
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[Qemu-devel] [PATCH v6 2/5] migration: Add a new option to enable only-migratable

[Ashijeet Acharya]

Add a new option "--only-migratable" in qemu which will allow to add
only those devices which will not fail qemu after migration. Devices
set with the flag 'unmigratable' cannot be added when this option will
be used.

Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
---
 include/migration/migration.h | 3 +++
 qemu-options.hx               | 9 +++++++++
 vl.c                          | 4 ++++
 3 files changed, 16 insertions(+)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index c309d23..40b3697 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -38,6 +38,9 @@
 #define QEMU_VM_COMMAND              0x08
 #define QEMU_VM_SECTION_FOOTER       0x7e
 
+/* for vl.c */
+extern int only_migratable;
+
 struct MigrationParams {
     bool blk;
     bool shared;
diff --git a/qemu-options.hx b/qemu-options.hx
index c534a2f..1e16ae8 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3574,6 +3574,15 @@ be used to change settings (such as migration parameters) prior to issuing
 the migrate_incoming to allow the migration to begin.
 ETEXI
 
+DEF("only-migratable", 0, QEMU_OPTION_only_migratable, \
+    "-only-migratable     allow only migratable devices\n", QEMU_ARCH_ALL)
+STEXI
+@item -only-migratable
+@findex -only-migratable
+Only allow migratable devices. Devices will not be allowed to enter an
+unmigratable state.
+ETEXI
+
 DEF("nodefaults", 0, QEMU_OPTION_nodefaults, \
     "-nodefaults     don't create default devices\n", QEMU_ARCH_ALL)
 STEXI
diff --git a/vl.c b/vl.c
index c643d3f..cbcb459 100644
--- a/vl.c
+++ b/vl.c
@@ -180,6 +180,7 @@ bool boot_strict;
 uint8_t *boot_splash_filedata;
 size_t boot_splash_filedata_size;
 uint8_t qemu_extra_params_fw[2];
+int only_migratable; /* turn it off unless user states otherwise */
 
 int icount_align_option;
 
@@ -3910,6 +3911,9 @@ int main(int argc, char **argv, char **envp)
                 }
                 incoming = optarg;
                 break;
+            case QEMU_OPTION_only_migratable:
+                only_migratable = 1;
+                break;
             case QEMU_OPTION_nodefaults:
                 has_defaults = 0;
                 break;
-- 
2.6.2

Re: [Qemu-devel] [PATCH v6 2/5] migration: Add a new option to enable only-migratable

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> Add a new option "--only-migratable" in qemu which will allow to add
> only those devices which will not fail qemu after migration. Devices
> set with the flag 'unmigratable' cannot be added when this option will
> be used.
> 
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  include/migration/migration.h | 3 +++
>  qemu-options.hx               | 9 +++++++++
>  vl.c                          | 4 ++++
>  3 files changed, 16 insertions(+)
> 
> diff --git a/include/migration/migration.h b/include/migration/migration.h
> index c309d23..40b3697 100644
> --- a/include/migration/migration.h
> +++ b/include/migration/migration.h
> @@ -38,6 +38,9 @@
>  #define QEMU_VM_COMMAND              0x08
>  #define QEMU_VM_SECTION_FOOTER       0x7e
>  
> +/* for vl.c */
> +extern int only_migratable;
> +
>  struct MigrationParams {
>      bool blk;
>      bool shared;
> diff --git a/qemu-options.hx b/qemu-options.hx
> index c534a2f..1e16ae8 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -3574,6 +3574,15 @@ be used to change settings (such as migration parameters) prior to issuing
>  the migrate_incoming to allow the migration to begin.
>  ETEXI
>  
> +DEF("only-migratable", 0, QEMU_OPTION_only_migratable, \
> +    "-only-migratable     allow only migratable devices\n", QEMU_ARCH_ALL)
> +STEXI
> +@item -only-migratable
> +@findex -only-migratable
> +Only allow migratable devices. Devices will not be allowed to enter an
> +unmigratable state.
> +ETEXI
> +
>  DEF("nodefaults", 0, QEMU_OPTION_nodefaults, \
>      "-nodefaults     don't create default devices\n", QEMU_ARCH_ALL)
>  STEXI
> diff --git a/vl.c b/vl.c
> index c643d3f..cbcb459 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -180,6 +180,7 @@ bool boot_strict;
>  uint8_t *boot_splash_filedata;
>  size_t boot_splash_filedata_size;
>  uint8_t qemu_extra_params_fw[2];
> +int only_migratable; /* turn it off unless user states otherwise */
>  
>  int icount_align_option;
>  
> @@ -3910,6 +3911,9 @@ int main(int argc, char **argv, char **envp)
>                  }
>                  incoming = optarg;
>                  break;
> +            case QEMU_OPTION_only_migratable:
> +                only_migratable = 1;
> +                break;
>              case QEMU_OPTION_nodefaults:
>                  has_defaults = 0;
>                  break;
> -- 
> 2.6.2
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v6 2/5] migration: Add a new option to enable only-migratable

[Juan Quintela]

Ashijeet Acharya <ashijeetacharya@gmail.com> wrote:
> Add a new option "--only-migratable" in qemu which will allow to add
> only those devices which will not fail qemu after migration. Devices
> set with the flag 'unmigratable' cannot be added when this option will
> be used.
>
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

I don't comment on the block code, as I don't know it well enough.

[Qemu-devel] [PATCH v6 3/5] migration: Allow "device add" options to only add migratable devices

[Ashijeet Acharya]

Introduce checks for the unmigratable flag in the VMStateDescription
structs of respective devices when user attempts to add them. If the
"--only-migratable" was specified, all unmigratable devices will
rightly fail to add. This feature is made compatible for both "-device"
and "-usbdevice" command line options and covers their hmp and qmp
counterparts as well.

Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
---
 hw/usb/bus.c   | 19 +++++++++++++++++++
 qdev-monitor.c |  9 +++++++++
 2 files changed, 28 insertions(+)

diff --git a/hw/usb/bus.c b/hw/usb/bus.c
index 25913ad..1dcc35c 100644
--- a/hw/usb/bus.c
+++ b/hw/usb/bus.c
@@ -8,6 +8,7 @@
 #include "monitor/monitor.h"
 #include "trace.h"
 #include "qemu/cutils.h"
+#include "migration/migration.h"
 
 static void usb_bus_dev_print(Monitor *mon, DeviceState *qdev, int indent);
 
@@ -686,6 +687,8 @@ USBDevice *usbdevice_create(const char *cmdline)
     const char *params;
     int len;
     USBDevice *dev;
+    ObjectClass *klass;
+    DeviceClass *dc;
 
     params = strchr(cmdline,':');
     if (params) {
@@ -720,6 +723,22 @@ USBDevice *usbdevice_create(const char *cmdline)
         return NULL;
     }
 
+    klass = object_class_by_name(f->name);
+    if (klass == NULL) {
+        error_report("Device '%s' not found", f->name);
+        return NULL;
+    }
+
+    dc = DEVICE_CLASS(klass);
+
+    if (only_migratable) {
+        if (dc->vmsd->unmigratable) {
+            error_report("Device %s is not migratable, but --only-migratable "
+                         "was specified", f->name);
+            return NULL;
+        }
+    }
+
     if (f->usbdevice_init) {
         dev = f->usbdevice_init(bus, params);
     } else {
diff --git a/qdev-monitor.c b/qdev-monitor.c
index c73410c..81d01df 100644
--- a/qdev-monitor.c
+++ b/qdev-monitor.c
@@ -29,6 +29,7 @@
 #include "qemu/error-report.h"
 #include "qemu/help_option.h"
 #include "sysemu/block-backend.h"
+#include "migration/migration.h"
 
 /*
  * Aliases were a bad idea from the start.  Let's keep them
@@ -577,6 +578,14 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
         return NULL;
     }
 
+    if (only_migratable) {
+        if (dc->vmsd->unmigratable) {
+            error_setg(errp, "Device %s is not migratable, but "
+                       "--only-migratable was specified", driver);
+            return NULL;
+        }
+    }
+
     /* find bus */
     path = qemu_opt_get(opts, "bus");
     if (path != NULL) {
-- 
2.6.2

Re: [Qemu-devel] [PATCH v6 3/5] migration: Allow "device add" options to only add migratable devices

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> Introduce checks for the unmigratable flag in the VMStateDescription
> structs of respective devices when user attempts to add them. If the
> "--only-migratable" was specified, all unmigratable devices will
> rightly fail to add. This feature is made compatible for both "-device"
> and "-usbdevice" command line options and covers their hmp and qmp
> counterparts as well.
> 
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  hw/usb/bus.c   | 19 +++++++++++++++++++
>  qdev-monitor.c |  9 +++++++++
>  2 files changed, 28 insertions(+)
> 
> diff --git a/hw/usb/bus.c b/hw/usb/bus.c
> index 25913ad..1dcc35c 100644
> --- a/hw/usb/bus.c
> +++ b/hw/usb/bus.c
> @@ -8,6 +8,7 @@
>  #include "monitor/monitor.h"
>  #include "trace.h"
>  #include "qemu/cutils.h"
> +#include "migration/migration.h"
>  
>  static void usb_bus_dev_print(Monitor *mon, DeviceState *qdev, int indent);
>  
> @@ -686,6 +687,8 @@ USBDevice *usbdevice_create(const char *cmdline)
>      const char *params;
>      int len;
>      USBDevice *dev;
> +    ObjectClass *klass;
> +    DeviceClass *dc;
>  
>      params = strchr(cmdline,':');
>      if (params) {
> @@ -720,6 +723,22 @@ USBDevice *usbdevice_create(const char *cmdline)
>          return NULL;
>      }
>  
> +    klass = object_class_by_name(f->name);
> +    if (klass == NULL) {
> +        error_report("Device '%s' not found", f->name);
> +        return NULL;
> +    }
> +
> +    dc = DEVICE_CLASS(klass);
> +
> +    if (only_migratable) {
> +        if (dc->vmsd->unmigratable) {
> +            error_report("Device %s is not migratable, but --only-migratable "
> +                         "was specified", f->name);
> +            return NULL;
> +        }
> +    }
> +
>      if (f->usbdevice_init) {
>          dev = f->usbdevice_init(bus, params);
>      } else {
> diff --git a/qdev-monitor.c b/qdev-monitor.c
> index c73410c..81d01df 100644
> --- a/qdev-monitor.c
> +++ b/qdev-monitor.c
> @@ -29,6 +29,7 @@
>  #include "qemu/error-report.h"
>  #include "qemu/help_option.h"
>  #include "sysemu/block-backend.h"
> +#include "migration/migration.h"
>  
>  /*
>   * Aliases were a bad idea from the start.  Let's keep them
> @@ -577,6 +578,14 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
>          return NULL;
>      }
>  
> +    if (only_migratable) {
> +        if (dc->vmsd->unmigratable) {
> +            error_setg(errp, "Device %s is not migratable, but "
> +                       "--only-migratable was specified", driver);
> +            return NULL;
> +        }
> +    }
> +
>      /* find bus */
>      path = qemu_opt_get(opts, "bus");
>      if (path != NULL) {
> -- 
> 2.6.2
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[Qemu-devel] [PATCH v6 4/5] migration: disallow migrate_add_blocker during migration

[Ashijeet Acharya]

If a migration is already in progress and somebody attempts
to add a migration blocker, this should rightly fail.

Add an errp parameter and a retcode return value to migrate_add_blocker.

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
---
 block/qcow.c                  |  8 +++++++-
 block/vdi.c                   |  8 +++++++-
 block/vhdx.c                  | 17 +++++++++++------
 block/vmdk.c                  |  9 ++++++++-
 block/vpc.c                   | 11 ++++++++---
 block/vvfat.c                 | 19 ++++++++++++-------
 hw/9pfs/9p.c                  | 33 ++++++++++++++++++++++-----------
 hw/display/virtio-gpu.c       | 32 +++++++++++++++++++-------------
 hw/intc/arm_gic_kvm.c         | 17 +++++++++++------
 hw/intc/arm_gicv3_its_kvm.c   | 20 +++++++++++++-------
 hw/intc/arm_gicv3_kvm.c       | 19 ++++++++++++-------
 hw/misc/ivshmem.c             | 14 ++++++++++----
 hw/scsi/vhost-scsi.c          | 25 +++++++++++++++++++------
 hw/virtio/vhost.c             |  8 +++++++-
 include/migration/migration.h |  7 ++++++-
 migration/migration.c         | 37 +++++++++++++++++++++++++++++++++++--
 stubs/migr-blocker.c          |  3 ++-
 target/i386/kvm.c             | 16 +++++++++++++---
 18 files changed, 222 insertions(+), 81 deletions(-)

diff --git a/block/qcow.c b/block/qcow.c
index 7540f43..fb738fc 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -104,6 +104,7 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
     unsigned int len, i, shift;
     int ret;
     QCowHeader header;
+    Error *local_err = NULL;
 
     ret = bdrv_pread(bs->file, 0, &header, sizeof(header));
     if (ret < 0) {
@@ -252,7 +253,12 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
     error_setg(&s->migration_blocker, "The qcow format used by node '%s' "
                "does not support live migration",
                bdrv_get_device_or_node_name(bs));
-    migrate_add_blocker(s->migration_blocker);
+    ret = migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        goto fail;
+    }
 
     qemu_co_mutex_init(&s->lock);
     return 0;
diff --git a/block/vdi.c b/block/vdi.c
index 96b78d5..0aeb940 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -361,6 +361,7 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
     VdiHeader header;
     size_t bmap_size;
     int ret;
+    Error *local_err = NULL;
 
     logout("\n");
 
@@ -471,7 +472,12 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
     error_setg(&s->migration_blocker, "The vdi format used by node '%s' "
                "does not support live migration",
                bdrv_get_device_or_node_name(bs));
-    migrate_add_blocker(s->migration_blocker);
+    ret = migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        goto fail_free_bmap;
+    }
 
     qemu_co_mutex_init(&s->write_lock);
 
diff --git a/block/vhdx.c b/block/vhdx.c
index 0ba2f0a..68db9e0 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -991,6 +991,17 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
         }
     }
 
+    /* Disable migration when VHDX images are used */
+    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
+               "does not support live migration",
+               bdrv_get_device_or_node_name(bs));
+    ret = migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        goto fail;
+    }
+
     if (flags & BDRV_O_RDWR) {
         ret = vhdx_update_headers(bs, s, false, NULL);
         if (ret < 0) {
@@ -1000,12 +1011,6 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
 
     /* TODO: differencing files */
 
-    /* Disable migration when VHDX images are used */
-    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
-               "does not support live migration",
-               bdrv_get_device_or_node_name(bs));
-    migrate_add_blocker(s->migration_blocker);
-
     return 0;
 fail:
     vhdx_close(bs);
diff --git a/block/vmdk.c b/block/vmdk.c
index a11c27a..7750212 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -941,6 +941,7 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
     int ret;
     BDRVVmdkState *s = bs->opaque;
     uint32_t magic;
+    Error *local_err = NULL;
 
     buf = vmdk_read_desc(bs->file, 0, errp);
     if (!buf) {
@@ -976,7 +977,13 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
     error_setg(&s->migration_blocker, "The vmdk format used by node '%s' "
                "does not support live migration",
                bdrv_get_device_or_node_name(bs));
-    migrate_add_blocker(s->migration_blocker);
+    ret = migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        goto fail;
+    }
+
     g_free(buf);
     return 0;
 
diff --git a/block/vpc.c b/block/vpc.c
index 8d5886f..ed6353d 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -422,13 +422,18 @@ static int vpc_open(BlockDriverState *bs, QDict *options, int flags,
 #endif
     }
 
-    qemu_co_mutex_init(&s->lock);
-
     /* Disable migration when VHD images are used */
     error_setg(&s->migration_blocker, "The vpc format used by node '%s' "
                "does not support live migration",
                bdrv_get_device_or_node_name(bs));
-    migrate_add_blocker(s->migration_blocker);
+    ret = migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        goto fail;
+    }
+
+    qemu_co_mutex_init(&s->lock);
 
     return 0;
 
diff --git a/block/vvfat.c b/block/vvfat.c
index 7b706dc..c6bf67e 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -1185,21 +1185,26 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags,
 
     s->sector_count = s->faked_sectors + s->sectors_per_cluster*s->cluster_count;
 
-    if (s->first_sectors_number == 0x40) {
-        init_mbr(s, cyls, heads, secs);
-    }
-
-    qemu_co_mutex_init(&s->lock);
-
     /* Disable migration when vvfat is used rw */
     if (s->qcow) {
         error_setg(&s->migration_blocker,
                    "The vvfat (rw) format used by node '%s' "
                    "does not support live migration",
                    bdrv_get_device_or_node_name(bs));
-        migrate_add_blocker(s->migration_blocker);
+        ret = migrate_add_blocker(s->migration_blocker, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            error_free(s->migration_blocker);
+            goto fail;
+        }
     }
 
+    if (s->first_sectors_number == 0x40) {
+        init_mbr(s, cyls, heads, secs);
+    }
+
+    qemu_co_mutex_init(&s->lock);
+
     ret = 0;
 fail:
     qemu_opts_del(opts);
diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
index fa58877..06b6e7e 100644
--- a/hw/9pfs/9p.c
+++ b/hw/9pfs/9p.c
@@ -979,6 +979,7 @@ static void coroutine_fn v9fs_attach(void *opaque)
     size_t offset = 7;
     V9fsQID qid;
     ssize_t err;
+    Error *local_err = NULL;
 
     v9fs_string_init(&uname);
     v9fs_string_init(&aname);
@@ -1007,26 +1008,36 @@ static void coroutine_fn v9fs_attach(void *opaque)
         clunk_fid(s, fid);
         goto out;
     }
-    err = pdu_marshal(pdu, offset, "Q", &qid);
-    if (err < 0) {
-        clunk_fid(s, fid);
-        goto out;
-    }
-    err += offset;
-    memcpy(&s->root_qid, &qid, sizeof(qid));
-    trace_v9fs_attach_return(pdu->tag, pdu->id,
-                             qid.type, qid.version, qid.path);
+
     /*
      * disable migration if we haven't done already.
      * attach could get called multiple times for the same export.
      */
     if (!s->migration_blocker) {
-        s->root_fid = fid;
         error_setg(&s->migration_blocker,
                    "Migration is disabled when VirtFS export path '%s' is mounted in the guest using mount_tag '%s'",
                    s->ctx.fs_root ? s->ctx.fs_root : "NULL", s->tag);
-        migrate_add_blocker(s->migration_blocker);
+        err = migrate_add_blocker(s->migration_blocker, &local_err);
+        if (local_err) {
+            error_free(local_err);
+            error_free(s->migration_blocker);
+            s->migration_blocker = NULL;
+            clunk_fid(s, fid);
+            goto out;
+        }
+        s->root_fid = fid;
+    }
+
+    err = pdu_marshal(pdu, offset, "Q", &qid);
+    if (err < 0) {
+        clunk_fid(s, fid);
+        goto out;
     }
+    err += offset;
+
+    memcpy(&s->root_qid, &qid, sizeof(qid));
+    trace_v9fs_attach_return(pdu->tag, pdu->id,
+                             qid.type, qid.version, qid.path);
 out:
     put_fid(pdu, fidp);
 out_nofid:
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 7a15c61..d8e3d1c 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1132,6 +1132,7 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
     VirtIODevice *vdev = VIRTIO_DEVICE(qdev);
     VirtIOGPU *g = VIRTIO_GPU(qdev);
     bool have_virgl;
+    Error *local_err = NULL;
     int i;
 
     if (g->conf.max_outputs > VIRTIO_GPU_MAX_SCANOUTS) {
@@ -1139,14 +1140,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
         return;
     }
 
-    g->config_size = sizeof(struct virtio_gpu_config);
-    g->virtio_config.num_scanouts = g->conf.max_outputs;
-    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
-                g->config_size);
-
-    g->req_state[0].width = 1024;
-    g->req_state[0].height = 768;
-
     g->use_virgl_renderer = false;
 #if !defined(CONFIG_VIRGL) || defined(HOST_WORDS_BIGENDIAN)
     have_virgl = false;
@@ -1158,6 +1151,24 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
     }
 
     if (virtio_gpu_virgl_enabled(g->conf)) {
+        error_setg(&g->migration_blocker, "virgl is not yet migratable");
+        migrate_add_blocker(g->migration_blocker, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            error_free(g->migration_blocker);
+            return;
+        }
+    }
+
+    g->config_size = sizeof(struct virtio_gpu_config);
+    g->virtio_config.num_scanouts = g->conf.max_outputs;
+    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
+                g->config_size);
+
+    g->req_state[0].width = 1024;
+    g->req_state[0].height = 768;
+
+    if (virtio_gpu_virgl_enabled(g->conf)) {
         /* use larger control queue in 3d mode */
         g->ctrl_vq   = virtio_add_queue(vdev, 256, virtio_gpu_handle_ctrl_cb);
         g->cursor_vq = virtio_add_queue(vdev, 16, virtio_gpu_handle_cursor_cb);
@@ -1183,11 +1194,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
             dpy_gfx_replace_surface(g->scanout[i].con, NULL);
         }
     }
-
-    if (virtio_gpu_virgl_enabled(g->conf)) {
-        error_setg(&g->migration_blocker, "virgl is not yet migratable");
-        migrate_add_blocker(g->migration_blocker);
-    }
 }
 
 static void virtio_gpu_device_unrealize(DeviceState *qdev, Error **errp)
diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
index 11729ee..ec952ec 100644
--- a/hw/intc/arm_gic_kvm.c
+++ b/hw/intc/arm_gic_kvm.c
@@ -510,6 +510,17 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (!kvm_arm_gic_can_save_restore(s)) {
+        error_setg(&s->migration_blocker, "This operating system kernel does "
+                                          "not support vGICv2 migration");
+        migrate_add_blocker(s->migration_blocker, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            error_free(s->migration_blocker);
+            return;
+        }
+    }
+
     gic_init_irqs_and_mmio(s, kvm_arm_gicv2_set_irq, NULL);
 
     for (i = 0; i < s->num_irq - GIC_INTERNAL; i++) {
@@ -558,12 +569,6 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
                             KVM_VGIC_V2_ADDR_TYPE_CPU,
                             s->dev_fd);
 
-    if (!kvm_arm_gic_can_save_restore(s)) {
-        error_setg(&s->migration_blocker, "This operating system kernel does "
-                                          "not support vGICv2 migration");
-        migrate_add_blocker(s->migration_blocker);
-    }
-
     if (kvm_has_gsi_routing()) {
         /* set up irq routing */
         kvm_init_irq_routing(kvm_state);
diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
index fc246e0..bd4f3aa 100644
--- a/hw/intc/arm_gicv3_its_kvm.c
+++ b/hw/intc/arm_gicv3_its_kvm.c
@@ -56,6 +56,19 @@ static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
 static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
 {
     GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
+    Error *local_err = NULL;
+
+    /*
+     * Block migration of a KVM GICv3 ITS device: the API for saving and
+     * restoring the state in the kernel is not yet available
+     */
+    error_setg(&s->migration_blocker, "vITS migration is not implemented");
+    migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        return;
+    }
 
     s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
     if (s->dev_fd < 0) {
@@ -73,13 +86,6 @@ static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
 
     gicv3_its_init_mmio(s, NULL);
 
-    /*
-     * Block migration of a KVM GICv3 ITS device: the API for saving and
-     * restoring the state in the kernel is not yet available
-     */
-    error_setg(&s->migration_blocker, "vITS migration is not implemented");
-    migrate_add_blocker(s->migration_blocker);
-
     kvm_msi_use_devid = true;
     kvm_gsi_direct_mapping = false;
     kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 199a439..d69dc47 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -103,6 +103,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
 
     gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL);
 
+    /* Block migration of a KVM GICv3 device: the API for saving and restoring
+     * the state in the kernel is not yet finalised in the kernel or
+     * implemented in QEMU.
+     */
+    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
+    migrate_add_blocker(s->migration_blocker, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        error_free(s->migration_blocker);
+        return;
+    }
+
     /* Try to create the device via the device control API */
     s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_V3, false);
     if (s->dev_fd < 0) {
@@ -122,13 +134,6 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
     kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd);
 
-    /* Block migration of a KVM GICv3 device: the API for saving and restoring
-     * the state in the kernel is not yet finalised in the kernel or
-     * implemented in QEMU.
-     */
-    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
-    migrate_add_blocker(s->migration_blocker);
-
     if (kvm_has_gsi_routing()) {
         /* set up irq routing */
         kvm_init_irq_routing(kvm_state);
diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c
index abeaf3d..fd14d7a 100644
--- a/hw/misc/ivshmem.c
+++ b/hw/misc/ivshmem.c
@@ -840,6 +840,7 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
     uint8_t *pci_conf;
     uint8_t attr = PCI_BASE_ADDRESS_SPACE_MEMORY |
         PCI_BASE_ADDRESS_MEM_PREFETCH;
+    Error *local_err = NULL;
 
     /* IRQFD requires MSI */
     if (ivshmem_has_feature(s, IVSHMEM_IOEVENTFD) &&
@@ -903,9 +904,6 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
         }
     }
 
-    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
-    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
-
     if (s->master == ON_OFF_AUTO_AUTO) {
         s->master = s->vm_id == 0 ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
     }
@@ -913,8 +911,16 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
     if (!ivshmem_is_master(s)) {
         error_setg(&s->migration_blocker,
                    "Migration is disabled when using feature 'peer mode' in device 'ivshmem'");
-        migrate_add_blocker(s->migration_blocker);
+        migrate_add_blocker(s->migration_blocker, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            error_free(s->migration_blocker);
+            return;
+        }
     }
+
+    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
+    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
 }
 
 static void ivshmem_exit(PCIDevice *dev)
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index 5b26946..c491ece 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -238,8 +238,16 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
                                vhost_dummy_handle_output);
     if (err != NULL) {
         error_propagate(errp, err);
-        close(vhostfd);
-        return;
+        goto close_fd;
+    }
+
+    error_setg(&s->migration_blocker,
+               "vhost-scsi does not support migration");
+    migrate_add_blocker(s->migration_blocker, &err);
+    if (err) {
+        error_propagate(errp, err);
+        error_free(s->migration_blocker);
+        goto close_fd;
     }
 
     s->dev.nvqs = VHOST_SCSI_VQ_NUM_FIXED + vs->conf.num_queues;
@@ -252,7 +260,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
     if (ret < 0) {
         error_setg(errp, "vhost-scsi: vhost initialization failed: %s",
                    strerror(-ret));
-        return;
+        goto free_vqs;
     }
 
     /* At present, channel and lun both are 0 for bootable vhost-scsi disk */
@@ -261,9 +269,14 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
     /* Note: we can also get the minimum tpgt from kernel */
     s->target = vs->conf.boot_tpgt;
 
-    error_setg(&s->migration_blocker,
-            "vhost-scsi does not support migration");
-    migrate_add_blocker(s->migration_blocker);
+    return;
+
+ free_vqs:
+    migrate_del_blocker(s->migration_blocker);
+    g_free(s->dev.vqs);
+ close_fd:
+    close(vhostfd);
+    return;
 }
 
 static void vhost_scsi_unrealize(DeviceState *dev, Error **errp)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index d396b22..cca1788 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1082,6 +1082,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
 {
     uint64_t features;
     int i, r, n_initialized_vqs = 0;
+    Error *local_err = NULL;
 
     hdev->migration_blocker = NULL;
 
@@ -1158,7 +1159,12 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     }
 
     if (hdev->migration_blocker != NULL) {
-        migrate_add_blocker(hdev->migration_blocker);
+        r = migrate_add_blocker(hdev->migration_blocker, &local_err);
+        if (local_err) {
+            error_report_err(local_err);
+            error_free(hdev->migration_blocker);
+            goto fail_busyloop;
+        }
     }
 
     hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 40b3697..bcbdb03 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -243,6 +243,7 @@ void remove_migration_state_change_notifier(Notifier *notify);
 MigrationState *migrate_init(const MigrationParams *params);
 bool migration_is_blocked(Error **errp);
 bool migration_in_setup(MigrationState *);
+bool migration_is_idle(MigrationState *s);
 bool migration_has_finished(MigrationState *);
 bool migration_has_failed(MigrationState *);
 /* True if outgoing migration has entered postcopy phase */
@@ -287,8 +288,12 @@ int ram_postcopy_incoming_init(MigrationIncomingState *mis);
  * @migrate_add_blocker - prevent migration from proceeding
  *
  * @reason - an error to be returned whenever migration is attempted
+ *
+ * @errp - [out] The reason (if any) we cannot block migration right now.
+ *
+ * @returns - 0 on success, -EBUSY on failure, with errp set.
  */
-void migrate_add_blocker(Error *reason);
+int migrate_add_blocker(Error *reason, Error **errp);
 
 /**
  * @migrate_del_blocker - remove a blocking error from migration
diff --git a/migration/migration.c b/migration/migration.c
index f498ab8..0d88286 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1044,6 +1044,31 @@ bool migration_in_postcopy_after_devices(MigrationState *s)
     return migration_in_postcopy(s) && s->postcopy_after_devices;
 }
 
+bool migration_is_idle(MigrationState *s)
+{
+    if (!s) {
+        s = migrate_get_current();
+    }
+
+    switch (s->state) {
+    case MIGRATION_STATUS_NONE:
+    case MIGRATION_STATUS_CANCELLED:
+    case MIGRATION_STATUS_COMPLETED:
+    case MIGRATION_STATUS_FAILED:
+        return true;
+    case MIGRATION_STATUS_SETUP:
+    case MIGRATION_STATUS_CANCELLING:
+    case MIGRATION_STATUS_ACTIVE:
+    case MIGRATION_STATUS_POSTCOPY_ACTIVE:
+    case MIGRATION_STATUS_COLO:
+        return false;
+    case MIGRATION_STATUS__MAX:
+        g_assert_not_reached();
+    }
+
+    return false;
+}
+
 MigrationState *migrate_init(const MigrationParams *params)
 {
     MigrationState *s = migrate_get_current();
@@ -1086,9 +1111,17 @@ MigrationState *migrate_init(const MigrationParams *params)
 
 static GSList *migration_blockers;
 
-void migrate_add_blocker(Error *reason)
+int migrate_add_blocker(Error *reason, Error **errp)
 {
-    migration_blockers = g_slist_prepend(migration_blockers, reason);
+    if (migration_is_idle(NULL)) {
+        migration_blockers = g_slist_prepend(migration_blockers, reason);
+        return 0;
+    }
+
+    error_propagate(errp, error_copy(reason));
+    error_prepend(errp, "disallowing migration blocker (migration in "
+                      "progress) for: ");
+    return -EBUSY;
 }
 
 void migrate_del_blocker(Error *reason)
diff --git a/stubs/migr-blocker.c b/stubs/migr-blocker.c
index 8ab3604..a5ba18f 100644
--- a/stubs/migr-blocker.c
+++ b/stubs/migr-blocker.c
@@ -2,8 +2,9 @@
 #include "qemu-common.h"
 #include "migration/migration.h"
 
-void migrate_add_blocker(Error *reason)
+int migrate_add_blocker(Error *reason, Error **errp)
 {
+    return 0;
 }
 
 void migrate_del_blocker(Error *reason)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 10a9cd8..9887052 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -709,6 +709,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
     uint32_t signature[3];
     int kvm_base = KVM_CPUID_SIGNATURE;
     int r;
+    Error *local_err = NULL;
 
     memset(&cpuid_data, 0, sizeof(cpuid_data));
 
@@ -968,7 +969,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
         error_setg(&invtsc_mig_blocker,
                    "State blocked by non-migratable CPU device"
                    " (invtsc flag)");
-        migrate_add_blocker(invtsc_mig_blocker);
+        r = migrate_add_blocker(invtsc_mig_blocker, &local_err);
+        if (local_err) {
+            error_report_err(local_err);
+            error_free(invtsc_mig_blocker);
+            goto fail;
+        }
         /* for savevm */
         vmstate_x86_cpu.unmigratable = 1;
     }
@@ -976,12 +982,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
     cpuid_data.cpuid.padding = 0;
     r = kvm_vcpu_ioctl(cs, KVM_SET_CPUID2, &cpuid_data);
     if (r) {
-        return r;
+        goto fail;
     }
 
     r = kvm_arch_set_tsc_khz(cs);
     if (r < 0) {
-        return r;
+        goto fail;
     }
 
     /* vcpu's TSC frequency is either specified by user, or following
@@ -1008,6 +1014,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
     }
 
     return 0;
+
+ fail:
+    migrate_del_blocker(invtsc_mig_blocker);
+    return r;
 }
 
 void kvm_arch_reset_vcpu(X86CPU *cpu)
-- 
2.6.2

Re: [Qemu-devel] [PATCH v6 4/5] migration: disallow migrate_add_blocker during migration

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> If a migration is already in progress and somebody attempts
> to add a migration blocker, this should rightly fail.
> 
> Add an errp parameter and a retcode return value to migrate_add_blocker.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

armbru: Are you OK with the error manipulation?

Dave

> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
> ---
>  block/qcow.c                  |  8 +++++++-
>  block/vdi.c                   |  8 +++++++-
>  block/vhdx.c                  | 17 +++++++++++------
>  block/vmdk.c                  |  9 ++++++++-
>  block/vpc.c                   | 11 ++++++++---
>  block/vvfat.c                 | 19 ++++++++++++-------
>  hw/9pfs/9p.c                  | 33 ++++++++++++++++++++++-----------
>  hw/display/virtio-gpu.c       | 32 +++++++++++++++++++-------------
>  hw/intc/arm_gic_kvm.c         | 17 +++++++++++------
>  hw/intc/arm_gicv3_its_kvm.c   | 20 +++++++++++++-------
>  hw/intc/arm_gicv3_kvm.c       | 19 ++++++++++++-------
>  hw/misc/ivshmem.c             | 14 ++++++++++----
>  hw/scsi/vhost-scsi.c          | 25 +++++++++++++++++++------
>  hw/virtio/vhost.c             |  8 +++++++-
>  include/migration/migration.h |  7 ++++++-
>  migration/migration.c         | 37 +++++++++++++++++++++++++++++++++++--
>  stubs/migr-blocker.c          |  3 ++-
>  target/i386/kvm.c             | 16 +++++++++++++---
>  18 files changed, 222 insertions(+), 81 deletions(-)
> 
> diff --git a/block/qcow.c b/block/qcow.c
> index 7540f43..fb738fc 100644
> --- a/block/qcow.c
> +++ b/block/qcow.c
> @@ -104,6 +104,7 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
>      unsigned int len, i, shift;
>      int ret;
>      QCowHeader header;
> +    Error *local_err = NULL;
>  
>      ret = bdrv_pread(bs->file, 0, &header, sizeof(header));
>      if (ret < 0) {
> @@ -252,7 +253,12 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The qcow format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
>  
>      qemu_co_mutex_init(&s->lock);
>      return 0;
> diff --git a/block/vdi.c b/block/vdi.c
> index 96b78d5..0aeb940 100644
> --- a/block/vdi.c
> +++ b/block/vdi.c
> @@ -361,6 +361,7 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
>      VdiHeader header;
>      size_t bmap_size;
>      int ret;
> +    Error *local_err = NULL;
>  
>      logout("\n");
>  
> @@ -471,7 +472,12 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The vdi format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail_free_bmap;
> +    }
>  
>      qemu_co_mutex_init(&s->write_lock);
>  
> diff --git a/block/vhdx.c b/block/vhdx.c
> index 0ba2f0a..68db9e0 100644
> --- a/block/vhdx.c
> +++ b/block/vhdx.c
> @@ -991,6 +991,17 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
>          }
>      }
>  
> +    /* Disable migration when VHDX images are used */
> +    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
> +               "does not support live migration",
> +               bdrv_get_device_or_node_name(bs));
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
>      if (flags & BDRV_O_RDWR) {
>          ret = vhdx_update_headers(bs, s, false, NULL);
>          if (ret < 0) {
> @@ -1000,12 +1011,6 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      /* TODO: differencing files */
>  
> -    /* Disable migration when VHDX images are used */
> -    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
> -               "does not support live migration",
> -               bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> -
>      return 0;
>  fail:
>      vhdx_close(bs);
> diff --git a/block/vmdk.c b/block/vmdk.c
> index a11c27a..7750212 100644
> --- a/block/vmdk.c
> +++ b/block/vmdk.c
> @@ -941,6 +941,7 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
>      int ret;
>      BDRVVmdkState *s = bs->opaque;
>      uint32_t magic;
> +    Error *local_err = NULL;
>  
>      buf = vmdk_read_desc(bs->file, 0, errp);
>      if (!buf) {
> @@ -976,7 +977,13 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The vmdk format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
>      g_free(buf);
>      return 0;
>  
> diff --git a/block/vpc.c b/block/vpc.c
> index 8d5886f..ed6353d 100644
> --- a/block/vpc.c
> +++ b/block/vpc.c
> @@ -422,13 +422,18 @@ static int vpc_open(BlockDriverState *bs, QDict *options, int flags,
>  #endif
>      }
>  
> -    qemu_co_mutex_init(&s->lock);
> -
>      /* Disable migration when VHD images are used */
>      error_setg(&s->migration_blocker, "The vpc format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
> +    qemu_co_mutex_init(&s->lock);
>  
>      return 0;
>  
> diff --git a/block/vvfat.c b/block/vvfat.c
> index 7b706dc..c6bf67e 100644
> --- a/block/vvfat.c
> +++ b/block/vvfat.c
> @@ -1185,21 +1185,26 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      s->sector_count = s->faked_sectors + s->sectors_per_cluster*s->cluster_count;
>  
> -    if (s->first_sectors_number == 0x40) {
> -        init_mbr(s, cyls, heads, secs);
> -    }
> -
> -    qemu_co_mutex_init(&s->lock);
> -
>      /* Disable migration when vvfat is used rw */
>      if (s->qcow) {
>          error_setg(&s->migration_blocker,
>                     "The vvfat (rw) format used by node '%s' "
>                     "does not support live migration",
>                     bdrv_get_device_or_node_name(bs));
> -        migrate_add_blocker(s->migration_blocker);
> +        ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            goto fail;
> +        }
>      }
>  
> +    if (s->first_sectors_number == 0x40) {
> +        init_mbr(s, cyls, heads, secs);
> +    }
> +
> +    qemu_co_mutex_init(&s->lock);
> +
>      ret = 0;
>  fail:
>      qemu_opts_del(opts);
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index fa58877..06b6e7e 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -979,6 +979,7 @@ static void coroutine_fn v9fs_attach(void *opaque)
>      size_t offset = 7;
>      V9fsQID qid;
>      ssize_t err;
> +    Error *local_err = NULL;
>  
>      v9fs_string_init(&uname);
>      v9fs_string_init(&aname);
> @@ -1007,26 +1008,36 @@ static void coroutine_fn v9fs_attach(void *opaque)
>          clunk_fid(s, fid);
>          goto out;
>      }
> -    err = pdu_marshal(pdu, offset, "Q", &qid);
> -    if (err < 0) {
> -        clunk_fid(s, fid);
> -        goto out;
> -    }
> -    err += offset;
> -    memcpy(&s->root_qid, &qid, sizeof(qid));
> -    trace_v9fs_attach_return(pdu->tag, pdu->id,
> -                             qid.type, qid.version, qid.path);
> +
>      /*
>       * disable migration if we haven't done already.
>       * attach could get called multiple times for the same export.
>       */
>      if (!s->migration_blocker) {
> -        s->root_fid = fid;
>          error_setg(&s->migration_blocker,
>                     "Migration is disabled when VirtFS export path '%s' is mounted in the guest using mount_tag '%s'",
>                     s->ctx.fs_root ? s->ctx.fs_root : "NULL", s->tag);
> -        migrate_add_blocker(s->migration_blocker);
> +        err = migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_free(local_err);
> +            error_free(s->migration_blocker);
> +            s->migration_blocker = NULL;
> +            clunk_fid(s, fid);
> +            goto out;
> +        }
> +        s->root_fid = fid;
> +    }
> +
> +    err = pdu_marshal(pdu, offset, "Q", &qid);
> +    if (err < 0) {
> +        clunk_fid(s, fid);
> +        goto out;
>      }
> +    err += offset;
> +
> +    memcpy(&s->root_qid, &qid, sizeof(qid));
> +    trace_v9fs_attach_return(pdu->tag, pdu->id,
> +                             qid.type, qid.version, qid.path);
>  out:
>      put_fid(pdu, fidp);
>  out_nofid:
> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> index 7a15c61..d8e3d1c 100644
> --- a/hw/display/virtio-gpu.c
> +++ b/hw/display/virtio-gpu.c
> @@ -1132,6 +1132,7 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>      VirtIODevice *vdev = VIRTIO_DEVICE(qdev);
>      VirtIOGPU *g = VIRTIO_GPU(qdev);
>      bool have_virgl;
> +    Error *local_err = NULL;
>      int i;
>  
>      if (g->conf.max_outputs > VIRTIO_GPU_MAX_SCANOUTS) {
> @@ -1139,14 +1140,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>          return;
>      }
>  
> -    g->config_size = sizeof(struct virtio_gpu_config);
> -    g->virtio_config.num_scanouts = g->conf.max_outputs;
> -    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
> -                g->config_size);
> -
> -    g->req_state[0].width = 1024;
> -    g->req_state[0].height = 768;
> -
>      g->use_virgl_renderer = false;
>  #if !defined(CONFIG_VIRGL) || defined(HOST_WORDS_BIGENDIAN)
>      have_virgl = false;
> @@ -1158,6 +1151,24 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>      }
>  
>      if (virtio_gpu_virgl_enabled(g->conf)) {
> +        error_setg(&g->migration_blocker, "virgl is not yet migratable");
> +        migrate_add_blocker(g->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(g->migration_blocker);
> +            return;
> +        }
> +    }
> +
> +    g->config_size = sizeof(struct virtio_gpu_config);
> +    g->virtio_config.num_scanouts = g->conf.max_outputs;
> +    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
> +                g->config_size);
> +
> +    g->req_state[0].width = 1024;
> +    g->req_state[0].height = 768;
> +
> +    if (virtio_gpu_virgl_enabled(g->conf)) {
>          /* use larger control queue in 3d mode */
>          g->ctrl_vq   = virtio_add_queue(vdev, 256, virtio_gpu_handle_ctrl_cb);
>          g->cursor_vq = virtio_add_queue(vdev, 16, virtio_gpu_handle_cursor_cb);
> @@ -1183,11 +1194,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>              dpy_gfx_replace_surface(g->scanout[i].con, NULL);
>          }
>      }
> -
> -    if (virtio_gpu_virgl_enabled(g->conf)) {
> -        error_setg(&g->migration_blocker, "virgl is not yet migratable");
> -        migrate_add_blocker(g->migration_blocker);
> -    }
>  }
>  
>  static void virtio_gpu_device_unrealize(DeviceState *qdev, Error **errp)
> diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
> index 11729ee..ec952ec 100644
> --- a/hw/intc/arm_gic_kvm.c
> +++ b/hw/intc/arm_gic_kvm.c
> @@ -510,6 +510,17 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
>          return;
>      }
>  
> +    if (!kvm_arm_gic_can_save_restore(s)) {
> +        error_setg(&s->migration_blocker, "This operating system kernel does "
> +                                          "not support vGICv2 migration");
> +        migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            return;
> +        }
> +    }
> +
>      gic_init_irqs_and_mmio(s, kvm_arm_gicv2_set_irq, NULL);
>  
>      for (i = 0; i < s->num_irq - GIC_INTERNAL; i++) {
> @@ -558,12 +569,6 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
>                              KVM_VGIC_V2_ADDR_TYPE_CPU,
>                              s->dev_fd);
>  
> -    if (!kvm_arm_gic_can_save_restore(s)) {
> -        error_setg(&s->migration_blocker, "This operating system kernel does "
> -                                          "not support vGICv2 migration");
> -        migrate_add_blocker(s->migration_blocker);
> -    }
> -
>      if (kvm_has_gsi_routing()) {
>          /* set up irq routing */
>          kvm_init_irq_routing(kvm_state);
> diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
> index fc246e0..bd4f3aa 100644
> --- a/hw/intc/arm_gicv3_its_kvm.c
> +++ b/hw/intc/arm_gicv3_its_kvm.c
> @@ -56,6 +56,19 @@ static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
>  static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
>  {
>      GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
> +    Error *local_err = NULL;
> +
> +    /*
> +     * Block migration of a KVM GICv3 ITS device: the API for saving and
> +     * restoring the state in the kernel is not yet available
> +     */
> +    error_setg(&s->migration_blocker, "vITS migration is not implemented");
> +    migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        return;
> +    }
>  
>      s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
>      if (s->dev_fd < 0) {
> @@ -73,13 +86,6 @@ static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
>  
>      gicv3_its_init_mmio(s, NULL);
>  
> -    /*
> -     * Block migration of a KVM GICv3 ITS device: the API for saving and
> -     * restoring the state in the kernel is not yet available
> -     */
> -    error_setg(&s->migration_blocker, "vITS migration is not implemented");
> -    migrate_add_blocker(s->migration_blocker);
> -
>      kvm_msi_use_devid = true;
>      kvm_gsi_direct_mapping = false;
>      kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
> diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
> index 199a439..d69dc47 100644
> --- a/hw/intc/arm_gicv3_kvm.c
> +++ b/hw/intc/arm_gicv3_kvm.c
> @@ -103,6 +103,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>  
>      gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL);
>  
> +    /* Block migration of a KVM GICv3 device: the API for saving and restoring
> +     * the state in the kernel is not yet finalised in the kernel or
> +     * implemented in QEMU.
> +     */
> +    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
> +    migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        return;
> +    }
> +
>      /* Try to create the device via the device control API */
>      s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_V3, false);
>      if (s->dev_fd < 0) {
> @@ -122,13 +134,6 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>      kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
>                              KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd);
>  
> -    /* Block migration of a KVM GICv3 device: the API for saving and restoring
> -     * the state in the kernel is not yet finalised in the kernel or
> -     * implemented in QEMU.
> -     */
> -    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
> -    migrate_add_blocker(s->migration_blocker);
> -
>      if (kvm_has_gsi_routing()) {
>          /* set up irq routing */
>          kvm_init_irq_routing(kvm_state);
> diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c
> index abeaf3d..fd14d7a 100644
> --- a/hw/misc/ivshmem.c
> +++ b/hw/misc/ivshmem.c
> @@ -840,6 +840,7 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>      uint8_t *pci_conf;
>      uint8_t attr = PCI_BASE_ADDRESS_SPACE_MEMORY |
>          PCI_BASE_ADDRESS_MEM_PREFETCH;
> +    Error *local_err = NULL;
>  
>      /* IRQFD requires MSI */
>      if (ivshmem_has_feature(s, IVSHMEM_IOEVENTFD) &&
> @@ -903,9 +904,6 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>          }
>      }
>  
> -    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
> -    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
> -
>      if (s->master == ON_OFF_AUTO_AUTO) {
>          s->master = s->vm_id == 0 ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
>      }
> @@ -913,8 +911,16 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>      if (!ivshmem_is_master(s)) {
>          error_setg(&s->migration_blocker,
>                     "Migration is disabled when using feature 'peer mode' in device 'ivshmem'");
> -        migrate_add_blocker(s->migration_blocker);
> +        migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            return;
> +        }
>      }
> +
> +    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
> +    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
>  }
>  
>  static void ivshmem_exit(PCIDevice *dev)
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index 5b26946..c491ece 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -238,8 +238,16 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>                                 vhost_dummy_handle_output);
>      if (err != NULL) {
>          error_propagate(errp, err);
> -        close(vhostfd);
> -        return;
> +        goto close_fd;
> +    }
> +
> +    error_setg(&s->migration_blocker,
> +               "vhost-scsi does not support migration");
> +    migrate_add_blocker(s->migration_blocker, &err);
> +    if (err) {
> +        error_propagate(errp, err);
> +        error_free(s->migration_blocker);
> +        goto close_fd;
>      }
>  
>      s->dev.nvqs = VHOST_SCSI_VQ_NUM_FIXED + vs->conf.num_queues;
> @@ -252,7 +260,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>      if (ret < 0) {
>          error_setg(errp, "vhost-scsi: vhost initialization failed: %s",
>                     strerror(-ret));
> -        return;
> +        goto free_vqs;
>      }
>  
>      /* At present, channel and lun both are 0 for bootable vhost-scsi disk */
> @@ -261,9 +269,14 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>      /* Note: we can also get the minimum tpgt from kernel */
>      s->target = vs->conf.boot_tpgt;
>  
> -    error_setg(&s->migration_blocker,
> -            "vhost-scsi does not support migration");
> -    migrate_add_blocker(s->migration_blocker);
> +    return;
> +
> + free_vqs:
> +    migrate_del_blocker(s->migration_blocker);
> +    g_free(s->dev.vqs);
> + close_fd:
> +    close(vhostfd);
> +    return;
>  }
>  
>  static void vhost_scsi_unrealize(DeviceState *dev, Error **errp)
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index d396b22..cca1788 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1082,6 +1082,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>  {
>      uint64_t features;
>      int i, r, n_initialized_vqs = 0;
> +    Error *local_err = NULL;
>  
>      hdev->migration_blocker = NULL;
>  
> @@ -1158,7 +1159,12 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      }
>  
>      if (hdev->migration_blocker != NULL) {
> -        migrate_add_blocker(hdev->migration_blocker);
> +        r = migrate_add_blocker(hdev->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_report_err(local_err);
> +            error_free(hdev->migration_blocker);
> +            goto fail_busyloop;
> +        }
>      }
>  
>      hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
> diff --git a/include/migration/migration.h b/include/migration/migration.h
> index 40b3697..bcbdb03 100644
> --- a/include/migration/migration.h
> +++ b/include/migration/migration.h
> @@ -243,6 +243,7 @@ void remove_migration_state_change_notifier(Notifier *notify);
>  MigrationState *migrate_init(const MigrationParams *params);
>  bool migration_is_blocked(Error **errp);
>  bool migration_in_setup(MigrationState *);
> +bool migration_is_idle(MigrationState *s);
>  bool migration_has_finished(MigrationState *);
>  bool migration_has_failed(MigrationState *);
>  /* True if outgoing migration has entered postcopy phase */
> @@ -287,8 +288,12 @@ int ram_postcopy_incoming_init(MigrationIncomingState *mis);
>   * @migrate_add_blocker - prevent migration from proceeding
>   *
>   * @reason - an error to be returned whenever migration is attempted
> + *
> + * @errp - [out] The reason (if any) we cannot block migration right now.
> + *
> + * @returns - 0 on success, -EBUSY on failure, with errp set.
>   */
> -void migrate_add_blocker(Error *reason);
> +int migrate_add_blocker(Error *reason, Error **errp);
>  
>  /**
>   * @migrate_del_blocker - remove a blocking error from migration
> diff --git a/migration/migration.c b/migration/migration.c
> index f498ab8..0d88286 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1044,6 +1044,31 @@ bool migration_in_postcopy_after_devices(MigrationState *s)
>      return migration_in_postcopy(s) && s->postcopy_after_devices;
>  }
>  
> +bool migration_is_idle(MigrationState *s)
> +{
> +    if (!s) {
> +        s = migrate_get_current();
> +    }
> +
> +    switch (s->state) {
> +    case MIGRATION_STATUS_NONE:
> +    case MIGRATION_STATUS_CANCELLED:
> +    case MIGRATION_STATUS_COMPLETED:
> +    case MIGRATION_STATUS_FAILED:
> +        return true;
> +    case MIGRATION_STATUS_SETUP:
> +    case MIGRATION_STATUS_CANCELLING:
> +    case MIGRATION_STATUS_ACTIVE:
> +    case MIGRATION_STATUS_POSTCOPY_ACTIVE:
> +    case MIGRATION_STATUS_COLO:
> +        return false;
> +    case MIGRATION_STATUS__MAX:
> +        g_assert_not_reached();
> +    }
> +
> +    return false;
> +}
> +
>  MigrationState *migrate_init(const MigrationParams *params)
>  {
>      MigrationState *s = migrate_get_current();
> @@ -1086,9 +1111,17 @@ MigrationState *migrate_init(const MigrationParams *params)
>  
>  static GSList *migration_blockers;
>  
> -void migrate_add_blocker(Error *reason)
> +int migrate_add_blocker(Error *reason, Error **errp)
>  {
> -    migration_blockers = g_slist_prepend(migration_blockers, reason);
> +    if (migration_is_idle(NULL)) {
> +        migration_blockers = g_slist_prepend(migration_blockers, reason);
> +        return 0;
> +    }
> +
> +    error_propagate(errp, error_copy(reason));
> +    error_prepend(errp, "disallowing migration blocker (migration in "
> +                      "progress) for: ");
> +    return -EBUSY;
>  }
>  
>  void migrate_del_blocker(Error *reason)
> diff --git a/stubs/migr-blocker.c b/stubs/migr-blocker.c
> index 8ab3604..a5ba18f 100644
> --- a/stubs/migr-blocker.c
> +++ b/stubs/migr-blocker.c
> @@ -2,8 +2,9 @@
>  #include "qemu-common.h"
>  #include "migration/migration.h"
>  
> -void migrate_add_blocker(Error *reason)
> +int migrate_add_blocker(Error *reason, Error **errp)
>  {
> +    return 0;
>  }
>  
>  void migrate_del_blocker(Error *reason)
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 10a9cd8..9887052 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -709,6 +709,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      uint32_t signature[3];
>      int kvm_base = KVM_CPUID_SIGNATURE;
>      int r;
> +    Error *local_err = NULL;
>  
>      memset(&cpuid_data, 0, sizeof(cpuid_data));
>  
> @@ -968,7 +969,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
>          error_setg(&invtsc_mig_blocker,
>                     "State blocked by non-migratable CPU device"
>                     " (invtsc flag)");
> -        migrate_add_blocker(invtsc_mig_blocker);
> +        r = migrate_add_blocker(invtsc_mig_blocker, &local_err);
> +        if (local_err) {
> +            error_report_err(local_err);
> +            error_free(invtsc_mig_blocker);
> +            goto fail;
> +        }
>          /* for savevm */
>          vmstate_x86_cpu.unmigratable = 1;
>      }
> @@ -976,12 +982,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      cpuid_data.cpuid.padding = 0;
>      r = kvm_vcpu_ioctl(cs, KVM_SET_CPUID2, &cpuid_data);
>      if (r) {
> -        return r;
> +        goto fail;
>      }
>  
>      r = kvm_arch_set_tsc_khz(cs);
>      if (r < 0) {
> -        return r;
> +        goto fail;
>      }
>  
>      /* vcpu's TSC frequency is either specified by user, or following
> @@ -1008,6 +1014,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      }
>  
>      return 0;
> +
> + fail:
> +    migrate_del_blocker(invtsc_mig_blocker);
> +    return r;
>  }
>  
>  void kvm_arch_reset_vcpu(X86CPU *cpu)
> -- 
> 2.6.2
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v6 4/5] migration: disallow migrate_add_blocker during migration

[Greg Kurz]

On Mon, 16 Jan 2017 17:01:53 +0530
Ashijeet Acharya <ashijeetacharya@gmail.com> wrote:

> If a migration is already in progress and somebody attempts
> to add a migration blocker, this should rightly fail.
> 
> Add an errp parameter and a retcode return value to migrate_add_blocker.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
> ---

For the 9pfs part.

Acked-by: Greg Kurz <groug@kaod.org>

>  block/qcow.c                  |  8 +++++++-
>  block/vdi.c                   |  8 +++++++-
>  block/vhdx.c                  | 17 +++++++++++------
>  block/vmdk.c                  |  9 ++++++++-
>  block/vpc.c                   | 11 ++++++++---
>  block/vvfat.c                 | 19 ++++++++++++-------
>  hw/9pfs/9p.c                  | 33 ++++++++++++++++++++++-----------
>  hw/display/virtio-gpu.c       | 32 +++++++++++++++++++-------------
>  hw/intc/arm_gic_kvm.c         | 17 +++++++++++------
>  hw/intc/arm_gicv3_its_kvm.c   | 20 +++++++++++++-------
>  hw/intc/arm_gicv3_kvm.c       | 19 ++++++++++++-------
>  hw/misc/ivshmem.c             | 14 ++++++++++----
>  hw/scsi/vhost-scsi.c          | 25 +++++++++++++++++++------
>  hw/virtio/vhost.c             |  8 +++++++-
>  include/migration/migration.h |  7 ++++++-
>  migration/migration.c         | 37 +++++++++++++++++++++++++++++++++++--
>  stubs/migr-blocker.c          |  3 ++-
>  target/i386/kvm.c             | 16 +++++++++++++---
>  18 files changed, 222 insertions(+), 81 deletions(-)
> 
> diff --git a/block/qcow.c b/block/qcow.c
> index 7540f43..fb738fc 100644
> --- a/block/qcow.c
> +++ b/block/qcow.c
> @@ -104,6 +104,7 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
>      unsigned int len, i, shift;
>      int ret;
>      QCowHeader header;
> +    Error *local_err = NULL;
>  
>      ret = bdrv_pread(bs->file, 0, &header, sizeof(header));
>      if (ret < 0) {
> @@ -252,7 +253,12 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The qcow format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
>  
>      qemu_co_mutex_init(&s->lock);
>      return 0;
> diff --git a/block/vdi.c b/block/vdi.c
> index 96b78d5..0aeb940 100644
> --- a/block/vdi.c
> +++ b/block/vdi.c
> @@ -361,6 +361,7 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
>      VdiHeader header;
>      size_t bmap_size;
>      int ret;
> +    Error *local_err = NULL;
>  
>      logout("\n");
>  
> @@ -471,7 +472,12 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The vdi format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail_free_bmap;
> +    }
>  
>      qemu_co_mutex_init(&s->write_lock);
>  
> diff --git a/block/vhdx.c b/block/vhdx.c
> index 0ba2f0a..68db9e0 100644
> --- a/block/vhdx.c
> +++ b/block/vhdx.c
> @@ -991,6 +991,17 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
>          }
>      }
>  
> +    /* Disable migration when VHDX images are used */
> +    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
> +               "does not support live migration",
> +               bdrv_get_device_or_node_name(bs));
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
>      if (flags & BDRV_O_RDWR) {
>          ret = vhdx_update_headers(bs, s, false, NULL);
>          if (ret < 0) {
> @@ -1000,12 +1011,6 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      /* TODO: differencing files */
>  
> -    /* Disable migration when VHDX images are used */
> -    error_setg(&s->migration_blocker, "The vhdx format used by node '%s' "
> -               "does not support live migration",
> -               bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> -
>      return 0;
>  fail:
>      vhdx_close(bs);
> diff --git a/block/vmdk.c b/block/vmdk.c
> index a11c27a..7750212 100644
> --- a/block/vmdk.c
> +++ b/block/vmdk.c
> @@ -941,6 +941,7 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
>      int ret;
>      BDRVVmdkState *s = bs->opaque;
>      uint32_t magic;
> +    Error *local_err = NULL;
>  
>      buf = vmdk_read_desc(bs->file, 0, errp);
>      if (!buf) {
> @@ -976,7 +977,13 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
>      error_setg(&s->migration_blocker, "The vmdk format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
>      g_free(buf);
>      return 0;
>  
> diff --git a/block/vpc.c b/block/vpc.c
> index 8d5886f..ed6353d 100644
> --- a/block/vpc.c
> +++ b/block/vpc.c
> @@ -422,13 +422,18 @@ static int vpc_open(BlockDriverState *bs, QDict *options, int flags,
>  #endif
>      }
>  
> -    qemu_co_mutex_init(&s->lock);
> -
>      /* Disable migration when VHD images are used */
>      error_setg(&s->migration_blocker, "The vpc format used by node '%s' "
>                 "does not support live migration",
>                 bdrv_get_device_or_node_name(bs));
> -    migrate_add_blocker(s->migration_blocker);
> +    ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        goto fail;
> +    }
> +
> +    qemu_co_mutex_init(&s->lock);
>  
>      return 0;
>  
> diff --git a/block/vvfat.c b/block/vvfat.c
> index 7b706dc..c6bf67e 100644
> --- a/block/vvfat.c
> +++ b/block/vvfat.c
> @@ -1185,21 +1185,26 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      s->sector_count = s->faked_sectors + s->sectors_per_cluster*s->cluster_count;
>  
> -    if (s->first_sectors_number == 0x40) {
> -        init_mbr(s, cyls, heads, secs);
> -    }
> -
> -    qemu_co_mutex_init(&s->lock);
> -
>      /* Disable migration when vvfat is used rw */
>      if (s->qcow) {
>          error_setg(&s->migration_blocker,
>                     "The vvfat (rw) format used by node '%s' "
>                     "does not support live migration",
>                     bdrv_get_device_or_node_name(bs));
> -        migrate_add_blocker(s->migration_blocker);
> +        ret = migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            goto fail;
> +        }
>      }
>  
> +    if (s->first_sectors_number == 0x40) {
> +        init_mbr(s, cyls, heads, secs);
> +    }
> +
> +    qemu_co_mutex_init(&s->lock);
> +
>      ret = 0;
>  fail:
>      qemu_opts_del(opts);
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index fa58877..06b6e7e 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -979,6 +979,7 @@ static void coroutine_fn v9fs_attach(void *opaque)
>      size_t offset = 7;
>      V9fsQID qid;
>      ssize_t err;
> +    Error *local_err = NULL;
>  
>      v9fs_string_init(&uname);
>      v9fs_string_init(&aname);
> @@ -1007,26 +1008,36 @@ static void coroutine_fn v9fs_attach(void *opaque)
>          clunk_fid(s, fid);
>          goto out;
>      }
> -    err = pdu_marshal(pdu, offset, "Q", &qid);
> -    if (err < 0) {
> -        clunk_fid(s, fid);
> -        goto out;
> -    }
> -    err += offset;
> -    memcpy(&s->root_qid, &qid, sizeof(qid));
> -    trace_v9fs_attach_return(pdu->tag, pdu->id,
> -                             qid.type, qid.version, qid.path);
> +
>      /*
>       * disable migration if we haven't done already.
>       * attach could get called multiple times for the same export.
>       */
>      if (!s->migration_blocker) {
> -        s->root_fid = fid;
>          error_setg(&s->migration_blocker,
>                     "Migration is disabled when VirtFS export path '%s' is mounted in the guest using mount_tag '%s'",
>                     s->ctx.fs_root ? s->ctx.fs_root : "NULL", s->tag);
> -        migrate_add_blocker(s->migration_blocker);
> +        err = migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_free(local_err);
> +            error_free(s->migration_blocker);
> +            s->migration_blocker = NULL;
> +            clunk_fid(s, fid);
> +            goto out;
> +        }
> +        s->root_fid = fid;
> +    }
> +
> +    err = pdu_marshal(pdu, offset, "Q", &qid);
> +    if (err < 0) {
> +        clunk_fid(s, fid);
> +        goto out;
>      }
> +    err += offset;
> +
> +    memcpy(&s->root_qid, &qid, sizeof(qid));
> +    trace_v9fs_attach_return(pdu->tag, pdu->id,
> +                             qid.type, qid.version, qid.path);
>  out:
>      put_fid(pdu, fidp);
>  out_nofid:
> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> index 7a15c61..d8e3d1c 100644
> --- a/hw/display/virtio-gpu.c
> +++ b/hw/display/virtio-gpu.c
> @@ -1132,6 +1132,7 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>      VirtIODevice *vdev = VIRTIO_DEVICE(qdev);
>      VirtIOGPU *g = VIRTIO_GPU(qdev);
>      bool have_virgl;
> +    Error *local_err = NULL;
>      int i;
>  
>      if (g->conf.max_outputs > VIRTIO_GPU_MAX_SCANOUTS) {
> @@ -1139,14 +1140,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>          return;
>      }
>  
> -    g->config_size = sizeof(struct virtio_gpu_config);
> -    g->virtio_config.num_scanouts = g->conf.max_outputs;
> -    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
> -                g->config_size);
> -
> -    g->req_state[0].width = 1024;
> -    g->req_state[0].height = 768;
> -
>      g->use_virgl_renderer = false;
>  #if !defined(CONFIG_VIRGL) || defined(HOST_WORDS_BIGENDIAN)
>      have_virgl = false;
> @@ -1158,6 +1151,24 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>      }
>  
>      if (virtio_gpu_virgl_enabled(g->conf)) {
> +        error_setg(&g->migration_blocker, "virgl is not yet migratable");
> +        migrate_add_blocker(g->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(g->migration_blocker);
> +            return;
> +        }
> +    }
> +
> +    g->config_size = sizeof(struct virtio_gpu_config);
> +    g->virtio_config.num_scanouts = g->conf.max_outputs;
> +    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
> +                g->config_size);
> +
> +    g->req_state[0].width = 1024;
> +    g->req_state[0].height = 768;
> +
> +    if (virtio_gpu_virgl_enabled(g->conf)) {
>          /* use larger control queue in 3d mode */
>          g->ctrl_vq   = virtio_add_queue(vdev, 256, virtio_gpu_handle_ctrl_cb);
>          g->cursor_vq = virtio_add_queue(vdev, 16, virtio_gpu_handle_cursor_cb);
> @@ -1183,11 +1194,6 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
>              dpy_gfx_replace_surface(g->scanout[i].con, NULL);
>          }
>      }
> -
> -    if (virtio_gpu_virgl_enabled(g->conf)) {
> -        error_setg(&g->migration_blocker, "virgl is not yet migratable");
> -        migrate_add_blocker(g->migration_blocker);
> -    }
>  }
>  
>  static void virtio_gpu_device_unrealize(DeviceState *qdev, Error **errp)
> diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
> index 11729ee..ec952ec 100644
> --- a/hw/intc/arm_gic_kvm.c
> +++ b/hw/intc/arm_gic_kvm.c
> @@ -510,6 +510,17 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
>          return;
>      }
>  
> +    if (!kvm_arm_gic_can_save_restore(s)) {
> +        error_setg(&s->migration_blocker, "This operating system kernel does "
> +                                          "not support vGICv2 migration");
> +        migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            return;
> +        }
> +    }
> +
>      gic_init_irqs_and_mmio(s, kvm_arm_gicv2_set_irq, NULL);
>  
>      for (i = 0; i < s->num_irq - GIC_INTERNAL; i++) {
> @@ -558,12 +569,6 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
>                              KVM_VGIC_V2_ADDR_TYPE_CPU,
>                              s->dev_fd);
>  
> -    if (!kvm_arm_gic_can_save_restore(s)) {
> -        error_setg(&s->migration_blocker, "This operating system kernel does "
> -                                          "not support vGICv2 migration");
> -        migrate_add_blocker(s->migration_blocker);
> -    }
> -
>      if (kvm_has_gsi_routing()) {
>          /* set up irq routing */
>          kvm_init_irq_routing(kvm_state);
> diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
> index fc246e0..bd4f3aa 100644
> --- a/hw/intc/arm_gicv3_its_kvm.c
> +++ b/hw/intc/arm_gicv3_its_kvm.c
> @@ -56,6 +56,19 @@ static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
>  static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
>  {
>      GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
> +    Error *local_err = NULL;
> +
> +    /*
> +     * Block migration of a KVM GICv3 ITS device: the API for saving and
> +     * restoring the state in the kernel is not yet available
> +     */
> +    error_setg(&s->migration_blocker, "vITS migration is not implemented");
> +    migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        return;
> +    }
>  
>      s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
>      if (s->dev_fd < 0) {
> @@ -73,13 +86,6 @@ static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
>  
>      gicv3_its_init_mmio(s, NULL);
>  
> -    /*
> -     * Block migration of a KVM GICv3 ITS device: the API for saving and
> -     * restoring the state in the kernel is not yet available
> -     */
> -    error_setg(&s->migration_blocker, "vITS migration is not implemented");
> -    migrate_add_blocker(s->migration_blocker);
> -
>      kvm_msi_use_devid = true;
>      kvm_gsi_direct_mapping = false;
>      kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
> diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
> index 199a439..d69dc47 100644
> --- a/hw/intc/arm_gicv3_kvm.c
> +++ b/hw/intc/arm_gicv3_kvm.c
> @@ -103,6 +103,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>  
>      gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL);
>  
> +    /* Block migration of a KVM GICv3 device: the API for saving and restoring
> +     * the state in the kernel is not yet finalised in the kernel or
> +     * implemented in QEMU.
> +     */
> +    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
> +    migrate_add_blocker(s->migration_blocker, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        error_free(s->migration_blocker);
> +        return;
> +    }
> +
>      /* Try to create the device via the device control API */
>      s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_V3, false);
>      if (s->dev_fd < 0) {
> @@ -122,13 +134,6 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>      kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
>                              KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd);
>  
> -    /* Block migration of a KVM GICv3 device: the API for saving and restoring
> -     * the state in the kernel is not yet finalised in the kernel or
> -     * implemented in QEMU.
> -     */
> -    error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
> -    migrate_add_blocker(s->migration_blocker);
> -
>      if (kvm_has_gsi_routing()) {
>          /* set up irq routing */
>          kvm_init_irq_routing(kvm_state);
> diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c
> index abeaf3d..fd14d7a 100644
> --- a/hw/misc/ivshmem.c
> +++ b/hw/misc/ivshmem.c
> @@ -840,6 +840,7 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>      uint8_t *pci_conf;
>      uint8_t attr = PCI_BASE_ADDRESS_SPACE_MEMORY |
>          PCI_BASE_ADDRESS_MEM_PREFETCH;
> +    Error *local_err = NULL;
>  
>      /* IRQFD requires MSI */
>      if (ivshmem_has_feature(s, IVSHMEM_IOEVENTFD) &&
> @@ -903,9 +904,6 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>          }
>      }
>  
> -    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
> -    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
> -
>      if (s->master == ON_OFF_AUTO_AUTO) {
>          s->master = s->vm_id == 0 ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
>      }
> @@ -913,8 +911,16 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp)
>      if (!ivshmem_is_master(s)) {
>          error_setg(&s->migration_blocker,
>                     "Migration is disabled when using feature 'peer mode' in device 'ivshmem'");
> -        migrate_add_blocker(s->migration_blocker);
> +        migrate_add_blocker(s->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            error_free(s->migration_blocker);
> +            return;
> +        }
>      }
> +
> +    vmstate_register_ram(s->ivshmem_bar2, DEVICE(s));
> +    pci_register_bar(PCI_DEVICE(s), 2, attr, s->ivshmem_bar2);
>  }
>  
>  static void ivshmem_exit(PCIDevice *dev)
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index 5b26946..c491ece 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -238,8 +238,16 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>                                 vhost_dummy_handle_output);
>      if (err != NULL) {
>          error_propagate(errp, err);
> -        close(vhostfd);
> -        return;
> +        goto close_fd;
> +    }
> +
> +    error_setg(&s->migration_blocker,
> +               "vhost-scsi does not support migration");
> +    migrate_add_blocker(s->migration_blocker, &err);
> +    if (err) {
> +        error_propagate(errp, err);
> +        error_free(s->migration_blocker);
> +        goto close_fd;
>      }
>  
>      s->dev.nvqs = VHOST_SCSI_VQ_NUM_FIXED + vs->conf.num_queues;
> @@ -252,7 +260,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>      if (ret < 0) {
>          error_setg(errp, "vhost-scsi: vhost initialization failed: %s",
>                     strerror(-ret));
> -        return;
> +        goto free_vqs;
>      }
>  
>      /* At present, channel and lun both are 0 for bootable vhost-scsi disk */
> @@ -261,9 +269,14 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>      /* Note: we can also get the minimum tpgt from kernel */
>      s->target = vs->conf.boot_tpgt;
>  
> -    error_setg(&s->migration_blocker,
> -            "vhost-scsi does not support migration");
> -    migrate_add_blocker(s->migration_blocker);
> +    return;
> +
> + free_vqs:
> +    migrate_del_blocker(s->migration_blocker);
> +    g_free(s->dev.vqs);
> + close_fd:
> +    close(vhostfd);
> +    return;
>  }
>  
>  static void vhost_scsi_unrealize(DeviceState *dev, Error **errp)
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index d396b22..cca1788 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1082,6 +1082,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>  {
>      uint64_t features;
>      int i, r, n_initialized_vqs = 0;
> +    Error *local_err = NULL;
>  
>      hdev->migration_blocker = NULL;
>  
> @@ -1158,7 +1159,12 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      }
>  
>      if (hdev->migration_blocker != NULL) {
> -        migrate_add_blocker(hdev->migration_blocker);
> +        r = migrate_add_blocker(hdev->migration_blocker, &local_err);
> +        if (local_err) {
> +            error_report_err(local_err);
> +            error_free(hdev->migration_blocker);
> +            goto fail_busyloop;
> +        }
>      }
>  
>      hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
> diff --git a/include/migration/migration.h b/include/migration/migration.h
> index 40b3697..bcbdb03 100644
> --- a/include/migration/migration.h
> +++ b/include/migration/migration.h
> @@ -243,6 +243,7 @@ void remove_migration_state_change_notifier(Notifier *notify);
>  MigrationState *migrate_init(const MigrationParams *params);
>  bool migration_is_blocked(Error **errp);
>  bool migration_in_setup(MigrationState *);
> +bool migration_is_idle(MigrationState *s);
>  bool migration_has_finished(MigrationState *);
>  bool migration_has_failed(MigrationState *);
>  /* True if outgoing migration has entered postcopy phase */
> @@ -287,8 +288,12 @@ int ram_postcopy_incoming_init(MigrationIncomingState *mis);
>   * @migrate_add_blocker - prevent migration from proceeding
>   *
>   * @reason - an error to be returned whenever migration is attempted
> + *
> + * @errp - [out] The reason (if any) we cannot block migration right now.
> + *
> + * @returns - 0 on success, -EBUSY on failure, with errp set.
>   */
> -void migrate_add_blocker(Error *reason);
> +int migrate_add_blocker(Error *reason, Error **errp);
>  
>  /**
>   * @migrate_del_blocker - remove a blocking error from migration
> diff --git a/migration/migration.c b/migration/migration.c
> index f498ab8..0d88286 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1044,6 +1044,31 @@ bool migration_in_postcopy_after_devices(MigrationState *s)
>      return migration_in_postcopy(s) && s->postcopy_after_devices;
>  }
>  
> +bool migration_is_idle(MigrationState *s)
> +{
> +    if (!s) {
> +        s = migrate_get_current();
> +    }
> +
> +    switch (s->state) {
> +    case MIGRATION_STATUS_NONE:
> +    case MIGRATION_STATUS_CANCELLED:
> +    case MIGRATION_STATUS_COMPLETED:
> +    case MIGRATION_STATUS_FAILED:
> +        return true;
> +    case MIGRATION_STATUS_SETUP:
> +    case MIGRATION_STATUS_CANCELLING:
> +    case MIGRATION_STATUS_ACTIVE:
> +    case MIGRATION_STATUS_POSTCOPY_ACTIVE:
> +    case MIGRATION_STATUS_COLO:
> +        return false;
> +    case MIGRATION_STATUS__MAX:
> +        g_assert_not_reached();
> +    }
> +
> +    return false;
> +}
> +
>  MigrationState *migrate_init(const MigrationParams *params)
>  {
>      MigrationState *s = migrate_get_current();
> @@ -1086,9 +1111,17 @@ MigrationState *migrate_init(const MigrationParams *params)
>  
>  static GSList *migration_blockers;
>  
> -void migrate_add_blocker(Error *reason)
> +int migrate_add_blocker(Error *reason, Error **errp)
>  {
> -    migration_blockers = g_slist_prepend(migration_blockers, reason);
> +    if (migration_is_idle(NULL)) {
> +        migration_blockers = g_slist_prepend(migration_blockers, reason);
> +        return 0;
> +    }
> +
> +    error_propagate(errp, error_copy(reason));
> +    error_prepend(errp, "disallowing migration blocker (migration in "
> +                      "progress) for: ");
> +    return -EBUSY;
>  }
>  
>  void migrate_del_blocker(Error *reason)
> diff --git a/stubs/migr-blocker.c b/stubs/migr-blocker.c
> index 8ab3604..a5ba18f 100644
> --- a/stubs/migr-blocker.c
> +++ b/stubs/migr-blocker.c
> @@ -2,8 +2,9 @@
>  #include "qemu-common.h"
>  #include "migration/migration.h"
>  
> -void migrate_add_blocker(Error *reason)
> +int migrate_add_blocker(Error *reason, Error **errp)
>  {
> +    return 0;
>  }
>  
>  void migrate_del_blocker(Error *reason)
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 10a9cd8..9887052 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -709,6 +709,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      uint32_t signature[3];
>      int kvm_base = KVM_CPUID_SIGNATURE;
>      int r;
> +    Error *local_err = NULL;
>  
>      memset(&cpuid_data, 0, sizeof(cpuid_data));
>  
> @@ -968,7 +969,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
>          error_setg(&invtsc_mig_blocker,
>                     "State blocked by non-migratable CPU device"
>                     " (invtsc flag)");
> -        migrate_add_blocker(invtsc_mig_blocker);
> +        r = migrate_add_blocker(invtsc_mig_blocker, &local_err);
> +        if (local_err) {
> +            error_report_err(local_err);
> +            error_free(invtsc_mig_blocker);
> +            goto fail;
> +        }
>          /* for savevm */
>          vmstate_x86_cpu.unmigratable = 1;
>      }
> @@ -976,12 +982,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      cpuid_data.cpuid.padding = 0;
>      r = kvm_vcpu_ioctl(cs, KVM_SET_CPUID2, &cpuid_data);
>      if (r) {
> -        return r;
> +        goto fail;
>      }
>  
>      r = kvm_arch_set_tsc_khz(cs);
>      if (r < 0) {
> -        return r;
> +        goto fail;
>      }
>  
>      /* vcpu's TSC frequency is either specified by user, or following
> @@ -1008,6 +1014,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
>      }
>  
>      return 0;
> +
> + fail:
> +    migrate_del_blocker(invtsc_mig_blocker);
> +    return r;
>  }
>  
>  void kvm_arch_reset_vcpu(X86CPU *cpu)

[Qemu-devel] [PATCH v6 5/5] migration: Fail migration blocker for --only-migratable

[Ashijeet Acharya]

migrate_add_blocker should rightly fail if the '--only-migratable'
option was specified and the device in use should not be able to
perform the action which results in an unmigratable VM.

Make migrate_add_blocker return -EACCES in this case.

Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>
---
 include/migration/migration.h | 2 +-
 migration/migration.c         | 7 +++++++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index bcbdb03..7881e89 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -291,7 +291,7 @@ int ram_postcopy_incoming_init(MigrationIncomingState *mis);
  *
  * @errp - [out] The reason (if any) we cannot block migration right now.
  *
- * @returns - 0 on success, -EBUSY on failure, with errp set.
+ * @returns - 0 on success, -EBUSY/-EACCES on failure, with errp set.
  */
 int migrate_add_blocker(Error *reason, Error **errp);
 
diff --git a/migration/migration.c b/migration/migration.c
index 0d88286..7dcb7d7 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1113,6 +1113,13 @@ static GSList *migration_blockers;
 
 int migrate_add_blocker(Error *reason, Error **errp)
 {
+    if (only_migratable) {
+        error_propagate(errp, error_copy(reason));
+        error_prepend(errp, "disallowing migration blocker "
+                          "(--only_migratable) for: ");
+        return -EACCES;
+    }
+
     if (migration_is_idle(NULL)) {
         migration_blockers = g_slist_prepend(migration_blockers, reason);
         return 0;
-- 
2.6.2

Re: [Qemu-devel] [PATCH v6 5/5] migration: Fail migration blocker for --only-migratable

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> migrate_add_blocker should rightly fail if the '--only-migratable'
> option was specified and the device in use should not be able to
> perform the action which results in an unmigratable VM.
> 
> Make migrate_add_blocker return -EACCES in this case.
> 
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  include/migration/migration.h | 2 +-
>  migration/migration.c         | 7 +++++++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/include/migration/migration.h b/include/migration/migration.h
> index bcbdb03..7881e89 100644
> --- a/include/migration/migration.h
> +++ b/include/migration/migration.h
> @@ -291,7 +291,7 @@ int ram_postcopy_incoming_init(MigrationIncomingState *mis);
>   *
>   * @errp - [out] The reason (if any) we cannot block migration right now.
>   *
> - * @returns - 0 on success, -EBUSY on failure, with errp set.
> + * @returns - 0 on success, -EBUSY/-EACCES on failure, with errp set.
>   */
>  int migrate_add_blocker(Error *reason, Error **errp);
>  
> diff --git a/migration/migration.c b/migration/migration.c
> index 0d88286..7dcb7d7 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1113,6 +1113,13 @@ static GSList *migration_blockers;
>  
>  int migrate_add_blocker(Error *reason, Error **errp)
>  {
> +    if (only_migratable) {
> +        error_propagate(errp, error_copy(reason));
> +        error_prepend(errp, "disallowing migration blocker "
> +                          "(--only_migratable) for: ");
> +        return -EACCES;
> +    }
> +
>      if (migration_is_idle(NULL)) {
>          migration_blockers = g_slist_prepend(migration_blockers, reason);
>          return 0;
> -- 
> 2.6.2
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v6 5/5] migration: Fail migration blocker for --only-migratable

[Juan Quintela]

Ashijeet Acharya <ashijeetacharya@gmail.com> wrote:
> migrate_add_blocker should rightly fail if the '--only-migratable'
> option was specified and the device in use should not be able to
> perform the action which results in an unmigratable VM.
>
> Make migrate_add_blocker return -EACCES in this case.
>
> Signed-off-by: Ashijeet Acharya <ashijeetacharya@gmail.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PATCH v6 0/5] Introduce a new --only-migratable option

[Dr. David Alan Gilbert]

* Ashijeet Acharya (ashijeetacharya@gmail.com) wrote:
> Previously posted series patches:
> http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg02324.html
> http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg01277.html
> http://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg00320.html
> http://lists.nongnu.org/archive/html/qemu-devel/2016-12/msg02391.html
> http://lists.nongnu.org/archive/html/qemu-devel/2016-12/msg02062.html
> 
> This series adds a new command line option "--only-migratable" which will only
> allow addition of those devices to a QEMU instance which are migratable and do
> not abruptly fail QEMU after migration.

Queued.

Dave

> Patch 1 is a preparatory patch to remove an undesirable comment and make patchew
> happy.
> 
> Patch 2 adds the new option "-only-migratable".
> 
> Patch 3 adds compatibility for various "device adding" options for both via
> command line and hotplug methods.
> 
> Patch 4 helps to fail the migration blocker if the migration is already in
> progress and thus cannot be blocked.
> Note: This patch was originally written by John Snow and I have only made few
> changes.
> 
> Patch 5 handles the special case of devices which become unmigratable
> dynamically by making call to "migrate_add_blocker". Here we fail the
> migration blocker if --only-migratable was specified.
> Eg: 9pfs fails to mount the filesystem.
> 
> Note: I have not been able to test and compile the ARM drivers for KVM. They
> are:
> hw/intc/arm_gic_kvm.c
> hw/intc/arm_gicv3_its_kvm.c
> hw/intc/arm_gicv3_kvm.c
> 
> Changes in v6:
> -make Error *local_err = NULL at some places
> -free reason at call sites
> -rebase
> -pass error_copy(reason) in error_propagate()
> 
> Changes in v5:
> -drop =0 for global variable (Dave)
> -print error message if klass=NULL (Dave)
> -deal with migration blocker before pdu_marshal (Greg)
> -call migrate_add_blocker before kvm_create_device (Dave)
> -fix typo in subject line for 4/4 (Eric)
> -fix error_free in migrate_add_blocker (Dave)
> -make migrate_add_blocker to deal with error messages itself (Peter, Greg, Dave)
> 
> Changes in v4:
> - drop diff in 9pfs for patch 4/4
> - call clunk_fid() after freeing migration_blocker
> - drop ret and use err directly
> 
> Changes in v3:
> - set s->root_fid after migrate_add_blocker
> - free migration_blocker inside v9fs_attach()
> - change back ret<0 to just ret
> - free local_err
> 
> Changes in v2:
> - change the documentation for the new option
> - add a NULL check for ObjectClass
> - break patch 3 into patch 3 and 4
> - use error_append_hint
> - return -EACCES for only-migratable
> - fix the error messages
> 
> Ashijeet Acharya (5):
>   block/vvfat: Remove the undesirable comment
>   migration: Add a new option to enable only-migratable
>   migration: Allow "device add" options to only add migratable devices
>   migration: disallow migrate_add_blocker during migration
>   migration: Fail migration blocker for --only-migratable
> 
>  block/qcow.c                  |  8 +++++++-
>  block/vdi.c                   |  8 +++++++-
>  block/vhdx.c                  | 17 +++++++++++------
>  block/vmdk.c                  |  9 ++++++++-
>  block/vpc.c                   | 11 ++++++++---
>  block/vvfat.c                 | 20 ++++++++++++--------
>  hw/9pfs/9p.c                  | 33 +++++++++++++++++++++-----------
>  hw/display/virtio-gpu.c       | 32 ++++++++++++++++++-------------
>  hw/intc/arm_gic_kvm.c         | 17 +++++++++++------
>  hw/intc/arm_gicv3_its_kvm.c   | 20 +++++++++++++-------
>  hw/intc/arm_gicv3_kvm.c       | 19 ++++++++++++-------
>  hw/misc/ivshmem.c             | 14 ++++++++++----
>  hw/scsi/vhost-scsi.c          | 25 ++++++++++++++++++------
>  hw/usb/bus.c                  | 19 +++++++++++++++++++
>  hw/virtio/vhost.c             |  8 +++++++-
>  include/migration/migration.h | 10 +++++++++-
>  migration/migration.c         | 44 +++++++++++++++++++++++++++++++++++++++++--
>  qdev-monitor.c                |  9 +++++++++
>  qemu-options.hx               |  9 +++++++++
>  stubs/migr-blocker.c          |  3 ++-
>  target/i386/kvm.c             | 16 +++++++++++++---
>  vl.c                          |  4 ++++
>  22 files changed, 273 insertions(+), 82 deletions(-)
> 
> -- 
> 2.6.2
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK