[Qemu-devel] [PULL 0/8] Miscellaneous patches for 2019-04-02

[Qemu-devel] [PULL 0/8] Miscellaneous patches for 2019-04-02

[Markus Armbruster]

Two regression fixes for 4.0:

* Unbreak migration for Xen
* Unbreak accelerator fallback

Three safe bug fixes thrown in:

* Fix error location of qemu-system-FOO positional arguments
* COLOStatus minor doc fix
* COLOStatus member rename to conform to naming rules

Markus Armbruster (6):
  vl: Fix error location of positional arguments
  Revert "vl: Fix to create migration object before block backends
    again"
  Revert "migration: move only_migratable to MigrationState"
  migration: Support adding migration blockers earlier
  vl: Document dependencies hiding in global and compat props
  accel: Unbreak accelerator fallback

Zhang Chen (2):
  qapi/migration.json: Fix ColoStatus member last_mode's version
  qapi/migration.json: Rename COLOStatus last_mode to last-mode

 accel/accel.c           |  3 ++-
 include/sysemu/sysemu.h |  1 +
 migration/migration.c   | 11 ++++++----
 migration/migration.h   |  3 ---
 migration/savevm.c      |  2 +-
 qapi/migration.json     |  6 +++---
 vl.c                    | 45 +++++++++++++++++++++++++++--------------
 7 files changed, 44 insertions(+), 27 deletions(-)

-- 
2.17.2

[Qemu-devel] [PULL 1/8] vl: Fix error location of positional arguments

[Markus Armbruster]

We blame badness in positional arguments on the last option argument:

    $ qemu-system-x86_64 -vnc :1 bad.img
    qemu-system-x86_64: -vnc :1: Could not open 'foo': No such file or directory

I believe we've done this ever since we reported locations.  Fix it to

    qemu-system-x86_64: bad.img: Could not open 'bad.img': No such file or directory

Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190318183312.4684-1-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
---
 vl.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/vl.c b/vl.c
index d61d5604e5..24572de0bd 100644
--- a/vl.c
+++ b/vl.c
@@ -3119,6 +3119,7 @@ int main(int argc, char **argv, char **envp)
         if (optind >= argc)
             break;
         if (argv[optind][0] != '-') {
+            loc_set_cmdline(argv, optind, 1);
             drive_add(IF_DEFAULT, 0, argv[optind++], HD_OPTS);
         } else {
             const QEMUOption *popt;
-- 
2.17.2

[Qemu-devel] [PULL 2/8] qapi/migration.json: Fix ColoStatus member last_mode's version

[Markus Armbruster]

From: Zhang Chen <chen.zhang@intel.com>

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20190326174510.13303-1-chen.zhang@intel.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message tweaked as per Eric's review]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/migration.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qapi/migration.json b/qapi/migration.json
index cfde29acf8..798c6ac2df 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1382,7 +1382,7 @@
 #
 # @last_mode: COLO last running mode. If COLO is running, this field
 #             will return same like mode field, after failover we can
-#             use this field to get last colo mode. (since 4.1)
+#             use this field to get last colo mode. (since 4.0)
 #
 # @reason: describes the reason for the COLO exit.
 #
-- 
2.17.2

[Qemu-devel] [PULL 3/8] qapi/migration.json: Rename COLOStatus last_mode to last-mode

[Markus Armbruster]

From: Zhang Chen <chen.zhang@intel.com>

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20190402085521.17973-1-chen.zhang@intel.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message rephrased]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/migration.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qapi/migration.json b/qapi/migration.json
index 798c6ac2df..9cfbaf8c6c 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1380,7 +1380,7 @@
 # @mode: COLO running mode. If COLO is running, this field will return
 #        'primary' or 'secondary'.
 #
-# @last_mode: COLO last running mode. If COLO is running, this field
+# @last-mode: COLO last running mode. If COLO is running, this field
 #             will return same like mode field, after failover we can
 #             use this field to get last colo mode. (since 4.0)
 #
@@ -1389,7 +1389,7 @@
 # Since: 3.1
 ##
 { 'struct': 'COLOStatus',
-  'data': { 'mode': 'COLOMode', 'last_mode': 'COLOMode',
+  'data': { 'mode': 'COLOMode', 'last-mode': 'COLOMode',
             'reason': 'COLOExitReason' } }
 
 ##
-- 
2.17.2

[Qemu-devel] [PULL 4/8] Revert "vl: Fix to create migration object before block backends again"

[Markus Armbruster]

This reverts commit e60483f2f8498ae08ae79ca4c6fb03a3317f5e1e.

Recent commit cda4aa9a5a0 moved block backend creation before machine
property evaluation.  This broke block backends registering migration
blockers.  Commit e60483f2f84 fixed it by moving migration object
creation before block backend creation.  This broke migration with
Xen.  Turns out we need to configure the accelerator before we create
the migration object so that Xen's accelerator compat properties get
applied.  Revert the flawed commit.  This fixes the Xen regression,
but brings back the block backend regression.  The next commits will
fix it again.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190401090827.20793-2-armbru@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
 vl.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/vl.c b/vl.c
index 24572de0bd..9b215341a3 100644
--- a/vl.c
+++ b/vl.c
@@ -4277,17 +4277,10 @@ int main(int argc, char **argv, char **envp)
         exit(0);
     }
 
-    /*
-     * Migration object can only be created after global properties
-     * are applied correctly.
-     */
-    migration_object_init();
-
     /*
      * Note: we need to create block backends before
      * machine_set_property(), so machine properties can refer to
-     * them, and after migration_object_init(), so we can create
-     * migration blockers.
+     * them.
      */
     configure_blockdev(&bdo_queue, machine_class, snapshot);
 
@@ -4305,6 +4298,12 @@ int main(int argc, char **argv, char **envp)
                      machine_class->name, machine_class->deprecation_reason);
     }
 
+    /*
+     * Migration object can only be created after global properties
+     * are applied correctly.
+     */
+    migration_object_init();
+
     if (qtest_chrdev) {
         qtest_init(qtest_chrdev, qtest_log, &error_fatal);
     }
-- 
2.17.2

[Qemu-devel] [PULL 5/8] Revert "migration: move only_migratable to MigrationState"

[Markus Armbruster]

This reverts commit 3df663e575f1876d7f3bc684f80e72fca0703d39.
This reverts commit b605c47b57b58e61a901a50a0762dccf43d94783.

Command line option --only-migratable is for disallowing any
configuration that can block migration.

Initially, --only-migratable set global variable @only_migratable.

Commit 3df663e575 "migration: move only_migratable to MigrationState"
replaced it by MigrationState member @only_migratable.  That was a
mistake.

First, it doesn't make sense on the design level.  MigrationState
captures the state of an individual migration, but --only-migratable
isn't a property of an individual migration, it's a restriction on
QEMU configuration.  With fault tolerance, we could have several
migrations at once.  --only-migratable would certainly protect all of
them.  Storing it in MigrationState feels inappropriate.

Second, it contributes to a dependency cycle that manifests itself as
a bug now.

Putting @only_migratable into MigrationState means its available only
after migration_object_init().

We can't set it before migration_object_init(), so we delay setting it
with a global property (this is fixup commit b605c47b57 "migration:
fix handling for --only-migratable").

We can't get it before migration_object_init(), so anything that uses
it can only run afterwards.

Since migrate_add_blocker() needs to obey --only-migratable, any code
adding migration blockers can run only afterwards.  This contributes
to the following dependency cycle:

* configure_blockdev() must run before machine_set_property()
  so machine properties can refer to block backends

* machine_set_property() before configure_accelerator()
  so machine properties like kvm-irqchip get applied

* configure_accelerator() before migration_object_init()
  so that Xen's accelerator compat properties get applied.

* migration_object_init() before configure_blockdev()
  so configure_blockdev() can add migration blockers

The cycle was closed when recent commit cda4aa9a5a0 "Create block
backends before setting machine properties" added the first
dependency, and satisfied it by violating the last one.  Broke block
backends that add migration blockers.

Moving @only_migratable into MigrationState was a mistake.  Revert it.

This doesn't quite break the "migration_object_init() before
configure_blockdev() dependency, since migrate_add_blocker() still has
another dependency on migration_object_init().  To be addressed the
next commit.

Note that the reverted commit made -only-migratable sugar for -global
migration.only-migratable=on below the hood.  Documentation has only
ever mentioned -only-migratable.  This commit removes the arcane &
undocumented alternative to -only-migratable again.  Nobody should be
using it.

Conflicts:
	include/migration/misc.h
	migration/migration.c
	migration/migration.h
	vl.c

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190401090827.20793-3-armbru@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
 include/sysemu/sysemu.h | 1 +
 migration/migration.c   | 5 ++---
 migration/migration.h   | 3 ---
 migration/savevm.c      | 2 +-
 vl.c                    | 9 ++-------
 5 files changed, 6 insertions(+), 14 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 6065d9e420..5f133cae83 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -14,6 +14,7 @@
 /* vl.c */
 
 extern const char *bios_name;
+extern int only_migratable;
 extern const char *qemu_name;
 extern QemuUUID qemu_uuid;
 extern bool qemu_uuid_set;
diff --git a/migration/migration.c b/migration/migration.c
index 69f75124c9..f6076e5295 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1707,7 +1707,7 @@ static GSList *migration_blockers;
 
 int migrate_add_blocker(Error *reason, Error **errp)
 {
-    if (migrate_get_current()->only_migratable) {
+    if (only_migratable) {
         error_propagate_prepend(errp, error_copy(reason),
                                 "disallowing migration blocker "
                                 "(--only_migratable) for: ");
@@ -3337,7 +3337,7 @@ void migration_global_dump(Monitor *mon)
     monitor_printf(mon, "store-global-state: %s\n",
                    ms->store_global_state ? "on" : "off");
     monitor_printf(mon, "only-migratable: %s\n",
-                   ms->only_migratable ? "on" : "off");
+                   only_migratable ? "on" : "off");
     monitor_printf(mon, "send-configuration: %s\n",
                    ms->send_configuration ? "on" : "off");
     monitor_printf(mon, "send-section-footer: %s\n",
@@ -3352,7 +3352,6 @@ void migration_global_dump(Monitor *mon)
 static Property migration_properties[] = {
     DEFINE_PROP_BOOL("store-global-state", MigrationState,
                      store_global_state, true),
-    DEFINE_PROP_BOOL("only-migratable", MigrationState, only_migratable, false),
     DEFINE_PROP_BOOL("send-configuration", MigrationState,
                      send_configuration, true),
     DEFINE_PROP_BOOL("send-section-footer", MigrationState,
diff --git a/migration/migration.h b/migration/migration.h
index 0f986935e1..438f17edad 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -219,9 +219,6 @@ struct MigrationState
      */
     bool store_global_state;
 
-    /* Whether the VM is only allowing for migratable devices */
-    bool only_migratable;
-
     /* Whether we send QEMU_VM_CONFIGURATION during migration */
     bool send_configuration;
     /* Whether we send section footer during migration */
diff --git a/migration/savevm.c b/migration/savevm.c
index 1415001d1c..34bcad3807 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2844,7 +2844,7 @@ void vmstate_register_ram_global(MemoryRegion *mr)
 bool vmstate_check_only_migratable(const VMStateDescription *vmsd)
 {
     /* check needed if --only-migratable is specified */
-    if (!migrate_get_current()->only_migratable) {
+    if (!only_migratable) {
         return true;
     }
 
diff --git a/vl.c b/vl.c
index 9b215341a3..0d517a09e1 100644
--- a/vl.c
+++ b/vl.c
@@ -185,6 +185,7 @@ const char *prom_envs[MAX_PROM_ENVS];
 int boot_menu;
 bool boot_strict;
 uint8_t *boot_splash_filedata;
+int only_migratable; /* turn it off unless user states otherwise */
 bool wakeup_suspend_enabled;
 
 int icount_align_option;
@@ -3800,13 +3801,7 @@ int main(int argc, char **argv, char **envp)
                 incoming = optarg;
                 break;
             case QEMU_OPTION_only_migratable:
-                /*
-                 * TODO: we can remove this option one day, and we
-                 * should all use:
-                 *
-                 * "-global migration.only-migratable=true"
-                 */
-                qemu_global_option("migration.only-migratable=true");
+                only_migratable = 1;
                 break;
             case QEMU_OPTION_nodefaults:
                 has_defaults = 0;
-- 
2.17.2

[Qemu-devel] [PULL 6/8] migration: Support adding migration blockers earlier

[Markus Armbruster]

migrate_add_blocker() asserts we have a current_migration object, in
migrate_get_current().  We do only after migration_object_init().

This contributes to the following dependency cycle:

* configure_blockdev() must run before machine_set_property()
  so machine properties can refer to block backends

* machine_set_property() before configure_accelerator()
  so machine properties like kvm-irqchip get applied

* configure_accelerator() before migration_object_init()
  so that Xen's accelerator compat properties get applied.

* migration_object_init() before configure_blockdev()
  so configure_blockdev() can add migration blockers

The cycle was closed when recent commit cda4aa9a5a0 "Create block
backends before setting machine properties" added the first
dependency, and satisfied it by violating the last one.  Broke block
backends that add migration blockers, as demonstrated by qemu-iotests
055.

To fix it, break the last dependency: make migrate_add_blocker()
usable before migration_object_init().

The previous commit already removed the use of migrate_get_current()
from migrate_add_blocker() itself.  Didn't quite do the trick, as
there's another one hiding in migration_is_idle().

The use there isn't actually necessary: when no migration object has
been created yet, migration is surely idle.  Make migration_is_idle()
return true then.

Fixes: cda4aa9a5a08777cf13e164c0543bd4888b8adce
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190401090827.20793-4-armbru@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
 migration/migration.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index f6076e5295..609e0df5d0 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1646,7 +1646,11 @@ bool migration_in_postcopy_after_devices(MigrationState *s)
 
 bool migration_is_idle(void)
 {
-    MigrationState *s = migrate_get_current();
+    MigrationState *s = current_migration;
+
+    if (!s) {
+        return true;
+    }
 
     switch (s->state) {
     case MIGRATION_STATUS_NONE:
-- 
2.17.2

[Qemu-devel] [PULL 7/8] vl: Document dependencies hiding in global and compat props

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190401090827.20793-5-armbru@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
 vl.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/vl.c b/vl.c
index 0d517a09e1..c696ad2a13 100644
--- a/vl.c
+++ b/vl.c
@@ -4286,16 +4286,36 @@ int main(int argc, char **argv, char **envp)
     current_machine->maxram_size = maxram_size;
     current_machine->ram_slots = ram_slots;
 
+    /*
+     * Note: uses machine properties such as kernel-irqchip, must run
+     * after machine_set_property().
+     */
     configure_accelerator(current_machine, argv[0]);
 
+    /*
+     * Beware, QOM objects created before this point miss global and
+     * compat properties.
+     *
+     * Global properties get set up by qdev_prop_register_global(),
+     * called from user_register_global_props(), and certain option
+     * desugaring.  Also in CPU feature desugaring (buried in
+     * parse_cpu_model()), which happens below this point, but may
+     * only target the CPU type, which can only be created after
+     * parse_cpu_model() returned the type.
+     *
+     * Machine compat properties: object_set_machine_compat_props().
+     * Accelerator compat props: object_set_accelerator_compat_props(),
+     * called from configure_accelerator().
+     */
+
     if (!qtest_enabled() && machine_class->deprecation_reason) {
         error_report("Machine type '%s' is deprecated: %s",
                      machine_class->name, machine_class->deprecation_reason);
     }
 
     /*
-     * Migration object can only be created after global properties
-     * are applied correctly.
+     * Note: creates a QOM object, must run only after global and
+     * compat properties have been set up.
      */
     migration_object_init();
 
-- 
2.17.2

[Qemu-devel] [PULL 8/8] accel: Unbreak accelerator fallback

[Markus Armbruster]

When the user specifies a list of accelerators, we pick the first one
that initializes successfully.  Recent commit 1a3ec8c1564 broke that.
Reproducer:

    $ qemu-system-x86_64 --machine accel=xen:tcg
    xencall: error: Could not obtain handle on privileged command interface: No such file or directory
    xen be core: xen be core: can't open xen interface
    can't open xen interface
    qemu-system-x86_64: failed to initialize Xen: Operation not permitted
    qemu-system-x86_64: /home/armbru/work/qemu/qom/object.c:436: object_set_accelerator_compat_props: Assertion `!object_compat_props[0]' failed.

Root cause: we register accelerator compat properties even when the
accelerator fails.  The failed assertion is
object_set_accelerator_compat_props() telling us off.  Fix by calling
it only for the accelerator that succeeded.

Fixes: 1a3ec8c1564f51628cce10d435a2e22559ea29fd
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20190401090827.20793-6-armbru@redhat.com>
---
 accel/accel.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/accel/accel.c b/accel/accel.c
index 8deb475b5d..454fef9d92 100644
--- a/accel/accel.c
+++ b/accel/accel.c
@@ -65,8 +65,9 @@ static int accel_init_machine(AccelClass *acc, MachineState *ms)
         ms->accelerator = NULL;
         *(acc->allowed) = false;
         object_unref(OBJECT(accel));
+    } else {
+        object_set_accelerator_compat_props(acc->compat_props);
     }
-    object_set_accelerator_compat_props(acc->compat_props);
     return ret;
 }
 
-- 
2.17.2

Re: [Qemu-devel] [PULL 0/8] Miscellaneous patches for 2019-04-02

[Peter Maydell]

Hi Markus -- this pullreq seems to be missing the git url and tag
to merge. Could you fix it up and resend the cover letter, please?

thanks
-- PMM


On Tue, 2 Apr 2019 at 20:25, Markus Armbruster <armbru@redhat.com> wrote:
>
> Two regression fixes for 4.0:
>
> * Unbreak migration for Xen
> * Unbreak accelerator fallback
>
> Three safe bug fixes thrown in:
>
> * Fix error location of qemu-system-FOO positional arguments
> * COLOStatus minor doc fix
> * COLOStatus member rename to conform to naming rules
>
> Markus Armbruster (6):
>   vl: Fix error location of positional arguments
>   Revert "vl: Fix to create migration object before block backends
>     again"
>   Revert "migration: move only_migratable to MigrationState"
>   migration: Support adding migration blockers earlier
>   vl: Document dependencies hiding in global and compat props
>   accel: Unbreak accelerator fallback
>
> Zhang Chen (2):
>   qapi/migration.json: Fix ColoStatus member last_mode's version
>   qapi/migration.json: Rename COLOStatus last_mode to last-mode
>
>  accel/accel.c           |  3 ++-
>  include/sysemu/sysemu.h |  1 +
>  migration/migration.c   | 11 ++++++----
>  migration/migration.h   |  3 ---
>  migration/savevm.c      |  2 +-
>  qapi/migration.json     |  6 +++---
>  vl.c                    | 45 +++++++++++++++++++++++++++--------------
>  7 files changed, 44 insertions(+), 27 deletions(-)
>
> --
> 2.17.2
>
>