[Qemu-devel] [PULL 00/11] x86 queue, 2017-02-27

[Qemu-devel] [PULL 00/11] x86 queue, 2017-02-27

[Eduardo Habkost]

The following changes since commit 3b1d8169844fafee184366b0e0d7080534758b4d:

  tests-aio-multithread: use atomic_read properly (2017-02-27 12:54:08 +0000)

are available in the git repository at:

  git://github.com/ehabkost/qemu.git tags/x86-pull-request

for you to fetch changes up to b8097deb359bbbd92592b9670adfe9e245b2d0bd:

  i386: Improve query-cpu-model-expansion full mode (2017-02-27 13:23:35 -0300)

----------------------------------------------------------------
x86 queue, 2017-02-27

"-cpu max" and query-cpu-model-expansion support for x86. This
should be the last x86 pull request before 2.9 soft freeze.

----------------------------------------------------------------

Eduardo Habkost (11):
  i386: Unset cannot_destroy_with_object_finalize_yet on "host" model
  i386: Add ordering field to CPUClass
  i386: Rename X86CPU::host_features to X86CPU::max_features
  i386: Reorganize and document CPUID initialization steps
  qapi-schema: Comment about full expansion of non-migration-safe models
  i386: Create "max" CPU model
  i386: Make "max" model not use any host CPUID info on TCG
  i386: Don't set CPUClass::cpu_def on "max" model
  i386: Define static "base" CPU model
  i386: Implement query-cpu-model-expansion QMP command
  i386: Improve query-cpu-model-expansion full mode

 qapi-schema.json      |   9 +
 target/i386/cpu-qom.h |   8 +-
 target/i386/cpu.h     |   2 +-
 monitor.c             |   4 +-
 target/i386/cpu.c     | 455 +++++++++++++++++++++++++++++++++++++++++---------
 5 files changed, 397 insertions(+), 81 deletions(-)

-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 01/11] i386: Unset cannot_destroy_with_object_finalize_yet on "host" model

[Eduardo Habkost]

The class is now safe because the assert(kvm_enabled()) line was
removed by commit e435601058e656e6d24e3e87b187e5518f7bf16a.

Message-Id: <20170119210449.11991-2-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index b6f157dca3..2adee42035 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1571,8 +1571,6 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void *data)
      */
 
     dc->props = host_x86_cpu_properties;
-    /* Reason: host_x86_cpu_initfn() dies when !kvm_enabled() */
-    dc->cannot_destroy_with_object_finalize_yet = true;
 }
 
 static void host_x86_cpu_initfn(Object *obj)
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 02/11] i386: Add ordering field to CPUClass

[Eduardo Habkost]

Instead of using kvm_enabled to order the "-cpu help" list, use a
new "ordering" field for that.

Message-Id: <20170119210449.11991-3-ehabkost@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu-qom.h | 2 ++
 target/i386/cpu.c     | 8 ++++----
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/target/i386/cpu-qom.h b/target/i386/cpu-qom.h
index 8cd607e9a2..14abd0a8c1 100644
--- a/target/i386/cpu-qom.h
+++ b/target/i386/cpu-qom.h
@@ -48,6 +48,7 @@ typedef struct X86CPUDefinition X86CPUDefinition;
  * X86CPUClass:
  * @cpu_def: CPU model definition
  * @kvm_required: Whether CPU model requires KVM to be enabled.
+ * @ordering: Ordering on the "-cpu help" CPU model list.
  * @migration_safe: See CpuDefinitionInfo::migration_safe
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
@@ -63,6 +64,7 @@ typedef struct X86CPUClass {
     X86CPUDefinition *cpu_def;
 
     bool kvm_required;
+    int ordering;
     bool migration_safe;
 
     /* Optional description of CPU model.
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2adee42035..8efe4de89e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1550,6 +1550,7 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void *data)
     uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
 
     xcc->kvm_required = true;
+    xcc->ordering = 9;
 
     host_cpuid(0x0, 0, &eax, &ebx, &ecx, &edx);
     x86_cpu_vendor_words2str(host_cpudef.vendor, ebx, edx, ecx);
@@ -2126,7 +2127,7 @@ static void listflags(FILE *f, fprintf_function print, const char **featureset)
     }
 }
 
-/* Sort alphabetically by type name, listing kvm_required models last. */
+/* Sort alphabetically by type name, respecting X86CPUClass::ordering. */
 static gint x86_cpu_list_compare(gconstpointer a, gconstpointer b)
 {
     ObjectClass *class_a = (ObjectClass *)a;
@@ -2135,9 +2136,8 @@ static gint x86_cpu_list_compare(gconstpointer a, gconstpointer b)
     X86CPUClass *cc_b = X86_CPU_CLASS(class_b);
     const char *name_a, *name_b;
 
-    if (cc_a->kvm_required != cc_b->kvm_required) {
-        /* kvm_required items go last */
-        return cc_a->kvm_required ? 1 : -1;
+    if (cc_a->ordering != cc_b->ordering) {
+        return cc_a->ordering - cc_b->ordering;
     } else {
         name_a = object_class_get_name(class_a);
         name_b = object_class_get_name(class_b);
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 03/11] i386: Rename X86CPU::host_features to X86CPU::max_features

[Eduardo Habkost]

Rename the field and add a small comment to make its purpose
clearer.

Message-Id: <20170119210449.11991-4-ehabkost@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.h | 2 +-
 target/i386/cpu.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 8df124f332..5f75ab1b15 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1211,7 +1211,7 @@ struct X86CPU {
     bool enforce_cpuid;
     bool expose_kvm;
     bool migratable;
-    bool host_features;
+    bool max_features; /* Enable all supported features automatically */
     uint32_t apic_id;
 
     /* Enables publishing of TSC increment and Local APIC bus frequencies to
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 8efe4de89e..87affbfcd6 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1583,7 +1583,7 @@ static void host_x86_cpu_initfn(Object *obj)
     /* We can't fill the features array here because we don't know yet if
      * "migratable" is true or false.
      */
-    cpu->host_features = true;
+    cpu->max_features = true;
 
     /* If KVM is disabled, x86_cpu_realizefn() will report an error later */
     if (kvm_enabled()) {
@@ -3101,12 +3101,12 @@ static void x86_cpu_load_features(X86CPU *cpu, Error **errp)
     GList *l;
     Error *local_err = NULL;
 
-    /*TODO: cpu->host_features incorrectly overwrites features
+    /*TODO: cpu->max_features incorrectly overwrites features
      * set using "feat=on|off". Once we fix this, we can convert
      * plus_features & minus_features to global properties
      * inside x86_cpu_parse_featurestr() too.
      */
-    if (cpu->host_features) {
+    if (cpu->max_features) {
         for (w = 0; w < FEATURE_WORDS; w++) {
             env->features[w] =
                 x86_cpu_get_supported_feature_word(w, cpu->migratable);
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 04/11] i386: Reorganize and document CPUID initialization steps

[Eduardo Habkost]

CPU runnability checks and CPU model expansion have slightly
different requirements. Document the steps involved in loading a
CPU model and realizing a CPU, so their requirements and purpose
are clearly defined.

This patch doesn't change any implementation. It just add
comments, rename the x86_cpu_load_features() function for clarity
(so it won't be confused with x86_cpu_load_def()), and move
x86_cpu_filter_features() closer to it.

Message-Id: <20170116211124.29245-2-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.c | 102 +++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 71 insertions(+), 31 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 87affbfcd6..b614887a61 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -2059,7 +2059,7 @@ static void x86_cpu_parse_featurestr(const char *typename, char *features,
     }
 }
 
-static void x86_cpu_load_features(X86CPU *cpu, Error **errp);
+static void x86_cpu_expand_features(X86CPU *cpu, Error **errp);
 static int x86_cpu_filter_features(X86CPU *cpu);
 
 /* Check for missing features that may prevent the CPU class from
@@ -2082,9 +2082,9 @@ static void x86_cpu_class_check_missing_features(X86CPUClass *xcc,
 
     xc = X86_CPU(object_new(object_class_get_name(OBJECT_CLASS(xcc))));
 
-    x86_cpu_load_features(xc, &err);
+    x86_cpu_expand_features(xc, &err);
     if (err) {
-        /* Errors at x86_cpu_load_features should never happen,
+        /* Errors at x86_cpu_expand_features should never happen,
          * but in case it does, just report the model as not
          * runnable at all using the "type" property.
          */
@@ -2245,31 +2245,6 @@ static uint32_t x86_cpu_get_supported_feature_word(FeatureWord w,
     return r;
 }
 
-/*
- * Filters CPU feature words based on host availability of each feature.
- *
- * Returns: 0 if all flags are supported by the host, non-zero otherwise.
- */
-static int x86_cpu_filter_features(X86CPU *cpu)
-{
-    CPUX86State *env = &cpu->env;
-    FeatureWord w;
-    int rv = 0;
-
-    for (w = 0; w < FEATURE_WORDS; w++) {
-        uint32_t host_feat =
-            x86_cpu_get_supported_feature_word(w, false);
-        uint32_t requested_features = env->features[w];
-        env->features[w] &= host_feat;
-        cpu->filtered_features[w] = requested_features & ~env->features[w];
-        if (cpu->filtered_features[w]) {
-            rv = 1;
-        }
-    }
-
-    return rv;
-}
-
 static void x86_cpu_report_filtered_features(X86CPU *cpu)
 {
     FeatureWord w;
@@ -3093,8 +3068,47 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
     env->features[FEAT_XSAVE_COMP_HI] = mask >> 32;
 }
 
-/* Load CPUID data based on configured features */
-static void x86_cpu_load_features(X86CPU *cpu, Error **errp)
+/***** Steps involved on loading and filtering CPUID data
+ *
+ * When initializing and realizing a CPU object, the steps
+ * involved in setting up CPUID data are:
+ *
+ * 1) Loading CPU model definition (X86CPUDefinition). This is
+ *    implemented by x86_cpu_load_def() and should be completely
+ *    transparent, as it is done automatically by instance_init.
+ *    No code should need to look at X86CPUDefinition structs
+ *    outside instance_init.
+ *
+ * 2) CPU expansion. This is done by realize before CPUID
+ *    filtering, and will make sure host/accelerator data is
+ *    loaded for CPU models that depend on host capabilities
+ *    (e.g. "host"). Done by x86_cpu_expand_features().
+ *
+ * 3) CPUID filtering. This initializes extra data related to
+ *    CPUID, and checks if the host supports all capabilities
+ *    required by the CPU. Runnability of a CPU model is
+ *    determined at this step. Done by x86_cpu_filter_features().
+ *
+ * Some operations don't require all steps to be performed.
+ * More precisely:
+ *
+ * - CPU instance creation (instance_init) will run only CPU
+ *   model loading. CPU expansion can't run at instance_init-time
+ *   because host/accelerator data may be not available yet.
+ * - CPU realization will perform both CPU model expansion and CPUID
+ *   filtering, and return an error in case one of them fails.
+ * - query-cpu-definitions needs to run all 3 steps. It needs
+ *   to run CPUID filtering, as the 'unavailable-features'
+ *   field is set based on the filtering results.
+ * - The query-cpu-model-expansion QMP command only needs to run
+ *   CPU model loading and CPU expansion. It should not filter
+ *   any CPUID data based on host capabilities.
+ */
+
+/* Expand CPU configuration data, based on configured features
+ * and host/accelerator capabilities when appropriate.
+ */
+static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
     FeatureWord w;
@@ -3171,6 +3185,32 @@ out:
     }
 }
 
+/*
+ * Finishes initialization of CPUID data, filters CPU feature
+ * words based on host availability of each feature.
+ *
+ * Returns: 0 if all flags are supported by the host, non-zero otherwise.
+ */
+static int x86_cpu_filter_features(X86CPU *cpu)
+{
+    CPUX86State *env = &cpu->env;
+    FeatureWord w;
+    int rv = 0;
+
+    for (w = 0; w < FEATURE_WORDS; w++) {
+        uint32_t host_feat =
+            x86_cpu_get_supported_feature_word(w, false);
+        uint32_t requested_features = env->features[w];
+        env->features[w] &= host_feat;
+        cpu->filtered_features[w] = requested_features & ~env->features[w];
+        if (cpu->filtered_features[w]) {
+            rv = 1;
+        }
+    }
+
+    return rv;
+}
+
 #define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \
                            (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \
                            (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3)
@@ -3198,7 +3238,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         return;
     }
 
-    x86_cpu_load_features(cpu, &local_err);
+    x86_cpu_expand_features(cpu, &local_err);
     if (local_err) {
         goto out;
     }
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 05/11] qapi-schema: Comment about full expansion of non-migration-safe models

[Eduardo Habkost]

Add a note warning that static expansion may not be 100% accurate
when the CPU model is not migration-safe. This will be the case
on x86 when expansing the "host" CPU model, because there are
"host" features that can't have a migration-safe representation
(e.g. "host-cache-info").

Message-Id: <20170116211124.29245-3-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 qapi-schema.json | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/qapi-schema.json b/qapi-schema.json
index 150ee98e9e..2051c46cfc 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4274,6 +4274,15 @@
 #        migration-safe, but allows tooling to get an insight and work with
 #        model details.
 #
+# Note: When a non-migration-safe CPU model is expanded in static mode, some
+# features enabled by the CPU model may be omitted, because they can't be
+# implemented by a static CPU model definition (e.g. cache info passthrough and
+# PMU passthrough in x86). If you need an accurate representation of the
+# features enabled by a non-migration-safe CPU model, use @full. If you need a
+# static representation that will keep ABI compatibility even when changing QEMU
+# version or machine-type, use @static (but keep in mind that some features may
+# be omitted).
+#
 # Since: 2.8.0
 ##
 { 'enum': 'CpuModelExpansionType',
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 06/11] i386: Create "max" CPU model

[Eduardo Habkost]

Rename the existing "host" CPU model to "max, and set it to
kvm_enabled=false. The new "max" CPU model will be able to enable
all features supported by TCG out of the box, because its logic
is based on x86_cpu_get_supported_feature_word(), which already
works with TCG.

A new KVM-specific "host" class was added, that simply inherits
everything from "max" except the 'ordering' and 'description'
fields.

Message-Id: <20170222183919.11928-2-ehabkost@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.c | 46 ++++++++++++++++++++++++++++++++--------------
 1 file changed, 32 insertions(+), 14 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index b614887a61..a53fafa205 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1503,15 +1503,15 @@ void x86_cpu_change_kvm_default(const char *prop, const char *value)
 static uint32_t x86_cpu_get_supported_feature_word(FeatureWord w,
                                                    bool migratable_only);
 
-#ifdef CONFIG_KVM
-
 static bool lmce_supported(void)
 {
-    uint64_t mce_cap;
+    uint64_t mce_cap = 0;
 
+#ifdef CONFIG_KVM
     if (kvm_ioctl(kvm_state, KVM_X86_GET_MCE_CAP_SUPPORTED, &mce_cap) < 0) {
         return false;
     }
+#endif
 
     return !!(mce_cap & MCG_LMCE_P);
 }
@@ -1533,23 +1533,22 @@ static int cpu_x86_fill_model_id(char *str)
 
 static X86CPUDefinition host_cpudef;
 
-static Property host_x86_cpu_properties[] = {
+static Property max_x86_cpu_properties[] = {
     DEFINE_PROP_BOOL("migratable", X86CPU, migratable, true),
     DEFINE_PROP_BOOL("host-cache-info", X86CPU, cache_info_passthrough, false),
     DEFINE_PROP_END_OF_LIST()
 };
 
-/* class_init for the "host" CPU model
+/* class_init for the "max" CPU model
  *
  * This function may be called before KVM is initialized.
  */
-static void host_x86_cpu_class_init(ObjectClass *oc, void *data)
+static void max_x86_cpu_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
     uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
 
-    xcc->kvm_required = true;
     xcc->ordering = 9;
 
     host_cpuid(0x0, 0, &eax, &ebx, &ecx, &edx);
@@ -1564,17 +1563,16 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void *data)
 
     xcc->cpu_def = &host_cpudef;
     xcc->model_description =
-        "KVM processor with all supported host features "
-        "(only available in KVM mode)";
+        "Enables all features supported by the accelerator in the current host";
 
     /* level, xlevel, xlevel2, and the feature words are initialized on
      * instance_init, because they require KVM to be initialized.
      */
 
-    dc->props = host_x86_cpu_properties;
+    dc->props = max_x86_cpu_properties;
 }
 
-static void host_x86_cpu_initfn(Object *obj)
+static void max_x86_cpu_initfn(Object *obj)
 {
     X86CPU *cpu = X86_CPU(obj);
     CPUX86State *env = &cpu->env;
@@ -1585,7 +1583,6 @@ static void host_x86_cpu_initfn(Object *obj)
      */
     cpu->max_features = true;
 
-    /* If KVM is disabled, x86_cpu_realizefn() will report an error later */
     if (kvm_enabled()) {
         env->cpuid_min_level =
             kvm_arch_get_supported_cpuid(s, 0x0, 0, R_EAX);
@@ -1602,10 +1599,30 @@ static void host_x86_cpu_initfn(Object *obj)
     object_property_set_bool(OBJECT(cpu), true, "pmu", &error_abort);
 }
 
+static const TypeInfo max_x86_cpu_type_info = {
+    .name = X86_CPU_TYPE_NAME("max"),
+    .parent = TYPE_X86_CPU,
+    .instance_init = max_x86_cpu_initfn,
+    .class_init = max_x86_cpu_class_init,
+};
+
+#ifdef CONFIG_KVM
+
+static void host_x86_cpu_class_init(ObjectClass *oc, void *data)
+{
+    X86CPUClass *xcc = X86_CPU_CLASS(oc);
+
+    xcc->kvm_required = true;
+    xcc->ordering = 8;
+
+    xcc->model_description =
+        "KVM processor with all supported host features "
+        "(only available in KVM mode)";
+}
+
 static const TypeInfo host_x86_cpu_type_info = {
     .name = X86_CPU_TYPE_NAME("host"),
-    .parent = TYPE_X86_CPU,
-    .instance_init = host_x86_cpu_initfn,
+    .parent = X86_CPU_TYPE_NAME("max"),
     .class_init = host_x86_cpu_class_init,
 };
 
@@ -3820,6 +3837,7 @@ static void x86_cpu_register_types(void)
     for (i = 0; i < ARRAY_SIZE(builtin_x86_defs); i++) {
         x86_register_cpudef_type(&builtin_x86_defs[i]);
     }
+    type_register_static(&max_x86_cpu_type_info);
 #ifdef CONFIG_KVM
     type_register_static(&host_x86_cpu_type_info);
 #endif
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 07/11] i386: Make "max" model not use any host CPUID info on TCG

[Eduardo Habkost]

Instead of reporting host CPUID data on "max", use the qemu64 CPU
model as reference to initialize CPUID
vendor/family/model/stepping/model-id.

Message-Id: <20170222183919.11928-3-ehabkost@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index a53fafa205..ec22ed41b2 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1594,6 +1594,15 @@ static void max_x86_cpu_initfn(Object *obj)
         if (lmce_supported()) {
             object_property_set_bool(OBJECT(cpu), true, "lmce", &error_abort);
         }
+    } else {
+        object_property_set_str(OBJECT(cpu), CPUID_VENDOR_AMD,
+                                "vendor", &error_abort);
+        object_property_set_int(OBJECT(cpu), 6, "family", &error_abort);
+        object_property_set_int(OBJECT(cpu), 6, "model", &error_abort);
+        object_property_set_int(OBJECT(cpu), 3, "stepping", &error_abort);
+        object_property_set_str(OBJECT(cpu),
+                                "QEMU TCG CPU version " QEMU_HW_VERSION,
+                                "model-id", &error_abort);
     }
 
     object_property_set_bool(OBJECT(cpu), true, "pmu", &error_abort);
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 08/11] i386: Don't set CPUClass::cpu_def on "max" model

[Eduardo Habkost]

Host CPUID info is used by the "max" CPU model only in KVM mode.
Move the initialization of CPUID data for "max" from class_init
to instance_init, and don't set CPUClass::cpu_def for "max".

Message-Id: <20170222183919.11928-4-ehabkost@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu-qom.h |  4 +++-
 target/i386/cpu.c     | 45 +++++++++++++++++++++------------------------
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/target/i386/cpu-qom.h b/target/i386/cpu-qom.h
index 14abd0a8c1..f6c704c3a9 100644
--- a/target/i386/cpu-qom.h
+++ b/target/i386/cpu-qom.h
@@ -60,7 +60,9 @@ typedef struct X86CPUClass {
     CPUClass parent_class;
     /*< public >*/
 
-    /* Should be eventually replaced by subclass-specific property defaults. */
+    /* CPU definition, automatically loaded by instance_init if not NULL.
+     * Should be eventually replaced by subclass-specific property defaults.
+     */
     X86CPUDefinition *cpu_def;
 
     bool kvm_required;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index ec22ed41b2..366253cd4c 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1531,47 +1531,27 @@ static int cpu_x86_fill_model_id(char *str)
     return 0;
 }
 
-static X86CPUDefinition host_cpudef;
-
 static Property max_x86_cpu_properties[] = {
     DEFINE_PROP_BOOL("migratable", X86CPU, migratable, true),
     DEFINE_PROP_BOOL("host-cache-info", X86CPU, cache_info_passthrough, false),
     DEFINE_PROP_END_OF_LIST()
 };
 
-/* class_init for the "max" CPU model
- *
- * This function may be called before KVM is initialized.
- */
 static void max_x86_cpu_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
-    uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
 
     xcc->ordering = 9;
 
-    host_cpuid(0x0, 0, &eax, &ebx, &ecx, &edx);
-    x86_cpu_vendor_words2str(host_cpudef.vendor, ebx, edx, ecx);
-
-    host_cpuid(0x1, 0, &eax, &ebx, &ecx, &edx);
-    host_cpudef.family = ((eax >> 8) & 0x0F) + ((eax >> 20) & 0xFF);
-    host_cpudef.model = ((eax >> 4) & 0x0F) | ((eax & 0xF0000) >> 12);
-    host_cpudef.stepping = eax & 0x0F;
-
-    cpu_x86_fill_model_id(host_cpudef.model_id);
-
-    xcc->cpu_def = &host_cpudef;
     xcc->model_description =
         "Enables all features supported by the accelerator in the current host";
 
-    /* level, xlevel, xlevel2, and the feature words are initialized on
-     * instance_init, because they require KVM to be initialized.
-     */
-
     dc->props = max_x86_cpu_properties;
 }
 
+static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp);
+
 static void max_x86_cpu_initfn(Object *obj)
 {
     X86CPU *cpu = X86_CPU(obj);
@@ -1584,6 +1564,21 @@ static void max_x86_cpu_initfn(Object *obj)
     cpu->max_features = true;
 
     if (kvm_enabled()) {
+        X86CPUDefinition host_cpudef = { };
+        uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
+
+        host_cpuid(0x0, 0, &eax, &ebx, &ecx, &edx);
+        x86_cpu_vendor_words2str(host_cpudef.vendor, ebx, edx, ecx);
+
+        host_cpuid(0x1, 0, &eax, &ebx, &ecx, &edx);
+        host_cpudef.family = ((eax >> 8) & 0x0F) + ((eax >> 20) & 0xFF);
+        host_cpudef.model = ((eax >> 4) & 0x0F) | ((eax & 0xF0000) >> 12);
+        host_cpudef.stepping = eax & 0x0F;
+
+        cpu_x86_fill_model_id(host_cpudef.model_id);
+
+        x86_cpu_load_def(cpu, &host_cpudef, &error_abort);
+
         env->cpuid_min_level =
             kvm_arch_get_supported_cpuid(s, 0x0, 0, R_EAX);
         env->cpuid_min_xlevel =
@@ -2185,7 +2180,7 @@ static void x86_cpu_list_entry(gpointer data, gpointer user_data)
     CPUListState *s = user_data;
     char *name = x86_cpu_class_get_model_name(cc);
     const char *desc = cc->model_description;
-    if (!desc) {
+    if (!desc && cc->cpu_def) {
         desc = cc->cpu_def->model_id;
     }
 
@@ -3683,7 +3678,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add_alias(obj, "sse4_1", obj, "sse4.1", &error_abort);
     object_property_add_alias(obj, "sse4_2", obj, "sse4.2", &error_abort);
 
-    x86_cpu_load_def(cpu, xcc->cpu_def, &error_abort);
+    if (xcc->cpu_def) {
+        x86_cpu_load_def(cpu, xcc->cpu_def, &error_abort);
+    }
 }
 
 static int64_t x86_cpu_get_arch_id(CPUState *cs)
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 09/11] i386: Define static "base" CPU model

[Eduardo Habkost]

The query-cpu-model-expand QMP command needs at least one static
model, to allow the "static" expansion mode to be implemented.
Instead of defining static versions of every CPU model, define a
"base" CPU model that has absolutely no feature flag enabled.

Despite having no CPUID data set at all, "-cpu base" is even a
functional CPU:

* It can boot a Slackware Linux 1.01 image with a Linux 0.99.12
  kernel[1].
* It is even possible to boot[2] a modern Fedora x86_64 guest by
  manually enabling the following CPU features:
  -cpu base,+lm,+msr,+pae,+fpu,+cx8,+cmov,+sse,+sse2,+fxsr

[1] http://www.qemu-advent-calendar.org/2014/#day-1
[2] This is what can be seen in the guest:
    [root@localhost ~]# cat /proc/cpuinfo
    processor       : 0
    vendor_id       : unknown
    cpu family      : 0
    model           : 0
    model name      : 00/00
    stepping        : 0
    physical id     : 0
    siblings        : 1
    core id         : 0
    cpu cores       : 1
    apicid          : 0
    initial apicid  : 0
    fpu             : yes
    fpu_exception   : yes
    cpuid level     : 1
    wp              : yes
    flags           : fpu msr pae cx8 cmov fxsr sse sse2 lm nopl
    bugs            :
    bogomips        : 5832.70
    clflush size    : 64
    cache_alignment : 64
    address sizes   : 36 bits physical, 48 bits virtual
    power management:

    [root@localhost ~]# x86info -v -a
    x86info v1.30.  Dave Jones 2001-2011
    Feedback to <davej@redhat.com>.

    No TSC, MHz calculation cannot be performed.
    Unknown vendor (0)
    MP Table:

    Family: 0 Model: 0 Stepping: 0
    CPU Model (x86info's best guess):

    eax in: 0x00000000, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 00000000
    eax in: 0x00000001, eax = 00000000 ebx = 00000800 ecx = 00000000 edx = 07008161

    eax in: 0x80000000, eax = 80000001 ebx = 00000000 ecx = 00000000 edx = 00000000
    eax in: 0x80000001, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 20000000

    Feature flags:
     fpu            Onboard FPU
     msr            Model-Specific Registers
     pae            Physical Address Extensions
     cx8            CMPXCHG8 instruction
     cmov           CMOV instruction
     fxsr           FXSAVE and FXRSTOR instructions
     sse            SSE support
     sse2           SSE2 support

    Long NOPs supported: yes

    Address sizes : 0 bits physical, 0 bits virtual
    0MHz processor (estimate).

     running at an estimated 0MHz
    [root@localhost ~]#

Message-Id: <20170222190029.17243-2-ehabkost@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu-qom.h |  2 ++
 target/i386/cpu.c     | 20 ++++++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/target/i386/cpu-qom.h b/target/i386/cpu-qom.h
index f6c704c3a9..c2205e6077 100644
--- a/target/i386/cpu-qom.h
+++ b/target/i386/cpu-qom.h
@@ -50,6 +50,7 @@ typedef struct X86CPUDefinition X86CPUDefinition;
  * @kvm_required: Whether CPU model requires KVM to be enabled.
  * @ordering: Ordering on the "-cpu help" CPU model list.
  * @migration_safe: See CpuDefinitionInfo::migration_safe
+ * @static_model: See CpuDefinitionInfo::static
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
  *
@@ -68,6 +69,7 @@ typedef struct X86CPUClass {
     bool kvm_required;
     int ordering;
     bool migration_safe;
+    bool static_model;
 
     /* Optional description of CPU model.
      * If unavailable, cpu_def->model_id is used */
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 366253cd4c..0a71594445 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -2229,6 +2229,7 @@ static void x86_cpu_definition_entry(gpointer data, gpointer user_data)
     info->q_typename = g_strdup(object_class_get_name(oc));
     info->migration_safe = cc->migration_safe;
     info->has_migration_safe = true;
+    info->q_static = cc->static_model;
 
     entry = g_malloc0(sizeof(*entry));
     entry->value = info;
@@ -3835,6 +3836,24 @@ static const TypeInfo x86_cpu_type_info = {
     .class_init = x86_cpu_common_class_init,
 };
 
+
+/* "base" CPU model, used by query-cpu-model-expansion */
+static void x86_cpu_base_class_init(ObjectClass *oc, void *data)
+{
+    X86CPUClass *xcc = X86_CPU_CLASS(oc);
+
+    xcc->static_model = true;
+    xcc->migration_safe = true;
+    xcc->model_description = "base CPU model type with no features enabled";
+    xcc->ordering = 8;
+}
+
+static const TypeInfo x86_base_cpu_type_info = {
+        .name = X86_CPU_TYPE_NAME("base"),
+        .parent = TYPE_X86_CPU,
+        .class_init = x86_cpu_base_class_init,
+};
+
 static void x86_cpu_register_types(void)
 {
     int i;
@@ -3844,6 +3863,7 @@ static void x86_cpu_register_types(void)
         x86_register_cpudef_type(&builtin_x86_defs[i]);
     }
     type_register_static(&max_x86_cpu_type_info);
+    type_register_static(&x86_base_cpu_type_info);
 #ifdef CONFIG_KVM
     type_register_static(&host_x86_cpu_type_info);
 #endif
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 10/11] i386: Implement query-cpu-model-expansion QMP command

[Eduardo Habkost]

Implement query-cpu-model-expansion for target-i386.

This should meet all the requirements while being simple. In the
case of static expansion, it will use the new "base" CPU model,
and in the case of full expansion, it will keep the original CPU
model name+props, and append extra properties.

A future follow-up should improve the implementation of
type=full, so that it returns more detailed data, including every
writable QOM property in the CPU object.

Cc: libvir-list@redhat.com
Cc: Jiri Denemark <jdenemar@redhat.com>
Message-Id: <20170222190029.17243-3-ehabkost@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 monitor.c         |   4 +-
 target/i386/cpu.c | 191 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 193 insertions(+), 2 deletions(-)

diff --git a/monitor.c b/monitor.c
index f8f4a07cfb..b68944d93c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -984,8 +984,10 @@ static void qmp_unregister_commands_hack(void)
 #ifndef TARGET_ARM
     qmp_unregister_command("query-gic-capabilities");
 #endif
-#if !defined(TARGET_S390X)
+#if !defined(TARGET_S390X) && !defined(TARGET_I386)
     qmp_unregister_command("query-cpu-model-expansion");
+#endif
+#if !defined(TARGET_S390X)
     qmp_unregister_command("query-cpu-model-baseline");
     qmp_unregister_command("query-cpu-model-comparison");
 #endif
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 0a71594445..139b7ea12e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -29,10 +29,16 @@
 #include "qemu/option.h"
 #include "qemu/config-file.h"
 #include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qstring.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qbool.h"
+#include "qapi/qmp/qint.h"
+#include "qapi/qmp/qfloat.h"
 
 #include "qapi-types.h"
 #include "qapi-visit.h"
 #include "qapi/visitor.h"
+#include "qom/qom-qobject.h"
 #include "sysemu/arch_init.h"
 
 #if defined(CONFIG_KVM)
@@ -2288,7 +2294,7 @@ static void x86_cpu_apply_props(X86CPU *cpu, PropValue *props)
     }
 }
 
-/* Load data from X86CPUDefinition
+/* Load data from X86CPUDefinition into a X86CPU object
  */
 static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp)
 {
@@ -2297,6 +2303,11 @@ static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp)
     char host_vendor[CPUID_VENDOR_SZ + 1];
     FeatureWord w;
 
+    /*NOTE: any property set by this function should be returned by
+     * x86_cpu_static_props(), so static expansion of
+     * query-cpu-model-expansion is always complete.
+     */
+
     /* CPU models only set _minimum_ values for level/xlevel: */
     object_property_set_int(OBJECT(cpu), def->level, "min-level", errp);
     object_property_set_int(OBJECT(cpu), def->xlevel, "min-xlevel", errp);
@@ -2341,6 +2352,184 @@ static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp)
 
 }
 
+/* Return a QDict containing keys for all properties that can be included
+ * in static expansion of CPU models. All properties set by x86_cpu_load_def()
+ * must be included in the dictionary.
+ */
+static QDict *x86_cpu_static_props(void)
+{
+    FeatureWord w;
+    int i;
+    static const char *props[] = {
+        "min-level",
+        "min-xlevel",
+        "family",
+        "model",
+        "stepping",
+        "model-id",
+        "vendor",
+        "lmce",
+        NULL,
+    };
+    static QDict *d;
+
+    if (d) {
+        return d;
+    }
+
+    d = qdict_new();
+    for (i = 0; props[i]; i++) {
+        qdict_put_obj(d, props[i], qnull());
+    }
+
+    for (w = 0; w < FEATURE_WORDS; w++) {
+        FeatureWordInfo *fi = &feature_word_info[w];
+        int bit;
+        for (bit = 0; bit < 32; bit++) {
+            if (!fi->feat_names[bit]) {
+                continue;
+            }
+            qdict_put_obj(d, fi->feat_names[bit], qnull());
+        }
+    }
+
+    return d;
+}
+
+/* Add an entry to @props dict, with the value for property. */
+static void x86_cpu_expand_prop(X86CPU *cpu, QDict *props, const char *prop)
+{
+    QObject *value = object_property_get_qobject(OBJECT(cpu), prop,
+                                                 &error_abort);
+
+    qdict_put_obj(props, prop, value);
+}
+
+/* Convert CPU model data from X86CPU object to a property dictionary
+ * that can recreate exactly the same CPU model.
+ */
+static void x86_cpu_to_dict(X86CPU *cpu, QDict *props)
+{
+    QDict *sprops = x86_cpu_static_props();
+    const QDictEntry *e;
+
+    for (e = qdict_first(sprops); e; e = qdict_next(sprops, e)) {
+        const char *prop = qdict_entry_key(e);
+        x86_cpu_expand_prop(cpu, props, prop);
+    }
+}
+
+static void object_apply_props(Object *obj, QDict *props, Error **errp)
+{
+    const QDictEntry *prop;
+    Error *err = NULL;
+
+    for (prop = qdict_first(props); prop; prop = qdict_next(props, prop)) {
+        object_property_set_qobject(obj, qdict_entry_value(prop),
+                                         qdict_entry_key(prop), &err);
+        if (err) {
+            break;
+        }
+    }
+
+    error_propagate(errp, err);
+}
+
+/* Create X86CPU object according to model+props specification */
+static X86CPU *x86_cpu_from_model(const char *model, QDict *props, Error **errp)
+{
+    X86CPU *xc = NULL;
+    X86CPUClass *xcc;
+    Error *err = NULL;
+
+    xcc = X86_CPU_CLASS(cpu_class_by_name(TYPE_X86_CPU, model));
+    if (xcc == NULL) {
+        error_setg(&err, "CPU model '%s' not found", model);
+        goto out;
+    }
+
+    xc = X86_CPU(object_new(object_class_get_name(OBJECT_CLASS(xcc))));
+    if (props) {
+        object_apply_props(OBJECT(xc), props, &err);
+        if (err) {
+            goto out;
+        }
+    }
+
+    x86_cpu_expand_features(xc, &err);
+    if (err) {
+        goto out;
+    }
+
+out:
+    if (err) {
+        error_propagate(errp, err);
+        object_unref(OBJECT(xc));
+        xc = NULL;
+    }
+    return xc;
+}
+
+CpuModelExpansionInfo *
+arch_query_cpu_model_expansion(CpuModelExpansionType type,
+                                                      CpuModelInfo *model,
+                                                      Error **errp)
+{
+    X86CPU *xc = NULL;
+    Error *err = NULL;
+    CpuModelExpansionInfo *ret = g_new0(CpuModelExpansionInfo, 1);
+    QDict *props = NULL;
+    const char *base_name;
+
+    xc = x86_cpu_from_model(model->name,
+                            model->has_props ?
+                                qobject_to_qdict(model->props) :
+                                NULL, &err);
+    if (err) {
+        goto out;
+    }
+
+
+    switch (type) {
+    case CPU_MODEL_EXPANSION_TYPE_STATIC:
+        /* Static expansion will be based on "base" only */
+        base_name = "base";
+    break;
+    case CPU_MODEL_EXPANSION_TYPE_FULL:
+        /* As we don't return every single property, full expansion needs
+         * to keep the original model name+props, and add extra
+         * properties on top of that.
+         */
+        base_name = model->name;
+        if (model->has_props && model->props) {
+            props = qdict_clone_shallow(qobject_to_qdict(model->props));
+        }
+    break;
+    default:
+        error_setg(&err, "Unsupportted expansion type");
+        goto out;
+    }
+
+    if (!props) {
+        props = qdict_new();
+    }
+    x86_cpu_to_dict(xc, props);
+
+    ret->model = g_new0(CpuModelInfo, 1);
+    ret->model->name = g_strdup(base_name);
+    ret->model->props = QOBJECT(props);
+    ret->model->has_props = true;
+
+out:
+    object_unref(OBJECT(xc));
+    if (err) {
+        error_propagate(errp, err);
+        qapi_free_CpuModelExpansionInfo(ret);
+        ret = NULL;
+    }
+    return ret;
+}
+
 X86CPU *cpu_x86_init(const char *cpu_model)
 {
     return X86_CPU(cpu_generic_init(TYPE_X86_CPU, cpu_model));
-- 
2.11.0.259.g40922b1

[Qemu-devel] [PULL 11/11] i386: Improve query-cpu-model-expansion full mode

[Eduardo Habkost]

This keeps the same results on type=static expansion, but make
type=full expansion return every single QOM property on the CPU
object that have a different value from the "base' CPU model,
plus all the CPU feature flag properties.

Cc: Jiri Denemark <jdenemar@redhat.com>
Message-Id: <20170222190029.17243-4-ehabkost@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target/i386/cpu.c | 34 +++++++++++++++++++++++++++++++---
 1 file changed, 31 insertions(+), 3 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 139b7ea12e..89421c893b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -2419,6 +2419,34 @@ static void x86_cpu_to_dict(X86CPU *cpu, QDict *props)
     }
 }
 
+/* Convert CPU model data from X86CPU object to a property dictionary
+ * that can recreate exactly the same CPU model, including every
+ * writeable QOM property.
+ */
+static void x86_cpu_to_dict_full(X86CPU *cpu, QDict *props)
+{
+    ObjectPropertyIterator iter;
+    ObjectProperty *prop;
+
+    object_property_iter_init(&iter, OBJECT(cpu));
+    while ((prop = object_property_iter_next(&iter))) {
+        /* skip read-only or write-only properties */
+        if (!prop->get || !prop->set) {
+            continue;
+        }
+
+        /* "hotplugged" is the only property that is configurable
+         * on the command-line but will be set differently on CPUs
+         * created using "-cpu ... -smp ..." and by CPUs created
+         * on the fly by x86_cpu_from_model() for querying. Skip it.
+         */
+        if (!strcmp(prop->name, "hotplugged")) {
+            continue;
+        }
+        x86_cpu_expand_prop(cpu, props, prop->name);
+    }
+}
+
 static void object_apply_props(Object *obj, QDict *props, Error **errp)
 {
     const QDictEntry *prop;
@@ -2489,11 +2517,13 @@ arch_query_cpu_model_expansion(CpuModelExpansionType type,
         goto out;
     }
 
+    props = qdict_new();
 
     switch (type) {
     case CPU_MODEL_EXPANSION_TYPE_STATIC:
         /* Static expansion will be based on "base" only */
         base_name = "base";
+        x86_cpu_to_dict(xc, props);
     break;
     case CPU_MODEL_EXPANSION_TYPE_FULL:
         /* As we don't return every single property, full expansion needs
@@ -2501,9 +2531,7 @@ arch_query_cpu_model_expansion(CpuModelExpansionType type,
          * properties on top of that.
          */
         base_name = model->name;
-        if (model->has_props && model->props) {
-            props = qdict_clone_shallow(qobject_to_qdict(model->props));
-        }
+        x86_cpu_to_dict_full(xc, props);
     break;
     default:
         error_setg(&err, "Unsupportted expansion type");
-- 
2.11.0.259.g40922b1

Re: [Qemu-devel] [PULL 00/11] x86 queue, 2017-02-27

[Peter Maydell]

On 27 February 2017 at 16:24, Eduardo Habkost <ehabkost@redhat.com> wrote:
> The following changes since commit 3b1d8169844fafee184366b0e0d7080534758b4d:
>
>   tests-aio-multithread: use atomic_read properly (2017-02-27 12:54:08 +0000)
>
> are available in the git repository at:
>
>   git://github.com/ehabkost/qemu.git tags/x86-pull-request
>
> for you to fetch changes up to b8097deb359bbbd92592b9670adfe9e245b2d0bd:
>
>   i386: Improve query-cpu-model-expansion full mode (2017-02-27 13:23:35 -0300)
>
> ----------------------------------------------------------------
> x86 queue, 2017-02-27
>
> "-cpu max" and query-cpu-model-expansion support for x86. This
> should be the last x86 pull request before 2.9 soft freeze.
>
> ----------------------------------------------------------------
>
> Eduardo Habkost (11):
>   i386: Unset cannot_destroy_with_object_finalize_yet on "host" model
>   i386: Add ordering field to CPUClass
>   i386: Rename X86CPU::host_features to X86CPU::max_features
>   i386: Reorganize and document CPUID initialization steps
>   qapi-schema: Comment about full expansion of non-migration-safe models
>   i386: Create "max" CPU model
>   i386: Make "max" model not use any host CPUID info on TCG
>   i386: Don't set CPUClass::cpu_def on "max" model
>   i386: Define static "base" CPU model
>   i386: Implement query-cpu-model-expansion QMP command
>   i386: Improve query-cpu-model-expansion full mode

This seemed to hang in 'make check' on x86. I didn't have time to
investigate, so I'll have another go with it later.

thanks
-- PMM

[Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Eduardo Habkost]

On Mon, Feb 27, 2017 at 07:16:03PM +0000, Peter Maydell wrote:
[...]
> This seemed to hang in 'make check' on x86. I didn't have time to
> investigate, so I'll have another go with it later.

I have been seeing lots of travis-ci failures recently because of
'make check' taking too long. The jobs seem to timeout while
running tests/test-aio-multithread.

I see similar failures on recent qemu.git master jobs, too:
https://travis-ci.org/qemu/qemu/builds

Are they similar to the hang you have seen?

-- 
Eduardo

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

On Mon, 27 Feb 2017 16:41:34 -0300
Eduardo Habkost <ehabkost@redhat.com> wrote:

> On Mon, Feb 27, 2017 at 07:16:03PM +0000, Peter Maydell wrote:
> [...]
> > This seemed to hang in 'make check' on x86. I didn't have time to
> > investigate, so I'll have another go with it later.  
> 
> I have been seeing lots of travis-ci failures recently because of
> 'make check' taking too long. The jobs seem to timeout while
> running tests/test-aio-multithread.
> 
> I see similar failures on recent qemu.git master jobs, too:
> https://travis-ci.org/qemu/qemu/builds
> 
> Are they similar to the hang you have seen?
> 

The problem with tests/test-aio-multithread got fixed today with the
following commit:

http://git.qemu-project.org/?p=qemu.git;a=commit;h=3b1d8169844fafee184366b0e0d7080534758b4d

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Eduardo Habkost]

On Mon, Feb 27, 2017 at 08:57:53PM +0100, Greg Kurz wrote:
> On Mon, 27 Feb 2017 16:41:34 -0300
> Eduardo Habkost <ehabkost@redhat.com> wrote:
> 
> > On Mon, Feb 27, 2017 at 07:16:03PM +0000, Peter Maydell wrote:
> > [...]
> > > This seemed to hang in 'make check' on x86. I didn't have time to
> > > investigate, so I'll have another go with it later.  
> > 
> > I have been seeing lots of travis-ci failures recently because of
> > 'make check' taking too long. The jobs seem to timeout while
> > running tests/test-aio-multithread.
> > 
> > I see similar failures on recent qemu.git master jobs, too:
> > https://travis-ci.org/qemu/qemu/builds
> > 
> > Are they similar to the hang you have seen?
> > 
> 
> The problem with tests/test-aio-multithread got fixed today with the
> following commit:
> 
> http://git.qemu-project.org/?p=qemu.git;a=commit;h=3b1d8169844fafee184366b0e0d7080534758b4d

I see. The failures really stopped after I rebased to that
commit.

I saw a failure on x86-pull-request that seemed to be because of
vhost-user-test[1]. However, after restarting the job, it
passed[2]. (I can't confirm the actual cause of the first
failure, because the log files seem to be temporarily unavailable
on Travis.)

Now, the problem with vhost-user-test is that it is not
guaranteed to work without KVM[3], so we need to add a mechanism
to skip vhost-user-test (or at least make errors non-fatal) if
KVM is unavailable.

[1] https://travis-ci.org/ehabkost/qemu/builds/205849528
[2] https://travis-ci.org/ehabkost/qemu/builds/205850173
[3] See commit cdafe929615ec5eca71bcd5a3d12bab5678e5886

-- 
Eduardo

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Peter Maydell]

On 28 February 2017 at 19:12, Eduardo Habkost <ehabkost@redhat.com> wrote:
> I saw a failure on x86-pull-request that seemed to be because of
> vhost-user-test[1]. However, after restarting the job, it
> passed[2].

I'm currently processing a patch which (hopefully) fixes
vhost-user-test's intermittent failures:
http://patchwork.ozlabs.org/patch/732747/

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/11] x86 queue, 2017-02-27

[Peter Maydell]

On 27 February 2017 at 19:16, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 27 February 2017 at 16:24, Eduardo Habkost <ehabkost@redhat.com> wrote:
>> The following changes since commit 3b1d8169844fafee184366b0e0d7080534758b4d:
>>
>>   tests-aio-multithread: use atomic_read properly (2017-02-27 12:54:08 +0000)
>>
>> are available in the git repository at:
>>
>>   git://github.com/ehabkost/qemu.git tags/x86-pull-request
>>
>> for you to fetch changes up to b8097deb359bbbd92592b9670adfe9e245b2d0bd:
>>
>>   i386: Improve query-cpu-model-expansion full mode (2017-02-27 13:23:35 -0300)
>>
>> ----------------------------------------------------------------
>> x86 queue, 2017-02-27
>>
>> "-cpu max" and query-cpu-model-expansion support for x86. This
>> should be the last x86 pull request before 2.9 soft freeze.
>>
>> ----------------------------------------------------------------
>>
>> Eduardo Habkost (11):
>>   i386: Unset cannot_destroy_with_object_finalize_yet on "host" model
>>   i386: Add ordering field to CPUClass
>>   i386: Rename X86CPU::host_features to X86CPU::max_features
>>   i386: Reorganize and document CPUID initialization steps
>>   qapi-schema: Comment about full expansion of non-migration-safe models
>>   i386: Create "max" CPU model
>>   i386: Make "max" model not use any host CPUID info on TCG
>>   i386: Don't set CPUClass::cpu_def on "max" model
>>   i386: Define static "base" CPU model
>>   i386: Implement query-cpu-model-expansion QMP command
>>   i386: Improve query-cpu-model-expansion full mode
>
> This seemed to hang in 'make check' on x86. I didn't have time to
> investigate, so I'll have another go with it later.

Retrying worked ok, so I've applied it, on the assumption the
hang was not something in this patchset.

thanks
-- PMM

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Eduardo Habkost]

On Tue, Feb 28, 2017 at 07:17:39PM +0000, Peter Maydell wrote:
> On 28 February 2017 at 19:12, Eduardo Habkost <ehabkost@redhat.com> wrote:
> > I saw a failure on x86-pull-request that seemed to be because of
> > vhost-user-test[1]. However, after restarting the job, it
> > passed[2].
> 
> I'm currently processing a patch which (hopefully) fixes
> vhost-user-test's intermittent failures:
> http://patchwork.ozlabs.org/patch/732747/

I'm not sure it will solve the issues on hosts without KVM. As
far as I can see, if vhost-user-test is working without KVM, it
is working by accident.

See the thread at:
http://www.mail-archive.com/qemu-devel@nongnu.org/msg394258.html

-- 
Eduardo

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Paolo Bonzini]

On 02/03/2017 16:39, Eduardo Habkost wrote:
> On Tue, Feb 28, 2017 at 07:17:39PM +0000, Peter Maydell wrote:
>> On 28 February 2017 at 19:12, Eduardo Habkost <ehabkost@redhat.com> wrote:
>>> I saw a failure on x86-pull-request that seemed to be because of
>>> vhost-user-test[1]. However, after restarting the job, it
>>> passed[2].
>>
>> I'm currently processing a patch which (hopefully) fixes
>> vhost-user-test's intermittent failures:
>> http://patchwork.ozlabs.org/patch/732747/
> 
> I'm not sure it will solve the issues on hosts without KVM. As
> far as I can see, if vhost-user-test is working without KVM, it
> is working by accident.

Well, it has worked for a while before the patch.  As long as you don't
overwrite code with vhost-user data and then try to run that data,
things will be fine.  Just not something you can use in practice, but it
works in tests.

Paolo

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Eduardo Habkost]

On Thu, Mar 02, 2017 at 04:54:26PM +0100, Paolo Bonzini wrote:
> On 02/03/2017 16:39, Eduardo Habkost wrote:
> > On Tue, Feb 28, 2017 at 07:17:39PM +0000, Peter Maydell wrote:
> >> On 28 February 2017 at 19:12, Eduardo Habkost <ehabkost@redhat.com> wrote:
> >>> I saw a failure on x86-pull-request that seemed to be because of
> >>> vhost-user-test[1]. However, after restarting the job, it
> >>> passed[2].
> >>
> >> I'm currently processing a patch which (hopefully) fixes
> >> vhost-user-test's intermittent failures:
> >> http://patchwork.ozlabs.org/patch/732747/
> > 
> > I'm not sure it will solve the issues on hosts without KVM. As
> > far as I can see, if vhost-user-test is working without KVM, it
> > is working by accident.
> 
> Well, it has worked for a while before the patch.

Before which patch?

>                                                    As long as you don't
> overwrite code with vhost-user data and then try to run that data,
> things will be fine.  Just not something you can use in practice, but it
> works in tests.

Earlier this week I saw the wait_for_fds assertion (mentioned at
the thread above) on a travis-ci job again, and I was suspecting
it was the same vhost_set_mem_table() + TCG error seen at the
thread above.

Unfortunately travis-ci overwrote the previous logs when I
restarted the job, and now I can't confirm if it was really the
same vhost_set_mem_table() error. I guess we'll have to simply
wait and see if it fails again.

-- 
Eduardo

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Peter Maydell]

On 2 March 2017 at 16:07, Eduardo Habkost <ehabkost@redhat.com> wrote:
> Earlier this week I saw the wait_for_fds assertion (mentioned at
> the thread above) on a travis-ci job again, and I was suspecting
> it was the same vhost_set_mem_table() + TCG error seen at the
> thread above.
>
> Unfortunately travis-ci overwrote the previous logs when I
> restarted the job, and now I can't confirm if it was really the
> same vhost_set_mem_table() error. I guess we'll have to simply
> wait and see if it fails again.

Here's a fresh new travis job failing like that:
https://travis-ci.org/qemu/qemu/builds/207005044
(specifically https://travis-ci.org/qemu/qemu/jobs/207005050)

thanks
-- PMM

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Paolo Bonzini]

On 02/03/2017 17:07, Eduardo Habkost wrote:
> On Thu, Mar 02, 2017 at 04:54:26PM +0100, Paolo Bonzini wrote:
>> On 02/03/2017 16:39, Eduardo Habkost wrote:
>>> On Tue, Feb 28, 2017 at 07:17:39PM +0000, Peter Maydell wrote:
>>>> On 28 February 2017 at 19:12, Eduardo Habkost <ehabkost@redhat.com> wrote:
>>>>> I saw a failure on x86-pull-request that seemed to be because of
>>>>> vhost-user-test[1]. However, after restarting the job, it
>>>>> passed[2].
>>>>
>>>> I'm currently processing a patch which (hopefully) fixes
>>>> vhost-user-test's intermittent failures:
>>>> http://patchwork.ozlabs.org/patch/732747/
>>>
>>> I'm not sure it will solve the issues on hosts without KVM. As
>>> far as I can see, if vhost-user-test is working without KVM, it
>>> is working by accident.
>>
>> Well, it has worked for a while before the patch.
> 
> Before which patch?

The one mentioned in the commit message by Marc-André:
b0a335e351103bf92f3f9d0bd5759311be8156ac.

Paolo

> 
>>                                                    As long as you don't
>> overwrite code with vhost-user data and then try to run that data,
>> things will be fine.  Just not something you can use in practice, but it
>> works in tests.
> 
> Earlier this week I saw the wait_for_fds assertion (mentioned at
> the thread above) on a travis-ci job again, and I was suspecting
> it was the same vhost_set_mem_table() + TCG error seen at the
> thread above.
> 
> Unfortunately travis-ci overwrote the previous logs when I
> restarted the job, and now I can't confirm if it was really the
> same vhost_set_mem_table() error. I guess we'll have to simply
> wait and see if it fails again.
> 

Re: [Qemu-devel] travis-ci 'make check' timeouts (was Re: [PULL 00/11] x86 queue, 2017-02-27)

[Eduardo Habkost]

On Thu, Mar 02, 2017 at 04:16:53PM +0000, Peter Maydell wrote:
> On 2 March 2017 at 16:07, Eduardo Habkost <ehabkost@redhat.com> wrote:
> > Earlier this week I saw the wait_for_fds assertion (mentioned at
> > the thread above) on a travis-ci job again, and I was suspecting
> > it was the same vhost_set_mem_table() + TCG error seen at the
> > thread above.
> >
> > Unfortunately travis-ci overwrote the previous logs when I
> > restarted the job, and now I can't confirm if it was really the
> > same vhost_set_mem_table() error. I guess we'll have to simply
> > wait and see if it fails again.
> 
> Here's a fresh new travis job failing like that:
> https://travis-ci.org/qemu/qemu/builds/207005044
> (specifically https://travis-ci.org/qemu/qemu/jobs/207005050)

Indeed, it is not exactly the same TCG-related error we've seen
in August. It looks like vhost-user-test + TCG is not as broken
as I thought.

-- 
Eduardo