Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option

[Igor Mammedov <imammedo@redhat.com>]

On Fri, 15 Jan 2021 10:20:23 +0100
Vitaly Kuznetsov <vkuznets@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> writes:
> 
> > On Thu,  7 Jan 2021 16:14:49 +0100
> > Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
> >  
> >> Enabling Hyper-V emulation for a Windows VM is a tiring experience as it
> >> requires listing all currently supported enlightenments ("hv-*" CPU
> >> features) explicitly. We do have 'hv-passthrough' mode enabling
> >> everything but it can't be used in production as it prevents migration.
> >> 
> >> Introduce a simple 'hv-default=on' CPU flag enabling all currently supported
> >> Hyper-V enlightenments. Later, when new enlightenments get implemented,
> >> compat_props mechanism will be used to disable them for legacy machine types,
> >> this will keep 'hv-default=on' configurations migratable.
> >> 
> >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> >> ---
> >>  docs/hyperv.txt   | 16 +++++++++++++---
> >>  target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++
> >>  target/i386/cpu.h |  5 +++++
> >>  3 files changed, 56 insertions(+), 3 deletions(-)
> >> 
> >> diff --git a/docs/hyperv.txt b/docs/hyperv.txt
> >> index 5df00da54fc4..a54c066cab09 100644
> >> --- a/docs/hyperv.txt
> >> +++ b/docs/hyperv.txt
> >> @@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features.
> >>  
> >>  2. Setup
> >>  =========
> >> -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In
> >> -QEMU, individual enlightenments can be enabled through CPU flags, e.g:
> >> +All currently supported Hyper-V enlightenments can be enabled by specifying
> >> +'hv-default=on' CPU flag:
> >>  
> >> -  qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ...
> >> +  qemu-system-x86_64 --enable-kvm --cpu host,hv-default ...
> >> +
> >> +Alternatively, it is possible to do fine-grained enablement through CPU flags,
> >> +e.g:
> >> +
> >> +  qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...  
> >
> > I'd put here not '...' but rather recommended list of flags, and update
> > it every time when new feature added if necessary.
> >  

1)
 
> This is an example of fine-grained enablement, there is no point to put
> all the existing flags there (hv-default is the only recommended way
> now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend
on machine version on top that). Doesn't look like a good documentation to me
(sure everyone can go and read source code for it and try to figure out how
it's supposed to work)

>
> > (not to mention that if we had it to begin with, then new 'hv-default' won't
> > be necessary, I still see it as functionality duplication but I will not oppose it)
> >  
> 
> Unfortunately, upper layer tools don't read this doc and update
> themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level.
why stop there, just merge with yet another upper layer, it would save us a lot
on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.)
There should be limit somewhere, where QEMU job ends and others pile hw abstraction
layers on top of it.

> Similarly, if when these tools use '-machine q35' they get all the new features we add
> automatically, right?
it depends, in case of CPUs, new features usually 'off' by default
for existing models. In case of bugs, features sometimes could be
flipped and versioned machines were used to keep broken CPU models
on old machine types.

   
> >> +It is also possible to disable individual enlightenments from the default list,
> >> +this can be used for debugging purposes:
> >> +
> >> +  qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ...
> >>  
> >>  Sometimes there are dependencies between enlightenments, QEMU is supposed to
> >>  check that the supplied configuration is sane.
> >> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> >> index 48007a876e32..99338de00f78 100644
> >> --- a/target/i386/cpu.c
> >> +++ b/target/i386/cpu.c
> >> @@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name,
> >>      cpu->env.tsc_khz = cpu->env.user_tsc_khz = value / 1000;
> >>  }
> >>  
> >> +static bool x86_hv_default_get(Object *obj, Error **errp)
> >> +{
> >> +    X86CPU *cpu = X86_CPU(obj);
> >> +
> >> +    return cpu->hyperv_default;
> >> +}
> >> +
> >> +static void x86_hv_default_set(Object *obj, bool value, Error **errp)
> >> +{
> >> +    X86CPU *cpu = X86_CPU(obj);
> >> +
> >> +    cpu->hyperv_default = value;
> >> +
> >> +    if (value) {
> >> +        cpu->hyperv_features |= cpu->hyperv_default_features;  
> >
> > s/|="/=/ please,
> > i.e. no option overrides whatever was specified before to keep semantics consistent.
> >  
> 
> Hm,
> 

> this doesn't matter for the most recent machine type as
> hyperv_default_features has all the features but imagine you're running
> an older machine type which doesn't have 'hv_feature'. Now your
normally one shouldn't use new feature with old machine type as it makes
VM non-migratable to older QEMU that has this machine type but not this feature.

nitpicking:
  according to (1) user should not use 'hv_feature' on old machine since
  hv_default should cover all their needs (well they don't know what hv_default actually is).

> suggestion is 
> 
> if I do:
> 
> 'hv_default,hv_feature=on' I will get "hyperv_default_features | hv_feature"
> 
> but if I do
> 
> 'hv_feature=on,hv_default' I will just get 'hyperv_default_features'
> (as hv_default enablement will overwrite everything)
> 
> How is this consistent?
usual semantics for properties, is that the latest property overwrites,
the previous property value parsed from left to right.
(i.e. if one asked for hv_default, one gets it related CPUID bit set/unset,
if one needs more than that one should add more related features after that.


> >> +    }
> >> +}
> >> +
> >>  /* Generic getter for "feature-words" and "filtered-features" properties */
> >>  static void x86_cpu_get_feature_words(Object *obj, Visitor *v,
> >>                                        const char *name, void *opaque,
> >> @@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)
> >>      object_property_add_alias(obj, "pause_filter", obj, "pause-filter");
> >>      object_property_add_alias(obj, "sse4_1", obj, "sse4.1");
> >>      object_property_add_alias(obj, "sse4_2", obj, "sse4.2");
> >> +    object_property_add_alias(obj, "hv_default", obj, "hv-default");
> >>  
> >>      if (xcc->model) {
> >>          x86_cpu_load_model(cpu, xcc->model);
> >>      }
> >> +
> >> +    /* Hyper-V features enabled with 'hv-default=on' */
> >> +    cpu->hyperv_default_features = BIT(HYPERV_FEAT_RELAXED) |
> >> +        BIT(HYPERV_FEAT_VAPIC) | BIT(HYPERV_FEAT_TIME) |
> >> +        BIT(HYPERV_FEAT_CRASH) | BIT(HYPERV_FEAT_RESET) |
> >> +        BIT(HYPERV_FEAT_VPINDEX) | BIT(HYPERV_FEAT_RUNTIME) |
> >> +        BIT(HYPERV_FEAT_SYNIC) | BIT(HYPERV_FEAT_STIMER) |
> >> +        BIT(HYPERV_FEAT_FREQUENCIES) | BIT(HYPERV_FEAT_REENLIGHTENMENT) |
> >> +        BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |
> >> +        BIT(HYPERV_FEAT_STIMER_DIRECT);
> >> +
> >> +    /* Enlightened VMCS is only available on Intel/VMX */
> >> +    if (kvm_hv_evmcs_available()) {
> >> +        cpu->hyperv_default_features |= BIT(HYPERV_FEAT_EVMCS);
> >> +    }  
> > what if VVM is migrated to another host without evmcs,
> > will it change CPUID?
> >  
> 
> Evmcs is tightly coupled with VMX, we can't migrate when it's not
> there.

Are you saying mgmt will check and refuse to migrate to such host?

> 
> >>  }
> >>  
> >>  static int64_t x86_cpu_get_arch_id(CPUState *cs)
> >> @@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
> >>                                x86_cpu_get_crash_info_qom, NULL, NULL, NULL);
> >>  #endif
> >>  
> >> +    object_class_property_add_bool(oc, "hv-default",
> >> +                              x86_hv_default_get,
> >> +                              x86_hv_default_set);
> >> +
> >>      for (w = 0; w < FEATURE_WORDS; w++) {
> >>          int bitnr;
> >>          for (bitnr = 0; bitnr < 64; bitnr++) {
> >> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> >> index 6220cb2cabb9..8a484becb6b9 100644
> >> --- a/target/i386/cpu.h
> >> +++ b/target/i386/cpu.h
> >> @@ -1657,6 +1657,11 @@ struct X86CPU {
> >>      bool hyperv_synic_kvm_only;
> >>      uint64_t hyperv_features;
> >>      bool hyperv_passthrough;
> >> +
> >> +    /* 'hv-default' enablement */
> >> +    uint64_t hyperv_default_features;
> >> +    bool hyperv_default;
> >> +
> >>      OnOffAuto hyperv_no_nonarch_cs;
> >>      uint32_t hyperv_vendor_id[3];
> >>      uint32_t hyperv_interface_id[4];  
> >  
>