Re: [Qemu-devel] [PATCH RESEND v2] i386/kvm: add support for KVM_CAP_X86_DISABLE_EXITS

[Eduardo Habkost <ehabkost@redhat.com>]

On Thu, May 17, 2018 at 08:57:04PM +0300, Michael S. Tsirkin wrote:
> On Thu, May 17, 2018 at 02:34:28PM -0300, Eduardo Habkost wrote:
> > On Wed, May 16, 2018 at 05:26:40PM +0300, Michael S. Tsirkin wrote:
> > > On Wed, May 16, 2018 at 09:34:52AM -0300, Eduardo Habkost wrote:
> > > > On Sat, May 12, 2018 at 01:12:59AM +0300, Michael S. Tsirkin wrote:
> > > > > On Thu, Apr 19, 2018 at 06:53:20PM -0300, Eduardo Habkost wrote:
> > > > > > On Thu, Apr 19, 2018 at 11:32:16PM +0200, Paolo Bonzini wrote:
> > > > > > > On 19/04/2018 21:56, Eduardo Habkost wrote:
> > > > > > > > On Thu, Apr 19, 2018 at 05:48:57PM +0200, Paolo Bonzini wrote:
> > > > > > > >> On 17/04/2018 22:59, Eduardo Habkost wrote:
> > > > > > > >>>> +        if (disable_exits) {
> > > > > > > >>>> +            disable_exits &= (KVM_X86_DISABLE_EXITS_MWAIT |
> > > > > > > >>>> +                              KVM_X86_DISABLE_EXITS_HLT |
> > > > > > > >>>> +                              KVM_X86_DISABLE_EXITS_PAUSE);
> > > > > > > >>>> +            if (env->user_features[FEAT_KVM] & KVM_PV_UNHALT) {
> > > > > > > >>>> +                disable_exits &= ~KVM_X86_DISABLE_EXITS_HLT;
> > > > > > > >>>> +            }
> > > > > > > >>>
> > > > > > > >>> In the future, if we decide to enable kvm-pv-unhalt by default,
> > > > > > > >>> should "-cpu ...,kvm-hint-dedicated=on" disable kvm-pv-unhalt
> > > > > > > >>> automatically, or should we require an explicit
> > > > > > > >>> "kvm-hint-dedicated=on,kvm-pv-unhalt=off" option?
> > > > > > > >>
> > > > > > > >> It should be automatic.
> > > > > > > >>
> > > > > > > >>> For today's defaults, this patch solves the problem, only one
> > > > > > > >>> thing is missing before I give my R-b: we need to clearly
> > > > > > > >>> document what exactly are the consequences and requirements of
> > > > > > > >>> setting kvm-hint-dedicated=on (I'm not sure if the best place for
> > > > > > > >>> this is qemu-options.hx, x86_cpu_list(), or somewhere else).
> > > > > > > >>
> > > > > > > >> I don't think we have a good place for this kind of documentation,
> > > > > > > >> unfortunately.  Right now it is mentioned in
> > > > > > > >> Documentation/virtual/kvm/cpuid.txt.
> > > > > > > > 
> > > > > > > > With this patch, the QEMU option will do more than just setting
> > > > > > > > the CPUID bit, that's why I miss more detailed documentation on
> > > > > > > > the QEMU side.  But I agree we have no obvious place for that
> > > > > > > > documentation.
> > > > > > > > 
> > > > > > > > In the worst case we can just add a code comment on top of
> > > > > > > > feature_word_info[FEAT_KVM_HINTS].feat_names warning that
> > > > > > > > kvm-hint-dedicated won't just enable the flag on CPUID and has
> > > > > > > > other side-effects.
> > > > > > > 
> > > > > > > Maybe we should use "-realtime dedicated=on" instead of, or in addition
> > > > > > > to kvm-hint-dedicated=on?
> > > > > > 
> > > > > > Maybe it's a better idea than overloading an option that is only
> > > > > > expected to control a CPUID bit.
> > > > > 
> > > > > Well -realtime would be a bit confusing in that it enables mlock by
> > > > > default.
> > > > > 
> > > > > From pure API point of view, hint-dedicated looks good since
> > > > > it seems to say "optimize for a dedicated host CPU" and
> > > > > it's a hint in that guests keep working if you violate this
> > > > > slightly once in a while.
> > > > > 
> > > > > But I agree there's a problem: right now "kvm-" means "KVM PV"
> > > > > as opposed to e.g. HV enlightened gusts.
> > > > > So if you enable hyperv and also want to disable halt existing,
> > > > > what then? What should kvm-hint-dedicated=on do?
> > > > > 
> > > > > So how about a new hint-dedicated=on cpu flag?  We can have that set
> > > > > kvm-hint-dedicated if kvm PV is enabled.
> > > > 
> > > > Using a boolean flag that is _not_ considered a CPUID feature
> > > > flag would be better.  Using the CPUID feature flag name risks
> > > > having management software enabling the flag by accident (as it
> > > > will get included in query-cpu-model-* queries).  A separate
> > > > boolean flag would make this clearer.
> > > 
> > > Can we remove all hints from query-cpu-model queries?
> > 
> > We already do (see usage of EatureWordInfo::no_autoenable_flags).
> > This is just a matter of making the configuration option
> > decoupled from the CPUID code, to avoid surprises elsewhere.
> 
> It it too late to drop the hint flag and rename to a -realtime option?

We can't remove support for "-cpu ...,kvm-pv-dedicated=on"
without a deprecation notice, as it's already in v2.12.0.

But it's not too late to make other side-effects (e.g. disabling
VM exits) be controlled by -realtime or other command-line
option.  It's also not too late to move kvm-pv-dedicated from
the feat_names array to x86_cpu_properties to avoid confusion.

The existing behavior of "-cpu ...,kvm-hint-dedicated=on" is to
only set the CPUID bit and do nothing else that could have
unwanted side-effects (like disabling VM exits).
Do you see a problem in simply keeping the existing behavior?

-- 
Eduardo