From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28861C43334 for ; Thu, 7 Jul 2022 16:18:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236280AbiGGQSG (ORCPT ); Thu, 7 Jul 2022 12:18:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236210AbiGGQSF (ORCPT ); Thu, 7 Jul 2022 12:18:05 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA4AD1570A for ; Thu, 7 Jul 2022 09:18:03 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id bf13so6343592pgb.11 for ; Thu, 07 Jul 2022 09:18:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=g4b8V9nVein02SaaRGxSEkwK4bSbPWw2wYDtDzzfHkA=; b=S1wqm/1JGuzFumHfm8KgkZ/ZmuKwtDo7fOVy8TDe4S/6DZZF3DOLf/U+gCQWD7X5dW mOlMPSMknLjb/VmC2bG7nJf/9Oyzf7DWQ5dfsGmm0+816UKihjMTzDE4R7trWFTzTZeR NWvkfug0M9MD2cvqAjyz2cOMPYl1YxrQSj9H5UTOUFTo3AddFXwj+fk6Cw58HSbuADVc rmx1tsUG28aI4sdXBlAgR9llS9ESRbYQcIbH3zOCC8h+hIP1vOJGTApjsKon1xSF0J0/ 5MAOO5uFIvqcZ61Yybujt6Q0k/6WH1YWdQIbTnVc2y7815JWlP4Uk2siX41L4Xknj8uj ZsdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=g4b8V9nVein02SaaRGxSEkwK4bSbPWw2wYDtDzzfHkA=; b=owS/XXhF+RIRdNv/xttCNHBe8/q4UE4haSaGstKqdB3CNOpw/IsMm+q3vRzmvx6XRd y5RKZc+B6fn3kPqbD+FPWZo/cxUyPloApGu6dLVc6opqJ4KtYJ9lhzNB6mc1yGeqXdKM 6ECi6uBMgEsS3/j802OP8k/0H9lZ5R8liABclbCkv07kSPgCaFi7diyKBJy1ZWu08Ofp 0qB5c/O3kN1z61AaDZ0MJk9HFUCehtugBUmHdBCtKQ/wxpRsCG9JvTN8koxRW2Bc+ZKn cEXXV+6rcGWPCfLMJOUkZjDEd8fqfUeC4hcpEAFwjra0l9diYgFS5kMoJ0j4MrGmVVw9 3hHw== X-Gm-Message-State: AJIora9RFwulUvq1cREJsHwOXOoXwSkk2Xt1zrUJlYSkqeHFVjepKmLx 1z4PveRDFRs0TxSWV5z1iY6E9Q== X-Google-Smtp-Source: AGRyM1uBhqcLgaC/a3j9kzPspaK/sXU9dO4WK60ec34W+P36XvLkfgxxzr991p2xaa3KxJ2iq2PdCg== X-Received: by 2002:a17:90a:db96:b0:1ef:8c86:eb09 with SMTP id h22-20020a17090adb9600b001ef8c86eb09mr6043973pjv.22.1657210683045; Thu, 07 Jul 2022 09:18:03 -0700 (PDT) Received: from google.com (123.65.230.35.bc.googleusercontent.com. [35.230.65.123]) by smtp.gmail.com with ESMTPSA id cq13-20020a056a00330d00b005255489187fsm27060777pfb.135.2022.07.07.09.18.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Jul 2022 09:18:02 -0700 (PDT) Date: Thu, 7 Jul 2022 16:17:58 +0000 From: Sean Christopherson To: Vitaly Kuznetsov Cc: kvm@vger.kernel.org, Paolo Bonzini , Anirudh Rayabharam , Wanpeng Li , Jim Mattson , Maxim Levitsky , linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 06/28] KVM: nVMX: Introduce KVM_CAP_HYPERV_ENLIGHTENED_VMCS2 Message-ID: References: <20220629150625.238286-1-vkuznets@redhat.com> <20220629150625.238286-7-vkuznets@redhat.com> <87o7y1qm5t.fsf@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87o7y1qm5t.fsf@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-hyperv@vger.kernel.org On Thu, Jul 07, 2022, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > > On Wed, Jun 29, 2022, Vitaly Kuznetsov wrote: > >> Turns out Enlightened VMCS can gain new fields without version change > >> and KVM_CAP_HYPERV_ENLIGHTENED_VMCS which KVM currently has cant's > >> handle this reliably. In particular, just updating the current definition > >> of eVMCSv1 with the new fields and adjusting the VMX MSR filtering will > >> inevitably break live migration to older KVMs. Note: enabling eVMCS and > >> setting VMX feature MSR can happen in any order. > >> > >> Introduce a notion of KVM internal "Enlightened VMCS revision" and add > >> a new capability allowing to add fields to Enlightened VMCS while keeping > >> its version. > > > > Bumping a "minor" version number in KVM is going to be a nightmare. KVM is going > > to be stuck "supporting" old revisions in perpetuity, and userspace will be forced > > to keep track of which features are available with which arbitrary revision (is > > that information even communicated to userspace?). > > My brain is certainly tainted with how we enable this in QEMU but why > would userspace be interested in which features are actually filtered > out? For all the same reasons userspace wants to know what hardware features are supported by hardware and KVM. > Currently (again, by QEMU), eVMCS is treated as a purely software > feature. When enabled, certain controls are filtered out "under the > hood" as VMX MSRs reported to VMM remain unfiltered (see > '!msr_info->host_initiated' in vmx_get_msr()). Same stays true with any > new revision: VMM's job is just to check that a) all hardware features > are supported on both source and destination and b) the requested 'eVMCS > revision' is supported by both. No need to know what's filtered out and > what isn't. But users will inevitably want to know exactly what features a platform supports for a given configuration. E.g. if KVM only supported pre-configured CPU models, KVM would need to document what features are supported by each model, and userspace would have very little flexibility in terms of what features are exposed to the guest. That's an exaggerated example as there are far more CPUID features than eVMCS features, but it's the same underlying concept. > > I think a more maintainable approach would be to expose the "filtered" VMX MSRs to > > userspace, e.g. add KVM_GET_EVMCS_VMX_MSRS. Then KVM just needs to document what > > the "filters" are for KVM versions that don't support KVM_GET_EVMCS_VMX_MSRS. > > KVM itself doesn't need to maintain version information because it's userspace's > > responsibility to ensure that userspace doesn't try to migrate to a KVM that doesn't > > support the desired feature set. > > That would be a reasonable (but complex for VMM) approach too but I > don't think we need this (and this patch introducing 'eVMCS revisions' > to this matter): But userspace already has to deal with that complexity in raw VMX MSRs. The filtered values will always be a subset of the unfiltered values, so if eVMCS will be exposed to the guest, userspace can simply treat the filtered set as the baseline supported set, i.e. the userspace logic could simply be: if (expose_evmcs) get_evmcs_vmx_msrs(&msrs); else get_vmx_msrs(&msrs); Userspace will need to take on more complexity if userspace wants to expose features that are supported in hardware but not with eVMCS active, but I don't see the point in doing so because AFAICT KVM will just override and filter the VMX MSRs anyways when eVMCS is enabled, i.e. userspace _can't_ expose the unfiltered VMX MSRs to the guest when eVMCS is enabled. > luckily, Microsoft added a new PV CPUID feature bit inidicating the support > for the new features in eVMCSv1 so KVM can just observe whether the bit was > set by VMM or not and filter accordingly. If there's a CPUID feature bit, why does KVM need to invent its own revision scheme? > > That also avoids messes like unnecessarily blocking migration from "incompatible" > > revisions when running on hardware that doesn't even support the control. > > Well yea, in case the difference between 'eVMCS revisions' is void > because the hardware doesn't support these, it would still be possible > to migrate to an older KVM which doesn't support the new revision but > I'd stay strict: if a newer revision was requested it must be supported, > no matter the hardware.