From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59FE4C4321E for ; Thu, 1 Dec 2022 15:44:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Wu5DwVDB2fmgFPCaA7pYCDuSn/JM6yRC6TVSXxaEmsc=; b=dspXDDdvYmX59n D8B4t31ep6CGUZCFq3sRtLeZgCRWnvnGLkp4WfVsqLesuSMR5q34f/yxrKiNVKgFiDaC5yXHXoLDT /+Ue5bOR0P5aAX83ZHy9YuLanosKcyF8PaQsJFNePXjn5bDao9ZguR9SdVSsWKvWVPFr80Lzc50uP QAGJqT8+aSw30DirLGGjLxdwHPTGeg7PdVpIpTCbQPDqop3J841oUh+IwdiBwY6hfdGExXZ/fbqtd 5uyLuQUsLLLMW85vsv0VtRevqmkD0gBDZLG+tzWXzBdLs0WfmhtsFacQIcO7IrIPgMkVW4PDSAg8A oCQF+gVjrdj8KQg3jCrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p0liS-008JG8-KI; Thu, 01 Dec 2022 15:43:12 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p0liO-008JDm-Uy for linux-arm-kernel@lists.infradead.org; Thu, 01 Dec 2022 15:43:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669909387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=uICPaEENN8EB00poBiDynBWP9unBSRZ6Eo0ZeloiILw=; b=eAROI184uodfpk+SZ7s0fh84ZmhUbDoncx223gqLkBBLNzgC5P5x6W5TbJtpfZu/T7HbQW Gn8l3mJ5uZJu6vWg8jNId8OMGXXVlYJt6u6lVbJs9Gq0lYEQtCGooSFLHVE1DocP/tOf9n D7GB1Bf/UCpY3EpGIWwzyLL+6g+F2nc= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-637-tw-SdeAuNW2SnkmKnY6epQ-1; Thu, 01 Dec 2022 10:43:03 -0500 X-MC-Unique: tw-SdeAuNW2SnkmKnY6epQ-1 Received: by mail-wr1-f70.google.com with SMTP id k7-20020adfc707000000b002416f2e9ad5so533819wrg.6 for ; Thu, 01 Dec 2022 07:43:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uICPaEENN8EB00poBiDynBWP9unBSRZ6Eo0ZeloiILw=; b=4hipvCS5SAXJ6Y0RP9Qortn9XCUZiNQdFEQ01xOiM8JspO3ob9H31gODWVoQKkJcde qWitKKxalu4JzEObNQrIH1PNSKZ8QIvonVeHr6OURSurW5oto+zSMTApsvVyXh7dKtT7 cncTZmWNlYKsGJvrYMcDpx7masNDo6LJ6jx8K2Gq1j4bsQA2sUx0/rcYmEUr6ltdlYH1 oo2FtF7VZYM0pH5QUbCHP8dOuqSyXREBsit6FwYS43X5kJYtzKzHmHGRzRYXzaQpXPK2 fRJoAwF8spLAQk0b7AC66tP4ZzxuwzqsFukMw/g3u3GHV6/rS67MGuEQkGX+lFE14gsF YdIw== X-Gm-Message-State: ANoB5plpnCP7deKkEk1UWjop1kPRhRA8c1OmoWNRMy9hNRaKoEiXJXhz tgrapBzNsQf9zVW28l4P8eYyMFtwZPog/nGlghIC5rcANbiuHiqwXSoa3CJUVP1z8pkZw9O+Gtx 7X6sLlRIf6oeWWgK2pRKeKiB4/McuTePPlak= X-Received: by 2002:adf:ecd2:0:b0:236:6fd9:9efa with SMTP id s18-20020adfecd2000000b002366fd99efamr39370660wro.101.1669909382065; Thu, 01 Dec 2022 07:43:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf4imAxqHpsHzf4lwMnv25YvuYKadYTHsioCJaPqnZlFtnFG20xMn8tkXOhL/ZrHUpQYnEbeIQ== X-Received: by 2002:adf:ecd2:0:b0:236:6fd9:9efa with SMTP id s18-20020adfecd2000000b002366fd99efamr39370625wro.101.1669909381800; Thu, 01 Dec 2022 07:43:01 -0800 (PST) Received: from ovpn-194-141.brq.redhat.com (nat-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id bg28-20020a05600c3c9c00b003cfa3a12660sm9307122wmb.1.2022.12.01.07.42.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Dec 2022 07:43:00 -0800 (PST) From: Vitaly Kuznetsov To: Sean Christopherson Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Atish Patra , David Hildenbrand , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, Yuan Yao , Cornelia Huck , Isaku Yamahata , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= , Fabiano Rosas , Michael Ellerman , Kai Huang , Chao Gao , Thomas Gleixner , Paolo Bonzini , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Matthew Rosato , Eric Farman , Sean Christopherson , David Woodhouse , Paul Durrant Subject: Re: [PATCH v2 10/50] KVM: VMX: Reset eVMCS controls in VP assist page during hardware disabling In-Reply-To: <20221130230934.1014142-11-seanjc@google.com> References: <20221130230934.1014142-1-seanjc@google.com> <20221130230934.1014142-11-seanjc@google.com> Date: Thu, 01 Dec 2022 16:42:58 +0100 Message-ID: <87h6yff7ul.fsf@ovpn-194-141.brq.redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221201_074309_105009_CCC47B5F X-CRM114-Status: GOOD ( 31.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Sean Christopherson writes: > Reset the eVMCS controls in the per-CPU VP assist page during hardware > disabling instead of waiting until kvm-intel's module exit. The controls > are activated if and only if KVM creates a VM, i.e. don't need to be > reset if hardware is never enabled. > > Doing the reset during hardware disabling will naturally fix a potential > NULL pointer deref bug once KVM disables CPU hotplug while enabling and > disabling hardware (which is necessary to fix a variety of bugs). If the > kernel is running as the root partition, the VP assist page is unmapped > during CPU hot unplug, and so KVM's clearing of the eVMCS controls needs > to occur with CPU hot(un)plug disabled, otherwise KVM could attempt to > write to a CPU's VP assist page after it's unmapped. > > Reported-by: Vitaly Kuznetsov > Signed-off-by: Sean Christopherson > --- > arch/x86/kvm/vmx/vmx.c | 50 +++++++++++++++++++++++++----------------- > 1 file changed, 30 insertions(+), 20 deletions(-) > > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > index cea8c07f5229..d85d175dca70 100644 > --- a/arch/x86/kvm/vmx/vmx.c > +++ b/arch/x86/kvm/vmx/vmx.c > @@ -551,6 +551,33 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) > return 0; > } > > +static void hv_reset_evmcs(void) > +{ > + struct hv_vp_assist_page *vp_ap; > + > + if (!static_branch_unlikely(&enable_evmcs)) > + return; > + > + /* > + * KVM should enable eVMCS if and only if all CPUs have a VP assist > + * page, and should reject CPU onlining if eVMCS is enabled the CPU > + * doesn't have a VP assist page allocated. > + */ > + vp_ap = hv_get_vp_assist_page(smp_processor_id()); > + if (WARN_ON_ONCE(!vp_ap)) > + return; > + In case my understanding is correct, this may actually get triggered for Hyper-V root partition: vmx_hardware_disable() gets called from kvm_dying_cpu() which has its own CPUHP_AP_KVM_STARTING stage. VP page unmapping happens in hv_cpu_die() which uses generic CPUHP_AP_ONLINE_DYN (happens first on CPU oflining AFAIR). I believe we need to introduce a new CPUHP_AP_HYPERV_STARTING stage and put it before CPUHP_AP_KVM_STARTING so it happens after it upon offlining. The issue is likely theoretical as Hyper-V root partition is a very special case, I'm not sure whether KVM is used there and whether CPU offlining is possible. In any case, WARN_ON_ONCE() is much better than NULL pointer dereference we have now :-) > + /* > + * Reset everything to support using non-enlightened VMCS access later > + * (e.g. when we reload the module with enlightened_vmcs=0) > + */ > + vp_ap->nested_control.features.directhypercall = 0; > + vp_ap->current_nested_vmcs = 0; > + vp_ap->enlighten_vmentry = 0; > +} > + > +#else /* IS_ENABLED(CONFIG_HYPERV) */ > +static void hv_reset_evmcs(void) {} > #endif /* IS_ENABLED(CONFIG_HYPERV) */ > > /* > @@ -2496,6 +2523,8 @@ static void vmx_hardware_disable(void) > if (cpu_vmxoff()) > kvm_spurious_fault(); > > + hv_reset_evmcs(); > + > intel_pt_handle_vmx(0); > } > > @@ -8462,27 +8491,8 @@ static void vmx_exit(void) > kvm_exit(); > > #if IS_ENABLED(CONFIG_HYPERV) > - if (static_branch_unlikely(&enable_evmcs)) { > - int cpu; > - struct hv_vp_assist_page *vp_ap; > - /* > - * Reset everything to support using non-enlightened VMCS > - * access later (e.g. when we reload the module with > - * enlightened_vmcs=0) > - */ > - for_each_online_cpu(cpu) { > - vp_ap = hv_get_vp_assist_page(cpu); > - > - if (!vp_ap) > - continue; > - > - vp_ap->nested_control.features.directhypercall = 0; > - vp_ap->current_nested_vmcs = 0; > - vp_ap->enlighten_vmentry = 0; > - } > - > + if (static_branch_unlikely(&enable_evmcs)) > static_branch_disable(&enable_evmcs); > - } > #endif > vmx_cleanup_l1d_flush(); Reviewed-by: Vitaly Kuznetsov -- Vitaly _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel