From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65039C021B2 for ; Tue, 25 Feb 2025 23:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=I+5VgZvS5RLhaGwFZILFRfiBrL6y7Ik6GC5bOnzJSKo=; b=14rf36CTaxjrx79vFpl5nz5r/K 4FTE9JZSyXxuOCiw3m967QmwMOVLby4ljseqsAxLvDVL6hXPGIdQIAAqHuh9pOwljVX9absrZfWD5 jVtNSm/XbZIn0A3lfqpjHiXQy8YMXysVkta5zCOTFr01jYLO+fhNouoLCXqWwGwwiQdtVMeBxt6iS snqTjan9OhiJnby5Rqh8BYXUh/MTHLE5R91ASgQcpFd6VyB4poOCFHSX/c4fs30RJGc/yCZpbV3Lf TtjMOcRJW+tcGPOvgGzBFcbEloIIFO+R+F2S2f8hcHukmCM2Us3W+7WjpVaIu40oE3HbNBcq1Si8G YQFXoMpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tn4fv-00000001pAV-0X9N; Tue, 25 Feb 2025 23:49:19 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tn4eM-00000001ouz-2Km7 for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 23:47:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740527261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=I+5VgZvS5RLhaGwFZILFRfiBrL6y7Ik6GC5bOnzJSKo=; b=a76CfblVNFPOHDrRAxJzja0S3ZS9PJI43qrAqoPpGvn9o7UfknbjNfr367JzkRioW8k66I oY5+MBwfekrMOXD9GSm+NIGJJVz86gcv20uaGEf/VZz1sWMLEQVjazInB1JQmGIHAAIF++ zhdWZ5pkwd74xFYr7xl3ALBiUvibeHc= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-381-6RTi_4OKOHSkIKD9MMrmqA-1; Tue, 25 Feb 2025 18:47:38 -0500 X-MC-Unique: 6RTi_4OKOHSkIKD9MMrmqA-1 X-Mimecast-MFC-AGG-ID: 6RTi_4OKOHSkIKD9MMrmqA_1740527257 Received: by mail-ed1-f69.google.com with SMTP id 4fb4d7f45d1cf-5dedd3ee338so9349257a12.2 for ; Tue, 25 Feb 2025 15:47:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740527257; x=1741132057; h=content-transfer-encoding:in-reply-to:autocrypt:content-language :from:references:cc:to:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I+5VgZvS5RLhaGwFZILFRfiBrL6y7Ik6GC5bOnzJSKo=; b=iuyzyqWJelbmc4heNbwaUOGuuEy4uZbp6nzkLzTATUq4NVWecIR1reMDmsBUm3WO9o F08dE7rSKixcRP/2cmagwXKKZia+UWg9KNBe8xjBZFAjII6IcPt7A3dT/PyUKFtVrnEH GeGvmErdkikUAsV2wDDenx6ol+YSH7iU/ttIbu2iarlM2bvrPnT7emFGiCcgS9oRUBUq bbl04Ig0YBHHMSx7xDxmzRrrT13eJld6y0pUcKFlfoF7uWTS4PjKOIZ5oQhmZ6C3T0+v /BEwuSJSQconTqVuOF7zZzu2patCmU3O2osE8CGKTCbJkDpcs9b1Mt/0C3V7rMwRMjIm HNMg== X-Gm-Message-State: AOJu0YyUxkp/FWS0O5dMHeYNIX1z1CqK7OwirDLbTu2HIXmjlOR3P9lj uLPEMoZoyHFSJ+cflXwJbEUChlaSrmXUrqeTSCC284L4bVkbrPo1LOEMB8acMCj3XLCib+qJ4VT ycrhrqDzCPGSmGGqTvXgTNGSey2TrE69FWl4bZ6jsaDJudC0ntUqqfzj/g2+TnfgoZtA2d0D9 X-Gm-Gg: ASbGncs5F5hCgISgZ4Otg0Eh+kq9Uozi9zTonCve7PmQTeQCQ5OejfFlowJw3HVWAqQ Z/bxruUpcb5tg4CcdxPXV2VaFzus7VNaKJv/ANcZlzYyoYFN7kanHlYKXkBw8vuPviZynmTHwgK pok/BqFe+mw4vqyaT+fK+YifI5SZbeFqbNKhN/SJ6ZeQCgFA0LaUqUvSE1y4tTUhWbRDeR/fD7t lD2yw88TUARcYsce+UcWsDZFemFKIHd+LMl83gUARxExiJF4sg6uIpRz/IJuES8gMPHO/mNCEdq sCtf4krD3ehJU5dLHKAA X-Received: by 2002:a05:6402:35d1:b0:5e0:750a:5c30 with SMTP id 4fb4d7f45d1cf-5e4a0dfc708mr1644320a12.20.1740527256942; Tue, 25 Feb 2025 15:47:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IGIdTNjyooRvOwtF86c7hT5C/AWZUuE2nxzDPKhqPyR/bEqM/yRWSn2G4IdfhjgLnOUUDk8gQ== X-Received: by 2002:a05:6402:35d1:b0:5e0:750a:5c30 with SMTP id 4fb4d7f45d1cf-5e4a0dfc708mr1644299a12.20.1740527256453; Tue, 25 Feb 2025 15:47:36 -0800 (PST) Received: from [192.168.10.81] ([176.206.102.52]) by smtp.googlemail.com with ESMTPSA id 4fb4d7f45d1cf-5e460ff8629sm1891405a12.59.2025.02.25.15.47.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 25 Feb 2025 15:47:35 -0800 (PST) Message-ID: <6475f9c7-304a-4e0b-8000-3dc5c8e718e9@redhat.com> Date: Wed, 26 Feb 2025 00:47:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/7] KVM: x86: Free vCPUs before freeing VM state To: Sean Christopherson , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Madhavan Srinivasan , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Aaron Lewis , Jim Mattson , Yan Zhao , Rick P Edgecombe , Kai Huang , Isaku Yamahata References: <20250224235542.2562848-1-seanjc@google.com> <20250224235542.2562848-2-seanjc@google.com> From: Paolo Bonzini Autocrypt: addr=pbonzini@redhat.com; keydata= xsEhBFRCcBIBDqDGsz4K0zZun3jh+U6Z9wNGLKQ0kSFyjN38gMqU1SfP+TUNQepFHb/Gc0E2 CxXPkIBTvYY+ZPkoTh5xF9oS1jqI8iRLzouzF8yXs3QjQIZ2SfuCxSVwlV65jotcjD2FTN04 hVopm9llFijNZpVIOGUTqzM4U55sdsCcZUluWM6x4HSOdw5F5Utxfp1wOjD/v92Lrax0hjiX DResHSt48q+8FrZzY+AUbkUS+Jm34qjswdrgsC5uxeVcLkBgWLmov2kMaMROT0YmFY6A3m1S P/kXmHDXxhe23gKb3dgwxUTpENDBGcfEzrzilWueOeUWiOcWuFOed/C3SyijBx3Av/lbCsHU Vx6pMycNTdzU1BuAroB+Y3mNEuW56Yd44jlInzG2UOwt9XjjdKkJZ1g0P9dwptwLEgTEd3Fo UdhAQyRXGYO8oROiuh+RZ1lXp6AQ4ZjoyH8WLfTLf5g1EKCTc4C1sy1vQSdzIRu3rBIjAvnC tGZADei1IExLqB3uzXKzZ1BZ+Z8hnt2og9hb7H0y8diYfEk2w3R7wEr+Ehk5NQsT2MPI2QBd wEv1/Aj1DgUHZAHzG1QN9S8wNWQ6K9DqHZTBnI1hUlkp22zCSHK/6FwUCuYp1zcAEQEAAc0j UGFvbG8gQm9uemluaSA8cGJvbnppbmlAcmVkaGF0LmNvbT7CwU0EEwECACMFAlRCcBICGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRB+FRAMzTZpsbceDp9IIN6BIA0Ol7MoB15E 11kRz/ewzryFY54tQlMnd4xxfH8MTQ/mm9I482YoSwPMdcWFAKnUX6Yo30tbLiNB8hzaHeRj jx12K+ptqYbg+cevgOtbLAlL9kNgLLcsGqC2829jBCUTVeMSZDrzS97ole/YEez2qFpPnTV0 VrRWClWVfYh+JfzpXmgyhbkuwUxNFk421s4Ajp3d8nPPFUGgBG5HOxzkAm7xb1cjAuJ+oi/K CHfkuN+fLZl/u3E/fw7vvOESApLU5o0icVXeakfSz0LsygEnekDbxPnE5af/9FEkXJD5EoYG SEahaEtgNrR4qsyxyAGYgZlS70vkSSYJ+iT2rrwEiDlo31MzRo6Ba2FfHBSJ7lcYdPT7bbk9 AO3hlNMhNdUhoQv7M5HsnqZ6unvSHOKmReNaS9egAGdRN0/GPDWr9wroyJ65ZNQsHl9nXBqE AukZNr5oJO5vxrYiAuuTSd6UI/xFkjtkzltG3mw5ao2bBpk/V/YuePrJsnPFHG7NhizrxttB nTuOSCMo45pfHQ+XYd5K1+Cv/NzZFNWscm5htJ0HznY+oOsZvHTyGz3v91pn51dkRYN0otqr bQ4tlFFuVjArBZcapSIe6NV8C4cEiSTOwE0EVEJx7gEIAMeHcVzuv2bp9HlWDp6+RkZe+vtl KwAHplb/WH59j2wyG8V6i33+6MlSSJMOFnYUCCL77bucx9uImI5nX24PIlqT+zasVEEVGSRF m8dgkcJDB7Tps0IkNrUi4yof3B3shR+vMY3i3Ip0e41zKx0CvlAhMOo6otaHmcxr35sWq1Jk tLkbn3wG+fPQCVudJJECvVQ//UAthSSEklA50QtD2sBkmQ14ZryEyTHQ+E42K3j2IUmOLriF dNr9NvE1QGmGyIcbw2NIVEBOK/GWxkS5+dmxM2iD4Jdaf2nSn3jlHjEXoPwpMs0KZsgdU0pP JQzMUMwmB1wM8JxovFlPYrhNT9MAEQEAAcLBMwQYAQIACQUCVEJx7gIbDAAKCRB+FRAMzTZp sadRDqCctLmYICZu4GSnie4lKXl+HqlLanpVMOoFNnWs9oRP47MbE2wv8OaYh5pNR9VVgyhD OG0AU7oidG36OeUlrFDTfnPYYSF/mPCxHttosyt8O5kabxnIPv2URuAxDByz+iVbL+RjKaGM GDph56ZTswlx75nZVtIukqzLAQ5fa8OALSGum0cFi4ptZUOhDNz1onz61klD6z3MODi0sBZN Aj6guB2L/+2ZwElZEeRBERRd/uommlYuToAXfNRdUwrwl9gRMiA0WSyTb190zneRRDfpSK5d usXnM/O+kr3Dm+Ui+UioPf6wgbn3T0o6I5BhVhs4h4hWmIW7iNhPjX1iybXfmb1gAFfjtHfL xRUr64svXpyfJMScIQtBAm0ihWPltXkyITA92ngCmPdHa6M1hMh4RDX+Jf1fiWubzp1voAg0 JBrdmNZSQDz0iKmSrx8xkoXYfA3bgtFN8WJH2xgFL28XnqY4M6dLhJwV3z08tPSRqYFm4NMP dRsn0/7oymhneL8RthIvjDDQ5ktUjMe8LtHr70OZE/TT88qvEdhiIVUogHdo4qBrk41+gGQh b906Dudw5YhTJFU3nC6bbF2nrLlB4C/XSiH76ZvqzV0Z/cAMBo5NF/w= In-Reply-To: <20250224235542.2562848-2-seanjc@google.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: w-BNuHNFBCjBFGHz1shJeiNqzl2sKTBUvcTVsWcoxe0_1740527257 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_154742_663475_A93E80D6 X-CRM114-Status: GOOD ( 20.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2/25/25 00:55, Sean Christopherson wrote: > Free vCPUs before freeing any VM state, as both SVM and VMX may access > VM state when "freeing" a vCPU that is currently "in" L2, i.e. that needs > to be kicked out of nested guest mode. > > Commit 6fcee03df6a1 ("KVM: x86: avoid loading a vCPU after .vm_destroy was > called") partially fixed the issue, but for unknown reasons only moved the > MMU unloading before VM destruction. Complete the change, and free all > vCPU state prior to destroying VM state, as nVMX accesses even more state > than nSVM. I applied this to kvm-coco-queue, I will place it in kvm/master too unless you shout. Paolo > In addition to the AVIC, KVM can hit a use-after-free on MSR filters: > > kvm_msr_allowed+0x4c/0xd0 > __kvm_set_msr+0x12d/0x1e0 > kvm_set_msr+0x19/0x40 > load_vmcs12_host_state+0x2d8/0x6e0 [kvm_intel] > nested_vmx_vmexit+0x715/0xbd0 [kvm_intel] > nested_vmx_free_vcpu+0x33/0x50 [kvm_intel] > vmx_free_vcpu+0x54/0xc0 [kvm_intel] > kvm_arch_vcpu_destroy+0x28/0xf0 > kvm_vcpu_destroy+0x12/0x50 > kvm_arch_destroy_vm+0x12c/0x1c0 > kvm_put_kvm+0x263/0x3c0 > kvm_vm_release+0x21/0x30 > > and an upcoming fix to process injectable interrupts on nested VM-Exit > will access the PIC: > > BUG: kernel NULL pointer dereference, address: 0000000000000090 > #PF: supervisor read access in kernel mode > #PF: error_code(0x0000) - not-present page > CPU: 23 UID: 1000 PID: 2658 Comm: kvm-nx-lpage-re > RIP: 0010:kvm_cpu_has_extint+0x2f/0x60 [kvm] > Call Trace: > > kvm_cpu_has_injectable_intr+0xe/0x60 [kvm] > nested_vmx_vmexit+0x2d7/0xdf0 [kvm_intel] > nested_vmx_free_vcpu+0x40/0x50 [kvm_intel] > vmx_vcpu_free+0x2d/0x80 [kvm_intel] > kvm_arch_vcpu_destroy+0x2d/0x130 [kvm] > kvm_destroy_vcpus+0x8a/0x100 [kvm] > kvm_arch_destroy_vm+0xa7/0x1d0 [kvm] > kvm_destroy_vm+0x172/0x300 [kvm] > kvm_vcpu_release+0x31/0x50 [kvm] > > Inarguably, both nSVM and nVMX need to be fixed, but punt on those > cleanups for the moment. Conceptually, vCPUs should be freed before VM > state. Assets like the I/O APIC and PIC _must_ be allocated before vCPUs > are created, so it stands to reason that they must be freed _after_ vCPUs > are destroyed. > > Reported-by: Aaron Lewis > Closes: https://lore.kernel.org/all/20240703175618.2304869-2-aaronlewis@google.com > Cc: Jim Mattson > Cc: Yan Zhao > Cc: Rick P Edgecombe > Cc: Kai Huang > Cc: Isaku Yamahata > Signed-off-by: Sean Christopherson > --- > arch/x86/kvm/x86.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 58b82d6fd77c..045c61cc7e54 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -12890,11 +12890,11 @@ void kvm_arch_destroy_vm(struct kvm *kvm) > mutex_unlock(&kvm->slots_lock); > } > kvm_unload_vcpu_mmus(kvm); > + kvm_destroy_vcpus(kvm); > kvm_x86_call(vm_destroy)(kvm); > kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1)); > kvm_pic_destroy(kvm); > kvm_ioapic_destroy(kvm); > - kvm_destroy_vcpus(kvm); > kvfree(rcu_dereference_check(kvm->arch.apic_map, 1)); > kfree(srcu_dereference_check(kvm->arch.pmu_event_filter, &kvm->srcu, 1)); > kvm_mmu_uninit_vm(kvm);