From mboxrd@z Thu Jan 1 00:00:00 1970 From: Isaku Yamahata Date: Mon, 7 Nov 2022 21:43:54 -0800 Subject: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling In-Reply-To: References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Tue, Nov 08, 2022 at 01:09:27AM +0000, "Huang, Kai" wrote: > On Mon, 2022-11-07 at 13:46 -0800, Isaku Yamahata wrote: > > > On Fri, Nov 04, 2022, Isaku Yamahata wrote: > > > > Thanks for the patch series. I the rebased TDX KVM patch series and it > > > > worked. > > > > Since cpu offline needs to be rejected in some cases(To keep at least one > > > > cpu > > > > on a package), arch hook for cpu offline is needed. > > > > > > I hate to bring this up because I doubt there's a real use case for SUSPEND > > > with > > > TDX, but the CPU offline path isn't just for true offlining of CPUs.? When > > > the > > > system enters SUSPEND, only the initiating CPU goes through > > > kvm_suspend()+kvm_resume(), > > > all responding CPUs go through CPU offline+online.? I.e. disallowing all > > > CPUs from > > > going "offline" will prevent suspending the system. > > > > The current TDX KVM implementation disallows CPU package from offline only > > when > > TDs are running.? If no TD is running, CPU offline is allowed.? So before > > SUSPEND, TDs need to be killed via systemd or something.? After killing TDs, > > the > > system can enter into SUSPEND state. > > This seems not correct. You need one cpu for each to be online in order to > create TD as well, as TDH.MNG.KEY.CONFIG needs to be called on all packages, > correct? That's correct. In such case, the creation of TD fails. TD creation checks if at least one cpu is online on all CPU packages. If no, error. -- Isaku Yamahata From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id A36DFC4167E for ; Tue, 8 Nov 2022 05:44:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 115134B8C9; Tue, 8 Nov 2022 00:44:00 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@gmail.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HzGQY2V-YbvX; Tue, 8 Nov 2022 00:43:58 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C79464B8CA; Tue, 8 Nov 2022 00:43:58 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E0EA64B8C6 for ; Tue, 8 Nov 2022 00:43:57 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JRp2LyOlk0cI for ; Tue, 8 Nov 2022 00:43:56 -0500 (EST) Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 8DBB14B8C5 for ; Tue, 8 Nov 2022 00:43:56 -0500 (EST) Received: by mail-pl1-f176.google.com with SMTP id io19so13227587plb.8 for ; Mon, 07 Nov 2022 21:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=XNdeC13pb4LjOBaRWcKMBNqFRJ+RgsbfVoNw18z5L+vfkg1SZceZjUxbLhiIy1qySy Y4hI02da8j9ev8qt2sUsGeUiOSLlDAUgz5JUplPGm8kAk4jLtNdmDFZCCStM1EGIelNU pjjtL4FSGAVwgisQucoSj4vLVpkgLqzZdzX9wYytiy3sfYNB3aYKj8S/Nug5gaKAa8gA /5oDb+KCNmSa3bC/jNyCQmeij+quPN3cyVSSr1ovXCPVJxrqYAAydTsm3Q4bRt6yNhxm 6FVSyKTuVZN23UWZkdV6Uc7djgZpboRwjQPImKvKrchYImM3eB6pVBMtJk1jJ8Nfk6ob kIoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=kXXHLvT/UIkLM9YSnn1YN/mNjTd8SctGVJwGJgMiWsk/cjLOgiyjqiu+9RoiXWi6IT CF5dNvpuSpgUvnkP6yk++a5aIkmbIxHEyQg4I1l5hnSA54/gkB+5BL3mGsU5/axHgSdS wLYZqNDlu9pXXNNSYfV+LuvyVZ1tNKJS8R2xF6k2OtaFKKUqGGHG++MxZM1PTkgPSnT7 S5Bl2wLlI6yUPEZUz8nGqcUpq7fJU2i3zVtgJr6CPG76npic7zpOf8vgq648tgOBQPZr 4weoyC6UYlVDiLx+nXi2w/ICZNagsXwYguVD3wAbjV8wSiaCk7TFMnKHGe/2t8+aCZ3x FBzQ== X-Gm-Message-State: ACrzQf21o8DBHItMIte+Z/AGJdkIuuhazsOvsr9hxgS7Q99Ngahx3PXx ps+7kF0stU1RlVNUaRiS0gY= X-Google-Smtp-Source: AMsMyM6Ssf7qu6Zt1cmqzeY+6ksfJF4jyx1t2euaZBhwzvq5JPVY+8zsaODdG48vWXQYQDWTdtSyvw== X-Received: by 2002:a17:90b:4a04:b0:213:587b:204e with SMTP id kk4-20020a17090b4a0400b00213587b204emr55088612pjb.98.1667886235549; Mon, 07 Nov 2022 21:43:55 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id k21-20020a628415000000b0056bb06ce1cfsm5614872pfd.97.2022.11.07.21.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 21:43:55 -0800 (PST) Date: Mon, 7 Nov 2022 21:43:54 -0800 From: Isaku Yamahata To: "Huang, Kai" Subject: Re: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Cc: "mjrosato@linux.ibm.com" , "david@redhat.com" , "paul.walmsley@sifive.com" , "linux-kernel@vger.kernel.org" , "linux-riscv@lists.infradead.org" , "imbrenda@linux.ibm.com" , "kvmarm@lists.cs.columbia.edu" , "isaku.yamahata@gmail.com" , "linux-s390@vger.kernel.org" , "kvm@vger.kernel.org" , "mpe@ellerman.id.au" , "chenhuacai@kernel.org" , "aleksandar.qemu.devel@gmail.com" , "borntraeger@linux.ibm.com" , "Gao, Chao" , "farman@linux.ibm.com" , "aou@eecs.berkeley.edu" , "Yao, Yuan" , "kvmarm@lists.linux.dev" , "tglx@linutronix.de" , "linux-arm-kernel@lists.infradead.org" , "frankja@linux.ibm.com" , "Yamahata, Isaku" , "atishp@atishpatra.org" , "farosas@linux.ibm.com" , "linux-mips@vger.kernel.org" , "palmer@dabbelt.com" , "kvm-riscv@lists.infradead.org" , "maz@kernel.org" , "pbonzini@redhat.com" , "vkuznets@redhat.com" , "linuxppc-dev@lists.ozlabs.org" X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu T24gVHVlLCBOb3YgMDgsIDIwMjIgYXQgMDE6MDk6MjdBTSArMDAwMCwKIkh1YW5nLCBLYWkiIDxr YWkuaHVhbmdAaW50ZWwuY29tPiB3cm90ZToKCj4gT24gTW9uLCAyMDIyLTExLTA3IGF0IDEzOjQ2 IC0wODAwLCBJc2FrdSBZYW1haGF0YSB3cm90ZToKPiA+ID4gT24gRnJpLCBOb3YgMDQsIDIwMjIs IElzYWt1IFlhbWFoYXRhIHdyb3RlOgo+ID4gPiA+IFRoYW5rcyBmb3IgdGhlIHBhdGNoIHNlcmll cy4gSSB0aGUgcmViYXNlZCBURFggS1ZNIHBhdGNoIHNlcmllcyBhbmQgaXQKPiA+ID4gPiB3b3Jr ZWQuCj4gPiA+ID4gU2luY2UgY3B1IG9mZmxpbmUgbmVlZHMgdG8gYmUgcmVqZWN0ZWQgaW4gc29t ZSBjYXNlcyhUbyBrZWVwIGF0IGxlYXN0IG9uZQo+ID4gPiA+IGNwdQo+ID4gPiA+IG9uIGEgcGFj a2FnZSksIGFyY2ggaG9vayBmb3IgY3B1IG9mZmxpbmUgaXMgbmVlZGVkLgo+ID4gPiAKPiA+ID4g SSBoYXRlIHRvIGJyaW5nIHRoaXMgdXAgYmVjYXVzZSBJIGRvdWJ0IHRoZXJlJ3MgYSByZWFsIHVz ZSBjYXNlIGZvciBTVVNQRU5ECj4gPiA+IHdpdGgKPiA+ID4gVERYLCBidXQgdGhlIENQVSBvZmZs aW5lIHBhdGggaXNuJ3QganVzdCBmb3IgdHJ1ZSBvZmZsaW5pbmcgb2YgQ1BVcy7CoCBXaGVuCj4g PiA+IHRoZQo+ID4gPiBzeXN0ZW0gZW50ZXJzIFNVU1BFTkQsIG9ubHkgdGhlIGluaXRpYXRpbmcg Q1BVIGdvZXMgdGhyb3VnaAo+ID4gPiBrdm1fc3VzcGVuZCgpK2t2bV9yZXN1bWUoKSwKPiA+ID4g YWxsIHJlc3BvbmRpbmcgQ1BVcyBnbyB0aHJvdWdoIENQVSBvZmZsaW5lK29ubGluZS7CoCBJLmUu IGRpc2FsbG93aW5nIGFsbAo+ID4gPiBDUFVzIGZyb20KPiA+ID4gZ29pbmcgIm9mZmxpbmUiIHdp bGwgcHJldmVudCBzdXNwZW5kaW5nIHRoZSBzeXN0ZW0uCj4gPiAKPiA+IFRoZSBjdXJyZW50IFRE WCBLVk0gaW1wbGVtZW50YXRpb24gZGlzYWxsb3dzIENQVSBwYWNrYWdlIGZyb20gb2ZmbGluZSBv bmx5Cj4gPiB3aGVuCj4gPiBURHMgYXJlIHJ1bm5pbmcuwqAgSWYgbm8gVEQgaXMgcnVubmluZywg Q1BVIG9mZmxpbmUgaXMgYWxsb3dlZC7CoCBTbyBiZWZvcmUKPiA+IFNVU1BFTkQsIFREcyBuZWVk IHRvIGJlIGtpbGxlZCB2aWEgc3lzdGVtZCBvciBzb21ldGhpbmcuwqAgQWZ0ZXIga2lsbGluZyBU RHMsCj4gPiB0aGUKPiA+IHN5c3RlbSBjYW4gZW50ZXIgaW50byBTVVNQRU5EIHN0YXRlLgo+IAo+ IFRoaXMgc2VlbXMgbm90IGNvcnJlY3QuICBZb3UgbmVlZCBvbmUgY3B1IGZvciBlYWNoIHRvIGJl IG9ubGluZSBpbiBvcmRlciB0bwo+IGNyZWF0ZSBURCBhcyB3ZWxsLCBhcyBUREguTU5HLktFWS5D T05GSUcgbmVlZHMgdG8gYmUgY2FsbGVkIG9uIGFsbCBwYWNrYWdlcywKPiBjb3JyZWN0PwoKVGhh dCdzIGNvcnJlY3QuIEluIHN1Y2ggY2FzZSwgdGhlIGNyZWF0aW9uIG9mIFREIGZhaWxzLiAgVEQg Y3JlYXRpb24gY2hlY2tzIGlmCmF0IGxlYXN0IG9uZSBjcHUgaXMgb25saW5lIG9uIGFsbCBDUFUg cGFja2FnZXMuICBJZiBubywgZXJyb3IuCi0tIApJc2FrdSBZYW1haGF0YSA8aXNha3UueWFtYWhh dGFAZ21haWwuY29tPgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwprdm1hcm0gbWFpbGluZyBsaXN0Cmt2bWFybUBsaXN0cy5jcy5jb2x1bWJpYS5lZHUKaHR0 cHM6Ly9saXN0cy5jcy5jb2x1bWJpYS5lZHUvbWFpbG1hbi9saXN0aW5mby9rdm1hcm0K From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4410D5CB0 for ; Tue, 8 Nov 2022 05:43:56 +0000 (UTC) Received: by mail-pl1-f179.google.com with SMTP id l2so13204591pld.13 for ; Mon, 07 Nov 2022 21:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=XNdeC13pb4LjOBaRWcKMBNqFRJ+RgsbfVoNw18z5L+vfkg1SZceZjUxbLhiIy1qySy Y4hI02da8j9ev8qt2sUsGeUiOSLlDAUgz5JUplPGm8kAk4jLtNdmDFZCCStM1EGIelNU pjjtL4FSGAVwgisQucoSj4vLVpkgLqzZdzX9wYytiy3sfYNB3aYKj8S/Nug5gaKAa8gA /5oDb+KCNmSa3bC/jNyCQmeij+quPN3cyVSSr1ovXCPVJxrqYAAydTsm3Q4bRt6yNhxm 6FVSyKTuVZN23UWZkdV6Uc7djgZpboRwjQPImKvKrchYImM3eB6pVBMtJk1jJ8Nfk6ob kIoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=1G3fsbH1GhvXaz8hz+nfg/tLNJJaZ815+rp6TvhxvB/X1GJ+PERZPlTTMIVfW/lIqB iyv5ylmcGFRiSsO0El2+ZlAy7zW+hcfRdeEa4Inh5KOGkYubRMHAETYYfDZIXseuCqyI 2QvxsNTf0H4NKBHc5T+uJVEcm/6B4V2RfwAVZPjZ8PhbrOty1C+Tl8pzxsYR36y1iqzC gmptWbuvUD6izYZlm928sujy+NBkzXTTm8HkYcEHTtN/BqAqAeRiiQ1EAcAT8YIyTWmS J0tn0QcIm8fCdO5aUbLNgyfo16ltTs+XllmRRiq+PHWwk7UFVk40SLtXb535KF0yKOMr svWQ== X-Gm-Message-State: ACrzQf1P0OLvROBsPsuARzXo2bRS+B/s0gBI1lbfANMMjQaCEdvaIrMT T4FmYvbULMW4RYr6TxWvWWg= X-Google-Smtp-Source: AMsMyM6Ssf7qu6Zt1cmqzeY+6ksfJF4jyx1t2euaZBhwzvq5JPVY+8zsaODdG48vWXQYQDWTdtSyvw== X-Received: by 2002:a17:90b:4a04:b0:213:587b:204e with SMTP id kk4-20020a17090b4a0400b00213587b204emr55088612pjb.98.1667886235549; Mon, 07 Nov 2022 21:43:55 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id k21-20020a628415000000b0056bb06ce1cfsm5614872pfd.97.2022.11.07.21.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 21:43:55 -0800 (PST) Date: Mon, 7 Nov 2022 21:43:54 -0800 From: Isaku Yamahata To: "Huang, Kai" Cc: "Christopherson,, Sean" , "isaku.yamahata@gmail.com" , "borntraeger@linux.ibm.com" , "kvm-riscv@lists.infradead.org" , "tglx@linutronix.de" , "Yao, Yuan" , "kvm@vger.kernel.org" , "Yamahata, Isaku" , "suzuki.poulose@arm.com" , "pbonzini@redhat.com" , "david@redhat.com" , "linux-mips@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , "linux-riscv@lists.infradead.org" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "mjrosato@linux.ibm.com" , "oliver.upton@linux.dev" , "farosas@linux.ibm.com" , "palmer@dabbelt.com" , "linux-s390@vger.kernel.org" , "chenhuacai@kernel.org" , "aou@eecs.berkeley.edu" , "alexandru.elisei@arm.com" , "mpe@ellerman.id.au" , "vkuznets@redhat.com" , "maz@kernel.org" , "anup@brainfault.org" , "frankja@linux.ibm.com" , "farman@linux.ibm.com" , "james.morse@arm.com" , "kvmarm@lists.cs.columbia.edu" , "aleksandar.qemu.devel@gmail.com" , "paul.walmsley@sifive.com" , "linux-arm-kernel@lists.infradead.org" , "atishp@atishpatra.org" , "imbrenda@linux.ibm.com" , "Gao, Chao" Subject: Re: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Message-ID: <20221108054354.rrjjOuSezHxNS6jyt3LdkV0L1V_D2fYlNz2nE-vJhTw@z> On Tue, Nov 08, 2022 at 01:09:27AM +0000, "Huang, Kai" wrote: > On Mon, 2022-11-07 at 13:46 -0800, Isaku Yamahata wrote: > > > On Fri, Nov 04, 2022, Isaku Yamahata wrote: > > > > Thanks for the patch series. I the rebased TDX KVM patch series and it > > > > worked. > > > > Since cpu offline needs to be rejected in some cases(To keep at least one > > > > cpu > > > > on a package), arch hook for cpu offline is needed. > > > > > > I hate to bring this up because I doubt there's a real use case for SUSPEND > > > with > > > TDX, but the CPU offline path isn't just for true offlining of CPUs.  When > > > the > > > system enters SUSPEND, only the initiating CPU goes through > > > kvm_suspend()+kvm_resume(), > > > all responding CPUs go through CPU offline+online.  I.e. disallowing all > > > CPUs from > > > going "offline" will prevent suspending the system. > > > > The current TDX KVM implementation disallows CPU package from offline only > > when > > TDs are running.  If no TD is running, CPU offline is allowed.  So before > > SUSPEND, TDs need to be killed via systemd or something.  After killing TDs, > > the > > system can enter into SUSPEND state. > > This seems not correct. You need one cpu for each to be online in order to > create TD as well, as TDH.MNG.KEY.CONFIG needs to be called on all packages, > correct? That's correct. In such case, the creation of TD fails. TD creation checks if at least one cpu is online on all CPU packages. If no, error. -- Isaku Yamahata From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C685C433FE for ; Tue, 8 Nov 2022 05:44:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=u6ehrpyGwkFXlH6rYjdoUVFJXoSJdKFqmYaVbX0OOvE=; b=tf630/r0pBXrfQ lvYDfjB+/tUxScG6aWaXCQRQ1OKPULgr9kfq0iFwpx6pjJMrRZCKF33HshT8MOuVLLVXRASStvoYD bKCYUx7gibZyZKLAaaXu2lizSsgJfzxb075k39TcO4RfhhtjgSS69b22Dz6FHeCO2SfJrGpEtqFZi UQAQ20E4QnXdnUVfIJGZEwZs/JbQN5fOH/43S+oCJaC/2Bpe3RpoFkkbRfPvzuG1boVOIojOoS7/6 MqhIDQQLXzhpqJHhpzzNUpPDwekW3IyZmmaOyQ/VSTPy4+5LLaDrkVlrtHT1W7r29VsfeeuC/UnDy ETU6qpaLRP1wMKNZ7ZVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1osHP1-002vWh-54; Tue, 08 Nov 2022 05:44:03 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1osHOy-002vUM-Bx; Tue, 08 Nov 2022 05:44:01 +0000 Received: by mail-pl1-x62c.google.com with SMTP id b21so13229100plc.9; Mon, 07 Nov 2022 21:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=XNdeC13pb4LjOBaRWcKMBNqFRJ+RgsbfVoNw18z5L+vfkg1SZceZjUxbLhiIy1qySy Y4hI02da8j9ev8qt2sUsGeUiOSLlDAUgz5JUplPGm8kAk4jLtNdmDFZCCStM1EGIelNU pjjtL4FSGAVwgisQucoSj4vLVpkgLqzZdzX9wYytiy3sfYNB3aYKj8S/Nug5gaKAa8gA /5oDb+KCNmSa3bC/jNyCQmeij+quPN3cyVSSr1ovXCPVJxrqYAAydTsm3Q4bRt6yNhxm 6FVSyKTuVZN23UWZkdV6Uc7djgZpboRwjQPImKvKrchYImM3eB6pVBMtJk1jJ8Nfk6ob kIoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=JaLkEfL7XnTBc0JIT82t+Mg7eW2Sy3flENaVZyynXYOt97Y6mcYbBMKdkCOzJdZwzs PRs9gBkvSyq/eTPeDn2LdselQ/pmEMIIsMZqMH3zE0nn6e5DG9h317g/3P9jL+9g+h98 sCc6qG0dvSQR0wJCK/+6TjnIM1SgIWKEcdWM5/uMfVHfStHpeT5v+VKV3YyjjJ9hQoDA 4Eucml+JFpqnMlFU4ggup/4feZOpBY3bTu2lFPKJQtnsa/2h/0ryTGh7/23Jm+6HTz7E 5+bwIwOFGZsJ6oy5eQ/FRkxQETh7Iz7NeUnkcxuu8mSE/tL8tu/EZW5GKhzwTCX/kBYG D/JQ== X-Gm-Message-State: ACrzQf0EkP4utXkqow4lFjXHJm84jAbgAk6PPAj0FagUMQkbBVrDMHwb GYEDRkjf7OpcJLH/iRmtTmk= X-Google-Smtp-Source: AMsMyM6Ssf7qu6Zt1cmqzeY+6ksfJF4jyx1t2euaZBhwzvq5JPVY+8zsaODdG48vWXQYQDWTdtSyvw== X-Received: by 2002:a17:90b:4a04:b0:213:587b:204e with SMTP id kk4-20020a17090b4a0400b00213587b204emr55088612pjb.98.1667886235549; Mon, 07 Nov 2022 21:43:55 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id k21-20020a628415000000b0056bb06ce1cfsm5614872pfd.97.2022.11.07.21.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 21:43:55 -0800 (PST) Date: Mon, 7 Nov 2022 21:43:54 -0800 From: Isaku Yamahata To: "Huang, Kai" Cc: "Christopherson,, Sean" , "isaku.yamahata@gmail.com" , "borntraeger@linux.ibm.com" , "kvm-riscv@lists.infradead.org" , "tglx@linutronix.de" , "Yao, Yuan" , "kvm@vger.kernel.org" , "Yamahata, Isaku" , "suzuki.poulose@arm.com" , "pbonzini@redhat.com" , "david@redhat.com" , "linux-mips@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , "linux-riscv@lists.infradead.org" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "mjrosato@linux.ibm.com" , "oliver.upton@linux.dev" , "farosas@linux.ibm.com" , "palmer@dabbelt.com" , "linux-s390@vger.kernel.org" , "chenhuacai@kernel.org" , "aou@eecs.berkeley.edu" , "alexandru.elisei@arm.com" , "mpe@ellerman.id.au" , "vkuznets@redhat.com" , "maz@kernel.org" , "anup@brainfault.org" , "frankja@linux.ibm.com" , "farman@linux.ibm.com" , "james.morse@arm.com" , "kvmarm@lists.cs.columbia.edu" , "aleksandar.qemu.devel@gmail.com" , "paul.walmsley@sifive.com" , "linux-arm-kernel@lists.infradead.org" , "atishp@atishpatra.org" , "imbrenda@linux.ibm.com" , "Gao, Chao" Subject: Re: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221107_214400_449613_57CA35A4 X-CRM114-Status: GOOD ( 21.60 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gVHVlLCBOb3YgMDgsIDIwMjIgYXQgMDE6MDk6MjdBTSArMDAwMCwKIkh1YW5nLCBLYWkiIDxr YWkuaHVhbmdAaW50ZWwuY29tPiB3cm90ZToKCj4gT24gTW9uLCAyMDIyLTExLTA3IGF0IDEzOjQ2 IC0wODAwLCBJc2FrdSBZYW1haGF0YSB3cm90ZToKPiA+ID4gT24gRnJpLCBOb3YgMDQsIDIwMjIs IElzYWt1IFlhbWFoYXRhIHdyb3RlOgo+ID4gPiA+IFRoYW5rcyBmb3IgdGhlIHBhdGNoIHNlcmll cy4gSSB0aGUgcmViYXNlZCBURFggS1ZNIHBhdGNoIHNlcmllcyBhbmQgaXQKPiA+ID4gPiB3b3Jr ZWQuCj4gPiA+ID4gU2luY2UgY3B1IG9mZmxpbmUgbmVlZHMgdG8gYmUgcmVqZWN0ZWQgaW4gc29t ZSBjYXNlcyhUbyBrZWVwIGF0IGxlYXN0IG9uZQo+ID4gPiA+IGNwdQo+ID4gPiA+IG9uIGEgcGFj a2FnZSksIGFyY2ggaG9vayBmb3IgY3B1IG9mZmxpbmUgaXMgbmVlZGVkLgo+ID4gPiAKPiA+ID4g SSBoYXRlIHRvIGJyaW5nIHRoaXMgdXAgYmVjYXVzZSBJIGRvdWJ0IHRoZXJlJ3MgYSByZWFsIHVz ZSBjYXNlIGZvciBTVVNQRU5ECj4gPiA+IHdpdGgKPiA+ID4gVERYLCBidXQgdGhlIENQVSBvZmZs aW5lIHBhdGggaXNuJ3QganVzdCBmb3IgdHJ1ZSBvZmZsaW5pbmcgb2YgQ1BVcy7CoCBXaGVuCj4g PiA+IHRoZQo+ID4gPiBzeXN0ZW0gZW50ZXJzIFNVU1BFTkQsIG9ubHkgdGhlIGluaXRpYXRpbmcg Q1BVIGdvZXMgdGhyb3VnaAo+ID4gPiBrdm1fc3VzcGVuZCgpK2t2bV9yZXN1bWUoKSwKPiA+ID4g YWxsIHJlc3BvbmRpbmcgQ1BVcyBnbyB0aHJvdWdoIENQVSBvZmZsaW5lK29ubGluZS7CoCBJLmUu IGRpc2FsbG93aW5nIGFsbAo+ID4gPiBDUFVzIGZyb20KPiA+ID4gZ29pbmcgIm9mZmxpbmUiIHdp bGwgcHJldmVudCBzdXNwZW5kaW5nIHRoZSBzeXN0ZW0uCj4gPiAKPiA+IFRoZSBjdXJyZW50IFRE WCBLVk0gaW1wbGVtZW50YXRpb24gZGlzYWxsb3dzIENQVSBwYWNrYWdlIGZyb20gb2ZmbGluZSBv bmx5Cj4gPiB3aGVuCj4gPiBURHMgYXJlIHJ1bm5pbmcuwqAgSWYgbm8gVEQgaXMgcnVubmluZywg Q1BVIG9mZmxpbmUgaXMgYWxsb3dlZC7CoCBTbyBiZWZvcmUKPiA+IFNVU1BFTkQsIFREcyBuZWVk IHRvIGJlIGtpbGxlZCB2aWEgc3lzdGVtZCBvciBzb21ldGhpbmcuwqAgQWZ0ZXIga2lsbGluZyBU RHMsCj4gPiB0aGUKPiA+IHN5c3RlbSBjYW4gZW50ZXIgaW50byBTVVNQRU5EIHN0YXRlLgo+IAo+ IFRoaXMgc2VlbXMgbm90IGNvcnJlY3QuICBZb3UgbmVlZCBvbmUgY3B1IGZvciBlYWNoIHRvIGJl IG9ubGluZSBpbiBvcmRlciB0bwo+IGNyZWF0ZSBURCBhcyB3ZWxsLCBhcyBUREguTU5HLktFWS5D T05GSUcgbmVlZHMgdG8gYmUgY2FsbGVkIG9uIGFsbCBwYWNrYWdlcywKPiBjb3JyZWN0PwoKVGhh dCdzIGNvcnJlY3QuIEluIHN1Y2ggY2FzZSwgdGhlIGNyZWF0aW9uIG9mIFREIGZhaWxzLiAgVEQg Y3JlYXRpb24gY2hlY2tzIGlmCmF0IGxlYXN0IG9uZSBjcHUgaXMgb25saW5lIG9uIGFsbCBDUFUg cGFja2FnZXMuICBJZiBubywgZXJyb3IuCi0tIApJc2FrdSBZYW1haGF0YSA8aXNha3UueWFtYWhh dGFAZ21haWwuY29tPgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVh ZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1y aXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6E2FC433FE for ; Tue, 8 Nov 2022 05:44:53 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4N5xrN0LWLz3cK7 for ; Tue, 8 Nov 2022 16:44:52 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=XNdeC13p; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::62b; helo=mail-pl1-x62b.google.com; envelope-from=isaku.yamahata@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=XNdeC13p; dkim-atps=neutral Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4N5xqM3wSDz3c1K for ; Tue, 8 Nov 2022 16:43:57 +1100 (AEDT) Received: by mail-pl1-x62b.google.com with SMTP id v17so13254950plo.1 for ; Mon, 07 Nov 2022 21:43:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=XNdeC13pb4LjOBaRWcKMBNqFRJ+RgsbfVoNw18z5L+vfkg1SZceZjUxbLhiIy1qySy Y4hI02da8j9ev8qt2sUsGeUiOSLlDAUgz5JUplPGm8kAk4jLtNdmDFZCCStM1EGIelNU pjjtL4FSGAVwgisQucoSj4vLVpkgLqzZdzX9wYytiy3sfYNB3aYKj8S/Nug5gaKAa8gA /5oDb+KCNmSa3bC/jNyCQmeij+quPN3cyVSSr1ovXCPVJxrqYAAydTsm3Q4bRt6yNhxm 6FVSyKTuVZN23UWZkdV6Uc7djgZpboRwjQPImKvKrchYImM3eB6pVBMtJk1jJ8Nfk6ob kIoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=S2CmnEPxXvCEcbzHhTRN23lLhCg76X8qXN/3Y5F0RKYRxbb26uWqEiQ0/6xSXWkbeX Q+qVYzMdFUhC9Pfhvlcmzgb0UJwpHAoKIRZNoUIN9r4Ty0h1opF5t8dvy7cvu8hCYlwm 9KTCf9yVPM5Dg1KVlMqWA/2tMfXHA084nZIvteREvq+lMgO4CSBZtCKX8YBlN6ok1fUn sbUJ3Mg8O3doTnwNdREUzk59zFTKbrMnrGw751xBKI0sCe3xZFnFrwPNxPdcvraqPuga UkvYGW1jYU+OvanHIgQNKlD1n5RQxVdejY2/s5aemKrFFMwvGBb58Hs87UGDWr2HwdPZ LODQ== X-Gm-Message-State: ACrzQf3d+6hIjF2Wa/oSzO6vOL6qal+5RcvvOY9ER3mw2a0TgWbOcDn7 26NDpuxPrzEZmfN43FuT5hM= X-Google-Smtp-Source: AMsMyM6Ssf7qu6Zt1cmqzeY+6ksfJF4jyx1t2euaZBhwzvq5JPVY+8zsaODdG48vWXQYQDWTdtSyvw== X-Received: by 2002:a17:90b:4a04:b0:213:587b:204e with SMTP id kk4-20020a17090b4a0400b00213587b204emr55088612pjb.98.1667886235549; Mon, 07 Nov 2022 21:43:55 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id k21-20020a628415000000b0056bb06ce1cfsm5614872pfd.97.2022.11.07.21.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 21:43:55 -0800 (PST) Date: Mon, 7 Nov 2022 21:43:54 -0800 From: Isaku Yamahata To: "Huang, Kai" Subject: Re: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "mjrosato@linux.ibm.com" , "david@redhat.com" , "anup@brainfault.org" , "paul.walmsley@sifive.com" , "linux-kernel@vger.kernel.org" , "linux-riscv@lists.infradead.org" , "imbrenda@linux.ibm.com" , "kvmarm@lists.cs.columbia.edu" , "isaku.yamahata@gmail.com" , "linux-s390@vger.kernel.org" , "kvm@vger.kernel.org" , "chenhuacai@kernel.org" , "aleksandar.qemu.devel@gmail.com" , "james.morse@arm.com" , "borntraeger@linux.ibm.com" , "Gao, Chao" , "farman@linux.ibm.com" , "aou@eecs.berkeley.edu" , "suzuki.poulose@arm.com" , "Yao, Yuan" , "kvmarm@lists.linux.dev" , "tglx@linutronix.de" , "alexandru.elisei@arm.com" , "linux-arm-kernel@lists.infradead.org" , "frankja@linux.ibm.com" , "Yamahata, Isaku" , "atishp@atishpatra.org" , "farosas@linux.ibm.com" , "Christopherson,, Sean" , "linux-mips@vger.kernel.org" , "oliver.upton@linux.dev" , "palmer@dabbelt.com" , "kvm-riscv@lists.infradead.org" , "maz@kernel.org" , "pbonzini@redhat.com" , "vkuznets@redhat.com" , "linuxppc-dev@lists.ozlabs.org" Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue, Nov 08, 2022 at 01:09:27AM +0000, "Huang, Kai" wrote: > On Mon, 2022-11-07 at 13:46 -0800, Isaku Yamahata wrote: > > > On Fri, Nov 04, 2022, Isaku Yamahata wrote: > > > > Thanks for the patch series. I the rebased TDX KVM patch series and it > > > > worked. > > > > Since cpu offline needs to be rejected in some cases(To keep at least one > > > > cpu > > > > on a package), arch hook for cpu offline is needed. > > > > > > I hate to bring this up because I doubt there's a real use case for SUSPEND > > > with > > > TDX, but the CPU offline path isn't just for true offlining of CPUs.  When > > > the > > > system enters SUSPEND, only the initiating CPU goes through > > > kvm_suspend()+kvm_resume(), > > > all responding CPUs go through CPU offline+online.  I.e. disallowing all > > > CPUs from > > > going "offline" will prevent suspending the system. > > > > The current TDX KVM implementation disallows CPU package from offline only > > when > > TDs are running.  If no TD is running, CPU offline is allowed.  So before > > SUSPEND, TDs need to be killed via systemd or something.  After killing TDs, > > the > > system can enter into SUSPEND state. > > This seems not correct. You need one cpu for each to be online in order to > create TD as well, as TDH.MNG.KEY.CONFIG needs to be called on all packages, > correct? That's correct. In such case, the creation of TD fails. TD creation checks if at least one cpu is online on all CPU packages. If no, error. -- Isaku Yamahata From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CE6DCC433FE for ; Tue, 8 Nov 2022 05:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7H7r4ppZDHNNyKpGp9Y5mp9i4/B3HS0wruOeDXVa5fg=; b=2KryoKhHEouI60 9Cd0PdE7OC0UjEaPG6PrAUgtlzTqfM47t0vU9yBRX5tr+sMJk6xCnGNDSbJWEpnLBrp5+PMJU/jKN FbWIpc6+uBS2SyGj9nKRzygoWGu0NelRdTFc2ZmHwW43RBeOZdtnQ21v8bSmxfsiot0BLooQLiQ4R xmz0kRS+TG/QPTu6wpQPvL8B+JTRemVt3gEEpsmLA8ypEKanTOe+g+XVOU+KvAwqZ8vRnNWsD3M3o LuwTqGb36hGfFyOvWEVadyjXVqB699IVB2YNKZomRhbE+YuF54Cnlt/k+VA+EfHzd08h1BVSHdxVK DhwjfxQ2dBajPWg793xA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1osHP2-002vWx-Sy; Tue, 08 Nov 2022 05:44:05 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1osHOy-002vUM-Bx; Tue, 08 Nov 2022 05:44:01 +0000 Received: by mail-pl1-x62c.google.com with SMTP id b21so13229100plc.9; Mon, 07 Nov 2022 21:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=XNdeC13pb4LjOBaRWcKMBNqFRJ+RgsbfVoNw18z5L+vfkg1SZceZjUxbLhiIy1qySy Y4hI02da8j9ev8qt2sUsGeUiOSLlDAUgz5JUplPGm8kAk4jLtNdmDFZCCStM1EGIelNU pjjtL4FSGAVwgisQucoSj4vLVpkgLqzZdzX9wYytiy3sfYNB3aYKj8S/Nug5gaKAa8gA /5oDb+KCNmSa3bC/jNyCQmeij+quPN3cyVSSr1ovXCPVJxrqYAAydTsm3Q4bRt6yNhxm 6FVSyKTuVZN23UWZkdV6Uc7djgZpboRwjQPImKvKrchYImM3eB6pVBMtJk1jJ8Nfk6ob kIoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3UicMHRBXNKZ6klzBYtNlGfdpT3ipHtBcp9O7LuoFbE=; b=JaLkEfL7XnTBc0JIT82t+Mg7eW2Sy3flENaVZyynXYOt97Y6mcYbBMKdkCOzJdZwzs PRs9gBkvSyq/eTPeDn2LdselQ/pmEMIIsMZqMH3zE0nn6e5DG9h317g/3P9jL+9g+h98 sCc6qG0dvSQR0wJCK/+6TjnIM1SgIWKEcdWM5/uMfVHfStHpeT5v+VKV3YyjjJ9hQoDA 4Eucml+JFpqnMlFU4ggup/4feZOpBY3bTu2lFPKJQtnsa/2h/0ryTGh7/23Jm+6HTz7E 5+bwIwOFGZsJ6oy5eQ/FRkxQETh7Iz7NeUnkcxuu8mSE/tL8tu/EZW5GKhzwTCX/kBYG D/JQ== X-Gm-Message-State: ACrzQf0EkP4utXkqow4lFjXHJm84jAbgAk6PPAj0FagUMQkbBVrDMHwb GYEDRkjf7OpcJLH/iRmtTmk= X-Google-Smtp-Source: AMsMyM6Ssf7qu6Zt1cmqzeY+6ksfJF4jyx1t2euaZBhwzvq5JPVY+8zsaODdG48vWXQYQDWTdtSyvw== X-Received: by 2002:a17:90b:4a04:b0:213:587b:204e with SMTP id kk4-20020a17090b4a0400b00213587b204emr55088612pjb.98.1667886235549; Mon, 07 Nov 2022 21:43:55 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id k21-20020a628415000000b0056bb06ce1cfsm5614872pfd.97.2022.11.07.21.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 21:43:55 -0800 (PST) Date: Mon, 7 Nov 2022 21:43:54 -0800 From: Isaku Yamahata To: "Huang, Kai" Cc: "Christopherson,, Sean" , "isaku.yamahata@gmail.com" , "borntraeger@linux.ibm.com" , "kvm-riscv@lists.infradead.org" , "tglx@linutronix.de" , "Yao, Yuan" , "kvm@vger.kernel.org" , "Yamahata, Isaku" , "suzuki.poulose@arm.com" , "pbonzini@redhat.com" , "david@redhat.com" , "linux-mips@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , "linux-riscv@lists.infradead.org" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "mjrosato@linux.ibm.com" , "oliver.upton@linux.dev" , "farosas@linux.ibm.com" , "palmer@dabbelt.com" , "linux-s390@vger.kernel.org" , "chenhuacai@kernel.org" , "aou@eecs.berkeley.edu" , "alexandru.elisei@arm.com" , "mpe@ellerman.id.au" , "vkuznets@redhat.com" , "maz@kernel.org" , "anup@brainfault.org" , "frankja@linux.ibm.com" , "farman@linux.ibm.com" , "james.morse@arm.com" , "kvmarm@lists.cs.columbia.edu" , "aleksandar.qemu.devel@gmail.com" , "paul.walmsley@sifive.com" , "linux-arm-kernel@lists.infradead.org" , "atishp@atishpatra.org" , "imbrenda@linux.ibm.com" , "Gao, Chao" Subject: Re: [PATCH 00/44] KVM: Rework kvm_init() and hardware enabling Message-ID: <20221108054354.GA1708572@ls.amr.corp.intel.com> References: <20221102231911.3107438-1-seanjc@google.com> <20221104071749.GC1063309@ls.amr.corp.intel.com> <20221107214634.GE1063309@ls.amr.corp.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221107_214400_449613_57CA35A4 X-CRM114-Status: GOOD ( 21.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVHVlLCBOb3YgMDgsIDIwMjIgYXQgMDE6MDk6MjdBTSArMDAwMCwKIkh1YW5nLCBLYWkiIDxr YWkuaHVhbmdAaW50ZWwuY29tPiB3cm90ZToKCj4gT24gTW9uLCAyMDIyLTExLTA3IGF0IDEzOjQ2 IC0wODAwLCBJc2FrdSBZYW1haGF0YSB3cm90ZToKPiA+ID4gT24gRnJpLCBOb3YgMDQsIDIwMjIs IElzYWt1IFlhbWFoYXRhIHdyb3RlOgo+ID4gPiA+IFRoYW5rcyBmb3IgdGhlIHBhdGNoIHNlcmll cy4gSSB0aGUgcmViYXNlZCBURFggS1ZNIHBhdGNoIHNlcmllcyBhbmQgaXQKPiA+ID4gPiB3b3Jr ZWQuCj4gPiA+ID4gU2luY2UgY3B1IG9mZmxpbmUgbmVlZHMgdG8gYmUgcmVqZWN0ZWQgaW4gc29t ZSBjYXNlcyhUbyBrZWVwIGF0IGxlYXN0IG9uZQo+ID4gPiA+IGNwdQo+ID4gPiA+IG9uIGEgcGFj a2FnZSksIGFyY2ggaG9vayBmb3IgY3B1IG9mZmxpbmUgaXMgbmVlZGVkLgo+ID4gPiAKPiA+ID4g SSBoYXRlIHRvIGJyaW5nIHRoaXMgdXAgYmVjYXVzZSBJIGRvdWJ0IHRoZXJlJ3MgYSByZWFsIHVz ZSBjYXNlIGZvciBTVVNQRU5ECj4gPiA+IHdpdGgKPiA+ID4gVERYLCBidXQgdGhlIENQVSBvZmZs aW5lIHBhdGggaXNuJ3QganVzdCBmb3IgdHJ1ZSBvZmZsaW5pbmcgb2YgQ1BVcy7CoCBXaGVuCj4g PiA+IHRoZQo+ID4gPiBzeXN0ZW0gZW50ZXJzIFNVU1BFTkQsIG9ubHkgdGhlIGluaXRpYXRpbmcg Q1BVIGdvZXMgdGhyb3VnaAo+ID4gPiBrdm1fc3VzcGVuZCgpK2t2bV9yZXN1bWUoKSwKPiA+ID4g YWxsIHJlc3BvbmRpbmcgQ1BVcyBnbyB0aHJvdWdoIENQVSBvZmZsaW5lK29ubGluZS7CoCBJLmUu IGRpc2FsbG93aW5nIGFsbAo+ID4gPiBDUFVzIGZyb20KPiA+ID4gZ29pbmcgIm9mZmxpbmUiIHdp bGwgcHJldmVudCBzdXNwZW5kaW5nIHRoZSBzeXN0ZW0uCj4gPiAKPiA+IFRoZSBjdXJyZW50IFRE WCBLVk0gaW1wbGVtZW50YXRpb24gZGlzYWxsb3dzIENQVSBwYWNrYWdlIGZyb20gb2ZmbGluZSBv bmx5Cj4gPiB3aGVuCj4gPiBURHMgYXJlIHJ1bm5pbmcuwqAgSWYgbm8gVEQgaXMgcnVubmluZywg Q1BVIG9mZmxpbmUgaXMgYWxsb3dlZC7CoCBTbyBiZWZvcmUKPiA+IFNVU1BFTkQsIFREcyBuZWVk IHRvIGJlIGtpbGxlZCB2aWEgc3lzdGVtZCBvciBzb21ldGhpbmcuwqAgQWZ0ZXIga2lsbGluZyBU RHMsCj4gPiB0aGUKPiA+IHN5c3RlbSBjYW4gZW50ZXIgaW50byBTVVNQRU5EIHN0YXRlLgo+IAo+ IFRoaXMgc2VlbXMgbm90IGNvcnJlY3QuICBZb3UgbmVlZCBvbmUgY3B1IGZvciBlYWNoIHRvIGJl IG9ubGluZSBpbiBvcmRlciB0bwo+IGNyZWF0ZSBURCBhcyB3ZWxsLCBhcyBUREguTU5HLktFWS5D T05GSUcgbmVlZHMgdG8gYmUgY2FsbGVkIG9uIGFsbCBwYWNrYWdlcywKPiBjb3JyZWN0PwoKVGhh dCdzIGNvcnJlY3QuIEluIHN1Y2ggY2FzZSwgdGhlIGNyZWF0aW9uIG9mIFREIGZhaWxzLiAgVEQg Y3JlYXRpb24gY2hlY2tzIGlmCmF0IGxlYXN0IG9uZSBjcHUgaXMgb25saW5lIG9uIGFsbCBDUFUg cGFja2FnZXMuICBJZiBubywgZXJyb3IuCi0tIApJc2FrdSBZYW1haGF0YSA8aXNha3UueWFtYWhh dGFAZ21haWwuY29tPgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0 cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGlu Zm8vbGludXgtYXJtLWtlcm5lbAo=