From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19AEB241667; Sat, 26 Jul 2025 10:34:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753526070; cv=none; b=SrXWKdd/hTjjvt6RMUqbOTBjvO1cQOReGcA/QWrIjjJ+0QF/5yrfHzFI6RecgE58e7KlN7i1suo94ATUw8Gowj9Ru2Mkt1Leh4PxmUNWHYrYyaFoR9pmlW+OhN4+h/KtiH9b/quYH1zHI/LC1bA442wFBxGD8/WGIuVqIMm6m80= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753526070; c=relaxed/simple; bh=FYJxXIsnGPguGntCrMrA8ZIl5Irvi5pt8i95/ZZPs3A=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=LfkkNajqEYd8t3fsQ1dXP5/hnFvy+BfkWL/SJF0CDOM6+EgOdvyD8Sh5ozOt0SwPz9bDIfwChz2w1+j/AFSu4TC1hzirveblYHrkFqOFOU0bxlTKaSIDB5voqs5QUemXOehz1D40JpKS3frHjt/hObGtU+278gY2KSntQUGQ/rQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=o6ueX9EK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="o6ueX9EK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89E13C4CEED; Sat, 26 Jul 2025 10:34:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753526069; bh=FYJxXIsnGPguGntCrMrA8ZIl5Irvi5pt8i95/ZZPs3A=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=o6ueX9EKPj4OmAp9prP4eOor1kcfzNTxNa6KCVmDf0wsD+rHkvdUvcnDu+D6uFG8t OT32rGJn+KqPyrrmhZU82EfKAo7fasSuaZ4/+Q4VUlL7CIV2uULjJTzAuZwYR0AzuQ I1i1U2Yv/TxWwK/l8LIgSltYzf5WSVH9tb4ibldmGAa9c0NZOplj8uV/oTMgSmOax4 TrA1e0kK0wNkY2hjzhE71FKolHlTp0oDWMxaU5HNITr6ceegbkMoJz8zsUx3oOYYhv ryjYuh3szHf3LIptHEYTJ4QN7pVV+T7tqbBv3n5TOTe0PFptMDHISqmI1Wa+QXhsnH F+IukygPkMTcA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1ufcEV-001adU-E7; Sat, 26 Jul 2025 11:34:27 +0100 Date: Sat, 26 Jul 2025 11:34:26 +0100 Message-ID: <868qkb8clp.wl-maz@kernel.org> From: Marc Zyngier To: Wei-Lin Chang Cc: Andre Przywara , Will Deacon , Julien Thierry , kvm@vger.kernel.org, kvmarm@lists.linux.dev, Alexandru Elisei Subject: Re: [PATCH kvmtool v2 5/6] arm64: add FEAT_E2H0 support (TBC) In-Reply-To: References: <20250725144100.2944226-1-andre.przywara@arm.com> <20250725144100.2944226-6-andre.przywara@arm.com> <86cy9o8bwn.wl-maz@kernel.org> <87jz3vtils.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: r09922117@csie.ntu.edu.tw, andre.przywara@arm.com, will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev, alexandru.elisei@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Sat, 26 Jul 2025 11:11:43 +0100, Wei-Lin Chang wrote: > > On Sat, Jul 26, 2025 at 10:19:11AM +0100, Marc Zyngier wrote: > > On Sat, 26 Jul 2025 10:01:25 +0100, > > Wei-Lin Chang wrote: > > > > > > Hi all, > > > > > > On Fri, Jul 25, 2025 at 05:37:12PM +0100, Marc Zyngier wrote: > > > > Hi Andre, > > > > > > > > Thanks for picking this. A few nits below. > > > > > > > > On Fri, 25 Jul 2025 15:40:59 +0100, > > > > Andre Przywara wrote: > > > > > > > > > > From: Marc Zyngier > > > > > > > > > > To reduce code complexity, KVM only supports nested virtualisation in > > > > > VHE mode. So to allow recursive nested virtualisation, and be able to > > > > > expose FEAT_NV2 to a guest, we must prevent a guest from turning off > > > > > HCR_EL2.E2H, which is covered by not advertising the FEAT_E2H0 architecture > > > > > feature. > > > > > > > > > > To allow people to run a guest in non-VHE mode, KVM introduced the > > > > > KVM_ARM_VCPU_HAS_EL2_E2H0 feature flag, which will allow control over > > > > > HCR_EL2.E2H, but at the cost of turning off FEAT_NV2. > > > > > > > > All of that has been captured at length in the kernel code, and I > > > > think this is "too much information" for userspace. I'd rather we > > > > stick to a pure description of what the various options mean to the > > > > user. > > > > > > > > > Add a kvmtool command line option "--e2h0" to set that feature bit when > > > > > creating a guest, to gain non-VHE, but lose recursive nested virt. > > > > > > > > How about: > > > > > > > > "The --nested option allows a guest to boot at EL2 without FEAT_E2H0 > > > > (i.e. mandating VHE support). While this is great for "modern" > > > > operating systems and hypervisors, a few legacy guests are stuck in a > > > > distant past. > > > > > > > > To support those, the --e2h0 option exposes FEAT_E2H0 to the guest, > > > > at the expense of a number of other features, such as FEAT_NV2. This > > > > > > Just a very small thing: > > > > > > Will only mentioning FEAT_NV2 here lead people to think that FEAT_NV is > > > still available with --e2h0? > > > Maybe s/FEAT_NV2/FEAT_NV/ makes it clearer? > > > > Maybe. On the other hand, we never advertise the old FEAT_NV as such, > > irrespective of the state of E2H. This is indicated by > > ID_AA64MMFR4_EL1.NV_frac==0b0001 when NV is advertised. So I'm not > > sure this changes anything, really. > > Right, thanks for the input. Even if the user doesn't know what's up the L1 > hypervisor will when it checks ID_AA64MMFR4_EL1 :) Exactly. The idea behind this relaxation to the architecture was that SW that is relying on FEAT_NV being advertised through ID_AA64MMFR2_EL1 would find that the "old style NV" isn't supported, while SW that has followed the architecture would also look at NV_frac, and find that the support that matters actually exists. It means that we potentially leave behind some hypervisors that haven't caught up with NV_frac yet, but since we don't know of any other NV-capable hypervisor that could run under KVM, that's not really a problem! ;-) Thanks, M. -- Without deviation from the norm, progress is not possible.