From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B4B7CE8D6B for ; Mon, 17 Nov 2025 11:34:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pZ65++Hz38XdQioXm8+T8DOmbWL16djPgjyuQhqKZ4U=; b=XIVITySA0BgBBdX7GEGxHXu4VH GdAm6z9r2Uca1pBqNSbksGDzfjHTYCGov53ZtNKF5t6jKtAniukLaYWnqQUJP0xYPc832aksmwVnz BaR3YV6chbIKpA5DaQueZB4NPFcSL3N6/06HGS1uyaYs+1d3orwHE5WXX4bhzCUqsihFwrGyotuJO 7IT/GEVgKgbwCMszZvhcNgXfM/FIHZyWpxRi74fI6DxTvpo54mJm5wkiFtpY4LTM4ZSNuY3eyEbL4 P7pkGol0uOWkxsIIf2N4+F1fzOWXSlpmYB6bDFaCiVa08nremC+zmy4Db4EDGrpsisP0WPqxtGUy+ oXErk0MQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vKxUk-0000000Fyx6-35Sx; Mon, 17 Nov 2025 11:34:06 +0000 Received: from tor.source.kernel.org ([2600:3c04:e001:324:0:1991:8:25]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vKxUj-0000000Fywh-1L6f for linux-arm-kernel@lists.infradead.org; Mon, 17 Nov 2025 11:34:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id D87E560143; Mon, 17 Nov 2025 11:34:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 813E1C4CEF1; Mon, 17 Nov 2025 11:34:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763379243; bh=G2XSX5R44t4v3J1yTweOP2/+Y1hZ5/nw0s9qbif0Qfc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PFMfQHpJkJGwZcolZSlLrjAkA9oxFJCSCweqjn4iO/bO6kIT9NR0HLEidaLBpPS9j zR/cT6zCVA/WpBEk59z+2xr1TTonTJ54la/5bMHDqygs+mAZA9FARd0qYyPomI7QJE nqWewcacHQN1HF8kw9E/91Pg2ioGxBAOvSDKDo5zRiQCEJ2G/9pycR060yCwP24q2A x1d26vScEaVVSfK5zVpRsDkKyTry//DlTTxWFh2aOUQczKRY9uFveV9Ffhx25pkldA +4hr/NsLH3NHG1EWkNygRupWhXebdzj8/4D+X1oJQyuwMABbGBsaIrpM6kyOxRDTQU QBYcw59HJMGwQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vKxUf-00000005nnE-2NPR; Mon, 17 Nov 2025 11:34:01 +0000 Date: Mon, 17 Nov 2025 11:34:01 +0000 Message-ID: <864iqsuc46.wl-maz@kernel.org> From: Marc Zyngier To: Fuad Tabba Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Christoffer Dall , Mark Brown Subject: Re: [PATCH v3 3/5] KVM: arm64: GICv3: nv: Resync LRs/VMCR/HCR early for better MI emulation In-Reply-To: References: <20251117091527.1119213-1-maz@kernel.org> <20251117091527.1119213-4-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tabba@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oupton@kernel.org, yuzenghui@huawei.com, christoffer.dall@arm.com, broonie@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 17 Nov 2025 11:24:24 +0000, Fuad Tabba wrote: > > Hi Marc, > > > On Mon, 17 Nov 2025 at 09:15, Marc Zyngier wrote: > > > > The current approach to nested GICv3 support is to not do anything > > while L2 is running, wait a transition from L2 to L1 to resync > > LRs, VMCR and HCR, and only then evaluate the state to decide > > whether to generate a maintenance interrupt. > > > > This doesn't provide a good quality of emulation, and it would be > > far preferable to find out early that we need to perform a switch. > > > > Move the LRs/VMCR and HCR resync into vgic_v3_sync_nested(), so > > that we have most of the state available. As we turning the vgic > > off at this stage to avoid a screaming host MI, add a new helper > > vgic_v3_flush_nested() that switches the vgic on again. The MI can > > then be directly injected as required. > > > > Signed-off-by: Marc Zyngier > > --- > > arch/arm64/include/asm/kvm_hyp.h | 1 + > > arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +- > > arch/arm64/kvm/vgic/vgic-v3-nested.c | 69 ++++++++++++++++------------ > > arch/arm64/kvm/vgic/vgic.c | 6 ++- > > arch/arm64/kvm/vgic/vgic.h | 1 + > > 5 files changed, 46 insertions(+), 33 deletions(-) > > > > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h > > index dbf16a9f67728..76ce2b94bd97e 100644 > > --- a/arch/arm64/include/asm/kvm_hyp.h > > +++ b/arch/arm64/include/asm/kvm_hyp.h > > @@ -77,6 +77,7 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); > > int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu); > > > > u64 __gic_v3_get_lr(unsigned int lr); > > +void __gic_v3_set_lr(u64 val, int lr); > > > > void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if); > > void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if); > > diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c > > index 71199e1a92940..99342c13e1794 100644 > > --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c > > +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c > > @@ -60,7 +60,7 @@ u64 __gic_v3_get_lr(unsigned int lr) > > unreachable(); > > } > > > > -static void __gic_v3_set_lr(u64 val, int lr) > > +void __gic_v3_set_lr(u64 val, int lr) > > { > > switch (lr & 0xf) { > > case 0: > > diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c > > index 17bceef83269e..bf37fd3198ba7 100644 > > --- a/arch/arm64/kvm/vgic/vgic-v3-nested.c > > +++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c > > @@ -70,13 +70,14 @@ static int lr_map_idx_to_shadow_idx(struct shadow_if *shadow_if, int idx) > > * - on L2 put: perform the inverse transformation, so that the result of L2 > > * running becomes visible to L1 in the VNCR-accessible registers. > > * > > - * - there is nothing to do on L2 entry, as everything will have happened > > - * on load. However, this is the point where we detect that an interrupt > > - * targeting L1 and prepare the grand switcheroo. > > + * - there is nothing to do on L2 entry apart from enabling the vgic, as > > + * everything will have happened on load. However, this is the point where > > + * we detect that an interrupt targeting L1 and prepare the grand > > + * switcheroo. > > * > > - * - on L2 exit: emulate the HW bit, and deactivate corresponding the L1 > > - * interrupt. The L0 active state will be cleared by the HW if the L1 > > - * interrupt was itself backed by a HW interrupt. > > + * - on L2 exit: resync the LRs and VMCR, emulate the HW bit, and deactivate > > + * corresponding the L1 interrupt. The L0 active state will be cleared by > > + * the HW if the L1 interrupt was itself backed by a HW interrupt. > > * > > * Maintenance Interrupt (MI) management: > > * > > @@ -265,15 +266,30 @@ static void vgic_v3_create_shadow_lr(struct kvm_vcpu *vcpu, > > s_cpu_if->used_lrs = hweight16(shadow_if->lr_map); > > } > > > > +void vgic_v3_flush_nested(struct kvm_vcpu *vcpu) > > +{ > > + u64 val = __vcpu_sys_reg(vcpu, ICH_HCR_EL2); > > + > > + write_sysreg_s(val | vgic_ich_hcr_trap_bits(), SYS_ICH_HCR_EL2); > > +} > > + > > void vgic_v3_sync_nested(struct kvm_vcpu *vcpu) > > { > > struct shadow_if *shadow_if = get_shadow_if(); > > int i; > > > > for_each_set_bit(i, &shadow_if->lr_map, kvm_vgic_global_state.nr_lr) { > > - u64 lr = __vcpu_sys_reg(vcpu, ICH_LRN(i)); > > + u64 val, host_lr, lr; > > struct vgic_irq *irq; > > > > + host_lr = __gic_v3_get_lr(lr_map_idx_to_shadow_idx(shadow_if, i)); > > + > > + /* Propagate the new LR state */ > > + lr = __vcpu_sys_reg(vcpu, ICH_LRN(i)); > > + val = lr & ~ICH_LR_STATE; > > + val |= host_lr & ICH_LR_STATE; > > + __vcpu_assign_sys_reg(vcpu, ICH_LRN(i), val); > > + > > As I said before, I am outside of my comfort zone here. However, > should the following check be changed to use the merged 'val', rather > than the guest lr as it was? [...] > > > if (!(lr & ICH_LR_HW) || !(lr & ICH_LR_STATE)) > > continue; No, this decision must be taken based on the *original* state, before the L2 guest was run. If the LR was in an invalid state the first place, there is nothing to do. > > > > @@ -286,12 +302,21 @@ void vgic_v3_sync_nested(struct kvm_vcpu *vcpu) > > if (WARN_ON(!irq)) /* Shouldn't happen as we check on load */ > > continue; > > > > - lr = __gic_v3_get_lr(lr_map_idx_to_shadow_idx(shadow_if, i)); > > - if (!(lr & ICH_LR_STATE)) > > + if (!(host_lr & ICH_LR_STATE)) > > irq->active = false; And here, if we see that the *new* state (as fished out of the HW LRs) is now invalid, this means that a deactivation has taken place in L2, and we must propagate it to L1. Thanks, M. -- Without deviation from the norm, progress is not possible.