From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDF4C3148AB; Fri, 14 Nov 2025 15:02:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132556; cv=none; b=BvH4yc7tiCofOPIkkm/QG4Q3Bs604T6rGzqKGnF6l/7aaEbgTflx+0HXjN9bgHERqT/ZAm9HMgMQfP/NK14Z9w8Jsr0ZO4a2yZkbkGhDi/hIGQJb0elCZ0U74cdtnRF1+I+ovRapDHZTP36+o9+qe1aDqu25XIA3erP5BiuGeyw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132556; c=relaxed/simple; bh=bnQzitv+bcx/vm2e9QneP7g5jpvlqREWphFL7TxpTU4=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=noyDsIPkxMRgNPNi2IgYJvQfT7IRFR313XR/OIOQQACQ94zcTWhntHpuD0hqmV9TL1cro84Kzn8Jx35UNmJL+MIoQnJP8QxAbRWTeQ7Q1N0uNY0jIDuaBeqJeyAvnRwyfIneQ5mQVu0LE7oSCz1/5ACCY1UykKHHFYqk2roEqsI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ky5GRlAl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ky5GRlAl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 418F1C2BCAF; Fri, 14 Nov 2025 15:02:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763132555; bh=bnQzitv+bcx/vm2e9QneP7g5jpvlqREWphFL7TxpTU4=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Ky5GRlAlhAqlckUwc8+x2G2tZ/UWYE7+cFntEDUbaAf+Bh34DB1pF6XBVehYerHOm +pSVYMfoGGXif7VC+8vQ8H9g/vVsaIqPLg/X0ZsnI0O8jm06s49mTU+amHx3K3Dl7y 7NFv/ytgc/hdoA7qID1VOnBm9YHOIpgLCffgB4tQxWo4df383whExqrfr50XqLaJcU ZhOdmpWDtJHsOlfro8GuvbRgT2BPZ33/kzRyvXe0wcondMXxveg9OKFlcrLAz4oRic SZXIvXDU9gVpWzKE6X8WJv0+3VYa3lRxvu3Xz7WKMMRmdO4pKDMlXolASOoLNYrftB Ey2NK6nWwmXSA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vJvJo-00000005F1H-31IK; Fri, 14 Nov 2025 15:02:32 +0000 Date: Fri, 14 Nov 2025 15:02:32 +0000 Message-ID: <86cy5ku06v.wl-maz@kernel.org> From: Marc Zyngier To: Fuad Tabba Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Christoffer Dall , Volodymyr Babchuk , Yao Yuan Subject: Re: [PATCH v2 29/45] KVM: arm64: GICv3: Set ICH_HCR_EL2.TDIR when interrupts overflow LR capacity In-Reply-To: References: <20251109171619.1507205-1-maz@kernel.org> <20251109171619.1507205-30-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tabba@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oupton@kernel.org, yuzenghui@huawei.com, christoffer.dall@arm.com, Volodymyr_Babchuk@epam.com, yaoyuan@linux.alibaba.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Fri, 14 Nov 2025 14:20:46 +0000, Fuad Tabba wrote: > > Hi Marc, > > On Sun, 9 Nov 2025 at 17:17, Marc Zyngier wrote: > > > > Now that we are ready to handle deactivation through ICV_DIR_EL1, > > set the trap bit if we have active interrupts outside of the LRs. > > > > Signed-off-by: Marc Zyngier > > --- > > arch/arm64/kvm/vgic/vgic-v3.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c > > index 1026031f22ff9..26e17ed057f00 100644 > > --- a/arch/arm64/kvm/vgic/vgic-v3.c > > +++ b/arch/arm64/kvm/vgic/vgic-v3.c > > @@ -42,6 +42,13 @@ void vgic_v3_configure_hcr(struct kvm_vcpu *vcpu, > > ICH_HCR_EL2_VGrp0DIE : ICH_HCR_EL2_VGrp0EIE; > > cpuif->vgic_hcr |= (cpuif->vgic_vmcr & ICH_VMCR_ENG1_MASK) ? > > ICH_HCR_EL2_VGrp1DIE : ICH_HCR_EL2_VGrp1EIE; > > + > > + /* > > + * Note that we set the trap irrespective of EOIMode, as that > > + * can change behind our back without any warning... > > + */ > > + if (irqs_active_outside_lrs(als)) > > + cpuif->vgic_hcr |= ICH_HCR_EL2_TDIR; > > } > > I just tested these patches as they are on kvmarm/next > 2ea7215187c5759fc5d277280e3095b350ca6a50 ("Merge branch > 'kvm-arm64/vgic-lr-overflow' into kvmarm/next"), without any > additional pKVM patches. I tried running it with pKVM (non-protected) > and with just plain nVHE. In both cases, I get a trap to EL2 (0x18) > when booting a non-protected guest, which triggers a bug in > handle_trap() arch/arm64/kvm/hyp/nvhe/hyp-main.c:706 > > This trap is happening because of setting this particular trap (TDIR). > Just removing this trap from vgic_v3_configure_hcr() from the ToT on > kvmarm/next boots fine. This is surprising, as I'm not hitting this on actual HW. Are you getting a 0x18 trap? If so, is it coming from the host? Can you correlate the PC with what the host is doing? It would indicate that we are leaking trap bits on exit, and that QEMU is trapping ICC_DIR_EL1 on top of ICV_DIR_EL1 (which the HW I have access to doesn't seem to do). > I'm running this on QEMU with '-machine virt,gic-version=3 -cpu max' > and the kernel with 'kvm-arm.mode=protected' and with > 'kvm-arm.mode=nvhe'. > > Let me know if you need any more info or help testing. On top of the above, could you give the hack below a go? I haven't tested it at all (I'm in the middle of a bisect from hell...) Thanks, M. diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c index e950efa22547..71199e1a9294 100644 --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c @@ -243,7 +243,7 @@ void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if) cpu_if->vgic_hcr |= val & ICH_HCR_EL2_EOIcount; } - write_gicreg(compute_ich_hcr(cpu_if) & ~ICH_HCR_EL2_En, ICH_HCR_EL2); + write_gicreg(0, ICH_HCR_EL2); } void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if) -- Without deviation from the norm, progress is not possible.