From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8C8CCE8D40 for ; Fri, 14 Nov 2025 15:02:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=udhFmhZtlG3TBzZAohN4eP9yg2+7Zj3K5xQejSUTbLs=; b=NX/mVpVIyxHOtUMw9paYO9bBkn JHnPOW9Hft9+7NtjN04nes76HeO5VzXmnyYqRBbDb9HV92NH3y0Me6xaZ45dHtNzeEcDgTTzzaPNF yqKCEukSsgTKYgIjSxvikco2PzqyjcaqcWz6JqHAyvYFSreqk3iKHVcZBeWtsS+JScShuyf8ujEg1 9UqNdcp1t64p1orkJYjCm5ejMX2UHVYaMbhQvKbXv9rGDX9CPHqsm5n/aRXsb6jWMHRVSXySm/C9I Z13+rp/HeORKIa8cxMa9P5YUj/6UUYV+NaDYyZ4IO6fs3xWnsNJT99R3EA1KRvnAgOQ50xkWFtBWR 9FhiJWAA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vJvJz-0000000CPjy-2QDX; Fri, 14 Nov 2025 15:02:43 +0000 Received: from sea.source.kernel.org ([172.234.252.31]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vJvJs-0000000CPhK-0iff for linux-arm-kernel@lists.infradead.org; Fri, 14 Nov 2025 15:02:39 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id 83712443B5; Fri, 14 Nov 2025 15:02:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 418F1C2BCAF; Fri, 14 Nov 2025 15:02:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763132555; bh=bnQzitv+bcx/vm2e9QneP7g5jpvlqREWphFL7TxpTU4=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Ky5GRlAlhAqlckUwc8+x2G2tZ/UWYE7+cFntEDUbaAf+Bh34DB1pF6XBVehYerHOm +pSVYMfoGGXif7VC+8vQ8H9g/vVsaIqPLg/X0ZsnI0O8jm06s49mTU+amHx3K3Dl7y 7NFv/ytgc/hdoA7qID1VOnBm9YHOIpgLCffgB4tQxWo4df383whExqrfr50XqLaJcU ZhOdmpWDtJHsOlfro8GuvbRgT2BPZ33/kzRyvXe0wcondMXxveg9OKFlcrLAz4oRic SZXIvXDU9gVpWzKE6X8WJv0+3VYa3lRxvu3Xz7WKMMRmdO4pKDMlXolASOoLNYrftB Ey2NK6nWwmXSA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vJvJo-00000005F1H-31IK; Fri, 14 Nov 2025 15:02:32 +0000 Date: Fri, 14 Nov 2025 15:02:32 +0000 Message-ID: <86cy5ku06v.wl-maz@kernel.org> From: Marc Zyngier To: Fuad Tabba Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Christoffer Dall , Volodymyr Babchuk , Yao Yuan Subject: Re: [PATCH v2 29/45] KVM: arm64: GICv3: Set ICH_HCR_EL2.TDIR when interrupts overflow LR capacity In-Reply-To: References: <20251109171619.1507205-1-maz@kernel.org> <20251109171619.1507205-30-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tabba@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oupton@kernel.org, yuzenghui@huawei.com, christoffer.dall@arm.com, Volodymyr_Babchuk@epam.com, yaoyuan@linux.alibaba.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20251114_070236_245690_8BDAF42D X-CRM114-Status: GOOD ( 33.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 14 Nov 2025 14:20:46 +0000, Fuad Tabba wrote: > > Hi Marc, > > On Sun, 9 Nov 2025 at 17:17, Marc Zyngier wrote: > > > > Now that we are ready to handle deactivation through ICV_DIR_EL1, > > set the trap bit if we have active interrupts outside of the LRs. > > > > Signed-off-by: Marc Zyngier > > --- > > arch/arm64/kvm/vgic/vgic-v3.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c > > index 1026031f22ff9..26e17ed057f00 100644 > > --- a/arch/arm64/kvm/vgic/vgic-v3.c > > +++ b/arch/arm64/kvm/vgic/vgic-v3.c > > @@ -42,6 +42,13 @@ void vgic_v3_configure_hcr(struct kvm_vcpu *vcpu, > > ICH_HCR_EL2_VGrp0DIE : ICH_HCR_EL2_VGrp0EIE; > > cpuif->vgic_hcr |= (cpuif->vgic_vmcr & ICH_VMCR_ENG1_MASK) ? > > ICH_HCR_EL2_VGrp1DIE : ICH_HCR_EL2_VGrp1EIE; > > + > > + /* > > + * Note that we set the trap irrespective of EOIMode, as that > > + * can change behind our back without any warning... > > + */ > > + if (irqs_active_outside_lrs(als)) > > + cpuif->vgic_hcr |= ICH_HCR_EL2_TDIR; > > } > > I just tested these patches as they are on kvmarm/next > 2ea7215187c5759fc5d277280e3095b350ca6a50 ("Merge branch > 'kvm-arm64/vgic-lr-overflow' into kvmarm/next"), without any > additional pKVM patches. I tried running it with pKVM (non-protected) > and with just plain nVHE. In both cases, I get a trap to EL2 (0x18) > when booting a non-protected guest, which triggers a bug in > handle_trap() arch/arm64/kvm/hyp/nvhe/hyp-main.c:706 > > This trap is happening because of setting this particular trap (TDIR). > Just removing this trap from vgic_v3_configure_hcr() from the ToT on > kvmarm/next boots fine. This is surprising, as I'm not hitting this on actual HW. Are you getting a 0x18 trap? If so, is it coming from the host? Can you correlate the PC with what the host is doing? It would indicate that we are leaking trap bits on exit, and that QEMU is trapping ICC_DIR_EL1 on top of ICV_DIR_EL1 (which the HW I have access to doesn't seem to do). > I'm running this on QEMU with '-machine virt,gic-version=3 -cpu max' > and the kernel with 'kvm-arm.mode=protected' and with > 'kvm-arm.mode=nvhe'. > > Let me know if you need any more info or help testing. On top of the above, could you give the hack below a go? I haven't tested it at all (I'm in the middle of a bisect from hell...) Thanks, M. diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c index e950efa22547..71199e1a9294 100644 --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c @@ -243,7 +243,7 @@ void __vgic_v3_save_state(struct vgic_v3_cpu_if *cpu_if) cpu_if->vgic_hcr |= val & ICH_HCR_EL2_EOIcount; } - write_gicreg(compute_ich_hcr(cpu_if) & ~ICH_HCR_EL2_En, ICH_HCR_EL2); + write_gicreg(0, ICH_HCR_EL2); } void __vgic_v3_restore_state(struct vgic_v3_cpu_if *cpu_if) -- Without deviation from the norm, progress is not possible.