From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD9C2CE8D6B for ; Mon, 17 Nov 2025 11:57:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Ll1uXMxWvxmz95IRsYiLS+6peKBMZCQ88LeHFHRfFpk=; b=qkuDpmmV5o8K+lDFMcpyixfq+S Vrw7X0CUUOxUQWh/yk3Cx4EwSXPBktQlterUobD92yHBegIgnS5DerAfDpIVK/FWa3vJAbYlfiX1a Y4y9VLo7cZlxyxprplJZtV/WaJSOlG/Z4tFkuFRYNQL63Qc+YZ6ULt3Zm7Tt9i1pE0ObNWB5YTqaY lddx4pG2EussZ7VQGZJUSvXNBpAhtYiGJFdXfyTGN6j+a3NNlEkIarM6iuzXcuVKf6E6zq4XWIepC vflGG/hLZyww0bZ/63DTqfKNNAONNR1ivUcJ7pWaZKIciih4erpQgazjKnYfdg2a5/EHEtLshKugc yERTGk7w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vKxqr-0000000G1Cz-1uCE; Mon, 17 Nov 2025 11:56:57 +0000 Received: from sea.source.kernel.org ([2600:3c0a:e001:78e:0:1991:8:25]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vKxqo-0000000G1Cc-3ESG for linux-arm-kernel@lists.infradead.org; Mon, 17 Nov 2025 11:56:55 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id 66238439F7; Mon, 17 Nov 2025 11:56:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FF00C4CEF1; Mon, 17 Nov 2025 11:56:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763380614; bh=kB9ZXKohVrn5IEX6S+MiFF63aw8Py7ecNIx3DP7Yi+E=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=aSN0iYkknKkvL0mij60oWERa1I1nFDaXLZGicletaKLuv+pgGNmTLofH7Xf409+u2 0SGzgm1GEyLD3Sn/ZtmvWY/mSdJ94pmrBc5Y0NmFUosSrN6kuTZjDd8HFDK056E5xe rRSubGAb3LKV1vwS/S4rbVoG4QGoTSQRTBrldmLUwfG7Bm6wHZNepJiO1emO/1/Ohn AS6ykqNckSmvEwmPREXaC3xg/QTrxhdi4fOJ1s9IBRhoQpB23WVGvY8zQTibIRJstd pIELReB4DkVkdcwBN+17uFx2D0/QEPLfCaOaog1rh2nooNzKLpiwQvbTQgLLRqweh6 RVj+Pp0Q9Anlw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vKxql-00000005oR5-4B8h; Mon, 17 Nov 2025 11:56:52 +0000 Date: Mon, 17 Nov 2025 11:56:51 +0000 Message-ID: <861plwub24.wl-maz@kernel.org> From: Marc Zyngier To: Fuad Tabba Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Christoffer Dall , Volodymyr Babchuk , Yao Yuan Subject: Re: [PATCH v2 29/45] KVM: arm64: GICv3: Set ICH_HCR_EL2.TDIR when interrupts overflow LR capacity In-Reply-To: References: <20251109171619.1507205-1-maz@kernel.org> <20251109171619.1507205-30-maz@kernel.org> <86cy5ku06v.wl-maz@kernel.org> <86a50otsuh.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tabba@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oupton@kernel.org, yuzenghui@huawei.com, christoffer.dall@arm.com, Volodymyr_Babchuk@epam.com, yaoyuan@linux.alibaba.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20251117_035654_854051_9351674C X-CRM114-Status: GOOD ( 43.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 17 Nov 2025 08:22:05 +0000, Fuad Tabba wrote: > > Hi Marc, > > On Fri, 14 Nov 2025 at 17:41, Marc Zyngier wrote: > > > > On Fri, 14 Nov 2025 15:53:33 +0000, > > Fuad Tabba wrote: > > > > > > Hi Marc, > > > > > > On Fri, 14 Nov 2025 at 15:02, Marc Zyngier wrote: > > > > > > > > On Fri, 14 Nov 2025 14:20:46 +0000, > > > > Fuad Tabba wrote: > > > > > > > > > > Hi Marc, > > > > > > > > > > On Sun, 9 Nov 2025 at 17:17, Marc Zyngier wrote: > > > > > > > > > > > > Now that we are ready to handle deactivation through ICV_DIR_EL1, > > > > > > set the trap bit if we have active interrupts outside of the LRs. > > > > > > > > > > > > Signed-off-by: Marc Zyngier > > > > > > --- > > > > > > arch/arm64/kvm/vgic/vgic-v3.c | 7 +++++++ > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c > > > > > > index 1026031f22ff9..26e17ed057f00 100644 > > > > > > --- a/arch/arm64/kvm/vgic/vgic-v3.c > > > > > > +++ b/arch/arm64/kvm/vgic/vgic-v3.c > > > > > > @@ -42,6 +42,13 @@ void vgic_v3_configure_hcr(struct kvm_vcpu *vcpu, > > > > > > ICH_HCR_EL2_VGrp0DIE : ICH_HCR_EL2_VGrp0EIE; > > > > > > cpuif->vgic_hcr |= (cpuif->vgic_vmcr & ICH_VMCR_ENG1_MASK) ? > > > > > > ICH_HCR_EL2_VGrp1DIE : ICH_HCR_EL2_VGrp1EIE; > > > > > > + > > > > > > + /* > > > > > > + * Note that we set the trap irrespective of EOIMode, as that > > > > > > + * can change behind our back without any warning... > > > > > > + */ > > > > > > + if (irqs_active_outside_lrs(als)) > > > > > > + cpuif->vgic_hcr |= ICH_HCR_EL2_TDIR; > > > > > > } > > > > > > > > > > I just tested these patches as they are on kvmarm/next > > > > > 2ea7215187c5759fc5d277280e3095b350ca6a50 ("Merge branch > > > > > 'kvm-arm64/vgic-lr-overflow' into kvmarm/next"), without any > > > > > additional pKVM patches. I tried running it with pKVM (non-protected) > > > > > and with just plain nVHE. In both cases, I get a trap to EL2 (0x18) > > > > > when booting a non-protected guest, which triggers a bug in > > > > > handle_trap() arch/arm64/kvm/hyp/nvhe/hyp-main.c:706 > > > > > > > > > > This trap is happening because of setting this particular trap (TDIR). > > > > > Just removing this trap from vgic_v3_configure_hcr() from the ToT on > > > > > kvmarm/next boots fine. > > > > > > > > This is surprising, as I'm not hitting this on actual HW. Are you > > > > getting a 0x18 trap? If so, is it coming from the host? Can you > > > > correlate the PC with what the host is doing? > > > > > > I should have given you that earlier, sorry. > > > > > > Yes, it's an 0x18 trap from the host (although it happens when I boot > > > a guest). Here is the relevant part of the backtrace addr2lined and > > > the full one below. > > > > > > handle_percpu_devid_irq+0x90/0x120 (kernel/irq/chip.c:930) > > > generic_handle_domain_irq+0x40/0x64 (include/linux/irqdesc.h:?) > > > gic_handle_irq+0x4c/0x110 (include/linux/irqdesc.h:?) > > > call_on_irq_stack+0x30/0x48 (arch/arm64/kernel/entry.S:893) > > > > > > [ 28.454804] Code: d65f03c0 92800008 f9000008 17fffffa (d4210000) > > > [ 28.454873] kvm [266]: Hyp Offset: 0xfff1205c3fe00000 > > > [ 28.455157] Kernel panic - not syncing: HYP panic: > > > [ 28.455157] PS:204023c9 PC:000e5fa4413e39bc ESR:00000000f2000800 > > > [ 28.455157] FAR:ffff800082733d3c HPFAR:0000000000500000 PAR:0000000000000000 > > > > I expect you have a write to ICC_DIR_EL1 at this address? > > It almost surely must be, but tracking it down hasn't been that easy. > That said, I think it's ending up in gic_eoimode1_eoi_irq(), which > calls gic_write_dir() if !gic_arm64_erratum_2941627_needed(d). > > I wonder if your hardware needs that erratum. No, it doesn't. And this erratum only kicks in if the deactivated interrupt is an SPI that has been moved to another CPU while being handled (i.e. never). It really isn't related to the GIC itself (which the erratum above is about), but to the CPU interface, and whether TDIR applies to ICC_DIR_EL1 on top of the virtual variant. The architecture "deprecates" not trapping, and QEMU abides by this deprecation. HW that predates the deprecation didn't get the message, oddly enough... Neither did I, until you reported the crash! Thanks, M. -- Without deviation from the norm, progress is not possible.