From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 73C15CAC5BB for ; Wed, 1 Oct 2025 16:17:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=O3xFGfSJUrzTYy5HFRvEf7UVQ3lgG70Ni6Ds4LtkEdo=; b=4w7xsaTnUEMLU5iPR7wkhrruDR 7o6HPzNMd5U+sy8ZTxJjvzjFbZp9sofX/mpitGRbd09pggdsv3VsQZlqsYkYzJCVFl+1gapAHoze2 3h15M01POxm9qljC3lbxg8ShvJbVFvQrWyLtpx2jjIaq94bjMpxqcPQCaFKvMgipsbaV7pexOSzt6 2dDJ+Tg00rZBhFFZrCpqjmGGtq/O2okZIXRqSjOq1YH9WyZ1Qp0TLr1lkDOI5csOWBpBbn/RTBP5Q p4U8LLDMKDNKp5aIkUwQnZCa3xpczZFeHlJlWABQopQWzWgT9LrU4PVvYoVtl9Hl1mBNSZjdG7Vv9 m6ko219Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1v3zWZ-00000008VJn-1SDW; Wed, 01 Oct 2025 16:17:51 +0000 Received: from sea.source.kernel.org ([172.234.252.31]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1v3zWW-00000008VIG-0ADt for linux-arm-kernel@lists.infradead.org; Wed, 01 Oct 2025 16:17:49 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id BB8CB417BF; Wed, 1 Oct 2025 16:17:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 98736C4CEF1; Wed, 1 Oct 2025 16:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1759335467; bh=bMA07Ln4hDU10o5W440wFOk9DddcSBtkg4Pl8kMatM0=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=tjgUy+a0m+LLagSHoLjoo97tMk8+HnZ8x+7Pj8e1t1ZSA823s7A1uAhNp8mQrlODf 6keLUOqRgGD2Xnda5HZ4ctk1TsMH5FUL7UHUZiaH6u4r2FPop46y9dikzYj11YWvzp y/27ZWamaCVEBtDkofkynE18MCu2dS+98T0q7o8cAmVxVoC3MoTD4+HomvbLMuLV1Q wsuKK0hUdS9aoimdBtL3iIDyjVgyZ8WgABvG0d9nMuQiPM/WrcbEbECb1rDMQjz2O7 K86+OMPUWVFkkU2+WgbLPg/6uGJu9ZoNAB+gOEYdtNVjH/LqvnNwsQLt1bLk5DfOu7 CIlmH/J/TxGHA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1v3zWT-0000000ArZi-1KXr; Wed, 01 Oct 2025 16:17:45 +0000 Date: Wed, 01 Oct 2025 17:17:44 +0100 Message-ID: <86cy76zjdj.wl-maz@kernel.org> From: Marc Zyngier To: Volodymyr Babchuk Cc: "linux-arm-kernel@lists.infradead.org" , Dmytro Terletskyi , kvmarm Subject: Re: KVM: Nested VGIC emulation leads to infinite IRQ exceptions In-Reply-To: <87bjmrprvq.fsf@epam.com> References: <87bjmrprvq.fsf@epam.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: Volodymyr_Babchuk@epam.com, linux-arm-kernel@lists.infradead.org, Dmytro_Terletskyi@epam.com, kvmarm@lists.linux.dev X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20251001_091748_117295_30B3A028 X-CRM114-Status: GOOD ( 15.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 30 Sep 2025 22:11:54 +0100, Volodymyr Babchuk wrote: [...] I spent some time to look at this again. > > This is a part of the KVM trace, where you can see that vCPU in question > tries to perform ERET to Linux in DomU but is being brought back to > vEL2. In this particular case this is vCPU1 / vvCPU0. I filtered out > other vCPUs to reduce clutter. > > qemu-system-aar-41290 [000] d.... 12023.695620: kvm_entry: PC: 0x00000a0000267c80 > qemu-system-aar-41290 [000] d.... 12023.695620: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0 > qemu-system-aar-41290 [000] d.... 12023.695621: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0 > qemu-system-aar-41290 [000] d.... 12023.695621: kvm_timer_emulate: arch_timer_ctx_index: 1 (should_fire: 1) > qemu-system-aar-41290 [000] d.... 12023.695621: kvm_timer_emulate: arch_timer_ctx_index: 0 (should_fire: 0) > qemu-system-aar-41290 [000] ..... 12023.695621: kvm_exit: TRAP: HSR_EC: 0x001a (ERET), PC: 0x00000a00002674e0 Wants to ERET to EL1 > qemu-system-aar-41290 [000] ..... 12023.695621: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0 > qemu-system-aar-41290 [000] d.... 12023.695622: kvm_timer_save_state: CTL: 0x000000 CVAL: 0x0 arch_timer_ctx_index: 2 > qemu-system-aar-41290 [000] d.... 12023.695622: kvm_timer_save_state: CTL: 0x000005 CVAL: 0x426f7d24736c arch_timer_ctx_index: 3 EL2 physical timer is pending > qemu-system-aar-41290 [000] ..... 12023.695622: kvm_nested_eret: elr_el2: 0xffffffc0010ac5a4 spsr_el2: 0x024000c5 (M: EL1h) hcr_el2: 807c663f Return to EL1, reload the EL1 context > qemu-system-aar-41290 [000] ..... 12023.695622: kvm_get_timer_map: VCPU: 1, dv: 1, dp: 0, ev: 2, ep: 3 > qemu-system-aar-41290 [000] ..... 12023.695622: kvm_timer_update_irq: VCPU: 1, IRQ 27, level 1 > qemu-system-aar-41290 [000] ..... 12023.695623: vgic_update_irq_pending: VCPU: 1, IRQ 27, level: 1 > qemu-system-aar-41290 [000] ..... 12023.695623: kvm_timer_update_irq: VCPU: 1, IRQ 30, level 0 > qemu-system-aar-41290 [000] ..... 12023.695623: vgic_update_irq_pending: VCPU: 1, IRQ 30, level: 0 > qemu-system-aar-41290 [000] d.... 12023.695623: kvm_timer_restore_state: CTL: 0x000005 CVAL: 0x48aac64bd arch_timer_ctx_index: 1 EL1 virtual timer is pending > qemu-system-aar-41290 [000] d.... 12023.695624: kvm_timer_restore_state: CTL: 0x000000 CVAL: 0x0 arch_timer_ctx_index: 0 > qemu-system-aar-41290 [000] ..... 12023.695624: kvm_timer_emulate: arch_timer_ctx_index: 2 (should_fire: 0) > qemu-system-aar-41290 [000] ..... 12023.695624: kvm_timer_emulate: arch_timer_ctx_index: 3 (should_fire: 1) EL2 physical timer still pending > qemu-system-aar-41290 [000] ..... 12023.695626: kvm_get_timer_map: VCPU: 1, dv: 1, dp: 0, ev: 2, ep: 3 > qemu-system-aar-41290 [000] d.... 12023.695626: kvm_timer_save_state: CTL: 0x000005 CVAL: 0x48aac64bd arch_timer_ctx_index: 1 > qemu-system-aar-41290 [000] d.... 12023.695627: kvm_timer_save_state: CTL: 0x000000 CVAL: 0x0 arch_timer_ctx_index: 0 HW without FEAT_ECV, I presume? > qemu-system-aar-41290 [000] ..... 12023.695627: kvm_inject_nested_exception: IRQ: esr_el2 0x0 elr_el2: 0xffffffc0010ac5a4 spsr_el2: 0x024000c5 (M: EL1h) hcr_el2: 807c663f Take an interrupt from EL1 to EL2, flip the world again. > qemu-system-aar-41290 [000] ..... 12023.695627: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0 > qemu-system-aar-41290 [000] ..... 12023.695627: kvm_timer_update_irq: VCPU: 1, IRQ 28, level 0 > qemu-system-aar-41290 [000] ..... 12023.695627: vgic_update_irq_pending: VCPU: 1, IRQ 28, level: 0 > qemu-system-aar-41290 [000] ..... 12023.695628: kvm_timer_update_irq: VCPU: 1, IRQ 26, level 1 > qemu-system-aar-41290 [000] ..... 12023.695628: vgic_update_irq_pending: VCPU: 1, IRQ 26, level: 1 > qemu-system-aar-41290 [000] d.... 12023.695628: kvm_timer_restore_state: CTL: 0x000000 CVAL: 0x0 arch_timer_ctx_index: 2 > qemu-system-aar-41290 [000] d.... 12023.695628: kvm_timer_restore_state: CTL: 0x000005 CVAL: 0x426f7d24736c arch_timer_ctx_index: 3 Yup, EL2 timer still pending > qemu-system-aar-41290 [000] ..... 12023.695629: kvm_timer_emulate: arch_timer_ctx_index: 1 (should_fire: 1) > qemu-system-aar-41290 [000] ..... 12023.695629: kvm_timer_emulate: arch_timer_ctx_index: 0 (should_fire: 0) > qemu-system-aar-41290 [000] d.... 12023.695632: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0 > qemu-system-aar-41290 [000] d.... 12023.695632: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0 > qemu-system-aar-41290 [000] d.... 12023.695633: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0 > qemu-system-aar-41290 [000] d.... 12023.695633: kvm_entry: PC: 0x00000a0000267c80 and we go again. So the MI doesn't seem to be the cause of this, as empty LRs are not likely to be the problem. However, we definitely see timer interrupts firing, EL2 being entered, and yet, El2 doesn't seem to acknowledge the interrupt. So something is wrong there, either in Xen on in KVM. You want to instrument what is happening at this stage (I don't see anything of the like, but my machines have FEAT_ECV). M. -- Without deviation from the norm, progress is not possible.