From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECBFA606A329 for ; Mon, 2 Nov 2026 18:41:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=Xjc5KiYX; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=SCW3pZSQ DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Unsubscribe-Post:List-Owner; bh=1siRmUO4vM6mmsoG9PiDb9+LFZeVOS+fG7WHL62iNvw=; b=Xjc5KiYXtd5Snfr0m1dnY+gePL MMozuve7hGGsrXitZkZ5CDK0cha5Yw2W0WUTJSSqNG74fWNuCrhzlvN3naFWp35iRIfHAaPKq7OzZ UHEt7V5MsyKVsXd+yGnhA+RaBelRW9dT43Lv5LJvv9/B4YWz+mLwBmuHB5/YfGBuDXpvyZYptNMZs +m63HBTHCkK2NVtZv+1I0n28QR+tPoDj7d1IaZoSiJ0fINo4yoy+kzO/562UxUGo8RCmwjZQb/wA2 3b0tyvVZ5Hbvoaas7JRDOVXkcTvUuRV7tladRGf95OsYXYpeEFQpwNkocvQE/4qWIxUzod8C0wepb aQgvzyYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xNwy8-000000000dI-21ik; Mon, 02 Nov 2026 18:41:20 +0000 Received: from sea.source.kernel.org ([172.234.252.31]) by bombadil.infradead.org with esmtps (Exim 4.100.1 #2 (Red Hat Linux)) id 1xNwy7-000000000d4-01c7 for linux-arm-kernel@lists.infradead.org; Mon, 02 Nov 2026 18:41:19 +0000 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by sea.source.kernel.org (Postfix) with ESMTP id 666EB437F3; Mon, 2 Nov 2026 18:41:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4AF831F00A1A; Mon, 2 Nov 2026 18:41:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1793644878; bh=1siRmUO4vM6mmsoG9PiDb9+LFZeVOS+fG7WHL62iNvw=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=SCW3pZSQiuuCCSRrJEbYUc/Rtta1yJfKnOc7PlPInpy6+CvR4M5f94reuHeIOq20p eV8ZksdI+7U3MTC+Sh4es8lXO5ej8E2aT8tkViYrN8qSFVLE3GwnoyjZt0/dFAD2kO qEKyvVrXiwX90O6FW+3NhoBoE5Jdk2WPUt7zPwVx7EuscnxvQMM91BAPfTqx/ZsBs0 jICmqnf5y+TVgNJ8TVmAKljm2/dHOd+v20IedDJmNPVyLs58TgAul6/6u3qehoH6Fd HjBPxKUCp73f82UApmNo9WkX6JVs8FUW2l9bm/DSxBgjyKlbMpWC2mcioLXMfgdkBH s80etyyTOYvKQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xNwy4-0000000A2b6-0jFh; Mon, 02 Nov 2026 18:41:16 +0000 Date: Mon, 02 Nov 2026 18:41:15 +0000 Message-ID: <86a4nr2i0k.wl-maz@kernel.org> From: Marc Zyngier To: "Ard Biesheuvel" Cc: "Fuad Tabba" , "Will Deacon" , "Ard Biesheuvel" , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, "Oliver Upton" , "Joey Gouly" , "Steffen Eiden" , "Suzuki K Poulose" , "Zenghui Yu" Subject: Re: [PATCH] KVM: arm64: Don't livelock when AT can't resolve a fault IPA In-Reply-To: <08db10ef-c3d8-44b8-9d80-e7505400bf12@app.fastmail.com> References: <20261102094748.2638468-2-ardb+git@google.com> <86bj872n1b.wl-maz@kernel.org> <08db10ef-c3d8-44b8-9d80-e7505400bf12@app.fastmail.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) X-Archived-By: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: ardb@kernel.org, fuad.tabba@linux.dev, will@kernel.org, ardb+git@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, oupton@kernel.org, joey.gouly@arm.com, seiden@linux.ibm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" X-Spam-Status: No, score=-10.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on aws-us-west-2-korg-listserv-alma10-1 On Mon, 02 Nov 2026 18:23:09 +0000, "Ard Biesheuvel" wrote: > > > > On Mon, 2 Nov 2026, at 17:52, Marc Zyngier wrote: > > On Mon, 02 Nov 2026 15:57:51 +0000, > > Fuad Tabba wrote: > >> > >> Hi Will, > >> > >> On Mon, 02 Nov 2026 14:17:10 +0000, Will Deacon wrote: > >> > On Mon, Nov 02, 2026 at 02:09:25PM +0000, Marc Zyngier wrote: > >> > > On Mon, 02 Nov 2026 13:59:12 +0000, > >> > > Will Deacon wrote: > >> [...] > >> > > > > diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h > >> [...] > >> > > > > static inline bool kvm_hyp_handle_memory_fault(struct kvm_vcpu *vcpu, > >> > > > > u64 *exit_code) > >> > > > > { > >> > > > > - if (!__populate_fault_info(vcpu)) > >> > > > > + if (!__populate_fault_info(vcpu)) { > >> > > > > >> > > > Presumably we need a barrier (DSB?) here, to ensure that the TLBI happens > >> > > > after the AT instruction? > >> > >> I don't think the TLBI requires a barrier before it, because it > >> doesn't matter if the AT's walk allocates entries after the TLBI. > >> Anything it allocates matches memory, and the stage-2 descriptor the > >> AT faulted on has AF clear, so that descriptor can't be cached: > >> > >> R_XCLRD: Translation table entries that generate a Translation > >> fault, an Address size fault, or an Access flag fault are never cached > >> in a TLB. > >> > >> The stale entries the guest keeps hitting were already there before > >> the TLBI, and the dsb(nsh) and isb after it are what complete it > >> against them. > >> > >> [... your other reply ... ] > >> > > Alternatively, VMALLE1 or even VAE1 should be enough to get rid of the > >> > > S1 TLB. > >> > > >> > Ah, sorry, VMALLE1 is what I meant to ask about. I basically haven't > >> > understood the need to invalidate the stage-2. > >> > >> The guest's walk can also go through a stage-2 entry for its > >> page-table page, cached before the aging cleared that page's AF, and > >> nothing requires the AT to use that entry: > >> > >> R_NPZWX: When an address translation instruction is executed, it is > >> IMPLEMENTATION DEFINED whether the result can be returned from a TLB, > >> or a translation table walk occurs. > >> > >> So if my reading is correct, on a CPU whose AT walks memory instead, > >> the AT keeps failing while the guest's walk goes through the stale > >> entry, and the guest takes the same permission fault again. VMALLE1 > >> and VAE1 only cover stage 1, so they'd leave that loop in place. > > > > If we assume that the TLB covers VA->PA (which is a very likely case), > > TLBI VAE1 should nuke the permission fault that brought the vcpu to > > EL2, and instead walk the page tables, getting the access fault. > > > > The only case I can imagine where invalidating S2 would matter is if > > you are running under NV, because there is an independent S2 TLB (the > > shadow S2), and then VMALLS12E1 makes total sense. > > > > Ard, was this solely observed under NV? Or did you see it on a non NV > > guest as well? > > > > No it happens with guests running at EL1 too. Damn! > My experiments seem to indicate that on this uarch, the AT instruction > does not hit in the TLB, and so it can fail on AF==0 for an IPA that > is covered by a S2-only TLB entry (with AF==1 and XN==1 due to lazy exec). > > So after the VAAE1, the S1 walk happily produces the same translation > we just invalidated, without taking a S1PTW fault, and the guest faults > again on the I-fetch that took us to hyp the first time around. Ah, I think I see what can happen, even without split TLBs. If you have a S2 walk cache that skips the descriptor where AF=0, then the TLB refill just succeeds, and the cycle repeats. OK, I'm now convinced we need S2 invalidation as well even in the VA->PA case. We needed it for NV anyway, but hey... Hopefully this doesn't happen too often. Thanks, M. -- Without deviation from the norm, progress is not possible.