From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4B024229D0; Thu, 6 Aug 2026 09:10:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786007436; cv=none; b=iv4XFQOFpA+hIIOGPvnl4aqFPxT++7YI2FgSgn6ba91YpCqbvnlfJ7WdZMHJB4sdMKiBdFxj5WhnywsxHFRxFZXOd5rWSwYf9TztmTRy9j9FQriwYkbV9tGh3eBRF8ZPZuh83C/AwO9MD+MzIb2bn4PtY6/f2UAszR5tKHyDuXI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786007436; c=relaxed/simple; bh=UKJi7F6Ti2WX/mFEgYxUB+ZxN2Uzlw5SDNU2jlgxLsI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fXumik4NmJxzVbWM3f1MSFlIg4EyaoPXGikJbltzHSS1xjfQdvihDFmscwxHlRqE09ciSrE2x4Ub8H2I/WWvi1ioXjXevSTokVYK/SiR0j4DMO52cPRZqzvnm0qflIr9XBlc6O9tDy0V0j7jocEa7LDcR6vXV84BcWIL/AuV2sY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hrlYiL6A; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hrlYiL6A" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7047D1F01559; Thu, 6 Aug 2026 09:10:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786007434; bh=gQ8zq70y0787azP95WYE6ddYaI5CvHh9/VpExmEvz+w=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=hrlYiL6AUHSwVsGrZy5BQKpHYGfFiVMP+eiN1kClFvgN7Hy2nKce9wuEa5tJDtsDQ xNxbyoL+N0ycBsX4at6VXk9WEnX6CMrKjrIL1FRn8vLrpm0HIdTgK52fvnsE82/3bN q79LKl4awvIPHZCILgwY34u8eqgpE4ODq44vrQmVx/z5RMrbFV+a0GMYJNELSCG0Lk jC1eQ6qH6pPNij5jKlFAW31JGGFp79LxMSqh/9ngbWVovbxNgtE6t0CdaVBaAscljO jJDD7R9QoewanwYAcoDcIv4epE+s9loQdE8lRSM0LbC0xh6eLx5soNvMsADZ7zR1rg cd57302UT8qNw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wru7U-0000000CtHi-2vCZ; Thu, 06 Aug 2026 09:10:32 +0000 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Steffen Eiden , Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Fuad Tabba , Hyunwoo Kim , Yao Yuan , ljs@kernel.org, stable@vger.kernel.org Subject: [PATCH v2 4/8] KVM: arm64: Correctly handle end of VA space TLBI invalidation Date: Thu, 6 Aug 2026 10:10:22 +0100 Message-ID: <20260806091026.620700-5-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260806091026.620700-1-maz@kernel.org> References: <20260806091026.620700-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, seiden@linux.ibm.com, joey.gouly@arm.com, suzuki.poulose@arm.com, oupton@kernel.org, yuzenghui@huawei.com, fuad.tabba@linux.dev, imv4bel@gmail.com, yaoyuan@linux.alibaba.com, ljs@kernel.org, stable@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Our TLB invalidation by VA code is based on comparing two ranges, one defined by the TLB, and one defined by the TLBI instruction. Each range is defined by a start and a size. However, the way the comparison is done doesn't account for address rollover, as it compares an address with (base + size). This works nicely until this expression represent the last page/block in the TTBR1 VA space, as the result is a big fat 0. And a failed TLB invalidation. Rewrite the comparison in a way that is immune to the address rollover (making the end address inclusive instead of exclusive), and move this into a common helper that is used by both VA and IPA invalidations, as suggested by Hyunwoo Kim (although the IPA version didn't suffer from this particular problem, obviously). Fixes: 4ffa72ad8f37e ("KVM: arm64: nv: Add S1 TLB invalidation primitive for VNCR_EL2") Reviewed-by: Yuan Yao Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org --- arch/arm64/kvm/nested.c | 43 ++++++++++++++++++----------------------- 1 file changed, 19 insertions(+), 24 deletions(-) diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 27bc7ee4b3382..8a602d074dbb4 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -999,6 +999,20 @@ static void invalidate_vncr(struct vncr_tlb *vt) clear_fixmap(vncr_fixmap(vt->cpu)); } +static bool vncr_tlb_intersects(struct vncr_tlb *vt, u64 addr, + u64 scope_start, u64 scope_size) +{ + u64 tlb_size, tlb_start, tlb_end, scope_end; + + tlb_size = ttl_to_size(pgshift_level_to_ttl(vt->wi.pgshift, vt->wr.level)); + + tlb_start = addr & ~(tlb_size - 1); + tlb_end = tlb_start + tlb_size - 1; + scope_end = scope_start + scope_size - 1; + + return !(tlb_end < scope_start || tlb_start > scope_end); +} + /* * VNCR TLB invalidation occurs from MMU notifiers or TLBI instructions, and * either can race against a vcpu not being onlined yet (no pseudo-TLB @@ -1021,19 +1035,9 @@ static void kvm_invalidate_vncr_ipa(struct kvm *kvm, u64 start, u64 end) if (!kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY)) return; - kvm_for_each_vncr_tlb(i, vcpu, vt, kvm) { - u64 ipa_start, ipa_end, ipa_size; - - ipa_size = ttl_to_size(pgshift_level_to_ttl(vt->wi.pgshift, - vt->wr.level)); - ipa_start = vt->wr.pa & ~(ipa_size - 1); - ipa_end = ipa_start + ipa_size; - - if (ipa_end <= start || ipa_start >= end) - continue; - - invalidate_vncr(vt); - } + kvm_for_each_vncr_tlb(i, vcpu, vt, kvm) + if (vncr_tlb_intersects(vt, vt->wr.pa, start, end - start)) + invalidate_vncr(vt); } struct s1e2_tlbi_scope { @@ -1059,28 +1063,19 @@ static void invalidate_vncr_va(struct kvm *kvm, lockdep_assert_held_write(&kvm->mmu_lock); kvm_for_each_vncr_tlb(i, vcpu, vt, kvm) { - u64 va_start, va_end, va_size; - - va_size = ttl_to_size(pgshift_level_to_ttl(vt->wi.pgshift, - vt->wr.level)); - va_start = vt->gva & ~(va_size - 1); - va_end = va_start + va_size; - switch (scope->type) { case TLBI_ALL: break; case TLBI_VA: - if (va_end <= scope->va || - va_start >= (scope->va + scope->size)) + if (!vncr_tlb_intersects(vt, vt->gva, scope->va, scope->size)) continue; if (vt->wr.nG && vt->wr.asid != scope->asid) continue; break; case TLBI_VAA: - if (va_end <= scope->va || - va_start >= (scope->va + scope->size)) + if (!vncr_tlb_intersects(vt, vt->gva, scope->va, scope->size)) continue; break; -- 2.47.3