From: Yosry Ahmed <yosry@kernel.org>
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Jim Mattson <jmattson@google.com>,
Maxim Levitsky <mlevitsk@redhat.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Tom Lendacky <thomas.lendacky@amd.com>,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
Yosry Ahmed <yosry@kernel.org>
Subject: [PATCH v2 25/29] KVM: x86/mmu: Refactor kvm_mmu_invlpg() to allow skipping the GVA flush
Date: Thu, 8 Oct 2026 00:14:21 +0000 [thread overview]
Message-ID: <20261008001425.2458927-26-yosry@kernel.org> (raw)
In-Reply-To: <20261008001425.2458927-1-yosry@kernel.org>
Refactor helpers out of kvm_mmu_invalidate_addr() and kvm_mmu_invlpg()
that take in an extra argument to skip the GVA flush.
This will be used when invalidating GVAs in a different context than the
current one (i.e. invalidating an L2 GVA from L1), so flushing the
current context would flush the wrong TLB entries.
No functional change intended.
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 24 ++++++++++++++++++------
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index b3085f409d0de..433ed69b1e1e4 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -6769,8 +6769,8 @@ static void kvm_mmu_sync_addr(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,
write_unlock(&vcpu->kvm->mmu_lock);
}
-void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,
- u64 addr, unsigned long roots)
+static void __kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,
+ u64 addr, unsigned long roots, bool flush_gva)
{
struct kvm_mmu *mmu;
int i;
@@ -6783,7 +6783,8 @@ void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,
if (is_noncanonical_invlpg_address(addr, vcpu))
return;
- kvm_x86_call(flush_tlb_gva)(vcpu, addr);
+ if (flush_gva)
+ kvm_x86_call(flush_tlb_gva)(vcpu, addr);
if (tdp_enabled)
return;
@@ -6805,9 +6806,15 @@ void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,
kvm_mmu_sync_addr(vcpu, mmu, addr, mmu->prev_roots[i].hpa);
}
}
+
+void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,
+ u64 addr, unsigned long roots)
+{
+ __kvm_mmu_invalidate_addr(vcpu, w, addr, roots, true);
+}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_mmu_invalidate_addr);
-void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
+static void __kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva, bool flush_gva)
{
/*
* INVLPG is required to invalidate any global mappings for the VA,
@@ -6819,11 +6826,16 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
* be synced when switching to that new cr3, so nothing needs to be
* done here for them.
*/
- kvm_mmu_invalidate_addr(vcpu, &vcpu->arch.gva_walk, gva, KVM_MMU_ROOTS_ALL);
+ __kvm_mmu_invalidate_addr(vcpu, &vcpu->arch.gva_walk, gva,
+ KVM_MMU_ROOTS_ALL, flush_gva);
++vcpu->stat.invlpg;
}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_mmu_invlpg);
+void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
+{
+ __kvm_mmu_invlpg(vcpu, gva, true);
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_mmu_invlpg);
void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid)
{
--
2.56.0.360.g66cac248cb-goog
next prev parent reply other threads:[~2026-10-08 0:14 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-08 0:13 [PATCH v2 00/29] KVM: nSVM: Optimize nSVM TLB flushes Yosry Ahmed
2026-10-08 0:13 ` [PATCH v2 01/29] KVM: nSVM: Flush the TLB after forcefully leaving nested Yosry Ahmed
2026-10-08 0:13 ` [PATCH v2 02/29] KVM: SVM: Document number of ASIDs CPUID setting Yosry Ahmed
2026-10-08 0:13 ` [PATCH v2 03/29] KVM: VMX: Generalize VPID allocation to be vendor-neutral Yosry Ahmed
2026-10-08 0:30 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 04/29] KVM: x86/mmu: Support specifying reserved TLB tags Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 05/29] KVM: SVM: Add helpers to set/clear ASID flush in VMCB Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 06/29] KVM: SVM: Fallback to flush everything if FLUSHBYASID is not available Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 07/29] KVM: SEV: Do ASID initialization at VMCB initialization Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 08/29] KVM: SEV: Expose sev_get_asid() outside of sev.c Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 09/29] KVM: SEV: Explicitly initialize the per vCPU ASID on SEV VM migration Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 10/29] KVM: SVM: Use a static ASID per vCPU Yosry Ahmed
2026-10-08 0:30 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 11/29] KVM: SVM: Only flush the fallback ASID when used by a different vCPU Yosry Ahmed
2026-10-08 0:33 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 12/29] KVM: nSVM: Drop svm->nested.initialized Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 13/29] KVM: nSVM: Add a placeholder ASID for L2 Yosry Ahmed
2026-10-08 0:42 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 14/29] KVM: x86: hyper-v: Rename kvm_hv_vcpu_purge_flush_tlb() Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 15/29] KVM: x86: hyper-v: Allow purging all TLB flush FIFOs Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 16/29] KVM: nSVM: Flush both L1 and L2 ASIDs on KVM_REQ_TLB_FLUSH Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 17/29] KVM: nSVM: Always switch VMCB before leaving guest mode Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 18/29] KVM: nSVM: Split nested_svm_transition_tlb_flush() into entry/exit fns Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 19/29] KVM: nSVM: Service local TLB flushes before nested transitions Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 20/29] KVM: x86: Add KVM_REQ_MMU_SYNC_ALL_ROOTS Yosry Ahmed
2026-10-08 3:29 ` Lai Jiangshan
2026-10-08 0:14 ` [PATCH v2 21/29] KVM: nSVM: Handle nested TLB flush requests through TLB_CONTROL Yosry Ahmed
2026-10-08 0:36 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 22/29] KVM: nSVM: Flush the TLB if L1 changes L2's ASID in vmcb12 Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 23/29] KVM: nSVM: Do not reset TLB_CONTROL in vmcb02 on nested VM-Enter Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 24/29] KVM: x86/mmu: Rename __kvm_mmu_invalidate_addr() to kvm_mmu_sync_addr() Yosry Ahmed
2026-10-08 0:14 ` Yosry Ahmed [this message]
2026-10-08 0:14 ` [PATCH v2 26/29] KVM: nSVM: Flush L2's ASID when emulating INVLPGA Yosry Ahmed
2026-10-08 0:45 ` sashiko-bot
2026-10-08 0:14 ` [PATCH v2 27/29] KVM: nSVM: Flush the ASID on nested transitions if shared by L1 and L2 Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 28/29] KVM: nSVM: Use different ASIDs for " Yosry Ahmed
2026-10-08 0:14 ` [PATCH v2 29/29] KVM: selftests: Add a test for nested TLB flushes Yosry Ahmed
2026-10-08 0:35 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261008001425.2458927-26-yosry@kernel.org \
--to=yosry@kernel.org \
--cc=jmattson@google.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mlevitsk@redhat.com \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=thomas.lendacky@amd.com \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox