* FAILED: patch "[PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU" failed to apply to 5.15-stable tree
@ 2026-07-25 6:52 gregkh
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
0 siblings, 2 replies; 21+ messages in thread
From: gregkh @ 2026-07-25 6:52 UTC (permalink / raw)
To: seanjc, imv4bel, pbonzini; +Cc: stable
The patch below does not apply to the 5.15-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <stable@vger.kernel.org>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-5.15.y
git checkout FETCH_HEAD
git cherry-pick -x 2abd5287f08319fa35764566b15c6e22cb1068db
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<stable@vger.kernel.org>' --in-reply-to '2026072557-gem-defensive-a757@gregkh' --subject-prefix 'PATCH 5.15.y' 'HEAD^..'
Possible dependencies:
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 2abd5287f08319fa35764566b15c6e22cb1068db Mon Sep 17 00:00:00 2001
From: Sean Christopherson <seanjc@google.com>
Date: Mon, 13 Jul 2026 08:15:33 -0700
Subject: [PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU
pages available
Check for a "stale" page fault, i.e. for an invalid and/or obsolete root,
after making MMU pages available for the shadow MMU. If reclaiming shadow
pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to
map memory into an invalid root. On its own, populating an invalid root is
"fine", but because child shadow pages inherit their parent's role, any
children created during the map/fetch will be created as invalid pages,
thus violating KVM's invariant that invalid pages are never on the list of
active MMU pages.
Note, the underlying flaw has existed since KVM first started tracking
invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root
pagetables"), but the true badness only came along in 2020 (Linux 5.9)
with the invariant that invalid shadow pages can't be on the list of
active pages.
Note #2, inheriting role.invalid when creating child shadow pages is also
far from ideal; that flaw will be addressed separately.
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 234d0a95abf5..41f92ed1ca37 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4852,16 +4852,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
if (r != RET_PF_CONTINUE)
return r;
- r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (is_page_fault_stale(vcpu, fault))
- goto out_unlock;
-
r = make_mmu_pages_available(vcpu);
if (r)
goto out_unlock;
+ if (is_page_fault_stale(vcpu, fault)) {
+ r = RET_PF_RETRY;
+ goto out_unlock;
+ }
+
r = direct_map(vcpu, fault);
out_unlock:
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index df3ae0c7ec2c..1ba840a73b7a 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -864,15 +864,17 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
}
#endif
- r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (is_page_fault_stale(vcpu, fault))
- goto out_unlock;
-
r = make_mmu_pages_available(vcpu);
if (r)
goto out_unlock;
+
+ if (is_page_fault_stale(vcpu, fault)) {
+ r = RET_PF_RETRY;
+ goto out_unlock;
+ }
+
r = FNAME(fetch)(vcpu, fault, &walker);
out_unlock:
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps
2026-07-25 6:52 FAILED: patch "[PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU" failed to apply to 5.15-stable tree gregkh
@ 2026-07-30 19:29 ` Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 2/6] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Sasha Levin
` (4 more replies)
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
1 sibling, 5 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit a42989e7fbb0186d9fee05b29e0ea9cb639d0bd3 ]
Use pte_list_destroy() directly when recycling rmaps instead of bouncing
through kvm_unmap_rmapp() and kvm_zap_rmapp(). Calling kvm_unmap_rmapp()
is unnecessary and odd as it requires passing dummy parameters; passing
NULL for @slot when __rmap_add() already has a valid slot is especially
weird and confusing.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715224226.3749507-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 3e95e39d4fb4..a84d9ddfdc2c 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -1639,7 +1639,7 @@ static void rmap_add(struct kvm_vcpu *vcpu, u64 *spte, gfn_t gfn)
rmap_count = pte_list_add(vcpu, spte, rmap_head);
if (rmap_count > RMAP_RECYCLE_THRESHOLD) {
- kvm_unmap_rmapp(vcpu->kvm, rmap_head, NULL, gfn, sp->role.level, __pte(0));
+ pte_list_destroy(vcpu->kvm, rmap_head);
kvm_flush_remote_tlbs_with_address(
vcpu->kvm, sp->gfn, KVM_PAGES_PER_HPAGE(sp->role.level));
}
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 2/6] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
@ 2026-07-30 19:29 ` Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 3/6] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Sasha Levin
` (3 subsequent siblings)
4 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit 9202aee816c84d69179f94193c5dd321bb0e8530 ]
Rename pte_list_remove() and pte_list_destroy() to kvm_zap_one_rmap_spte()
and kvm_zap_all_rmap_sptes() respectively to document that (a) they zap
SPTEs and (b) to better document how they differ (remove vs. destroy does
not exactly scream "one vs. all").
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715224226.3749507-7-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index a84d9ddfdc2c..415293bc6d1b 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -999,15 +999,16 @@ static void __pte_list_remove(u64 *spte, struct kvm_rmap_head *rmap_head)
}
}
-static void pte_list_remove(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
- u64 *sptep)
+static void kvm_zap_one_rmap_spte(struct kvm *kvm,
+ struct kvm_rmap_head *rmap_head, u64 *sptep)
{
mmu_spte_clear_track_bits(kvm, sptep);
__pte_list_remove(sptep, rmap_head);
}
-/* Return true if rmap existed, false otherwise */
-static bool pte_list_destroy(struct kvm *kvm, struct kvm_rmap_head *rmap_head)
+/* Return true if at least one SPTE was zapped, false otherwise */
+static bool kvm_zap_all_rmap_sptes(struct kvm *kvm,
+ struct kvm_rmap_head *rmap_head)
{
struct pte_list_desc *desc, *next;
int i;
@@ -1434,7 +1435,7 @@ static bool rmap_write_protect(struct kvm_vcpu *vcpu, u64 gfn)
static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
const struct kvm_memory_slot *slot)
{
- return pte_list_destroy(kvm, rmap_head);
+ return kvm_zap_all_rmap_sptes(kvm, rmap_head);
}
static bool kvm_unmap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
@@ -1465,7 +1466,7 @@ static bool kvm_set_pte_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
need_flush = 1;
if (pte_write(pte)) {
- pte_list_remove(kvm, rmap_head, sptep);
+ kvm_zap_one_rmap_spte(kvm, rmap_head, sptep);
goto restart;
} else {
new_spte = kvm_mmu_changed_pte_notifier_make_spte(
@@ -1639,7 +1640,7 @@ static void rmap_add(struct kvm_vcpu *vcpu, u64 *spte, gfn_t gfn)
rmap_count = pte_list_add(vcpu, spte, rmap_head);
if (rmap_count > RMAP_RECYCLE_THRESHOLD) {
- pte_list_destroy(vcpu->kvm, rmap_head);
+ kvm_zap_all_rmap_sptes(vcpu->kvm, rmap_head);
kvm_flush_remote_tlbs_with_address(
vcpu->kvm, sp->gfn, KVM_PAGES_PER_HPAGE(sp->role.level));
}
@@ -5855,7 +5856,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
!kvm_is_reserved_pfn(pfn) &&
sp->role.level < kvm_mmu_max_mapping_level(kvm, slot, sp->gfn,
pfn, PG_LEVEL_NUM)) {
- pte_list_remove(kvm, rmap_head, sptep);
+ kvm_zap_one_rmap_spte(kvm, rmap_head, sptep);
if (kvm_available_flush_tlb_with_range())
kvm_flush_remote_tlbs_with_address(kvm, sp->gfn,
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 3/6] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level()
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 2/6] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Sasha Levin
@ 2026-07-30 19:29 ` Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 4/6] KVM: Rename mmu_notifier_* to mmu_invalidate_* Sasha Levin
` (2 subsequent siblings)
4 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit 65e3b446bcceaac7448cb25a2a5bf4adbcf25fe6 ]
Add a comment to document how host_pfn_mapping_level() can be used safely,
as the line between safe and dangerous is quite thin. E.g. if KVM were
to ever support in-place promotion to create huge pages, consuming the
level is safe if the caller holds mmu_lock and checks that there's an
existing _leaf_ SPTE, but unsafe if the caller only checks that there's a
non-leaf SPTE.
Opportunistically tweak the existing comments to explicitly document why
KVM needs to use READ_ONCE().
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715232107.3775620-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 415293bc6d1b..c00bcfadc068 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2869,6 +2869,31 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
__direct_pte_prefetch(vcpu, sp, sptep);
}
+/*
+ * Lookup the mapping level for @gfn in the current mm.
+ *
+ * WARNING! Use of host_pfn_mapping_level() requires the caller and the end
+ * consumer to be tied into KVM's handlers for MMU notifier events!
+ *
+ * There are several ways to safely use this helper:
+ *
+ * - Check mmu_notifier_retry_hva() after grabbing the mapping level, before
+ * consuming it. In this case, mmu_lock doesn't need to be held during the
+ * lookup, but it does need to be held while checking the MMU notifier.
+ *
+ * - Hold mmu_lock AND ensure there is no in-progress MMU notifier invalidation
+ * event for the hva. This can be done by explicit checking the MMU notifier
+ * or by ensuring that KVM already has a valid mapping that covers the hva.
+ *
+ * - Do not use the result to install new mappings, e.g. use the host mapping
+ * level only to decide whether or not to zap an entry. In this case, it's
+ * not required to hold mmu_lock (though it's highly likely the caller will
+ * want to hold mmu_lock anyways, e.g. to modify SPTEs).
+ *
+ * Note! The lookup can still race with modifications to host page tables, but
+ * the above "rules" ensure KVM will not _consume_ the result of the walk if a
+ * race with the primary MMU occurs.
+ */
static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
const struct kvm_memory_slot *slot)
{
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 4/6] KVM: Rename mmu_notifier_* to mmu_invalidate_*
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 2/6] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 3/6] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Sasha Levin
@ 2026-07-30 19:29 ` Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 5/6] KVM: x86/mmu: Split out TDP MMU page fault handling Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map() Sasha Levin
4 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: Chao Peng, Paolo Bonzini, Sasha Levin
From: Chao Peng <chao.p.peng@linux.intel.com>
[ Upstream commit 20ec3ebd707c77fb9b11b37193449193d4649f33 ]
The motivation of this renaming is to make these variables and related
helper functions less mmu_notifier bound and can also be used for non
mmu_notifier based page invalidation. mmu_invalidate_* was chosen to
better describe the purpose of 'invalidating' a page that those
variables are used for.
- mmu_notifier_seq/range_start/range_end are renamed to
mmu_invalidate_seq/range_start/range_end.
- mmu_notifier_retry{_hva} helper functions are renamed to
mmu_invalidate_retry{_hva}.
- mmu_notifier_count is renamed to mmu_invalidate_in_progress to
avoid confusion with mn_active_invalidate_count.
- While here, also update kvm_inc/dec_notifier_count() to
kvm_mmu_invalidate_begin/end() to match the change for
mmu_notifier_count.
No functional change intended.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Message-Id: <20220816125322.1110439-3-chao.p.peng@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/arm64/kvm/mmu.c | 10 ++--
arch/mips/kvm/mmu.c | 12 ++---
arch/powerpc/include/asm/kvm_book3s_64.h | 2 +-
arch/powerpc/kvm/book3s_64_mmu_host.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 +--
arch/powerpc/kvm/book3s_64_vio_hv.c | 2 +-
arch/powerpc/kvm/book3s_hv_nested.c | 2 +-
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 8 ++--
arch/powerpc/kvm/e500_mmu_host.c | 4 +-
arch/x86/kvm/mmu/mmu.c | 14 +++---
arch/x86/kvm/mmu/paging_tmpl.h | 6 +--
include/linux/kvm_host.h | 60 ++++++++++++------------
virt/kvm/kvm_main.c | 49 +++++++++----------
14 files changed, 93 insertions(+), 90 deletions(-)
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index db667b4ad103..7d04f9670644 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -881,7 +881,7 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
* THP doesn't start to split while we are adjusting the
* refcounts.
*
- * We are sure this doesn't happen, because mmu_notifier_retry
+ * We are sure this doesn't happen, because mmu_invalidate_retry
* was successful and we are holding the mmu_lock, so if this
* THP is trying to split, it will be blocked in the mmu
* notifier before touching any of the pages, specifically
@@ -1074,14 +1074,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
gfn = fault_ipa >> PAGE_SHIFT;
/*
- * Read mmu_notifier_seq so that KVM can detect if the results of
+ * Read mmu_invalidate_seq so that KVM can detect if the results of
* vma_lookup() or __gfn_to_pfn_memslot() become stale prior to
* acquiring kvm->mmu_lock.
*
* Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs
- * with the smp_wmb() in kvm_dec_notifier_count().
+ * with the smp_wmb() in kvm_mmu_invalidate_end().
*/
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
mmap_read_unlock(current->mm);
pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL,
@@ -1118,7 +1118,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
spin_lock(&kvm->mmu_lock);
pgt = vcpu->arch.hw_mmu->pgt;
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
goto out_unlock;
/*
diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c
index 72b524945227..755b7a0a5595 100644
--- a/arch/mips/kvm/mmu.c
+++ b/arch/mips/kvm/mmu.c
@@ -615,17 +615,17 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa,
* Used to check for invalidations in progress, of the pfn that is
* returned by pfn_to_pfn_prot below.
*/
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
/*
- * Ensure the read of mmu_notifier_seq isn't reordered with PTE reads in
- * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't
+ * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads
+ * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't
* risk the page we get a reference to getting unmapped before we have a
- * chance to grab the mmu_lock without mmu_notifier_retry() noticing.
+ * chance to grab the mmu_lock without mmu_invalidate_retry() noticing.
*
* This smp_rmb() pairs with the effective smp_wmb() of the combination
* of the pte_unmap_unlock() after the PTE is zapped, and the
* spin_lock() in kvm_mmu_notifier_invalidate_<page|range_end>() before
- * mmu_notifier_seq is incremented.
+ * mmu_invalidate_seq is incremented.
*/
smp_rmb();
@@ -638,7 +638,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa,
spin_lock(&kvm->mmu_lock);
/* Check if an invalidation has taken place since we got pfn */
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
/*
* This can happen when mappings are changed asynchronously, but
* also synchronously if a COW is triggered by
diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h
index eaf3a562bf1e..0b418df7e3f8 100644
--- a/arch/powerpc/include/asm/kvm_book3s_64.h
+++ b/arch/powerpc/include/asm/kvm_book3s_64.h
@@ -673,7 +673,7 @@ static inline pte_t *find_kvm_host_pte(struct kvm *kvm, unsigned long mmu_seq,
VM_WARN(!spin_is_locked(&kvm->mmu_lock),
"%s called with kvm mmu_lock not held \n", __func__);
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
return NULL;
pte = __find_linux_pte(kvm->mm->pgd, ea, NULL, hshift);
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index c3e31fef0be1..1d0f3f9d8508 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -90,7 +90,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte,
unsigned long pfn;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* Get host physical address for gpa */
@@ -151,7 +151,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte,
cpte = kvmppc_mmu_hpte_cache_next(vcpu);
spin_lock(&kvm->mmu_lock);
- if (!cpte || mmu_notifier_retry(kvm, mmu_seq)) {
+ if (!cpte || mmu_invalidate_retry(kvm, mmu_seq)) {
r = -EAGAIN;
goto out_unlock;
}
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index c63e263312a4..42cc2de2e3b5 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -570,7 +570,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu,
return -EFAULT;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
ret = -EFAULT;
@@ -685,7 +685,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu,
/* Check if we might have been invalidated; let the guest retry if so */
ret = RESUME_GUEST;
- if (mmu_notifier_retry(vcpu->kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(vcpu->kvm, mmu_seq)) {
unlock_rmap(rmap);
goto out_unlock;
}
diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
index 16359525a40f..6dc8dad1be99 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -634,7 +634,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
/* Check if we might have been invalidated; let the guest retry if so */
spin_lock(&kvm->mmu_lock);
ret = -EAGAIN;
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
goto out_unlock;
/* Now traverse again under the lock and change the tree */
@@ -824,7 +824,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu,
bool large_enable;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/*
@@ -1185,7 +1185,7 @@ void kvmppc_radix_flush_memslot(struct kvm *kvm,
* Increase the mmu notifier sequence number to prevent any page
* fault that read the memslot earlier from writing a PTE.
*/
- kvm->mmu_notifier_seq++;
+ kvm->mmu_invalidate_seq++;
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index fdeda6a9cff4..26ee88843f76 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -499,7 +499,7 @@ long kvmppc_rm_h_put_tce_indirect(struct kvm_vcpu *vcpu,
/*
* used to check for invalidations in progress
*/
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
stt = kvmppc_find_table(vcpu->kvm, liobn);
diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c
index ddea14e5cb5e..e31b3c20a9d4 100644
--- a/arch/powerpc/kvm/book3s_hv_nested.c
+++ b/arch/powerpc/kvm/book3s_hv_nested.c
@@ -1574,7 +1574,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu,
/* 2. Find the host pte for this L1 guest real address */
/* Used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* See if can find translation in our partition scoped tables for L1 */
diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
index 632b2545072b..95fb2965945f 100644
--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
@@ -216,7 +216,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
g_ptel = ptel;
/* used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* Find the memslot (if any) for this address */
@@ -363,7 +363,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
rmap = real_vmalloc_addr(rmap);
lock_rmap(rmap);
/* Check for pending invalidations under the rmap chain lock */
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
/* inval in progress, write a non-present HPTE */
pteh |= HPTE_V_ABSENT;
pteh &= ~HPTE_V_VALID;
@@ -929,7 +929,7 @@ static long kvmppc_do_h_page_init_zero(struct kvm_vcpu *vcpu,
int i;
/* Used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
arch_spin_lock(&kvm->mmu_lock.rlock.raw_lock);
@@ -957,7 +957,7 @@ static long kvmppc_do_h_page_init_copy(struct kvm_vcpu *vcpu,
long ret = H_SUCCESS;
/* Used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
arch_spin_lock(&kvm->mmu_lock.rlock.raw_lock);
diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c
index 7f16afc331ef..05668e964140 100644
--- a/arch/powerpc/kvm/e500_mmu_host.c
+++ b/arch/powerpc/kvm/e500_mmu_host.c
@@ -339,7 +339,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
unsigned long flags;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/*
@@ -460,7 +460,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
}
spin_lock(&kvm->mmu_lock);
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
ret = -EAGAIN;
goto out;
}
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index c00bcfadc068..e52940bf1892 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2863,7 +2863,7 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
* If addresses are being invalidated, skip prefetching to avoid
* accidentally prefetching those addresses.
*/
- if (unlikely(vcpu->kvm->mmu_notifier_count))
+ if (unlikely(vcpu->kvm->mmu_invalidate_in_progress))
return;
__direct_pte_prefetch(vcpu, sp, sptep);
@@ -2877,7 +2877,7 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
*
* There are several ways to safely use this helper:
*
- * - Check mmu_notifier_retry_hva() after grabbing the mapping level, before
+ * - Check mmu_invalidate_retry_hva() after grabbing the mapping level, before
* consuming it. In this case, mmu_lock doesn't need to be held during the
* lookup, but it does need to be held while checking the MMU notifier.
*
@@ -2972,7 +2972,7 @@ int kvm_mmu_hugepage_adjust(struct kvm_vcpu *vcpu, gfn_t gfn,
return PG_LEVEL_4K;
/*
- * mmu_notifier_retry() was successful and mmu_lock is held, so
+ * mmu_invalidate_retry() was successful and mmu_lock is held, so
* the pmd can't be split from under us.
*/
mask = KVM_PAGES_PER_HPAGE(level) - 1;
@@ -4018,7 +4018,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
if (r)
return r;
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
smp_rmb();
if (kvm_faultin_pfn(vcpu, prefault, gfn, gpa, &pfn, &hva,
@@ -4035,7 +4035,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
else
write_lock(&vcpu->kvm->mmu_lock);
- if (!is_noslot_pfn(pfn) && mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva))
+ if (!is_noslot_pfn(pfn) && mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, hva))
goto out_unlock;
if (is_tdp_mmu_fault) {
@@ -5767,7 +5767,7 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
write_lock(&kvm->mmu_lock);
- kvm_inc_notifier_count(kvm, gfn_start, gfn_end);
+ kvm_mmu_invalidate_begin(kvm, gfn_start, gfn_end);
if (kvm_memslots_have_rmaps(kvm)) {
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
@@ -5802,7 +5802,7 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
gfn_end - gfn_start);
- kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
+ kvm_mmu_invalidate_end(kvm, gfn_start, gfn_end);
write_unlock(&kvm->mmu_lock);
}
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index a1811f51eda9..6ebabb9afdb2 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -634,7 +634,7 @@ static void FNAME(pte_prefetch)(struct kvm_vcpu *vcpu, struct guest_walker *gw,
* If addresses are being invalidated, skip prefetching to avoid
* accidentally prefetching those addresses.
*/
- if (unlikely(vcpu->kvm->mmu_notifier_count))
+ if (unlikely(vcpu->kvm->mmu_invalidate_in_progress))
return;
if (sp->role.direct)
@@ -894,7 +894,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,
else
max_level = walker.level;
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
smp_rmb();
if (kvm_faultin_pfn(vcpu, prefault, walker.gfn, addr, &pfn, &hva,
@@ -925,7 +925,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,
r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (!is_noslot_pfn(pfn) && mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva))
+ if (!is_noslot_pfn(pfn) && mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, hva))
goto out_unlock;
kvm_mmu_audit(vcpu, AUDIT_PRE_PAGE_FAULT);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 118a3905afb0..24ce84739f03 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -710,10 +710,10 @@ struct kvm {
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
struct mmu_notifier mmu_notifier;
- unsigned long mmu_notifier_seq;
- long mmu_notifier_count;
- unsigned long mmu_notifier_range_start;
- unsigned long mmu_notifier_range_end;
+ unsigned long mmu_invalidate_seq;
+ long mmu_invalidate_in_progress;
+ unsigned long mmu_invalidate_range_start;
+ unsigned long mmu_invalidate_range_end;
#endif
struct list_head devices;
u64 manual_dirty_log_protect;
@@ -1100,10 +1100,10 @@ void kvm_mmu_free_memory_cache(struct kvm_mmu_memory_cache *mc);
void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc);
#endif
-void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end);
-void kvm_dec_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end);
+void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start,
+ unsigned long end);
+void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start,
+ unsigned long end);
long kvm_arch_dev_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg);
@@ -1631,42 +1631,44 @@ extern const struct kvm_stats_header kvm_vcpu_stats_header;
extern const struct _kvm_stats_desc kvm_vcpu_stats_desc[];
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
-static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
+static inline int mmu_invalidate_retry(struct kvm *kvm, unsigned long mmu_seq)
{
- if (unlikely(kvm->mmu_notifier_count))
+ if (unlikely(kvm->mmu_invalidate_in_progress))
return 1;
/*
- * Ensure the read of mmu_notifier_count happens before the read
- * of mmu_notifier_seq. This interacts with the smp_wmb() in
- * mmu_notifier_invalidate_range_end to make sure that the caller
- * either sees the old (non-zero) value of mmu_notifier_count or
- * the new (incremented) value of mmu_notifier_seq.
- * PowerPC Book3s HV KVM calls this under a per-page lock
- * rather than under kvm->mmu_lock, for scalability, so
- * can't rely on kvm->mmu_lock to keep things ordered.
+ * Ensure the read of mmu_invalidate_in_progress happens before
+ * the read of mmu_invalidate_seq. This interacts with the
+ * smp_wmb() in mmu_notifier_invalidate_range_end to make sure
+ * that the caller either sees the old (non-zero) value of
+ * mmu_invalidate_in_progress or the new (incremented) value of
+ * mmu_invalidate_seq.
+ *
+ * PowerPC Book3s HV KVM calls this under a per-page lock rather
+ * than under kvm->mmu_lock, for scalability, so can't rely on
+ * kvm->mmu_lock to keep things ordered.
*/
smp_rmb();
- if (kvm->mmu_notifier_seq != mmu_seq)
+ if (kvm->mmu_invalidate_seq != mmu_seq)
return 1;
return 0;
}
-static inline int mmu_notifier_retry_hva(struct kvm *kvm,
- unsigned long mmu_seq,
- unsigned long hva)
+static inline int mmu_invalidate_retry_hva(struct kvm *kvm,
+ unsigned long mmu_seq,
+ unsigned long hva)
{
lockdep_assert_held(&kvm->mmu_lock);
/*
- * If mmu_notifier_count is non-zero, then the range maintained by
- * kvm_mmu_notifier_invalidate_range_start contains all addresses that
- * might be being invalidated. Note that it may include some false
+ * If mmu_invalidate_in_progress is non-zero, then the range maintained
+ * by kvm_mmu_notifier_invalidate_range_start contains all addresses
+ * that might be being invalidated. Note that it may include some false
* positives, due to shortcuts when handing concurrent invalidations.
*/
- if (unlikely(kvm->mmu_notifier_count) &&
- hva >= kvm->mmu_notifier_range_start &&
- hva < kvm->mmu_notifier_range_end)
+ if (unlikely(kvm->mmu_invalidate_in_progress) &&
+ hva >= kvm->mmu_invalidate_range_start &&
+ hva < kvm->mmu_invalidate_range_end)
return 1;
- if (kvm->mmu_notifier_seq != mmu_seq)
+ if (kvm->mmu_invalidate_seq != mmu_seq)
return 1;
return 0;
}
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 79422642ae00..1482aeebe745 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -665,30 +665,31 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn,
/*
* .change_pte() must be surrounded by .invalidate_range_{start,end}().
- * If mmu_notifier_count is zero, then no in-progress invalidations,
- * including this one, found a relevant memslot at start(); rechecking
- * memslots here is unnecessary. Note, a false positive (count elevated
- * by a different invalidation) is sub-optimal but functionally ok.
+ * If mmu_invalidate_in_progress is zero, then no in-progress
+ * invalidations, including this one, found a relevant memslot at
+ * start(); rechecking memslots here is unnecessary. Note, a false
+ * positive (count elevated by a different invalidation) is sub-optimal
+ * but functionally ok.
*/
WARN_ON_ONCE(!READ_ONCE(kvm->mn_active_invalidate_count));
- if (!READ_ONCE(kvm->mmu_notifier_count))
+ if (!READ_ONCE(kvm->mmu_invalidate_in_progress))
return;
kvm_handle_hva_range(mn, address, address + 1, pte, kvm_change_spte_gfn);
}
-void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end)
+void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start,
+ unsigned long end)
{
/*
* The count increase must become visible at unlock time as no
* spte can be established without taking the mmu_lock and
* count is also read inside the mmu_lock critical section.
*/
- kvm->mmu_notifier_count++;
- if (likely(kvm->mmu_notifier_count == 1)) {
- kvm->mmu_notifier_range_start = start;
- kvm->mmu_notifier_range_end = end;
+ kvm->mmu_invalidate_in_progress++;
+ if (likely(kvm->mmu_invalidate_in_progress == 1)) {
+ kvm->mmu_invalidate_range_start = start;
+ kvm->mmu_invalidate_range_end = end;
} else {
/*
* Fully tracking multiple concurrent ranges has dimishing
@@ -699,10 +700,10 @@ void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
* accumulate and persist until all outstanding invalidates
* complete.
*/
- kvm->mmu_notifier_range_start =
- min(kvm->mmu_notifier_range_start, start);
- kvm->mmu_notifier_range_end =
- max(kvm->mmu_notifier_range_end, end);
+ kvm->mmu_invalidate_range_start =
+ min(kvm->mmu_invalidate_range_start, start);
+ kvm->mmu_invalidate_range_end =
+ max(kvm->mmu_invalidate_range_end, end);
}
}
@@ -715,7 +716,7 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
.end = range->end,
.pte = __pte(0),
.handler = kvm_unmap_gfn_range,
- .on_lock = kvm_inc_notifier_count,
+ .on_lock = kvm_mmu_invalidate_begin,
.on_unlock = kvm_arch_guest_memory_reclaimed,
.flush_on_ret = true,
.may_block = mmu_notifier_range_blockable(range),
@@ -726,7 +727,7 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
/*
* Prevent memslot modification between range_start() and range_end()
* so that conditionally locking provides the same result in both
- * functions. Without that guarantee, the mmu_notifier_count
+ * functions. Without that guarantee, the mmu_invalidate_in_progress
* adjustments will be imbalanced.
*
* Pairs with the decrement in range_end().
@@ -740,22 +741,22 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
return 0;
}
-void kvm_dec_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end)
+void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start,
+ unsigned long end)
{
/*
* This sequence increase will notify the kvm page fault that
* the page that is going to be mapped in the spte could have
* been freed.
*/
- kvm->mmu_notifier_seq++;
+ kvm->mmu_invalidate_seq++;
smp_wmb();
/*
* The above sequence increase must be visible before the
* below count decrease, which is ensured by the smp_wmb above
- * in conjunction with the smp_rmb in mmu_notifier_retry().
+ * in conjunction with the smp_rmb in mmu_invalidate_retry().
*/
- kvm->mmu_notifier_count--;
+ kvm->mmu_invalidate_in_progress--;
}
static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
@@ -767,7 +768,7 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
.end = range->end,
.pte = __pte(0),
.handler = (void *)kvm_null_fn,
- .on_lock = kvm_dec_notifier_count,
+ .on_lock = kvm_mmu_invalidate_end,
.on_unlock = (void *)kvm_null_fn,
.flush_on_ret = false,
.may_block = mmu_notifier_range_blockable(range),
@@ -788,7 +789,7 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
if (wake)
rcuwait_wake_up(&kvm->mn_memslots_update_rcuwait);
- BUG_ON(kvm->mmu_notifier_count < 0);
+ BUG_ON(kvm->mmu_invalidate_in_progress < 0);
}
static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn,
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 5/6] KVM: x86/mmu: Split out TDP MMU page fault handling
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
` (2 preceding siblings ...)
2026-07-30 19:29 ` [PATCH 5.15.y 4/6] KVM: Rename mmu_notifier_* to mmu_invalidate_* Sasha Levin
@ 2026-07-30 19:29 ` Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map() Sasha Levin
4 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: David Matlack, Isaku Yamahata, Paolo Bonzini, Sasha Levin
From: David Matlack <dmatlack@google.com>
[ Upstream commit 9aa8ab43b38146029de807a8ff2696f51e15b226 ]
Split out the page fault handling for the TDP MMU to a separate
function. This creates some duplicate code, but makes the TDP MMU fault
handler simpler to read by eliminating branches and will enable future
cleanups by allowing the TDP MMU and non-TDP MMU fault paths to diverge.
Only compile in the TDP MMU fault handler for 64-bit builds since
kvm_tdp_mmu_map() does not exist in 32-bit builds.
No functional change intended.
Signed-off-by: David Matlack <dmatlack@google.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220921173546.2674386-9-dmatlack@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 84 ++++++++++++++++++++++++++++++++----------
1 file changed, 64 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index e52940bf1892..04e814816b17 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -3997,7 +3997,6 @@ static bool kvm_faultin_pfn(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn,
static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
bool prefault, int max_level, bool is_tdp)
{
- bool is_tdp_mmu_fault = is_tdp_mmu(vcpu->arch.mmu);
bool write = error_code & PFERR_WRITE_MASK;
bool map_writable;
@@ -4029,31 +4028,20 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
return r;
r = RET_PF_RETRY;
-
- if (is_tdp_mmu_fault)
- read_lock(&vcpu->kvm->mmu_lock);
- else
- write_lock(&vcpu->kvm->mmu_lock);
+ write_lock(&vcpu->kvm->mmu_lock);
if (!is_noslot_pfn(pfn) && mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, hva))
goto out_unlock;
- if (is_tdp_mmu_fault) {
- r = kvm_tdp_mmu_map(vcpu, gpa, error_code, map_writable, max_level,
- pfn, prefault);
- } else {
- r = make_mmu_pages_available(vcpu);
- if (r)
- goto out_unlock;
- r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
- prefault, is_tdp);
- }
+ r = make_mmu_pages_available(vcpu);
+ if (r)
+ goto out_unlock;
+
+ r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
+ prefault, is_tdp);
out_unlock:
- if (is_tdp_mmu_fault)
- read_unlock(&vcpu->kvm->mmu_lock);
- else
- write_unlock(&vcpu->kvm->mmu_lock);
+ write_unlock(&vcpu->kvm->mmu_lock);
kvm_release_pfn_clean(pfn);
return r;
}
@@ -4101,6 +4089,56 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code,
}
EXPORT_SYMBOL_GPL(kvm_handle_page_fault);
+#ifdef CONFIG_X86_64
+static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa,
+ u32 error_code, bool prefault, int max_level)
+{
+ bool write = error_code & PFERR_WRITE_MASK;
+ bool map_writable;
+
+ gfn_t gfn = gpa >> PAGE_SHIFT;
+ unsigned long mmu_seq;
+ kvm_pfn_t pfn;
+ hva_t hva;
+ int r;
+
+ if (page_fault_handle_page_track(vcpu, error_code, gfn))
+ return RET_PF_EMULATE;
+
+ r = fast_page_fault(vcpu, gpa, error_code);
+ if (r != RET_PF_INVALID)
+ return r;
+
+ r = mmu_topup_memory_caches(vcpu, false);
+ if (r)
+ return r;
+
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
+ smp_rmb();
+
+ if (kvm_faultin_pfn(vcpu, prefault, gfn, gpa, &pfn, &hva,
+ write, &map_writable, &r))
+ return r;
+
+ if (handle_abnormal_pfn(vcpu, 0, gfn, pfn, ACC_ALL, &r))
+ return r;
+
+ r = RET_PF_RETRY;
+ read_lock(&vcpu->kvm->mmu_lock);
+
+ if (!is_noslot_pfn(pfn) && mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, hva))
+ goto out_unlock;
+
+ r = kvm_tdp_mmu_map(vcpu, gpa, error_code, map_writable, max_level,
+ pfn, prefault);
+
+out_unlock:
+ read_unlock(&vcpu->kvm->mmu_lock);
+ kvm_release_pfn_clean(pfn);
+ return r;
+}
+#endif
+
int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
bool prefault)
{
@@ -4116,6 +4154,12 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
break;
}
+#ifdef CONFIG_X86_64
+ if (is_tdp_mmu(vcpu->arch.mmu))
+ return kvm_tdp_mmu_page_fault(vcpu, gpa, error_code, prefault,
+ max_level);
+#endif
+
return direct_page_fault(vcpu, gpa, error_code, prefault,
max_level, true);
}
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
` (3 preceding siblings ...)
2026-07-30 19:29 ` [PATCH 5.15.y 5/6] KVM: x86/mmu: Split out TDP MMU page fault handling Sasha Levin
@ 2026-07-30 19:29 ` Sasha Levin
2026-08-06 16:19 ` Harshit Mogalapalli
2026-08-21 12:38 ` Kenta Akagi
4 siblings, 2 replies; 21+ messages in thread
From: Sasha Levin @ 2026-07-30 19:29 UTC (permalink / raw)
To: stable; +Cc: David Matlack, Isaku Yamahata, Paolo Bonzini, Sasha Levin
From: David Matlack <dmatlack@google.com>
[ Upstream commit 6c882ef4fc7bd99b67ad152e75428b669281c521 ]
Rename __direct_map() to direct_map() since the leading underscores are
unnecessary. This also makes the page fault handler names more
consistent: kvm_tdp_mmu_page_fault() calls kvm_tdp_mmu_map() and
direct_page_fault() calls direct_map().
Opportunistically make some trivial cleanups to comments that had to be
modified anyway since they mentioned __direct_map(). Specifically, use
"()" when referring to functions, and include kvm_tdp_mmu_map() among
the various callers of disallowed_hugepage_adjust().
No functional change intended.
Signed-off-by: David Matlack <dmatlack@google.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220921173546.2674386-11-dmatlack@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kvm/mmu/mmu.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 04e814816b17..c274ab0ceff9 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2991,11 +2991,11 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
is_shadow_present_pte(spte) &&
!is_large_pte(spte)) {
/*
- * A small SPTE exists for this pfn, but FNAME(fetch)
- * and __direct_map would like to create a large PTE
- * instead: just force them to go down another level,
- * patching back for them into pfn the next 9 bits of
- * the address.
+ * A small SPTE exists for this pfn, but FNAME(fetch),
+ * direct_map(), or kvm_tdp_mmu_map() would like to create a
+ * large PTE instead: just force them to go down another level,
+ * patching back for them into pfn the next 9 bits of the
+ * address.
*/
u64 page_mask = KVM_PAGES_PER_HPAGE(level) -
KVM_PAGES_PER_HPAGE(level - 1);
@@ -3004,9 +3004,9 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
}
}
-static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
- int map_writable, int max_level, kvm_pfn_t pfn,
- bool prefault, bool is_tdp)
+static int direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
+ int map_writable, int max_level, kvm_pfn_t pfn,
+ bool prefault, bool is_tdp)
{
bool nx_huge_page_workaround_enabled = is_nx_huge_page_enabled();
bool write = error_code & PFERR_WRITE_MASK;
@@ -4037,8 +4037,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
if (r)
goto out_unlock;
- r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
- prefault, is_tdp);
+ r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
+ prefault, is_tdp);
out_unlock:
write_unlock(&vcpu->kvm->mmu_lock);
--
2.53.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-07-30 19:29 ` [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map() Sasha Levin
@ 2026-08-06 16:19 ` Harshit Mogalapalli
2026-08-06 16:50 ` Sasha Levin
2026-08-21 12:38 ` Kenta Akagi
1 sibling, 1 reply; 21+ messages in thread
From: Harshit Mogalapalli @ 2026-08-06 16:19 UTC (permalink / raw)
To: Sasha Levin, stable; +Cc: David Matlack, Isaku Yamahata, Paolo Bonzini
Hi Sasha
On 31/07/26 12:59 am, Sasha Levin wrote:
> From: David Matlack <dmatlack@google.com>
>
> [ Upstream commit 6c882ef4fc7bd99b67ad152e75428b669281c521 ]
>
> Rename __direct_map() to direct_map() since the leading underscores are
> unnecessary. This also makes the page fault handler names more
> consistent: kvm_tdp_mmu_page_fault() calls kvm_tdp_mmu_map() and
> direct_page_fault() calls direct_map().
>
> Opportunistically make some trivial cleanups to comments that had to be
> modified anyway since they mentioned __direct_map(). Specifically, use
> "()" when referring to functions, and include kvm_tdp_mmu_map() among
> the various callers of disallowed_hugepage_adjust().
>
> No functional change intended.
>
> Signed-off-by: David Matlack <dmatlack@google.com>
> Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20220921173546.2674386-11-dmatlack@google.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
> Signed-off-by: Sasha Levin <sashal@kernel.org>
I don't see them in queue-5.15 , FYI.
Also I think along with 6 prerequisites and final 7th patch, we also
might need: commit: 9f33697ac7ff ("KVM: x86/mmu: Stop needlessly making
MMU pages available for TDP MMU faults") should be a clean cherru-pick.
thanks,
harshit
> ---
> arch/x86/kvm/mmu/mmu.c | 20 ++++++++++----------
> 1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index 04e814816b17..c274ab0ceff9 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -2991,11 +2991,11 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
> is_shadow_present_pte(spte) &&
> !is_large_pte(spte)) {
> /*
> - * A small SPTE exists for this pfn, but FNAME(fetch)
> - * and __direct_map would like to create a large PTE
> - * instead: just force them to go down another level,
> - * patching back for them into pfn the next 9 bits of
> - * the address.
> + * A small SPTE exists for this pfn, but FNAME(fetch),
> + * direct_map(), or kvm_tdp_mmu_map() would like to create a
> + * large PTE instead: just force them to go down another level,
> + * patching back for them into pfn the next 9 bits of the
> + * address.
> */
> u64 page_mask = KVM_PAGES_PER_HPAGE(level) -
> KVM_PAGES_PER_HPAGE(level - 1);
> @@ -3004,9 +3004,9 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
> }
> }
>
> -static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> - int map_writable, int max_level, kvm_pfn_t pfn,
> - bool prefault, bool is_tdp)
> +static int direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> + int map_writable, int max_level, kvm_pfn_t pfn,
> + bool prefault, bool is_tdp)
> {
> bool nx_huge_page_workaround_enabled = is_nx_huge_page_enabled();
> bool write = error_code & PFERR_WRITE_MASK;
> @@ -4037,8 +4037,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> if (r)
> goto out_unlock;
>
> - r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
> - prefault, is_tdp);
> + r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
> + prefault, is_tdp);
>
> out_unlock:
> write_unlock(&vcpu->kvm->mmu_lock);
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-08-06 16:19 ` Harshit Mogalapalli
@ 2026-08-06 16:50 ` Sasha Levin
0 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-08-06 16:50 UTC (permalink / raw)
To: Harshit Mogalapalli; +Cc: stable, David Matlack, Isaku Yamahata, Paolo Bonzini
On Thu, Aug 06, 2026 at 09:49:23PM +0530, Harshit Mogalapalli wrote:
>Hi Sasha
>
>On 31/07/26 12:59 am, Sasha Levin wrote:
>>From: David Matlack <dmatlack@google.com>
>>
>>[ Upstream commit 6c882ef4fc7bd99b67ad152e75428b669281c521 ]
>>
>>Rename __direct_map() to direct_map() since the leading underscores are
>>unnecessary. This also makes the page fault handler names more
>>consistent: kvm_tdp_mmu_page_fault() calls kvm_tdp_mmu_map() and
>>direct_page_fault() calls direct_map().
>>
>>Opportunistically make some trivial cleanups to comments that had to be
>>modified anyway since they mentioned __direct_map(). Specifically, use
>>"()" when referring to functions, and include kvm_tdp_mmu_map() among
>>the various callers of disallowed_hugepage_adjust().
>>
>>No functional change intended.
>>
>>Signed-off-by: David Matlack <dmatlack@google.com>
>>Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>Message-Id: <20220921173546.2674386-11-dmatlack@google.com>
>>Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
>>Signed-off-by: Sasha Levin <sashal@kernel.org>
>
>I don't see them in queue-5.15 , FYI.
I don't think Greg got to these yet. We normally let these sit on the mailing
list for a bit, for review.
>Also I think along with 6 prerequisites and final 7th patch, we also
>might need: commit: 9f33697ac7ff ("KVM: x86/mmu: Stop needlessly
>making MMU pages available for TDP MMU faults") should be a clean
>cherru-pick.
Possibly, but there's no fixes or stable tag on it, so without an ack from the
KVM folks it won't go in.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-07-30 19:29 ` [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map() Sasha Levin
2026-08-06 16:19 ` Harshit Mogalapalli
@ 2026-08-21 12:38 ` Kenta Akagi
2026-08-23 3:42 ` Sasha Levin
1 sibling, 1 reply; 21+ messages in thread
From: Kenta Akagi @ 2026-08-21 12:38 UTC (permalink / raw)
To: sashal, stable; +Cc: dmatlack, isaku.yamahata, pbonzini
Hi Sasha,
On 2026/07/31 4:29, Sasha Levin wrote:
> From: David Matlack <dmatlack@google.com>
>
> [ Upstream commit 6c882ef4fc7bd99b67ad152e75428b669281c521 ]
>
> Rename __direct_map() to direct_map() since the leading underscores are
> unnecessary. This also makes the page fault handler names more
> consistent: kvm_tdp_mmu_page_fault() calls kvm_tdp_mmu_map() and
> direct_page_fault() calls direct_map().
>
> Opportunistically make some trivial cleanups to comments that had to be
> modified anyway since they mentioned __direct_map(). Specifically, use
> "()" when referring to functions, and include kvm_tdp_mmu_map() among
> the various callers of disallowed_hugepage_adjust().
>
> No functional change intended.
>
> Signed-off-by: David Matlack <dmatlack@google.com>
> Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20220921173546.2674386-11-dmatlack@google.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Maybe I'm missing something, but commit 2abd5287f083 ("KVM: x86: Check for
invalid/obsolete root *after* making MMU pages available") cannot be backported
as-is. It depends on is_page_fault_stale(), introduced by commit a955cad84cda
("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update"),
which is missing in 5.15.y.
Furthermore, a955cad84cda is required to fix a regression in 5.15.y when
applying the CVE-2026-46113 and CVE-2026-53359 fixes, but its backport remains
pending [1][2].
I plan to send a manual backport of a955cad84cda and 2abd5287f083.
In doing so, the interface of is_page_fault_stale() will end up incompatible
with mainline. While not ideal, making the interface compatible requires
struct kvm_page_fault and its 50+ prerequisite commits.
[1] https://lore.kernel.org/all/20260626174620.1819772-7-pbonzini@redhat.com/
[2] https://lore.kernel.org/all/CABgObfaKqT0gMUB_XRP5DW5HmpgaoTgmD-A8_zQVG2Vp5xZE4g@mail.gmail.com/
Thanks,
akagi
> Signed-off-by: Sasha Levin <sashal@kernel.org>
> ---
> arch/x86/kvm/mmu/mmu.c | 20 ++++++++++----------
> 1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index 04e814816b17..c274ab0ceff9 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -2991,11 +2991,11 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
> is_shadow_present_pte(spte) &&
> !is_large_pte(spte)) {
> /*
> - * A small SPTE exists for this pfn, but FNAME(fetch)
> - * and __direct_map would like to create a large PTE
> - * instead: just force them to go down another level,
> - * patching back for them into pfn the next 9 bits of
> - * the address.
> + * A small SPTE exists for this pfn, but FNAME(fetch),
> + * direct_map(), or kvm_tdp_mmu_map() would like to create a
> + * large PTE instead: just force them to go down another level,
> + * patching back for them into pfn the next 9 bits of the
> + * address.
> */
> u64 page_mask = KVM_PAGES_PER_HPAGE(level) -
> KVM_PAGES_PER_HPAGE(level - 1);
> @@ -3004,9 +3004,9 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
> }
> }
>
> -static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> - int map_writable, int max_level, kvm_pfn_t pfn,
> - bool prefault, bool is_tdp)
> +static int direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> + int map_writable, int max_level, kvm_pfn_t pfn,
> + bool prefault, bool is_tdp)
> {
> bool nx_huge_page_workaround_enabled = is_nx_huge_page_enabled();
> bool write = error_code & PFERR_WRITE_MASK;
> @@ -4037,8 +4037,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
> if (r)
> goto out_unlock;
>
> - r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
> - prefault, is_tdp);
> + r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
> + prefault, is_tdp);
>
> out_unlock:
> write_unlock(&vcpu->kvm->mmu_lock);
> --
> 2.53.0
>
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-08-21 12:38 ` Kenta Akagi
@ 2026-08-23 3:42 ` Sasha Levin
0 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-08-23 3:42 UTC (permalink / raw)
To: stable; +Cc: Sasha Levin, Kenta Akagi, dmatlack, isaku.yamahata, pbonzini
On Fri, Aug 21, 2026 at 12:38:49PM +0000, Kenta Akagi wrote:
>Maybe I'm missing something, but commit 2abd5287f083 ("KVM: x86: Check for
>invalid/obsolete root *after* making MMU pages available") cannot be backported
>as-is. It depends on is_page_fault_stale(), introduced by commit a955cad84cda
>("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update"),
>which is missing in 5.15.y.
You're not missing anything - you're right.
I've dropped all six from the 5.15 queue.
>I plan to send a manual backport of a955cad84cda and 2abd5287f083.
Please do, and thanks for offering.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561
2026-07-25 6:52 FAILED: patch "[PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU" failed to apply to 5.15-stable tree gregkh
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 1/8] KVM: x86/mmu: Retry page fault if root is invalidated by memslot update Kenta Akagi
` (8 more replies)
1 sibling, 9 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Kenta Akagi
This is v2 of the backport to fix CVE-2026-64561, v1 is Sasha's series [1].
Patches 2-7 in this series correspond to patches 1-6 in v1; Patches 1 and 8 are new in v2.
Changes in v2:
- Added Patch 1 to backport is_page_fault_stale() as a prerequisite
for Patch 8 and for the CVE-2026-46113 and CVE-2026-53359 fixes,
which will be sent in a separate series [2].
- Added Patch 8 as the fix for CVE-2026-64561.
- Adjusted Patch 5 to rename mmu_notifier_retry_hva() inside
is_page_fault_stale().
- Adjusted Patch 6 to use is_page_fault_stale() in
kvm_tdp_mmu_page_fault().
[1] https://lore.kernel.org/all/20260730192933.3123642-1-sashal@kernel.org/
[2] https://lore.kernel.org/all/CABgObfaKqT0gMUB_XRP5DW5HmpgaoTgmD-A8_zQVG2Vp5xZE4g@mail.gmail.com/
Chao Peng (1):
KVM: Rename mmu_notifier_* to mmu_invalidate_*
David Matlack (2):
KVM: x86/mmu: Split out TDP MMU page fault handling
KVM: x86/mmu: Rename __direct_map() to direct_map()
Sean Christopherson (5):
KVM: x86/mmu: Retry page fault if root is invalidated by memslot
update
KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps
KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap
SPTEs
KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level()
KVM: x86: Check for invalid/obsolete root *after* making MMU pages
available
arch/arm64/kvm/mmu.c | 10 +-
arch/mips/kvm/mmu.c | 12 +-
arch/powerpc/include/asm/kvm_book3s_64.h | 2 +-
arch/powerpc/kvm/book3s_64_mmu_host.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 +-
arch/powerpc/kvm/book3s_64_vio_hv.c | 2 +-
arch/powerpc/kvm/book3s_hv_nested.c | 2 +-
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 8 +-
arch/powerpc/kvm/e500_mmu_host.c | 4 +-
arch/x86/kvm/mmu/mmu.c | 175 +++++++++++++++++------
arch/x86/kvm/mmu/paging_tmpl.h | 13 +-
include/linux/kvm_host.h | 60 ++++----
virt/kvm/kvm_main.c | 49 +++----
14 files changed, 223 insertions(+), 128 deletions(-)
--
2.51.2
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 1/8] KVM: x86/mmu: Retry page fault if root is invalidated by memslot update
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 2/8] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Kenta Akagi
` (7 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Ben Gardon, Paolo Bonzini, Kenta Akagi
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit a955cad84cdaffa282b3cf8f5ce69e9e5655e585 ]
Bail from the page fault handler if the root shadow page was obsoleted by
a memslot update. Do the check _after_ acuiring mmu_lock, as the TDP MMU
doesn't rely on the memslot/MMU generation, and instead relies on the
root being explicit marked invalid by kvm_mmu_zap_all_fast(), which takes
mmu_lock for write.
For the TDP MMU, inserting a SPTE into an obsolete root can leak a SP if
kvm_tdp_mmu_zap_invalidated_roots() has already zapped the SP, i.e. has
moved past the gfn associated with the SP.
For other MMUs, the resulting behavior is far more convoluted, though
unlikely to be truly problematic. Installing SPs/SPTEs into the obsolete
root isn't directly problematic, as the obsolete root will be unloaded
and dropped before the vCPU re-enters the guest. But because the legacy
MMU tracks shadow pages by their role, any SP created by the fault can
can be reused in the new post-reload root. Again, that _shouldn't_ be
problematic as any leaf child SPTEs will be created for the current/valid
memslot generation, and kvm_mmu_get_page() will not reuse child SPs from
the old generation as they will be flagged as obsolete. But, given that
continuing with the fault is pointess (the root will be unloaded), apply
the check to all MMUs.
Fixes: b7cccd397f31 ("KVM: x86/mmu: Fast invalidation for TDP MMU")
Cc: stable@vger.kernel.org
Cc: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20211120045046.3940942-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
[backport note: is_page_fault_stale() adapted to take individual parameters
instead of struct kvm_page_fault, since 4326e57ef40a ("KVM: MMU: change
direct_page_fault() arguments to kvm_page_fault") is not in 5.15.y;
required by the following backport of 2abd5287f083 and by 0cb2af2ea66a
to be backported separately]
Assisted-by: Claude:claude-sonnet-4.6
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 22 ++++++++++++++++++++--
arch/x86/kvm/mmu/paging_tmpl.h | 3 ++-
2 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 84eeed9a9615..2fbe71dbab6f 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -1942,7 +1942,11 @@ static void mmu_audit_disable(void) { }
static bool is_obsolete_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
{
- return sp->role.invalid ||
+ if (sp->role.invalid)
+ return true;
+
+ /* TDP MMU pages due not use the MMU generation. */
+ return !sp->tdp_mmu_page &&
unlikely(sp->mmu_valid_gen != kvm->arch.mmu_valid_gen);
}
@@ -3968,6 +3972,20 @@ static bool kvm_faultin_pfn(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn,
return true;
}
+/*
+ * Returns true if the page fault is stale and needs to be retried, i.e. if the
+ * root was invalidated by a memslot update or a relevant mmu_notifier fired.
+ */
+static bool is_page_fault_stale(struct kvm_vcpu *vcpu,
+ kvm_pfn_t pfn, unsigned long mmu_seq, hva_t hva)
+{
+ if (is_obsolete_sp(vcpu->kvm, to_shadow_page(vcpu->arch.mmu->root_hpa)))
+ return true;
+
+ return !is_noslot_pfn(pfn) &&
+ mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva);
+}
+
static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
bool prefault, int max_level, bool is_tdp)
{
@@ -4009,7 +4027,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
else
write_lock(&vcpu->kvm->mmu_lock);
- if (!is_noslot_pfn(pfn) && mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva))
+ if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
goto out_unlock;
if (is_tdp_mmu_fault) {
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index a1811f51eda9..d1e7ac58b735 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -925,7 +925,8 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,
r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (!is_noslot_pfn(pfn) && mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva))
+
+ if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
goto out_unlock;
kvm_mmu_audit(vcpu, AUDIT_PRE_PAGE_FAULT);
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 2/8] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 1/8] KVM: x86/mmu: Retry page fault if root is invalidated by memslot update Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 3/8] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Kenta Akagi
` (6 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin, Kenta Akagi
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit a42989e7fbb0186d9fee05b29e0ea9cb639d0bd3 ]
Use pte_list_destroy() directly when recycling rmaps instead of bouncing
through kvm_unmap_rmapp() and kvm_zap_rmapp(). Calling kvm_unmap_rmapp()
is unnecessary and odd as it requires passing dummy parameters; passing
NULL for @slot when __rmap_add() already has a valid slot is especially
weird and confusing.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715224226.3749507-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 2fbe71dbab6f..0addde562ed6 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -1639,7 +1639,7 @@ static void rmap_add(struct kvm_vcpu *vcpu, u64 *spte, gfn_t gfn)
rmap_count = pte_list_add(vcpu, spte, rmap_head);
if (rmap_count > RMAP_RECYCLE_THRESHOLD) {
- kvm_unmap_rmapp(vcpu->kvm, rmap_head, NULL, gfn, sp->role.level, __pte(0));
+ pte_list_destroy(vcpu->kvm, rmap_head);
kvm_flush_remote_tlbs_with_address(
vcpu->kvm, sp->gfn, KVM_PAGES_PER_HPAGE(sp->role.level));
}
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 3/8] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 1/8] KVM: x86/mmu: Retry page fault if root is invalidated by memslot update Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 2/8] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 4/8] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Kenta Akagi
` (5 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin, Kenta Akagi
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit 9202aee816c84d69179f94193c5dd321bb0e8530 ]
Rename pte_list_remove() and pte_list_destroy() to kvm_zap_one_rmap_spte()
and kvm_zap_all_rmap_sptes() respectively to document that (a) they zap
SPTEs and (b) to better document how they differ (remove vs. destroy does
not exactly scream "one vs. all").
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715224226.3749507-7-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 0addde562ed6..610125291a48 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -999,15 +999,16 @@ static void __pte_list_remove(u64 *spte, struct kvm_rmap_head *rmap_head)
}
}
-static void pte_list_remove(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
- u64 *sptep)
+static void kvm_zap_one_rmap_spte(struct kvm *kvm,
+ struct kvm_rmap_head *rmap_head, u64 *sptep)
{
mmu_spte_clear_track_bits(kvm, sptep);
__pte_list_remove(sptep, rmap_head);
}
-/* Return true if rmap existed, false otherwise */
-static bool pte_list_destroy(struct kvm *kvm, struct kvm_rmap_head *rmap_head)
+/* Return true if at least one SPTE was zapped, false otherwise */
+static bool kvm_zap_all_rmap_sptes(struct kvm *kvm,
+ struct kvm_rmap_head *rmap_head)
{
struct pte_list_desc *desc, *next;
int i;
@@ -1434,7 +1435,7 @@ static bool rmap_write_protect(struct kvm_vcpu *vcpu, u64 gfn)
static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
const struct kvm_memory_slot *slot)
{
- return pte_list_destroy(kvm, rmap_head);
+ return kvm_zap_all_rmap_sptes(kvm, rmap_head);
}
static bool kvm_unmap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
@@ -1465,7 +1466,7 @@ static bool kvm_set_pte_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,
need_flush = 1;
if (pte_write(pte)) {
- pte_list_remove(kvm, rmap_head, sptep);
+ kvm_zap_one_rmap_spte(kvm, rmap_head, sptep);
goto restart;
} else {
new_spte = kvm_mmu_changed_pte_notifier_make_spte(
@@ -1639,7 +1640,7 @@ static void rmap_add(struct kvm_vcpu *vcpu, u64 *spte, gfn_t gfn)
rmap_count = pte_list_add(vcpu, spte, rmap_head);
if (rmap_count > RMAP_RECYCLE_THRESHOLD) {
- pte_list_destroy(vcpu->kvm, rmap_head);
+ kvm_zap_all_rmap_sptes(vcpu->kvm, rmap_head);
kvm_flush_remote_tlbs_with_address(
vcpu->kvm, sp->gfn, KVM_PAGES_PER_HPAGE(sp->role.level));
}
@@ -5873,7 +5874,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
!kvm_is_reserved_pfn(pfn) &&
sp->role.level < kvm_mmu_max_mapping_level(kvm, slot, sp->gfn,
pfn, PG_LEVEL_NUM)) {
- pte_list_remove(kvm, rmap_head, sptep);
+ kvm_zap_one_rmap_spte(kvm, rmap_head, sptep);
if (kvm_available_flush_tlb_with_range())
kvm_flush_remote_tlbs_with_address(kvm, sp->gfn,
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 4/8] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level()
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (2 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 3/8] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 5/8] KVM: Rename mmu_notifier_* to mmu_invalidate_* Kenta Akagi
` (4 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Paolo Bonzini, Sasha Levin, Kenta Akagi
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit 65e3b446bcceaac7448cb25a2a5bf4adbcf25fe6 ]
Add a comment to document how host_pfn_mapping_level() can be used safely,
as the line between safe and dangerous is quite thin. E.g. if KVM were
to ever support in-place promotion to create huge pages, consuming the
level is safe if the caller holds mmu_lock and checks that there's an
existing _leaf_ SPTE, but unsafe if the caller only checks that there's a
non-leaf SPTE.
Opportunistically tweak the existing comments to explicitly document why
KVM needs to use READ_ONCE().
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220715232107.3775620-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 610125291a48..5cb3362e1653 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2873,6 +2873,31 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
__direct_pte_prefetch(vcpu, sp, sptep);
}
+/*
+ * Lookup the mapping level for @gfn in the current mm.
+ *
+ * WARNING! Use of host_pfn_mapping_level() requires the caller and the end
+ * consumer to be tied into KVM's handlers for MMU notifier events!
+ *
+ * There are several ways to safely use this helper:
+ *
+ * - Check mmu_notifier_retry_hva() after grabbing the mapping level, before
+ * consuming it. In this case, mmu_lock doesn't need to be held during the
+ * lookup, but it does need to be held while checking the MMU notifier.
+ *
+ * - Hold mmu_lock AND ensure there is no in-progress MMU notifier invalidation
+ * event for the hva. This can be done by explicit checking the MMU notifier
+ * or by ensuring that KVM already has a valid mapping that covers the hva.
+ *
+ * - Do not use the result to install new mappings, e.g. use the host mapping
+ * level only to decide whether or not to zap an entry. In this case, it's
+ * not required to hold mmu_lock (though it's highly likely the caller will
+ * want to hold mmu_lock anyways, e.g. to modify SPTEs).
+ *
+ * Note! The lookup can still race with modifications to host page tables, but
+ * the above "rules" ensure KVM will not _consume_ the result of the walk if a
+ * race with the primary MMU occurs.
+ */
static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
const struct kvm_memory_slot *slot)
{
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 5/8] KVM: Rename mmu_notifier_* to mmu_invalidate_*
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (3 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 4/8] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 6/8] KVM: x86/mmu: Split out TDP MMU page fault handling Kenta Akagi
` (3 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Chao Peng, Paolo Bonzini, Sasha Levin, Kenta Akagi
From: Chao Peng <chao.p.peng@linux.intel.com>
[ Upstream commit 20ec3ebd707c77fb9b11b37193449193d4649f33 ]
The motivation of this renaming is to make these variables and related
helper functions less mmu_notifier bound and can also be used for non
mmu_notifier based page invalidation. mmu_invalidate_* was chosen to
better describe the purpose of 'invalidating' a page that those
variables are used for.
- mmu_notifier_seq/range_start/range_end are renamed to
mmu_invalidate_seq/range_start/range_end.
- mmu_notifier_retry{_hva} helper functions are renamed to
mmu_invalidate_retry{_hva}.
- mmu_notifier_count is renamed to mmu_invalidate_in_progress to
avoid confusion with mn_active_invalidate_count.
- While here, also update kvm_inc/dec_notifier_count() to
kvm_mmu_invalidate_begin/end() to match the change for
mmu_notifier_count.
No functional change intended.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Message-Id: <20220816125322.1110439-3-chao.p.peng@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/arm64/kvm/mmu.c | 10 ++--
arch/mips/kvm/mmu.c | 12 ++---
arch/powerpc/include/asm/kvm_book3s_64.h | 2 +-
arch/powerpc/kvm/book3s_64_mmu_host.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 +--
arch/powerpc/kvm/book3s_64_vio_hv.c | 2 +-
arch/powerpc/kvm/book3s_hv_nested.c | 2 +-
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 8 ++--
arch/powerpc/kvm/e500_mmu_host.c | 4 +-
arch/x86/kvm/mmu/mmu.c | 14 +++---
arch/x86/kvm/mmu/paging_tmpl.h | 4 +-
include/linux/kvm_host.h | 60 ++++++++++++------------
virt/kvm/kvm_main.c | 49 +++++++++----------
14 files changed, 92 insertions(+), 89 deletions(-)
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index db667b4ad103..7d04f9670644 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -881,7 +881,7 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
* THP doesn't start to split while we are adjusting the
* refcounts.
*
- * We are sure this doesn't happen, because mmu_notifier_retry
+ * We are sure this doesn't happen, because mmu_invalidate_retry
* was successful and we are holding the mmu_lock, so if this
* THP is trying to split, it will be blocked in the mmu
* notifier before touching any of the pages, specifically
@@ -1074,14 +1074,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
gfn = fault_ipa >> PAGE_SHIFT;
/*
- * Read mmu_notifier_seq so that KVM can detect if the results of
+ * Read mmu_invalidate_seq so that KVM can detect if the results of
* vma_lookup() or __gfn_to_pfn_memslot() become stale prior to
* acquiring kvm->mmu_lock.
*
* Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs
- * with the smp_wmb() in kvm_dec_notifier_count().
+ * with the smp_wmb() in kvm_mmu_invalidate_end().
*/
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
mmap_read_unlock(current->mm);
pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL,
@@ -1118,7 +1118,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
spin_lock(&kvm->mmu_lock);
pgt = vcpu->arch.hw_mmu->pgt;
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
goto out_unlock;
/*
diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c
index 72b524945227..755b7a0a5595 100644
--- a/arch/mips/kvm/mmu.c
+++ b/arch/mips/kvm/mmu.c
@@ -615,17 +615,17 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa,
* Used to check for invalidations in progress, of the pfn that is
* returned by pfn_to_pfn_prot below.
*/
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
/*
- * Ensure the read of mmu_notifier_seq isn't reordered with PTE reads in
- * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't
+ * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads
+ * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't
* risk the page we get a reference to getting unmapped before we have a
- * chance to grab the mmu_lock without mmu_notifier_retry() noticing.
+ * chance to grab the mmu_lock without mmu_invalidate_retry() noticing.
*
* This smp_rmb() pairs with the effective smp_wmb() of the combination
* of the pte_unmap_unlock() after the PTE is zapped, and the
* spin_lock() in kvm_mmu_notifier_invalidate_<page|range_end>() before
- * mmu_notifier_seq is incremented.
+ * mmu_invalidate_seq is incremented.
*/
smp_rmb();
@@ -638,7 +638,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa,
spin_lock(&kvm->mmu_lock);
/* Check if an invalidation has taken place since we got pfn */
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
/*
* This can happen when mappings are changed asynchronously, but
* also synchronously if a COW is triggered by
diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h
index eaf3a562bf1e..0b418df7e3f8 100644
--- a/arch/powerpc/include/asm/kvm_book3s_64.h
+++ b/arch/powerpc/include/asm/kvm_book3s_64.h
@@ -673,7 +673,7 @@ static inline pte_t *find_kvm_host_pte(struct kvm *kvm, unsigned long mmu_seq,
VM_WARN(!spin_is_locked(&kvm->mmu_lock),
"%s called with kvm mmu_lock not held \n", __func__);
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
return NULL;
pte = __find_linux_pte(kvm->mm->pgd, ea, NULL, hshift);
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index c3e31fef0be1..1d0f3f9d8508 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -90,7 +90,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte,
unsigned long pfn;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* Get host physical address for gpa */
@@ -151,7 +151,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte,
cpte = kvmppc_mmu_hpte_cache_next(vcpu);
spin_lock(&kvm->mmu_lock);
- if (!cpte || mmu_notifier_retry(kvm, mmu_seq)) {
+ if (!cpte || mmu_invalidate_retry(kvm, mmu_seq)) {
r = -EAGAIN;
goto out_unlock;
}
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index c63e263312a4..42cc2de2e3b5 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -570,7 +570,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu,
return -EFAULT;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
ret = -EFAULT;
@@ -685,7 +685,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu,
/* Check if we might have been invalidated; let the guest retry if so */
ret = RESUME_GUEST;
- if (mmu_notifier_retry(vcpu->kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(vcpu->kvm, mmu_seq)) {
unlock_rmap(rmap);
goto out_unlock;
}
diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
index 16359525a40f..6dc8dad1be99 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -634,7 +634,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
/* Check if we might have been invalidated; let the guest retry if so */
spin_lock(&kvm->mmu_lock);
ret = -EAGAIN;
- if (mmu_notifier_retry(kvm, mmu_seq))
+ if (mmu_invalidate_retry(kvm, mmu_seq))
goto out_unlock;
/* Now traverse again under the lock and change the tree */
@@ -824,7 +824,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu,
bool large_enable;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/*
@@ -1185,7 +1185,7 @@ void kvmppc_radix_flush_memslot(struct kvm *kvm,
* Increase the mmu notifier sequence number to prevent any page
* fault that read the memslot earlier from writing a PTE.
*/
- kvm->mmu_notifier_seq++;
+ kvm->mmu_invalidate_seq++;
spin_unlock(&kvm->mmu_lock);
}
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index fdeda6a9cff4..26ee88843f76 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -499,7 +499,7 @@ long kvmppc_rm_h_put_tce_indirect(struct kvm_vcpu *vcpu,
/*
* used to check for invalidations in progress
*/
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
stt = kvmppc_find_table(vcpu->kvm, liobn);
diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c
index ddea14e5cb5e..e31b3c20a9d4 100644
--- a/arch/powerpc/kvm/book3s_hv_nested.c
+++ b/arch/powerpc/kvm/book3s_hv_nested.c
@@ -1574,7 +1574,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu,
/* 2. Find the host pte for this L1 guest real address */
/* Used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* See if can find translation in our partition scoped tables for L1 */
diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
index 632b2545072b..95fb2965945f 100644
--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
@@ -216,7 +216,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
g_ptel = ptel;
/* used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/* Find the memslot (if any) for this address */
@@ -363,7 +363,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
rmap = real_vmalloc_addr(rmap);
lock_rmap(rmap);
/* Check for pending invalidations under the rmap chain lock */
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
/* inval in progress, write a non-present HPTE */
pteh |= HPTE_V_ABSENT;
pteh &= ~HPTE_V_VALID;
@@ -929,7 +929,7 @@ static long kvmppc_do_h_page_init_zero(struct kvm_vcpu *vcpu,
int i;
/* Used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
arch_spin_lock(&kvm->mmu_lock.rlock.raw_lock);
@@ -957,7 +957,7 @@ static long kvmppc_do_h_page_init_copy(struct kvm_vcpu *vcpu,
long ret = H_SUCCESS;
/* Used later to detect if we might have been invalidated */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
arch_spin_lock(&kvm->mmu_lock.rlock.raw_lock);
diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c
index 7f16afc331ef..05668e964140 100644
--- a/arch/powerpc/kvm/e500_mmu_host.c
+++ b/arch/powerpc/kvm/e500_mmu_host.c
@@ -339,7 +339,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
unsigned long flags;
/* used to check for invalidations in progress */
- mmu_seq = kvm->mmu_notifier_seq;
+ mmu_seq = kvm->mmu_invalidate_seq;
smp_rmb();
/*
@@ -460,7 +460,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
}
spin_lock(&kvm->mmu_lock);
- if (mmu_notifier_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
ret = -EAGAIN;
goto out;
}
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 5cb3362e1653..479f43859612 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2867,7 +2867,7 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
* If addresses are being invalidated, skip prefetching to avoid
* accidentally prefetching those addresses.
*/
- if (unlikely(vcpu->kvm->mmu_notifier_count))
+ if (unlikely(vcpu->kvm->mmu_invalidate_in_progress))
return;
__direct_pte_prefetch(vcpu, sp, sptep);
@@ -2881,7 +2881,7 @@ static void direct_pte_prefetch(struct kvm_vcpu *vcpu, u64 *sptep)
*
* There are several ways to safely use this helper:
*
- * - Check mmu_notifier_retry_hva() after grabbing the mapping level, before
+ * - Check mmu_invalidate_retry_hva() after grabbing the mapping level, before
* consuming it. In this case, mmu_lock doesn't need to be held during the
* lookup, but it does need to be held while checking the MMU notifier.
*
@@ -2976,7 +2976,7 @@ int kvm_mmu_hugepage_adjust(struct kvm_vcpu *vcpu, gfn_t gfn,
return PG_LEVEL_4K;
/*
- * mmu_notifier_retry() was successful and mmu_lock is held, so
+ * mmu_invalidate_retry() was successful and mmu_lock is held, so
* the pmd can't be split from under us.
*/
mask = KVM_PAGES_PER_HPAGE(level) - 1;
@@ -4009,7 +4009,7 @@ static bool is_page_fault_stale(struct kvm_vcpu *vcpu,
return true;
return !is_noslot_pfn(pfn) &&
- mmu_notifier_retry_hva(vcpu->kvm, mmu_seq, hva);
+ mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, hva);
}
static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
@@ -4036,7 +4036,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
if (r)
return r;
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
smp_rmb();
if (kvm_faultin_pfn(vcpu, prefault, gfn, gpa, &pfn, &hva,
@@ -5785,7 +5785,7 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
write_lock(&kvm->mmu_lock);
- kvm_inc_notifier_count(kvm, gfn_start, gfn_end);
+ kvm_mmu_invalidate_begin(kvm, gfn_start, gfn_end);
if (kvm_memslots_have_rmaps(kvm)) {
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
@@ -5820,7 +5820,7 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
gfn_end - gfn_start);
- kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
+ kvm_mmu_invalidate_end(kvm, gfn_start, gfn_end);
write_unlock(&kvm->mmu_lock);
}
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index d1e7ac58b735..c67b51bc666e 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -634,7 +634,7 @@ static void FNAME(pte_prefetch)(struct kvm_vcpu *vcpu, struct guest_walker *gw,
* If addresses are being invalidated, skip prefetching to avoid
* accidentally prefetching those addresses.
*/
- if (unlikely(vcpu->kvm->mmu_notifier_count))
+ if (unlikely(vcpu->kvm->mmu_invalidate_in_progress))
return;
if (sp->role.direct)
@@ -894,7 +894,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,
else
max_level = walker.level;
- mmu_seq = vcpu->kvm->mmu_notifier_seq;
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
smp_rmb();
if (kvm_faultin_pfn(vcpu, prefault, walker.gfn, addr, &pfn, &hva,
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 118a3905afb0..24ce84739f03 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -710,10 +710,10 @@ struct kvm {
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
struct mmu_notifier mmu_notifier;
- unsigned long mmu_notifier_seq;
- long mmu_notifier_count;
- unsigned long mmu_notifier_range_start;
- unsigned long mmu_notifier_range_end;
+ unsigned long mmu_invalidate_seq;
+ long mmu_invalidate_in_progress;
+ unsigned long mmu_invalidate_range_start;
+ unsigned long mmu_invalidate_range_end;
#endif
struct list_head devices;
u64 manual_dirty_log_protect;
@@ -1100,10 +1100,10 @@ void kvm_mmu_free_memory_cache(struct kvm_mmu_memory_cache *mc);
void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc);
#endif
-void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end);
-void kvm_dec_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end);
+void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start,
+ unsigned long end);
+void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start,
+ unsigned long end);
long kvm_arch_dev_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg);
@@ -1631,42 +1631,44 @@ extern const struct kvm_stats_header kvm_vcpu_stats_header;
extern const struct _kvm_stats_desc kvm_vcpu_stats_desc[];
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
-static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
+static inline int mmu_invalidate_retry(struct kvm *kvm, unsigned long mmu_seq)
{
- if (unlikely(kvm->mmu_notifier_count))
+ if (unlikely(kvm->mmu_invalidate_in_progress))
return 1;
/*
- * Ensure the read of mmu_notifier_count happens before the read
- * of mmu_notifier_seq. This interacts with the smp_wmb() in
- * mmu_notifier_invalidate_range_end to make sure that the caller
- * either sees the old (non-zero) value of mmu_notifier_count or
- * the new (incremented) value of mmu_notifier_seq.
- * PowerPC Book3s HV KVM calls this under a per-page lock
- * rather than under kvm->mmu_lock, for scalability, so
- * can't rely on kvm->mmu_lock to keep things ordered.
+ * Ensure the read of mmu_invalidate_in_progress happens before
+ * the read of mmu_invalidate_seq. This interacts with the
+ * smp_wmb() in mmu_notifier_invalidate_range_end to make sure
+ * that the caller either sees the old (non-zero) value of
+ * mmu_invalidate_in_progress or the new (incremented) value of
+ * mmu_invalidate_seq.
+ *
+ * PowerPC Book3s HV KVM calls this under a per-page lock rather
+ * than under kvm->mmu_lock, for scalability, so can't rely on
+ * kvm->mmu_lock to keep things ordered.
*/
smp_rmb();
- if (kvm->mmu_notifier_seq != mmu_seq)
+ if (kvm->mmu_invalidate_seq != mmu_seq)
return 1;
return 0;
}
-static inline int mmu_notifier_retry_hva(struct kvm *kvm,
- unsigned long mmu_seq,
- unsigned long hva)
+static inline int mmu_invalidate_retry_hva(struct kvm *kvm,
+ unsigned long mmu_seq,
+ unsigned long hva)
{
lockdep_assert_held(&kvm->mmu_lock);
/*
- * If mmu_notifier_count is non-zero, then the range maintained by
- * kvm_mmu_notifier_invalidate_range_start contains all addresses that
- * might be being invalidated. Note that it may include some false
+ * If mmu_invalidate_in_progress is non-zero, then the range maintained
+ * by kvm_mmu_notifier_invalidate_range_start contains all addresses
+ * that might be being invalidated. Note that it may include some false
* positives, due to shortcuts when handing concurrent invalidations.
*/
- if (unlikely(kvm->mmu_notifier_count) &&
- hva >= kvm->mmu_notifier_range_start &&
- hva < kvm->mmu_notifier_range_end)
+ if (unlikely(kvm->mmu_invalidate_in_progress) &&
+ hva >= kvm->mmu_invalidate_range_start &&
+ hva < kvm->mmu_invalidate_range_end)
return 1;
- if (kvm->mmu_notifier_seq != mmu_seq)
+ if (kvm->mmu_invalidate_seq != mmu_seq)
return 1;
return 0;
}
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 79422642ae00..1482aeebe745 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -665,30 +665,31 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn,
/*
* .change_pte() must be surrounded by .invalidate_range_{start,end}().
- * If mmu_notifier_count is zero, then no in-progress invalidations,
- * including this one, found a relevant memslot at start(); rechecking
- * memslots here is unnecessary. Note, a false positive (count elevated
- * by a different invalidation) is sub-optimal but functionally ok.
+ * If mmu_invalidate_in_progress is zero, then no in-progress
+ * invalidations, including this one, found a relevant memslot at
+ * start(); rechecking memslots here is unnecessary. Note, a false
+ * positive (count elevated by a different invalidation) is sub-optimal
+ * but functionally ok.
*/
WARN_ON_ONCE(!READ_ONCE(kvm->mn_active_invalidate_count));
- if (!READ_ONCE(kvm->mmu_notifier_count))
+ if (!READ_ONCE(kvm->mmu_invalidate_in_progress))
return;
kvm_handle_hva_range(mn, address, address + 1, pte, kvm_change_spte_gfn);
}
-void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end)
+void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start,
+ unsigned long end)
{
/*
* The count increase must become visible at unlock time as no
* spte can be established without taking the mmu_lock and
* count is also read inside the mmu_lock critical section.
*/
- kvm->mmu_notifier_count++;
- if (likely(kvm->mmu_notifier_count == 1)) {
- kvm->mmu_notifier_range_start = start;
- kvm->mmu_notifier_range_end = end;
+ kvm->mmu_invalidate_in_progress++;
+ if (likely(kvm->mmu_invalidate_in_progress == 1)) {
+ kvm->mmu_invalidate_range_start = start;
+ kvm->mmu_invalidate_range_end = end;
} else {
/*
* Fully tracking multiple concurrent ranges has dimishing
@@ -699,10 +700,10 @@ void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
* accumulate and persist until all outstanding invalidates
* complete.
*/
- kvm->mmu_notifier_range_start =
- min(kvm->mmu_notifier_range_start, start);
- kvm->mmu_notifier_range_end =
- max(kvm->mmu_notifier_range_end, end);
+ kvm->mmu_invalidate_range_start =
+ min(kvm->mmu_invalidate_range_start, start);
+ kvm->mmu_invalidate_range_end =
+ max(kvm->mmu_invalidate_range_end, end);
}
}
@@ -715,7 +716,7 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
.end = range->end,
.pte = __pte(0),
.handler = kvm_unmap_gfn_range,
- .on_lock = kvm_inc_notifier_count,
+ .on_lock = kvm_mmu_invalidate_begin,
.on_unlock = kvm_arch_guest_memory_reclaimed,
.flush_on_ret = true,
.may_block = mmu_notifier_range_blockable(range),
@@ -726,7 +727,7 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
/*
* Prevent memslot modification between range_start() and range_end()
* so that conditionally locking provides the same result in both
- * functions. Without that guarantee, the mmu_notifier_count
+ * functions. Without that guarantee, the mmu_invalidate_in_progress
* adjustments will be imbalanced.
*
* Pairs with the decrement in range_end().
@@ -740,22 +741,22 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
return 0;
}
-void kvm_dec_notifier_count(struct kvm *kvm, unsigned long start,
- unsigned long end)
+void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start,
+ unsigned long end)
{
/*
* This sequence increase will notify the kvm page fault that
* the page that is going to be mapped in the spte could have
* been freed.
*/
- kvm->mmu_notifier_seq++;
+ kvm->mmu_invalidate_seq++;
smp_wmb();
/*
* The above sequence increase must be visible before the
* below count decrease, which is ensured by the smp_wmb above
- * in conjunction with the smp_rmb in mmu_notifier_retry().
+ * in conjunction with the smp_rmb in mmu_invalidate_retry().
*/
- kvm->mmu_notifier_count--;
+ kvm->mmu_invalidate_in_progress--;
}
static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
@@ -767,7 +768,7 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
.end = range->end,
.pte = __pte(0),
.handler = (void *)kvm_null_fn,
- .on_lock = kvm_dec_notifier_count,
+ .on_lock = kvm_mmu_invalidate_end,
.on_unlock = (void *)kvm_null_fn,
.flush_on_ret = false,
.may_block = mmu_notifier_range_blockable(range),
@@ -788,7 +789,7 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
if (wake)
rcuwait_wake_up(&kvm->mn_memslots_update_rcuwait);
- BUG_ON(kvm->mmu_notifier_count < 0);
+ BUG_ON(kvm->mmu_invalidate_in_progress < 0);
}
static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn,
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 6/8] KVM: x86/mmu: Split out TDP MMU page fault handling
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (4 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 5/8] KVM: Rename mmu_notifier_* to mmu_invalidate_* Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 7/8] KVM: x86/mmu: Rename __direct_map() to direct_map() Kenta Akagi
` (2 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable
Cc: David Matlack, Isaku Yamahata, Paolo Bonzini, Sasha Levin,
Kenta Akagi
From: David Matlack <dmatlack@google.com>
[ Upstream commit 9aa8ab43b38146029de807a8ff2696f51e15b226 ]
Split out the page fault handling for the TDP MMU to a separate
function. This creates some duplicate code, but makes the TDP MMU fault
handler simpler to read by eliminating branches and will enable future
cleanups by allowing the TDP MMU and non-TDP MMU fault paths to diverge.
Only compile in the TDP MMU fault handler for 64-bit builds since
kvm_tdp_mmu_map() does not exist in 32-bit builds.
No functional change intended.
Signed-off-by: David Matlack <dmatlack@google.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220921173546.2674386-9-dmatlack@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 84 ++++++++++++++++++++++++++++++++----------
1 file changed, 64 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 479f43859612..8c1061b2f12f 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4015,7 +4015,6 @@ static bool is_page_fault_stale(struct kvm_vcpu *vcpu,
static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
bool prefault, int max_level, bool is_tdp)
{
- bool is_tdp_mmu_fault = is_tdp_mmu(vcpu->arch.mmu);
bool write = error_code & PFERR_WRITE_MASK;
bool map_writable;
@@ -4047,31 +4046,20 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
return r;
r = RET_PF_RETRY;
-
- if (is_tdp_mmu_fault)
- read_lock(&vcpu->kvm->mmu_lock);
- else
- write_lock(&vcpu->kvm->mmu_lock);
+ write_lock(&vcpu->kvm->mmu_lock);
if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
goto out_unlock;
- if (is_tdp_mmu_fault) {
- r = kvm_tdp_mmu_map(vcpu, gpa, error_code, map_writable, max_level,
- pfn, prefault);
- } else {
- r = make_mmu_pages_available(vcpu);
- if (r)
- goto out_unlock;
- r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
- prefault, is_tdp);
- }
+ r = make_mmu_pages_available(vcpu);
+ if (r)
+ goto out_unlock;
+
+ r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
+ prefault, is_tdp);
out_unlock:
- if (is_tdp_mmu_fault)
- read_unlock(&vcpu->kvm->mmu_lock);
- else
- write_unlock(&vcpu->kvm->mmu_lock);
+ write_unlock(&vcpu->kvm->mmu_lock);
kvm_release_pfn_clean(pfn);
return r;
}
@@ -4119,6 +4107,56 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code,
}
EXPORT_SYMBOL_GPL(kvm_handle_page_fault);
+#ifdef CONFIG_X86_64
+static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa,
+ u32 error_code, bool prefault, int max_level)
+{
+ bool write = error_code & PFERR_WRITE_MASK;
+ bool map_writable;
+
+ gfn_t gfn = gpa >> PAGE_SHIFT;
+ unsigned long mmu_seq;
+ kvm_pfn_t pfn;
+ hva_t hva;
+ int r;
+
+ if (page_fault_handle_page_track(vcpu, error_code, gfn))
+ return RET_PF_EMULATE;
+
+ r = fast_page_fault(vcpu, gpa, error_code);
+ if (r != RET_PF_INVALID)
+ return r;
+
+ r = mmu_topup_memory_caches(vcpu, false);
+ if (r)
+ return r;
+
+ mmu_seq = vcpu->kvm->mmu_invalidate_seq;
+ smp_rmb();
+
+ if (kvm_faultin_pfn(vcpu, prefault, gfn, gpa, &pfn, &hva,
+ write, &map_writable, &r))
+ return r;
+
+ if (handle_abnormal_pfn(vcpu, 0, gfn, pfn, ACC_ALL, &r))
+ return r;
+
+ r = RET_PF_RETRY;
+ read_lock(&vcpu->kvm->mmu_lock);
+
+ if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
+ goto out_unlock;
+
+ r = kvm_tdp_mmu_map(vcpu, gpa, error_code, map_writable, max_level,
+ pfn, prefault);
+
+out_unlock:
+ read_unlock(&vcpu->kvm->mmu_lock);
+ kvm_release_pfn_clean(pfn);
+ return r;
+}
+#endif
+
int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
bool prefault)
{
@@ -4134,6 +4172,12 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
break;
}
+#ifdef CONFIG_X86_64
+ if (is_tdp_mmu(vcpu->arch.mmu))
+ return kvm_tdp_mmu_page_fault(vcpu, gpa, error_code, prefault,
+ max_level);
+#endif
+
return direct_page_fault(vcpu, gpa, error_code, prefault,
max_level, true);
}
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 7/8] KVM: x86/mmu: Rename __direct_map() to direct_map()
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (5 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 6/8] KVM: x86/mmu: Split out TDP MMU page fault handling Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 8/8] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available Kenta Akagi
2026-08-25 11:49 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Sasha Levin
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable
Cc: David Matlack, Isaku Yamahata, Paolo Bonzini, Sasha Levin,
Kenta Akagi
From: David Matlack <dmatlack@google.com>
[ Upstream commit 6c882ef4fc7bd99b67ad152e75428b669281c521 ]
Rename __direct_map() to direct_map() since the leading underscores are
unnecessary. This also makes the page fault handler names more
consistent: kvm_tdp_mmu_page_fault() calls kvm_tdp_mmu_map() and
direct_page_fault() calls direct_map().
Opportunistically make some trivial cleanups to comments that had to be
modified anyway since they mentioned __direct_map(). Specifically, use
"()" when referring to functions, and include kvm_tdp_mmu_map() among
the various callers of disallowed_hugepage_adjust().
No functional change intended.
Signed-off-by: David Matlack <dmatlack@google.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220921173546.2674386-11-dmatlack@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stable-dep-of: 2abd5287f083 ("KVM: x86: Check for invalid/obsolete root *after* making MMU pages available")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 8c1061b2f12f..019e24a76a96 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2995,11 +2995,11 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
is_shadow_present_pte(spte) &&
!is_large_pte(spte)) {
/*
- * A small SPTE exists for this pfn, but FNAME(fetch)
- * and __direct_map would like to create a large PTE
- * instead: just force them to go down another level,
- * patching back for them into pfn the next 9 bits of
- * the address.
+ * A small SPTE exists for this pfn, but FNAME(fetch),
+ * direct_map(), or kvm_tdp_mmu_map() would like to create a
+ * large PTE instead: just force them to go down another level,
+ * patching back for them into pfn the next 9 bits of the
+ * address.
*/
u64 page_mask = KVM_PAGES_PER_HPAGE(level) -
KVM_PAGES_PER_HPAGE(level - 1);
@@ -3008,9 +3008,9 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level,
}
}
-static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
- int map_writable, int max_level, kvm_pfn_t pfn,
- bool prefault, bool is_tdp)
+static int direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
+ int map_writable, int max_level, kvm_pfn_t pfn,
+ bool prefault, bool is_tdp)
{
bool nx_huge_page_workaround_enabled = is_nx_huge_page_enabled();
bool write = error_code & PFERR_WRITE_MASK;
@@ -4055,8 +4055,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
if (r)
goto out_unlock;
- r = __direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
- prefault, is_tdp);
+ r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
+ prefault, is_tdp);
out_unlock:
write_unlock(&vcpu->kvm->mmu_lock);
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5.15.y v2 8/8] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (6 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 7/8] KVM: x86/mmu: Rename __direct_map() to direct_map() Kenta Akagi
@ 2026-08-23 13:07 ` Kenta Akagi
2026-08-25 11:49 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Sasha Levin
8 siblings, 0 replies; 21+ messages in thread
From: Kenta Akagi @ 2026-08-23 13:07 UTC (permalink / raw)
To: stable; +Cc: Sean Christopherson, Hyunwoo Kim, Paolo Bonzini, Kenta Akagi
From: Sean Christopherson <seanjc@google.com>
[ Upstream commit 2abd5287f08319fa35764566b15c6e22cb1068db ]
Check for a "stale" page fault, i.e. for an invalid and/or obsolete root,
after making MMU pages available for the shadow MMU. If reclaiming shadow
pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to
map memory into an invalid root. On its own, populating an invalid root is
"fine", but because child shadow pages inherit their parent's role, any
children created during the map/fetch will be created as invalid pages,
thus violating KVM's invariant that invalid pages are never on the list of
active MMU pages.
Note, the underlying flaw has existed since KVM first started tracking
invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root
pagetables"), but the true badness only came along in 2020 (Linux 5.9)
with the invariant that invalid shadow pages can't be on the list of
active pages.
Note #2, inheriting role.invalid when creating child shadow pages is also
far from ideal; that flaw will be addressed separately.
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
[backport note: upstream passes struct kvm_page_fault to is_page_fault_stale();
here individual parameters are used, as is_page_fault_stale() was
backported with that interface in the preceding commit]
Signed-off-by: Kenta Akagi <k@mgml.me>
---
arch/x86/kvm/mmu/mmu.c | 9 +++++----
arch/x86/kvm/mmu/paging_tmpl.h | 10 ++++++----
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 019e24a76a96..8c381cf61e21 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4045,16 +4045,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
if (handle_abnormal_pfn(vcpu, is_tdp ? 0 : gpa, gfn, pfn, ACC_ALL, &r))
return r;
- r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
- goto out_unlock;
-
r = make_mmu_pages_available(vcpu);
if (r)
goto out_unlock;
+ if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) {
+ r = RET_PF_RETRY;
+ goto out_unlock;
+ }
+
r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn,
prefault, is_tdp);
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index c67b51bc666e..fd3f2c5bb7ed 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -923,16 +923,18 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,
walker.pte_access &= ~ACC_EXEC_MASK;
}
- r = RET_PF_RETRY;
write_lock(&vcpu->kvm->mmu_lock);
- if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva))
- goto out_unlock;
-
kvm_mmu_audit(vcpu, AUDIT_PRE_PAGE_FAULT);
r = make_mmu_pages_available(vcpu);
if (r)
goto out_unlock;
+
+ if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) {
+ r = RET_PF_RETRY;
+ goto out_unlock;
+ }
+
r = FNAME(fetch)(vcpu, addr, &walker, error_code, max_level, pfn,
map_writable, prefault);
kvm_mmu_audit(vcpu, AUDIT_POST_PAGE_FAULT);
--
2.51.2
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
` (7 preceding siblings ...)
2026-08-23 13:07 ` [PATCH 5.15.y v2 8/8] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available Kenta Akagi
@ 2026-08-25 11:49 ` Sasha Levin
8 siblings, 0 replies; 21+ messages in thread
From: Sasha Levin @ 2026-08-25 11:49 UTC (permalink / raw)
To: stable; +Cc: Sasha Levin, Kenta Akagi
On Sun, Aug 23, 2026 at 10:07:43PM +0900, Kenta Akagi wrote:
> This is v2 of the backport to fix CVE-2026-64561, v1 is Sasha's series [1].
>
> Patches 2-7 in this series correspond to patches 1-6 in v1; Patches 1 and 8 are new in v2.
Queued the series for 5.15, thanks.
One addition on my side: I inserted 18c841e1f411 ("KVM: x86: Retry page
fault if MMU reload is pending and root has no sp") immediately after
patch 1/8. It is the Cc: stable follow-up to a955cad84cda and the two
need to ship together.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2026-08-25 11:49 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-25 6:52 FAILED: patch "[PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU" failed to apply to 5.15-stable tree gregkh
2026-07-30 19:29 ` [PATCH 5.15.y 1/6] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 2/6] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 3/6] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 4/6] KVM: Rename mmu_notifier_* to mmu_invalidate_* Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 5/6] KVM: x86/mmu: Split out TDP MMU page fault handling Sasha Levin
2026-07-30 19:29 ` [PATCH 5.15.y 6/6] KVM: x86/mmu: Rename __direct_map() to direct_map() Sasha Levin
2026-08-06 16:19 ` Harshit Mogalapalli
2026-08-06 16:50 ` Sasha Levin
2026-08-21 12:38 ` Kenta Akagi
2026-08-23 3:42 ` Sasha Levin
2026-08-23 13:07 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 1/8] KVM: x86/mmu: Retry page fault if root is invalidated by memslot update Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 2/8] KVM: x86/mmu: Directly "destroy" PTE list when recycling rmaps Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 3/8] KVM: x86/mmu: Rename pte_list_{destroy,remove}() to show they zap SPTEs Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 4/8] KVM: x86/mmu: Document the "rules" for using host_pfn_mapping_level() Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 5/8] KVM: Rename mmu_notifier_* to mmu_invalidate_* Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 6/8] KVM: x86/mmu: Split out TDP MMU page fault handling Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 7/8] KVM: x86/mmu: Rename __direct_map() to direct_map() Kenta Akagi
2026-08-23 13:07 ` [PATCH 5.15.y v2 8/8] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available Kenta Akagi
2026-08-25 11:49 ` [PATCH 5.15.y v2 0/8] backport for CVE-2026-64561 Sasha Levin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).