Re: [PATCH 3/4] KVM: x86/mmu: Add automatic prefaulting/prefetching for TDP MMU faults

[Sean Christopherson <seanjc@google.com>]

On Fri, Oct 30, 2026, Marco Marangoni wrote:
> 
> On Thu, Oct 29, 2026, Sean Christopherson wrote:
> >> From my perspective, using fast GUP has the problem that you cannot prefault
> >> "around" your page, but just ahead of it, so I don't mind going with normal
> >> GUP if it makes things easier for you.
> >
> > I think I'm missing something, why would fast GUP prevent prefaulting "behind"?
> > E.g. is that effectively a limitation of the underlying backing store you're
> > using, a fundamental limitation somewhere in mm/, or a limitation in KVM itself?
> 
> I didn't explain myself properly, sorry.
> 
> What I had in mind is that kvm_mmu_auto_prefault() returns on the first
> kvm_tdp_page_prefault() failure, and starts from the ALIGN_DOWN'd window start
> rather than from the faulting page.  So if a hole is below the faulting page,
> the resident pages above it don't get prefetched either.
> 
> E.g. faulting at page 5, window [0..7].  If [4..7] are resident and [0..3] are
> UFFD-missing, the loop fails on page 0 and nothing gets prefetched, even though
> 6 and 7 are right there.
> 
> It doesn't matter when nr_pages matches the UFFD chunk size and alignment, but
> it would with smaller chunks or a non-linear access pattern.

Oh, that's easy enough to "fix", the only question is whether or not that would
be a net positive.

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 08b6a90c7131..61cae6f1d9a9 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5179,8 +5179,7 @@ static void kvm_mmu_auto_prefault(struct kvm_vcpu *vcpu, gpa_t start,
 		if (gpa < start || gpa_to_gfn(gpa) > kvm_mmu_max_gfn())
 			return;
 
-		if (kvm_tdp_page_prefault(vcpu, gpa, error_code, KVM_PREFETCH_AUTO, &level))
-			return;
+		kvm_tdp_page_prefault(vcpu, gpa, error_code, KVM_PREFETCH_AUTO, &level);
 	}
 }

I generally don't like heuristics, but they're practically unavoidable when it
comes to stuff like this, so the other option would be to set a threshold for
when to bail.  Failure should be cheap?  So we could set the threshold to be the
default PTE_PREFETCH_NUM, that way the default behavior is that KVM agressively
tries to prefault everything.

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 08b6a90c7131..5dc820a32f8c 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5156,8 +5156,8 @@ static void kvm_mmu_auto_prefault(struct kvm_vcpu *vcpu, gpa_t start,
 				  u64 error_code, u8 level)
 {
 	gfn_t nr_pages = READ_ONCE(auto_prefault_nr_pages);
+	gfn_t i, nr_failures;
 	int nr_pages_msb;
-	gfn_t i;
 
 	if (error_code & (PFERR_RSVD_MASK | PFERR_PRESENT_MASK))
 		return;
@@ -5179,7 +5179,10 @@ static void kvm_mmu_auto_prefault(struct kvm_vcpu *vcpu, gpa_t start,
 		if (gpa < start || gpa_to_gfn(gpa) > kvm_mmu_max_gfn())
 			return;
 
-		if (kvm_tdp_page_prefault(vcpu, gpa, error_code, KVM_PREFETCH_AUTO, &level))
+		if (!kvm_tdp_page_prefault(vcpu, gpa, error_code, KVM_PREFETCH_AUTO, &level))
+			continue;
+
+		if (++nr_failures > PTE_PREFETCH_NUM)
 			return;
 	}
 }

> FOLL_NOFAULT is fine by me; it just makes "hole" mean "not mapped" rather than
> "not resident", so slightly more cases hit this.