[PATCH v7 06/12] KVM: guest_memfd: add module param for disabling TLB flushing

Roy, Patrick
2025-09-24T15:22:51Z

Add an option to not perform TLB flushes after direct map manipulations.
TLB flushes result in a up to 40x elongation of page faults in
guest_memfd (scaling with the number of CPU cores), or a 5x elongation
of memory population, which is inacceptable when wanting to use direct
map removed guest_memfd as a drop-in replacement for existing workloads.

TLB flushes are not needed for functional correctness (the virt->phys
mapping technically stays "correct", the kernel should simply not use it
for a while), so we can skip them to keep performance in-line with
"traditional" VMs.

Enabling this option means that the desired protection from
Spectre-style attacks is not perfect, as an attacker could try to
prevent a stale TLB entry from getting evicted, keeping it alive until
the page it refers to is used by the guest for some sensitive data, and
then targeting it using a spectre-gadget.

Cc: Will Deacon <will@kernel.org>
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
---
include/linux/kvm_host.h | 1 +
virt/kvm/guest_memfd.c | 3 ++-
virt/kvm/kvm_main.c | 3 +++
3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 73a15cade54a..4d2bc18860fc 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2298,6 +2298,7 @@ extern unsigned int halt_poll_ns;
extern unsigned int halt_poll_ns_grow;
extern unsigned int halt_poll_ns_grow_start;
extern unsigned int halt_poll_ns_shrink;
+extern bool guest_memfd_tlb_flush;

struct kvm_device {
const struct kvm_device_ops *ops;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index b7129c4868c5..d8dd24459f0d 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -63,7 +63,8 @@ static int kvm_gmem_folio_zap_direct_map(struct folio *folio)
if (!r) {
unsigned long addr = (unsigned long) folio_address(folio);
folio->private = (void *) ((u64) folio->private & KVM_GMEM_FOLIO_NO_DIRECT_MAP);
- flush_tlb_kernel_range(addr, addr + folio_size(folio));
+ if (guest_memfd_tlb_flush)
+ flush_tlb_kernel_range(addr, addr + folio_size(folio));
}

return r;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index b5e702d95230..753c06ebba7f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -95,6 +95,9 @@ unsigned int halt_poll_ns_shrink = 2;
module_param(halt_poll_ns_shrink, uint, 0644);
EXPORT_SYMBOL_GPL(halt_poll_ns_shrink);

+bool guest_memfd_tlb_flush = true;
+module_param(guest_memfd_tlb_flush, bool, 0444);
+
/*
* Allow direct access (from KVM or the CPU) without MMU notifier protection
* to unpinned pages.
--
2.51.0