[PATCH v5 0/7] KVM: arm64: pKVM stage-2 mapping and memcache fixes

[PATCH v5 0/7] KVM: arm64: pKVM stage-2 mapping and memcache fixes

[Fuad Tabba]

Hi folks,

Changes since v4 [1]:
  - Restored the permission-fault and dirty-logging memcache top-ups
    that v4 dropped. Both fix real bugs.
  - Patch 1 uses the anonymous bitfield encoding rather than the
    open-coded mask and helpers. (Marc)
  - Reshaped the permission-fault top-up to stage the full memcache
    under pKVM, not just the mapping object. The object-only form still
    returned -ENOMEM and tripped a WARN under the hypervisor's
    unconditional min-pages check.
  - Re-scoped the dirty-logging top-up to its generic, non-pKVM failure
    mode, now that the permission-fault patch covers pKVM.
  - Added three adjacent fixes found while going through the series, and
    a selftest for the block transitions.
  - Patches 1 and 3 keep Bradley's Signed-off-by from v3; my changes
    to each are noted in a [tabba: ...] line.

I picked this up while reviewing Bradley's "mapping cache" series [2]:
v4 dropped two fixes from v3 that address real bugs, so I've collected
the three fixes back together, reshaped patch 1 per Marc's review [3],
added a few adjacent fixes I found along the way and a selftest, and am
reposting as v5.

Most of the fixes are in the pKVM stage-2 walker. On a pKVM host a
non-protected guest's stage-2 faults go through pkvm_pgtable_*(), which
diverges from the generic walker in several ways that are bugs: cache
maintenance on non-cacheable mappings (patch 1), a missing memcache
top-up on permission faults that under pKVM still allocate (patch 2), a
full flush walk on FWB hardware (patch 4), a WARN on a guest-reachable
map failure (patch 5), and an eager-split capability whose pKVM backend
is only a stub (patch 6). These affect non-protected guests only, since
dispatch and memcache selection key on the host-global pKVM mode, and
protected guests take no permission faults.

Patch 3 is not pKVM-specific. During dirty logging a non-write
permission fault, an instruction fetch, still needs a page-table page
to split a block, but the memcache top-up is gated on write faults.
That fault path is generic, so the fix is too.

The two memcache top-ups came from sashiko review-bot findings [4][5],
and both check out against the code.

The series is structured as follows:

  1:    Skip cache maintenance for non-cacheable mappings.
  2-3:  Top up the memcache for the permission and dirty-logging faults
        that force stage-2 block transitions.
  4-6:  Adjacent pkvm_pgtable_*() fixes: FWB flush early-out, drop a
        spurious map WARN, and gate the eager-split capability.
  7:    Selftest for the block-collapse and block-split transitions.

Testing: the selftest in patch 7 covers the two dirty-logging block
transitions (page->block collapse and block->page split) that patch 2
stages; on the base kernel the collapse oopses the host in
pkvm_pgtable_stage2_map() with a NULL dereference under mmu_lock, and
with the series applied it passes. It is a standalone test rather than
an extension of kvm_page_table_test, whose default anonymous-4K backing
forms no huge-page blocks, so an automated run never exercises these
transitions, and whose worker/stage harness does not fit the multi-stage
logging sequence. Run it with 2M hugepages
reserved and, for the split half, on a CPU with CTR_EL0.DIC == 0 (e.g.
-cpu cortex-a710 under QEMU); it self-skips those otherwise. It also
passes on a non-pKVM host (VHE and nVHE), where patch 3's generic change
applies. The other fixes are not exercised by the selftest and rest on
the analysis in their commit messages: patch 3's fault path is generic
and non-pKVM (under pKVM patch 2 already tops it up, and its failure is
a WARN_ON(!nobjs) in kvm_mmu_memory_cache_alloc(), not a KVM_RUN error),
and patches 1 and 4-6 each need a specific pKVM configuration to hit.

Based on Linux v7.2-rc3 (a13c140cc289).

Cheers,
/fuad

[1] https://lore.kernel.org/r/20260701192428.17430-1-include@grrlz.net
[2] https://lore.kernel.org/r/20260624160028.15591-1-include@grrlz.net
[3] https://lore.kernel.org/r/86qzllpy1g.wl-maz@kernel.org
[4] https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
[5] https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/

Bradley Morgan (2):
  KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
  KVM: arm64: Top up stage-2 memcache for dirty logging faults

Fuad Tabba (5):
  KVM: arm64: Top up the memcache for pKVM permission faults
  KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
  KVM: arm64: Don't WARN on pKVM stage-2 map failures
  KVM: arm64: Don't advertise eager page splitting under pKVM
  KVM: arm64: selftests: Add stage-2 block transition test

 arch/arm64/include/asm/kvm_pkvm.h             |   8 +-
 arch/arm64/kvm/mmu.c                          |   9 +-
 arch/arm64/kvm/pkvm.c                         |  21 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/arm64/stage2_block_transitions.c      | 226 ++++++++++++++++++
 5 files changed, 254 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/arm64/stage2_block_transitions.c


base-commit: a13c140cc289c0b7b3770bce5b3ad42ab35074aa
-- 
2.39.5

[PATCH v5 1/7] KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings

[Fuad Tabba]

From: Bradley Morgan <include@grrlz.net>

The pKVM flush path walks its own pkvm_mappings list and cleans the
data cache for every mapping, unlike the generic stage-2 walker it
shadows, which skips non-cacheable leaves. Cleaning the cacheable
alias of a non-cacheable mapping is pointless and can corrupt a
device endpoint. Record whether a mapping is non-cacheable in spare
bits of nr_pages and skip cache maintenance for it.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: use Marc's anonymous bitfield in place of the open-coded mask and helpers]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_pkvm.h |  5 ++++-
 arch/arm64/kvm/pkvm.c             | 15 +++++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 74fedd9c5ff0..57afb07d6b13 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -195,7 +195,10 @@ struct pkvm_mapping {
 	struct rb_node node;
 	u64 gfn;
 	u64 pfn;
-	u64 nr_pages;
+	struct {
+		u64 nr_pages:48;
+		u64 nc:1;
+	};
 	u64 __subtree_last;	/* Internal member for interval tree */
 };
 
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 053e4f733e4b..f70c601dcf4c 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -369,7 +369,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64
 
 	for_each_mapping_in_range_safe(pgt, start, end, mapping) {
 		ret = kvm_call_hyp_nvhe(__pkvm_host_unshare_guest, handle, mapping->gfn,
-					mapping->nr_pages);
+					(u64)mapping->nr_pages);
 		if (WARN_ON(ret))
 			return ret;
 		pkvm_mapping_remove(mapping, &pgt->pkvm_mappings);
@@ -473,6 +473,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
 	mapping->gfn = gfn;
 	mapping->pfn = pfn;
 	mapping->nr_pages = size / PAGE_SIZE;
+	mapping->nc = !!(prot & (KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC));
 	pkvm_mapping_insert(mapping, &pgt->pkvm_mappings);
 
 	return ret;
@@ -503,7 +504,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	lockdep_assert_held(&kvm->mmu_lock);
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
 		ret = kvm_call_hyp_nvhe(__pkvm_host_wrprotect_guest, handle, mapping->gfn,
-					mapping->nr_pages);
+					(u64)mapping->nr_pages);
 		if (WARN_ON(ret))
 			break;
 	}
@@ -517,9 +518,11 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	struct pkvm_mapping *mapping;
 
 	lockdep_assert_held(&kvm->mmu_lock);
-	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
-		__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
-					  PAGE_SIZE * mapping->nr_pages);
+	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
+		if (!mapping->nc)
+			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
+						  PAGE_SIZE * mapping->nr_pages);
+	}
 
 	return 0;
 }
@@ -537,7 +540,7 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64
 	lockdep_assert_held(&kvm->mmu_lock);
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
 		young |= kvm_call_hyp_nvhe(__pkvm_host_test_clear_young_guest, handle, mapping->gfn,
-					   mapping->nr_pages, mkold);
+					   (u64)mapping->nr_pages, mkold);
 
 	return young;
 }
-- 
2.39.5

[PATCH v5 2/7] KVM: arm64: Top up the memcache for pKVM permission faults

[Fuad Tabba]

A permission fault normally only relaxes a leaf, so user_mem_abort()
skips the memcache top-up. Under pKVM such a fault can instead collapse
pages into a block. That needs a fresh pkvm_mapping object, and without
it cache->mapping is NULL, so pkvm_pgtable_stage2_map() dereferences NULL
and faults the host under mmu_lock. Staging only the object is not
enough: the hypervisor requires kvm_mmu_cache_min_pages in the memcache
even for the allocation-free install, so under memcache pressure the
collapse returns -ENOMEM and trips the WARN_ON(ret) in
pkvm_pgtable_stage2_map() where a non-pKVM guest succeeds.

Top up the full memcache for pKVM permission faults so both the mapping
object and the min-pages are staged before mmu_lock.

Fixes: db14091d8f75 ("KVM: arm64: Stage-2 huge mappings for np-guests")
Reported-by: Bradley Morgan <include@grrlz.net>
Link: https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6c941aaa10c6..4d7c9bdcef00 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2114,10 +2114,14 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 	 * and so normally don't require allocations from the memcache. The
 	 * only exception to this is when dirty logging is enabled at runtime
 	 * and a write fault needs to collapse a block entry into a table.
+	 * Under pKVM a permission fault can also collapse pages into a block,
+	 * which needs a fresh mapping object, and the hypervisor requires the
+	 * min-pages memcache even when the install allocates nothing.
 	 */
 	memcache = get_mmu_memcache(s2fd->vcpu);
 	if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
-			    kvm_is_write_fault(s2fd->vcpu))) {
+			    kvm_is_write_fault(s2fd->vcpu)) ||
+	    is_protected_kvm_enabled()) {
 		ret = topup_mmu_memcache(s2fd->vcpu, memcache);
 		if (ret)
 			return ret;
-- 
2.39.5

[PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults

[Fuad Tabba]

From: Bradley Morgan <include@grrlz.net>

Dirty logging forces new stage-2 mappings to page size but does not
always split an existing block first (eager splitting is best effort
and off by default). A non-write permission fault on such a block, an
instruction fetch, still needs a page-table page to split it, but the
top-up is gated on write faults.

With the cache empty, kvm_mmu_memory_cache_alloc() hits its
guest-triggerable WARN_ON(!nobjs) and falls back to a GFP_ATOMIC
allocation under mmu_lock, with a BUG_ON() if that fails.

Top up the memcache for any permission fault while dirty logging is
active.

Fixes: 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to generic page-table API")
Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: reword the commit message for the generic, non-pKVM failure mode]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 4d7c9bdcef00..74e7e7f7564c 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2113,14 +2113,13 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 	 * Permission faults just need to update the existing leaf entry,
 	 * and so normally don't require allocations from the memcache. The
 	 * only exception to this is when dirty logging is enabled at runtime
-	 * and a write fault needs to collapse a block entry into a table.
+	 * and a fault needs to collapse a block entry into a table.
 	 * Under pKVM a permission fault can also collapse pages into a block,
 	 * which needs a fresh mapping object, and the hypervisor requires the
 	 * min-pages memcache even when the install allocates nothing.
 	 */
 	memcache = get_mmu_memcache(s2fd->vcpu);
-	if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
-			    kvm_is_write_fault(s2fd->vcpu)) ||
+	if (!perm_fault || memslot_is_logging(s2fd->memslot) ||
 	    is_protected_kvm_enabled()) {
 		ret = topup_mmu_memcache(s2fd->vcpu, memcache);
 		if (ret)
-- 
2.39.5

[PATCH v5 4/7] KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled

[Fuad Tabba]

pkvm_pgtable_stage2_flush() cleans the D-cache for every mapping in the
range even on hardware with stage-2 Force Write-Back, where FWB keeps
guest memory coherent to the PoC and the maintenance is unnecessary. The
generic kvm_pgtable_stage2_flush() returns early in that case, but the
pKVM MMU does not, so it needlessly cleans the whole range on, e.g.,
every set/way trap.

Return early when FWB is enabled, matching the generic walker.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/pkvm.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index f70c601dcf4c..ba6570e37545 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -518,6 +518,10 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	struct pkvm_mapping *mapping;
 
 	lockdep_assert_held(&kvm->mmu_lock);
+
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return 0;
+
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
 		if (!mapping->nc)
 			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
-- 
2.39.5

[PATCH v5 5/7] KVM: arm64: Don't WARN on pKVM stage-2 map failures

[Fuad Tabba]

pkvm_pgtable_stage2_map() wraps the __pkvm_host_share_guest() and
__pkvm_host_donate_guest() return in WARN_ON(), but those hypercalls
fail for reasons that are not EL1 invariant violations: -EINVAL for a
pfn that is not memblock RAM (check_range_allowed_memory() rejects a
device page mapped into a non-protected guest) and -ENOMEM under
memcache pressure. Both are reachable from a guest fault, so the WARN
splats on host input.

Return the error without warning. The unshare and write-protect WARNs
stay, since a failure there does signal a broken EL1 invariant.

Fixes: 3669ddd8fa8b5 ("KVM: arm64: Add a range to pkvm_mappings")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/pkvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index ba6570e37545..1212a2c5613d 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -466,7 +466,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
 					size / PAGE_SIZE, prot);
 	}
 
-	if (WARN_ON(ret))
+	if (ret)
 		return ret;
 
 	swap(mapping, cache->mapping);
-- 
2.39.5

[PATCH v5 6/7] KVM: arm64: Don't advertise eager page splitting under pKVM

[Fuad Tabba]

Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
enabled and reported for non-protected guests: the capability check keys
on the per-VM protected state while the walker dispatch keys on the
host-global mode. Enabling the cap and then dirty-logging the guest
reaches the stub, splatting a userspace-reachable WARN.

Reject the capability, and stop reporting a chunk size and the
supported block sizes, for every VM once pKVM is enabled, keyed on the
host-global mode like the split dispatch. Gating only protected VMs
would leave the non-protected guests that reach the stub still able to
enable it. Userspace decides whether eager splitting is available from
the block-size bitmap (QEMU falls back to no eager splitting when it
reads 0), so leaving it advertised steers an explicit request into the
enable failure instead of the fallback.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_pkvm.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 57afb07d6b13..beea00e693a0 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -45,6 +45,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
 		return true;
 	case KVM_CAP_ARM_MTE:
 		return false;
+	case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
+	case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
+		return false;
 	default:
 		return !kvm || !kvm_vm_is_protected(kvm);
 	}
-- 
2.39.5

[PATCH v5 7/7] KVM: arm64: selftests: Add stage-2 block transition test

[Fuad Tabba]

Add a test for the two stage-2 granularity changes dirty logging forces
at fault time, asserting the guest completes with no KVM_RUN error. The
first scenario collapses a page into a hugetlb-backed block: it writes
under logging, re-write-protects the page via GET_DIRTY_LOG, then writes
again with logging off. The second splits blocks: it faults in several
non-executable 2M blocks, enables logging, then executes in each block so
an execute permission fault splits it. It is skipped when CTR_EL0.DIC is
set, since mappings are then executable and no execute fault occurs.

Both paths make the fault handler allocate under mmu_lock, so a backend
that fails to stage that memory returns a KVM_RUN error or crashes the
host. The property holds on any host. On a pKVM host, where a
non-protected guest uses the pkvm_pgtable_*() backend, it also guards
that backend's fault-time staging.

Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/arm64/stage2_block_transitions.c      | 226 ++++++++++++++++++
 2 files changed, 227 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/stage2_block_transitions.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index d28a057fa6c2..ed2877825eee 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -179,6 +179,7 @@ TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
 TEST_GEN_PROGS_arm64 += arm64/smccc_filter
+TEST_GEN_PROGS_arm64 += arm64/stage2_block_transitions
 TEST_GEN_PROGS_arm64 += arm64/vcpu_width_config
 TEST_GEN_PROGS_arm64 += arm64/vgic_init
 TEST_GEN_PROGS_arm64 += arm64/vgic_irq
diff --git a/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
new file mode 100644
index 000000000000..5fd47f4ada1f
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ *
+ * stage2_block_transitions - Exercise stage-2 block/page granularity changes
+ * that dirty logging forces at fault time, and assert the guest completes.
+ *
+ * Both scenarios need the fault handler to allocate at fault time (a fresh
+ * mapping and/or page-table pages while holding mmu_lock), so a fault path
+ * that fails to stage that memory manifests as a KVM_RUN error or, worse, a
+ * host crash. The asserted property is host-agnostic: the guest runs the
+ * sequence to completion and every KVM_RUN succeeds. On a pKVM host, where a
+ * non-protected guest's stage-2 faults are serviced by the pkvm_pgtable_*()
+ * backend, the same sequences also guard that backend's fault-time staging.
+ *
+ * Scenario 1 - block collapse on dirty-logging disable:
+ *   A write under dirty logging installs a 4K page; GET_DIRTY_LOG
+ *   re-write-protects it; logging is disabled; a second write takes a
+ *   permission fault that collapses the page into a hugetlb-backed block,
+ *   which requires a fresh mapping object under mmu_lock.
+ *
+ * Scenario 2 - block split under dirty logging:
+ *   Several hugetlb-backed blocks are faulted in as non-executable blocks,
+ *   dirty logging is enabled (write-protect only), then the guest executes
+ *   into each block. Each instruction fetch takes an execute permission
+ *   fault that must split the block into pages during logging, draining
+ *   page-table pages. Skipped on CTR_EL0.DIC hardware, where mappings are
+ *   made executable eagerly and the execute fault never occurs.
+ */
+#include <linux/bitfield.h>
+#include <linux/bitmap.h>
+#include <linux/mman.h>
+#include <linux/sizes.h>
+#include <sys/mman.h>
+
+#include <asm/sysreg.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+#include "ucall.h"
+
+#define DATA_SLOT		1
+#define TEST_GVA		0xc0000000UL
+#define BLOCK_SIZE		SZ_2M
+
+/* AArch64 "ret" (ret x30): a self-contained, returnable executable payload. */
+#define RET_INSN		0xd65f03c0U
+
+/*
+ * A non-protected guest's per-VM stage-2 pool is seeded only with the PGD
+ * donation, which stage-2 init immediately consumes, so the page-table budget
+ * for a fault that does not top up is just the handful (~2x the stage-2 min
+ * pages) of memcache leftovers. Executing into this many distinct blocks
+ * demands far more than that budget: a fault path that tops up on every fault
+ * completes all of them, one that skips non-write faults runs out mid-sequence.
+ */
+#define NR_BLOCKS		16
+
+/* Scenario 2 guest -> host sync stages. */
+#define STAGE_SKIP_DIC		1
+#define STAGE_BLOCKS_READY	2
+
+static void collapse_guest_code(u64 gva)
+{
+	u64 *data = (u64 *)gva;
+
+	/* Under dirty logging: install a 4K writable page. */
+	WRITE_ONCE(*data, 0x1);
+	GUEST_SYNC(1);
+
+	/* Logging disabled: a permission fault collapses the page into a block. */
+	WRITE_ONCE(*data, 0x2);
+	GUEST_SYNC(2);
+
+	GUEST_DONE();
+}
+
+static void test_block_collapse(void)
+{
+	struct kvm_vcpu *vcpu;
+	unsigned long *bmap;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	size_t npages;
+	u64 gpa;
+
+	vm = vm_create_with_one_vcpu(&vcpu, collapse_guest_code);
+	npages = BLOCK_SIZE / vm->page_size;
+
+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - BLOCK_SIZE;
+	gpa = align_down(gpa, BLOCK_SIZE);
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
+				    DATA_SLOT, npages, KVM_MEM_LOG_DIRTY_PAGES);
+	virt_map(vm, TEST_GVA, gpa, npages);
+	vcpu_args_set(vcpu, 1, TEST_GVA);
+
+	bmap = bitmap_zalloc(BLOCK_SIZE / getpagesize());
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 1,
+		    "Expected first sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
+
+	/* GET_DIRTY_LOG re-write-protects the dirtied page; then stop logging. */
+	kvm_vm_get_dirty_log(vm, DATA_SLOT, bmap);
+	vm_mem_region_set_flags(vm, DATA_SLOT, 0);
+
+	/* The collapsing permission fault: a broken fault path faults here. */
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 2,
+		    "Expected second sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
+		    "Expected done, got cmd %lu", uc.cmd);
+
+	free(bmap);
+	kvm_vm_free(vm);
+}
+
+static void guest_sync_insn(u64 va)
+{
+	/* Make the just-written instruction coherent for execution (!DIC). */
+	asm volatile("dc cvau, %0\n"
+		     "dsb ish\n"
+		     "ic ivau, %0\n"
+		     "dsb ish\n"
+		     "isb\n"
+		     :: "r" (va) : "memory");
+}
+
+static void split_guest_code(u64 base_gva, u64 nblocks)
+{
+	u64 i, va;
+
+	if (FIELD_GET(CTR_EL0_DIC_MASK, read_sysreg(ctr_el0))) {
+		GUEST_SYNC(STAGE_SKIP_DIC);
+		GUEST_DONE();
+		return;
+	}
+
+	/* Fault in each block (non-executable) and stage an executable payload. */
+	for (i = 0; i < nblocks; i++) {
+		va = base_gva + i * BLOCK_SIZE;
+		WRITE_ONCE(*(u32 *)va, RET_INSN);
+		guest_sync_insn(va);
+	}
+	GUEST_SYNC(STAGE_BLOCKS_READY);
+
+	/* Logging is now on: executing into each block splits it into pages. */
+	for (i = 0; i < nblocks; i++) {
+		va = base_gva + i * BLOCK_SIZE;
+		((void (*)(void))va)();
+	}
+
+	GUEST_DONE();
+}
+
+static void test_exec_split_drain(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	size_t npages;
+	u64 gpa;
+
+	vm = vm_create_with_one_vcpu(&vcpu, split_guest_code);
+	npages = NR_BLOCKS * (BLOCK_SIZE / vm->page_size);
+
+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - NR_BLOCKS * BLOCK_SIZE;
+	gpa = align_down(gpa, BLOCK_SIZE);
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
+				    DATA_SLOT, npages, 0);
+	virt_map(vm, TEST_GVA, gpa, npages);
+	vcpu_args_set(vcpu, 2, TEST_GVA, (u64)NR_BLOCKS);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC,
+		    "Expected sync, got cmd %lu", uc.cmd);
+	if (uc.args[1] == STAGE_SKIP_DIC) {
+		ksft_print_msg("SKIP block split: CTR_EL0.DIC == 1\n");
+		kvm_vm_free(vm);
+		return;
+	}
+	TEST_ASSERT(uc.args[1] == STAGE_BLOCKS_READY,
+		    "Expected blocks-ready sync, got arg %lu", uc.args[1]);
+
+	/* Write-protect the blocks; the guest then splits them by executing. */
+	vm_mem_region_set_flags(vm, DATA_SLOT, KVM_MEM_LOG_DIRTY_PAGES);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
+		    "Expected done, got cmd %lu", uc.cmd);
+
+	kvm_vm_free(vm);
+}
+
+/*
+ * The explicit-size hugetlb backing hard-fails region creation if the pages
+ * are not already reserved, so probe here and skip rather than abort. The
+ * peak reservation is scenario 2's; the two scenarios run and free in turn.
+ */
+static void require_hugepages(size_t bytes)
+{
+	void *mem = mmap(NULL, bytes, PROT_READ | PROT_WRITE,
+			 MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_2MB,
+			 -1, 0);
+
+	if (mem == MAP_FAILED)
+		ksft_exit_skip("Need %zu bytes of reserved 2M hugepages\n", bytes);
+	munmap(mem, bytes);
+}
+
+int main(void)
+{
+	require_hugepages(NR_BLOCKS * BLOCK_SIZE);
+
+	test_block_collapse();
+	test_exec_split_drain();
+
+	ksft_print_msg("All ok!\n");
+	return 0;
+}
-- 
2.39.5

Re: [PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] A malicious nested (L1) guest can crash the KVM host via a Break-Before-Make (BBM) violation that triggers a BUG_ON() due to an empty memory cache during a permission fault.
--

--- Patch [3]: [PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults ---
commit bac5488834df4c0a16b08356c8263e8eb67e0fe4
Author: Bradley Morgan <include@grrlz.net>

KVM: arm64: Top up stage-2 memcache for dirty logging faults

Dirty logging forces new stage-2 mappings to page size but does not always
split an existing block first. A non-write permission fault on such a block
still needs a page-table page to split it, but the top-up was previously
gated on write faults. This patch tops up the memcache for any permission
fault while dirty logging is active.

Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/

> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 4d7c9bdcef004..74e7e7f7564c8 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -2113,14 +2113,13 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
>  	 * Permission faults just need to update the existing leaf entry,
>  	 * and so normally don't require allocations from the memcache. The
>  	 * only exception to this is when dirty logging is enabled at runtime
> -	 * and a write fault needs to collapse a block entry into a table.
> +	 * and a fault needs to collapse a block entry into a table.
>  	 * Under pKVM a permission fault can also collapse pages into a block,
>  	 * which needs a fresh mapping object, and the hypervisor requires the
>  	 * min-pages memcache even when the install allocates nothing.
>  	 */
>  	memcache = get_mmu_memcache(s2fd->vcpu);
> -	if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
> -			    kvm_is_write_fault(s2fd->vcpu)) ||
> +	if (!perm_fault || memslot_is_logging(s2fd->memslot) ||
>  	    is_protected_kvm_enabled()) {

[Severity: High]
This is a pre-existing issue, but does skipping the memcache top-up here
leave the host vulnerable to a crash from a malicious nested L1 guest?

If a nested L1 guest establishes a 2MB stage-2 block mapping, and then
maliciously overwrites it with a 4KB page table descriptor without issuing
a TLBI (an architectural Break-Before-Make violation), an L2 guest can
trigger a permission fault on the stale 2MB shadow mapping.

Since KVM optimizes permission faults by assuming they only relax
permissions, the memcache top-up is skipped here.

However, in kvm_s2_fault_map(), if the mapping size and fault granule
don't match:

arch/arm64/kvm/mmu.c:kvm_s2_fault_map()
    ...
	} else {
		ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(gfn), mapping_size,
							 __pfn_to_phys(pfn), prot,
							 memcache, flags);
	}
    ...

kvm_pgtable_stage2_map() will attempt to split the block, which requires
a new page table allocation. Because the cache wasn't topped up,
kvm_mmu_memory_cache_alloc() will hit WARN_ON(!mc->nobjs) and fall back
to a GFP_ATOMIC allocation. If this atomic allocation fails under memory
pressure, it hits BUG_ON(!p) and crashes the host.

>  		ret = topup_mmu_memcache(s2fd->vcpu, memcache);
>  		if (ret)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260717130317.1953574-1-fuad.tabba@linux.dev?part=3

Re: [PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults

[Fuad Tabba]

On Fri, 17 Jul 2026 at 14:15, <sashiko-bot@kernel.org> wrote:
>
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
>
> Pre-existing issues:
> - [High] A malicious nested (L1) guest can crash the KVM host via a Break-Before-Make (BBM) violation that triggers a BUG_ON() due to an empty memory cache during a permission fault.

From my understanding of NV code, this looks like a real issue. I
don't think it changes this patch, though: this one only widens the
top-up for the dirty-logging case, and the path you describe is a
non-logging permission fault. So, it would be a separate fix, a top up
for nested permission faults too, the same way we already do for pKVM?

        if (!perm_fault || memslot_is_logging(s2fd->memslot) ||
            is_protected_kvm_enabled() || s2fd->nested) {

If that's right, I'll send a separate fix later. Marc, what do you think?

/fuad



/fuad

> --
>
> --- Patch [3]: [PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults ---
> commit bac5488834df4c0a16b08356c8263e8eb67e0fe4
> Author: Bradley Morgan <include@grrlz.net>
>
> KVM: arm64: Top up stage-2 memcache for dirty logging faults
>
> Dirty logging forces new stage-2 mappings to page size but does not always
> split an existing block first. A non-write permission fault on such a block
> still needs a page-table page to split it, but the top-up was previously
> gated on write faults. This patch tops up the memcache for any permission
> fault while dirty logging is active.
>
> Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/
>
> > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> > index 4d7c9bdcef004..74e7e7f7564c8 100644
> > --- a/arch/arm64/kvm/mmu.c
> > +++ b/arch/arm64/kvm/mmu.c
> > @@ -2113,14 +2113,13 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
> >        * Permission faults just need to update the existing leaf entry,
> >        * and so normally don't require allocations from the memcache. The
> >        * only exception to this is when dirty logging is enabled at runtime
> > -      * and a write fault needs to collapse a block entry into a table.
> > +      * and a fault needs to collapse a block entry into a table.
> >        * Under pKVM a permission fault can also collapse pages into a block,
> >        * which needs a fresh mapping object, and the hypervisor requires the
> >        * min-pages memcache even when the install allocates nothing.
> >        */
> >       memcache = get_mmu_memcache(s2fd->vcpu);
> > -     if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
> > -                         kvm_is_write_fault(s2fd->vcpu)) ||
> > +     if (!perm_fault || memslot_is_logging(s2fd->memslot) ||
> >           is_protected_kvm_enabled()) {
>
> [Severity: High]
> This is a pre-existing issue, but does skipping the memcache top-up here
> leave the host vulnerable to a crash from a malicious nested L1 guest?
>
> If a nested L1 guest establishes a 2MB stage-2 block mapping, and then
> maliciously overwrites it with a 4KB page table descriptor without issuing
> a TLBI (an architectural Break-Before-Make violation), an L2 guest can
> trigger a permission fault on the stale 2MB shadow mapping.
>
> Since KVM optimizes permission faults by assuming they only relax
> permissions, the memcache top-up is skipped here.
>
> However, in kvm_s2_fault_map(), if the mapping size and fault granule
> don't match:
>
> arch/arm64/kvm/mmu.c:kvm_s2_fault_map()
>     ...
>         } else {
>                 ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(gfn), mapping_size,
>                                                          __pfn_to_phys(pfn), prot,
>                                                          memcache, flags);
>         }
>     ...
>
> kvm_pgtable_stage2_map() will attempt to split the block, which requires
> a new page table allocation. Because the cache wasn't topped up,
> kvm_mmu_memory_cache_alloc() will hit WARN_ON(!mc->nobjs) and fall back
> to a GFP_ATOMIC allocation. If this atomic allocation fails under memory
> pressure, it hits BUG_ON(!p) and crashes the host.
>
> >               ret = topup_mmu_memcache(s2fd->vcpu, memcache);
> >               if (ret)
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20260717130317.1953574-1-fuad.tabba@linux.dev?part=3
>