[PATCH v5 0/7] KVM: arm64: pKVM stage-2 mapping and memcache fixes

[PATCH v5 0/7] KVM: arm64: pKVM stage-2 mapping and memcache fixes

[Fuad Tabba]

Hi folks,

Changes since v4 [1]:
  - Restored the permission-fault and dirty-logging memcache top-ups
    that v4 dropped. Both fix real bugs.
  - Patch 1 uses the anonymous bitfield encoding rather than the
    open-coded mask and helpers. (Marc)
  - Reshaped the permission-fault top-up to stage the full memcache
    under pKVM, not just the mapping object. The object-only form still
    returned -ENOMEM and tripped a WARN under the hypervisor's
    unconditional min-pages check.
  - Re-scoped the dirty-logging top-up to its generic, non-pKVM failure
    mode, now that the permission-fault patch covers pKVM.
  - Added three adjacent fixes found while going through the series, and
    a selftest for the block transitions.
  - Patches 1 and 3 keep Bradley's Signed-off-by from v3; my changes
    to each are noted in a [tabba: ...] line.

I picked this up while reviewing Bradley's "mapping cache" series [2]:
v4 dropped two fixes from v3 that address real bugs, so I've collected
the three fixes back together, reshaped patch 1 per Marc's review [3],
added a few adjacent fixes I found along the way and a selftest, and am
reposting as v5.

Most of the fixes are in the pKVM stage-2 walker. On a pKVM host a
non-protected guest's stage-2 faults go through pkvm_pgtable_*(), which
diverges from the generic walker in several ways that are bugs: cache
maintenance on non-cacheable mappings (patch 1), a missing memcache
top-up on permission faults that under pKVM still allocate (patch 2), a
full flush walk on FWB hardware (patch 4), a WARN on a guest-reachable
map failure (patch 5), and an eager-split capability whose pKVM backend
is only a stub (patch 6). These affect non-protected guests only, since
dispatch and memcache selection key on the host-global pKVM mode, and
protected guests take no permission faults.

Patch 3 is not pKVM-specific. During dirty logging a non-write
permission fault, an instruction fetch, still needs a page-table page
to split a block, but the memcache top-up is gated on write faults.
That fault path is generic, so the fix is too.

The two memcache top-ups came from sashiko review-bot findings [4][5],
and both check out against the code.

The series is structured as follows:

  1:    Skip cache maintenance for non-cacheable mappings.
  2-3:  Top up the memcache for the permission and dirty-logging faults
        that force stage-2 block transitions.
  4-6:  Adjacent pkvm_pgtable_*() fixes: FWB flush early-out, drop a
        spurious map WARN, and gate the eager-split capability.
  7:    Selftest for the block-collapse and block-split transitions.

Testing: the selftest in patch 7 covers the two dirty-logging block
transitions (page->block collapse and block->page split) that patch 2
stages; on the base kernel the collapse oopses the host in
pkvm_pgtable_stage2_map() with a NULL dereference under mmu_lock, and
with the series applied it passes. It is a standalone test rather than
an extension of kvm_page_table_test, whose default anonymous-4K backing
forms no huge-page blocks, so an automated run never exercises these
transitions, and whose worker/stage harness does not fit the multi-stage
logging sequence. Run it with 2M hugepages
reserved and, for the split half, on a CPU with CTR_EL0.DIC == 0 (e.g.
-cpu cortex-a710 under QEMU); it self-skips those otherwise. It also
passes on a non-pKVM host (VHE and nVHE), where patch 3's generic change
applies. The other fixes are not exercised by the selftest and rest on
the analysis in their commit messages: patch 3's fault path is generic
and non-pKVM (under pKVM patch 2 already tops it up, and its failure is
a WARN_ON(!nobjs) in kvm_mmu_memory_cache_alloc(), not a KVM_RUN error),
and patches 1 and 4-6 each need a specific pKVM configuration to hit.

Based on Linux v7.2-rc3 (a13c140cc289).

Cheers,
/fuad

[1] https://lore.kernel.org/r/20260701192428.17430-1-include@grrlz.net
[2] https://lore.kernel.org/r/20260624160028.15591-1-include@grrlz.net
[3] https://lore.kernel.org/r/86qzllpy1g.wl-maz@kernel.org
[4] https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
[5] https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/

Bradley Morgan (2):
  KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
  KVM: arm64: Top up stage-2 memcache for dirty logging faults

Fuad Tabba (5):
  KVM: arm64: Top up the memcache for pKVM permission faults
  KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
  KVM: arm64: Don't WARN on pKVM stage-2 map failures
  KVM: arm64: Don't advertise eager page splitting under pKVM
  KVM: arm64: selftests: Add stage-2 block transition test

 arch/arm64/include/asm/kvm_pkvm.h             |   8 +-
 arch/arm64/kvm/mmu.c                          |   9 +-
 arch/arm64/kvm/pkvm.c                         |  21 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/arm64/stage2_block_transitions.c      | 226 ++++++++++++++++++
 5 files changed, 254 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/arm64/stage2_block_transitions.c


base-commit: a13c140cc289c0b7b3770bce5b3ad42ab35074aa
-- 
2.39.5

[PATCH v5 1/7] KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings

[Fuad Tabba]

From: Bradley Morgan <include@grrlz.net>

The pKVM flush path walks its own pkvm_mappings list and cleans the
data cache for every mapping, unlike the generic stage-2 walker it
shadows, which skips non-cacheable leaves. Cleaning the cacheable
alias of a non-cacheable mapping is pointless and can corrupt a
device endpoint. Record whether a mapping is non-cacheable in spare
bits of nr_pages and skip cache maintenance for it.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: use Marc's anonymous bitfield in place of the open-coded mask and helpers]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_pkvm.h |  5 ++++-
 arch/arm64/kvm/pkvm.c             | 15 +++++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 74fedd9c5ff0..57afb07d6b13 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -195,7 +195,10 @@ struct pkvm_mapping {
 	struct rb_node node;
 	u64 gfn;
 	u64 pfn;
-	u64 nr_pages;
+	struct {
+		u64 nr_pages:48;
+		u64 nc:1;
+	};
 	u64 __subtree_last;	/* Internal member for interval tree */
 };
 
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 053e4f733e4b..f70c601dcf4c 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -369,7 +369,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64
 
 	for_each_mapping_in_range_safe(pgt, start, end, mapping) {
 		ret = kvm_call_hyp_nvhe(__pkvm_host_unshare_guest, handle, mapping->gfn,
-					mapping->nr_pages);
+					(u64)mapping->nr_pages);
 		if (WARN_ON(ret))
 			return ret;
 		pkvm_mapping_remove(mapping, &pgt->pkvm_mappings);
@@ -473,6 +473,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
 	mapping->gfn = gfn;
 	mapping->pfn = pfn;
 	mapping->nr_pages = size / PAGE_SIZE;
+	mapping->nc = !!(prot & (KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC));
 	pkvm_mapping_insert(mapping, &pgt->pkvm_mappings);
 
 	return ret;
@@ -503,7 +504,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	lockdep_assert_held(&kvm->mmu_lock);
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
 		ret = kvm_call_hyp_nvhe(__pkvm_host_wrprotect_guest, handle, mapping->gfn,
-					mapping->nr_pages);
+					(u64)mapping->nr_pages);
 		if (WARN_ON(ret))
 			break;
 	}
@@ -517,9 +518,11 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	struct pkvm_mapping *mapping;
 
 	lockdep_assert_held(&kvm->mmu_lock);
-	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
-		__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
-					  PAGE_SIZE * mapping->nr_pages);
+	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
+		if (!mapping->nc)
+			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
+						  PAGE_SIZE * mapping->nr_pages);
+	}
 
 	return 0;
 }
@@ -537,7 +540,7 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64
 	lockdep_assert_held(&kvm->mmu_lock);
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
 		young |= kvm_call_hyp_nvhe(__pkvm_host_test_clear_young_guest, handle, mapping->gfn,
-					   mapping->nr_pages, mkold);
+					   (u64)mapping->nr_pages, mkold);
 
 	return young;
 }
-- 
2.39.5

[PATCH v5 2/7] KVM: arm64: Top up the memcache for pKVM permission faults

[Fuad Tabba]

A permission fault normally only relaxes a leaf, so user_mem_abort()
skips the memcache top-up. Under pKVM such a fault can instead collapse
pages into a block. That needs a fresh pkvm_mapping object, and without
it cache->mapping is NULL, so pkvm_pgtable_stage2_map() dereferences NULL
and faults the host under mmu_lock. Staging only the object is not
enough: the hypervisor requires kvm_mmu_cache_min_pages in the memcache
even for the allocation-free install, so under memcache pressure the
collapse returns -ENOMEM and trips the WARN_ON(ret) in
pkvm_pgtable_stage2_map() where a non-pKVM guest succeeds.

Top up the full memcache for pKVM permission faults so both the mapping
object and the min-pages are staged before mmu_lock.

Fixes: db14091d8f75 ("KVM: arm64: Stage-2 huge mappings for np-guests")
Reported-by: Bradley Morgan <include@grrlz.net>
Link: https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6c941aaa10c6..4d7c9bdcef00 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2114,10 +2114,14 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 	 * and so normally don't require allocations from the memcache. The
 	 * only exception to this is when dirty logging is enabled at runtime
 	 * and a write fault needs to collapse a block entry into a table.
+	 * Under pKVM a permission fault can also collapse pages into a block,
+	 * which needs a fresh mapping object, and the hypervisor requires the
+	 * min-pages memcache even when the install allocates nothing.
 	 */
 	memcache = get_mmu_memcache(s2fd->vcpu);
 	if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
-			    kvm_is_write_fault(s2fd->vcpu))) {
+			    kvm_is_write_fault(s2fd->vcpu)) ||
+	    is_protected_kvm_enabled()) {
 		ret = topup_mmu_memcache(s2fd->vcpu, memcache);
 		if (ret)
 			return ret;
-- 
2.39.5

[PATCH v5 3/7] KVM: arm64: Top up stage-2 memcache for dirty logging faults

[Fuad Tabba]

From: Bradley Morgan <include@grrlz.net>

Dirty logging forces new stage-2 mappings to page size but does not
always split an existing block first (eager splitting is best effort
and off by default). A non-write permission fault on such a block, an
instruction fetch, still needs a page-table page to split it, but the
top-up is gated on write faults.

With the cache empty, kvm_mmu_memory_cache_alloc() hits its
guest-triggerable WARN_ON(!nobjs) and falls back to a GFP_ATOMIC
allocation under mmu_lock, with a BUG_ON() if that fails.

Top up the memcache for any permission fault while dirty logging is
active.

Fixes: 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to generic page-table API")
Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: reword the commit message for the generic, non-pKVM failure mode]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 4d7c9bdcef00..74e7e7f7564c 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2113,14 +2113,13 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 	 * Permission faults just need to update the existing leaf entry,
 	 * and so normally don't require allocations from the memcache. The
 	 * only exception to this is when dirty logging is enabled at runtime
-	 * and a write fault needs to collapse a block entry into a table.
+	 * and a fault needs to collapse a block entry into a table.
 	 * Under pKVM a permission fault can also collapse pages into a block,
 	 * which needs a fresh mapping object, and the hypervisor requires the
 	 * min-pages memcache even when the install allocates nothing.
 	 */
 	memcache = get_mmu_memcache(s2fd->vcpu);
-	if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
-			    kvm_is_write_fault(s2fd->vcpu)) ||
+	if (!perm_fault || memslot_is_logging(s2fd->memslot) ||
 	    is_protected_kvm_enabled()) {
 		ret = topup_mmu_memcache(s2fd->vcpu, memcache);
 		if (ret)
-- 
2.39.5

[PATCH v5 4/7] KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled

[Fuad Tabba]

pkvm_pgtable_stage2_flush() cleans the D-cache for every mapping in the
range even on hardware with stage-2 Force Write-Back, where FWB keeps
guest memory coherent to the PoC and the maintenance is unnecessary. The
generic kvm_pgtable_stage2_flush() returns early in that case, but the
pKVM MMU does not, so it needlessly cleans the whole range on, e.g.,
every set/way trap.

Return early when FWB is enabled, matching the generic walker.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/pkvm.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index f70c601dcf4c..ba6570e37545 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -518,6 +518,10 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
 	struct pkvm_mapping *mapping;
 
 	lockdep_assert_held(&kvm->mmu_lock);
+
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return 0;
+
 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
 		if (!mapping->nc)
 			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
-- 
2.39.5

[PATCH v5 5/7] KVM: arm64: Don't WARN on pKVM stage-2 map failures

[Fuad Tabba]

pkvm_pgtable_stage2_map() wraps the __pkvm_host_share_guest() and
__pkvm_host_donate_guest() return in WARN_ON(), but those hypercalls
fail for reasons that are not EL1 invariant violations: -EINVAL for a
pfn that is not memblock RAM (check_range_allowed_memory() rejects a
device page mapped into a non-protected guest) and -ENOMEM under
memcache pressure. Both are reachable from a guest fault, so the WARN
splats on host input.

Return the error without warning. The unshare and write-protect WARNs
stay, since a failure there does signal a broken EL1 invariant.

Fixes: 3669ddd8fa8b5 ("KVM: arm64: Add a range to pkvm_mappings")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/pkvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index ba6570e37545..1212a2c5613d 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -466,7 +466,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
 					size / PAGE_SIZE, prot);
 	}
 
-	if (WARN_ON(ret))
+	if (ret)
 		return ret;
 
 	swap(mapping, cache->mapping);
-- 
2.39.5

[PATCH v5 6/7] KVM: arm64: Don't advertise eager page splitting under pKVM

[Fuad Tabba]

Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
enabled and reported for non-protected guests: the capability check keys
on the per-VM protected state while the walker dispatch keys on the
host-global mode. Enabling the cap and then dirty-logging the guest
reaches the stub, splatting a userspace-reachable WARN.

Reject the capability, and stop reporting a chunk size and the
supported block sizes, for every VM once pKVM is enabled, keyed on the
host-global mode like the split dispatch. Gating only protected VMs
would leave the non-protected guests that reach the stub still able to
enable it. Userspace decides whether eager splitting is available from
the block-size bitmap (QEMU falls back to no eager splitting when it
reads 0), so leaving it advertised steers an explicit request into the
enable failure instead of the fallback.

Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_pkvm.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 57afb07d6b13..beea00e693a0 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -45,6 +45,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
 		return true;
 	case KVM_CAP_ARM_MTE:
 		return false;
+	case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
+	case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
+		return false;
 	default:
 		return !kvm || !kvm_vm_is_protected(kvm);
 	}
-- 
2.39.5

[PATCH v5 7/7] KVM: arm64: selftests: Add stage-2 block transition test

[Fuad Tabba]

Add a test for the two stage-2 granularity changes dirty logging forces
at fault time, asserting the guest completes with no KVM_RUN error. The
first scenario collapses a page into a hugetlb-backed block: it writes
under logging, re-write-protects the page via GET_DIRTY_LOG, then writes
again with logging off. The second splits blocks: it faults in several
non-executable 2M blocks, enables logging, then executes in each block so
an execute permission fault splits it. It is skipped when CTR_EL0.DIC is
set, since mappings are then executable and no execute fault occurs.

Both paths make the fault handler allocate under mmu_lock, so a backend
that fails to stage that memory returns a KVM_RUN error or crashes the
host. The property holds on any host. On a pKVM host, where a
non-protected guest uses the pkvm_pgtable_*() backend, it also guards
that backend's fault-time staging.

Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/arm64/stage2_block_transitions.c      | 226 ++++++++++++++++++
 2 files changed, 227 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/stage2_block_transitions.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index d28a057fa6c2..ed2877825eee 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -179,6 +179,7 @@ TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
 TEST_GEN_PROGS_arm64 += arm64/smccc_filter
+TEST_GEN_PROGS_arm64 += arm64/stage2_block_transitions
 TEST_GEN_PROGS_arm64 += arm64/vcpu_width_config
 TEST_GEN_PROGS_arm64 += arm64/vgic_init
 TEST_GEN_PROGS_arm64 += arm64/vgic_irq
diff --git a/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
new file mode 100644
index 000000000000..5fd47f4ada1f
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ *
+ * stage2_block_transitions - Exercise stage-2 block/page granularity changes
+ * that dirty logging forces at fault time, and assert the guest completes.
+ *
+ * Both scenarios need the fault handler to allocate at fault time (a fresh
+ * mapping and/or page-table pages while holding mmu_lock), so a fault path
+ * that fails to stage that memory manifests as a KVM_RUN error or, worse, a
+ * host crash. The asserted property is host-agnostic: the guest runs the
+ * sequence to completion and every KVM_RUN succeeds. On a pKVM host, where a
+ * non-protected guest's stage-2 faults are serviced by the pkvm_pgtable_*()
+ * backend, the same sequences also guard that backend's fault-time staging.
+ *
+ * Scenario 1 - block collapse on dirty-logging disable:
+ *   A write under dirty logging installs a 4K page; GET_DIRTY_LOG
+ *   re-write-protects it; logging is disabled; a second write takes a
+ *   permission fault that collapses the page into a hugetlb-backed block,
+ *   which requires a fresh mapping object under mmu_lock.
+ *
+ * Scenario 2 - block split under dirty logging:
+ *   Several hugetlb-backed blocks are faulted in as non-executable blocks,
+ *   dirty logging is enabled (write-protect only), then the guest executes
+ *   into each block. Each instruction fetch takes an execute permission
+ *   fault that must split the block into pages during logging, draining
+ *   page-table pages. Skipped on CTR_EL0.DIC hardware, where mappings are
+ *   made executable eagerly and the execute fault never occurs.
+ */
+#include <linux/bitfield.h>
+#include <linux/bitmap.h>
+#include <linux/mman.h>
+#include <linux/sizes.h>
+#include <sys/mman.h>
+
+#include <asm/sysreg.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+#include "ucall.h"
+
+#define DATA_SLOT		1
+#define TEST_GVA		0xc0000000UL
+#define BLOCK_SIZE		SZ_2M
+
+/* AArch64 "ret" (ret x30): a self-contained, returnable executable payload. */
+#define RET_INSN		0xd65f03c0U
+
+/*
+ * A non-protected guest's per-VM stage-2 pool is seeded only with the PGD
+ * donation, which stage-2 init immediately consumes, so the page-table budget
+ * for a fault that does not top up is just the handful (~2x the stage-2 min
+ * pages) of memcache leftovers. Executing into this many distinct blocks
+ * demands far more than that budget: a fault path that tops up on every fault
+ * completes all of them, one that skips non-write faults runs out mid-sequence.
+ */
+#define NR_BLOCKS		16
+
+/* Scenario 2 guest -> host sync stages. */
+#define STAGE_SKIP_DIC		1
+#define STAGE_BLOCKS_READY	2
+
+static void collapse_guest_code(u64 gva)
+{
+	u64 *data = (u64 *)gva;
+
+	/* Under dirty logging: install a 4K writable page. */
+	WRITE_ONCE(*data, 0x1);
+	GUEST_SYNC(1);
+
+	/* Logging disabled: a permission fault collapses the page into a block. */
+	WRITE_ONCE(*data, 0x2);
+	GUEST_SYNC(2);
+
+	GUEST_DONE();
+}
+
+static void test_block_collapse(void)
+{
+	struct kvm_vcpu *vcpu;
+	unsigned long *bmap;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	size_t npages;
+	u64 gpa;
+
+	vm = vm_create_with_one_vcpu(&vcpu, collapse_guest_code);
+	npages = BLOCK_SIZE / vm->page_size;
+
+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - BLOCK_SIZE;
+	gpa = align_down(gpa, BLOCK_SIZE);
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
+				    DATA_SLOT, npages, KVM_MEM_LOG_DIRTY_PAGES);
+	virt_map(vm, TEST_GVA, gpa, npages);
+	vcpu_args_set(vcpu, 1, TEST_GVA);
+
+	bmap = bitmap_zalloc(BLOCK_SIZE / getpagesize());
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 1,
+		    "Expected first sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
+
+	/* GET_DIRTY_LOG re-write-protects the dirtied page; then stop logging. */
+	kvm_vm_get_dirty_log(vm, DATA_SLOT, bmap);
+	vm_mem_region_set_flags(vm, DATA_SLOT, 0);
+
+	/* The collapsing permission fault: a broken fault path faults here. */
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 2,
+		    "Expected second sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
+		    "Expected done, got cmd %lu", uc.cmd);
+
+	free(bmap);
+	kvm_vm_free(vm);
+}
+
+static void guest_sync_insn(u64 va)
+{
+	/* Make the just-written instruction coherent for execution (!DIC). */
+	asm volatile("dc cvau, %0\n"
+		     "dsb ish\n"
+		     "ic ivau, %0\n"
+		     "dsb ish\n"
+		     "isb\n"
+		     :: "r" (va) : "memory");
+}
+
+static void split_guest_code(u64 base_gva, u64 nblocks)
+{
+	u64 i, va;
+
+	if (FIELD_GET(CTR_EL0_DIC_MASK, read_sysreg(ctr_el0))) {
+		GUEST_SYNC(STAGE_SKIP_DIC);
+		GUEST_DONE();
+		return;
+	}
+
+	/* Fault in each block (non-executable) and stage an executable payload. */
+	for (i = 0; i < nblocks; i++) {
+		va = base_gva + i * BLOCK_SIZE;
+		WRITE_ONCE(*(u32 *)va, RET_INSN);
+		guest_sync_insn(va);
+	}
+	GUEST_SYNC(STAGE_BLOCKS_READY);
+
+	/* Logging is now on: executing into each block splits it into pages. */
+	for (i = 0; i < nblocks; i++) {
+		va = base_gva + i * BLOCK_SIZE;
+		((void (*)(void))va)();
+	}
+
+	GUEST_DONE();
+}
+
+static void test_exec_split_drain(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	size_t npages;
+	u64 gpa;
+
+	vm = vm_create_with_one_vcpu(&vcpu, split_guest_code);
+	npages = NR_BLOCKS * (BLOCK_SIZE / vm->page_size);
+
+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - NR_BLOCKS * BLOCK_SIZE;
+	gpa = align_down(gpa, BLOCK_SIZE);
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
+				    DATA_SLOT, npages, 0);
+	virt_map(vm, TEST_GVA, gpa, npages);
+	vcpu_args_set(vcpu, 2, TEST_GVA, (u64)NR_BLOCKS);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC,
+		    "Expected sync, got cmd %lu", uc.cmd);
+	if (uc.args[1] == STAGE_SKIP_DIC) {
+		ksft_print_msg("SKIP block split: CTR_EL0.DIC == 1\n");
+		kvm_vm_free(vm);
+		return;
+	}
+	TEST_ASSERT(uc.args[1] == STAGE_BLOCKS_READY,
+		    "Expected blocks-ready sync, got arg %lu", uc.args[1]);
+
+	/* Write-protect the blocks; the guest then splits them by executing. */
+	vm_mem_region_set_flags(vm, DATA_SLOT, KVM_MEM_LOG_DIRTY_PAGES);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
+		    "Expected done, got cmd %lu", uc.cmd);
+
+	kvm_vm_free(vm);
+}
+
+/*
+ * The explicit-size hugetlb backing hard-fails region creation if the pages
+ * are not already reserved, so probe here and skip rather than abort. The
+ * peak reservation is scenario 2's; the two scenarios run and free in turn.
+ */
+static void require_hugepages(size_t bytes)
+{
+	void *mem = mmap(NULL, bytes, PROT_READ | PROT_WRITE,
+			 MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_2MB,
+			 -1, 0);
+
+	if (mem == MAP_FAILED)
+		ksft_exit_skip("Need %zu bytes of reserved 2M hugepages\n", bytes);
+	munmap(mem, bytes);
+}
+
+int main(void)
+{
+	require_hugepages(NR_BLOCKS * BLOCK_SIZE);
+
+	test_block_collapse();
+	test_exec_split_drain();
+
+	ksft_print_msg("All ok!\n");
+	return 0;
+}
-- 
2.39.5

Re: [PATCH v5 4/7] KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled

[Bradley Morgan]

On July 17, 2026 2:03:14 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>pkvm_pgtable_stage2_flush() cleans the D-cache for every mapping in the
>range even on hardware with stage-2 Force Write-Back, where FWB keeps
>guest memory coherent to the PoC and the maintenance is unnecessary. The
>generic kvm_pgtable_stage2_flush() returns early in that case, but the
>pKVM MMU does not, so it needlessly cleans the whole range on, e.g.,
>every set/way trap.
>
>Return early when FWB is enabled, matching the generic walker.
>
>Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
>Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

LGTM,

Reviewed-by: Bradley Morgan <include@grrlz.net>

>---
> arch/arm64/kvm/pkvm.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
>diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
>index f70c601dcf4c..ba6570e37545 100644
>--- a/arch/arm64/kvm/pkvm.c
>+++ b/arch/arm64/kvm/pkvm.c
>@@ -518,6 +518,10 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
> 	struct pkvm_mapping *mapping;
> 
> 	lockdep_assert_held(&kvm->mmu_lock);
>+
>+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
>+		return 0;
>+
> 	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> 		if (!mapping->nc)
> 			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
>

Thanks!

Re: [PATCH v5 5/7] KVM: arm64: Don't WARN on pKVM stage-2 map failures

[Bradley Morgan]

On July 17, 2026 2:03:15 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>pkvm_pgtable_stage2_map() wraps the __pkvm_host_share_guest() and
>__pkvm_host_donate_guest() return in WARN_ON(), but those hypercalls
>fail for reasons that are not EL1 invariant violations: -EINVAL for a
>pfn that is not memblock RAM (check_range_allowed_memory() rejects a
>device page mapped into a non-protected guest) and -ENOMEM under
>memcache pressure. Both are reachable from a guest fault, so the WARN
>splats on host input.
>
>Return the error without warning. The unshare and write-protect WARNs
>stay, since a failure there does signal a broken EL1 invariant.
>
>Fixes: 3669ddd8fa8b5 ("KVM: arm64: Add a range to pkvm_mappings")
>Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

LGTM,

Reviewed-by: Bradley Morgan <include@grrlz.net>


>---
> arch/arm64/kvm/pkvm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
>index ba6570e37545..1212a2c5613d 100644
>--- a/arch/arm64/kvm/pkvm.c
>+++ b/arch/arm64/kvm/pkvm.c
>@@ -466,7 +466,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
> 					size / PAGE_SIZE, prot);
> 	}
> 
>-	if (WARN_ON(ret))
>+	if (ret)
> 		return ret;
> 
> 	swap(mapping, cache->mapping);
>

Thanks!

Re: [PATCH v5 6/7] KVM: arm64: Don't advertise eager page splitting under pKVM

[Bradley Morgan]

On July 17, 2026 2:03:16 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
>WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
>enabled and reported for non-protected guests: the capability check keys
>on the per-VM protected state while the walker dispatch keys on the
>host-global mode. Enabling the cap and then dirty-logging the guest
>reaches the stub, splatting a userspace-reachable WARN.
>
>Reject the capability, and stop reporting a chunk size and the
>supported block sizes, for every VM once pKVM is enabled, keyed on the
>host-global mode like the split dispatch. Gating only protected VMs
>would leave the non-protected guests that reach the stub still able to
>enable it. Userspace decides whether eager splitting is available from
>the block-size bitmap (QEMU falls back to no eager splitting when it
>reads 0), so leaving it advertised steers an explicit request into the
>enable failure instead of the fallback.
>
>Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
>Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Do you think CC stable would be good? I do but sometimes I'm
embarrassingly wrong

>---
> arch/arm64/include/asm/kvm_pkvm.h | 3 +++
> 1 file changed, 3 insertions(+)
>
>diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
>index 57afb07d6b13..beea00e693a0 100644
>--- a/arch/arm64/include/asm/kvm_pkvm.h
>+++ b/arch/arm64/include/asm/kvm_pkvm.h
>@@ -45,6 +45,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
> 		return true;
> 	case KVM_CAP_ARM_MTE:
> 		return false;
>+	case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
>+	case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
>+		return false;


Nit: KVM_CAP_ARM_MTE right above already returns false, fold the two
new cases into that and drop the duplicate return. And please put a small
comment saying eager splitting is unimplemented under pKVM (WARN
stub), so nobody reads the list later and "fixes" it back, here's a
example, feel free to bikeshed


	case KVM_CAP_ARM_MTE:
	/* eager splitting is unimplemented in pKVM, please don't refix this */
	case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
	case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
		return false;
	default:
		return !kvm || !kvm_vm_is_protected(kvm);
	}




Killing the reporting too is right imo. Qemu reads a 0 block size
bitmap and quietly falls back, an explicit enable eating EINVAL is the
worse failure mode. Good.

if u choose not to, it's ok, please add 

Reviewed-by: Bradley Morgan
<include@grrlz.net>


> 	default:
> 		return !kvm || !kvm_vm_is_protected(kvm);
> 	}
>

Thanks!

Re: [PATCH v5 7/7] KVM: arm64: selftests: Add stage-2 block transition test

[Bradley Morgan]

On July 17, 2026 2:03:17 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>Add a test for the two stage-2 granularity changes dirty logging forces
>at fault time, asserting the guest completes with no KVM_RUN error. The
>first scenario collapses a page into a hugetlb-backed block: it writes
>under logging, re-write-protects the page via GET_DIRTY_LOG, then writes
>again with logging off. The second splits blocks: it faults in several
>non-executable 2M blocks, enables logging, then executes in each block so
>an execute permission fault splits it. It is skipped when CTR_EL0.DIC is
>set, since mappings are then executable and no execute fault occurs.
>
>Both paths make the fault handler allocate under mmu_lock, so a backend
>that fails to stage that memory returns a KVM_RUN error or crashes the
>host. The property holds on any host. On a pKVM host, where a
>non-protected guest uses the pkvm_pgtable_*() backend, it also guards
>that backend's fault-time staging.
>
>Assisted-by: Antigravity:gemini-3.1-pro
>Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Tested-by: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host




>---
> tools/testing/selftests/kvm/Makefile.kvm      |   1 +
> .../kvm/arm64/stage2_block_transitions.c      | 226 ++++++++++++++++++
> 2 files changed, 227 insertions(+)
> create mode 100644
> tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
>
>diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
>index d28a057fa6c2..ed2877825eee 100644
>--- a/tools/testing/selftests/kvm/Makefile.kvm
>+++ b/tools/testing/selftests/kvm/Makefile.kvm
>@@ -179,6 +179,7 @@ TEST_GEN_PROGS_arm64 += arm64/psci_test
> TEST_GEN_PROGS_arm64 += arm64/sea_to_user
> TEST_GEN_PROGS_arm64 += arm64/set_id_regs
> TEST_GEN_PROGS_arm64 += arm64/smccc_filter
>+TEST_GEN_PROGS_arm64 += arm64/stage2_block_transitions
> TEST_GEN_PROGS_arm64 += arm64/vcpu_width_config
> TEST_GEN_PROGS_arm64 += arm64/vgic_init
> TEST_GEN_PROGS_arm64 += arm64/vgic_irq
>diff --git a/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
>new file mode 100644
>index 000000000000..5fd47f4ada1f
>--- /dev/null
>+++ b/tools/testing/selftests/kvm/arm64/stage2_block_transitions.c
>@@ -0,0 +1,226 @@
>+// SPDX-License-Identifier: GPL-2.0-only
>+/*
>+ * Copyright (c) 2026 Google LLC
>+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
>+ *
>+ * stage2_block_transitions - Exercise stage-2 block/page granularity changes
>+ * that dirty logging forces at fault time, and assert the guest completes.
>+ *
>+ * Both scenarios need the fault handler to allocate at fault time (a fresh
>+ * mapping and/or page-table pages while holding mmu_lock), so a fault path
>+ * that fails to stage that memory manifests as a KVM_RUN error or, worse, a
>+ * host crash. The asserted property is host-agnostic: the guest runs the
>+ * sequence to completion and every KVM_RUN succeeds. On a pKVM host, where a
>+ * non-protected guest's stage-2 faults are serviced by the pkvm_pgtable_*()
>+ * backend, the same sequences also guard that backend's fault-time staging.
>+ *
>+ * Scenario 1 - block collapse on dirty-logging disable:
>+ *   A write under dirty logging installs a 4K page; GET_DIRTY_LOG
>+ *   re-write-protects it; logging is disabled; a second write takes a
>+ *   permission fault that collapses the page into a hugetlb-backed block,
>+ *   which requires a fresh mapping object under mmu_lock.
>+ *
>+ * Scenario 2 - block split under dirty logging:
>+ *   Several hugetlb-backed blocks are faulted in as non-executable blocks,
>+ *   dirty logging is enabled (write-protect only), then the guest executes
>+ *   into each block. Each instruction fetch takes an execute permission
>+ *   fault that must split the block into pages during logging, draining
>+ *   page-table pages. Skipped on CTR_EL0.DIC hardware, where mappings are
>+ *   made executable eagerly and the execute fault never occurs.
>+ */
>+#include <linux/bitfield.h>
>+#include <linux/bitmap.h>
>+#include <linux/mman.h>
>+#include <linux/sizes.h>
>+#include <sys/mman.h>
>+
>+#include <asm/sysreg.h>
>+
>+#include "kvm_util.h"
>+#include "processor.h"
>+#include "test_util.h"
>+#include "ucall.h"
>+
>+#define DATA_SLOT		1
>+#define TEST_GVA		0xc0000000UL
>+#define BLOCK_SIZE		SZ_2M
>+
>+/* AArch64 "ret" (ret x30): a self-contained, returnable executable payload. */
>+#define RET_INSN		0xd65f03c0U
>+
>+/*
>+ * A non-protected guest's per-VM stage-2 pool is seeded only with the PGD
>+ * donation, which stage-2 init immediately consumes, so the page-table budget
>+ * for a fault that does not top up is just the handful (~2x the stage-2 min
>+ * pages) of memcache leftovers. Executing into this many distinct blocks
>+ * demands far more than that budget: a fault path that tops up on every fault
>+ * completes all of them, one that skips non-write faults runs out mid-sequence.
>+ */
>+#define NR_BLOCKS		16
>+
>+/* Scenario 2 guest -> host sync stages. */
>+#define STAGE_SKIP_DIC		1
>+#define STAGE_BLOCKS_READY	2
>+
>+static void collapse_guest_code(u64 gva)
>+{
>+	u64 *data = (u64 *)gva;
>+
>+	/* Under dirty logging: install a 4K writable page. */
>+	WRITE_ONCE(*data, 0x1);
>+	GUEST_SYNC(1);
>+
>+	/* Logging disabled: a permission fault collapses the page into a block. */
>+	WRITE_ONCE(*data, 0x2);
>+	GUEST_SYNC(2);
>+
>+	GUEST_DONE();
>+}
>+
>+static void test_block_collapse(void)
>+{
>+	struct kvm_vcpu *vcpu;
>+	unsigned long *bmap;
>+	struct kvm_vm *vm;
>+	struct ucall uc;
>+	size_t npages;
>+	u64 gpa;
>+
>+	vm = vm_create_with_one_vcpu(&vcpu, collapse_guest_code);
>+	npages = BLOCK_SIZE / vm->page_size;
>+
>+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - BLOCK_SIZE;
>+	gpa = align_down(gpa, BLOCK_SIZE);
>+
>+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
>+				    DATA_SLOT, npages, KVM_MEM_LOG_DIRTY_PAGES);
>+	virt_map(vm, TEST_GVA, gpa, npages);
>+	vcpu_args_set(vcpu, 1, TEST_GVA);
>+
>+	bmap = bitmap_zalloc(BLOCK_SIZE / getpagesize());
>+
>+	vcpu_run(vcpu);
>+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 1,
>+		    "Expected first sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
>+
>+	/* GET_DIRTY_LOG re-write-protects the dirtied page; then stop logging. */
>+	kvm_vm_get_dirty_log(vm, DATA_SLOT, bmap);
>+	vm_mem_region_set_flags(vm, DATA_SLOT, 0);
>+
>+	/* The collapsing permission fault: a broken fault path faults here. */
>+	vcpu_run(vcpu);
>+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == 2,
>+		    "Expected second sync, got cmd %lu arg %lu", uc.cmd, uc.args[1]);
>+
>+	vcpu_run(vcpu);
>+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
>+		    "Expected done, got cmd %lu", uc.cmd);
>+
>+	free(bmap);
>+	kvm_vm_free(vm);
>+}
>+
>+static void guest_sync_insn(u64 va)
>+{
>+	/* Make the just-written instruction coherent for execution (!DIC). */
>+	asm volatile("dc cvau, %0\n"
>+		     "dsb ish\n"
>+		     "ic ivau, %0\n"
>+		     "dsb ish\n"
>+		     "isb\n"
>+		     :: "r" (va) : "memory");
>+}
>+
>+static void split_guest_code(u64 base_gva, u64 nblocks)
>+{
>+	u64 i, va;
>+
>+	if (FIELD_GET(CTR_EL0_DIC_MASK, read_sysreg(ctr_el0))) {
>+		GUEST_SYNC(STAGE_SKIP_DIC);
>+		GUEST_DONE();
>+		return;
>+	}
>+
>+	/* Fault in each block (non-executable) and stage an executable payload. */
>+	for (i = 0; i < nblocks; i++) {
>+		va = base_gva + i * BLOCK_SIZE;
>+		WRITE_ONCE(*(u32 *)va, RET_INSN);
>+		guest_sync_insn(va);
>+	}
>+	GUEST_SYNC(STAGE_BLOCKS_READY);
>+
>+	/* Logging is now on: executing into each block splits it into pages. */
>+	for (i = 0; i < nblocks; i++) {
>+		va = base_gva + i * BLOCK_SIZE;
>+		((void (*)(void))va)();
>+	}
>+
>+	GUEST_DONE();
>+}
>+
>+static void test_exec_split_drain(void)
>+{
>+	struct kvm_vcpu *vcpu;
>+	struct kvm_vm *vm;
>+	struct ucall uc;
>+	size_t npages;
>+	u64 gpa;
>+
>+	vm = vm_create_with_one_vcpu(&vcpu, split_guest_code);
>+	npages = NR_BLOCKS * (BLOCK_SIZE / vm->page_size);
>+
>+	gpa = (vm_compute_max_gfn(vm) * vm->page_size) - NR_BLOCKS * BLOCK_SIZE;
>+	gpa = align_down(gpa, BLOCK_SIZE);
>+
>+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, gpa,
>+				    DATA_SLOT, npages, 0);
>+	virt_map(vm, TEST_GVA, gpa, npages);
>+	vcpu_args_set(vcpu, 2, TEST_GVA, (u64)NR_BLOCKS);
>+
>+	vcpu_run(vcpu);
>+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC,
>+		    "Expected sync, got cmd %lu", uc.cmd);
>+	if (uc.args[1] == STAGE_SKIP_DIC) {
>+		ksft_print_msg("SKIP block split: CTR_EL0.DIC == 1\n");
>+		kvm_vm_free(vm);
>+		return;
>+	}
>+	TEST_ASSERT(uc.args[1] == STAGE_BLOCKS_READY,
>+		    "Expected blocks-ready sync, got arg %lu", uc.args[1]);
>+
>+	/* Write-protect the blocks; the guest then splits them by executing. */
>+	vm_mem_region_set_flags(vm, DATA_SLOT, KVM_MEM_LOG_DIRTY_PAGES);
>+
>+	vcpu_run(vcpu);
>+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE,
>+		    "Expected done, got cmd %lu", uc.cmd);
>+
>+	kvm_vm_free(vm);
>+}
>+
>+/*
>+ * The explicit-size hugetlb backing hard-fails region creation if the pages
>+ * are not already reserved, so probe here and skip rather than abort. The
>+ * peak reservation is scenario 2's; the two scenarios run and free in turn.
>+ */
>+static void require_hugepages(size_t bytes)
>+{
>+	void *mem = mmap(NULL, bytes, PROT_READ | PROT_WRITE,
>+			 MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_2MB,
>+			 -1, 0);
>+
>+	if (mem == MAP_FAILED)
>+		ksft_exit_skip("Need %zu bytes of reserved 2M hugepages\n", bytes);
>+	munmap(mem, bytes);
>+}
>+
>+int main(void)
>+{
>+	require_hugepages(NR_BLOCKS * BLOCK_SIZE);
>+
>+	test_block_collapse();
>+	test_exec_split_drain();
>+
>+	ksft_print_msg("All ok!\n");
>+	return 0;
>+}
>

Thanks!

Re: [PATCH v5 0/7] KVM: arm64: pKVM stage-2 mapping and memcache fixes

[Bradley Morgan]

On July 17, 2026 2:03:10 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>Hi folks,
>
>Changes since v4 [1]:
>  - Restored the permission-fault and dirty-logging memcache top-ups
>    that v4 dropped. Both fix real bugs.
>  - Patch 1 uses the anonymous bitfield encoding rather than the
>    open-coded mask and helpers. (Marc)
>  - Reshaped the permission-fault top-up to stage the full memcache
>    under pKVM, not just the mapping object. The object-only form still
>    returned -ENOMEM and tripped a WARN under the hypervisor's
>    unconditional min-pages check.
>  - Re-scoped the dirty-logging top-up to its generic, non-pKVM failure
>    mode, now that the permission-fault patch covers pKVM.
>  - Added three adjacent fixes found while going through the series, and
>    a selftest for the block transitions.
>  - Patches 1 and 3 keep Bradley's Signed-off-by from v3; my changes
>    to each are noted in a [tabba: ...] line.
>
>I picked this up while reviewing Bradley's "mapping cache" series [2]:
>v4 dropped two fixes from v3 that address real bugs, so I've collected
>the three fixes back together, reshaped patch 1 per Marc's review [3],
>added a few adjacent fixes I found along the way and a selftest, and am
>reposting as v5.
>
>Most of the fixes are in the pKVM stage-2 walker. On a pKVM host a
>non-protected guest's stage-2 faults go through pkvm_pgtable_*(), which
>diverges from the generic walker in several ways that are bugs: cache
>maintenance on non-cacheable mappings (patch 1), a missing memcache
>top-up on permission faults that under pKVM still allocate (patch 2), a
>full flush walk on FWB hardware (patch 4), a WARN on a guest-reachable
>map failure (patch 5), and an eager-split capability whose pKVM backend
>is only a stub (patch 6). These affect non-protected guests only, since
>dispatch and memcache selection key on the host-global pKVM mode, and
>protected guests take no permission faults.
>
>Patch 3 is not pKVM-specific. During dirty logging a non-write
>permission fault, an instruction fetch, still needs a page-table page
>to split a block, but the memcache top-up is gated on write faults.
>That fault path is generic, so the fix is too.
>
>The two memcache top-ups came from sashiko review-bot findings [4][5],
>and both check out against the code.
>
>The series is structured as follows:
>
>  1:    Skip cache maintenance for non-cacheable mappings.
>  2-3:  Top up the memcache for the permission and dirty-logging faults
>        that force stage-2 block transitions.
>  4-6:  Adjacent pkvm_pgtable_*() fixes: FWB flush early-out, drop a
>        spurious map WARN, and gate the eager-split capability.
>  7:    Selftest for the block-collapse and block-split transitions.
>
>Testing: the selftest in patch 7 covers the two dirty-logging block
>transitions (page->block collapse and block->page split) that patch 2
>stages; on the base kernel the collapse oopses the host in
>pkvm_pgtable_stage2_map() with a NULL dereference under mmu_lock, and
>with the series applied it passes. It is a standalone test rather than
>an extension of kvm_page_table_test, whose default anonymous-4K backing
>forms no huge-page blocks, so an automated run never exercises these
>transitions, and whose worker/stage harness does not fit the multi-stage
>logging sequence. Run it with 2M hugepages
>reserved and, for the split half, on a CPU with CTR_EL0.DIC == 0 (e.g.
>-cpu cortex-a710 under QEMU); it self-skips those otherwise. It also
>passes on a non-pKVM host (VHE and nVHE), where patch 3's generic change
>applies. The other fixes are not exercised by the selftest and rest on
>the analysis in their commit messages: patch 3's fault path is generic
>and non-pKVM (under pKVM patch 2 already tops it up, and its failure is
>a WARN_ON(!nobjs) in kvm_mmu_memory_cache_alloc(), not a KVM_RUN error),
>and patches 1 and 4-6 each need a specific pKVM configuration to hit.
>
>Based on Linux v7.2-rc3 (a13c140cc289).
>
>Cheers,
>/fuad
>
>[1] https://lore.kernel.org/r/20260701192428.17430-1-include@grrlz.net
>[2] https://lore.kernel.org/r/20260624160028.15591-1-include@grrlz.net
>[3] https://lore.kernel.org/r/86qzllpy1g.wl-maz@kernel.org
>[4]
>https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
>[5]
>https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/

Whole series


PKVM.c/.h
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006

Test: Tested-by: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
 

Thanks!

Re: [PATCH v5 6/7] KVM: arm64: Don't advertise eager page splitting under pKVM

[Fuad Tabba]

Hi Bradley,

On Tue, 21 Jul 2026 at 12:19, Bradley Morgan <include@grrlz.net> wrote:
>
> On July 17, 2026 2:03:16 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
> wrote:
> >Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
> >WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
> >enabled and reported for non-protected guests: the capability check keys
> >on the per-VM protected state while the walker dispatch keys on the
> >host-global mode. Enabling the cap and then dirty-logging the guest
> >reaches the stub, splatting a userspace-reachable WARN.
> >
> >Reject the capability, and stop reporting a chunk size and the
> >supported block sizes, for every VM once pKVM is enabled, keyed on the
> >host-global mode like the split dispatch. Gating only protected VMs
> >would leave the non-protected guests that reach the stub still able to
> >enable it. Userspace decides whether eager splitting is available from
> >the block-size bitmap (QEMU falls back to no eager splitting when it
> >reads 0), so leaving it advertised steers an explicit request into the
> >enable failure instead of the fallback.
> >
> >Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
> >Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
>
> Do you think CC stable would be good? I do but sometimes I'm
> embarrassingly wrong

I'd lean against it here. The "Fixes" commit has been in mainline
since v6.14, the symptom is a WARN rather than a crash or data
corruption.

>
> >---
> > arch/arm64/include/asm/kvm_pkvm.h | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> >diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
> >index 57afb07d6b13..beea00e693a0 100644
> >--- a/arch/arm64/include/asm/kvm_pkvm.h
> >+++ b/arch/arm64/include/asm/kvm_pkvm.h
> >@@ -45,6 +45,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
> >               return true;
> >       case KVM_CAP_ARM_MTE:
> >               return false;
> >+      case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
> >+      case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
> >+              return false;
>
>
> Nit: KVM_CAP_ARM_MTE right above already returns false, fold the two
> new cases into that and drop the duplicate return. And please put a small
> comment saying eager splitting is unimplemented under pKVM (WARN
> stub), so nobody reads the list later and "fixes" it back, here's a
> example, feel free to bikeshed

I'd prefer to keep the separate return. The grouping is deliberate:
MTE and eager splitting are unrelated features, and the two eager
splitting caps belong together. The function is self-explanatory from
its name and header comment, so I don't think an inline comment is
needed.

Thanks for the reviews!
/fuad

>
>
>         case KVM_CAP_ARM_MTE:
>         /* eager splitting is unimplemented in pKVM, please don't refix this */
>         case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
>         case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
>                 return false;
>         default:
>                 return !kvm || !kvm_vm_is_protected(kvm);
>         }
>
>
>
>
> Killing the reporting too is right imo. Qemu reads a 0 block size
> bitmap and quietly falls back, an explicit enable eating EINVAL is the
> worse failure mode. Good.
>
> if u choose not to, it's ok, please add
>
> Reviewed-by: Bradley Morgan
> <include@grrlz.net>
>
>
> >       default:
> >               return !kvm || !kvm_vm_is_protected(kvm);
> >       }
> >
>
> Thanks!

Re: [PATCH v5 6/7] KVM: arm64: Don't advertise eager page splitting under pKVM

[Bradley Morgan]

On July 21, 2026 1:24:04 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
wrote:
>Hi Bradley,
>
>On Tue, 21 Jul 2026 at 12:19, Bradley Morgan <include@grrlz.net> wrote:
>>
>> On July 17, 2026 2:03:16 PM GMT+01:00, Fuad Tabba <fuad.tabba@linux.dev>
>> wrote:
>> >Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(),
>a
>> >WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
>> >enabled and reported for non-protected guests: the capability check
>keys
>> >on the per-VM protected state while the walker dispatch keys on the
>> >host-global mode. Enabling the cap and then dirty-logging the guest
>> >reaches the stub, splatting a userspace-reachable WARN.
>> >
>> >Reject the capability, and stop reporting a chunk size and the
>> >supported block sizes, for every VM once pKVM is enabled, keyed on the
>> >host-global mode like the split dispatch. Gating only protected VMs
>> >would leave the non-protected guests that reach the stub still able to
>> >enable it. Userspace decides whether eager splitting is available from
>> >the block-size bitmap (QEMU falls back to no eager splitting when it
>> >reads 0), so leaving it advertised steers an explicit request into the
>> >enable failure instead of the fallback.
>> >
>> >Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
>> >Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
>>
>> Do you think CC stable would be good? I do but sometimes I'm
>> embarrassingly wrong
>
>I'd lean against it here. The "Fixes" commit has been in mainline
>since v6.14, the symptom is a WARN rather than a crash or data
>corruption.

Don't worry, I mean. It could be back ported to ACK, right? Or anything
against it?

If you want I'll have to sign up for the gerrit and I'll do the backports


>>
>> >---
>> > arch/arm64/include/asm/kvm_pkvm.h | 3 +++
>> > 1 file changed, 3 insertions(+)
>> >
>> >diff --git a/arch/arm64/include/asm/kvm_pkvm.h
>b/arch/arm64/include/asm/kvm_pkvm.h
>> >index 57afb07d6b13..beea00e693a0 100644
>> >--- a/arch/arm64/include/asm/kvm_pkvm.h
>> >+++ b/arch/arm64/include/asm/kvm_pkvm.h
>> >@@ -45,6 +45,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm
>*kvm, long ext)
>> >               return true;
>> >       case KVM_CAP_ARM_MTE:
>> >               return false;
>> >+      case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
>> >+      case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
>> >+              return false;
>>
>>
>> Nit: KVM_CAP_ARM_MTE right above already returns false, fold the two
>> new cases into that and drop the duplicate return. And please put a
>small
>> comment saying eager splitting is unimplemented under pKVM (WARN
>> stub), so nobody reads the list later and "fixes" it back, here's a
>> example, feel free to bikeshed
>
>I'd prefer to keep the separate return. The grouping is deliberate:
>MTE and eager splitting are unrelated features, and the two eager
>splitting caps belong together. The function is self-explanatory from
>its name and header comment, so I don't think an inline comment is
>needed.

hmm. ok fair enough, this comment opinion is just my opinion anyway, not a
blocker or anything.


>Thanks for the reviews!
>/fuad
>
>>
>>
>>         case KVM_CAP_ARM_MTE:
>>         /* eager splitting is unimplemented in pKVM, please don't refix
>this */
>>         case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE:
>>         case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
>>                 return false;
>>         default:
>>                 return !kvm || !kvm_vm_is_protected(kvm);
>>         }
>>
>>
>>
>>
>> Killing the reporting too is right imo. Qemu reads a 0 block size
>> bitmap and quietly falls back, an explicit enable eating EINVAL is the
>> worse failure mode. Good.
>>
>> if u choose not to, it's ok, please add
>>
>> Reviewed-by: Bradley Morgan
>> <include@grrlz.net>
>>
>>
>> >       default:
>> >               return !kvm || !kvm_vm_is_protected(kvm);
>> >       }
>> >
>>
>> Thanks!
>

Thanks!

Re: [PATCH v5 1/7] KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings

[Vincent Donnefort]

On Fri, Jul 17, 2026 at 02:03:11PM +0100, Fuad Tabba wrote:
> From: Bradley Morgan <include@grrlz.net>
> 
> The pKVM flush path walks its own pkvm_mappings list and cleans the
> data cache for every mapping, unlike the generic stage-2 walker it
> shadows, which skips non-cacheable leaves. Cleaning the cacheable
> alias of a non-cacheable mapping is pointless and can corrupt a
> device endpoint. Record whether a mapping is non-cacheable in spare
> bits of nr_pages and skip cache maintenance for it.
> 
> Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Bradley Morgan <include@grrlz.net>
> [tabba: use Marc's anonymous bitfield in place of the open-coded mask and helpers]
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Reviewed-by: Vincent Donnefort <vdonnefort@google.com>

> ---
>  arch/arm64/include/asm/kvm_pkvm.h |  5 ++++-
>  arch/arm64/kvm/pkvm.c             | 15 +++++++++------
>  2 files changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
> index 74fedd9c5ff0..57afb07d6b13 100644
> --- a/arch/arm64/include/asm/kvm_pkvm.h
> +++ b/arch/arm64/include/asm/kvm_pkvm.h
> @@ -195,7 +195,10 @@ struct pkvm_mapping {
>  	struct rb_node node;
>  	u64 gfn;
>  	u64 pfn;
> -	u64 nr_pages;
> +	struct {
> +		u64 nr_pages:48;
> +		u64 nc:1;
> +	};
>  	u64 __subtree_last;	/* Internal member for interval tree */
>  };
>  
> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
> index 053e4f733e4b..f70c601dcf4c 100644
> --- a/arch/arm64/kvm/pkvm.c
> +++ b/arch/arm64/kvm/pkvm.c
> @@ -369,7 +369,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64
>  
>  	for_each_mapping_in_range_safe(pgt, start, end, mapping) {
>  		ret = kvm_call_hyp_nvhe(__pkvm_host_unshare_guest, handle, mapping->gfn,
> -					mapping->nr_pages);
> +					(u64)mapping->nr_pages);
>  		if (WARN_ON(ret))
>  			return ret;
>  		pkvm_mapping_remove(mapping, &pgt->pkvm_mappings);
> @@ -473,6 +473,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
>  	mapping->gfn = gfn;
>  	mapping->pfn = pfn;
>  	mapping->nr_pages = size / PAGE_SIZE;
> +	mapping->nc = !!(prot & (KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC));
>  	pkvm_mapping_insert(mapping, &pgt->pkvm_mappings);
>  
>  	return ret;
> @@ -503,7 +504,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)
>  	lockdep_assert_held(&kvm->mmu_lock);
>  	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
>  		ret = kvm_call_hyp_nvhe(__pkvm_host_wrprotect_guest, handle, mapping->gfn,
> -					mapping->nr_pages);
> +					(u64)mapping->nr_pages);
>  		if (WARN_ON(ret))
>  			break;
>  	}
> @@ -517,9 +518,11 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
>  	struct pkvm_mapping *mapping;
>  
>  	lockdep_assert_held(&kvm->mmu_lock);
> -	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
> -		__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
> -					  PAGE_SIZE * mapping->nr_pages);
> +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> +		if (!mapping->nc)
> +			__clean_dcache_guest_page(pfn_to_kaddr(mapping->pfn),
> +						  PAGE_SIZE * mapping->nr_pages);
> +	}
>  
>  	return 0;
>  }
> @@ -537,7 +540,7 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64
>  	lockdep_assert_held(&kvm->mmu_lock);
>  	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping)
>  		young |= kvm_call_hyp_nvhe(__pkvm_host_test_clear_young_guest, handle, mapping->gfn,
> -					   mapping->nr_pages, mkold);
> +					   (u64)mapping->nr_pages, mkold);
>  
>  	return young;
>  }
> -- 
> 2.39.5
> 

-- 
-- 
Vincent