[PATCH 0/3] KVM: arm64: Fix teardown of non-protected VMs with pKVM

[PATCH 0/3] KVM: arm64: Fix teardown of non-protected VMs with pKVM

[Will Deacon]

Hi folks,

Sashiko continues to throw rocks at me.

This time, it spotted that my fix (introduced in v5 [1] of the pKVM
series) to prevent taking a reference on a VM in the 'is_dying' state
also prevents unsharing of pages shared with a non-protected VM if that
VM is torn down by its VM fd being destroyed, rather than the usual path
via the MMU notifiers.

Rather than send a v6 of the whole series, here are three patches that
apply on top of v5 and fix the issue by (a) preventing teardown of a
referenced VM and (b) allowing some references to be taken on a dying
VM. As an added bonus, this simplifies the locking on the reclaim path
because now a VM reference is enough to stop the page-tables from going
away.

Cheers,

Will

[1] https://lore.kernel.org/kvmarm/20260330144841.26181-1-will@kernel.org

Cc: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oupton@kernel.org>
Cc: Joey Gouly <joey.gouly@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Zenghui Yu <yuzenghui@huawei.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Quentin Perret <qperret@google.com>
Cc: Fuad Tabba <tabba@google.com>
Cc: Vincent Donnefort <vdonnefort@google.com>
Cc: Mostafa Saleh <smostafa@google.com>
Cc: Alexandru Elisei <alexandru.elisei@arm.com>

--->8

Will Deacon (3):
  KVM: arm64: Prevent teardown finalisation of referenced 'hyp_vm'
  KVM: arm64: Allow get_pkvm_hyp_vm() to take a reference to a dying VM
  KVM: arm64: Don't hold 'vm_table_lock' across guest page reclaim

 arch/arm64/kvm/hyp/nvhe/pkvm.c | 53 +++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 27 deletions(-)

-- 
2.53.0.1118.gaef5881109-goog

[PATCH 1/3] KVM: arm64: Prevent teardown finalisation of referenced 'hyp_vm'

[Will Deacon]

Destroying a 'hyp_vm' with an elevated referenced count in
__pkvm_finalize_teardown_vm() is only going to lead to tears.

In preparation for allowing limited references to be acquired on dying
VMs during the teardown process, factor out the handle-to-vm logic for
the teardown path and reuse it for both the 'start' and 'finalise'
stages of the teardown process.

Signed-off-by: Will Deacon <will@kernel.org>
---
 arch/arm64/kvm/hyp/nvhe/pkvm.c | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 8b906217c4c3..3fd3b930beeb 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -936,20 +936,27 @@ int __pkvm_reclaim_dying_guest_page(pkvm_handle_t handle, u64 gfn)
 	return ret;
 }
 
+static struct pkvm_hyp_vm *get_pkvm_unref_hyp_vm_locked(pkvm_handle_t handle)
+{
+	struct pkvm_hyp_vm *hyp_vm;
+
+	hyp_assert_lock_held(&vm_table_lock);
+
+	hyp_vm = get_vm_by_handle(handle);
+	if (!hyp_vm || hyp_page_count(hyp_vm))
+		return NULL;
+
+	return hyp_vm;
+}
+
 int __pkvm_start_teardown_vm(pkvm_handle_t handle)
 {
 	struct pkvm_hyp_vm *hyp_vm;
 	int ret = 0;
 
 	hyp_spin_lock(&vm_table_lock);
-	hyp_vm = get_vm_by_handle(handle);
-	if (!hyp_vm) {
-		ret = -ENOENT;
-		goto unlock;
-	} else if (WARN_ON(hyp_page_count(hyp_vm))) {
-		ret = -EBUSY;
-		goto unlock;
-	} else if (hyp_vm->kvm.arch.pkvm.is_dying) {
+	hyp_vm = get_pkvm_unref_hyp_vm_locked(handle);
+	if (!hyp_vm || hyp_vm->kvm.arch.pkvm.is_dying) {
 		ret = -EINVAL;
 		goto unlock;
 	}
@@ -971,12 +978,9 @@ int __pkvm_finalize_teardown_vm(pkvm_handle_t handle)
 	int err;
 
 	hyp_spin_lock(&vm_table_lock);
-	hyp_vm = get_vm_by_handle(handle);
-	if (!hyp_vm) {
-		err = -ENOENT;
-		goto err_unlock;
-	} else if (!hyp_vm->kvm.arch.pkvm.is_dying) {
-		err = -EBUSY;
+	hyp_vm = get_pkvm_unref_hyp_vm_locked(handle);
+	if (!hyp_vm || !hyp_vm->kvm.arch.pkvm.is_dying) {
+		err = -EINVAL;
 		goto err_unlock;
 	}
 
-- 
2.53.0.1118.gaef5881109-goog

[PATCH 2/3] KVM: arm64: Allow get_pkvm_hyp_vm() to take a reference to a dying VM

[Will Deacon]

Now that completion of the teardown path requires a refcount of zero for
the target VM, we can allow get_pkvm_hyp_vm() to take a reference on a
dying VM, which is necessary to unshare pages with a non-protected VM
during the teardown process itself.

Note that vCPUs belonging to a dying VM cannot be loaded and pages can
only be reclaimed from a protected VM (via
__pkvm_reclaim_dying_guest_page()) if the target VM is in the dying
state.

Signed-off-by: Will Deacon <will@kernel.org>
---
 arch/arm64/kvm/hyp/nvhe/pkvm.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 3fd3b930beeb..b955da0e50bc 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -309,14 +309,8 @@ struct pkvm_hyp_vm *get_pkvm_hyp_vm(pkvm_handle_t handle)
 
 	hyp_spin_lock(&vm_table_lock);
 	hyp_vm = get_vm_by_handle(handle);
-	if (!hyp_vm)
-		goto unlock;
-
-	if (hyp_vm->kvm.arch.pkvm.is_dying)
-		hyp_vm = NULL;
-	else
+	if (hyp_vm)
 		hyp_page_ref_inc(hyp_virt_to_page(hyp_vm));
-unlock:
 	hyp_spin_unlock(&vm_table_lock);
 
 	return hyp_vm;
-- 
2.53.0.1118.gaef5881109-goog

[PATCH 3/3] KVM: arm64: Don't hold 'vm_table_lock' across guest page reclaim

[Will Deacon]

Now that the teardown of a VM cannot be finalised as long as a reference
is held on the VM, rework __pkvm_reclaim_dying_guest_page() to hold a
reference to the dying VM rather than take the global 'vm_table_lock'
during the reclaim operation.

Signed-off-by: Will Deacon <will@kernel.org>
---
 arch/arm64/kvm/hyp/nvhe/pkvm.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index b955da0e50bc..7ed96d64d611 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -918,15 +918,16 @@ teardown_donated_memory(struct kvm_hyp_memcache *mc, void *addr, size_t size)
 
 int __pkvm_reclaim_dying_guest_page(pkvm_handle_t handle, u64 gfn)
 {
-	struct pkvm_hyp_vm *hyp_vm;
+	struct pkvm_hyp_vm *hyp_vm = get_pkvm_hyp_vm(handle);
 	int ret = -EINVAL;
 
-	hyp_spin_lock(&vm_table_lock);
-	hyp_vm = get_vm_by_handle(handle);
-	if (hyp_vm && hyp_vm->kvm.arch.pkvm.is_dying)
-		ret = __pkvm_host_reclaim_page_guest(gfn, hyp_vm);
-	hyp_spin_unlock(&vm_table_lock);
+	if (!hyp_vm)
+		return ret;
 
+	if (hyp_vm->kvm.arch.pkvm.is_dying)
+		ret = __pkvm_host_reclaim_page_guest(gfn, hyp_vm);
+
+	put_pkvm_hyp_vm(hyp_vm);
 	return ret;
 }
 
-- 
2.53.0.1118.gaef5881109-goog

Re: [PATCH 0/3] KVM: arm64: Fix teardown of non-protected VMs with pKVM

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 867 bytes --]

On Tue, Mar 31, 2026 at 04:50:52PM +0100, Will Deacon wrote:

> This time, it spotted that my fix (introduced in v5 [1] of the pKVM
> series) to prevent taking a reference on a VM in the 'is_dying' state
> also prevents unsharing of pages shared with a non-protected VM if that
> VM is torn down by its VM fd being destroyed, rather than the usual path
> via the MMU notifiers.

> Rather than send a v6 of the whole series, here are three patches that
> apply on top of v5 and fix the issue by (a) preventing teardown of a
> referenced VM and (b) allowing some references to be taken on a dying
> VM. As an added bonus, this simplifies the locking on the reclaim path
> because now a VM reference is enough to stop the page-tables from going
> away.

This series fixes the warnings I reported with pKVM on N1DSP yesterday:

Tested-by: Mark Brown <broonie@kernel.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/3] KVM: arm64: Fix teardown of non-protected VMs with pKVM

[Marc Zyngier]

On Tue, 31 Mar 2026 16:50:52 +0100, Will Deacon wrote:
> Sashiko continues to throw rocks at me.
> 
> This time, it spotted that my fix (introduced in v5 [1] of the pKVM
> series) to prevent taking a reference on a VM in the 'is_dying' state
> also prevents unsharing of pages shared with a non-protected VM if that
> VM is torn down by its VM fd being destroyed, rather than the usual path
> via the MMU notifiers.
> 
> [...]

Applied to next, thanks!

[1/3] KVM: arm64: Prevent teardown finalisation of referenced 'hyp_vm'
      commit: 760299a1d8102b36bed5c25c5a3f94b5a0eee081
[2/3] KVM: arm64: Allow get_pkvm_hyp_vm() to take a reference to a dying VM
      commit: 2400696883870ec3fb0fb9925426c62a3383ca36
[3/3] KVM: arm64: Don't hold 'vm_table_lock' across guest page reclaim
      commit: bc20692f528b2ac8226bafe5b1db9a1f8be96dbf

Cheers,

	M.
-- 
Without deviation from the norm, progress is not possible.