From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E0E3F86F8 for ; Mon, 27 Jul 2026 11:11:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785150681; cv=none; b=PbuxF2K78EyOEq7MbCbLEEDNBYPqLivIXIkK6GXO/DRRzp3xyuY5As2s7tYeCFvi4aJyyPSUA2ofy4+KBDDZJiVYMrQWRj2H71VSy2bsBxNPrOb2sq+6hZmsHXDGvQUSR8JcY/1Fojrbok8SZRjc0OwzYVW+2XdhoKrI85tmiYg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785150681; c=relaxed/simple; bh=DVYIRT5CnFGgz13+W9Ujakfd2/8RVKXgzZbl38sw+Uw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=emvCx5sZi8nnCT6xj2LWP38W60jG7FahX2WT3KTVm48Uq7aFr1fuBEEY1Ml5/R2EEPwwmD2jjD+Omn5Eg126bGWc6Cn0MxOtJTK1DGxCK2HIt1yPXP26AeT/mddCkI+LZnH4zWBJubU7EqyfDLcTj6ReAPi8EP5J6mhP4/bp5TA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=musLjB3r; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="musLjB3r" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-c1691e4ec40so338862566b.3 for ; Mon, 27 Jul 2026 04:11:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1785150676; x=1785755476; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=rIRh8rLxbYh0lNXtq2f3kgP4YXDt3QK8yefpI/5/BFA=; b=musLjB3rzJQmuXudm6sBAHDyuqoqgvVso3YFu8onx+F4nvwLjMkgQssh6QSUAwTM/i GpCRXuRIJmGCYWoPLAait5RumQdp5I80Y/O+x32zJSg/vriKQzdBS0Fy3Zz+lm9TeSRI LjSlKMwSXK2kNmTGm+7fqzvtWRAjnwfFyFoT3dv6PiagesxdbsXzC+7wnR3SBy9P74gW jdEQwfK6kbRyeoTgzWoDP9A09Iaxpa2JD/h1m+h8yPoHGJWkVJb8fc+BbjviXhMdSHqr x6n6pXwfe125ByUSqvmEPpRxCb1RDIvz3jgFBGuxnslkjBVsW40Myo+odZQeTffx+UTt ZknA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785150676; x=1785755476; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=rIRh8rLxbYh0lNXtq2f3kgP4YXDt3QK8yefpI/5/BFA=; b=ReMaUtxljRtd3nq9PmxLRSRpBic+m6qB5rcUozFtcct2+v8rEg5fWEj8Bcf815Y5Xf 75AsDObAGd8zmvXFkPaJ+d6eIfj6pfS2zEioo3YPzyAnCpH8elPqpIKtFfLymNE3Euxa 1hfs+/HJZAzm9/LwlcQrQl+9KvUjUmC4ksXGhqqFHebX2goVh0MT0jGSo0FZPfLZlfVR FYbFeTO+5OjhAdj/aCK6KzgfuyB3cNAyhn6stSMwmFOBtZudV+2W+ddLWJglpn1KsVCU RPmqO/UJrZEsUVMiFsNlEmFrq0yk6yJLa4ZkEBfqxTBdDvmZFyj27hWxSUQDIFditTLn WVnA== X-Forwarded-Encrypted: i=1; AHgh+RqmFkFeyiVWlq5nQxi3r5sU9rB5XwetJz0XO9UqPvgzn6MaAzhrCFiv04BMgA7RpTzM0oIIMa0s8KbPG+k=@vger.kernel.org X-Gm-Message-State: AOJu0YxoGQPFvfkyiWhzj7xm3KinUM2q28fNsDkErPAfpGpdRAUmzQmy pS0iyTxtc4Yi//hifDgpBQqVdrEcqY4j+4ukb3opGQfTceHszDn9Wcc6APAhAGkRNg== X-Gm-Gg: AR+sD11HksgtOO93ohiMTCSrWSg5nTzBBp/nO5BrqmTpYFvXAOwBazNGzBbvLmX83vU Zy3KKcNcLPFSkhk2TqrBXGmoW7mz+OKvoeRE3E/em0Ev4nuO6G8wAxa/UacmZFeVjaqowG0fjyC hGuwwiN/MvBY+exF0fd4b2BGuPLSv15b2ofFHoPpIn2YGWaePV6cNiqzNKNzJHJtzXlCNWF4rF6 t5ouXXBkegKHAQbM6BMByPEP9I+vmBTi3Hb1T7jY65bKqpqQJIYXGTsvz4+JvUP9YU+g2eFAfGj Gwmzzr71DsQh70xIza/Z5e3g1NAX2nqBiIcGZUHxGIvbxTlXk9GmUj8ypio7b1Whi1CreIvV8/0 0ak2l2JbNMfVybakg2iqS4W++Bi5NYWpfhoySRgz9VZ4TAjAxEMZnPT4d8UpCv8h/6Zm+VHbvrT QgLuvB0fQdlYMy8PEh4hZ5GJAv73jgI9nocubb3ZzmyKvl+A== X-Received: by 2002:a17:907:c0a:b0:c16:9f61:854b with SMTP id a640c23a62f3a-c1f1f153f26mr394532266b.42.1785150675763; Mon, 27 Jul 2026 04:11:15 -0700 (PDT) Received: from google.com (135.91.155.104.bc.googleusercontent.com. [104.155.91.135]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c1c32af45ecsm611801266b.28.2026.07.27.04.11.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jul 2026 04:11:14 -0700 (PDT) Date: Mon, 27 Jul 2026 12:11:10 +0100 From: Vincent Donnefort To: Jack Thomson Cc: maz@kernel.org, oupton@kernel.org, pbonzini@redhat.com, joey.gouly@arm.com, seiden@linux.ibm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, corbet@lwn.net, vladimir.murzin@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, isaku.yamahata@intel.com, Jack Thomson Subject: Re: [PATCH v5 2/5] KVM: arm64: Add pre_fault_memory implementation Message-ID: References: <20260612162354.73378-1-jackabt.amazon@gmail.com> <20260612162354.73378-3-jackabt.amazon@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260612162354.73378-3-jackabt.amazon@gmail.com> Hi, On Fri, Jun 12, 2026 at 05:23:50PM +0100, Jack Thomson wrote: > From: Jack Thomson > > Add arm64 support for KVM_PRE_FAULT_MEMORY by synthesizing a read data > abort and routing it through the existing stage-2 fault handlers. Treat > the requested GPA as an IPA in the userspace-owned VM's memslot space > and always target the canonical stage-2, even if the vCPU last ran with > a nested/shadow MMU selected. > > If the vCPU last ran in a nested context, switch to the canonical > stage-2 with the vCPU put/load helpers so VMID, VNCR and shadow-MMU > refcount state stay consistent. Leave the switch in place for the ioctl; > vcpu_put() at ioctl exit drops the hw_mmu and the next vcpu_load() > reselects the correct MMU from vCPU state. > > Check existing mappings with a shared page-table walk under the MMU read > lock, and use the resulting walk level when constructing the synthetic > fault. Report poisoned pages through the ioctl return path with > -EHWPOISON instead of also queueing SIGBUS, and use the installed > mapping size to advance the prefault range. > > Advertise KVM_CAP_PRE_FAULT_MEMORY on arm64. Protected VMs remain > unsupported: pKVM filters the capability, and the ioctl returns > -EOPNOTSUPP if invoked anyway. > > Signed-off-by: Jack Thomson > --- > Documentation/virt/kvm/api.rst | 18 +++- > arch/arm64/kvm/Kconfig | 1 + > arch/arm64/kvm/arm.c | 1 + > arch/arm64/kvm/mmu.c | 162 +++++++++++++++++++++++++++++++++ > 4 files changed, 178 insertions(+), 4 deletions(-) > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index 52bbbb553ce1..657e05656fa6 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -6462,7 +6462,7 @@ See KVM_SET_USER_MEMORY_REGION2 for additional details. > --------------------------- > > :Capability: KVM_CAP_PRE_FAULT_MEMORY > -:Architectures: none > +:Architectures: x86, arm64 > :Type: vcpu ioctl > :Parameters: struct kvm_pre_fault_memory (in/out) > :Returns: 0 if at least one page is processed, < 0 on error > @@ -6470,11 +6470,14 @@ See KVM_SET_USER_MEMORY_REGION2 for additional details. > Errors: > > ========== =============================================================== > + EAGAIN A memslot update raced with the ioctl before any page was > + processed. > EINVAL The specified `gpa` and `size` were invalid (e.g. not > page aligned, causes an overflow, or size is zero). > ENOENT The specified `gpa` is outside defined memslots. > EINTR An unmasked signal is pending and no page was processed. > EFAULT The parameter address was invalid. > + EHWPOISON A poisoned host page was encountered. > EOPNOTSUPP Mapping memory for a GPA is unsupported by the > hypervisor, and/or for the current vCPU state/mode. > EIO unexpected error conditions (also causes a WARN) > @@ -6494,7 +6497,14 @@ Errors: > KVM_PRE_FAULT_MEMORY populates KVM's stage-2 page tables used to map memory > for the current vCPU state. KVM maps memory as if the vCPU generated a > stage-2 read page fault, e.g. faults in memory as needed, but doesn't break > -CoW. However, KVM does not mark any newly created stage-2 PTE as Accessed. > +CoW. However, on x86, KVM does not mark any newly created stage-2 PTE as > +Accessed. On arm64, newly created stage-2 PTEs are marked Accessed. > + > +On arm64, `gpa` is interpreted as an IPA in the userspace-owned VM's > +memslot address space. If the vCPU most recently ran a nested guest, KVM > +still targets the VM's canonical stage-2, and does not interpret `gpa` as > +a nested guest IPA or target the nested/shadow stage-2 selected by the > +vCPU's last run state. > > In the case of confidential VM types where there is an initial set up of > private guest memory before the guest is 'finalized'/measured, this ioctl > @@ -6507,9 +6517,9 @@ case, the ioctl can be called in parallel. > > When the ioctl returns, the input values are updated to point to the > remaining range. If `size` > 0 on return, the caller can just issue > -the ioctl again with the same `struct kvm_map_memory` argument. > +the ioctl again with the same `struct kvm_pre_fault_memory` argument. > > -Shadow page tables cannot support this ioctl because they > +On x86, shadow page tables cannot support this ioctl because they > are indexed by virtual address or nested guest physical address. > Calling this ioctl when the guest is using shadow page tables (for > example because it is running a nested guest with nested page tables) > diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig > index 449154f9a485..6b89262e8ba7 100644 > --- a/arch/arm64/kvm/Kconfig > +++ b/arch/arm64/kvm/Kconfig > @@ -24,6 +24,7 @@ menuconfig KVM > select HAVE_KVM_CPU_RELAX_INTERCEPT > select KVM_MMIO > select KVM_GENERIC_DIRTYLOG_READ_PROTECT > + select KVM_GENERIC_PRE_FAULT_MEMORY > select VIRT_XFER_TO_GUEST_WORK > select KVM_VFIO > select HAVE_KVM_DIRTY_RING_ACQ_REL > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 9453321ef8c6..dcb92bee13af 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -392,6 +392,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) > case KVM_CAP_COUNTER_OFFSET: > case KVM_CAP_ARM_WRITABLE_IMP_ID_REGS: > case KVM_CAP_ARM_SEA_TO_USER: > + case KVM_CAP_PRE_FAULT_MEMORY: > r = 1; > break; > case KVM_CAP_SET_GUEST_DEBUG2: > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > index c720f07cb82e..4bf048bbcf8b 100644 > --- a/arch/arm64/kvm/mmu.c > +++ b/arch/arm64/kvm/mmu.c > @@ -1571,6 +1571,8 @@ struct kvm_s2_fault_desc { > struct kvm_s2_trans *nested; > struct kvm_memory_slot *memslot; > unsigned long hva; > + unsigned long *page_size; It looks a bit odd in that struct. Also, transparent_hugepage_adjust() modifies pfn and gfn. So perhaps it'd be better to introduce a struct kvm_s2_fault_mapping that can be used to get what has actually been mapped in kvm_s2_fault_map()? > + bool prefault; > }; > > static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) > @@ -1882,6 +1884,13 @@ static int kvm_s2_fault_pin_pfn(const struct kvm_s2_fault_desc *s2fd, > &s2vi->map_writable, &s2vi->page); > if (unlikely(is_error_noslot_pfn(s2vi->pfn))) { > if (s2vi->pfn == KVM_PFN_ERR_HWPOISON) { > + /* > + * When prefaulting, report the poison via -EHWPOISON > + * only; don't also queue a SIGBUS as the run path > + * does for the faulting vCPU thread. > + */ > + if (s2fd->prefault) > + return -EHWPOISON; > kvm_send_hwpoison_signal(s2fd->hva, __ffs(s2vi->vma_pagesize)); > return 0; > } > @@ -2053,6 +2062,9 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd, > kvm_release_faultin_page(kvm, s2vi->page, !!ret, writable); > kvm_fault_unlock(kvm); > > + if (s2fd->page_size && !ret) > + *s2fd->page_size = mapping_size; > + > /* > * Mark the page dirty only if the fault is handled successfully, > * making sure we adjust the canonical IPA if the mapping size has > @@ -2757,3 +2769,153 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled) > > trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled); > } > + > +/* > + * Prefaulting always targets the canonical stage-2. If the vCPU last ran > + * in a nested context, swap in the canonical MMU via the vCPU put/load > + * helpers so that preemption, VMID, VNCR fixmap and shadow-MMU refcount > + * state stay consistent. > + * > + * The swap is deliberately not undone: nothing runs in between the > + * per-page invocations of kvm_arch_vcpu_pre_fault_memory() except the > + * generic prefault loop, and the vcpu_put() at ioctl exit discards > + * vcpu->arch.hw_mmu anyway (see kvm_vcpu_put_hw_mmu()), so the next > + * vcpu_load() re-derives the correct MMU from the vCPU's context. If the > + * prefault task is preempted in the meantime, kvm_vcpu_put_hw_mmu() > + * keeps the canonical MMU in place for the reload. Leaving the swap in > + * place also bounds the cost to at most one put/load pair per ioctl, > + * rather than two pairs per prefaulted page. > + */ > +static void kvm_pre_fault_load_canonical_mmu(struct kvm_vcpu *vcpu) > +{ > + if (!vcpu_has_nv(vcpu) || vcpu->arch.hw_mmu == &vcpu->kvm->arch.mmu) > + return; > + > + preempt_disable(); > + kvm_arch_vcpu_put(vcpu); > + vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu; > + kvm_arch_vcpu_load(vcpu, smp_processor_id()); > + preempt_enable(); > +} > + > +long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, > + struct kvm_pre_fault_memory *range) > +{ > + struct kvm_vcpu_fault_info *fault_info = &vcpu->arch.fault; > + struct kvm_vcpu_fault_info fault_backup = *fault_info; > + s8 walk_level = KVM_PGTABLE_LAST_LEVEL; > + unsigned long page_size = PAGE_SIZE; > + struct kvm_memory_slot *memslot; > + phys_addr_t gpa = range->gpa; > + struct kvm_pgtable *pgt; > + phys_addr_t end; > + kvm_pte_t pte; > + hva_t hva; > + gfn_t gfn; > + long ret; > + > + if (vcpu_is_protected(vcpu)) > + return -EOPNOTSUPP; It could just call pkvm_mem_abort() when the VM is protected? > + > + /* > + * Interpret range->gpa in the userspace-owned VM's IPA space, not in > + * any nested guest IPA space that may have been active on the vCPU's > + * last run. Always target the canonical stage-2. > + */ > + kvm_pre_fault_load_canonical_mmu(vcpu); > + > + if (gpa >= kvm_phys_size(vcpu->arch.hw_mmu)) { > + ret = -ENOENT; > + goto out; > + } > + > + gfn = gpa_to_gfn(gpa); > + memslot = gfn_to_memslot(vcpu->kvm, gfn); > + if (!memslot) { > + ret = -ENOENT; > + goto out; > + } > + > + /* > + * A racing memslot deletion or move installs an invalid slot before > + * zapping stage-2. Ask userspace to retry once the update settles. > + */ > + if (memslot->flags & KVM_MEMSLOT_INVALID) { > + ret = -EAGAIN; > + goto out; > + } > + > + /* > + * pKVM stage-2 mappings aren't directly walkable from the host; let > + * the fault path handle both new and existing mappings. > + */ pKVM has a host interval-tree where we keep track of what has been mapped and at what level. This tree is also protected with the mmu_lock. > + if (!is_protected_kvm_enabled()) { > + pgt = vcpu->arch.hw_mmu->pgt; > + scoped_guard(read_lock, &vcpu->kvm->mmu_lock) { > + ret = kvm_pgtable_get_leaf(pgt, gpa, &pte, &walk_level, > + KVM_PGTABLE_WALK_SHARED); > + } > + if (ret) > + goto out; > + > + if (kvm_pte_valid(pte)) { > + page_size = kvm_granule_size(walk_level); > + if (!(pte & KVM_PTE_LEAF_ATTR_LO_S2_AF)) > + handle_access_fault(vcpu, gpa); > + goto out_success; > + } > + } > + > + /* > + * Synthesize a read translation fault for the canonical IPA, at the > + * level where the stage-2 walk currently ends (the last level under > + * pKVM, where stage-2 isn't walkable from the host). > + */ > + fault_info->esr_el2 = (ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT) | > + ESR_ELx_IL | ESR_ELx_FSC_FAULT_L(walk_level); > + fault_info->hpfar_el2 = HPFAR_EL2_NS | > + FIELD_PREP(HPFAR_EL2_FIPA, gpa >> 12); > + > + struct kvm_s2_fault_desc s2fd = { > + .vcpu = vcpu, > + .fault_ipa = gpa, > + .nested = NULL, > + .memslot = memslot, > + .page_size = &page_size, > + .prefault = true, > + }; > + > + /* > + * As in the run path, -EAGAIN from the abort handlers is treated as > + * progress: either a parallel fault installed the mapping, or a racing > + * invalidation is in flight and the next access will refault. > + */ > + if (kvm_slot_has_gmem(memslot)) { > + ret = gmem_abort(&s2fd); > + } else { > + hva = gfn_to_hva_memslot_prot(memslot, gfn, NULL); > + if (kvm_is_error_hva(hva)) { > + ret = -EFAULT; > + goto out; > + } > + > + s2fd.hva = hva; > + ret = user_mem_abort(&s2fd); > + } > + > + if (ret < 0) > + goto out; > + > +out_success: > + end = ALIGN_DOWN(gpa, page_size) + page_size; > + ret = min_t(u64, range->size, end - gpa); > +out: > + /* > + * Restore the synthetic fault state so a subsequent KVM_RUN does not > + * observe it. kvm_handle_mmio_return() runs before guest entry can > + * refresh fault.esr_el2 from hardware, so leaving the synthetic ESR > + * in place would corrupt the completion of a pending MMIO exit. > + */ > + *fault_info = fault_backup; > + return ret; > +} > -- > 2.43.0 > >