* [RFC PATCH v5 01/16] KVM: guest_memfd: Add a handle for provider PFNs
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 02/16] KVM: guest_memfd: Get PFNs through common provider operations Aneesh Kumar K.V (Arm)
` (14 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Add an MMIO PFN flag, separate from KVM memory attributes.
Add struct guest_memfd_handle to hold a PFN, its attributes, the order
of the contiguous range and an optional folio reference. Later patches
use this handle for both folio and PFN providers.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 3e2c0714ca33..659d7d4baa58 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -2,13 +2,30 @@
#ifndef _LINUX_GUEST_MEMFD_H
#define _LINUX_GUEST_MEMFD_H
+#include <linux/bits.h>
#include <linux/types.h>
+#define GUEST_MEMFD_PFN_MMIO BIT_ULL(0)
+
struct file;
struct folio;
struct inode;
struct mempolicy;
+/**
+ * struct guest_memfd_handle - PFN information and an optional folio reference
+ * @pfn: Physical frame number
+ * @pfn_attrs: GUEST_MEMFD_PFN_* memory type and permissions
+ * @max_order: Largest order of a contiguous range with the same PFN attributes
+ * @folio: Optional locked folio reference, managed by the core
+ */
+struct guest_memfd_handle {
+ unsigned long pfn;
+ u64 pfn_attrs;
+ int max_order;
+ struct folio *folio;
+};
+
/**
* struct guest_memfd_provider_operations - Operations for external memory providers
* @attach: Attach to a resource file, perform filesystem-specific validation,
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 02/16] KVM: guest_memfd: Get PFNs through common provider operations
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 01/16] KVM: guest_memfd: Add a handle for provider PFNs Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 03/16] KVM: guest_memfd: Let providers coordinate mapping changes Aneesh Kumar K.V (Arm)
` (13 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Use get_pfn() for both folios and external PFNs. Folio helpers return
a locked folio reference in the handle. The core unlocks and releases
that reference. PFN providers keep memory stable under the inode
invalidate lock.
Declare native provider operations in their static table. Install the
folio get_pfn() helper inside KVM for external folio providers, so
built-in shmem does not depend on modular KVM.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 23 +--
samples/kvm/mem_provider_sample.c | 10 +-
virt/kvm/guest_memfd.c | 245 +++++++++++++++++++-----------
3 files changed, 176 insertions(+), 102 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 659d7d4baa58..3d84fd0adec1 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -18,6 +18,11 @@ struct mempolicy;
* @pfn_attrs: GUEST_MEMFD_PFN_* memory type and permissions
* @max_order: Largest order of a contiguous range with the same PFN attributes
* @folio: Optional locked folio reference, managed by the core
+ *
+ * The core unlocks the folio before dropping its reference. Population keeps
+ * the reference while its callback runs without the folio lock. A host folio
+ * fault instead transfers the locked reference to the MM. PFN providers leave
+ * @folio NULL and keep their PFNs stable under the inode invalidate lock.
*/
struct guest_memfd_handle {
unsigned long pfn;
@@ -35,13 +40,13 @@ struct guest_memfd_handle {
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
* @invalidate_folio: Notify provider that a folio has been invalidated.
- * @get_pfn: Obtain a PFN for a given index. The provider must ensure backing
- * memory is present and is responsible for its contents, such as
- * zeroing if necessary. On success, return 0 with the physical frame
- * number in @pfn and, if @max_order is non-NULL, the maximum page
- * order of contiguous backing memory containing @pfn. Returns a
- * negative error code on failure. The provider must define
- * @lookup_pfn_range when @get_pfn is defined.
+ * @get_pfn: Get the PFN for initialized memory at an index. Return zero on
+ * success or a negative error code. Called with the inode
+ * invalidate lock held. Fill the handle with its PFN, attributes
+ * and order. Folio helpers also return a locked folio reference in
+ * guest_memfd_handle.folio for the core to unlock and release.
+ * PFN providers leave folio NULL and keep their PFNs stable until
+ * the caller releases the inode invalidate lock.
* @lookup_pfn_range: Look up the PFN for a given index. Returns 0 on success
* with the physical frame number in @pfn and the number of
* contiguous pages the mapping is valid for in @nr_pages.
@@ -64,8 +69,8 @@ struct guest_memfd_provider_operations {
struct folio *(*alloc_folio)(void *provider, pgoff_t index,
struct mempolicy *mpol);
void (*invalidate_folio)(void *provider, struct folio *folio);
- int (*get_pfn)(void *provider, pgoff_t index, unsigned long *pfn,
- int *max_order);
+ int (*get_pfn)(struct inode *inode, void *provider, pgoff_t index,
+ struct guest_memfd_handle *handle);
int (*lookup_pfn_range)(void *provider, pgoff_t index,
unsigned long *pfn, size_t *nr_pages);
void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
index 5496e0c7864b..f1b0698268c0 100644
--- a/samples/kvm/mem_provider_sample.c
+++ b/samples/kvm/mem_provider_sample.c
@@ -178,8 +178,8 @@ static void mps_gmem_release(void *provider)
fput(resource_file);
}
-static int mps_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
- int *max_order)
+static int mps_gmem_get_pfn(struct inode *inode, void *provider,
+ pgoff_t index, struct guest_memfd_handle *handle)
{
struct file *resource_file = provider;
struct mps_info *info = resource_file->private_data;
@@ -190,9 +190,9 @@ static int mps_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
if (test_bit(index, info->absent))
return -EFAULT;
- *pfn = info->base_pfn + index;
- if (max_order)
- *max_order = 0;
+ handle->pfn = info->base_pfn + index;
+ handle->max_order = 0;
+ handle->pfn_attrs = 0;
return 0;
}
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 48d14a20fc0b..026a7bd0fe76 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -38,6 +38,7 @@ struct gmem_inode {
void *provider;
const struct guest_memfd_provider_operations *provider_ops;
+ struct guest_memfd_provider_operations provider_operations;
u64 flags;
/*
@@ -49,6 +50,9 @@ struct gmem_inode {
struct maple_tree attributes;
};
+static int gmem_folio_get_pfn(struct inode *inode, void *provider, pgoff_t index,
+ struct guest_memfd_handle *handle);
+
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
{
return container_of(inode, struct gmem_inode, vfs_inode);
@@ -83,6 +87,7 @@ static struct folio *gmem_native_alloc_folio(void *provider, pgoff_t index,
static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.attach = gmem_native_attach,
.alloc_folio = gmem_native_alloc_folio,
+ .get_pfn = gmem_folio_get_pfn,
};
static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -91,16 +96,47 @@ static bool gmem_provider_ops_valid(struct gmem_inode *gi,
if (!ops || !ops->attach)
return false;
+ if (ops == &gmem_native_provider_ops)
+ return !kvm_gmem_is_pfn_mode(gi);
+
+ /* get_pfn() and lookup_pfn_range() must describe the same memory. */
+ if (!!ops->get_pfn != !!ops->lookup_pfn_range)
+ return false;
+
+ if (ops->invalidate_folio && !ops->alloc_folio)
+ return false;
+
+ /* Custom folio get_pfn() needs matching host mapping callbacks. */
+ if (ops->alloc_folio && ops->get_pfn &&
+ (gi->flags & GUEST_MEMFD_FLAG_MMAP))
+ return false;
+
if (kvm_gmem_is_pfn_mode(gi))
return ops->get_pfn && ops->lookup_pfn_range;
return ops->alloc_folio != NULL;
}
+/*
+ * Complete external folio providers here so built-in shmem does not reference
+ * helpers in a potentially modular KVM. Native operations are declared statically.
+ */
+static void gmem_provider_init_ops(struct gmem_inode *gi,
+ const struct guest_memfd_provider_operations *ops)
+{
+ gi->provider_operations = *ops;
+ if (ops->alloc_folio) {
+ if (!ops->get_pfn)
+ gi->provider_operations.get_pfn = gmem_folio_get_pfn;
+ }
+ gi->provider_ops = &gi->provider_operations;
+}
+
#define GMEM_RESOURCE_FD_NATIVE (-1)
static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
{
+ struct gmem_inode *gi = GMEM_I(inode);
const struct guest_memfd_provider_operations *ops = NULL;
void *provider = ERR_PTR(-EOPNOTSUPP);
struct file *resource_file = NULL;
@@ -116,7 +152,7 @@ static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
ops = resource_file->f_op->gmem_provider_ops;
}
- if (gmem_provider_ops_valid(GMEM_I(inode), ops))
+ if (gmem_provider_ops_valid(gi, ops))
provider = ops->attach(inode, resource_file);
if (resource_file)
@@ -125,8 +161,11 @@ static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
if (IS_ERR(provider))
return PTR_ERR(provider);
- GMEM_I(inode)->provider = provider;
- GMEM_I(inode)->provider_ops = ops;
+ gi->provider = provider;
+ if (ops == &gmem_native_provider_ops)
+ gi->provider_ops = ops;
+ else
+ gmem_provider_init_ops(gi, ops);
return 0;
}
@@ -141,18 +180,58 @@ static struct folio *gmem_provider_alloc_folio(struct gmem_inode *gi,
return ERR_PTR(-EOPNOTSUPP);
}
+/**
+ * gmem_provider_get_pfn - Get a handle for a file index
+ * @gi: guest_memfd inode containing the attached provider
+ * @index: Page index within the file
+ * @handle: Receives PFN information and an optional locked folio reference
+ *
+ * Folio helpers return a locked folio with a reference. After success, call
+ * gmem_unlock_handle(), then gmem_put_handle(). On failure, the provider
+ * releases any folio lock or reference taken before the error.
+ *
+ * A host folio fault instead passes the locked reference to the MM, which
+ * handles unlocking and releasing it. Do not call the cleanup helpers then.
+ *
+ * PFN providers leave handle->folio NULL and keep their PFNs stable while
+ * the caller holds the inode invalidate lock. They must not leave per-lookup
+ * locks or references requiring provider cleanup.
+ *
+ * Return: Zero on success, or a negative error code on failure.
+ */
static int gmem_provider_get_pfn(struct gmem_inode *gi, pgoff_t index,
- kvm_pfn_t *pfn, int *max_order)
+ struct guest_memfd_handle *handle)
{
- unsigned long raw_pfn;
- int r;
+ int ret;
- r = gi->provider_ops->get_pfn(gi->provider, index, &raw_pfn, max_order);
- if (r)
- return r;
+ *handle = (struct guest_memfd_handle){};
+ ret = gi->provider_ops->get_pfn(&gi->vfs_inode, gi->provider, index, handle);
+ return ret;
+}
- *pfn = raw_pfn;
- return 0;
+/**
+ * gmem_unlock_handle - Unlock the folio returned in a handle
+ * @handle: Result returned by gmem_provider_get_pfn()
+ *
+ * Keep the folio reference until gmem_put_handle(). PFN providers return no
+ * folio, so there is nothing to unlock.
+ */
+static void gmem_unlock_handle(struct guest_memfd_handle *handle)
+{
+ if (handle->folio)
+ folio_unlock(handle->folio);
+}
+
+/**
+ * gmem_put_handle - Release the folio reference returned in a handle
+ * @handle: Result already passed to gmem_unlock_handle()
+ *
+ * PFN providers return no folio, so there is no temporary reference to drop.
+ */
+static void gmem_put_handle(struct guest_memfd_handle *handle)
+{
+ if (handle->folio)
+ folio_put(handle->folio);
}
#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM)
@@ -523,19 +602,20 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
for (index = start; index < end; ) {
pgoff_t next_boundary;
pgoff_t max_nr_pages;
- int max_order = 0;
- kvm_pfn_t pfn;
+ struct guest_memfd_handle handle;
if (signal_pending(current)) {
r = -EINTR;
break;
}
- r = gmem_provider_get_pfn(gi, index, &pfn, &max_order);
+ r = gmem_provider_get_pfn(gi, index, &handle);
if (r)
break;
- max_nr_pages = 1UL << max_order;
+ max_nr_pages = 1UL << handle.max_order;
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
next_boundary = ALIGN_DOWN(index + max_nr_pages, max_nr_pages);
index += min_t(size_t, next_boundary - index, end - index);
cond_resched();
@@ -712,7 +792,7 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
struct gmem_inode *gi = GMEM_I(inode);
vm_fault_t ret = VM_FAULT_LOCKED;
struct folio *folio = NULL;
- kvm_pfn_t pfn;
+ struct guest_memfd_handle handle;
int r = 0;
if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
@@ -722,9 +802,12 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) {
r = -EACCES;
} else if (kvm_gmem_is_pfn_mode(gi)) {
- r = gmem_provider_get_pfn(gi, vmf->pgoff, &pfn, NULL);
- if (!r)
- ret = vmf_insert_pfn(vmf->vma, vmf->address, pfn);
+ r = gmem_provider_get_pfn(gi, vmf->pgoff, &handle);
+ if (!r) {
+ ret = vmf_insert_pfn(vmf->vma, vmf->address, handle.pfn);
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
+ }
} else {
folio = kvm_gmem_get_folio(inode, vmf->pgoff);
if (IS_ERR(folio))
@@ -1484,43 +1567,20 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
filemap_invalidate_unlock(file->f_mapping);
}
-/* On success, return a locked folio in folio mode and NULL in PFN mode. */
-static struct folio *__kvm_gmem_get_pfn(struct file *file,
- struct kvm_memory_slot *slot,
- pgoff_t index, kvm_pfn_t *pfn,
- int *max_order)
+/* Return a locked folio reference for the core to manage. */
+static int gmem_folio_get_pfn(struct inode *inode, void *provider, pgoff_t index,
+ struct guest_memfd_handle *handle)
{
- struct file *slot_file = READ_ONCE(slot->gmem.file);
- struct gmem_inode *gi = GMEM_I(file_inode(file));
- struct gmem_file *f = file->private_data;
struct folio *folio;
- int r;
-
- if (file != slot_file) {
- WARN_ON_ONCE(slot_file);
- return ERR_PTR(-EFAULT);
- }
- if (xa_load(&f->bindings, index) != slot) {
- WARN_ON_ONCE(xa_load(&f->bindings, index));
- return ERR_PTR(-EIO);
- }
-
- if (kvm_gmem_is_pfn_mode(gi)) {
- r = gmem_provider_get_pfn(gi, index, pfn, max_order);
- if (r)
- return ERR_PTR(r);
- return NULL;
- }
-
- folio = kvm_gmem_get_folio(file_inode(file), index);
+ folio = kvm_gmem_get_folio(inode, index);
if (IS_ERR(folio))
- return folio;
+ return PTR_ERR(folio);
if (folio_test_hwpoison(folio)) {
folio_unlock(folio);
folio_put(folio);
- return ERR_PTR(-EHWPOISON);
+ return -EHWPOISON;
}
if (!folio_test_uptodate(folio)) {
@@ -1528,47 +1588,64 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
folio_mark_uptodate(folio);
}
- *pfn = folio_file_pfn(folio, index);
- if (max_order)
- *max_order = 0;
+ handle->pfn = folio_file_pfn(folio, index);
+ handle->max_order = 0;
- return folio;
+ handle->folio = folio;
+ return 0;
+}
+
+static int kvm_gmem_resolve_pfn(struct file *file, struct kvm_memory_slot *slot,
+ pgoff_t index, struct guest_memfd_handle *handle)
+{
+ struct file *slot_file = READ_ONCE(slot->gmem.file);
+ struct gmem_inode *gi = GMEM_I(file_inode(file));
+ struct gmem_file *f = file->private_data;
+
+ if (file != slot_file) {
+ WARN_ON_ONCE(slot_file);
+ return -EFAULT;
+ }
+
+ if (xa_load(&f->bindings, index) != slot) {
+ WARN_ON_ONCE(xa_load(&f->bindings, index));
+ return -EIO;
+ }
+
+ return gmem_provider_get_pfn(gi, index, handle);
}
int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
gfn_t gfn, kvm_pfn_t *pfn, int *max_order)
{
pgoff_t index = kvm_gmem_get_index(slot, gfn);
- struct folio *folio;
- int r = 0, __order;
-
- max_order = max_order ?: &__order;
+ struct guest_memfd_handle handle;
+ struct inode *inode;
+ int ret;
CLASS(gmem_get_file, file)(slot);
if (!file)
return -EFAULT;
- filemap_invalidate_lock_shared(file_inode(file)->i_mapping);
+ inode = file_inode(file);
+ filemap_invalidate_lock_shared(inode->i_mapping);
- folio = __kvm_gmem_get_pfn(file, slot, index, pfn, max_order);
- if (IS_ERR(folio)) {
- r = PTR_ERR(folio);
+ ret = kvm_gmem_resolve_pfn(file, slot, index, &handle);
+ if (ret)
goto out;
- }
- if (kvm_arch_has_gmem_convert() &&
- kvm_gmem_is_private_mem(file_inode(file), index))
- r = kvm_arch_gmem_make_private(kvm, gfn, *pfn,
- (kvm_pfn_t)1 << *max_order);
-
- if (folio) {
- folio_unlock(folio);
- folio_put(folio);
- }
+ if (kvm_gmem_is_private_mem(inode, index) && kvm_arch_has_gmem_convert())
+ ret = kvm_arch_gmem_make_private(kvm, gfn, handle.pfn,
+ (kvm_pfn_t)1 << handle.max_order);
+ *pfn = handle.pfn;
+ if (max_order)
+ *max_order = handle.max_order;
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
out:
- filemap_invalidate_unlock_shared(file_inode(file)->i_mapping);
- return r;
+ filemap_invalidate_unlock_shared(inode->i_mapping);
+ return ret;
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
@@ -1579,31 +1656,23 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
kvm_gmem_populate_cb post_populate, void *opaque)
{
pgoff_t index = kvm_gmem_get_index(slot, gfn);
- struct folio *folio;
- kvm_pfn_t pfn;
+ struct guest_memfd_handle handle;
int ret;
filemap_invalidate_lock(file->f_mapping);
-
- folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, NULL);
- if (IS_ERR(folio)) {
- ret = PTR_ERR(folio);
+ ret = kvm_gmem_resolve_pfn(file, slot, index, &handle);
+ if (ret)
goto out_unlock;
- }
-
- if (folio)
- folio_unlock(folio);
+ gmem_unlock_handle(&handle);
if (!kvm_is_private_gfn(kvm, gfn)) {
ret = -EINVAL;
- goto out_put_folio;
+ goto out_put;
}
- ret = post_populate(kvm, gfn, pfn, src_page, opaque);
-
-out_put_folio:
- if (folio)
- folio_put(folio);
+ ret = post_populate(kvm, gfn, handle.pfn, src_page, opaque);
+out_put:
+ gmem_put_handle(&handle);
out_unlock:
filemap_invalidate_unlock(file->f_mapping);
return ret;
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 03/16] KVM: guest_memfd: Let providers coordinate mapping changes
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 01/16] KVM: guest_memfd: Add a handle for provider PFNs Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 02/16] KVM: guest_memfd: Get PFNs through common provider operations Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 04/16] KVM: guest_memfd: Add provider PFN range lookup Aneesh Kumar K.V (Arm)
` (12 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Add optional paired lock() and unlock() callbacks so providers can prevent
memory changes during guest invalidation and attribute conversion. Reject
providers that supply only one callback.
Take the provider lock with the inode invalidate lock held for write,
before guest invalidation. Release it after invalidation and any
successful attribute publication. Use the same locking for revocation
and final inode cleanup.
This lock is separate from the lock taken by get_pfn() for an individual
PFN. Existing providers can omit both callbacks.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: kvm@vger.kernel.org
Cc: patches@lists.linux.dev
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 8 ++++++
virt/kvm/guest_memfd.c | 50 +++++++++++++++++++++++++++++++++++++
2 files changed, 58 insertions(+)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 3d84fd0adec1..b63ca59773f9 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -36,6 +36,12 @@ struct guest_memfd_handle {
* @attach: Attach to a resource file, perform filesystem-specific validation,
* and return an opaque provider context.
* @release: Release provider context and unpin any resources.
+ * @lock: Optional provider lock for mapping and attribute changes. Called
+ * with the inode invalidate lock held for write, before guest
+ * invalidation starts.
+ * @unlock: Release the provider lock taken by lock(). Called after guest
+ * invalidation ends and any successful attribute publication, with
+ * the inode invalidate lock still held.
* @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
@@ -66,6 +72,8 @@ struct guest_memfd_handle {
struct guest_memfd_provider_operations {
void *(*attach)(struct inode *gmem_inode, struct file *resource_file);
void (*release)(void *provider);
+ void (*lock)(void *provider);
+ void (*unlock)(void *provider);
struct folio *(*alloc_folio)(void *provider, pgoff_t index,
struct mempolicy *mpol);
void (*invalidate_folio)(void *provider, struct folio *folio);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 026a7bd0fe76..5fda14a53488 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -96,6 +96,9 @@ static bool gmem_provider_ops_valid(struct gmem_inode *gi,
if (!ops || !ops->attach)
return false;
+ if (!!ops->lock != !!ops->unlock)
+ return false;
+
if (ops == &gmem_native_provider_ops)
return !kvm_gmem_is_pfn_mode(gi);
@@ -269,6 +272,42 @@ static void gmem_provider_put_pfn_range(struct gmem_inode *gi, pgoff_t index,
gi->provider_ops->put_pfn_range(gi->provider, index, nr_pages);
}
+/**
+ * gmem_provider_lock_updates - Take the provider lock for memory updates
+ * @gi: guest_memfd inode containing the attached provider
+ *
+ * Prevent providers from moving or reassigning memory during guest
+ * invalidation, attribute conversion/publication, revocation, unbinding and
+ * final cleanup. Pair with gmem_provider_unlock_updates(). Providers
+ * requiring no additional locking may omit both lock() and unlock().
+ *
+ * This is separate from locking inside get_pfn(); it neither acquires a PFN
+ * reference nor pairs with gmem_unlock_handle().
+ *
+ * Caller holds the inode invalidate lock for write through
+ * gmem_provider_unlock_updates().
+ */
+static void gmem_provider_lock_updates(struct gmem_inode *gi)
+{
+ if (gi->provider_ops && gi->provider_ops->lock)
+ gi->provider_ops->lock(gi->provider);
+}
+
+/**
+ * gmem_provider_unlock_updates - Release the provider lock for memory updates
+ * @gi: guest_memfd inode passed to gmem_provider_lock_updates()
+ *
+ * Release the provider lock after guest invalidation and any attribute
+ * publication are complete. This does not unlock or put an individual PFN reference.
+ *
+ * Caller holds the inode invalidate lock for write.
+ */
+static void gmem_provider_unlock_updates(struct gmem_inode *gi)
+{
+ if (gi->provider_ops && gi->provider_ops->unlock)
+ gi->provider_ops->unlock(gi->provider);
+}
+
static void gmem_provider_release(struct gmem_inode *gi)
{
if (gi->provider_ops && gi->provider_ops->release)
@@ -506,6 +545,7 @@ static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
{
enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
pgoff_t max_pages = i_size_read(inode) >> PAGE_SHIFT;
+ struct gmem_inode *gi = GMEM_I(inode);
loff_t offset, len;
pgoff_t end;
@@ -526,6 +566,8 @@ static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
*/
filemap_invalidate_lock(inode->i_mapping);
+ gmem_provider_lock_updates(gi);
+
kvm_gmem_invalidate_start(inode, start, end, filter);
if (kvm_gmem_is_pfn_mode(GMEM_I(inode))) {
@@ -543,6 +585,7 @@ static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
}
kvm_gmem_invalidate_end(inode, start, end);
+ gmem_provider_unlock_updates(gi);
filemap_invalidate_unlock(inode->i_mapping);
@@ -1118,6 +1161,8 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
* checks and can proceed to do guest-breaking changes.
*/
+ gmem_provider_lock_updates(gi);
+
filter = to_private ? KVM_FILTER_SHARED : KVM_FILTER_PRIVATE;
kvm_gmem_invalidate_start(inode, start, end, filter);
@@ -1127,6 +1172,7 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
mas_store_prealloc(&mas, xa_mk_value(attrs));
kvm_gmem_invalidate_end(inode, start, end);
+ gmem_provider_unlock_updates(gi);
out:
filemap_invalidate_unlock(mapping);
return r;
@@ -1793,8 +1839,12 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi)) {
pgoff_t nr_pages = i_size_read(inode) >> PAGE_SHIFT;
+ filemap_invalidate_lock(inode->i_mapping);
+ gmem_provider_lock_updates(gi);
kvm_gmem_pfn_reclaim(gi, 0, nr_pages);
gmem_provider_put_pfn_range(gi, 0, nr_pages);
+ gmem_provider_unlock_updates(gi);
+ filemap_invalidate_unlock(inode->i_mapping);
}
gmem_provider_release(gi);
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 04/16] KVM: guest_memfd: Add provider PFN range lookup
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (2 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 03/16] KVM: guest_memfd: Let providers coordinate mapping changes Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 05/16] KVM: guest_memfd: Let providers validate attribute changes Aneesh Kumar K.V (Arm)
` (11 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Provide nonallocating folio range lookup and carry PFN attributes separately
from requested KVM memory attributes
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 10 ++++--
samples/kvm/mem_provider_sample.c | 6 ++--
virt/kvm/guest_memfd.c | 60 ++++++++++++++++++++++++++-----
3 files changed, 63 insertions(+), 13 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index b63ca59773f9..d3b153a3d891 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -60,7 +60,10 @@ struct guest_memfd_handle {
* is present. If backing memory is not available, return
* -ENOENT. Return a negative error code on other failure.
* The provider must define @get_pfn when @lookup_pfn_range
- * is defined.
+ * is defined, or omit both to use the folio helper.
+ * Lookup must not allocate memory or prepare it for access.
+ * PFNs remain stable under the inode invalidate lock and
+ * any provider lock held by conversion/reclaim callers.
* @put_pfn_range: Release a range of PFNs previously obtained for a given
* index. The provider does not need to ensure that backing
* memory is present. If backing memory is not available, the
@@ -79,8 +82,9 @@ struct guest_memfd_provider_operations {
void (*invalidate_folio)(void *provider, struct folio *folio);
int (*get_pfn)(struct inode *inode, void *provider, pgoff_t index,
struct guest_memfd_handle *handle);
- int (*lookup_pfn_range)(void *provider, pgoff_t index,
- unsigned long *pfn, size_t *nr_pages);
+ int (*lookup_pfn_range)(struct inode *inode, void *provider, pgoff_t index,
+ unsigned long *pfn, size_t *nr_pages,
+ u64 *pfn_attrs);
void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
};
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
index f1b0698268c0..8eca914c49ef 100644
--- a/samples/kvm/mem_provider_sample.c
+++ b/samples/kvm/mem_provider_sample.c
@@ -197,8 +197,9 @@ static int mps_gmem_get_pfn(struct inode *inode, void *provider,
return 0;
}
-static int mps_gmem_lookup_pfn_range(void *provider, pgoff_t index,
- unsigned long *pfn, size_t *nr_pages)
+static int mps_gmem_lookup_pfn_range(struct inode *inode, void *provider,
+ pgoff_t index, unsigned long *pfn,
+ size_t *nr_pages, u64 *pfn_attrs)
{
struct file *resource_file = provider;
struct mps_info *info = resource_file->private_data;
@@ -211,6 +212,7 @@ static int mps_gmem_lookup_pfn_range(void *provider, pgoff_t index,
*pfn = info->base_pfn + index;
*nr_pages = 1;
+ *pfn_attrs = 0;
return 0;
}
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 5fda14a53488..ad9cc1bbe5d7 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -52,6 +52,8 @@ struct gmem_inode {
static int gmem_folio_get_pfn(struct inode *inode, void *provider, pgoff_t index,
struct guest_memfd_handle *handle);
+static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
+ pgoff_t index, unsigned long *pfn, size_t *nr_pages, u64 *pfn_attrs);
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
{
@@ -88,6 +90,7 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.attach = gmem_native_attach,
.alloc_folio = gmem_native_alloc_folio,
.get_pfn = gmem_folio_get_pfn,
+ .lookup_pfn_range = gmem_folio_lookup_pfn_range,
};
static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -129,8 +132,10 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
{
gi->provider_operations = *ops;
if (ops->alloc_folio) {
- if (!ops->get_pfn)
+ if (!ops->get_pfn) {
gi->provider_operations.get_pfn = gmem_folio_get_pfn;
+ gi->provider_operations.lookup_pfn_range = gmem_folio_lookup_pfn_range;
+ }
}
gi->provider_ops = &gi->provider_operations;
}
@@ -239,13 +244,15 @@ static void gmem_put_handle(struct guest_memfd_handle *handle)
#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM)
static int gmem_provider_lookup_pfn_range(struct gmem_inode *gi, pgoff_t index,
- kvm_pfn_t *pfn, size_t *nr_pages)
+ kvm_pfn_t *pfn, size_t *nr_pages, u64 *pfn_attrs)
{
unsigned long raw_pfn;
int r;
- r = gi->provider_ops->lookup_pfn_range(gi->provider, index, &raw_pfn,
- nr_pages);
+ /* Initialize the length in case an error leaves it unset by the provider. */
+ *nr_pages = 0;
+ r = gi->provider_ops->lookup_pfn_range(&gi->vfs_inode, gi->provider, index, &raw_pfn,
+ nr_pages, pfn_attrs);
if (r)
return r;
@@ -433,6 +440,38 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
return folio;
}
+/* Conversion holds invalidate_lock for write, excluding truncation/allocation. */
+static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
+ pgoff_t index, unsigned long *pfn, size_t *nr_pages, u64 *pfn_attrs)
+{
+ struct folio *folio;
+
+ lockdep_assert_held_write(&inode->i_mapping->invalidate_lock);
+ folio = filemap_get_folio(inode->i_mapping, index);
+ if (IS_ERR(folio)) {
+ struct folio_batch fbatch;
+ pgoff_t end = i_size_read(inode) >> PAGE_SHIFT;
+ pgoff_t next = index;
+
+ if (PTR_ERR(folio) != -ENOENT)
+ return PTR_ERR(folio);
+
+ folio_batch_init(&fbatch);
+ if (filemap_get_folios(inode->i_mapping, &next, end - 1, &fbatch)) {
+ end = fbatch.folios[0]->index;
+ folio_batch_release(&fbatch);
+ }
+ *nr_pages = end - index;
+ return -ENOENT;
+ }
+
+ *pfn = folio_file_pfn(folio, index);
+ *nr_pages = folio_next_index(folio) - index;
+ *pfn_attrs = 0;
+ folio_put(folio);
+ return 0;
+}
+
static enum kvm_gfn_range_filter kvm_gmem_get_all_gfns_filter(struct inode *inode)
{
if (gmem_in_place_conversion)
@@ -525,11 +564,14 @@ static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t e
size_t step = 1;
size_t nr_pages;
kvm_pfn_t pfn;
+ u64 pfn_attrs;
+ int ret;
- if (!gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages)) {
- step = min_t(size_t, nr_pages, end - index);
+ ret = gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages,
+ &pfn_attrs);
+ step = min_t(size_t, max_t(size_t, nr_pages, 1), end - index);
+ if (!ret && !(pfn_attrs & GUEST_MEMFD_PFN_MMIO))
kvm_arch_gmem_reclaim(pfn, step);
- }
index += step;
cond_resched();
@@ -1068,8 +1110,10 @@ static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff_t end
size_t step = 1;
size_t nr_pages;
kvm_pfn_t pfn;
+ u64 pfn_attrs;
- if (!gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages)) {
+ if (!gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages,
+ &pfn_attrs)) {
step = min_t(size_t, nr_pages, end - index);
kvm_arch_gmem_make_shared(pfn, step);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 05/16] KVM: guest_memfd: Let providers validate attribute changes
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (3 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 04/16] KVM: guest_memfd: Add provider PFN range lookup Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 06/16] KVM: guest_memfd: Share memory through provider range lookup Aneesh Kumar K.V (Arm)
` (10 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Call prepare_attributes() before invalidating guest mappings or changing
file attributes. Remove host mappings first when converting to private,
so folio validation can check for outstanding references.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 9 ++++++++
virt/kvm/guest_memfd.c | 43 ++++++++++++++++++++++++-------------
2 files changed, 37 insertions(+), 15 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index d3b153a3d891..06f094a2fd49 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -42,6 +42,13 @@ struct guest_memfd_handle {
* @unlock: Release the provider lock taken by lock(). Called after guest
* invalidation ends and any successful attribute publication, with
* the inode invalidate lock still held.
+ * @prepare_attributes: Optional memory validation before conversion, under
+ * the inode invalidate lock for write, before taking
+ * the provider lock or starting guest invalidation. Host mappings have
+ * been removed for private conversion. Set @err_index on
+ * failure; it initially points to the range start. NULL
+ * selects folio validation for folio providers; for
+ * other providers, NULL skips this validation.
* @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
@@ -77,6 +84,8 @@ struct guest_memfd_provider_operations {
void (*release)(void *provider);
void (*lock)(void *provider);
void (*unlock)(void *provider);
+ int (*prepare_attributes)(struct inode *inode, void *provider, pgoff_t start,
+ size_t nr_pages, u64 memory_attrs, pgoff_t *err_index);
struct folio *(*alloc_folio)(void *provider, pgoff_t index,
struct mempolicy *mpol);
void (*invalidate_folio)(void *provider, struct folio *folio);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ad9cc1bbe5d7..c69a8b7d046c 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -54,6 +54,8 @@ static int gmem_folio_get_pfn(struct inode *inode, void *provider, pgoff_t index
struct guest_memfd_handle *handle);
static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
pgoff_t index, unsigned long *pfn, size_t *nr_pages, u64 *pfn_attrs);
+static int gmem_folio_prepare_attributes(struct inode *inode, void *provider,
+ pgoff_t start, size_t nr_pages, u64 attrs, pgoff_t *err_index);
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
{
@@ -91,6 +93,7 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.alloc_folio = gmem_native_alloc_folio,
.get_pfn = gmem_folio_get_pfn,
.lookup_pfn_range = gmem_folio_lookup_pfn_range,
+ .prepare_attributes = gmem_folio_prepare_attributes,
};
static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -136,6 +139,8 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
gi->provider_operations.get_pfn = gmem_folio_get_pfn;
gi->provider_operations.lookup_pfn_range = gmem_folio_lookup_pfn_range;
}
+ if (!ops->prepare_attributes)
+ gi->provider_operations.prepare_attributes = gmem_folio_prepare_attributes;
}
gi->provider_ops = &gi->provider_operations;
}
@@ -1095,6 +1100,16 @@ static bool kvm_gmem_has_outstanding_references(struct inode *inode,
return false;
}
+static int gmem_folio_prepare_attributes(struct inode *inode, void *provider,
+ pgoff_t start, size_t nr_pages, u64 attrs, pgoff_t *err_index)
+{
+ if ((attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE) &&
+ kvm_gmem_has_outstanding_references(inode, start, nr_pages, err_index))
+ return -EAGAIN;
+
+ return 0;
+}
+
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff_t end)
{
@@ -1169,6 +1184,7 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
mt = &gi->attributes;
+ *err_index = start;
filemap_invalidate_lock(mapping);
if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs))
@@ -1176,26 +1192,23 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
mas_init(&mas, mt, start);
r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
- if (r) {
- *err_index = start;
+ if (r)
goto out;
- }
- if (to_private) {
- /*
- * Forcefully unmap the pages from all userspace page tables,
- * and then verify there are no outstanding references, e.g.
- * acquired via GUP or similar. Tell userspace to try again if
- * there are outstanding references and hope that whatever has
- * pinned the page will put its reference "soon".
- */
+ /*
+ * Before private conversion, forcefully unmap the pages from all userspace
+ * page tables. The folio provider then checks for outstanding references,
+ * e.g. from GUP, and returns -EAGAIN so userspace can retry after those
+ * references are released.
+ */
+ if (to_private)
unmap_mapping_pages(mapping, start, nr_pages, false);
- if (!kvm_gmem_is_pfn_mode(gi) &&
- kvm_gmem_has_outstanding_references(inode, start, nr_pages,
- err_index)) {
+ if (gi->provider_ops->prepare_attributes) {
+ r = gi->provider_ops->prepare_attributes(inode, gi->provider, start,
+ nr_pages, attrs, err_index);
+ if (r) {
mas_destroy(&mas);
- r = -EAGAIN;
goto out;
}
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 06/16] KVM: guest_memfd: Share memory through provider range lookup
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (4 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 05/16] KVM: guest_memfd: Let providers validate attribute changes Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 07/16] KVM: guest_memfd: Release and revoke memory through provider operations Aneesh Kumar K.V (Arm)
` (9 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Use provider range lookup for both folios and PFNs when converting memory
to shared. Skip holes without allocating pages and return lookup errors.
Invalidate mappings before conversion and publish shared attributes only
after conversion succeeds.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
virt/kvm/guest_memfd.c | 126 ++++++++++++++++++++---------------------
1 file changed, 62 insertions(+), 64 deletions(-)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index c69a8b7d046c..ffe0a8e0f2dd 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1110,87 +1110,79 @@ static int gmem_folio_prepare_attributes(struct inode *inode, void *provider,
return 0;
}
-#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
-static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff_t end)
+static int gmem_provider_make_shared(struct inode *inode, pgoff_t start,
+ pgoff_t end)
{
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
struct gmem_inode *gi = GMEM_I(inode);
- struct folio_batch fbatch;
- pgoff_t next = start;
- int i;
+ void *entry;
- if (kvm_gmem_is_pfn_mode(gi)) {
- pgoff_t index;
+ MA_STATE(mas, &gi->attributes, start, start);
- for (index = start; index < end; ) {
- size_t step = 1;
+ if (!kvm_arch_has_gmem_convert())
+ return 0;
+
+ mas_for_each(&mas, entry, end - 1) {
+ pgoff_t first = max(start, mas.index);
+ pgoff_t last = min(end, mas.last + 1);
+
+ if (!(kvm_gmem_get_attributes(inode, entry) & KVM_MEMORY_ATTRIBUTE_PRIVATE))
+ continue;
+
+ while (first < last) {
size_t nr_pages;
- kvm_pfn_t pfn;
u64 pfn_attrs;
-
- if (!gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages,
- &pfn_attrs)) {
- step = min_t(size_t, nr_pages, end - index);
- kvm_arch_gmem_make_shared(pfn, step);
+ kvm_pfn_t pfn;
+ int ret;
+
+ ret = gmem_provider_lookup_pfn_range(gi, first, &pfn, &nr_pages,
+ &pfn_attrs);
+ if (ret == -ENOENT) {
+ /*
+ * Skip the reported hole, advancing at least one page
+ * even if nr_pages is zero, but stop at the end of this
+ * attribute range.
+ */
+ first += min_t(size_t, max_t(size_t, nr_pages, 1),
+ last - first);
+ cond_resched();
+ continue;
}
+ if (ret)
+ return ret;
- index += step;
- cond_resched();
- }
- return;
- }
+ nr_pages = min_t(size_t, nr_pages, last - first);
+ kvm_arch_gmem_make_shared(pfn, nr_pages);
+ ret = 0;
+ if (ret)
+ return ret;
- folio_batch_init(&fbatch);
- while (filemap_get_folios(inode->i_mapping, &next, end - 1, &fbatch)) {
- for (i = 0; i < folio_batch_count(&fbatch); ++i) {
- struct folio *folio = fbatch.folios[i];
- pgoff_t start_index, end_index;
- kvm_pfn_t start_pfn;
- kvm_pfn_t nr_pages;
-
- start_index = max(start, folio->index);
- end_index = min(end, folio_next_index(folio));
- /*
- * end_index is either in folio or points to
- * the first page of the next folio. Hence,
- * all pages in range [start_index, end_index)
- * are contiguous.
- */
- start_pfn = folio_file_pfn(folio, start_index);
- nr_pages = end_index - start_index;
-
- kvm_arch_gmem_make_shared(start_pfn, nr_pages);
+ first += nr_pages;
+ cond_resched();
}
-
- folio_batch_release(&fbatch);
- cond_resched();
}
-}
-#else
-static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff_t end) {}
#endif
+ return 0;
+}
static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
- size_t nr_pages, uint64_t attrs,
- pgoff_t *err_index)
+ size_t nr_pages, u64 attrs, pgoff_t *err_index)
{
bool to_private = attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE;
struct address_space *mapping = inode->i_mapping;
struct gmem_inode *gi = GMEM_I(inode);
enum kvm_gfn_range_filter filter;
pgoff_t end = start + nr_pages;
- struct maple_tree *mt;
struct ma_state mas;
int r = 0;
- mt = &gi->attributes;
-
*err_index = start;
filemap_invalidate_lock(mapping);
if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs))
goto out;
- mas_init(&mas, mt, start);
+ mas_init(&mas, &gi->attributes, start);
r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
if (r)
goto out;
@@ -1207,29 +1199,35 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
if (gi->provider_ops->prepare_attributes) {
r = gi->provider_ops->prepare_attributes(inode, gi->provider, start,
nr_pages, attrs, err_index);
- if (r) {
- mas_destroy(&mas);
- goto out;
- }
+ if (r)
+ goto destroy;
}
- /*
- * From this point on guest_memfd has performed necessary
- * checks and can proceed to do guest-breaking changes.
- */
-
gmem_provider_lock_updates(gi);
filter = to_private ? KVM_FILTER_SHARED : KVM_FILTER_PRIVATE;
kvm_gmem_invalidate_start(inode, start, end, filter);
- if (!to_private && kvm_arch_has_gmem_convert())
- kvm_gmem_make_shared(inode, start, end);
+ /*
+ * Make existing pages host-accessible before publishing shared attributes.
+ * Host faults map shared pages without performing architecture conversion.
+ * Private conversion revokes host mappings here, but can defer making
+ * pages private until a guest fault or prefault.
+ */
+ if (!to_private)
+ r = gmem_provider_make_shared(inode, start, end);
- mas_store_prealloc(&mas, xa_mk_value(attrs));
+ if (!r)
+ mas_store_prealloc(&mas, xa_mk_value(attrs));
+ if (r)
+ mas_destroy(&mas);
kvm_gmem_invalidate_end(inode, start, end);
gmem_provider_unlock_updates(gi);
+ goto out;
+
+destroy:
+ mas_destroy(&mas);
out:
filemap_invalidate_unlock(mapping);
return r;
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 07/16] KVM: guest_memfd: Release and revoke memory through provider operations
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (5 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 06/16] KVM: guest_memfd: Share memory through provider range lookup Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 08/16] KVM: guest_memfd: Use provider operations for fallocate allocation Aneesh Kumar K.V (Arm)
` (8 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Add release_range() helpers for folios and PFNs. Keep guest invalidation
in the core. Folio release truncates memory; ->free_folio() calls
kvm_arch_gmem_reclaim() when the folio is removed from the page cache.
PFN release removes host mappings and calls kvm_arch_gmem_reclaim()
before put_pfn_range() releases the memory.
Use these operations for hole punching, provider revocation and final
inode cleanup.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 13 +++-
samples/kvm/mem_provider_sample.c | 3 +-
virt/kvm/guest_memfd.c | 106 +++++++++++++++++++++---------
3 files changed, 88 insertions(+), 34 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 06f094a2fd49..2ed929a9dca5 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -49,6 +49,12 @@ struct guest_memfd_handle {
* failure; it initially points to the range start. NULL
* selects folio validation for folio providers; for
* other providers, NULL skips this validation.
+ * @release_range: Remove host mappings and release memory in [start, end),
+ * after core guest invalidation, under the inode invalidate
+ * write lock and any provider lock. Also called at final
+ * inode cleanup, when no bindings remain. The optional pfn
+ * identifies memory revoked by the provider. release_backing
+ * is false for provider-initiated revocation, true otherwise.
* @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
@@ -86,6 +92,9 @@ struct guest_memfd_provider_operations {
void (*unlock)(void *provider);
int (*prepare_attributes)(struct inode *inode, void *provider, pgoff_t start,
size_t nr_pages, u64 memory_attrs, pgoff_t *err_index);
+ int (*release_range)(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs, bool release_backing);
struct folio *(*alloc_folio)(void *provider, pgoff_t index,
struct mempolicy *mpol);
void (*invalidate_folio)(void *provider, struct folio *folio);
@@ -99,10 +108,10 @@ struct guest_memfd_provider_operations {
#ifdef CONFIG_KVM_GUEST_MEMFD
int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
- const unsigned long *pfn);
+ const unsigned long *pfn, u64 pfn_attrs);
#else
static inline int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
- const unsigned long *pfn)
+ const unsigned long *pfn, u64 pfn_attrs)
{
return -EOPNOTSUPP;
}
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
index 8eca914c49ef..4b7a8f760fdf 100644
--- a/samples/kvm/mem_provider_sample.c
+++ b/samples/kvm/mem_provider_sample.c
@@ -88,7 +88,8 @@ static void mps_set_present(struct mps_info *info, unsigned long start,
if (info->gmem_inode) {
unsigned long pfn = info->base_pfn + start;
- kvm_gmem_revoke(info->gmem_inode, start, end - start, &pfn);
+ kvm_gmem_revoke(info->gmem_inode, start, end - start,
+ &pfn, 0);
mps_zero_range(info, start, end - start);
}
}
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ffe0a8e0f2dd..a224ae969d79 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -56,6 +56,12 @@ static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
pgoff_t index, unsigned long *pfn, size_t *nr_pages, u64 *pfn_attrs);
static int gmem_folio_prepare_attributes(struct inode *inode, void *provider,
pgoff_t start, size_t nr_pages, u64 attrs, pgoff_t *err_index);
+static int gmem_folio_release_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs, bool release_backing);
+static int gmem_pfn_release_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs, bool release_backing);
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
{
@@ -94,6 +100,7 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.get_pfn = gmem_folio_get_pfn,
.lookup_pfn_range = gmem_folio_lookup_pfn_range,
.prepare_attributes = gmem_folio_prepare_attributes,
+ .release_range = gmem_folio_release_range,
};
static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -141,6 +148,11 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
}
if (!ops->prepare_attributes)
gi->provider_operations.prepare_attributes = gmem_folio_prepare_attributes;
+ if (!ops->release_range)
+ gi->provider_operations.release_range = gmem_folio_release_range;
+ } else {
+ if (!ops->release_range)
+ gi->provider_operations.release_range = gmem_pfn_release_range;
}
gi->provider_ops = &gi->provider_operations;
}
@@ -582,19 +594,66 @@ static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t e
cond_resched();
}
}
-#else
-static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t end) {}
#endif
+/*
+ * Folios are managed by the page cache. Truncation removes them, and the MM
+ * invokes ->free_folio(), which calls kvm_arch_gmem_reclaim(). Keep that
+ * cleanup in ->free_folio() instead of duplicating it in the release path.
+ */
+static int gmem_folio_release_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs, bool release_backing)
+{
+ loff_t offset = (loff_t)start << PAGE_SHIFT;
+ loff_t len = (loff_t)(end - start) << PAGE_SHIFT;
+
+ truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
+ return 0;
+}
+
+static void gmem_provider_reclaim(struct gmem_inode *gi, pgoff_t start,
+ pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs)
+{
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
+ if (pfn_attrs & GUEST_MEMFD_PFN_MMIO)
+ return;
+ if (pfn)
+ kvm_arch_gmem_reclaim(*pfn, end - start);
+ else
+ kvm_gmem_pfn_reclaim(gi, start, end);
+#endif
+}
+
+static int gmem_pfn_release_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end, const unsigned long *pfn,
+ u64 pfn_attrs, bool release_backing)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+
+ unmap_mapping_pages(inode->i_mapping, start, end - start, true);
+ gmem_provider_reclaim(gi, start, end, pfn, pfn_attrs);
+
+ if (release_backing)
+ gmem_provider_put_pfn_range(gi, start, end - start);
+
+ return 0;
+}
+
static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
size_t nr_pages, const unsigned long *pfn,
- bool put_pfn_range)
+ u64 pfn_attrs, bool release_backing)
{
enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
pgoff_t max_pages = i_size_read(inode) >> PAGE_SHIFT;
struct gmem_inode *gi = GMEM_I(inode);
- loff_t offset, len;
+ const struct guest_memfd_provider_operations *ops = gi->provider_ops;
pgoff_t end;
+ int r;
+
+ if (!ops->release_range)
+ return -EOPNOTSUPP;
if (start >= max_pages)
return 0;
@@ -603,8 +662,6 @@ static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
if (!nr_pages)
return 0;
- offset = (loff_t)start << PAGE_SHIFT;
- len = (loff_t)nr_pages << PAGE_SHIFT;
end = start + nr_pages;
/*
@@ -617,32 +674,20 @@ static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
kvm_gmem_invalidate_start(inode, start, end, filter);
- if (kvm_gmem_is_pfn_mode(GMEM_I(inode))) {
- unmap_mapping_range(inode->i_mapping, offset, len, 0);
-#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
- if (pfn)
- kvm_arch_gmem_reclaim(*pfn, nr_pages);
- else
- kvm_gmem_pfn_reclaim(GMEM_I(inode), start, end);
-#endif
- if (put_pfn_range)
- gmem_provider_put_pfn_range(GMEM_I(inode), start, nr_pages);
- } else {
- truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
- }
+ r = ops->release_range(inode, gi->provider, start, end, pfn, pfn_attrs,
+ release_backing);
kvm_gmem_invalidate_end(inode, start, end);
gmem_provider_unlock_updates(gi);
-
filemap_invalidate_unlock(inode->i_mapping);
- return 0;
+ return r;
}
static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
{
return __kvm_gmem_revoke(inode, offset >> PAGE_SHIFT, len >> PAGE_SHIFT,
- NULL, true);
+ NULL, 0, true);
}
/**
@@ -653,6 +698,7 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
* @pfn: The physical address / frame number being revoked. Directly
* reclaims this physical range without querying the provider's
* .lookup_pfn_range().
+ * @pfn_attrs: GUEST_MEMFD_PFN_* attributes of the supplied PFNs.
*
* guest_memfd will do everything to make sure the range revoked is no longer in
* use across guest and host mappings, as well as reclaim any confidential
@@ -665,9 +711,9 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
* Return: 0 on success, or a negative error code on failure.
*/
int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
- const unsigned long *pfn)
+ const unsigned long *pfn, u64 pfn_attrs)
{
- return __kvm_gmem_revoke(inode, start, nr_pages, pfn, false);
+ return __kvm_gmem_revoke(inode, start, nr_pages, pfn, pfn_attrs, false);
}
EXPORT_SYMBOL_GPL(kvm_gmem_revoke);
@@ -1887,17 +1933,15 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
{
struct gmem_inode *gi = GMEM_I(inode);
- /*
- * Skip if the provider was not yet set up, as iput() can reach this on
- * error paths before provider_ops is initialized.
- */
- if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi)) {
+ /* Provider setup may have failed before this inode was published. */
+ if (gi->provider_ops) {
pgoff_t nr_pages = i_size_read(inode) >> PAGE_SHIFT;
filemap_invalidate_lock(inode->i_mapping);
gmem_provider_lock_updates(gi);
- kvm_gmem_pfn_reclaim(gi, 0, nr_pages);
- gmem_provider_put_pfn_range(gi, 0, nr_pages);
+ if (nr_pages)
+ gi->provider_ops->release_range(inode, gi->provider, 0,
+ nr_pages, NULL, 0, true);
gmem_provider_unlock_updates(gi);
filemap_invalidate_unlock(inode->i_mapping);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 08/16] KVM: guest_memfd: Use provider operations for fallocate allocation
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (6 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 07/16] KVM: guest_memfd: Release and revoke memory through provider operations Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 09/16] KVM: guest_memfd: Use provider operations for host mappings and faults Aneesh Kumar K.V (Arm)
` (7 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Use allocate_range() for fallocate allocation so providers can supply
allocation and release callback. When a provider supplies neither callback,
guest_memfd handles allocation and release for it. Providers that supply
only release_range() cannot use fallocate(), including hole punching.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 8 +++
virt/kvm/guest_memfd.c | 122 +++++++++++++++++++++++-------------
2 files changed, 87 insertions(+), 43 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 2ed929a9dca5..e11b2392b001 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -49,12 +49,18 @@ struct guest_memfd_handle {
* failure; it initially points to the range start. NULL
* selects folio validation for folio providers; for
* other providers, NULL skips this validation.
+ * @allocate_range: Preallocate [start, end), without architecture private
+ * preparation. Called with the inode invalidate lock held
+ * for read.
* @release_range: Remove host mappings and release memory in [start, end),
* after core guest invalidation, under the inode invalidate
* write lock and any provider lock. Also called at final
* inode cleanup, when no bindings remain. The optional pfn
* identifies memory revoked by the provider. release_backing
* is false for provider-initiated revocation, true otherwise.
+ * Supplying this without allocate_range()
+ * supports a fixed memory range and disables fallocate(),
+ * including hole punching.
* @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
@@ -92,6 +98,8 @@ struct guest_memfd_provider_operations {
void (*unlock)(void *provider);
int (*prepare_attributes)(struct inode *inode, void *provider, pgoff_t start,
size_t nr_pages, u64 memory_attrs, pgoff_t *err_index);
+ int (*allocate_range)(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end);
int (*release_range)(struct inode *inode, void *provider,
pgoff_t start, pgoff_t end, const unsigned long *pfn,
u64 pfn_attrs, bool release_backing);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index a224ae969d79..c6b1814b5e7e 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -56,6 +56,10 @@ static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
pgoff_t index, unsigned long *pfn, size_t *nr_pages, u64 *pfn_attrs);
static int gmem_folio_prepare_attributes(struct inode *inode, void *provider,
pgoff_t start, size_t nr_pages, u64 attrs, pgoff_t *err_index);
+static int gmem_folio_allocate_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end);
+static int gmem_pfn_allocate_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end);
static int gmem_folio_release_range(struct inode *inode, void *provider,
pgoff_t start, pgoff_t end, const unsigned long *pfn,
u64 pfn_attrs, bool release_backing);
@@ -100,6 +104,7 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.get_pfn = gmem_folio_get_pfn,
.lookup_pfn_range = gmem_folio_lookup_pfn_range,
.prepare_attributes = gmem_folio_prepare_attributes,
+ .allocate_range = gmem_folio_allocate_range,
.release_range = gmem_folio_release_range,
};
@@ -122,6 +127,10 @@ static bool gmem_provider_ops_valid(struct gmem_inode *gi,
if (ops->invalidate_folio && !ops->alloc_folio)
return false;
+ /* A fixed memory range may support release without supporting allocation. */
+ if (ops->allocate_range && !ops->release_range)
+ return false;
+
/* Custom folio get_pfn() needs matching host mapping callbacks. */
if (ops->alloc_folio && ops->get_pfn &&
(gi->flags & GUEST_MEMFD_FLAG_MMAP))
@@ -148,11 +157,15 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
}
if (!ops->prepare_attributes)
gi->provider_operations.prepare_attributes = gmem_folio_prepare_attributes;
- if (!ops->release_range)
+ if (!ops->release_range) {
+ gi->provider_operations.allocate_range = gmem_folio_allocate_range;
gi->provider_operations.release_range = gmem_folio_release_range;
+ }
} else {
- if (!ops->release_range)
+ if (!ops->release_range) {
+ gi->provider_operations.allocate_range = gmem_pfn_allocate_range;
gi->provider_operations.release_range = gmem_pfn_release_range;
+ }
}
gi->provider_ops = &gi->provider_operations;
}
@@ -717,47 +730,11 @@ int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
}
EXPORT_SYMBOL_GPL(kvm_gmem_revoke);
-static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
+static int gmem_folio_allocate_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end)
{
- struct address_space *mapping = inode->i_mapping;
- struct gmem_inode *gi = GMEM_I(inode);
- pgoff_t start, index, end;
- int r;
-
- /* Dedicated guest is immutable by default. */
- if (offset + len > i_size_read(inode))
- return -EINVAL;
-
- filemap_invalidate_lock_shared(mapping);
-
- start = offset >> PAGE_SHIFT;
- end = (offset + len) >> PAGE_SHIFT;
-
- r = 0;
- if (kvm_gmem_is_pfn_mode(gi)) {
- for (index = start; index < end; ) {
- pgoff_t next_boundary;
- pgoff_t max_nr_pages;
- struct guest_memfd_handle handle;
-
- if (signal_pending(current)) {
- r = -EINTR;
- break;
- }
-
- r = gmem_provider_get_pfn(gi, index, &handle);
- if (r)
- break;
-
- max_nr_pages = 1UL << handle.max_order;
- gmem_unlock_handle(&handle);
- gmem_put_handle(&handle);
- next_boundary = ALIGN_DOWN(index + max_nr_pages, max_nr_pages);
- index += min_t(size_t, next_boundary - index, end - index);
- cond_resched();
- }
- goto out_unlock;
- }
+ pgoff_t index;
+ int r = 0;
for (index = start; index < end; ) {
struct folio *folio;
@@ -785,7 +762,62 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
cond_resched();
}
-out_unlock:
+ return r;
+}
+
+static int gmem_pfn_allocate_range(struct inode *inode, void *provider,
+ pgoff_t start, pgoff_t end)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+ pgoff_t index;
+ int r = 0;
+
+ for (index = start; index < end; ) {
+ pgoff_t next_boundary;
+ pgoff_t max_nr_pages;
+ struct guest_memfd_handle handle;
+
+ if (signal_pending(current)) {
+ r = -EINTR;
+ break;
+ }
+
+ r = gmem_provider_get_pfn(gi, index, &handle);
+ if (r)
+ break;
+
+ max_nr_pages = 1UL << handle.max_order;
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
+ next_boundary = ALIGN_DOWN(index + max_nr_pages, max_nr_pages);
+ index += min_t(size_t, next_boundary - index, end - index);
+ cond_resched();
+ }
+
+ return r;
+}
+
+static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
+{
+ struct address_space *mapping = inode->i_mapping;
+ struct gmem_inode *gi = GMEM_I(inode);
+ const struct guest_memfd_provider_operations *ops = gi->provider_ops;
+ pgoff_t start, end;
+ int r;
+
+ if (!ops->allocate_range)
+ return -EOPNOTSUPP;
+
+ /* Dedicated guest is immutable by default. */
+ if (offset + len > i_size_read(inode))
+ return -EINVAL;
+
+ filemap_invalidate_lock_shared(mapping);
+
+ start = offset >> PAGE_SHIFT;
+ end = (offset + len) >> PAGE_SHIFT;
+ r = ops->allocate_range(inode, gi->provider, start, end);
+
filemap_invalidate_unlock_shared(mapping);
return r;
@@ -796,6 +828,10 @@ static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset,
{
int ret;
+ /* Providers without allocation support cannot be hole punched either. */
+ if (!GMEM_I(file_inode(file))->provider_ops->allocate_range)
+ return -EOPNOTSUPP;
+
if (!(mode & FALLOC_FL_KEEP_SIZE))
return -EOPNOTSUPP;
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 09/16] KVM: guest_memfd: Use provider operations for host mappings and faults
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (7 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 08/16] KVM: guest_memfd: Use provider operations for fallocate allocation Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 10/16] KVM: guest_memfd: Let providers prepare memory regions Aneesh Kumar K.V (Arm)
` (6 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Add mmap() and fault() callbacks so providers can set up host mappings
and handle faults. Keep common permission checks, private/shared checks
and inode locking in guest_memfd. Require providers to supply both
callbacks together, or let guest_memfd handle both.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 14 ++++
virt/kvm/guest_memfd.c | 146 +++++++++++++++++++++++-------------
2 files changed, 106 insertions(+), 54 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index e11b2392b001..777507191603 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -3,6 +3,7 @@
#define _LINUX_GUEST_MEMFD_H
#include <linux/bits.h>
+#include <linux/mm_types.h>
#include <linux/types.h>
#define GUEST_MEMFD_PFN_MMIO BIT_ULL(0)
@@ -61,6 +62,14 @@ struct guest_memfd_handle {
* Supplying this without allocate_range()
* supports a fixed memory range and disables fallocate(),
* including hole punching.
+ * @mmap: Configure provider-specific VMA flags and operations after common
+ * mapping permission checks. Supply with fault(), or omit both to
+ * select folio/PFN host-mapping defaults.
+ * @fault: Resolve a shared host fault under the inode invalidate read lock.
+ * Default folio helpers transfer a locked page reference to the MM
+ * and return VM_FAULT_LOCKED; default PFN helpers insert the PFN with
+ * its memory type and permissions, then release their temporary
+ * reference.
* @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
* Must return a pointer to a struct folio on success, or an
* ERR_PTR() error code on failure. Must never return NULL.
@@ -72,6 +81,9 @@ struct guest_memfd_handle {
* guest_memfd_handle.folio for the core to unlock and release.
* PFN providers leave folio NULL and keep their PFNs stable until
* the caller releases the inode invalidate lock.
+ * Folio providers supplying get_pfn() must also supply mmap() and
+ * fault() when host mmap is enabled. The default fault expects a
+ * locked folio reference in guest_memfd_handle.folio.
* @lookup_pfn_range: Look up the PFN for a given index. Returns 0 on success
* with the physical frame number in @pfn and the number of
* contiguous pages the mapping is valid for in @nr_pages.
@@ -103,6 +115,8 @@ struct guest_memfd_provider_operations {
int (*release_range)(struct inode *inode, void *provider,
pgoff_t start, pgoff_t end, const unsigned long *pfn,
u64 pfn_attrs, bool release_backing);
+ int (*mmap)(struct inode *inode, void *provider, struct vm_area_struct *vma);
+ vm_fault_t (*fault)(struct inode *inode, void *provider, struct vm_fault *vmf);
struct folio *(*alloc_folio)(void *provider, pgoff_t index,
struct mempolicy *mpol);
void (*invalidate_folio)(void *provider, struct folio *folio);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index c6b1814b5e7e..3a92f2453e8b 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -66,6 +66,14 @@ static int gmem_folio_release_range(struct inode *inode, void *provider,
static int gmem_pfn_release_range(struct inode *inode, void *provider,
pgoff_t start, pgoff_t end, const unsigned long *pfn,
u64 pfn_attrs, bool release_backing);
+static int gmem_folio_mmap(struct inode *inode, void *provider,
+ struct vm_area_struct *vma);
+static int gmem_pfn_mmap(struct inode *inode, void *provider,
+ struct vm_area_struct *vma);
+static vm_fault_t gmem_folio_fault(struct inode *inode, void *provider,
+ struct vm_fault *vmf);
+static vm_fault_t gmem_pfn_fault(struct inode *inode, void *provider,
+ struct vm_fault *vmf);
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
{
@@ -106,6 +114,8 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.prepare_attributes = gmem_folio_prepare_attributes,
.allocate_range = gmem_folio_allocate_range,
.release_range = gmem_folio_release_range,
+ .mmap = gmem_folio_mmap,
+ .fault = gmem_folio_fault,
};
static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -131,8 +141,11 @@ static bool gmem_provider_ops_valid(struct gmem_inode *gi,
if (ops->allocate_range && !ops->release_range)
return false;
- /* Custom folio get_pfn() needs matching host mapping callbacks. */
- if (ops->alloc_folio && ops->get_pfn &&
+ if (!!ops->mmap != !!ops->fault)
+ return false;
+
+ /* The default host fault expects a locked folio in guest_memfd_handle.folio. */
+ if (ops->alloc_folio && ops->get_pfn && !ops->fault &&
(gi->flags & GUEST_MEMFD_FLAG_MMAP))
return false;
@@ -161,11 +174,19 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
gi->provider_operations.allocate_range = gmem_folio_allocate_range;
gi->provider_operations.release_range = gmem_folio_release_range;
}
+ if (!ops->mmap) {
+ gi->provider_operations.mmap = gmem_folio_mmap;
+ gi->provider_operations.fault = gmem_folio_fault;
+ }
} else {
if (!ops->release_range) {
gi->provider_operations.allocate_range = gmem_pfn_allocate_range;
gi->provider_operations.release_range = gmem_pfn_release_range;
}
+ if (!ops->mmap) {
+ gi->provider_operations.mmap = gmem_pfn_mmap;
+ gi->provider_operations.fault = gmem_pfn_fault;
+ }
}
gi->provider_ops = &gi->provider_operations;
}
@@ -958,63 +979,69 @@ static bool kvm_gmem_supports_mmap(struct inode *inode)
return GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_MMAP;
}
-static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
+static vm_fault_t gmem_fault_error(int ret)
+{
+ return ret == -EAGAIN ? VM_FAULT_RETRY : vmf_error(ret);
+}
+
+static vm_fault_t gmem_folio_fault(struct inode *inode, void *provider,
+ struct vm_fault *vmf)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
struct gmem_inode *gi = GMEM_I(inode);
- vm_fault_t ret = VM_FAULT_LOCKED;
- struct folio *folio = NULL;
struct guest_memfd_handle handle;
- int r = 0;
+ struct folio *folio;
+ int ret;
- if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
- return VM_FAULT_SIGBUS;
+ ret = gmem_provider_get_pfn(gi, vmf->pgoff, &handle);
+ if (ret)
+ return gmem_fault_error(ret);
- filemap_invalidate_lock_shared(inode->i_mapping);
- if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) {
- r = -EACCES;
- } else if (kvm_gmem_is_pfn_mode(gi)) {
- r = gmem_provider_get_pfn(gi, vmf->pgoff, &handle);
- if (!r) {
- ret = vmf_insert_pfn(vmf->vma, vmf->address, handle.pfn);
- gmem_unlock_handle(&handle);
- gmem_put_handle(&handle);
- }
- } else {
- folio = kvm_gmem_get_folio(inode, vmf->pgoff);
- if (IS_ERR(folio))
- r = PTR_ERR(folio);
+ folio = handle.folio;
+ if (WARN_ON_ONCE(folio_test_large(folio))) {
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
+ return VM_FAULT_SIGBUS;
}
- filemap_invalidate_unlock_shared(inode->i_mapping);
-
- if (r) {
- if (r == -EAGAIN)
- return VM_FAULT_RETRY;
- return vmf_error(r);
- }
+ /* Pass the locked folio reference to the MM fault handler. */
+ vmf->page = folio_file_page(folio, vmf->pgoff);
+ return VM_FAULT_LOCKED;
+}
- if (kvm_gmem_is_pfn_mode(gi))
- return ret;
+static vm_fault_t gmem_pfn_fault(struct inode *inode, void *provider,
+ struct vm_fault *vmf)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+ pgprot_t prot = vmf->vma->vm_page_prot;
+ struct guest_memfd_handle handle;
+ vm_fault_t ret;
+ int r;
- if (WARN_ON_ONCE(folio_test_large(folio))) {
- ret = VM_FAULT_SIGBUS;
- goto out_folio;
- }
+ r = gmem_provider_get_pfn(gi, vmf->pgoff, &handle);
+ if (r)
+ return gmem_fault_error(r);
- if (!folio_test_uptodate(folio)) {
- clear_highpage(folio_page(folio, 0));
- folio_mark_uptodate(folio);
- }
+ ret = vmf_insert_pfn_prot(vmf->vma, vmf->address, handle.pfn, prot);
+ gmem_unlock_handle(&handle);
+ gmem_put_handle(&handle);
+ return ret;
+}
- vmf->page = folio_file_page(folio, vmf->pgoff);
+static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
+{
+ struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct gmem_inode *gi = GMEM_I(inode);
+ vm_fault_t ret;
-out_folio:
- if (ret != VM_FAULT_LOCKED) {
- folio_unlock(folio);
- folio_put(folio);
- }
+ if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
+ return VM_FAULT_SIGBUS;
+ filemap_invalidate_lock_shared(inode->i_mapping);
+ if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff))
+ ret = VM_FAULT_SIGBUS;
+ else
+ ret = gi->provider_ops->fault(inode, gi->provider, vmf);
+ filemap_invalidate_unlock_shared(inode->i_mapping);
return ret;
}
@@ -1055,11 +1082,27 @@ static const struct vm_operations_struct kvm_gmem_vm_ops = {
#endif
};
+static int gmem_folio_mmap(struct inode *inode, void *provider,
+ struct vm_area_struct *vma)
+{
+ vma->vm_ops = &kvm_gmem_vm_ops;
+ return 0;
+}
+
+static int gmem_pfn_mmap(struct inode *inode, void *provider,
+ struct vm_area_struct *vma)
+{
+ vm_flags_set(vma, VM_PFNMAP);
+ vma->vm_ops = &kvm_gmem_vm_ops;
+ return 0;
+}
+
static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
{
- struct gmem_inode *gi = GMEM_I(file_inode(file));
+ struct inode *inode = file_inode(file);
+ struct gmem_inode *gi = GMEM_I(inode);
- if (!kvm_gmem_supports_mmap(file_inode(file)))
+ if (!kvm_gmem_supports_mmap(inode))
return -ENODEV;
if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) !=
@@ -1067,12 +1110,7 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
return -EINVAL;
}
- if (kvm_gmem_is_pfn_mode(gi))
- vm_flags_set(vma, VM_PFNMAP);
-
- vma->vm_ops = &kvm_gmem_vm_ops;
-
- return 0;
+ return gi->provider_ops->mmap(inode, gi->provider, vma);
}
bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t gfn)
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 10/16] KVM: guest_memfd: Let providers prepare memory regions
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (8 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 09/16] KVM: guest_memfd: Use provider operations for host mappings and faults Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 11/16] KVM: guest_memfd: Pass KVM explicitly to provider attach Aneesh Kumar K.V (Arm)
` (5 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Memory providers may need to check memslot flags and memory ranges before
KVM installs a memslot. Add an optional prepare_memory_region() callback to
the common provider operations
Pass the byte offset, size and requested dirty-logging and read-only
flags, and return any error before publishing the slot. Providers do not
need access to KVM's memslot structure or internal flags. Existing
providers that do not implement the callback keep their current
behavior.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: kvm@vger.kernel.org
Cc: patches@lists.linux.dev
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 8 ++++++++
virt/kvm/guest_memfd.c | 14 ++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 777507191603..79e2983c3b26 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -37,6 +37,12 @@ struct guest_memfd_handle {
* @attach: Attach to a resource file, perform filesystem-specific validation,
* and return an opaque provider context.
* @release: Release provider context and unpin any resources.
+ * @prepare_memory_region: Optional check of a memory range and preparation
+ * for access. Called after common file checks and
+ * before publishing the memslot. Offset and size are
+ * in bytes. The flags contain only
+ * KVM_MEM_LOG_DIRTY_PAGES and KVM_MEM_READONLY bits.
+ * Return 0 on success or a negative error code.
* @lock: Optional provider lock for mapping and attribute changes. Called
* with the inode invalidate lock held for write, before guest
* invalidation starts.
@@ -106,6 +112,8 @@ struct guest_memfd_handle {
struct guest_memfd_provider_operations {
void *(*attach)(struct inode *gmem_inode, struct file *resource_file);
void (*release)(void *provider);
+ int (*prepare_memory_region)(void *provider, u64 offset, u64 size,
+ u32 flags);
void (*lock)(void *provider);
void (*unlock)(void *provider);
int (*prepare_attributes)(struct inode *inode, void *provider, pgoff_t start,
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 3a92f2453e8b..ffecc4b19a17 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1671,9 +1671,11 @@ int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot
unsigned int fd, uoff_t offset)
{
uoff_t size = slot->npages << PAGE_SHIFT;
+ struct gmem_inode *gi;
struct gmem_file *f;
struct inode *inode;
struct file *file;
+ int ret;
BUILD_BUG_ON(sizeof(gpa_t) != sizeof(offset));
@@ -1694,10 +1696,22 @@ int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot
goto err;
inode = file_inode(file);
+ gi = GMEM_I(inode);
if (!PAGE_ALIGNED(offset) || offset + size > i_size_read(inode))
goto err;
+ if (gi->provider_ops->prepare_memory_region) {
+ u32 flags = slot->flags & (KVM_MEM_LOG_DIRTY_PAGES | KVM_MEM_READONLY);
+
+ ret = gi->provider_ops->prepare_memory_region(gi->provider, offset,
+ size, flags);
+ if (ret) {
+ fput(file);
+ return ret;
+ }
+ }
+
/*
* memslots of flag KVM_MEM_GUEST_MEMFD are immutable to change, so
* kvm_gmem_bind() must occur on a new memslot. Because the memslot
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 11/16] KVM: guest_memfd: Pass KVM explicitly to provider attach
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (9 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 10/16] KVM: guest_memfd: Let providers prepare memory regions Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 12/16] KVM: guest_memfd: Support devices through common provider operations Aneesh Kumar K.V (Arm)
` (4 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Pass the attaching VM to the resource-provider attach callbacks. The
argument is valid during attachment and must not be retained. This
avoids storing the VM in the device binding context introduced next.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 3 ++-
mm/shmem.c | 2 +-
samples/kvm/mem_provider_sample.c | 2 +-
virt/kvm/guest_memfd.c | 9 +++++----
4 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 79e2983c3b26..012174ad8ef8 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -12,6 +12,7 @@ struct file;
struct folio;
struct inode;
struct mempolicy;
+struct kvm;
/**
* struct guest_memfd_handle - PFN information and an optional folio reference
@@ -110,7 +111,7 @@ struct guest_memfd_handle {
* Used by filesystems and device drivers that provide memory for guest_memfd.
*/
struct guest_memfd_provider_operations {
- void *(*attach)(struct inode *gmem_inode, struct file *resource_file);
+ void *(*attach)(struct inode *gmem_inode, struct file *resource_file, struct kvm *kvm);
void (*release)(void *provider);
int (*prepare_memory_region)(void *provider, u64 offset, u64 size,
u32 flags);
diff --git a/mm/shmem.c b/mm/shmem.c
index a1c1cc92515b..f652a6c1009f 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -5462,7 +5462,7 @@ static const struct inode_operations shmem_special_inode_operations = {
#ifdef CONFIG_KVM_GUEST_MEMFD
static void *shmem_gmem_attach(struct inode *gmem_inode,
- struct file *resource_file)
+ struct file *resource_file, struct kvm *kvm)
{
struct inode *inode = file_inode(resource_file);
struct shmem_sb_info *sbinfo;
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
index 4b7a8f760fdf..2eb9748cec5b 100644
--- a/samples/kvm/mem_provider_sample.c
+++ b/samples/kvm/mem_provider_sample.c
@@ -152,7 +152,7 @@ static long mps_file_ioctl(struct file *file, unsigned int cmd,
}
static void *mps_gmem_attach(struct inode *gmem_inode,
- struct file *resource_file)
+ struct file *resource_file, struct kvm *kvm)
{
struct mps_info *info = resource_file->private_data;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ffecc4b19a17..178ac5ad123f 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -86,7 +86,7 @@ static bool kvm_gmem_is_pfn_mode(struct gmem_inode *gi)
}
static void *gmem_native_attach(struct inode *gmem_inode,
- struct file *resource_file)
+ struct file *resource_file, struct kvm *kvm)
{
return GMEM_I(gmem_inode);
}
@@ -193,7 +193,8 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
#define GMEM_RESOURCE_FD_NATIVE (-1)
-static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
+static int kvm_gmem_attach_resource(struct kvm *kvm,
+ struct inode *inode, int resource_fd)
{
struct gmem_inode *gi = GMEM_I(inode);
const struct guest_memfd_provider_operations *ops = NULL;
@@ -212,7 +213,7 @@ static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
}
if (gmem_provider_ops_valid(gi, ops))
- provider = ops->attach(inode, resource_file);
+ provider = ops->attach(inode, resource_file, kvm);
if (resource_file)
fput(resource_file);
@@ -1619,7 +1620,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int resour
*/
smp_wmb();
- err = kvm_gmem_attach_resource(inode, resource_fd);
+ err = kvm_gmem_attach_resource(kvm, inode, resource_fd);
if (err) {
fput(file);
put_unused_fd(fd);
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 12/16] KVM: guest_memfd: Support devices through common provider operations
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (10 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 11/16] KVM: guest_memfd: Pass KVM explicitly to provider attach Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 13/16] KVM: guest_memfd: Notify providers after private attribute updates Aneesh Kumar K.V (Arm)
` (3 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Allow device MMIO to use the PFN provider interface on architectures
that support protected device mappings. Pass device information during
attachment. Invalidate both shared and private device mappings during
revocation and file cleanup.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
Documentation/virt/kvm/api.rst | 6 ++-
include/linux/guest_memfd.h | 34 ++++++++++++++--
include/linux/kvm_host.h | 5 +++
mm/shmem.c | 3 +-
samples/kvm/mem_provider_sample.c | 3 +-
virt/kvm/guest_memfd.c | 68 +++++++++++++++++++++++++++----
6 files changed, 104 insertions(+), 15 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index bb550e4f11cd..1ccd166a4cff 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6478,8 +6478,10 @@ that refers to it. guest_memfd files are roughly analogous to files created
via memfd_create(), e.g. guest_memfd files live in RAM, have volatile storage,
and are automatically released when the last reference is dropped. Unlike
"regular" memfd_create() files, guest_memfd files are bound to their owning
-virtual machine (see below), cannot be mapped, read, or written by userspace,
-and cannot be resized (guest_memfd files do however support PUNCH_HOLE).
+virtual machine (see below), cannot be read or written by userspace, and
+cannot be resized. Userspace mappings depend on the creation flags and memory
+resource. Ordinary guest_memfd files support PUNCH_HOLE; files using device
+memory reject fallocate().
::
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 012174ad8ef8..ebfb3199a302 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -14,6 +14,26 @@ struct inode;
struct mempolicy;
struct kvm;
+/* Binding data stays valid only while the provider is active. */
+struct guest_memfd_device_binding {
+ /* Architecture-specific mapping owner, e.g. an RMM vDEVICE handle. */
+ unsigned long owner;
+ u64 vdev_id;
+};
+
+/**
+ * struct guest_memfd_device - Device memory context supplied during attach()
+ * @size: File size in bytes
+ * @flags: GUEST_MEMFD_FLAG_* creation flags
+ * @binding: Mapping owner and device ID set by attach(), with the underlying
+ * resources pinned until release()
+ */
+struct guest_memfd_device {
+ u64 size;
+ u64 flags;
+ struct guest_memfd_device_binding binding;
+};
+
/**
* struct guest_memfd_handle - PFN information and an optional folio reference
* @pfn: Physical frame number
@@ -36,8 +56,14 @@ struct guest_memfd_handle {
/**
* struct guest_memfd_provider_operations - Operations for external memory providers
* @attach: Attach to a resource file, perform filesystem-specific validation,
- * and return an opaque provider context.
- * @release: Release provider context and unpin any resources.
+ * and return an opaque provider context. Pin resources and callback
+ * code until release(). Device providers receive their core context
+ * in the device argument, which is NULL for other providers.
+ * The kvm argument identifies the VM and is valid during attach();
+ * providers must not retain it.
+ * @release: Release provider context and unpin any resources. Must not access
+ * the VM here; its file reference may be gone.
+ * @device_memory: Memory is device MMIO rather than RAM.
* @prepare_memory_region: Optional check of a memory range and preparation
* for access. Called after common file checks and
* before publishing the memslot. Offset and size are
@@ -111,7 +137,8 @@ struct guest_memfd_handle {
* Used by filesystems and device drivers that provide memory for guest_memfd.
*/
struct guest_memfd_provider_operations {
- void *(*attach)(struct inode *gmem_inode, struct file *resource_file, struct kvm *kvm);
+ void *(*attach)(struct inode *gmem_inode, struct file *resource_file,
+ struct kvm *kvm, struct guest_memfd_device *device);
void (*release)(void *provider);
int (*prepare_memory_region)(void *provider, u64 offset, u64 size,
u32 flags);
@@ -135,6 +162,7 @@ struct guest_memfd_provider_operations {
unsigned long *pfn, size_t *nr_pages,
u64 *pfn_attrs);
void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
+ bool device_memory;
};
#ifdef CONFIG_KVM_GUEST_MEMFD
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 345f97b110fa..f2dd387ab0fa 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -60,6 +60,7 @@
*/
#define KVM_MEMSLOT_INVALID (1UL << 16)
#define KVM_MEMSLOT_GMEM_ONLY (1UL << 17)
+#define KVM_MEMSLOT_GMEM_DEVICE (1UL << 18)
/*
* Bit 63 of the memslot generation number is an "update in-progress flag",
@@ -2613,6 +2614,10 @@ static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn)
#endif /* kvm_arch_has_private_mem */
#ifdef CONFIG_KVM_GUEST_MEMFD
+struct guest_memfd_device_binding;
+bool kvm_arch_gmem_device_supported(struct kvm *kvm);
+void kvm_arch_gmem_device_bind(struct kvm_memory_slot *slot,
+ const struct guest_memfd_device_binding *binding);
bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t gfn);
bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm);
diff --git a/mm/shmem.c b/mm/shmem.c
index f652a6c1009f..58e3d73f348d 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -5462,7 +5462,8 @@ static const struct inode_operations shmem_special_inode_operations = {
#ifdef CONFIG_KVM_GUEST_MEMFD
static void *shmem_gmem_attach(struct inode *gmem_inode,
- struct file *resource_file, struct kvm *kvm)
+ struct file *resource_file, struct kvm *kvm,
+ struct guest_memfd_device *device)
{
struct inode *inode = file_inode(resource_file);
struct shmem_sb_info *sbinfo;
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
index 2eb9748cec5b..c9ecea72854b 100644
--- a/samples/kvm/mem_provider_sample.c
+++ b/samples/kvm/mem_provider_sample.c
@@ -152,7 +152,8 @@ static long mps_file_ioctl(struct file *file, unsigned int cmd,
}
static void *mps_gmem_attach(struct inode *gmem_inode,
- struct file *resource_file, struct kvm *kvm)
+ struct file *resource_file, struct kvm *kvm,
+ struct guest_memfd_device *device)
{
struct mps_info *info = resource_file->private_data;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 178ac5ad123f..a9609ddccafd 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -40,6 +40,8 @@ struct gmem_inode {
const struct guest_memfd_provider_operations *provider_ops;
struct guest_memfd_provider_operations provider_operations;
+ struct guest_memfd_device device;
+
u64 flags;
/*
* Every index in this inode, whether memory is populated or
@@ -80,13 +82,20 @@ static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
return container_of(inode, struct gmem_inode, vfs_inode);
}
+static bool gmem_provider_is_device(struct gmem_inode *gi)
+{
+ return gi->provider_ops && gi->provider_ops->device_memory;
+}
+
+
static bool kvm_gmem_is_pfn_mode(struct gmem_inode *gi)
{
return gi->flags & GUEST_MEMFD_FLAG_PFN_MODE;
}
static void *gmem_native_attach(struct inode *gmem_inode,
- struct file *resource_file, struct kvm *kvm)
+ struct file *resource_file, struct kvm *kvm,
+ struct guest_memfd_device *device)
{
return GMEM_I(gmem_inode);
}
@@ -118,7 +127,7 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
.fault = gmem_folio_fault,
};
-static bool gmem_provider_ops_valid(struct gmem_inode *gi,
+static bool gmem_provider_ops_valid(struct kvm *kvm, struct gmem_inode *gi,
const struct guest_memfd_provider_operations *ops)
{
if (!ops || !ops->attach)
@@ -149,6 +158,14 @@ static bool gmem_provider_ops_valid(struct gmem_inode *gi,
(gi->flags & GUEST_MEMFD_FLAG_MMAP))
return false;
+ if (ops->device_memory) {
+ if (!kvm_gmem_is_pfn_mode(gi) ||
+ !(gi->flags & GUEST_MEMFD_FLAG_INIT_SHARED) ||
+ !kvm_arch_gmem_device_supported(kvm) ||
+ !ops->release || ops->alloc_folio || ops->invalidate_folio)
+ return false;
+ }
+
if (kvm_gmem_is_pfn_mode(gi))
return ops->get_pfn && ops->lookup_pfn_range;
@@ -180,7 +197,8 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
}
} else {
if (!ops->release_range) {
- gi->provider_operations.allocate_range = gmem_pfn_allocate_range;
+ if (!ops->device_memory)
+ gi->provider_operations.allocate_range = gmem_pfn_allocate_range;
gi->provider_operations.release_range = gmem_pfn_release_range;
}
if (!ops->mmap) {
@@ -194,8 +212,9 @@ static void gmem_provider_init_ops(struct gmem_inode *gi,
#define GMEM_RESOURCE_FD_NATIVE (-1)
static int kvm_gmem_attach_resource(struct kvm *kvm,
- struct inode *inode, int resource_fd)
+ struct file *file, int resource_fd)
{
+ struct inode *inode = file_inode(file);
struct gmem_inode *gi = GMEM_I(inode);
const struct guest_memfd_provider_operations *ops = NULL;
void *provider = ERR_PTR(-EOPNOTSUPP);
@@ -212,8 +231,14 @@ static int kvm_gmem_attach_resource(struct kvm *kvm,
ops = resource_file->f_op->gmem_provider_ops;
}
- if (gmem_provider_ops_valid(gi, ops))
- provider = ops->attach(inode, resource_file, kvm);
+ if (gmem_provider_ops_valid(kvm, gi, ops)) {
+ if (ops->device_memory) {
+ gi->device.size = i_size_read(inode);
+ gi->device.flags = gi->flags;
+ }
+ provider = ops->attach(inode, resource_file, kvm,
+ ops->device_memory ? &gi->device : NULL);
+ }
if (resource_file)
fput(resource_file);
@@ -524,9 +549,15 @@ static int gmem_folio_lookup_pfn_range(struct inode *inode, void *provider,
return 0;
}
+/*
+ * Select the mapping types that should be covered by generic guest_memfd
+ * invalidation. Device memory can have both shared and private mappings. For
+ * RAM, this depends on the file's initial attributes and whether in-place
+ * conversion allows the same file to back both shared and private mappings.
+ */
static enum kvm_gfn_range_filter kvm_gmem_get_all_gfns_filter(struct inode *inode)
{
- if (gmem_in_place_conversion)
+ if (gmem_provider_is_device(GMEM_I(inode)) || gmem_in_place_conversion)
return KVM_FILTER_SHARED | KVM_FILTER_PRIVATE;
if (GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED)
@@ -1565,6 +1596,16 @@ static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags)
return r;
}
+bool __weak kvm_arch_gmem_device_supported(struct kvm *kvm)
+{
+ return false;
+}
+
+void __weak kvm_arch_gmem_device_bind(struct kvm_memory_slot *slot,
+ const struct guest_memfd_device_binding *binding)
+{
+}
+
static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int resource_fd)
{
static const char *name = "[kvm-gmem]";
@@ -1620,7 +1661,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int resour
*/
smp_wmb();
- err = kvm_gmem_attach_resource(kvm, inode, resource_fd);
+ err = kvm_gmem_attach_resource(kvm, file, resource_fd);
if (err) {
fput(file);
put_unused_fd(fd);
@@ -1713,6 +1754,16 @@ int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot
}
}
+ /*
+ * Device slots resolve both shared and private faults through guest_memfd.
+ * Identify them as device memory and pass the provider's binding to the
+ * architecture so it can associate mappings with the protected device.
+ */
+ if (gmem_provider_is_device(gi)) {
+ slot->flags |= KVM_MEMSLOT_GMEM_ONLY | KVM_MEMSLOT_GMEM_DEVICE;
+ kvm_arch_gmem_device_bind(slot, &gi->device.binding);
+ }
+
/*
* memslots of flag KVM_MEM_GUEST_MEMFD are immutable to change, so
* kvm_gmem_bind() must occur on a new memslot. Because the memslot
@@ -2014,6 +2065,7 @@ static struct inode *kvm_gmem_alloc_inode(struct super_block *sb)
gi->flags = 0;
gi->provider = NULL;
gi->provider_ops = NULL;
+ memset(&gi->device, 0, sizeof(gi->device));
INIT_LIST_HEAD(&gi->gmem_file_list);
return &gi->vfs_inode;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 13/16] KVM: guest_memfd: Notify providers after private attribute updates
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (11 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 12/16] KVM: guest_memfd: Support devices through common provider operations Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 14/16] KVM: guest_memfd: Enable shared host mappings for device providers Aneesh Kumar K.V (Arm)
` (2 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Publish whether committed private ranges remain through an optional
update_private_ranges() provider operation. Scan the attribute tree
after successful publication while holding the inode invalidate lock and
any provider lock.
Keep the cached summary in the provider that uses it. Providers
initialize their summary during attach() before exposing access checks.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 5 +++++
virt/kvm/guest_memfd.c | 25 ++++++++++++++++++++++++-
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index ebfb3199a302..31cd5142e5f1 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -76,6 +76,10 @@ struct guest_memfd_handle {
* @unlock: Release the provider lock taken by lock(). Called after guest
* invalidation ends and any successful attribute publication, with
* the inode invalidate lock still held.
+ * @update_private_ranges: Optional notification of whether committed private
+ * ranges remain. Called after attribute publication,
+ * and with false after final file cleanup, under the
+ * inode invalidate lock and any provider lock.
* @prepare_attributes: Optional memory validation before conversion, under
* the inode invalidate lock for write, before taking
* the provider lock or starting guest invalidation. Host mappings have
@@ -144,6 +148,7 @@ struct guest_memfd_provider_operations {
u32 flags);
void (*lock)(void *provider);
void (*unlock)(void *provider);
+ void (*update_private_ranges)(void *provider, bool has_private_ranges);
int (*prepare_attributes)(struct inode *inode, void *provider, pgoff_t start,
size_t nr_pages, u64 memory_attrs, pgoff_t *err_index);
int (*allocate_range)(struct inode *inode, void *provider,
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index a9609ddccafd..1c90f59f480e 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1317,6 +1317,27 @@ static int gmem_provider_make_shared(struct inode *inode, pgoff_t start,
return 0;
}
+static void gmem_provider_update_attributes(struct inode *inode)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+ void *entry;
+ bool private = false;
+
+ MA_STATE(mas, &gi->attributes, 0, 0);
+
+ if (!gi->provider_ops->update_private_ranges)
+ return;
+
+ mas_for_each(&mas, entry, ULONG_MAX) {
+ if (kvm_gmem_get_attributes(&gi->vfs_inode, entry) &
+ KVM_MEMORY_ATTRIBUTE_PRIVATE) {
+ private = true;
+ break;
+ }
+ }
+ gi->provider_ops->update_private_ranges(gi->provider, private);
+}
+
static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
size_t nr_pages, u64 attrs, pgoff_t *err_index)
{
@@ -1369,8 +1390,10 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
if (!to_private)
r = gmem_provider_make_shared(inode, start, end);
- if (!r)
+ if (!r) {
mas_store_prealloc(&mas, xa_mk_value(attrs));
+ gmem_provider_update_attributes(inode);
+ }
if (r)
mas_destroy(&mas);
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 14/16] KVM: guest_memfd: Enable shared host mappings for device providers
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (12 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 13/16] KVM: guest_memfd: Notify providers after private attribute updates Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 15/16] KVM: guest_memfd: Notify providers during file release Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 16/16] KVM: guest_memfd: Invalidate mappings before memslot unbind Aneesh Kumar K.V (Arm)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Use the PFN provider mmap and fault handlers for shared device memory.
Apply device VMA flags and noncached, decrypted protections, retaining
the inode invalidate lock across PFN lookup and PTE insertion.
Private ranges remain inaccessible to host faults. Attribute conversion
revokes host mappings before making memory private.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/guest_memfd.h | 4 +++-
virt/kvm/guest_memfd.c | 18 ++++++++++++++++--
2 files changed, 19 insertions(+), 3 deletions(-)
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 31cd5142e5f1..1fcce1240bb6 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -63,7 +63,9 @@ struct guest_memfd_handle {
* providers must not retain it.
* @release: Release provider context and unpin any resources. Must not access
* the VM here; its file reference may be gone.
- * @device_memory: Memory is device MMIO rather than RAM.
+ * @device_memory: Memory is device MMIO rather than RAM. The core maps its
+ * userspace VMAs noncached and decrypted, with device flags.
+ * Shared userspace faults are resolved through @get_pfn
* @prepare_memory_region: Optional check of a memory range and preparation
* for access. Called after common file checks and
* before publishing the memslot. Offset and size are
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 1c90f59f480e..d3ae01aa6fe6 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1053,6 +1053,8 @@ static vm_fault_t gmem_pfn_fault(struct inode *inode, void *provider,
if (r)
return gmem_fault_error(r);
+ if (gmem_provider_is_device(gi))
+ prot = pgprot_decrypted(pgprot_noncached(prot));
ret = vmf_insert_pfn_prot(vmf->vma, vmf->address, handle.pfn, prot);
gmem_unlock_handle(&handle);
gmem_put_handle(&handle);
@@ -1114,6 +1116,11 @@ static const struct vm_operations_struct kvm_gmem_vm_ops = {
#endif
};
+/* Device PFNs have fixed addresses and do not use NUMA allocation policy. */
+static const struct vm_operations_struct kvm_gmem_device_vm_ops = {
+ .fault = kvm_gmem_fault_user_mapping,
+};
+
static int gmem_folio_mmap(struct inode *inode, void *provider,
struct vm_area_struct *vma)
{
@@ -1124,8 +1131,15 @@ static int gmem_folio_mmap(struct inode *inode, void *provider,
static int gmem_pfn_mmap(struct inode *inode, void *provider,
struct vm_area_struct *vma)
{
- vm_flags_set(vma, VM_PFNMAP);
- vma->vm_ops = &kvm_gmem_vm_ops;
+ struct gmem_inode *gi = GMEM_I(inode);
+
+ /* Keep device memory protections in the VMA for mprotect(). */
+ if (gmem_provider_is_device(gi))
+ vma->vm_page_prot = pgprot_decrypted(pgprot_noncached(vma->vm_page_prot));
+
+ vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
+ vma->vm_ops = gmem_provider_is_device(gi) ? &kvm_gmem_device_vm_ops :
+ &kvm_gmem_vm_ops;
return 0;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 15/16] KVM: guest_memfd: Notify providers during file release
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (13 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 14/16] KVM: guest_memfd: Enable shared host mappings for device providers Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
2026-11-12 7:41 ` [RFC PATCH v5 16/16] KVM: guest_memfd: Invalidate mappings before memslot unbind Aneesh Kumar K.V (Arm)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
When releasing the last file associated with an inode, invalidate its
mappings and notify the provider that no private ranges remain. Hold the
provider lock across both operations so device unlock waits for cleanup.
Skip notification if provider attachment failed.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
virt/kvm/guest_memfd.c | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index d3ae01aa6fe6..bdee42ff6b26 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -87,7 +87,6 @@ static bool gmem_provider_is_device(struct gmem_inode *gi)
return gi->provider_ops && gi->provider_ops->device_memory;
}
-
static bool kvm_gmem_is_pfn_mode(struct gmem_inode *gi)
{
return gi->flags & GUEST_MEMFD_FLAG_PFN_MODE;
@@ -904,12 +903,28 @@ static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset,
return ret;
}
+/* No file users remain, so private attributes cannot create new mappings. */
+static void gmem_provider_release_views(struct inode *inode)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+
+ /* File release also runs after a failed provider attachment. */
+ if (!gi->provider_ops)
+ return;
+
+ if (gi->provider_ops->update_private_ranges &&
+ list_is_singular(&gi->gmem_file_list))
+ gi->provider_ops->update_private_ranges(gi->provider, false);
+}
+
static int kvm_gmem_release(struct inode *inode, struct file *file)
{
struct gmem_file *f = file->private_data;
+ struct gmem_inode *gi = GMEM_I(inode);
struct kvm_memory_slot *slot;
struct kvm *kvm = f->kvm;
unsigned long index;
+ enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
/*
* Prevent concurrent attempts to *unbind* a memslot. This is the last
@@ -922,6 +937,8 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
filemap_invalidate_lock(inode->i_mapping);
+ gmem_provider_lock_updates(gi);
+
/*
* Note! synchronize_srcu() is _not_ needed after nullifying memslot
* bindings as slot->gmem.file cannot be set back to a non-null value
@@ -975,9 +992,10 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
* Zap all SPTEs pointed at by this file. Do not free the backing
* memory, as its lifetime is associated with the inode, not the file.
*/
- __kvm_gmem_invalidate_start(f, 0, -1ul,
- kvm_gmem_get_all_gfns_filter(inode));
+ __kvm_gmem_invalidate_start(f, 0, -1ul, filter);
__kvm_gmem_invalidate_end(f, 0, -1ul);
+ gmem_provider_release_views(inode);
+ gmem_provider_unlock_updates(gi);
list_del(&f->entry);
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [RFC PATCH v5 16/16] KVM: guest_memfd: Invalidate mappings before memslot unbind
2026-11-12 7:41 [RFC PATCH v5 00/16] KVM: guest_memfd: Add a common framework for device memory providers Aneesh Kumar K.V (Arm)
` (14 preceding siblings ...)
2026-11-12 7:41 ` [RFC PATCH v5 15/16] KVM: guest_memfd: Notify providers during file release Aneesh Kumar K.V (Arm)
@ 2026-11-12 7:41 ` Aneesh Kumar K.V (Arm)
15 siblings, 0 replies; 17+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-12 7:41 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Ackerley Tng, David Woodhouse,
David Hildenbrand, Jason Gunthorpe, Kevin Tian, Paolo Bonzini,
Sean Christopherson, Alexey Kardashevskiy, Xu Yilun,
Catalin Marinas, Suzuki K Poulose, Steven Price, Fred Griffoul,
iommu, kvm, patches
Use the generic invalidate_start/end helpers to remove shared and
private device mappings before removing the memslot's guest_memfd
binding.
Hold gmem_provider_lock_updates() across invalidation and binding removal to
prevent device unlock from racing with teardown.
Keep the file attributes and has_private_ranges unchanged. Deleting a
memslot removes mappings but does not share the memory range. Device unlock
remains blocked until userspace shares the private ranges or closes the
last guest_memfd reference.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: kvm@vger.kernel.org
Cc: patches@lists.linux.dev
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
virt/kvm/guest_memfd.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index bdee42ff6b26..93e2ccfc4540 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1877,6 +1877,8 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
unsigned long start = slot->gmem.pgoff;
unsigned long end = start + slot->npages;
struct gmem_file *f;
+ struct gmem_inode *gi;
+ enum kvm_gfn_range_filter filter;
/*
* Nothing to do if the underlying file was _already_ closed, as
@@ -1896,8 +1898,13 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
* drops slots_lock, so there's no need to verify the file is live.
*/
f = file->private_data;
+ gi = GMEM_I(file_inode(file));
+ filter = kvm_gmem_get_all_gfns_filter(file_inode(file));
filemap_invalidate_lock(file->f_mapping);
+ gmem_provider_lock_updates(gi);
+ __kvm_gmem_invalidate_start(f, start, end, filter);
+ __kvm_gmem_invalidate_end(f, start, end);
xa_store_range(&f->bindings, start, end - 1, NULL, GFP_KERNEL);
/*
@@ -1906,6 +1913,7 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
* to guarantee kvm_gmem_get_pfn() can't see the slot+file.
*/
WRITE_ONCE(slot->gmem.file, NULL);
+ gmem_provider_unlock_updates(gi);
filemap_invalidate_unlock(file->f_mapping);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread