[RFC PATCH 0/6] KVM: guest_memfd: back guest_memfd with an imported dma-buf

[RFC PATCH 0/6] KVM: guest_memfd: back guest_memfd with an imported dma-buf

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

This extends David Woodhouse's "KVM: Allow alternative providers of
guest_memfd backed by PFNMAP memory" RFC (v2) to let guest_memfd import
a dma-buf as its page source:

  https://lore.kernel.org/kvm/20260720111259.122911-1-dwmw2@infradead.org/

The memory owner is a plain dma-buf exporter, linking against neither
KVM nor iommufd. guest_memfd imports it for the guest (stage-2) plane
and iommufd imports the same buffer for the device (DMA) plane, so one
object and one invalidation protocol serve both. This is a concrete
answer to the "how do we plug into iommufd without masquerading as a
dma-buf" question from David's cover: the provider does not pretend to
be anything; the dma-buf is the shared object.

Applies on top of David's v2 series.

Patches:

 1 KVM: guest_memfd: Add a writable result to get_pfn()
   Give get_pfn() a writable out-param so a backing can map a page
   present but read-only (KVM_MEM_READONLY is rejected on guest_memfd
   slots). A guest write then exits as a memory fault. Stage-2 only;
   KVM's own host-mapped accesses are unaffected.

 2 dma-buf: Add get_phys() to describe a physical run
   New exporter op: for an offset+len, return one physically contiguous
   backed run and an attribute word (RAM/MMIO, plus READONLY). -ENOENT
   for an unbacked byte, -ENODEV when revoked. Lets a page-table-building
   importer ask where the memory is instead of taking a scatterlist.
   Converts vfio-pci, the iommufd selftest exporter and the sample;
   iommufd behaviour is unchanged.

 3 dma-buf: Add ranged mapping invalidation
   Invalidate a byte range rather than the whole buffer. The callback
   means "re-query this range" and covers loss, replacement and
   read-only flips. Importers without the callback still get a
   whole-buffer invalidation.

 4 dma-buf: Allow dynamic attach without a device
   Allow a dynamic importer with no struct device, for a consumer (KVM)
   that only needs the physical layout and builds its own tables. Such
   an importer must not map the attachment for DMA.

 5 KVM: guest_memfd: Add dma-buf backing
   Add GUEST_MEMFD_FLAG_USE_DMABUF and a dmabuf_fd to
   KVM_CREATE_GUEST_MEMFD. guest_memfd attaches as a revocable importer
   without a device, serves faults from get_phys() (largest aligned RAM
   run, read-only honoured), and zaps the range on the invalidation
   callback.

 6 samples/kvm, selftests/kvm: Exercise dma-buf backing
   Turn the sample into a dma-buf exporter (a toy memory owner: one root
   region, a child per VM, move/donate/reclaim, absent and read-only
   pages, a scratch page, per-child ioctl allowlists) and move the
   selftests to feed one dma-buf fd to both guest_memfd and iommufd.

Scope: non-confidential VMs, RAM backings only; iommufd keeps its
single-range import for now. As with David's series, this is largely
AI-assisted and is posted to get the interface shape reviewed.

Fred Griffoul (6):
  KVM: guest_memfd: Add a writable result to get_pfn()
  dma-buf: Add get_phys() to describe a physical run
  dma-buf: Add ranged mapping invalidation
  dma-buf: Allow dynamic attach without a device
  KVM: guest_memfd: Add dma-buf backing
  samples/kvm, selftests/kvm: Exercise dma-buf backing

 arch/arm64/kvm/mmu.c                          |   13 +-
 arch/arm64/kvm/nested.c                       |   10 +-
 arch/x86/kvm/mmu/mmu.c                        |   18 +-
 arch/x86/kvm/svm/sev.c                        |    4 +-
 drivers/dma-buf/dma-buf.c                     |   92 +-
 drivers/iommu/iommufd/iommufd_private.h       |    8 -
 drivers/iommu/iommufd/iommufd_test.h          |   16 +
 drivers/iommu/iommufd/pages.c                 |   74 +-
 drivers/iommu/iommufd/selftest.c              |  106 +-
 drivers/vfio/pci/vfio_pci_dmabuf.c            |   58 +-
 include/linux/dma-buf.h                       |   70 +
 include/linux/kvm_host.h                      |   12 +-
 include/linux/vfio_pci_core.h                 |    3 -
 include/trace/events/dma_buf.h                |    2 +-
 include/uapi/linux/kvm.h                      |    6 +-
 samples/kvm/gmem_provider.c                   | 1311 ++++++++++-------
 samples/kvm/gmem_provider.h                   |  144 +-
 tools/include/uapi/linux/kvm.h                |    6 +-
 tools/testing/selftests/kvm/Makefile.kvm      |    3 +-
 .../kvm/gmem_provider_nvme_dma_test.c         |   28 +-
 .../testing/selftests/kvm/include/kvm_util.h  |   25 +-
 .../testing/selftests/kvm/x86/gmem_poc_test.c |  824 +++++++++++
 .../kvm/x86/gmem_provider_hugepage_test.c     |   30 +-
 .../kvm/x86/gmem_provider_iommufd_test.c      |   42 +-
 .../kvm/x86/gmem_provider_readonly_test.c     |  172 +++
 .../kvm/x86/gmem_provider_revoke_test.c       |   34 +-
 .../selftests/kvm/x86/gmem_provider_test.c    |  190 +--
 .../kvm/x86/gmem_provider_vfio_test.c         |   45 +-
 virt/kvm/Kconfig                              |    1 +
 virt/kvm/guest_memfd.c                        |  252 +++-
 30 files changed, 2703 insertions(+), 896 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_poc_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c

--
2.47.3

[RFC PATCH 1/6] KVM: guest_memfd: Add a writable result to get_pfn()

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A guest_memfd backing has no way to tell KVM that a page must stay
mapped but must not be written by the guest. get_pfn() returns only a
frame, and KVM takes write access from the memslot flags.
KVM_MEM_READONLY is not an option: KVM refuses it on guest_memfd slots,
because it emulates a write to a read-only slot as MMIO, which private
memory cannot support.

Add a writable output to get_pfn(). KVM sets it to true before the
call, and the backing may clear it. When it is false, the x86 and arm64
stage-2 fault paths map the page without write permission. A guest
write to that page then exits to userspace as a memory fault; KVM first
releases any page that the backing returned. Callers that do not map
the page pass NULL.

The result applies to stage-2 mappings only. When KVM writes guest
memory through the memslot's host address, the permissions of the host
mapping apply.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 arch/arm64/kvm/mmu.c        | 13 ++++++++-----
 arch/arm64/kvm/nested.c     | 10 +++++++---
 arch/x86/kvm/mmu/mmu.c      | 18 ++++++++++++++++--
 arch/x86/kvm/svm/sev.c      |  4 ++--
 include/linux/kvm_host.h    | 11 ++++++++---
 samples/kvm/gmem_provider.c |  3 ++-
 virt/kvm/guest_memfd.c      | 13 ++++++++-----
 7 files changed, 51 insertions(+), 21 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6c941aaa10c6..32e591edc69d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1607,7 +1607,7 @@ struct kvm_s2_fault_desc {
 
 static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
 {
-	bool write_fault, exec_fault;
+	bool write_fault, exec_fault, writable;
 	bool perm_fault = kvm_vcpu_trap_is_permission_fault(s2fd->vcpu);
 	enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
 	enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
@@ -1641,14 +1641,17 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
 	/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
 	smp_rmb();
 
-	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL);
-	if (ret) {
+	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL,
+			       &writable);
+	if (ret || (write_fault && !writable)) {
 		kvm_prepare_memory_fault_exit(s2fd->vcpu, s2fd->fault_ipa, PAGE_SIZE,
 					      write_fault, exec_fault, false);
-		return ret;
+		if (!ret)
+			kvm_release_faultin_page(kvm, page, true, false);
+		return ret ?: -EFAULT;
 	}
 
-	if (!(s2fd->memslot->flags & KVM_MEM_READONLY))
+	if (!(s2fd->memslot->flags & KVM_MEM_READONLY) && writable)
 		prot |= KVM_PGTABLE_PROT_W;
 
 	if (s2fd->nested)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index fb54f6dad995..9ad1fa029835 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1411,12 +1411,16 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 		if (is_error_noslot_pfn(pfn) || (write_fault && !writable))
 			return -EFAULT;
 	} else {
-		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, &page, NULL);
-		if (ret) {
+		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, &page, NULL,
+				       &writable);
+		if (ret || (write_fault && !writable)) {
 			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
 					      write_fault, false, false);
-			return ret;
+			if (!ret)
+				kvm_release_faultin_page(vcpu->kvm, page, true, false);
+			return ret ?: -EFAULT;
 		}
+		vt->wr.pw &= writable;
 	}
 
 	scoped_guard(write_lock, &vcpu->kvm->mmu_lock) {
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 234d0a95abf5..17ac2bfdc164 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4612,6 +4612,7 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu,
 static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 				    struct kvm_page_fault *fault)
 {
+	bool writable;
 	int max_order, r;
 
 	if (!kvm_slot_has_gmem(fault->slot)) {
@@ -4620,13 +4621,26 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 	}
 
 	r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn,
-			     &fault->refcounted_page, &max_order);
+			     &fault->refcounted_page, &max_order, &writable);
 	if (r) {
 		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
 		return r;
 	}
 
-	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY);
+	/*
+	 * The memory's owner has the final say on writability, on top of the
+	 * memslot flag: a page it reports read-only is mapped read-only, and a
+	 * guest write to it exits to userspace rather than being installed.
+	 */
+	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY) &&
+			      writable;
+	if (fault->write && !fault->map_writable) {
+		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
+		kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page,
+				 true, false);
+		fault->refcounted_page = NULL;
+		return -EFAULT;
+	}
 	fault->max_level = kvm_max_level_for_order(max_order);
 
 	return RET_PF_CONTINUE;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 125779c82bc4..983e19f7dbd2 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4063,7 +4063,7 @@ static void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu)
 	 * The new VMSA will be private memory guest memory, so retrieve the
 	 * PFN from the gmem backend.
 	 */
-	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL))
+	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL, NULL))
 		return;
 
 	/*
@@ -4996,7 +4996,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 		return;
 	}
 
-	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order);
+	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order, NULL);
 	if (ret) {
 		pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n",
 				    gpa);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 04fa0cb126f6..f84f3ab44acf 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -660,9 +660,14 @@ struct kvm_gmem_ops {
 		    struct kvm_memory_slot *slot, loff_t offset);
 	void (*unbind)(struct file *file, struct kvm *kvm,
 		       struct kvm_memory_slot *slot);
+	/*
+	 * @writable: [out] clear to have KVM map the page read-only; a guest
+	 * write then exits as a memory fault.  NULL if the caller does not care.
+	 */
 	int (*get_pfn)(struct file *file, struct kvm *kvm,
 		       struct kvm_memory_slot *slot, gfn_t gfn,
-		       kvm_pfn_t *pfn, struct page **page, int *max_order);
+		       kvm_pfn_t *pfn, struct page **page, int *max_order,
+		       bool *writable);
 	int (*populate)(struct file *file, struct kvm *kvm,
 			struct kvm_memory_slot *slot, gfn_t gfn,
 			kvm_pfn_t *pfn, struct page *src_page, int order);
@@ -2651,12 +2656,12 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
 #ifdef CONFIG_KVM_GUEST_MEMFD
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		     gfn_t gfn, kvm_pfn_t *pfn, struct page **page,
-		     int *max_order);
+		     int *max_order, bool *writable);
 #else
 static inline int kvm_gmem_get_pfn(struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
 				   kvm_pfn_t *pfn, struct page **page,
-				   int *max_order)
+				   int *max_order, bool *writable)
 {
 	KVM_BUG_ON(1, kvm);
 	return -EIO;
diff --git a/samples/kvm/gmem_provider.c b/samples/kvm/gmem_provider.c
index 9728f5a8029b..75197c088762 100644
--- a/samples/kvm/gmem_provider.c
+++ b/samples/kvm/gmem_provider.c
@@ -157,7 +157,8 @@ static int gmem_max_order(struct gmem_info *info, gfn_t gfn, unsigned long index
 
 static int gmem_get_pfn(struct file *file, struct kvm *kvm,
 			struct kvm_memory_slot *slot, gfn_t gfn,
-			kvm_pfn_t *pfn, struct page **page, int *max_order)
+			kvm_pfn_t *pfn, struct page **page, int *max_order,
+			bool *writable)
 {
 	struct gmem_info *info = to_gmem_info(file);
 	pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index d284bb70fe05..0f4bf2cc5e8e 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -624,7 +624,7 @@ static void kvm_gmem_native_unbind(struct file *slot_file, struct kvm *kvm,
 static int  kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 				    struct kvm_memory_slot *slot, gfn_t gfn,
 				    kvm_pfn_t *pfn, struct page **page,
-				    int *max_order);
+				    int *max_order, bool *writable);
 static void kvm_gmem_native_release(struct file *file);
 static int  kvm_gmem_native_mmap(struct file *file,
 				 struct vm_area_struct *vma);
@@ -988,7 +988,7 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
 static int kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
 				   kvm_pfn_t *pfn, struct page **page,
-				   int *max_order)
+				   int *max_order, bool *writable)
 {
 	pgoff_t index = kvm_gmem_get_index(slot, gfn);
 	struct folio *folio;
@@ -1016,7 +1016,7 @@ static int kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		     gfn_t gfn, kvm_pfn_t *pfn, struct page **page,
-		     int *max_order)
+		     int *max_order, bool *writable)
 {
 	const struct kvm_gmem_ops *ops;
 
@@ -1029,7 +1029,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		return -EFAULT;
 
 	*page = NULL;
-	return ops->get_pfn(file, kvm, slot, gfn, pfn, page, max_order);
+	if (writable)
+		*writable = true;
+	return ops->get_pfn(file, kvm, slot, gfn, pfn, page, max_order,
+			    writable);
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
 
@@ -1093,7 +1096,7 @@ static int kvm_gmem_populate_one(const struct kvm_gmem_ops *ops,
 				    src_page, 0);
 	else
 		ret = ops->get_pfn(file, kvm, slot, gfn, &pfn,
-				   &ignored_page, NULL);
+				   &ignored_page, NULL, NULL);
 	if (ret)
 		return ret;
 
-- 
2.47.3

[RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

iommufd and KVM write physical addresses into their own page tables.
To do so, they must ask the exporter which frames back an offset of a
dma-buf, whether that memory is RAM or MMIO, and whether it may be
written.

Add a get_phys() operation. The importer passes an offset and a maximum
length. The exporter reports one run: the frames that start at the
offset, are backed and physically contiguous, and share one attribute
word. The run never exceeds the length. The exporter may end it early,
so importers must not assume that it is the longest possible run.

get_phys() returns -ENOENT when the byte at the offset is not backed,
and -ENODEV when the buffer is revoked. The caller holds the
reservation, and either pins the attachment or handles revocation. A
reported frame stays valid until an invalidation that covers it
returns.

The attribute word holds the memory type and a READONLY flag. Zero
means writable RAM. Importers refuse unknown types, reserved bits and
unknown flags, so attributes added later fail safely. Two flag bits are
reserved: one for holes and one for confidential memory.

Convert vfio-pci, the iommufd selftest exporter and the KVM sample.
iommufd behaves as before: it maps a buffer only when one writable run
covers all of it.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/dma-buf/dma-buf.c               |  47 +++++++++++
 drivers/iommu/iommufd/iommufd_private.h |   8 --
 drivers/iommu/iommufd/iommufd_test.h    |  16 ++++
 drivers/iommu/iommufd/pages.c           |  74 ++++-------------
 drivers/iommu/iommufd/selftest.c        | 106 ++++++++++++++++++++----
 drivers/vfio/pci/vfio_pci_dmabuf.c      |  58 ++++++-------
 include/linux/dma-buf.h                 |  54 ++++++++++++
 include/linux/vfio_pci_core.h           |   3 -
 samples/kvm/gmem_provider.c             |  39 ++++-----
 9 files changed, 267 insertions(+), 138 deletions(-)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index d504c636dc29..66b85d53ed22 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -1389,6 +1389,53 @@ void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings, "DMA_BUF");
 
+/**
+ * dma_buf_get_phys - describe the run that starts at an offset
+ * @attach: attachment to query
+ * @offset: first buffer byte to describe
+ * @len: maximum number of bytes to describe
+ * @phys: physical address and length of the run
+ * @attr: DMA_BUF_PHYS_ATTR_* word of the run
+ *
+ * A run is the longest stretch of backed bytes starting at @offset whose
+ * frames are physically contiguous and share one attribute word. On success
+ * *@phys starts at the byte at @offset and covers at most @len bytes; it may
+ * be shorter than the run.
+ *
+ * The dma-buf reservation must be held. The attachment must be pinned or have
+ * revocable importer operations. A frame remains valid until a covering
+ * invalidation callback returns; the exporter must invalidate a changed range
+ * before reusing its old frames.
+ *
+ * Returns:
+ *
+ * 0 on success, -ENOENT if the byte at @offset is not backed, -ENODEV if the
+ * buffer is revoked, -EOPNOTSUPP if the exporter cannot describe itself this
+ * way, or another negative error code.
+ */
+int dma_buf_get_phys(struct dma_buf_attachment *attach, u64 offset, u64 len,
+		     struct phys_vec *phys, u32 *attr)
+{
+	u64 end;
+	int ret;
+
+	if (WARN_ON_ONCE(!attach || !attach->dmabuf || !phys || !attr))
+		return -EINVAL;
+	if (!len || check_add_overflow(offset, len, &end) ||
+	    end > attach->dmabuf->size)
+		return -EINVAL;
+
+	dma_resv_assert_held(attach->dmabuf->resv);
+	if (!attach->dmabuf->ops->get_phys)
+		return -EOPNOTSUPP;
+
+	ret = attach->dmabuf->ops->get_phys(attach, offset, len, phys, attr);
+	if (!ret && WARN_ON_ONCE(!phys->len || phys->len > len))
+		return -EIO;
+	return ret;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_get_phys, "DMA_BUF");
+
 /**
  * DOC: cpu access
  *
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 43fbc5bed8de..5cded585c227 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -716,8 +716,6 @@ bool iommufd_should_fail(void);
 int __init iommufd_test_init(void);
 void iommufd_test_exit(void);
 bool iommufd_selftest_is_mock_dev(struct device *dev);
-int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				     struct phys_vec *phys);
 #else
 static inline void iommufd_test_syz_conv_iova_id(struct iommufd_ucmd *ucmd,
 						 unsigned int ioas_id,
@@ -739,11 +737,5 @@ static inline bool iommufd_selftest_is_mock_dev(struct device *dev)
 {
 	return false;
 }
-static inline int
-iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				 struct phys_vec *phys)
-{
-	return -EOPNOTSUPP;
-}
 #endif
 #endif
diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h
index 52b78cbcc920..28fd9c43edc4 100644
--- a/drivers/iommu/iommufd/iommufd_test.h
+++ b/drivers/iommu/iommufd/iommufd_test.h
@@ -31,6 +31,8 @@ enum {
 	IOMMU_TEST_OP_PASID_CHECK_HWPT,
 	IOMMU_TEST_OP_DMABUF_GET,
 	IOMMU_TEST_OP_DMABUF_REVOKE,
+	IOMMU_TEST_OP_MD_CHECK_MAPPED,
+	IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
 };
 
 enum {
@@ -193,6 +195,20 @@ struct iommu_test_cmd {
 			__s32 dmabuf_fd;
 			__u32 revoked;
 		} dmabuf_revoke;
+		struct {
+			/*
+			 * 1: every page in [iova, iova+length) must be mapped;
+			 * 0: none of them may be. Mixed is an error.
+			 */
+			__u32 mapped;
+			__u32 __reserved;
+			__aligned_u64 iova;
+			__aligned_u64 length;
+		} check_mapped;
+		struct {
+			__aligned_u64 iova;
+			__aligned_u64 out_phys;	/* 0 if unmapped */
+		} iova_to_phys;
 	};
 	__u32 last;
 };
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index f9b2ae6d7e96..196d1bb330c2 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1463,68 +1463,12 @@ static const struct dma_buf_attach_ops iopt_dmabuf_attach_revoke_ops = {
 	.invalidate_mappings = iopt_revoke_notify,
 };
 
-/*
- * iommufd and vfio have a circular dependency. Future work for a phys
- * based private interconnect will remove this.
- */
-/*
- * Look up the exporter's phys accessor for iommufd's private-interconnect
- * path.  Also fills *is_cpu_ram: true if the exporter's memory is normal
- * cache-coherent RAM (needs BATCH_CPU_MEMORY / IOMMU_CACHE), false for MMIO
- * (needs BATCH_MMIO / IOMMU_MMIO).  This will be replaced by a formal
- * exporter op that returns phys + memory type together.
- */
-static int
-sym_vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				 struct phys_vec *phys, bool *is_cpu_ram)
-{
-	typeof(&vfio_pci_dma_buf_iommufd_map) fn;
-	int rc;
-
-	rc = iommufd_test_dma_buf_iommufd_map(attachment, phys);
-	if (rc != -EOPNOTSUPP) {
-		*is_cpu_ram = false;	/* test hook mimics VFIO MMIO */
-		return rc;
-	}
-
-	/*
-	 * Prototype: try the sample gmem provider's dma-buf exporter.  This
-	 * mirrors the vfio-pci private-interconnect hook, and (like it) is
-	 * meant to be replaced by a formal negotiated exporter op returning
-	 * phys + memory type.  The provider serves RAM, so mark it CPU_RAM.
-	 */
-	{
-		extern int gmem_provider_dma_buf_iommufd_map(
-			struct dma_buf_attachment *, struct phys_vec *);
-		typeof(&gmem_provider_dma_buf_iommufd_map) gfn;
-
-		gfn = symbol_get(gmem_provider_dma_buf_iommufd_map);
-		if (gfn) {
-			rc = gfn(attachment, phys);
-			symbol_put(gmem_provider_dma_buf_iommufd_map);
-			if (rc != -EOPNOTSUPP) {
-				*is_cpu_ram = true;
-				return rc;
-			}
-		}
-	}
-
-	if (!IS_ENABLED(CONFIG_VFIO_PCI_DMABUF))
-		return -EOPNOTSUPP;
-
-	fn = symbol_get(vfio_pci_dma_buf_iommufd_map);
-	if (!fn)
-		return -EOPNOTSUPP;
-	rc = fn(attachment, phys);
-	symbol_put(vfio_pci_dma_buf_iommufd_map);
-	*is_cpu_ram = false;	/* VFIO PCI dma-buf carries BAR (MMIO) memory */
-	return rc;
-}
-
 static int iopt_map_dmabuf(struct iommufd_ctx *ictx, struct iopt_pages *pages,
 			   struct dma_buf *dmabuf)
 {
 	struct dma_buf_attachment *attach;
+	struct phys_vec pv;
+	u32 attr;
 	int rc;
 
 	attach = dma_buf_dynamic_attach(dmabuf, iommufd_global_device(),
@@ -1546,10 +1490,20 @@ static int iopt_map_dmabuf(struct iommufd_ctx *ictx, struct iopt_pages *pages,
 	if (rc)
 		goto err_detach;
 
-	rc = sym_vfio_pci_dma_buf_iommufd_map(attach, &pages->dmabuf.phys,
-					      &pages->dmabuf.is_cpu_ram);
+	/* One backed, writable run covering the buffer: refuse the rest. */
+	rc = dma_buf_get_phys(attach, 0, dmabuf->size, &pv, &attr);
+	if (rc == -ENOENT)
+		rc = -EOPNOTSUPP;
 	if (rc)
 		goto err_unpin;
+	if (pv.len != dmabuf->size || !dma_buf_phys_attr_known(attr) ||
+	    (attr & DMA_BUF_PHYS_ATTR_FLAGS_MASK)) {
+		rc = -EOPNOTSUPP;
+		goto err_unpin;
+	}
+	pages->dmabuf.phys = pv;
+	pages->dmabuf.is_cpu_ram =
+		dma_buf_phys_attr_type(attr) == DMA_BUF_PHYS_ATTR_RAM;
 
 	dma_resv_unlock(dmabuf->resv);
 
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index af07c642a526..0899272d1e66 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -1962,32 +1962,31 @@ static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
 	kfree(priv);
 }
 
-static const struct dma_buf_ops iommufd_test_dmabuf_ops = {
-	.attach = iommufd_test_dma_buf_attach,
-	.detach = iommufd_test_dma_buf_detach,
-	.map_dma_buf = iommufd_test_dma_buf_map,
-	.release = iommufd_test_dma_buf_release,
-	.unmap_dma_buf = iommufd_test_dma_buf_unmap,
-};
-
-int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				     struct phys_vec *phys)
+static int iommufd_test_dma_buf_get_phys(struct dma_buf_attachment *attachment,
+					 u64 offset, u64 len,
+					 struct phys_vec *phys, u32 *attr)
 {
 	struct iommufd_test_dma_buf *priv = attachment->dmabuf->priv;
 
 	dma_resv_assert_held(attachment->dmabuf->resv);
-
-	if (attachment->dmabuf->ops != &iommufd_test_dmabuf_ops)
-		return -EOPNOTSUPP;
-
 	if (priv->revoked)
 		return -ENODEV;
 
-	phys->paddr = virt_to_phys(priv->memory);
-	phys->len = priv->length;
+	phys->paddr = virt_to_phys(priv->memory) + offset;
+	phys->len = len;
+	*attr = DMA_BUF_PHYS_ATTR_MMIO;
 	return 0;
 }
 
+static const struct dma_buf_ops iommufd_test_dmabuf_ops = {
+	.attach = iommufd_test_dma_buf_attach,
+	.detach = iommufd_test_dma_buf_detach,
+	.map_dma_buf = iommufd_test_dma_buf_map,
+	.release = iommufd_test_dma_buf_release,
+	.unmap_dma_buf = iommufd_test_dma_buf_unmap,
+	.get_phys = iommufd_test_dma_buf_get_phys,
+};
+
 static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
 				   unsigned int open_flags,
 				   size_t len)
@@ -2031,6 +2030,73 @@ static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
 	return rc;
 }
 
+/*
+ * Report the physical address the mock domain resolves @iova to, or 0 if
+ * it is unmapped.  Lets a test check that two IOVAs share one frame (a
+ * scratch substitution) without knowing the frame in advance.
+ */
+static int iommufd_test_md_iova_to_phys(struct iommufd_ucmd *ucmd,
+					unsigned int mockpt_id,
+					unsigned long iova)
+{
+	struct iommu_test_cmd *cmd = ucmd->cmd;
+	struct iommufd_hw_pagetable *hwpt;
+	struct mock_iommu_domain *mock;
+	unsigned int page_size;
+	int rc;
+
+	hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
+	if (IS_ERR(hwpt))
+		return PTR_ERR(hwpt);
+
+	page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
+	if (iova % page_size) {
+		rc = -EINVAL;
+		goto out_put;
+	}
+	cmd->iova_to_phys.out_phys =
+		mock->domain.ops->iova_to_phys(&mock->domain, iova);
+	rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
+out_put:
+	iommufd_put_object(ucmd->ictx, &hwpt->obj);
+	return rc;
+}
+
+static int iommufd_test_md_check_mapped(struct iommufd_ucmd *ucmd,
+					unsigned int mockpt_id,
+					unsigned long iova, size_t length,
+					bool mapped)
+{
+	struct iommufd_hw_pagetable *hwpt;
+	struct mock_iommu_domain *mock;
+	unsigned int page_size;
+	int rc = 0;
+
+	hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
+	if (IS_ERR(hwpt))
+		return PTR_ERR(hwpt);
+
+	page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
+	if (iova % page_size || length % page_size || !length) {
+		rc = -EINVAL;
+		goto out_put;
+	}
+
+	for (; length; length -= page_size, iova += page_size) {
+		bool is_mapped =
+			mock->domain.ops->iova_to_phys(&mock->domain, iova) != 0;
+
+		if (is_mapped != mapped) {
+			rc = -ENOENT;
+			goto out_put;
+		}
+	}
+
+out_put:
+	iommufd_put_object(ucmd->ictx, &hwpt->obj);
+	return rc;
+}
+
 static int iommufd_test_dmabuf_revoke(struct iommufd_ucmd *ucmd, int fd,
 				      bool revoked)
 {
@@ -2143,6 +2209,14 @@ int iommufd_test(struct iommufd_ucmd *ucmd)
 		return iommufd_test_dmabuf_revoke(ucmd,
 						  cmd->dmabuf_revoke.dmabuf_fd,
 						  cmd->dmabuf_revoke.revoked);
+	case IOMMU_TEST_OP_MD_CHECK_MAPPED:
+		return iommufd_test_md_check_mapped(ucmd, cmd->id,
+						    cmd->check_mapped.iova,
+						    cmd->check_mapped.length,
+						    cmd->check_mapped.mapped);
+	case IOMMU_TEST_OP_MD_IOVA_TO_PHYS:
+		return iommufd_test_md_iova_to_phys(ucmd, cmd->id,
+						    cmd->iova_to_phys.iova);
 	default:
 		return -EOPNOTSUPP;
 	}
diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
index c16f460c01d6..381c3d338e8c 100644
--- a/drivers/vfio/pci/vfio_pci_dmabuf.c
+++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
@@ -99,46 +99,46 @@ static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf)
 	kfree(priv);
 }
 
-static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
-	.attach = vfio_pci_dma_buf_attach,
-	.map_dma_buf = vfio_pci_dma_buf_map,
-	.unmap_dma_buf = vfio_pci_dma_buf_unmap,
-	.release = vfio_pci_dma_buf_release,
-};
-
 /*
- * This is a temporary "private interconnect" between VFIO DMABUF and iommufd.
- * It allows the two co-operating drivers to exchange the physical address of
- * the BAR. This is to be replaced with a formal DMABUF system for negotiated
- * interconnect types.
+ * Report the BAR's physical range for importers which program it into their own
+ * translation tables, such as iommufd. A BAR is MMIO, never cache-coherent RAM.
  *
- * If this function succeeds the following are true:
- *  - There is one physical range and it is pointing to MMIO
- *  - When move_notify is called it means revoke, not move, vfio_dma_buf_map
- *    will fail if it is currently revoked
+ * When move_notify is called it means revoke, not move, so this fails while
+ * revoked and vfio_dma_buf_map() does the same.
  */
-int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				 struct phys_vec *phys)
+static int vfio_pci_dma_buf_get_phys(struct dma_buf_attachment *attachment,
+				     u64 offset, u64 len,
+				     struct phys_vec *phys, u32 *attr)
 {
-	struct vfio_pci_dma_buf *priv;
+	struct vfio_pci_dma_buf *priv = attachment->dmabuf->priv;
+	u32 i;
 
 	dma_resv_assert_held(attachment->dmabuf->resv);
-
-	if (attachment->dmabuf->ops != &vfio_pci_dmabuf_ops)
-		return -EOPNOTSUPP;
-
-	priv = attachment->dmabuf->priv;
 	if (priv->revoked)
 		return -ENODEV;
 
-	/* More than one range to iommufd will require proper DMABUF support */
-	if (priv->nr_ranges != 1)
-		return -EOPNOTSUPP;
-
-	*phys = priv->phys_vec[0];
+	/* Report from @offset to the end of the BAR range containing it. */
+	for (i = 0; i < priv->nr_ranges; i++) {
+		if (offset < priv->phys_vec[i].len)
+			break;
+		offset -= priv->phys_vec[i].len;
+	}
+	if (i == priv->nr_ranges)
+		return -EINVAL;
+	phys->paddr = priv->phys_vec[i].paddr + offset;
+	phys->len = min_t(u64, priv->phys_vec[i].len - offset, len);
+	*attr = DMA_BUF_PHYS_ATTR_MMIO;
 	return 0;
 }
-EXPORT_SYMBOL_FOR_MODULES(vfio_pci_dma_buf_iommufd_map, "iommufd");
+
+static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
+	.attach = vfio_pci_dma_buf_attach,
+	.map_dma_buf = vfio_pci_dma_buf_map,
+	.unmap_dma_buf = vfio_pci_dma_buf_unmap,
+	.release = vfio_pci_dma_buf_release,
+	.get_phys = vfio_pci_dma_buf_get_phys,
+};
+
 
 int vfio_pci_core_fill_phys_vec(struct phys_vec *phys_vec,
 				struct vfio_region_dma_range *dma_ranges,
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index d1203da56fc5..b223962e20c2 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -13,6 +13,7 @@
 #ifndef __DMA_BUF_H__
 #define __DMA_BUF_H__
 
+#include <linux/bitfield.h>
 #include <linux/iosys-map.h>
 #include <linux/file.h>
 #include <linux/err.h>
@@ -23,6 +24,7 @@
 #include <linux/dma-fence.h>
 #include <linux/wait.h>
 #include <linux/pci-p2pdma.h>
+#include <linux/types.h>
 
 struct device;
 struct dma_buf;
@@ -182,6 +184,29 @@ struct dma_buf_ops {
 			      struct sg_table *,
 			      enum dma_data_direction);
 
+	/**
+	 * @get_phys:
+	 *
+	 * Describe the run that starts at @offset, for an importer that
+	 * programs its own translation tables. A run is the longest stretch
+	 * of backed bytes whose frames are physically contiguous and share
+	 * one attribute word. Report it in *@phys, starting at the byte at
+	 * @offset and clipped at @offset + @len, with its DMA_BUF_PHYS_ATTR_*
+	 * word in *@attr. The exporter may stop before the end of the run;
+	 * importers must not assume the reported run is maximal.
+	 *
+	 * Return 0 on success, -ENOENT if the byte at @offset is not backed,
+	 * -ENODEV if the buffer is revoked, or another negative error. Do not
+	 * wait for memory to become available.
+	 *
+	 * The dma-buf reservation is held. The attachment must be pinned or
+	 * have revocable importer operations. A reported frame remains valid
+	 * until a covering invalidation callback returns; an exporter must
+	 * invalidate every change before reusing an old frame.
+	 */
+	int (*get_phys)(struct dma_buf_attachment *attach, u64 offset, u64 len,
+			struct phys_vec *phys, u32 *attr);
+
 	/* TODO: Add try_map_dma_buf version, to return immed with -EBUSY
 	 * if the call would block.
 	 */
@@ -576,6 +601,35 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
 				enum dma_data_direction);
 void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
 bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
+/* bits 0-7: memory type (a value, not flags) */
+#define DMA_BUF_PHYS_ATTR_TYPE_MASK	GENMASK(7, 0)
+#define DMA_BUF_PHYS_ATTR_RAM		0x00	/* cache-coherent system RAM */
+#define DMA_BUF_PHYS_ATTR_MMIO		0x01	/* device MMIO, uncached */
+/* bits 8-15: reserved for a second value field; must be zero */
+#define DMA_BUF_PHYS_ATTR_RSVD_MASK	GENMASK(15, 8)
+/* bits 16-31: flags; undefined bits must be zero */
+#define DMA_BUF_PHYS_ATTR_READONLY	BIT(16)
+/* BIT(17): reserved (hole, for importers that walk across gaps) */
+/* BIT(18): reserved (private, for confidential computing) */
+#define DMA_BUF_PHYS_ATTR_FLAGS_MASK	(DMA_BUF_PHYS_ATTR_READONLY)
+
+static inline u32 dma_buf_phys_attr_type(u32 attrs)
+{
+	return FIELD_GET(DMA_BUF_PHYS_ATTR_TYPE_MASK, attrs);
+}
+
+static inline bool dma_buf_phys_attr_known(u32 attrs)
+{
+	return dma_buf_phys_attr_type(attrs) <= DMA_BUF_PHYS_ATTR_MMIO &&
+	       !(attrs & DMA_BUF_PHYS_ATTR_RSVD_MASK) &&
+	       !(attrs & ~(DMA_BUF_PHYS_ATTR_TYPE_MASK |
+			  DMA_BUF_PHYS_ATTR_RSVD_MASK |
+			  DMA_BUF_PHYS_ATTR_FLAGS_MASK));
+}
+
+int dma_buf_get_phys(struct dma_buf_attachment *attach, u64 offset, u64 len,
+		     struct phys_vec *phys, u32 *attr);
+
 int dma_buf_begin_cpu_access(struct dma_buf *dma_buf,
 			     enum dma_data_direction dir);
 int dma_buf_end_cpu_access(struct dma_buf *dma_buf,
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa..2a1d13abdb0c 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -257,7 +257,4 @@ vfio_pci_core_get_iomap(struct vfio_pci_core_device *vdev, unsigned int bar)
 	return vdev->barmap[bar];
 }
 
-int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
-				 struct phys_vec *phys);
-
 #endif /* VFIO_PCI_CORE_H */
diff --git a/samples/kvm/gmem_provider.c b/samples/kvm/gmem_provider.c
index 75197c088762..b6824fe5d228 100644
--- a/samples/kvm/gmem_provider.c
+++ b/samples/kvm/gmem_provider.c
@@ -487,35 +487,30 @@ static void gmem_dma_buf_release(struct dma_buf *dmabuf)
 	kfree(priv);
 }
 
-static const struct dma_buf_ops gmem_dma_buf_ops = {
-	.attach		= gmem_dma_buf_attach,
-	.map_dma_buf	= gmem_dma_buf_map,
-	.unmap_dma_buf	= gmem_dma_buf_unmap,
-	.release	= gmem_dma_buf_release,
-};
-
-/*
- * Private interconnect for iommufd (mirrors vfio_pci_dma_buf_iommufd_map).
- * Returns the single contiguous phys range for the exported region so iommufd
- * can program the IOMMU directly, bypassing the DMA API.
- */
-int gmem_provider_dma_buf_iommufd_map(struct dma_buf_attachment *attach,
-				      struct phys_vec *phys);
-int gmem_provider_dma_buf_iommufd_map(struct dma_buf_attachment *attach,
-				      struct phys_vec *phys)
+/* Report this flat sample region through the generic dma-buf operation. */
+static int gmem_dma_buf_get_phys(struct dma_buf_attachment *attach,
+				 u64 offset, u64 len,
+				 struct phys_vec *phys, u32 *attr)
 {
-	struct gmem_dmabuf *priv;
+	struct gmem_dmabuf *priv = attach->dmabuf->priv;
 
 	dma_resv_assert_held(attach->dmabuf->resv);
-	if (attach->dmabuf->ops != &gmem_dma_buf_ops)
-		return -EOPNOTSUPP;
-	priv = attach->dmabuf->priv;
 	if (priv->revoked)
 		return -ENODEV;
-	*phys = priv->phys;
+
+	phys->paddr = priv->phys.paddr + offset;
+	phys->len = len;
+	*attr = DMA_BUF_PHYS_ATTR_RAM;
 	return 0;
 }
-EXPORT_SYMBOL_FOR_MODULES(gmem_provider_dma_buf_iommufd_map, "iommufd");
+
+static const struct dma_buf_ops gmem_dma_buf_ops = {
+	.attach		= gmem_dma_buf_attach,
+	.map_dma_buf	= gmem_dma_buf_map,
+	.unmap_dma_buf	= gmem_dma_buf_unmap,
+	.release	= gmem_dma_buf_release,
+	.get_phys	= gmem_dma_buf_get_phys,
+};
 
 /* Called with info->dmabufs_lock held on the revoke path. */
 static void gmem_dma_buf_revoke_all(struct gmem_info *info)
-- 
2.47.3

Re: [RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Christian König]

On 10/5/26 11:55, Fred Griffoul wrote:
> From: Fred Griffoul <fgriffo@amazon.co.uk>
> 
> iommufd and KVM write physical addresses into their own page tables.
> To do so, they must ask the exporter which frames back an offset of a
> dma-buf, whether that memory is RAM or MMIO, and whether it may be
> written.

Well filling page tables by the importer is an absolutely clear NO-GO for the DMA-buf design, we have gotten down that path already and it took us years to remove this functionality again.

The problem you are facing here is that dma_buf_mmap() doesn't work because you don't have a VMA.

I can understand the reasoning that you don't want to have a VMA, but I don't think that this is a valid justification to add complexity to DMA-buf and especially bring an approach back which we have already deprecated.

A possible solution might be Jasons patch set to directly negotiate exposing PCI BAR regions as DMA-buf, but that certainly needs more discussion.

But the approach outlined here is an absolutely clear NAK from my side.

Regards,
Christian.

> 
> Add a get_phys() operation. The importer passes an offset and a maximum
> length. The exporter reports one run: the frames that start at the
> offset, are backed and physically contiguous, and share one attribute
> word. The run never exceeds the length. The exporter may end it early,
> so importers must not assume that it is the longest possible run.
> 
> get_phys() returns -ENOENT when the byte at the offset is not backed,
> and -ENODEV when the buffer is revoked. The caller holds the
> reservation, and either pins the attachment or handles revocation. A
> reported frame stays valid until an invalidation that covers it
> returns.
> 
> The attribute word holds the memory type and a READONLY flag. Zero
> means writable RAM. Importers refuse unknown types, reserved bits and
> unknown flags, so attributes added later fail safely. Two flag bits are
> reserved: one for holes and one for confidential memory.
> 
> Convert vfio-pci, the iommufd selftest exporter and the KVM sample.
> iommufd behaves as before: it maps a buffer only when one writable run
> covers all of it.
> 
> Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
> ---
>  drivers/dma-buf/dma-buf.c               |  47 +++++++++++
>  drivers/iommu/iommufd/iommufd_private.h |   8 --
>  drivers/iommu/iommufd/iommufd_test.h    |  16 ++++
>  drivers/iommu/iommufd/pages.c           |  74 ++++-------------
>  drivers/iommu/iommufd/selftest.c        | 106 ++++++++++++++++++++----
>  drivers/vfio/pci/vfio_pci_dmabuf.c      |  58 ++++++-------
>  include/linux/dma-buf.h                 |  54 ++++++++++++
>  include/linux/vfio_pci_core.h           |   3 -
>  samples/kvm/gmem_provider.c             |  39 ++++-----
>  9 files changed, 267 insertions(+), 138 deletions(-)
> 
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index d504c636dc29..66b85d53ed22 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -1389,6 +1389,53 @@ void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
>  }
>  EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings, "DMA_BUF");
> 
> +/**
> + * dma_buf_get_phys - describe the run that starts at an offset
> + * @attach: attachment to query
> + * @offset: first buffer byte to describe
> + * @len: maximum number of bytes to describe
> + * @phys: physical address and length of the run
> + * @attr: DMA_BUF_PHYS_ATTR_* word of the run
> + *
> + * A run is the longest stretch of backed bytes starting at @offset whose
> + * frames are physically contiguous and share one attribute word. On success
> + * *@phys starts at the byte at @offset and covers at most @len bytes; it may
> + * be shorter than the run.
> + *
> + * The dma-buf reservation must be held. The attachment must be pinned or have
> + * revocable importer operations. A frame remains valid until a covering
> + * invalidation callback returns; the exporter must invalidate a changed range
> + * before reusing its old frames.
> + *
> + * Returns:
> + *
> + * 0 on success, -ENOENT if the byte at @offset is not backed, -ENODEV if the
> + * buffer is revoked, -EOPNOTSUPP if the exporter cannot describe itself this
> + * way, or another negative error code.
> + */
> +int dma_buf_get_phys(struct dma_buf_attachment *attach, u64 offset, u64 len,
> +                    struct phys_vec *phys, u32 *attr)
> +{
> +       u64 end;
> +       int ret;
> +
> +       if (WARN_ON_ONCE(!attach || !attach->dmabuf || !phys || !attr))
> +               return -EINVAL;
> +       if (!len || check_add_overflow(offset, len, &end) ||
> +           end > attach->dmabuf->size)
> +               return -EINVAL;
> +
> +       dma_resv_assert_held(attach->dmabuf->resv);
> +       if (!attach->dmabuf->ops->get_phys)
> +               return -EOPNOTSUPP;
> +
> +       ret = attach->dmabuf->ops->get_phys(attach, offset, len, phys, attr);
> +       if (!ret && WARN_ON_ONCE(!phys->len || phys->len > len))
> +               return -EIO;
> +       return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_get_phys, "DMA_BUF");
> +
>  /**
>   * DOC: cpu access
>   *
> diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
> index 43fbc5bed8de..5cded585c227 100644
> --- a/drivers/iommu/iommufd/iommufd_private.h
> +++ b/drivers/iommu/iommufd/iommufd_private.h
> @@ -716,8 +716,6 @@ bool iommufd_should_fail(void);
>  int __init iommufd_test_init(void);
>  void iommufd_test_exit(void);
>  bool iommufd_selftest_is_mock_dev(struct device *dev);
> -int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                    struct phys_vec *phys);
>  #else
>  static inline void iommufd_test_syz_conv_iova_id(struct iommufd_ucmd *ucmd,
>                                                  unsigned int ioas_id,
> @@ -739,11 +737,5 @@ static inline bool iommufd_selftest_is_mock_dev(struct device *dev)
>  {
>         return false;
>  }
> -static inline int
> -iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                struct phys_vec *phys)
> -{
> -       return -EOPNOTSUPP;
> -}
>  #endif
>  #endif
> diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h
> index 52b78cbcc920..28fd9c43edc4 100644
> --- a/drivers/iommu/iommufd/iommufd_test.h
> +++ b/drivers/iommu/iommufd/iommufd_test.h
> @@ -31,6 +31,8 @@ enum {
>         IOMMU_TEST_OP_PASID_CHECK_HWPT,
>         IOMMU_TEST_OP_DMABUF_GET,
>         IOMMU_TEST_OP_DMABUF_REVOKE,
> +       IOMMU_TEST_OP_MD_CHECK_MAPPED,
> +       IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
>  };
> 
>  enum {
> @@ -193,6 +195,20 @@ struct iommu_test_cmd {
>                         __s32 dmabuf_fd;
>                         __u32 revoked;
>                 } dmabuf_revoke;
> +               struct {
> +                       /*
> +                        * 1: every page in [iova, iova+length) must be mapped;
> +                        * 0: none of them may be. Mixed is an error.
> +                        */
> +                       __u32 mapped;
> +                       __u32 __reserved;
> +                       __aligned_u64 iova;
> +                       __aligned_u64 length;
> +               } check_mapped;
> +               struct {
> +                       __aligned_u64 iova;
> +                       __aligned_u64 out_phys; /* 0 if unmapped */
> +               } iova_to_phys;
>         };
>         __u32 last;
>  };
> diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
> index f9b2ae6d7e96..196d1bb330c2 100644
> --- a/drivers/iommu/iommufd/pages.c
> +++ b/drivers/iommu/iommufd/pages.c
> @@ -1463,68 +1463,12 @@ static const struct dma_buf_attach_ops iopt_dmabuf_attach_revoke_ops = {
>         .invalidate_mappings = iopt_revoke_notify,
>  };
> 
> -/*
> - * iommufd and vfio have a circular dependency. Future work for a phys
> - * based private interconnect will remove this.
> - */
> -/*
> - * Look up the exporter's phys accessor for iommufd's private-interconnect
> - * path.  Also fills *is_cpu_ram: true if the exporter's memory is normal
> - * cache-coherent RAM (needs BATCH_CPU_MEMORY / IOMMU_CACHE), false for MMIO
> - * (needs BATCH_MMIO / IOMMU_MMIO).  This will be replaced by a formal
> - * exporter op that returns phys + memory type together.
> - */
> -static int
> -sym_vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                struct phys_vec *phys, bool *is_cpu_ram)
> -{
> -       typeof(&vfio_pci_dma_buf_iommufd_map) fn;
> -       int rc;
> -
> -       rc = iommufd_test_dma_buf_iommufd_map(attachment, phys);
> -       if (rc != -EOPNOTSUPP) {
> -               *is_cpu_ram = false;    /* test hook mimics VFIO MMIO */
> -               return rc;
> -       }
> -
> -       /*
> -        * Prototype: try the sample gmem provider's dma-buf exporter.  This
> -        * mirrors the vfio-pci private-interconnect hook, and (like it) is
> -        * meant to be replaced by a formal negotiated exporter op returning
> -        * phys + memory type.  The provider serves RAM, so mark it CPU_RAM.
> -        */
> -       {
> -               extern int gmem_provider_dma_buf_iommufd_map(
> -                       struct dma_buf_attachment *, struct phys_vec *);
> -               typeof(&gmem_provider_dma_buf_iommufd_map) gfn;
> -
> -               gfn = symbol_get(gmem_provider_dma_buf_iommufd_map);
> -               if (gfn) {
> -                       rc = gfn(attachment, phys);
> -                       symbol_put(gmem_provider_dma_buf_iommufd_map);
> -                       if (rc != -EOPNOTSUPP) {
> -                               *is_cpu_ram = true;
> -                               return rc;
> -                       }
> -               }
> -       }
> -
> -       if (!IS_ENABLED(CONFIG_VFIO_PCI_DMABUF))
> -               return -EOPNOTSUPP;
> -
> -       fn = symbol_get(vfio_pci_dma_buf_iommufd_map);
> -       if (!fn)
> -               return -EOPNOTSUPP;
> -       rc = fn(attachment, phys);
> -       symbol_put(vfio_pci_dma_buf_iommufd_map);
> -       *is_cpu_ram = false;    /* VFIO PCI dma-buf carries BAR (MMIO) memory */
> -       return rc;
> -}
> -
>  static int iopt_map_dmabuf(struct iommufd_ctx *ictx, struct iopt_pages *pages,
>                            struct dma_buf *dmabuf)
>  {
>         struct dma_buf_attachment *attach;
> +       struct phys_vec pv;
> +       u32 attr;
>         int rc;
> 
>         attach = dma_buf_dynamic_attach(dmabuf, iommufd_global_device(),
> @@ -1546,10 +1490,20 @@ static int iopt_map_dmabuf(struct iommufd_ctx *ictx, struct iopt_pages *pages,
>         if (rc)
>                 goto err_detach;
> 
> -       rc = sym_vfio_pci_dma_buf_iommufd_map(attach, &pages->dmabuf.phys,
> -                                             &pages->dmabuf.is_cpu_ram);
> +       /* One backed, writable run covering the buffer: refuse the rest. */
> +       rc = dma_buf_get_phys(attach, 0, dmabuf->size, &pv, &attr);
> +       if (rc == -ENOENT)
> +               rc = -EOPNOTSUPP;
>         if (rc)
>                 goto err_unpin;
> +       if (pv.len != dmabuf->size || !dma_buf_phys_attr_known(attr) ||
> +           (attr & DMA_BUF_PHYS_ATTR_FLAGS_MASK)) {
> +               rc = -EOPNOTSUPP;
> +               goto err_unpin;
> +       }
> +       pages->dmabuf.phys = pv;
> +       pages->dmabuf.is_cpu_ram =
> +               dma_buf_phys_attr_type(attr) == DMA_BUF_PHYS_ATTR_RAM;
> 
>         dma_resv_unlock(dmabuf->resv);
> 
> diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
> index af07c642a526..0899272d1e66 100644
> --- a/drivers/iommu/iommufd/selftest.c
> +++ b/drivers/iommu/iommufd/selftest.c
> @@ -1962,32 +1962,31 @@ static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
>         kfree(priv);
>  }
> 
> -static const struct dma_buf_ops iommufd_test_dmabuf_ops = {
> -       .attach = iommufd_test_dma_buf_attach,
> -       .detach = iommufd_test_dma_buf_detach,
> -       .map_dma_buf = iommufd_test_dma_buf_map,
> -       .release = iommufd_test_dma_buf_release,
> -       .unmap_dma_buf = iommufd_test_dma_buf_unmap,
> -};
> -
> -int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                    struct phys_vec *phys)
> +static int iommufd_test_dma_buf_get_phys(struct dma_buf_attachment *attachment,
> +                                        u64 offset, u64 len,
> +                                        struct phys_vec *phys, u32 *attr)
>  {
>         struct iommufd_test_dma_buf *priv = attachment->dmabuf->priv;
> 
>         dma_resv_assert_held(attachment->dmabuf->resv);
> -
> -       if (attachment->dmabuf->ops != &iommufd_test_dmabuf_ops)
> -               return -EOPNOTSUPP;
> -
>         if (priv->revoked)
>                 return -ENODEV;
> 
> -       phys->paddr = virt_to_phys(priv->memory);
> -       phys->len = priv->length;
> +       phys->paddr = virt_to_phys(priv->memory) + offset;
> +       phys->len = len;
> +       *attr = DMA_BUF_PHYS_ATTR_MMIO;
>         return 0;
>  }
> 
> +static const struct dma_buf_ops iommufd_test_dmabuf_ops = {
> +       .attach = iommufd_test_dma_buf_attach,
> +       .detach = iommufd_test_dma_buf_detach,
> +       .map_dma_buf = iommufd_test_dma_buf_map,
> +       .release = iommufd_test_dma_buf_release,
> +       .unmap_dma_buf = iommufd_test_dma_buf_unmap,
> +       .get_phys = iommufd_test_dma_buf_get_phys,
> +};
> +
>  static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
>                                    unsigned int open_flags,
>                                    size_t len)
> @@ -2031,6 +2030,73 @@ static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
>         return rc;
>  }
> 
> +/*
> + * Report the physical address the mock domain resolves @iova to, or 0 if
> + * it is unmapped.  Lets a test check that two IOVAs share one frame (a
> + * scratch substitution) without knowing the frame in advance.
> + */
> +static int iommufd_test_md_iova_to_phys(struct iommufd_ucmd *ucmd,
> +                                       unsigned int mockpt_id,
> +                                       unsigned long iova)
> +{
> +       struct iommu_test_cmd *cmd = ucmd->cmd;
> +       struct iommufd_hw_pagetable *hwpt;
> +       struct mock_iommu_domain *mock;
> +       unsigned int page_size;
> +       int rc;
> +
> +       hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
> +       if (IS_ERR(hwpt))
> +               return PTR_ERR(hwpt);
> +
> +       page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
> +       if (iova % page_size) {
> +               rc = -EINVAL;
> +               goto out_put;
> +       }
> +       cmd->iova_to_phys.out_phys =
> +               mock->domain.ops->iova_to_phys(&mock->domain, iova);
> +       rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
> +out_put:
> +       iommufd_put_object(ucmd->ictx, &hwpt->obj);
> +       return rc;
> +}
> +
> +static int iommufd_test_md_check_mapped(struct iommufd_ucmd *ucmd,
> +                                       unsigned int mockpt_id,
> +                                       unsigned long iova, size_t length,
> +                                       bool mapped)
> +{
> +       struct iommufd_hw_pagetable *hwpt;
> +       struct mock_iommu_domain *mock;
> +       unsigned int page_size;
> +       int rc = 0;
> +
> +       hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
> +       if (IS_ERR(hwpt))
> +               return PTR_ERR(hwpt);
> +
> +       page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
> +       if (iova % page_size || length % page_size || !length) {
> +               rc = -EINVAL;
> +               goto out_put;
> +       }
> +
> +       for (; length; length -= page_size, iova += page_size) {
> +               bool is_mapped =
> +                       mock->domain.ops->iova_to_phys(&mock->domain, iova) != 0;
> +
> +               if (is_mapped != mapped) {
> +                       rc = -ENOENT;
> +                       goto out_put;
> +               }
> +       }
> +
> +out_put:
> +       iommufd_put_object(ucmd->ictx, &hwpt->obj);
> +       return rc;
> +}
> +
>  static int iommufd_test_dmabuf_revoke(struct iommufd_ucmd *ucmd, int fd,
>                                       bool revoked)
>  {
> @@ -2143,6 +2209,14 @@ int iommufd_test(struct iommufd_ucmd *ucmd)
>                 return iommufd_test_dmabuf_revoke(ucmd,
>                                                   cmd->dmabuf_revoke.dmabuf_fd,
>                                                   cmd->dmabuf_revoke.revoked);
> +       case IOMMU_TEST_OP_MD_CHECK_MAPPED:
> +               return iommufd_test_md_check_mapped(ucmd, cmd->id,
> +                                                   cmd->check_mapped.iova,
> +                                                   cmd->check_mapped.length,
> +                                                   cmd->check_mapped.mapped);
> +       case IOMMU_TEST_OP_MD_IOVA_TO_PHYS:
> +               return iommufd_test_md_iova_to_phys(ucmd, cmd->id,
> +                                                   cmd->iova_to_phys.iova);
>         default:
>                 return -EOPNOTSUPP;
>         }
> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
> index c16f460c01d6..381c3d338e8c 100644
> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
> @@ -99,46 +99,46 @@ static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf)
>         kfree(priv);
>  }
> 
> -static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
> -       .attach = vfio_pci_dma_buf_attach,
> -       .map_dma_buf = vfio_pci_dma_buf_map,
> -       .unmap_dma_buf = vfio_pci_dma_buf_unmap,
> -       .release = vfio_pci_dma_buf_release,
> -};
> -
>  /*
> - * This is a temporary "private interconnect" between VFIO DMABUF and iommufd.
> - * It allows the two co-operating drivers to exchange the physical address of
> - * the BAR. This is to be replaced with a formal DMABUF system for negotiated
> - * interconnect types.
> + * Report the BAR's physical range for importers which program it into their own
> + * translation tables, such as iommufd. A BAR is MMIO, never cache-coherent RAM.
>   *
> - * If this function succeeds the following are true:
> - *  - There is one physical range and it is pointing to MMIO
> - *  - When move_notify is called it means revoke, not move, vfio_dma_buf_map
> - *    will fail if it is currently revoked
> + * When move_notify is called it means revoke, not move, so this fails while
> + * revoked and vfio_dma_buf_map() does the same.
>   */
> -int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                struct phys_vec *phys)
> +static int vfio_pci_dma_buf_get_phys(struct dma_buf_attachment *attachment,
> +                                    u64 offset, u64 len,
> +                                    struct phys_vec *phys, u32 *attr)
>  {
> -       struct vfio_pci_dma_buf *priv;
> +       struct vfio_pci_dma_buf *priv = attachment->dmabuf->priv;
> +       u32 i;
> 
>         dma_resv_assert_held(attachment->dmabuf->resv);
> -
> -       if (attachment->dmabuf->ops != &vfio_pci_dmabuf_ops)
> -               return -EOPNOTSUPP;
> -
> -       priv = attachment->dmabuf->priv;
>         if (priv->revoked)
>                 return -ENODEV;
> 
> -       /* More than one range to iommufd will require proper DMABUF support */
> -       if (priv->nr_ranges != 1)
> -               return -EOPNOTSUPP;
> -
> -       *phys = priv->phys_vec[0];
> +       /* Report from @offset to the end of the BAR range containing it. */
> +       for (i = 0; i < priv->nr_ranges; i++) {
> +               if (offset < priv->phys_vec[i].len)
> +                       break;
> +               offset -= priv->phys_vec[i].len;
> +       }
> +       if (i == priv->nr_ranges)
> +               return -EINVAL;
> +       phys->paddr = priv->phys_vec[i].paddr + offset;
> +       phys->len = min_t(u64, priv->phys_vec[i].len - offset, len);
> +       *attr = DMA_BUF_PHYS_ATTR_MMIO;
>         return 0;
>  }
> -EXPORT_SYMBOL_FOR_MODULES(vfio_pci_dma_buf_iommufd_map, "iommufd");
> +
> +static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
> +       .attach = vfio_pci_dma_buf_attach,
> +       .map_dma_buf = vfio_pci_dma_buf_map,
> +       .unmap_dma_buf = vfio_pci_dma_buf_unmap,
> +       .release = vfio_pci_dma_buf_release,
> +       .get_phys = vfio_pci_dma_buf_get_phys,
> +};
> +
> 
>  int vfio_pci_core_fill_phys_vec(struct phys_vec *phys_vec,
>                                 struct vfio_region_dma_range *dma_ranges,
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index d1203da56fc5..b223962e20c2 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -13,6 +13,7 @@
>  #ifndef __DMA_BUF_H__
>  #define __DMA_BUF_H__
> 
> +#include <linux/bitfield.h>
>  #include <linux/iosys-map.h>
>  #include <linux/file.h>
>  #include <linux/err.h>
> @@ -23,6 +24,7 @@
>  #include <linux/dma-fence.h>
>  #include <linux/wait.h>
>  #include <linux/pci-p2pdma.h>
> +#include <linux/types.h>
> 
>  struct device;
>  struct dma_buf;
> @@ -182,6 +184,29 @@ struct dma_buf_ops {
>                               struct sg_table *,
>                               enum dma_data_direction);
> 
> +       /**
> +        * @get_phys:
> +        *
> +        * Describe the run that starts at @offset, for an importer that
> +        * programs its own translation tables. A run is the longest stretch
> +        * of backed bytes whose frames are physically contiguous and share
> +        * one attribute word. Report it in *@phys, starting at the byte at
> +        * @offset and clipped at @offset + @len, with its DMA_BUF_PHYS_ATTR_*
> +        * word in *@attr. The exporter may stop before the end of the run;
> +        * importers must not assume the reported run is maximal.
> +        *
> +        * Return 0 on success, -ENOENT if the byte at @offset is not backed,
> +        * -ENODEV if the buffer is revoked, or another negative error. Do not
> +        * wait for memory to become available.
> +        *
> +        * The dma-buf reservation is held. The attachment must be pinned or
> +        * have revocable importer operations. A reported frame remains valid
> +        * until a covering invalidation callback returns; an exporter must
> +        * invalidate every change before reusing an old frame.
> +        */
> +       int (*get_phys)(struct dma_buf_attachment *attach, u64 offset, u64 len,
> +                       struct phys_vec *phys, u32 *attr);
> +
>         /* TODO: Add try_map_dma_buf version, to return immed with -EBUSY
>          * if the call would block.
>          */
> @@ -576,6 +601,35 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
>                                 enum dma_data_direction);
>  void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
>  bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
> +/* bits 0-7: memory type (a value, not flags) */
> +#define DMA_BUF_PHYS_ATTR_TYPE_MASK    GENMASK(7, 0)
> +#define DMA_BUF_PHYS_ATTR_RAM          0x00    /* cache-coherent system RAM */
> +#define DMA_BUF_PHYS_ATTR_MMIO         0x01    /* device MMIO, uncached */
> +/* bits 8-15: reserved for a second value field; must be zero */
> +#define DMA_BUF_PHYS_ATTR_RSVD_MASK    GENMASK(15, 8)
> +/* bits 16-31: flags; undefined bits must be zero */
> +#define DMA_BUF_PHYS_ATTR_READONLY     BIT(16)
> +/* BIT(17): reserved (hole, for importers that walk across gaps) */
> +/* BIT(18): reserved (private, for confidential computing) */
> +#define DMA_BUF_PHYS_ATTR_FLAGS_MASK   (DMA_BUF_PHYS_ATTR_READONLY)
> +
> +static inline u32 dma_buf_phys_attr_type(u32 attrs)
> +{
> +       return FIELD_GET(DMA_BUF_PHYS_ATTR_TYPE_MASK, attrs);
> +}
> +
> +static inline bool dma_buf_phys_attr_known(u32 attrs)
> +{
> +       return dma_buf_phys_attr_type(attrs) <= DMA_BUF_PHYS_ATTR_MMIO &&
> +              !(attrs & DMA_BUF_PHYS_ATTR_RSVD_MASK) &&
> +              !(attrs & ~(DMA_BUF_PHYS_ATTR_TYPE_MASK |
> +                         DMA_BUF_PHYS_ATTR_RSVD_MASK |
> +                         DMA_BUF_PHYS_ATTR_FLAGS_MASK));
> +}
> +
> +int dma_buf_get_phys(struct dma_buf_attachment *attach, u64 offset, u64 len,
> +                    struct phys_vec *phys, u32 *attr);
> +
>  int dma_buf_begin_cpu_access(struct dma_buf *dma_buf,
>                              enum dma_data_direction dir);
>  int dma_buf_end_cpu_access(struct dma_buf *dma_buf,
> diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
> index 9a1674c152aa..2a1d13abdb0c 100644
> --- a/include/linux/vfio_pci_core.h
> +++ b/include/linux/vfio_pci_core.h
> @@ -257,7 +257,4 @@ vfio_pci_core_get_iomap(struct vfio_pci_core_device *vdev, unsigned int bar)
>         return vdev->barmap[bar];
>  }
> 
> -int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
> -                                struct phys_vec *phys);
> -
>  #endif /* VFIO_PCI_CORE_H */
> diff --git a/samples/kvm/gmem_provider.c b/samples/kvm/gmem_provider.c
> index 75197c088762..b6824fe5d228 100644
> --- a/samples/kvm/gmem_provider.c
> +++ b/samples/kvm/gmem_provider.c
> @@ -487,35 +487,30 @@ static void gmem_dma_buf_release(struct dma_buf *dmabuf)
>         kfree(priv);
>  }
> 
> -static const struct dma_buf_ops gmem_dma_buf_ops = {
> -       .attach         = gmem_dma_buf_attach,
> -       .map_dma_buf    = gmem_dma_buf_map,
> -       .unmap_dma_buf  = gmem_dma_buf_unmap,
> -       .release        = gmem_dma_buf_release,
> -};
> -
> -/*
> - * Private interconnect for iommufd (mirrors vfio_pci_dma_buf_iommufd_map).
> - * Returns the single contiguous phys range for the exported region so iommufd
> - * can program the IOMMU directly, bypassing the DMA API.
> - */
> -int gmem_provider_dma_buf_iommufd_map(struct dma_buf_attachment *attach,
> -                                     struct phys_vec *phys);
> -int gmem_provider_dma_buf_iommufd_map(struct dma_buf_attachment *attach,
> -                                     struct phys_vec *phys)
> +/* Report this flat sample region through the generic dma-buf operation. */
> +static int gmem_dma_buf_get_phys(struct dma_buf_attachment *attach,
> +                                u64 offset, u64 len,
> +                                struct phys_vec *phys, u32 *attr)
>  {
> -       struct gmem_dmabuf *priv;
> +       struct gmem_dmabuf *priv = attach->dmabuf->priv;
> 
>         dma_resv_assert_held(attach->dmabuf->resv);
> -       if (attach->dmabuf->ops != &gmem_dma_buf_ops)
> -               return -EOPNOTSUPP;
> -       priv = attach->dmabuf->priv;
>         if (priv->revoked)
>                 return -ENODEV;
> -       *phys = priv->phys;
> +
> +       phys->paddr = priv->phys.paddr + offset;
> +       phys->len = len;
> +       *attr = DMA_BUF_PHYS_ATTR_RAM;
>         return 0;
>  }
> -EXPORT_SYMBOL_FOR_MODULES(gmem_provider_dma_buf_iommufd_map, "iommufd");
> +
> +static const struct dma_buf_ops gmem_dma_buf_ops = {
> +       .attach         = gmem_dma_buf_attach,
> +       .map_dma_buf    = gmem_dma_buf_map,
> +       .unmap_dma_buf  = gmem_dma_buf_unmap,
> +       .release        = gmem_dma_buf_release,
> +       .get_phys       = gmem_dma_buf_get_phys,
> +};
> 
>  /* Called with info->dmabufs_lock held on the revoke path. */
>  static void gmem_dma_buf_revoke_all(struct gmem_info *info)
> --
> 2.47.3
> 

Re: [RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Fred Griffoul]

On 10/5/26 Christian König wrote:
> Well filling page tables by the importer is an absolutely clear NO-GO
> for the DMA-buf design, we have gotten down that path already and it
> took us years to remove this functionality again.

Understood, thanks for the quick and clear answers on this patch and on
patch 3.

> The problem you are facing here is that dma_buf_mmap() doesn't work
> because you don't have a VMA.

Right: the memory has no struct page and is deliberately not mapped in
the host, so there is no VMA to hand to KVM.

I'll post a new version of this series. It drops get_phys(), the ranged
invalidation and the guest_memfd dma-buf import (patches 2-5), and it
does not touch drivers/dma-buf, as you suggested in the PAL discussion:

  https://lore.kernel.org/all/c413710b-4c28-4ed8-88ec-aeb8c4482011@amd.com/

KVM gets its frames through the KVM-private guest_memfd provider
interface from David's series. iommufd gets them through a private
interface with the exporter, as it already does for vfio-pci. The new
version adds a small registry in iommufd for that, which replaces the
symbol_get() of the sample in David's series. The dma-buf is then only
the handle and the existing revoke.

Thanks,
Fred

Re: [RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Christian König]

On 10/5/26 15:20, Fred Griffoul wrote:
> On 10/5/26 Christian König wrote:
>> Well filling page tables by the importer is an absolutely clear NO-GO
>> for the DMA-buf design, we have gotten down that path already and it
>> took us years to remove this functionality again.
> 
> Understood, thanks for the quick and clear answers on this patch and on
> patch 3.
> 
>> The problem you are facing here is that dma_buf_mmap() doesn't work
>> because you don't have a VMA.
> 
> Right: the memory has no struct page and is deliberately not mapped in
> the host, so there is no VMA to hand to KVM.
> 
> I'll post a new version of this series. It drops get_phys(), the ranged
> invalidation and the guest_memfd dma-buf import (patches 2-5), and it
> does not touch drivers/dma-buf, as you suggested in the PAL discussion:
> 
>   https://lore.kernel.org/all/c413710b-4c28-4ed8-88ec-aeb8c4482011@amd.com/
> 
> KVM gets its frames through the KVM-private guest_memfd provider
> interface from David's series. iommufd gets them through a private
> interface with the exporter, as it already does for vfio-pci. The new
> version adds a small registry in iommufd for that, which replaces the
> symbol_get() of the sample in David's series. The dma-buf is then only
> the handle and the existing revoke.

Yeah that approach sounds totally sane to me.

We have drivers which stuff all kind of resources, not just memory, into a DMA-buf. That ranges from MMIO BARs to trigger FW operations (doorbells) over full HW blocks (ordered appends, global wave sync etc...) where two or more applications need to share which one is used.

That you then have a exporter private interface is for accessing this is perfectly ok.

The problem you are facing here is more that this is not limited to one driver/module but multiple ones and you don't want module inter dependencies because of the symbols.

So symbol_get() is probably the right thing to do as long as we don't have something like weak symbols or similar.

Regards,
Christian.

> 
> Thanks,
> Fred

Re: [RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Jason Gunthorpe]

On Mon, Oct 05, 2026 at 04:53:34PM +0200, Christian König wrote:
> > KVM gets its frames through the KVM-private guest_memfd provider
> > interface from David's series. iommufd gets them through a private
> > interface with the exporter, as it already does for vfio-pci. The new
> > version adds a small registry in iommufd for that, which replaces the
> > symbol_get() of the sample in David's series. The dma-buf is then only
> > the handle and the existing revoke.
> 
> Yeah that approach sounds totally sane to me.
> 
> We have drivers which stuff all kind of resources, not just memory,
> into a DMA-buf. That ranges from MMIO BARs to trigger FW operations
> (doorbells) over full HW blocks (ordered appends, global wave sync
> etc...) where two or more applications need to share which one is
> used.
> 
> That you then have a exporter private interface is for accessing
> this is perfectly ok.
> 
> The problem you are facing here is more that this is not limited to
> one driver/module but multiple ones and you don't want module inter
> dependencies because of the symbols.
> 
> So symbol_get() is probably the right thing to do as long as we
> don't have something like weak symbols or similar.

I don't want to keep expanding this hack in iommufd. If dmabuf is to
be used then dmabuf needs to provide a pluggable way to fill the
iommu_domain that many exporters can plug into. I don't want to
hardwire every exporter into iommufd, that makes no sense.

The existing arrangement is to motivate solving this problem
generically, not something to be expanded on.

Jason

Re: [RFC PATCH 2/6] dma-buf: Add get_phys() to describe a physical run

[Fred Griffoul]

On Mon, Oct 12, 2026 at 11:46:16AM -0300, Jason Gunthorpe wrote:
> I don't want to keep expanding this hack in iommufd. If dmabuf is to
> be used then dmabuf needs to provide a pluggable way to fill the
> iommu_domain that many exporters can plug into. I don't want to
> hardwire every exporter into iommufd, that makes no sense.
>
> The existing arrangement is to motivate solving this problem
> generically, not something to be expanded on.

Agreed. The iommufd registry I mentioned above was never posted, and I
have dropped it.

The series I posted since takes another route, without dma-buf:

  https://lore.kernel.org/all/20261006183235.16576-1-griffoul@gmail.com/

It adds a small interface in mm for drivers that own memory. Such a
driver hands out files, and for each page of a file it tells the caller
which frame backs it. iommufd maps the file with IOMMU_IOAS_MAP_FILE and
builds the iommu_domain itself, and the driver revokes a range when it
changes. So iommufd never knows which driver it is talking to, and there
is no symbol_get(). guest_memfd uses the same files for the guest. The
iommufd part is patch 5:

  https://lore.kernel.org/all/20261006183235.16576-6-griffoul@gmail.com/

The iommufd part is mainly there to show the interface, and it needs
more work. Its revoke unmaps and then remaps, so a device can hit a
hole. I would rebuild it on the REPLACE operation David discussed with
you at LPC, with the scratch page as a quirk on the IOAS.

Is this the kind of generic interface you had in mind, or would you
rather have it hang off dma-buf?

Thanks,
Fred

[RFC PATCH 3/6] dma-buf: Add ranged mapping invalidation

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

dma_buf_invalidate_mappings() tells every importer that the whole
buffer changed. An exporter that changes one part of its memory cannot
say which bytes changed, so importers throw away mappings that are
still valid.

Add an exporter helper that invalidates a byte range, and an importer
callback that receives it. The callback means that the address, the
attributes or the backing of the range changed. If part of the range is
no longer backed, get_phys() returns -ENOENT for it. Importers must stop
using their old answer before the callback returns. Importers that do
not implement the callback still receive a whole-buffer invalidation.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/dma-buf/dma-buf.c | 30 ++++++++++++++++++++++++++++++
 include/linux/dma-buf.h   | 16 ++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 66b85d53ed22..e7010163eb2f 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -1389,6 +1389,36 @@ void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings, "DMA_BUF");
 
+/**
+ * dma_buf_invalidate_mappings_range - notify attachments that a range changed
+ * @dmabuf: buffer whose layout changed
+ * @offset: first changed byte
+ * @length: number of changed bytes
+ *
+ * Importers with a ranged callback stop using their old mappings of the range
+ * before returning. Other importers receive the existing whole-buffer
+ * callback, which is correct but coarser. The reservation lock must be held.
+ */
+void dma_buf_invalidate_mappings_range(struct dma_buf *dmabuf,
+				       unsigned long offset,
+				       unsigned long length)
+{
+	struct dma_buf_attachment *attach;
+
+	dma_resv_assert_held(dmabuf->resv);
+	list_for_each_entry(attach, &dmabuf->attachments, node) {
+		const struct dma_buf_attach_ops *ops = attach->importer_ops;
+
+		if (!ops)
+			continue;
+		if (ops->invalidate_mappings_range)
+			ops->invalidate_mappings_range(attach, offset, length);
+		else if (ops->invalidate_mappings)
+			ops->invalidate_mappings(attach);
+	}
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings_range, "DMA_BUF");
+
 /**
  * dma_buf_get_phys - describe the run that starts at an offset
  * @attach: attachment to query
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index b223962e20c2..55c3fe60a0ba 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -485,6 +485,19 @@ struct dma_buf_attach_ops {
 	 * required behavior.
 	 */
 	void (*invalidate_mappings)(struct dma_buf_attachment *attach);
+
+	/**
+	 * @invalidate_mappings_range: [optional] a byte range changed
+	 *
+	 * The exporter changed the address, attributes or backing of
+	 * [@offset, @offset + @length). The importer must stop using its old
+	 * answer for that range before returning.
+	 * Importers without this callback receive @invalidate_mappings for
+	 * the whole buffer instead.
+	 */
+	void (*invalidate_mappings_range)(struct dma_buf_attachment *attach,
+					  unsigned long offset,
+					  unsigned long length);
 };
 
 /**
@@ -600,6 +613,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
 void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
 				enum dma_data_direction);
 void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
+void dma_buf_invalidate_mappings_range(struct dma_buf *dma_buf,
+				       unsigned long offset,
+				       unsigned long length);
 bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
 /* bits 0-7: memory type (a value, not flags) */
 #define DMA_BUF_PHYS_ATTR_TYPE_MASK	GENMASK(7, 0)
-- 
2.47.3

Re: [RFC PATCH 3/6] dma-buf: Add ranged mapping invalidation

[Christian König]

On 10/5/26 11:55, Fred Griffoul wrote:
> From: Fred Griffoul <fgriffo@amazon.co.uk>
> 
> dma_buf_invalidate_mappings() tells every importer that the whole
> buffer changed. An exporter that changes one part of its memory cannot
> say which bytes changed, so importers throw away mappings that are
> still valid.
> 
> Add an exporter helper that invalidates a byte range, and an importer
> callback that receives it. The callback means that the address, the
> attributes or the backing of the range changed. If part of the range is
> no longer backed, get_phys() returns -ENOENT for it. Importers must stop
> using their old answer before the callback returns. Importers that do
> not implement the callback still receive a whole-buffer invalidation.

Yeah that is exactly one of the reasons why we don't allow that.

Clear NAK to the whole approach. See the reply to patch #2 for a detailed description.

Regards,
Christian.

> 
> Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
> ---
>  drivers/dma-buf/dma-buf.c | 30 ++++++++++++++++++++++++++++++
>  include/linux/dma-buf.h   | 16 ++++++++++++++++
>  2 files changed, 46 insertions(+)
> 
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 66b85d53ed22..e7010163eb2f 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -1389,6 +1389,36 @@ void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
>  }
>  EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings, "DMA_BUF");
> 
> +/**
> + * dma_buf_invalidate_mappings_range - notify attachments that a range changed
> + * @dmabuf: buffer whose layout changed
> + * @offset: first changed byte
> + * @length: number of changed bytes
> + *
> + * Importers with a ranged callback stop using their old mappings of the range
> + * before returning. Other importers receive the existing whole-buffer
> + * callback, which is correct but coarser. The reservation lock must be held.
> + */
> +void dma_buf_invalidate_mappings_range(struct dma_buf *dmabuf,
> +                                      unsigned long offset,
> +                                      unsigned long length)
> +{
> +       struct dma_buf_attachment *attach;
> +
> +       dma_resv_assert_held(dmabuf->resv);
> +       list_for_each_entry(attach, &dmabuf->attachments, node) {
> +               const struct dma_buf_attach_ops *ops = attach->importer_ops;
> +
> +               if (!ops)
> +                       continue;
> +               if (ops->invalidate_mappings_range)
> +                       ops->invalidate_mappings_range(attach, offset, length);
> +               else if (ops->invalidate_mappings)
> +                       ops->invalidate_mappings(attach);
> +       }
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_invalidate_mappings_range, "DMA_BUF");
> +
>  /**
>   * dma_buf_get_phys - describe the run that starts at an offset
>   * @attach: attachment to query
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index b223962e20c2..55c3fe60a0ba 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -485,6 +485,19 @@ struct dma_buf_attach_ops {
>          * required behavior.
>          */
>         void (*invalidate_mappings)(struct dma_buf_attachment *attach);
> +
> +       /**
> +        * @invalidate_mappings_range: [optional] a byte range changed
> +        *
> +        * The exporter changed the address, attributes or backing of
> +        * [@offset, @offset + @length). The importer must stop using its old
> +        * answer for that range before returning.
> +        * Importers without this callback receive @invalidate_mappings for
> +        * the whole buffer instead.
> +        */
> +       void (*invalidate_mappings_range)(struct dma_buf_attachment *attach,
> +                                         unsigned long offset,
> +                                         unsigned long length);
>  };
> 
>  /**
> @@ -600,6 +613,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>  void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
>                                 enum dma_data_direction);
>  void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
> +void dma_buf_invalidate_mappings_range(struct dma_buf *dma_buf,
> +                                      unsigned long offset,
> +                                      unsigned long length);
>  bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
>  /* bits 0-7: memory type (a value, not flags) */
>  #define DMA_BUF_PHYS_ATTR_TYPE_MASK    GENMASK(7, 0)
> --
> 2.47.3
> 

[RFC PATCH 4/6] dma-buf: Allow dynamic attach without a device

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

dma_buf_dynamic_attach() requires a struct device: the device for which
the buffer is mapped for DMA. Exporters may also check it before they
accept the attachment. An importer that does not perform DMA has no
such device. iommufd already attaches with a global placeholder device
for this reason. KVM would need one too, although it only needs to know
where the memory is so that it can map it into a guest.

Allow @dev to be NULL for a dynamic importer. Such an importer learns
where the memory is from get_phys() and builds its own mappings. It
must not map the attachment for DMA, and dma_buf_map_attachment()
refuses an attachment without a device. An exporter that needs a device
to answer can refuse the attachment in its attach op, as it can for any
other reason.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/dma-buf/dma-buf.c      | 15 +++++++++++++--
 include/trace/events/dma_buf.h |  2 +-
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index e7010163eb2f..3d43b529a523 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -1007,6 +1007,13 @@ dma_buf_pin_on_map(struct dma_buf_attachment *attach)
  * Note that this can fail if the backing storage of @dmabuf is in a place not
  * accessible to @dev, and cannot be moved to a more suitable place. This is
  * indicated with the error code -EBUSY.
+ *
+ * @dev may be NULL for a dynamic importer that is not a DMA master: one that
+ * only asks the exporter where the memory is, through &dma_buf_ops.get_phys,
+ * and builds its own mappings from the answer.  Such an importer must never
+ * call dma_buf_map_attachment(), and an exporter that needs a device to
+ * answer (a peer-to-peer check, say) refuses the attachment in its
+ * &dma_buf_ops.attach.
  */
 struct dma_buf_attachment *
 dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
@@ -1016,7 +1023,7 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 	struct dma_buf_attachment *attach;
 	int ret;
 
-	if (WARN_ON(!dmabuf || !dev))
+	if (WARN_ON(!dmabuf || (!dev && !importer_ops)))
 		return ERR_PTR(-EINVAL);
 
 	attach = kzalloc_obj(*attach);
@@ -1175,6 +1182,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 
 	if (WARN_ON(!attach || !attach->dmabuf))
 		return ERR_PTR(-EINVAL);
+	/* An importer without a device cannot be a DMA master. */
+	if (WARN_ON(!attach->dev))
+		return ERR_PTR(-EINVAL);
 
 	dma_resv_assert_held(attach->dmabuf->resv);
 
@@ -1847,7 +1857,8 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused)
 		attach_count = 0;
 
 		list_for_each_entry(attach_obj, &buf_obj->attachments, node) {
-			seq_printf(s, "\t%s\n", dev_name(attach_obj->dev));
+			seq_printf(s, "\t%s\n", attach_obj->dev ?
+				   dev_name(attach_obj->dev) : "none");
 			attach_count++;
 		}
 		dma_resv_unlock(buf_obj->resv);
diff --git a/include/trace/events/dma_buf.h b/include/trace/events/dma_buf.h
index 3bb88d05bcc8..7cabca53b798 100644
--- a/include/trace/events/dma_buf.h
+++ b/include/trace/events/dma_buf.h
@@ -40,7 +40,7 @@ DECLARE_EVENT_CLASS(dma_buf_attach_dev,
 	TP_ARGS(dmabuf, attach, is_dynamic, dev),
 
 	TP_STRUCT__entry(
-		__string(	dev_name,			dev_name(dev))
+		__string(dev_name, dev ? dev_name(dev) : "none")
 		__string(	exp_name,			dmabuf->exp_name)
 		__field(	size_t,				size)
 		__field(	ino_t,				ino)
-- 
2.47.3

[RFC PATCH 5/6] KVM: guest_memfd: Add dma-buf backing

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A memory owner that exports a dma-buf should be able to use that one
object for both guest stage-2 mappings and device IOMMU mappings. A
second, KVM-specific backing interface would duplicate the description
of the memory layout and the rules for invalidating it.

Add GUEST_MEMFD_FLAG_USE_DMABUF and a dmabuf_fd field to
KVM_CREATE_GUEST_MEMFD. This backing has its own kvm_gmem_ops.
guest_memfd attaches to the buffer as a revocable importer without a
device. At creation, it asks for the run at offset 0 and refuses MMIO
or unknown attributes, so a buffer that KVM can never map fails early.
Offset 0 may be unbacked.

On a fault, guest_memfd takes the reservation once. It asks get_phys()
about the PUD-sized, PMD-sized and page-sized blocks around the faulting
page, in that order, and maps the first block that one aligned RAM run
covers. If the page is not backed or the run is not supported, the
fault returns -EFAULT. A READONLY run is mapped without stage-2 write
permission.

The invalidation callback only runs KVM's invalidation over the
affected range. It does not allocate memory or call the exporter. A
fault that read the layout during the change sees mmu_invalidate_seq
move and retries.

Callbacks can start as soon as guest_memfd attaches. A dma-buf-backed
file is therefore added to the inode's list of guest_memfd files under
filemap_invalidate_lock.

mmap() uses the exporter's dma-buf mmap. fallocate() and populate are
not supported. Import the DMA_BUF symbol namespace so that a
CONFIG_KVM=m build passes modpost.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 include/linux/kvm_host.h       |   1 +
 include/uapi/linux/kvm.h       |   6 +-
 tools/include/uapi/linux/kvm.h |   6 +-
 virt/kvm/Kconfig               |   1 +
 virt/kvm/guest_memfd.c         | 239 +++++++++++++++++++++++++++++++--
 5 files changed, 243 insertions(+), 10 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index f84f3ab44acf..0158dc46580b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -834,6 +834,7 @@ static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
 
 	if (!kvm || kvm_arch_supports_gmem_init_shared(kvm))
 		flags |= GUEST_MEMFD_FLAG_INIT_SHARED;
+	flags |= GUEST_MEMFD_FLAG_USE_DMABUF;
 
 	return flags;
 }
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8..f8c83013773d 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1654,11 +1654,15 @@ struct kvm_memory_attributes {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_DMABUF	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	/* With USE_DMABUF, memory comes from this dma-buf's leading bytes. */
+	__s32 dmabuf_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index d0c0c8605976..ebeb2bc353ed 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1644,11 +1644,15 @@ struct kvm_memory_attributes {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_DMABUF	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	/* With USE_DMABUF, memory comes from this dma-buf's leading bytes. */
+	__s32 dmabuf_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 794976b88c6f..7ac4c8a371eb 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -105,6 +105,7 @@ config KVM_GENERIC_MEMORY_ATTRIBUTES
 
 config KVM_GUEST_MEMFD
        select XARRAY_MULTI
+       select DMA_SHARED_BUFFER
        bool
 
 config HAVE_KVM_ARCH_GMEM_PREPARE
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 0f4bf2cc5e8e..f401cc560eb4 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1,4 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
+#include <linux/module.h>
 #include <linux/anon_inodes.h>
 #include <linux/backing-dev.h>
 #include <linux/falloc.h>
@@ -44,6 +47,8 @@ struct gmem_inode {
 	struct list_head gmem_file_list;
 
 	u64 flags;
+	struct dma_buf *dmabuf;
+	struct dma_buf_attachment *attach;
 };
 
 static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
@@ -477,21 +482,37 @@ static const struct vm_operations_struct kvm_gmem_vm_ops = {
 #endif
 };
 
-static int kvm_gmem_native_mmap(struct file *file, struct vm_area_struct *vma)
+static int kvm_gmem_validate_mmap(struct file *file,
+				  struct vm_area_struct *vma)
 {
 	if (!kvm_gmem_supports_mmap(file_inode(file)))
 		return -ENODEV;
-
 	if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) !=
-	    (VM_SHARED | VM_MAYSHARE)) {
+	    (VM_SHARED | VM_MAYSHARE))
 		return -EINVAL;
-	}
+	return 0;
+}
 
-	vma->vm_ops = &kvm_gmem_vm_ops;
+static int kvm_gmem_native_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	int ret = kvm_gmem_validate_mmap(file, vma);
 
+	if (ret)
+		return ret;
+	vma->vm_ops = &kvm_gmem_vm_ops;
 	return 0;
 }
 
+static int kvm_gmem_dmabuf_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct dma_buf *dmabuf = GMEM_I(file_inode(file))->dmabuf;
+	int ret = kvm_gmem_validate_mmap(file, vma);
+
+	if (ret)
+		return ret;
+	return dma_buf_mmap(dmabuf, vma, vma->vm_pgoff);
+}
+
 /* File-op dispatchers — thin: they all go through the backing's ops table. */
 
 static int kvm_gmem_fops_release(struct inode *inode, struct file *file)
@@ -617,6 +638,172 @@ bool __weak kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
 	return true;
 }
 
+/* ---- a dma-buf behind the file ----------------------------------------- */
+
+/*
+ * Refuse a buffer whose first run guest_memfd can never map, such as MMIO,
+ * at creation rather than on every fault. A gap at offset 0 is legal. This
+ * is advisory: each fault checks the run it uses.
+ */
+static int gmem_dmabuf_check(struct dma_buf_attachment *attach, u64 size)
+{
+	struct phys_vec pv;
+	u32 attr;
+	int ret;
+
+	dma_resv_lock(attach->dmabuf->resv, NULL);
+	ret = dma_buf_get_phys(attach, 0, size, &pv, &attr);
+	dma_resv_unlock(attach->dmabuf->resv);
+	if (ret == -ENOENT)
+		return 0;
+	if (ret)
+		return ret;
+	if (!dma_buf_phys_attr_known(attr) ||
+	    dma_buf_phys_attr_type(attr) != DMA_BUF_PHYS_ATTR_RAM)
+		return -EOPNOTSUPP;
+	return 0;
+}
+
+static void gmem_dmabuf_invalidate_range(struct dma_buf_attachment *attach,
+					 unsigned long offset,
+					 unsigned long len)
+{
+	struct inode *inode = attach->importer_priv;
+	pgoff_t npages = i_size_read(inode) >> PAGE_SHIFT;
+	pgoff_t start = offset >> PAGE_SHIFT;
+	pgoff_t end = min_t(pgoff_t, DIV_ROUND_UP(offset + len, PAGE_SIZE),
+			    npages);
+
+	if (start >= end)
+		return;
+	/* Lock order: dma-buf reservation, file invalidation, then mmu_lock. */
+	filemap_invalidate_lock(inode->i_mapping);
+	kvm_gmem_invalidate_start(inode, start, end);
+	kvm_gmem_invalidate_end(inode, start, end);
+	filemap_invalidate_unlock(inode->i_mapping);
+}
+
+static void gmem_dmabuf_invalidate(struct dma_buf_attachment *attach)
+{
+	struct inode *inode = attach->importer_priv;
+
+	gmem_dmabuf_invalidate_range(attach, 0, i_size_read(inode));
+}
+
+static const struct dma_buf_attach_ops gmem_dmabuf_attach_ops = {
+	.allow_peer2peer	= true,
+	.invalidate_mappings	= gmem_dmabuf_invalidate,
+	.invalidate_mappings_range = gmem_dmabuf_invalidate_range,
+};
+
+static int kvm_gmem_dmabuf_attach(struct inode *inode, int dmabuf_fd)
+{
+	struct gmem_inode *gi = GMEM_I(inode);
+	struct dma_buf_attachment *attach;
+	struct dma_buf *dmabuf;
+	int ret;
+
+	dmabuf = dma_buf_get(dmabuf_fd);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+	if (dmabuf->size < i_size_read(inode)) {
+		ret = -EINVAL;
+		goto put;
+	}
+
+	gi->dmabuf = dmabuf;
+	attach = dma_buf_dynamic_attach(dmabuf, NULL, &gmem_dmabuf_attach_ops,
+					inode);
+	if (IS_ERR(attach)) {
+		ret = PTR_ERR(attach);
+		goto clear_put;
+	}
+	gi->attach = attach;
+	ret = gmem_dmabuf_check(attach, i_size_read(inode));
+	if (ret)
+		goto detach;
+	return 0;
+
+detach:
+	dma_buf_detach(dmabuf, attach);
+	gi->attach = NULL;
+clear_put:
+	gi->dmabuf = NULL;
+put:
+	dma_buf_put(dmabuf);
+	return ret;
+}
+
+static void kvm_gmem_dmabuf_release(struct inode *inode)
+{
+	struct gmem_inode *gi = GMEM_I(inode);
+
+	if (!gi->dmabuf)
+		return;
+	dma_buf_detach(gi->dmabuf, gi->attach);
+	dma_buf_put(gi->dmabuf);
+	gi->attach = NULL;
+	gi->dmabuf = NULL;
+}
+
+static int gmem_dmabuf_get_pfn(struct file *file, struct kvm *kvm,
+			       struct kvm_memory_slot *slot, gfn_t gfn,
+			       kvm_pfn_t *pfn, struct page **page,
+			       int *max_order, bool *writable)
+{
+	static const int orders[] = { PUD_ORDER, PMD_ORDER, 0 };
+	struct gmem_inode *gi = GMEM_I(file_inode(file));
+	struct gmem_file *f = gmem_file_of(file);
+	pgoff_t index = kvm_gmem_get_index(slot, gfn);
+	pgoff_t npages = i_size_read(file_inode(file)) >> PAGE_SHIFT;
+	unsigned int oi;
+	int ret = -EFAULT;
+
+	if (file != READ_ONCE(slot->gmem.file))
+		return -EFAULT;
+	if (xa_load(&f->bindings, index) != slot)
+		return -EIO;
+
+	/* A block maps at an order only if one aligned RAM run covers it. */
+	dma_resv_lock(gi->dmabuf->resv, NULL);
+	for (oi = max_order ? 0 : ARRAY_SIZE(orders) - 1;
+	     oi < ARRAY_SIZE(orders); oi++) {
+		int order = orders[oi];
+		pgoff_t bs = ALIGN_DOWN(index, 1UL << order);
+		u64 blen = (u64)PAGE_SIZE << order;
+		struct phys_vec pv;
+		u32 attr;
+
+		if (bs + (1UL << order) > npages)
+			continue;
+		if (dma_buf_get_phys(gi->attach, (u64)bs << PAGE_SHIFT, blen,
+				     &pv, &attr) ||
+		    pv.len != blen || !IS_ALIGNED(pv.paddr, blen) ||
+		    !dma_buf_phys_attr_known(attr) ||
+		    dma_buf_phys_attr_type(attr) != DMA_BUF_PHYS_ATTR_RAM)
+			continue;
+
+		*pfn = PHYS_PFN(pv.paddr) + (index - bs);
+		*page = NULL;
+		if (max_order)
+			*max_order = order;
+		if (writable && (attr & DMA_BUF_PHYS_ATTR_READONLY))
+			*writable = false;
+		ret = 0;
+		break;
+	}
+	dma_resv_unlock(gi->dmabuf->resv);
+	return ret;
+}
+
+static int kvm_gmem_dmabuf_populate(struct file *file, struct kvm *kvm,
+				    struct kvm_memory_slot *slot, gfn_t gfn,
+				    kvm_pfn_t *pfn, struct page *src_page,
+				    int order)
+{
+	return -EOPNOTSUPP;
+}
+
 static int  kvm_gmem_native_bind(struct file *file, struct kvm *kvm,
 				 struct kvm_memory_slot *slot, loff_t offset);
 static void kvm_gmem_native_unbind(struct file *slot_file, struct kvm *kvm,
@@ -654,7 +841,17 @@ static const struct kvm_gmem_ops kvm_gmem_native_ops = {
 	.fallocate = kvm_gmem_native_fallocate,
 };
 
-static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
+static const struct kvm_gmem_ops kvm_gmem_dmabuf_ops = {
+	.bind      = kvm_gmem_native_bind,
+	.unbind    = kvm_gmem_native_unbind,
+	.get_pfn   = gmem_dmabuf_get_pfn,
+	.populate  = kvm_gmem_dmabuf_populate,
+	.release   = kvm_gmem_native_release,
+	.mmap      = kvm_gmem_dmabuf_mmap,
+};
+
+static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags,
+			     int dmabuf_fd)
 {
 	static const char *name = "[kvm-gmem]";
 	struct gmem_file *f;
@@ -695,6 +892,12 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 
 	GMEM_I(inode)->flags = flags;
 
+	if (flags & GUEST_MEMFD_FLAG_USE_DMABUF) {
+		err = kvm_gmem_dmabuf_attach(inode, dmabuf_fd);
+		if (err)
+			goto err_inode;
+	}
+
 	file = alloc_file_pseudo(inode, kvm_gmem_mnt, name, O_RDWR, &kvm_gmem_fops);
 	if (IS_ERR(file)) {
 		err = PTR_ERR(file);
@@ -703,12 +906,17 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 
 	file->f_flags |= O_LARGEFILE;
 	file->private_data = &f->backing;
-	f->backing.ops = &kvm_gmem_native_ops;
+	f->backing.ops = flags & GUEST_MEMFD_FLAG_USE_DMABUF ?
+			 &kvm_gmem_dmabuf_ops : &kvm_gmem_native_ops;
 
 	kvm_get_kvm(kvm);
 	f->kvm = kvm;
 	xa_init(&f->bindings);
+	if (flags & GUEST_MEMFD_FLAG_USE_DMABUF)
+		filemap_invalidate_lock(inode->i_mapping);
 	list_add(&f->entry, &GMEM_I(inode)->gmem_file_list);
+	if (flags & GUEST_MEMFD_FLAG_USE_DMABUF)
+		filemap_invalidate_unlock(inode->i_mapping);
 
 	fd_install(fd, file);
 	return fd;
@@ -734,8 +942,11 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
 
 	if (size <= 0 || !PAGE_ALIGNED(size))
 		return -EINVAL;
+	if (args->pad ||
+	    (!(flags & GUEST_MEMFD_FLAG_USE_DMABUF) && args->dmabuf_fd))
+		return -EINVAL;
 
-	return __kvm_gmem_create(kvm, size, flags);
+	return __kvm_gmem_create(kvm, size, flags, args->dmabuf_fd);
 }
 
 /*
@@ -1211,6 +1422,8 @@ static struct inode *kvm_gmem_alloc_inode(struct super_block *sb)
 	mpol_shared_policy_init(&gi->policy, NULL);
 
 	gi->flags = 0;
+	gi->dmabuf = NULL;
+	gi->attach = NULL;
 	INIT_LIST_HEAD(&gi->gmem_file_list);
 	return &gi->vfs_inode;
 }
@@ -1225,11 +1438,19 @@ static void kvm_gmem_free_inode(struct inode *inode)
 	kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
 }
 
+static void kvm_gmem_evict_inode(struct inode *inode)
+{
+	kvm_gmem_dmabuf_release(inode);
+	truncate_inode_pages_final(&inode->i_data);
+	clear_inode(inode);
+}
+
 static const struct super_operations kvm_gmem_super_operations = {
 	.statfs		= simple_statfs,
 	.alloc_inode	= kvm_gmem_alloc_inode,
 	.destroy_inode	= kvm_gmem_destroy_inode,
 	.free_inode	= kvm_gmem_free_inode,
+	.evict_inode	= kvm_gmem_evict_inode,
 };
 
 static int kvm_gmem_init_fs_context(struct fs_context *fc)
@@ -1292,3 +1513,5 @@ void kvm_gmem_exit(void)
 	rcu_barrier();
 	kmem_cache_destroy(kvm_gmem_inode_cachep);
 }
+
+MODULE_IMPORT_NS("DMA_BUF");
-- 
2.47.3

[RFC PATCH 6/6] samples/kvm, selftests/kvm: Exercise dma-buf backing

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Turn the sample into a dma-buf exporter, and move every supported test
to dma-buf-backed guest_memfd. Doing both in one commit avoids a state
where the sample and the tests use different interfaces.

The sample owns one root region and creates a child descriptor for each
VM. It supports moving, donating and reclaiming pages, absent pages,
read-only ranges, a scratch page and per-child ioctl allowlists.

The sample's get_phys() reports the run at the requested offset. It
uses bitmap searches to find where presence or read-only state changes.
An absent page returns -ENOENT. In scratch mode, an absent page of a
root child is reported as the read-only scratch page instead. Scratch
mode applies only to root children, because a mode change invalidates
only them. Every change of ownership sends a ranged invalidation.

Each test passes one dma-buf fd to both guest_memfd and iommufd. New
tests cover:

- two VMMs, checking what the guest, the device and the host see;
- SET_PRESENT and SET_READONLY racing with one KVM_CREATE_GUEST_MEMFD,
  so that lockdep checks attachment setup;
- a 2 MiB-aligned region in which every other page is replaced by the
  scratch page. The guest reads zeros and its own data, and KVM maps
  the region with 4 KiB pages. An untouched aligned region still maps
  at 2 MiB.

The sample sizes the CMA root before children overlap, and publishes
each child only after its ownership and allowlist are set. Confidential
VMs are out of scope; their test is an explicit skip.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/kvm/gmem_provider.c                   | 1287 ++++++++++-------
 samples/kvm/gmem_provider.h                   |  144 +-
 tools/testing/selftests/kvm/Makefile.kvm      |    3 +-
 .../kvm/gmem_provider_nvme_dma_test.c         |   28 +-
 .../testing/selftests/kvm/include/kvm_util.h  |   25 +-
 .../testing/selftests/kvm/x86/gmem_poc_test.c |  824 +++++++++++
 .../kvm/x86/gmem_provider_hugepage_test.c     |   30 +-
 .../kvm/x86/gmem_provider_iommufd_test.c      |   42 +-
 .../kvm/x86/gmem_provider_readonly_test.c     |  172 +++
 .../kvm/x86/gmem_provider_revoke_test.c       |   34 +-
 .../selftests/kvm/x86/gmem_provider_test.c    |  190 +--
 .../kvm/x86/gmem_provider_vfio_test.c         |   45 +-
 12 files changed, 2091 insertions(+), 733 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_poc_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c

diff --git a/samples/kvm/gmem_provider.c b/samples/kvm/gmem_provider.c
index b6824fe5d228..2cde5bcb71ca 100644
--- a/samples/kvm/gmem_provider.c
+++ b/samples/kvm/gmem_provider.c
@@ -1,43 +1,46 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * gmem_provider - sample external guest_memfd provider.
+ * gmem_provider - sample owner of page-less memory, shared as a dma-buf.
  *
- * Demonstrates the KVM guest_memfd provider ABI with two backing modes:
+ * A toy owner of physical memory: one root region carved into children, one
+ * child per VM, ranges that MOVE between children or are DONATEd to the root
+ * and RECLAIMed, a shared scratch frame for revoked pages, and a per-fd
+ * ioctl allowlist fixed by the creator.  The provider owns the frames and is
+ * the single authority for who may map them.
  *
- *  - External (page-less): loaded with addr=/len=, backs guest memory with a
- *    fixed physical range that has no struct page -- e.g. memory carved out of
- *    the kernel with mem= on the command line.  This is the case the provider
- *    ABI exists for; get_pfn() returns bare PFNs KVM treats as non-refcounted.
+ * It is a dma-buf exporter and nothing else.  A child fd hands out a dma-buf
+ * for the child's memory; a VMM gives that dma-buf to KVM_CREATE_GUEST_MEMFD
+ * for the guest and to iommufd for its devices.  An ownership change here is
+ * one ranged invalidation of the child's dma-bufs; every importer drops the
+ * changed view and re-reads the layout. The owner knows nothing of KVM, or which
+ * importer is a guest and which is a device.
  *
- *  - CMA fallback (page-backed): when addr=/len= are not given, allocates a
- *    physically contiguous region via alloc_contig_pages() sized by the setup
- *    ioctl.  Easier to run (no mem= boot param), and still exercises 2M/1G
- *    mappings.  Not preserved across kexec/live update.
- *
- * On SEV-SNP hosts, bind() resets the range's RMP entries to 4K shared so a
- * (new) SNP VM can re-encrypt it, which is what allows re-binding the range to
- * a fresh VM across a live update.
+ * Backing:
+ *  - External (page-less): loaded with addr=/len=, a fixed physical range with
+ *    no struct page, e.g. carved out with memmap= on the command line.
+ *  - CMA fallback: without addr=/len=, alloc_contig_pages() sized by SETUP.
  *
  * Usage:
- *   # page-less external range:
- *   insmod gmem_provider.ko addr=0x5D40000000 len=0x1000000
- *   # or CMA fallback (no params); size comes from the ioctl
- *   insmod gmem_provider.ko
- *   fd = open("/dev/gmem_provider"); ioctl(fd, GMEM_PROVIDER_SETUP, {kvm_fd, size});
- *   pass the returned fd + KVM_MEM_GUEST_MEMFD to KVM_SET_USER_MEMORY_REGION2.
+ *   insmod gmem_provider.ko addr=0x180000000 len=0x42000000
+ *   ctl = open("/dev/gmem_provider");
+ *   child = ioctl(ctl, GMEM_PROVIDER_NEW_CHILD, {offset, len, allow}); # control
+ *   dmabuf = ioctl(child, GMEM_PROVIDER_GET_DMABUF);                     # VMM
+ *   gmem = ioctl(vm, KVM_CREATE_GUEST_MEMFD, {size, USE_DMABUF, dmabuf});
+ *   KVM_SET_USER_MEMORY_REGION2(..., KVM_MEM_GUEST_MEMFD, gmem);
+ *   IOMMU_IOAS_MAP_FILE(ioas, dmabuf, ...);
+ *
+ * The one rule: change ownership state, drop the provider lock, then revoke.
  */
 
 #include <linux/anon_inodes.h>
-#include <linux/bitmap.h>
 #include <linux/dma-buf.h>
-#include <linux/dma-buf-mapping.h>
 #include <linux/dma-resv.h>
+#include <linux/bitmap.h>
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/gfp.h>
 #include <linux/highmem.h>
 #include <linux/io.h>
-#include <linux/kvm_host.h>
 #include <linux/miscdevice.h>
 #include <linux/mm.h>
 #include <linux/module.h>
@@ -50,7 +53,6 @@
 
 #include "gmem_provider.h"
 
-MODULE_IMPORT_NS("DMA_BUF");
 
 static unsigned long long addr;
 module_param(addr, ullong, 0444);
@@ -63,444 +65,294 @@ MODULE_PARM_DESC(len, "size in bytes of the external backing region (optional)")
 
 struct gmem_info {
 	/*
-	 * MUST be first: file->private_data points here.  is_kvm_gmem_file()
-	 * on the KVM side proves the reinterpretation is safe.
-	 */
-	struct kvm_gmem_backing backing;
-
-	/*
-	 * Protects everything below (except the immutable base_pfn/npages/
-	 * cma_pages fields set at setup).  Ordering: info->lock is a leaf;
-	 * do not acquire other locks under it.  KVM's slots_lock is already
-	 * held on the .bind/.unbind paths so info->lock is only needed to
-	 * serialise those against ioctl(SET_PRESENT).
+	 * Protects everything below except the immutable fields set at setup
+	 * and the dma-buf list.  Ordering: gmem_root.lock, then
+	 * info->dmabufs_lock, then a dma-buf reservation.  info->lock is a
+	 * leaf: it is dropped before any invalidation.
 	 */
 	struct mutex lock;
 
-	/* Kept locally: kvm_gmem_ops does not carry a kvm pointer. */
-	struct kvm *kvm;
-
-	/*
-	 * Single active memslot binding.  Used at release time to zap any
-	 * still-active guest mappings before the memory disappears.  NULL
-	 * when no memslot is bound (or the last one has been unbound).
-	 */
-	struct kvm_memory_slot *bound_slot;
 
-	bool mmap_capable;			/* -> KVM_MEMSLOT_GMEM_ONLY at bind */
+	bool mmap_capable;			/* child and dma-buf may be mmap()ed */
 
 	unsigned long base_pfn;
 	unsigned long npages;
-	struct page *cma_pages;			/* non-NULL if CMA-allocated */
-	gfn_t base_gfn;				/* recorded at bind, for revoke */
-	pgoff_t pgoff;				/* provider offset (pages) of the slot */
+	struct page *cma_pages;			/* non-NULL if CMA-allocated (SETUP path only) */
+
+	/*
+	 * Toy descriptor tree.  A child created by NEW_CHILD is a sub-range
+	 * of the root region: @root_index is its first page within the root,
+	 * @owned marks which of its pages are currently granted to it (a page
+	 * MOVEd out or DONATEd is not owned; a page revoked by SET_PRESENT is
+	 * owned but absent).  @allow is the ioctl allowlist.  SETUP-created
+	 * providers have no root and own everything.
+	 */
+	unsigned long root_index;
+	unsigned long *owned;			/* NULL for SETUP-created providers */
+	u32 allow;
+	struct list_head root_link;		/* gmem_root.children */
 	unsigned long *absent;			/* bitmap of currently-revoked pages */
-	struct list_head dmabufs;		/* struct gmem_dmabuf entries */
-	struct mutex dmabufs_lock;
+	unsigned long *readonly;		/* bitmap of pages the guest may not write */
+	struct address_space *mapping;		/* our file's: host windows live here */
+	struct list_head dmabufs;		/* exported dma-bufs (struct gmem_dmabuf) */
+	struct mutex dmabufs_lock;		/* protects @dmabufs */
 };
 
 static struct gmem_info *to_gmem_info(struct file *file)
 {
-	return container_of(file->private_data, struct gmem_info, backing);
+	return file->private_data;
 }
 
-/* Map a backing PFN for CPU access: page-backed via kmap, page-less via memremap. */
-static void *gmem_map_pfn(kvm_pfn_t pfn)
-{
-	if (pfn_valid(pfn))
-		return kmap_local_pfn(pfn);
-	return memremap(PFN_PHYS(pfn), PAGE_SIZE, MEMREMAP_WB);
-}
+/*
+ * The root of the toy descriptor tree: one backing region owned by the
+ * control device.  Children carve sub-ranges out of it.  All ownership
+ * transitions (NEW_CHILD, MOVE, DONATE, RECLAIM) run under root.lock, which
+ * is taken before any child's info->lock.
+ */
+static struct gmem_root {
+	struct mutex lock;			/* protects the fields below */
+	unsigned long base_pfn;
+	unsigned long npages;
+	struct page *cma_pages;
+	unsigned long *owned;			/* pages some child currently owns */
+	unsigned long *donated;			/* pages parked at the root */
+	struct list_head children;
+	bool scratch_enabled;
+} gmem_root;
 
-static void gmem_unmap_pfn(kvm_pfn_t pfn, void *vaddr)
-{
-	if (!vaddr)
-		return;
-	if (pfn_valid(pfn))
-		kunmap_local(vaddr);
-	else
-		memunmap(vaddr);
-}
+/*
+ * One zeroed scratch frame for the whole module.  When scratch mode is on,
+ * a revoked page of a root child reports this frame read-only to every
+ * importer, so a device that cannot tolerate an IOMMU fault lands here.
+ */
+static struct page *gmem_scratch_page;
 
-/* Largest order KVM may map at @gfn, snapped to 4K/2M/1G. */
-static int gmem_max_order(struct gmem_info *info, gfn_t gfn, unsigned long index)
+static inline unsigned long gmem_scratch_pfn(void)
 {
-	unsigned long pfn = info->base_pfn + index;
-	unsigned long remaining = info->npages - index;
-	unsigned int pud_order = PUD_SHIFT - PAGE_SHIFT;
-	unsigned int pmd_order = PMD_SHIFT - PAGE_SHIFT;
-	unsigned long absent_next;
-
-	/*
-	 * A hugepage may not span any revoked page.  Clamp by the distance to
-	 * the next absent bit; scanning is cheap because @absent is a plain
-	 * bitmap and the caller already checked test_bit(index).
-	 */
-	if (info->absent) {
-		absent_next = find_next_bit(info->absent, info->npages,
-					    index + 1);
-		remaining = min(remaining, absent_next - index);
-	}
-
-	if (IS_ALIGNED(pfn, 1UL << pud_order) &&
-	    IS_ALIGNED(gfn, 1UL << pud_order) &&
-	    remaining >= (1UL << pud_order))
-		return pud_order;
-
-	if (IS_ALIGNED(pfn, 1UL << pmd_order) &&
-	    IS_ALIGNED(gfn, 1UL << pmd_order) &&
-	    remaining >= (1UL << pmd_order))
-		return pmd_order;
-
-	return 0;
+	return page_to_pfn(gmem_scratch_page);
 }
 
-static int gmem_get_pfn(struct file *file, struct kvm *kvm,
-			struct kvm_memory_slot *slot, gfn_t gfn,
-			kvm_pfn_t *pfn, struct page **page, int *max_order,
-			bool *writable)
+/*
+ * Is page @index of @info currently reachable by the guest and devices?
+ * Callable with or without info->lock: every change to the bitmaps is
+ * followed by an invalidation of the range, so a lock-free answer is either
+ * current or about to be superseded.
+ */
+static inline bool gmem_page_present(struct gmem_info *info, unsigned long index)
 {
-	struct gmem_info *info = to_gmem_info(file);
-	pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff;
-
-	if (index >= info->npages)
-		return -EINVAL;
-
-	/* Revoked (absent) page: behave like not-present so the fault fails. */
+	if (info->owned && !test_bit(index, info->owned))
+		return false;
 	if (info->absent && test_bit(index, info->absent))
-		return -EFAULT;
-
-	*pfn = info->base_pfn + index;
-	if (max_order)
-		*max_order = gmem_max_order(info, gfn, index);
-	return 0;
+		return false;
+	return true;
 }
 
-static int gmem_populate(struct file *file, struct kvm *kvm,
-			 struct kvm_memory_slot *slot, gfn_t gfn,
-			 kvm_pfn_t *pfn, struct page *src_page, int order)
-{
-	struct gmem_info *info = to_gmem_info(file);
-	pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff;
-
-	if (index >= info->npages)
-		return -EINVAL;
-
-	*pfn = info->base_pfn + index;
+static void gmem_release(struct file *file);
+static int gmem_mmap(struct file *file, struct vm_area_struct *vma);
+static long gmem_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
 
-	if (src_page) {
-		void *dst, *src;
 
-		/* Map dst first: memremap() may sleep, kmap_local_page() must not. */
-		dst = gmem_map_pfn(*pfn);
-		if (!dst)
-			return -ENOMEM;
-		src = kmap_local_page(src_page);
-		memcpy(dst, src, PAGE_SIZE);
-		kunmap_local(src);
-		gmem_unmap_pfn(*pfn, dst);
-	}
-	return 0;
-}
-
-static int gmem_bind(struct file *file, struct kvm *kvm,
-		     struct kvm_memory_slot *slot, loff_t offset)
+static void gmem_release(struct file *file)
 {
 	struct gmem_info *info = to_gmem_info(file);
-	struct kvm *old_kvm = NULL;
-	unsigned long start = offset >> PAGE_SHIFT;
-
-	if (offset < 0 || !PAGE_ALIGNED(offset) ||
-	    start + slot->npages > info->npages)
-		return -EINVAL;
 
 	/*
-	 * An mmap-capable backing hands the VMM a host mapping onto pages
-	 * that a hardware-encrypted VM (SEV, SEV-ES, SEV-SNP, TDX) will mark
-	 * private in the RMP/EPT; the mmap can only ever fault on those
-	 * pages.  Refuse rather than hand the VMM a useless (and misleading)
-	 * shared view.  SW_PROTECTED_VM has no hardware encryption and is
-	 * fine.
+	 * Every exported dma-buf holds a reference on this file, so none can
+	 * be alive here: no device and no guest_memfd still maps our frames.
 	 */
-#ifdef CONFIG_X86
-	if (info->mmap_capable &&
-	    (kvm->arch.vm_type == KVM_X86_SEV_VM ||
-	     kvm->arch.vm_type == KVM_X86_SEV_ES_VM ||
-	     kvm->arch.vm_type == KVM_X86_SNP_VM ||
-	     kvm->arch.vm_type == KVM_X86_TDX_VM))
-		return -EACCES;
-#endif
+	WARN_ON(!list_empty(&info->dmabufs));
 
-	/* Record the binding so the revoke ioctl can translate offset -> gfn. */
-	mutex_lock(&info->lock);
-	info->base_gfn = slot->base_gfn;
-	info->pgoff = start;
-	info->bound_slot = slot;
-	mutex_unlock(&info->lock);
-
-#if IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)
 	/*
-	 * Reset the RMP for the range to 4K shared so a (new) SEV-SNP VM can
-	 * transition it to private and re-encrypt it.  PSMASH any 2M entries
-	 * first.  Harmless on non-SNP hosts, where these return -ENODEV.
+	 * A child returns its carved range to the root.  Pages it still owned
+	 * become free again; pages it had DONATEd stay parked at the root
+	 * (still in gmem_root.donated) until RECLAIM or module exit; pages
+	 * MOVEd out belong to another child and are not ours to free.
 	 */
-	{
+	if (info->owned) {
 		unsigned long i;
-		unsigned long first_pmd_pfn = ALIGN(info->base_pfn + start,
-						    PTRS_PER_PMD);
-
-		for (i = first_pmd_pfn - (info->base_pfn + start);
-		     i + PTRS_PER_PMD <= slot->npages;
-		     i += PTRS_PER_PMD)
-			psmash(info->base_pfn + start + i);
 
-		for (i = 0; i < slot->npages; i++) {
-			unsigned long pfn = info->base_pfn + start + i;
-			int ret = rmp_make_shared(pfn, PG_LEVEL_4K);
-
-			if (ret && ret != -ENODEV)
-				pr_info_once("gmem_provider: rmp_make_shared(0x%lx) = %d\n",
-					     pfn, ret);
-		}
+		mutex_lock(&gmem_root.lock);
+		list_del(&info->root_link);
+		for_each_set_bit(i, info->owned, info->npages)
+			__clear_bit(info->root_index + i, gmem_root.owned);
+		mutex_unlock(&gmem_root.lock);
+		kvfree(info->owned);
 	}
-#endif
-
-	/*
-	 * Claim (or, on re-bind, transfer) VM ownership; pins the VM.
-	 * kvm_put_kvm(old_kvm) MUST run outside info->lock: if the put
-	 * drops the last ref, kvm_destroy_vm() runs inline and calls back
-	 * into our gmem_unbind() (via kvm_gmem_unbind() on each memslot),
-	 * which needs info->lock -- taking it here would self-deadlock.
-	 */
-	mutex_lock(&info->lock);
-	if (info->kvm != kvm) {
-		old_kvm = info->kvm;
-		kvm_get_kvm(kvm);
-		info->kvm = kvm;
-	}
-	mutex_unlock(&info->lock);
-	if (old_kvm)
-		kvm_put_kvm(old_kvm);
-
-	/*
-	 * Record the file on the slot (KVM's outer bind no longer does this
-	 * for us) and mark the slot gmem-only if this backing serves host
-	 * accesses through its own mmap.
-	 */
-	WRITE_ONCE(slot->gmem.file, file);
-	slot->gmem.pgoff = start;
-	if (info->mmap_capable)
-		slot->flags |= KVM_MEMSLOT_GMEM_ONLY;
-
-	return 0;
+	if (info->cma_pages)
+		free_contig_range(info->base_pfn, info->npages);
+	kvfree(info->absent);
+	kvfree(info->readonly);
+	kfree(info);
+	module_put(THIS_MODULE);
 }
 
-static void gmem_unbind(struct file *file, struct kvm *kvm,
-			struct kvm_memory_slot *slot)
+static vm_fault_t gmem_vm_fault(struct vm_fault *vmf)
 {
-	struct gmem_info *info = to_gmem_info(file);
-
-	mutex_lock(&info->lock);
-	if (info->bound_slot == slot)
-		info->bound_slot = NULL;
-	mutex_unlock(&info->lock);
-
-#if IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)
-	{
-		unsigned long start = slot->gmem.pgoff;
-		unsigned long first_pmd_pfn = ALIGN(info->base_pfn + start,
-						    PTRS_PER_PMD);
-		unsigned long i;
-
-		/*
-		 * Symmetric with gmem_bind(): PSMASH any 2M RMP entries first
-		 * so rmp_make_shared(PG_LEVEL_4K) can succeed on SNP hosts.
-		 * Harmless on non-SNP: psmash() returns -ENODEV.
-		 */
-		for (i = first_pmd_pfn - (info->base_pfn + start);
-		     i + PTRS_PER_PMD <= slot->npages;
-		     i += PTRS_PER_PMD)
-			psmash(info->base_pfn + start + i);
+	/* The VMA's file is ours or a dma-buf's; the child rides in vm_private_data. */
+	struct gmem_info *info = vmf->vma->vm_private_data;
+	unsigned long index = vmf->pgoff;
 
-		for (i = 0; i < slot->npages; i++) {
-			unsigned long pfn = info->base_pfn + start + i;
-
-			rmp_make_shared(pfn, PG_LEVEL_4K);
-		}
-	}
-#endif
+	if (index >= info->npages || !gmem_page_present(info, index))
+		return VM_FAULT_SIGBUS;
+	return vmf_insert_pfn(vmf->vma, vmf->address, info->base_pfn + index);
 }
 
-static void gmem_release(struct file *file);
-static int gmem_mmap(struct file *file, struct vm_area_struct *vma);
-static long gmem_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
-
-static const struct kvm_gmem_ops gmem_ops = {
-	.bind		= gmem_bind,
-	.unbind		= gmem_unbind,
-	.get_pfn	= gmem_get_pfn,
-	.populate	= gmem_populate,
-	.release	= gmem_release,
-	.mmap		= gmem_mmap,
-	.ioctl		= gmem_fd_ioctl,
+static const struct vm_operations_struct gmem_vm_ops = {
+	.fault = gmem_vm_fault,
 };
 
-static void gmem_release(struct file *file)
-{
-	struct gmem_info *info = to_gmem_info(file);
-	struct kvm_memory_slot *slot;
-
-	/*
-	 * If a memslot is still bound at close time, KVM has not yet had a
-	 * chance to call ops->unbind.  Zap the guest mappings for the range
-	 * and clear slot->gmem.file so the eventual unbind is a no-op.  This
-	 * matches native gmem's kvm_gmem_release() and prevents the guest
-	 * from continuing to hit backing memory after we free it below.
-	 */
-	mutex_lock(&info->lock);
-	slot = info->bound_slot;
-	if (slot && info->kvm) {
-		kvm_gmem_invalidate_range(info->kvm, slot->base_gfn,
-					  slot->base_gfn + slot->npages);
-		WRITE_ONCE(slot->gmem.file, NULL);
-		info->bound_slot = NULL;
-	}
-	mutex_unlock(&info->lock);
-
-	if (info->kvm)
-		kvm_put_kvm(info->kvm);
-	if (info->cma_pages)
-		free_contig_range(info->base_pfn, info->npages);
-	kvfree(info->absent);
-	kfree(info);
-	module_put(THIS_MODULE);
-}
-
 static int gmem_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct gmem_info *info = to_gmem_info(file);
 	unsigned long npages = vma_pages(vma);
 
-	/*
-	 * gmem_bind() refuses to bind an mmap-capable fd to a coco VM.  A fd
-	 * that was created without GMEM_PROVIDER_FLAG_MMAP_CAPABLE has no
-	 * such gate at bind time, so its mmap must not succeed at any point.
-	 */
+	/* A fd created without GMEM_PROVIDER_FLAG_MMAP_CAPABLE is never host-mappable. */
 	if (!info->mmap_capable)
 		return -EPERM;
 
 	if (vma->vm_pgoff + npages > info->npages)
 		return -EINVAL;
 
-	/* Page-less backing: map raw PFNs, not folios. */
+	/*
+	 * Page-less backing: raw PFNs, inserted on fault, never at mmap()
+	 * time, so that a window torn down on a revoke comes back by itself
+	 * once the page is the child's again.
+	 */
 	vm_flags_set(vma, VM_PFNMAP | VM_IO | VM_DONTEXPAND | VM_DONTDUMP);
-	return remap_pfn_range(vma, vma->vm_start, info->base_pfn + vma->vm_pgoff,
-			       npages << PAGE_SHIFT, vma->vm_page_prot);
+	vma->vm_private_data = info;
+	vma->vm_ops = &gmem_vm_ops;
+	return 0;
 }
 
 /*
- * Dynamic dma-buf exporter over the provider's backing.
- *
- * Follows the same shape as drivers/vfio/pci/vfio_pci_dmabuf.c: a per-dmabuf
- * priv holding a phys_vec, a revocable dynamic attach, and a "private
- * interconnect" symbol iommufd looks up to fetch phys directly (instead of
- * mapping through the DMA API).
+ * The dma-buf a child exports.
  *
- * A revoke on the provider (SET_PRESENT present=0) fans out to every exported
- * dma-buf via dma_buf_invalidate_mappings(), so iommufd (which registered a
- * revocable importer) tears down the IOMMU mapping alongside KVM's NPT zap.
+ * A child may hand out several dma-bufs over its life (one per holder of the
+ * fd who asks); each covers the whole child and pins this file until it is
+ * released. Importers must accept ranged invalidation: every change to the
+ * child's pages is sent to them, and they re-read get_phys().
  */
 struct gmem_dmabuf {
 	struct dma_buf *dmabuf;
 	struct gmem_info *info;
 	struct file *provider_file;		/* holds info alive */
 	struct list_head list;			/* info->dmabufs */
-	struct phys_vec phys;			/* single contiguous range */
-	struct kref kref;
-	struct completion comp;
-	bool revoked;
 };
 
 static int gmem_dma_buf_attach(struct dma_buf *dmabuf,
 			       struct dma_buf_attachment *attach)
 {
-	struct gmem_dmabuf *priv = dmabuf->priv;
-
-	if (!attach->peer2peer)
-		return -EOPNOTSUPP;
-	if (priv->revoked)
-		return -ENODEV;
-	if (!dma_buf_attach_revocable(attach))
+	/* Only importers that can be told to re-read may attach. */
+	if (!attach->peer2peer || !dma_buf_attach_revocable(attach))
 		return -EOPNOTSUPP;
 	return 0;
 }
 
-static void gmem_dma_buf_done(struct kref *kref)
-{
-	struct gmem_dmabuf *priv = container_of(kref, struct gmem_dmabuf, kref);
-
-	complete(&priv->comp);
-}
-
 static struct sg_table *gmem_dma_buf_map(struct dma_buf_attachment *attach,
 					 enum dma_data_direction dir)
 {
-	struct gmem_dmabuf *priv = attach->dmabuf->priv;
-	struct sg_table *sgt;
-
-	dma_resv_assert_held(priv->dmabuf->resv);
-	if (priv->revoked)
-		return ERR_PTR(-ENODEV);
-
-	/* RAM, not P2P MMIO: no p2pdma_provider. */
-	sgt = dma_buf_phys_vec_to_sgt(attach, NULL, &priv->phys, 1,
-				      priv->phys.len, dir);
-	if (IS_ERR(sgt))
-		return sgt;
-
-	kref_get(&priv->kref);
-	return sgt;
+	/* DMA-API importers are not served; use get_phys() (iommufd does). */
+	return ERR_PTR(-EOPNOTSUPP);
 }
 
 static void gmem_dma_buf_unmap(struct dma_buf_attachment *attach,
-			       struct sg_table *sgt,
-			       enum dma_data_direction dir)
+			       struct sg_table *sgt, enum dma_data_direction dir)
 {
-	struct gmem_dmabuf *priv = attach->dmabuf->priv;
+}
+
+/* A host window through the dma-buf: the same rules as a window on the child. */
+static int gmem_dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
+{
+	struct gmem_dmabuf *priv = dmabuf->priv;
 
-	dma_resv_assert_held(priv->dmabuf->resv);
-	dma_buf_free_sgt(attach, sgt, dir);
-	kref_put(&priv->kref, gmem_dma_buf_done);
+	return gmem_mmap(priv->provider_file, vma);
 }
 
 static void gmem_dma_buf_release(struct dma_buf *dmabuf)
 {
 	struct gmem_dmabuf *priv = dmabuf->priv;
+	struct gmem_info *info = priv->info;
 
-	if (priv->info) {
-		mutex_lock(&priv->info->dmabufs_lock);
-		list_del_init(&priv->list);
-		mutex_unlock(&priv->info->dmabufs_lock);
-	}
-	if (priv->provider_file)
-		fput(priv->provider_file);
+	mutex_lock(&info->dmabufs_lock);
+	list_del(&priv->list);
+	mutex_unlock(&info->dmabufs_lock);
+	fput(priv->provider_file);
 	kfree(priv);
 }
 
-/* Report this flat sample region through the generic dma-buf operation. */
+/* Return the first bit whose value differs from @index. */
+static unsigned long gmem_bitmap_next_change(const unsigned long *bitmap,
+					     unsigned long nbits,
+					     unsigned long index)
+{
+	if (!bitmap)
+		return nbits;
+	if (test_bit(index, bitmap))
+		return find_next_zero_bit(bitmap, nbits, index + 1);
+	return find_next_bit(bitmap, nbits, index + 1);
+}
+
+/* Find a run with one present/read-only disposition, without a per-bit scan. */
+static unsigned long gmem_disposition_end(struct gmem_info *info,
+					  unsigned long index)
+{
+	bool present = gmem_page_present(info, index);
+	bool readonly = info->readonly && test_bit(index, info->readonly);
+	unsigned long pos = index;
+
+	for (;;) {
+		unsigned long next = info->npages;
+
+		next = min(next, gmem_bitmap_next_change(info->owned,
+						   info->npages, pos));
+		next = min(next, gmem_bitmap_next_change(info->absent,
+						   info->npages, pos));
+		next = min(next, gmem_bitmap_next_change(info->readonly,
+						   info->npages, pos));
+		if (next >= info->npages ||
+		    gmem_page_present(info, next) != present ||
+		    (!!(info->readonly && test_bit(next, info->readonly))) != readonly)
+			return next;
+		pos = next;
+	}
+}
+
+/*
+ * Describe the run at @offset from the child's ownership bitmaps. A present
+ * page extends to the next page whose presence or read-only state differs,
+ * found with bitmap searches. An absent page is not backed, unless scratch
+ * mode substitutes the shared read-only scratch frame for that one page.
+ * Scratch mode covers only children of the root: SET_SCRATCH invalidates
+ * those, so a SETUP provider must never report the scratch frame.
+ */
 static int gmem_dma_buf_get_phys(struct dma_buf_attachment *attach,
 				 u64 offset, u64 len,
 				 struct phys_vec *phys, u32 *attr)
 {
 	struct gmem_dmabuf *priv = attach->dmabuf->priv;
+	struct gmem_info *info = priv->info;
+	unsigned long index = offset >> PAGE_SHIFT;
+	unsigned long end, run_end;
 
-	dma_resv_assert_held(attach->dmabuf->resv);
-	if (priv->revoked)
-		return -ENODEV;
+	if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len))
+		return -EINVAL;
+	end = (offset + len) >> PAGE_SHIFT;
+
+	if (!gmem_page_present(info, index)) {
+		if (!info->owned || !READ_ONCE(gmem_root.scratch_enabled))
+			return -ENOENT;
+		phys->paddr = PFN_PHYS(gmem_scratch_pfn());
+		phys->len = PAGE_SIZE;
+		*attr = DMA_BUF_PHYS_ATTR_RAM | DMA_BUF_PHYS_ATTR_READONLY;
+		return 0;
+	}
 
-	phys->paddr = priv->phys.paddr + offset;
-	phys->len = len;
+	run_end = min(gmem_disposition_end(info, index), end);
+	phys->paddr = PFN_PHYS(info->base_pfn + index);
+	phys->len = (u64)(run_end - index) << PAGE_SHIFT;
 	*attr = DMA_BUF_PHYS_ATTR_RAM;
+	if (info->readonly && test_bit(index, info->readonly))
+		*attr |= DMA_BUF_PHYS_ATTR_READONLY;
 	return 0;
 }
 
@@ -510,23 +362,9 @@ static const struct dma_buf_ops gmem_dma_buf_ops = {
 	.unmap_dma_buf	= gmem_dma_buf_unmap,
 	.release	= gmem_dma_buf_release,
 	.get_phys	= gmem_dma_buf_get_phys,
+	.mmap		= gmem_dma_buf_mmap,
 };
 
-/* Called with info->dmabufs_lock held on the revoke path. */
-static void gmem_dma_buf_revoke_all(struct gmem_info *info)
-{
-	struct gmem_dmabuf *priv;
-
-	list_for_each_entry(priv, &info->dmabufs, list) {
-		dma_resv_lock(priv->dmabuf->resv, NULL);
-		if (!priv->revoked) {
-			priv->revoked = true;
-			dma_buf_invalidate_mappings(priv->dmabuf);
-		}
-		dma_resv_unlock(priv->dmabuf->resv);
-	}
-}
-
 static int gmem_provider_get_dmabuf(struct file *file)
 {
 	struct gmem_info *info = to_gmem_info(file);
@@ -537,27 +375,19 @@ static int gmem_provider_get_dmabuf(struct file *file)
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
-
 	priv->info = info;
 	priv->provider_file = get_file(file);
-	priv->phys.paddr = (u64)info->base_pfn << PAGE_SHIFT;
-	priv->phys.len = (u64)info->npages << PAGE_SHIFT;
-	kref_init(&priv->kref);
-	init_completion(&priv->comp);
-	INIT_LIST_HEAD(&priv->list);
-
 	exp_info.ops = &gmem_dma_buf_ops;
-	exp_info.size = priv->phys.len;
+	exp_info.size = (u64)info->npages << PAGE_SHIFT;
 	exp_info.flags = O_RDWR;
 	exp_info.priv = priv;
-
 	priv->dmabuf = dma_buf_export(&exp_info);
 	if (IS_ERR(priv->dmabuf)) {
 		fd = PTR_ERR(priv->dmabuf);
+		fput(priv->provider_file);
 		kfree(priv);
 		return fd;
 	}
-
 	mutex_lock(&info->dmabufs_lock);
 	list_add(&priv->list, &info->dmabufs);
 	mutex_unlock(&info->dmabufs_lock);
@@ -568,14 +398,132 @@ static int gmem_provider_get_dmabuf(struct file *file)
 	return fd;
 }
 
+/*
+ * Tell every importer of @info that [start, end) changed.  The caller has
+ * already updated the bitmaps and dropped info->lock: an importer re-reads
+ * get_phys() from inside this.
+ */
+static void gmem_revoke_range(struct gmem_info *info,
+			      unsigned long start, unsigned long end)
+{
+	struct gmem_dmabuf *priv;
+
+	lockdep_assert_not_held(&info->lock);
+	mutex_lock(&info->dmabufs_lock);
+	list_for_each_entry(priv, &info->dmabufs, list) {
+		dma_resv_lock(priv->dmabuf->resv, NULL);
+		dma_buf_invalidate_mappings_range(priv->dmabuf,
+						  (u64)start << PAGE_SHIFT,
+						  (u64)(end - start) << PAGE_SHIFT);
+		dma_resv_unlock(priv->dmabuf->resv);
+	}
+	mutex_unlock(&info->dmabufs_lock);
+}
+
+/*
+ * The child lost the frames behind [start, end): host windows over them go
+ * too.  Not for a read-only flip or a scratch toggle, where the VMM may be
+ * the page's writer; the VMM re-maps after a grant.
+ */
+static void gmem_unmap_host(struct gmem_info *info, unsigned long start,
+			    unsigned long end)
+{
+	loff_t off = (loff_t)start << PAGE_SHIFT, len = (loff_t)(end - start) << PAGE_SHIFT;
+	struct gmem_dmabuf *priv;
+
+	if (end <= start)
+		return;
+	unmap_mapping_range(info->mapping, off, len, 1);
+	/* Windows through a dma-buf live on the dma-buf's own file. */
+	mutex_lock(&info->dmabufs_lock);
+	list_for_each_entry(priv, &info->dmabufs, list)
+		unmap_mapping_range(priv->dmabuf->file->f_mapping, off, len, 1);
+	mutex_unlock(&info->dmabufs_lock);
+}
+
+/* Which allowlist bit gates each child-fd ioctl.  0 = not gated. */
+static u32 gmem_ioctl_allow_bit(unsigned int cmd)
+{
+	switch (cmd) {
+	case GMEM_PROVIDER_SET_PRESENT:	return GMEM_ALLOW_SET_PRESENT;
+	case GMEM_PROVIDER_SET_READONLY: return GMEM_ALLOW_SET_READONLY;
+	case GMEM_PROVIDER_GET_DMABUF:	return GMEM_ALLOW_GET_DMABUF;
+	case GMEM_PROVIDER_GET_STATS:	return GMEM_ALLOW_GET_STATS;
+	default:			return 0;
+	}
+}
+
+static long gmem_get_stats(struct gmem_info *info, void __user *uarg)
+{
+	struct gmem_provider_stats st = {};
+
+	mutex_lock(&info->lock);
+	st.region_offset = (u64)info->root_index << PAGE_SHIFT;
+	st.region_len = (u64)info->npages << PAGE_SHIFT;
+	st.owned_pages = info->owned ?
+		bitmap_weight(info->owned, info->npages) : info->npages;
+	st.absent_pages = info->absent ?
+		bitmap_weight(info->absent, info->npages) : 0;
+	st.readonly_pages = info->readonly ?
+		bitmap_weight(info->readonly, info->npages) : 0;
+	st.allow = info->allow;
+	mutex_unlock(&info->lock);
+
+	return copy_to_user(uarg, &st, sizeof(st)) ? -EFAULT : 0;
+}
+
 static long gmem_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 {
 	struct gmem_info *info = to_gmem_info(file);
 	struct gmem_provider_present p;
 	unsigned long start_index, end_index;
+	u32 need = gmem_ioctl_allow_bit(cmd);
+
+	/*
+	 * The allowlist is fixed by the creator at NEW_CHILD and checked here
+	 * before dispatch, so it also governs ioctls added later.  A SETUP
+	 * provider has no root and allows everything, as before.
+	 */
+	if (need && !(info->allow & need))
+		return -EPERM;
 
 	if (cmd == GMEM_PROVIDER_GET_DMABUF)
 		return gmem_provider_get_dmabuf(file);
+	if (cmd == GMEM_PROVIDER_GET_STATS)
+		return gmem_get_stats(info, (void __user *)arg);
+
+	if (cmd == GMEM_PROVIDER_SET_READONLY) {
+		struct gmem_provider_readonly r;
+
+		if (copy_from_user(&r, (void __user *)arg, sizeof(r)))
+			return -EFAULT;
+		if (!r.len || !PAGE_ALIGNED(r.offset) || !PAGE_ALIGNED(r.len) ||
+		    r.pad)
+			return -EINVAL;
+		start_index = r.offset >> PAGE_SHIFT;
+		end_index = start_index + (r.len >> PAGE_SHIFT);
+		if (end_index > info->npages || end_index < start_index)
+			return -EINVAL;
+
+		/*
+		 * Flip the bits, then invalidate the range so importers drop
+		 * their mappings and the next access re-reads get_phys() with
+		 * the new permission.  Making a range read-only must
+		 * tear down writable mappings; making it writable again is
+		 * also invalidated so a stale read-only mapping does not keep
+		 * exiting.
+		 */
+		mutex_lock(&info->lock);
+		if (r.readonly)
+			bitmap_set(info->readonly, start_index,
+				   end_index - start_index);
+		else
+			bitmap_clear(info->readonly, start_index,
+				     end_index - start_index);
+		mutex_unlock(&info->lock);
+		gmem_revoke_range(info, start_index, end_index);
+		return 0;
+	}
 
 	if (cmd != GMEM_PROVIDER_SET_PRESENT)
 		return -ENOTTY;
@@ -589,152 +537,495 @@ static long gmem_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg
 	if (end_index > info->npages || end_index < start_index)
 		return -EINVAL;
 
-	if (p.present) {
-		/* Restore: next guest fault calls get_pfn() and re-maps. */
-		mutex_lock(&info->lock);
+	mutex_lock(&info->lock);
+	if (p.present)
 		bitmap_clear(info->absent, start_index, end_index - start_index);
-		mutex_unlock(&info->lock);
-	} else {
-		unsigned long clamped_start;
-
-		/* Revoke: mark absent, then zap the guest NPT/EPT for the range. */
-		mutex_lock(&info->lock);
+	else
 		bitmap_set(info->absent, start_index, end_index - start_index);
+	/*
+	 * Both directions invalidate: on revoke so the guest and devices stop
+	 * using the pages, on restore so importers that were shown a hole or
+	 * the scratch frame re-read and map the real frames again.
+	 */
+	mutex_unlock(&info->lock);
+	gmem_revoke_range(info, start_index, end_index);
+	if (!p.present)
+		gmem_unmap_host(info, start_index, end_index);
+	return 0;
+}
 
-		/*
-		 * Translate provider offset -> guest gfn.  Skip any part of
-		 * the range that falls outside the currently bound slot; a
-		 * naive subtraction would underflow.
-		 */
-		clamped_start = max_t(unsigned long, start_index, info->pgoff);
-		if (info->kvm && end_index > clamped_start)
-			kvm_gmem_invalidate_range(info->kvm,
-				info->base_gfn + clamped_start - info->pgoff,
-				info->base_gfn + end_index - info->pgoff);
-		mutex_unlock(&info->lock);
-
-		/*
-		 * Fan out to iommufd (and any other dma-buf importer): mark the
-		 * exported dma-buf(s) revoked and invalidate any active mappings.
-		 * The provider stays ignorant of scratch-page policy; that lives
-		 * in the importer.
-		 */
-		mutex_lock(&info->dmabufs_lock);
-		gmem_dma_buf_revoke_all(info);
-		mutex_unlock(&info->dmabufs_lock);
-	}
+static int gmem_fops_release(struct inode *inode, struct file *file)
+{
+	gmem_release(file);
 	return 0;
 }
 
-static long gmem_ctl_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+/* A child file owns its controls, host windows and dma-buf exports. */
+static const struct file_operations gmem_provider_fops = {
+	.owner		= THIS_MODULE,
+	.release	= gmem_fops_release,
+	.mmap		= gmem_mmap,
+	.unlocked_ioctl	= gmem_fd_ioctl,
+	.compat_ioctl	= gmem_fd_ioctl,
+};
+
+/*
+ * Allocate a child over [base_pfn, +npages) and return its fd. Shared by
+ * SETUP (standalone region) and NEW_CHILD (a root sub-range). The fd owns the
+ * module pin on success.
+ */
+static int gmem_new_provider_fd(unsigned long base_pfn,
+				unsigned long npages, struct page *cma_pages,
+				u32 flags, unsigned long root_index,
+				unsigned long *owned, u32 allow,
+				struct gmem_info **out)
 {
-	struct gmem_provider_setup setup;
+	struct file *file;
 	struct gmem_info *info;
-	struct file *kvm_file;
-	struct page *pages = NULL;
-	struct kvm *kvm;
-	unsigned long npages;
 	int fd, ret;
 
-	if (cmd != GMEM_PROVIDER_SETUP)
-		return -ENOTTY;
+	info = kzalloc_obj(*info);
+	if (!info)
+		return -ENOMEM;
+	info->base_pfn = base_pfn;
+	info->npages = npages;
+	info->cma_pages = cma_pages;
+	info->root_index = root_index;
+	info->owned = owned;
+	info->allow = allow;
+	info->absent = kvzalloc_objs(unsigned long, BITS_TO_LONGS(npages));
+	info->readonly = kvzalloc_objs(unsigned long, BITS_TO_LONGS(npages));
+	if (!info->absent || !info->readonly) {
+		ret = -ENOMEM;
+		goto err_free;
+	}
+	INIT_LIST_HEAD(&info->dmabufs);
+	INIT_LIST_HEAD(&info->root_link);
+	mutex_init(&info->dmabufs_lock);
+	mutex_init(&info->lock);
 
-	if (copy_from_user(&setup, (void __user *)arg, sizeof(setup)))
-		return -EFAULT;
-	if (setup.flags & ~GMEM_PROVIDER_FLAG_MMAP_CAPABLE)
-		return -EINVAL;
+	/* Pin THIS_MODULE while the provider fd is alive (release drops it). */
+	if (!try_module_get(THIS_MODULE)) {
+		ret = -ENODEV;
+		goto err_free;
+	}
+	info->mmap_capable = !!(flags & GMEM_PROVIDER_FLAG_MMAP_CAPABLE);
 
-	kvm_file = fget(setup.kvm_fd);
-	if (!kvm_file)
-		return -EBADF;
-	if (!file_is_kvm(kvm_file)) {
-		fput(kvm_file);
-		return -EINVAL;
+	/*
+	 * A private inode, not the shared anon one: host windows are torn
+	 * down by file range on revoke, which must not touch other files.
+	 */
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0) {
+		ret = fd;
+		module_put(THIS_MODULE);
+		goto err_free;
 	}
-	kvm = kvm_file->private_data;
-	if (!kvm) {
-		fput(kvm_file);
-		return -EINVAL;
+	file = anon_inode_create_getfile("[gmem-provider]", &gmem_provider_fops,
+					 info, O_RDWR, NULL);
+	if (IS_ERR(file)) {
+		put_unused_fd(fd);
+		ret = PTR_ERR(file);
+		module_put(THIS_MODULE);
+		goto err_free;
 	}
-	kvm_get_kvm(kvm);
-	fput(kvm_file);
+	info->mapping = file->f_mapping;
+	if (owned)
+		list_add(&info->root_link, &gmem_root.children);
+	fd_install(fd, file);
+	if (out)
+		*out = info;
+	return fd;
 
-	info = kzalloc(sizeof(*info), GFP_KERNEL);
-	if (!info) {
-		ret = -ENOMEM;
-		goto err_put_kvm;
-	}
+err_free:
+	kvfree(info->absent);
+	kvfree(info->readonly);
+	kfree(info);
+	return ret;
+}
+
+static long gmem_ctl_setup(void __user *uarg)
+{
+	struct gmem_provider_setup setup;
+	struct page *pages = NULL;
+	unsigned long base_pfn, npages;
+	int fd;
+
+	if (copy_from_user(&setup, uarg, sizeof(setup)))
+		return -EFAULT;
+	if (setup.flags & ~GMEM_PROVIDER_FLAG_MMAP_CAPABLE)
+		return -EINVAL;
+	/* kvm_fd is legacy and ignored: binding happens in KVM_CREATE_GUEST_MEMFD. */
 
 	if (addr && len) {
 		/* External page-less range from module params. */
-		info->base_pfn = addr >> PAGE_SHIFT;
-		info->npages = len >> PAGE_SHIFT;
+		base_pfn = addr >> PAGE_SHIFT;
+		npages = len >> PAGE_SHIFT;
 	} else {
 		/* CMA fallback: allocate a contiguous, page-backed region. */
-		if (!setup.size || !PAGE_ALIGNED(setup.size)) {
-			ret = -EINVAL;
-			goto err_free_info;
-		}
+		if (!setup.size || !PAGE_ALIGNED(setup.size))
+			return -EINVAL;
 		npages = setup.size >> PAGE_SHIFT;
 		pages = alloc_contig_pages(npages, GFP_KERNEL, numa_node_id(), NULL);
-		if (!pages) {
-			ret = -ENOMEM;
-			goto err_free_info;
-		}
+		if (!pages)
+			return -ENOMEM;
 		/* Provider path skips KVM's folio-clear; zero to avoid data leak. */
 		memset(page_to_virt(pages), 0, (size_t)npages << PAGE_SHIFT);
-		info->base_pfn = page_to_pfn(pages);
-		info->npages = npages;
-		info->cma_pages = pages;
+		base_pfn = page_to_pfn(pages);
 	}
 
-	info->absent = kvzalloc(BITS_TO_LONGS(info->npages) * sizeof(unsigned long),
-				GFP_KERNEL);
-	if (!info->absent) {
-		ret = -ENOMEM;
-		goto err_free_pages;
+	fd = gmem_new_provider_fd(base_pfn, npages, pages, setup.flags, 0,
+				  NULL, GMEM_ALLOW_ALL, NULL);
+	if (fd < 0 && pages)
+		free_contig_range(page_to_pfn(pages), npages);
+	return fd;
+}
+
+/*
+ * Lazily create the root region on first NEW_CHILD.  Uses the module params
+ * if given (page-less), else a CMA region of @size bytes.  Idempotent once
+ * created; a later different @size is ignored.
+ */
+static int gmem_root_ensure(u64 size)
+{
+	unsigned long npages;
+	struct page *pages = NULL;
+
+	lockdep_assert_held(&gmem_root.lock);
+	if (gmem_root.npages)
+		return 0;
+
+	if (addr && len) {
+		gmem_root.base_pfn = addr >> PAGE_SHIFT;
+		npages = len >> PAGE_SHIFT;
+	} else {
+		if (!size || !PAGE_ALIGNED(size))
+			return -EINVAL;
+		npages = size >> PAGE_SHIFT;
+		pages = alloc_contig_pages(npages, GFP_KERNEL, numa_node_id(), NULL);
+		if (!pages)
+			return -ENOMEM;
+		memset(page_to_virt(pages), 0, (size_t)npages << PAGE_SHIFT);
+		gmem_root.base_pfn = page_to_pfn(pages);
 	}
+	gmem_root.owned = kvzalloc_objs(unsigned long, BITS_TO_LONGS(npages));
+	gmem_root.donated = kvzalloc_objs(unsigned long, BITS_TO_LONGS(npages));
+	if (!gmem_root.owned || !gmem_root.donated) {
+		kvfree(gmem_root.owned);
+		kvfree(gmem_root.donated);
+		gmem_root.owned = NULL;
+		gmem_root.donated = NULL;
+		if (pages)
+			free_contig_range(page_to_pfn(pages), npages);
+		return -ENOMEM;
+	}
+	gmem_root.cma_pages = pages;
+	gmem_root.npages = npages;
+	return 0;
+}
 
-	INIT_LIST_HEAD(&info->dmabufs);
-	mutex_init(&info->dmabufs_lock);
-	mutex_init(&info->lock);
+static void gmem_root_teardown(void)
+{
+	if (!gmem_root.npages)
+		return;
+	WARN_ON(!list_empty(&gmem_root.children));
+	if (gmem_root.cma_pages)
+		free_contig_range(gmem_root.base_pfn, gmem_root.npages);
+	kvfree(gmem_root.owned);
+	kvfree(gmem_root.donated);
+	memset(&gmem_root, 0, sizeof(gmem_root));
+	mutex_init(&gmem_root.lock);
+	INIT_LIST_HEAD(&gmem_root.children);
+}
+
+/* Validate a page-aligned [offset, +len) against the root; return page bounds. */
+static int gmem_root_range(u64 offset, u64 length,
+			   unsigned long *first, unsigned long *last)
+{
+	if (!length || !PAGE_ALIGNED(offset) || !PAGE_ALIGNED(length))
+		return -EINVAL;
+	*first = offset >> PAGE_SHIFT;
+	*last = *first + (length >> PAGE_SHIFT);		/* exclusive */
+	if (*last <= *first || *last > gmem_root.npages)
+		return -EINVAL;
+	return 0;
+}
+
+/* Resolve a child fd created by NEW_CHILD; returns a referenced file. */
+static struct file *gmem_get_child(int fd, struct gmem_info **infop)
+{
+	struct file *f = fget(fd);
+	struct gmem_info *info;
+
+	if (!f)
+		return ERR_PTR(-EBADF);
+	if (f->f_op != &gmem_provider_fops) {
+		fput(f);
+		return ERR_PTR(-EINVAL);
+	}
+	info = to_gmem_info(f);
+	if (!info->owned) {
+		fput(f);
+		return ERR_PTR(-EINVAL);
+	}
+	*infop = info;
+	return f;
+}
 
+static long gmem_ctl_new_child(void __user *uarg)
+{
+	struct gmem_provider_new_child nc;
+	struct gmem_info *info;
+	unsigned long first, last;
+	unsigned long *owned;
+	int fd, ret;
+
+	if (copy_from_user(&nc, uarg, sizeof(nc)))
+		return -EFAULT;
+	if ((nc.flags & ~GMEM_PROVIDER_FLAG_MMAP_CAPABLE) || nc.pad ||
+	    (nc.allow & ~GMEM_ALLOW_ALL))
+		return -EINVAL;
+
+	/* nc.kvm_fd is legacy and ignored: binding happens in KVM_CREATE_GUEST_MEMFD. */
+
+	mutex_lock(&gmem_root.lock);
+	ret = gmem_root_ensure(nc.offset + nc.len);
+	if (ret)
+		goto out_unlock;
+	ret = gmem_root_range(nc.offset, nc.len, &first, &last);
+	if (ret)
+		goto out_unlock;
 	/*
-	 * Pin THIS_MODULE while the provider fd is alive.  The fd is created
-	 * with kvm_gmem_fops (owned by kvm.ko), which does not pin us, so
-	 * rmmod of gmem_provider is otherwise free to run behind our ops.
+	 * A carve is the range this child may ever hold; it is the VM's whole
+	 * view of memory and becomes its memslot.  Carves may overlap: that is
+	 * how a range can later MOVE from one VM to another.  Ownership is
+	 * per page and exclusive.  The new child is granted every page of its
+	 * carve that no other child owns and the root has not parked; the
+	 * rest it can only receive by MOVE or RECLAIM.  A carve with nothing
+	 * to grant is refused as a likely mistake.
 	 */
-	if (!try_module_get(THIS_MODULE)) {
-		ret = -ENODEV;
-		goto err_free_pages;
+	if (find_next_zero_bit(gmem_root.owned, last, first) >= last) {
+		ret = -EBUSY;
+		goto out_unlock;
 	}
 
-	info->backing.ops = &gmem_ops;
-	info->kvm = kvm;
-	info->mmap_capable = !!(setup.flags & GMEM_PROVIDER_FLAG_MMAP_CAPABLE);
+	/*
+	 * Allocate the ownership bitmap before the fd exists, so a failure
+	 * here has nothing to unwind.  It is handed to the new info below.
+	 */
+	owned = kvzalloc_objs(unsigned long, BITS_TO_LONGS(last - first));
+	if (!owned) {
+		ret = -ENOMEM;
+		goto out_unlock;
+	}
+	/* Establish the grant before publishing the fd. */
+	{
+		unsigned long i;
 
-	fd = anon_inode_getfd("[gmem-provider]", &kvm_gmem_fops,
-			      &info->backing, O_RDWR | O_CLOEXEC);
+		for (i = first; i < last; i++) {
+			if (test_bit(i, gmem_root.owned) ||
+			    test_bit(i, gmem_root.donated))
+				continue;
+			__set_bit(i - first, owned);
+			__set_bit(i, gmem_root.owned);
+		}
+	}
+	fd = gmem_new_provider_fd(gmem_root.base_pfn + first,
+				  last - first, NULL, nc.flags, first,
+				  owned, nc.allow, &info);
 	if (fd < 0) {
+		unsigned long i;
+
+		for_each_set_bit(i, owned, last - first)
+			__clear_bit(first + i, gmem_root.owned);
+		kvfree(owned);
 		ret = fd;
-		goto err_module_put;
+		goto out_unlock;
 	}
+	mutex_unlock(&gmem_root.lock);
 	return fd;
 
-err_module_put:
-	module_put(THIS_MODULE);
+out_unlock:
+	mutex_unlock(&gmem_root.lock);
+	return ret;
+}
 
-err_free_pages:
-	if (pages)
-		free_contig_range(page_to_pfn(pages), npages);
-err_free_info:
-	kvfree(info->absent);
-	kfree(info);
-err_put_kvm:
-	kvm_put_kvm(kvm);
+/*
+ * Take [first, last) away from @info: clear ownership, then revoke from both
+ * of its importers.  Ownership is cleared before the revoke so a racing fault
+ * that slips in re-reads "not owned" and gets the scratch frame or fails.
+ * Caller holds gmem_root.lock; we take info->lock inside it.
+ */
+static void gmem_child_lose(struct gmem_info *info,
+			    unsigned long first, unsigned long last)
+{
+	unsigned long s = first - info->root_index, e = last - info->root_index;
+
+	mutex_lock(&info->lock);
+	bitmap_clear(info->owned, s, e - s);
+	mutex_unlock(&info->lock);
+	gmem_revoke_range(info, s, e);
+	gmem_unmap_host(info, s, e);
+	bitmap_clear(gmem_root.owned, first, last - first);
+}
+
+/* Give [first, last) to @info and invalidate so its importers pick it up. */
+static void gmem_child_gain(struct gmem_info *info,
+			    unsigned long first, unsigned long last)
+{
+	unsigned long s = first - info->root_index, e = last - info->root_index;
+
+	mutex_lock(&info->lock);
+	bitmap_set(info->owned, s, e - s);
+	mutex_unlock(&info->lock);
+	gmem_revoke_range(info, s, e);
+	bitmap_set(gmem_root.owned, first, last - first);
+}
+
+/* Does child @info's carved range contain [first, last)? */
+static bool gmem_child_covers(struct gmem_info *info,
+			      unsigned long first, unsigned long last)
+{
+	return first >= info->root_index &&
+	       last <= info->root_index + info->npages;
+}
+
+/* Does child @info currently own every page of [first, last)? */
+static bool gmem_child_owns(struct gmem_info *info,
+			    unsigned long first, unsigned long last)
+{
+	unsigned long s = first - info->root_index, e = last - info->root_index;
+
+	return gmem_child_covers(info, first, last) &&
+	       find_next_zero_bit(info->owned, e, s) >= e;
+}
+
+static long gmem_ctl_move(void __user *uarg)
+{
+	struct gmem_provider_move mv;
+	struct gmem_info *src, *dst;
+	struct file *sf, *df;
+	unsigned long first, last;
+	long ret;
+
+	if (copy_from_user(&mv, uarg, sizeof(mv)))
+		return -EFAULT;
+	sf = gmem_get_child(mv.src_fd, &src);
+	if (IS_ERR(sf))
+		return PTR_ERR(sf);
+	df = gmem_get_child(mv.dst_fd, &dst);
+	if (IS_ERR(df)) {
+		fput(sf);
+		return PTR_ERR(df);
+	}
+
+	mutex_lock(&gmem_root.lock);
+	ret = gmem_root_range(mv.offset, mv.len, &first, &last);
+	if (ret)
+		goto out;
+	if (src == dst || !gmem_child_owns(src, first, last) ||
+	    !gmem_child_covers(dst, first, last)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	/*
+	 * Revoke on the source first, grant on the destination second.  No two
+	 * importers hold the range at once.  Both children's info->lock are
+	 * taken in turn under gmem_root.lock, never nested with each other.
+	 */
+	gmem_child_lose(src, first, last);
+	gmem_child_gain(dst, first, last);
+	ret = 0;
+out:
+	mutex_unlock(&gmem_root.lock);
+	fput(df);
+	fput(sf);
+	return ret;
+}
+
+static long gmem_ctl_donate(void __user *uarg, bool reclaim)
+{
+	struct gmem_provider_donate d;
+	struct gmem_info *info;
+	struct file *f;
+	unsigned long first, last;
+	long ret;
+
+	if (copy_from_user(&d, uarg, sizeof(d)))
+		return -EFAULT;
+	if (d.pad)
+		return -EINVAL;
+	f = gmem_get_child(d.fd, &info);
+	if (IS_ERR(f))
+		return PTR_ERR(f);
+
+	mutex_lock(&gmem_root.lock);
+	ret = gmem_root_range(d.offset, d.len, &first, &last);
+	if (ret)
+		goto out;
+	if (!reclaim) {
+		/* DONATE: the child must own it; park it at the root. */
+		if (!gmem_child_owns(info, first, last)) {
+			ret = -EINVAL;
+			goto out;
+		}
+		gmem_child_lose(info, first, last);
+		bitmap_set(gmem_root.donated, first, last - first);
+	} else {
+		/* RECLAIM: must be donated and inside this child's range. */
+		if (!gmem_child_covers(info, first, last) ||
+		    find_next_zero_bit(gmem_root.donated, last, first) < last) {
+			ret = -EINVAL;
+			goto out;
+		}
+		bitmap_clear(gmem_root.donated, first, last - first);
+		gmem_child_gain(info, first, last);
+	}
+	ret = 0;
+out:
+	mutex_unlock(&gmem_root.lock);
+	fput(f);
 	return ret;
 }
 
+static long gmem_ctl_set_scratch(void __user *uarg)
+{
+	struct gmem_provider_scratch sc;
+	struct gmem_info *info;
+
+	if (copy_from_user(&sc, uarg, sizeof(sc)))
+		return -EFAULT;
+	if (sc.pad || sc.enable > 1)
+		return -EINVAL;
+
+	/*
+	 * Flipping the mode changes what every revoked page reports, so every
+	 * child is re-invalidated in full: cheap for a PoC, and it guarantees
+	 * no importer keeps a stale hole or a stale scratch mapping.
+	 */
+	mutex_lock(&gmem_root.lock);
+	WRITE_ONCE(gmem_root.scratch_enabled, !!sc.enable);
+	list_for_each_entry(info, &gmem_root.children, root_link)
+		gmem_revoke_range(info, 0, info->npages);
+	mutex_unlock(&gmem_root.lock);
+	return 0;
+}
+
+static long gmem_ctl_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+	void __user *uarg = (void __user *)arg;
+
+	switch (cmd) {
+	case GMEM_PROVIDER_SETUP:	return gmem_ctl_setup(uarg);
+	case GMEM_PROVIDER_NEW_CHILD:	return gmem_ctl_new_child(uarg);
+	case GMEM_PROVIDER_MOVE:	return gmem_ctl_move(uarg);
+	case GMEM_PROVIDER_DONATE:	return gmem_ctl_donate(uarg, false);
+	case GMEM_PROVIDER_RECLAIM:	return gmem_ctl_donate(uarg, true);
+	case GMEM_PROVIDER_SET_SCRATCH:	return gmem_ctl_set_scratch(uarg);
+	default:			return -ENOTTY;
+	}
+}
+
 static const struct file_operations gmem_ctl_fops = {
 	.owner		= THIS_MODULE,
 	.unlocked_ioctl	= gmem_ctl_ioctl,
@@ -749,20 +1040,34 @@ static struct miscdevice gmem_dev = {
 
 static int __init gmem_provider_init(void)
 {
+	int ret;
+
 	if ((addr || len) &&
 	    (!addr || !len || !PAGE_ALIGNED(addr) || !PAGE_ALIGNED(len))) {
 		pr_err("gmem_provider: addr= and len= must both be set and page aligned\n");
 		return -EINVAL;
 	}
-	return misc_register(&gmem_dev);
+	gmem_scratch_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	if (!gmem_scratch_page)
+		return -ENOMEM;
+	mutex_init(&gmem_root.lock);
+	INIT_LIST_HEAD(&gmem_root.children);
+
+	ret = misc_register(&gmem_dev);
+	if (ret)
+		__free_page(gmem_scratch_page);
+	return ret;
 }
 module_init(gmem_provider_init);
 
 static void __exit gmem_provider_exit(void)
 {
 	misc_deregister(&gmem_dev);
+	gmem_root_teardown();
+	__free_page(gmem_scratch_page);
 }
 module_exit(gmem_provider_exit);
 
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS("DMA_BUF");
 MODULE_DESCRIPTION("Sample guest_memfd provider (external page-less range or CMA fallback)");
diff --git a/samples/kvm/gmem_provider.h b/samples/kvm/gmem_provider.h
index 45f1b8257f60..1c8f5cacd2be 100644
--- a/samples/kvm/gmem_provider.h
+++ b/samples/kvm/gmem_provider.h
@@ -6,19 +6,18 @@
 #include <linux/types.h>
 
 /*
- * ioctl on /dev/gmem_provider: create a guest_memfd provider fd and return it
- * for use with KVM_SET_USER_MEMORY_REGION2.
+ * ioctl on /dev/gmem_provider: create a memory-owner fd and return it. The
+ * holder may mmap it and request a dma-buf for KVM or iommufd.
  *
  * If the module was loaded with addr=/len=, the fd is backed by that fixed
  * page-less physical range and @size is ignored.  Otherwise the fd is backed
  * by a @size-byte physically contiguous region from alloc_contig_pages().
  */
 /* Flags for struct gmem_provider_setup.flags */
-#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)	/* fd is mmap()-able; slot becomes gmem-only.
-								   Refused for coco VMs (SEV-SNP/TDX). */
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)	/* fd and its dma-bufs are mmap()-able */
 
 struct gmem_provider_setup {
-	__s32 kvm_fd;	/* an open KVM VM fd */
+	__s32 kvm_fd;	/* ignored */
 	__u32 flags;	/* GMEM_PROVIDER_FLAG_* */
 	__u64 size;	/* CMA fallback size in bytes, page aligned */
 };
@@ -27,10 +26,9 @@ struct gmem_provider_setup {
 #define GMEM_PROVIDER_SETUP		_IOW(GMEM_PROVIDER_IOCTL_BASE, 1, struct gmem_provider_setup)
 
 /*
- * ioctl on a provider fd (returned by SETUP): flip a byte range of the backing
- * between present and absent.  Revoking (present=0) marks the range absent and
- * zaps the guest's NPT/EPT so the next access re-faults; get_pfn() then refuses
- * the range until it is restored (present=1).  Models overcommit page reclaim.
+ * ioctl on an owner fd: flip a byte range between present and absent.
+ * Revoking a range invalidates every exported dma-buf, so importers discard
+ * their old mappings and observe a hole until the range is restored.
  */
 struct gmem_provider_present {
 	__u64 offset;	/* byte offset into the provider region, page aligned */
@@ -42,12 +40,134 @@ struct gmem_provider_present {
 #define GMEM_PROVIDER_SET_PRESENT	_IOW(GMEM_PROVIDER_IOCTL_BASE, 2, struct gmem_provider_present)
 
 /*
- * ioctl on a provider fd (returned by SETUP): export the backing region as a
- * dynamic dma-buf and return an fd for it, suitable for
+ * ioctl on a provider fd: make a byte range read-only for the guest, or
+ * writable again.  KVM maps a read-only page without write permission and a
+ * guest write to it exits to userspace with KVM_EXIT_MEMORY_FAULT.  Existing
+ * mappings of the range are dropped so the change takes effect on the next
+ * access.
+ */
+struct gmem_provider_readonly {
+	__u64 offset;	/* byte offset into the provider region, page aligned */
+	__u64 len;	/* byte length, page aligned */
+	__u32 readonly;	/* 1 = guest may not write, 0 = guest may write */
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_SET_READONLY \
+	_IOW(GMEM_PROVIDER_IOCTL_BASE, 4, struct gmem_provider_readonly)
+
+/*
+ * ioctl on an owner fd: export its memory as a dynamic dma-buf and return an
+ * fd suitable for
  * IOMMU_IOAS_MAP_FILE.  Revoking the region (SET_PRESENT present=0) fans out
- * to the exported dma-buf via dma_buf_invalidate_mappings(), causing iommufd
+ * to the exported dma-buf via dma_buf_invalidate_mappings_range(), causing importers
  * to tear down the IOMMU mapping so DMA to the reclaimed range faults.
  */
 #define GMEM_PROVIDER_GET_DMABUF	_IO(GMEM_PROVIDER_IOCTL_BASE, 3)
 
+/*
+ * ---- Toy descriptor tree (a memory owner as a provider) -------------------
+ *
+ * The control device owns one backing region (the "root").  NEW_CHILD carves a
+ * sub-range of it into a new provider fd bound to one VM.  A child's pages can
+ * be MOVEd to another child, DONATEd to the root (absent from every consumer)
+ * and RECLAIMed.  Every change revokes the range from KVM and from the
+ * child's exported dma-bufs.
+ *
+ * Every child fd carries an ioctl allowlist, fixed at NEW_CHILD by the
+ * creator.  Ioctls not in the list fail with -EPERM.
+ */
+
+/* Allowlist bits for struct gmem_provider_new_child.allow */
+#define GMEM_ALLOW_SET_PRESENT		(1u << 0)
+#define GMEM_ALLOW_SET_READONLY		(1u << 1)
+#define GMEM_ALLOW_GET_DMABUF		(1u << 2)
+#define GMEM_ALLOW_GET_STATS		(1u << 3)
+#define GMEM_ALLOW_ALL			(GMEM_ALLOW_SET_PRESENT | \
+					 GMEM_ALLOW_SET_READONLY | \
+					 GMEM_ALLOW_GET_DMABUF | \
+					 GMEM_ALLOW_GET_STATS)
+
+/*
+ * ioctl on /dev/gmem_provider: carve @len bytes at @offset of the root into a
+ * new child fd. The range must be available. Without addr=/len=, the first
+ * child fixes the CMA root size; create and close a sizing child first if
+ * later children extend beyond it. @allow becomes the child's ioctl allowlist.
+ */
+struct gmem_provider_new_child {
+	__s32 kvm_fd;	/* ignored */
+	__u32 flags;	/* GMEM_PROVIDER_FLAG_* */
+	__u64 offset;	/* byte offset into the root region, page aligned */
+	__u64 len;	/* byte length, page aligned */
+	__u32 allow;	/* GMEM_ALLOW_* */
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_NEW_CHILD	_IOW(GMEM_PROVIDER_IOCTL_BASE, 5, struct gmem_provider_new_child)
+
+/*
+ * ioctl on /dev/gmem_provider: move [@offset, @offset+@len) of the root region
+ * from child @src_fd to child @dst_fd.  The range must currently be owned by
+ * @src_fd and must fall inside @dst_fd's carved range. The source is
+ * invalidated first, then the destination is granted, so no two children own
+ * the range at once.
+ */
+struct gmem_provider_move {
+	__s32 src_fd;
+	__s32 dst_fd;
+	__u64 offset;	/* byte offset into the root region, page aligned */
+	__u64 len;	/* byte length, page aligned */
+};
+
+#define GMEM_PROVIDER_MOVE	_IOW(GMEM_PROVIDER_IOCTL_BASE, 6, struct gmem_provider_move)
+
+/*
+ * ioctl on /dev/gmem_provider: DONATE revokes [@offset, +@len) from the child
+ * that owns it and parks it at the root; RECLAIM returns a donated range to
+ * @fd, which must be the child whose carved range contains it.  The toy does
+ * no scrub and no hotplug; it models only the ownership state.
+ */
+struct gmem_provider_donate {
+	__s32 fd;	/* DONATE: owning child (checked); RECLAIM: recipient */
+	__u32 pad;
+	__u64 offset;	/* byte offset into the root region, page aligned */
+	__u64 len;	/* byte length, page aligned */
+};
+
+#define GMEM_PROVIDER_DONATE	_IOW(GMEM_PROVIDER_IOCTL_BASE, 7, struct gmem_provider_donate)
+#define GMEM_PROVIDER_RECLAIM	_IOW(GMEM_PROVIDER_IOCTL_BASE, 8, struct gmem_provider_donate)
+
+/*
+ * ioctl on /dev/gmem_provider: select what a revoked page reports to its
+ * consumers.  With @enable=0 (default) a revoked page is absent: get_phys()
+ * reports it as not backed, so a guest access exits and a device DMA faults.  With
+ * @enable=1 a revoked page reports the module's scratch frame, read-only, on
+ * both paths, so a device that cannot tolerate a fault lands on a harmless
+ * page.  Scratch mode applies to children created by NEW_CHILD only; a
+ * SETUP provider always reports a revoked page as absent.  Changing the mode
+ * re-invalidates every child so consumers re-read.
+ */
+struct gmem_provider_scratch {
+	__u32 enable;
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_SET_SCRATCH \
+	_IOW(GMEM_PROVIDER_IOCTL_BASE, 9, struct gmem_provider_scratch)
+
+/*
+ * ioctl on a child fd: read back the child's ownership state, for tests.
+ */
+struct gmem_provider_stats {
+	__u64 region_offset;	/* child's carved range within the root */
+	__u64 region_len;
+	__u64 owned_pages;	/* pages currently granted to this child */
+	__u64 absent_pages;	/* pages revoked (moved out, donated, or SET_PRESENT 0) */
+	__u64 readonly_pages;
+	__u32 allow;
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_GET_STATS	_IOR(GMEM_PROVIDER_IOCTL_BASE, 10, struct gmem_provider_stats)
+
 #endif /* _SAMPLES_KVM_GMEM_PROVIDER_H */
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 12004a487c32..6accbc56ae69 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -78,11 +78,12 @@ TEST_GEN_PROGS_x86 += x86/evmcs_smm_controls_test
 TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test
 TEST_GEN_PROGS_x86 += x86/fastops_test
 TEST_GEN_PROGS_x86 += x86/fix_hypercall_test
-TEST_GEN_PROGS_x86 += x86/gmem_provider_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_hugepage_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_revoke_test
+TEST_GEN_PROGS_x86 += x86/gmem_provider_readonly_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_iommufd_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_vfio_test
+TEST_GEN_PROGS_x86 += x86/gmem_poc_test
 TEST_GEN_PROGS_x86 += x86/hwcr_msr_test
 TEST_GEN_PROGS_x86 += x86/hyperv_clock
 TEST_GEN_PROGS_x86 += x86/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/gmem_provider_nvme_dma_test.c b/tools/testing/selftests/kvm/gmem_provider_nvme_dma_test.c
index 665b19028e13..b5ef6b9b1385 100644
--- a/tools/testing/selftests/kvm/gmem_provider_nvme_dma_test.c
+++ b/tools/testing/selftests/kvm/gmem_provider_nvme_dma_test.c
@@ -35,8 +35,8 @@ struct gmem_provider_setup {
 	__u64 size;
 };
 #define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
-#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
 #define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
 
 struct gmem_provider_present {
 	__u64 offset;
@@ -94,7 +94,7 @@ int main(void)
 	struct vfio_device_attach_iommufd_pt att = {};
 	struct vfio_region_info reg = {};
 	const char *cdev_path;
-	int gmem_ctl, gmem_fd, dmabuf_fd, iommufd_fd, vfio_fd;
+	int gmem_ctl, gmem_fd, dmabuf_fd, prov_fd, vm_fd = -1, iommufd_fd, vfio_fd;
 	void *provider_hva, *sq_buf, *cq_buf, *bar;
 	uint16_t pci_cmd;
 	uint16_t expected_vid;
@@ -120,22 +120,34 @@ int main(void)
 		vm = ioctl(kvm, KVM_CREATE_VM, 0);
 		TEST_ASSERT(vm >= 0, "KVM_CREATE_VM errno=%d", errno);
 		setup.kvm_fd = vm;
+		vm_fd = vm;
 		close(kvm);
 	}
 	setup.size = PROVIDER_SIZE;
-	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
-	TEST_ASSERT(gmem_fd >= 0, "SETUP errno=%d", errno);
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "SETUP errno=%d", errno);
+
+	/* Export once; KVM and iommufd import the same dma-buf. */
+	dmabuf_fd = ioctl(prov_fd, GMEM_PROVIDER_GET_DMABUF);
+	TEST_ASSERT(dmabuf_fd >= 0, "GET_DMABUF errno=%d", errno);
+	{
+		struct kvm_create_guest_memfd cgm = {
+			.size = PROVIDER_SIZE,
+			.flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_DMABUF,
+			.dmabuf_fd = dmabuf_fd,
+		};
+		gmem_fd = ioctl(vm_fd, KVM_CREATE_GUEST_MEMFD, &cgm);
+		TEST_ASSERT(gmem_fd >= 0, "KVM_CREATE_GUEST_MEMFD(dmabuf) errno=%d", errno);
+	}
 
 	provider_hva = mmap(NULL, PROVIDER_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
 			    gmem_fd, 0);
 	TEST_ASSERT(provider_hva != MAP_FAILED, "provider mmap errno=%d", errno);
 	memset(provider_hva, 0xcc, 0x1000);	/* poison first page */
 
-	/* IOAS + provider dma-buf. */
+	/* IOAS + the same dma-buf KVM imported. */
 	alloc.size = sizeof(alloc);
 	TEST_ASSERT(!ioctl(iommufd_fd, IOMMU_IOAS_ALLOC, &alloc), "IOAS_ALLOC");
-	dmabuf_fd = ioctl(gmem_fd, GMEM_PROVIDER_GET_DMABUF);
-	TEST_ASSERT(dmabuf_fd >= 0, "GET_DMABUF");
 	mapf.size = sizeof(mapf);
 	mapf.flags = IOMMU_IOAS_MAP_FIXED_IOVA | IOMMU_IOAS_MAP_READABLE |
 		     IOMMU_IOAS_MAP_WRITEABLE;
@@ -367,10 +379,10 @@ int main(void)
 
 	munmap(bar, reg.size);
 	close(vfio_fd);
-	close(dmabuf_fd);
 	close(iommufd_fd);
 	munmap(provider_hva, PROVIDER_SIZE);
 	close(gmem_fd);
+	close(dmabuf_fd);
 	close(gmem_ctl);
 	return 0;
 }
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 04a910164a29..c168c0880ebf 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -664,17 +664,38 @@ static inline bool is_smt_on(void)
 
 void vm_create_irqchip(struct kvm_vm *vm);
 
-static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
-					  u64 flags)
+static inline int __vm_create_guest_memfd_dmabuf(struct kvm_vm *vm,
+						 u64 size, u64 flags,
+						 int dmabuf_fd)
 {
 	struct kvm_create_guest_memfd guest_memfd = {
 		.size = size,
 		.flags = flags,
+		.dmabuf_fd = dmabuf_fd,
 	};
 
 	return __vm_ioctl(vm, KVM_CREATE_GUEST_MEMFD, &guest_memfd);
 }
 
+static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
+					  u64 flags)
+{
+	return __vm_create_guest_memfd_dmabuf(vm, size, flags, 0);
+}
+
+/* A guest_memfd that imports the dma-buf @dmabuf_fd for its memory. */
+static inline int vm_create_guest_memfd_dmabuf(struct kvm_vm *vm,
+					       u64 size, u64 flags,
+					       int dmabuf_fd)
+{
+	int fd = __vm_create_guest_memfd_dmabuf(vm, size,
+						  flags | GUEST_MEMFD_FLAG_USE_DMABUF,
+						  dmabuf_fd);
+
+	TEST_ASSERT(fd >= 0, KVM_IOCTL_ERROR(KVM_CREATE_GUEST_MEMFD, fd));
+	return fd;
+}
+
 static inline int vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
 					u64 flags)
 {
diff --git a/tools/testing/selftests/kvm/x86/gmem_poc_test.c b/tools/testing/selftests/kvm/x86/gmem_poc_test.c
new file mode 100644
index 000000000000..bdb64fe949ad
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/gmem_poc_test.c
@@ -0,0 +1,824 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * gmem_poc_test - one provider, a control process and two VMMs.
+ *
+ * Two roles in one process, kept apart on purpose:
+ *
+ *  ctl:  owns /dev/gmem_provider (the root).  Creates a child per VM with a
+ *        narrowed allowlist, MOVEs a range between children, DONATEs and
+ *        RECLAIMs, flips scratch mode.  Never touches a VM.
+ *  vmm:  owns one child fd, one KVM VM and one iommufd IOAS.  Creates the
+ *        VM's guest_memfd with the child as provider fd, binds it to the
+ *        memslot, and maps the guest_memfd's dma-buf into the IOAS on a
+ *        mock domain.  The provider revokes into guest_memfd; KVM and the
+ *        device follow.  The VMM never touches the control fd, and its
+ *        allowlist would stop it.
+ *
+ * guest_memfd sees one flat fd per VM; every relationship between fds lives
+ * in the provider.  The scenarios:
+ *
+ *  1. Launch:    each guest runs on its child and its write lands.
+ *  2. Move:      a range leaves A for B.  A faults on it and exits to its VMM
+ *                with KVM_EXIT_MEMORY_FAULT; A's device mapping of it is a
+ *                hole and the rest is intact; B reads what A wrote there.
+ *  3. Donate:    a range leaves A for the root and comes back on RECLAIM.
+ *  4. Read-only: a guest write to a read-only page exits; after clearing
+ *                the bit the same write lands.
+ *  5. Scratch:   with scratch mode on, A's donated range resolves in the
+ *                IOAS to one frame that is none of A's, and reads as zero.
+ *  6. Fragment: alternate scratch and owned pages in one 2 MiB block read
+ *                correctly and map at 4K; an untouched block maps at 2M.
+ *  7. Allowlist: B's fd cannot reach an ioctl the control process did not grant.
+ *  8. Window:    a VMM window mmap()ed through the guest_memfd fd over a
+ *                range the provider donates faults afterwards: the
+ *                revoke tore the host PTEs down too.
+ *
+ * A's guest_memfd is created while another thread flips SET_PRESENT and
+ * SET_READONLY on its child, so attachment races exporter invalidation.
+ *
+ * Runs on the iommufd mock domain: no device needed.  Requires
+ * /dev/gmem_provider (samples/kvm/gmem_provider.ko) and /dev/iommu.
+ */
+#include <fcntl.h>
+#include <errno.h>
+#include <setjmp.h>
+#include <pthread.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/iommufd.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+/*
+ * The iommufd mock domain's test ABI.  Only the three probes we need; the
+ * iommufd selftest helper header brings the kselftest harness and its own
+ * bitops with it, which do not coexist with the KVM selftest library.
+ */
+#include "../../../../../drivers/iommu/iommufd/iommufd_test.h"
+
+/* Mirrors samples/kvm/gmem_provider.h */
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
+#define GMEM_ALLOW_SET_PRESENT		(1u << 0)
+#define GMEM_ALLOW_SET_READONLY		(1u << 1)
+#define GMEM_ALLOW_GET_DMABUF		(1u << 2)
+#define GMEM_ALLOW_GET_STATS		(1u << 3)
+
+struct gmem_provider_present { __u64 offset, len; __u32 present, pad; };
+struct gmem_provider_readonly { __u64 offset, len; __u32 readonly, pad; };
+struct gmem_provider_new_child {
+	__s32 kvm_fd; __u32 flags; __u64 offset, len; __u32 allow, pad;
+};
+
+struct gmem_provider_move { __s32 src_fd, dst_fd; __u64 offset, len; };
+
+struct gmem_provider_donate { __s32 fd; __u32 pad; __u64 offset, len; };
+
+struct gmem_provider_scratch { __u32 enable, pad; };
+
+struct gmem_provider_stats {
+	__u64 region_offset, region_len, owned_pages, absent_pages, readonly_pages;
+	__u32 allow, pad;
+};
+
+#define GMEM_PROVIDER_SET_PRESENT	_IOW('G', 2, struct gmem_provider_present)
+#define GMEM_PROVIDER_SET_READONLY	_IOW('G', 4, struct gmem_provider_readonly)
+#define GMEM_PROVIDER_NEW_CHILD		_IOW('G', 5, struct gmem_provider_new_child)
+#define GMEM_PROVIDER_MOVE		_IOW('G', 6, struct gmem_provider_move)
+#define GMEM_PROVIDER_DONATE		_IOW('G', 7, struct gmem_provider_donate)
+#define GMEM_PROVIDER_RECLAIM		_IOW('G', 8, struct gmem_provider_donate)
+#define GMEM_PROVIDER_SET_SCRATCH	_IOW('G', 9, struct gmem_provider_scratch)
+#define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
+#define GMEM_PROVIDER_GET_STATS		_IOR('G', 10, struct gmem_provider_stats)
+
+#define PAGE		0x1000ULL
+#define CHILD_SIZE	0x800000ULL		/* 8 MiB per child */
+#define GPA		(1ULL << 32)		/* each VM maps its child here */
+#define IOVA		(1ULL << 28)		/* inside the mock domain's aperture */
+#define MAGIC_A		0xa11ce000a11ce000ULL
+#define MAGIC_B		0xb0bb0bb0b0bb0bb0ULL
+
+/*
+ * Layout of the root region.  A and B are carved so that they OVERLAP on
+ * [SHARED_OFF, +SHARED_LEN): a carve is the range a child may ever hold,
+ * ownership within it is per page.  the control process creates A owning its whole carve and
+ * B owning its carve minus the shared window, so the window starts with A
+ * and can MOVE to B.  This is the carve-out shape: two VMs, two fds,
+ * one range that changes hands, no shared object in guest_memfd.
+ */
+#define A_OFF		0ULL
+#define SHARED_LEN	(4 * PAGE)
+#define B_OFF		(CHILD_SIZE - SHARED_LEN)	/* B's carve begins at the window */
+#define SHARED_OFF	B_OFF				/* root offset of the window */
+#define ROOT_SIZE	(B_OFF + CHILD_SIZE)
+
+/* The shared window as each VM sees it. */
+#define A_SHARED_GPA	(GPA + (SHARED_OFF - A_OFF))
+#define B_SHARED_GPA	(GPA + (SHARED_OFF - B_OFF))
+#define A_SHARED_IOVA	(IOVA + (SHARED_OFF - A_OFF))
+
+/* A page of A's that is never moved, for read-only and control checks. */
+#define RO_OFF		(64 * PAGE)
+#define RO_GPA		(GPA + RO_OFF)
+
+/* A page B owns from the start (B's offset 0 is the window it lacks). */
+#define B_HOME_OFF	(CHILD_SIZE / 2)
+#define B_HOME_GPA	(GPA + B_HOME_OFF)
+
+#define FRAG_OFF	0ULL
+#define FRAG_LEN	0x200000ULL		/* one 2 MiB-aligned block */
+#define HUGE_OFF	0x200000ULL		/* untouched 2 MiB block */
+#define FRAG_MAGIC	0x5a5a5a5a5a5a5a5aULL
+
+/* ------------------------------------------------------------------------ */
+/* Guest: read args from a fixed GVA, act, report.                           */
+
+struct guest_args {
+	uint64_t write_gpa;	/* 0 = skip */
+	uint64_t write_val;
+	uint64_t read_gpa;	/* 0 = skip; value returned via GUEST_SYNC */
+	uint64_t scan_gpa;
+	uint64_t scan_pages;
+	uint64_t scan_value;
+};
+
+static void guest_code(struct guest_args *a)
+{
+	if (a->write_gpa)
+		*(volatile uint64_t *)a->write_gpa = a->write_val;
+	if (a->read_gpa)
+		GUEST_SYNC(*(volatile uint64_t *)a->read_gpa);
+	if (a->scan_pages) {
+		uint64_t i;
+
+		for (i = 0; i < a->scan_pages; i++) {
+			uint64_t value = *(volatile uint64_t *)(a->scan_gpa + i * PAGE);
+
+			GUEST_ASSERT_EQ(value, i & 1 ? a->scan_value : 0);
+		}
+	}
+	GUEST_DONE();
+}
+
+/* ------------------------------------------------------------------------ */
+/* the control process role: the only holder of the control fd.                              */
+
+struct vmm_control { int ctl; };
+
+static void ctl_open(struct vmm_control *c)
+{
+	c->ctl = open("/dev/gmem_provider", O_RDWR);
+	__TEST_REQUIRE(c->ctl >= 0, "gmem_provider not loaded");
+}
+
+static int ctl_new_child(struct vmm_control *c, int kvm_fd, uint64_t off, uint64_t len,
+			 uint32_t allow)
+{
+	struct gmem_provider_new_child nc = {
+		.kvm_fd = kvm_fd, .flags = GMEM_PROVIDER_FLAG_MMAP_CAPABLE,
+		.offset = off, .len = len, .allow = allow,
+	};
+	int fd = ioctl(c->ctl, GMEM_PROVIDER_NEW_CHILD, &nc);
+
+	TEST_ASSERT(fd >= 0, "NEW_CHILD(%#llx,%#llx) errno=%d",
+		    (unsigned long long)off, (unsigned long long)len, errno);
+	return fd;
+}
+
+static int ctl_move(struct vmm_control *c, int src, int dst, uint64_t off, uint64_t len)
+{
+	struct gmem_provider_move mv = { .src_fd = src, .dst_fd = dst,
+					 .offset = off, .len = len };
+
+	return ioctl(c->ctl, GMEM_PROVIDER_MOVE, &mv) ? -errno : 0;
+}
+
+static void ctl_donate(struct vmm_control *c, int fd, uint64_t off, uint64_t len, bool reclaim)
+{
+	struct gmem_provider_donate d = { .fd = fd, .offset = off, .len = len };
+
+	TEST_ASSERT(!ioctl(c->ctl, reclaim ? GMEM_PROVIDER_RECLAIM
+					   : GMEM_PROVIDER_DONATE, &d),
+		    "%s errno=%d", reclaim ? "RECLAIM" : "DONATE", errno);
+}
+
+static void ctl_scratch(struct vmm_control *c, bool on)
+{
+	struct gmem_provider_scratch sc = { .enable = on };
+
+	TEST_ASSERT(!ioctl(c->ctl, GMEM_PROVIDER_SET_SCRATCH, &sc),
+		    "SET_SCRATCH errno=%d", errno);
+}
+
+/* ------------------------------------------------------------------------ */
+/* VMM role: one child fd, one VM, one IOAS.                                 */
+
+struct vmm {
+	const char *name;
+	int child;			/* control fd: windows, stats, allowed ioctls */
+	int gmem;			/* the VM's memory object: memslot fd, window mmaps */
+	int dmabuf;			/* the child's dma-buf: what KVM and iommufd both import */
+	int iommufd;
+	uint32_t ioas, stdev, hwpt;
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	void *hva;			/* host window over the whole child */
+	gva_t args_gva;
+};
+
+static void vmm_create(struct vmm *v)
+{
+	struct vm_shape shape = { .mode = VM_MODE_DEFAULT,
+				  .type = KVM_X86_SW_PROTECTED_VM };
+
+	v->vm = vm_create_shape_with_one_vcpu(shape, &v->vcpu, guest_code);
+	v->args_gva = vm_alloc_page(v->vm);
+}
+
+static int mock_domain_create(struct vmm *v)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd), .op = IOMMU_TEST_OP_MOCK_DOMAIN, .id = v->ioas,
+	};
+
+	if (ioctl(v->iommufd, IOMMU_TEST_CMD, &cmd))
+		return -errno;
+	v->stdev = cmd.mock_domain.out_stdev_id;
+	v->hwpt = cmd.mock_domain.out_hwpt_id;
+	return 0;
+}
+
+static bool vmm_iova_mapped(struct vmm *v, uint64_t iova)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd), .op = IOMMU_TEST_OP_MD_CHECK_MAPPED, .id = v->hwpt,
+		.check_mapped = { .mapped = true, .iova = iova, .length = PAGE },
+	};
+
+	return !ioctl(v->iommufd, IOMMU_TEST_CMD, &cmd);
+}
+
+static uint64_t vmm_iova_phys(struct vmm *v, uint64_t iova)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd), .op = IOMMU_TEST_OP_MD_IOVA_TO_PHYS, .id = v->hwpt,
+		.iova_to_phys = { .iova = iova },
+	};
+
+	if (ioctl(v->iommufd, IOMMU_TEST_CMD, &cmd))
+		return 0;
+	return cmd.iova_to_phys.out_phys;
+}
+
+/*
+ * Whether the importer understands the full get_phys() contract: several
+ * ranges, holes, and a ranged invalidation it answers by re-reading the
+ * layout.  Without that, iommufd maps only a buffer whose layout is one
+ * range (-EOPNOTSUPP otherwise), and any change to the buffer costs the
+ * device its whole mapping for good.  Detected at attach time from B,
+ * whose child has a hole at launch; the device-plane checks below say
+ * what to expect either way.
+ */
+static bool ranged_import = true;
+
+/* Map [off, off+len) of the child into the IOAS at the matching IOVA. */
+static int vmm_map_range(struct vmm *v, uint64_t off, uint64_t len)
+{
+	struct iommu_ioas_map_file map = {
+		.size = sizeof(map),
+		.flags = IOMMU_IOAS_MAP_FIXED_IOVA | IOMMU_IOAS_MAP_READABLE |
+			 IOMMU_IOAS_MAP_WRITEABLE,
+		.ioas_id = v->ioas, .fd = v->dmabuf,
+		.start = off, .length = len, .iova = IOVA + off,
+	};
+
+	return ioctl(v->iommufd, IOMMU_IOAS_MAP_FILE, &map) ? -errno : 0;
+}
+
+/*
+ * the control process hands the VMM its child fd and the range it owns today.  The VMM
+ * creates the VM's guest_memfd from it (provider fd), binds the whole view
+ * to the memslot, and maps the owned part into the IOAS with
+ * IOMMU_IOAS_MAP_FILE on the guest_memfd's dma-buf.  iommufd refuses to map a range
+ * guest_memfd reports as a hole, so a VMM maps what it has and maps more
+ * when the control process tells it a range arrived (see scenario_move).
+ */
+struct create_race {
+	int child;
+	atomic_bool stop;
+	atomic_int error;
+};
+
+static void *create_race_thread(void *arg)
+{
+	struct create_race *race = arg;
+	struct gmem_provider_present present = { .offset = RO_OFF, .len = PAGE };
+	struct gmem_provider_readonly readonly = { .offset = RO_OFF, .len = PAGE };
+
+	while (!atomic_load(&race->stop)) {
+		present.present = 0;
+		if (ioctl(race->child, GMEM_PROVIDER_SET_PRESENT, &present))
+			break;
+		present.present = 1;
+		if (ioctl(race->child, GMEM_PROVIDER_SET_PRESENT, &present))
+			break;
+		readonly.readonly = 1;
+		if (ioctl(race->child, GMEM_PROVIDER_SET_READONLY, &readonly))
+			break;
+		readonly.readonly = 0;
+		if (ioctl(race->child, GMEM_PROVIDER_SET_READONLY, &readonly))
+			break;
+	}
+	if (!atomic_load(&race->stop))
+		atomic_store(&race->error, errno ?: EIO);
+	return NULL;
+}
+
+static void finish_create_race(struct create_race *race, pthread_t thread)
+{
+	struct gmem_provider_present present = {
+		.offset = RO_OFF, .len = PAGE, .present = 1,
+	};
+	struct gmem_provider_readonly readonly = {
+		.offset = RO_OFF, .len = PAGE, .readonly = 0,
+	};
+
+	atomic_store(&race->stop, true);
+	TEST_ASSERT(!pthread_join(thread, NULL), "pthread_join");
+	TEST_ASSERT(!atomic_load(&race->error), "create race errno=%d",
+		    atomic_load(&race->error));
+	TEST_ASSERT(!ioctl(race->child, GMEM_PROVIDER_SET_PRESENT, &present),
+		    "restore present errno=%d", errno);
+	TEST_ASSERT(!ioctl(race->child, GMEM_PROVIDER_SET_READONLY, &readonly),
+		    "restore writable errno=%d", errno);
+}
+
+static void vmm_attach(struct vmm *v, int child_fd, uint64_t own_off,
+		       uint64_t own_len, uint64_t gmem_flags, bool race_create)
+{
+	struct iommu_ioas_alloc alloc = { .size = sizeof(alloc) };
+	struct create_race race = { .child = child_fd };
+	void *reservation;
+	pthread_t thread;
+	int r;
+
+	v->child = child_fd;
+
+	/*
+	 * KVM leg: a guest_memfd whose memory the child provides.  The child
+	 * is the provider fd; guest_memfd is the VM's one memory object.  The
+	 * host window is the provider fd's mmap.
+	 */
+	v->dmabuf = ioctl(v->child, GMEM_PROVIDER_GET_DMABUF);
+	TEST_ASSERT(v->dmabuf >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	if (race_create)
+		TEST_ASSERT(!pthread_create(&thread, NULL, create_race_thread, &race),
+			    "pthread_create");
+	v->gmem = vm_create_guest_memfd_dmabuf(v->vm, CHILD_SIZE, gmem_flags,
+					       v->dmabuf);
+	if (race_create)
+		finish_create_race(&race, thread);
+	reservation = mmap(NULL, CHILD_SIZE + FRAG_LEN, PROT_NONE,
+			   MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
+	TEST_ASSERT(reservation != MAP_FAILED, "%s: reserve HVA errno=%d",
+		    v->name, errno);
+	v->hva = (void *)(((uintptr_t)reservation + FRAG_LEN - 1) &
+			  ~((uintptr_t)FRAG_LEN - 1));
+	v->hva = mmap(v->hva, CHILD_SIZE, PROT_READ | PROT_WRITE,
+		      MAP_SHARED | MAP_FIXED, v->child, 0);
+	TEST_ASSERT(v->hva != MAP_FAILED, "%s: mmap child errno=%d", v->name,
+		    errno);
+	r = __vm_set_user_memory_region2(v->vm, 10, KVM_MEM_GUEST_MEMFD, GPA,
+					 CHILD_SIZE, v->hva, v->gmem, 0);
+	TEST_ASSERT(!r, "%s: SET_USER_MEMORY_REGION2 r=%d errno=%d", v->name, r, errno);
+	virt_map(v->vm, GPA, GPA, CHILD_SIZE / PAGE);
+	/* Private, so every guest access goes through get_pfn(), not the HVA. */
+	vm_mem_set_private(v->vm, GPA, CHILD_SIZE);
+
+	/* DMA leg: the same guest_memfd, into an IOAS on a mock domain. */
+	v->iommufd = open("/dev/iommu", O_RDWR);
+	__TEST_REQUIRE(v->iommufd >= 0, "iommufd unavailable");
+	TEST_ASSERT(!ioctl(v->iommufd, IOMMU_IOAS_ALLOC, &alloc), "IOAS_ALLOC errno=%d", errno);
+	v->ioas = alloc.out_ioas_id;
+
+	/* A mock device on a mock domain, so the IOAS is really programmed. */
+	r = mock_domain_create(v);
+	TEST_ASSERT(!r, "%s: mock domain r=%d (CONFIG_IOMMUFD_TEST?)", v->name, r);
+
+	r = vmm_map_range(v, own_off, own_len);
+	if (r == -EOPNOTSUPP && own_off) {
+		/* B's layout has a hole at launch; only a ranged importer takes it. */
+		ranged_import = false;
+		pr_info("   iommufd maps one range; device checks expect full revoke\n");
+		return;
+	}
+	TEST_ASSERT(!r, "%s: IOAS_MAP_FILE(dma-buf) r=%d", v->name, r);
+}
+
+/*
+ * Continue the guest where it stopped.  KVM_RUN returns -EFAULT for
+ * KVM_EXIT_MEMORY_FAULT, which several scenarios expect, so do not assert
+ * on it here; the expect_*() helpers check the exit.
+ */
+static void vmm_resume(struct vmm *v)
+{
+	int r = _vcpu_run(v->vcpu);
+
+	TEST_ASSERT(!r || (errno == EFAULT &&
+			   v->vcpu->run->exit_reason == KVM_EXIT_MEMORY_FAULT),
+		    "%s: KVM_RUN r=%d errno=%d exit=%s", v->name, r, errno,
+		    exit_reason_str(v->vcpu->run->exit_reason));
+}
+
+/* Run the guest from the top with fresh args. */
+static void vmm_run(struct vmm *v, uint64_t write_gpa, uint64_t write_val,
+		    uint64_t read_gpa)
+{
+	struct guest_args *a = addr_gva2hva(v->vm, v->args_gva);
+
+	a->write_gpa = write_gpa;
+	a->write_val = write_val;
+	a->read_gpa = read_gpa;
+	a->scan_pages = 0;
+	vcpu_arch_set_entry_point(v->vcpu, guest_code);
+	vcpu_args_set(v->vcpu, 1, v->args_gva);
+	vmm_resume(v);
+}
+
+static void vmm_scan(struct vmm *v, uint64_t gpa, uint64_t pages,
+		     uint64_t value)
+{
+	struct guest_args *a = addr_gva2hva(v->vm, v->args_gva);
+
+	memset(a, 0, sizeof(*a));
+	a->scan_gpa = gpa;
+	a->scan_pages = pages;
+	a->scan_value = value;
+	vcpu_arch_set_entry_point(v->vcpu, guest_code);
+	vcpu_args_set(v->vcpu, 1, v->args_gva);
+	vmm_resume(v);
+}
+
+static void expect_done(struct vmm *v)
+{
+	struct kvm_run *run = v->vcpu->run;
+	struct ucall uc;
+
+	TEST_ASSERT(run->exit_reason != KVM_EXIT_MEMORY_FAULT,
+		    "%s: unexpected memory fault gpa=%#llx size=%#llx flags=%#llx",
+		    v->name, (unsigned long long)run->memory_fault.gpa,
+		    (unsigned long long)run->memory_fault.size,
+		    (unsigned long long)run->memory_fault.flags);
+	TEST_ASSERT(get_ucall(v->vcpu, &uc) == UCALL_DONE, "%s: guest exit %s",
+		    v->name, exit_reason_str(run->exit_reason));
+}
+
+static uint64_t expect_sync_then_done(struct vmm *v)
+{
+	struct ucall uc;
+	uint64_t val;
+
+	TEST_ASSERT(get_ucall(v->vcpu, &uc) == UCALL_SYNC, "%s: guest exit %s",
+		    v->name, exit_reason_str(v->vcpu->run->exit_reason));
+	val = uc.args[1];
+	vmm_resume(v);
+	expect_done(v);
+	return val;
+}
+
+static void expect_memory_fault(struct vmm *v, uint64_t gpa)
+{
+	struct kvm_run *run = v->vcpu->run;
+
+	TEST_ASSERT(run->exit_reason == KVM_EXIT_MEMORY_FAULT,
+		    "%s: want KVM_EXIT_MEMORY_FAULT, got %s", v->name,
+		    exit_reason_str(run->exit_reason));
+	TEST_ASSERT(run->memory_fault.gpa == gpa, "%s: fault gpa %#llx, want %#llx",
+		    v->name, (unsigned long long)run->memory_fault.gpa,
+		    (unsigned long long)gpa);
+}
+
+static void vmm_stats(struct vmm *v, struct gmem_provider_stats *st)
+{
+	TEST_ASSERT(!ioctl(v->child, GMEM_PROVIDER_GET_STATS, st),
+		    "%s: GET_STATS errno=%d", v->name, errno);
+}
+
+static void vmm_destroy(struct vmm *v)
+{
+	kvm_vm_free(v->vm);
+	close(v->gmem);
+	close(v->iommufd);
+	munmap(v->hva, CHILD_SIZE);
+	close(v->child);
+}
+
+/* ------------------------------------------------------------------------ */
+/* Scenarios.                                                                 */
+
+static void scenario_launch(struct vmm *a, struct vmm *b)
+{
+	pr_info("1. launch\n");
+	vmm_run(a, GPA, MAGIC_A, 0);
+	expect_done(a);
+	vmm_run(b, B_HOME_GPA, MAGIC_B, 0);
+	expect_done(b);
+	TEST_ASSERT(*(volatile uint64_t *)a->hva == MAGIC_A, "A's write missing");
+	TEST_ASSERT(*(volatile uint64_t *)(b->hva + B_HOME_OFF) == MAGIC_B, "B's write missing");
+	TEST_ASSERT(vmm_iova_mapped(a, IOVA) && vmm_iova_mapped(a, A_SHARED_IOVA),
+		    "A: IOAS not fully mapped after launch");
+}
+
+static void scenario_move(struct vmm_control *c, struct vmm *a, struct vmm *b)
+{
+	struct gmem_provider_stats st;
+	int r;
+
+	pr_info("2. move A -> B\n");
+
+	/* A writes into the window while it still owns it. */
+	vmm_run(a, A_SHARED_GPA, MAGIC_A, 0);
+	expect_done(a);
+
+	/* B does not own the window yet: a read faults out to B's VMM ... */
+	vmm_run(b, 0, 0, B_SHARED_GPA);
+	expect_memory_fault(b, B_SHARED_GPA);
+	/* ... and iommufd refuses to map a hole (or a layout with one in it). */
+	r = vmm_map_range(b, SHARED_OFF - B_OFF, SHARED_LEN);
+	TEST_ASSERT(r == (ranged_import ? -EFAULT : -EOPNOTSUPP),
+		    "B: mapping an unowned window returned %d", r);
+
+	/* A move outside the destination's carve is refused. */
+	r = ctl_move(c, a->child, b->child, RO_OFF, PAGE);
+	TEST_ASSERT(r == -EINVAL, "MOVE outside B's carve returned %d", r);
+
+	/* The real move. */
+	r = ctl_move(c, a->child, b->child, SHARED_OFF, SHARED_LEN);
+	TEST_ASSERT(!r, "MOVE returned %d", r);
+
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE - SHARED_LEN / PAGE,
+		    "A owns %llu pages after move", (unsigned long long)st.owned_pages);
+	vmm_stats(b, &st);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE,
+		    "B owns %llu pages after move", (unsigned long long)st.owned_pages);
+
+	/* A's device mapping: exactly the window is a hole, neighbours intact. */
+	TEST_ASSERT(!vmm_iova_mapped(a, A_SHARED_IOVA), "A: moved IOVA still mapped");
+	TEST_ASSERT(!vmm_iova_mapped(a, A_SHARED_IOVA + SHARED_LEN - PAGE),
+		    "A: last moved IOVA still mapped");
+	if (ranged_import) {
+		TEST_ASSERT(vmm_iova_mapped(a, A_SHARED_IOVA - PAGE), "A: page before window lost");
+		TEST_ASSERT(vmm_iova_mapped(a, IOVA), "A: base lost");
+	} else {
+		/* Whole-buffer revocation: the untouched pages went with the window. */
+		TEST_ASSERT(!vmm_iova_mapped(a, IOVA), "A: base survived a whole-buffer revoke");
+	}
+
+	/* A's guest: the window is gone. */
+	vmm_run(a, 0, 0, A_SHARED_GPA);
+	expect_memory_fault(a, A_SHARED_GPA);
+
+	/* B's guest: the window is here, and it holds what A wrote. */
+	vmm_run(b, 0, 0, B_SHARED_GPA);
+	TEST_ASSERT(expect_sync_then_done(b) == MAGIC_A, "B does not see A's write");
+
+	/*
+	 * B's device: the control process told B's VMM the window arrived; now
+	 * it maps.  B's layout is one range from here on, so this works with
+	 * either importer.
+	 */
+	r = vmm_map_range(b, SHARED_OFF - B_OFF, SHARED_LEN);
+	TEST_ASSERT(!r, "B: mapping the received window r=%d", r);
+	TEST_ASSERT(vmm_iova_mapped(b, IOVA + (SHARED_OFF - B_OFF)), "B: window not mapped");
+}
+
+static void scenario_donate(struct vmm_control *c, struct vmm *a)
+{
+	struct gmem_provider_stats st;
+	uint64_t off = A_OFF + 8 * PAGE, gpa = GPA + 8 * PAGE, iova = IOVA + 8 * PAGE;
+
+	pr_info("3. donate + reclaim\n");
+	ctl_donate(c, a->child, off, 2 * PAGE, false);
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE - SHARED_LEN / PAGE - 2,
+		    "A owns %llu after donate", (unsigned long long)st.owned_pages);
+	if (ranged_import) {
+		TEST_ASSERT(!vmm_iova_mapped(a, iova) && !vmm_iova_mapped(a, iova + PAGE),
+			    "A: donated IOVAs still mapped");
+		TEST_ASSERT(vmm_iova_mapped(a, iova - PAGE) && vmm_iova_mapped(a, iova + 2 * PAGE),
+			    "A: neighbours of donation lost");
+	}
+	vmm_run(a, 0, 0, gpa);
+	expect_memory_fault(a, gpa);
+
+	ctl_donate(c, a->child, off, 2 * PAGE, true);
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE - SHARED_LEN / PAGE,
+		    "A owns %llu after reclaim", (unsigned long long)st.owned_pages);
+	if (ranged_import)
+		TEST_ASSERT(vmm_iova_mapped(a, iova), "A: reclaimed IOVA not remapped");
+	vmm_run(a, 0, 0, gpa);
+	expect_sync_then_done(a);
+}
+
+static void scenario_readonly(struct vmm *a)
+{
+	struct gmem_provider_readonly ro = { .offset = RO_OFF, .len = PAGE, .readonly = 1 };
+
+	pr_info("4. read-only\n");
+	TEST_ASSERT(!ioctl(a->child, GMEM_PROVIDER_SET_READONLY, &ro),
+		    "SET_READONLY errno=%d", errno);
+
+	vmm_run(a, RO_GPA, MAGIC_A, 0);
+	expect_memory_fault(a, RO_GPA);
+	TEST_ASSERT(*(volatile uint64_t *)(a->hva + RO_OFF) != MAGIC_A, "RO write landed");
+
+	/* Clear the bit and let the guest retry the same instruction. */
+	ro.readonly = 0;
+	TEST_ASSERT(!ioctl(a->child, GMEM_PROVIDER_SET_READONLY, &ro), "clear RO errno=%d", errno);
+	vmm_resume(a);
+	expect_done(a);
+	TEST_ASSERT(*(volatile uint64_t *)(a->hva + RO_OFF) == MAGIC_A,
+		    "write after clearing RO did not land");
+}
+
+static void scenario_scratch(struct vmm_control *c, struct vmm *a)
+{
+	uint64_t off = A_OFF + 8 * PAGE, gpa = GPA + 8 * PAGE, iova = IOVA + 8 * PAGE;
+	uint64_t own0, own_before, p0, p1;
+
+	pr_info("5. scratch\n");
+	if (!ranged_import) {
+		pr_info("   skipped: needs the device mapping A lost in scenario 2\n");
+		return;
+	}
+	own0 = vmm_iova_phys(a, IOVA);
+	own_before = vmm_iova_phys(a, iova - PAGE);
+	TEST_ASSERT(own0 && own_before, "A: expected pages unmapped before scratch test");
+
+	ctl_scratch(c, true);
+	ctl_donate(c, a->child, off, 2 * PAGE, false);
+
+	p0 = vmm_iova_phys(a, iova);
+	p1 = vmm_iova_phys(a, iova + PAGE);
+	TEST_ASSERT(p0 && p0 == p1, "scratch: donated IOVAs -> %#llx, %#llx; want one frame",
+		    (unsigned long long)p0, (unsigned long long)p1);
+	TEST_ASSERT(p0 != own0 && p0 != own_before, "scratch frame is one of A's own");
+	TEST_ASSERT(vmm_iova_phys(a, IOVA) == own0, "scratch: unrelated IOVA changed");
+
+	/* The guest reads the scratch page: no fault, and it is zero. */
+	vmm_run(a, 0, 0, gpa);
+	TEST_ASSERT(expect_sync_then_done(a) == 0, "scratch page must read as zero");
+
+	ctl_donate(c, a->child, off, 2 * PAGE, true);
+	ctl_scratch(c, false);
+	TEST_ASSERT(vmm_iova_phys(a, iova) != p0, "after reclaim IOVA still on scratch");
+}
+
+static void scenario_fragmented(struct vmm_control *c, struct vmm *a)
+{
+	uint64_t p4k_before, p2m_before, p4k_after, p2m_after;
+	unsigned long i;
+
+	pr_info("6. fragmented layout\n");
+	for (i = 0; i < (FRAG_LEN / PAGE); i++)
+		*(uint64_t *)(a->hva + FRAG_OFF + i * PAGE) = FRAG_MAGIC;
+	*(uint64_t *)(a->hva + HUGE_OFF) = FRAG_MAGIC;
+	p4k_before = vm_get_stat(a->vm, pages_4k);
+	p2m_before = vm_get_stat(a->vm, pages_2m);
+	ctl_scratch(c, true);
+	for (i = 0; i < (FRAG_LEN / PAGE); i += 2)
+		ctl_donate(c, a->child, A_OFF + FRAG_OFF + i * PAGE, PAGE, false);
+
+	vmm_scan(a, GPA + FRAG_OFF, FRAG_LEN / PAGE, FRAG_MAGIC);
+	expect_done(a);
+	p4k_after = vm_get_stat(a->vm, pages_4k);
+	TEST_ASSERT(p4k_after > p4k_before,
+		    "fragmented block did not add 4K mappings");
+	/* The fragmented block's own 2M mapping was zapped; measure from here. */
+	p2m_before = vm_get_stat(a->vm, pages_2m);
+
+	vmm_run(a, 0, 0, GPA + HUGE_OFF);
+	TEST_ASSERT(expect_sync_then_done(a) == FRAG_MAGIC,
+		    "untouched block data changed");
+	p2m_after = vm_get_stat(a->vm, pages_2m);
+	pr_info("   fragmented levels: 4K %llu->%llu, 2M %llu->%llu\n",
+		(unsigned long long)p4k_before, (unsigned long long)p4k_after,
+		(unsigned long long)p2m_before, (unsigned long long)p2m_after);
+	TEST_ASSERT(p2m_after > p2m_before,
+		    "untouched block did not add a 2M mapping");
+
+	for (i = 0; i < (FRAG_LEN / PAGE); i += 2)
+		ctl_donate(c, a->child, A_OFF + FRAG_OFF + i * PAGE, PAGE, true);
+	ctl_scratch(c, false);
+}
+
+static void scenario_allowlist(struct vmm *b)
+{
+	struct gmem_provider_present p = { .offset = 0, .len = PAGE, .present = 0 };
+	struct gmem_provider_stats st;
+
+	pr_info("7. allowlist\n");
+	TEST_ASSERT(ioctl(b->child, GMEM_PROVIDER_SET_PRESENT, &p) && errno == EPERM,
+		    "B: SET_PRESENT must be denied");
+	vmm_stats(b, &st);
+	TEST_ASSERT(!(st.allow & GMEM_ALLOW_SET_PRESENT), "B allow=%#x", st.allow);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE, "B lost pages to a denied ioctl");
+}
+
+/* 8. A window through the gmem fd dies with the range it covers. */
+static sigjmp_buf window_jmp;
+static void window_sig(int sig)
+{
+	siglongjmp(window_jmp, sig);
+}
+
+static void scenario_window(struct vmm_control *c, struct vmm *a)
+{
+	uint64_t off = A_OFF + 8 * PAGE;
+	struct sigaction sa = { .sa_handler = window_sig }, old_bus, old_segv;
+	volatile uint64_t *win;
+	int sig;
+
+	pr_info("8. host window teardown\n");
+	win = mmap(NULL, PAGE, PROT_READ | PROT_WRITE, MAP_SHARED, a->gmem, off);
+	TEST_ASSERT(win != MAP_FAILED, "mmap(gmem, window) errno=%d", errno);
+	*win = MAGIC_A;
+	TEST_ASSERT(*(volatile uint64_t *)(a->hva + off) == MAGIC_A,
+		    "window write not visible through the child mapping");
+
+	ctl_donate(c, a->child, off, PAGE, false);
+
+	sigaction(SIGBUS, &sa, &old_bus);
+	sigaction(SIGSEGV, &sa, &old_segv);
+	sig = sigsetjmp(window_jmp, 1);
+	if (!sig) {
+		(void)*win;		/* must fault: the PTE was torn down */
+		sigaction(SIGBUS, &old_bus, NULL);
+		sigaction(SIGSEGV, &old_segv, NULL);
+		TEST_FAIL("window still readable after the range was donated");
+	}
+	sigaction(SIGBUS, &old_bus, NULL);
+	sigaction(SIGSEGV, &old_segv, NULL);
+	pr_info("   window access after donate: signal %d, as expected\n", sig);
+
+	ctl_donate(c, a->child, off, PAGE, true);
+	munmap((void *)win, PAGE);
+}
+
+int main(void)
+{
+	struct vmm_control ctl;
+	struct vmm a = { .name = "A" }, b = { .name = "B" };
+	int a_fd, b_fd, sizing_fd;
+
+	TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM));
+	ctl_open(&ctl);
+	vmm_create(&a);
+	vmm_create(&b);
+
+	/* Size a CMA-backed root for both overlapping child windows. */
+	sizing_fd = ctl_new_child(&ctl, a.vm->fd, 0, ROOT_SIZE, 0);
+	close(sizing_fd);
+
+	/*
+	 * The control process carves the root. A carve is a VM's whole view;
+	 * carves may overlap, but page ownership is exclusive. A is created
+	 * first and gets its full carve, including the shared window. B is
+	 * created second and gets only unowned pages, so its view has a hole
+	 * until the control process MOVEs the window over.
+	 * Neither VMM gets a management ioctl; those live on the control fd.
+	 */
+	a_fd = ctl_new_child(&ctl, a.vm->fd, A_OFF, CHILD_SIZE,
+			     GMEM_ALLOW_SET_PRESENT | GMEM_ALLOW_SET_READONLY |
+			     GMEM_ALLOW_GET_DMABUF |
+			     GMEM_ALLOW_GET_STATS);
+	b_fd = ctl_new_child(&ctl, b.vm->fd, B_OFF, CHILD_SIZE,
+			     GMEM_ALLOW_GET_DMABUF | GMEM_ALLOW_GET_STATS);
+	vmm_attach(&a, a_fd, 0, CHILD_SIZE, GUEST_MEMFD_FLAG_MMAP, true);
+	vmm_attach(&b, b_fd, SHARED_LEN, CHILD_SIZE - SHARED_LEN, 0, false);
+
+	scenario_launch(&a, &b);
+	scenario_move(&ctl, &a, &b);
+	scenario_donate(&ctl, &a);
+	scenario_readonly(&a);
+	scenario_scratch(&ctl, &a);
+	scenario_fragmented(&ctl, &a);
+	scenario_allowlist(&b);
+	scenario_window(&ctl, &a);
+
+	vmm_destroy(&a);
+	vmm_destroy(&b);
+	close(ctl.ctl);
+	pr_info("gmem_poc: all scenarios passed\n");
+	return 0;
+}
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_hugepage_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_hugepage_test.c
index fbbef9761e64..9c3bcfbe2cae 100644
--- a/tools/testing/selftests/kvm/x86/gmem_provider_hugepage_test.c
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_hugepage_test.c
@@ -36,8 +36,18 @@ struct gmem_provider_setup {
 	__u64 size;
 };
 #define GMEM_PROVIDER_SETUP _IOW('G', 1, struct gmem_provider_setup)
+#define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
 #define GMEM_PROVIDER_FLAG_MMAP_CAPABLE (1u << 0)
 
+/* The dma-buf for a sample-provider child: what KVM and iommufd both import. */
+static int child_dmabuf(int child_fd)
+{
+	int fd = ioctl(child_fd, GMEM_PROVIDER_GET_DMABUF);
+
+	TEST_ASSERT(fd >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	return fd;
+}
+
 #define DATA_SLOT	10
 #define DATA_GPA	(1ULL << 32)		/* 4G: 1G-aligned */
 #define DATA_SIZE	((uint64_t)SZ_1G + SZ_2M)
@@ -59,7 +69,8 @@ int main(void)
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
 	struct ucall uc;
-	int gmem_ctl, gmem_fd, r;
+	int gmem_ctl, gmem_fd, dmabuf_fd, r;
+	int prov_fd;
 	void *resv, *hva;
 	uint64_t p4k, p2m, p1g;
 
@@ -73,8 +84,17 @@ int main(void)
 
 	setup.kvm_fd = vm->fd;
 	setup.size = DATA_SIZE;
-	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
-	TEST_ASSERT(gmem_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+
+	/*
+	 * The provider fd is not a guest_memfd; it is the provider fd.  The
+	 * guest_memfd KVM returns is what the memslot binds, what the host
+	 * mmaps (through the provider's mmap), and what iommufd would map.
+	 */
+	dmabuf_fd = child_dmabuf(prov_fd);
+	gmem_fd = vm_create_guest_memfd_dmabuf(vm, DATA_SIZE,
+					       GUEST_MEMFD_FLAG_MMAP, dmabuf_fd);
 
 	/*
 	 * Map the provider at a 1G-aligned host VA so the slot's userspace_addr
@@ -86,7 +106,7 @@ int main(void)
 	TEST_ASSERT(resv != MAP_FAILED, "reserve VA failed, errno %d", errno);
 	hva = (void *)(((uintptr_t)resv + SZ_1G - 1) & ~((uintptr_t)SZ_1G - 1));
 	hva = mmap(hva, DATA_SIZE, PROT_READ | PROT_WRITE,
-		   MAP_SHARED | MAP_FIXED, gmem_fd, 0);
+		   MAP_SHARED | MAP_FIXED, prov_fd, 0);
 	TEST_ASSERT(hva != MAP_FAILED,
 		    "mmap(provider) failed, errno %d", errno);
 
@@ -125,6 +145,8 @@ int main(void)
 	kvm_vm_free(vm);
 	munmap(hva, DATA_SIZE);
 	close(gmem_fd);
+	close(dmabuf_fd);
+	close(prov_fd);
 	close(gmem_ctl);
 	return 0;
 }
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_iommufd_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_iommufd_test.c
index 885dffa0b659..b5f4afae5383 100644
--- a/tools/testing/selftests/kvm/x86/gmem_provider_iommufd_test.c
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_iommufd_test.c
@@ -6,8 +6,8 @@
  * write is visible via the host mmap.
  *
  * The iommufd side exercises exactly the provider->iommufd path we just wired:
- * GET_DMABUF on the provider fd -> IOMMU_IOAS_MAP_FILE, which walks
- * iopt_map_dmabuf -> sym_..._iommufd_map -> gmem_provider_dma_buf_iommufd_map.
+ * GMEM_PROVIDER_GET_DMABUF on the child -> IOMMU_IOAS_MAP_FILE, which walks
+ * iopt_map_dmabuf -> dma_buf_get_phys -> the provider's get_phys op.
  *
  * The test opens the provider with GMEM_PROVIDER_FLAG_MMAP_CAPABLE at SETUP time; requires iommufd
  * available at /dev/iommu.  Actual IOMMU page-table programming happens once
@@ -36,8 +36,17 @@ struct gmem_provider_setup {
 	__u64 size;
 };
 #define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
-#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
 #define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
+
+/* The dma-buf for a sample-provider child: what KVM and iommufd both import. */
+static int child_dmabuf(int child_fd)
+{
+	int fd = ioctl(child_fd, GMEM_PROVIDER_GET_DMABUF);
+
+	TEST_ASSERT(fd >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	return fd;
+}
 
 #define DATA_SLOT	10
 #define DATA_GPA	(1ULL << 32)
@@ -64,6 +73,7 @@ int main(void)
 	struct kvm_vm *vm;
 	struct ucall uc;
 	int gmem_ctl, gmem_fd, dmabuf_fd, iommufd, r;
+	int prov_fd;
 	void *hva;
 
 	TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM));
@@ -80,13 +90,23 @@ int main(void)
 	vm = vm_create_shape_with_one_vcpu(shape, &vcpu, guest_code);
 	setup.kvm_fd = vm->fd;
 	setup.size = DATA_SIZE;
-	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
-	TEST_ASSERT(gmem_fd >= 0, "GMEM_PROVIDER_SETUP failed errno=%d", errno);
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "GMEM_PROVIDER_SETUP failed errno=%d", errno);
+
+	/*
+	 * The provider fd is not a guest_memfd.  Export its dma-buf and hand
+	 * that to KVM as the provider fd; the guest_memfd KVM returns is what
+	 * the memslot binds and what the host mmaps (mmap goes through the
+	 * exporter).  iommufd would import the very same dma-buf.
+	 */
+	dmabuf_fd = child_dmabuf(prov_fd);
+	gmem_fd = vm_create_guest_memfd_dmabuf(vm, DATA_SIZE,
+					       GUEST_MEMFD_FLAG_MMAP, dmabuf_fd);
 
 	/*
 	 * 2) KVM side: host mmap + guest_memfd memslot (gmem-only when mmap-capable).
 	 */
-	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, gmem_fd, 0);
+	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, prov_fd, 0);
 	TEST_ASSERT(hva != MAP_FAILED,
 		    "provider mmap failed errno=%d", errno);
 	r = __vm_set_user_memory_region2(vm, DATA_SLOT, KVM_MEM_GUEST_MEMFD,
@@ -105,16 +125,15 @@ int main(void)
 	/*
 	 * 3) iommufd side: allocate IOAS, get a dma-buf from the SAME provider fd,
 	 *    and map it into the IOAS via IOMMU_IOAS_MAP_FILE.  This drives
-	 *    iopt_map_dmabuf -> gmem_provider_dma_buf_iommufd_map.
+	 *    iopt_map_dmabuf -> dma_buf_get_phys -> the get_phys op.
 	 */
 	alloc.size = sizeof(alloc);
 	r = ioctl(iommufd, IOMMU_IOAS_ALLOC, &alloc);
 	TEST_ASSERT(!r, "IOMMU_IOAS_ALLOC failed errno=%d", errno);
 	pr_info("iommufd: allocated ioas id=%u\n", alloc.out_ioas_id);
 
-	dmabuf_fd = ioctl(gmem_fd, GMEM_PROVIDER_GET_DMABUF);
-	TEST_ASSERT(dmabuf_fd >= 0, "GMEM_PROVIDER_GET_DMABUF failed errno=%d", errno);
-	pr_info("provider: exported dma-buf fd=%d\n", dmabuf_fd);
+	/* IOMMU_IOAS_MAP_FILE takes the same dma-buf KVM imported. */
+	pr_info("guest_memfd fd=%d imports dma-buf fd=%d\n", gmem_fd, dmabuf_fd);
 
 	map.size = sizeof(map);
 	map.flags = IOMMU_IOAS_MAP_FIXED_IOVA |
@@ -127,7 +146,7 @@ int main(void)
 	r = ioctl(iommufd, IOMMU_IOAS_MAP_FILE, &map);
 	TEST_ASSERT(!r,
 		    "IOMMU_IOAS_MAP_FILE(dma-buf) failed r=%d errno=%d\n"
-		    "	 (gmem_provider_dma_buf_iommufd_map path)",
+		    "	 (dma_buf_get_phys path)",
 		    r, errno);
 	pr_info("iommufd: mapped provider dma-buf @ IOVA 0x%llx (0x%llx bytes)\n",
 		(unsigned long long)map.iova, (unsigned long long)map.length);
@@ -153,6 +172,7 @@ int main(void)
 	close(iommufd);
 	munmap(hva, DATA_SIZE);
 	close(gmem_fd);
+	close(prov_fd);
 	close(gmem_ctl);
 	return 0;
 }
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c
new file mode 100644
index 000000000000..a1b8b7ba5640
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c
@@ -0,0 +1,172 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * gmem_provider_readonly_test - exercise per-range read-only from a provider.
+ *
+ * Marks a provider-backed page read-only via an ioctl on the provider fd and
+ * checks that KVM honours the provider's answer: the guest can still read the
+ * page, a guest write exits to userspace with KVM_EXIT_MEMORY_FAULT rather than
+ * landing, and clearing the bit lets the write through. This is the mechanism
+ * a hypervisor uses to protect a page it shares with the guest, such as a
+ * information page a helper VM reads, without giving up the mapping.
+ *
+ * The test opens the provider with GMEM_PROVIDER_FLAG_MMAP_CAPABLE at SETUP time
+ * (gmem-only). Load the module with a backing region of at least DATA_SIZE.
+ */
+#include <fcntl.h>
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+/* Mirrors samples/kvm/gmem_provider.h */
+struct gmem_provider_setup {
+	__s32 kvm_fd;
+	__u32 flags;
+	__u64 size;
+};
+
+struct gmem_provider_readonly {
+	__u64 offset;
+	__u64 len;
+	__u32 readonly;
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
+#define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
+#define GMEM_PROVIDER_SET_READONLY	_IOW('G', 4, struct gmem_provider_readonly)
+
+/* The dma-buf for a sample-provider child: what KVM and iommufd both import. */
+static int child_dmabuf(int child_fd)
+{
+	int fd = ioctl(child_fd, GMEM_PROVIDER_GET_DMABUF);
+
+	TEST_ASSERT(fd >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	return fd;
+}
+
+#define DATA_SLOT	10
+#define DATA_GPA	(1ULL << 32)
+#define DATA_SIZE	0x200000ULL		/* 2 MiB region */
+#define MAGIC		0x1234abcdULL
+#define MAGIC2		0xfeedf00dULL
+
+/*
+ * Phase 1: read the page and report it.
+ * Phase 2: write to it.  With the page read-only this never returns to the
+ *          guest until userspace clears the bit; then it completes and the
+ *          guest reports what it wrote.
+ */
+static void guest_code(void)
+{
+	GUEST_SYNC(*(volatile uint64_t *)DATA_GPA);
+	*(volatile uint64_t *)DATA_GPA = MAGIC2;
+	GUEST_SYNC(*(volatile uint64_t *)DATA_GPA);
+	GUEST_DONE();
+}
+
+int main(void)
+{
+	struct vm_shape shape = {
+		.mode = VM_MODE_DEFAULT,
+		.type = KVM_X86_SW_PROTECTED_VM,
+	};
+	struct gmem_provider_setup setup = { .flags = GMEM_PROVIDER_FLAG_MMAP_CAPABLE };
+	struct gmem_provider_readonly req;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	int gmem_ctl, gmem_fd, dmabuf_fd, r;
+	int prov_fd;
+	void *hva;
+
+	TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM));
+
+	gmem_ctl = open("/dev/gmem_provider", O_RDWR);
+	__TEST_REQUIRE(gmem_ctl >= 0,
+		       "gmem_provider module not loaded (/dev/gmem_provider absent)");
+
+	vm = vm_create_shape_with_one_vcpu(shape, &vcpu, guest_code);
+
+	setup.kvm_fd = vm->fd;
+	setup.size = DATA_SIZE;
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+
+	/*
+	 * The provider fd is not a guest_memfd; it is the provider fd.  The
+	 * guest_memfd KVM returns is what the memslot binds, what the host
+	 * mmaps (through the provider's mmap), and what iommufd would map.
+	 */
+	dmabuf_fd = child_dmabuf(prov_fd);
+	gmem_fd = vm_create_guest_memfd_dmabuf(vm, DATA_SIZE,
+					       GUEST_MEMFD_FLAG_MMAP, dmabuf_fd);
+
+	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, prov_fd, 0);
+	TEST_ASSERT(hva != MAP_FAILED, "mmap(provider) failed, errno %d", errno);
+
+	r = __vm_set_user_memory_region2(vm, DATA_SLOT, KVM_MEM_GUEST_MEMFD,
+					 DATA_GPA, DATA_SIZE, hva, gmem_fd, 0);
+	TEST_ASSERT(!r, "KVM_SET_USER_MEMORY_REGION2 failed: %d errno %d", r, errno);
+	virt_map(vm, DATA_GPA, DATA_GPA, 1);
+
+	/* Seed the page from the host before the guest ever touches it. */
+	*(volatile uint64_t *)hva = MAGIC;
+
+	/* 1) Make the page read-only for the guest. */
+	req = (struct gmem_provider_readonly){ .offset = 0, .len = 4096, .readonly = 1 };
+	r = ioctl(prov_fd, GMEM_PROVIDER_SET_READONLY, &req);
+	TEST_ASSERT(!r, "set readonly ioctl failed, errno %d", errno);
+
+	/* 2) Guest read must still work and see the host's value. */
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC, "expected UCALL_SYNC");
+	TEST_ASSERT(uc.args[1] == MAGIC, "guest read 0x%lx, want MAGIC",
+		    (unsigned long)uc.args[1]);
+	pr_info("read-only: guest read 0x%llx\n", MAGIC);
+
+	/* 3) Guest write must exit to userspace, not land. */
+	r = _vcpu_run(vcpu);
+	TEST_ASSERT(r == -1 && errno == EFAULT &&
+		    vcpu->run->exit_reason == KVM_EXIT_MEMORY_FAULT,
+		    "read-only write: expected KVM_EXIT_MEMORY_FAULT (r=%d errno=%d exit_reason=%u %s)",
+		    r, errno, vcpu->run->exit_reason,
+		    exit_reason_str(vcpu->run->exit_reason));
+	TEST_ASSERT(*(volatile uint64_t *)hva == MAGIC,
+		    "guest write landed on a read-only page: host sees 0x%lx",
+		    (unsigned long)*(volatile uint64_t *)hva);
+	pr_info("read-only: guest write exited with KVM_EXIT_MEMORY_FAULT, page unchanged\n");
+
+	/* 4) Make it writable again; the retried write must complete. */
+	req.readonly = 0;
+	r = ioctl(prov_fd, GMEM_PROVIDER_SET_READONLY, &req);
+	TEST_ASSERT(!r, "clear readonly ioctl failed, errno %d", errno);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC, "expected UCALL_SYNC after clear");
+	TEST_ASSERT(uc.args[1] == MAGIC2, "after clear guest read 0x%lx, want MAGIC2",
+		    (unsigned long)uc.args[1]);
+	TEST_ASSERT(*(volatile uint64_t *)hva == MAGIC2,
+		    "host sees 0x%lx after guest write, want MAGIC2",
+		    (unsigned long)*(volatile uint64_t *)hva);
+	pr_info("writable: guest write 0x%llx landed -- read-only path works\n", MAGIC2);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE, "expected UCALL_DONE");
+
+	kvm_vm_free(vm);
+	munmap(hva, DATA_SIZE);
+	close(gmem_fd);
+	close(dmabuf_fd);
+	close(prov_fd);
+	close(gmem_ctl);
+	return 0;
+}
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_revoke_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_revoke_test.c
index 415972aa8a7e..df2e9fbd91d8 100644
--- a/tools/testing/selftests/kvm/x86/gmem_provider_revoke_test.c
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_revoke_test.c
@@ -38,9 +38,19 @@ struct gmem_provider_present {
 	__u32 pad;
 };
 #define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
+#define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
 #define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
 #define GMEM_PROVIDER_SET_PRESENT	_IOW('G', 2, struct gmem_provider_present)
 
+/* The dma-buf for a sample-provider child: what KVM and iommufd both import. */
+static int child_dmabuf(int child_fd)
+{
+	int fd = ioctl(child_fd, GMEM_PROVIDER_GET_DMABUF);
+
+	TEST_ASSERT(fd >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	return fd;
+}
+
 #define DATA_SLOT	10
 #define DATA_GPA	(1ULL << 32)
 #define DATA_SIZE	0x200000ULL		/* 2 MiB region */
@@ -64,7 +74,8 @@ int main(void)
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
 	struct ucall uc;
-	int gmem_ctl, gmem_fd, r;
+	int gmem_ctl, gmem_fd, dmabuf_fd, r;
+	int prov_fd;
 	void *hva;
 
 	TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM));
@@ -77,10 +88,19 @@ int main(void)
 
 	setup.kvm_fd = vm->fd;
 	setup.size = DATA_SIZE;
-	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
-	TEST_ASSERT(gmem_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+
+	/*
+	 * The provider fd is not a guest_memfd; it is the provider fd.  The
+	 * guest_memfd KVM returns is what the memslot binds, what the host
+	 * mmaps (through the provider's mmap), and what iommufd would map.
+	 */
+	dmabuf_fd = child_dmabuf(prov_fd);
+	gmem_fd = vm_create_guest_memfd_dmabuf(vm, DATA_SIZE,
+					       GUEST_MEMFD_FLAG_MMAP, dmabuf_fd);
 
-	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, gmem_fd, 0);
+	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, prov_fd, 0);
 	TEST_ASSERT(hva != MAP_FAILED,
 		    "mmap(provider) failed, errno %d", errno);
 
@@ -98,7 +118,7 @@ int main(void)
 
 	/* 2) Revoke: mark absent and zap the guest NPT (provider->KVM). */
 	req = (struct gmem_provider_present){ .offset = 0, .len = 4096, .present = 0 };
-	r = ioctl(gmem_fd, GMEM_PROVIDER_SET_PRESENT, &req);
+	r = ioctl(prov_fd, GMEM_PROVIDER_SET_PRESENT, &req);
 	TEST_ASSERT(!r, "revoke ioctl failed, errno %d", errno);
 
 	/* 3) Guest re-reads -> re-fault into absent get_pfn -> must NOT see MAGIC. */
@@ -117,7 +137,7 @@ int main(void)
 
 	/* 4) Restore: mark present again. */
 	req.present = 1;
-	r = ioctl(gmem_fd, GMEM_PROVIDER_SET_PRESENT, &req);
+	r = ioctl(prov_fd, GMEM_PROVIDER_SET_PRESENT, &req);
 	TEST_ASSERT(!r, "restore ioctl failed, errno %d", errno);
 
 	/* 5) Re-enter: the fault re-maps via get_pfn, guest reads MAGIC again. */
@@ -130,6 +150,8 @@ int main(void)
 	kvm_vm_free(vm);
 	munmap(hva, DATA_SIZE);
 	close(gmem_fd);
+	close(dmabuf_fd);
+	close(prov_fd);
 	close(gmem_ctl);
 	return 0;
 }
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_test.c
index d7caa11616df..d926b2033843 100644
--- a/tools/testing/selftests/kvm/x86/gmem_provider_test.c
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_test.c
@@ -1,195 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0
-/*
- * gmem_provider_test - exercise the samples/kvm gmem_provider module through a
- * full SEV-SNP guest launch, then re-bind the same provider fd to a second VM
- * (the live-update path).
- *
- * The provider module must be loaded first, in either mode:
- *   insmod gmem_provider.ko addr=0x5D40000000 len=0x1000000   # page-less
- *   insmod gmem_provider.ko                                   # CMA fallback
- *
- * The test skips (KSFT_SKIP) if /dev/gmem_provider or SNP support is absent.
- *
- * Note: with the current kvm_gmem_populate() ABI the launch source is an
- * ordinary anonymous buffer (pinned via get_user_pages_fast); the provider's
- * populate() copies it into the backing.  No /dev/mem mapping is needed.
- */
+/* CoCo support for dma-buf-backed guest_memfd is outside this series. */
 #include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <errno.h>
-#include <sys/ioctl.h>
-#include <sys/mman.h>
-#include <stdint.h>
-#include <linux/kvm.h>
-#include <asm/kvm.h>
 
-/* Mirrors samples/kvm/gmem_provider.h */
-struct gmem_provider_setup {
-	int32_t kvm_fd;
-	uint32_t pad;
-	uint64_t size;
-};
-#define GMEM_PROVIDER_SETUP _IOW('G', 1, struct gmem_provider_setup)
-
-#define KSFT_PASS 0
-#define KSFT_FAIL 1
 #define KSFT_SKIP 4
 
-#define GUEST_MEM_SIZE	(16UL * 1024 * 1024)
-#define PAGE_SIZE_4K	4096UL
-
-static int sev_ioctl(int vm_fd, int sev_fd, int cmd, void *data)
-{
-	struct kvm_sev_cmd sev_cmd = {
-		.id = cmd,
-		.data = (uint64_t)(unsigned long)data,
-		.sev_fd = sev_fd,
-	};
-	return ioctl(vm_fd, KVM_MEMORY_ENCRYPT_OP, &sev_cmd);
-}
-
-static int launch_snp_vm(int kvm_fd, int gmem_fd, void *src, int vm_num)
-{
-	int vm_fd, vcpu_fd, sev_fd, ret;
-
-	printf("[VM%d] KVM_CREATE_VM (SNP)\n", vm_num);
-	vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, KVM_X86_SNP_VM);
-	if (vm_fd < 0) { perror("KVM_CREATE_VM"); return -1; }
-
-	sev_fd = open("/dev/sev", O_RDWR);
-	if (sev_fd < 0) { perror("open /dev/sev"); close(vm_fd); return -1; }
-
-	struct kvm_sev_init init = { 0 };
-	ret = sev_ioctl(vm_fd, sev_fd, KVM_SEV_INIT2, &init);
-	if (ret) { perror("KVM_SEV_INIT2"); goto out; }
-
-	struct kvm_userspace_memory_region2 region = {
-		.slot = 0,
-		.flags = KVM_MEM_GUEST_MEMFD,
-		.guest_phys_addr = 0,
-		.memory_size = GUEST_MEM_SIZE,
-		.userspace_addr = (uint64_t)(unsigned long)src,
-		.guest_memfd = gmem_fd,
-		.guest_memfd_offset = 0,
-	};
-	ret = ioctl(vm_fd, KVM_SET_USER_MEMORY_REGION2, &region);
-	if (ret) { perror("KVM_SET_USER_MEMORY_REGION2"); goto out; }
-
-	struct kvm_memory_attributes attrs = {
-		.address = 0,
-		.size = GUEST_MEM_SIZE,
-		.attributes = KVM_MEMORY_ATTRIBUTE_PRIVATE,
-	};
-	ret = ioctl(vm_fd, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
-	if (ret) { perror("KVM_SET_MEMORY_ATTRIBUTES"); goto out; }
-
-	struct kvm_sev_snp_launch_start start = { .policy = 0x30000 };
-	ret = sev_ioctl(vm_fd, sev_fd, KVM_SEV_SNP_LAUNCH_START, &start);
-	if (ret) { perror("SNP_LAUNCH_START"); goto out; }
-
-	printf("[VM%d] SNP_LAUNCH_UPDATE (code + zero, %luMB)\n",
-	       vm_num, GUEST_MEM_SIZE >> 20);
-	struct kvm_sev_snp_launch_update update = {
-		.gfn_start = 0,
-		.uaddr = (uint64_t)(unsigned long)src,
-		.len = PAGE_SIZE_4K,
-		.type = KVM_SEV_SNP_PAGE_TYPE_NORMAL,
-	};
-	ret = sev_ioctl(vm_fd, sev_fd, KVM_SEV_SNP_LAUNCH_UPDATE, &update);
-	if (ret) { perror("SNP_LAUNCH_UPDATE code"); goto out; }
-
-	struct kvm_sev_snp_launch_update update_zero = {
-		.gfn_start = 1,
-		.uaddr = (uint64_t)(unsigned long)(src + PAGE_SIZE_4K),
-		.len = GUEST_MEM_SIZE - PAGE_SIZE_4K,
-		.type = KVM_SEV_SNP_PAGE_TYPE_ZERO,
-	};
-	ret = sev_ioctl(vm_fd, sev_fd, KVM_SEV_SNP_LAUNCH_UPDATE, &update_zero);
-	if (ret) { perror("SNP_LAUNCH_UPDATE zero"); goto out; }
-
-	struct kvm_sev_snp_launch_finish finish = { 0 };
-	ret = sev_ioctl(vm_fd, sev_fd, KVM_SEV_SNP_LAUNCH_FINISH, &finish);
-	if (ret) { perror("SNP_LAUNCH_FINISH"); goto out; }
-
-	vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0);
-	if (vcpu_fd < 0) { perror("KVM_CREATE_VCPU"); ret = -1; goto out; }
-
-	printf("[VM%d] *** launch succeeded ***\n", vm_num);
-	close(vcpu_fd);
-	ret = 0;
-out:
-	close(sev_fd);
-	close(vm_fd);
-	return ret;
-}
-
 int main(void)
 {
-	int kvm_fd, gmem_ctl_fd, gmem_fd, tmp_vm;
-	void *src;
-
-	setbuf(stdout, NULL);
-
-	kvm_fd = open("/dev/kvm", O_RDWR);
-	if (kvm_fd < 0) {
-		printf("SKIP: cannot open /dev/kvm (%s)\n", strerror(errno));
-		return KSFT_SKIP;
-	}
-
-	gmem_ctl_fd = open("/dev/gmem_provider", O_RDWR);
-	if (gmem_ctl_fd < 0) {
-		printf("SKIP: /dev/gmem_provider not present -- load gmem_provider.ko (%s)\n",
-		       strerror(errno));
-		return KSFT_SKIP;
-	}
-
-	/* Provider setup needs a VM fd; ownership transfers to each VM on bind. */
-	tmp_vm = ioctl(kvm_fd, KVM_CREATE_VM, KVM_X86_SNP_VM);
-	if (tmp_vm < 0) {
-		printf("SKIP: cannot create SNP VM -- host not SNP-capable? (%s)\n",
-		       strerror(errno));
-		return KSFT_SKIP;
-	}
-
-	struct gmem_provider_setup setup = {
-		.kvm_fd = tmp_vm,
-		.size = GUEST_MEM_SIZE,
-	};
-	gmem_fd = ioctl(gmem_ctl_fd, GMEM_PROVIDER_SETUP, &setup);
-	if (gmem_fd < 0) {
-		perror("GMEM_PROVIDER_SETUP");
-		return KSFT_FAIL;
-	}
-	close(tmp_vm);
-	printf("provider gmem_fd = %d (persists across VMs)\n", gmem_fd);
-
-	/* Launch source: ordinary anonymous memory holding a HLT at gfn 0. */
-	src = mmap(NULL, GUEST_MEM_SIZE, PROT_READ | PROT_WRITE,
-		   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
-	if (src == MAP_FAILED) { perror("mmap src"); return KSFT_FAIL; }
-	memset(src, 0, GUEST_MEM_SIZE);
-	((uint8_t *)src)[0] = 0xf4; /* HLT */
-
-	if (launch_snp_vm(kvm_fd, gmem_fd, src, 1) != 0) {
-		printf("FAIL: VM1 launch failed\n");
-		return KSFT_FAIL;
-	}
-
-	usleep(100000);
-
-	/* Re-bind the SAME provider fd to a fresh VM (live-update path). */
-	if (launch_snp_vm(kvm_fd, gmem_fd, src, 2) != 0) {
-		printf("FAIL: VM2 re-bind launch failed\n");
-		return KSFT_FAIL;
-	}
-
-	printf("PASS: page-less/provider-backed SNP launch + re-bind succeeded\n");
-	munmap(src, GUEST_MEM_SIZE);
-	close(gmem_fd);
-	close(gmem_ctl_fd);
-	close(kvm_fd);
-	return KSFT_PASS;
+	puts("SKIP: dma-buf-backed CoCo is outside this series");
+	return KSFT_SKIP;
 }
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_vfio_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_vfio_test.c
index 97a66a295b14..7f3c640aea78 100644
--- a/tools/testing/selftests/kvm/x86/gmem_provider_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_vfio_test.c
@@ -5,7 +5,7 @@
  *
  * Steps:
  *   1. SETUP provider (KVM VM shape not required for this test).
- *   2. iommufd IOAS + GET_DMABUF + IOMMU_IOAS_MAP_FILE - IOAS holds the
+ *   2. guest_memfd from the provider fd; IOMMU_IOAS_MAP_FILE on its dma-buf - IOAS holds the
  *      provider region.
  *   3. Open a vfio-pci cdev (default /dev/vfio/devices/vfio0, overridable
  *      via GMEM_VFIO_CDEV env), VFIO_DEVICE_BIND_IOMMUFD, then
@@ -39,8 +39,17 @@ struct gmem_provider_setup {
 	__u64 size;
 };
 #define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
-#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
 #define GMEM_PROVIDER_GET_DMABUF	_IO('G', 3)
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
+
+/* The dma-buf for a sample-provider child: what KVM and iommufd both import. */
+static int child_dmabuf(int child_fd)
+{
+	int fd = ioctl(child_fd, GMEM_PROVIDER_GET_DMABUF);
+
+	TEST_ASSERT(fd >= 0, "GMEM_PROVIDER_GET_DMABUF errno=%d", errno);
+	return fd;
+}
 
 #define DATA_SIZE	0x200000ULL
 #define IOVA_BASE	(1ULL << 34)
@@ -53,7 +62,7 @@ int main(void)
 	struct vfio_device_bind_iommufd bind = {};
 	struct vfio_device_attach_iommufd_pt att = {};
 	const char *cdev_path;
-	int gmem_ctl, gmem_fd, dmabuf_fd, iommufd_fd, vfio_fd, r;
+	int gmem_ctl, gmem_fd, dmabuf_fd, prov_fd, vm_fd = -1, iommufd_fd, vfio_fd, r;
 
 	gmem_ctl = open("/dev/gmem_provider", O_RDWR);
 	__TEST_REQUIRE(gmem_ctl >= 0, "gmem_provider module not loaded");
@@ -80,20 +89,34 @@ int main(void)
 		vm = ioctl(kvm, KVM_CREATE_VM, 0);
 		TEST_ASSERT(vm >= 0, "KVM_CREATE_VM errno=%d", errno);
 		setup.kvm_fd = vm;
+		vm_fd = vm;
 		close(kvm);
 	}
 	setup.size = DATA_SIZE;
-	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
-	TEST_ASSERT(gmem_fd >= 0, "GMEM_PROVIDER_SETUP errno=%d", errno);
+	prov_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(prov_fd >= 0, "GMEM_PROVIDER_SETUP errno=%d", errno);
+
+	/*
+	 * The provider fd is the provider fd of a guest_memfd, and iommufd maps
+	 * the dma-buf; a device and a guest import the same one.
+	 */
+	{
+		struct kvm_create_guest_memfd cgm;
+
+		dmabuf_fd = child_dmabuf(prov_fd);
+		cgm = (struct kvm_create_guest_memfd) {
+			.size = DATA_SIZE,
+			.flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_DMABUF,
+			.dmabuf_fd = dmabuf_fd,
+		};
+		gmem_fd = ioctl(vm_fd, KVM_CREATE_GUEST_MEMFD, &cgm);
+		TEST_ASSERT(gmem_fd >= 0, "KVM_CREATE_GUEST_MEMFD(dmabuf) errno=%d", errno);
+	}
 
-	/* IOAS + provider dma-buf */
+	/* IOAS + the same dma-buf KVM imported. */
 	alloc.size = sizeof(alloc);
 	r = ioctl(iommufd_fd, IOMMU_IOAS_ALLOC, &alloc);
 	TEST_ASSERT(!r, "IOMMU_IOAS_ALLOC errno=%d", errno);
-
-	dmabuf_fd = ioctl(gmem_fd, GMEM_PROVIDER_GET_DMABUF);
-	TEST_ASSERT(dmabuf_fd >= 0, "GET_DMABUF errno=%d", errno);
-
 	map.size = sizeof(map);
 	map.flags = IOMMU_IOAS_MAP_FIXED_IOVA |
 		    IOMMU_IOAS_MAP_READABLE | IOMMU_IOAS_MAP_WRITEABLE;
@@ -126,8 +149,8 @@ int main(void)
 	pr_info("real passthrough device now has IOMMU domain covering provider region\n");
 
 	close(vfio_fd);
-	close(dmabuf_fd);
 	close(iommufd_fd);
+	close(dmabuf_fd);
 	close(gmem_fd);
 	close(gmem_ctl);
 	return 0;
-- 
2.47.3

[RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Some drivers manage RAM outside the page allocator: a carve-out, device
memory, or memory that a host component moves between VMs while they
run. This memory often has no struct page. The driver wants to lend
pages to a VM and to its devices, and to take any of them back later.

David Woodhouse's series "KVM: Allow alternative providers of
guest_memfd backed by PFNMAP memory" lets such a driver back a
guest_memfd through struct kvm_gmem_ops:

  https://lore.kernel.org/kvm/20260720111259.122911-1-dwmw2@infradead.org/

That covers the guest, but not the VM's devices, which his cover
letter left as an open question. This series adds a small interface in
mm: the driver that owns the memory becomes a provider, and the code
that maps it attaches as a consumer. There are two consumers, a
guest_memfd backend and IOMMU_IOAS_MAP_FILE in iommufd.

The series is on top of David's v2 (base 0e35b9b6ec0f). Patch 1 was
part of the earlier dma-buf RFC. The provider backend depends on it, so
this series carries it.

Why not dma-buf
===============

The earlier RFC shared the memory as a dma-buf. Christian König
rejected that: an importer must not build its own page tables from a
dma-buf, and the use case should stay out of drivers/dma-buf. I
acknowledged that and dropped it; this series does not touch dma-buf.

Why a new interface
===================

kvm_gmem_ops gives KVM a way in. For devices, the only option today is
a dma-buf plus a private call that iommufd looks up with symbol_get(),
which is how vfio-pci works. Each new owner would need another such
lookup, and iommufd would end up knowing each owner by name. With a
common interface, a provider module only calls into mm, and nothing
needs symbol_get().

It also means the provider revokes a range once. The VMM hands the
same file to KVM and to iommufd; on a revoke the core forwards it to
every consumer, so KVM clears the range from stage-2, iommufd from the
IOMMU page tables, and guest_memfd from the VMM's own mapping, all
before the revoke returns.

The contract
============

For a page of a provider file, get_page() returns the frame, its type
(RAM or MMIO, read-only or not), and the largest aligned block of frames
of the same type, so that consumers can use large mappings. A page
without a frame is a hole.

Consumers take no references. A frame stays valid until the revoke that
removes it returns. To guarantee that, a consumer either holds a lock
across get_page() and the mapping that its revoke callback also takes,
or detects a racing revoke and retries.

The interface does not deal with folios. The native guest_memfd backend
and memfd pinning in iommufd handle pages from the page allocator, and
pages that can move or swap would need references. A backend that needs
full control over the file can still write its own kvm_gmem_ops.

Backends
========

The motivating user is a host-side memory manager. It reserves a region
of host RAM, lends parts of it to VMs and their devices, moves pages
between VMs, and has to survive a live update of the host kernel. The
sample module in this series models it.

Device-DAX could also be a provider: its range never moves, get_page()
is short, and it only revokes on unbind.  iommufd can map MMIO, but KVM
takes only RAM, because it picks the guest memory type itself.

Neither consumer keeps state about the provider's frames, which helps
with live update.  guest_memfd has nothing to save across kexec, since
KVM refills stage-2 on faults. The provider saves its frames and who
owns them. Devices cannot fault, so their mappings stay part of the
existing iommufd live update work, and the provider only has to give
back the same frames.

Confidential VMs
================

This series does not support them yet. For now only x86 VMs of type
KVM_X86_DEFAULT_VM can use a provider.

I don't think a provider should back private pages. Revoking a private
page loses its contents, and only guest_memfd knows which pages are
private. The interface could still be useful there later, with
guest_memfd acting as a provider for its own files so that iommufd only
maps shared pages. I left that for a later series.

Patches
=======

  Patch 1 lets a kvm_gmem_ops backend map a page read-only for the
  guest.  get_pfn() gets a writable output, and a guest write to such a
  page exits with KVM_EXIT_MEMORY_FAULT. KVM_MEM_READONLY can't be
  used for this, because guest_memfd slots don't allow it.

  Patch 2 adds the interface and the core, and the FOP_MEM_PROVIDER
  flag in struct file_operations.

  Patch 3 adds the provider backend to guest_memfd, with the
  GUEST_MEMFD_FLAG_USE_PROVIDER flag and a provider_fd field.
  guest_memfd also handles the userspace mapping of the file, if the
  provider allows it, so providers don't each have to get the fault and
  revoke handling right.

  Patch 4 prepares iommufd for tracking pages it does not pin. No
  functional change.

  Patch 5 lets IOMMU_IOAS_MAP_FILE take a provider file. iommufd maps
  one PAGE_SIZE entry per page and pins nothing. It leaves holes
  unmapped and honours read-only and MMIO pages. On a revoke, it
  unmaps the range and maps whatever the provider backs now.

  The PAGE_SIZE entries are deliberate for now: a partial revoke then
  never has to split a large IOMMU page. Using the block size the
  provider reports would be better, and needs the revoke to unmap and
  remap whole blocks. A device that accesses the range between the
  unmap and the map faults; replacing the entries in place would avoid
  that, but needs new support in the IOMMU drivers. Both are left for
  later.

  Patches 6 and 7 add iommufd selftests: two mock-domain queries and a
  mock provider.

  Patch 8 adds a sample provider. It gives each VM a child file, can
  move, donate and reclaim pages, and supports read-only pages. A child
  file is read-only to its holder; the owner changes it through the
  control device. It only uses the mm interface.

  Patch 9 adds a KVM selftest with one provider and two VMMs. After
  each change it checks what the guest, the device and the VMM's mapping
  see.

Testing
=======

x86_64, in QEMU with KASAN, PROVE_LOCKING and DEBUG_ATOMIC_SLEEP, with
two ranges hidden from the host with memmap=, one for each sample. No
KASAN report, lockdep splat or warning in any run.

  - mem_provider_test, the selftest in patch 9: pass.
  - guest_memfd_test, including the USE_PROVIDER flag check: pass.
  - iommufd_selftest, iommufd_ioas: the 12 provider tests pass in all
    four variants. The 4 failures are access_domain_destory, which the
    series does not touch: it needs hugetlb pages, which this VM has
    none of.
  - David's gmem_provider tests, which this series does not change:
    hugepage, revoke, iommufd and readonly pass; the SNP and vfio-pci
    tests skip for lack of hardware.

Not tested: arm64, where guest_memfd refuses providers; a real IOMMU,
since only the mock domain was used.

I wrote most of this series with AI help, and I am posting it as an RFC
to get feedback on the interface.

Fred Griffoul (9):
  KVM: guest_memfd: Add a writable result to get_pfn()
  mm: Add memory providers
  KVM: guest_memfd: Add a memory provider backing
  iommufd: Track the domains of pages that are not pinned
  iommufd: Map memory provider files
  iommufd/selftest: Add mock-domain IOVA queries
  iommufd/selftest: Add a mock memory provider
  samples/kvm: Add a memory provider sample
  KVM: selftests: Test a memory provider shared by KVM and iommufd

 Documentation/virt/kvm/api.rst                |  45 +-
 MAINTAINERS                                   |   8 +
 arch/arm64/kvm/mmu.c                          |  13 +-
 arch/arm64/kvm/nested.c                       |  16 +-
 arch/x86/kvm/mmu/mmu.c                        |  18 +-
 arch/x86/kvm/svm/sev.c                        |  13 +-
 arch/x86/kvm/x86.c                            |  10 +
 drivers/iommu/iommufd/Kconfig                 |   1 +
 drivers/iommu/iommufd/io_pagetable.c          |  37 +-
 drivers/iommu/iommufd/io_pagetable.h          |  54 +-
 drivers/iommu/iommufd/iommufd_test.h          |  36 +
 drivers/iommu/iommufd/pages.c                 | 340 ++++++-
 drivers/iommu/iommufd/selftest.c              | 286 ++++++
 include/linux/fs.h                            |   2 +
 include/linux/kvm_host.h                      |  20 +-
 include/linux/mem_provider.h                  | 211 +++++
 include/uapi/linux/iommufd.h                  |  11 +-
 include/uapi/linux/kvm.h                      |  11 +-
 mm/Kconfig                                    |   7 +
 mm/Makefile                                   |   1 +
 mm/mem_provider.c                             | 177 ++++
 samples/Kconfig                               |  17 +
 samples/Makefile                              |   1 +
 samples/kvm/Makefile                          |   1 +
 samples/kvm/gmem_provider.c                   |  67 +-
 samples/kvm/gmem_provider.h                   |  17 +
 samples/kvm/mem_provider_sample.c             | 855 ++++++++++++++++++
 samples/kvm/mem_provider_sample.h             | 135 +++
 tools/include/uapi/linux/kvm.h                |  11 +-
 tools/testing/selftests/iommu/iommufd.c       | 119 +++
 tools/testing/selftests/iommu/iommufd_utils.h |  64 ++
 tools/testing/selftests/kvm/Makefile.kvm      |   2 +
 .../testing/selftests/kvm/guest_memfd_test.c  |   6 +
 .../kvm/x86/gmem_provider_readonly_test.c     | 150 +++
 .../selftests/kvm/x86/mem_provider_test.c     | 582 ++++++++++++
 virt/kvm/Kconfig                              |   1 +
 virt/kvm/guest_memfd.c                        | 286 +++++-
 37 files changed, 3531 insertions(+), 100 deletions(-)
 create mode 100644 include/linux/mem_provider.h
 create mode 100644 mm/mem_provider.c
 create mode 100644 samples/kvm/mem_provider_sample.c
 create mode 100644 samples/kvm/mem_provider_sample.h
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/mem_provider_test.c


base-commit: 0e35b9b6ec0ffcc5e23cbdec09f5c622ad532b53
prerequisite-patch-id: 2a0016e90f0690baef841a8c34c07b96e3f1b941
prerequisite-patch-id: c497c1ff1e9de8e9c470c58b163ccbfce4827ae7
prerequisite-patch-id: a16b61afd172662bde725e5c12805758f76b89fa
prerequisite-patch-id: 0ddee6c1b48fa853e6a94f2746340e525477d287
prerequisite-patch-id: 4dc9a395a2eb73283b20b75e2ba763ec72b2e22d
prerequisite-patch-id: a9758d7f8f6959dae6ad154902fa1269234dbc3c
prerequisite-patch-id: 0537bcc3ca5e3bb9b86fe7c99a245a9ff9d67831
prerequisite-patch-id: d5feb18b6630c99973259804418243d856c37ce7
prerequisite-patch-id: 2a08a105b6b5d46240a69f5bbf646211d44f8412
prerequisite-patch-id: 760b34834d5d8dcdf1ae1a09cc6aae267dd7760c
prerequisite-patch-id: dd94ecc7ae9472c2bf005c9cb1c483994eacd163
-- 
2.47.3

[PATCH 1/9] KVM: guest_memfd: Add a writable result to get_pfn()

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A guest_memfd backing cannot map a page read-only for the guest:
get_pfn() returns only a frame, and KVM_MEM_READONLY is refused on
guest_memfd slots. A hypervisor therefore cannot give a guest a page it
may read but not write.

Let get_pfn() clear a writable result. The x86 and arm64 fault paths
then map the page without write permission, and a guest write exits
with KVM_EXIT_MEMORY_FAULT. The result covers the whole block that
max_order allows.

Callers that hand the page to something that writes it must refuse a
read-only page: the arm64 VNCR page and the SEV-SNP VMSA. populate() has
no writable result, so a backing cannot report read-only state through
it. The result does not cover KVM's own writes through the slot's host
address.

David's gmem_provider sample gains a read-only ioctl, and a selftest
checks that a guest write to such a page exits and lands once the page
is writable again.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 arch/arm64/kvm/mmu.c                          |  13 +-
 arch/arm64/kvm/nested.c                       |  16 +-
 arch/x86/kvm/mmu/mmu.c                        |  18 ++-
 arch/x86/kvm/svm/sev.c                        |  13 +-
 include/linux/kvm_host.h                      |  16 +-
 samples/kvm/gmem_provider.c                   |  67 +++++++-
 samples/kvm/gmem_provider.h                   |  17 ++
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/x86/gmem_provider_readonly_test.c     | 150 ++++++++++++++++++
 virt/kvm/guest_memfd.c                        |  21 ++-
 10 files changed, 309 insertions(+), 23 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6c941aaa10c6..32e591edc69d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1607,7 +1607,7 @@ struct kvm_s2_fault_desc {
 
 static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
 {
-	bool write_fault, exec_fault;
+	bool write_fault, exec_fault, writable;
 	bool perm_fault = kvm_vcpu_trap_is_permission_fault(s2fd->vcpu);
 	enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
 	enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
@@ -1641,14 +1641,17 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
 	/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
 	smp_rmb();
 
-	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL);
-	if (ret) {
+	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL,
+			       &writable);
+	if (ret || (write_fault && !writable)) {
 		kvm_prepare_memory_fault_exit(s2fd->vcpu, s2fd->fault_ipa, PAGE_SIZE,
 					      write_fault, exec_fault, false);
-		return ret;
+		if (!ret)
+			kvm_release_faultin_page(kvm, page, true, false);
+		return ret ?: -EFAULT;
 	}
 
-	if (!(s2fd->memslot->flags & KVM_MEM_READONLY))
+	if (!(s2fd->memslot->flags & KVM_MEM_READONLY) && writable)
 		prot |= KVM_PGTABLE_PROT_W;
 
 	if (s2fd->nested)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index fb54f6dad995..fa86705df949 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1411,11 +1411,19 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 		if (is_error_noslot_pfn(pfn) || (write_fault && !writable))
 			return -EFAULT;
 	} else {
-		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, &page, NULL);
-		if (ret) {
+		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, &page, NULL,
+				       &writable);
+		/*
+		 * The VNCR page is always written, so a read-only page cannot
+		 * back it.  Exit on the first access, even a read, rather than
+		 * map it read-only and fault on the next write.
+		 */
+		if (ret || !writable) {
 			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
-					      write_fault, false, false);
-			return ret;
+					      true, false, false);
+			if (!ret)
+				kvm_release_faultin_page(vcpu->kvm, page, true, false);
+			return ret ?: -EFAULT;
 		}
 	}
 
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 234d0a95abf5..f689ef5c2b46 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4612,6 +4612,7 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu,
 static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 				    struct kvm_page_fault *fault)
 {
+	bool writable;
 	int max_order, r;
 
 	if (!kvm_slot_has_gmem(fault->slot)) {
@@ -4620,13 +4621,26 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 	}
 
 	r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn,
-			     &fault->refcounted_page, &max_order);
+			     &fault->refcounted_page, &max_order, &writable);
 	if (r) {
 		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
 		return r;
 	}
 
-	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY);
+	/*
+	 * get_pfn() may clear writable, on top of the memslot's read-only flag:
+	 * a read-only page is mapped read-only, and a guest write to it exits to
+	 * userspace rather than being installed.
+	 */
+	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY) &&
+			      writable;
+	if (fault->write && !fault->map_writable) {
+		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
+		kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page,
+				 true, false);
+		fault->refcounted_page = NULL;
+		return -EFAULT;
+	}
 	fault->max_level = kvm_max_level_for_order(max_order);
 
 	return RET_PF_CONTINUE;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 125779c82bc4..f4f944c7da10 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4026,6 +4026,7 @@ static void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu)
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct kvm_memory_slot *slot;
 	struct page *page;
+	bool writable;
 	kvm_pfn_t pfn;
 	gfn_t gfn;
 
@@ -4063,9 +4064,17 @@ static void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu)
 	 * The new VMSA will be private memory guest memory, so retrieve the
 	 * PFN from the gmem backend.
 	 */
-	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL))
+	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL,
+			     &writable))
 		return;
 
+	/* The CPU writes the VMSA on every VMRUN: a read-only page cannot be one. */
+	if (!writable) {
+		if (page)
+			kvm_release_page_clean(page);
+		return;
+	}
+
 	/*
 	 * From this point forward, the VMSA will always be a guest-mapped page
 	 * rather than the initial one allocated by KVM in svm->sev_es.vmsa. In
@@ -4996,7 +5005,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 		return;
 	}
 
-	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order);
+	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order, NULL);
 	if (ret) {
 		pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n",
 				    gpa);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 04fa0cb126f6..7281d0e94121 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -649,6 +649,15 @@ static inline bool kvm_slot_has_gmem(const struct kvm_memory_slot *slot)
  *    is responsible for put_page() after use.  If *page is left NULL the PFN
  *    is treated as non-refcounted, and its lifetime is owned by the
  *    implementation across bind()/unbind().
+ *  - *writable, when the caller passes it, is true on entry.  Clear it to
+ *    have KVM map the page read-only; a guest write then exits as a memory
+ *    fault.  It covers the whole block that *max_order allows, so that block
+ *    must be all writable or all read-only.  It applies to stage-2 mappings
+ *    only, not to KVM's own writes through the memslot's host address.  A
+ *    caller that hands the page to hardware that writes it passes @writable
+ *    and refuses a read-only page.  Only a caller that never writes the page
+ *    itself passes NULL.  populate() has no writable result, so KVM cannot
+ *    learn read-only state through it; it is used only to fill a page.
  *
  * Memory intended to back guest RAM MUST be reported as E820_TYPE_RAM by the
  * host so KVM maps it write-back (and applies the memory-encryption bit on
@@ -662,7 +671,8 @@ struct kvm_gmem_ops {
 		       struct kvm_memory_slot *slot);
 	int (*get_pfn)(struct file *file, struct kvm *kvm,
 		       struct kvm_memory_slot *slot, gfn_t gfn,
-		       kvm_pfn_t *pfn, struct page **page, int *max_order);
+		       kvm_pfn_t *pfn, struct page **page, int *max_order,
+		       bool *writable);
 	int (*populate)(struct file *file, struct kvm *kvm,
 			struct kvm_memory_slot *slot, gfn_t gfn,
 			kvm_pfn_t *pfn, struct page *src_page, int order);
@@ -2651,12 +2661,12 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
 #ifdef CONFIG_KVM_GUEST_MEMFD
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		     gfn_t gfn, kvm_pfn_t *pfn, struct page **page,
-		     int *max_order);
+		     int *max_order, bool *writable);
 #else
 static inline int kvm_gmem_get_pfn(struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
 				   kvm_pfn_t *pfn, struct page **page,
-				   int *max_order)
+				   int *max_order, bool *writable)
 {
 	KVM_BUG_ON(1, kvm);
 	return -EIO;
diff --git a/samples/kvm/gmem_provider.c b/samples/kvm/gmem_provider.c
index 9728f5a8029b..99b288302d98 100644
--- a/samples/kvm/gmem_provider.c
+++ b/samples/kvm/gmem_provider.c
@@ -95,6 +95,7 @@ struct gmem_info {
 	gfn_t base_gfn;				/* recorded at bind, for revoke */
 	pgoff_t pgoff;				/* provider offset (pages) of the slot */
 	unsigned long *absent;			/* bitmap of currently-revoked pages */
+	unsigned long *readonly;		/* bitmap of pages the guest may not write */
 	struct list_head dmabufs;		/* struct gmem_dmabuf entries */
 	struct mutex dmabufs_lock;
 };
@@ -142,6 +143,22 @@ static int gmem_max_order(struct gmem_info *info, gfn_t gfn, unsigned long index
 		remaining = min(remaining, absent_next - index);
 	}
 
+	/*
+	 * Likewise a hugepage must be uniformly writable or uniformly
+	 * read-only: clamp at the next page whose read-only bit differs.
+	 */
+	if (info->readonly) {
+		unsigned long next;
+
+		if (test_bit(index, info->readonly))
+			next = find_next_zero_bit(info->readonly, info->npages,
+						  index + 1);
+		else
+			next = find_next_bit(info->readonly, info->npages,
+					     index + 1);
+		remaining = min(remaining, next - index);
+	}
+
 	if (IS_ALIGNED(pfn, 1UL << pud_order) &&
 	    IS_ALIGNED(gfn, 1UL << pud_order) &&
 	    remaining >= (1UL << pud_order))
@@ -157,7 +174,8 @@ static int gmem_max_order(struct gmem_info *info, gfn_t gfn, unsigned long index
 
 static int gmem_get_pfn(struct file *file, struct kvm *kvm,
 			struct kvm_memory_slot *slot, gfn_t gfn,
-			kvm_pfn_t *pfn, struct page **page, int *max_order)
+			kvm_pfn_t *pfn, struct page **page, int *max_order,
+			bool *writable)
 {
 	struct gmem_info *info = to_gmem_info(file);
 	pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff;
@@ -172,6 +190,8 @@ static int gmem_get_pfn(struct file *file, struct kvm *kvm,
 	*pfn = info->base_pfn + index;
 	if (max_order)
 		*max_order = gmem_max_order(info, gfn, index);
+	if (writable)
+		*writable = !(info->readonly && test_bit(index, info->readonly));
 	return 0;
 }
 
@@ -371,6 +391,7 @@ static void gmem_release(struct file *file)
 	if (info->cma_pages)
 		free_contig_range(info->base_pfn, info->npages);
 	kvfree(info->absent);
+	kvfree(info->readonly);
 	kfree(info);
 	module_put(THIS_MODULE);
 }
@@ -581,6 +602,46 @@ static long gmem_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg
 	if (cmd == GMEM_PROVIDER_GET_DMABUF)
 		return gmem_provider_get_dmabuf(file);
 
+	if (cmd == GMEM_PROVIDER_SET_READONLY) {
+		struct gmem_provider_readonly r;
+		unsigned long clamped_start;
+
+		if (copy_from_user(&r, (void __user *)arg, sizeof(r)))
+			return -EFAULT;
+		if (!r.len || !PAGE_ALIGNED(r.offset) || !PAGE_ALIGNED(r.len) ||
+		    r.pad)
+			return -EINVAL;
+		start_index = r.offset >> PAGE_SHIFT;
+		end_index = start_index + (r.len >> PAGE_SHIFT);
+		if (end_index > info->npages || end_index < start_index)
+			return -EINVAL;
+
+		/*
+		 * Flip the bits, then drop the guest's existing mappings of the
+		 * range so the next access re-faults through get_pfn() and
+		 * picks up the new permission.  Making a range read-only must
+		 * tear down writable mappings; making it writable again is
+		 * also invalidated so a stale read-only mapping does not keep
+		 * exiting.
+		 */
+		mutex_lock(&info->lock);
+		if (r.readonly)
+			bitmap_set(info->readonly, start_index,
+				   end_index - start_index);
+		else
+			bitmap_clear(info->readonly, start_index,
+				     end_index - start_index);
+		clamped_start = max_t(unsigned long, start_index, info->pgoff);
+		if (info->kvm && end_index > clamped_start)
+			kvm_gmem_invalidate_range(info->kvm,
+						  info->base_gfn + clamped_start -
+							  info->pgoff,
+						  info->base_gfn + end_index -
+							  info->pgoff);
+		mutex_unlock(&info->lock);
+		return 0;
+	}
+
 	if (cmd != GMEM_PROVIDER_SET_PRESENT)
 		return -ENOTTY;
 	if (copy_from_user(&p, (void __user *)arg, sizeof(p)))
@@ -694,7 +755,8 @@ static long gmem_ctl_ioctl(struct file *file, unsigned int cmd, unsigned long ar
 
 	info->absent = kvzalloc(BITS_TO_LONGS(info->npages) * sizeof(unsigned long),
 				GFP_KERNEL);
-	if (!info->absent) {
+	info->readonly = kvzalloc_objs(unsigned long, BITS_TO_LONGS(info->npages));
+	if (!info->absent || !info->readonly) {
 		ret = -ENOMEM;
 		goto err_free_pages;
 	}
@@ -733,6 +795,7 @@ static long gmem_ctl_ioctl(struct file *file, unsigned int cmd, unsigned long ar
 		free_contig_range(page_to_pfn(pages), npages);
 err_free_info:
 	kvfree(info->absent);
+	kvfree(info->readonly);
 	kfree(info);
 err_put_kvm:
 	kvm_put_kvm(kvm);
diff --git a/samples/kvm/gmem_provider.h b/samples/kvm/gmem_provider.h
index 45f1b8257f60..51b49ef9a3c5 100644
--- a/samples/kvm/gmem_provider.h
+++ b/samples/kvm/gmem_provider.h
@@ -41,6 +41,23 @@ struct gmem_provider_present {
 
 #define GMEM_PROVIDER_SET_PRESENT	_IOW(GMEM_PROVIDER_IOCTL_BASE, 2, struct gmem_provider_present)
 
+/*
+ * ioctl on a provider fd: make a byte range read-only for the guest, or
+ * writable again.  KVM maps a read-only page without write permission and a
+ * guest write to it exits to userspace with KVM_EXIT_MEMORY_FAULT.  Existing
+ * mappings of the range are dropped so the change takes effect on the next
+ * access.
+ */
+struct gmem_provider_readonly {
+	__u64 offset;	/* byte offset into the provider region, page aligned */
+	__u64 len;	/* byte length, page aligned */
+	__u32 readonly;	/* 1 = guest may not write, 0 = guest may write */
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_SET_READONLY \
+	_IOW(GMEM_PROVIDER_IOCTL_BASE, 4, struct gmem_provider_readonly)
+
 /*
  * ioctl on a provider fd (returned by SETUP): export the backing region as a
  * dynamic dma-buf and return an fd for it, suitable for
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 12004a487c32..4d7082448cea 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -81,6 +81,7 @@ TEST_GEN_PROGS_x86 += x86/fix_hypercall_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_hugepage_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_revoke_test
+TEST_GEN_PROGS_x86 += x86/gmem_provider_readonly_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_iommufd_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_vfio_test
 TEST_GEN_PROGS_x86 += x86/hwcr_msr_test
diff --git a/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c b/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c
new file mode 100644
index 000000000000..c3eedba7a16d
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/gmem_provider_readonly_test.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * gmem_provider_readonly_test - exercise per-range read-only from a provider.
+ *
+ * Marks a provider-backed page read-only via an ioctl on the provider fd and
+ * checks that KVM honours the provider's answer: the guest can still read the
+ * page, a guest write exits to userspace with KVM_EXIT_MEMORY_FAULT rather than
+ * landing, and clearing the bit lets the write through. This is the mechanism
+ * a hypervisor uses to protect a page it shares with the guest, such as a
+ * sidecar's info page, without giving up the mapping.
+ *
+ * The test opens the provider with GMEM_PROVIDER_FLAG_MMAP_CAPABLE at SETUP time
+ * (gmem-only). Load the module with a backing region of at least DATA_SIZE.
+ */
+#include <fcntl.h>
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+/* Mirrors samples/kvm/gmem_provider.h */
+struct gmem_provider_setup {
+	__s32 kvm_fd;
+	__u32 flags;
+	__u64 size;
+};
+
+struct gmem_provider_readonly {
+	__u64 offset;
+	__u64 len;
+	__u32 readonly;
+	__u32 pad;
+};
+
+#define GMEM_PROVIDER_SETUP		_IOW('G', 1, struct gmem_provider_setup)
+#define GMEM_PROVIDER_FLAG_MMAP_CAPABLE	(1u << 0)
+#define GMEM_PROVIDER_SET_READONLY	_IOW('G', 4, struct gmem_provider_readonly)
+
+#define DATA_SLOT	10
+#define DATA_GPA	(1ULL << 32)
+#define DATA_SIZE	0x200000ULL		/* 2 MiB region */
+#define MAGIC		0x1234abcdULL
+#define MAGIC2		0xfeedf00dULL
+
+/*
+ * Phase 1: read the page and report it.
+ * Phase 2: write to it.  With the page read-only this never returns to the
+ *          guest until userspace clears the bit; then it completes and the
+ *          guest reports what it wrote.
+ */
+static void guest_code(void)
+{
+	GUEST_SYNC(*(volatile uint64_t *)DATA_GPA);
+	*(volatile uint64_t *)DATA_GPA = MAGIC2;
+	GUEST_SYNC(*(volatile uint64_t *)DATA_GPA);
+	GUEST_DONE();
+}
+
+int main(void)
+{
+	struct vm_shape shape = {
+		.mode = VM_MODE_DEFAULT,
+		.type = KVM_X86_SW_PROTECTED_VM,
+	};
+	struct gmem_provider_setup setup = { .flags = GMEM_PROVIDER_FLAG_MMAP_CAPABLE };
+	struct gmem_provider_readonly req;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	int gmem_ctl, gmem_fd, r;
+	void *hva;
+
+	TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM));
+
+	gmem_ctl = open("/dev/gmem_provider", O_RDWR);
+	__TEST_REQUIRE(gmem_ctl >= 0,
+		       "gmem_provider module not loaded (/dev/gmem_provider absent)");
+
+	vm = vm_create_shape_with_one_vcpu(shape, &vcpu, guest_code);
+
+	setup.kvm_fd = vm->fd;
+	setup.size = DATA_SIZE;
+	gmem_fd = ioctl(gmem_ctl, GMEM_PROVIDER_SETUP, &setup);
+	TEST_ASSERT(gmem_fd >= 0, "GMEM_PROVIDER_SETUP failed, errno %d", errno);
+
+	hva = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, gmem_fd, 0);
+	TEST_ASSERT(hva != MAP_FAILED, "mmap(provider) failed, errno %d", errno);
+
+	r = __vm_set_user_memory_region2(vm, DATA_SLOT, KVM_MEM_GUEST_MEMFD,
+					 DATA_GPA, DATA_SIZE, hva, gmem_fd, 0);
+	TEST_ASSERT(!r, "KVM_SET_USER_MEMORY_REGION2 failed: %d errno %d", r, errno);
+	virt_map(vm, DATA_GPA, DATA_GPA, 1);
+
+	/* Seed the page from the host before the guest ever touches it. */
+	*(volatile uint64_t *)hva = MAGIC;
+
+	/* 1) Make the page read-only for the guest. */
+	req = (struct gmem_provider_readonly){ .offset = 0, .len = 4096, .readonly = 1 };
+	r = ioctl(gmem_fd, GMEM_PROVIDER_SET_READONLY, &req);
+	TEST_ASSERT(!r, "set readonly ioctl failed, errno %d", errno);
+
+	/* 2) Guest read must still work and see the host's value. */
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC, "expected UCALL_SYNC");
+	TEST_ASSERT(uc.args[1] == MAGIC, "guest read 0x%lx, want MAGIC",
+		    (unsigned long)uc.args[1]);
+	pr_info("read-only: guest read 0x%llx\n", MAGIC);
+
+	/* 3) Guest write must exit to userspace, not land. */
+	r = _vcpu_run(vcpu);
+	TEST_ASSERT(r == -1 && errno == EFAULT &&
+		    vcpu->run->exit_reason == KVM_EXIT_MEMORY_FAULT,
+		    "read-only write: expected KVM_EXIT_MEMORY_FAULT (r=%d errno=%d exit_reason=%u %s)",
+		    r, errno, vcpu->run->exit_reason,
+		    exit_reason_str(vcpu->run->exit_reason));
+	TEST_ASSERT(*(volatile uint64_t *)hva == MAGIC,
+		    "guest write landed on a read-only page: host sees 0x%lx",
+		    (unsigned long)*(volatile uint64_t *)hva);
+	pr_info("read-only: guest write exited with KVM_EXIT_MEMORY_FAULT, page unchanged\n");
+
+	/* 4) Make it writable again; the retried write must complete. */
+	req.readonly = 0;
+	r = ioctl(gmem_fd, GMEM_PROVIDER_SET_READONLY, &req);
+	TEST_ASSERT(!r, "clear readonly ioctl failed, errno %d", errno);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC, "expected UCALL_SYNC after clear");
+	TEST_ASSERT(uc.args[1] == MAGIC2, "after clear guest read 0x%lx, want MAGIC2",
+		    (unsigned long)uc.args[1]);
+	TEST_ASSERT(*(volatile uint64_t *)hva == MAGIC2,
+		    "host sees 0x%lx after guest write, want MAGIC2",
+		    (unsigned long)*(volatile uint64_t *)hva);
+	pr_info("writable: guest write 0x%llx landed -- read-only path works\n", MAGIC2);
+
+	vcpu_run(vcpu);
+	TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_DONE, "expected UCALL_DONE");
+
+	kvm_vm_free(vm);
+	munmap(hva, DATA_SIZE);
+	close(gmem_fd);
+	close(gmem_ctl);
+	return 0;
+}
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index d284bb70fe05..a509f1a96c0b 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -624,7 +624,7 @@ static void kvm_gmem_native_unbind(struct file *slot_file, struct kvm *kvm,
 static int  kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 				    struct kvm_memory_slot *slot, gfn_t gfn,
 				    kvm_pfn_t *pfn, struct page **page,
-				    int *max_order);
+				    int *max_order, bool *writable);
 static void kvm_gmem_native_release(struct file *file);
 static int  kvm_gmem_native_mmap(struct file *file,
 				 struct vm_area_struct *vma);
@@ -988,7 +988,7 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
 static int kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
 				   kvm_pfn_t *pfn, struct page **page,
-				   int *max_order)
+				   int *max_order, bool *writable)
 {
 	pgoff_t index = kvm_gmem_get_index(slot, gfn);
 	struct folio *folio;
@@ -1016,7 +1016,7 @@ static int kvm_gmem_native_get_pfn(struct file *file, struct kvm *kvm,
 
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		     gfn_t gfn, kvm_pfn_t *pfn, struct page **page,
-		     int *max_order)
+		     int *max_order, bool *writable)
 {
 	const struct kvm_gmem_ops *ops;
 
@@ -1029,7 +1029,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		return -EFAULT;
 
 	*page = NULL;
-	return ops->get_pfn(file, kvm, slot, gfn, pfn, page, max_order);
+	if (writable)
+		*writable = true;
+	return ops->get_pfn(file, kvm, slot, gfn, pfn, page, max_order,
+			    writable);
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
 
@@ -1080,6 +1083,7 @@ static int kvm_gmem_populate_one(const struct kvm_gmem_ops *ops,
 				      void *opaque)
 {
 	struct page *ignored_page = NULL;
+	bool writable = true;
 	kvm_pfn_t pfn;
 	int ret;
 
@@ -1093,10 +1097,17 @@ static int kvm_gmem_populate_one(const struct kvm_gmem_ops *ops,
 				    src_page, 0);
 	else
 		ret = ops->get_pfn(file, kvm, slot, gfn, &pfn,
-				   &ignored_page, NULL);
+				   &ignored_page, NULL, &writable);
 	if (ret)
 		return ret;
 
+	/* post_populate() writes the page, so it cannot be read-only. */
+	if (!writable) {
+		if (ignored_page)
+			put_page(ignored_page);
+		return -EPERM;
+	}
+
 	ret = post_populate(kvm, gfn, pfn, src_page, opaque);
 
 	/*

[PATCH 2/9] mm: Add memory providers

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A driver that owns memory outside the page allocator, often without
struct page, has no common way to lend it to the code that maps it.
guest_memfd and iommufd each need their own hooks, and the owner must
keep them consistent when it takes memory back.

Add an interface between such an owner, the provider, and its
consumers. A consumer attaches to a provider file, asks for the frame
and attributes behind each page, and makes every mapping itself. When
the provider changes a range, it revokes it, and every consumer of the
file removes its mappings before the revoke returns.

A frame is therefore valid until its revoke returns, and consumers hold
no references. A revoke reaches every consumer of the file, so a
provider cannot take a frame from one consumer while another still maps
it.

A provider is found from its file: its file_operations sit in a struct
mem_provider_fops with the provider's operations, and FOP_MEM_PROVIDER
is set. Each provider file embeds a struct mem_provider_file that holds
its consumers, and the provider revokes through it. The core has no
registry and no global state, and struct file_operations does not grow.

The provider also owns the host memory type of its frames. Memory that
is not System RAM must have its type reserved, or PAT makes it uncached
on x86.

Suggested-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 MAINTAINERS                  |   7 ++
 include/linux/fs.h           |   2 +
 include/linux/mem_provider.h | 211 +++++++++++++++++++++++++++++++++++
 mm/Kconfig                   |   7 ++
 mm/Makefile                  |   1 +
 mm/mem_provider.c            | 177 +++++++++++++++++++++++++++++
 6 files changed, 405 insertions(+)
 create mode 100644 include/linux/mem_provider.h
 create mode 100644 mm/mem_provider.c

diff --git a/MAINTAINERS b/MAINTAINERS
index f37a81950e25..6cab075a3ff7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17360,6 +17360,13 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
 F:	include/uapi/asm-generic/mman-common.h
 F:	mm/madvise.c
 
+MEMORY PROVIDERS
+M:	Fred Griffoul <fgriffo@amazon.co.uk>
+L:	linux-mm@kvack.org
+S:	Maintained
+F:	include/linux/mem_provider.h
+F:	mm/mem_provider.c
+
 MEMORY TECHNOLOGY DEVICES (MTD)
 M:	Miquel Raynal <miquel.raynal@bootlin.com>
 M:	Richard Weinberger <richard@nod.at>
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 50ce731a2b78..f459f0e34ca5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1980,6 +1980,8 @@ struct file_operations {
 #define FOP_ASYNC_LOCK		((__force fop_flags_t)(1 << 6))
 /* File system supports uncached read/write buffered IO */
 #define FOP_DONTCACHE		((__force fop_flags_t)(1 << 7))
+/* Lends memory to consumers: embedded in a struct mem_provider_fops */
+#define FOP_MEM_PROVIDER	((__force fop_flags_t)(1 << 8))
 
 /* Wrap a directory iterator that needs exclusive inode access */
 int wrap_directory_iterator(struct file *, struct dir_context *,
diff --git a/include/linux/mem_provider.h b/include/linux/mem_provider.h
new file mode 100644
index 000000000000..94e9352ebeb0
--- /dev/null
+++ b/include/linux/mem_provider.h
@@ -0,0 +1,211 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEM_PROVIDER_H
+#define _LINUX_MEM_PROVIDER_H
+
+#include <linux/bits.h>
+#include <linux/fs.h>
+#include <linux/list.h>
+#include <linux/rwsem.h>
+#include <linux/types.h>
+
+/*
+ * Memory providers
+ *
+ * A provider owns physical memory that the page allocator does not manage,
+ * and lends it to consumers through files that it hands to userspace.  A
+ * consumer is given such a file, attaches to it, and asks the provider for
+ * the frame behind each page.  The consumer makes every mapping of the
+ * frames itself, including the mappings into userspace; the provider makes
+ * none of them.
+ *
+ * When the frames or attributes behind a range change, the provider revokes
+ * the range.  The core calls every consumer attached to the file, and each
+ * one removes all its mappings of the range before the revoke returns.  A
+ * frame therefore stays valid for a consumer until the revoke that removes
+ * it has returned, and consumers hold no references.
+ *
+ * A consumer must not install a mapping of a frame after the revoke that
+ * removes it has returned.  It either holds a lock across get_page() and
+ * the map that its revoke callback also takes, or detects a revoke that ran
+ * in between and retries, as KVM does with its invalidation sequence.
+ *
+ * A revoke may also change the contents behind the range, for example when
+ * the provider moves a frame from one file to another.  A consumer that
+ * reports the pages written through its mappings, for live migration, must
+ * therefore treat the whole revoked range as written.
+ *
+ * A provider cannot take a frame back from one consumer only: a revoke
+ * reaches every consumer of the file.
+ *
+ * A provider gives its files a struct mem_provider_fops, which holds their
+ * file_operations, with FOP_MEM_PROVIDER and an owner set, and the provider's
+ * operations.  It embeds a struct mem_provider_file in the state of each
+ * file, returns it from attach(), and passes it to mem_provider_revoke().
+ * An attachment holds a reference to the file, so neither the file's state
+ * nor the module can go away while a consumer is attached.  MEM_PROVIDER
+ * has no prompt: a provider selects it, or depends on a consumer that does.
+ */
+
+/*
+ * Memory type of a frame, in the low bits of the attributes.  The provider
+ * makes the host's memory type for its frames match what it reports: for
+ * memory that is not System RAM, it reserves the type of the range, for
+ * example with memremap() or ioremap_wc(), before it hands frames out.  The
+ * architecture otherwise chooses the type of an unknown range, uncached on
+ * x86, and the VMM's mapping of a guest_memfd would differ from the guest's.
+ * On x86 the guest's type also depends on the host: KVM maps a frame uncached
+ * for the guest unless it is RAM in the firmware's E820 map, or a struct-page
+ * frame (DAX, ZONE_DEVICE) that PAT does not force uncached.  So the
+ * reservation matters for the guest too, not only for the VMM's mapping.
+ */
+#define MEM_PROVIDER_ATTR_TYPE		GENMASK(3, 0)
+#define MEM_PROVIDER_TYPE_RAM		0	/* cacheable system memory */
+#define MEM_PROVIDER_TYPE_MMIO		1	/* uncached device memory */
+#define MEM_PROVIDER_TYPE_MMIO_WC	2	/* write-combining device memory */
+						/* 3 to 15 are reserved */
+
+/*
+ * No consumer may map the frame writable: not for a guest, a device or
+ * userspace.
+ */
+#define MEM_PROVIDER_ATTR_READONLY	BIT(4)
+
+/*
+ * No consumer may map the frame into userspace.  Guest and device mappings
+ * are not affected.  The provider decides this for each page.
+ */
+#define MEM_PROVIDER_ATTR_NO_USER_MAP	BIT(5)
+
+#define MEM_PROVIDER_ATTR_VALID		(MEM_PROVIDER_ATTR_TYPE | \
+					 MEM_PROVIDER_ATTR_READONLY | \
+					 MEM_PROVIDER_ATTR_NO_USER_MAP)
+
+static inline unsigned int mem_provider_type(u32 attrs)
+{
+	return attrs & MEM_PROVIDER_ATTR_TYPE;
+}
+
+/*
+ * The provider side of one provider file, embedded in the provider's state
+ * for the file and set up with mem_provider_file_init().  It must stay until
+ * the file is released.  Its fields are private to the core.
+ */
+struct mem_provider_file {
+	struct rw_semaphore lock;
+	struct list_head consumers;
+};
+
+/**
+ * struct mem_provider_ops - What a provider implements.
+ */
+struct mem_provider_ops {
+	/**
+	 * @attach: A consumer starts to use @file.
+	 *
+	 * Called once for each consumer, and possibly for several at once.
+	 *
+	 * @size is the end of the range of the file that the consumer will
+	 * use, from offset 0, as the consumer's user asked for it: the size
+	 * of a guest_memfd, or the end of an iommufd mapping.  Return the
+	 * file's struct mem_provider_file, which is passed to the other
+	 * operations, or an ERR_PTR().  Return ERR_PTR(-EINVAL) if the file
+	 * is smaller than @size, so that the consumer fails the request at
+	 * once.  May sleep.
+	 */
+	struct mem_provider_file *(*attach)(struct file *file, loff_t size);
+
+	/**
+	 * @detach: A consumer is gone.
+	 *
+	 * Called once for each successful attach().  The consumer has removed
+	 * every mapping of the frames, and the core no longer calls its
+	 * revoke callback.  May sleep.
+	 */
+	void (*detach)(struct mem_provider_file *mpf);
+
+	/**
+	 * @get_page: Return the frame behind page @index.
+	 *
+	 * @pfn:       [out] The frame.
+	 * @max_order: [in, out] On entry, the largest order the consumer
+	 *             can use.  On return, the largest order for which the
+	 *             aligned block that contains @index is physically
+	 *             contiguous and has the same attributes.  It must not
+	 *             be larger than on entry.
+	 * @attrs:     [out] MEM_PROVIDER_ATTR_* for the block.
+	 *
+	 * Return 0, -EFAULT if the provider does not back the page now, or
+	 * another negative errno.
+	 *
+	 * May sleep, and may run at the same time as the provider changes
+	 * its state.  A result that is out of date is harmless, because the
+	 * provider revokes the range after the change.  Must not call into
+	 * the consumer, must not call mem_provider_revoke(), and must not take
+	 * a lock that the provider holds across mem_provider_revoke().
+	 */
+	int (*get_page)(struct mem_provider_file *mpf, pgoff_t index,
+			unsigned long *pfn, int *max_order, u32 *attrs);
+};
+
+/*
+ * The file_operations of a provider's files, with FOP_MEM_PROVIDER set in
+ * fops.fop_flags, and the provider's operations.
+ */
+struct mem_provider_fops {
+	struct file_operations fops;
+	const struct mem_provider_ops *ops;
+};
+
+/**
+ * struct mem_provider_attachment - One consumer's attachment to a provider
+ * file.  Embedded in the consumer's object, set by mem_provider_attach() and
+ * owned by the consumer until mem_provider_detach().  @ops must be NULL before
+ * the first mem_provider_attach(), for example by zeroing the attachment, so
+ * that mem_provider_detach() on an attachment that was never made does
+ * nothing.
+ * @ops: The provider's operations.  Read-only to the consumer.
+ * @mpf: The provider's per-file state.  Read-only to the consumer.
+ * @file: The provider file.  Read-only to the consumer.
+ * @size: The end of the range the consumer uses.  Read-only to the consumer.
+ * @revoke: Called when the frames behind a range change; see below.
+ * @node: Private to the core.
+ */
+struct mem_provider_attachment {
+	const struct mem_provider_ops *ops;
+	struct mem_provider_file *mpf;
+	struct file *file;
+	loff_t size;
+
+	/*
+	 * @revoke: The frames behind [@offset, @offset + @len) changed.
+	 *
+	 * The consumer removes every mapping it made of the range and asks
+	 * get_page() again when it next needs a page.  It must not return
+	 * while a mapping of the old frames remains, including one being
+	 * installed from an earlier get_page(): it excludes such a mapping or
+	 * makes it retry.  Revokes of the same file may call this at the same
+	 * time, and may call it before mem_provider_attach() returns.  May
+	 * sleep, and may call get_page().  Must not call
+	 * mem_provider_revoke(), and must not attach or detach any provider.
+	 */
+	void (*revoke)(struct mem_provider_attachment *att, loff_t offset,
+		       loff_t len);
+
+	struct list_head node;
+};
+
+/* Provider side. */
+void mem_provider_file_init(struct mem_provider_file *mpf);
+void mem_provider_revoke(struct mem_provider_file *mpf, loff_t offset,
+			 loff_t len);
+
+/* Consumer side. */
+int mem_provider_attach(struct mem_provider_attachment *att, struct file *file,
+			loff_t size,
+			void (*revoke)(struct mem_provider_attachment *att,
+				       loff_t offset, loff_t len));
+void mem_provider_detach(struct mem_provider_attachment *att);
+int mem_provider_get_page(struct mem_provider_attachment *att, pgoff_t index,
+			  unsigned long *pfn, int *max_order, u32 *attrs);
+
+#endif /* _LINUX_MEM_PROVIDER_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index 9e0ca4824905..7085bf6666e5 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -774,6 +774,13 @@ config DEFAULT_MMAP_MIN_ADDR
 config ARCH_SUPPORTS_MEMORY_FAILURE
 	bool
 
+config MEM_PROVIDER
+	bool
+	help
+	  An interface that lets a driver lend memory that the page allocator
+	  does not manage to consumers such as guest_memfd and iommufd, and
+	  take it back.  See include/linux/mem_provider.h.
+
 config MEMORY_FAILURE
 	depends on MMU
 	depends on ARCH_SUPPORTS_MEMORY_FAILURE
diff --git a/mm/Makefile b/mm/Makefile
index eff9f9e7e061..35420b587f89 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -110,6 +110,7 @@ obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o
 obj-$(CONFIG_GUP_TEST) += gup_test.o
 obj-$(CONFIG_DMAPOOL_TEST) += dmapool_test.o
 obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o
+obj-$(CONFIG_MEM_PROVIDER) += mem_provider.o
 obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o
 obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o
 obj-$(CONFIG_DEBUG_RODATA_TEST) += rodata_test.o
diff --git a/mm/mem_provider.c b/mm/mem_provider.c
new file mode 100644
index 000000000000..5a2984afde10
--- /dev/null
+++ b/mm/mem_provider.c
@@ -0,0 +1,177 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Memory providers: lend memory that the page allocator does not manage to
+ * consumers that map it.  See include/linux/mem_provider.h.
+ *
+ * Locking: the @lock of a struct mem_provider_file protects its list of
+ * consumers.  mem_provider_revoke() holds it for reading while it calls the
+ * consumers, so a consumer cannot detach until every revoke of its file has
+ * returned.  It follows that a revoke callback must not attach, detach or
+ * revoke, and that a consumer must not detach while it holds a lock that its
+ * revoke callback takes.  A revoke callback may call get_page(), which takes
+ * no lock of the core.
+ */
+#include <linux/export.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/mem_provider.h>
+#include <linux/mm.h>
+#include <linux/rwsem.h>
+
+/**
+ * mem_provider_file_init() - Set up the provider side of a provider file.
+ * @mpf: Embedded in the provider's state for the file.
+ */
+void mem_provider_file_init(struct mem_provider_file *mpf)
+{
+	init_rwsem(&mpf->lock);
+	INIT_LIST_HEAD(&mpf->consumers);
+}
+EXPORT_SYMBOL_GPL(mem_provider_file_init);
+
+/**
+ * mem_provider_attach() - Attach a consumer to a provider file.
+ * @att: The consumer's attachment, filled in on success.
+ * @file: A file handed out by a provider.
+ * @size: The end of the range of @file that the consumer will use, in bytes
+ *        from offset 0.  The provider refuses a file that is smaller.
+ * @revoke: Called when the frames behind a range of @file change.
+ *
+ * Holds a reference to @file, and so to the provider's module, until
+ * mem_provider_detach().
+ *
+ * Return: 0, -EINVAL if @size or @revoke is invalid or the provider lacks an
+ * operation, -ENODEV if @file is not a provider file, or the error returned
+ * by the provider.
+ */
+int mem_provider_attach(struct mem_provider_attachment *att, struct file *file,
+			loff_t size,
+			void (*revoke)(struct mem_provider_attachment *att,
+				       loff_t offset, loff_t len))
+{
+	const struct mem_provider_ops *ops;
+	struct mem_provider_file *mpf;
+
+	if (size <= 0 || !revoke)
+		return -EINVAL;
+
+	if (!(file->f_op->fop_flags & FOP_MEM_PROVIDER))
+		return -ENODEV;
+	ops = container_of(file->f_op, struct mem_provider_fops, fops)->ops;
+	if (WARN_ON_ONCE(!ops->attach || !ops->detach || !ops->get_page))
+		return -EINVAL;
+
+	mpf = ops->attach(file, size);
+	if (IS_ERR(mpf))
+		return PTR_ERR(mpf);
+
+	att->ops = ops;
+	att->mpf = mpf;
+	att->file = get_file(file);
+	att->size = size;
+	att->revoke = revoke;
+
+	down_write(&mpf->lock);
+	list_add_tail(&att->node, &mpf->consumers);
+	up_write(&mpf->lock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mem_provider_attach);
+
+/**
+ * mem_provider_detach() - Detach a consumer from a provider file.
+ * @att: An attachment from mem_provider_attach().
+ *
+ * The consumer must have removed every mapping of the provider's frames.
+ * Waits for revokes of the file that are in progress.  Does nothing if @att
+ * is not attached.
+ */
+void mem_provider_detach(struct mem_provider_attachment *att)
+{
+	struct mem_provider_file *mpf = att->mpf;
+
+	if (!att->ops)
+		return;
+
+	down_write(&mpf->lock);
+	list_del(&att->node);
+	up_write(&mpf->lock);
+
+	att->ops->detach(mpf);
+	fput(att->file);
+
+	att->ops = NULL;
+	att->mpf = NULL;
+	att->file = NULL;
+}
+EXPORT_SYMBOL_GPL(mem_provider_detach);
+
+/**
+ * mem_provider_get_page() - Get the frame behind a page of an attachment.
+ * @att: The attachment.
+ * @index: The page, in units of PAGE_SIZE from offset 0.
+ * @pfn: [out] The frame.
+ * @max_order: [in, out] See &mem_provider_ops.get_page.
+ * @attrs: [out] MEM_PROVIDER_ATTR_* for the frame.
+ *
+ * On return, @max_order is also limited so that the block is aligned in
+ * physical memory.  The frame stays valid until the consumer's revoke
+ * callback for the page has returned.
+ *
+ * Return: 0, -EFAULT if the provider does not back the page now, -EINVAL if
+ * @index is beyond the attachment, -EIO if the provider returned an invalid
+ * result, or another negative errno from the provider.
+ */
+int mem_provider_get_page(struct mem_provider_attachment *att, pgoff_t index,
+			  unsigned long *pfn, int *max_order, u32 *attrs)
+{
+	int order = *max_order;
+	int ret;
+
+	if (index >= DIV_ROUND_UP(att->size, PAGE_SIZE))
+		return -EINVAL;
+
+	*attrs = 0;
+	ret = att->ops->get_page(att->mpf, index, pfn, max_order, attrs);
+	if (ret)
+		return ret;
+
+	if (WARN_ON_ONCE(*max_order < 0 || *max_order > order ||
+			 (*attrs & ~MEM_PROVIDER_ATTR_VALID) ||
+			 mem_provider_type(*attrs) > MEM_PROVIDER_TYPE_MMIO_WC))
+		return -EIO;
+
+	/* A large mapping also needs the first frame of the block aligned. */
+	if (*max_order && ((*pfn ^ index) & ((1UL << *max_order) - 1)))
+		*max_order = __ffs(*pfn ^ index);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mem_provider_get_page);
+
+/**
+ * mem_provider_revoke() - Tell the consumers that frames changed.
+ * @mpf: The provider side of the file.
+ * @offset: The start of the range, in bytes.
+ * @len: The length of the range, in bytes.
+ *
+ * Call this after the provider has changed the frames or attributes behind
+ * the range.  Returns when every consumer of the file has removed its
+ * mappings of the range.  The provider must not hold a lock that its
+ * get_page() takes, because a consumer may call get_page() before it
+ * returns.  May sleep.
+ */
+void mem_provider_revoke(struct mem_provider_file *mpf, loff_t offset,
+			 loff_t len)
+{
+	struct mem_provider_attachment *att;
+
+	might_sleep();
+	if (len <= 0)
+		return;
+
+	down_read(&mpf->lock);
+	list_for_each_entry(att, &mpf->consumers, node)
+		att->revoke(att, offset, len);
+	up_read(&mpf->lock);
+}
+EXPORT_SYMBOL_GPL(mem_provider_revoke);

[PATCH 3/9] KVM: guest_memfd: Add a memory provider backing

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A driver can back a guest_memfd today only by implementing
kvm_gmem_ops itself, and must then keep any device mapping of the same
memory consistent on its own.

Add GUEST_MEMFD_FLAG_USE_PROVIDER. KVM_CREATE_GUEST_MEMFD then takes a
memory provider file in provider_fd, which uses the first reserved word
and must be zero without the flag. Guest faults ask the provider for
each frame, and a revoke removes the range from the guest and from the
VMM's mapping. A page that is not backed, or that is not RAM, exits with
KVM_EXIT_MEMORY_FAULT. Read-only pages are mapped read only, and
fallocate() is not supported.

guest_memfd maps itself into userspace from the same frames, so that
a provider needs no fault handler. Pages marked NO_USER_MAP raise
SIGBUS there.

USE_PROVIDER requires GUEST_MEMFD_FLAG_MMAP, so that the memory slot
is gmem-only. An architecture opts in; x86 does so for VMs without
private or encrypted memory, and other architectures refuse the flag
for now. Native guest_memfd files now take the invalidate lock when
a file is added and when a closing file is unbound.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 Documentation/virt/kvm/api.rst                |  45 ++-
 arch/x86/kvm/x86.c                            |  10 +
 include/linux/kvm_host.h                      |   4 +
 include/uapi/linux/kvm.h                      |  11 +-
 tools/include/uapi/linux/kvm.h                |  11 +-
 .../testing/selftests/kvm/guest_memfd_test.c  |   6 +
 virt/kvm/Kconfig                              |   1 +
 virt/kvm/guest_memfd.c                        | 265 +++++++++++++++++-
 8 files changed, 338 insertions(+), 15 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index a5f9ee92f43e..d9aaf8a76b59 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6431,7 +6431,9 @@ and cannot be resized  (guest_memfd files do however support PUNCH_HOLE).
   struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	__s32 provider_fd;
+	__u32 pad;
+	__u64 reserved[5];
   };
 
 Conceptually, the inode backing a guest_memfd file represents physical memory,
@@ -6453,15 +6455,38 @@ a single guest_memfd file, but the bound ranges must not overlap).
 The capability KVM_CAP_GUEST_MEMFD_FLAGS enumerates the `flags` that can be
 specified via KVM_CREATE_GUEST_MEMFD.  Currently defined flags:
 
-  ============================ ================================================
-  GUEST_MEMFD_FLAG_MMAP        Enable using mmap() on the guest_memfd file
-                               descriptor.
-  GUEST_MEMFD_FLAG_INIT_SHARED Make all memory in the file shared during
-                               KVM_CREATE_GUEST_MEMFD (memory files created
-                               without INIT_SHARED will be marked private).
-                               Shared memory can be faulted into host userspace
-                               page tables. Private memory cannot.
-  ============================ ================================================
+  ============================= ================================================
+  GUEST_MEMFD_FLAG_MMAP         Enable using mmap() on the guest_memfd file
+                                descriptor.
+  GUEST_MEMFD_FLAG_INIT_SHARED  Make all memory in the file shared during
+                                KVM_CREATE_GUEST_MEMFD (memory files created
+                                without INIT_SHARED will be marked private).
+                                Shared memory can be faulted into host userspace
+                                page tables. Private memory cannot.
+  GUEST_MEMFD_FLAG_USE_PROVIDER Take the memory of the file from the memory
+                                provider behind `provider_fd`, instead of
+                                allocating it.  Requires
+                                GUEST_MEMFD_FLAG_MMAP.  Not supported for VMs
+                                with private or encrypted memory.
+  ============================= ================================================
+
+`pad` must be zero.  With GUEST_MEMFD_FLAG_USE_PROVIDER, `provider_fd` is a file
+handed out by a memory provider (see include/linux/mem_provider.h); without it,
+`provider_fd` must be zero.
+The provider decides which pages exist and what backs them, and can change
+this at any time.  The guest and any host mapping of the guest_memfd follow
+the change.  A guest access to a page that the provider does not back exits
+to userspace with KVM_EXIT_MEMORY_FAULT, and so does an access to a page that
+is not RAM.  A page that the provider marks read only is mapped read only for
+the guest.  fallocate() is not supported.  Only x86 VMs of type
+KVM_X86_DEFAULT_VM support GUEST_MEMFD_FLAG_USE_PROVIDER.
+
+mmap() of the guest_memfd maps the provider's pages into userspace on fault.
+An access raises SIGBUS if the provider does not back the page, if the page is
+not RAM, if the provider marks it as not to be mapped into userspace, or if it
+is a write to a read-only page.
+KVM's own accesses through the memslot's userspace address fail in the same
+cases.
 
 When the KVM MMU performs a PFN lookup to service a guest fault and the backing
 guest_memfd has the GUEST_MEMFD_FLAG_MMAP set, then the fault will always be
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index afcac1042947..3669316f5712 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14124,6 +14124,16 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
 	return !kvm_arch_has_private_mem(kvm);
 }
 
+/*
+ * Only VMs without encrypted memory: SEV and SEV-ES guests have no private
+ * memory in KVM's sense, but their memory would need reclaiming when a
+ * provider takes it back.
+ */
+bool kvm_arch_gmem_supports_provider(struct kvm *kvm)
+{
+	return !kvm || kvm->arch.vm_type == KVM_X86_DEFAULT_VM;
+}
+
 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE
 int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order)
 {
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 7281d0e94121..3ea591642542 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -832,11 +832,15 @@ static inline bool kvm_arch_has_private_mem(struct kvm *kvm)
 
 #ifdef CONFIG_KVM_GUEST_MEMFD
 bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm);
+bool kvm_arch_gmem_supports_provider(struct kvm *kvm);
 
 static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
 {
 	u64 flags = GUEST_MEMFD_FLAG_MMAP;
 
+	if (kvm_arch_gmem_supports_provider(kvm))
+		flags |= GUEST_MEMFD_FLAG_USE_PROVIDER;
+
 	if (!kvm || kvm_arch_supports_gmem_init_shared(kvm))
 		flags |= GUEST_MEMFD_FLAG_INIT_SHARED;
 
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8..820448f82e22 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1654,11 +1654,20 @@ struct kvm_memory_attributes {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_PROVIDER	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	/*
+	 * With GUEST_MEMFD_FLAG_USE_PROVIDER: a memory provider file whose
+	 * pages back this guest_memfd.  The provider may take
+	 * any range back at any time; the guest and every host mapping of
+	 * this guest_memfd follow.  Must be 0 otherwise.
+	 */
+	__s32 provider_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index d0c0c8605976..0f4d1dd0e931 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1644,11 +1644,20 @@ struct kvm_memory_attributes {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_PROVIDER	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	/*
+	 * With GUEST_MEMFD_FLAG_USE_PROVIDER: a memory provider file whose
+	 * pages back this guest_memfd.  The provider may take
+	 * any range back at any time; the guest and every host mapping of
+	 * this guest_memfd follow.  Must be 0 otherwise.
+	 */
+	__s32 provider_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 2233d871a38f..91ff10ac6274 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -405,6 +405,12 @@ static void test_guest_memfd_flags(struct kvm_vm *vm)
 
 	for (flag = BIT(0); flag; flag <<= 1) {
 		fd = __vm_create_guest_memfd(vm, page_size, flag);
+		/* USE_PROVIDER also needs MMAP and a provider file. */
+		if (flag == GUEST_MEMFD_FLAG_USE_PROVIDER && (flag & valid_flags)) {
+			TEST_ASSERT(fd < 0 && errno == EINVAL,
+				    "guest_memfd() with USE_PROVIDER alone should fail with EINVAL");
+			continue;
+		}
 		if (flag & valid_flags) {
 			TEST_ASSERT(fd >= 0,
 				    "guest_memfd() with flag '0x%lx' should succeed",
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 794976b88c6f..cfb6c4e51128 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -105,6 +105,7 @@ config KVM_GENERIC_MEMORY_ATTRIBUTES
 
 config KVM_GUEST_MEMFD
        select XARRAY_MULTI
+       select MEM_PROVIDER
        bool
 
 config HAVE_KVM_ARCH_GMEM_PREPARE
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index a509f1a96c0b..aedd8630e3ea 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -4,6 +4,7 @@
 #include <linux/falloc.h>
 #include <linux/fs.h>
 #include <linux/kvm_host.h>
+#include <linux/mem_provider.h>
 #include <linux/mempolicy.h>
 #include <linux/pseudo_fs.h>
 #include <linux/pagemap.h>
@@ -44,6 +45,9 @@ struct gmem_inode {
 	struct list_head gmem_file_list;
 
 	u64 flags;
+
+	/* The provider of the pages, with GUEST_MEMFD_FLAG_USE_PROVIDER. */
+	struct mem_provider_attachment att;
 };
 
 static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
@@ -654,7 +658,222 @@ static const struct kvm_gmem_ops kvm_gmem_native_ops = {
 	.fallocate = kvm_gmem_native_fallocate,
 };
 
-static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
+/*
+ * Can @kvm use a memory provider?  @kvm is NULL for the system-wide capability.
+ * An architecture opts in by overriding this.  VMs whose memory is private or
+ * encrypted are not supported: each frame would need preparing before use and
+ * reclaiming when the provider takes it back.
+ */
+bool __weak kvm_arch_gmem_supports_provider(struct kvm *kvm)
+{
+	return false;
+}
+
+/*
+ * A guest_memfd backed by a memory provider (include/linux/mem_provider.h).
+ *
+ * The provider owns the frames, and guest_memfd keeps no state for them.  A
+ * guest fault asks the provider for the frame each time.  A fault that
+ * races with a change in the provider is retried by KVM, because the
+ * provider revokes the range after the change and the revoke opens and
+ * closes KVM's invalidation window.  Host mappings are made by guest_memfd
+ * from the same frames and attributes, and removed by the revoke.
+ *
+ * Bindings and release are the same as for the native guest_memfd.
+ */
+static void kvm_gmem_provider_revoke(struct mem_provider_attachment *att,
+				     loff_t offset, loff_t len)
+{
+	struct gmem_inode *gi = container_of(att, struct gmem_inode, att);
+	struct inode *inode = &gi->vfs_inode;
+	pgoff_t start, end;
+	struct gmem_file *f;
+
+	if (offset >= att->size)
+		return;
+	len = min(len, att->size - offset);
+	start = offset >> PAGE_SHIFT;
+	end = DIV_ROUND_UP(offset + len, PAGE_SIZE);
+
+	/*
+	 * The bindings must be stable so that each start is matched by an
+	 * end.  Only VMs without private memory use a provider, but remove
+	 * both kinds of mapping so that none is missed.
+	 */
+	filemap_invalidate_lock(inode->i_mapping);
+	kvm_gmem_for_each_file(f, inode)
+		__kvm_gmem_invalidate_start(f, start, end,
+					    KVM_FILTER_SHARED | KVM_FILTER_PRIVATE);
+	unmap_mapping_range(inode->i_mapping, (loff_t)start << PAGE_SHIFT,
+			    (loff_t)(end - start) << PAGE_SHIFT, 1);
+	kvm_gmem_for_each_file(f, inode)
+		__kvm_gmem_invalidate_end(f, start, end);
+	filemap_invalidate_unlock(inode->i_mapping);
+}
+
+static int kvm_gmem_provider_get_pfn(struct file *file, struct kvm *kvm,
+				     struct kvm_memory_slot *slot, gfn_t gfn,
+				     kvm_pfn_t *pfn, struct page **page,
+				     int *max_order, bool *writable)
+{
+	struct gmem_inode *gi = GMEM_I(file_inode(file));
+	pgoff_t index = kvm_gmem_get_index(slot, gfn);
+	unsigned long frame;
+	int order, ret;
+	u32 attrs;
+
+	if (file != READ_ONCE(slot->gmem.file))
+		return -EFAULT;
+	if (xa_load(&gmem_file_of(file)->bindings, index) != slot)
+		return -EIO;
+
+	/* A large mapping needs the block aligned in both index and gfn. */
+	order = PUD_ORDER;
+	if (index != gfn)
+		order = min_t(int, order, __ffs(index ^ gfn));
+
+	ret = mem_provider_get_page(&gi->att, index, &frame, &order, &attrs);
+	if (ret)
+		return ret;
+
+	/*
+	 * KVM decides the guest's memory type itself, so accept RAM only.
+	 * -EFAULT, so that the VMM gets a memory fault exit for the access.
+	 */
+	if (mem_provider_type(attrs) != MEM_PROVIDER_TYPE_RAM)
+		return -EFAULT;
+
+	if (attrs & MEM_PROVIDER_ATTR_READONLY) {
+		if (!writable)
+			return -EPERM;
+		*writable = false;
+	}
+
+	*pfn = frame;
+	if (max_order)
+		*max_order = order;
+	return 0;
+}
+
+/* The protection for a host mapping of a frame with @attrs. */
+static pgprot_t kvm_gmem_provider_prot(struct vm_area_struct *vma, u32 attrs)
+{
+	vm_flags_t flags = vma->vm_flags;
+
+	if (attrs & MEM_PROVIDER_ATTR_READONLY)
+		flags &= ~VM_WRITE;
+	return vm_get_page_prot(flags);
+}
+
+/*
+ * Map page @vmf->pgoff of a provider-backed guest_memfd for the host.
+ *
+ * The invalidate lock is held shared across get_page() and the insert, and
+ * a revoke holds it exclusive while it unmaps the range, so a frame is never
+ * mapped after the revoke that removes it.  A writable page is mapped
+ * writable, and a read-only page without write permission.  With @mkwrite,
+ * the PTE is read only and the write must be checked: replace it under the
+ * lock, so that a change to read only cannot slip between the check and the
+ * upgrade.
+ */
+static vm_fault_t kvm_gmem_provider_map_host(struct vm_fault *vmf, bool mkwrite)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct inode *inode = file_inode(vma->vm_file);
+	unsigned long uaddr = vmf->address & PAGE_MASK;
+	bool write = vmf->flags & FAULT_FLAG_WRITE;
+	unsigned long pfn;
+	int order = 0;
+	vm_fault_t ret;
+	u32 attrs;
+
+	if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
+		return VM_FAULT_SIGBUS;
+
+	filemap_invalidate_lock_shared(inode->i_mapping);
+	if (mem_provider_get_page(&GMEM_I(inode)->att, vmf->pgoff, &pfn,
+				  &order, &attrs)) {
+		ret = VM_FAULT_SIGBUS;
+	} else if (mem_provider_type(attrs) != MEM_PROVIDER_TYPE_RAM ||
+		   (attrs & MEM_PROVIDER_ATTR_NO_USER_MAP) ||
+		   (write && (attrs & MEM_PROVIDER_ATTR_READONLY))) {
+		ret = VM_FAULT_SIGBUS;
+	} else {
+		if (mkwrite)
+			zap_special_vma_range(vma, uaddr, PAGE_SIZE);
+		ret = vmf_insert_pfn_prot(vma, uaddr, pfn,
+					  kvm_gmem_provider_prot(vma, attrs));
+	}
+	filemap_invalidate_unlock_shared(inode->i_mapping);
+	return ret;
+}
+
+static vm_fault_t kvm_gmem_provider_fault(struct vm_fault *vmf)
+{
+	return kvm_gmem_provider_map_host(vmf, false);
+}
+
+/* A write to a page that a read fault, or mprotect(), left read only. */
+static vm_fault_t kvm_gmem_provider_pfn_mkwrite(struct vm_fault *vmf)
+{
+	return kvm_gmem_provider_map_host(vmf, true);
+}
+
+static const struct vm_operations_struct kvm_gmem_provider_vm_ops = {
+	.fault		= kvm_gmem_provider_fault,
+	.pfn_mkwrite	= kvm_gmem_provider_pfn_mkwrite,
+};
+
+static int kvm_gmem_provider_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct inode *inode = file_inode(file);
+	pgoff_t npages = i_size_read(inode) >> PAGE_SHIFT;
+
+	if (!kvm_gmem_supports_mmap(inode))
+		return -ENODEV;
+
+	if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) !=
+	    (VM_SHARED | VM_MAYSHARE))
+		return -EINVAL;
+
+	if (vma->vm_pgoff >= npages || vma_pages(vma) > npages - vma->vm_pgoff)
+		return -EINVAL;
+
+	/*
+	 * The frames may have no struct page.  They are inserted on fault, so
+	 * that a range that is revoked and comes back is reached again.
+	 */
+	vm_flags_set(vma, VM_PFNMAP | VM_IO | VM_DONTEXPAND | VM_DONTDUMP);
+	vma->vm_ops = &kvm_gmem_provider_vm_ops;
+	return 0;
+}
+
+static const struct kvm_gmem_ops kvm_gmem_provider_ops = {
+	.bind    = kvm_gmem_native_bind,
+	.unbind  = kvm_gmem_native_unbind,
+	.get_pfn = kvm_gmem_provider_get_pfn,
+	.release = kvm_gmem_native_release,
+	.mmap    = kvm_gmem_provider_mmap,
+	/* The provider decides which pages exist: no fallocate(). */
+};
+
+static int kvm_gmem_provider_attach(struct inode *inode, int provider_fd)
+{
+	struct file *file;
+	int ret;
+
+	file = fget(provider_fd);
+	if (!file)
+		return -EBADF;
+
+	ret = mem_provider_attach(&GMEM_I(inode)->att, file,
+				  i_size_read(inode), kvm_gmem_provider_revoke);
+	fput(file);
+	return ret;
+}
+
+static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags,
+			     int provider_fd)
 {
 	static const char *name = "[kvm-gmem]";
 	struct gmem_file *f;
@@ -695,6 +914,12 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 
 	GMEM_I(inode)->flags = flags;
 
+	if (flags & GUEST_MEMFD_FLAG_USE_PROVIDER) {
+		err = kvm_gmem_provider_attach(inode, provider_fd);
+		if (err)
+			goto err_inode;
+	}
+
 	file = alloc_file_pseudo(inode, kvm_gmem_mnt, name, O_RDWR, &kvm_gmem_fops);
 	if (IS_ERR(file)) {
 		err = PTR_ERR(file);
@@ -703,12 +928,18 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 
 	file->f_flags |= O_LARGEFILE;
 	file->private_data = &f->backing;
-	f->backing.ops = &kvm_gmem_native_ops;
+	if (flags & GUEST_MEMFD_FLAG_USE_PROVIDER)
+		f->backing.ops = &kvm_gmem_provider_ops;
+	else
+		f->backing.ops = &kvm_gmem_native_ops;
 
 	kvm_get_kvm(kvm);
 	f->kvm = kvm;
 	xa_init(&f->bindings);
+	/* A provider can revoke, and walk the list, as soon as it is attached. */
+	filemap_invalidate_lock(inode->i_mapping);
 	list_add(&f->entry, &GMEM_I(inode)->gmem_file_list);
+	filemap_invalidate_unlock(inode->i_mapping);
 
 	fd_install(fd, file);
 	return fd;
@@ -735,7 +966,20 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
 	if (size <= 0 || !PAGE_ALIGNED(size))
 		return -EINVAL;
 
-	return __kvm_gmem_create(kvm, size, flags);
+	if (args->pad ||
+	    (!(flags & GUEST_MEMFD_FLAG_USE_PROVIDER) && args->provider_fd))
+		return -EINVAL;
+
+	/*
+	 * Without MMAP the memslot is not gmem-only, and a VM without private
+	 * memory would fault through the slot's userspace address instead of
+	 * the provider.
+	 */
+	if ((flags & GUEST_MEMFD_FLAG_USE_PROVIDER) &&
+	    !(flags & GUEST_MEMFD_FLAG_MMAP))
+		return -EINVAL;
+
+	return __kvm_gmem_create(kvm, size, flags, args->provider_fd);
 }
 
 /*
@@ -911,9 +1155,14 @@ static void kvm_gmem_native_unbind(struct file *slot_file, struct kvm *kvm,
 	 * bindings.  I.e. reaching this point means kvm_gmem_release() hasn't
 	 * yet destroyed the bindings or freed the gmem_file, and can't do so
 	 * until the caller drops slots_lock.
+	 *
+	 * A memory provider can still revoke, and walk the bindings, until
+	 * the inode is evicted, so take the invalidate lock in this case too.
 	 */
 	if (!file) {
+		filemap_invalidate_lock(slot_file->f_mapping);
 		__kvm_gmem_unbind(slot, gmem_file_of(slot_file));
+		filemap_invalidate_unlock(slot_file->f_mapping);
 		return;
 	}
 
@@ -1219,6 +1468,7 @@ static struct inode *kvm_gmem_alloc_inode(struct super_block *sb)
 	mpol_shared_policy_init(&gi->policy, NULL);
 
 	gi->flags = 0;
+	gi->att.ops = NULL;
 	INIT_LIST_HEAD(&gi->gmem_file_list);
 	return &gi->vfs_inode;
 }
@@ -1233,10 +1483,19 @@ static void kvm_gmem_free_inode(struct inode *inode)
 	kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
 }
 
+static void kvm_gmem_evict_inode(struct inode *inode)
+{
+	/* Every file is closed, so nothing maps the provider's frames. */
+	mem_provider_detach(&GMEM_I(inode)->att);
+	truncate_inode_pages_final(&inode->i_data);
+	clear_inode(inode);
+}
+
 static const struct super_operations kvm_gmem_super_operations = {
 	.statfs		= simple_statfs,
 	.alloc_inode	= kvm_gmem_alloc_inode,
 	.destroy_inode	= kvm_gmem_destroy_inode,
+	.evict_inode	= kvm_gmem_evict_inode,
 	.free_inode	= kvm_gmem_free_inode,
 };
 

[PATCH 4/9] iommufd: Track the domains of pages that are not pinned

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Memory provider pages, added in the next patch, are not pinned and can
change under the domains that map them, as a dma-buf can. They need the
same list of (area, domain) pairs that the dma-buf revoke uses.

Move that list from struct iopt_pages_dmabuf to struct iopt_pages.

No functional change.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/iommu/iommufd/io_pagetable.c | 24 +++++-----
 drivers/iommu/iommufd/io_pagetable.h | 33 +++++++++-----
 drivers/iommu/iommufd/pages.c        | 67 ++++++++++++++--------------
 3 files changed, 67 insertions(+), 57 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 24d4917105d9..bcd531acc9dd 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -1008,14 +1008,14 @@ static void iopt_unfill_domain(struct io_pagetable *iopt,
 				WARN_ON(!area->storage_domain);
 			if (area->storage_domain == domain)
 				area->storage_domain = storage_domain;
-			if (iopt_is_dmabuf(pages)) {
+			if (iopt_pages_tracked(pages)) {
 				if (!iopt_dmabuf_revoked(pages))
 					iopt_area_unmap_domain(area, domain);
-				iopt_dmabuf_untrack_domain(pages, area, domain);
+				iopt_pages_untrack_domain(pages, area, domain);
 			}
 			mutex_unlock(&pages->mutex);
 
-			if (!iopt_is_dmabuf(pages))
+			if (!iopt_pages_tracked(pages))
 				iopt_area_unmap_domain(area, domain);
 		}
 		return;
@@ -1033,8 +1033,8 @@ static void iopt_unfill_domain(struct io_pagetable *iopt,
 		WARN_ON(area->storage_domain != domain);
 		area->storage_domain = NULL;
 		iopt_area_unfill_domain(area, pages, domain);
-		if (iopt_is_dmabuf(pages))
-			iopt_dmabuf_untrack_domain(pages, area, domain);
+		if (iopt_pages_tracked(pages))
+			iopt_pages_untrack_domain(pages, area, domain);
 		mutex_unlock(&pages->mutex);
 	}
 }
@@ -1065,15 +1065,15 @@ static int iopt_fill_domain(struct io_pagetable *iopt,
 			continue;
 
 		guard(mutex)(&pages->mutex);
-		if (iopt_is_dmabuf(pages)) {
-			rc = iopt_dmabuf_track_domain(pages, area, domain);
+		if (iopt_pages_tracked(pages)) {
+			rc = iopt_pages_track_domain(pages, area, domain);
 			if (rc)
 				goto out_unfill;
 		}
 		rc = iopt_area_fill_domain(area, domain);
 		if (rc) {
-			if (iopt_is_dmabuf(pages))
-				iopt_dmabuf_untrack_domain(pages, area, domain);
+			if (iopt_pages_tracked(pages))
+				iopt_pages_untrack_domain(pages, area, domain);
 			goto out_unfill;
 		}
 		if (!area->storage_domain) {
@@ -1102,8 +1102,8 @@ static int iopt_fill_domain(struct io_pagetable *iopt,
 			area->storage_domain = NULL;
 		}
 		iopt_area_unfill_domain(area, pages, domain);
-		if (iopt_is_dmabuf(pages))
-			iopt_dmabuf_untrack_domain(pages, area, domain);
+		if (iopt_pages_tracked(pages))
+			iopt_pages_untrack_domain(pages, area, domain);
 		mutex_unlock(&pages->mutex);
 	}
 	return rc;
@@ -1315,7 +1315,7 @@ static int iopt_area_split(struct iopt_area *area, unsigned long iova)
 		return -EBUSY;
 
 	/* Maintaining the domains_itree below is a bit complicated */
-	if (iopt_is_dmabuf(pages))
+	if (iopt_pages_tracked(pages))
 		return -EOPNOTSUPP;
 
 	if (new_start & (alignment - 1) ||
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index 887ed94474c7..5389227eb6ff 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -70,15 +70,14 @@ void iopt_area_unfill_domain(struct iopt_area *area, struct iopt_pages *pages,
 void iopt_area_unmap_domain(struct iopt_area *area,
 			    struct iommu_domain *domain);
 
-int iopt_dmabuf_track_domain(struct iopt_pages *pages, struct iopt_area *area,
-			     struct iommu_domain *domain);
-void iopt_dmabuf_untrack_domain(struct iopt_pages *pages,
-				struct iopt_area *area,
-				struct iommu_domain *domain);
-int iopt_dmabuf_track_all_domains(struct iopt_area *area,
-				  struct iopt_pages *pages);
-void iopt_dmabuf_untrack_all_domains(struct iopt_area *area,
-				     struct iopt_pages *pages);
+int iopt_pages_track_domain(struct iopt_pages *pages, struct iopt_area *area,
+			    struct iommu_domain *domain);
+void iopt_pages_untrack_domain(struct iopt_pages *pages, struct iopt_area *area,
+			       struct iommu_domain *domain);
+int iopt_pages_track_all_domains(struct iopt_area *area,
+				 struct iopt_pages *pages);
+void iopt_pages_untrack_all_domains(struct iopt_area *area,
+				    struct iopt_pages *pages);
 
 static inline unsigned long iopt_area_index(struct iopt_area *area)
 {
@@ -194,7 +193,8 @@ enum iopt_address_type {
 	IOPT_ADDRESS_DMABUF,
 };
 
-struct iopt_pages_dmabuf_track {
+/* An area of the pages mapped into a domain, for pages that are not pinned. */
+struct iopt_pages_track {
 	struct iommu_domain *domain;
 	struct iopt_area *area;
 	struct list_head elm;
@@ -205,7 +205,6 @@ struct iopt_pages_dmabuf {
 	struct phys_vec phys;
 	/* Always PAGE_SIZE aligned */
 	unsigned long start;
-	struct list_head tracker;
 	/*
 	 * true if the exporter's phys is CPU RAM (map with IOMMU_CACHE, no
 	 * IOMMU_MMIO); false for MMIO/BAR memory (map with IOMMU_MMIO).  Set
@@ -253,6 +252,12 @@ struct iopt_pages {
 	struct rb_root_cached access_itree;
 	/* Of iopt_area::pages_node */
 	struct rb_root_cached domains_itree;
+	/*
+	 * Of iopt_pages_track::elm.  Pages that are not pinned can change
+	 * under the domains that map them, so every (area, domain) that maps
+	 * them is listed here.  See iopt_pages_tracked().
+	 */
+	struct list_head tracker;
 };
 
 static inline bool iopt_is_dmabuf(struct iopt_pages *pages)
@@ -262,6 +267,12 @@ static inline bool iopt_is_dmabuf(struct iopt_pages *pages)
 	return pages->type == IOPT_ADDRESS_DMABUF;
 }
 
+/* The pages are not pinned, so their domains are tracked in pages->tracker. */
+static inline bool iopt_pages_tracked(struct iopt_pages *pages)
+{
+	return iopt_is_dmabuf(pages);
+}
+
 static inline bool iopt_dmabuf_revoked(struct iopt_pages *pages)
 {
 	lockdep_assert_held(&pages->mutex);
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index f9b2ae6d7e96..d68f6eea836d 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1392,6 +1392,7 @@ static struct iopt_pages *iopt_alloc_pages(unsigned long start_byte,
 	pages->npages = DIV_ROUND_UP(length + start_byte, PAGE_SIZE);
 	pages->access_itree = RB_ROOT_CACHED;
 	pages->domains_itree = RB_ROOT_CACHED;
+	INIT_LIST_HEAD(&pages->tracker);
 	pages->writable = writable;
 	if (capable(CAP_IPC_LOCK))
 		pages->account_mode = IOPT_PAGES_ACCOUNT_NONE;
@@ -1442,13 +1443,13 @@ struct iopt_pages *iopt_alloc_file_pages(struct file *file,
 static void iopt_revoke_notify(struct dma_buf_attachment *attach)
 {
 	struct iopt_pages *pages = attach->importer_priv;
-	struct iopt_pages_dmabuf_track *track;
+	struct iopt_pages_track *track;
 
 	guard(mutex)(&pages->mutex);
 	if (iopt_dmabuf_revoked(pages))
 		return;
 
-	list_for_each_entry(track, &pages->dmabuf.tracker, elm) {
+	list_for_each_entry(track, &pages->tracker, elm) {
 		struct iopt_area *area = track->area;
 
 		iopt_area_unmap_domain_range(area, track->domain,
@@ -1598,7 +1599,6 @@ struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 	pages->account_mode = IOPT_PAGES_ACCOUNT_NONE;
 	pages->type = IOPT_ADDRESS_DMABUF;
 	pages->dmabuf.start = start - start_byte;
-	INIT_LIST_HEAD(&pages->dmabuf.tracker);
 
 	rc = iopt_map_dmabuf(ictx, pages, dmabuf);
 	if (rc) {
@@ -1609,16 +1609,16 @@ struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 	return pages;
 }
 
-int iopt_dmabuf_track_domain(struct iopt_pages *pages, struct iopt_area *area,
-			     struct iommu_domain *domain)
+int iopt_pages_track_domain(struct iopt_pages *pages, struct iopt_area *area,
+			    struct iommu_domain *domain)
 {
-	struct iopt_pages_dmabuf_track *track;
+	struct iopt_pages_track *track;
 
 	lockdep_assert_held(&pages->mutex);
-	if (WARN_ON(!iopt_is_dmabuf(pages)))
+	if (WARN_ON(!iopt_pages_tracked(pages)))
 		return -EINVAL;
 
-	list_for_each_entry(track, &pages->dmabuf.tracker, elm)
+	list_for_each_entry(track, &pages->tracker, elm)
 		if (WARN_ON(track->domain == domain && track->area == area))
 			return -EINVAL;
 
@@ -1627,21 +1627,20 @@ int iopt_dmabuf_track_domain(struct iopt_pages *pages, struct iopt_area *area,
 		return -ENOMEM;
 	track->domain = domain;
 	track->area = area;
-	list_add_tail(&track->elm, &pages->dmabuf.tracker);
+	list_add_tail(&track->elm, &pages->tracker);
 
 	return 0;
 }
 
-void iopt_dmabuf_untrack_domain(struct iopt_pages *pages,
-				struct iopt_area *area,
-				struct iommu_domain *domain)
+void iopt_pages_untrack_domain(struct iopt_pages *pages, struct iopt_area *area,
+			       struct iommu_domain *domain)
 {
-	struct iopt_pages_dmabuf_track *track;
+	struct iopt_pages_track *track;
 
 	lockdep_assert_held(&pages->mutex);
-	WARN_ON(!iopt_is_dmabuf(pages));
+	WARN_ON(!iopt_pages_tracked(pages));
 
-	list_for_each_entry(track, &pages->dmabuf.tracker, elm) {
+	list_for_each_entry(track, &pages->tracker, elm) {
 		if (track->domain == domain && track->area == area) {
 			list_del(&track->elm);
 			kfree(track);
@@ -1651,36 +1650,36 @@ void iopt_dmabuf_untrack_domain(struct iopt_pages *pages,
 	WARN_ON(true);
 }
 
-int iopt_dmabuf_track_all_domains(struct iopt_area *area,
-				  struct iopt_pages *pages)
+int iopt_pages_track_all_domains(struct iopt_area *area,
+				 struct iopt_pages *pages)
 {
-	struct iopt_pages_dmabuf_track *track;
+	struct iopt_pages_track *track;
 	struct iommu_domain *domain;
 	unsigned long index;
 	int rc;
 
-	list_for_each_entry(track, &pages->dmabuf.tracker, elm)
+	list_for_each_entry(track, &pages->tracker, elm)
 		if (WARN_ON(track->area == area))
 			return -EINVAL;
 
 	xa_for_each(&area->iopt->domains, index, domain) {
-		rc = iopt_dmabuf_track_domain(pages, area, domain);
+		rc = iopt_pages_track_domain(pages, area, domain);
 		if (rc)
 			goto err_untrack;
 	}
 	return 0;
 err_untrack:
-	iopt_dmabuf_untrack_all_domains(area, pages);
+	iopt_pages_untrack_all_domains(area, pages);
 	return rc;
 }
 
-void iopt_dmabuf_untrack_all_domains(struct iopt_area *area,
-				     struct iopt_pages *pages)
+void iopt_pages_untrack_all_domains(struct iopt_area *area,
+				    struct iopt_pages *pages)
 {
-	struct iopt_pages_dmabuf_track *track;
-	struct iopt_pages_dmabuf_track *tmp;
+	struct iopt_pages_track *track;
+	struct iopt_pages_track *tmp;
 
-	list_for_each_entry_safe(track, tmp, &pages->dmabuf.tracker,
+	list_for_each_entry_safe(track, tmp, &pages->tracker,
 				 elm) {
 		if (track->area == area) {
 			list_del(&track->elm);
@@ -1697,6 +1696,7 @@ void iopt_release_pages(struct kref *kref)
 	WARN_ON(!RB_EMPTY_ROOT(&pages->domains_itree.rb_root));
 	WARN_ON(pages->npinned);
 	WARN_ON(!xa_empty(&pages->pinned_pfns));
+	WARN_ON(!list_empty(&pages->tracker));
 	if (iopt_is_dmabuf(pages) && pages->dmabuf.attach) {
 		struct dma_buf *dmabuf = pages->dmabuf.attach->dmabuf;
 
@@ -1705,7 +1705,6 @@ void iopt_release_pages(struct kref *kref)
 		dma_resv_unlock(dmabuf->resv);
 		dma_buf_detach(dmabuf, pages->dmabuf.attach);
 		dma_buf_put(dmabuf);
-		WARN_ON(!list_empty(&pages->dmabuf.tracker));
 	} else if (pages->type == IOPT_ADDRESS_FILE) {
 		fput(pages->file);
 	}
@@ -1790,7 +1789,7 @@ static void __iopt_area_unfill_domain(struct iopt_area *area,
 
 	lockdep_assert_held(&pages->mutex);
 
-	if (iopt_is_dmabuf(pages)) {
+	if (iopt_pages_tracked(pages)) {
 		if (WARN_ON(iopt_dmabuf_revoked(pages)))
 			return;
 		iopt_area_unmap_domain_range(area, domain, start_index,
@@ -1958,8 +1957,8 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		return 0;
 
 	mutex_lock(&pages->mutex);
-	if (iopt_is_dmabuf(pages)) {
-		rc = iopt_dmabuf_track_all_domains(area, pages);
+	if (iopt_pages_tracked(pages)) {
+		rc = iopt_pages_track_all_domains(area, pages);
 		if (rc)
 			goto out_unlock;
 	}
@@ -2024,8 +2023,8 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 	}
 	pfn_reader_destroy(&pfns);
 out_untrack:
-	if (iopt_is_dmabuf(pages))
-		iopt_dmabuf_untrack_all_domains(area, pages);
+	if (iopt_pages_tracked(pages))
+		iopt_pages_untrack_all_domains(area, pages);
 out_unlock:
 	mutex_unlock(&pages->mutex);
 	return rc;
@@ -2065,8 +2064,8 @@ void iopt_area_unfill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		WARN_ON(RB_EMPTY_NODE(&area->pages_node.rb));
 	interval_tree_remove(&area->pages_node, &pages->domains_itree);
 	iopt_area_unfill_domain(area, pages, area->storage_domain);
-	if (iopt_is_dmabuf(pages))
-		iopt_dmabuf_untrack_all_domains(area, pages);
+	if (iopt_pages_tracked(pages))
+		iopt_pages_untrack_all_domains(area, pages);
 	area->storage_domain = NULL;
 out_unlock:
 	mutex_unlock(&pages->mutex);

[PATCH 5/9] iommufd: Map memory provider files

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

IOMMU_IOAS_MAP_FILE pins the pages it maps, which cannot work for a
memory provider file: the frames may have no struct page, and the
provider can take any of them back at any time.

Accept a provider file. iommufd neither pins nor accounts its pages.
It asks the provider for each frame when it fills a domain, and on a
revoke it unmaps the range in every domain and maps what the provider
backs now. A device that accesses the range in between faults.

Each frame gets a PAGE_SIZE entry, so a partial revoke never splits a
large IOMMU page. Holes stay unmapped, read-only pages are mapped
without IOMMU_WRITE, and MMIO pages with IOMMU_MMIO. A provider that
returns frame 0 makes the map fail with -EINVAL.

A revoke would race a dirty bitmap read and lose the dirty bits of the
range, so provider pages and a dirty tracking domain cannot share an
IOAS: whichever comes second fails with -EOPNOTSUPP. A provider
mapping cannot be split by a partial unmap, and in-kernel accesses to
it are refused.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/iommu/iommufd/Kconfig        |   1 +
 drivers/iommu/iommufd/io_pagetable.c |  13 +-
 drivers/iommu/iommufd/io_pagetable.h |  23 ++-
 drivers/iommu/iommufd/pages.c        | 273 ++++++++++++++++++++++++++-
 include/uapi/linux/iommufd.h         |  11 +-
 5 files changed, 315 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig
index 455bac0351f2..65d71e28c12b 100644
--- a/drivers/iommu/iommufd/Kconfig
+++ b/drivers/iommu/iommufd/Kconfig
@@ -8,6 +8,7 @@ config IOMMUFD
 	select INTERVAL_TREE
 	select INTERVAL_TREE_SPAN_ITER
 	select IOMMU_API
+	select MEM_PROVIDER
 	default n
 	help
 	  Provides /dev/iommu, the user API to control the IOMMU subsystem as
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index bcd531acc9dd..3eca7f4f83d3 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -217,6 +217,7 @@ static int iopt_insert_area(struct io_pagetable *iopt, struct iopt_area *area,
 		return -EPERM;
 
 	area->iommu_prot = iommu_prot;
+	area->provider = iopt_is_provider(pages);
 	area->page_offset = start_byte % PAGE_SIZE;
 	if (area->page_offset & (iopt->iova_alignment - 1))
 		return -EINVAL;
@@ -289,6 +290,9 @@ static int iopt_alloc_area_pages(struct io_pagetable *iopt,
 		case IOPT_ADDRESS_DMABUF:
 			start = elm->start_byte + elm->pages->dmabuf.start;
 			break;
+		case IOPT_ADDRESS_PROVIDER:
+			start = elm->start_byte + elm->pages->provider.start;
+			break;
 		}
 		rc = iopt_alloc_iova(iopt, dst_iova, start, length);
 		if (rc)
@@ -515,8 +519,13 @@ int iopt_map_file_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
 		if (!file)
 			return -EBADF;
 
-		pages = iopt_alloc_file_pages(file, start_byte, start, length,
-					      iommu_prot & IOMMU_WRITE);
+		/* A memory provider file, or else a memfd. */
+		pages = iopt_alloc_provider_pages(file, start, length,
+						  iommu_prot & IOMMU_WRITE);
+		if (PTR_ERR(pages) == -ENODEV)
+			pages = iopt_alloc_file_pages(file, start_byte, start,
+						      length,
+						      iommu_prot & IOMMU_WRITE);
 		fput(file);
 		if (IS_ERR(pages))
 			return PTR_ERR(pages);
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index 5389227eb6ff..3f65380e91d2 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -8,6 +8,7 @@
 #include <linux/dma-buf.h>
 #include <linux/interval_tree.h>
 #include <linux/kref.h>
+#include <linux/mem_provider.h>
 #include <linux/mutex.h>
 #include <linux/xarray.h>
 
@@ -48,6 +49,8 @@ struct iopt_area {
 	/* IOMMU_READ, IOMMU_WRITE, etc */
 	int iommu_prot;
 	bool prevent_access : 1;
+	/* The pages come from a memory provider and may have holes */
+	bool provider : 1;
 	unsigned int num_accesses;
 	unsigned int num_locks;
 };
@@ -191,6 +194,7 @@ enum iopt_address_type {
 	IOPT_ADDRESS_USER = 0,
 	IOPT_ADDRESS_FILE,
 	IOPT_ADDRESS_DMABUF,
+	IOPT_ADDRESS_PROVIDER,
 };
 
 /* An area of the pages mapped into a domain, for pages that are not pinned. */
@@ -214,6 +218,12 @@ struct iopt_pages_dmabuf {
 	bool is_cpu_ram;
 };
 
+struct iopt_pages_provider {
+	struct mem_provider_attachment att;
+	/* Byte offset in the provider file, always PAGE_SIZE aligned */
+	unsigned long start;
+};
+
 /*
  * This holds a pinned page list for multiple areas of IO address space. The
  * pages always originate from a linear chunk of userspace VA. Multiple
@@ -243,6 +253,8 @@ struct iopt_pages {
 		};
 		/* IOPT_ADDRESS_DMABUF */
 		struct iopt_pages_dmabuf dmabuf;
+		/* IOPT_ADDRESS_PROVIDER */
+		struct iopt_pages_provider provider;
 	};
 	bool writable:1;
 	u8 account_mode;
@@ -267,10 +279,15 @@ static inline bool iopt_is_dmabuf(struct iopt_pages *pages)
 	return pages->type == IOPT_ADDRESS_DMABUF;
 }
 
+static inline bool iopt_is_provider(struct iopt_pages *pages)
+{
+	return pages->type == IOPT_ADDRESS_PROVIDER;
+}
+
 /* The pages are not pinned, so their domains are tracked in pages->tracker. */
 static inline bool iopt_pages_tracked(struct iopt_pages *pages)
 {
-	return iopt_is_dmabuf(pages);
+	return iopt_is_dmabuf(pages) || iopt_is_provider(pages);
 }
 
 static inline bool iopt_dmabuf_revoked(struct iopt_pages *pages)
@@ -292,6 +309,10 @@ struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 					   unsigned long start_byte,
 					   unsigned long start,
 					   unsigned long length, bool writable);
+struct iopt_pages *iopt_alloc_provider_pages(struct file *file,
+					     unsigned long start,
+					     unsigned long length,
+					     bool writable);
 void iopt_release_pages(struct kref *kref);
 static inline void iopt_put_pages(struct iopt_pages *pages)
 {
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index d68f6eea836d..99c95602ac4e 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -52,6 +52,7 @@
 #include <linux/iommu.h>
 #include <linux/iommufd.h>
 #include <linux/kthread.h>
+#include <linux/mem_provider.h>
 #include <linux/overflow.h>
 #include <linux/slab.h>
 #include <linux/sched/mm.h>
@@ -238,6 +239,35 @@ static void iommu_unmap_nofail(struct iommu_domain *domain, unsigned long iova,
 	WARN_ON(ret != size);
 }
 
+/*
+ * Provider pages are mapped with PAGE_SIZE entries, and may have holes.  Some
+ * IOMMU drivers warn when asked to unmap an IOVA that is not mapped, so find
+ * the runs that are mapped and unmap each one, without touching the holes.
+ * iopt_provider_map() refuses frame 0, which iova_to_phys() reports for
+ * an IOVA that is not mapped.
+ */
+static void iopt_provider_unmap(struct iopt_area *area,
+				struct iommu_domain *domain,
+				unsigned long start_index,
+				unsigned long last_index)
+{
+	unsigned long iova = iopt_area_index_to_iova(area, start_index);
+	size_t left = (last_index - start_index + 1) * PAGE_SIZE;
+
+	while (left) {
+		size_t len = 0;
+
+		while (len < left && iommu_iova_to_phys(domain, iova + len))
+			len += PAGE_SIZE;
+		if (len)
+			iommu_unmap_nofail(domain, iova, len);
+		/* Step over the run and the hole page that ended it. */
+		len = min(len + PAGE_SIZE, left);
+		iova += len;
+		left -= len;
+	}
+}
+
 static void iopt_area_unmap_domain_range(struct iopt_area *area,
 					 struct iommu_domain *domain,
 					 unsigned long start_index,
@@ -245,6 +275,11 @@ static void iopt_area_unmap_domain_range(struct iopt_area *area,
 {
 	unsigned long start_iova = iopt_area_index_to_iova(area, start_index);
 
+	if (area->provider) {
+		iopt_provider_unmap(area, domain, start_index, last_index);
+		return;
+	}
+
 	iommu_unmap_nofail(domain, start_iova,
 			   iopt_area_index_to_iova_last(area, last_index) -
 				   start_iova + 1);
@@ -1357,6 +1392,10 @@ static int pfn_reader_first(struct pfn_reader *pfns, struct iopt_pages *pages,
 	    WARN_ON(last_index < start_index))
 		return -EINVAL;
 
+	/* Provider pages are read from the provider, see iopt_provider_map() */
+	if (WARN_ON(iopt_is_provider(pages)))
+		return -EINVAL;
+
 	rc = pfn_reader_init(pfns, pages, start_index, last_index);
 	if (rc)
 		return rc;
@@ -1688,6 +1727,215 @@ void iopt_pages_untrack_all_domains(struct iopt_area *area,
 	}
 }
 
+static int iopt_provider_prot(struct iopt_area *area, u32 attrs)
+{
+	int prot = area->iommu_prot;
+
+	if (attrs & MEM_PROVIDER_ATTR_READONLY)
+		prot &= ~IOMMU_WRITE;
+	if (mem_provider_type(attrs) != MEM_PROVIDER_TYPE_RAM) {
+		prot &= ~IOMMU_CACHE;
+		prot |= IOMMU_MMIO;
+	}
+	return prot;
+}
+
+/*
+ * Map what the provider backs in [start_index, last_index] of the area into
+ * @domain, and leave the holes unmapped.  Each frame is mapped with a
+ * PAGE_SIZE entry, so that a revoke of part of a large block never has to
+ * split a larger IOMMU page.  On failure nothing in the range is mapped.
+ */
+static int iopt_provider_map(struct iopt_area *area, struct iopt_pages *pages,
+			     struct iommu_domain *domain,
+			     unsigned long start_index,
+			     unsigned long last_index)
+{
+	unsigned long base = pages->provider.start >> PAGE_SHIFT;
+	unsigned long index = start_index;
+	int rc;
+
+	lockdep_assert_held(&pages->mutex);
+
+	if ((1UL << __ffs(domain->pgsize_bitmap)) > PAGE_SIZE)
+		return -EOPNOTSUPP;
+
+	/*
+	 * A revoke unmaps under pages->mutex only, so it would race with a
+	 * dirty bitmap read, and lose the dirty bits of what it unmaps.
+	 * Provider pages are not mapped in a domain with dirty tracking.
+	 */
+	if (domain->dirty_ops)
+		return -EOPNOTSUPP;
+
+	while (index <= last_index) {
+		unsigned long pfn, iova, block_end, nr, i;
+		int order = PUD_ORDER;
+		u32 attrs;
+		int prot;
+
+		rc = mem_provider_get_page(&pages->provider.att, base + index,
+					   &pfn, &order, &attrs);
+		if (rc == -EFAULT) {
+			index++;
+			continue;
+		}
+		/* Frame 0 would look unmapped to iopt_provider_unmap(). */
+		if (!rc && !pfn)
+			rc = -EINVAL;
+		if (rc)
+			goto err_unmap;
+
+		block_end = ALIGN_DOWN(base + index, 1UL << order) +
+			    (1UL << order) - base;
+		nr = min(block_end, last_index + 1) - index;
+		iova = iopt_area_index_to_iova(area, index);
+		prot = iopt_provider_prot(area, attrs);
+
+		for (i = 0; i < nr; i++) {
+			rc = iommu_map_nosync(domain, iova + i * PAGE_SIZE,
+					      PFN_PHYS(pfn + i), PAGE_SIZE, prot,
+					      GFP_KERNEL_ACCOUNT);
+			if (rc)
+				break;
+		}
+		if (i) {
+			int sync_rc = iommu_sync_map(domain, iova, i * PAGE_SIZE);
+
+			if (!rc)
+				rc = sync_rc;
+		}
+		index += i;
+		if (rc)
+			goto err_unmap;
+	}
+	return 0;
+
+err_unmap:
+	if (index > start_index)
+		iopt_provider_unmap(area, domain, start_index, index - 1);
+	return rc;
+}
+
+/* Map the whole area into every domain of its io_pagetable. */
+static int iopt_provider_fill_domains(struct iopt_area *area,
+				      struct iopt_pages *pages)
+{
+	struct iommu_domain *domain, *undo;
+	unsigned long index, undo_index;
+	int rc;
+
+	xa_for_each(&area->iopt->domains, index, domain) {
+		rc = iopt_provider_map(area, pages, domain,
+				       iopt_area_index(area),
+				       iopt_area_last_index(area));
+		if (rc)
+			goto err_unmap;
+	}
+	return 0;
+
+err_unmap:
+	xa_for_each(&area->iopt->domains, undo_index, undo) {
+		if (undo_index >= index)
+			break;
+		iopt_provider_unmap(area, undo, iopt_area_index(area),
+				    iopt_area_last_index(area));
+	}
+	return rc;
+}
+
+/*
+ * The provider changed the frames behind [offset, offset + len).  In every
+ * domain, unmap the range and map what the provider backs now.  A device that
+ * accesses the range in between faults.
+ */
+static void iopt_provider_revoke(struct mem_provider_attachment *att,
+				 loff_t offset, loff_t len)
+{
+	struct iopt_pages *pages =
+		container_of(att, struct iopt_pages, provider.att);
+	u64 start = pages->provider.start;
+	u64 end = start + (u64)pages->npages * PAGE_SIZE;
+	struct iopt_pages_track *track;
+	unsigned long first, last;
+
+	if (offset < 0 || len <= 0 || offset >= end || offset + len <= start)
+		return;
+	first = (max_t(u64, offset, start) - start) >> PAGE_SHIFT;
+	last = (min_t(u64, offset + len, end) - start - 1) >> PAGE_SHIFT;
+
+	guard(mutex)(&pages->mutex);
+	list_for_each_entry(track, &pages->tracker, elm) {
+		struct iopt_area *area = track->area;
+		unsigned long s = max(first, iopt_area_index(area));
+		unsigned long l = min(last, iopt_area_last_index(area));
+
+		if (s > l)
+			continue;
+		iopt_provider_unmap(area, track->domain, s, l);
+		if (iopt_provider_map(area, pages, track->domain, s, l))
+			pr_warn_ratelimited("iommufd: cannot map provider pages after a revoke\n");
+	}
+}
+
+/**
+ * iopt_alloc_provider_pages() - Pages backed by a memory provider file
+ * @file: The provider file
+ * @start: Byte offset in the file, PAGE_SIZE aligned
+ * @length: Number of bytes
+ * @writable: The pages may be mapped writable
+ *
+ * The pages are not pinned.  The provider can change the frames behind them
+ * at any time, and every domain that maps them follows.
+ *
+ * Return: the pages, ERR_PTR(-ENODEV) if @file is not a provider file,
+ * ERR_PTR(-EINVAL) if @start is not page aligned, or another ERR_PTR().
+ */
+struct iopt_pages *iopt_alloc_provider_pages(struct file *file,
+					     unsigned long start,
+					     unsigned long length,
+					     bool writable)
+{
+	static struct lock_class_key pages_provider_mutex_key;
+	struct iopt_pages *pages;
+	int rc;
+
+	if (length / PAGE_SIZE >= MAX_NPFNS)
+		return ERR_PTR(-EINVAL);
+
+	pages = iopt_alloc_pages(0, length, writable);
+	if (IS_ERR(pages))
+		return pages;
+
+	/*
+	 * The pages mutex of provider pages is never held while taking the
+	 * mmap_lock, but is taken from the provider's revoke.  Split the lock
+	 * class from the pinned pages.
+	 */
+	lockdep_set_class(&pages->mutex, &pages_provider_mutex_key);
+
+	/* Provider pages are not pinned, so they are not accounted. */
+	pages->account_mode = IOPT_PAGES_ACCOUNT_NONE;
+	pages->type = IOPT_ADDRESS_PROVIDER;
+	pages->provider.start = start;
+
+	/*
+	 * The provider must cover the whole mapping, so the size is its end.
+	 * Check the alignment only once the file is known to be a provider
+	 * file: on -ENODEV the caller maps it as a memfd, which may start
+	 * anywhere.
+	 */
+	rc = mem_provider_attach(&pages->provider.att, file, start + length,
+				 iopt_provider_revoke);
+	if (!rc && !PAGE_ALIGNED(start))
+		rc = -EINVAL;
+	if (rc) {
+		iopt_put_pages(pages);
+		return ERR_PTR(rc);
+	}
+	return pages;
+}
+
 void iopt_release_pages(struct kref *kref)
 {
 	struct iopt_pages *pages = container_of(kref, struct iopt_pages, kref);
@@ -1705,6 +1953,8 @@ void iopt_release_pages(struct kref *kref)
 		dma_resv_unlock(dmabuf->resv);
 		dma_buf_detach(dmabuf, pages->dmabuf.attach);
 		dma_buf_put(dmabuf);
+	} else if (iopt_is_provider(pages)) {
+		mem_provider_detach(&pages->provider.att);
 	} else if (pages->type == IOPT_ADDRESS_FILE) {
 		fput(pages->file);
 	}
@@ -1855,6 +2105,12 @@ static void iopt_area_unfill_partial_domain(struct iopt_area *area,
  */
 void iopt_area_unmap_domain(struct iopt_area *area, struct iommu_domain *domain)
 {
+	if (area->provider) {
+		iopt_provider_unmap(area, domain, iopt_area_index(area),
+				    iopt_area_last_index(area));
+		return;
+	}
+
 	iommu_unmap_nofail(domain, iopt_area_iova(area),
 			   iopt_area_length(area));
 }
@@ -1898,6 +2154,11 @@ int iopt_area_fill_domain(struct iopt_area *area, struct iommu_domain *domain)
 	if (iopt_dmabuf_revoked(area->pages))
 		return 0;
 
+	if (iopt_is_provider(area->pages))
+		return iopt_provider_map(area, area->pages, domain,
+					 iopt_area_index(area),
+					 iopt_area_last_index(area));
+
 	rc = pfn_reader_first(&pfns, area->pages, iopt_area_index(area),
 			      iopt_area_last_index(area));
 	if (rc)
@@ -1963,7 +2224,11 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 			goto out_unlock;
 	}
 
-	if (!iopt_dmabuf_revoked(pages)) {
+	if (iopt_is_provider(pages)) {
+		rc = iopt_provider_fill_domains(area, pages);
+		if (rc)
+			goto out_untrack;
+	} else if (!iopt_dmabuf_revoked(pages)) {
 		rc = pfn_reader_first(&pfns, pages, iopt_area_index(area),
 				      iopt_area_last_index(area));
 		if (rc)
@@ -2402,7 +2667,7 @@ int iopt_pages_rw_access(struct iopt_pages *pages, unsigned long start_byte,
 	if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable)
 		return -EPERM;
 
-	if (iopt_is_dmabuf(pages))
+	if (iopt_is_dmabuf(pages) || iopt_is_provider(pages))
 		return -EINVAL;
 
 	if (pages->type != IOPT_ADDRESS_USER)
@@ -2491,6 +2756,10 @@ int iopt_area_add_access(struct iopt_area *area, unsigned long start_index,
 	if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable)
 		return -EPERM;
 
+	/* Provider frames may have no struct page, and are not pinned. */
+	if (iopt_is_provider(pages))
+		return -EOPNOTSUPP;
+
 	mutex_lock(&pages->mutex);
 	access = iopt_pages_get_exact_access(pages, start_index, last_index);
 	if (access) {
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 0425d452d41e..b4500fc21368 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -224,7 +224,7 @@ struct iommu_ioas_map {
  * @size: sizeof(struct iommu_ioas_map_file)
  * @flags: same as for iommu_ioas_map
  * @ioas_id: same as for iommu_ioas_map
- * @fd: the memfd or supported dma-buf file to map
+ * @fd: the memfd, memory provider file or supported dma-buf file to map
  * @start: byte offset from start of the file to map from
  * @length: same as for iommu_ioas_map
  * @iova: same as for iommu_ioas_map
@@ -235,6 +235,15 @@ struct iommu_ioas_map {
  * VFIO PCI dma-bufs exported through VFIO_DEVICE_FEATURE_DMA_BUF, and
  * other dma-bufs may be rejected. All other arguments and semantics match
  * those of IOMMU_IOAS_MAP.
+ *
+ * A file from a memory provider is also accepted; @start must then be
+ * page aligned. Its pages are not pinned. The provider may change the memory
+ * behind any range at any time, and the mapping follows: pages that the
+ * provider does not back are left unmapped, read-only pages are mapped
+ * without write permission, and device memory is mapped as MMIO. A device
+ * access to a page that is being changed, or that is not backed, faults. Such
+ * a mapping cannot be split by a partial unmap, and in-kernel accesses to it
+ * are refused. It is not supported in an IOAS with a dirty tracking domain.
  */
 struct iommu_ioas_map_file {
 	__u32 size;

[PATCH 6/9] iommufd/selftest: Add mock-domain IOVA queries

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

A test of memory that can be taken back or replaced does not know in
advance which frame a device reaches. IOMMU_TEST_OP_MD_CHECK_MAP only
compares against a known buffer.

Add MD_CHECK_MAPPED, which checks that a range is fully mapped or fully
unmapped, and MD_IOVA_TO_PHYS, which returns the frame behind an IOVA,
or 0 if it is unmapped. Both refuse an IOVA outside the domain's
aperture.

Both hold domains_rwsem for writing so that a userspace unmap cannot
free a page-table level during the walk.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/iommu/iommufd/iommufd_test.h |  16 ++++
 drivers/iommu/iommufd/selftest.c     | 109 +++++++++++++++++++++++++++
 2 files changed, 125 insertions(+)

diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h
index 52b78cbcc920..28fd9c43edc4 100644
--- a/drivers/iommu/iommufd/iommufd_test.h
+++ b/drivers/iommu/iommufd/iommufd_test.h
@@ -31,6 +31,8 @@ enum {
 	IOMMU_TEST_OP_PASID_CHECK_HWPT,
 	IOMMU_TEST_OP_DMABUF_GET,
 	IOMMU_TEST_OP_DMABUF_REVOKE,
+	IOMMU_TEST_OP_MD_CHECK_MAPPED,
+	IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
 };
 
 enum {
@@ -193,6 +195,20 @@ struct iommu_test_cmd {
 			__s32 dmabuf_fd;
 			__u32 revoked;
 		} dmabuf_revoke;
+		struct {
+			/*
+			 * 1: every page in [iova, iova+length) must be mapped;
+			 * 0: none of them may be. Mixed is an error.
+			 */
+			__u32 mapped;
+			__u32 __reserved;
+			__aligned_u64 iova;
+			__aligned_u64 length;
+		} check_mapped;
+		struct {
+			__aligned_u64 iova;
+			__aligned_u64 out_phys;	/* 0 if unmapped */
+		} iova_to_phys;
 	};
 	__u32 last;
 };
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index af07c642a526..1f5cd2d00fda 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -2031,6 +2031,107 @@ static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
 	return rc;
 }
 
+/*
+ * True if [iova, iova + length) lies inside the domain's aperture.  Outside
+ * it the page table returns an error code from iova_to_phys(), not 0.
+ */
+static bool mock_domain_covers(struct mock_iommu_domain *mock,
+			       unsigned long iova, size_t length)
+{
+	struct iommu_domain_geometry *geo = &mock->domain.geometry;
+	unsigned long last;
+
+	if (check_add_overflow(iova, length - 1, &last))
+		return false;
+	return iova >= geo->aperture_start && last <= geo->aperture_end;
+}
+
+/*
+ * iova_to_phys() walks the page table, so it must not run while an unmap
+ * frees a level of it.  A userspace unmap holds the IOAS domains_rwsem for
+ * reading while it unmaps the domains, so holding it for writing keeps such
+ * unmaps away.  A memory provider or dma-buf revoke unmaps under its pages
+ * mutex only, so tests must not run these queries while a revoke of the
+ * range is in progress.
+ */
+static struct rw_semaphore *
+mock_domain_unmap_lock(struct iommufd_hw_pagetable *hwpt)
+{
+	return &to_hwpt_paging(hwpt)->ioas->iopt.domains_rwsem;
+}
+
+/*
+ * Report the physical address the mock domain resolves @iova to, or 0 if
+ * it is unmapped.  Lets a test check that two IOVAs share one frame, or that
+ * an IOVA moved to another frame, without knowing the frames in advance.
+ */
+static int iommufd_test_md_iova_to_phys(struct iommufd_ucmd *ucmd,
+					unsigned int mockpt_id,
+					unsigned long iova)
+{
+	struct iommu_test_cmd *cmd = ucmd->cmd;
+	struct iommufd_hw_pagetable *hwpt;
+	struct mock_iommu_domain *mock;
+	unsigned int page_size;
+	int rc;
+
+	hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
+	if (IS_ERR(hwpt))
+		return PTR_ERR(hwpt);
+
+	page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
+	if (iova % page_size || !mock_domain_covers(mock, iova, page_size)) {
+		rc = -EINVAL;
+		goto out_put;
+	}
+	down_write(mock_domain_unmap_lock(hwpt));
+	cmd->iova_to_phys.out_phys =
+		mock->domain.ops->iova_to_phys(&mock->domain, iova);
+	up_write(mock_domain_unmap_lock(hwpt));
+	rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
+out_put:
+	iommufd_put_object(ucmd->ictx, &hwpt->obj);
+	return rc;
+}
+
+static int iommufd_test_md_check_mapped(struct iommufd_ucmd *ucmd,
+					unsigned int mockpt_id,
+					unsigned long iova, size_t length,
+					bool mapped)
+{
+	struct iommufd_hw_pagetable *hwpt;
+	struct mock_iommu_domain *mock;
+	unsigned int page_size;
+	int rc = 0;
+
+	hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
+	if (IS_ERR(hwpt))
+		return PTR_ERR(hwpt);
+
+	page_size = 1 << __ffs(mock->domain.pgsize_bitmap);
+	if (iova % page_size || length % page_size || !length ||
+	    !mock_domain_covers(mock, iova, length)) {
+		rc = -EINVAL;
+		goto out_put;
+	}
+
+	down_write(mock_domain_unmap_lock(hwpt));
+	for (; length; length -= page_size, iova += page_size) {
+		bool is_mapped =
+			mock->domain.ops->iova_to_phys(&mock->domain, iova) != 0;
+
+		if (is_mapped != mapped) {
+			rc = -ENOENT;
+			break;
+		}
+	}
+	up_write(mock_domain_unmap_lock(hwpt));
+
+out_put:
+	iommufd_put_object(ucmd->ictx, &hwpt->obj);
+	return rc;
+}
+
 static int iommufd_test_dmabuf_revoke(struct iommufd_ucmd *ucmd, int fd,
 				      bool revoked)
 {
@@ -2143,6 +2244,14 @@ int iommufd_test(struct iommufd_ucmd *ucmd)
 		return iommufd_test_dmabuf_revoke(ucmd,
 						  cmd->dmabuf_revoke.dmabuf_fd,
 						  cmd->dmabuf_revoke.revoked);
+	case IOMMU_TEST_OP_MD_CHECK_MAPPED:
+		return iommufd_test_md_check_mapped(ucmd, cmd->id,
+						    cmd->check_mapped.iova,
+						    cmd->check_mapped.length,
+						    cmd->check_mapped.mapped);
+	case IOMMU_TEST_OP_MD_IOVA_TO_PHYS:
+		return iommufd_test_md_iova_to_phys(ucmd, cmd->id,
+						    cmd->iova_to_phys.iova);
 	default:
 		return -EOPNOTSUPP;
 	}

[PATCH 7/9] iommufd/selftest: Add a mock memory provider

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

The provider path of IOMMU_IOAS_MAP_FILE needs a provider to test
against.

Add a mock provider whose pages can be made holes, read only, MMIO or
another shared frame, with a revoke after each change. Test that holes
stay unmapped, that a revoke changes only its range, that new domains
see the current state, and that provider pages and dirty tracking
domains cannot share an IOAS.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 drivers/iommu/iommufd/iommufd_test.h          |  20 ++
 drivers/iommu/iommufd/selftest.c              | 177 ++++++++++++++++++
 tools/testing/selftests/iommu/iommufd.c       | 119 ++++++++++++
 tools/testing/selftests/iommu/iommufd_utils.h |  64 +++++++
 4 files changed, 380 insertions(+)

diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h
index 28fd9c43edc4..5e5023dc8070 100644
--- a/drivers/iommu/iommufd/iommufd_test.h
+++ b/drivers/iommu/iommufd/iommufd_test.h
@@ -33,6 +33,17 @@ enum {
 	IOMMU_TEST_OP_DMABUF_REVOKE,
 	IOMMU_TEST_OP_MD_CHECK_MAPPED,
 	IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
+	IOMMU_TEST_OP_PROVIDER_GET,
+	IOMMU_TEST_OP_PROVIDER_SET,
+};
+
+/* Page states of the mock memory provider, IOMMU_TEST_OP_PROVIDER_SET */
+enum {
+	MOCK_PROVIDER_BACKED = 0,
+	MOCK_PROVIDER_HOLE,
+	MOCK_PROVIDER_READONLY,
+	MOCK_PROVIDER_OTHER,
+	MOCK_PROVIDER_MMIO,
 };
 
 enum {
@@ -209,6 +220,15 @@ struct iommu_test_cmd {
 			__aligned_u64 iova;
 			__aligned_u64 out_phys;	/* 0 if unmapped */
 		} iova_to_phys;
+		struct {
+			__u32 length;
+		} provider_get;
+		struct {
+			__s32 fd;
+			__u32 state;
+			__aligned_u64 offset;
+			__aligned_u64 length;
+		} provider_set;
 	};
 	__u32 last;
 };
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index 1f5cd2d00fda..270c41e6dd77 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -10,6 +10,7 @@
 #include <linux/fault-inject.h>
 #include <linux/file.h>
 #include <linux/iommu.h>
+#include <linux/mem_provider.h>
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 #include <linux/xarray.h>
@@ -2132,6 +2133,174 @@ static int iommufd_test_md_check_mapped(struct iommufd_ucmd *ucmd,
 	return rc;
 }
 
+/*
+ * A mock memory provider.  Each page of the file is backed by its own page of
+ * RAM, or is a hole, read only, the one page that every page in the OTHER
+ * state shares, or reported as MMIO.
+ * IOMMU_TEST_OP_PROVIDER_SET changes the state of a range and revokes it.
+ */
+struct iommufd_test_provider {
+	struct mem_provider_file mpf;
+	size_t npages;
+	struct page **pages;
+	struct page *other;
+	u8 *state;
+};
+
+static void iommufd_test_provider_free(struct iommufd_test_provider *tp)
+{
+	size_t i;
+
+	for (i = 0; i != tp->npages; i++)
+		if (tp->pages[i])
+			__free_page(tp->pages[i]);
+	if (tp->other)
+		__free_page(tp->other);
+	kfree(tp->state);
+	kfree(tp->pages);
+	kfree(tp);
+}
+
+static int iommufd_test_provider_release(struct inode *inode,
+					 struct file *file)
+{
+	iommufd_test_provider_free(file->private_data);
+	return 0;
+}
+
+static struct mem_provider_file *
+iommufd_test_provider_attach(struct file *file, loff_t size)
+{
+	struct iommufd_test_provider *tp = file->private_data;
+
+	if (size > (loff_t)tp->npages * PAGE_SIZE)
+		return ERR_PTR(-EINVAL);
+	return &tp->mpf;
+}
+
+static void iommufd_test_provider_detach(struct mem_provider_file *mpf)
+{
+}
+
+static int iommufd_test_provider_get_page(struct mem_provider_file *mpf,
+					  pgoff_t index, unsigned long *pfn,
+					  int *max_order, u32 *attrs)
+{
+	struct iommufd_test_provider *tp =
+		container_of(mpf, struct iommufd_test_provider, mpf);
+
+	*max_order = 0;
+	switch (READ_ONCE(tp->state[index])) {
+	case MOCK_PROVIDER_HOLE:
+		return -EFAULT;
+	case MOCK_PROVIDER_READONLY:
+		*attrs = MEM_PROVIDER_ATTR_READONLY;
+		break;
+	case MOCK_PROVIDER_OTHER:
+		*pfn = page_to_pfn(tp->other);
+		return 0;
+	case MOCK_PROVIDER_MMIO:
+		*attrs = MEM_PROVIDER_TYPE_MMIO;
+		break;
+	}
+	*pfn = page_to_pfn(tp->pages[index]);
+	return 0;
+}
+
+static const struct mem_provider_ops iommufd_test_provider_ops = {
+	.attach = iommufd_test_provider_attach,
+	.detach = iommufd_test_provider_detach,
+	.get_page = iommufd_test_provider_get_page,
+};
+
+static const struct mem_provider_fops iommufd_test_provider_fops = {
+	.fops = {
+		.owner = THIS_MODULE,
+		.fop_flags = FOP_MEM_PROVIDER,
+		.release = iommufd_test_provider_release,
+	},
+	.ops = &iommufd_test_provider_ops,
+};
+
+static int iommufd_test_provider_get(struct iommufd_ucmd *ucmd, size_t len)
+{
+	struct iommufd_test_provider *tp;
+	struct file *file;
+	size_t i;
+	int fd, rc;
+
+	len = ALIGN(len, PAGE_SIZE);
+	if (len == 0 || len > PAGE_SIZE * 512)
+		return -EINVAL;
+
+	tp = kzalloc_obj(*tp);
+	if (!tp)
+		return -ENOMEM;
+	mem_provider_file_init(&tp->mpf);
+	tp->npages = len / PAGE_SIZE;
+	tp->pages = kcalloc(tp->npages, sizeof(*tp->pages), GFP_KERNEL);
+	tp->state = kcalloc(tp->npages, sizeof(*tp->state), GFP_KERNEL);
+	tp->other = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	if (!tp->pages || !tp->state || !tp->other) {
+		rc = -ENOMEM;
+		goto err_free;
+	}
+	for (i = 0; i != tp->npages; i++) {
+		tp->pages[i] = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!tp->pages[i]) {
+			rc = -ENOMEM;
+			goto err_free;
+		}
+	}
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0) {
+		rc = fd;
+		goto err_free;
+	}
+	file = anon_inode_getfile("[iommufd-test-provider]",
+				  &iommufd_test_provider_fops.fops, tp, O_RDWR);
+	if (IS_ERR(file)) {
+		put_unused_fd(fd);
+		rc = PTR_ERR(file);
+		goto err_free;
+	}
+	fd_install(fd, file);
+	return fd;
+
+err_free:
+	iommufd_test_provider_free(tp);
+	return rc;
+}
+
+static int iommufd_test_provider_set(struct iommufd_ucmd *ucmd, int fd,
+				     unsigned int state, u64 offset, u64 length)
+{
+	struct iommufd_test_provider *tp;
+	u64 end;
+	size_t i;
+
+	if (state > MOCK_PROVIDER_MMIO)
+		return -EINVAL;
+
+	CLASS(fd, f)(fd);
+	if (fd_empty(f))
+		return -EBADF;
+	if (fd_file(f)->f_op != &iommufd_test_provider_fops.fops)
+		return -EINVAL;
+	tp = fd_file(f)->private_data;
+
+	if (!length || check_add_overflow(offset, length, &end) ||
+	    !PAGE_ALIGNED(offset) || !PAGE_ALIGNED(length) ||
+	    end > (u64)tp->npages * PAGE_SIZE)
+		return -EINVAL;
+
+	for (i = offset / PAGE_SIZE; i != end / PAGE_SIZE; i++)
+		WRITE_ONCE(tp->state[i], state);
+	mem_provider_revoke(&tp->mpf, offset, length);
+	return 0;
+}
+
 static int iommufd_test_dmabuf_revoke(struct iommufd_ucmd *ucmd, int fd,
 				      bool revoked)
 {
@@ -2252,6 +2421,14 @@ int iommufd_test(struct iommufd_ucmd *ucmd)
 	case IOMMU_TEST_OP_MD_IOVA_TO_PHYS:
 		return iommufd_test_md_iova_to_phys(ucmd, cmd->id,
 						    cmd->iova_to_phys.iova);
+	case IOMMU_TEST_OP_PROVIDER_GET:
+		return iommufd_test_provider_get(ucmd,
+						 cmd->provider_get.length);
+	case IOMMU_TEST_OP_PROVIDER_SET:
+		return iommufd_test_provider_set(ucmd, cmd->provider_set.fd,
+						 cmd->provider_set.state,
+						 cmd->provider_set.offset,
+						 cmd->provider_set.length);
 	default:
 		return -EOPNOTSUPP;
 	}
diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c
index d44b34b05757..ebf81ca3af18 100644
--- a/tools/testing/selftests/iommu/iommufd.c
+++ b/tools/testing/selftests/iommu/iommufd.c
@@ -1627,6 +1627,125 @@ TEST_F(iommufd_ioas, dmabuf_revoke)
 	close(dfd);
 }
 
+TEST_F(iommufd_ioas, provider_simple)
+{
+	size_t buf_size = PAGE_SIZE * 4;
+	__u64 iova;
+	int pfd;
+
+	test_cmd_get_provider(buf_size, &pfd);
+	test_err_ioctl_ioas_map_file(EINVAL, pfd, 0, 0, &iova);
+	test_err_ioctl_ioas_map_file(EINVAL, pfd, 0, buf_size + PAGE_SIZE,
+				     &iova);
+	test_err_ioctl_ioas_map_file(EINVAL, pfd, 1, PAGE_SIZE, &iova);
+	test_ioctl_ioas_map_file(pfd, 0, buf_size, &iova);
+	if (variant->mock_domains)
+		test_cmd_md_check_mapped(self->hwpt_id, iova, buf_size, true);
+
+	/* The mapping keeps the provider file alive */
+	close(pfd);
+	if (variant->mock_domains)
+		test_cmd_md_check_mapped(self->hwpt_id, iova, buf_size, true);
+	test_ioctl_ioas_unmap(iova, buf_size);
+}
+
+TEST_F(iommufd_ioas, provider_revoke)
+{
+	size_t buf_size = PAGE_SIZE * 8;
+	__u64 phys0, phys1;
+	__u32 hwpt_id;
+	__u64 iova;
+	int pfd;
+
+	if (!variant->mock_domains)
+		SKIP(return, "Needs a mock domain");
+
+	test_cmd_get_provider(buf_size, &pfd);
+
+	/* A page that is a hole when the file is mapped stays unmapped */
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_HOLE, PAGE_SIZE, PAGE_SIZE);
+	test_ioctl_ioas_map_file(pfd, 0, buf_size, &iova);
+	test_cmd_md_check_mapped(self->hwpt_id, iova, PAGE_SIZE, true);
+	test_cmd_md_check_mapped(self->hwpt_id, iova + PAGE_SIZE, PAGE_SIZE,
+				 false);
+	test_cmd_md_check_mapped(self->hwpt_id, iova + 2 * PAGE_SIZE,
+				 buf_size - 2 * PAGE_SIZE, true);
+
+	/* Give the page back */
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_BACKED, PAGE_SIZE, PAGE_SIZE);
+	test_cmd_md_check_mapped(self->hwpt_id, iova, buf_size, true);
+
+	/* Take back the middle, and only the middle goes */
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_HOLE, 2 * PAGE_SIZE,
+			      4 * PAGE_SIZE);
+	test_cmd_md_check_mapped(self->hwpt_id, iova, 2 * PAGE_SIZE, true);
+	test_cmd_md_check_mapped(self->hwpt_id, iova + 2 * PAGE_SIZE,
+				 4 * PAGE_SIZE, false);
+	test_cmd_md_check_mapped(self->hwpt_id, iova + 6 * PAGE_SIZE,
+				 2 * PAGE_SIZE, true);
+
+	/* A domain added now sees the same holes */
+	test_cmd_hwpt_alloc(self->device_id, self->ioas_id, 0, &hwpt_id);
+	test_cmd_md_check_mapped(hwpt_id, iova, 2 * PAGE_SIZE, true);
+	test_cmd_md_check_mapped(hwpt_id, iova + 2 * PAGE_SIZE, 4 * PAGE_SIZE,
+				 false);
+	test_cmd_md_check_mapped(hwpt_id, iova + 6 * PAGE_SIZE, 2 * PAGE_SIZE,
+				 true);
+	test_ioctl_destroy(hwpt_id);
+
+	/* Put the same other frame behind two pages */
+	phys0 = test_cmd_md_iova_to_phys(self->hwpt_id, iova);
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_OTHER, 0, 2 * PAGE_SIZE);
+	phys1 = test_cmd_md_iova_to_phys(self->hwpt_id, iova);
+	ASSERT_NE(0, phys1);
+	ASSERT_NE(phys0, phys1);
+	ASSERT_EQ(phys1,
+		  test_cmd_md_iova_to_phys(self->hwpt_id, iova + PAGE_SIZE));
+
+	/* Read-only and MMIO pages stay mapped */
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_READONLY, 0, buf_size / 2);
+	test_cmd_provider_set(pfd, MOCK_PROVIDER_MMIO, buf_size / 2,
+			      buf_size / 2);
+	test_cmd_md_check_mapped(self->hwpt_id, iova, buf_size, true);
+
+	/* A change past the end of the file is refused */
+	EXPECT_ERRNO(EINVAL, _test_cmd_provider_set(self->fd, pfd,
+						    MOCK_PROVIDER_HOLE,
+						    buf_size, PAGE_SIZE));
+
+	test_ioctl_ioas_unmap(iova, buf_size);
+	close(pfd);
+}
+
+TEST_F(iommufd_ioas, provider_dirty_tracking)
+{
+	size_t buf_size = PAGE_SIZE * 4;
+	__u32 hwpt_id;
+	__u64 iova;
+	int pfd;
+
+	if (!variant->mock_domains)
+		SKIP(return, "Needs a mock domain");
+
+	test_cmd_get_provider(buf_size, &pfd);
+
+	/* No dirty tracking domain over provider pages ... */
+	test_ioctl_ioas_map_file(pfd, 0, buf_size, &iova);
+	test_err_hwpt_alloc(EOPNOTSUPP, self->device_id, self->ioas_id,
+			    IOMMU_HWPT_ALLOC_DIRTY_TRACKING, &hwpt_id);
+	test_ioctl_ioas_unmap(iova, buf_size);
+
+	/* ... and no provider pages under a dirty tracking domain. */
+	test_cmd_hwpt_alloc(self->device_id, self->ioas_id,
+			    IOMMU_HWPT_ALLOC_DIRTY_TRACKING, &hwpt_id);
+	test_err_ioctl_ioas_map_file(EOPNOTSUPP, pfd, 0, buf_size, &iova);
+	test_ioctl_destroy(hwpt_id);
+
+	test_ioctl_ioas_map_file(pfd, 0, buf_size, &iova);
+	test_ioctl_ioas_unmap(iova, buf_size);
+	close(pfd);
+}
+
 FIXTURE(iommufd_mock_domain)
 {
 	int fd;
diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h
index b4928cbd4d9c..d1bd0a283db2 100644
--- a/tools/testing/selftests/iommu/iommufd_utils.h
+++ b/tools/testing/selftests/iommu/iommufd_utils.h
@@ -593,6 +593,70 @@ static int _test_cmd_revoke_dmabuf(int fd, int dmabuf_fd, bool revoked)
 #define test_cmd_revoke_dmabuf(dmabuf_fd, revoke) \
 	ASSERT_EQ(0, _test_cmd_revoke_dmabuf(self->fd, dmabuf_fd, revoke))
 
+static int _test_cmd_get_provider(int fd, size_t len, int *out_fd)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_PROVIDER_GET,
+		.provider_get = { .length = len },
+	};
+
+	*out_fd = ioctl(fd, IOMMU_TEST_CMD, &cmd);
+	if (*out_fd < 0)
+		return -1;
+	return 0;
+}
+#define test_cmd_get_provider(len, out_fd) \
+	ASSERT_EQ(0, _test_cmd_get_provider(self->fd, len, out_fd))
+
+static int _test_cmd_provider_set(int fd, int pfd, unsigned int state,
+				  __u64 offset, __u64 length)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_PROVIDER_SET,
+		.provider_set = { .fd = pfd, .state = state,
+				  .offset = offset, .length = length },
+	};
+
+	return ioctl(fd, IOMMU_TEST_CMD, &cmd);
+}
+#define test_cmd_provider_set(pfd, state, offset, length) \
+	ASSERT_EQ(0, _test_cmd_provider_set(self->fd, pfd, state, offset, length))
+
+static int _test_cmd_md_check_mapped(int fd, __u32 hwpt_id, __u64 iova,
+				     __u64 length, bool mapped)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_MD_CHECK_MAPPED,
+		.id = hwpt_id,
+		.check_mapped = { .mapped = mapped, .iova = iova,
+				  .length = length },
+	};
+
+	return ioctl(fd, IOMMU_TEST_CMD, &cmd);
+}
+#define test_cmd_md_check_mapped(hwpt_id, iova, length, mapped)		\
+	ASSERT_EQ(0, _test_cmd_md_check_mapped(self->fd, hwpt_id, iova,	\
+					       length, mapped))
+
+static __u64 _test_cmd_md_iova_to_phys(int fd, __u32 hwpt_id, __u64 iova)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
+		.id = hwpt_id,
+		.iova_to_phys = { .iova = iova },
+	};
+
+	if (ioctl(fd, IOMMU_TEST_CMD, &cmd))
+		return -1;
+	return cmd.iova_to_phys.out_phys;
+}
+#define test_cmd_md_iova_to_phys(hwpt_id, iova) \
+	_test_cmd_md_iova_to_phys(self->fd, hwpt_id, iova)
+
 static int _test_ioctl_destroy(int fd, unsigned int id)
 {
 	struct iommu_destroy cmd = {

[PATCH 8/9] samples/kvm: Add a memory provider sample

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

The memory provider interface has no in-tree provider that a VMM can
use.

Add mem_provider_sample. It lends a region, either a fixed range given
with addr= and len= or memory from alloc_contig_pages(), as child files
that a VMM passes to both guest_memfd and iommufd. A control device
moves pages between children, takes them back and makes them read
only, and every change revokes the range. It uses no KVM, dma-buf or
iommufd symbol.

The fixed range is not System RAM, so the sample keeps a write-back
memremap() of it while a file over it may be mapped into userspace.
Without it, PAT would make the range uncached on x86, and the VMM's
mapping would be uncached while the guest's is write-back. The sample
is x86 only, as guest_memfd accepts providers only there.

David's gmem_provider sample is unchanged.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 MAINTAINERS                       |   1 +
 samples/Kconfig                   |  17 +
 samples/Makefile                  |   1 +
 samples/kvm/Makefile              |   1 +
 samples/kvm/mem_provider_sample.c | 855 ++++++++++++++++++++++++++++++
 samples/kvm/mem_provider_sample.h | 135 +++++
 6 files changed, 1010 insertions(+)
 create mode 100644 samples/kvm/mem_provider_sample.c
 create mode 100644 samples/kvm/mem_provider_sample.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 6cab075a3ff7..b6b47e132e16 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17366,6 +17366,7 @@ L:	linux-mm@kvack.org
 S:	Maintained
 F:	include/linux/mem_provider.h
 F:	mm/mem_provider.c
+F:	samples/kvm/mem_provider_sample.*
 
 MEMORY TECHNOLOGY DEVICES (MTD)
 M:	Miquel Raynal <miquel.raynal@bootlin.com>
diff --git a/samples/Kconfig b/samples/Kconfig
index d26a03dea072..e4c4aface3ab 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -344,6 +344,23 @@ config SAMPLE_KVM_GMEM_PROVIDER
 
 	  If unsure, say N.
 
+config SAMPLE_KVM_MEM_PROVIDER
+	tristate "Build sample memory provider -- loadable module only"
+	depends on MEM_PROVIDER && CONTIG_ALLOC && X86_64 && m
+	help
+	  This builds a sample memory provider (include/linux/mem_provider.h).
+	  Its files can back a guest_memfd and be mapped by iommufd.  Loaded
+	  with addr= and len=, it lends a fixed physical range that has no
+	  struct page, for example memory hidden with memmap= on the command
+	  line.  Without them, it allocates a contiguous region with
+	  alloc_contig_pages().
+
+	  It shows how a memory owner moves pages between VMs, takes them
+	  back or makes them read only, and how guest_memfd and iommufd
+	  follow each change.
+
+	  If unsure, say N.
+
 endif # SAMPLES
 
 config HAVE_SAMPLE_FTRACE_DIRECT
diff --git a/samples/Makefile b/samples/Makefile
index e85397e5e34f..0565141d1f45 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -38,6 +38,7 @@ subdir-$(CONFIG_SAMPLE_WATCHDOG)	+= watchdog
 subdir-$(CONFIG_SAMPLE_WATCH_QUEUE)	+= watch_queue
 obj-$(CONFIG_SAMPLE_KMEMLEAK)		+= kmemleak/
 obj-$(CONFIG_SAMPLE_KVM_GMEM_PROVIDER)	+= kvm/
+obj-$(CONFIG_SAMPLE_KVM_MEM_PROVIDER)	+= kvm/
 obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG)	+= coresight/
 obj-$(CONFIG_SAMPLE_FPROBE)		+= fprobe/
 obj-$(CONFIG_SAMPLES_RUST)		+= rust/
diff --git a/samples/kvm/Makefile b/samples/kvm/Makefile
index dcad6e53ea78..a885b5313804 100644
--- a/samples/kvm/Makefile
+++ b/samples/kvm/Makefile
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_SAMPLE_KVM_GMEM_PROVIDER) += gmem_provider.o
+obj-$(CONFIG_SAMPLE_KVM_MEM_PROVIDER) += mem_provider_sample.o
diff --git a/samples/kvm/mem_provider_sample.c b/samples/kvm/mem_provider_sample.c
new file mode 100644
index 000000000000..8180959ab78f
--- /dev/null
+++ b/samples/kvm/mem_provider_sample.c
@@ -0,0 +1,855 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * mem_provider_sample - a sample memory provider.
+ *
+ * The module owns a region of physical memory and lends it through provider
+ * files (include/linux/mem_provider.h).  A VMM passes the same file to
+ * KVM_CREATE_GUEST_MEMFD, for the guest, and to IOMMU_IOAS_MAP_FILE, for the
+ * guest's devices.  The module calls neither KVM nor iommufd: it answers
+ * "what is page N" and revokes a range when the answer changes.
+ *
+ * The region is either:
+ *
+ *  - a fixed range given with addr= and len=, which has no struct page, for
+ *    example memory hidden from the kernel with memmap= on the command line;
+ *  - or, without those parameters, a range from alloc_contig_pages().
+ *
+ * The control device /dev/mem_provider_sample creates provider files.
+ * SETUP makes one file over the whole region.  NEW_CHILD carves part of the
+ * region into a child file, and MOVE, DONATE and RECLAIM change which child
+ * has which page.  A page can also be taken away or made read only.  See
+ * mem_provider_sample.h.
+ *
+ * Confidential VMs are not supported.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/bitmap.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/gfp.h>
+#include <linux/io.h>
+#include <linux/mem_provider.h>
+#include <linux/miscdevice.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+
+#include "mem_provider_sample.h"
+
+static unsigned long long addr;
+module_param(addr, ullong, 0444);
+MODULE_PARM_DESC(addr,
+		 "Physical base of a region with no struct page (optional)");
+
+static unsigned long long len;
+module_param(len, ullong, 0444);
+MODULE_PARM_DESC(len, "Size in bytes of that region (optional)");
+
+/*
+ * One provider file.
+ *
+ * Lock order: mps_root.lock, then mps_info.lock, then the consumers' locks
+ * (taken by their revoke callbacks).  get_page() takes no lock, so a
+ * consumer can call it from its fault paths and its revoke callback.
+ */
+struct mps_info {
+	struct mem_provider_file mpf;
+	struct mutex lock;		/* the bitmaps */
+
+	unsigned long base_pfn;
+	unsigned long npages;
+	struct page *cma_pages;		/* from alloc_contig_pages(), or NULL */
+	bool fixed;			/* a SETUP file over addr=/len= */
+	bool mmap_capable;
+	bool fixed_wb;			/* holds a ref on the fixed-region WB alias */
+
+	unsigned long *absent;		/* taken away by CTL_SET_PRESENT */
+	unsigned long *readonly;
+
+	/*
+	 * A child of the root: its first page in the root, and the pages it
+	 * has.  NULL @owned for a file made by SETUP, which has all its pages.
+	 */
+	unsigned long root_index;
+	unsigned long *owned;
+	struct list_head root_link;	/* mps_root.children, mps_root.lock */
+};
+
+/*
+ * The root region, owned by the control device.  Changes of ownership run
+ * under @lock, which is taken before any child's lock.
+ */
+static struct mps_root {
+	struct mutex lock;		/* everything below */
+	unsigned long base_pfn;
+	unsigned long npages;
+	struct page *cma_pages;
+	unsigned long *owned;		/* pages that a child has */
+	unsigned long *donated;		/* pages kept at the root */
+	struct list_head children;
+	/*
+	 * The addr=/len= region is used either by one SETUP file or by the
+	 * root, never by both, so that its frames have one owner.
+	 */
+	bool fixed_setup;
+	/* Files over the fixed region that may be mapped into userspace. */
+	unsigned int fixed_wb_users;
+} mps_root;
+
+/*
+ * A write-back mapping of the addr=/len= region, held while any file over it
+ * may be mapped into userspace (counted by mps_root.fixed_wb_users).  Without
+ * it, PAT makes the range uncached on x86, and the VMM's mapping would be
+ * uncached while the guest's is write-back.  PAT may still give a weaker type
+ * if the MTRRs do not mark the range write-back.
+ */
+static void *mps_fixed_wb;
+
+/* A file over the fixed region whose pages the host may map. */
+static bool mps_over_fixed_mappable(struct mps_info *info)
+{
+	return addr && len && !info->cma_pages && info->mmap_capable;
+}
+
+/*
+ * Hold a write-back mapping of the fixed region while any file over it may be
+ * mapped into userspace.  Called under mps_root.lock.
+ */
+static int mps_fixed_wb_get(void)
+{
+	if (mps_root.fixed_wb_users == 0) {
+		mps_fixed_wb = memremap(addr, len, MEMREMAP_WB);
+		if (!mps_fixed_wb)
+			return -ENOMEM;
+	}
+	mps_root.fixed_wb_users++;
+	return 0;
+}
+
+static void mps_fixed_wb_put(void)
+{
+	if (--mps_root.fixed_wb_users == 0) {
+		memunmap(mps_fixed_wb);
+		mps_fixed_wb = NULL;
+	}
+}
+
+/* Does @info have page @index now?  Called with or without info->lock. */
+static bool mps_page_present(struct mps_info *info, unsigned long index)
+{
+	if (info->owned && !test_bit(index, info->owned))
+		return false;
+	return !test_bit(index, info->absent);
+}
+
+/*
+ * The length of the run from @index of pages with the same state as @index:
+ * present, and the same read-only bit.
+ */
+static unsigned long mps_run(struct mps_info *info, unsigned long index)
+{
+	unsigned long end = info->npages;
+
+	end = min(end, find_next_bit(info->absent, info->npages, index + 1));
+	if (info->owned)
+		end = min(end, find_next_zero_bit(info->owned, info->npages,
+						  index + 1));
+	if (test_bit(index, info->readonly))
+		end = min(end, find_next_zero_bit(info->readonly, info->npages,
+						  index + 1));
+	else
+		end = min(end, find_next_bit(info->readonly, info->npages,
+					     index + 1));
+	return end - index;
+}
+
+/* ---- Provider operations ------------------------------------------------ */
+
+static struct mem_provider_file *
+mps_mp_attach(struct file *file, loff_t size)
+{
+	struct mps_info *info = file->private_data;
+
+	if (size > (loff_t)info->npages << PAGE_SHIFT)
+		return ERR_PTR(-EINVAL);
+	return &info->mpf;
+}
+
+static void mps_mp_detach(struct mem_provider_file *mpf)
+{
+}
+
+/*
+ * Reads the bitmaps without info->lock.  A change runs under the lock and
+ * revokes the range afterwards, so an answer that a change makes out of date
+ * is dropped by the consumer.
+ */
+static int mps_mp_get_page(struct mem_provider_file *mpf, pgoff_t index,
+			   unsigned long *pfn, int *max_order, u32 *attrs)
+{
+	struct mps_info *info = container_of(mpf, struct mps_info, mpf);
+	unsigned long run;
+
+	if (index >= info->npages)
+		return -EINVAL;
+
+	if (!mps_page_present(info, index))
+		return -EFAULT;
+
+	/* The aligned block around @index must be one run. */
+	run = mps_run(info, index);
+	while (*max_order) {
+		unsigned long first = ALIGN_DOWN(index, 1UL << *max_order);
+
+		if (first + (1UL << *max_order) <= index + run &&
+		    (first == index || mps_page_present(info, first)) &&
+		    mps_run(info, first) >= 1UL << *max_order)
+			break;
+		(*max_order)--;
+	}
+
+	*pfn = info->base_pfn + index;
+	if (test_bit(index, info->readonly))
+		*attrs |= MEM_PROVIDER_ATTR_READONLY;
+	/* The creator of the file decides whether the host may map it. */
+	if (!info->mmap_capable)
+		*attrs |= MEM_PROVIDER_ATTR_NO_USER_MAP;
+	return 0;
+}
+
+static const struct mem_provider_ops mps_mp_ops = {
+	.attach		= mps_mp_attach,
+	.detach		= mps_mp_detach,
+	.get_page	= mps_mp_get_page,
+};
+
+/* Tell the consumers that pages [start, end) of @info changed. */
+static void mps_revoke(struct mps_info *info, unsigned long start,
+		       unsigned long end)
+{
+	if (end > start)
+		mem_provider_revoke(&info->mpf, (loff_t)start << PAGE_SHIFT,
+				    (loff_t)(end - start) << PAGE_SHIFT);
+}
+
+/* ---- Provider file ------------------------------------------------------ */
+
+/* Byte range [offset, offset + len) of @info as page indices. */
+static int mps_range(struct mps_info *info, u64 offset, u64 len,
+		     unsigned long *start, unsigned long *end)
+{
+	if (!len || !PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len))
+		return -EINVAL;
+	*start = offset >> PAGE_SHIFT;
+	*end = *start + (len >> PAGE_SHIFT);
+	if (*end > info->npages || *end < *start)
+		return -EINVAL;
+	return 0;
+}
+
+static void mps_set_readonly(struct mps_info *info, unsigned long start,
+			     unsigned long end, bool readonly)
+{
+	mutex_lock(&info->lock);
+	if (readonly)
+		bitmap_set(info->readonly, start, end - start);
+	else
+		bitmap_clear(info->readonly, start, end - start);
+	mps_revoke(info, start, end);
+	mutex_unlock(&info->lock);
+}
+
+static void mps_set_present(struct mps_info *info, unsigned long start,
+			    unsigned long end, bool present)
+{
+	mutex_lock(&info->lock);
+	if (present)
+		bitmap_clear(info->absent, start, end - start);
+	else
+		bitmap_set(info->absent, start, end - start);
+	mps_revoke(info, start, end);
+	mutex_unlock(&info->lock);
+}
+
+static long mps_get_stats(struct mps_info *info, void __user *uarg)
+{
+	struct mps_stats st = {};
+
+	mutex_lock(&info->lock);
+	st.region_offset = (u64)info->root_index << PAGE_SHIFT;
+	st.region_len = (u64)info->npages << PAGE_SHIFT;
+	st.owned_pages = info->owned ?
+		bitmap_weight(info->owned, info->npages) : info->npages;
+	st.absent_pages = bitmap_weight(info->absent, info->npages);
+	st.readonly_pages = bitmap_weight(info->readonly, info->npages);
+	mutex_unlock(&info->lock);
+
+	return copy_to_user(uarg, &st, sizeof(st)) ? -EFAULT : 0;
+}
+
+/*
+ * A child fd is read-only to its holder: only GET_STATS.  The owner changes a
+ * child's pages through the control device (MPS_CTL_SET_PRESENT and
+ * MPS_CTL_SET_READONLY), so a VMM that holds a child cannot.
+ */
+static long mps_file_ioctl(struct file *file, unsigned int cmd,
+			   unsigned long arg)
+{
+	struct mps_info *info = file->private_data;
+	void __user *uarg = (void __user *)arg;
+
+	if (cmd == MPS_GET_STATS)
+		return mps_get_stats(info, uarg);
+	return -ENOTTY;
+}
+
+/* Every consumer has detached: each one holds a reference to the file. */
+static int mps_file_release(struct inode *inode, struct file *file)
+{
+	struct mps_info *info = file->private_data;
+	unsigned long i;
+
+	/*
+	 * A child gives the pages it has back to the root.  Pages it donated
+	 * stay at the root, and pages moved out belong to another child.
+	 */
+	if (info->owned) {
+		mutex_lock(&mps_root.lock);
+		list_del(&info->root_link);
+		for_each_set_bit(i, info->owned, info->npages)
+			__clear_bit(info->root_index + i, mps_root.owned);
+		mutex_unlock(&mps_root.lock);
+		bitmap_free(info->owned);
+	}
+	if (addr && len && !info->cma_pages) {
+		mutex_lock(&mps_root.lock);
+		if (info->fixed)
+			mps_root.fixed_setup = false;
+		if (info->fixed_wb)
+			mps_fixed_wb_put();
+		mutex_unlock(&mps_root.lock);
+	}
+
+	if (info->cma_pages)
+		free_contig_range(info->base_pfn, info->npages);
+	bitmap_free(info->absent);
+	bitmap_free(info->readonly);
+	kfree(info);
+	return 0;
+}
+
+static const struct mem_provider_fops mps_file_fops = {
+	.fops = {
+		.owner		= THIS_MODULE,
+		.fop_flags	= FOP_MEM_PROVIDER,
+		.release	= mps_file_release,
+		.unlocked_ioctl	= mps_file_ioctl,
+		.compat_ioctl	= compat_ptr_ioctl,
+	},
+	.ops = &mps_mp_ops,
+};
+
+/*
+ * Make a provider file over [base_pfn, base_pfn + npages).  The fd is
+ * reserved but not installed, so that the caller can finish setting up
+ * @info before another thread can reach it.
+ */
+static int mps_new_file(unsigned long base_pfn, unsigned long npages,
+			u32 flags, struct mps_info **infop,
+			 struct file **filep)
+{
+	struct mps_info *info;
+	struct file *file;
+	int fd, ret;
+
+	info = kzalloc_obj(*info);
+	if (!info)
+		return -ENOMEM;
+	mem_provider_file_init(&info->mpf);
+	info->base_pfn = base_pfn;
+	info->npages = npages;
+	info->mmap_capable = flags & MPS_FLAG_MMAP_CAPABLE;
+	info->absent = bitmap_zalloc(npages, GFP_KERNEL);
+	info->readonly = bitmap_zalloc(npages, GFP_KERNEL);
+	if (!info->absent || !info->readonly) {
+		ret = -ENOMEM;
+		goto err_free;
+	}
+	mutex_init(&info->lock);
+	INIT_LIST_HEAD(&info->root_link);
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0) {
+		ret = fd;
+		goto err_free;
+	}
+	file = anon_inode_getfile("[mem-provider-sample]", &mps_file_fops.fops,
+				  info, O_RDWR);
+	if (IS_ERR(file)) {
+		put_unused_fd(fd);
+		ret = PTR_ERR(file);
+		goto err_free;
+	}
+	*infop = info;
+	*filep = file;
+	return fd;
+
+err_free:
+	bitmap_free(info->absent);
+	bitmap_free(info->readonly);
+	kfree(info);
+	return ret;
+}
+
+static struct page *mps_alloc_region(unsigned long npages)
+{
+	struct page *pages;
+
+	pages = alloc_contig_pages(npages, GFP_KERNEL | __GFP_ZERO,
+				   numa_node_id(), NULL);
+	return pages;
+}
+
+static long mps_ctl_setup(void __user *uarg)
+{
+	struct mps_setup setup;
+	unsigned long base_pfn, npages;
+	struct page *pages = NULL;
+	struct mps_info *info;
+	struct file *file;
+	int fd;
+
+	if (copy_from_user(&setup, uarg, sizeof(setup)))
+		return -EFAULT;
+	if ((setup.flags & ~MPS_FLAG_MMAP_CAPABLE) || setup.pad)
+		return -EINVAL;
+
+	if (addr && len) {
+		/* The fixed region has one owner: this file or the root. */
+		mutex_lock(&mps_root.lock);
+		if (mps_root.fixed_setup || mps_root.npages) {
+			mutex_unlock(&mps_root.lock);
+			return -EBUSY;
+		}
+		fd = mps_new_file(addr >> PAGE_SHIFT, len >> PAGE_SHIFT,
+				  setup.flags, &info, &file);
+		if (fd >= 0 && mps_over_fixed_mappable(info)) {
+			if (mps_fixed_wb_get()) {
+				put_unused_fd(fd);
+				fput(file);
+				fd = -ENOMEM;
+			} else {
+				info->fixed_wb = true;
+			}
+		}
+		if (fd >= 0) {
+			info->fixed = true;
+			mps_root.fixed_setup = true;
+		}
+		mutex_unlock(&mps_root.lock);
+		if (fd >= 0)
+			fd_install(fd, file);
+		return fd;
+	}
+
+	if (!setup.size || !PAGE_ALIGNED(setup.size))
+		return -EINVAL;
+	npages = setup.size >> PAGE_SHIFT;
+	pages = mps_alloc_region(npages);
+	if (!pages)
+		return -ENOMEM;
+	base_pfn = page_to_pfn(pages);
+
+	fd = mps_new_file(base_pfn, npages, setup.flags, &info, &file);
+	if (fd < 0) {
+		free_contig_range(base_pfn, npages);
+		return fd;
+	}
+	info->cma_pages = pages;
+	fd_install(fd, file);
+	return fd;
+}
+
+/* ---- Root and children -------------------------------------------------- */
+
+/* Create the root on the first NEW_CHILD, from addr=/len= or of @size. */
+static int mps_root_ensure(u64 size)
+{
+	struct page *pages = NULL;
+	unsigned long npages;
+
+	lockdep_assert_held(&mps_root.lock);
+	if (mps_root.npages)
+		return 0;
+
+	if (addr && len) {
+		if (mps_root.fixed_setup)
+			return -EBUSY;
+		mps_root.base_pfn = addr >> PAGE_SHIFT;
+		npages = len >> PAGE_SHIFT;
+	} else {
+		if (!size || !PAGE_ALIGNED(size))
+			return -EINVAL;
+		npages = size >> PAGE_SHIFT;
+		pages = mps_alloc_region(npages);
+		if (!pages)
+			return -ENOMEM;
+		mps_root.base_pfn = page_to_pfn(pages);
+	}
+	mps_root.owned = bitmap_zalloc(npages, GFP_KERNEL);
+	mps_root.donated = bitmap_zalloc(npages, GFP_KERNEL);
+	if (!mps_root.owned || !mps_root.donated) {
+		bitmap_free(mps_root.owned);
+		bitmap_free(mps_root.donated);
+		mps_root.owned = NULL;
+		mps_root.donated = NULL;
+		if (pages)
+			free_contig_range(page_to_pfn(pages), npages);
+		return -ENOMEM;
+	}
+	mps_root.cma_pages = pages;
+	mps_root.npages = npages;
+	return 0;
+}
+
+static void mps_root_teardown(void)
+{
+	if (!mps_root.npages)
+		return;
+	WARN_ON(!list_empty(&mps_root.children));
+	if (mps_root.cma_pages)
+		free_contig_range(mps_root.base_pfn, mps_root.npages);
+	bitmap_free(mps_root.owned);
+	bitmap_free(mps_root.donated);
+}
+
+/* A page-aligned byte range of the root as page indices [first, last). */
+static int mps_root_range(u64 offset, u64 length, unsigned long *first,
+			  unsigned long *last)
+{
+	if (!length || !PAGE_ALIGNED(offset) || !PAGE_ALIGNED(length))
+		return -EINVAL;
+	*first = offset >> PAGE_SHIFT;
+	*last = *first + (length >> PAGE_SHIFT);
+	if (*last <= *first || *last > mps_root.npages)
+		return -EINVAL;
+	return 0;
+}
+
+/* Return a referenced child file made by NEW_CHILD. */
+static struct file *mps_get_child(int fd, struct mps_info **infop)
+{
+	struct file *file = fget(fd);
+
+	if (!file)
+		return ERR_PTR(-EBADF);
+	if (file->f_op != &mps_file_fops.fops ||
+	    !((struct mps_info *)file->private_data)->owned) {
+		fput(file);
+		return ERR_PTR(-EINVAL);
+	}
+	*infop = file->private_data;
+	return file;
+}
+
+static long mps_ctl_new_child(void __user *uarg)
+{
+	struct mps_new_child nc;
+	unsigned long first, last, i, grant = 0;
+	struct mps_info *info;
+	unsigned long *owned;
+	struct file *file;
+	int fd, ret;
+
+	if (copy_from_user(&nc, uarg, sizeof(nc)))
+		return -EFAULT;
+	if ((nc.flags & ~MPS_FLAG_MMAP_CAPABLE) || nc.pad)
+		return -EINVAL;
+
+	mutex_lock(&mps_root.lock);
+	ret = mps_root_ensure(nc.offset + nc.len);
+	if (ret)
+		goto out_unlock;
+	ret = mps_root_range(nc.offset, nc.len, &first, &last);
+	if (ret)
+		goto out_unlock;
+
+	/* A child that would get no page is likely a mistake. */
+	for (i = first; i < last; i++)
+		if (!test_bit(i, mps_root.owned) &&
+		    !test_bit(i, mps_root.donated))
+			grant++;
+	if (!grant) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	owned = bitmap_zalloc(last - first, GFP_KERNEL);
+	if (!owned) {
+		ret = -ENOMEM;
+		goto out_unlock;
+	}
+	fd = mps_new_file(mps_root.base_pfn + first, last - first, nc.flags,
+			  &info, &file);
+	if (fd < 0) {
+		bitmap_free(owned);
+		ret = fd;
+		goto out_unlock;
+	}
+	if (mps_over_fixed_mappable(info)) {
+		if (mps_fixed_wb_get()) {
+			put_unused_fd(fd);
+			fput(file);
+			bitmap_free(owned);
+			ret = -ENOMEM;
+			goto out_unlock;
+		}
+		info->fixed_wb = true;
+	}
+
+	/* No other thread can reach the child until fd_install(). */
+	for (i = first; i < last; i++) {
+		if (test_bit(i, mps_root.owned) ||
+		    test_bit(i, mps_root.donated))
+			continue;
+		__set_bit(i - first, owned);
+		__set_bit(i, mps_root.owned);
+	}
+	info->owned = owned;
+	info->root_index = first;
+	list_add(&info->root_link, &mps_root.children);
+	mutex_unlock(&mps_root.lock);
+	fd_install(fd, file);
+	return fd;
+
+out_unlock:
+	mutex_unlock(&mps_root.lock);
+	return ret;
+}
+
+/* Take [first, last) of the root from @info.  Called under mps_root.lock. */
+static void mps_child_lose(struct mps_info *info, unsigned long first,
+			   unsigned long last)
+{
+	unsigned long s = first - info->root_index;
+	unsigned long e = last - info->root_index;
+
+	mutex_lock(&info->lock);
+	bitmap_clear(info->owned, s, e - s);
+	mps_revoke(info, s, e);
+	mutex_unlock(&info->lock);
+	bitmap_clear(mps_root.owned, first, last - first);
+}
+
+/* Give [first, last) of the root to @info.  Called under mps_root.lock. */
+static void mps_child_gain(struct mps_info *info, unsigned long first,
+			   unsigned long last)
+{
+	unsigned long s = first - info->root_index;
+	unsigned long e = last - info->root_index;
+
+	mutex_lock(&info->lock);
+	bitmap_set(info->owned, s, e - s);
+	mps_revoke(info, s, e);
+	mutex_unlock(&info->lock);
+	bitmap_set(mps_root.owned, first, last - first);
+}
+
+static bool mps_child_covers(struct mps_info *info, unsigned long first,
+			     unsigned long last)
+{
+	return first >= info->root_index &&
+	       last <= info->root_index + info->npages;
+}
+
+static bool mps_child_owns(struct mps_info *info, unsigned long first,
+			   unsigned long last)
+{
+	unsigned long s = first - info->root_index;
+	unsigned long e = last - info->root_index;
+
+	return mps_child_covers(info, first, last) &&
+	       find_next_zero_bit(info->owned, e, s) >= e;
+}
+
+static long mps_ctl_move(void __user *uarg)
+{
+	struct mps_move mv;
+	struct mps_info *src, *dst;
+	unsigned long first, last;
+	struct file *sf, *df;
+	long ret;
+
+	if (copy_from_user(&mv, uarg, sizeof(mv)))
+		return -EFAULT;
+	sf = mps_get_child(mv.src_fd, &src);
+	if (IS_ERR(sf))
+		return PTR_ERR(sf);
+	df = mps_get_child(mv.dst_fd, &dst);
+	if (IS_ERR(df)) {
+		fput(sf);
+		return PTR_ERR(df);
+	}
+
+	mutex_lock(&mps_root.lock);
+	ret = mps_root_range(mv.offset, mv.len, &first, &last);
+	if (ret)
+		goto out;
+	if (src == dst || !mps_child_owns(src, first, last) ||
+	    !mps_child_covers(dst, first, last)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	/* Revoke from the source before the destination gets the pages. */
+	mps_child_lose(src, first, last);
+	mps_child_gain(dst, first, last);
+out:
+	mutex_unlock(&mps_root.lock);
+	fput(df);
+	fput(sf);
+	return ret;
+}
+
+static long mps_ctl_donate(void __user *uarg, bool reclaim)
+{
+	struct mps_donate d;
+	unsigned long first, last;
+	struct mps_info *info;
+	struct file *file;
+	long ret;
+
+	if (copy_from_user(&d, uarg, sizeof(d)))
+		return -EFAULT;
+	if (d.pad)
+		return -EINVAL;
+	file = mps_get_child(d.fd, &info);
+	if (IS_ERR(file))
+		return PTR_ERR(file);
+
+	mutex_lock(&mps_root.lock);
+	ret = mps_root_range(d.offset, d.len, &first, &last);
+	if (ret)
+		goto out;
+	if (!reclaim) {
+		if (!mps_child_owns(info, first, last)) {
+			ret = -EINVAL;
+			goto out;
+		}
+		mps_child_lose(info, first, last);
+		bitmap_set(mps_root.donated, first, last - first);
+	} else {
+		if (!mps_child_covers(info, first, last) ||
+		    find_next_zero_bit(mps_root.donated, last, first) < last) {
+			ret = -EINVAL;
+			goto out;
+		}
+		bitmap_clear(mps_root.donated, first, last - first);
+		mps_child_gain(info, first, last);
+	}
+out:
+	mutex_unlock(&mps_root.lock);
+	fput(file);
+	return ret;
+}
+
+/*
+ * CTL_SET_READONLY and CTL_SET_PRESENT change a child's pages, by the owner on
+ * the control device.  A VMM that holds the child fd cannot, so it cannot undo
+ * them.
+ */
+static long mps_ctl_set_range(void __user *uarg, bool readonly)
+{
+	struct mps_ctl_range cr;
+	unsigned long start, end;
+	struct mps_info *info;
+	struct file *file;
+	int ret;
+
+	if (copy_from_user(&cr, uarg, sizeof(cr)))
+		return -EFAULT;
+	if (cr.value > 1)
+		return -EINVAL;
+	file = mps_get_child(cr.fd, &info);
+	if (IS_ERR(file))
+		return PTR_ERR(file);
+	ret = mps_range(info, cr.offset, cr.len, &start, &end);
+	if (!ret) {
+		if (readonly)
+			mps_set_readonly(info, start, end, cr.value);
+		else
+			mps_set_present(info, start, end, cr.value);
+	}
+	fput(file);
+	return ret;
+}
+
+static long mps_ctl_ioctl(struct file *file, unsigned int cmd,
+			  unsigned long arg)
+{
+	void __user *uarg = (void __user *)arg;
+
+	switch (cmd) {
+	case MPS_SETUP:
+		return mps_ctl_setup(uarg);
+	case MPS_NEW_CHILD:
+		return mps_ctl_new_child(uarg);
+	case MPS_MOVE:
+		return mps_ctl_move(uarg);
+	case MPS_DONATE:
+		return mps_ctl_donate(uarg, false);
+	case MPS_RECLAIM:
+		return mps_ctl_donate(uarg, true);
+	case MPS_CTL_SET_READONLY:
+		return mps_ctl_set_range(uarg, true);
+	case MPS_CTL_SET_PRESENT:
+		return mps_ctl_set_range(uarg, false);
+	default:
+		return -ENOTTY;
+	}
+}
+
+static const struct file_operations mps_ctl_fops = {
+	.owner		= THIS_MODULE,
+	.unlocked_ioctl	= mps_ctl_ioctl,
+	.compat_ioctl	= compat_ptr_ioctl,
+};
+
+static struct miscdevice mps_dev = {
+	.minor	= MISC_DYNAMIC_MINOR,
+	.name	= "mem_provider_sample",
+	.fops	= &mps_ctl_fops,
+};
+
+static int __init mps_init(void)
+{
+	int ret;
+
+	if ((addr || len) &&
+	    (!addr || !len || !PAGE_ALIGNED(addr) || !PAGE_ALIGNED(len))) {
+		pr_err("mem_provider_sample: addr= and len= must both be set and page aligned\n");
+		return -EINVAL;
+	}
+
+	mutex_init(&mps_root.lock);
+	INIT_LIST_HEAD(&mps_root.children);
+
+	ret = misc_register(&mps_dev);
+	if (ret)
+		return ret;
+	return 0;
+}
+module_init(mps_init);
+
+static void __exit mps_exit(void)
+{
+	misc_deregister(&mps_dev);
+	mps_root_teardown();
+	if (mps_fixed_wb)
+		memunmap(mps_fixed_wb);
+}
+module_exit(mps_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Sample memory provider for guest_memfd and iommufd");
diff --git a/samples/kvm/mem_provider_sample.h b/samples/kvm/mem_provider_sample.h
new file mode 100644
index 000000000000..3be453756d84
--- /dev/null
+++ b/samples/kvm/mem_provider_sample.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _SAMPLES_KVM_MEM_PROVIDER_SAMPLE_H
+#define _SAMPLES_KVM_MEM_PROVIDER_SAMPLE_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+/*
+ * A sample memory provider (include/linux/mem_provider.h).
+ *
+ * Every fd that this module returns is a provider file.  Pass it as
+ * provider_fd to KVM_CREATE_GUEST_MEMFD with GUEST_MEMFD_FLAG_USE_PROVIDER,
+ * and as fd to IOMMU_IOAS_MAP_FILE.  Both consumers follow every change that
+ * the ioctls below make.
+ */
+
+/* Flags for struct mps_setup and struct mps_new_child */
+/* The host may map the pages; otherwise they are NO_USER_MAP */
+#define MPS_FLAG_MMAP_CAPABLE	(1u << 0)
+
+/*
+ * ioctl on /dev/mem_provider_sample: create a provider fd.
+ *
+ * If the module was loaded with addr= and len=, the fd covers that fixed
+ * range, which has no struct page, and @size is ignored.  The fixed range
+ * then has one owner.  SETUP fails with -EBUSY while a SETUP file uses it,
+ * or once the first MPS_NEW_CHILD has made it the root of the children, which
+ * lasts until the module is unloaded.  MPS_NEW_CHILD fails with -EBUSY while
+ * a SETUP file uses it.  Otherwise the fd covers @size bytes from
+ * alloc_contig_pages().
+ */
+struct mps_setup {
+	__u32 flags;	/* MPS_FLAG_* */
+	__u32 pad;
+	__u64 size;	/* bytes, page aligned */
+};
+
+#define MPS_IOCTL_BASE		'P'
+#define MPS_SETUP \
+	_IOW(MPS_IOCTL_BASE, 1, struct mps_setup)
+
+/*
+ * ---- A tree of owners --------------------------------------------------
+ *
+ * The control device owns one region, the root.  NEW_CHILD carves part of
+ * it into a new provider fd for one VM.  The owner can MOVE pages between
+ * children, DONATE pages to the root, so that no child has them, and
+ * RECLAIM them.  Each change revokes the range, so KVM and iommufd follow.
+ *
+ * The owner changes a child's pages through the control device
+ * (CTL_SET_READONLY, CTL_SET_PRESENT).  A child fd is read-only to its
+ * holder: a VMM can only GET_STATS, never change the memory.
+ */
+
+/*
+ * ioctl on /dev/mem_provider_sample: carve [@offset, @offset + @len) of the
+ * root into a new child fd.  The child gets every page of the range that no
+ * other child has and that is not donated.  Ranges of children may overlap,
+ * which is how a page can MOVE between them.  Returns the child fd.
+ */
+struct mps_new_child {
+	__u32 flags;	/* MPS_FLAG_* */
+	__u32 pad;
+	__u64 offset;	/* bytes into the root, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MPS_NEW_CHILD \
+	_IOW(MPS_IOCTL_BASE, 5, struct mps_new_child)
+
+/*
+ * ioctl on /dev/mem_provider_sample: move [@offset, @offset + @len) of the root
+ * from child @src_fd to child @dst_fd.  @src_fd must have every page of the
+ * range, and the range must be inside @dst_fd's.  The pages are revoked from
+ * the source before the destination gets them, so the consumers of the two
+ * children never map them at the same time.
+ */
+struct mps_move {
+	__s32 src_fd;
+	__s32 dst_fd;
+	__u64 offset;	/* bytes into the root, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MPS_MOVE \
+	_IOW(MPS_IOCTL_BASE, 6, struct mps_move)
+
+/*
+ * ioctls on /dev/mem_provider_sample: DONATE takes [@offset, @offset + @len) of
+ * the root from child @fd, which must have all of it, and keeps it at the
+ * root.  RECLAIM gives a donated range to child @fd, whose range must
+ * contain it.  The sample does not scrub the pages.
+ */
+struct mps_donate {
+	__s32 fd;
+	__u32 pad;
+	__u64 offset;	/* bytes into the root, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MPS_DONATE \
+	_IOW(MPS_IOCTL_BASE, 7, struct mps_donate)
+#define MPS_RECLAIM \
+	_IOW(MPS_IOCTL_BASE, 8, struct mps_donate)
+
+/* ioctl on a child fd: read the child's state, for tests. */
+struct mps_stats {
+	__u64 region_offset;	/* the child's range in the root */
+	__u64 region_len;
+	__u64 owned_pages;	/* pages the child has */
+	__u64 absent_pages;	/* pages taken away by CTL_SET_PRESENT */
+	__u64 readonly_pages;
+};
+
+#define MPS_GET_STATS \
+	_IOR(MPS_IOCTL_BASE, 10, struct mps_stats)
+
+/*
+ * ioctls on /dev/mem_provider_sample: CTL_SET_READONLY or CTL_SET_PRESENT on
+ * child @fd, by the owner.  @value is the readonly or present value, and
+ * @offset and @len are a byte range within the child, page aligned.
+ */
+struct mps_ctl_range {
+	__s32 fd;
+	__u32 value;
+	__u64 offset;	/* bytes into the child, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MPS_CTL_SET_READONLY \
+	_IOW(MPS_IOCTL_BASE, 11, struct mps_ctl_range)
+#define MPS_CTL_SET_PRESENT \
+	_IOW(MPS_IOCTL_BASE, 12, struct mps_ctl_range)
+
+#endif /* _SAMPLES_KVM_MEM_PROVIDER_SAMPLE_H */

[PATCH 9/9] KVM: selftests: Test a memory provider shared by KVM and iommufd

[Fred Griffoul]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Test that guest, host and device mappings of one provider file follow
the provider's changes.

Two VMs get child files of the sample provider, each passed to
guest_memfd and to an iommufd mock domain. The test moves a range
between them, donates and reclaims pages, and makes a page read only.
After each change it checks the guest, a host mapping and the device
mapping. Pages are mapped on the host before a change, so the test
checks that the revoke zaps them.

Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/mem_provider_test.c     | 582 ++++++++++++++++++
 2 files changed, 583 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/mem_provider_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 4d7082448cea..ccee373b4c13 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -84,6 +84,7 @@ TEST_GEN_PROGS_x86 += x86/gmem_provider_revoke_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_readonly_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_iommufd_test
 TEST_GEN_PROGS_x86 += x86/gmem_provider_vfio_test
+TEST_GEN_PROGS_x86 += x86/mem_provider_test
 TEST_GEN_PROGS_x86 += x86/hwcr_msr_test
 TEST_GEN_PROGS_x86 += x86/hyperv_clock
 TEST_GEN_PROGS_x86 += x86/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/x86/mem_provider_test.c b/tools/testing/selftests/kvm/x86/mem_provider_test.c
new file mode 100644
index 000000000000..bd7ae44f05d7
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/mem_provider_test.c
@@ -0,0 +1,582 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * mem_provider_test - one memory provider shared by KVM and iommufd.
+ *
+ * A control process and two VMMs, in one process:
+ *
+ *  ctl:  holds /dev/mem_provider_sample, the owner of the memory.  Creates a
+ *        child provider file per VM, moves pages between children, donates
+ *        and reclaims them, and makes them read only.
+ *        Never touches a VM.
+ *  vmm:  holds one child file, one VM and one iommufd IOAS on a mock domain.
+ *        Passes the child file to KVM_CREATE_GUEST_MEMFD and to
+ *        IOMMU_IOAS_MAP_FILE.  Never touches the control device.
+ *
+ * Scenarios:
+ *
+ *  1. Launch:    each guest writes to its memory, the host sees the write,
+ *                and the device mapping covers the pages the child has.
+ *  2. Move:      a range leaves A for B.  A's guest and A's host mapping
+ *                fault on it and A's device loses only that range.  B's
+ *                guest and host read what A wrote, and B's device reaches
+ *                the frame A had.
+ *  3. Donate:    a range leaves A for the root and comes back on reclaim,
+ *                for the guest, the host mapping and the device.
+ *  4. Read only: a guest write to a read-only page exits to the VMM and a
+ *                host write raises SIGBUS; both land once the page is
+ *                writable again.
+ *  5. Window:    a host mapping of the guest_memfd faults after the range it
+ *                covers is donated.
+ *
+ * Requires samples/kvm/mem_provider_sample.ko and CONFIG_IOMMUFD_TEST.
+ */
+#include <fcntl.h>
+#include <errno.h>
+#include <setjmp.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/iommufd.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+/*
+ * The iommufd mock domain test interface.  The iommufd selftest helpers
+ * bring their own harness, which does not mix with the KVM selftest library.
+ */
+#include "../../../../../drivers/iommu/iommufd/iommufd_test.h"
+
+/* Mirrors samples/kvm/mem_provider_sample.h */
+#define MPS_FLAG_MMAP_CAPABLE	(1u << 0)
+
+struct mps_new_child { __u32 flags, pad; __u64 offset, len; };
+struct mps_move { __s32 src_fd, dst_fd; __u64 offset, len; };
+struct mps_donate { __s32 fd; __u32 pad; __u64 offset, len; };
+struct mps_ctl_range { __s32 fd; __u32 value; __u64 offset, len; };
+struct mps_stats {
+	__u64 region_offset, region_len, owned_pages, absent_pages, readonly_pages;
+};
+
+#define MPS_NEW_CHILD		_IOW('P', 5, struct mps_new_child)
+#define MPS_MOVE		_IOW('P', 6, struct mps_move)
+#define MPS_DONATE		_IOW('P', 7, struct mps_donate)
+#define MPS_RECLAIM		_IOW('P', 8, struct mps_donate)
+#define MPS_GET_STATS		_IOR('P', 10, struct mps_stats)
+#define MPS_CTL_SET_READONLY	_IOW('P', 11, struct mps_ctl_range)
+
+#define PAGE		0x1000ULL
+#define CHILD_SIZE	0x400000ULL		/* 4 MiB per child */
+#define GPA		(1ULL << 32)		/* each VM maps its child here */
+#define IOVA		(1ULL << 28)		/* inside the mock aperture */
+#define MAGIC_A		0xa11ce000a11ce000ULL
+#define MAGIC_B		0xb0bb0bb0b0bb0bb0ULL
+
+/*
+ * The ranges of A and B in the root overlap on [SHARED_OFF, +SHARED_LEN).
+ * A is created first and has its whole range, B is created second and has
+ * its range without the overlap.  The overlap can then move from A to B.
+ */
+#define A_OFF		0ULL
+#define SHARED_LEN	(4 * PAGE)
+#define B_OFF		(CHILD_SIZE - SHARED_LEN)
+#define SHARED_OFF	B_OFF
+#define ROOT_SIZE	(B_OFF + CHILD_SIZE)
+
+#define A_SHARED_GPA	(GPA + (SHARED_OFF - A_OFF))
+#define B_SHARED_GPA	(GPA + (SHARED_OFF - B_OFF))
+#define A_SHARED_IOVA	(IOVA + (SHARED_OFF - A_OFF))
+#define B_SHARED_IOVA	(IOVA + (SHARED_OFF - B_OFF))
+
+/* A page of A that never moves. */
+#define RO_OFF		(64 * PAGE)
+#define RO_GPA		(GPA + RO_OFF)
+
+/* Pages of A that are donated and reclaimed. */
+#define DON_OFF		(8 * PAGE)
+#define DON_GPA		(GPA + DON_OFF)
+#define DON_IOVA	(IOVA + DON_OFF)
+
+/* A page that B has from the start. */
+#define B_HOME_OFF	(CHILD_SIZE / 2)
+#define B_HOME_GPA	(GPA + B_HOME_OFF)
+
+struct guest_args {
+	uint64_t write_gpa;	/* 0 to skip */
+	uint64_t write_val;
+	uint64_t read_gpa;	/* 0 to skip; the value goes to GUEST_SYNC */
+};
+
+static void guest_code(struct guest_args *a)
+{
+	if (a->write_gpa)
+		*(volatile uint64_t *)a->write_gpa = a->write_val;
+	if (a->read_gpa)
+		GUEST_SYNC(*(volatile uint64_t *)a->read_gpa);
+	GUEST_DONE();
+}
+
+/* ---- Control process ---------------------------------------------------- */
+
+static int ctl;
+
+static int ctl_new_child(uint64_t off, uint64_t len)
+{
+	struct mps_new_child nc = {
+		.flags = MPS_FLAG_MMAP_CAPABLE,
+		.offset = off, .len = len,
+	};
+	int fd = ioctl(ctl, MPS_NEW_CHILD, &nc);
+
+	TEST_ASSERT(fd >= 0, "NEW_CHILD(%#llx, %#llx) errno=%d",
+		    (unsigned long long)off, (unsigned long long)len, errno);
+	return fd;
+}
+
+static int ctl_move(int src, int dst, uint64_t off, uint64_t len)
+{
+	struct mps_move mv = { .src_fd = src, .dst_fd = dst,
+			       .offset = off, .len = len };
+
+	return ioctl(ctl, MPS_MOVE, &mv) ? -errno : 0;
+}
+
+static void ctl_donate(int fd, uint64_t off, uint64_t len, bool reclaim)
+{
+	struct mps_donate d = { .fd = fd, .offset = off, .len = len };
+
+	TEST_ASSERT(!ioctl(ctl, reclaim ? MPS_RECLAIM : MPS_DONATE, &d),
+		    "%s errno=%d", reclaim ? "RECLAIM" : "DONATE", errno);
+}
+
+static void ctl_readonly(int fd, uint64_t off, bool ro)
+{
+	struct mps_ctl_range cr = { .fd = fd, .value = ro, .offset = off,
+				    .len = PAGE };
+
+	TEST_ASSERT(!ioctl(ctl, MPS_CTL_SET_READONLY, &cr),
+		    "CTL_SET_READONLY errno=%d", errno);
+}
+
+/* ---- VMM ---------------------------------------------------------------- */
+
+struct vmm {
+	const char *name;
+	int child;		/* the provider file */
+	int gmem;		/* the guest_memfd backed by it */
+	int iommufd;
+	uint32_t ioas, stdev, hwpt;
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	void *hva;		/* host mapping of the whole guest_memfd */
+	gva_t args_gva;
+};
+
+static void vmm_create(struct vmm *v)
+{
+	v->vm = vm_create_with_one_vcpu(&v->vcpu, guest_code);
+	v->args_gva = vm_alloc_page(v->vm);
+}
+
+static bool vmm_iova_mapped(struct vmm *v, uint64_t iova, uint64_t len,
+			    bool mapped)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd), .op = IOMMU_TEST_OP_MD_CHECK_MAPPED,
+		.id = v->hwpt,
+		.check_mapped = { .mapped = mapped, .iova = iova,
+				  .length = len },
+	};
+
+	return !ioctl(v->iommufd, IOMMU_TEST_CMD, &cmd);
+}
+
+static uint64_t vmm_iova_phys(struct vmm *v, uint64_t iova)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd), .op = IOMMU_TEST_OP_MD_IOVA_TO_PHYS,
+		.id = v->hwpt,
+		.iova_to_phys = { .iova = iova },
+	};
+
+	if (ioctl(v->iommufd, IOMMU_TEST_CMD, &cmd))
+		return 0;
+	return cmd.iova_to_phys.out_phys;
+}
+
+/* Give the child file to KVM and to iommufd. */
+static void vmm_attach(struct vmm *v, int child_fd)
+{
+	struct iommu_ioas_alloc alloc = { .size = sizeof(alloc) };
+	struct kvm_create_guest_memfd gm = {
+		.size = CHILD_SIZE,
+		.flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_PROVIDER,
+		.provider_fd = child_fd,
+	};
+	struct iommu_test_cmd mock = {
+		.size = sizeof(mock), .op = IOMMU_TEST_OP_MOCK_DOMAIN,
+	};
+	struct iommu_ioas_map_file map = {
+		.size = sizeof(map),
+		.flags = IOMMU_IOAS_MAP_FIXED_IOVA | IOMMU_IOAS_MAP_READABLE |
+			 IOMMU_IOAS_MAP_WRITEABLE,
+		.fd = child_fd, .start = 0, .length = CHILD_SIZE, .iova = IOVA,
+	};
+	int r;
+
+	v->child = child_fd;
+
+	/* KVM: a guest_memfd backed by the child, behind one memslot. */
+	v->gmem = __vm_ioctl(v->vm, KVM_CREATE_GUEST_MEMFD, &gm);
+	TEST_ASSERT(v->gmem >= 0, "%s: KVM_CREATE_GUEST_MEMFD errno=%d",
+		    v->name, errno);
+	v->hva = mmap(NULL, CHILD_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
+		      v->gmem, 0);
+	TEST_ASSERT(v->hva != MAP_FAILED, "%s: mmap(guest_memfd) errno=%d",
+		    v->name, errno);
+	r = __vm_set_user_memory_region2(v->vm, 10, KVM_MEM_GUEST_MEMFD, GPA,
+					 CHILD_SIZE, v->hva, v->gmem, 0);
+	TEST_ASSERT(!r, "%s: SET_USER_MEMORY_REGION2 errno=%d", v->name,
+		    errno);
+	virt_map(v->vm, GPA, GPA, CHILD_SIZE / PAGE);
+
+	/* iommufd: the same child file, in an IOAS on a mock domain. */
+	v->iommufd = open("/dev/iommu", O_RDWR);
+	__TEST_REQUIRE(v->iommufd >= 0, "iommufd unavailable");
+	TEST_ASSERT(!ioctl(v->iommufd, IOMMU_IOAS_ALLOC, &alloc),
+		    "IOAS_ALLOC errno=%d", errno);
+	v->ioas = alloc.out_ioas_id;
+
+	mock.id = v->ioas;
+	__TEST_REQUIRE(!ioctl(v->iommufd, IOMMU_TEST_CMD, &mock),
+		       "no iommufd mock domain (CONFIG_IOMMUFD_TEST?)");
+	v->stdev = mock.mock_domain.out_stdev_id;
+	v->hwpt = mock.mock_domain.out_hwpt_id;
+
+	map.ioas_id = v->ioas;
+	TEST_ASSERT(!ioctl(v->iommufd, IOMMU_IOAS_MAP_FILE, &map),
+		    "%s: IOAS_MAP_FILE(provider) errno=%d", v->name, errno);
+}
+
+/*
+ * Continue the guest.  KVM_RUN fails with EFAULT for KVM_EXIT_MEMORY_FAULT,
+ * which several scenarios expect; the expect_*() helpers check the exit.
+ */
+static void vmm_resume(struct vmm *v)
+{
+	int r = _vcpu_run(v->vcpu);
+
+	TEST_ASSERT(!r || (errno == EFAULT &&
+			   v->vcpu->run->exit_reason == KVM_EXIT_MEMORY_FAULT),
+		    "%s: KVM_RUN r=%d errno=%d exit=%s", v->name, r, errno,
+		    exit_reason_str(v->vcpu->run->exit_reason));
+}
+
+static void vmm_run(struct vmm *v, uint64_t write_gpa, uint64_t write_val,
+		    uint64_t read_gpa)
+{
+	struct guest_args *a = addr_gva2hva(v->vm, v->args_gva);
+
+	a->write_gpa = write_gpa;
+	a->write_val = write_val;
+	a->read_gpa = read_gpa;
+	vcpu_arch_set_entry_point(v->vcpu, guest_code);
+	vcpu_args_set(v->vcpu, 1, v->args_gva);
+	vmm_resume(v);
+}
+
+static void expect_done(struct vmm *v)
+{
+	struct kvm_run *run = v->vcpu->run;
+	struct ucall uc;
+
+	TEST_ASSERT(run->exit_reason != KVM_EXIT_MEMORY_FAULT,
+		    "%s: unexpected memory fault at gpa %#llx", v->name,
+		    (unsigned long long)run->memory_fault.gpa);
+	TEST_ASSERT(get_ucall(v->vcpu, &uc) == UCALL_DONE, "%s: guest exit %s",
+		    v->name, exit_reason_str(run->exit_reason));
+}
+
+static uint64_t expect_sync_then_done(struct vmm *v)
+{
+	struct ucall uc;
+	uint64_t val;
+
+	TEST_ASSERT(get_ucall(v->vcpu, &uc) == UCALL_SYNC, "%s: guest exit %s",
+		    v->name, exit_reason_str(v->vcpu->run->exit_reason));
+	val = uc.args[1];
+	vmm_resume(v);
+	expect_done(v);
+	return val;
+}
+
+static void expect_memory_fault(struct vmm *v, uint64_t gpa)
+{
+	struct kvm_run *run = v->vcpu->run;
+
+	TEST_ASSERT(run->exit_reason == KVM_EXIT_MEMORY_FAULT,
+		    "%s: want KVM_EXIT_MEMORY_FAULT, got %s", v->name,
+		    exit_reason_str(run->exit_reason));
+	TEST_ASSERT(run->memory_fault.gpa == gpa,
+		    "%s: fault at gpa %#llx, want %#llx", v->name,
+		    (unsigned long long)run->memory_fault.gpa,
+		    (unsigned long long)gpa);
+}
+
+static void vmm_stats(struct vmm *v, struct mps_stats *st)
+{
+	TEST_ASSERT(!ioctl(v->child, MPS_GET_STATS, st),
+		    "%s: GET_STATS errno=%d", v->name, errno);
+}
+
+static void vmm_destroy(struct vmm *v)
+{
+	close(v->iommufd);
+	kvm_vm_free(v->vm);
+	munmap(v->hva, CHILD_SIZE);
+	close(v->gmem);
+	close(v->child);
+}
+
+static sigjmp_buf host_jmp;
+
+static void host_sig(int sig)
+{
+	siglongjmp(host_jmp, sig);
+}
+
+/*
+ * Read @addr, or write @val to it, through a host mapping.  Return the
+ * signal that the access raised, or 0.
+ */
+static int host_access(volatile uint64_t *addr, bool write, uint64_t val)
+{
+	struct sigaction sa = { .sa_handler = host_sig }, old_bus, old_segv;
+	int sig;
+
+	sigaction(SIGBUS, &sa, &old_bus);
+	sigaction(SIGSEGV, &sa, &old_segv);
+	sig = sigsetjmp(host_jmp, 1);
+	if (!sig) {
+		if (write)
+			*addr = val;
+		else
+			(void)*addr;
+	}
+	sigaction(SIGBUS, &old_bus, NULL);
+	sigaction(SIGSEGV, &old_segv, NULL);
+	return sig;
+}
+
+#define host_read_faults(p)	(host_access(p, false, 0) == SIGBUS)
+#define host_write_faults(p, v)	(host_access(p, true, v) == SIGBUS)
+
+/* ---- Scenarios ---------------------------------------------------------- */
+
+static void scenario_launch(struct vmm *a, struct vmm *b)
+{
+	pr_info("1. launch\n");
+	vmm_run(a, GPA, MAGIC_A, 0);
+	expect_done(a);
+	vmm_run(b, B_HOME_GPA, MAGIC_B, 0);
+	expect_done(b);
+	TEST_ASSERT(*(volatile uint64_t *)a->hva == MAGIC_A,
+		    "A: the guest write is not visible to the host");
+	TEST_ASSERT(*(volatile uint64_t *)(b->hva + B_HOME_OFF) == MAGIC_B,
+		    "B: the guest write is not visible to the host");
+
+	/* A has its whole range; B lacks the shared window. */
+	TEST_ASSERT(vmm_iova_mapped(a, IOVA, CHILD_SIZE, true),
+		    "A: device mapping incomplete");
+	TEST_ASSERT(vmm_iova_mapped(b, B_SHARED_IOVA, SHARED_LEN, false),
+		    "B: device maps a window that B does not have");
+	TEST_ASSERT(vmm_iova_mapped(b, B_SHARED_IOVA + SHARED_LEN,
+				    CHILD_SIZE - SHARED_LEN, true),
+		    "B: device mapping incomplete");
+}
+
+static void scenario_move(struct vmm *a, struct vmm *b)
+{
+	struct mps_stats st;
+	uint64_t frame;
+	int r;
+
+	pr_info("2. move A -> B\n");
+	vmm_run(a, A_SHARED_GPA, MAGIC_A, 0);
+	expect_done(a);
+	frame = vmm_iova_phys(a, A_SHARED_IOVA);
+	TEST_ASSERT(frame, "A: window not mapped before the move");
+	/* Map the window on the host too, so that the move must zap it. */
+	TEST_ASSERT(*(volatile uint64_t *)(a->hva + SHARED_OFF - A_OFF) ==
+		    MAGIC_A, "A: host mapping does not see the guest write");
+
+	/* B does not have the window yet. */
+	vmm_run(b, 0, 0, B_SHARED_GPA);
+	expect_memory_fault(b, B_SHARED_GPA);
+
+	/* A move outside the destination's range is refused. */
+	r = ctl_move(a->child, b->child, RO_OFF, PAGE);
+	TEST_ASSERT(r == -EINVAL, "MOVE outside B's range returned %d", r);
+
+	r = ctl_move(a->child, b->child, SHARED_OFF, SHARED_LEN);
+	TEST_ASSERT(!r, "MOVE returned %d", r);
+
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == (CHILD_SIZE - SHARED_LEN) / PAGE,
+		    "A has %llu pages after the move",
+		    (unsigned long long)st.owned_pages);
+	vmm_stats(b, &st);
+	TEST_ASSERT(st.owned_pages == CHILD_SIZE / PAGE,
+		    "B has %llu pages after the move",
+		    (unsigned long long)st.owned_pages);
+
+	/* A's device lost the window and only the window. */
+	TEST_ASSERT(vmm_iova_mapped(a, A_SHARED_IOVA, SHARED_LEN, false),
+		    "A: moved window still mapped");
+	TEST_ASSERT(vmm_iova_mapped(a, IOVA, CHILD_SIZE - SHARED_LEN, true),
+		    "A: the rest of the mapping went too");
+
+	/* A's guest and A's host mapping fault on the window. */
+	vmm_run(a, 0, 0, A_SHARED_GPA);
+	expect_memory_fault(a, A_SHARED_GPA);
+	TEST_ASSERT(host_read_faults(a->hva + SHARED_OFF - A_OFF),
+		    "A: host mapping of the moved window still readable");
+
+	/* B reads what A wrote, and B's device reaches A's frame. */
+	vmm_run(b, 0, 0, B_SHARED_GPA);
+	TEST_ASSERT(expect_sync_then_done(b) == MAGIC_A,
+		    "B does not see A's write");
+	TEST_ASSERT(*(volatile uint64_t *)(b->hva + SHARED_OFF - B_OFF) ==
+		    MAGIC_A, "B: host mapping does not see A's write");
+	TEST_ASSERT(vmm_iova_phys(b, B_SHARED_IOVA) == frame,
+		    "B: window is not on the frame A had");
+}
+
+static void scenario_donate(struct vmm *a)
+{
+	struct mps_stats st;
+
+	pr_info("3. donate and reclaim\n");
+	/* Map the page on the host, so that the donation must zap it. */
+	TEST_ASSERT(!host_access(a->hva + DON_OFF, false, 0),
+		    "A: host mapping faults before the donation");
+	ctl_donate(a->child, DON_OFF, 2 * PAGE, false);
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == (CHILD_SIZE - SHARED_LEN) / PAGE - 2,
+		    "A has %llu pages after the donation",
+		    (unsigned long long)st.owned_pages);
+	TEST_ASSERT(vmm_iova_mapped(a, DON_IOVA, 2 * PAGE, false),
+		    "A: donated pages still mapped for the device");
+	TEST_ASSERT(host_read_faults(a->hva + DON_OFF),
+		    "A: host mapping of a donated page still readable");
+	vmm_run(a, 0, 0, DON_GPA);
+	expect_memory_fault(a, DON_GPA);
+
+	ctl_donate(a->child, DON_OFF, 2 * PAGE, true);
+	vmm_stats(a, &st);
+	TEST_ASSERT(st.owned_pages == (CHILD_SIZE - SHARED_LEN) / PAGE,
+		    "A has %llu pages after the reclaim",
+		    (unsigned long long)st.owned_pages);
+	TEST_ASSERT(vmm_iova_mapped(a, DON_IOVA, 2 * PAGE, true),
+		    "A: reclaimed pages not mapped again for the device");
+	TEST_ASSERT(!host_access(a->hva + DON_OFF, false, 0),
+		    "A: host mapping of a reclaimed page faults");
+	vmm_run(a, 0, 0, DON_GPA);
+	expect_sync_then_done(a);
+}
+
+static void scenario_readonly(struct vmm *a)
+{
+	volatile uint64_t *page = (volatile uint64_t *)(a->hva + RO_OFF);
+
+	pr_info("4. read only\n");
+	*page = MAGIC_B;
+
+	ctl_readonly(a->child, RO_OFF, true);
+	TEST_ASSERT(vmm_iova_mapped(a, IOVA + RO_OFF, PAGE, true),
+		    "A: read-only page not mapped for the device");
+
+	vmm_run(a, RO_GPA, MAGIC_A, 0);
+	expect_memory_fault(a, RO_GPA);
+	TEST_ASSERT(host_write_faults(page, MAGIC_A),
+		    "A: host write to a read-only page did not fault");
+	TEST_ASSERT(*page == MAGIC_B, "write to a read-only page landed");
+
+	/* Clear the bit; the guest retries the same write. */
+	ctl_readonly(a->child, RO_OFF, false);
+	vmm_resume(a);
+	expect_done(a);
+	TEST_ASSERT(*page == MAGIC_A, "write after clearing read only lost");
+	TEST_ASSERT(!host_write_faults(page, MAGIC_B),
+		    "A: host write after clearing read only faulted");
+	TEST_ASSERT(*page == MAGIC_B, "host write after clearing read only lost");
+}
+
+static void scenario_window(struct vmm *a)
+{
+	volatile uint64_t *win;
+
+	pr_info("5. host mapping\n");
+	win = mmap(NULL, PAGE, PROT_READ | PROT_WRITE, MAP_SHARED, a->gmem,
+		   DON_OFF);
+	TEST_ASSERT(win != MAP_FAILED, "mmap(guest_memfd, page) errno=%d",
+		    errno);
+	*win = MAGIC_A;
+	TEST_ASSERT(*(volatile uint64_t *)(a->hva + DON_OFF) == MAGIC_A,
+		    "the two host mappings disagree");
+
+	ctl_donate(a->child, DON_OFF, PAGE, false);
+
+	TEST_ASSERT(host_read_faults(win),
+		    "host mapping still readable after the donation");
+
+	ctl_donate(a->child, DON_OFF, PAGE, true);
+	TEST_ASSERT(*win == MAGIC_A, "host mapping did not come back");
+	munmap((void *)win, PAGE);
+}
+
+int main(void)
+{
+	struct vmm a = { .name = "A" }, b = { .name = "B" };
+	struct mps_new_child probe = { .offset = ROOT_SIZE - PAGE, .len = PAGE };
+	int fd;
+
+	TEST_REQUIRE(kvm_check_cap(KVM_CAP_GUEST_MEMFD_FLAGS) &
+		     GUEST_MEMFD_FLAG_USE_PROVIDER);
+
+	ctl = open("/dev/mem_provider_sample", O_RDWR);
+	__TEST_REQUIRE(ctl >= 0, "mem_provider_sample not loaded");
+
+	/*
+	 * Without addr= and len=, the first NEW_CHILD sizes the root, so ask
+	 * for its last page first and give it back.  A fixed root that is too
+	 * small makes the test skip.
+	 */
+	fd = ioctl(ctl, MPS_NEW_CHILD, &probe);
+	__TEST_REQUIRE(fd >= 0, "provider root smaller than %#llx bytes",
+		       (unsigned long long)ROOT_SIZE);
+	close(fd);
+
+	vmm_create(&a);
+	vmm_create(&b);
+	vmm_attach(&a, ctl_new_child(A_OFF, CHILD_SIZE));
+	vmm_attach(&b, ctl_new_child(B_OFF, CHILD_SIZE));
+
+	scenario_launch(&a, &b);
+	scenario_move(&a, &b);
+	scenario_donate(&a);
+	scenario_readonly(&a);
+	scenario_window(&a);
+
+	vmm_destroy(&a);
+	vmm_destroy(&b);
+	close(ctl);
+	pr_info("mem_provider: all scenarios passed\n");
+	return 0;
+}

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Aneesh Kumar K.V]

Fred Griffoul <griffoul@gmail.com> writes:

> From: Fred Griffoul <fgriffo@amazon.co.uk>
>
> Some drivers manage RAM outside the page allocator: a carve-out, device
> memory, or memory that a host component moves between VMs while they
> run. This memory often has no struct page. The driver wants to lend
> pages to a VM and to its devices, and to take any of them back later.
>
> David Woodhouse's series "KVM: Allow alternative providers of
> guest_memfd backed by PFNMAP memory" lets such a driver back a
> guest_memfd through struct kvm_gmem_ops:
>
>   https://lore.kernel.org/kvm/20260720111259.122911-1-dwmw2@infradead.org/
>
> That covers the guest, but not the VM's devices, which his cover
> letter left as an open question. This series adds a small interface in
> mm: the driver that owns the memory becomes a provider, and the code
> that maps it attaches as a consumer. There are two consumers, a
> guest_memfd backend and IOMMU_IOAS_MAP_FILE in iommufd.
>
> The series is on top of David's v2 (base 0e35b9b6ec0f). Patch 1 was
> part of the earlier dma-buf RFC. The provider backend depends on it, so
> this series carries it.
>


I have been working on something similar for confidential computing use
cases. There is likely some overlap between this series and what I posted
at [1].

I have not reviewed this series in detail yet, but I will compare the two
approaches and identify what can be shared between the use cases. Please
let me know how you would like to proceed.

[1] https://lore.kernel.org/all/20261010072504.536230-1-aneesh.kumar@kernel.org/

-aneesh

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

On Sat, Oct 10, 2026, Aneesh Kumar K.V wrote:
> I have been working on something similar for confidential computing use
> cases. There is likely some overlap between this series and what I posted
> at [1].
>
> I have not reviewed this series in detail yet, but I will compare the two
> approaches and identify what can be shared between the use cases. Please
> let me know how you would like to proceed.
>
> [1] https://lore.kernel.org/all/20261010072504.536230-1-aneesh.kumar@kernel.org/

Thanks for the pointer. I read your series, and Ackerley's resource fd
RFC that it builds on. The use cases differ, but the interfaces overlap,
and it would be good to agree on the common parts.

All three series add a flag and an fd to KVM_CREATE_GUEST_MEMFD. I
think one resource_fd is enough, with guest_memfd picking the backend
from the kind of file it gets. I am happy to drop provider_fd for
Ackerley's flag and field. We also find the provider from the file in
three different ways; I would like us to pick one, preferably without
a registry.

The main constraint on our side is that the provider stays the authority
on which frame backs which page: it moves pages between VMs and has to
survive a live update. What would not work for us is guest_memfd
becoming the primary manager of PFN memory, as Ackerley suggests. Your
get_pfn() leaves VFIO as the authority for the BAR, so I think we agree
there.

You refuse dma-buf exports while guest_memfd is attached. That is
roughly what the memory provider interface is for: one owner, several
users, and one revoke that reaches all of them. A BAR exposed as a
provider file could be mapped by both guest_memfd and iommufd.

I plan to change the revoke anyway, because iommufd must not leave the
device with a hole: it should replace mappings in place instead of
unmapping and remapping. Would that fit your case? I suspect it depends
on locking, for example whether you have to revoke while holding
memory_lock.

Shared/private state and the conversion hooks would stay in
guest_memfd and the arch code, as in your series.

Does that sound reasonable? Ackerley, David, any thoughts?

Thanks,
Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

On Mon, 2026-10-12 at 09:58 +0000, Fred Griffoul wrote:
> 
> I plan to change the revoke anyway, because iommufd must not leave the
> device with a hole: it should replace mappings in place instead of
> unmapping and remapping. Would that fit your case? I suspect it depends
> on locking, for example whether you have to revoke while holding
> memory_lock.

I was keeping that hack *entirely* separate from the guest_memfd work.

That's a quirk on the IOAS in iommufd: "whenever you unmap, put *this*
page in its place instead". It doesn't belong anywhere near what we're
doing here, ideally.

I even talked to Jason about it about it at LPC, and he didn't hurt me.
If we implement a hitless¹ REPLACE operation then the scratch-page ends
up being a fairly trivial thing to redirect UNMAP to REPLACE (the
tricksy detail being that it uses the *same* scratch page for every
IOVA in the range).

¹ why do we call it that? Shouldn't it be 'missless'?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Ackerley Tng]

Fred Griffoul <griffoul@gmail.com> writes:

> On Sat, Oct 10, 2026, Aneesh Kumar K.V wrote:
>> I have been working on something similar for confidential computing use
>> cases. There is likely some overlap between this series and what I posted
>> at [1].
>>
>> I have not reviewed this series in detail yet, but I will compare the two
>> approaches and identify what can be shared between the use cases. Please
>> let me know how you would like to proceed.
>>
>> [1] https://lore.kernel.org/all/20261010072504.536230-1-aneesh.kumar@kernel.org/
>
> Thanks for the pointer. I read your series, and Ackerley's resource fd
> RFC that it builds on. The use cases differ, but the interfaces overlap,
> and it would be good to agree on the common parts.
>
> All three series add a flag and an fd to KVM_CREATE_GUEST_MEMFD. I
> think one resource_fd is enough, with guest_memfd picking the backend
> from the kind of file it gets. I am happy to drop provider_fd for
> Ackerley's flag and field. We also find the provider from the file in
> three different ways; I would like us to pick one, preferably without
> a registry.
>

David brought up many good points at the LPC discussion, I need to wrap
my head around your model. I'll look closely at Fred and David's
versions this week.

These thoughts are just from reading this email, hope to get more
clarification :)

> The main constraint on our side is that the provider stays the authority
> on which frame backs which page:

Wait, is there some magic done to make struct pages represent different
frames, as in PFNs? In your model, the struct page <=> PFN mapping
doesn't stay constant?

> it moves pages between VMs

Do multiple VMs map the same physical memory at the same time, or is it
a move like below?

I'm thinking that if guest_memfd wraps your provider, then your provider
can issue a revocation to guest_memfd (ctrl-f for
kvm_gmem_provider_revoke in [1]) on the move source guest_memfd inode,
then wait for an allocation request from the move target guest_memfd
inode.

[1] https://lpc.events/event/20/contributions/2551/attachments/2047/4693/lpc-2026-gmem-other-backends.pdf

> and has to
> survive a live update.

guest_memfd is going to support preservation [2]. Tarun's series just
preserves memory from guest_memfd's default provider (buddy allocator)
now. For your provider, I imagine the preservation flow would preserve
whatever metadata in guest_memfd (like the attributes in the maple tree)
and then go on to notify the provider to preserve some range of PFNs (or
folios). Your provider would then basically do nothing, since those PFNs
already survive a live update.

[2] https://lore.kernel.org/all/20260728121138.1103610-1-tarunsahu@google.com/

> What would not work for us is guest_memfd
> becoming the primary manager of PFN memory, as Ackerley suggests. Your
> get_pfn() leaves VFIO as the authority for the BAR, so I think we agree
> there.
>
> You refuse dma-buf exports while guest_memfd is attached. That is
> roughly what the memory provider interface is for: one owner, several
> users, and one revoke that reaches all of them. A BAR exposed as a
> provider file could be mapped by both guest_memfd and iommufd.
>

Will read the rest of this series and reply again :)

> I plan to change the revoke anyway, because iommufd must not leave the
> device with a hole: it should replace mappings in place instead of
> unmapping and remapping. Would that fit your case? I suspect it depends
> on locking, for example whether you have to revoke while holding
> memory_lock.
>
> Shared/private state and the conversion hooks would stay in
> guest_memfd and the arch code, as in your series.
>
> Does that sound reasonable? Ackerley, David, any thoughts?
>
> Thanks,
> Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

Ackerley Tng <ackerleytng@google.com> writes:

> Wait, is there some magic done to make struct pages represent different
> frames, as in PFNs? In your model, the struct page <=> PFN mapping
> doesn't stay constant?

No, our memory has no struct page: it is a carve-out hidden from the
host with memmap=. I meant the mapping from guest_memfd file offset to
PFN. The PFNs never change, only which file and offset they back.

> Do multiple VMs map the same physical memory at the same time, or is it
> a move like below?

Mostly moves: a page has one owner at a time. There is also a share,
where a parent keeps a range and gives a child a view of it, so two VMs
can map the same frames for a while.

> I'm thinking that if guest_memfd wraps your provider, then your provider
> can issue a revocation to guest_memfd (ctrl-f for
> kvm_gmem_provider_revoke in [1]) on the move source guest_memfd inode,
> then wait for an allocation request from the move target guest_memfd
> inode.

For the guest, this series works the same way: the provider revokes the
range on the source, and the target's next guest fault asks the provider
for the frame. Devices are different, because a device may not support
faults. So the provider also revokes the range on the target once it
owns the frames, and iommufd maps them right away. In other words, a
revoke means that the frames behind a range changed, whether they went
away or arrived.

> guest_memfd is going to support preservation [2]. Tarun's series just
> preserves memory from guest_memfd's default provider (buddy allocator)
> now. For your provider, I imagine the preservation flow would preserve
> whatever metadata in guest_memfd (like the attributes in the maple tree)
> and then go on to notify the provider to preserve some range of PFNs (or
> folios). Your provider would then basically do nothing, since those PFNs
> already survive a live update.

Yes, that would work for us, as long as guest_memfd does not also have
to save the offset to PFN layout: the provider keeps it across the
update, and guest_memfd can ask again afterwards.

Thanks,
Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On Mon, 2026-10-12 at 15:07 +0000, Fred Griffoul wrote:
>  Devices are different, because a device may not support
> faults. So the provider also revokes the range on the target once it
> owns the frames, and iommufd maps them right away. In other words, a
> revoke means that the frames behind a range changed, whether they went
> away or arrived.

That would map to the IOMMUFD 'REPLACE' operation that hitlessly
replaces a PTE, but does anyone actually need that?

As I said, assume that the "crappy devices can't take faults and need a
scratch patch" issue is dealt with *elsewhere* and not relevant to the
guest_memfd/memprovider work. That's purely a quirk on a given IOAS
within iommufd.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

On Mon, 2026-10-12 at 16:13 +0100, David Woodhouse wrote:
> On Mon, 2026-10-12 at 15:07 +0000, Fred Griffoul wrote:
> >  Devices are different, because a device may not support
> > faults. So the provider also revokes the range on the target once it
> > owns the frames, and iommufd maps them right away. In other words, a
> > revoke means that the frames behind a range changed, whether they went
> > away or arrived.
>
> That would map to the IOMMUFD 'REPLACE' operation that hitlessly
> replaces a PTE, but does anyone actually need that?

No, not on the target. Before the move the child has nothing mapped
in that range, so when the frames arrive iommufd just maps them. That
is a plain map, not a REPLACE.

> As I said, assume that the "crappy devices can't take faults and need a
> scratch patch" issue is dealt with *elsewhere* and not relevant to the
> guest_memfd/memprovider work. That's purely a quirk on a given IOAS
> within iommufd.

Agreed. The replace is only needed on the source, where the frames go
away and a device that cannot fault must see the scratch page instead.
With that quirk on the IOAS, iommufd does it, and the provider does not
need to know about it.

Thanks,
Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Ackerley Tng]

Hey David/Fred,

I took some time to read the patches, hope I got everything :) The
biggest difference between your approach and mine is who (provider or
guest_memfd) manages the zapping across all the users.

The memory users here are

1. [KVM] The guest (KVM manages the stage 2 page tables)
2. [IOMMU] The IOMMU (iommufd manages the stage 2 page tables)
3. [USERSPACE] Host userspace (guest_memfd handles faults, I guess you
   could say core mm manages host userspace page tables)
4. [DIRECTMAP] Kernel direct map. Your provider already provides PFNs
   that are presumably not in the direct map. For other providers, the
   direct map is another mapping that may need to be zapped on
   private/shared conversions. (Listing this here for completeness, I
   think this should be tackled separately.)

I'll use [LABEL] to illustrate the way notifications flow in your
approach vs mine.

When the provider wants to revoke the page, the notifications/function
calls flow this way, the last [ENTITY] in the chain actually does the
zapping.

+ [PROVIDER] -> [KVM]
+ [PROVIDER] -> [DMABUF] -> [IOMMU]
+ [PROVIDER] -> [GUEST_MEMFD] -> [USERSPACE] (AFAICT this was not
  implemented in the RFC.)

That's when the provider wants to revoke the page. IIUC guest_memfd
conversions was not illustrated in the RFC: what happens on a shared
-> private conversion?

+ [KVM]: Zap only shared pages, keep private pages mapped.
+ [IOMMU]:
    + Before Confidential IO: so far, IOMMU mappings are mapped
      indirectly via [USERSPACE] mappings. guest_memfd zaps
      [USERSPACE] mappings, then checks refcounts to ensure no other
      mappings (which includes IOMMU). See __kvm_gmem_set_attributes()
      on kvm-x86/coco [2].
    + With Confidential IO: shared IOMMU mapping need to be zapped,
      keep mappings of private pages.
+ [USERSPACE]: Zap all mappings on shared->private conversions.
+ [DIRECTMAP]: Can zap these mappings (not implemented yet, but
  discussed in the context of Direct Map Removal support for
  guest_memfd [3])

To have the conversions work, in your model, we'd need to do

+ [GUEST_MEMFD] -> [PROVIDER] -> [KVM] (zap only shared pages)
+ [GUEST_MEMFD] -> [PROVIDER] -> [DMABUF] -> [IOMMU] (zap only shared pages)
+ [GUEST_MEMFD] -> [USERSPACE] (IIUC the provider doesn't directly
  allow mapping to userspace, also in your approach the provider
  pretty much replaces all the innards of guest_memfd anyway, hence
  I'm showing this as gmem going direct to zap userspace)

With my proposal [1], we could do

+ [GUEST_MEMFD] -> [KVM] (zap only shared pages, implemented in [2])
+ [GUEST_MEMFD] -> [IOMMU] (zap only shared pages)
+ [GUEST_MEMFD] -> [USERSPACE] (implemented in [2])
+ [GUEST_MEMFD] -> [DIRECTMAP] (future)

In my proposal, I agree with you completely that the provider must be
allowed to revoke memory. Using the sample in this RFC, gmem_fd_ioctl()
called with !GMEM_PROVIDER_SET_PRESENT is where the provider wants to
revoke memory, and the revocation in my model would look like

  [PROVIDER] -> [GUEST_MEMFD]

and [GUEST_MEMFD] would then fan it out to [KVM], [IOMMU], [USERSPACE]
and [DIRECTMAP].

The [GUEST_MEMFD] -> [IOMMU] is missing from my proposal because I was
seeing that as a separate patch series. I think what we want is to have
IOMMU_IOAS_MAP_FILE be able to accept a guest_memfd file. Currently,
during the mapping process(), memfd_pin_folios() is used, which has this
check

	if (!shmem_file(memfd) && !is_file_hugepages(memfd))
		return -EINVAL;

that disqualifies guest_memfd. Refcounting won't work for guest_memfd()
folios because of conversions (see kvm_gmem_has_outstanding_references
at [4]). The implementation I have in mind is to have the the iommufd
instance be tracked in guest_memfd during the pinning process, and
that's the pointer guest_memfd will use to get iommufd to zap during
[GUEST_MEMFD] -> [IOMMU]. This [GUEST_MEMFD] -> [IOMMU] zapping was
previously also proposed in the context of Confidential IO [5].

[1] https://lore.kernel.org/all/20260925-gmem-tmpfs-backend-v1-0-d36159822d18@google.com/
[2] https://github.com/kvm-x86/linux/blob/a6270ed4bec0caa25913082e71ce22254e9cd3ed/virt/kvm/guest_memfd.c#L685
[3] https://lore.kernel.org/all/20260410151746.61150-1-kalyazin@amazon.com/
[4] https://github.com/kvm-x86/linux/blob/a6270ed4bec0caa25913082e71ce22254e9cd3ed/virt/kvm/guest_memfd.c#L722
[5] https://lore.kernel.org/all/20260225075211.3353194-1-aik@amd.com/

I'm intentionally avoiding the "primary manager" terminology since
that's overly broad and doesn't really explain what processes exactly
are being managed. It's really just which entity tells which other
entity to zap pages, and since shared/private-ness of memory is core to
guest_memfd, I think guest_memfd is in the best position to tell all the
other entities to zap, and what (shared/private to zap).

Does your provider need to know if some memory was transitioned between
shared/private states? The provider can be notified too, if
necessary. If not, then isn't guest_memfd doing all the coordination
better than going via the provider?

Does the revocation model work for you? If it helps, I can rebuild the
sample provider to illustrate how the revocation can be done. Let me
know what else is missing from my model [1]. :)

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2807 bytes --]

> In my proposal, I agree with you completely that the provider must be
> allowed to revoke memory. Using the sample in this RFC, gmem_fd_ioctl()
> called with !GMEM_PROVIDER_SET_PRESENT is where the provider wants to
> revoke memory, and the revocation in my model would look like
>
>   [PROVIDER] -> [GUEST_MEMFD]
>
> and [GUEST_MEMFD] would then fan it out to [KVM], [IOMMU], [USERSPACE]
> and [DIRECTMAP].

(Fred is the one with his head in this space right now; I mostly wanted
to get the ball rolling from our side. So if he contradicts me, take
his word for it and don't necessarily wait for me to concede.)

Your model above works for me, I think. I just don't *like* it very
much. 

I had a nice clean (in my head) model where the memory provider is the
owner of that memory, responsible for who owns what and whether it's
read-only/shared/private/revoked, and for persisting it across live
update.

Anyone who wants a piece of it can come and ask, and it interfaced
directly to KVM, IOMMU and anything else.

And it was then theoretically usable for other things, too. Fred has
already been working on ways for a guest to donate memory to the host
for caches to support more efficient nesting, for example, and that
could have used a memory provider (which, on the back end, has had its
pages donated from the guest).

The model of pushing it all into gmemfd probably works... but kind of
makes my skin itch in ways I can't fully express.

It does absolutely need to support PFN-based operation though.

And one of the main design points is that we want to support the memory
provider being the *trusted* arbiter of who owns what. The provider
will be the part that gets the most careful review, that some people
will be suggesting that we write in Rust because it magically fixes all
bugs. Where in some platforms we have EL2 literally *enforcing* the
fact that a rogue Linux kernel *cannot* access the wrong guests'
memory, the model we want to support here is the next best thing, where
a small and trusted piece of code provides and revokes those PFNs.

As long as that's preserved, I guess it's probably OK to push more of
the coordination into gmemfd than I had originally imagined.

> Does your provider need to know if some memory was transitioned between
> shared/private states? The provider can be notified too, if
> necessary. If not, then isn't guest_memfd doing all the coordination
> better than going via the provider?

I think it at least needs to know, doesn't it? If a guest page is being
taken away (donated to another guest, or something like Fred's nested
cache), then we do kind of have opinions on whether it's back in
'shared' mode or whether it's still allegedly private to the guest
which claimed to donate it.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

Ackerley Tng <ackerleytng@google.com> writes:

> When the provider wants to revoke the page, the notifications/function
> calls flow this way, the last [ENTITY] in the chain actually does the
> zapping.
>
> + [PROVIDER] -> [KVM]
> + [PROVIDER] -> [DMABUF] -> [IOMMU]
> + [PROVIDER] -> [GUEST_MEMFD] -> [USERSPACE] (AFAICT this was not
>   implemented in the RFC.)

Thanks for going through it. I have just posted v2:

  https://lore.kernel.org/all/20261014093733.90312-1-griffoul@gmail.com/

It changes two of these chains. dma-buf is gone: iommufd attaches to
the provider file itself, through a small interface in mm. And
guest_memfd now maps the file into userspace from the provider's
frames, so a revoke also zaps the VMM's mapping:

  [PROVIDER] -> [GUEST_MEMFD] -> [KVM], [USERSPACE]
  [PROVIDER] -> [IOMMU]

> IIUC guest_memfd conversions was not illustrated in the RFC: what
> happens on a shared -> private conversion?

There are none yet. Only VMs of type KVM_X86_DEFAULT_VM can use a
provider, so all their memory is shared.

> Does your provider need to know if some memory was transitioned between
> shared/private states? The provider can be notified too, if
> necessary. If not, then isn't guest_memfd doing all the coordination
> better than going via the provider?

Our provider does not need to know, and I agree: once there is private
memory, guest_memfd converts the pages, so it should tell the others
what to zap.

I think the two models meet there, and the v2 cover letter now says so.
The iommufd side of v2 does not pin and keeps no references: it asks for
the frame behind each page, treats a page with no frame as a hole, and
maps again on a revoke. guest_memfd can offer that for its own file:
shared pages return their frames, private pages are holes, and
guest_memfd revokes a range whenever it converts it, in either
direction. That is your [GUEST_MEMFD] -> [IOMMU], and it should work for
folio-backed guest_memfd without memfd_pin_folios() or refcounts, as
long as guest_memfd also revokes a range before it frees its folios.

The one difference is that v2 also lets iommufd map a provider file
directly. Going through guest_memfd would add nothing without private
memory: it would only pass each revoke on. And memory that only devices
use, such as MMIO, does not go through guest_memfd at all. Both paths
would use the same iommufd code.

> Does the revocation model work for you? If it helps, I can rebuild the
> sample provider to illustrate how the revocation can be done.

Yes, the revocation model works for us. v2 replaces David's sample
with one that moves, donates and makes pages read-only, and a selftest
that checks the guest, the device and the VMM's mapping after each
change. It may be a good base if you want to try guest_memfd as the
provider for iommufd on top of it.

> 4. [DIRECTMAP] Kernel direct map.

Agreed, that is separate. The memory we have in mind is a range hidden
from the host with memmap=, so it is not in the direct map.

Thanks,
Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Ackerley Tng]

Fred Griffoul <griffoul@gmail.com> writes:

> Ackerley Tng <ackerleytng@google.com> writes:
>
>> When the provider wants to revoke the page, the notifications/function
>> calls flow this way, the last [ENTITY] in the chain actually does the
>> zapping.
>>
>> + [PROVIDER] -> [KVM]
>> + [PROVIDER] -> [DMABUF] -> [IOMMU]
>> + [PROVIDER] -> [GUEST_MEMFD] -> [USERSPACE] (AFAICT this was not
>>   implemented in the RFC.)
>
> Thanks for going through it. I have just posted v2:
>
>   https://lore.kernel.org/all/20261014093733.90312-1-griffoul@gmail.com/
>
> It changes two of these chains. dma-buf is gone: iommufd attaches to
> the provider file itself, through a small interface in mm. And
> guest_memfd now maps the file into userspace from the provider's
> frames, so a revoke also zaps the VMM's mapping:
>
>   [PROVIDER] -> [GUEST_MEMFD] -> [KVM], [USERSPACE]
>   [PROVIDER] -> [IOMMU]
>

Thanks :)

>> IIUC guest_memfd conversions was not illustrated in the RFC: what
>> happens on a shared -> private conversion?
>
> There are none yet. Only VMs of type KVM_X86_DEFAULT_VM can use a
> provider, so all their memory is shared.
>

To borrow David's words, the part about this that makes me itch is
conversion. We have to support conversions. Even if it doesn't merge in
the first patch series, we have to have line of sight to how conversions
would work.

Conversion requests originate from the guest and then the host handles
the guest request by making calling an ioctl on guest_memfd. Hopefully
this part doesn't change with alternative provider support.

Suppose guest_memfd still gets the conversion request first, in RFCv2,
to reach all the mappings in various page tables, it would have to be

+ [GUEST_MEMFD] -> [KVM], [USERSPACE]
+ [GUEST_MEMFD] -> [PROVIDER] -> [IOMMU]

since guest_memfd must ensure no IOMMU mappings to guest private pages?
There's that extra step through the [PROVIDER]...

>> Does your provider need to know if some memory was transitioned between
>> shared/private states? The provider can be notified too, if
>> necessary. If not, then isn't guest_memfd doing all the coordination
>> better than going via the provider?
>
> Our provider does not need to know, and I agree: once there is private
> memory, guest_memfd converts the pages, so it should tell the others
> what to zap.
>
> I think the two models meet there, and the v2 cover letter now says so.
> The iommufd side of v2 does not pin and keeps no references: it asks for
> the frame behind each page, treats a page with no frame as a hole, and
> maps again on a revoke. guest_memfd can offer that for its own file:
> shared pages return their frames, private pages are holes, and
> guest_memfd revokes a range whenever it converts it, in either
> direction. That is your [GUEST_MEMFD] -> [IOMMU], and it should work for
> folio-backed guest_memfd without memfd_pin_folios() or refcounts, as
> long as guest_memfd also revokes a range before it frees its folios.
>

This part makes sense, if I'm understanding you correctly.

> The one difference is that v2 also lets iommufd map a provider file
> directly. Going through guest_memfd would add nothing without private
> memory: it would only pass each revoke on. And memory that only devices
> use, such as MMIO, does not go through guest_memfd at all. Both paths
> would use the same iommufd code.
>
>> Does the revocation model work for you? If it helps, I can rebuild the
>> sample provider to illustrate how the revocation can be done.
>
> Yes, the revocation model works for us. v2 replaces David's sample
> with one that moves, donates and makes pages read-only, and a selftest
> that checks the guest, the device and the VMM's mapping after each
> change. It may be a good base if you want to try guest_memfd as the
> provider for iommufd on top of it.
>

Thanks, I will prototype what I have in mind too to make sure things
work as I think they will :)

>> 4. [DIRECTMAP] Kernel direct map.
>
> Agreed, that is separate. The memory we have in mind is a range hidden
> from the host with memmap=, so it is not in the direct map.
>
> Thanks,
> Fred

Re: [RFC PATCH 0/9] mm: Memory providers for guest_memfd and iommufd

[Fred Griffoul]

Ackerley Tng <ackerleytng@google.com> writes:

> To borrow David's words, the part about this that makes me itch is
> conversion. We have to support conversions. Even if it doesn't merge in
> the first patch series, we have to have line of sight to how conversions
> would work.
>
> Conversion requests originate from the guest and then the host handles
> the guest request by making calling an ioctl on guest_memfd. Hopefully
> this part doesn't change with alternative provider support.

Agreed, and it doesn't change. Conversions still go through
guest_memfd. It is the only one that knows which pages are private.

> Suppose guest_memfd still gets the conversion request first, in RFCv2,
> to reach all the mappings in various page tables, it would have to be
>
> + [GUEST_MEMFD] -> [KVM], [USERSPACE]
> + [GUEST_MEMFD] -> [PROVIDER] -> [IOMMU]
>
> since guest_memfd must ensure no IOMMU mappings to guest private pages?
> There's that extra step through the [PROVIDER]...

Right, there is a step through [PROVIDER]. But it can be a step through
the mem_provider code in mm, which every provider shares, rather than
through each provider driver. Then guest_memfd doesn't need to know who
else maps the frames, and providers don't have to get it right one by
one.

Here is the idea, on top of your in-place conversion. The VMM gives the
same provider file to guest_memfd and to iommufd, as in v2. When
guest_memfd converts a range to private, it claims the range:

  int mem_provider_claim(struct mem_provider_attachment *att,
                         loff_t offset, loff_t len);
  void mem_provider_unclaim(struct mem_provider_attachment *att,
                            loff_t offset, loff_t len);

The claim revokes the range for every other consumer of the file. After
that, get_page() returns a hole to them. So iommufd unmaps the pages
and can't map them again, and nothing changes on the iommufd side.
guest_memfd zaps its own mappings, as it does today.

Going back to shared is the reverse. guest_memfd returns the frames to
the host first, then unclaims the range. The other consumers then map
the frames again. So a conversion looks like:

  [GUEST_MEMFD] -> [KVM], [USERSPACE]
  [GUEST_MEMFD] -> [PROVIDER] -> [IOMMU]

That is your second chain, but the [PROVIDER] step is the common code
in mm/mem_provider.c. It does the revokes and hands out the holes. The
provider driver itself only gets one callback, and it can refuse a
claim. For example, it would refuse if it also lends those frames
through another file. Conversions can already fail, so this fits. If a
later step fails, guest_memfd just unclaims the range.

This also answers your earlier question. The provider doesn't need to
know what is shared or private. It only needs to know which ranges are
claimed, so that it doesn't move or share them, and only takes them
back once the guest has given them up.

I haven't implemented this yet. My plan is to rebase on kvm-x86 next,
which has your conversion code. Then I'd demo it with a
KVM_X86_SW_PROTECTED_VM and the sample provider. A device sees a shared
page, loses it when the page becomes private, and sees the same frame
again when it goes back to shared.

Does that match what you have in mind?

> Thanks, I will prototype what I have in mind too to make sure things
> work as I think they will :)

Thanks, that would really help. The sample provider and the KVM
selftest in the series could be a starting point.

Thanks,
Fred