[PATCH RFC v3 00/88] guest_memfd alternative providers

[PATCH RFC v3 00/88] guest_memfd alternative providers

[Ackerley Tng via B4 Relay]

Here's RFC v3 of guest_memfd alternative providers.

This RFC is meant to show line of sight to a few things, and will have to be
split into a few series for upstreaming:

1. Supporting alternative folio providers (tmpfs used for illustration)
     ends at "KVM: selftests: Test guest_memfd conversions with tmpfs backing"
2. Introducing a sample memory provider
     ends at "KVM: selftests: Test marking page ranges read-only"
3. Supporting pfn-mode providers in KVM
     ends at "KVM: selftests: Test flag dependency of PFN_MODE on USE_RESOURCE"
4. Making sample memory provider a guest_memfd pfn-mode provider
     ends at "KVM: selftests: Test guest_memfd conversions with sample memory provider backing"
5. Making VFIO a guest_memfd pfn-mode provider
     ends at the last patch

I was trying to get this to work at a conceptual level:

In this version, after working with VFIO, the provider fd should be clarified as
the provider control/config plane fd.

* tmpfs: the directory fd is used as the control plane, since it indicates which
  mount to get folios from, what the mount config is, etc
* sample memory provider: the provider file fd doesn't allow access to memory
  (data plane), but is a handle for memory that is pre-carved out using
  alloc_contig_pages()
* VFIO: I added a VFIO_DEVICE_BIND_FLAG_CTRL_ONLY flag for
  VFIO_DEVICE_BIND_IOMMUFD, which partitions VFIO operations into data plane
  (mmap_supported + some ioctls) and control plane (everything else, which I
  think shouldn't be used for Confidential IO, and gmem doesn't want to manage
  anyway)

The split between vfio fd and gmem fd is this:

* gmem fd: Read/write/mmap mappable BARs 0-5 from userspace and stage 2, and
           conversions
* vfio fd: Read/write everything else. mmap, ioeventfd, and dmabufs are blocked
           for CTRL_ONLY

I also formalized Sean's suggestion of gmem requiring exclusive access to the
resource, which shows up in the locking requirements (I updated the docstrings
for all the gmem provider ops accordingly).

For VFIO, this is implemented as a .bars_available field. On attach, we take
dev_set->lock in vfio to store .gmem_inode, while vfio-pci takes ->memory_lock
and initializes .bars_available based on whether memory decoding is enabled and
the device is awake.

After setting .bars_available = true, gmem has exclusive access to the data
plane for VFIO, which should be everything needed for Confidential IO. gmem will
use its filemap_invalidate_lock for VFIO operations. Everything on the VFIO side
is locklessly checked.

Revoking from the VFIO side first sets .bars_available = false to block racing
accesses from gmem, then calls kvm_gmem_revoke().

For testing VFIO, I selected igb as the test driver since it is the most
full-featured yet can work with my qemu testing setup :). mdpy and mtty don't
use vfio-pci so they don't use ->memory_lock.

And then when I introduced the last test on read/write I realized this
implementation has a slightly different behavior between vfio and gmem-wrapped
vfio, which is that the reads/writes to the gmem-wrapped version doesn't
automatically get the device out of low power mode. I don't like this slight
difference required.

It doesn't get the device out of low power mode because getting the device out
of low power mode involves taking vdev->memory_lock, and this model commits to
not taking the lock while holding the filemap_invalidate_lock() in gmem.

The problem here is that read/write needs to take a provider side lock, but
guest_memfd, the caller, needs to hold the filemap_invalidate_lock() to make
sure shared/private state doesn't change during the read. Holding
filemap_invalidate_lock() while taking the provider side lock
(vdev->memory_lock) causes an ABBA locking issue. Not taking the provider lock
is possible but results in slightly different functionality.

Aneesh's version [1] didn't implement read/write. I think David mentioned during
gmem biweekly it shouldn't be that hard, so I tried and that turned out to be
_the_ challenge, lol. From a gmem provider perspective, the goal is to try and
get line of sight to a broad set of providers, hence VFIO's read/write is a good
challenge. Either way, I do think we should try to preserve as much vfio
functionality even after wrapping with gmem.

Aneesh, I think I'm finally at a point where I have enough background
understanding to compare our approaches, so I'll get to that after KVM forum.

I didn't get around to prototyping the Confidential stuff but I still want to
post this because our proposals differ very much in the exclusive access and
locking setup, and I'd like to get your help to also see if the
Confidential-specific parts of gmem+VFIO works, and if there are other parts we
can converge on. I left a patch prefixed QUESTION for the CoCo community.

At a higher level, I'm really trying to make VFIO not that different a provider
from other PFN providers.

Sean/Aneesh, it turns out that ARM relies on the VMA to determine cacheability
in stage-2 page tables. Since guest_memfd memory may not have a VMA, reporting
cacheability somehow from provider (VFIO) via guest_memfd to KVM makes sense
imo. In "KVM: guest_memfd: Support reporting PFN cacheability", for x86, I just
left a WARN-ing if there's a discrepancy to be resolved.

Fred, these are the other changes more relevant for the regular PFN provider:

* Cleaned up the sample memory provider (formerly MPS) implementation and added
  selftests.
* Fixed bug in the RFCv2, MPS' .lookup_pfn_range() implementation always
  returned nr_pages = 1, but it can return much more than nr_pages = 1. The
  intent was to have nr_pages be as large as possible to indicate the full
  contiguous block beginning at the returned PFN.
* .lookup_pfn_range() will now set *nr_pages on -ENOENT to indicate the size of
  the hole. This helps with VFIO since the holes are huge and iterating would be
  sad.
* Added the readonly support within KVM ("KVM: guest_memfd: Support reporting
  PFN attributes")
* Didn't get around to verifying proper support for more than order 0 PFN ranges

This was built on kvm/next, and the series is also available here for your
convenience:

https://github.com/googleprodkernel/linux-cc/tree/gmem-alternative-providers-rfc-v3

Testing: these pass:

+ ./tools/testing/selftests/kvm/mem_provider_sample_test
+ ./tools/testing/selftests/kvm/guest_memfd_test
+ ./tools/testing/selftests/kvm/x86/guest_memfd_conversions_test
+ ./tools/testing/selftests/kvm/x86/guest_memfd_vfio_test

RFC v1: https://lore.kernel.org/all/20260925-gmem-tmpfs-backend-v1-0-d36159822d18@google.com/
RFC v2: https://lore.kernel.org/all/20261022-gmem-alternative-providers-v2-0-5dd1dc6c660a@google.com/

[1] https://lore.kernel.org/all/20261104150322.3516320-1-aneesh.kumar@kernel.org/

---
Changes in v3:
- EDITME: describe what is new in this series revision.
- EDITME: use bulletpoints and terse descriptions.
- Link to v2: https://patch.msgid.link/20261022-gmem-alternative-providers-v2-0-5dd1dc6c660a@google.com

---
Ackerley Tng (82):
      KVM: guest_memfd: Split folio allocation and insertion in kvm_gmem_get_folio
      KVM: guest_memfd: Route folio allocation through native provider operations
      mm: shmem: Implement guest_memfd provider operations for tmpfs
      KVM: guest_memfd: Support provider folio invalidation
      KVM: guest_memfd: Support using resource provider
      KVM: selftests: Add helper to create guest_memfd with a resource file
      KVM: selftests: Test negative validation of resource_fd argument
      KVM: selftests: Test rejection of unsupported filesystem for resource_fd
      KVM: selftests: Test rejection of memfd for resource_fd
      KVM: selftests: Test rejection of swap tmpfs mounts
      KVM: selftests: Test rejection of hugepage tmpfs mounts
      KVM: selftests: Test rejection of regular file in tmpfs mount
      KVM: selftests: Test guest_memfd with tmpfs resource pool
      KVM: selftests: Test guest_memfd resource pool sharing across instances
      KVM: selftests: Test memory allocation against shared tmpfs resource pool
      KVM: selftests: Test that shared tmpfs resource pool size limit is respected
      KVM: selftests: Test guest execution with tmpfs-backed guest_memfd
      KVM: selftests: Document testing TODOs
      KVM: selftests: Test guest_memfd conversions with tmpfs backing
      KVM: selftests: Add basic test for sample memory provider
      KVM: selftests: Test marking page ranges present and absent
      KVM: selftests: Test marking page ranges read-only
      KVM: guest_memfd: Support getting PFNs for stage 2 from providers
      KVM: guest_memfd: Refactor error handling in kvm_gmem_fault_user_mapping
      KVM: guest_memfd: Set VM_PFNMAP on VMA in PFN mode
      KVM: guest_memfd: Support user fault handling in PFN mode
      KVM: guest_memfd: Skip checking outstanding references in PFN mode
      KVM: guest_memfd: Support converting to shared in PFN mode
      KVM: guest_memfd: Support preallocating memory via fallocate in PFN mode
      KVM: guest_memfd: Clamp hole punch range to inode size
      KVM: guest_memfd: Support reclaim during hole punch in PFN mode
      KVM: guest_memfd: Support arch reclaim on inode destruction in PFN mode
      KVM: guest_memfd: Support put_pfn_range provider op in PFN mode
      KVM: guest_memfd: Add TODO for memory failure handling in PFN mode
      KVM: guest_memfd: Allow attaching PFN-backed providers
      KVM: guest_memfd: Export kvm_gmem_revoke for provider-driven revocation
      KVM: selftests: Test flag dependency of PFN_MODE on USE_RESOURCE
      samples/kvm: Implement attach and release provider ops in sample provider
      samples/kvm: Disallow MP_SET_READONLY after attach
      samples/kvm: Implement get_pfn provider operation
      samples/kvm: Implement put_pfn_range provider operation
      samples/kvm: Support dynamic revocation via kvm_gmem_revoke
      samples/kvm: Implement lookup_pfn_range provider operation
      KVM: selftests: Test guest_memfd conversions with sample memory provider backing
      KVM: guest_memfd: Support reporting PFN cacheability
      KVM: guest_memfd: Support mmap provider operation
      KVM: guest_memfd: Support check_set_attributes provider operation
      KVM: guest_memfd: Support read and write operations and provider callbacks
      vfio: Track device file access mode with an enum
      vfio: Support VFIO_DEVICE_BIND_FLAG_CTRL_ONLY
      vfio: Dispatch guest_memfd provider operations via vfio_device_ops
      vfio/pci: Implement attach and release provider ops for guest_memfd
      vfio: Dispatch guest_memfd get_pfn provider op via vfio_device_ops
      vfio/pci: Implement get_pfn provider op for guest_memfd
      vfio: Implement mmap provider op for guest_memfd
      vfio: Dispatch guest_memfd check_set_attributes via vfio_device_ops
      vfio/pci: Implement check_set_attributes provider op for guest_memfd
      vfio: Add vfio_device_gmem_revoke helper for guest_memfd
      vfio/pci: Revoke guest_memfd memory on BAR invalidation and reset
      vfio: Dispatch guest_memfd lookup_pfn_range via vfio_device_ops
      vfio/pci: Implement lookup_pfn_range provider op for guest_memfd
      vfio/pci: Use iov_iter for MMIO read and write
      vfio: Dispatch guest_memfd read and write provider ops via vfio_device_ops
      vfio/pci: Introduce vfio_pci_bar_rw_kbuf for kernel buffer MMIO access
      vfio/pci: Implement read and write provider ops for guest_memfd
      KVM: selftests: Add initial VFIO test verifying device info
      KVM: selftests: Verify VFIO region info for igb device
      KVM: selftests: Verify VFIO IRQ info for igb device
      KVM: selftests: Verify VFIO MMIO read and write for igb device
      KVM: selftests: Verify VFIO MMIO mmap access for igb device
      KVM: selftests: Verify VFIO device reset for igb device
      KVM: selftests: Verify VFIO access directly and via guest_memfd
      KVM: selftests: Verify BAR revocation and access gating in low power state
      KVM: selftests: Verify host VMA flags for VFIO BAR
      KVM: selftests: Verify guest mapping for VFIO BAR
      KVM: selftests: Verify VFIO dmabuf creation for igb device
      KVM: selftests: Verify VFIO ioeventfd creation for igb device
      KVM: selftests: Verify memory conversions for VFIO BARs
      KVM: selftests: Verify shared and private mmap access for VFIO BARs
      KVM: selftests: Expect successful conversion on MSI-X BARs in VFIO test
      QUESTION: KVM: guest_memfd: Disallow private mappings of shared-only PFNs
      KVM: selftests: Verify shared and private pread and pwrite for VFIO BARs

Fred Griffoul (6):
      samples/kvm: Introduce memory provider module skeleton
      samples/kvm: Support creating memory provider files
      samples/kvm: Support marking page ranges present or absent
      samples/kvm: Support marking page ranges read-only
      samples/kvm: Support querying provider file statistics
      KVM: guest_memfd: Support reporting PFN attributes

 Documentation/virt/kvm/api.rst                     |   34 +-
 arch/arm64/kvm/mmu.c                               |   43 +-
 arch/arm64/kvm/nested.c                            |   19 +-
 arch/x86/kvm/Makefile                              |    3 +-
 arch/x86/kvm/mmu/mmu.c                             |   18 +-
 arch/x86/kvm/mmu/spte.c                            |    2 +-
 arch/x86/kvm/mmu/spte.h                            |    1 +
 arch/x86/kvm/svm/sev.c                             |   29 +-
 drivers/vfio/device_cdev.c                         |   15 +-
 drivers/vfio/group.c                               |    2 +-
 drivers/vfio/pci/vfio_pci.c                        |    9 +
 drivers/vfio/pci/vfio_pci_config.c                 |   16 +-
 drivers/vfio/pci/vfio_pci_core.c                   |  211 +++-
 drivers/vfio/pci/vfio_pci_priv.h                   |   13 +
 drivers/vfio/pci/vfio_pci_rdwr.c                   |  145 ++-
 drivers/vfio/vfio.h                                |    8 +-
 drivers/vfio/vfio_main.c                           |  273 ++++-
 fs/libfs.c                                         |    4 +-
 include/linux/fs.h                                 |    6 +
 include/linux/guest_memfd.h                        |  125 +++
 include/linux/kvm_host.h                           |   10 +-
 include/linux/vfio.h                               |   30 +
 include/linux/vfio_pci_core.h                      |   39 +
 include/uapi/linux/kvm.h                           |    6 +-
 include/uapi/linux/vfio.h                          |    7 +
 mm/shmem.c                                         |  126 +++
 samples/Kconfig                                    |   11 +
 samples/Makefile                                   |    1 +
 samples/kvm/Makefile                               |    2 +
 samples/kvm/memory_provider.c                      |  405 ++++++++
 samples/kvm/memory_provider.h                      |   70 ++
 tools/include/uapi/linux/kvm.h                     |    6 +-
 tools/testing/selftests/kvm/Makefile.kvm           |    2 +
 tools/testing/selftests/kvm/guest_memfd_test.c     |  277 ++++-
 tools/testing/selftests/kvm/include/kvm_util.h     |   24 +-
 tools/testing/selftests/kvm/include/test_util.h    |    2 +
 tools/testing/selftests/kvm/lib/kvm_util.c         |    9 +-
 tools/testing/selftests/kvm/lib/test_util.c        |   56 +
 .../selftests/kvm/mem_provider_sample_test.c       |  215 ++++
 .../kvm/x86/guest_memfd_conversions_test.c         |   91 +-
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 1092 ++++++++++++++++++++
 virt/kvm/guest_memfd.c                             |  658 +++++++++++-
 42 files changed, 3956 insertions(+), 159 deletions(-)
---
base-commit: 1689c80fca308978ea1c9f03d15a500f4e44e2e1
change-id: 20261015-gmem-alternative-providers-5d98d72efabd

Best regards,
--  
Ackerley Tng <ackerleytng@google.com>

[PATCH RFC v3 01/88] KVM: guest_memfd: Split folio allocation and insertion in kvm_gmem_get_folio

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Separate __filemap_get_folio_mpol() call in kvm_gmem_get_folio() into two
distinct steps:

1. Allocating a folio via filemap_alloc_folio() with the NUMA policy.
2. Inserting the allocated folio into the inode filemap.

Handle -EEXIST from filemap_add_folio() by retrying filemap_lock_folio()
safely to manage concurrent insertions.

Decouple folio allocation from filemap insertion to prepare for supporting
other memory providers.

No functional change intended.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 93f3d29d5751a..f865fa0e46840 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -131,7 +131,9 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
 	/* TODO: Support huge pages. */
 	struct mempolicy *policy;
 	struct folio *folio;
+	int r;
 
+retry:
 	/*
 	 * Fast-path: See if folio is already present in mapping to avoid
 	 * policy_lookup.
@@ -141,10 +143,19 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
 		return folio;
 
 	policy = mpol_shared_policy_lookup(&GMEM_I(inode)->policy, index);
-	folio = __filemap_get_folio_mpol(inode->i_mapping, index,
-					 FGP_LOCK | FGP_CREAT,
-					 mapping_gfp_mask(inode->i_mapping), policy);
+	folio = filemap_alloc_folio(mapping_gfp_mask(inode->i_mapping), 0,
+				    policy);
 	mpol_cond_put(policy);
+	if (!folio)
+		return ERR_PTR(-ENOMEM);
+
+	r = filemap_add_folio(inode->i_mapping, folio, index, GFP_KERNEL);
+	if (r) {
+		folio_put(folio);
+		if (r == -EEXIST)
+			goto retry;
+		return ERR_PTR(r);
+	}
 
 	/*
 	 * External interfaces like kvm_gmem_get_pfn() support dealing
@@ -153,7 +164,7 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
 	 * to be updated for anything otherwise (e.g. page-clearing
 	 * operations).
 	 */
-	WARN_ON_ONCE(!IS_ERR(folio) && folio_order(folio));
+	WARN_ON_ONCE(folio_order(folio));
 
 	return folio;
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 02/88] KVM: guest_memfd: Route folio allocation through native provider operations

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Conceptually refactor the buddy allocator providing guest_memfd with
order-0 folios as the native provider.

Define operations with an alloc_folio callback for the native provider,
which still uses filemap_alloc_folio to allocate folios as before. Also
define the attach callback to establish provider context. Route folio
allocation in the get-folio path through provider ops, and set up the
native provider during creation using a sentinel fd and attach.

Attach the provider only after the entire inode and pseudo-file have been
initialized. Exposing the inode to external providers before allocating the
file or populating the file list risks race conditions and use-after-free
bugs if an external provider immediately triggers operations on the
guest_memfd inode.

Now that the inode is exposed for other providers, add a write barrier to
commit everything in the inode before the provider (and other racing
provider threads) gets to use the gmem inode.

Expose provider operations so external providers across the kernel can
implement them.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h | 28 +++++++++++++++
 virt/kvm/guest_memfd.c      | 86 ++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 110 insertions(+), 4 deletions(-)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
new file mode 100644
index 0000000000000..218de509769c1
--- /dev/null
+++ b/include/linux/guest_memfd.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LINUX_GUEST_MEMFD_H
+#define _LINUX_GUEST_MEMFD_H
+
+#include <linux/types.h>
+
+struct file;
+struct folio;
+struct inode;
+struct mempolicy;
+
+/**
+ * struct guest_memfd_provider_operations - Operations for external memory providers
+ * @attach: Attach to a resource file, perform filesystem-specific validation,
+ *          and return an opaque provider context.
+ * @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
+ *               Must return a pointer to a struct folio on success, or an
+ *               ERR_PTR() error code on failure. Must never return NULL.
+ *
+ * Used by filesystems and device drivers that provide memory for guest_memfd.
+ */
+struct guest_memfd_provider_operations {
+	void *(*attach)(struct inode *gmem_inode, struct file *resource_file);
+	struct folio *(*alloc_folio)(void *provider, pgoff_t index,
+				     struct mempolicy *mpol);
+};
+
+#endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index f865fa0e46840..545c3e646e9c1 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -3,6 +3,7 @@
 #include <linux/backing-dev.h>
 #include <linux/falloc.h>
 #include <linux/fs.h>
+#include <linux/guest_memfd.h>
 #include <linux/kvm_host.h>
 #include <linux/maple_tree.h>
 #include <linux/mempolicy.h>
@@ -35,6 +36,9 @@ struct gmem_inode {
 	struct inode vfs_inode;
 	struct list_head gmem_file_list;
 
+	void *provider;
+	const struct guest_memfd_provider_operations *provider_ops;
+
 	u64 flags;
 	/*
 	 * Every index in this inode, whether memory is populated or
@@ -50,6 +54,65 @@ static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
 	return container_of(inode, struct gmem_inode, vfs_inode);
 }
 
+static void *gmem_native_attach(struct inode *gmem_inode,
+				struct file *resource_file)
+{
+	return GMEM_I(gmem_inode);
+}
+
+static struct folio *gmem_native_alloc_folio(void *provider, pgoff_t index,
+					     struct mempolicy *mpol)
+{
+	struct gmem_inode *gi = provider;
+	struct folio *folio;
+	gfp_t gfp;
+
+	gfp = mapping_gfp_mask(gi->vfs_inode.i_mapping);
+	folio = filemap_alloc_folio(gfp, 0, mpol);
+	if (!folio)
+		return ERR_PTR(-ENOMEM);
+
+	return folio;
+}
+
+static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
+	.attach = gmem_native_attach,
+	.alloc_folio = gmem_native_alloc_folio,
+};
+
+#define GMEM_RESOURCE_FD_NATIVE (-1)
+
+static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
+{
+	const struct guest_memfd_provider_operations *ops = NULL;
+	struct file *resource_file = NULL;
+	void *provider;
+
+	if (resource_fd == GMEM_RESOURCE_FD_NATIVE)
+		ops = &gmem_native_provider_ops;
+	else
+		return -EOPNOTSUPP;
+
+	provider = ops->attach(inode, resource_file);
+	if (IS_ERR(provider))
+		return PTR_ERR(provider);
+
+	GMEM_I(inode)->provider = provider;
+	GMEM_I(inode)->provider_ops = ops;
+
+	return 0;
+}
+
+static struct folio *gmem_provider_alloc_folio(struct gmem_inode *gi,
+					       pgoff_t index,
+					       struct mempolicy *mpol)
+{
+	if (gi->provider_ops && gi->provider_ops->alloc_folio)
+		return gi->provider_ops->alloc_folio(gi->provider, index, mpol);
+
+	return ERR_PTR(-EOPNOTSUPP);
+}
+
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -143,11 +206,10 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
 		return folio;
 
 	policy = mpol_shared_policy_lookup(&GMEM_I(inode)->policy, index);
-	folio = filemap_alloc_folio(mapping_gfp_mask(inode->i_mapping), 0,
-				    policy);
+	folio = gmem_provider_alloc_folio(GMEM_I(inode), index, policy);
 	mpol_cond_put(policy);
-	if (!folio)
-		return ERR_PTR(-ENOMEM);
+	if (IS_ERR(folio))
+		return folio;
 
 	r = filemap_add_folio(inode->i_mapping, folio, index, GFP_KERNEL);
 	if (r) {
@@ -996,6 +1058,20 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 	xa_init(&f->bindings);
 	list_add(&f->entry, &GMEM_I(inode)->gmem_file_list);
 
+	/*
+	 * Ensure all inode and file initialization (such as gmem_file_list and
+	 * bindings) is globally visible before exposing the inode to the
+	 * provider in ->attach().
+	 */
+	smp_wmb();
+
+	err = kvm_gmem_attach_resource(inode, GMEM_RESOURCE_FD_NATIVE);
+	if (err) {
+		fput(file);
+		put_unused_fd(fd);
+		return err;
+	}
+
 	fd_install(fd, file);
 	return fd;
 
@@ -1355,6 +1431,8 @@ static struct inode *kvm_gmem_alloc_inode(struct super_block *sb)
 	mt_init_flags(&gi->attributes, MT_FLAGS_LOCK_EXTERN | MT_FLAGS_USE_RCU);
 
 	gi->flags = 0;
+	gi->provider = NULL;
+	gi->provider_ops = NULL;
 	INIT_LIST_HEAD(&gi->gmem_file_list);
 	return &gi->vfs_inode;
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 03/88] mm: shmem: Implement guest_memfd provider operations for tmpfs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement guest_memfd provider operations to declare that tmpfs supports
providing memory to guest_memfd.

I put guest_memfd_provider_operations as a pointer in struct
file_operations. Being in file_operations generalizes well to other kinds
of providers, like devices, which don't have their own super_operations.

Setting it in shmem_dir_operations allows directory file descriptors
opened on a tmpfs mount to provide memory to guest_memfd.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 fs/libfs.c                  |   4 +-
 include/linux/fs.h          |   6 +++
 include/linux/guest_memfd.h |   4 ++
 mm/shmem.c                  | 126 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 138 insertions(+), 2 deletions(-)

diff --git a/fs/libfs.c b/fs/libfs.c
index 27d7dc16fcb0a..77d3a4fcb5c13 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -429,7 +429,7 @@ void simple_offset_destroy(struct offset_ctx *octx)
  * Returns the updated read position if successful; otherwise a
  * negative errno is returned and the read position remains unchanged.
  */
-static loff_t offset_dir_llseek(struct file *file, loff_t offset, int whence)
+loff_t offset_dir_llseek(struct file *file, loff_t offset, int whence)
 {
 	switch (whence) {
 	case SEEK_CUR:
@@ -551,7 +551,7 @@ static void offset_iterate_dir(struct file *file, struct dir_context *ctx)
  * Return values:
  *   %0 - Complete
  */
-static int offset_readdir(struct file *file, struct dir_context *ctx)
+int offset_readdir(struct file *file, struct dir_context *ctx)
 {
 	struct dentry *dir = file->f_path.dentry;
 
diff --git a/include/linux/fs.h b/include/linux/fs.h
index deb411e866617..4f1494c4fbfde 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1915,6 +1915,7 @@ struct dir_context {
 
 struct io_uring_cmd;
 struct offset_ctx;
+struct guest_memfd_provider_operations;
 
 struct file_operations {
 	struct module *owner;
@@ -1960,6 +1961,9 @@ struct file_operations {
 	int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *,
 				unsigned int poll_flags);
 	int (*mmap_prepare)(struct vm_area_desc *);
+#ifdef CONFIG_KVM_GUEST_MEMFD
+	const struct guest_memfd_provider_operations *gmem_provider_ops;
+#endif
 } __randomize_layout;
 
 /* Supports async buffered reads */
@@ -3337,6 +3341,8 @@ int simple_offset_rename_exchange(struct inode *old_dir,
 				  struct dentry *new_dentry);
 void simple_offset_destroy(struct offset_ctx *octx);
 
+loff_t offset_dir_llseek(struct file *file, loff_t offset, int whence);
+int offset_readdir(struct file *file, struct dir_context *ctx);
 extern const struct file_operations simple_offset_dir_operations;
 
 extern int simple_fsync_noflush(struct file *, loff_t, loff_t, int);
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 218de509769c1..f3e6077c39ac3 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -13,16 +13,20 @@ struct mempolicy;
  * struct guest_memfd_provider_operations - Operations for external memory providers
  * @attach: Attach to a resource file, perform filesystem-specific validation,
  *          and return an opaque provider context.
+ * @release: Release provider context and unpin any resources.
  * @alloc_folio: Allocate an uninserted folio for a given index and NUMA policy.
  *               Must return a pointer to a struct folio on success, or an
  *               ERR_PTR() error code on failure. Must never return NULL.
+ * @invalidate_folio: Notify provider that a folio has been invalidated.
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
 struct guest_memfd_provider_operations {
 	void *(*attach)(struct inode *gmem_inode, struct file *resource_file);
+	void (*release)(void *provider);
 	struct folio *(*alloc_folio)(void *provider, pgoff_t index,
 				     struct mempolicy *mpol);
+	void (*invalidate_folio)(void *provider, struct folio *folio);
 };
 
 #endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/mm/shmem.c b/mm/shmem.c
index 848316eaa7f4f..2a9faf8be7382 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -38,6 +38,7 @@
 #include <linux/uio.h>
 #include <linux/hugetlb.h>
 #include <linux/fs_parser.h>
+#include <linux/guest_memfd.h>
 #include <linux/swapfile.h>
 #include <linux/iversion.h>
 #include <linux/unicode.h>
@@ -268,6 +269,9 @@ static void shmem_inode_unacct_blocks(struct inode *inode, long pages)
 static const struct super_operations shmem_ops;
 static const struct address_space_operations shmem_aops;
 static const struct file_operations shmem_file_operations;
+#ifdef CONFIG_KVM_GUEST_MEMFD
+static const struct file_operations shmem_dir_operations;
+#endif
 static const struct inode_operations shmem_inode_operations;
 static const struct inode_operations shmem_dir_inode_operations;
 static const struct inode_operations shmem_special_inode_operations;
@@ -3088,7 +3092,11 @@ static struct inode *__shmem_get_inode(struct mnt_idmap *idmap,
 		/* Some things misbehave if size == 0 on a directory */
 		inode->i_size = 2 * BOGO_DIRENT_SIZE;
 		inode->i_op = &shmem_dir_inode_operations;
+#ifdef CONFIG_KVM_GUEST_MEMFD
+		inode->i_fop = &shmem_dir_operations;
+#else
 		inode->i_fop = &simple_offset_dir_operations;
+#endif
 		simple_offset_init(shmem_get_offset_ctx(inode));
 		break;
 	case S_IFLNK:
@@ -5237,6 +5245,124 @@ static const struct inode_operations shmem_special_inode_operations = {
 #endif
 };
 
+#ifdef CONFIG_KVM_GUEST_MEMFD
+static void *shmem_gmem_attach(struct inode *gmem_inode,
+			       struct file *resource_file)
+{
+	struct inode *inode = file_inode(resource_file);
+	struct shmem_sb_info *sbinfo;
+
+	if (!S_ISDIR(inode->i_mode))
+		return ERR_PTR(-ENOTDIR);
+
+	/*
+	 * Would like comments: Requiring the root directory of the mount
+	 * provides a less confusing interface, although this is not strictly
+	 * necessary.
+	 */
+	if (resource_file->f_path.dentry != resource_file->f_path.mnt->mnt_root)
+		return ERR_PTR(-EINVAL);
+
+	if (__mnt_is_readonly(resource_file->f_path.mnt))
+		return ERR_PTR(-EROFS);
+
+	if (inode_permission(file_mnt_idmap(resource_file), inode,
+			     MAY_WRITE | MAY_EXEC))
+		return ERR_PTR(-EACCES);
+
+	/* Enforce noswap because guest_memfd does not support swapping. */
+	sbinfo = SHMEM_SB(inode->i_sb);
+	if (!sbinfo->noswap)
+		return ERR_PTR(-EINVAL);
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+	/*
+	 * guest_memfd does not support huge pages yet; this restriction can be
+	 * relaxed in the future.
+	 *
+	 * TODO: Block rebinding with huge page support.
+	 */
+	if (sbinfo->huge != SHMEM_HUGE_NEVER)
+		return ERR_PTR(-EINVAL);
+#endif
+
+	return (void *)mntget(resource_file->f_path.mnt);
+}
+
+static void shmem_gmem_release(void *provider)
+{
+	struct vfsmount *mnt = provider;
+
+	mntput(mnt);
+}
+
+/*
+ * TODO: Consider refactoring shmem allocation helpers to share code between
+ * internal tmpfs folio allocation and guest_memfd provider allocation.
+ */
+static struct folio *shmem_gmem_alloc_folio(void *provider, pgoff_t index,
+					    struct mempolicy *mpol)
+{
+	struct mempolicy *sb_mpol = NULL;
+	struct vfsmount *mnt = provider;
+	struct shmem_sb_info *sbinfo;
+	struct folio *folio;
+
+	sbinfo = SHMEM_SB(mnt->mnt_sb);
+	if (sbinfo->max_blocks &&
+	    !percpu_counter_limited_add(&sbinfo->used_blocks,
+					sbinfo->max_blocks, 1))
+		return ERR_PTR(-ENOSPC);
+
+	if (!mpol) {
+		sb_mpol = shmem_get_sbmpol(sbinfo);
+		mpol = sb_mpol;
+	}
+
+	if (mpol)
+		folio = folio_alloc_mpol(GFP_HIGHUSER_MOVABLE, 0, mpol, index,
+					 numa_node_id());
+	else
+		folio = folio_alloc(GFP_HIGHUSER_MOVABLE, 0);
+
+	mpol_cond_put(sb_mpol);
+
+	if (!folio) {
+		if (sbinfo->max_blocks)
+			percpu_counter_sub(&sbinfo->used_blocks, 1);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	return folio;
+}
+
+static void shmem_gmem_invalidate_folio(void *provider, struct folio *folio)
+{
+	struct vfsmount *mnt = provider;
+	struct shmem_sb_info *sbinfo;
+
+	sbinfo = SHMEM_SB(mnt->mnt_sb);
+	if (sbinfo->max_blocks)
+		percpu_counter_sub(&sbinfo->used_blocks, folio_nr_pages(folio));
+}
+
+static const struct guest_memfd_provider_operations shmem_gmem_provider_ops = {
+	.attach			= shmem_gmem_attach,
+	.release		= shmem_gmem_release,
+	.alloc_folio		= shmem_gmem_alloc_folio,
+	.invalidate_folio	= shmem_gmem_invalidate_folio,
+};
+
+static const struct file_operations shmem_dir_operations = {
+	.llseek		= offset_dir_llseek,
+	.iterate_shared	= offset_readdir,
+	.read		= generic_read_dir,
+	.fsync		= noop_fsync,
+	.setlease	= generic_setlease,
+	.gmem_provider_ops = &shmem_gmem_provider_ops,
+};
+#endif /* CONFIG_KVM_GUEST_MEMFD */
+
 static const struct super_operations shmem_ops = {
 	.alloc_inode	= shmem_alloc_inode,
 	.free_inode	= shmem_free_in_core_inode,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 04/88] KVM: guest_memfd: Support provider folio invalidation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

tmpfs increments the total number of used pages on the mount at allocation
time, so that number has to be decremented when the folio is removed from
tmpfs' ownership.

In this model where guest_memfd allocates from a tmpfs mount, I believe we
should respect the limits set in the mount, hence I require .alloc_folio()
to increment.

.invalidate_folio() is then the counterpart to .alloc_folio(), for undoing
any provider-specific per-folio work during allocations.

.invalidate_folio() requires mapping_set_release_always(). Not sure how odd
it is to use that mapping flag.

Using .free_folio() is hard because folio->mapping is NULL by then, so I
can't reach provider information. The provider information could also be
stuffed in folio->private? What do people think of that?

Another alternative would be to use a custom truncation function in
guest_memfd, just like shmem_truncate_range() or shmem_undo_range(). The
con is that guest_memfd might at some point support more generic mm stuff
like swap/reclaim, of some form? Or maybe never? Shall we go custom now and
unify later? Or just not think too far ahead?

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 545c3e646e9c1..ee823f2ef0368 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -113,6 +113,13 @@ static struct folio *gmem_provider_alloc_folio(struct gmem_inode *gi,
 	return ERR_PTR(-EOPNOTSUPP);
 }
 
+static void gmem_provider_invalidate_folio(struct gmem_inode *gi,
+					   struct folio *folio)
+{
+	if (gi->provider_ops && gi->provider_ops->invalidate_folio)
+		gi->provider_ops->invalidate_folio(gi->provider, folio);
+}
+
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -213,6 +220,7 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
 
 	r = filemap_add_folio(inode->i_mapping, folio, index, GFP_KERNEL);
 	if (r) {
+		gmem_provider_invalidate_folio(GMEM_I(inode), folio);
 		folio_put(folio);
 		if (r == -EEXIST)
 			goto retry;
@@ -942,10 +950,24 @@ static void kvm_gmem_free_folio(struct folio *folio)
 }
 #endif
 
+static void kvm_gmem_invalidate_folio(struct folio *folio, size_t offset,
+				      size_t len)
+{
+	struct inode *inode = folio->mapping->host;
+	struct gmem_inode *gi = GMEM_I(inode);
+
+	/* guest_memfd only allows truncating full folios. */
+	if (WARN_ON_ONCE(offset != 0 || len != folio_size(folio)))
+		return;
+
+	gmem_provider_invalidate_folio(gi, folio);
+}
+
 static const struct address_space_operations kvm_gmem_aops = {
 	.dirty_folio = noop_dirty_folio,
 	.migrate_folio	= kvm_gmem_migrate_folio,
 	.error_remove_folio = kvm_gmem_error_folio,
+	.invalidate_folio = kvm_gmem_invalidate_folio,
 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
 	.free_folio = kvm_gmem_free_folio,
 #endif
@@ -984,6 +1006,7 @@ static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags)
 	 */
 	mapping_set_inaccessible(inode->i_mapping);
 	WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping));
+	mapping_set_release_always(inode->i_mapping);
 
 	gi->flags = flags;
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 05/88] KVM: guest_memfd: Support using resource provider

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

A refcount on the resource_fd is taken only to keep the reference stable
for guest_memfd to figure out the existence of a provider. The provider is
responsible for pinning anything required to keep the provider's ops and
the actual provided resource alive.

.release() is called when this guest_memfd inode is destroyed to allow the
provider to clean up.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 Documentation/virt/kvm/api.rst | 28 +++++++++++++++++----------
 include/linux/kvm_host.h       |  2 +-
 include/uapi/linux/kvm.h       |  5 ++++-
 virt/kvm/guest_memfd.c         | 44 ++++++++++++++++++++++++++++++++++--------
 4 files changed, 59 insertions(+), 20 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index e95c9cda085d9..9f5313d405d9c 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6486,7 +6486,9 @@ and cannot be resized  (guest_memfd files do however support PUNCH_HOLE).
   struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	__s32 resource_fd;
+	__u32 pad;
+	__u64 reserved[5];
   };
 
 Conceptually, the inode backing a guest_memfd file represents physical memory,
@@ -6508,15 +6510,21 @@ a single guest_memfd file, but the bound ranges must not overlap).
 The capability KVM_CAP_GUEST_MEMFD_FLAGS enumerates the `flags` that can be
 specified via KVM_CREATE_GUEST_MEMFD.  Currently defined flags:
 
-  ============================ ================================================
-  GUEST_MEMFD_FLAG_MMAP        Enable using mmap() on the guest_memfd file
-                               descriptor.
-  GUEST_MEMFD_FLAG_INIT_SHARED Make all memory in the file shared during
-                               KVM_CREATE_GUEST_MEMFD (memory files created
-                               without INIT_SHARED will be marked private).
-                               Shared memory can be faulted into host userspace
-                               page tables. Private memory cannot.
-  ============================ ================================================
+  ============================= ================================================
+  GUEST_MEMFD_FLAG_MMAP         Enable using mmap() on the guest_memfd file
+                                descriptor.
+  GUEST_MEMFD_FLAG_INIT_SHARED  Make all memory in the file shared during
+                                KVM_CREATE_GUEST_MEMFD (memory files created
+                                without INIT_SHARED will be marked private).
+                                Shared memory can be faulted into host userspace
+                                page tables. Private memory cannot.
+  GUEST_MEMFD_FLAG_USE_RESOURCE Consume resource_fd as an external memory
+                                provider or resource pool. When not set,
+                                resource_fd must be '0'. When set, the provider
+                                must be a supported resource (e.g. tmpfs mount
+                                root directory). Currently, tmpfs resource pools
+                                require noswap and huge=never mount options.
+  ============================= ================================================
 
 When the KVM MMU performs a PFN lookup to service a guest fault, the fault will
 always be consumed from guest_memfd, regardless of whether it is a shared or a
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index e7fc94e2910a8..d820afac49252 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2618,7 +2618,7 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm);
 
 static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
 {
-	u64 flags = GUEST_MEMFD_FLAG_MMAP;
+	u64 flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_RESOURCE;
 
 	if (!kvm || kvm_arch_supports_gmem_init_shared(kvm))
 		flags |= GUEST_MEMFD_FLAG_INIT_SHARED;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 8dff2fc1972e9..f4108bf7e3192 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1674,11 +1674,14 @@ struct kvm_memory_attributes2 {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_RESOURCE	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	__s32 resource_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ee823f2ef0368..84877beed6701 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -85,15 +85,26 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
 static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
 {
 	const struct guest_memfd_provider_operations *ops = NULL;
+	void *provider = ERR_PTR(-EOPNOTSUPP);
 	struct file *resource_file = NULL;
-	void *provider;
 
-	if (resource_fd == GMEM_RESOURCE_FD_NATIVE)
+	if (resource_fd == GMEM_RESOURCE_FD_NATIVE) {
 		ops = &gmem_native_provider_ops;
-	else
-		return -EOPNOTSUPP;
+	} else {
+		resource_file = fget_raw(resource_fd);
+		if (!resource_file)
+			return -EBADF;
+
+		if (resource_file->f_op)
+			ops = resource_file->f_op->gmem_provider_ops;
+	}
+
+	if (ops && ops->attach && ops->alloc_folio)
+		provider = ops->attach(inode, resource_file);
+
+	if (resource_file)
+		fput(resource_file);
 
-	provider = ops->attach(inode, resource_file);
 	if (IS_ERR(provider))
 		return PTR_ERR(provider);
 
@@ -120,6 +131,12 @@ static void gmem_provider_invalidate_folio(struct gmem_inode *gi,
 		gi->provider_ops->invalidate_folio(gi->provider, folio);
 }
 
+static void gmem_provider_release(struct gmem_inode *gi)
+{
+	if (gi->provider_ops && gi->provider_ops->release)
+		gi->provider_ops->release(gi->provider);
+}
+
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -1033,7 +1050,7 @@ static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags)
 	return r;
 }
 
-static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
+static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int resource_fd)
 {
 	static const char *name = "[kvm-gmem]";
 	struct gmem_file *f;
@@ -1088,7 +1105,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 	 */
 	smp_wmb();
 
-	err = kvm_gmem_attach_resource(inode, GMEM_RESOURCE_FD_NATIVE);
+	err = kvm_gmem_attach_resource(inode, resource_fd);
 	if (err) {
 		fput(file);
 		put_unused_fd(fd);
@@ -1111,6 +1128,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 
 int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
 {
+	int resource_fd = GMEM_RESOURCE_FD_NATIVE;
 	loff_t size = args->size;
 	u64 flags = args->flags;
 
@@ -1120,7 +1138,15 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
 	if (size <= 0 || !PAGE_ALIGNED(size))
 		return -EINVAL;
 
-	return __kvm_gmem_create(kvm, size, flags);
+	if (flags & GUEST_MEMFD_FLAG_USE_RESOURCE) {
+		if (args->resource_fd < 0)
+			return -EBADF;
+		resource_fd = args->resource_fd;
+	} else if (args->resource_fd) {
+		return -EINVAL;
+	}
+
+	return __kvm_gmem_create(kvm, size, flags, resource_fd);
 }
 
 int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot,
@@ -1464,6 +1490,8 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
 {
 	struct gmem_inode *gi = GMEM_I(inode);
 
+	gmem_provider_release(gi);
+
 	mpol_free_shared_policy(&gi->policy);
 
 	/*

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 06/88] KVM: selftests: Add helper to create guest_memfd with a resource file

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a helper to create guest_memfd instances with a resource file
descriptor, and update the existing creation helper to pass a zero
resource file descriptor.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/include/uapi/linux/kvm.h                 |  5 ++++-
 tools/testing/selftests/kvm/include/kvm_util.h | 23 ++++++++++++++++++-----
 2 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index 419011097fa8e..66c1f40b50c29 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1654,11 +1654,14 @@ struct kvm_memory_attributes {
 #define KVM_CREATE_GUEST_MEMFD	_IOWR(KVMIO,  0xd4, struct kvm_create_guest_memfd)
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
+#define GUEST_MEMFD_FLAG_USE_RESOURCE	(1ULL << 2)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
 	__u64 flags;
-	__u64 reserved[6];
+	__s32 resource_fd;
+	__u32 pad;
+	__u64 reserved[5];
 };
 
 #define KVM_PRE_FAULT_MEMORY	_IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index e2618877619e1..eca296539db79 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -809,26 +809,39 @@ static inline bool is_smt_on(void)
 
 void vm_create_irqchip(struct kvm_vm *vm);
 
-static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
-					  u64 flags)
+static inline int __vm_create_guest_memfd_resource(struct kvm_vm *vm, u64 size,
+						   u64 flags, int resource_fd)
 {
 	struct kvm_create_guest_memfd guest_memfd = {
 		.size = size,
 		.flags = flags,
+		.resource_fd = resource_fd,
 	};
 
 	return __vm_ioctl(vm, KVM_CREATE_GUEST_MEMFD, &guest_memfd);
 }
 
-static inline int vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
-					u64 flags)
+static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
+					  u64 flags)
+{
+	return __vm_create_guest_memfd_resource(vm, size, flags, 0);
+}
+
+static inline int vm_create_guest_memfd_resource(struct kvm_vm *vm, u64 size,
+						 u64 flags, int resource_fd)
 {
-	int fd = __vm_create_guest_memfd(vm, size, flags);
+	int fd = __vm_create_guest_memfd_resource(vm, size, flags, resource_fd);
 
 	TEST_ASSERT(fd >= 0, KVM_IOCTL_ERROR(KVM_CREATE_GUEST_MEMFD, fd));
 	return fd;
 }
 
+static inline int vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
+					u64 flags)
+{
+	return vm_create_guest_memfd_resource(vm, size, flags, 0);
+}
+
 void vm_set_user_memory_region(struct kvm_vm *vm, u32 slot, u32 flags,
 			       gpa_t gpa, u64 size, void *hva);
 int __vm_set_user_memory_region(struct kvm_vm *vm, u32 slot, u32 flags,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 07/88] KVM: selftests: Test negative validation of resource_fd argument

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add negative test coverage verifying that creating a guest_memfd
instance with a non-zero resource_fd when GUEST_MEMFD_FLAG_USE_RESOURCE
is clear fails with -EINVAL, and creating with an invalid resource_fd
fails with -EBADF.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index cd5df88bc6427..3313d306c0bdc 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -404,6 +404,8 @@ static void test_guest_memfd_flags(struct kvm_vm *vm)
 	int fd;
 
 	for (flag = BIT(0); flag; flag <<= 1) {
+		if (flag == GUEST_MEMFD_FLAG_USE_RESOURCE)
+			continue;
 		fd = __vm_create_guest_memfd(vm, page_size, flag);
 		if (flag & valid_flags) {
 			TEST_ASSERT(fd >= 0,
@@ -418,6 +420,22 @@ static void test_guest_memfd_flags(struct kvm_vm *vm)
 	}
 }
 
+static void test_resource_fd_invalid(struct kvm_vm *vm)
+{
+	int fd;
+
+	/* Non-zero resource_fd without GUEST_MEMFD_FLAG_USE_RESOURCE fails */
+	fd = __vm_create_guest_memfd_resource(vm, page_size, 0, 1);
+	TEST_ASSERT(fd < 0, "guest_memfd with resource_fd but without flag should fail");
+	TEST_ASSERT_EQ(errno, EINVAL);
+
+	/* Bad file descriptor with GUEST_MEMFD_FLAG_USE_RESOURCE fails */
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE, -1);
+	TEST_ASSERT(fd < 0, "guest_memfd with -1 resource_fd should fail");
+	TEST_ASSERT_EQ(errno, EBADF);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
 do {									\
 	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
@@ -476,6 +494,8 @@ static void test_guest_memfd(unsigned long vm_type)
 
 	test_guest_memfd_flags(vm);
 
+	test_resource_fd_invalid(vm);
+
 	__test_guest_memfd(vm, 0);
 
 	flags = vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 08/88] KVM: selftests: Test rejection of unsupported filesystem for resource_fd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add test coverage verifying that passing a directory file descriptor
from an unsupported filesystem without provider operations as a resource
file fails with EOPNOTSUPP.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 3313d306c0bdc..445396860d819 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -436,6 +436,27 @@ static void test_resource_fd_invalid(struct kvm_vm *vm)
 	TEST_ASSERT_EQ(errno, EBADF);
 }
 
+static void test_resource_fd_unsupported_fs(struct kvm_vm *vm)
+{
+	int fd, file_fd;
+
+	/*
+	 * Passing an FD from an unsupported filesystem without provider ops
+	 * (e.g. the /proc mount root directory) is rejected by KVM with
+	 * EOPNOTSUPP.
+	 */
+	file_fd = open("/proc", O_DIRECTORY | O_RDONLY);
+	TEST_ASSERT(file_fd >= 0, "open /proc failed");
+
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      file_fd);
+	TEST_ASSERT(fd < 0, "guest_memfd with /proc fd should fail");
+	TEST_ASSERT_EQ(errno, EOPNOTSUPP);
+
+	close(file_fd);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
 do {									\
 	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
@@ -495,6 +516,7 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_guest_memfd_flags(vm);
 
 	test_resource_fd_invalid(vm);
+	test_resource_fd_unsupported_fs(vm);
 
 	__test_guest_memfd(vm, 0);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 09/88] KVM: selftests: Test rejection of memfd for resource_fd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add test coverage verifying that passing an fd created via memfd_create
as a resource file fails with EOPNOTSUPP, because regular files do not
implement gmem_provider_ops. Only directory file descriptors representing
the mount root are supported as resource pools.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 445396860d819..d62ae23256083 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -457,6 +457,27 @@ static void test_resource_fd_unsupported_fs(struct kvm_vm *vm)
 	close(file_fd);
 }
 
+static void test_resource_fd_memfd(struct kvm_vm *vm)
+{
+	int fd, file_fd;
+
+	/*
+	 * Passing an fd returned by memfd_create is rejected with EOPNOTSUPP
+	 * because regular files do not have gmem_provider_ops defined on their
+	 * f_op. Only directory FDs representing the mount root are supported
+	 * resource pools.
+	 */
+	file_fd = memfd_create("test_memfd", 0);
+	TEST_ASSERT(file_fd >= 0, "memfd_create failed");
+
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      file_fd);
+	TEST_ASSERT(fd < 0, "guest_memfd with memfd should fail");
+	TEST_ASSERT_EQ(errno, EOPNOTSUPP);
+	close(file_fd);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
 do {									\
 	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
@@ -517,6 +538,7 @@ static void test_guest_memfd(unsigned long vm_type)
 
 	test_resource_fd_invalid(vm);
 	test_resource_fd_unsupported_fs(vm);
+	test_resource_fd_memfd(vm);
 
 	__test_guest_memfd(vm, 0);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 10/88] KVM: selftests: Test rejection of swap tmpfs mounts

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Test that guest_memfd creation fails with -EINVAL when passed a resource
pool fd for a mount that has swap enabled.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 70 ++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index d62ae23256083..205f822e10c6c 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -13,7 +13,9 @@
 
 #include <linux/bitmap.h>
 #include <linux/falloc.h>
+#include <linux/mount.h>
 #include <linux/sizes.h>
+#include <sys/mount.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 
@@ -478,6 +480,73 @@ static void test_resource_fd_memfd(struct kvm_vm *vm)
 	close(file_fd);
 }
 
+static int create_tmpfs_pool_fd(const char *huge, bool noswap, size_t size)
+{
+	char size_str[32];
+	char template[] = "/tmp/gmem_test_dir_XXXXXX";
+	char *mount_dir;
+	int fs_fd, mnt_fd, dir_fd;
+
+	TEST_ASSERT(size && IS_ALIGNED(size, page_size),
+		    "Pool size '0x%zx' must be positive and page-aligned", size);
+
+	fs_fd = syscall(__NR_fsopen, "tmpfs", FSOPEN_CLOEXEC);
+	TEST_ASSERT(fs_fd >= 0, "fsopen failed");
+
+	if (noswap)
+		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_FLAG,
+				     "noswap", NULL, 0), "fsconfig noswap failed");
+
+	if (huge)
+		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
+				     "huge", huge, 0), "fsconfig huge failed");
+
+	snprintf(size_str, sizeof(size_str), "%zu", size);
+	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
+			     "size", size_str, 0), "fsconfig size failed");
+
+	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_CMD_CREATE,
+			     NULL, NULL, 0), "fsconfig create failed");
+
+	mnt_fd = syscall(__NR_fsmount, fs_fd, FSMOUNT_CLOEXEC, 0);
+	TEST_ASSERT(mnt_fd > 0, "fsmount failed");
+	close(fs_fd);
+
+	mount_dir = mkdtemp(template);
+	TEST_ASSERT(mount_dir, "mkdtemp failed");
+
+	TEST_ASSERT(!syscall(__NR_move_mount, mnt_fd, "", AT_FDCWD, mount_dir,
+			     MOVE_MOUNT_F_EMPTY_PATH), "move_mount failed");
+	close(mnt_fd);
+
+	dir_fd = open(mount_dir, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
+	TEST_ASSERT(dir_fd >= 0, "open tmpfs mount directory failed");
+
+	/*
+	 * Detach mount from namespace so removing mount_dir upon exit or
+	 * unmount doesn't conflict, while dir_fd retains the active mount.
+	 */
+	TEST_ASSERT(!umount2(mount_dir, MNT_DETACH), "umount2 failed");
+	TEST_ASSERT(!rmdir(mount_dir), "rmdir failed");
+
+	return dir_fd;
+}
+
+static void test_resource_fd_tmpfs_swap(struct kvm_vm *vm)
+{
+	int pool_fd, fd;
+
+	pool_fd = create_tmpfs_pool_fd("never", false, page_size);
+
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      pool_fd);
+	TEST_ASSERT(fd < 0, "guest_memfd with swap-enabled tmpfs should fail");
+	TEST_ASSERT_EQ(errno, EINVAL);
+
+	close(pool_fd);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
 do {									\
 	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
@@ -539,6 +608,7 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_resource_fd_invalid(vm);
 	test_resource_fd_unsupported_fs(vm);
 	test_resource_fd_memfd(vm);
+	test_resource_fd_tmpfs_swap(vm);
 
 	__test_guest_memfd(vm, 0);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 11/88] KVM: selftests: Test rejection of hugepage tmpfs mounts

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Ensure that guest_memfd creation fails with -EINVAL when passed a tmpfs
resource pool mount descriptor that has huge pages enabled, as required
by the tmpfs guest_memfd provider implementation.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 205f822e10c6c..541d8cdd0bbf7 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -547,6 +547,21 @@ static void test_resource_fd_tmpfs_swap(struct kvm_vm *vm)
 	close(pool_fd);
 }
 
+static void test_resource_fd_tmpfs_huge(struct kvm_vm *vm)
+{
+	int pool_fd, fd;
+
+	pool_fd = create_tmpfs_pool_fd("always", true, page_size);
+
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      pool_fd);
+	TEST_ASSERT(fd < 0, "guest_memfd with huge-enabled tmpfs should fail");
+	TEST_ASSERT_EQ(errno, EINVAL);
+
+	close(pool_fd);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
 do {									\
 	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
@@ -609,6 +624,7 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_resource_fd_unsupported_fs(vm);
 	test_resource_fd_memfd(vm);
 	test_resource_fd_tmpfs_swap(vm);
+	test_resource_fd_tmpfs_huge(vm);
 
 	__test_guest_memfd(vm, 0);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 12/88] KVM: selftests: Test rejection of regular file in tmpfs mount

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add test coverage verifying that creating a regular file inside an
actual tmpfs mount and passing that file descriptor as resource_fd
fails with EOPNOTSUPP, because tmpfs regular files do not have
gmem_provider_ops defined on their file_operations.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 541d8cdd0bbf7..bef1acead4a6d 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -613,6 +613,25 @@ static void __test_guest_memfd(struct kvm_vm *vm, u64 flags)
 	gmem_test_vm(invalid_binding, vm, flags);
 }
 
+static void test_resource_fd_tmpfs_regular_file(struct kvm_vm *vm)
+{
+	int pool_fd, file_fd, fd;
+
+	pool_fd = create_tmpfs_pool_fd("never", true, page_size);
+
+	file_fd = openat(pool_fd, "regular_file", O_CREAT | O_RDWR | O_CLOEXEC, 0644);
+	TEST_ASSERT(file_fd >= 0, "openat regular_file in tmpfs pool failed");
+
+	fd = __vm_create_guest_memfd_resource(vm, page_size,
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      file_fd);
+	TEST_ASSERT(fd < 0, "guest_memfd with tmpfs regular file should fail");
+	TEST_ASSERT_EQ(errno, EOPNOTSUPP);
+
+	close(file_fd);
+	close(pool_fd);
+}
+
 static void test_guest_memfd(unsigned long vm_type)
 {
 	struct kvm_vm *vm = vm_create_barebones_type(vm_type);
@@ -625,6 +644,7 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_resource_fd_memfd(vm);
 	test_resource_fd_tmpfs_swap(vm);
 	test_resource_fd_tmpfs_huge(vm);
+	test_resource_fd_tmpfs_regular_file(vm);
 
 	__test_guest_memfd(vm, 0);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 13/88] KVM: selftests: Test guest_memfd with tmpfs resource pool

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add test coverage verifying that creating a guest_memfd backed by a
tmpfs resource pool directory succeeds across all guest_memfd subtests
and flag combinations, and reports the requested file size.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 35 ++++++++++++++++++++------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index bef1acead4a6d..2250c3e9a583f 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -562,12 +562,23 @@ static void test_resource_fd_tmpfs_huge(struct kvm_vm *vm)
 	close(pool_fd);
 }
 
-#define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)	\
-do {									\
-	int fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
-									\
-	test_##__test(args);						\
-	close(fd);							\
+#define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)		\
+do {										\
+	int pool_fd = -1;							\
+	int fd;									\
+										\
+	if ((__flags) & GUEST_MEMFD_FLAG_USE_RESOURCE) {			\
+		pool_fd = create_tmpfs_pool_fd("never", true, __gmem_size);	\
+		fd = vm_create_guest_memfd_resource(__vm, __gmem_size,		\
+						    __flags, pool_fd);		\
+	} else {								\
+		fd = vm_create_guest_memfd(__vm, __gmem_size, __flags);	\
+	}									\
+										\
+	test_##__test(args);							\
+	close(fd);								\
+	if (pool_fd >= 0)							\
+		close(pool_fd);							\
 } while (0)
 
 #define __gmem_test(__test, __vm, __flags, __gmem_size)			\
@@ -647,15 +658,23 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_resource_fd_tmpfs_regular_file(vm);
 
 	__test_guest_memfd(vm, 0);
+	__test_guest_memfd(vm, GUEST_MEMFD_FLAG_USE_RESOURCE);
 
 	flags = vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS);
-	if (flags & GUEST_MEMFD_FLAG_MMAP)
+	if (flags & GUEST_MEMFD_FLAG_MMAP) {
 		__test_guest_memfd(vm, GUEST_MEMFD_FLAG_MMAP);
+		__test_guest_memfd(vm, GUEST_MEMFD_FLAG_MMAP |
+				       GUEST_MEMFD_FLAG_USE_RESOURCE);
+	}
 
 	/* MMAP should always be supported if INIT_SHARED is supported. */
-	if (flags & GUEST_MEMFD_FLAG_INIT_SHARED)
+	if (flags & GUEST_MEMFD_FLAG_INIT_SHARED) {
 		__test_guest_memfd(vm, GUEST_MEMFD_FLAG_MMAP |
 				       GUEST_MEMFD_FLAG_INIT_SHARED);
+		__test_guest_memfd(vm, GUEST_MEMFD_FLAG_MMAP |
+				       GUEST_MEMFD_FLAG_INIT_SHARED |
+				       GUEST_MEMFD_FLAG_USE_RESOURCE);
+	}
 
 	kvm_vm_free(vm);
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 14/88] KVM: selftests: Test guest_memfd resource pool sharing across instances

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Test that multiple guest_memfd instances can be created backed by the
same tmpfs resource pool descriptor, that each file descriptor reflects
its requested size, and that they have distinct inodes.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 34 ++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 2250c3e9a583f..310a45b6656aa 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -562,6 +562,39 @@ static void test_resource_fd_tmpfs_huge(struct kvm_vm *vm)
 	close(pool_fd);
 }
 
+static void test_resource_fd_tmpfs_shared_pool(struct kvm_vm *vm)
+{
+	int pool_fd, fd1, fd2;
+	struct stat st1, st2;
+
+	pool_fd = create_tmpfs_pool_fd("never", true, page_size * 2);
+
+	fd1 = vm_create_guest_memfd_resource(vm, page_size,
+					     GUEST_MEMFD_FLAG_USE_RESOURCE,
+					     pool_fd);
+
+	fd2 = vm_create_guest_memfd_resource(vm, page_size * 2,
+					     GUEST_MEMFD_FLAG_USE_RESOURCE,
+					     pool_fd);
+
+	TEST_ASSERT(!fstat(fd1, &st1), "fstat on fd1 should succeed");
+	TEST_ASSERT(st1.st_size == page_size,
+		    "fd1 st_size (%lu) should match requested size (%lu)",
+		    st1.st_size, page_size);
+
+	TEST_ASSERT(!fstat(fd2, &st2), "fstat on fd2 should succeed");
+	TEST_ASSERT(st2.st_size == page_size * 2,
+		    "fd2 st_size (%lu) should match requested size (%lu)",
+		    st2.st_size, page_size * 2);
+
+	TEST_ASSERT(st1.st_ino != st2.st_ino,
+		    "different guest_memfd instances should have distinct inodes");
+
+	close(fd2);
+	close(fd1);
+	close(pool_fd);
+}
+
 #define ____gmem_test(__test, __vm, __flags, __gmem_size, args...)		\
 do {										\
 	int pool_fd = -1;							\
@@ -656,6 +689,7 @@ static void test_guest_memfd(unsigned long vm_type)
 	test_resource_fd_tmpfs_swap(vm);
 	test_resource_fd_tmpfs_huge(vm);
 	test_resource_fd_tmpfs_regular_file(vm);
+	test_resource_fd_tmpfs_shared_pool(vm);
 
 	__test_guest_memfd(vm, 0);
 	__test_guest_memfd(vm, GUEST_MEMFD_FLAG_USE_RESOURCE);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 15/88] KVM: selftests: Test memory allocation against shared tmpfs resource pool

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Verify that allocating folios in multiple guest_memfd instances sharing
the same tmpfs resource pool consumes blocks from the underlying pool as
reported by filesystem statistics.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 310a45b6656aa..639b1b0c204ed 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -18,6 +18,7 @@
 #include <sys/mount.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <sys/statvfs.h>
 
 #include "kvm_syscalls.h"
 #include "kvm_util.h"
@@ -564,7 +565,8 @@ static void test_resource_fd_tmpfs_huge(struct kvm_vm *vm)
 
 static void test_resource_fd_tmpfs_shared_pool(struct kvm_vm *vm)
 {
-	int pool_fd, fd1, fd2;
+	int pool_fd, fd1, fd2, ret;
+	struct statvfs svfs;
 	struct stat st1, st2;
 
 	pool_fd = create_tmpfs_pool_fd("never", true, page_size * 2);
@@ -590,6 +592,18 @@ static void test_resource_fd_tmpfs_shared_pool(struct kvm_vm *vm)
 	TEST_ASSERT(st1.st_ino != st2.st_ino,
 		    "different guest_memfd instances should have distinct inodes");
 
+	ret = fallocate(fd1, FALLOC_FL_KEEP_SIZE, 0, page_size);
+	TEST_ASSERT(!ret, "fallocate on fd1 should succeed");
+
+	TEST_ASSERT(!fstatvfs(pool_fd, &svfs), "fstatvfs on pool_fd should succeed");
+	TEST_ASSERT_EQ(svfs.f_blocks - svfs.f_bfree, 1);
+
+	ret = fallocate(fd2, FALLOC_FL_KEEP_SIZE, 0, page_size);
+	TEST_ASSERT(!ret, "fallocate on fd2 should succeed");
+
+	TEST_ASSERT(!fstatvfs(pool_fd, &svfs), "fstatvfs on pool_fd should succeed");
+	TEST_ASSERT_EQ(svfs.f_blocks - svfs.f_bfree, 2);
+
 	close(fd2);
 	close(fd1);
 	close(pool_fd);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 16/88] KVM: selftests: Test that shared tmpfs resource pool size limit is respected

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Verify that allocating memory across guest_memfd instances sharing a
tmpfs resource pool fails with ENOSPC when the total allocation exceeds
the pool capacity.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 639b1b0c204ed..20de24ad535f1 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -604,6 +604,10 @@ static void test_resource_fd_tmpfs_shared_pool(struct kvm_vm *vm)
 	TEST_ASSERT(!fstatvfs(pool_fd, &svfs), "fstatvfs on pool_fd should succeed");
 	TEST_ASSERT_EQ(svfs.f_blocks - svfs.f_bfree, 2);
 
+	ret = fallocate(fd2, FALLOC_FL_KEEP_SIZE, page_size, page_size);
+	TEST_ASSERT(ret < 0, "fallocate exceeding pool capacity should fail");
+	TEST_ASSERT_EQ(errno, ENOSPC);
+
 	close(fd2);
 	close(fd1);
 	close(pool_fd);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 17/88] KVM: selftests: Test guest execution with tmpfs-backed guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Test that a guest vCPU can execute instructions and read/write memory
mapped into a guest_memfd instance that is backed by a tmpfs resource
pool descriptor.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 56 ++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 20de24ad535f1..8e9261db23354 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -799,6 +799,61 @@ static void test_guest_memfd_guest(void)
 	kvm_vm_free(vm);
 }
 
+static void test_guest_memfd_guest_resource(void)
+{
+	const gpa_t gpa = SZ_4G;
+	const int slot = 1;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int pool_fd, fd, i;
+	size_t size;
+	u8 *mem;
+
+	if (!kvm_check_cap(KVM_CAP_GUEST_MEMFD_FLAGS))
+		return;
+
+	pool_fd = create_tmpfs_pool_fd("never", true, page_size);
+	if (pool_fd < 0)
+		TEST_REQUIRE(false);
+
+	vm = __vm_create_shape_with_one_vcpu(VM_SHAPE_DEFAULT, &vcpu, 1, guest_code);
+
+	TEST_ASSERT(vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS) & GUEST_MEMFD_FLAG_MMAP,
+		    "Default VM type should support MMAP, supported flags = 0x%x",
+		    vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS));
+	TEST_ASSERT(vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS) & GUEST_MEMFD_FLAG_INIT_SHARED,
+		    "Default VM type should support INIT_SHARED, supported flags = 0x%x",
+		    vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS));
+
+	size = max_t(size_t, vm->page_size, page_size);
+	fd = __vm_create_guest_memfd_resource(vm, size,
+					      GUEST_MEMFD_FLAG_MMAP |
+					      GUEST_MEMFD_FLAG_INIT_SHARED |
+					      GUEST_MEMFD_FLAG_USE_RESOURCE,
+					      pool_fd);
+	TEST_ASSERT(fd >= 0, "guest_memfd with tmpfs pool should succeed");
+
+	vm_set_user_memory_region2(vm, slot, KVM_MEM_GUEST_MEMFD, gpa, size, NULL, fd, 0);
+
+	mem = kvm_mmap(size, PROT_READ | PROT_WRITE, MAP_SHARED, fd);
+	memset(mem, 0xaa, size);
+	kvm_munmap(mem, size);
+
+	virt_map(vm, gpa, gpa, size / vm->page_size);
+	vcpu_args_set(vcpu, 2, gpa, size);
+	vcpu_run(vcpu);
+
+	TEST_ASSERT_EQ(get_ucall(vcpu, NULL), UCALL_DONE);
+
+	mem = kvm_mmap(size, PROT_READ | PROT_WRITE, MAP_SHARED, fd);
+	for (i = 0; i < size; i++)
+		TEST_ASSERT_EQ(mem[i], 0xff);
+
+	close(fd);
+	close(pool_fd);
+	kvm_vm_free(vm);
+}
+
 int main(int argc, char *argv[])
 {
 	unsigned long vm_types, vm_type;
@@ -819,4 +874,5 @@ int main(int argc, char *argv[])
 		test_guest_memfd(vm_type);
 
 	test_guest_memfd_guest();
+	test_guest_memfd_guest_resource();
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 18/88] KVM: selftests: Document testing TODOs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Remind myself about all the tests (A)I did not write.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index 8e9261db23354..fd249445ce3cc 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -701,6 +701,16 @@ static void test_guest_memfd(unsigned long vm_type)
 
 	test_guest_memfd_flags(vm);
 
+	/*
+	 * TODO: Test that guest_memfd creation rejects a read-only tmpfs mount
+	 * (e.g. MOUNT_ATTR_RDONLY) with EROFS.
+	 *
+	 * TODO: Test that guest_memfd creation rejects an ID-mapped tmpfs mount
+	 * (e.g. MOUNT_ATTR_IDMAP) lacking write or execute/search permissions
+	 * with EACCES.
+	 *
+	 * TODO: Test mpol fallback to mount config.
+	 */
 	test_resource_fd_invalid(vm);
 	test_resource_fd_unsupported_fs(vm);
 	test_resource_fd_memfd(vm);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 19/88] KVM: selftests: Test guest_memfd conversions with tmpfs backing

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Move create_tmpfs_pool_fd() into lib/test_util.c and declare it in
include/test_util.h so it can be reused across selftests.

Extend guest_memfd_conversions_test using fixture variants to run all
conversion test cases against both native guest_memfd and tmpfs-backed
guest_memfd instances.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/guest_memfd_test.c     | 52 --------------------
 tools/testing/selftests/kvm/include/test_util.h    |  2 +
 tools/testing/selftests/kvm/lib/test_util.c        | 56 ++++++++++++++++++++++
 .../kvm/x86/guest_memfd_conversions_test.c         | 49 +++++++++++++++++--
 4 files changed, 102 insertions(+), 57 deletions(-)

diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index fd249445ce3cc..cbc2f309664b2 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -481,58 +481,6 @@ static void test_resource_fd_memfd(struct kvm_vm *vm)
 	close(file_fd);
 }
 
-static int create_tmpfs_pool_fd(const char *huge, bool noswap, size_t size)
-{
-	char size_str[32];
-	char template[] = "/tmp/gmem_test_dir_XXXXXX";
-	char *mount_dir;
-	int fs_fd, mnt_fd, dir_fd;
-
-	TEST_ASSERT(size && IS_ALIGNED(size, page_size),
-		    "Pool size '0x%zx' must be positive and page-aligned", size);
-
-	fs_fd = syscall(__NR_fsopen, "tmpfs", FSOPEN_CLOEXEC);
-	TEST_ASSERT(fs_fd >= 0, "fsopen failed");
-
-	if (noswap)
-		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_FLAG,
-				     "noswap", NULL, 0), "fsconfig noswap failed");
-
-	if (huge)
-		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
-				     "huge", huge, 0), "fsconfig huge failed");
-
-	snprintf(size_str, sizeof(size_str), "%zu", size);
-	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
-			     "size", size_str, 0), "fsconfig size failed");
-
-	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_CMD_CREATE,
-			     NULL, NULL, 0), "fsconfig create failed");
-
-	mnt_fd = syscall(__NR_fsmount, fs_fd, FSMOUNT_CLOEXEC, 0);
-	TEST_ASSERT(mnt_fd > 0, "fsmount failed");
-	close(fs_fd);
-
-	mount_dir = mkdtemp(template);
-	TEST_ASSERT(mount_dir, "mkdtemp failed");
-
-	TEST_ASSERT(!syscall(__NR_move_mount, mnt_fd, "", AT_FDCWD, mount_dir,
-			     MOVE_MOUNT_F_EMPTY_PATH), "move_mount failed");
-	close(mnt_fd);
-
-	dir_fd = open(mount_dir, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
-	TEST_ASSERT(dir_fd >= 0, "open tmpfs mount directory failed");
-
-	/*
-	 * Detach mount from namespace so removing mount_dir upon exit or
-	 * unmount doesn't conflict, while dir_fd retains the active mount.
-	 */
-	TEST_ASSERT(!umount2(mount_dir, MNT_DETACH), "umount2 failed");
-	TEST_ASSERT(!rmdir(mount_dir), "rmdir failed");
-
-	return dir_fd;
-}
-
 static void test_resource_fd_tmpfs_swap(struct kvm_vm *vm)
 {
 	int pool_fd, fd;
diff --git a/tools/testing/selftests/kvm/include/test_util.h b/tools/testing/selftests/kvm/include/test_util.h
index c80019e24b2d0..e6707534f1951 100644
--- a/tools/testing/selftests/kvm/include/test_util.h
+++ b/tools/testing/selftests/kvm/include/test_util.h
@@ -244,4 +244,6 @@ char *strdup_printf(const char *fmt, ...) __attribute__((format(printf, 1, 2), n
 
 char *sys_get_cur_clocksource(void);
 
+int create_tmpfs_pool_fd(const char *huge, bool noswap, size_t size);
+
 #endif /* SELFTEST_KVM_TEST_UTIL_H */
diff --git a/tools/testing/selftests/kvm/lib/test_util.c b/tools/testing/selftests/kvm/lib/test_util.c
index f5f7189f96a84..c5b264f533ad0 100644
--- a/tools/testing/selftests/kvm/lib/test_util.c
+++ b/tools/testing/selftests/kvm/lib/test_util.c
@@ -10,10 +10,14 @@
 #include <ctype.h>
 #include <limits.h>
 #include <stdlib.h>
+#include <fcntl.h>
 #include <time.h>
+#include <sys/mount.h>
 #include <sys/stat.h>
 #include <sys/syscall.h>
+#include <linux/align.h>
 #include <linux/mman.h>
+#include <linux/mount.h>
 #include "linux/kernel.h"
 
 #include "test_util.h"
@@ -483,3 +487,55 @@ char *sys_get_cur_clocksource(void)
 
 	return clk_name;
 }
+
+int create_tmpfs_pool_fd(const char *huge, bool noswap, size_t size)
+{
+	char size_str[32];
+	char template[] = "/tmp/gmem_test_dir_XXXXXX";
+	char *mount_dir;
+	int fs_fd, mnt_fd, dir_fd;
+
+	TEST_ASSERT(size && IS_ALIGNED(size, getpagesize()),
+		    "Pool size '0x%zx' must be positive and page-aligned", size);
+
+	fs_fd = syscall(__NR_fsopen, "tmpfs", FSOPEN_CLOEXEC);
+	TEST_ASSERT(fs_fd >= 0, "fsopen failed");
+
+	if (noswap)
+		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_FLAG,
+				     "noswap", NULL, 0), "fsconfig noswap failed");
+
+	if (huge)
+		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
+				     "huge", huge, 0), "fsconfig huge failed");
+
+	snprintf(size_str, sizeof(size_str), "%zu", size);
+	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_STRING,
+			     "size", size_str, 0), "fsconfig size failed");
+
+	TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_CMD_CREATE,
+			     NULL, NULL, 0), "fsconfig create failed");
+
+	mnt_fd = syscall(__NR_fsmount, fs_fd, FSMOUNT_CLOEXEC, 0);
+	TEST_ASSERT(mnt_fd > 0, "fsmount failed");
+	close(fs_fd);
+
+	mount_dir = mkdtemp(template);
+	TEST_ASSERT(mount_dir, "mkdtemp failed");
+
+	TEST_ASSERT(!syscall(__NR_move_mount, mnt_fd, "", AT_FDCWD, mount_dir,
+			     MOVE_MOUNT_F_EMPTY_PATH), "move_mount failed");
+	close(mnt_fd);
+
+	dir_fd = open(mount_dir, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
+	TEST_ASSERT(dir_fd >= 0, "open tmpfs mount directory failed");
+
+	/*
+	 * Detach mount from namespace so removing mount_dir upon exit or
+	 * unmount doesn't conflict, while dir_fd retains the active mount.
+	 */
+	TEST_ASSERT(!umount2(mount_dir, MNT_DETACH), "umount2 failed");
+	TEST_ASSERT(!rmdir(mount_dir), "rmdir failed");
+
+	return dir_fd;
+}
diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
index e1c84df0f0596..023df9cb9d4a6 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
@@ -20,12 +20,30 @@
 FIXTURE(gmem_conversions) {
 	struct kvm_vcpu *vcpu;
 	int gmem_fd;
+	int resource_fd;
 	/* HVA of the first byte of the memory mmap()-ed from gmem_fd. */
 	char *mem;
 };
 
 typedef FIXTURE_DATA(gmem_conversions) test_data_t;
 
+FIXTURE_VARIANT(gmem_conversions) {
+	int (*create_resource)(size_t size);
+};
+
+FIXTURE_VARIANT_ADD(gmem_conversions, native) {
+	.create_resource = NULL,
+};
+
+static int create_tmpfs_resource(size_t size)
+{
+	return create_tmpfs_pool_fd("never", true, size);
+}
+
+FIXTURE_VARIANT_ADD(gmem_conversions, tmpfs) {
+	.create_resource = create_tmpfs_resource,
+};
+
 FIXTURE_SETUP(gmem_conversions) { }
 
 static size_t page_size;
@@ -37,8 +55,9 @@ static void guest_do_rmw(void);
  * Defer setup until the individual test is invoked so that tests can specify
  * the number of pages and flags for the guest_memfd instance.
  */
-static void gmem_conversions_do_setup(test_data_t *t, int nr_pages,
-				      int gmem_flags)
+static void gmem_conversions_do_setup(test_data_t *t,
+				      const FIXTURE_VARIANT(gmem_conversions) * variant,
+				      int nr_pages, int gmem_flags)
 {
 	const struct vm_shape shape = {
 		.mode = VM_MODE_DEFAULT,
@@ -50,12 +69,27 @@ static void gmem_conversions_do_setup(test_data_t *t, int nr_pages,
 	 */
 	const gpa_t gpa = SZ_4G;
 	const u32 slot = 1;
+	size_t size = nr_pages * page_size;
 	struct kvm_vm *vm;
+	int gmem_fd = -1;
 
 	vm = __vm_create_shape_with_one_vcpu(shape, &t->vcpu, nr_pages, guest_do_rmw);
 
+	t->resource_fd = -1;
+
+	if (variant->create_resource) {
+		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE;
+
+		t->resource_fd = variant->create_resource(size);
+		TEST_ASSERT(t->resource_fd >= 0, "Failed to create backing resource");
+
+		gmem_fd = __vm_create_guest_memfd_resource(vm, size, flags,
+							   t->resource_fd);
+		TEST_ASSERT(gmem_fd >= 0, "Failed to create guest_memfd with resource");
+	}
+
 	vm_mem_add(vm, VM_MEM_SRC_SHMEM, gpa, slot, nr_pages,
-		   KVM_MEM_GUEST_MEMFD, -1, 0, gmem_flags);
+		   KVM_MEM_GUEST_MEMFD, gmem_fd, 0, gmem_flags);
 
 	t->gmem_fd = kvm_slot_to_fd(vm, slot);
 	t->mem = addr_gpa2hva(vm, gpa);
@@ -68,6 +102,11 @@ static void gmem_conversions_do_teardown(test_data_t *t)
 	if (!t->vcpu)
 		return;
 
+	if (t->resource_fd >= 0) {
+		close(t->resource_fd);
+		t->resource_fd = -1;
+	}
+
 	/* No need to close gmem_fd, it's owned by the VM structure. */
 	kvm_vm_free(t->vcpu->vm);
 	t->vcpu = NULL;
@@ -91,7 +130,7 @@ TEST_F(gmem_conversions, test)							\
 {										\
 	const int nr = (__nr_pages);						\
 										\
-	gmem_conversions_do_setup(self, nr, flags);				\
+	gmem_conversions_do_setup(self, variant, nr, flags);			\
 	__gmem_conversions_##test(self, nr);					\
 }										\
 static void __gmem_conversions_##test(test_data_t *t, int nr_pages)		\
@@ -127,7 +166,7 @@ TEST_F(gmem_conversions, test)							\
 	int i;									\
 										\
 	for (i = 0; i < nr; ++i) {						\
-		gmem_conversions_do_setup(self, nr, flags);			\
+		gmem_conversions_do_setup(self, variant, nr, flags);		\
 		__gmem_conversions_multipage_##test(self, nr, i);		\
 		gmem_conversions_do_teardown(self);				\
 	}									\

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 20/88] samples/kvm: Introduce memory provider module skeleton

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Introduce the memory provider sample module skeleton under samples/kvm/.

The memory provider module will serve as a sample in-tree memory provider
for guest_memfd and iommufd. Register a miscdevice at /dev/memory_provider
upon module initialization and deregister it upon module exit.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/Kconfig               | 11 +++++++++++
 samples/Makefile              |  1 +
 samples/kvm/Makefile          |  2 ++
 samples/kvm/memory_provider.c | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 49 insertions(+)

diff --git a/samples/Kconfig b/samples/Kconfig
index a75e8e78330da..d4e79e1a9286b 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -326,6 +326,17 @@ source "samples/rust/Kconfig"
 
 source "samples/damon/Kconfig"
 
+config SAMPLE_KVM_MEMORY_PROVIDER
+	tristate "Build sample memory provider -- loadable module only"
+	depends on CONTIG_ALLOC && X86_64 && m
+	help
+	  This builds a sample memory provider.
+	  Its files can back a guest_memfd and be mapped by iommufd.
+	  It allocates a contiguous region with alloc_contig_pages()
+	  and carves out child provider files on the fly.
+
+	  If unsure, say N.
+
 endif # SAMPLES
 
 config HAVE_SAMPLE_FTRACE_DIRECT
diff --git a/samples/Makefile b/samples/Makefile
index 07641e177bd8b..3933ef6a323ed 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -37,6 +37,7 @@ obj-$(CONFIG_SAMPLE_TPS6594_PFSM)	+= pfsm/
 subdir-$(CONFIG_SAMPLE_WATCHDOG)	+= watchdog
 subdir-$(CONFIG_SAMPLE_WATCH_QUEUE)	+= watch_queue
 obj-$(CONFIG_SAMPLE_KMEMLEAK)		+= kmemleak/
+obj-$(CONFIG_SAMPLE_KVM_MEMORY_PROVIDER)	+= kvm/
 obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG)	+= coresight/
 obj-$(CONFIG_SAMPLE_FPROBE)		+= fprobe/
 obj-$(CONFIG_SAMPLES_RUST)		+= rust/
diff --git a/samples/kvm/Makefile b/samples/kvm/Makefile
new file mode 100644
index 0000000000000..9668dc57aa458
--- /dev/null
+++ b/samples/kvm/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_SAMPLE_KVM_MEMORY_PROVIDER) += memory_provider.o
diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
new file mode 100644
index 0000000000000..d5272cb88b758
--- /dev/null
+++ b/samples/kvm/memory_provider.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * memory_provider - a sample memory provider.
+ *
+ * Confidential VMs are not supported.
+ */
+
+#include <linux/fs.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+
+static const struct file_operations mp_ctl_fops = {
+	.owner		= THIS_MODULE,
+};
+
+static struct miscdevice mp_dev = {
+	.minor	= MISC_DYNAMIC_MINOR,
+	.name	= "memory_provider",
+	.fops	= &mp_ctl_fops,
+};
+
+static int __init mp_init(void)
+{
+	return misc_register(&mp_dev);
+}
+module_init(mp_init);
+
+static void __exit mp_exit(void)
+{
+	misc_deregister(&mp_dev);
+}
+module_exit(mp_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Sample memory provider for guest_memfd and iommufd");

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 21/88] samples/kvm: Support creating memory provider files

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Support creating memory provider files via the MP_NEW_FILE ioctl on the
control device.

Allocating contiguous physical memory at file creation time ensures that
every provider file descriptor exists in a fully initialized state with a
fixed physical page range throughout its lifetime. When the provider file
is closed, free the contiguous memory range and release the associated
tracking metadata.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/kvm/memory_provider.c | 110 ++++++++++++++++++++++++++++++++++++++++++
 samples/kvm/memory_provider.h |  30 ++++++++++++
 2 files changed, 140 insertions(+)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index d5272cb88b758..e89ce55a357c0 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -2,15 +2,125 @@
 /*
  * memory_provider - a sample memory provider.
  *
+ * The module provides physical memory through provider files.
+ *
+ * The control device /dev/memory_provider creates provider files.
+ * Each file allocates its own contiguous memory from alloc_contig_pages().
+ * See memory_provider.h.
+ *
  * Confidential VMs are not supported.
  */
 
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
 #include <linux/fs.h>
+#include <linux/gfp.h>
 #include <linux/miscdevice.h>
+#include <linux/mm.h>
 #include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include "memory_provider.h"
+
+/*
+ * One provider file.
+ */
+struct mp_info {
+	unsigned long base_pfn;
+	unsigned long npages;
+	struct page *cma_pages;		/* from alloc_contig_pages() */
+};
+
+static int mp_file_release(struct inode *inode, struct file *file)
+{
+	struct mp_info *info = file->private_data;
+
+	if (info->cma_pages)
+		free_contig_range(info->base_pfn, info->npages);
+	kfree(info);
+	return 0;
+}
+
+static const struct file_operations mp_file_fops = {
+	.owner		= THIS_MODULE,
+	.release	= mp_file_release,
+};
+
+static long mp_ctl_new_file(void __user *uarg)
+{
+	struct mp_new_file nf;
+	struct mp_info *info;
+	struct page *pages;
+	struct file *file;
+	unsigned long npages;
+	int fd, ret;
+
+	if (copy_from_user(&nf, uarg, sizeof(nf)))
+		return -EFAULT;
+	if (nf.flags || nf.pad)
+		return -EINVAL;
+	if (!nf.size || !PAGE_ALIGNED(nf.size))
+		return -EINVAL;
+
+	npages = nf.size >> PAGE_SHIFT;
+
+	info = kzalloc_obj(*info);
+	if (!info)
+		return -ENOMEM;
+
+	pages = alloc_contig_pages(npages, GFP_KERNEL | __GFP_ZERO,
+				   numa_node_id(), NULL);
+	if (!pages) {
+		ret = -ENOMEM;
+		goto err_free_info;
+	}
+
+	info->cma_pages = pages;
+	info->base_pfn = page_to_pfn(pages);
+	info->npages = npages;
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0) {
+		ret = fd;
+		goto err_free_pages;
+	}
+
+	file = anon_inode_getfile("[memory-provider]", &mp_file_fops,
+				  info, O_RDWR);
+	if (IS_ERR(file)) {
+		put_unused_fd(fd);
+		ret = PTR_ERR(file);
+		goto err_free_pages;
+	}
+
+	fd_install(fd, file);
+	return fd;
+
+err_free_pages:
+	free_contig_range(page_to_pfn(pages), npages);
+err_free_info:
+	kfree(info);
+	return ret;
+}
+
+static long mp_ctl_ioctl(struct file *file, unsigned int cmd,
+			 unsigned long arg)
+{
+	void __user *uarg = (void __user *)arg;
+
+	switch (cmd) {
+	case MP_NEW_FILE:
+		return mp_ctl_new_file(uarg);
+	default:
+		return -ENOTTY;
+	}
+}
 
 static const struct file_operations mp_ctl_fops = {
 	.owner		= THIS_MODULE,
+	.unlocked_ioctl	= mp_ctl_ioctl,
+	.compat_ioctl	= compat_ptr_ioctl,
 };
 
 static struct miscdevice mp_dev = {
diff --git a/samples/kvm/memory_provider.h b/samples/kvm/memory_provider.h
new file mode 100644
index 0000000000000..0bd9e250aeaf5
--- /dev/null
+++ b/samples/kvm/memory_provider.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _SAMPLES_KVM_MEMORY_PROVIDER_H
+#define _SAMPLES_KVM_MEMORY_PROVIDER_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+/*
+ * A sample memory provider.
+ *
+ * Every fd that this module returns is a provider file. Pass it to
+ * KVM_CREATE_GUEST_MEMFD.
+ */
+
+#define MP_IOCTL_BASE		'P'
+
+/*
+ * ioctl on /dev/memory_provider: create a new provider fd of @size.
+ * Returns the provider fd.
+ */
+struct mp_new_file {
+	__u32 flags;	/* Reserved, must be 0 */
+	__u32 pad;
+	__u64 size;	/* bytes, page aligned */
+};
+
+#define MP_NEW_FILE \
+	_IOW(MP_IOCTL_BASE, 5, struct mp_new_file)
+
+#endif /* _SAMPLES_KVM_MEMORY_PROVIDER_H */

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 22/88] samples/kvm: Support marking page ranges present or absent

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Support the MP_SET_PRESENT ioctl on provider files to allow userspace to
dynamically mark page-aligned ranges of a provider file as present or
absent.

Track absent pages using a zero-initialized bitmap so that the entire
physical page range starts in the present state upon creation. Serialize
bitmap updates with a per-file mutex so concurrent ioctl calls do not race
when modifying page presence state.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/kvm/memory_provider.c | 83 ++++++++++++++++++++++++++++++++++++++++++-
 samples/kvm/memory_provider.h | 15 ++++++++
 2 files changed, 97 insertions(+), 1 deletion(-)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index e89ce55a357c0..6b8364afc5ed9 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -6,18 +6,21 @@
  *
  * The control device /dev/memory_provider creates provider files.
  * Each file allocates its own contiguous memory from alloc_contig_pages().
+ * Pages can be dynamically marked present or absent.
  * See memory_provider.h.
  *
  * Confidential VMs are not supported.
  */
 
 #include <linux/anon_inodes.h>
+#include <linux/bitmap.h>
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/gfp.h>
 #include <linux/miscdevice.h>
 #include <linux/mm.h>
 #include <linux/module.h>
+#include <linux/mutex.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
 
@@ -30,21 +33,88 @@ struct mp_info {
 	unsigned long base_pfn;
 	unsigned long npages;
 	struct page *cma_pages;		/* from alloc_contig_pages() */
+
+	struct mutex lock;		/* the bitmaps */
+	unsigned long *absent;		/* taken away by MP_SET_PRESENT */
 };
 
+/* Byte range [offset, offset + len) of @info as page indices. */
+static int mp_range(struct mp_info *info, u64 offset, u64 len,
+		    unsigned long *start, unsigned long *end)
+{
+	if (!len || !PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len))
+		return -EINVAL;
+
+	*start = offset >> PAGE_SHIFT;
+	*end = *start + (len >> PAGE_SHIFT);
+
+	if (*end > info->npages || *end < *start)
+		return -EINVAL;
+
+	return 0;
+}
+
+static void mp_set_present(struct mp_info *info, unsigned long start,
+			   unsigned long end, bool present)
+{
+	mutex_lock(&info->lock);
+
+	if (present)
+		bitmap_clear(info->absent, start, end - start);
+	else
+		bitmap_set(info->absent, start, end - start);
+
+	mutex_unlock(&info->lock);
+}
+
 static int mp_file_release(struct inode *inode, struct file *file)
 {
 	struct mp_info *info = file->private_data;
 
 	if (info->cma_pages)
 		free_contig_range(info->base_pfn, info->npages);
+	bitmap_free(info->absent);
 	kfree(info);
 	return 0;
 }
 
+static long mp_file_set_present(struct mp_info *info, void __user *uarg)
+{
+	struct mp_set_present sp;
+	unsigned long start, end;
+	int ret;
+
+	if (copy_from_user(&sp, uarg, sizeof(sp)))
+		return -EFAULT;
+	if (sp.value > 1 || sp.pad)
+		return -EINVAL;
+
+	ret = mp_range(info, sp.offset, sp.len, &start, &end);
+	if (!ret)
+		mp_set_present(info, start, end, sp.value);
+
+	return ret;
+}
+
+static long mp_file_ioctl(struct file *file, unsigned int cmd,
+			  unsigned long arg)
+{
+	struct mp_info *info = file->private_data;
+	void __user *uarg = (void __user *)arg;
+
+	switch (cmd) {
+	case MP_SET_PRESENT:
+		return mp_file_set_present(info, uarg);
+	default:
+		return -ENOTTY;
+	}
+}
+
 static const struct file_operations mp_file_fops = {
 	.owner		= THIS_MODULE,
 	.release	= mp_file_release,
+	.unlocked_ioctl	= mp_file_ioctl,
+	.compat_ioctl	= compat_ptr_ioctl,
 };
 
 static long mp_ctl_new_file(void __user *uarg)
@@ -64,21 +134,30 @@ static long mp_ctl_new_file(void __user *uarg)
 		return -EINVAL;
 
 	npages = nf.size >> PAGE_SHIFT;
+	if (npages > KMALLOC_MAX_SIZE * BITS_PER_BYTE)
+		return -EINVAL;
 
 	info = kzalloc_obj(*info);
 	if (!info)
 		return -ENOMEM;
 
+	info->absent = bitmap_zalloc(npages, GFP_KERNEL);
+	if (!info->absent) {
+		ret = -ENOMEM;
+		goto err_free_info;
+	}
+
 	pages = alloc_contig_pages(npages, GFP_KERNEL | __GFP_ZERO,
 				   numa_node_id(), NULL);
 	if (!pages) {
 		ret = -ENOMEM;
-		goto err_free_info;
+		goto err_free_absent;
 	}
 
 	info->cma_pages = pages;
 	info->base_pfn = page_to_pfn(pages);
 	info->npages = npages;
+	mutex_init(&info->lock);
 
 	fd = get_unused_fd_flags(O_CLOEXEC);
 	if (fd < 0) {
@@ -99,6 +178,8 @@ static long mp_ctl_new_file(void __user *uarg)
 
 err_free_pages:
 	free_contig_range(page_to_pfn(pages), npages);
+err_free_absent:
+	bitmap_free(info->absent);
 err_free_info:
 	kfree(info);
 	return ret;
diff --git a/samples/kvm/memory_provider.h b/samples/kvm/memory_provider.h
index 0bd9e250aeaf5..de07ffa482283 100644
--- a/samples/kvm/memory_provider.h
+++ b/samples/kvm/memory_provider.h
@@ -27,4 +27,19 @@ struct mp_new_file {
 #define MP_NEW_FILE \
 	_IOW(MP_IOCTL_BASE, 5, struct mp_new_file)
 
+/*
+ * ioctl on a provider fd: SET_PRESENT on [@offset, @offset + @len).
+ * @value is the present value (1 = present, 0 = absent), and @offset
+ * and @len are a byte range within the file, page aligned.
+ */
+struct mp_set_present {
+	__u32 value;
+	__u32 pad;
+	__u64 offset;	/* bytes into the file, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MP_SET_PRESENT \
+	_IOW(MP_IOCTL_BASE, 12, struct mp_set_present)
+
 #endif /* _SAMPLES_KVM_MEMORY_PROVIDER_H */

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 23/88] samples/kvm: Support marking page ranges read-only

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Support the MP_SET_READONLY ioctl on provider files to allow userspace to
dynamically mark page-aligned ranges of a provider file as read-only or
writable.

Track read-only pages in a per-file bitmap that starts cleared so all pages
default to writable upon creation.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/kvm/memory_provider.c | 45 ++++++++++++++++++++++++++++++++++++++++++-
 samples/kvm/memory_provider.h | 15 +++++++++++++++
 2 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 6b8364afc5ed9..39aeb1a0b93f6 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -36,6 +36,7 @@ struct mp_info {
 
 	struct mutex lock;		/* the bitmaps */
 	unsigned long *absent;		/* taken away by MP_SET_PRESENT */
+	unsigned long *readonly;	/* marked read-only by MP_SET_READONLY */
 };
 
 /* Byte range [offset, offset + len) of @info as page indices. */
@@ -54,6 +55,19 @@ static int mp_range(struct mp_info *info, u64 offset, u64 len,
 	return 0;
 }
 
+static void mp_set_readonly(struct mp_info *info, unsigned long start,
+			    unsigned long end, bool readonly)
+{
+	mutex_lock(&info->lock);
+
+	if (readonly)
+		bitmap_set(info->readonly, start, end - start);
+	else
+		bitmap_clear(info->readonly, start, end - start);
+
+	mutex_unlock(&info->lock);
+}
+
 static void mp_set_present(struct mp_info *info, unsigned long start,
 			   unsigned long end, bool present)
 {
@@ -73,11 +87,30 @@ static int mp_file_release(struct inode *inode, struct file *file)
 
 	if (info->cma_pages)
 		free_contig_range(info->base_pfn, info->npages);
+	bitmap_free(info->readonly);
 	bitmap_free(info->absent);
 	kfree(info);
 	return 0;
 }
 
+static long mp_file_set_readonly(struct mp_info *info, void __user *uarg)
+{
+	struct mp_set_readonly sr;
+	unsigned long start, end;
+	int ret;
+
+	if (copy_from_user(&sr, uarg, sizeof(sr)))
+		return -EFAULT;
+	if (sr.value > 1 || sr.pad)
+		return -EINVAL;
+
+	ret = mp_range(info, sr.offset, sr.len, &start, &end);
+	if (!ret)
+		mp_set_readonly(info, start, end, sr.value);
+
+	return ret;
+}
+
 static long mp_file_set_present(struct mp_info *info, void __user *uarg)
 {
 	struct mp_set_present sp;
@@ -105,6 +138,8 @@ static long mp_file_ioctl(struct file *file, unsigned int cmd,
 	switch (cmd) {
 	case MP_SET_PRESENT:
 		return mp_file_set_present(info, uarg);
+	case MP_SET_READONLY:
+		return mp_file_set_readonly(info, uarg);
 	default:
 		return -ENOTTY;
 	}
@@ -147,11 +182,17 @@ static long mp_ctl_new_file(void __user *uarg)
 		goto err_free_info;
 	}
 
+	info->readonly = bitmap_zalloc(npages, GFP_KERNEL);
+	if (!info->readonly) {
+		ret = -ENOMEM;
+		goto err_free_absent;
+	}
+
 	pages = alloc_contig_pages(npages, GFP_KERNEL | __GFP_ZERO,
 				   numa_node_id(), NULL);
 	if (!pages) {
 		ret = -ENOMEM;
-		goto err_free_absent;
+		goto err_free_readonly;
 	}
 
 	info->cma_pages = pages;
@@ -178,6 +219,8 @@ static long mp_ctl_new_file(void __user *uarg)
 
 err_free_pages:
 	free_contig_range(page_to_pfn(pages), npages);
+err_free_readonly:
+	bitmap_free(info->readonly);
 err_free_absent:
 	bitmap_free(info->absent);
 err_free_info:
diff --git a/samples/kvm/memory_provider.h b/samples/kvm/memory_provider.h
index de07ffa482283..f3772f57f5bf5 100644
--- a/samples/kvm/memory_provider.h
+++ b/samples/kvm/memory_provider.h
@@ -27,6 +27,21 @@ struct mp_new_file {
 #define MP_NEW_FILE \
 	_IOW(MP_IOCTL_BASE, 5, struct mp_new_file)
 
+/*
+ * ioctl on a provider fd: SET_READONLY on [@offset, @offset + @len).
+ * @value is the read-only value (1 = read-only, 0 = writable), and @offset
+ * and @len are a byte range within the file, page aligned.
+ */
+struct mp_set_readonly {
+	__u32 value;
+	__u32 pad;
+	__u64 offset;	/* bytes into the file, page aligned */
+	__u64 len;	/* bytes, page aligned */
+};
+
+#define MP_SET_READONLY \
+	_IOW(MP_IOCTL_BASE, 11, struct mp_set_readonly)
+
 /*
  * ioctl on a provider fd: SET_PRESENT on [@offset, @offset + @len).
  * @value is the present value (1 = present, 0 = absent), and @offset

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 24/88] samples/kvm: Support querying provider file statistics

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Support the MP_GET_STATS ioctl on provider files so userspace and selftests
can inspect the total capacity, number of absent pages, and number of
read-only pages of a provider file.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 samples/kvm/memory_provider.c | 16 ++++++++++++++++
 samples/kvm/memory_provider.h | 10 ++++++++++
 2 files changed, 26 insertions(+)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 39aeb1a0b93f6..81ac94ae1a4e9 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -81,6 +81,20 @@ static void mp_set_present(struct mp_info *info, unsigned long start,
 	mutex_unlock(&info->lock);
 }
 
+static long mp_get_stats(struct mp_info *info, void __user *uarg)
+{
+	struct mp_stats st = {
+		.size = (u64)info->npages << PAGE_SHIFT,
+	};
+
+	mutex_lock(&info->lock);
+	st.absent_pages = bitmap_weight(info->absent, info->npages);
+	st.readonly_pages = bitmap_weight(info->readonly, info->npages);
+	mutex_unlock(&info->lock);
+
+	return copy_to_user(uarg, &st, sizeof(st)) ? -EFAULT : 0;
+}
+
 static int mp_file_release(struct inode *inode, struct file *file)
 {
 	struct mp_info *info = file->private_data;
@@ -136,6 +150,8 @@ static long mp_file_ioctl(struct file *file, unsigned int cmd,
 	void __user *uarg = (void __user *)arg;
 
 	switch (cmd) {
+	case MP_GET_STATS:
+		return mp_get_stats(info, uarg);
 	case MP_SET_PRESENT:
 		return mp_file_set_present(info, uarg);
 	case MP_SET_READONLY:
diff --git a/samples/kvm/memory_provider.h b/samples/kvm/memory_provider.h
index f3772f57f5bf5..24a201a58e83b 100644
--- a/samples/kvm/memory_provider.h
+++ b/samples/kvm/memory_provider.h
@@ -27,6 +27,16 @@ struct mp_new_file {
 #define MP_NEW_FILE \
 	_IOW(MP_IOCTL_BASE, 5, struct mp_new_file)
 
+/* ioctl on a provider fd: read the file's state, for tests. */
+struct mp_stats {
+	__u64 size;
+	__u64 absent_pages;	/* pages taken away by MP_SET_PRESENT */
+	__u64 readonly_pages;	/* pages marked read-only by MP_SET_READONLY */
+};
+
+#define MP_GET_STATS \
+	_IOR(MP_IOCTL_BASE, 10, struct mp_stats)
+
 /*
  * ioctl on a provider fd: SET_READONLY on [@offset, @offset + @len).
  * @value is the read-only value (1 = read-only, 0 = writable), and @offset

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 25/88] KVM: selftests: Add basic test for sample memory provider

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a selftest for the sample memory provider to verify lifecycle
management and file creation.

Verify creating provider files across multiple sizes and ensure that
initial statistics accurately reflect file capacity, zero absent pages,
and zero read-only pages.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/Makefile.kvm           |  1 +
 .../selftests/kvm/mem_provider_sample_test.c       | 85 ++++++++++++++++++++++
 2 files changed, 86 insertions(+)

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 17a4256e90819..5f7118bddfc36 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -72,6 +72,7 @@ TEST_GEN_PROGS_COMMON += set_memory_region_test
 TEST_GEN_PROGS_COMMON += memslot_modification_stress_test
 TEST_GEN_PROGS_COMMON += memslot_perf_test
 TEST_GEN_PROGS_COMMON += vm_types_test
+TEST_GEN_PROGS_COMMON += mem_provider_sample_test
 
 # Compiled test targets
 TEST_GEN_PROGS_x86 = $(TEST_GEN_PROGS_COMMON)
diff --git a/tools/testing/selftests/kvm/mem_provider_sample_test.c b/tools/testing/selftests/kvm/mem_provider_sample_test.c
new file mode 100644
index 0000000000000..b39161f4933d3
--- /dev/null
+++ b/tools/testing/selftests/kvm/mem_provider_sample_test.c
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Tests for the sample memory provider (/dev/memory_provider).
+ */
+#include <fcntl.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/ioctl.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "test_util.h"
+#include "../../../../samples/kvm/memory_provider.h"
+
+#define DEV_PATH "/dev/memory_provider"
+
+static int open_provider_dev(void)
+{
+	int fd = open(DEV_PATH, O_RDWR);
+
+	if (fd < 0) {
+		TEST_REQUIRE(errno != ENOENT && errno != ENODEV);
+		TEST_FAIL("Failed to open %s: %d (%s)", DEV_PATH, errno,
+			  strerror(errno));
+	}
+	return fd;
+}
+
+static int create_provider_file(int dev_fd, uint64_t size)
+{
+	struct mp_new_file nf = {
+		.size = size,
+	};
+	int fd;
+
+	fd = ioctl(dev_fd, MP_NEW_FILE, &nf);
+	TEST_ASSERT(fd >= 0, "MP_NEW_FILE failed for size 0x%lx: %d (%s)",
+		    size, errno, strerror(errno));
+	return fd;
+}
+
+static void test_lifecycle_and_creation(int dev_fd)
+{
+	size_t page_size = getpagesize();
+	uint64_t sizes[] = {
+		page_size,
+		2 * page_size,
+		2 * 1024 * 1024,
+	};
+	struct mp_stats st;
+	int i, fd;
+
+	for (i = 0; i < ARRAY_SIZE(sizes); i++) {
+		fd = create_provider_file(dev_fd, sizes[i]);
+
+		memset(&st, 0, sizeof(st));
+		TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+			    "MP_GET_STATS failed: %d (%s)",
+			    errno, strerror(errno));
+		TEST_ASSERT(st.size == sizes[i],
+			    "size mismatch: expected 0x%lx, got 0x%llx",
+			    sizes[i], st.size);
+		TEST_ASSERT(st.absent_pages == 0,
+			    "absent_pages mismatch: expected 0, got %llu",
+			    st.absent_pages);
+		TEST_ASSERT(st.readonly_pages == 0,
+			    "readonly_pages mismatch: expected 0, got %llu",
+			    st.readonly_pages);
+
+		close(fd);
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	int dev_fd;
+
+	dev_fd = open_provider_dev();
+
+	test_lifecycle_and_creation(dev_fd);
+
+	close(dev_fd);
+	return 0;
+}

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 26/88] KVM: selftests: Test marking page ranges present and absent

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Test dynamically changing page presence in provider files via the
sample memory provider interface.

Verify marking pages absent and restoring them back to present, ensuring
that absent page counts in statistics reflect each change accurately.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/mem_provider_sample_test.c       | 65 ++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/tools/testing/selftests/kvm/mem_provider_sample_test.c b/tools/testing/selftests/kvm/mem_provider_sample_test.c
index b39161f4933d3..8fe634b0b4f2e 100644
--- a/tools/testing/selftests/kvm/mem_provider_sample_test.c
+++ b/tools/testing/selftests/kvm/mem_provider_sample_test.c
@@ -72,6 +72,70 @@ static void test_lifecycle_and_creation(int dev_fd)
 	}
 }
 
+static void test_set_present(int dev_fd)
+{
+	size_t page_size = getpagesize();
+	uint64_t nr_pages = 4;
+	uint64_t file_size = nr_pages * page_size;
+	struct mp_set_present sp;
+	struct mp_stats st;
+	int ret, fd;
+
+	fd = create_provider_file(dev_fd, file_size);
+
+	/* Initially all pages are present */
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.absent_pages == 0,
+		    "Initial absent_pages mismatch: expected 0, got %llu",
+		    st.absent_pages);
+
+	/* Mark page 1 absent */
+	memset(&sp, 0, sizeof(sp));
+	sp.value = 0;
+	sp.offset = page_size;
+	sp.len = page_size;
+	ret = ioctl(fd, MP_SET_PRESENT, &sp);
+	TEST_ASSERT(!ret, "MP_SET_PRESENT (absent) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.size == file_size,
+		    "size changed unexpectedly: 0x%llx", st.size);
+	TEST_ASSERT(st.absent_pages == 1,
+		    "absent_pages mismatch after absent: expected 1, got %llu",
+		    st.absent_pages);
+
+	/* Mark page 1 back to present */
+	sp.value = 1;
+	ret = ioctl(fd, MP_SET_PRESENT, &sp);
+	TEST_ASSERT(!ret, "MP_SET_PRESENT (present) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.absent_pages == 0,
+		    "absent_pages mismatch after restore: expected 0, got %llu",
+		    st.absent_pages);
+
+	/* Mark all pages absent */
+	sp.value = 0;
+	sp.offset = 0;
+	sp.len = file_size;
+	ret = ioctl(fd, MP_SET_PRESENT, &sp);
+	TEST_ASSERT(!ret, "MP_SET_PRESENT (all absent) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.absent_pages == nr_pages,
+		    "absent_pages mismatch after all absent: expected %lu, got %llu",
+		    nr_pages, st.absent_pages);
+
+	close(fd);
+}
+
 int main(int argc, char *argv[])
 {
 	int dev_fd;
@@ -79,6 +143,7 @@ int main(int argc, char *argv[])
 	dev_fd = open_provider_dev();
 
 	test_lifecycle_and_creation(dev_fd);
+	test_set_present(dev_fd);
 
 	close(dev_fd);
 	return 0;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 27/88] KVM: selftests: Test marking page ranges read-only

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Test dynamically marking page ranges read-only and writable via the
sample memory provider interface.

Verify changing page write permissions and ensure that read-only page
counts in statistics reflect each change accurately.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/mem_provider_sample_test.c       | 65 ++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/tools/testing/selftests/kvm/mem_provider_sample_test.c b/tools/testing/selftests/kvm/mem_provider_sample_test.c
index 8fe634b0b4f2e..942c0519836de 100644
--- a/tools/testing/selftests/kvm/mem_provider_sample_test.c
+++ b/tools/testing/selftests/kvm/mem_provider_sample_test.c
@@ -136,6 +136,70 @@ static void test_set_present(int dev_fd)
 	close(fd);
 }
 
+static void test_set_readonly(int dev_fd)
+{
+	size_t page_size = getpagesize();
+	uint64_t nr_pages = 4;
+	uint64_t file_size = nr_pages * page_size;
+	struct mp_set_readonly sr;
+	struct mp_stats st;
+	int ret, fd;
+
+	fd = create_provider_file(dev_fd, file_size);
+
+	/* Initially all pages are writable */
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.readonly_pages == 0,
+		    "Initial readonly_pages mismatch: expected 0, got %llu",
+		    st.readonly_pages);
+
+	/* Mark page 1 read-only */
+	memset(&sr, 0, sizeof(sr));
+	sr.value = 1;
+	sr.offset = page_size;
+	sr.len = page_size;
+	ret = ioctl(fd, MP_SET_READONLY, &sr);
+	TEST_ASSERT(!ret, "MP_SET_READONLY (read-only) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.size == file_size,
+		    "size changed unexpectedly: 0x%llx", st.size);
+	TEST_ASSERT(st.readonly_pages == 1,
+		    "readonly_pages mismatch after read-only: expected 1, got %llu",
+		    st.readonly_pages);
+
+	/* Mark page 1 back to writable */
+	sr.value = 0;
+	ret = ioctl(fd, MP_SET_READONLY, &sr);
+	TEST_ASSERT(!ret, "MP_SET_READONLY (writable) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.readonly_pages == 0,
+		    "readonly_pages mismatch after writable: expected 0, got %llu",
+		    st.readonly_pages);
+
+	/* Mark all pages read-only */
+	sr.value = 1;
+	sr.offset = 0;
+	sr.len = file_size;
+	ret = ioctl(fd, MP_SET_READONLY, &sr);
+	TEST_ASSERT(!ret, "MP_SET_READONLY (all read-only) failed: %d (%s)",
+		    errno, strerror(errno));
+
+	TEST_ASSERT(!ioctl(fd, MP_GET_STATS, &st),
+		    "MP_GET_STATS failed: %d (%s)", errno, strerror(errno));
+	TEST_ASSERT(st.readonly_pages == nr_pages,
+		    "readonly_pages mismatch after all read-only: expected %lu, got %llu",
+		    nr_pages, st.readonly_pages);
+
+	close(fd);
+}
+
 int main(int argc, char *argv[])
 {
 	int dev_fd;
@@ -144,6 +208,7 @@ int main(int argc, char *argv[])
 
 	test_lifecycle_and_creation(dev_fd);
 	test_set_present(dev_fd);
+	test_set_readonly(dev_fd);
 
 	close(dev_fd);
 	return 0;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 28/88] KVM: guest_memfd: Support reporting PFN attributes

[Ackerley Tng via B4 Relay]

From: Fred Griffoul <fgriffo@amazon.co.uk>

Support reporting physical frame attributes when retrieving PFNs from
guest_memfd backends.

Add a pfn_attrs output parameter to kvm_gmem_get_pfn and
__kvm_gmem_get_pfn. Define GUEST_MEMFD_PFN_ATTRS_READONLY as the initial
attribute bit so backends can report read-only restrictions on physical
memory ranges.

Handle GUEST_MEMFD_PFN_ATTRS_READONLY across stage 2 fault handlers in
x86 and arm64 to ensure that guest page table mappings prevent writes
when the backing physical memory is marked read-only. Exit to userspace
with a memory fault if the guest attempts to write to a read-only
frame.

Ensure that private memory invariants are preserved: reject read-only
frames for private memory faults with -EFAULT, and reject populating
read-only frames with -EPERM.

Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
---
 arch/arm64/kvm/mmu.c        | 13 +++++++++----
 arch/arm64/kvm/nested.c     | 14 ++++++++++++--
 arch/x86/kvm/mmu/mmu.c      | 10 ++++++++--
 arch/x86/kvm/svm/sev.c      | 15 +++++++++++++--
 include/linux/guest_memfd.h |  2 ++
 include/linux/kvm_host.h    |  7 +++++--
 virt/kvm/guest_memfd.c      | 32 ++++++++++++++++++++++++++++----
 7 files changed, 77 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 705f6d84770e9..8f0217185973c 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1738,6 +1738,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd,
 {
 	bool write_fault, exec_fault;
 	bool perm_fault = kvm_s2_fault_is_perm(s2fd);
+	bool writable;
 	enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
 	enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
 	struct kvm_pgtable *pgt = s2fd->mmu->pgt;
@@ -1746,6 +1747,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd,
 	struct kvm *kvm = s2fd->vcpu->kvm;
 	void *memcache = NULL;
 	gfn_t canonical_gfn;
+	u64 pfn_attrs = 0;
 	int ret;
 
 	if (!perm_fault) {
@@ -1773,18 +1775,21 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd,
 	/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
 	smp_rmb();
 
-	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, NULL);
-	if (ret) {
+	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, NULL,
+			       &pfn_attrs);
+	writable = !(s2fd->memslot->flags & KVM_MEM_READONLY) &&
+		   !(pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY);
+	if (ret || (write_fault && !writable)) {
 		/* If result is non-NULL this is a synthetic fault. */
 		if (!result)
 			kvm_prepare_memory_fault_exit(s2fd->vcpu, gfn_to_gpa(canonical_gfn),
 						      s2vi.vma_pagesize, write_fault, exec_fault,
 						      false);
 		kfree(mapping);
-		return ret;
+		return ret ?: -EFAULT;
 	}
 
-	if (!(s2fd->memslot->flags & KVM_MEM_READONLY))
+	if (writable)
 		prot |= KVM_PGTABLE_PROT_W;
 
 	if (s2fd->nested)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 4d83281a7a28b..4519e92d89d66 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1615,14 +1615,24 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 			return -EFAULT;
 		}
 	} else {
-		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, NULL);
+		u64 pfn_attrs;
+
+		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, NULL,
+				       &pfn_attrs);
 		if (ret) {
 			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
 					      write_fault, false, false);
 			return ret;
 		}
 
-		writable = !(memslot->flags & KVM_MEM_READONLY);
+		writable = !(memslot->flags & KVM_MEM_READONLY) &&
+			   !(pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY);
+		/* TODO: Explain why vt->wr.pw needs to be checked. */
+		if (write_fault && vt->wr.pw && !writable) {
+			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
+						      true, false, false);
+			return -EFAULT;
+		}
 	}
 
 	/*
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 5a2cc6a17a9b3..87deeeffd9919 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4674,6 +4674,7 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu,
 static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 				    struct kvm_page_fault *fault)
 {
+	u64 pfn_attrs;
 	int max_order, r;
 
 	if (!kvm_slot_has_gmem(fault->slot)) {
@@ -4682,13 +4683,18 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 	}
 
 	r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn,
-			     &max_order);
+			     &max_order, &pfn_attrs);
 	if (r) {
 		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
 		return r;
 	}
 
-	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY);
+	fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY) &&
+			      !(pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY);
+	if (fault->write && !fault->map_writable) {
+		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
+		return -EFAULT;
+	}
 	fault->max_level = kvm_max_level_for_order(max_order);
 
 	return RET_PF_CONTINUE;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 6427db390b1cd..5e37ea4bc7937 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4053,6 +4053,7 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	struct kvm *kvm = vcpu->kvm;
 	gfn_t gfn = gpa_to_gfn(gpa);
 	unsigned long mmu_seq;
+	u64 pfn_attrs;
 	kvm_pfn_t pfn;
 
 	lockdep_assert_held(&svm->sev_es.snp_vmsa_mutex);
@@ -4096,7 +4097,10 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	 * The new VMSA will be private memory guest memory, so retrieve the
 	 * PFN from the gmem backend.
 	 */
-	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL))
+	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL, &pfn_attrs))
+		return;
+
+	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)
 		return;
 
 	read_lock(&kvm->mmu_lock);
@@ -5063,6 +5067,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 	struct kvm *kvm = vcpu->kvm;
 	int order, rmp_level, ret;
 	unsigned long mmu_seq;
+	u64 pfn_attrs;
 	bool assigned;
 	kvm_pfn_t pfn;
 	gfn_t gfn;
@@ -5092,13 +5097,19 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 	mmu_seq = kvm->mmu_invalidate_seq;
 	smp_rmb();
 
-	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order);
+	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order, &pfn_attrs);
 	if (ret) {
 		pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n",
 				    gpa);
 		return;
 	}
 
+	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) {
+		pr_warn_ratelimited("SEV: Unexpected RMP fault, read-only backing page for GPA 0x%llx\n",
+				    gpa);
+		return;
+	}
+
 	ret = snp_lookup_rmpentry(pfn, &assigned, &rmp_level);
 	if (ret || !assigned) {
 		guard(read_lock)(&kvm->mmu_lock);
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index f3e6077c39ac3..df2db2a0bae6e 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -9,6 +9,8 @@ struct folio;
 struct inode;
 struct mempolicy;
 
+#define GUEST_MEMFD_PFN_ATTRS_READONLY	(1ULL << 0)
+
 /**
  * struct guest_memfd_provider_operations - Operations for external memory providers
  * @attach: Attach to a resource file, perform filesystem-specific validation,
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index d820afac49252..cf679efcf3260 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -35,6 +35,7 @@
 #include <linux/interval_tree.h>
 #include <linux/rbtree.h>
 #include <linux/xarray.h>
+#include <linux/guest_memfd.h>
 #include <asm/signal.h>
 
 #include <linux/kvm.h>
@@ -2627,11 +2628,13 @@ static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
 }
 
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
-		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order);
+		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order,
+		     u64 *pfn_attrs);
 #else
 static inline int kvm_gmem_get_pfn(struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
-				   kvm_pfn_t *pfn, int *max_order)
+				   kvm_pfn_t *pfn, int *max_order,
+				   u64 *pfn_attrs)
 {
 	KVM_BUG_ON(1, kvm);
 	return -EIO;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 84877beed6701..f0bd90fd9c986 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1274,7 +1274,7 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
 static struct folio *__kvm_gmem_get_pfn(struct file *file,
 					struct kvm_memory_slot *slot,
 					pgoff_t index, kvm_pfn_t *pfn,
-					int *max_order)
+					int *max_order, u64 *pfn_attrs)
 {
 	struct file *slot_file = READ_ONCE(slot->gmem.file);
 	struct gmem_file *f = file->private_data;
@@ -1308,18 +1308,23 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
 	*pfn = folio_file_pfn(folio, index);
 	if (max_order)
 		*max_order = 0;
+	if (pfn_attrs)
+		*pfn_attrs = 0;
 
 	return folio;
 }
 
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
-		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order)
+		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order,
+		     u64 *pfn_attrs)
 {
 	pgoff_t index = kvm_gmem_get_index(slot, gfn);
 	struct folio *folio;
 	int r = 0, __order;
+	u64 __attrs;
 
 	max_order = max_order ?: &__order;
+	pfn_attrs = pfn_attrs ?: &__attrs;
 
 	CLASS(gmem_get_file, file)(slot);
 	if (!file)
@@ -1327,17 +1332,29 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 	filemap_invalidate_lock_shared(file_inode(file)->i_mapping);
 
-	folio = __kvm_gmem_get_pfn(file, slot, index, pfn, max_order);
+	folio = __kvm_gmem_get_pfn(file, slot, index, pfn, max_order,
+				   pfn_attrs);
 	if (IS_ERR(folio)) {
 		r = PTR_ERR(folio);
 		goto out;
 	}
 
+	/*
+	 * Private memory requires exclusive, writable guest ownership and
+	 * cannot be backed by a read-only frame.
+	 */
+	if (kvm_gmem_is_private_mem(file_inode(file), index) &&
+	    (*pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)) {
+		r = -EFAULT;
+		goto out_put_folio;
+	}
+
 	if (kvm_arch_has_gmem_convert() &&
 	    kvm_gmem_is_private_mem(file_inode(file), index))
 		r = kvm_arch_gmem_make_private(kvm, gfn, *pfn,
 					       (kvm_pfn_t)1 << *max_order);
 
+out_put_folio:
 	folio_unlock(folio);
 	folio_put(folio);
 
@@ -1355,12 +1372,13 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 {
 	pgoff_t index = kvm_gmem_get_index(slot, gfn);
 	struct folio *folio;
+	u64 pfn_attrs;
 	kvm_pfn_t pfn;
 	int ret;
 
 	filemap_invalidate_lock(file->f_mapping);
 
-	folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, NULL);
+	folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, NULL, &pfn_attrs);
 	if (IS_ERR(folio)) {
 		ret = PTR_ERR(folio);
 		goto out_unlock;
@@ -1368,6 +1386,12 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 	folio_unlock(folio);
 
+	/* Populating pages initializes private memory, which cannot be read-only. */
+	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) {
+		ret = -EPERM;
+		goto out_put_folio;
+	}
+
 	if (!kvm_is_private_gfn(kvm, gfn)) {
 		ret = -EINVAL;
 		goto out_put_folio;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 29/88] KVM: guest_memfd: Support getting PFNs for stage 2 from providers

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add .get_pfn callback to struct guest_memfd_provider_operations to
support physical memory providers (e.g. devices, DAX, or pre-allocated
ranges) that supply PFNs directly without folio allocation or page cache
insertion.

Define GUEST_MEMFD_FLAG_PFN_MODE in uapi/linux/kvm.h and define
kvm_gmem_is_pfn_mode() to check for this flag in gi->flags.

In __kvm_gmem_get_pfn(), query .get_pfn() directly when in PFN mode
and return NULL on success (instead of a struct folio *).

Note that .alloc_folio is still required by kvm_gmem_attach_resource() and
GUEST_MEMFD_FLAG_PFN_MODE is not yet enabled for userspace.  Subsequent
patches will update the rest of guest_memfd (e.g. mmap, fallocate,
truncation, and attributes) to support PFN mode before enabling the flag
and allowing external PFN-backed providers to be attached.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h | 11 +++++++++++
 include/uapi/linux/kvm.h    |  1 +
 virt/kvm/guest_memfd.c      | 48 ++++++++++++++++++++++++++++++++++++++++-----
 3 files changed, 55 insertions(+), 5 deletions(-)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index df2db2a0bae6e..9fdc0bd1a7190 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -20,6 +20,15 @@ struct mempolicy;
  *               Must return a pointer to a struct folio on success, or an
  *               ERR_PTR() error code on failure. Must never return NULL.
  * @invalidate_folio: Notify provider that a folio has been invalidated.
+ * @get_pfn: Obtain a PFN for a given index. The provider must ensure backing
+ *           memory is present and is responsible for its contents, such as
+ *           zeroing if necessary. On success, return 0 with the physical frame
+ *           number in @pfn and, if non-NULL, the PFN attributes in @pfn_attrs
+ *           and the maximum page order of contiguous backing memory containing
+ *           @pfn with uniform @pfn_attrs in @max_order. Once @get_pfn succeeds,
+ *           the provider must keep the @index-to-@pfn mapping and @pfn_attrs
+ *           constant, and the memory accessible, until guest_memfd stops using
+ *           this page. Returns a negative error code on failure.
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -29,6 +38,8 @@ struct guest_memfd_provider_operations {
 	struct folio *(*alloc_folio)(void *provider, pgoff_t index,
 				     struct mempolicy *mpol);
 	void (*invalidate_folio)(void *provider, struct folio *folio);
+	int (*get_pfn)(void *provider, pgoff_t index, unsigned long *pfn,
+		       int *max_order, u64 *pfn_attrs);
 };
 
 #endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index f4108bf7e3192..6798056f50e32 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1675,6 +1675,7 @@ struct kvm_memory_attributes2 {
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
 #define GUEST_MEMFD_FLAG_USE_RESOURCE	(1ULL << 2)
+#define GUEST_MEMFD_FLAG_PFN_MODE	(1ULL << 3)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index f0bd90fd9c986..eedf03df12c55 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -54,6 +54,11 @@ static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
 	return container_of(inode, struct gmem_inode, vfs_inode);
 }
 
+static bool kvm_gmem_is_pfn_mode(struct gmem_inode *gi)
+{
+	return gi->flags & GUEST_MEMFD_FLAG_PFN_MODE;
+}
+
 static void *gmem_native_attach(struct inode *gmem_inode,
 				struct file *resource_file)
 {
@@ -124,6 +129,26 @@ static struct folio *gmem_provider_alloc_folio(struct gmem_inode *gi,
 	return ERR_PTR(-EOPNOTSUPP);
 }
 
+static int gmem_provider_get_pfn(struct gmem_inode *gi, pgoff_t index,
+				 kvm_pfn_t *pfn, int *max_order,
+				 u64 *pfn_attrs)
+{
+	unsigned long raw_pfn;
+	u64 __attrs = 0;
+	int r;
+
+	pfn_attrs = pfn_attrs ?: &__attrs;
+	*pfn_attrs = 0;
+
+	r = gi->provider_ops->get_pfn(gi->provider, index, &raw_pfn, max_order,
+				      pfn_attrs);
+	if (r)
+		return r;
+
+	*pfn = raw_pfn;
+	return 0;
+}
+
 static void gmem_provider_invalidate_folio(struct gmem_inode *gi,
 					   struct folio *folio)
 {
@@ -1270,15 +1295,17 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
 	filemap_invalidate_unlock(file->f_mapping);
 }
 
-/* Returns a locked folio on success.  */
+/* On success, return a locked folio in folio mode and NULL in PFN mode. */
 static struct folio *__kvm_gmem_get_pfn(struct file *file,
 					struct kvm_memory_slot *slot,
 					pgoff_t index, kvm_pfn_t *pfn,
 					int *max_order, u64 *pfn_attrs)
 {
 	struct file *slot_file = READ_ONCE(slot->gmem.file);
+	struct gmem_inode *gi = GMEM_I(file_inode(file));
 	struct gmem_file *f = file->private_data;
 	struct folio *folio;
+	int r;
 
 	if (file != slot_file) {
 		WARN_ON_ONCE(slot_file);
@@ -1290,6 +1317,13 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
 		return ERR_PTR(-EIO);
 	}
 
+	if (kvm_gmem_is_pfn_mode(gi)) {
+		r = gmem_provider_get_pfn(gi, index, pfn, max_order, pfn_attrs);
+		if (r)
+			return ERR_PTR(r);
+		return NULL;
+	}
+
 	folio = kvm_gmem_get_folio(file_inode(file), index);
 	if (IS_ERR(folio))
 		return folio;
@@ -1355,8 +1389,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 					       (kvm_pfn_t)1 << *max_order);
 
 out_put_folio:
-	folio_unlock(folio);
-	folio_put(folio);
+	if (folio) {
+		folio_unlock(folio);
+		folio_put(folio);
+	}
 
 out:
 	filemap_invalidate_unlock_shared(file_inode(file)->i_mapping);
@@ -1384,7 +1420,8 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 		goto out_unlock;
 	}
 
-	folio_unlock(folio);
+	if (folio)
+		folio_unlock(folio);
 
 	/* Populating pages initializes private memory, which cannot be read-only. */
 	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) {
@@ -1400,7 +1437,8 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 	ret = post_populate(kvm, gfn, pfn, src_page, opaque);
 
 out_put_folio:
-	folio_put(folio);
+	if (folio)
+		folio_put(folio);
 out_unlock:
 	filemap_invalidate_unlock(file->f_mapping);
 	return ret;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 30/88] KVM: guest_memfd: Refactor error handling in kvm_gmem_fault_user_mapping

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Track the status of kvm_gmem_fault_user_mapping() using an explicit error
code rather than encoding errors into the folio pointer.

This moves away from relying on the folio pointer holding error states,
such as synthesizing ERR_PTR(-EACCES) when memory is not shared only to
extract PTR_ERR afterwards, and prepares kvm_gmem_fault_user_mapping() for
PFN mode in the next commit, where no folio is returned on success.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index eedf03df12c55..180418c65a1ae 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -568,24 +568,28 @@ static bool kvm_gmem_supports_mmap(struct inode *inode)
 static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
 {
 	struct inode *inode = file_inode(vmf->vma->vm_file);
-	struct folio *folio;
 	vm_fault_t ret = VM_FAULT_LOCKED;
+	struct folio *folio = NULL;
+	int r = 0;
 
 	if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
 		return VM_FAULT_SIGBUS;
 
 	filemap_invalidate_lock_shared(inode->i_mapping);
-	if (kvm_gmem_is_shared_mem(inode, vmf->pgoff))
+	if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) {
+		r = -EACCES;
+	} else {
 		folio = kvm_gmem_get_folio(inode, vmf->pgoff);
-	else
-		folio = ERR_PTR(-EACCES);
+		if (IS_ERR(folio))
+			r = PTR_ERR(folio);
+	}
 	filemap_invalidate_unlock_shared(inode->i_mapping);
 
-	if (IS_ERR(folio)) {
-		if (PTR_ERR(folio) == -EAGAIN)
+	if (r) {
+		if (r == -EAGAIN)
 			return VM_FAULT_RETRY;
 
-		return vmf_error(PTR_ERR(folio));
+		return vmf_error(r);
 	}
 
 	if (WARN_ON_ONCE(folio_test_large(folio))) {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 31/88] KVM: guest_memfd: Set VM_PFNMAP on VMA in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In PFN mode, guest_memfd memory is backed by raw PFNs rather than
page-cache folios.

Mark the VMA with VM_PFNMAP when setting up the VMA in PFN mode so that
the core memory management subsystem recognizes the mapping as PFN-backed.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 180418c65a1ae..183e15775523f 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -652,6 +652,8 @@ static const struct vm_operations_struct kvm_gmem_vm_ops = {
 
 static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 {
+	struct gmem_inode *gi = GMEM_I(file_inode(file));
+
 	if (!kvm_gmem_supports_mmap(file_inode(file)))
 		return -ENODEV;
 
@@ -660,6 +662,9 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
+	if (kvm_gmem_is_pfn_mode(gi))
+		vm_flags_set(vma, VM_PFNMAP);
+
 	vma->vm_ops = &kvm_gmem_vm_ops;
 
 	return 0;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 32/88] KVM: guest_memfd: Support user fault handling in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In PFN mode, guest_memfd memory is backed by raw PFNs rather than
page-cache folios.

When inserting the PFN into the user page tables during fault handling,
constrain the inserted protection by stripping write permissions if the
provider reports a read-only PFN attribute, and fail immediate write faults
on read-only frames with VM_FAULT_SIGBUS.

Introduce kvm_gmem_pfn_vm_ops with a .pfn_mkwrite callback that returns
VM_FAULT_SIGBUS to block subsequent write attempts or upgrades to read-only
PFN mappings. Without this, core MM would have marked the PTE writable and
the write would have succeeded.

There is no need to do another .get_pfn() call since the contract with the
provider requires the attributes to be kept constant until memory is
revoked, at which time the user page tables would have been zapped.

In folio mode, the fault handler obtains a locked folio under the shared
filemap invalidation lock and returns with the folio still locked so the
core fault handler installs the PTE before unlocking the folio. Any
concurrent truncation or invalidation must wait on the folio lock before
unmapping and removing the folio.

In PFN mode, there is no folio lock to bridge the gap after dropping the
shared filemap invalidation lock. Hold the shared filemap invalidation lock
across PFN insertion to prevent races with revocation.

Co-developed-by: Fred Griffoul <fgriffo@amazon.co.uk>
Signed-off-by: Fred Griffoul <fgriffo@amazon.co.uk>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 51 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 183e15775523f..e94f2359b7c9a 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -565,11 +565,40 @@ static bool kvm_gmem_supports_mmap(struct inode *inode)
 	return GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_MMAP;
 }
 
+/*
+ * In PFN mode, physical frame attributes are constant for the lifetime of the
+ * mapping. If a page was inserted read-only, it was because the provider
+ * reported a read-only PFN attribute, and write upgrades are never permitted.
+ * Write faults on such pages must fail with SIGBUS.
+ */
+static vm_fault_t kvm_gmem_pfn_mkwrite(struct vm_fault *vmf)
+{
+	return VM_FAULT_SIGBUS;
+}
+
+static vm_fault_t gmem_insert_pfn(struct vm_fault *vmf, kvm_pfn_t pfn,
+				 u64 pfn_attrs)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	vm_flags_t flags = vma->vm_flags;
+
+	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)
+		flags &= ~VM_WRITE;
+
+	if ((vmf->flags & FAULT_FLAG_WRITE) && !(flags & VM_WRITE))
+		return VM_FAULT_SIGBUS;
+
+	return vmf_insert_pfn_prot(vma, vmf->address, pfn,
+				   vm_get_page_prot(flags));
+}
+
 static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
 {
 	struct inode *inode = file_inode(vmf->vma->vm_file);
+	struct gmem_inode *gi = GMEM_I(inode);
 	vm_fault_t ret = VM_FAULT_LOCKED;
 	struct folio *folio = NULL;
+	kvm_pfn_t pfn;
 	int r = 0;
 
 	if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
@@ -578,6 +607,12 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
 	filemap_invalidate_lock_shared(inode->i_mapping);
 	if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) {
 		r = -EACCES;
+	} else if (kvm_gmem_is_pfn_mode(gi)) {
+		u64 pfn_attrs;
+
+		r = gmem_provider_get_pfn(gi, vmf->pgoff, &pfn, NULL, &pfn_attrs);
+		if (!r)
+			ret = gmem_insert_pfn(vmf, pfn, pfn_attrs);
 	} else {
 		folio = kvm_gmem_get_folio(inode, vmf->pgoff);
 		if (IS_ERR(folio))
@@ -592,6 +627,9 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
 		return vmf_error(r);
 	}
 
+	if (kvm_gmem_is_pfn_mode(gi))
+		return ret;
+
 	if (WARN_ON_ONCE(folio_test_large(folio))) {
 		ret = VM_FAULT_SIGBUS;
 		goto out_folio;
@@ -650,6 +688,15 @@ static const struct vm_operations_struct kvm_gmem_vm_ops = {
 #endif
 };
 
+static const struct vm_operations_struct kvm_gmem_pfn_vm_ops = {
+	.fault		= kvm_gmem_fault_user_mapping,
+	.pfn_mkwrite	= kvm_gmem_pfn_mkwrite,
+#ifdef CONFIG_NUMA
+	.get_policy	= kvm_gmem_get_policy,
+	.set_policy	= kvm_gmem_set_policy,
+#endif
+};
+
 static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct gmem_inode *gi = GMEM_I(file_inode(file));
@@ -665,7 +712,10 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 	if (kvm_gmem_is_pfn_mode(gi))
 		vm_flags_set(vma, VM_PFNMAP);
 
-	vma->vm_ops = &kvm_gmem_vm_ops;
+	if (kvm_gmem_is_pfn_mode(gi))
+		vma->vm_ops = &kvm_gmem_pfn_vm_ops;
+	else
+		vma->vm_ops = &kvm_gmem_vm_ops;
 
 	return 0;
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 33/88] KVM: guest_memfd: Skip checking outstanding references in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When converting memory attributes to private, guest_memfd verifies that
there are no outstanding references to folios in the page cache.

For PFN mode, retain the part where memory is unmapped from userspace,
since userspace should not be able to map private memory.

Since memory was not mapped into userspace using pages, there will not be
any other users that had GUP-ed the memory and there is no need to check
for outstanding references (that indicate other users of the memory).

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index e94f2359b7c9a..ceef61cb63a0d 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -901,7 +901,8 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
 		 */
 		unmap_mapping_pages(mapping, start, nr_pages, false);
 
-		if (kvm_gmem_has_outstanding_references(inode, start, nr_pages,
+		if (!kvm_gmem_is_pfn_mode(gi) &&
+		    kvm_gmem_has_outstanding_references(inode, start, nr_pages,
 							err_index)) {
 			mas_destroy(&mas);
 			r = -EAGAIN;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 34/88] KVM: guest_memfd: Support converting to shared in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When converting memory attributes to shared, guest_memfd iterates
through folios in the page cache to notify architecture code.

In PFN mode, memory is not backed by page cache folios. To notify
architecture code without unintentionally triggering memory allocation
in dynamic providers for unpopulated ranges, introduce the
lookup_pfn_range provider operation. It returns the PFN and number of
pages of contiguous backing memory, or -ENOENT if not allocated.

Add support for PFN mode when converting to shared by querying
lookup_pfn_range and converting contiguous chunks.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h | 19 +++++++++++++++++-
 virt/kvm/guest_memfd.c      | 47 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+), 1 deletion(-)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 9fdc0bd1a7190..e965aeca061a6 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -28,7 +28,22 @@ struct mempolicy;
  *           @pfn with uniform @pfn_attrs in @max_order. Once @get_pfn succeeds,
  *           the provider must keep the @index-to-@pfn mapping and @pfn_attrs
  *           constant, and the memory accessible, until guest_memfd stops using
- *           this page. Returns a negative error code on failure.
+ *           this page. Returns a negative error code on failure. The provider
+ *           must define @lookup_pfn_range when @get_pfn is defined.
+ * @lookup_pfn_range: Look up the PFN for a given index. This only checks for
+ *                    the existence of backing pages and should not allocate
+ *                    memory. Returns 0 on success with the physical frame
+ *                    number in @pfn and the number of physically contiguous
+ *                    pages in @nr_pages (@nr_pages need not be reduced for
+ *                    pages with different PFN attributes). Once
+ *                    @lookup_pfn_range succeeds, the provider must keep the
+ *                    existence of the pages and the @index-to-@pfn mapping
+ *                    constant until guest_memfd stops using the range. If
+ *                    backing memory is not available at @index, return -ENOENT
+ *                    with @nr_pages set to the number of consecutive indices
+ *                    starting from @index that have no entry. Return a negative
+ *                    error code on other failure. The provider must define
+ *                    @get_pfn when @lookup_pfn_range is defined.
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -40,6 +55,8 @@ struct guest_memfd_provider_operations {
 	void (*invalidate_folio)(void *provider, struct folio *folio);
 	int (*get_pfn)(void *provider, pgoff_t index, unsigned long *pfn,
 		       int *max_order, u64 *pfn_attrs);
+	int (*lookup_pfn_range)(void *provider, pgoff_t index,
+				unsigned long *pfn, size_t *nr_pages);
 };
 
 #endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ceef61cb63a0d..0e392b8735259 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -149,6 +149,27 @@ static int gmem_provider_get_pfn(struct gmem_inode *gi, pgoff_t index,
 	return 0;
 }
 
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
+static int gmem_provider_lookup_pfn_range(struct gmem_inode *gi, pgoff_t index,
+					  kvm_pfn_t *pfn, size_t *nr_pages)
+{
+	unsigned long raw_pfn;
+	int r;
+
+	r = gi->provider_ops->lookup_pfn_range(gi->provider, index, &raw_pfn,
+					       nr_pages);
+	if (!r || r == -ENOENT) {
+		if (WARN_ON_ONCE(!*nr_pages))
+			return -EINVAL;
+	}
+	if (r)
+		return r;
+
+	*pfn = raw_pfn;
+	return 0;
+}
+#endif
+
 static void gmem_provider_invalidate_folio(struct gmem_inode *gi,
 					   struct folio *folio)
 {
@@ -830,10 +851,36 @@ static bool kvm_gmem_has_outstanding_references(struct inode *inode,
 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
 static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff_t end)
 {
+	struct gmem_inode *gi = GMEM_I(inode);
 	struct folio_batch fbatch;
 	pgoff_t next = start;
 	int i;
 
+	if (kvm_gmem_is_pfn_mode(gi)) {
+		pgoff_t index;
+
+		for (index = start; index < end; ) {
+			size_t step = 1;
+			size_t nr_pages;
+			kvm_pfn_t pfn;
+			int r;
+
+			r = gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages);
+			if (!r || r == -ENOENT) {
+				step = min_t(size_t, nr_pages, end - index);
+				if (!r)
+					kvm_arch_gmem_make_shared(pfn, step);
+			} else {
+				WARN_ONCE(1, "unexpected lookup_pfn_range error %d at index 0x%lx\n",
+					  r, index);
+			}
+
+			index += step;
+			cond_resched();
+		}
+		return;
+	}
+
 	folio_batch_init(&fbatch);
 	while (filemap_get_folios(inode->i_mapping, &next, end - 1, &fbatch)) {
 		for (i = 0; i < folio_batch_count(&fbatch); ++i) {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 35/88] KVM: guest_memfd: Support preallocating memory via fallocate in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In PFN mode, memory is not backed by page cache folios, so attempting
to allocate folios during fallocate fails.

Support preallocating backing memory in PFN mode by querying the
provider for PFNs across the requested range. Step forward by the
contiguous order reported by the provider so that large contiguous
allocations do not require redundant per-page calls.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 0e392b8735259..c62e15e05a249 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -411,6 +411,7 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
 {
 	struct address_space *mapping = inode->i_mapping;
+	struct gmem_inode *gi = GMEM_I(inode);
 	pgoff_t start, index, end;
 	int r;
 
@@ -424,6 +425,30 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
 	end = (offset + len) >> PAGE_SHIFT;
 
 	r = 0;
+	if (kvm_gmem_is_pfn_mode(gi)) {
+		for (index = start; index < end; ) {
+			pgoff_t next_boundary;
+			pgoff_t max_nr_pages;
+			int max_order = 0;
+			kvm_pfn_t pfn;
+
+			if (signal_pending(current)) {
+				r = -EINTR;
+				break;
+			}
+
+			r = gmem_provider_get_pfn(gi, index, &pfn, &max_order, NULL);
+			if (r)
+				break;
+
+			max_nr_pages = 1UL << max_order;
+			next_boundary = ALIGN_DOWN(index + max_nr_pages, max_nr_pages);
+			index += min_t(size_t, next_boundary - index, end - index);
+			cond_resched();
+		}
+		goto out_unlock;
+	}
+
 	for (index = start; index < end; ) {
 		struct folio *folio;
 
@@ -450,6 +475,7 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
 		cond_resched();
 	}
 
+out_unlock:
 	filemap_invalidate_unlock_shared(mapping);
 
 	return r;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 36/88] KVM: guest_memfd: Clamp hole punch range to inode size

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When userspace punches a hole via fallocate, the VFS only validates that
the offset and length do not overflow the superblock maximum file size,
allowing ranges that extend past the end of the guest_memfd inode.

In folio mode, truncating page cache folios past the end of the inode is a
fast no-op because the page cache xarray has no entries beyond the inode
size.

Prepare for PFN mode by clamping the hole punch range to the inode size and
returning early if the starting offset is at or beyond the end of the file.

This avoids, in PFN mode, iterating across a potentially huge punched range
where the provider isn't even providing that memory to guest_memfd.

For folios, no functional change intended.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index c62e15e05a249..9e8204f5c2588 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -388,8 +388,15 @@ static void kvm_gmem_invalidate_end(struct inode *inode, pgoff_t start,
 static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 {
 	enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
-	pgoff_t start = offset >> PAGE_SHIFT;
-	pgoff_t end = (offset + len) >> PAGE_SHIFT;
+	loff_t size = i_size_read(inode);
+	pgoff_t start, end;
+
+	if (offset >= size)
+		return 0;
+
+	len = min(len, size - offset);
+	start = offset >> PAGE_SHIFT;
+	end = (offset + len) >> PAGE_SHIFT;
 
 	/*
 	 * Bindings must be stable across invalidation to ensure the start+end

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 37/88] KVM: guest_memfd: Support reclaim during hole punch in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In PFN mode, guest_memfd does not allocate or insert folios into the page
cache (inode->i_mapping). Calling truncate_inode_pages_range() is therefore
unnecessary.

Instead, call unmap_mapping_range() when in PFN mode to unmap any host
userspace PTEs over the punched range, while retaining
kvm_gmem_invalidate_start() and kvm_gmem_invalidate_end() to zap stage-2
KVM SPTEs.

Reclaim any confidential architecture state (e.g. SEV-SNP RMP) on provider
PFNs across the punched range via kvm_gmem_pfn_reclaim(), querying
.lookup_pfn_range() to skip unallocated holes without allocating memory.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 38 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 9e8204f5c2588..567735a564c7a 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -149,7 +149,7 @@ static int gmem_provider_get_pfn(struct gmem_inode *gi, pgoff_t index,
 	return 0;
 }
 
-#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
+#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM)
 static int gmem_provider_lookup_pfn_range(struct gmem_inode *gi, pgoff_t index,
 					  kvm_pfn_t *pfn, size_t *nr_pages)
 {
@@ -385,6 +385,35 @@ static void kvm_gmem_invalidate_end(struct inode *inode, pgoff_t start,
 		__kvm_gmem_invalidate_end(f, start, end);
 }
 
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
+static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t end)
+{
+	pgoff_t index;
+
+	for (index = start; index < end; ) {
+		size_t step = 1;
+		size_t nr_pages;
+		kvm_pfn_t pfn;
+		int r;
+
+		r = gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages);
+		if (!r || r == -ENOENT) {
+			step = min_t(size_t, nr_pages, end - index);
+			if (!r)
+				kvm_arch_gmem_reclaim(pfn, step);
+		} else {
+			WARN_ONCE(1, "unexpected lookup_pfn_range error %d at index 0x%lx\n",
+				  r, index);
+		}
+
+		index += step;
+		cond_resched();
+	}
+}
+#else
+static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t end) {}
+#endif
+
 static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 {
 	enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
@@ -406,7 +435,12 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 
 	kvm_gmem_invalidate_start(inode, start, end, filter);
 
-	truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
+	if (kvm_gmem_is_pfn_mode(GMEM_I(inode))) {
+		unmap_mapping_range(inode->i_mapping, offset, len, 0);
+		kvm_gmem_pfn_reclaim(GMEM_I(inode), start, end);
+	} else {
+		truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
+	}
 
 	kvm_gmem_invalidate_end(inode, start, end);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 38/88] KVM: guest_memfd: Support arch reclaim on inode destruction in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In folio mode, folios truncated on inode destruction are reclaimed via
aops->free_folio (kvm_gmem_free_folio), calling kvm_arch_gmem_reclaim()
to reset hardware confidential VM state (e.g. SEV-SNP RMP entries).

In PFN mode, memory is not backed by page cache folios, so
kvm_gmem_free_folio() is never invoked.

Do the reclaim during kvm_gmem_destroy_inode() when in PFN mode across the
entire file range, ensuring that any confidential architecture state on
provider PFNs is reclaimed before releasing the provider.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 567735a564c7a..b46907b480138 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1726,6 +1726,13 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
 {
 	struct gmem_inode *gi = GMEM_I(inode);
 
+	/*
+	 * Skip if the provider was not yet set up, as iput() can reach this on
+	 * error paths before provider_ops is initialized.
+	 */
+	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi))
+		kvm_gmem_pfn_reclaim(gi, 0, i_size_read(inode) >> PAGE_SHIFT);
+
 	gmem_provider_release(gi);
 
 	mpol_free_shared_policy(&gi->policy);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 39/88] KVM: guest_memfd: Support put_pfn_range provider op in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When punching holes or destroying guest_memfd files in folio mode,
folios are truncated and freed, allowing the backing filesystem to free
or reset memory.

In PFN mode, memory is not backed by page cache folios. To inform the
provider that a physical range has been released by guest_memfd,
introduce the .put_pfn_range callback to struct
guest_memfd_provider_operations.

Invoke .put_pfn_range() after kvm_gmem_pfn_reclaim() during both hole
punching and inode destruction, passing the file index and page count.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h |  7 +++++++
 virt/kvm/guest_memfd.c      | 16 ++++++++++++++--
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index e965aeca061a6..697f8f8c9b0ae 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -44,6 +44,12 @@ struct mempolicy;
  *                    starting from @index that have no entry. Return a negative
  *                    error code on other failure. The provider must define
  *                    @get_pfn when @lookup_pfn_range is defined.
+ * @put_pfn_range: Release a range of PFNs previously obtained for a given
+ *                 index. Called with the exclusive filemap invalidate lock held.
+ *                 This indicates to the provider that guest_memfd is no longer
+ *                 using the memory range. The provider does not need to ensure
+ *                 that backing memory is present; if backing memory is not
+ *                 available, the provider should skip the unbacked range.
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -57,6 +63,7 @@ struct guest_memfd_provider_operations {
 		       int *max_order, u64 *pfn_attrs);
 	int (*lookup_pfn_range)(void *provider, pgoff_t index,
 				unsigned long *pfn, size_t *nr_pages);
+	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
 };
 
 #endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index b46907b480138..501c3e250fc98 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -177,6 +177,13 @@ static void gmem_provider_invalidate_folio(struct gmem_inode *gi,
 		gi->provider_ops->invalidate_folio(gi->provider, folio);
 }
 
+static void gmem_provider_put_pfn_range(struct gmem_inode *gi, pgoff_t index,
+					size_t nr_pages)
+{
+	if (gi->provider_ops && gi->provider_ops->put_pfn_range)
+		gi->provider_ops->put_pfn_range(gi->provider, index, nr_pages);
+}
+
 static void gmem_provider_release(struct gmem_inode *gi)
 {
 	if (gi->provider_ops && gi->provider_ops->release)
@@ -438,6 +445,7 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 	if (kvm_gmem_is_pfn_mode(GMEM_I(inode))) {
 		unmap_mapping_range(inode->i_mapping, offset, len, 0);
 		kvm_gmem_pfn_reclaim(GMEM_I(inode), start, end);
+		gmem_provider_put_pfn_range(GMEM_I(inode), start, end - start);
 	} else {
 		truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
 	}
@@ -1730,8 +1738,12 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
 	 * Skip if the provider was not yet set up, as iput() can reach this on
 	 * error paths before provider_ops is initialized.
 	 */
-	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi))
-		kvm_gmem_pfn_reclaim(gi, 0, i_size_read(inode) >> PAGE_SHIFT);
+	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi)) {
+		pgoff_t nr_pages = i_size_read(inode) >> PAGE_SHIFT;
+
+		kvm_gmem_pfn_reclaim(gi, 0, nr_pages);
+		gmem_provider_put_pfn_range(gi, 0, nr_pages);
+	}
 
 	gmem_provider_release(gi);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 40/88] KVM: guest_memfd: Add TODO for memory failure handling in PFN mode

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a TODO comment noting that PFN mode does not use page-cache folios, so
->error_remove_folio() will not be called by the core memory failure
infrastructure when physical memory errors occur.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 virt/kvm/guest_memfd.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 501c3e250fc98..ffe1508985d61 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1127,6 +1127,11 @@ static int kvm_gmem_migrate_folio(struct address_space *mapping,
 	return -EINVAL;
 }
 
+/*
+ * TODO: Support memory failure notification for PFN mode. In PFN mode, memory
+ * is not backed by page cache folios, so ->error_remove_folio() will not be
+ * invoked by the core memory failure handler.
+ */
 static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *folio)
 {
 	struct inode *inode = mapping->host;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 41/88] KVM: guest_memfd: Allow attaching PFN-backed providers

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Expose GUEST_MEMFD_FLAG_PFN_MODE in supported flags to allow userspace
to explicitly request PFN mode when creating a guest_memfd instance with
GUEST_MEMFD_FLAG_USE_RESOURCE.

PFN mode is only valid when backed by an external resource provider, as
native guest_memfd relies on the page cache and manages memory as folios.
Reject creation if PFN mode is requested without an external resource.

While guest_memfd could try to infer PFN mode from whether PFN lookup
callbacks exist, an explicit creation flag ensures userspace selects the
operating mode directly, allowing providers to support both modes.

Validate the requested operating mode against the provider operations
when attaching a resource:
- In PFN mode, the provider must implement both .get_pfn and
  .lookup_pfn_range.
- In folio mode, the provider must implement .alloc_folio.

Document the new flag and its requirements in the KVM API documentation.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 Documentation/virt/kvm/api.rst |  6 ++++++
 include/linux/kvm_host.h       |  3 ++-
 virt/kvm/guest_memfd.c         | 18 +++++++++++++++++-
 3 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 9f5313d405d9c..bb550e4f11cd8 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6524,6 +6524,12 @@ specified via KVM_CREATE_GUEST_MEMFD.  Currently defined flags:
                                 must be a supported resource (e.g. tmpfs mount
                                 root directory). Currently, tmpfs resource pools
                                 require noswap and huge=never mount options.
+  GUEST_MEMFD_FLAG_PFN_MODE     Request PFN mode for guest_memfd backed
+                                by an external resource provider. When set,
+                                GUEST_MEMFD_FLAG_USE_RESOURCE must also be set,
+                                and the provider must support PFN lookup
+                                (.get_pfn and .lookup_pfn_range) rather than
+                                folio allocation (.alloc_folio).
   ============================= ================================================
 
 When the KVM MMU performs a PFN lookup to service a guest fault, the fault will
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index cf679efcf3260..980ce85cf7239 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2619,7 +2619,8 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm);
 
 static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
 {
-	u64 flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_RESOURCE;
+	u64 flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_USE_RESOURCE |
+		    GUEST_MEMFD_FLAG_PFN_MODE;
 
 	if (!kvm || kvm_arch_supports_gmem_init_shared(kvm))
 		flags |= GUEST_MEMFD_FLAG_INIT_SHARED;
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ffe1508985d61..1b6fc44ba00bb 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -85,6 +85,18 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
 	.alloc_folio = gmem_native_alloc_folio,
 };
 
+static bool gmem_provider_ops_valid(struct gmem_inode *gi,
+				     const struct guest_memfd_provider_operations *ops)
+{
+	if (!ops || !ops->attach)
+		return false;
+
+	if (kvm_gmem_is_pfn_mode(gi))
+		return ops->get_pfn && ops->lookup_pfn_range;
+
+	return ops->alloc_folio != NULL;
+}
+
 #define GMEM_RESOURCE_FD_NATIVE (-1)
 
 static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
@@ -104,7 +116,7 @@ static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
 			ops = resource_file->f_op->gmem_provider_ops;
 	}
 
-	if (ops && ops->attach && ops->alloc_folio)
+	if (gmem_provider_ops_valid(GMEM_I(inode), ops))
 		provider = ops->attach(inode, resource_file);
 
 	if (resource_file)
@@ -1350,6 +1362,10 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
 	if (size <= 0 || !PAGE_ALIGNED(size))
 		return -EINVAL;
 
+	if (flags & GUEST_MEMFD_FLAG_PFN_MODE &&
+	    !(flags & GUEST_MEMFD_FLAG_USE_RESOURCE))
+		return -EINVAL;
+
 	if (flags & GUEST_MEMFD_FLAG_USE_RESOURCE) {
 		if (args->resource_fd < 0)
 			return -EBADF;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 42/88] KVM: guest_memfd: Export kvm_gmem_revoke for provider-driven revocation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Providers may dynamically revoke physical memory ranges (e.g., dynamic
page removal or device teardown) and need a mechanism to sever guest and
host mappings and reclaim confidential architecture state.

Introduce __kvm_gmem_revoke() to share the core invalidation and
reclamation path between kvm_gmem_punch_hole() and kvm_gmem_revoke().

Introduce boolean to control whether to call .put_pfn_range().
.put_pfn_range() must not be called when revoke is used by a provider,
because the provider already initiated the revocation and calling back into
the provider could lead to self-deadlock or re-entrancy on provider locks.

Export kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
const unsigned long *pfn) via EXPORT_SYMBOL_GPL. Providers must pass
both start and pfn.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 arch/x86/kvm/Makefile       |  3 +-
 include/linux/guest_memfd.h | 31 ++++++++++++++++----
 virt/kvm/guest_memfd.c      | 70 +++++++++++++++++++++++++++++++++++++++------
 3 files changed, 88 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index c6bf463f50327..5fd01bd19d1c4 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -61,7 +61,8 @@ exports_grep_trailer := --include='*.[ch]' -nrw $(srctree)/virt/kvm $(srctree)/a
 				-e kvm_page_track_unregister_notifier \
 				-e kvm_write_track_add_gfn \
 				-e kvm_write_track_remove_gfn \
-				-e kvm_file_to_kvm_fn
+				-e kvm_file_to_kvm_fn \
+				-e kvm_gmem_revoke
 
 # Force grep to emit a goofy group separator that can in turn be replaced with
 # the above newline macro (newlines in Make are a nightmare).  Note, grep only
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 697f8f8c9b0ae..fb2279a001ebf 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -28,8 +28,13 @@ struct mempolicy;
  *           @pfn with uniform @pfn_attrs in @max_order. Once @get_pfn succeeds,
  *           the provider must keep the @index-to-@pfn mapping and @pfn_attrs
  *           constant, and the memory accessible, until guest_memfd stops using
- *           this page. Returns a negative error code on failure. The provider
- *           must define @lookup_pfn_range when @get_pfn is defined.
+ *           this page. Do not take locks held across kvm_gmem_revoke().
+ *           Returns a negative error code on failure. The provider must define
+ *           @lookup_pfn_range when @get_pfn is defined.
+ * @mmap: Optional callback to set up user mappings. Called when userspace calls
+ *        mmap() on guest_memfd. The provider can customize VMA flags,
+ *        cacheability/protections, or reject the mapping. If not defined,
+ *        guest_memfd falls back to default mapping behavior.
  * @lookup_pfn_range: Look up the PFN for a given index. This only checks for
  *                    the existence of backing pages and should not allocate
  *                    memory. Returns 0 on success with the physical frame
@@ -41,15 +46,18 @@ struct mempolicy;
  *                    constant until guest_memfd stops using the range. If
  *                    backing memory is not available at @index, return -ENOENT
  *                    with @nr_pages set to the number of consecutive indices
- *                    starting from @index that have no entry. Return a negative
- *                    error code on other failure. The provider must define
- *                    @get_pfn when @lookup_pfn_range is defined.
+ *                    starting from @index that have no entry. Do not take locks
+ *                    held across kvm_gmem_revoke(). Return a negative error
+ *                    code on other failure. The provider must define @get_pfn
+ *                    when @lookup_pfn_range is defined.
  * @put_pfn_range: Release a range of PFNs previously obtained for a given
  *                 index. Called with the exclusive filemap invalidate lock held.
  *                 This indicates to the provider that guest_memfd is no longer
  *                 using the memory range. The provider does not need to ensure
  *                 that backing memory is present; if backing memory is not
- *                 available, the provider should skip the unbacked range.
+ *                 available, the provider should skip the unbacked range. Not
+ *                 called on kvm_gmem_revoke(). Do not take locks held across
+ *                 kvm_gmem_revoke().
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -66,4 +74,15 @@ struct guest_memfd_provider_operations {
 	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
 };
 
+#ifdef CONFIG_KVM_GUEST_MEMFD
+int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
+		    const unsigned long *pfn);
+#else
+static inline int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
+				  const unsigned long *pfn)
+{
+	return -EOPNOTSUPP;
+}
+#endif
+
 #endif /* _LINUX_GUEST_MEMFD_H */
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 1b6fc44ba00bb..023bf095340be 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -433,18 +433,25 @@ static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t e
 static void kvm_gmem_pfn_reclaim(struct gmem_inode *gi, pgoff_t start, pgoff_t end) {}
 #endif
 
-static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
+static int __kvm_gmem_revoke(struct inode *inode, pgoff_t start,
+			      size_t nr_pages, const unsigned long *pfn,
+			      bool put_pfn_range)
 {
 	enum kvm_gfn_range_filter filter = kvm_gmem_get_all_gfns_filter(inode);
-	loff_t size = i_size_read(inode);
-	pgoff_t start, end;
+	pgoff_t max_pages = i_size_read(inode) >> PAGE_SHIFT;
+	loff_t offset, len;
+	pgoff_t end;
 
-	if (offset >= size)
+	if (start >= max_pages)
 		return 0;
 
-	len = min(len, size - offset);
-	start = offset >> PAGE_SHIFT;
-	end = (offset + len) >> PAGE_SHIFT;
+	nr_pages = min_t(size_t, nr_pages, max_pages - start);
+	if (!nr_pages)
+		return 0;
+
+	offset = (loff_t)start << PAGE_SHIFT;
+	len = (loff_t)nr_pages << PAGE_SHIFT;
+	end = start + nr_pages;
 
 	/*
 	 * Bindings must be stable across invalidation to ensure the start+end
@@ -456,8 +463,14 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 
 	if (kvm_gmem_is_pfn_mode(GMEM_I(inode))) {
 		unmap_mapping_range(inode->i_mapping, offset, len, 0);
-		kvm_gmem_pfn_reclaim(GMEM_I(inode), start, end);
-		gmem_provider_put_pfn_range(GMEM_I(inode), start, end - start);
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
+		if (pfn)
+			kvm_arch_gmem_reclaim(*pfn, nr_pages);
+		else
+			kvm_gmem_pfn_reclaim(GMEM_I(inode), start, end);
+#endif
+		if (put_pfn_range)
+			gmem_provider_put_pfn_range(GMEM_I(inode), start, nr_pages);
 	} else {
 		truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1);
 	}
@@ -469,6 +482,45 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
 	return 0;
 }
 
+static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
+{
+	return __kvm_gmem_revoke(inode, offset >> PAGE_SHIFT, len >> PAGE_SHIFT,
+				 NULL, true);
+}
+
+/**
+ * kvm_gmem_revoke - Revoke and reclaim a range of memory in guest_memfd.
+ * @inode:    The guest_memfd inode.
+ * @start:    Starting page index within the file.
+ * @nr_pages: Number of pages to revoke.
+ * @pfn:      The physical address / frame number being revoked. Directly
+ *            reclaims this physical range without querying the provider's
+ *            .lookup_pfn_range().
+ *
+ * guest_memfd will do everything to make sure the range revoked is no longer in
+ * use across guest and host mappings, as well as reclaim any confidential
+ * architecture state.
+ *
+ * Before calling kvm_gmem_revoke(), the caller must ensure that future
+ * .get_pfn() calls for the revoked range will fail so new mappings cannot be
+ * faulted back in after kvm_gmem_revoke() releases the inode's invalidate_lock.
+ * Must be called before accessing, freeing, resetting, or powering down the
+ * backing resource, and the caller must keep the @start-to-PFN mapping valid
+ * and accessible until kvm_gmem_revoke() returns. .put_pfn_range() will not be
+ * called.
+ *
+ * Acquires @inode->i_mapping->invalidate_lock exclusively; callers must not
+ * hold any lock that is also acquired inside provider callbacks.
+ *
+ * Return: 0 on success, or a negative error code on failure.
+ */
+int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
+		    const unsigned long *pfn)
+{
+	return __kvm_gmem_revoke(inode, start, nr_pages, pfn, false);
+}
+EXPORT_SYMBOL_GPL(kvm_gmem_revoke);
+
 static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len)
 {
 	struct address_space *mapping = inode->i_mapping;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 43/88] KVM: selftests: Test flag dependency of PFN_MODE on USE_RESOURCE

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Creating a guest_memfd with GUEST_MEMFD_FLAG_PFN_MODE requires
GUEST_MEMFD_FLAG_USE_RESOURCE.

Skip testing GUEST_MEMFD_FLAG_PFN_MODE as an isolated flag in
test_guest_memfd_flags() because creating a guest_memfd with PFN_MODE
alone without USE_RESOURCE is rejected by KVM with EINVAL.

In test_resource_fd_invalid(), verify that attempting to create a
guest_memfd with GUEST_MEMFD_FLAG_PFN_MODE without
GUEST_MEMFD_FLAG_USE_RESOURCE fails with EINVAL.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/include/uapi/linux/kvm.h                 | 1 +
 tools/testing/selftests/kvm/guest_memfd_test.c | 8 +++++++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index 66c1f40b50c29..3ef7bc37a7070 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1655,6 +1655,7 @@ struct kvm_memory_attributes {
 #define GUEST_MEMFD_FLAG_MMAP		(1ULL << 0)
 #define GUEST_MEMFD_FLAG_INIT_SHARED	(1ULL << 1)
 #define GUEST_MEMFD_FLAG_USE_RESOURCE	(1ULL << 2)
+#define GUEST_MEMFD_FLAG_PFN_MODE	(1ULL << 3)
 
 struct kvm_create_guest_memfd {
 	__u64 size;
diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
index cbc2f309664b2..fb9d50be2e3b6 100644
--- a/tools/testing/selftests/kvm/guest_memfd_test.c
+++ b/tools/testing/selftests/kvm/guest_memfd_test.c
@@ -407,7 +407,8 @@ static void test_guest_memfd_flags(struct kvm_vm *vm)
 	int fd;
 
 	for (flag = BIT(0); flag; flag <<= 1) {
-		if (flag == GUEST_MEMFD_FLAG_USE_RESOURCE)
+		if (flag == GUEST_MEMFD_FLAG_USE_RESOURCE ||
+		    flag == GUEST_MEMFD_FLAG_PFN_MODE)
 			continue;
 		fd = __vm_create_guest_memfd(vm, page_size, flag);
 		if (flag & valid_flags) {
@@ -432,6 +433,11 @@ static void test_resource_fd_invalid(struct kvm_vm *vm)
 	TEST_ASSERT(fd < 0, "guest_memfd with resource_fd but without flag should fail");
 	TEST_ASSERT_EQ(errno, EINVAL);
 
+	/* PFN_MODE without GUEST_MEMFD_FLAG_USE_RESOURCE fails */
+	fd = __vm_create_guest_memfd(vm, page_size, GUEST_MEMFD_FLAG_PFN_MODE);
+	TEST_ASSERT(fd < 0, "guest_memfd with PFN_MODE without USE_RESOURCE should fail");
+	TEST_ASSERT_EQ(errno, EINVAL);
+
 	/* Bad file descriptor with GUEST_MEMFD_FLAG_USE_RESOURCE fails */
 	fd = __vm_create_guest_memfd_resource(vm, page_size,
 					      GUEST_MEMFD_FLAG_USE_RESOURCE, -1);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 44/88] samples/kvm: Implement attach and release provider ops in sample provider

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement the guest_memfd provider operations in the sample memory
provider module, providing .attach and .release callbacks.

In .attach, take a reference to the provider file (get_file) to keep the
provider context alive while attached. In .release, release the file
reference via fput.

Attach the provider operations to mps_file_fops.gmem_provider_ops so that
provider file descriptors created by the sample module can be passed as
resource_fd to guest_memfd.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/kvm/memory_provider.c | 39 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 38 insertions(+), 1 deletion(-)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 81ac94ae1a4e9..6f86e3e3262e3 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -17,6 +17,7 @@
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/gfp.h>
+#include <linux/guest_memfd.h>
 #include <linux/miscdevice.h>
 #include <linux/mm.h>
 #include <linux/module.h>
@@ -34,9 +35,11 @@ struct mp_info {
 	unsigned long npages;
 	struct page *cma_pages;		/* from alloc_contig_pages() */
 
-	struct mutex lock;		/* the bitmaps */
+	struct mutex lock;		/* the bitmaps and gmem_inode */
 	unsigned long *absent;		/* taken away by MP_SET_PRESENT */
 	unsigned long *readonly;	/* marked read-only by MP_SET_READONLY */
+
+	struct inode *gmem_inode;
 };
 
 /* Byte range [offset, offset + len) of @info as page indices. */
@@ -161,11 +164,45 @@ static long mp_file_ioctl(struct file *file, unsigned int cmd,
 	}
 }
 
+static void *mp_gmem_attach(struct inode *gmem_inode,
+			    struct file *resource_file)
+{
+	struct mp_info *info = resource_file->private_data;
+
+	mutex_lock(&info->lock);
+	if (info->gmem_inode) {
+		mutex_unlock(&info->lock);
+		return ERR_PTR(-EBUSY);
+	}
+	info->gmem_inode = gmem_inode;
+	mutex_unlock(&info->lock);
+
+	return get_file(resource_file);
+}
+
+static void mp_gmem_release(void *provider)
+{
+	struct file *resource_file = provider;
+	struct mp_info *info = resource_file->private_data;
+
+	mutex_lock(&info->lock);
+	info->gmem_inode = NULL;
+	mutex_unlock(&info->lock);
+
+	fput(resource_file);
+}
+
+static const struct guest_memfd_provider_operations mp_gmem_provider_ops = {
+	.attach		= mp_gmem_attach,
+	.release	= mp_gmem_release,
+};
+
 static const struct file_operations mp_file_fops = {
 	.owner		= THIS_MODULE,
 	.release	= mp_file_release,
 	.unlocked_ioctl	= mp_file_ioctl,
 	.compat_ioctl	= compat_ptr_ioctl,
+	.gmem_provider_ops = &mp_gmem_provider_ops,
 };
 
 static long mp_ctl_new_file(void __user *uarg)

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 45/88] samples/kvm: Disallow MP_SET_READONLY after attach

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Once a provider file is attached to guest_memfd, its physical attributes
must remain constant throughout its lifetime.

Disallow changing page read-only state via MP_SET_READONLY if the provider
file is currently attached to a guest_memfd inode, returning -EBUSY.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/kvm/memory_provider.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 6f86e3e3262e3..168c1d90edacd 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -58,17 +58,23 @@ static int mp_range(struct mp_info *info, u64 offset, u64 len,
 	return 0;
 }
 
-static void mp_set_readonly(struct mp_info *info, unsigned long start,
+static int mp_set_readonly(struct mp_info *info, unsigned long start,
 			    unsigned long end, bool readonly)
 {
 	mutex_lock(&info->lock);
 
+	if (info->gmem_inode) {
+		mutex_unlock(&info->lock);
+		return -EBUSY;
+	}
+
 	if (readonly)
 		bitmap_set(info->readonly, start, end - start);
 	else
 		bitmap_clear(info->readonly, start, end - start);
 
 	mutex_unlock(&info->lock);
+	return 0;
 }
 
 static void mp_set_present(struct mp_info *info, unsigned long start,
@@ -123,7 +129,7 @@ static long mp_file_set_readonly(struct mp_info *info, void __user *uarg)
 
 	ret = mp_range(info, sr.offset, sr.len, &start, &end);
 	if (!ret)
-		mp_set_readonly(info, start, end, sr.value);
+		ret = mp_set_readonly(info, start, end, sr.value);
 
 	return ret;
 }

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 46/88] samples/kvm: Implement get_pfn provider operation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement the .get_pfn callback in mp_gmem_provider_ops to operate as a
direct PFN provider for guest_memfd.

Implementing .get_pfn satisfies guest_memfd's provider operations
requirement, enabling memory_provider files to be attached as
resource_fd memory providers for guest_memfd.

In mp_gmem_get_pfn(), return the physical frame number corresponding to
the requested index from the CMA reservation, returning -EFAULT if the
page was marked absent via MP_SET_PRESENT. Also report
GUEST_MEMFD_PFN_ATTRS_READONLY in pfn_attrs if the page was marked
read-only via MP_SET_READONLY.

Checking test_bit(index, info->absent) and info->read_only is done
locklessly without holding info->lock:
- test_bit() is an atomic memory read.
- Locklessness prevents lock inversion and deadlocks. In guest_memfd,
  the guest page fault path (kvm_gmem_get_pfn()) holds
  filemap_invalidate_lock_shared while calling .get_pfn(). Avoiding
  info->lock in .get_pfn() ensures no lock inversions exist between
  filemap_invalidate_lock and info->lock.
- Similarly, while kvm_gmem_revoke() now takes an explicit pfn and avoids
  calling back into .get_pfn(), avoiding info->lock in .get_pfn() keeps
  the lookup path completely lock-free and robust.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/kvm/memory_provider.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 168c1d90edacd..1db69a8f58758 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -198,9 +198,34 @@ static void mp_gmem_release(void *provider)
 	fput(resource_file);
 }
 
+static int mp_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
+			   int *max_order, u64 *pfn_attrs)
+{
+	struct file *resource_file = provider;
+	struct mp_info *info = resource_file->private_data;
+
+	if (index >= info->npages)
+		return -EINVAL;
+
+	if (test_bit(index, info->absent))
+		return -EFAULT;
+
+	*pfn = info->base_pfn + index;
+	if (max_order)
+		*max_order = 0;
+	if (pfn_attrs) {
+		*pfn_attrs = 0;
+		if (test_bit(index, info->readonly))
+			*pfn_attrs |= GUEST_MEMFD_PFN_ATTRS_READONLY;
+	}
+
+	return 0;
+}
+
 static const struct guest_memfd_provider_operations mp_gmem_provider_ops = {
 	.attach		= mp_gmem_attach,
 	.release	= mp_gmem_release,
+	.get_pfn	= mp_gmem_get_pfn,
 };
 
 static const struct file_operations mp_file_fops = {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 47/88] samples/kvm: Implement put_pfn_range provider operation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When guest_memfd hole punches or destroys an instance backed by a PFN
provider, it invokes .put_pfn_range to release the physical memory back
to the provider.

Implement the .put_pfn_range callback in mps_gmem_provider_ops to clear
the CMA pages across the released range via clear_highpage(). This
ensures that subsequent accesses to punched ranges observe zeroed memory,
matching POSIX fallocate hole-punching semantics and preventing stale data
from being observed when pages are reused.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/kvm/memory_provider.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 1db69a8f58758..0a03623492f4c 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -17,6 +17,7 @@
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/gfp.h>
+#include <linux/highmem.h>
 #include <linux/guest_memfd.h>
 #include <linux/miscdevice.h>
 #include <linux/mm.h>
@@ -198,6 +199,21 @@ static void mp_gmem_release(void *provider)
 	fput(resource_file);
 }
 
+static void mp_gmem_put_pfn_range(void *provider, pgoff_t index,
+				   size_t nr_pages)
+{
+	struct file *resource_file = provider;
+	struct mp_info *info = resource_file->private_data;
+	size_t i;
+
+	if (index >= info->npages)
+		return;
+
+	nr_pages = min_t(size_t, nr_pages, info->npages - index);
+	for (i = 0; i < nr_pages; i++)
+		clear_highpage(info->cma_pages + index + i);
+}
+
 static int mp_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
 			   int *max_order, u64 *pfn_attrs)
 {
@@ -226,6 +242,7 @@ static const struct guest_memfd_provider_operations mp_gmem_provider_ops = {
 	.attach		= mp_gmem_attach,
 	.release	= mp_gmem_release,
 	.get_pfn	= mp_gmem_get_pfn,
+	.put_pfn_range	= mp_gmem_put_pfn_range,
 };
 
 static const struct file_operations mp_file_fops = {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 48/88] samples/kvm: Support dynamic revocation via kvm_gmem_revoke

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Support dynamic revocation in the sample memory provider module by
calling kvm_gmem_revoke() when pages are marked absent via
MPS_SET_PRESENT.

Note down the attaching gmem inode during .attach() and disallow double
attachment to the same provider file.

Clear the revoked pages after kvm_gmem_revoke() using mps_zero_range()
to ensure that stale data is wiped and cannot be observed when pages are
later marked present or reused.

Regarding locking and ordering:
- In mps_set_present(), mark the absent bitmap first before revoking so
  any concurrent page fault immediately sees the page as absent and
  rejects the lookup with -EFAULT.
- kvm_gmem_revoke() is called under info->lock to fully serialize
  MPS_SET_PRESENT operations against each other and prevent races
  between marking pages absent and present.
- The PFN passed to kvm_gmem_revoke() is derived from info->base_pfn +
  start. The mapping between page index and physical PFN is static and
  immutable for the lifetime of the provider file, ensuring that the
  pfn mapping remains completely stable across the revoke call.
- Passing pfn directly to kvm_gmem_revoke() allows guest_memfd to
  reclaim the physical range directly without calling back into the
  provider. This keeps the lock hierarchy strictly top-down (info->lock
  -> filemap_invalidate_lock -> mmu_lock) with no callbacks or deadlocks.

For now, nothing needs to be done when setting a page present, since users
should (re-)fault and (re-)obtain the PFN via .get_pfn(), to be added in a
later patch.

For users that don't support faulting, like the IOMMU, the counterpart to
kvm_gmem_revoke() will be made available for the first user to indicate
that the memory is now available. guest_memfd should then do the page table
mappings as necessary, based on the shared/private status.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/Kconfig               |  2 +-
 samples/kvm/memory_provider.c | 33 ++++++++++++++++++++++-----------
 2 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/samples/Kconfig b/samples/Kconfig
index d4e79e1a9286b..4ef9328a91778 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -328,7 +328,7 @@ source "samples/damon/Kconfig"
 
 config SAMPLE_KVM_MEMORY_PROVIDER
 	tristate "Build sample memory provider -- loadable module only"
-	depends on CONTIG_ALLOC && X86_64 && m
+	depends on CONTIG_ALLOC && X86_64 && m && KVM_GUEST_MEMFD
 	help
 	  This builds a sample memory provider.
 	  Its files can back a guest_memfd and be mapped by iommufd.
diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 0a03623492f4c..926f3cf68ed49 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -43,6 +43,19 @@ struct mp_info {
 	struct inode *gmem_inode;
 };
 
+static void mp_zero_range(struct mp_info *info, pgoff_t index,
+			   size_t nr_pages)
+{
+	size_t i;
+
+	if (index >= info->npages)
+		return;
+
+	nr_pages = min_t(size_t, nr_pages, info->npages - index);
+	for (i = 0; i < nr_pages; i++)
+		clear_highpage(info->cma_pages + index + i);
+}
+
 /* Byte range [offset, offset + len) of @info as page indices. */
 static int mp_range(struct mp_info *info, u64 offset, u64 len,
 		    unsigned long *start, unsigned long *end)
@@ -82,12 +95,17 @@ static void mp_set_present(struct mp_info *info, unsigned long start,
 			   unsigned long end, bool present)
 {
 	mutex_lock(&info->lock);
-
-	if (present)
+	if (present) {
 		bitmap_clear(info->absent, start, end - start);
-	else
+	} else {
 		bitmap_set(info->absent, start, end - start);
+		if (info->gmem_inode) {
+			unsigned long pfn = info->base_pfn + start;
 
+			kvm_gmem_revoke(info->gmem_inode, start, end - start, &pfn);
+			mp_zero_range(info, start, end - start);
+		}
+	}
 	mutex_unlock(&info->lock);
 }
 
@@ -203,15 +221,8 @@ static void mp_gmem_put_pfn_range(void *provider, pgoff_t index,
 				   size_t nr_pages)
 {
 	struct file *resource_file = provider;
-	struct mp_info *info = resource_file->private_data;
-	size_t i;
 
-	if (index >= info->npages)
-		return;
-
-	nr_pages = min_t(size_t, nr_pages, info->npages - index);
-	for (i = 0; i < nr_pages; i++)
-		clear_highpage(info->cma_pages + index + i);
+	mp_zero_range(resource_file->private_data, index, nr_pages);
 }
 
 static int mp_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 49/88] samples/kvm: Implement lookup_pfn_range provider operation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement .lookup_pfn_range to allow looking up index -> pfn mappings to
satisfy guest_memfd requirements and make this sample memory provider a
valid PFN mode provider.

This sample memory provider performs allocations upfront, so
.lookup_pfn_range doesn't do any allocations anyway and is only slightly
different from .get_pfn.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 samples/kvm/memory_provider.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
index 926f3cf68ed49..7fb7365b65940 100644
--- a/samples/kvm/memory_provider.c
+++ b/samples/kvm/memory_provider.c
@@ -249,10 +249,34 @@ static int mp_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
 	return 0;
 }
 
+static int mp_gmem_lookup_pfn_range(void *provider, pgoff_t index,
+				     unsigned long *pfn, size_t *nr_pages)
+{
+	struct file *resource_file = provider;
+	struct mp_info *info = resource_file->private_data;
+
+	if (index >= info->npages)
+		return -EINVAL;
+
+	if (test_bit(index, info->absent)) {
+		unsigned long next_present;
+
+		next_present = find_next_zero_bit(info->absent, info->npages,
+						  index + 1);
+		*nr_pages = next_present - index;
+		return -ENOENT;
+	}
+
+	*pfn = info->base_pfn + index;
+	*nr_pages = find_next_bit(info->absent, info->npages, index + 1) - index;
+	return 0;
+}
+
 static const struct guest_memfd_provider_operations mp_gmem_provider_ops = {
 	.attach		= mp_gmem_attach,
 	.release	= mp_gmem_release,
 	.get_pfn	= mp_gmem_get_pfn,
+	.lookup_pfn_range = mp_gmem_lookup_pfn_range,
 	.put_pfn_range	= mp_gmem_put_pfn_range,
 };
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 50/88] KVM: selftests: Test guest_memfd conversions with sample memory provider backing

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add fixture variant for the sample memory provider.

If the sample memory provider cannot be opened because the module is not
loaded, the variant is skipped gracefully via TEST_REQUIRE().

Because PFN-backed memory cannot be pinned via GUP, expect pinning to fail
for the variant in the elevated_refcount test.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/include/kvm_util.h     |  1 +
 tools/testing/selftests/kvm/lib/kvm_util.c         |  9 ++++-
 .../kvm/x86/guest_memfd_conversions_test.c         | 44 +++++++++++++++++++++-
 3 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index eca296539db79..1a1387f36a52c 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -1350,6 +1350,7 @@ static inline int pin_self_to_any_cpu(void)
 	return pin_task_to_any_cpu(pthread_self());
 }
 
+int __pin_pages(void *vaddr, uint64_t size);
 void pin_pages(void *vaddr, uint64_t size);
 void unpin_pages(void);
 
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index d42e201eba763..9793818f54d23 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -674,7 +674,7 @@ int __pin_task_to_cpu(pthread_t task, int cpu)
 
 static int gup_test_fd = -1;
 
-void pin_pages(void *vaddr, uint64_t size)
+int __pin_pages(void *vaddr, uint64_t size)
 {
 	const struct pin_longterm_test args = {
 		.addr = (uint64_t)vaddr,
@@ -685,7 +685,12 @@ void pin_pages(void *vaddr, uint64_t size)
 	gup_test_fd = __open_path_or_exit("/sys/kernel/debug/gup_test", O_RDWR,
 					  "Is CONFIG_GUP_TEST enabled?");
 
-	TEST_ASSERT_EQ(ioctl(gup_test_fd, PIN_LONGTERM_TEST_START, &args), 0);
+	return ioctl(gup_test_fd, PIN_LONGTERM_TEST_START, &args);
+}
+
+void pin_pages(void *vaddr, uint64_t size)
+{
+	TEST_ASSERT_EQ(__pin_pages(vaddr, size), 0);
 }
 
 void unpin_pages(void)
diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
index 023df9cb9d4a6..d0cbb1353d7ee 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
@@ -16,11 +16,13 @@
 #include "kselftest_harness.h"
 #include "test_util.h"
 #include "ucall_common.h"
+#include "../../../../samples/kvm/memory_provider.h"
 
 FIXTURE(gmem_conversions) {
 	struct kvm_vcpu *vcpu;
 	int gmem_fd;
 	int resource_fd;
+	bool expect_pin_failure;
 	/* HVA of the first byte of the memory mmap()-ed from gmem_fd. */
 	char *mem;
 };
@@ -29,10 +31,14 @@ typedef FIXTURE_DATA(gmem_conversions) test_data_t;
 
 FIXTURE_VARIANT(gmem_conversions) {
 	int (*create_resource)(size_t size);
+	uint64_t extra_flags;
+	bool expect_pin_failure;
 };
 
 FIXTURE_VARIANT_ADD(gmem_conversions, native) {
 	.create_resource = NULL,
+	.extra_flags = 0,
+	.expect_pin_failure = false,
 };
 
 static int create_tmpfs_resource(size_t size)
@@ -42,6 +48,34 @@ static int create_tmpfs_resource(size_t size)
 
 FIXTURE_VARIANT_ADD(gmem_conversions, tmpfs) {
 	.create_resource = create_tmpfs_resource,
+	.extra_flags = 0,
+	.expect_pin_failure = false,
+};
+
+static int create_mp_resource(size_t size)
+{
+	struct mp_new_file nf = {
+		.size = size,
+	};
+	int dev_fd, fd;
+
+	dev_fd = open("/dev/memory_provider", O_RDWR);
+	if (dev_fd < 0) {
+		TEST_REQUIRE(errno != ENOENT && errno != ENODEV);
+		TEST_FAIL("Failed to open /dev/memory_provider: %d (%s)",
+			  errno, strerror(errno));
+	}
+
+	fd = ioctl(dev_fd, MP_NEW_FILE, &nf);
+	TEST_ASSERT(fd >= 0, "MP_NEW_FILE failed: %d (%s)", errno, strerror(errno));
+	close(dev_fd);
+	return fd;
+}
+
+FIXTURE_VARIANT_ADD(gmem_conversions, mp) {
+	.create_resource = create_mp_resource,
+	.extra_flags = GUEST_MEMFD_FLAG_PFN_MODE,
+	.expect_pin_failure = true,
 };
 
 FIXTURE_SETUP(gmem_conversions) { }
@@ -75,10 +109,12 @@ static void gmem_conversions_do_setup(test_data_t *t,
 
 	vm = __vm_create_shape_with_one_vcpu(shape, &t->vcpu, nr_pages, guest_do_rmw);
 
+	t->expect_pin_failure = variant->expect_pin_failure;
 	t->resource_fd = -1;
 
 	if (variant->create_resource) {
-		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE;
+		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE |
+				 variant->extra_flags;
 
 		t->resource_fd = variant->create_resource(size);
 		TEST_ASSERT(t->resource_fd >= 0, "Failed to create backing resource");
@@ -506,6 +542,12 @@ GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(elevated_refcount, 4)
 {
 	int i;
 
+	if (t->expect_pin_failure) {
+		TEST_ASSERT(__pin_pages(t->mem + test_page * page_size, page_size) < 0,
+			    "Pinning PFN-backed memory should fail");
+		return;
+	}
+
 	pin_pages(t->mem + test_page * page_size, page_size);
 
 	for (i = 0; i < nr_pages; i++)

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 51/88] KVM: guest_memfd: Support reporting PFN cacheability

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When memory slots are backed exclusively by guest_memfd without a host
userspace VMA, KVM cannot inspect VMA flags or page protections to
determine the caching requirements of the backing physical frames. Even
when a VMA exists, external PFN providers such as device drivers may
expose multiple regions with distinct caching properties within a single
provider instance.

Extend PFN attributes with a cacheability field whose zero value
represents normal write-back cacheable memory so existing RAM providers
require no changes. Distinguish three non-cacheable modes alongside
normal cacheable memory: strict uncached MMIO where guest
write-combining could trigger uncontained host faults, uncached MMIO on
buses such as PCIe where stage-2 can safely permit guest write-combining
while host mappings remain uncached, and write-combining memory.

Handle the reported cacheability in stage-2 fault paths:

- On arm64, select Device or Normal Non-Cacheable stage-2 attributes for
  non-cacheable frames, forbid instruction fetches on non-cacheable
  mappings, verify hardware cacheability support for non-linear-mapped
  cacheable frames, and reject non-cacheable frames for nested VNCR
  mappings which require cacheable host kernel mappings.
- On x86, stage-2 PTE construction already derives MMIO status from host
  PAT and E820 tables (and couples that status with host memory
  encryption bits), so warn once with a TODO if the provider-reported
  cacheability disagrees with x86's host MMIO detection.

Preserve confidential RAM invariants by rejecting non-cacheable frames
during initial payload population, and during SEV-SNP VMSA reloading or
RMP fault handling.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 arch/arm64/kvm/mmu.c        | 30 ++++++++++++++++++++++++++++--
 arch/arm64/kvm/nested.c     |  5 +++++
 arch/x86/kvm/mmu/mmu.c      |  8 ++++++++
 arch/x86/kvm/mmu/spte.c     |  2 +-
 arch/x86/kvm/mmu/spte.h     |  1 +
 arch/x86/kvm/svm/sev.c      |  9 ++++++++-
 include/linux/guest_memfd.h | 29 ++++++++++++++++++++++++++++-
 virt/kvm/guest_memfd.c      |  8 ++++++--
 8 files changed, 85 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 8f0217185973c..71fc823ed2076 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1789,14 +1789,40 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd,
 		return ret ?: -EFAULT;
 	}
 
+	if (!kvm_gmem_pfn_is_cacheable(pfn_attrs) && exec_fault) {
+		kfree(mapping);
+		return -ENOEXEC;
+	}
+
 	if (writable)
 		prot |= KVM_PGTABLE_PROT_W;
 
 	if (s2fd->nested)
 		prot = adjust_nested_fault_perms(s2fd->nested, prot);
 
-	if (exec_fault || cpus_have_final_cap(ARM64_HAS_CACHE_DIC))
-		prot |= KVM_PGTABLE_PROT_X;
+	switch (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_CACHE_MASK) {
+	case GUEST_MEMFD_PFN_ATTRS_CACHE_WB:
+		if (!pfn_is_map_memory(s2vi.pfn) &&
+		    !kvm_supports_cacheable_pfnmap()) {
+			if (!result)
+				kvm_prepare_memory_fault_exit(
+					s2fd->vcpu, gfn_to_gpa(canonical_gfn),
+					s2vi.vma_pagesize, write_fault,
+					exec_fault, false);
+			kfree(mapping);
+			return -EFAULT;
+		}
+		if (exec_fault || cpus_have_final_cap(ARM64_HAS_CACHE_DIC))
+			prot |= KVM_PGTABLE_PROT_X;
+		break;
+	case GUEST_MEMFD_PFN_ATTRS_CACHE_UC:
+		prot |= KVM_PGTABLE_PROT_DEVICE;
+		break;
+	case GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC:
+	case GUEST_MEMFD_PFN_ATTRS_CACHE_WC:
+		prot |= KVM_PGTABLE_PROT_NORMAL_NC;
+		break;
+	}
 
 	if (s2fd->nested)
 		prot = adjust_nested_exec_perms(kvm, s2fd->nested, prot);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 4519e92d89d66..a3818ece5c753 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1633,6 +1633,11 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 						      true, false, false);
 			return -EFAULT;
 		}
+
+		if (!kvm_gmem_pfn_is_cacheable(pfn_attrs)) {
+			fail_s1_walk(&vt->wr, ESR_ELx_FSC_EXTABT, false);
+			return -EINVAL;
+		}
 	}
 
 	/*
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 87deeeffd9919..23feb737c755b 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4695,6 +4695,14 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
 		kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
 		return -EFAULT;
 	}
+
+	/*
+	 * TODO: Perhaps resolve stage-2 cacheability from pfn_attrs instead of
+	 * relying on host MMIO PFN heuristics when constructing SPTEs?
+	 */
+	WARN_ON_ONCE(__kvm_is_mmio_pfn(fault->pfn) !=
+		     !kvm_gmem_pfn_is_cacheable(pfn_attrs));
+
 	fault->max_level = kvm_max_level_for_order(max_order);
 
 	return RET_PF_CONTINUE;
diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c
index 5fc27e9733b38..fd119ac2fad94 100644
--- a/arch/x86/kvm/mmu/spte.c
+++ b/arch/x86/kvm/mmu/spte.c
@@ -106,7 +106,7 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access)
 	return spte;
 }
 
-static bool __kvm_is_mmio_pfn(kvm_pfn_t pfn)
+bool __kvm_is_mmio_pfn(kvm_pfn_t pfn)
 {
 	if (pfn_valid(pfn))
 		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h
index e730717824b38..05c7bffa5d2ed 100644
--- a/arch/x86/kvm/mmu/spte.h
+++ b/arch/x86/kvm/mmu/spte.h
@@ -557,6 +557,7 @@ static inline u64 get_mmio_spte_generation(u64 spte)
 	return gen;
 }
 
+bool __kvm_is_mmio_pfn(kvm_pfn_t pfn);
 bool spte_needs_atomic_update(u64 spte);
 
 bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 5e37ea4bc7937..526f2a457cfe0 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4100,7 +4100,8 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL, &pfn_attrs))
 		return;
 
-	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)
+	if ((pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) ||
+	    !kvm_gmem_pfn_is_cacheable(pfn_attrs))
 		return;
 
 	read_lock(&kvm->mmu_lock);
@@ -5110,6 +5111,12 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 		return;
 	}
 
+	if (!kvm_gmem_pfn_is_cacheable(pfn_attrs)) {
+		pr_warn_ratelimited("SEV: Unexpected RMP fault, non-cacheable backing page for GPA 0x%llx\n",
+				    gpa);
+		return;
+	}
+
 	ret = snp_lookup_rmpentry(pfn, &assigned, &rmp_level);
 	if (ret || !assigned) {
 		guard(read_lock)(&kvm->mmu_lock);
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index fb2279a001ebf..8cdc334d1c9aa 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -2,6 +2,7 @@
 #ifndef _LINUX_GUEST_MEMFD_H
 #define _LINUX_GUEST_MEMFD_H
 
+#include <linux/bits.h>
 #include <linux/types.h>
 
 struct file;
@@ -9,7 +10,33 @@ struct folio;
 struct inode;
 struct mempolicy;
 
-#define GUEST_MEMFD_PFN_ATTRS_READONLY	(1ULL << 0)
+#define GUEST_MEMFD_PFN_ATTRS_READONLY		(1ULL << 0)
+
+/*
+ * Cacheability attributes for physical frames returned by providers
+ * (columns show x86 / arm64 memory types):
+ *
+ * Attribute                               | Host PTE       | Stage-2 PTE
+ * ----------------------------------------+----------------+---------------
+ * GUEST_MEMFD_PFN_ATTRS_CACHE_WB          | WB / Normal    | WB / Normal
+ * GUEST_MEMFD_PFN_ATTRS_CACHE_UC          | UC / Device    | UC / Device
+ * GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC | UC / Device    | UC / Normal-NC
+ * GUEST_MEMFD_PFN_ATTRS_CACHE_WC          | WC / Normal-NC | UC / Normal-NC
+ *
+ * GUEST_MEMFD_PFN_ATTRS_CACHE_WB is 0 so normal cacheable memory is the
+ * default when no cacheability bits are set.
+ */
+#define GUEST_MEMFD_PFN_ATTRS_CACHE_MASK	GENMASK_ULL(2, 1)
+#define GUEST_MEMFD_PFN_ATTRS_CACHE_WB		(0ULL << 1)
+#define GUEST_MEMFD_PFN_ATTRS_CACHE_UC		(1ULL << 1)
+#define GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC	(2ULL << 1)
+#define GUEST_MEMFD_PFN_ATTRS_CACHE_WC		(3ULL << 1)
+
+static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
+{
+	return (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_CACHE_MASK) ==
+	       GUEST_MEMFD_PFN_ATTRS_CACHE_WB;
+}
 
 /**
  * struct guest_memfd_provider_operations - Operations for external memory providers
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 023bf095340be..1fbb719f0e31f 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -1678,8 +1678,12 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 	if (folio)
 		folio_unlock(folio);
 
-	/* Populating pages initializes private memory, which cannot be read-only. */
-	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) {
+	/*
+	 * Populating pages initializes private memory, which cannot be
+	 * read-only or non-cacheable.
+	 */
+	if ((pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) ||
+	    !kvm_gmem_pfn_is_cacheable(pfn_attrs)) {
 		ret = -EPERM;
 		goto out_put_folio;
 	}

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 52/88] KVM: guest_memfd: Support mmap provider operation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Allow memory providers to customize VMA setup when userspace creates a
shared mapping on a guest_memfd file.

Add an optional .mmap callback to struct guest_memfd_provider_operations
and invoke it via gmem_provider_mmap() during mmap handling on
guest_memfd descriptors. If the provider does not define .mmap,
gmem_provider_mmap() simply returns 0.

Invoke the provider callback before setting VM_PFNMAP and guest_memfd
VMA operations so that PFN mode mappings always have VM_PFNMAP set and
remain mediated by guest_memfd fault handling even if the provider omits
the callback or configures its own VMA flags and operations.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h |  4 ++++
 virt/kvm/guest_memfd.c      | 13 +++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index 8cdc334d1c9aa..a2af279cd3c34 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -85,6 +85,9 @@ static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
  *                 available, the provider should skip the unbacked range. Not
  *                 called on kvm_gmem_revoke(). Do not take locks held across
  *                 kvm_gmem_revoke().
+ * @mmap: Optional callback to set up user mappings. Called when userspace calls
+ *        mmap() on guest_memfd. The provider can customize VMA flags, or reject
+ *        the mapping.
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -99,6 +102,7 @@ struct guest_memfd_provider_operations {
 	int (*lookup_pfn_range)(void *provider, pgoff_t index,
 				unsigned long *pfn, size_t *nr_pages);
 	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
+	int (*mmap)(void *provider, struct vm_area_struct *vma);
 };
 
 #ifdef CONFIG_KVM_GUEST_MEMFD
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 1fbb719f0e31f..ec908419e5faa 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -202,6 +202,14 @@ static void gmem_provider_release(struct gmem_inode *gi)
 		gi->provider_ops->release(gi->provider);
 }
 
+static int gmem_provider_mmap(struct gmem_inode *gi, struct vm_area_struct *vma)
+{
+	if (gi->provider_ops && gi->provider_ops->mmap)
+		return gi->provider_ops->mmap(gi->provider, vma);
+
+	return 0;
+}
+
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -860,6 +868,7 @@ static const struct vm_operations_struct kvm_gmem_pfn_vm_ops = {
 static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct gmem_inode *gi = GMEM_I(file_inode(file));
+	int ret;
 
 	if (!kvm_gmem_supports_mmap(file_inode(file)))
 		return -ENODEV;
@@ -869,6 +878,10 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
+	ret = gmem_provider_mmap(gi, vma);
+	if (ret)
+		return ret;
+
 	if (kvm_gmem_is_pfn_mode(gi))
 		vm_flags_set(vma, VM_PFNMAP);
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 53/88] KVM: guest_memfd: Support check_set_attributes provider operation

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Allow memory providers to validate or reject attribute changes on a
range of memory offsets before they are committed.

Add an optional .check_set_attributes callback to struct
guest_memfd_provider_operations returning an int error code, and invoke
it via gmem_provider_check_set_attributes() under the exclusive filemap
invalidation lock before preallocating maple tree entries or unmapping
userspace mappings. If the provider rejects the attribute change with an
error, fail the ioctl with that error code and report the error index
back to userspace.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h | 12 ++++++++++++
 virt/kvm/guest_memfd.c      | 16 ++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index a2af279cd3c34..cd634576c9af8 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -88,6 +88,16 @@ static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
  * @mmap: Optional callback to set up user mappings. Called when userspace calls
  *        mmap() on guest_memfd. The provider can customize VMA flags, or reject
  *        the mapping.
+ * @check_set_attributes: Optional callback to validate attribute changes on a
+ *                        range of indices before they are committed. Called
+ *                        with the exclusive filemap invalidate lock held. The
+ *                        provider can inspect the requested attributes and
+ *                        offset range and return 0 to allow the transition or
+ *                        a negative error code (e.g. -EINVAL or -EPERM) to
+ *                        reject it. Once @check_set_attributes succeeds, the
+ *                        provider must ensure this result stays valid and
+ *                        constant until guest_memfd stops using the range. Do
+ *                        not take locks held across kvm_gmem_revoke().
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -103,6 +113,8 @@ struct guest_memfd_provider_operations {
 				unsigned long *pfn, size_t *nr_pages);
 	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
 	int (*mmap)(void *provider, struct vm_area_struct *vma);
+	int (*check_set_attributes)(void *provider, pgoff_t index,
+				    size_t nr_pages, u64 attributes);
 };
 
 #ifdef CONFIG_KVM_GUEST_MEMFD
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index ec908419e5faa..61a3ff7fcd84c 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -210,6 +210,16 @@ static int gmem_provider_mmap(struct gmem_inode *gi, struct vm_area_struct *vma)
 	return 0;
 }
 
+static int gmem_provider_check_set_attributes(struct gmem_inode *gi, pgoff_t index,
+					      size_t nr_pages, u64 attributes)
+{
+	if (gi->provider_ops && gi->provider_ops->check_set_attributes)
+		return gi->provider_ops->check_set_attributes(gi->provider, index,
+							      nr_pages, attributes);
+
+	return 0;
+}
+
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -1083,6 +1093,12 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
 	if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs))
 		goto out;
 
+	r = gmem_provider_check_set_attributes(gi, start, nr_pages, attrs);
+	if (r) {
+		*err_index = start;
+		goto out;
+	}
+
 	mas_init(&mas, mt, start);
 	r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
 	if (r) {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 54/88] KVM: guest_memfd: Support read and write operations and provider callbacks

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Allow reading and writing to shared memory backed by guest_memfd
descriptors.

Consolidate locking, file offset bounds checking, and memory attribute
validation into a shared kvm_gmem_rw helper used by both kvm_gmem_read
and kvm_gmem_write.

To decouple low-level provider operations from userspace copy semantics
and avoid AB-BA deadlocks between mapping.invalidate_lock and
mm->mmap_lock, transfer data in chunks using a small stack bounce buffer:
- For writes, copy from userspace into the bounce buffer outside
  filemap_invalidate_lock_shared before invoking provider callbacks with
  a kernel buffer.
- For reads, invoke provider callbacks under filemap_invalidate_lock_shared
  into the bounce buffer, release the lock, and copy out to userspace.

Verify while holding the shared filemap invalidation lock that the
requested file offset range does not exceed inode size and consists
solely of shared memory.

Forward read and write calls to the provider via gmem_provider_read and
gmem_provider_write, providing placeholder stubs for the native provider
until reading and writing to native folios is implemented.

Enable FMODE_PREAD and FMODE_PWRITE on guest_memfd file descriptors so
pread64 and pwrite64 are permitted by the VFS.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 include/linux/guest_memfd.h |   4 ++
 virt/kvm/guest_memfd.c      | 115 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 119 insertions(+)

diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index cd634576c9af8..c85594b58112a 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -113,6 +113,10 @@ struct guest_memfd_provider_operations {
 				unsigned long *pfn, size_t *nr_pages);
 	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
 	int (*mmap)(void *provider, struct vm_area_struct *vma);
+	ssize_t (*read)(void *provider, void *buf, size_t count,
+			loff_t *ppos);
+	ssize_t (*write)(void *provider, const void *buf, size_t count,
+			 loff_t *ppos);
 	int (*check_set_attributes)(void *provider, pgoff_t index,
 				    size_t nr_pages, u64 attributes);
 };
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 61a3ff7fcd84c..385b517b49ed5 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -80,9 +80,25 @@ static struct folio *gmem_native_alloc_folio(void *provider, pgoff_t index,
 	return folio;
 }
 
+static ssize_t gmem_native_read(void *provider, void *buf, size_t count,
+				loff_t *ppos)
+{
+	/* TODO: Support reading from native provider folios. */
+	return -EINVAL;
+}
+
+static ssize_t gmem_native_write(void *provider, const void *buf,
+				 size_t count, loff_t *ppos)
+{
+	/* TODO: Support writing to native provider folios. */
+	return -EINVAL;
+}
+
 static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
 	.attach = gmem_native_attach,
 	.alloc_folio = gmem_native_alloc_folio,
+	.read = gmem_native_read,
+	.write = gmem_native_write,
 };
 
 static bool gmem_provider_ops_valid(struct gmem_inode *gi,
@@ -210,6 +226,25 @@ static int gmem_provider_mmap(struct gmem_inode *gi, struct vm_area_struct *vma)
 	return 0;
 }
 
+static ssize_t gmem_provider_read(struct gmem_inode *gi, void *buf,
+				  size_t count, loff_t *ppos)
+{
+	if (gi->provider_ops && gi->provider_ops->read)
+		return gi->provider_ops->read(gi->provider, buf, count, ppos);
+
+	return -EINVAL;
+}
+
+static ssize_t gmem_provider_write(struct gmem_inode *gi,
+				   const void *buf, size_t count,
+				   loff_t *ppos)
+{
+	if (gi->provider_ops && gi->provider_ops->write)
+		return gi->provider_ops->write(gi->provider, buf, count, ppos);
+
+	return -EINVAL;
+}
+
 static int gmem_provider_check_set_attributes(struct gmem_inode *gi, pgoff_t index,
 					      size_t nr_pages, u64 attributes)
 {
@@ -1204,7 +1239,86 @@ static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl,
 	}
 }
 
+static ssize_t kvm_gmem_rw(struct file *file, void __user *buf, size_t count,
+			   loff_t *ppos, bool is_write)
+{
+	struct inode *inode = file_inode(file);
+	struct gmem_inode *gi = GMEM_I(inode);
+	size_t total_done = 0;
+	loff_t isize;
+
+	if (!count)
+		return 0;
+
+	if (*ppos < 0)
+		return -EINVAL;
+
+	isize = i_size_read(inode);
+	if (*ppos >= isize)
+		return 0;
+
+	count = min_t(size_t, count, isize - *ppos);
+
+	while (count) {
+		u8 bounce[256];
+		size_t chunk = min_t(size_t, count, sizeof(bounce));
+		pgoff_t start = *ppos >> PAGE_SHIFT;
+		pgoff_t end = (*ppos + chunk - 1) >> PAGE_SHIFT;
+		pgoff_t nr_pages = end - start + 1;
+		ssize_t ret;
+
+		if (is_write) {
+			if (copy_from_user(bounce, (const char __user *)buf + total_done, chunk))
+				return total_done ?: -EFAULT;
+		}
+
+		filemap_invalidate_lock_shared(inode->i_mapping);
+
+		if (!kvm_gmem_range_has_attributes(inode, start, nr_pages, 0)) {
+			filemap_invalidate_unlock_shared(inode->i_mapping);
+			return total_done ?: -EFAULT;
+		}
+
+		if (is_write)
+			ret = gmem_provider_write(gi, bounce, chunk, ppos);
+		else
+			ret = gmem_provider_read(gi, bounce, chunk, ppos);
+
+		filemap_invalidate_unlock_shared(inode->i_mapping);
+
+		if (ret < 0)
+			return total_done ?: ret;
+
+		if (!ret)
+			break;
+
+		if (!is_write) {
+			if (copy_to_user((char __user *)buf + total_done, bounce, ret))
+				return total_done ?: -EFAULT;
+		}
+
+		total_done += ret;
+		count -= ret;
+	}
+
+	return total_done;
+}
+
+static ssize_t kvm_gmem_read(struct file *file, char __user *buf, size_t count,
+			     loff_t *ppos)
+{
+	return kvm_gmem_rw(file, buf, count, ppos, false);
+}
+
+static ssize_t kvm_gmem_write(struct file *file, const char __user *buf,
+			      size_t count, loff_t *ppos)
+{
+	return kvm_gmem_rw(file, (void __user *)buf, count, ppos, true);
+}
+
 static struct file_operations kvm_gmem_fops = {
+	.read		= kvm_gmem_read,
+	.write		= kvm_gmem_write,
 	.mmap		= kvm_gmem_mmap,
 	.open		= generic_file_open,
 	.release	= kvm_gmem_release,
@@ -1395,6 +1509,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int resour
 		goto err_inode;
 	}
 
+	file->f_mode |= FMODE_PREAD | FMODE_PWRITE;
 	file->f_flags |= O_LARGEFILE;
 	file->private_data = f;
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 55/88] vfio: Track device file access mode with an enum

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Device file operations currently track whether access has been granted
using a boolean flag, assuming that binding a device always unlocks both
control-plane operations and direct region access simultaneously.

Upcoming support for control-only device bindings requires distinguishing
unbound descriptors from descriptors bound with restricted vs. full
access, without introducing multiple flags that could get out of sync or
represent invalid combinations under lockless release-acquire ordering.

Replace the boolean flag with an enum representing the access mode so
that the entire access state continues to be published and read as a
single release-acquire value.

No functional change intended.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/device_cdev.c | 8 ++++----
 drivers/vfio/group.c       | 2 +-
 drivers/vfio/vfio.h        | 7 ++++++-
 drivers/vfio/vfio_main.c   | 8 ++++----
 4 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c
index c1c26d7a9aec9..19be3947fc74a 100644
--- a/drivers/vfio/device_cdev.c
+++ b/drivers/vfio/device_cdev.c
@@ -125,7 +125,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
 
 	mutex_lock(&device->dev_set->lock);
 	/* one device cannot be bound twice */
-	if (df->access_granted) {
+	if (df->access_mode != VFIO_DF_ACCESS_NONE) {
 		ret = -EINVAL;
 		goto out_unlock;
 	}
@@ -163,7 +163,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
 	 * Paired with smp_load_acquire() in vfio_device_fops::ioctl/
 	 * read/write/mmap
 	 */
-	smp_store_release(&df->access_granted, true);
+	smp_store_release(&df->access_mode, VFIO_DF_ACCESS_FULL);
 	mutex_unlock(&device->dev_set->lock);
 	return 0;
 
@@ -185,10 +185,10 @@ void vfio_df_unbind_iommufd(struct vfio_device_file *df)
 
 	/*
 	 * In the time of close, there is no contention with another one
-	 * changing this flag.  So read df->access_granted without lock
+	 * changing this mode.  So read df->access_mode without lock
 	 * and no smp_load_acquire() is ok.
 	 */
-	if (!df->access_granted)
+	if (df->access_mode == VFIO_DF_ACCESS_NONE)
 		return;
 
 	mutex_lock(&device->dev_set->lock);
diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c
index 185f359f75d38..0a33e0e025bd8 100644
--- a/drivers/vfio/group.c
+++ b/drivers/vfio/group.c
@@ -217,7 +217,7 @@ static int vfio_df_group_open(struct vfio_device_file *df)
 	 * Paired with smp_load_acquire() in vfio_device_fops::ioctl/
 	 * read/write/mmap and vfio_file_has_device_access()
 	 */
-	smp_store_release(&df->access_granted, true);
+	smp_store_release(&df->access_mode, VFIO_DF_ACCESS_FULL);
 
 	mutex_unlock(&device->dev_set->lock);
 	mutex_unlock(&device->group->group_lock);
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index 2eeb526a8da58..af2b8158b40c0 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -16,11 +16,16 @@ struct iommufd_ctx;
 struct iommu_group;
 struct vfio_container;
 
+enum vfio_df_access_mode {
+	VFIO_DF_ACCESS_NONE = 0,
+	VFIO_DF_ACCESS_FULL,
+};
+
 struct vfio_device_file {
 	struct vfio_device *device;
 	struct vfio_group *group;
 
-	u8 access_granted;
+	enum vfio_df_access_mode access_mode;
 	u32 devid; /* only valid when iommufd is valid */
 	spinlock_t kvm_ref_lock; /* protect kvm field */
 	struct file *kvm;
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index ea8e6cc13c457..fe126e137d5ca 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1349,7 +1349,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
 		return vfio_df_ioctl_bind_iommufd(df, uptr);
 
 	/* Paired with smp_store_release() following vfio_df_open() */
-	if (!smp_load_acquire(&df->access_granted))
+	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
 		return -EINVAL;
 
 	ret = vfio_device_pm_runtime_get(device);
@@ -1397,7 +1397,7 @@ static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
 	struct vfio_device *device = df->device;
 
 	/* Paired with smp_store_release() following vfio_df_open() */
-	if (!smp_load_acquire(&df->access_granted))
+	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
 		return -EINVAL;
 
 	if (unlikely(!device->ops->read))
@@ -1414,7 +1414,7 @@ static ssize_t vfio_device_fops_write(struct file *filep,
 	struct vfio_device *device = df->device;
 
 	/* Paired with smp_store_release() following vfio_df_open() */
-	if (!smp_load_acquire(&df->access_granted))
+	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
 		return -EINVAL;
 
 	if (unlikely(!device->ops->write))
@@ -1429,7 +1429,7 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
 	struct vfio_device *device = df->device;
 
 	/* Paired with smp_store_release() following vfio_df_open() */
-	if (!smp_load_acquire(&df->access_granted))
+	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
 		return -EINVAL;
 
 	if (unlikely(!device->ops->mmap))

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 56/88] vfio: Support VFIO_DEVICE_BIND_FLAG_CTRL_ONLY

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

In preparation for attaching a VFIO device descriptor to guest_memfd,
mappable device BAR accesses must be mediated exclusively through
guest_memfd so they obey guest_memfd's per-page shared and private
memory attribute tracking and revocation synchronization. In
confidential computing VMs, converting device pages to private assigns
the physical BAR frames to the guest in hardware protection tables
(such as RMP or GPT). If host userspace could bypass guest_memfd via
direct VFIO reads, writes, memory mappings, dma-buf exports, or
ioeventfds, host accesses to private device pages could trigger host
hardware protection faults and crash the host.

An alternative would be to override the file operations table on the
open descriptor to strip non-control handlers, or to block region access
later at guest_memfd attachment time. However, overriding file
operations on an open file is racy because VFS caches capability flags
at open time and dispatches file operations without synchronizing
against pointer updates, and deferring the restriction until attachment
races with concurrent memory mappings, reads, writes, or exports on an
already-enabled descriptor.

Instead, introduce VFIO_DEVICE_BIND_FLAG_CTRL_ONLY for
VFIO_DEVICE_BIND_IOMMUFD to restrict the bound device descriptor as soon
as the device is enabled for use by the bind ioctl, avoiding any race
window:

- Unconditionally disable memory mappings (mmap) on the descriptor with
  -EOPNOTSUPP, as memory mappings of BAR regions belong to guest_memfd.
- Add an is_ctrl_plane callback to struct vfio_device_ops to query
  whether a given file offset belongs to the control plane.
- For descriptors bound with VFIO_DEVICE_BIND_FLAG_CTRL_ONLY, gate
  direct reads and writes using is_ctrl_plane: permit reads and writes
  to control-plane regions (such as PCI configuration space) on the
  VFIO descriptor, while rejecting reads and writes to data-plane
  regions (mappable MMIO BARs) with -EINVAL. This preserves
  control-plane register configuration and virtualization outside
  guest_memfd's filemap invalidation lock while guaranteeing data-plane
  isolation.
- Disable BAR side-channel exports (ioeventfd, dma-buf and graphics plane
  export, live migration, and DMA logging) with -EOPNOTSUPP.

Control-plane ioctls for IOMMU page table management, capability
discovery, interrupt configuration, resets, power management, VF token
configuration, and bus-master control remain permitted via the VFIO
descriptor because none of them expose arbitrary BAR memory to
userspace:

- Capability discovery, IOMMU attachment, VF token setup, and bus-master
  control operate only on host metadata, IOMMU state, or PCI
  configuration space rather than BAR MMIO pages.
- Interrupt configuration is mediated by the host kernel, and any BAR
  pages backing hardware MSI-X tables must remain shared rather than
  private so host-managed interrupt updates never touch private frames.
- Device resets and power-state transitions do not grant userspace BAR
  access and coordinate with guest_memfd revocation before changing
  hardware state once attached.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/device_cdev.c       |   9 +++-
 drivers/vfio/pci/vfio_pci.c      |   1 +
 drivers/vfio/pci/vfio_pci_core.c |  13 +++++
 drivers/vfio/vfio.h              |   1 +
 drivers/vfio/vfio_main.c         | 105 ++++++++++++++++++++++++++++++++++-----
 include/linux/vfio.h             |   2 +
 include/linux/vfio_pci_core.h    |   1 +
 include/uapi/linux/vfio.h        |   7 +++
 8 files changed, 125 insertions(+), 14 deletions(-)

diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c
index 19be3947fc74a..e5b04e531ea97 100644
--- a/drivers/vfio/device_cdev.c
+++ b/drivers/vfio/device_cdev.c
@@ -92,7 +92,9 @@ static int vfio_df_check_token(struct vfio_device *device,
 long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
 				struct vfio_device_bind_iommufd __user *arg)
 {
-	const u32 VALID_FLAGS = VFIO_DEVICE_BIND_FLAG_TOKEN;
+	const u32 VALID_FLAGS = VFIO_DEVICE_BIND_FLAG_TOKEN |
+				VFIO_DEVICE_BIND_FLAG_CTRL_ONLY;
+	enum vfio_df_access_mode access_mode;
 	struct vfio_device *device = df->device;
 	struct vfio_device_bind_iommufd bind;
 	unsigned long minsz;
@@ -159,11 +161,14 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
 		goto out_close_device;
 
 	device->cdev_opened = true;
+	access_mode = (bind.flags & VFIO_DEVICE_BIND_FLAG_CTRL_ONLY) ?
+			      VFIO_DF_ACCESS_CTRL_ONLY :
+			      VFIO_DF_ACCESS_FULL;
 	/*
 	 * Paired with smp_load_acquire() in vfio_device_fops::ioctl/
 	 * read/write/mmap
 	 */
-	smp_store_release(&df->access_mode, VFIO_DF_ACCESS_FULL);
+	smp_store_release(&df->access_mode, access_mode);
 	mutex_unlock(&device->dev_set->lock);
 	return 0;
 
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 830369ff878db..3dc232ee65491 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -155,6 +155,7 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.ioctl		= vfio_pci_core_ioctl,
 	.get_region_info_caps = vfio_pci_ioctl_get_region_info,
 	.device_feature = vfio_pci_core_ioctl_feature,
+	.is_ctrl_plane	= vfio_pci_core_is_ctrl_plane,
 	.read		= vfio_pci_core_read,
 	.write		= vfio_pci_core_write,
 	.mmap		= vfio_pci_core_mmap,
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 6757054e9d875..907e6ab024acd 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1680,6 +1680,19 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 	return ret;
 }
 
+bool vfio_pci_core_is_ctrl_plane(struct vfio_device *core_vdev, loff_t ppos)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(ppos);
+
+	if (index < PCI_STD_NUM_BARS && vdev->bar_mmap_supported[index])
+		return false;
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_is_ctrl_plane);
+
 ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf,
 		size_t count, loff_t *ppos)
 {
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index af2b8158b40c0..62c5cbb77a899 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -18,6 +18,7 @@ struct vfio_container;
 
 enum vfio_df_access_mode {
 	VFIO_DF_ACCESS_NONE = 0,
+	VFIO_DF_ACCESS_CTRL_ONLY,
 	VFIO_DF_ACCESS_FULL,
 };
 
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index fe126e137d5ca..8734ae9d06c2b 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1231,8 +1231,23 @@ vfio_ioctl_device_feature_logging_report(struct vfio_device *device,
 	return ret;
 }
 
+static bool vfio_feature_ctrl_only_allowed(u32 flags)
+{
+	switch (flags & VFIO_DEVICE_FEATURE_MASK) {
+	case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN:
+	case VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY:
+	case VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY_WITH_WAKEUP:
+	case VFIO_DEVICE_FEATURE_LOW_POWER_EXIT:
+	case VFIO_DEVICE_FEATURE_BUS_MASTER:
+		return true;
+	default:
+		return false;
+	}
+}
+
 static int vfio_ioctl_device_feature(struct vfio_device *device,
-				     struct vfio_device_feature __user *arg)
+				     struct vfio_device_feature __user *arg,
+				     bool ctrl_only)
 {
 	size_t minsz = offsetofend(struct vfio_device_feature, flags);
 	struct vfio_device_feature feature;
@@ -1255,6 +1270,9 @@ static int vfio_ioctl_device_feature(struct vfio_device *device,
 	    (feature.flags & VFIO_DEVICE_FEATURE_GET))
 		return -EINVAL;
 
+	if (ctrl_only && !vfio_feature_ctrl_only_allowed(feature.flags))
+		return -EOPNOTSUPP;
+
 	switch (feature.flags & VFIO_DEVICE_FEATURE_MASK) {
 	case VFIO_DEVICE_FEATURE_MIGRATION:
 		return vfio_ioctl_device_feature_migration(
@@ -1337,20 +1355,63 @@ static long vfio_get_region_info(struct vfio_device *device,
 	return ret;
 }
 
+static bool vfio_ioctl_ctrl_only_allowed(unsigned int cmd)
+{
+	switch (cmd) {
+	case VFIO_DEVICE_ATTACH_IOMMUFD_PT:
+	case VFIO_DEVICE_DETACH_IOMMUFD_PT:
+	case VFIO_DEVICE_FEATURE:
+	case VFIO_DEVICE_GET_INFO:
+	case VFIO_DEVICE_GET_REGION_INFO:
+	case VFIO_DEVICE_GET_IRQ_INFO:
+	case VFIO_DEVICE_SET_IRQS:
+	case VFIO_DEVICE_RESET:
+	case VFIO_DEVICE_GET_PCI_HOT_RESET_INFO:
+	case VFIO_DEVICE_PCI_HOT_RESET:
+		return true;
+	default:
+		return false;
+	}
+}
+
+static int vfio_df_check_full_access(struct vfio_device_file *df)
+{
+	/* Paired with smp_store_release() following vfio_df_open() */
+	switch (smp_load_acquire(&df->access_mode)) {
+	case VFIO_DF_ACCESS_NONE:
+		return -EINVAL;
+	case VFIO_DF_ACCESS_CTRL_ONLY:
+		return -EOPNOTSUPP;
+	case VFIO_DF_ACCESS_FULL:
+		return 0;
+	}
+	return -EINVAL;
+}
+
 static long vfio_device_fops_unl_ioctl(struct file *filep,
 				       unsigned int cmd, unsigned long arg)
 {
 	struct vfio_device_file *df = filep->private_data;
 	struct vfio_device *device = df->device;
 	void __user *uptr = (void __user *)arg;
+	bool ctrl_only = false;
 	int ret;
 
 	if (cmd == VFIO_DEVICE_BIND_IOMMUFD)
 		return vfio_df_ioctl_bind_iommufd(df, uptr);
 
 	/* Paired with smp_store_release() following vfio_df_open() */
-	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
+	switch (smp_load_acquire(&df->access_mode)) {
+	case VFIO_DF_ACCESS_NONE:
 		return -EINVAL;
+	case VFIO_DF_ACCESS_CTRL_ONLY:
+		if (!vfio_ioctl_ctrl_only_allowed(cmd))
+			return -EOPNOTSUPP;
+		ctrl_only = true;
+		break;
+	case VFIO_DF_ACCESS_FULL:
+		break;
+	}
 
 	ret = vfio_device_pm_runtime_get(device);
 	if (ret)
@@ -1371,7 +1432,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
 
 	switch (cmd) {
 	case VFIO_DEVICE_FEATURE:
-		ret = vfio_ioctl_device_feature(device, uptr);
+		ret = vfio_ioctl_device_feature(device, uptr, ctrl_only);
 		break;
 
 	case VFIO_DEVICE_GET_REGION_INFO:
@@ -1390,15 +1451,33 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
 	return ret;
 }
 
+static int vfio_df_check_rw_access(struct vfio_device_file *df, loff_t pos)
+{
+	/* Paired with smp_store_release() following vfio_df_open() */
+	switch (smp_load_acquire(&df->access_mode)) {
+	case VFIO_DF_ACCESS_NONE:
+		return -EINVAL;
+	case VFIO_DF_ACCESS_CTRL_ONLY:
+		if (!df->device->ops->is_ctrl_plane ||
+		    !df->device->ops->is_ctrl_plane(df->device, pos))
+			return -EINVAL;
+		return 0;
+	case VFIO_DF_ACCESS_FULL:
+		return 0;
+	}
+	return -EINVAL;
+}
+
 static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
 				     size_t count, loff_t *ppos)
 {
 	struct vfio_device_file *df = filep->private_data;
 	struct vfio_device *device = df->device;
+	int ret;
 
-	/* Paired with smp_store_release() following vfio_df_open() */
-	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
-		return -EINVAL;
+	ret = vfio_df_check_rw_access(df, *ppos);
+	if (ret)
+		return ret;
 
 	if (unlikely(!device->ops->read))
 		return -EINVAL;
@@ -1412,10 +1491,11 @@ static ssize_t vfio_device_fops_write(struct file *filep,
 {
 	struct vfio_device_file *df = filep->private_data;
 	struct vfio_device *device = df->device;
+	int ret;
 
-	/* Paired with smp_store_release() following vfio_df_open() */
-	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
-		return -EINVAL;
+	ret = vfio_df_check_rw_access(df, *ppos);
+	if (ret)
+		return ret;
 
 	if (unlikely(!device->ops->write))
 		return -EINVAL;
@@ -1427,10 +1507,11 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
 {
 	struct vfio_device_file *df = filep->private_data;
 	struct vfio_device *device = df->device;
+	int ret;
 
-	/* Paired with smp_store_release() following vfio_df_open() */
-	if (smp_load_acquire(&df->access_mode) == VFIO_DF_ACCESS_NONE)
-		return -EINVAL;
+	ret = vfio_df_check_full_access(df);
+	if (ret)
+		return ret;
 
 	if (unlikely(!device->ops->mmap))
 		return -EINVAL;
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 0cc91c6f96d2b..c81373ddf2a9c 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -97,6 +97,7 @@ struct vfio_device {
  * @pasid_detach_ioas: Opposite of pasid_attach_ioas
  * @open_device: Called when the first file descriptor is opened for this device
  * @close_device: Opposite of open_device
+ * @is_ctrl_plane: Check if a region offset belongs to the control plane
  * @read: Perform read(2) on device file descriptor
  * @write: Perform write(2) on device file descriptor
  * @ioctl: Perform ioctl(2) on device file descriptor, supporting VFIO_DEVICE_*
@@ -127,6 +128,7 @@ struct vfio_device_ops {
 	void	(*pasid_detach_ioas)(struct vfio_device *vdev, u32 pasid);
 	int	(*open_device)(struct vfio_device *vdev);
 	void	(*close_device)(struct vfio_device *vdev);
+	bool	(*is_ctrl_plane)(struct vfio_device *vdev, loff_t ppos);
 	ssize_t	(*read)(struct vfio_device *vdev, char __user *buf,
 			size_t count, loff_t *ppos);
 	ssize_t	(*write)(struct vfio_device *vdev, const char __user *buf,
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa2..1053774d2d659 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -178,6 +178,7 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,
 				   struct vfio_region_info *info,
 				   struct vfio_info_cap *caps);
+bool vfio_pci_core_is_ctrl_plane(struct vfio_device *core_vdev, loff_t ppos);
 ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf,
 		size_t count, loff_t *ppos);
 ssize_t vfio_pci_core_write(struct vfio_device *core_vdev, const char __user *buf,
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index e41437fa17ad0..ea62d4e72d3bd 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -926,12 +926,19 @@ struct vfio_device_feature {
  * be provided the same token as VFIO_DEVICE_FEATURE_PCI_VF_TOKEN provided to
  * the PF.
  *
+ * VFIO_DEVICE_BIND_FLAG_CTRL_ONLY restricts the bound device fd to
+ * control-plane ioctls and non-mappable region read/write access (such as PCI
+ * configuration space), rejecting mmap(), read/write to mmap-supported BARs,
+ * and all other ioctls or device features with -EOPNOTSUPP. This should be used
+ * to prepare the device fd for attachment to guest_memfd.
+ *
  * Return: 0 on success, -errno on failure.
  */
 struct vfio_device_bind_iommufd {
 	__u32		argsz;
 	__u32		flags;
 #define VFIO_DEVICE_BIND_FLAG_TOKEN (1 << 0)
+#define VFIO_DEVICE_BIND_FLAG_CTRL_ONLY (1 << 1)
 	__s32		iommufd;
 	__u32		out_devid;
 	__aligned_u64	token_uuid_ptr;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 57/88] vfio: Dispatch guest_memfd provider operations via vfio_device_ops

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When a guest_memfd instance is created using a VFIO device descriptor as
its resource_fd, guest_memfd queries file_operations::gmem_provider_ops
on the resource file to attach the external memory provider.

Implement guest_memfd provider operations on vfio_device_fops to
dispatch .attach and .release to the bus driver via vfio_device_ops.

In .attach:
- Verify that the device descriptor was bound with
  VFIO_DEVICE_BIND_FLAG_CTRL_ONLY by checking that access_mode is
  VFIO_DF_ACCESS_CTRL_ONLY, returning -EINVAL if the descriptor was
  bound in full-access mode or has not been bound yet.
- Check and record attachment state centrally in struct vfio_device
  under dev_set->lock, rejecting attachment with -EBUSY if the device
  is already attached to a guest_memfd instance.
- Dispatch to the bus driver's .gmem_attach callback to let the driver
  perform bus-specific validation and resource reservation.
- Pin the device file across attachment via get_file(), dropping it on
  .release via fput().

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/vfio.h     |  7 ++++++
 2 files changed, 70 insertions(+)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 8734ae9d06c2b..618daf8bdaf80 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1535,6 +1535,66 @@ static void vfio_device_show_fdinfo(struct seq_file *m, struct file *filep)
 }
 #endif
 
+#ifdef CONFIG_KVM_GUEST_MEMFD
+static void *vfio_device_gmem_attach(struct inode *inode, struct file *file)
+{
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+	int ret;
+
+	/*
+	 * Attachment is only permitted if the device descriptor was bound
+	 * with VFIO_DEVICE_BIND_FLAG_CTRL_ONLY, ensuring direct region
+	 * accesses (read, write, mmap) and side-channel exports (such as
+	 * dma-buf or ioeventfd) were disabled from the outset.
+	 */
+	if (smp_load_acquire(&df->access_mode) != VFIO_DF_ACCESS_CTRL_ONLY)
+		return ERR_PTR(-EINVAL);
+
+	if (!device->ops->gmem_attach)
+		return ERR_PTR(-EOPNOTSUPP);
+
+	mutex_lock(&device->dev_set->lock);
+	if (device->gmem_inode) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	ret = device->ops->gmem_attach(device);
+	if (ret)
+		goto out_unlock;
+
+	device->gmem_inode = inode;
+out_unlock:
+	mutex_unlock(&device->dev_set->lock);
+
+	return ret ? ERR_PTR(ret) : get_file(file);
+}
+
+static void vfio_device_gmem_release(void *provider)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	mutex_lock(&device->dev_set->lock);
+
+	if (device->ops->gmem_release)
+		device->ops->gmem_release(device);
+
+	device->gmem_inode = NULL;
+
+	mutex_unlock(&device->dev_set->lock);
+
+	fput(file);
+}
+
+static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
+	.attach  = vfio_device_gmem_attach,
+	.release = vfio_device_gmem_release,
+};
+#endif
+
 const struct file_operations vfio_device_fops = {
 	.owner		= THIS_MODULE,
 	.open		= vfio_device_fops_cdev_open,
@@ -1547,6 +1607,9 @@ const struct file_operations vfio_device_fops = {
 #ifdef CONFIG_PROC_FS
 	.show_fdinfo	= vfio_device_show_fdinfo,
 #endif
+#ifdef CONFIG_KVM_GUEST_MEMFD
+	.gmem_provider_ops = &vfio_device_gmem_provider_ops,
+#endif
 };
 
 static struct vfio_device *vfio_device_from_file(struct file *file)
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index c81373ddf2a9c..65dc2cb523351 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -74,6 +74,9 @@ struct vfio_device {
 #endif
 	u8 cdev_opened:1;
 	u8 noiommu:1;
+#ifdef CONFIG_KVM_GUEST_MEMFD
+	struct inode *gmem_inode;
+#endif
 	/*
 	 * debug_root is a static property of the vfio_device
 	 * which must be set prior to registering the vfio_device.
@@ -114,7 +117,11 @@ struct vfio_device {
  *             this device is attached to.
  * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
  */
+struct guest_memfd_provider_operations;
+
 struct vfio_device_ops {
+	int (*gmem_attach)(struct vfio_device *device);
+	void (*gmem_release)(struct vfio_device *device);
 	char	*name;
 	int	(*init)(struct vfio_device *vdev);
 	void	(*release)(struct vfio_device *vdev);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 58/88] vfio/pci: Implement attach and release provider ops for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement guest_memfd provider operations in the generic PCI VFIO bus
driver to support attaching VFIO PCI devices to guest_memfd.

In the attachment path, acquire the device memory lock for write to
initialize the BAR availability flag according to whether PCI memory
decoding is active and runtime power management is not engaged. BAR
resources are already mapped and reserved during device open at bind
time.

In the release path, acquire the device memory lock for write and mark
BARs unavailable.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci.c      |  4 ++++
 drivers/vfio/pci/vfio_pci_core.c | 37 +++++++++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    |  5 +++++
 3 files changed, 46 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 3dc232ee65491..96f438671db29 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -168,6 +168,10 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.detach_ioas	= vfio_iommufd_physical_detach_ioas,
 	.pasid_attach_ioas	= vfio_iommufd_physical_pasid_attach_ioas,
 	.pasid_detach_ioas	= vfio_iommufd_physical_pasid_detach_ioas,
+#ifdef CONFIG_KVM_GUEST_MEMFD
+	.gmem_attach	= vfio_pci_core_gmem_attach,
+	.gmem_release	= vfio_pci_core_gmem_release,
+#endif
 };
 
 static const struct vfio_pci_device_ops vfio_pci_dev_ops = {
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 907e6ab024acd..c7b67fbce4cae 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -15,6 +15,7 @@
 #include <linux/device.h>
 #include <linux/eventfd.h>
 #include <linux/file.h>
+#include <linux/guest_memfd.h>
 #include <linux/interrupt.h>
 #include <linux/iommu.h>
 #include <linux/module.h>
@@ -2696,6 +2697,42 @@ static int __init vfio_pci_core_init(void)
 	return vfio_pci_init_perm_bits();
 }
 
+#ifdef CONFIG_KVM_GUEST_MEMFD
+int vfio_pci_core_gmem_attach(struct vfio_device *core_vdev)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	bool bars_available;
+
+	down_write(&vdev->memory_lock);
+
+	bars_available = __vfio_pci_memory_enabled(vdev) &&
+			 !vdev->pm_runtime_engaged;
+
+	/* Pairs with smp_load_acquire() in provider operations */
+	smp_store_release(&vdev->bars_available, bars_available);
+
+	up_write(&vdev->memory_lock);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_attach);
+
+void vfio_pci_core_gmem_release(struct vfio_device *core_vdev)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+
+	down_write(&vdev->memory_lock);
+
+	/* Pairs with smp_load_acquire() in provider operations */
+	smp_store_release(&vdev->bars_available, false);
+
+	up_write(&vdev->memory_lock);
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_release);
+#endif
+
 module_init(vfio_pci_core_init);
 module_exit(vfio_pci_core_cleanup);
 
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 1053774d2d659..c3c16459e14f4 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -149,6 +149,7 @@ struct vfio_pci_core_device {
 	struct notifier_block	nb;
 	struct rw_semaphore	memory_lock;
 	struct list_head	dmabufs;
+	bool			bars_available;
 };
 
 enum vfio_pci_io_width {
@@ -168,6 +169,10 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev);
 void vfio_pci_core_release_dev(struct vfio_device *core_vdev);
 int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev);
 void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev);
+#ifdef CONFIG_KVM_GUEST_MEMFD
+int vfio_pci_core_gmem_attach(struct vfio_device *core_vdev);
+void vfio_pci_core_gmem_release(struct vfio_device *core_vdev);
+#endif
 extern const struct pci_error_handlers vfio_pci_core_err_handlers;
 int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
 				  int nr_virtfn);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 59/88] vfio: Dispatch guest_memfd get_pfn provider op via vfio_device_ops

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When servicing guest stage-2 page faults or host userspace page faults
on guest_memfd memory backed by a VFIO resource descriptor, guest_memfd
invokes the .get_pfn provider operation under the shared filemap
invalidation lock.

Implement .get_pfn on vfio_device_gmem_provider_ops to dispatch to the
bus driver's .gmem_get_pfn callback on struct vfio_device_ops. Return
-EOPNOTSUPP if the bus driver does not implement the callback. The core
dispatch path avoids acquiring dev_set->lock to satisfy the requirement
that provider callbacks do not acquire locks held during revocation.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 16 ++++++++++++++++
 include/linux/vfio.h     |  3 +++
 2 files changed, 19 insertions(+)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 618daf8bdaf80..93bd0a200ad22 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1589,9 +1589,25 @@ static void vfio_device_gmem_release(void *provider)
 	fput(file);
 }
 
+static int vfio_device_gmem_get_pfn(void *provider, pgoff_t index,
+				    unsigned long *pfn, int *max_order,
+				    u64 *pfn_attrs)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	if (!device->ops->gmem_get_pfn)
+		return -EOPNOTSUPP;
+
+	return device->ops->gmem_get_pfn(device, index, pfn, max_order,
+					 pfn_attrs);
+}
+
 static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
 	.attach  = vfio_device_gmem_attach,
 	.release = vfio_device_gmem_release,
+	.get_pfn = vfio_device_gmem_get_pfn,
 };
 #endif
 
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 65dc2cb523351..51e04b980a381 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -122,6 +122,9 @@ struct guest_memfd_provider_operations;
 struct vfio_device_ops {
 	int (*gmem_attach)(struct vfio_device *device);
 	void (*gmem_release)(struct vfio_device *device);
+	int (*gmem_get_pfn)(struct vfio_device *device, pgoff_t index,
+			    unsigned long *pfn, int *max_order,
+			    u64 *pfn_attrs);
 	char	*name;
 	int	(*init)(struct vfio_device *vdev);
 	void	(*release)(struct vfio_device *vdev);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 60/88] vfio/pci: Implement get_pfn provider op for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement the .get_pfn provider operation in the generic PCI VFIO bus
driver to translate guest_memfd page offsets to physical BAR frames.

Check the BAR availability flag locklessly via smp_load_acquire() to
avoid acquiring the device memory lock on the fault path, returning
-EFAULT if memory decoding is disabled or the device is in low-power D3.

Decode the 40-bit VFIO region offset to identify the targeted BAR and
offset within the BAR, validating that the BAR is a standard memory BAR,
supports memory mapping, and the offset falls within the resource size.

Compute the physical frame number from the PCI resource start address
and report GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC as the frame's
caching attribute so stage-2 mappings permit guest write-combining while
host userspace mappings remain uncached.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci.c      |  1 +
 drivers/vfio/pci/vfio_pci_core.c | 38 ++++++++++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    |  3 +++
 3 files changed, 42 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 96f438671db29..9bc8009a9e766 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -171,6 +171,7 @@ static const struct vfio_device_ops vfio_pci_ops = {
 #ifdef CONFIG_KVM_GUEST_MEMFD
 	.gmem_attach	= vfio_pci_core_gmem_attach,
 	.gmem_release	= vfio_pci_core_gmem_release,
+	.gmem_get_pfn	= vfio_pci_core_gmem_get_pfn,
 #endif
 };
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index c7b67fbce4cae..9551a067b5689 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2731,6 +2731,44 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev)
 	up_write(&vdev->memory_lock);
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_release);
+
+int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
+			       unsigned long *pfn, int *max_order,
+			       u64 *pfn_attrs)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	struct pci_dev *pdev = vdev->pdev;
+	unsigned int bar;
+	u64 pgoff;
+
+	/*
+	 * Lockless check: if memory decoding is disabled or the device is in
+	 * low-power D3, fail immediately without blocking on memory_lock.
+	 */
+	if (!smp_load_acquire(&vdev->bars_available))
+		return -EFAULT;
+
+	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
+	pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+
+	if (bar >= PCI_STD_NUM_BARS || !vdev->bar_mmap_supported[bar])
+		return -EINVAL;
+
+	if (pgoff >= (pci_resource_len(pdev, bar) >> PAGE_SHIFT))
+		return -EINVAL;
+
+	*pfn = (pci_resource_start(pdev, bar) >> PAGE_SHIFT) + pgoff;
+
+	if (max_order)
+		*max_order = 0;
+
+	if (pfn_attrs)
+		*pfn_attrs = GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_get_pfn);
 #endif
 
 module_init(vfio_pci_core_init);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index c3c16459e14f4..3a6d039454472 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -172,6 +172,9 @@ void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev);
 #ifdef CONFIG_KVM_GUEST_MEMFD
 int vfio_pci_core_gmem_attach(struct vfio_device *core_vdev);
 void vfio_pci_core_gmem_release(struct vfio_device *core_vdev);
+int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
+			       unsigned long *pfn, int *max_order,
+			       u64 *pfn_attrs);
 #endif
 extern const struct pci_error_handlers vfio_pci_core_err_handlers;
 int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 61/88] vfio: Implement mmap provider op for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When userspace creates a shared mapping on a guest_memfd file backed by
a VFIO device descriptor, guest_memfd invokes the .mmap provider
operation to allow the provider to configure the VMA.

Implement .mmap on vfio_device_gmem_provider_ops by forwarding directly
to the bus driver's existing .mmap callback on struct vfio_device_ops.
This reuses existing driver mapping setup and region boundary
validation, such as PCI BAR checking and VMA flag configuration in
vfio-pci, without requiring bus drivers to duplicate mapping logic for
guest_memfd.

While the bus driver's mapping handler sets its own virtual memory
operations on the VMA, guest_memfd overrides them with its own PFN
virtual memory operations upon return, ensuring fault handling remains
mediated by guest_memfd attribute checks.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 93bd0a200ad22..979ed69c5ebf7 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1503,20 +1503,25 @@ static ssize_t vfio_device_fops_write(struct file *filep,
 	return device->ops->write(device, buf, count, ppos);
 }
 
+static int vfio_device_mmap(struct vfio_device *device,
+			    struct vm_area_struct *vma)
+{
+	if (unlikely(!device->ops->mmap))
+		return -EOPNOTSUPP;
+
+	return device->ops->mmap(device, vma);
+}
+
 static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
 {
 	struct vfio_device_file *df = filep->private_data;
-	struct vfio_device *device = df->device;
 	int ret;
 
 	ret = vfio_df_check_full_access(df);
 	if (ret)
 		return ret;
 
-	if (unlikely(!device->ops->mmap))
-		return -EINVAL;
-
-	return device->ops->mmap(device, vma);
+	return vfio_device_mmap(df->device, vma);
 }
 
 #ifdef CONFIG_PROC_FS
@@ -1604,10 +1609,23 @@ static int vfio_device_gmem_get_pfn(void *provider, pgoff_t index,
 					 pfn_attrs);
 }
 
+static int vfio_device_gmem_mmap(void *provider, struct vm_area_struct *vma)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+
+	/*
+	 * No control- vs. data-plane filtering is required because
+	 * vfio_device_mmap() already only allows mmap-ing the data plane.
+	 */
+	return vfio_device_mmap(df->device, vma);
+}
+
 static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
 	.attach  = vfio_device_gmem_attach,
 	.release = vfio_device_gmem_release,
 	.get_pfn = vfio_device_gmem_get_pfn,
+	.mmap    = vfio_device_gmem_mmap,
 };
 #endif
 

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 62/88] vfio: Dispatch guest_memfd check_set_attributes via vfio_device_ops

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When userspace requests attribute conversion on a guest_memfd file
backed by a VFIO resource descriptor, guest_memfd invokes the
.check_set_attributes provider operation under the exclusive filemap
invalidation lock before committing attribute changes.

Implement .check_set_attributes on vfio_device_gmem_provider_ops to
dispatch to the bus driver's .gmem_check_set_attributes callback on
struct vfio_device_ops if implemented, allowing the driver to inspect
and validate the requested attributes and offset range.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 24 ++++++++++++++++++++----
 include/linux/vfio.h     |  3 +++
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 979ed69c5ebf7..e4e7bfe77d03b 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1621,11 +1621,27 @@ static int vfio_device_gmem_mmap(void *provider, struct vm_area_struct *vma)
 	return vfio_device_mmap(df->device, vma);
 }
 
+static int vfio_device_gmem_check_set_attributes(void *provider, pgoff_t index,
+						 size_t nr_pages,
+						 u64 attributes)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	if (!device->ops->gmem_check_set_attributes)
+		return 0;
+
+	return device->ops->gmem_check_set_attributes(device, index, nr_pages,
+						      attributes);
+}
+
 static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
-	.attach  = vfio_device_gmem_attach,
-	.release = vfio_device_gmem_release,
-	.get_pfn = vfio_device_gmem_get_pfn,
-	.mmap    = vfio_device_gmem_mmap,
+	.attach               = vfio_device_gmem_attach,
+	.release              = vfio_device_gmem_release,
+	.get_pfn              = vfio_device_gmem_get_pfn,
+	.mmap                 = vfio_device_gmem_mmap,
+	.check_set_attributes = vfio_device_gmem_check_set_attributes,
 };
 #endif
 
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 51e04b980a381..ffb01e655735f 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -125,6 +125,9 @@ struct vfio_device_ops {
 	int (*gmem_get_pfn)(struct vfio_device *device, pgoff_t index,
 			    unsigned long *pfn, int *max_order,
 			    u64 *pfn_attrs);
+	int (*gmem_check_set_attributes)(struct vfio_device *device,
+					 pgoff_t index, size_t nr_pages,
+					 u64 attributes);
 	char	*name;
 	int	(*init)(struct vfio_device *vdev);
 	void	(*release)(struct vfio_device *vdev);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 63/88] vfio/pci: Implement check_set_attributes provider op for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement the .check_set_attributes provider operation in the generic
PCI VFIO bus driver to validate attribute conversion requests on
guest_memfd files backed by a VFIO PCI device.

When converting to private, decode the 40-bit VFIO region offset to
verify that the targeted BAR is a standard memory BAR and that the range
does not exceed the BAR's physical resource size.

Prevent private conversion of pages overlapping the device's MSI-X
vector table. The host kernel accesses MSI-X table entries directly to
configure and mask device interrupts; converting these pages to private
would cause subsequent host kernel interrupt operations to trigger fatal
hardware protection faults (such as RMP or GPT violations) on
confidential computing platforms.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci.c      |  7 ++++---
 drivers/vfio/pci/vfio_pci_core.c | 34 ++++++++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h | 10 ++++++++++
 drivers/vfio/pci/vfio_pci_rdwr.c |  4 +---
 include/linux/vfio_pci_core.h    |  3 +++
 5 files changed, 52 insertions(+), 6 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 9bc8009a9e766..f42422c1b7bc5 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -169,9 +169,10 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.pasid_attach_ioas	= vfio_iommufd_physical_pasid_attach_ioas,
 	.pasid_detach_ioas	= vfio_iommufd_physical_pasid_detach_ioas,
 #ifdef CONFIG_KVM_GUEST_MEMFD
-	.gmem_attach	= vfio_pci_core_gmem_attach,
-	.gmem_release	= vfio_pci_core_gmem_release,
-	.gmem_get_pfn	= vfio_pci_core_gmem_get_pfn,
+	.gmem_attach		= vfio_pci_core_gmem_attach,
+	.gmem_release		= vfio_pci_core_gmem_release,
+	.gmem_get_pfn		= vfio_pci_core_gmem_get_pfn,
+	.gmem_check_set_attributes = vfio_pci_core_gmem_check_set_attributes,
 #endif
 };
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 9551a067b5689..894b1332f8a0c 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -16,6 +16,7 @@
 #include <linux/eventfd.h>
 #include <linux/file.h>
 #include <linux/guest_memfd.h>
+#include <linux/kvm_host.h>
 #include <linux/interrupt.h>
 #include <linux/iommu.h>
 #include <linux/module.h>
@@ -2769,6 +2770,39 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 	return 0;
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_get_pfn);
+
+int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
+					    pgoff_t index, size_t nr_pages,
+					    u64 attributes)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	pgoff_t bar_pgoff;
+	size_t size;
+	loff_t pos;
+	int bar;
+
+	if (!(attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE))
+		return 0;
+
+	/*
+	 * Prevent private conversion of pages containing the MSI-X vector
+	 * table. The host kernel directly programs MSI-X table entries to
+	 * configure and mask interrupts; converting these pages to private
+	 * would cause host kernel writes to trigger fatal hardware protection
+	 * faults (e.g. RMP or GPT violations) on confidential compute platforms.
+	 */
+	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
+	bar_pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+	pos = (loff_t)bar_pgoff << PAGE_SHIFT;
+	size = nr_pages << PAGE_SHIFT;
+
+	if (vfio_pci_range_overlaps_msix(vdev, bar, pos, size))
+		return -EINVAL;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_check_set_attributes);
 #endif
 
 module_init(vfio_pci_core_init);
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 4e7162234a2eb..607a33270aca1 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -23,6 +23,16 @@ struct vfio_pci_ioeventfd {
 	bool			test_mem;
 };
 
+static inline bool vfio_pci_range_overlaps_msix(struct vfio_pci_core_device *vdev,
+						int bar, loff_t pos, size_t count)
+{
+	if (vdev->msix_bar == 0xFF || bar != vdev->msix_bar)
+		return false;
+
+	return !(pos + count <= vdev->msix_offset ||
+		 pos >= vdev->msix_offset + vdev->msix_size);
+}
+
 bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev);
 void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev);
 
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index 7f14dd46de17a..e5f1f60513709 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -418,9 +418,7 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset,
 		return -EINVAL;
 
 	/* Disallow ioeventfds working around MSI-X table writes */
-	if (bar == vdev->msix_bar &&
-	    !(pos + count <= vdev->msix_offset ||
-	      pos >= vdev->msix_offset + vdev->msix_size))
+	if (vfio_pci_range_overlaps_msix(vdev, bar, pos, count))
 		return -EINVAL;
 
 	if (count == 8)
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 3a6d039454472..6ff67e38eaf84 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -175,6 +175,9 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev);
 int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 			       unsigned long *pfn, int *max_order,
 			       u64 *pfn_attrs);
+int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
+					    pgoff_t index, size_t nr_pages,
+					    u64 attributes);
 #endif
 extern const struct pci_error_handlers vfio_pci_core_err_handlers;
 int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 64/88] vfio: Add vfio_device_gmem_revoke helper for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When physical device regions backing guest_memfd memory must be revoked
due to hardware reset, power state transitions, or memory decode state
changes, the bus driver must revoke and reclaim the memory ranges from
guest_memfd before modifying hardware state.

Add a vfio_device_gmem_revoke() helper in VFIO core that queries whether
the device is attached to a guest_memfd instance and invokes
kvm_gmem_revoke() with the specified range and physical frame address.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 10 ++++++++++
 include/linux/vfio.h     | 12 ++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index e4e7bfe77d03b..fd5f71dfc44b7 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1643,6 +1643,16 @@ static const struct guest_memfd_provider_operations vfio_device_gmem_provider_op
 	.mmap                 = vfio_device_gmem_mmap,
 	.check_set_attributes = vfio_device_gmem_check_set_attributes,
 };
+
+int vfio_device_gmem_revoke(struct vfio_device *device, pgoff_t start,
+			    size_t nr_pages, const unsigned long *pfn)
+{
+	if (!device->gmem_inode)
+		return 0;
+
+	return kvm_gmem_revoke(device->gmem_inode, start, nr_pages, pfn);
+}
+EXPORT_SYMBOL_GPL(vfio_device_gmem_revoke);
 #endif
 
 const struct file_operations vfio_device_fops = {
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index ffb01e655735f..aade091a91fe8 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -394,6 +394,18 @@ bool vfio_file_is_valid(struct file *file);
 bool vfio_file_enforced_coherent(struct file *file);
 void vfio_file_set_kvm(struct file *vfio_file, struct file *kvm);
 
+#ifdef CONFIG_KVM_GUEST_MEMFD
+int vfio_device_gmem_revoke(struct vfio_device *device, pgoff_t start,
+			    size_t nr_pages, const unsigned long *pfn);
+#else
+static inline int vfio_device_gmem_revoke(struct vfio_device *device,
+					  pgoff_t start, size_t nr_pages,
+					  const unsigned long *pfn)
+{
+	return -EOPNOTSUPP;
+}
+#endif
+
 #define VFIO_PIN_PAGES_MAX_ENTRIES	(PAGE_SIZE/sizeof(unsigned long))
 
 int vfio_pin_pages(struct vfio_device *device, dma_addr_t iova,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 65/88] vfio/pci: Revoke guest_memfd memory on BAR invalidation and reset

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When PCI hardware events occur (such as clearing PCI_COMMAND_MEMORY,
entering low-power D3, bus resets, or function-level resets), physical
BAR access is invalidated.

In vfio_pci_zap_bars(), if guest_memfd memory was marked available,
mark the BARs unavailable and invoke vfio_device_gmem_revoke() for each
mappable standard PCI BAR to revoke host userspace mappings and guest
stage-2 SPTEs before hardware state is modified.

When memory decoding is subsequently re-enabled and power state is
restored to D0 under the device memory lock, mark BARs available again
so future guest_memfd faults can proceed.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci_config.c | 16 ++++++++++----
 drivers/vfio/pci/vfio_pci_core.c   | 44 ++++++++++++++++++++++++++++++--------
 include/linux/vfio_pci_core.h      | 22 +++++++++++++++++++
 3 files changed, 69 insertions(+), 13 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 9914f3ac69aef..fae3980fb8d9c 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -630,8 +630,10 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
 		*virt_cmd &= cpu_to_le16(~mask);
 		*virt_cmd |= cpu_to_le16(new_cmd & mask);
 
-		if (__vfio_pci_memory_enabled(vdev))
+		if (__vfio_pci_memory_enabled(vdev)) {
 			vfio_pci_dma_buf_move(vdev, false);
+			vfio_pci_set_bars_available(vdev, true);
+		}
 		up_write(&vdev->memory_lock);
 	}
 
@@ -720,8 +722,10 @@ static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev,
 	}
 
 	vfio_pci_set_power_state(vdev, state);
-	if (__vfio_pci_memory_enabled(vdev))
+	if (__vfio_pci_memory_enabled(vdev)) {
 		vfio_pci_dma_buf_move(vdev, false);
+		vfio_pci_set_bars_available(vdev, true);
+	}
 	up_write(&vdev->memory_lock);
 }
 
@@ -911,8 +915,10 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
 			vfio_pci_dma_buf_move(vdev, true);
 			pci_try_reset_function(vdev->pdev);
-			if (__vfio_pci_memory_enabled(vdev))
+			if (__vfio_pci_memory_enabled(vdev)) {
 				vfio_pci_dma_buf_move(vdev, false);
+				vfio_pci_set_bars_available(vdev, true);
+			}
 			up_write(&vdev->memory_lock);
 		}
 	}
@@ -996,8 +1002,10 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
 			vfio_pci_dma_buf_move(vdev, true);
 			pci_try_reset_function(vdev->pdev);
-			if (__vfio_pci_memory_enabled(vdev))
+			if (__vfio_pci_memory_enabled(vdev)) {
 				vfio_pci_dma_buf_move(vdev, false);
+				vfio_pci_set_bars_available(vdev, true);
+			}
 			up_write(&vdev->memory_lock);
 		}
 	}
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 894b1332f8a0c..2b29d0ec6a0c9 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -464,8 +464,10 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev)
 	 */
 	down_write(&vdev->memory_lock);
 	__vfio_pci_runtime_pm_exit(vdev);
-	if (__vfio_pci_memory_enabled(vdev))
+	if (__vfio_pci_memory_enabled(vdev)) {
 		vfio_pci_dma_buf_move(vdev, false);
+		vfio_pci_set_bars_available(vdev, true);
+	}
 	up_write(&vdev->memory_lock);
 }
 
@@ -1337,8 +1339,10 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 
 	vfio_pci_dma_buf_move(vdev, true);
 	ret = pci_try_reset_function(vdev->pdev);
-	if (__vfio_pci_memory_enabled(vdev))
+	if (__vfio_pci_memory_enabled(vdev)) {
 		vfio_pci_dma_buf_move(vdev, false);
+		vfio_pci_set_bars_available(vdev, true);
+	}
 	up_write(&vdev->memory_lock);
 
 	return ret;
@@ -1727,8 +1731,31 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev)
 	loff_t start = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_BAR0_REGION_INDEX);
 	loff_t end = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_ROM_REGION_INDEX);
 	loff_t len = end - start;
+	int i;
 
 	unmap_mapping_range(core_vdev->inode->i_mapping, start, len, true);
+
+#ifdef CONFIG_KVM_GUEST_MEMFD
+	if (vfio_pci_get_bars_available(vdev)) {
+		vfio_pci_set_bars_available(vdev, false);
+
+		for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+			pgoff_t gmem_start;
+			unsigned long pfn;
+			size_t nr_pages;
+
+			if (!vdev->bar_mmap_supported[i])
+				continue;
+
+			gmem_start = VFIO_PCI_INDEX_TO_OFFSET(i) >> PAGE_SHIFT;
+			nr_pages = pci_resource_len(vdev->pdev, i) >> PAGE_SHIFT;
+			pfn = pci_resource_start(vdev->pdev, i) >> PAGE_SHIFT;
+
+			vfio_device_gmem_revoke(&vdev->vdev, gmem_start,
+						nr_pages, &pfn);
+		}
+	}
+#endif
 }
 
 void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev)
@@ -2621,8 +2648,10 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set,
 err_undo:
 	list_for_each_entry_from_reverse(vdev, &dev_set->device_list,
 					 vdev.dev_set_list) {
-		if (vdev->vdev.open_count && __vfio_pci_memory_enabled(vdev))
+		if (vdev->vdev.open_count && __vfio_pci_memory_enabled(vdev)) {
 			vfio_pci_dma_buf_move(vdev, false);
+			vfio_pci_set_bars_available(vdev, true);
+		}
 		up_write(&vdev->memory_lock);
 	}
 
@@ -2709,9 +2738,7 @@ int vfio_pci_core_gmem_attach(struct vfio_device *core_vdev)
 
 	bars_available = __vfio_pci_memory_enabled(vdev) &&
 			 !vdev->pm_runtime_engaged;
-
-	/* Pairs with smp_load_acquire() in provider operations */
-	smp_store_release(&vdev->bars_available, bars_available);
+	vfio_pci_set_bars_available(vdev, bars_available);
 
 	up_write(&vdev->memory_lock);
 
@@ -2726,8 +2753,7 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev)
 
 	down_write(&vdev->memory_lock);
 
-	/* Pairs with smp_load_acquire() in provider operations */
-	smp_store_release(&vdev->bars_available, false);
+	vfio_pci_set_bars_available(vdev, false);
 
 	up_write(&vdev->memory_lock);
 }
@@ -2747,7 +2773,7 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 	 * Lockless check: if memory decoding is disabled or the device is in
 	 * low-power D3, fail immediately without blocking on memory_lock.
 	 */
-	if (!smp_load_acquire(&vdev->bars_available))
+	if (!vfio_pci_get_bars_available(vdev))
 		return -EFAULT;
 
 	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 6ff67e38eaf84..af74766d7c90a 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -178,6 +178,28 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
 					    pgoff_t index, size_t nr_pages,
 					    u64 attributes);
+static inline void vfio_pci_set_bars_available(struct vfio_pci_core_device *vdev,
+					       bool available)
+{
+	/* Pairs with smp_load_acquire() in provider operations */
+	smp_store_release(&vdev->bars_available, available);
+}
+
+static inline bool vfio_pci_get_bars_available(struct vfio_pci_core_device *vdev)
+{
+	/* Pairs with smp_store_release() under memory_lock */
+	return smp_load_acquire(&vdev->bars_available);
+}
+#else
+static inline void vfio_pci_set_bars_available(struct vfio_pci_core_device *vdev,
+					       bool available)
+{
+}
+
+static inline bool vfio_pci_get_bars_available(struct vfio_pci_core_device *vdev)
+{
+	return false;
+}
 #endif
 extern const struct pci_error_handlers vfio_pci_core_err_handlers;
 int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 66/88] vfio: Dispatch guest_memfd lookup_pfn_range via vfio_device_ops

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When scanning guest_memfd memory ranges for confidential architecture
reclaim or conversion, guest_memfd invokes the .lookup_pfn_range
provider operation to locate backed PFN extents or skip unbacked holes
in bulk.

Implement .lookup_pfn_range on vfio_device_gmem_provider_ops to
dispatch to the bus driver's .gmem_lookup_pfn_range callback on struct
vfio_device_ops. Return -EOPNOTSUPP if the bus driver does not define
the callback.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 15 +++++++++++++++
 include/linux/vfio.h     |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index fd5f71dfc44b7..1c2811848c775 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1636,10 +1636,25 @@ static int vfio_device_gmem_check_set_attributes(void *provider, pgoff_t index,
 						      attributes);
 }
 
+static int vfio_device_gmem_lookup_pfn_range(void *provider, pgoff_t index,
+					     unsigned long *pfn,
+					     size_t *nr_pages)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	if (!device->ops->gmem_lookup_pfn_range)
+		return -EOPNOTSUPP;
+
+	return device->ops->gmem_lookup_pfn_range(device, index, pfn, nr_pages);
+}
+
 static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
 	.attach               = vfio_device_gmem_attach,
 	.release              = vfio_device_gmem_release,
 	.get_pfn              = vfio_device_gmem_get_pfn,
+	.lookup_pfn_range     = vfio_device_gmem_lookup_pfn_range,
 	.mmap                 = vfio_device_gmem_mmap,
 	.check_set_attributes = vfio_device_gmem_check_set_attributes,
 };
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index aade091a91fe8..8022adedc26a6 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -128,6 +128,8 @@ struct vfio_device_ops {
 	int (*gmem_check_set_attributes)(struct vfio_device *device,
 					 pgoff_t index, size_t nr_pages,
 					 u64 attributes);
+	int (*gmem_lookup_pfn_range)(struct vfio_device *device, pgoff_t index,
+				     unsigned long *pfn, size_t *nr_pages);
 	char	*name;
 	int	(*init)(struct vfio_device *vdev);
 	void	(*release)(struct vfio_device *vdev);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 67/88] vfio/pci: Implement lookup_pfn_range provider op for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Mappable PCI BARs are placed at sparse 40-bit VFIO region offsets
spanning 1 TiB per region slot, while unpopulated BARs and non-BAR
regions past the standard BAR namespace have no backing pages in
guest_memfd. Without reporting contiguous backed and unbacked extent
lengths, guest_memfd conversion and reclaim loops across sparse region
offsets would step one page at a time across multi-TiB holes.

Decode the 40-bit VFIO region offset to report contiguous extents:
- When the offset falls within a mappable standard PCI BAR, report the
  starting physical frame number and the remaining page count in that
  BAR.
- Otherwise, report the hole length up to the next mappable standard
  BAR, or to the end of the page-offset space when no further mappable
  BARs exist, so scan and reclaim loops skip multi-TiB gaps and
  trailing non-BAR regions in bulk.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci.c      |  1 +
 drivers/vfio/pci/vfio_pci_core.c | 42 ++++++++++++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    |  3 +++
 3 files changed, 46 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index f42422c1b7bc5..6f28f1ffbad0e 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -172,6 +172,7 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.gmem_attach		= vfio_pci_core_gmem_attach,
 	.gmem_release		= vfio_pci_core_gmem_release,
 	.gmem_get_pfn		= vfio_pci_core_gmem_get_pfn,
+	.gmem_lookup_pfn_range	= vfio_pci_core_gmem_lookup_pfn_range,
 	.gmem_check_set_attributes = vfio_pci_core_gmem_check_set_attributes,
 #endif
 };
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 2b29d0ec6a0c9..a9274d95e9066 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2829,6 +2829,48 @@ int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
 	return 0;
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_check_set_attributes);
+
+int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
+					pgoff_t index, unsigned long *pfn,
+					size_t *nr_pages)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	struct pci_dev *pdev = vdev->pdev;
+	u64 pgoff, bar_pages;
+	unsigned long bar;
+
+	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
+	pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+
+	if (bar < PCI_STD_NUM_BARS) {
+		bar_pages = pci_resource_len(pdev, bar) >> PAGE_SHIFT;
+		if (vdev->bar_mmap_supported[bar] && pgoff < bar_pages) {
+			*pfn = (pci_resource_start(pdev, bar) >> PAGE_SHIFT) +
+			       pgoff;
+			*nr_pages = bar_pages - pgoff;
+			return 0;
+		}
+
+		for (bar++; bar < PCI_STD_NUM_BARS; bar++) {
+			pgoff_t next_start;
+
+			if (!vdev->bar_mmap_supported[bar] ||
+			    !(pci_resource_len(pdev, bar) >> PAGE_SHIFT))
+				continue;
+
+			next_start = VFIO_PCI_INDEX_TO_OFFSET(bar) >>
+				     PAGE_SHIFT;
+			*nr_pages = next_start - index;
+			return -ENOENT;
+		}
+	}
+
+	/* No backed standard BARs remain at or above @index */
+	*nr_pages = max_t(size_t, ULONG_MAX - index, 1);
+	return -ENOENT;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_lookup_pfn_range);
 #endif
 
 module_init(vfio_pci_core_init);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index af74766d7c90a..b6e2fb93f9500 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -178,6 +178,9 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
 					    pgoff_t index, size_t nr_pages,
 					    u64 attributes);
+int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
+					pgoff_t index, unsigned long *pfn,
+					size_t *nr_pages);
 static inline void vfio_pci_set_bars_available(struct vfio_pci_core_device *vdev,
 					       bool available)
 {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 68/88] vfio/pci: Use iov_iter for MMIO read and write

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Refactor the internal MMIO read and write helpers in vfio_pci_rdwr to
operate on struct iov_iter rather than directly calling copy_to_user
and copy_from_user with user pointers.

Keep the existing vfio_pci_core_do_io_rw interface unchanged by
importing user buffers into an iov_iter via import_ubuf before calling
the internal implementation. The overhead of initializing the iov_iter
on userspace accesses is negligible compared to PCIe bus cycle latency.

This decouples low-level PCI MMIO access from userspace copy semantics,
allowing in-kernel callers to service I/O using kernel buffers (such
as ITER_KVEC) without user copy overhead or page fault recursion.

For delegation of read/write() to guest_memfd, this helps avoid AB-BA
deadlocks between mapping.invalidate_lock and mm->mmap_lock when MMIO read
and write operations are invoked under filemap invalidation locks.

No functional change intended.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci_rdwr.c | 107 ++++++++++++++++++++++++---------------
 1 file changed, 65 insertions(+), 42 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index e5f1f60513709..cc64722d1ab9a 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -91,34 +91,34 @@ VFIO_IOREAD(32)
 VFIO_IOREAD(64)
 
 #define VFIO_IORDWR(size)						\
-static int vfio_pci_iordwr##size(struct vfio_pci_core_device *vdev,\
-				bool iswrite, bool test_mem,		\
-				void __iomem *io, char __user *buf,	\
-				loff_t off, size_t *filled)		\
-{									\
-	u##size val;							\
-	int ret;							\
-									\
-	if (iswrite) {							\
-		if (copy_from_user(&val, buf, sizeof(val)))		\
-			return -EFAULT;					\
-									\
-		ret = vfio_pci_core_iowrite##size(vdev, test_mem,	\
-						  val, io + off);	\
-		if (ret)						\
-			return ret;					\
-	} else {							\
-		ret = vfio_pci_core_ioread##size(vdev, test_mem,	\
-						 &val, io + off);	\
-		if (ret)						\
-			return ret;					\
-									\
-		if (copy_to_user(buf, &val, sizeof(val)))		\
-			return -EFAULT;					\
-	}								\
-									\
-	*filled = sizeof(val);						\
-	return 0;							\
+static int vfio_pci_iordwr##size(struct vfio_pci_core_device *vdev,		\
+				bool iswrite, bool test_mem,			\
+				void __iomem *io, struct iov_iter *iter,	\
+				loff_t off, size_t *filled)			\
+{										\
+	u##size val;								\
+	int ret;								\
+										\
+	if (iswrite) {								\
+		if (!copy_from_iter_full(&val, sizeof(val), iter))		\
+			return -EFAULT;						\
+										\
+		ret = vfio_pci_core_iowrite##size(vdev, test_mem,		\
+						  val, io + off);		\
+		if (ret)							\
+			return ret;						\
+	} else {								\
+		ret = vfio_pci_core_ioread##size(vdev, test_mem,		\
+						 &val, io + off);		\
+		if (ret)							\
+			return ret;						\
+										\
+		if (!copy_to_iter_full(&val, sizeof(val), iter))		\
+			return -EFAULT;						\
+	}									\
+										\
+	*filled = sizeof(val);							\
+	return 0;								\
 }									\
 
 VFIO_IORDWR(8)
@@ -132,11 +132,12 @@ VFIO_IORDWR(64)
  * reads with -1.  This is intended for handling MSI-X vector tables and
  * leftover space for ROM BARs.
  */
-ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
-			       void __iomem *io, char __user *buf,
-			       loff_t off, size_t count, size_t x_start,
-			       size_t x_end, bool iswrite,
-			       enum vfio_pci_io_width max_width)
+static ssize_t __vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev,
+					bool test_mem, void __iomem *io,
+					struct iov_iter *iter, loff_t off,
+					size_t count, size_t x_start,
+					size_t x_end, bool iswrite,
+					enum vfio_pci_io_width max_width)
 {
 	ssize_t done = 0;
 	int ret;
@@ -153,25 +154,25 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
 
 		if (fillable >= 8 && !(off % 8) && max_width >= 8) {
 			ret = vfio_pci_iordwr64(vdev, iswrite, test_mem,
-						io, buf, off, &filled);
+						io, iter, off, &filled);
 			if (ret)
 				return ret;
 
 		} else if (fillable >= 4 && !(off % 4) && max_width >= 4) {
 			ret = vfio_pci_iordwr32(vdev, iswrite, test_mem,
-						io, buf, off, &filled);
+						io, iter, off, &filled);
 			if (ret)
 				return ret;
 
 		} else if (fillable >= 2 && !(off % 2) && max_width >= 2) {
 			ret = vfio_pci_iordwr16(vdev, iswrite, test_mem,
-						io, buf, off, &filled);
+						io, iter, off, &filled);
 			if (ret)
 				return ret;
 
 		} else if (fillable) {
 			ret = vfio_pci_iordwr8(vdev, iswrite, test_mem,
-					       io, buf, off, &filled);
+					       io, iter, off, &filled);
 			if (ret)
 				return ret;
 
@@ -179,23 +180,45 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
 			/* Fill reads with -1, drop writes */
 			filled = min(count, (size_t)(x_end - off));
 			if (!iswrite) {
-				u8 val = 0xFF;
-				size_t i;
+				static const u8 ff_buf[64] = { [0 ... 63] = 0xFF };
+				size_t cur = filled;
+
+				while (cur) {
+					size_t step = min(cur, sizeof(ff_buf));
 
-				for (i = 0; i < filled; i++)
-					if (copy_to_user(buf + i, &val, 1))
+					if (!copy_to_iter_full(ff_buf, step, iter))
 						return -EFAULT;
+					cur -= step;
+				}
+			} else {
+				iov_iter_advance(iter, filled);
 			}
 		}
 
 		count -= filled;
 		done += filled;
 		off += filled;
-		buf += filled;
 	}
 
 	return done;
 }
+
+ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
+			       void __iomem *io, char __user *buf,
+			       loff_t off, size_t count, size_t x_start,
+			       size_t x_end, bool iswrite,
+			       enum vfio_pci_io_width max_width)
+{
+	struct iov_iter iter;
+	int ret;
+
+	ret = import_ubuf(iswrite ? ITER_SOURCE : ITER_DEST, buf, count, &iter);
+	if (ret)
+		return ret;
+
+	return __vfio_pci_core_do_io_rw(vdev, test_mem, io, &iter, off, count,
+					x_start, x_end, iswrite, max_width);
+}
 EXPORT_SYMBOL_GPL(vfio_pci_core_do_io_rw);
 
 ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 69/88] vfio: Dispatch guest_memfd read and write provider ops via vfio_device_ops

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When userspace reads or writes to a guest_memfd descriptor backed by a
VFIO device descriptor, guest_memfd invokes the .read and .write
provider operations under the shared filemap invalidation lock.

Implement .read and .write on vfio_device_gmem_provider_ops to dispatch
to the bus driver's .gmem_read and .gmem_write callbacks on struct
vfio_device_ops. The callbacks operate on kernel buffers (void *buf and
const void *buf), as guest_memfd bounces userspace transfers outside
its filemap invalidation lock. Return -EINVAL if the bus driver does
not define the respective callback.

Enforce the partition between direct VFIO descriptor access and
guest_memfd delegation: query is_ctrl_plane on the target file offset,
rejecting reads and writes to control-plane offsets (such as PCI
configuration space) with -EINVAL so guest_memfd only services
data-plane regions complementary to what direct VFIO descriptor accesses
allow in control-only mode.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/vfio_main.c | 36 ++++++++++++++++++++++++++++++++++++
 include/linux/vfio.h     |  4 ++++
 2 files changed, 40 insertions(+)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 1c2811848c775..96a768b5ec71b 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1650,11 +1650,47 @@ static int vfio_device_gmem_lookup_pfn_range(void *provider, pgoff_t index,
 	return device->ops->gmem_lookup_pfn_range(device, index, pfn, nr_pages);
 }
 
+static ssize_t vfio_device_gmem_read(void *provider, void *buf,
+				     size_t count, loff_t *ppos)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	if (!device->ops->gmem_read)
+		return -EINVAL;
+
+	if (device->ops->is_ctrl_plane &&
+	    device->ops->is_ctrl_plane(device, *ppos))
+		return -EINVAL;
+
+	return device->ops->gmem_read(device, buf, count, ppos);
+}
+
+static ssize_t vfio_device_gmem_write(void *provider, const void *buf,
+				      size_t count, loff_t *ppos)
+{
+	struct file *file = provider;
+	struct vfio_device_file *df = file->private_data;
+	struct vfio_device *device = df->device;
+
+	if (!device->ops->gmem_write)
+		return -EINVAL;
+
+	if (device->ops->is_ctrl_plane &&
+	    device->ops->is_ctrl_plane(device, *ppos))
+		return -EINVAL;
+
+	return device->ops->gmem_write(device, buf, count, ppos);
+}
+
 static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
 	.attach               = vfio_device_gmem_attach,
 	.release              = vfio_device_gmem_release,
 	.get_pfn              = vfio_device_gmem_get_pfn,
 	.lookup_pfn_range     = vfio_device_gmem_lookup_pfn_range,
+	.read                 = vfio_device_gmem_read,
+	.write                = vfio_device_gmem_write,
 	.mmap                 = vfio_device_gmem_mmap,
 	.check_set_attributes = vfio_device_gmem_check_set_attributes,
 };
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 8022adedc26a6..012a233a375ab 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -130,6 +130,10 @@ struct vfio_device_ops {
 					 u64 attributes);
 	int (*gmem_lookup_pfn_range)(struct vfio_device *device, pgoff_t index,
 				     unsigned long *pfn, size_t *nr_pages);
+	ssize_t (*gmem_read)(struct vfio_device *device, void *buf,
+			     size_t count, loff_t *ppos);
+	ssize_t (*gmem_write)(struct vfio_device *device, const void *buf,
+			      size_t count, loff_t *ppos);
 	char	*name;
 	int	(*init)(struct vfio_device *vdev);
 	void	(*release)(struct vfio_device *vdev);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 70/88] vfio/pci: Introduce vfio_pci_bar_rw_kbuf for kernel buffer MMIO access

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Refactor BAR read and write handling to separate the core MMIO logic
from the buffer source.

Extract the core BAR read and write implementation into a static helper
that operates on an iov_iter and accepts a test_mem parameter controlling
whether memory decoding and power state should be validated under
memory_lock. Retain the userspace interface, importing the user buffer
into an iov_iter with test_mem enabled.

Introduce a counterpart helper taking a kernel buffer and setting up
an ITER_KVEC iov_iter, allowing callers with kernel buffers to read
and write to PCI BARs without userspace pointer conversion or copy
overhead, and to specify whether memory testing is required.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci_priv.h |  3 +++
 drivers/vfio/pci/vfio_pci_rdwr.c | 34 ++++++++++++++++++++++++++++++----
 2 files changed, 33 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 607a33270aca1..48a068e68aab8 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -53,6 +53,9 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,
 
 ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			size_t count, loff_t *ppos, bool iswrite);
+ssize_t vfio_pci_bar_rw_kbuf(struct vfio_pci_core_device *vdev, void *kbuf,
+			     size_t count, loff_t *ppos, bool iswrite,
+			     bool test_mem);
 
 #ifdef CONFIG_VFIO_PCI_VGA
 ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *buf,
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index cc64722d1ab9a..c44c78e48e6aa 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -221,8 +221,9 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_do_io_rw);
 
-ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
-			size_t count, loff_t *ppos, bool iswrite)
+static ssize_t __vfio_pci_bar_rw(struct vfio_pci_core_device *vdev,
+				 struct iov_iter *iter, size_t count,
+				 loff_t *ppos, bool iswrite, bool test_mem)
 {
 	struct pci_dev *pdev = vdev->pdev;
 	loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
@@ -284,8 +285,9 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 		x_end = vdev->msix_offset + vdev->msix_size;
 	}
 
-	done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM, io, buf, pos,
-				      count, x_start, x_end, iswrite, max_width);
+	done = __vfio_pci_core_do_io_rw(vdev, test_mem && (res->flags & IORESOURCE_MEM),
+					io, iter, pos, count, x_start, x_end,
+					iswrite, max_width);
 
 	if (done >= 0)
 		*ppos += done;
@@ -301,6 +303,30 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 	return done;
 }
 
+ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
+			size_t count, loff_t *ppos, bool iswrite)
+{
+	struct iov_iter iter;
+	int ret;
+
+	ret = import_ubuf(iswrite ? ITER_SOURCE : ITER_DEST, buf, count, &iter);
+	if (ret)
+		return ret;
+
+	return __vfio_pci_bar_rw(vdev, &iter, count, ppos, iswrite, true);
+}
+
+ssize_t vfio_pci_bar_rw_kbuf(struct vfio_pci_core_device *vdev, void *kbuf,
+			     size_t count, loff_t *ppos, bool iswrite,
+			     bool test_mem)
+{
+	struct kvec kvec = { .iov_base = kbuf, .iov_len = count };
+	struct iov_iter iter;
+
+	iov_iter_kvec(&iter, iswrite ? ITER_SOURCE : ITER_DEST, &kvec, 1, count);
+	return __vfio_pci_bar_rw(vdev, &iter, count, ppos, iswrite, test_mem);
+}
+
 #ifdef CONFIG_VFIO_PCI_VGA
 ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			       size_t count, loff_t *ppos, bool iswrite)

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 71/88] vfio/pci: Implement read and write provider ops for guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Implement the .gmem_read and .gmem_write provider operations in
vfio-pci to service read and write accesses forwarded by guest_memfd.

When guest_memfd delegates BAR access, it already holds the shared
filemap invalidation lock and serializes against memory attribute
conversions and hardware revocations.

Restrict guest_memfd reads and writes strictly to mappable MMIO BARs
(indices < PCI_STD_NUM_BARS with bar_mmap_supported[index] set),
rejecting all other regions with -EINVAL.

Verify locklessly that BARs are available via bars_available before
accessing MMIO space. Forward to vfio_pci_bar_rw_kbuf with test_mem
set to false to skip acquiring memory_lock and runtime PM, preventing
AB-BA deadlocks against concurrent revocations during resets, power
state transitions, or memory decoding disablement.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 drivers/vfio/pci/vfio_pci.c      |  2 ++
 drivers/vfio/pci/vfio_pci_core.c | 36 ++++++++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    |  5 +++++
 3 files changed, 43 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 6f28f1ffbad0e..e5e2dbca152ec 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -174,6 +174,8 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.gmem_get_pfn		= vfio_pci_core_gmem_get_pfn,
 	.gmem_lookup_pfn_range	= vfio_pci_core_gmem_lookup_pfn_range,
 	.gmem_check_set_attributes = vfio_pci_core_gmem_check_set_attributes,
+	.gmem_read		= vfio_pci_core_gmem_read,
+	.gmem_write		= vfio_pci_core_gmem_write,
 #endif
 };
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index a9274d95e9066..c976574782663 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2871,6 +2871,42 @@ int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
 	return -ENOENT;
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_lookup_pfn_range);
+
+static ssize_t vfio_pci_core_gmem_rw(struct vfio_device *core_vdev,
+				     void *kbuf, size_t count,
+				     loff_t *ppos, bool iswrite)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
+
+	if (!count)
+		return 0;
+
+	if (index >= PCI_STD_NUM_BARS || !vdev->bar_mmap_supported[index])
+		return -EINVAL;
+
+	/* Lockless check on availability of BARs. */
+	if (!vfio_pci_get_bars_available(vdev))
+		return -EIO;
+
+	return vfio_pci_bar_rw_kbuf(vdev, kbuf, count, ppos, iswrite, false);
+}
+
+ssize_t vfio_pci_core_gmem_read(struct vfio_device *core_vdev, void *buf,
+				size_t count, loff_t *ppos)
+{
+	return vfio_pci_core_gmem_rw(core_vdev, buf, count, ppos, false);
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_read);
+
+ssize_t vfio_pci_core_gmem_write(struct vfio_device *core_vdev,
+				 const void *buf, size_t count,
+				 loff_t *ppos)
+{
+	return vfio_pci_core_gmem_rw(core_vdev, (void *)buf, count, ppos, true);
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_write);
 #endif
 
 module_init(vfio_pci_core_init);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index b6e2fb93f9500..03cb0b41855dd 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -181,6 +181,11 @@ int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
 int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
 					pgoff_t index, unsigned long *pfn,
 					size_t *nr_pages);
+ssize_t vfio_pci_core_gmem_read(struct vfio_device *core_vdev, void *buf,
+				size_t count, loff_t *ppos);
+ssize_t vfio_pci_core_gmem_write(struct vfio_device *core_vdev,
+				 const void *buf, size_t count,
+				 loff_t *ppos);
 static inline void vfio_pci_set_bars_available(struct vfio_pci_core_device *vdev,
 					       bool available)
 {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 72/88] KVM: selftests: Add initial VFIO test verifying device info

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a standalone selftest to verify that an emulated or physical
Intel 82576 network card bound to the vfio cdev interface can be opened,
bound to iommufd, and queried for device information without requiring
an attached IOMMU page table or a running virtual machine.

Structure test setup and teardown into a kselftest harness fixture,
skipping cleanly if the device or /dev/iommu is unavailable, and
verifying that queried device flags reflect PCI and reset capabilities.
---
 tools/testing/selftests/kvm/Makefile.kvm           |   1 +
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 153 +++++++++++++++++++++
 2 files changed, 154 insertions(+)

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 5f7118bddfc36..05f87764b71e7 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -162,6 +162,7 @@ TEST_GEN_PROGS_x86 += x86/triple_fault_event_test
 TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test
 TEST_GEN_PROGS_x86 += x86/aperfmperf_test
 TEST_GEN_PROGS_x86 += x86/guest_memfd_conversions_test
+TEST_GEN_PROGS_x86 += x86/guest_memfd_vfio_test
 TEST_GEN_PROGS_x86 += access_tracking_perf_test
 TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
new file mode 100644
index 0000000000000..669acc6058b5b
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -0,0 +1,153 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Minimal standalone test to verify an emulated or physical Intel 82576 (igb)
+ * NIC bound to vfio-pci via IOMMUFD cdev, without creating a KVM guest.
+ */
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/types.h>
+
+#include <linux/iommufd.h>
+#include <linux/pci_regs.h>
+#include <linux/types.h>
+#include <linux/compiler.h>
+#include <linux/vfio.h>
+
+#include "kselftest_harness.h"
+
+#define PCI_VENDOR_ID_INTEL 0x8086
+#define PCI_DEVICE_ID_INTEL_82576 0x10C9
+
+/* Intel 82576 BAR0 register offsets */
+#define E1000_CTRL	0x00000 /* Device Control - RW */
+#define E1000_STATUS	0x00008 /* Device Status - RO */
+#define E1000_RAL0	0x05400 /* Receive Address Low (0) - RW */
+#define E1000_RAH0	0x05404 /* Receive Address High (0) - RW */
+
+static char *find_igb_cdev_path(void)
+{
+	const char *sys_pci = "/sys/bus/pci/devices";
+	struct dirent *dev_entry;
+	DIR *dir = opendir(sys_pci);
+	char cdev_path[PATH_MAX];
+	char path[PATH_MAX];
+
+	if (!dir)
+		return NULL;
+
+	while ((dev_entry = readdir(dir)) != NULL) {
+		FILE *f;
+		uint32_t vendor = 0, device = 0;
+		DIR *vfio_dir;
+		struct dirent *vfio_entry;
+
+		if (dev_entry->d_name[0] == '.')
+			continue;
+
+		snprintf(path, sizeof(path), "%s/%s/vendor", sys_pci, dev_entry->d_name);
+		f = fopen(path, "r");
+		if (!f)
+			continue;
+		if (fscanf(f, "0x%x", &vendor) != 1) {
+			fclose(f);
+			continue;
+		}
+		fclose(f);
+
+		snprintf(path, sizeof(path), "%s/%s/device", sys_pci, dev_entry->d_name);
+		f = fopen(path, "r");
+		if (!f)
+			continue;
+		if (fscanf(f, "0x%x", &device) != 1) {
+			fclose(f);
+			continue;
+		}
+		fclose(f);
+
+		if (vendor != PCI_VENDOR_ID_INTEL || device != PCI_DEVICE_ID_INTEL_82576)
+			continue;
+
+		/* Found an Intel 82576 device. Check if bound to vfio-pci cdev. */
+		snprintf(path, sizeof(path), "%s/%s/vfio-dev", sys_pci, dev_entry->d_name);
+		vfio_dir = opendir(path);
+		if (!vfio_dir)
+			continue;
+
+		while ((vfio_entry = readdir(vfio_dir)) != NULL) {
+			if (strncmp(vfio_entry->d_name, "vfio", 4) == 0) {
+				snprintf(cdev_path, sizeof(cdev_path), "/dev/vfio/devices/%s",
+					 vfio_entry->d_name);
+				closedir(vfio_dir);
+				closedir(dir);
+				return strdup(cdev_path);
+			}
+		}
+		closedir(vfio_dir);
+	}
+
+	closedir(dir);
+	return NULL;
+}
+
+FIXTURE(guest_memfd_vfio) {
+	char *cdev_path;
+	int iommufd;
+	int vfio_fd;
+};
+
+FIXTURE_SETUP(guest_memfd_vfio)
+{
+	struct vfio_device_bind_iommufd bind_args = {};
+	int ret;
+
+	self->cdev_path = find_igb_cdev_path();
+	if (!self->cdev_path)
+		SKIP(return, "Intel 82576 device bound to vfio not found\n");
+
+	self->iommufd = open("/dev/iommu", O_RDWR);
+	if (self->iommufd < 0) {
+		free(self->cdev_path);
+		self->cdev_path = NULL;
+		SKIP(return, "Failed to open /dev/iommu\n");
+	}
+
+	self->vfio_fd = open(self->cdev_path, O_RDWR);
+	ASSERT_GE(self->vfio_fd, 0);
+
+	bind_args.argsz = sizeof(bind_args);
+	bind_args.iommufd = self->iommufd;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_BIND_IOMMUFD, &bind_args);
+	ASSERT_EQ(0, ret);
+}
+
+FIXTURE_TEARDOWN(guest_memfd_vfio)
+{
+	if (self->vfio_fd >= 0)
+		close(self->vfio_fd);
+	if (self->iommufd >= 0)
+		close(self->iommufd);
+	free(self->cdev_path);
+}
+
+TEST_F(guest_memfd_vfio, get_device_info)
+{
+	struct vfio_device_info dev_info = {};
+	int ret;
+
+	dev_info.argsz = sizeof(dev_info);
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_INFO, &dev_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(dev_info.flags & VFIO_DEVICE_FLAGS_PCI);
+	ASSERT_TRUE(dev_info.flags & VFIO_DEVICE_FLAGS_RESET);
+	ASSERT_GE(dev_info.num_regions, VFIO_PCI_NUM_REGIONS);
+	ASSERT_GT(dev_info.num_irqs, 0);
+}
+
+TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 73/88] KVM: selftests: Verify VFIO region info for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest for VFIO region info queries on the bound device.

Verify that the operational register MMIO space (128 KiB), MSI-X table
and PBA space (16 KiB), and extended PCIe configuration space (4096 bytes)
report their exact sizes and expected access capability flags.

Testing multiple regions establishes baseline checks on the underlying
device descriptor so that when VFIO BARs are later delegated to or wrapped
by guest_memfd, native region querying on the device descriptor is
confirmed to remain unaffected.
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 38 ++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 669acc6058b5b..c76abe6d45140 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -150,4 +150,42 @@ TEST_F(guest_memfd_vfio, get_device_info)
 	ASSERT_GT(dev_info.num_irqs, 0);
 }
 
+TEST_F(guest_memfd_vfio, get_region_info)
+{
+	struct vfio_region_info reg_info = {};
+	int ret;
+
+	/* Test BAR0: 128 KiB MMIO register space */
+	reg_info.argsz = sizeof(reg_info);
+	reg_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_EQ(0x20000, reg_info.size);
+	ASSERT_EQ(VFIO_REGION_INFO_FLAG_READ |
+		  VFIO_REGION_INFO_FLAG_WRITE |
+		  VFIO_REGION_INFO_FLAG_MMAP,
+		  reg_info.flags & (VFIO_REGION_INFO_FLAG_READ |
+				    VFIO_REGION_INFO_FLAG_WRITE |
+				    VFIO_REGION_INFO_FLAG_MMAP));
+
+	/* Test BAR3: 16 KiB MSI-X table and PBA */
+	memset(&reg_info, 0, sizeof(reg_info));
+	reg_info.argsz = sizeof(reg_info);
+	reg_info.index = VFIO_PCI_BAR3_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_EQ(0x4000, reg_info.size);
+
+	/* Test PCIe Config Space: 4096 bytes extended config */
+	memset(&reg_info, 0, sizeof(reg_info));
+	reg_info.argsz = sizeof(reg_info);
+	reg_info.index = VFIO_PCI_CONFIG_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_EQ(PCI_CFG_SPACE_EXP_SIZE, reg_info.size);
+	ASSERT_EQ(VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE,
+		  reg_info.flags & (VFIO_REGION_INFO_FLAG_READ |
+				    VFIO_REGION_INFO_FLAG_WRITE));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 74/88] KVM: selftests: Verify VFIO IRQ info for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify interrupt capability queries on the bound
Intel 82576 device.

Check that MSI-X reports a positive vector count with eventfd signaling
support, that MSI reports eventfd capability, and that INTx reports
eventfd support along with maskable and auto-masked flags when present.

Testing multiple interrupt types establishes baseline checks on the
underlying device descriptor so that when VFIO resources are later
delegated to or wrapped by guest_memfd, native capability querying on
the device descriptor remains unaffected.
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 48 ++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index c76abe6d45140..6525f392b3930 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -188,4 +188,52 @@ TEST_F(guest_memfd_vfio, get_region_info)
 				    VFIO_REGION_INFO_FLAG_WRITE));
 }
 
+TEST_F(guest_memfd_vfio, get_irq_info)
+{
+	struct vfio_irq_info irq_info = {};
+	int ret;
+
+	/*
+	 * Test MSI-X interrupts (VFIO_PCI_MSIX_IRQ_INDEX):
+	 * Intel 82576 supports up to 10 MSI-X vectors.
+	 */
+	irq_info.argsz = sizeof(irq_info);
+	irq_info.index = VFIO_PCI_MSIX_IRQ_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_GT(irq_info.count, 0);
+	ASSERT_TRUE(irq_info.flags & VFIO_IRQ_INFO_EVENTFD);
+
+	/*
+	 * Test MSI interrupts (VFIO_PCI_MSI_IRQ_INDEX):
+	 * Should report eventfd capability.
+	 */
+	memset(&irq_info, 0, sizeof(irq_info));
+	irq_info.argsz = sizeof(irq_info);
+	irq_info.index = VFIO_PCI_MSI_IRQ_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_GT(irq_info.count, 0);
+	ASSERT_TRUE(irq_info.flags & VFIO_IRQ_INFO_EVENTFD);
+
+	/*
+	 * Test legacy INTx interrupts (VFIO_PCI_INTX_IRQ_INDEX):
+	 * Must support eventfd and masking when present.
+	 */
+	memset(&irq_info, 0, sizeof(irq_info));
+	irq_info.argsz = sizeof(irq_info);
+	irq_info.index = VFIO_PCI_INTX_IRQ_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_LE(irq_info.count, 1);
+	if (irq_info.count == 1) {
+		ASSERT_EQ(VFIO_IRQ_INFO_EVENTFD |
+			  VFIO_IRQ_INFO_MASKABLE |
+			  VFIO_IRQ_INFO_AUTOMASKED,
+			  irq_info.flags & (VFIO_IRQ_INFO_EVENTFD |
+					    VFIO_IRQ_INFO_MASKABLE |
+					    VFIO_IRQ_INFO_AUTOMASKED));
+	}
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 75/88] KVM: selftests: Verify VFIO MMIO read and write for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest verifying direct pread() and pwrite() operations on the
device descriptor targeting BAR0 MMIO space.

Enable memory decoding in the PCI Command register, verify that reading
the device status register returns an accessible value, and verify that
writing and reading back a scratch MAC register (RAL0) succeeds with
expected data.

This validates that kernel-mediated MMIO operations over the VFIO device
descriptor function properly, providing a baseline to compare against
when guest_memfd wraps the BAR.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 58 ++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 6525f392b3930..fa05b7cfb3185 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -236,4 +236,62 @@ TEST_F(guest_memfd_vfio, get_irq_info)
 	}
 }
 
+static void vfio_pci_enable_memory_and_bus_master(struct __test_metadata *_metadata,
+						  int vfio_fd)
+{
+	struct vfio_region_info cfg_info = {};
+	ssize_t ret_bytes;
+	uint16_t cmd;
+	int ret;
+
+	cfg_info.argsz = sizeof(cfg_info);
+	cfg_info.index = VFIO_PCI_CONFIG_REGION_INDEX;
+	ret = ioctl(vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &cfg_info);
+	ASSERT_EQ(0, ret);
+
+	/* Enable Memory Space and Bus Master in PCI Command register */
+	ret_bytes = pread(vfio_fd, &cmd, sizeof(cmd), cfg_info.offset + PCI_COMMAND);
+	ASSERT_EQ(sizeof(cmd), ret_bytes);
+	cmd |= (PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER);
+	ret_bytes = pwrite(vfio_fd, &cmd, sizeof(cmd), cfg_info.offset + PCI_COMMAND);
+	ASSERT_EQ(sizeof(cmd), ret_bytes);
+}
+
+TEST_F(guest_memfd_vfio, bar_rw)
+{
+	struct vfio_region_info bar0_info = {};
+	ssize_t ret_bytes;
+	uint32_t val;
+	int ret;
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * Read Device Status (E1000_STATUS) via pread() at BAR0 offset.
+	 * Confirm device is readable and does not return 0xFFFFFFFF.
+	 */
+	val = 0;
+	ret_bytes = pread(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_NE(0xFFFFFFFF, val);
+
+	/*
+	 * Write a distinct scratch test pattern to RAL0 via pwrite() and read
+	 * it back via pread().
+	 */
+	val = 0xdeadbeef;
+	ret_bytes = pwrite(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+
+	val = 0;
+	ret_bytes = pread(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0xdeadbeef, val);
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 76/88] KVM: selftests: Verify VFIO MMIO mmap access for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest verifying memory-mapped I/O (mmap) access to BAR0.

Map BAR0 into userspace address space, read the device status register
using a pointer dereference to confirm device responsiveness, and write
and read back a scratch test pattern on RAL0.

This validates that direct CPU page table mappings to device MMIO via
VFIO function properly, establishing an access baseline before wrapping
the BAR inside guest_memfd.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 37 ++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index fa05b7cfb3185..4d5211b6acf79 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -294,4 +294,41 @@ TEST_F(guest_memfd_vfio, bar_rw)
 	ASSERT_EQ(0xdeadbeef, val);
 }
 
+TEST_F(guest_memfd_vfio, bar_mmap)
+{
+	struct vfio_region_info bar0_info = {};
+	uint32_t val;
+	void *bar0;
+	int ret;
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	bar0 = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		    self->vfio_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0);
+
+	/*
+	 * Read Device Status (E1000_STATUS) via mapped pointer. Confirm device
+	 * is readable and does not return 0xFFFFFFFF.
+	 */
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_STATUS));
+	ASSERT_NE(0xFFFFFFFF, val);
+
+	/*
+	 * Write a distinct scratch test pattern to RAL0 via mapped pointer and
+	 * read it back.
+	 */
+	WRITE_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0), 0xdeadbeef);
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0));
+	ASSERT_EQ(0xdeadbeef, val);
+
+	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 77/88] KVM: selftests: Verify VFIO device reset for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest for VFIO_DEVICE_RESET on the bound Intel 82576 device.

Configure PCI command space to enable memory decode, map BAR0, and write
a distinct test pattern to the scratch receive address low (RAL0) register.

Trigger a device reset via the VFIO device ioctl, handling potential lock
contention with retry logic. Re-enable memory decode post-reset and assert
that the scratch register pattern reverted to its hardware reset value.

This verifies hardware reset lifecycle behavior, establishing a baseline
to later confirm that resetting a device revokes access when BAR0 is
wrapped by guest_memfd.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 45 ++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 4d5211b6acf79..91cba359e4392 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -331,4 +331,49 @@ TEST_F(guest_memfd_vfio, bar_mmap)
 	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
 }
 
+TEST_F(guest_memfd_vfio, device_reset)
+{
+	struct vfio_region_info bar0_info = {};
+	uint32_t val;
+	void *bar0;
+	int retries = 20;
+	int ret;
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+
+	bar0 = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		    self->vfio_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0);
+
+	/* Write scratch test pattern to RAL0 */
+	WRITE_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0), 0xdeadbeef);
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0));
+	ASSERT_EQ(0xdeadbeef, val);
+
+	/* Trigger VFIO_DEVICE_RESET, handling possible lock contention (-EAGAIN) */
+	do {
+		ret = ioctl(self->vfio_fd, VFIO_DEVICE_RESET, NULL);
+		if (ret != 0 && errno == EAGAIN)
+			usleep(10000);
+	} while (ret != 0 && errno == EAGAIN && retries-- > 0);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * After device reset, PCI Command register memory decode is cleared by
+	 * hardware. Re-enable it to access BAR0 registers again.
+	 */
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	/* Verify RAL0 was reset and reverted from scratch pattern */
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0));
+	ASSERT_NE(0xdeadbeef, val);
+
+	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 78/88] KVM: selftests: Verify VFIO access directly and via guest_memfd

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Extend the VFIO igb device selftest suite to run across fixture variants
testing both direct VFIO access and mediated access via guest_memfd.

Define two fixture variants:
- direct: binds the device normally, exercising direct descriptor read,
  write, mmap, and reset.
- guest_memfd: binds with the control-only flag and attaches the device
  as a backing resource for a guest_memfd instance, validating that BAR
  read, write, mmap, and reset succeed through the guest_memfd
  descriptor.

Because guest_memfd delegation of VFIO regions relies on in-place
conversion (per-gmem memory attributes) so the entire sparse region
namespace can be initialized as shared via INIT_SHARED, require
kvm_has_gmem_attributes when setting up the guest_memfd variant.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 77 ++++++++++++++++++++--
 1 file changed, 72 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 91cba359e4392..03c324f7f618d 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -21,6 +21,7 @@
 #include <linux/vfio.h>
 
 #include "kselftest_harness.h"
+#include "kvm_util.h"
 
 #define PCI_VENDOR_ID_INTEL 0x8086
 #define PCI_DEVICE_ID_INTEL_82576 0x10C9
@@ -96,10 +97,27 @@ static char *find_igb_cdev_path(void)
 	return NULL;
 }
 
+#define VFIO_PCI_OFFSET_SHIFT 40
+
 FIXTURE(guest_memfd_vfio) {
 	char *cdev_path;
+	struct kvm_vm *vm;
+	int gmem_fd;
 	int iommufd;
 	int vfio_fd;
+	int bar_fd;
+};
+
+FIXTURE_VARIANT(guest_memfd_vfio) {
+	bool use_gmem;
+};
+
+FIXTURE_VARIANT_ADD(guest_memfd_vfio, direct) {
+	.use_gmem = false,
+};
+
+FIXTURE_VARIANT_ADD(guest_memfd_vfio, guest_memfd) {
+	.use_gmem = true,
 };
 
 FIXTURE_SETUP(guest_memfd_vfio)
@@ -107,6 +125,10 @@ FIXTURE_SETUP(guest_memfd_vfio)
 	struct vfio_device_bind_iommufd bind_args = {};
 	int ret;
 
+	self->vm = NULL;
+	self->gmem_fd = -1;
+	self->bar_fd = -1;
+
 	self->cdev_path = find_igb_cdev_path();
 	if (!self->cdev_path)
 		SKIP(return, "Intel 82576 device bound to vfio not found\n");
@@ -123,12 +145,50 @@ FIXTURE_SETUP(guest_memfd_vfio)
 
 	bind_args.argsz = sizeof(bind_args);
 	bind_args.iommufd = self->iommufd;
+	if (variant->use_gmem)
+		bind_args.flags = VFIO_DEVICE_BIND_FLAG_CTRL_ONLY;
+
 	ret = ioctl(self->vfio_fd, VFIO_DEVICE_BIND_IOMMUFD, &bind_args);
 	ASSERT_EQ(0, ret);
+
+	/*
+	 * Create a SW_PROTECTED_VM with 1 runnable vCPU allocated for memslot0
+	 * and stack/page table allocation, so guest_mapping can execute vCPUs
+	 * and FIXTURE_TEARDOWN uniformly cleans up self->vm across both variants.
+	 */
+	self->vm = __vm_create((struct vm_shape){ .mode = VM_MODE_DEFAULT,
+						  .type = KVM_X86_SW_PROTECTED_VM },
+			       1, 0);
+
+	if (variant->use_gmem) {
+		uint64_t gmem_flags = GUEST_MEMFD_FLAG_USE_RESOURCE |
+				      GUEST_MEMFD_FLAG_PFN_MODE |
+				      GUEST_MEMFD_FLAG_MMAP |
+				      GUEST_MEMFD_FLAG_INIT_SHARED;
+		uint64_t gmem_size = (uint64_t)VFIO_PCI_NUM_REGIONS << VFIO_PCI_OFFSET_SHIFT;
+
+		/*
+		 * guest_memfd providers (including VFIO) is only intended to be
+		 * used with gmem_in_place_conversion.
+		 */
+		TEST_REQUIRE(kvm_has_gmem_attributes);
+
+		self->gmem_fd = vm_create_guest_memfd_resource(self->vm, gmem_size,
+							       gmem_flags,
+							       self->vfio_fd);
+		ASSERT_GE(self->gmem_fd, 0);
+		self->bar_fd = self->gmem_fd;
+	} else {
+		self->bar_fd = self->vfio_fd;
+	}
 }
 
 FIXTURE_TEARDOWN(guest_memfd_vfio)
 {
+	if (self->gmem_fd >= 0)
+		close(self->gmem_fd);
+	if (self->vm)
+		kvm_vm_free(self->vm);
 	if (self->vfio_fd >= 0)
 		close(self->vfio_fd);
 	if (self->iommufd >= 0)
@@ -236,6 +296,13 @@ TEST_F(guest_memfd_vfio, get_irq_info)
 	}
 }
 
+/*
+ * PCI Command register configuration belongs to the PCI configuration space
+ * region (VFIO_PCI_CONFIG_REGION_INDEX), which is accessed via the underlying
+ * VFIO device descriptor (vfio_fd). In guest_memfd mode, bar_fd refers to the
+ * guest_memfd descriptor representing memory BARs and cannot be used to read or
+ * write PCI configuration space.
+ */
 static void vfio_pci_enable_memory_and_bus_master(struct __test_metadata *_metadata,
 						  int vfio_fd)
 {
@@ -276,7 +343,7 @@ TEST_F(guest_memfd_vfio, bar_rw)
 	 * Confirm device is readable and does not return 0xFFFFFFFF.
 	 */
 	val = 0;
-	ret_bytes = pread(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
+	ret_bytes = pread(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
 	ASSERT_EQ(sizeof(val), ret_bytes);
 	ASSERT_NE(0xFFFFFFFF, val);
 
@@ -285,11 +352,11 @@ TEST_F(guest_memfd_vfio, bar_rw)
 	 * it back via pread().
 	 */
 	val = 0xdeadbeef;
-	ret_bytes = pwrite(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
+	ret_bytes = pwrite(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
 	ASSERT_EQ(sizeof(val), ret_bytes);
 
 	val = 0;
-	ret_bytes = pread(self->vfio_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
+	ret_bytes = pread(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
 	ASSERT_EQ(sizeof(val), ret_bytes);
 	ASSERT_EQ(0xdeadbeef, val);
 }
@@ -310,7 +377,7 @@ TEST_F(guest_memfd_vfio, bar_mmap)
 	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
 
 	bar0 = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE, MAP_SHARED,
-		    self->vfio_fd, bar0_info.offset);
+		    self->bar_fd, bar0_info.offset);
 	ASSERT_NE(MAP_FAILED, bar0);
 
 	/*
@@ -347,7 +414,7 @@ TEST_F(guest_memfd_vfio, device_reset)
 	ASSERT_EQ(0, ret);
 
 	bar0 = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE, MAP_SHARED,
-		    self->vfio_fd, bar0_info.offset);
+		    self->bar_fd, bar0_info.offset);
 	ASSERT_NE(MAP_FAILED, bar0);
 
 	/* Write scratch test pattern to RAL0 */

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 79/88] KVM: selftests: Verify BAR revocation and access gating in low power state

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

When a device transitions to low power state (D3hot), physical BAR
access is invalidated. The kernel revokes existing memory mappings and
marks the BARs unavailable.

Add a subtest to verify BAR revocation upon entering low power state.
Populate BAR0 mappings, write a test pattern, and enter low power mode.
Confirm that host mmap access to the revoked region raises SIGBUS and
that pread and pwrite on the BAR descriptor fail with -EIO while in low
power mode.

Exit low power state back to D0 and confirm that host mmap access and
pread/pwrite operations are restored and succeed again.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 86 ++++++++++++++++++++++
 1 file changed, 86 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 03c324f7f618d..cf2d46a365504 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -443,4 +443,90 @@ TEST_F(guest_memfd_vfio, device_reset)
 	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
 }
 
+TEST_F(guest_memfd_vfio, low_power_bar_revocation)
+{
+	struct vfio_device_feature feat = {};
+	struct vfio_region_info bar0_info = {};
+	ssize_t ret_bytes;
+	uint32_t val;
+	void *bar0;
+	int ret;
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	bar0 = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		    self->bar_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0);
+
+	/* Probe support for VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY */
+	feat.argsz = sizeof(feat);
+	feat.flags = VFIO_DEVICE_FEATURE_PROBE |
+		     VFIO_DEVICE_FEATURE_SET |
+		     VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, &feat);
+	if (ret != 0) {
+		ASSERT_EQ(0, munmap(bar0, bar0_info.size));
+		SKIP(return, "LOW_POWER_ENTRY feature not supported on device");
+	}
+
+	/* Populate mapping and write test pattern to scratch register */
+	WRITE_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0), 0xdeadbeef);
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0));
+	ASSERT_EQ(0xdeadbeef, val);
+
+	/* Enter low-power D3hot state */
+	feat.argsz = sizeof(feat);
+	feat.flags = VFIO_DEVICE_FEATURE_SET |
+		     VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, &feat);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * In low-power state, BARs are marked unavailable and mappings are
+	 * revoked.
+	 *
+	 * - Host mmap accesses trigger SIGBUS because page tables were zapped
+	 *   and get_pfn fails with -EFAULT while in low power state.
+	 * - Host pread() and pwrite() on the BAR descriptor fail with -EIO.
+	 */
+	TEST_EXPECT_SIGBUS((void)READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0)));
+	TEST_EXPECT_SIGBUS(WRITE_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0), 0x12345678));
+
+	val = 0;
+	ret_bytes = pread(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(-1, ret_bytes);
+	ASSERT_EQ(EIO, errno);
+
+	val = 0x12345678;
+	ret_bytes = pwrite(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(-1, ret_bytes);
+	ASSERT_EQ(EIO, errno);
+
+	/* Exit low-power state back to D0 */
+	feat.argsz = sizeof(feat);
+	feat.flags = VFIO_DEVICE_FEATURE_SET |
+		     VFIO_DEVICE_FEATURE_LOW_POWER_EXIT;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, &feat);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * Access must succeed again after returning to active D0.
+	 */
+	val = READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0));
+	ASSERT_EQ(0xdeadbeef, val);
+
+	val = 0;
+	ret_bytes = pread(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_NE(0xFFFFFFFF, val);
+
+	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 80/88] KVM: selftests: Verify host VMA flags for VFIO BAR

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a test verifying host userspace memory mapping characteristics
over a VFIO device BAR.

Map BAR0 of the bound VFIO device into host userspace and parse
/proc/self/smaps to verify that the kernel configured the expected
hardware MMIO VMA flags (io, pf, de).

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 65 ++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index cf2d46a365504..79a0001ca1733 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -529,4 +529,69 @@ TEST_F(guest_memfd_vfio, low_power_bar_revocation)
 	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
 }
 
+
+static void verify_smaps_vm_flags(struct __test_metadata *_metadata,
+				  void *addr, const char *flag_substr)
+{
+	unsigned long vma_start, vma_end;
+	bool found_flag = false;
+	bool in_vma = false;
+	char *line = NULL;
+	size_t size = 0;
+	FILE *file;
+
+	file = fopen("/proc/self/smaps", "r");
+	ASSERT_NE(NULL, file);
+
+	while (getline(&line, &size, file) > 0) {
+		if (sscanf(line, "%lx-%lx", &vma_start, &vma_end) == 2) {
+			if ((unsigned long)addr >= vma_start &&
+			    (unsigned long)addr < vma_end)
+				in_vma = true;
+			else
+				in_vma = false;
+			continue;
+		}
+
+		if (in_vma && !strncmp(line, "VmFlags:", 8)) {
+			if (strstr(line, flag_substr))
+				found_flag = true;
+			break;
+		}
+	}
+
+	free(line);
+	fclose(file);
+
+	ASSERT_TRUE(found_flag);
+}
+
+TEST_F(guest_memfd_vfio, host_vma_flags)
+{
+	struct vfio_region_info bar0_info = {};
+	void *bar0_mmap;
+	int ret;
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	/* mmap the VFIO BAR directly into host userspace */
+	bar0_mmap = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE,
+			 MAP_SHARED, self->bar_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0_mmap);
+
+	/*
+	 * Verify host VMA configuration in /proc/self/smaps:
+	 * VFIO MMIO mappings must have io, pf, and de flags set.
+	 */
+	verify_smaps_vm_flags(_metadata, bar0_mmap, "io");
+	verify_smaps_vm_flags(_metadata, bar0_mmap, "pf");
+	verify_smaps_vm_flags(_metadata, bar0_mmap, "de");
+
+	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 81/88] KVM: selftests: Verify guest mapping for VFIO BAR

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a test verifying guest execution and stage-2 mapping over a VFIO
device BAR.

Enable PCI memory space decoding on the bound device to allow the fault
handler to insert the PFN into host userspace page tables. Configure the
mapped memory region into a KVM memslot and execute a guest vCPU accessing
the BAR through its page tables.

Verify that the guest executes successfully without memory faults, read
the device status register directly via the guest virtual address, and
verify that the leaf page table entry for the BAR GPA is present.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 57 ++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 79a0001ca1733..0339794fb6dd3 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -22,6 +22,7 @@
 
 #include "kselftest_harness.h"
 #include "kvm_util.h"
+#include "processor.h"
 
 #define PCI_VENDOR_ID_INTEL 0x8086
 #define PCI_DEVICE_ID_INTEL_82576 0x10C9
@@ -594,4 +595,60 @@ TEST_F(guest_memfd_vfio, host_vma_flags)
 	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
 }
 
+static void guest_memslot_cacheability_code(uint64_t bar_gpa)
+{
+	uint32_t *reg = (uint32_t *)(bar_gpa + E1000_STATUS);
+	uint32_t val;
+
+	val = READ_ONCE(*reg);
+	GUEST_ASSERT(val != 0xFFFFFFFF);
+
+	GUEST_DONE();
+}
+
+TEST_F(guest_memfd_vfio, guest_mapping)
+{
+	struct vfio_region_info bar0_info = {};
+	uint64_t bar_gpa = 0xc0000000;
+	struct kvm_vcpu *vcpu;
+	void *bar0_mmap;
+	uint64_t *pte;
+	int ret;
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	/* mmap the VFIO BAR directly into host userspace for memslot config */
+	bar0_mmap = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE,
+			 MAP_SHARED, self->bar_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0_mmap);
+
+	vcpu = vm_vcpu_add(self->vm, 0, guest_memslot_cacheability_code);
+	if (variant->use_gmem)
+		vm_set_user_memory_region2(self->vm, 1, KVM_MEM_GUEST_MEMFD,
+					   bar_gpa, bar0_info.size, bar0_mmap,
+					   self->gmem_fd, bar0_info.offset);
+	else
+		vm_set_user_memory_region(self->vm, 1, 0, bar_gpa,
+					  bar0_info.size, bar0_mmap);
+
+	virt_map(self->vm, bar_gpa, bar_gpa, bar0_info.size >> self->vm->page_shift);
+
+	/* Run guest to fault in the page via stage 2, then verify S2 PTE */
+	vcpu_args_set(vcpu, 1, bar_gpa);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_DONE, get_ucall(vcpu, NULL));
+
+	pte = vm_get_pte(self->vm, bar_gpa);
+	ASSERT_NE(NULL, pte);
+	ASSERT_TRUE(is_present_pte(&self->vm->mmu, pte));
+
+	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 82/88] KVM: selftests: Verify VFIO dmabuf creation for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify dmabuf export via VFIO_DEVICE_FEATURE_DMA_BUF.

Probe device feature support using VFIO_DEVICE_FEATURE_PROBE, skipping
the test cleanly when kernel dmabuf support is absent
(CONFIG_VFIO_PCI_DMABUF or CONFIG_PCI_P2PDMA disabled).

Allocate and configure a vfio_device_feature request covering BAR0,
requesting a dma_buf file descriptor export via
VFIO_DEVICE_FEATURE_GET.

Verify that in direct mode a valid file descriptor is returned, confirm
that attempting to mmap the resulting dmabuf fd fails with -EINVAL (as
VFIO PCI dmabufs provide peer-to-peer kernel DMA rather than userspace
CPU mappings), and verify clean closure and resource release.
In guest_memfd mode, verify that dmabuf creation is rejected with
-EOPNOTSUPP.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 74 ++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 0339794fb6dd3..46d325228bda3 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -651,4 +651,78 @@ TEST_F(guest_memfd_vfio, guest_mapping)
 	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
 }
 
+TEST_F(guest_memfd_vfio, dmabuf_creation)
+{
+	struct vfio_device_feature_dma_buf *dmabuf_feature;
+	struct vfio_device_feature probe_feat = {};
+	struct vfio_region_info bar0_info = {};
+	struct vfio_device_feature *feat;
+	size_t feat_size;
+	int dmabuf_fd;
+	int ret;
+
+	/*
+	 * Probe support for VFIO_DEVICE_FEATURE_DMA_BUF. This requires kernel
+	 * support for CONFIG_VFIO_PCI_DMABUF (CONFIG_PCI_P2PDMA). In guest_memfd
+	 * mode, VFIO_DEVICE_BIND_FLAG_CTRL_ONLY is active, so the probe itself
+	 * is rejected with -EOPNOTSUPP by the control-only allowlist.
+	 */
+	probe_feat.argsz = sizeof(probe_feat);
+	probe_feat.flags = VFIO_DEVICE_FEATURE_PROBE |
+			   VFIO_DEVICE_FEATURE_GET |
+			   VFIO_DEVICE_FEATURE_DMA_BUF;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, &probe_feat);
+	if (variant->use_gmem) {
+		ASSERT_LT(ret, 0);
+		ASSERT_EQ(EOPNOTSUPP, errno);
+	} else if (ret != 0) {
+		SKIP(return, "Kernel lacks CONFIG_VFIO_PCI_DMABUF support\n");
+	}
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * Allocate vfio_device_feature with trailing vfio_device_feature_dma_buf
+	 * containing one dma_range covering BAR0.
+	 */
+	feat_size = sizeof(*feat) + sizeof(*dmabuf_feature) +
+		    sizeof(struct vfio_region_dma_range);
+	feat = calloc(1, feat_size);
+	ASSERT_NE(NULL, feat);
+
+	feat->argsz = feat_size;
+	feat->flags = VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_DMA_BUF;
+
+	dmabuf_feature = (struct vfio_device_feature_dma_buf *)feat->data;
+	dmabuf_feature->region_index = VFIO_PCI_BAR0_REGION_INDEX;
+	dmabuf_feature->open_flags = O_RDWR | O_CLOEXEC;
+	dmabuf_feature->nr_ranges = 1;
+	dmabuf_feature->dma_ranges[0].offset = 0;
+	dmabuf_feature->dma_ranges[0].length = bar0_info.size;
+
+	dmabuf_fd = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, feat);
+	if (variant->use_gmem) {
+		ASSERT_LT(dmabuf_fd, 0);
+		ASSERT_EQ(EOPNOTSUPP, errno);
+	} else {
+		ASSERT_GE(dmabuf_fd, 0);
+
+		/*
+		 * VFIO PCI dmabufs are intended for peer-to-peer kernel DMA and
+		 * do not implement mmap or read operations. Verify mmap returns
+		 * -EINVAL.
+		 */
+		ASSERT_EQ(MAP_FAILED, mmap(NULL, bar0_info.size,
+					   PROT_READ | PROT_WRITE,
+					   MAP_SHARED, dmabuf_fd, 0));
+		ASSERT_EQ(EINVAL, errno);
+		close(dmabuf_fd);
+	}
+
+	free(feat);
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 83/88] KVM: selftests: Verify VFIO ioeventfd creation for igb device

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify ioeventfd registration and unregistration on
the bound Intel 82576 device.

Allocate an eventfd and configure a VFIO_DEVICE_IOEVENTFD ioctl targeting
the scratch receive address low (RAL0) register in BAR0 with 4-byte width
and a test trigger value.

Assert that in direct mode registration succeeds and that unregistration
(passing fd = -1 with matching register criteria) succeeds cleanly.
In guest_memfd mode, verify that ioeventfd registration is rejected with
-EOPNOTSUPP.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 39 ++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 46d325228bda3..dde249571ebc4 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -10,6 +10,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
+#include <sys/eventfd.h>
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 #include <sys/types.h>
@@ -725,4 +726,42 @@ TEST_F(guest_memfd_vfio, dmabuf_creation)
 	free(feat);
 }
 
+TEST_F(guest_memfd_vfio, ioeventfd_creation)
+{
+	struct vfio_device_ioeventfd ioeventfd = {};
+	struct vfio_region_info bar0_info = {};
+	int ret;
+	int efd;
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+
+	efd = eventfd(0, EFD_CLOEXEC);
+	ASSERT_GE(efd, 0);
+
+	/* Register ioeventfd on BAR0 targeting RAL0 with 4-byte width */
+	ioeventfd.argsz = sizeof(ioeventfd);
+	ioeventfd.flags = VFIO_DEVICE_IOEVENTFD_32;
+	ioeventfd.offset = bar0_info.offset + E1000_RAL0;
+	ioeventfd.data = 0xdeadbeef;
+	ioeventfd.fd = efd;
+
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_IOEVENTFD, &ioeventfd);
+	if (variant->use_gmem) {
+		ASSERT_LT(ret, 0);
+		ASSERT_EQ(EOPNOTSUPP, errno);
+	} else {
+		ASSERT_EQ(0, ret);
+
+		/* Unregister ioeventfd by passing fd = -1 with identical match criteria */
+		ioeventfd.fd = -1;
+		ret = ioctl(self->vfio_fd, VFIO_DEVICE_IOEVENTFD, &ioeventfd);
+		ASSERT_EQ(0, ret);
+	}
+
+	close(efd);
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 84/88] KVM: selftests: Verify memory conversions for VFIO BARs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify memory attribute conversions on guest_memfd
files backed by a VFIO PCI device.

Verify that memory attribute conversion to private succeeds on
operational register BAR space that does not host an interrupt table.
Verify that converting back to shared also succeeds cleanly.

Verify that converting a BAR that hosts the MSI-X vector table and PBA
to private is rejected with -EINVAL, and that the returned error offset
identifies the start of the invalid range. Host kernel drivers program
MSI-X table entries directly to mask and unmask interrupts, so
converting these pages to private would trigger fatal hardware
protection faults on confidential computing platforms. Verify that
shared conversions on this BAR remain permitted.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 50 ++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index dde249571ebc4..18fab1a39041e 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -764,4 +764,54 @@ TEST_F(guest_memfd_vfio, ioeventfd_creation)
 	close(efd);
 }
 
+TEST_F(guest_memfd_vfio, conversions)
+{
+	struct vfio_region_info bar0_info = {};
+	struct vfio_region_info bar3_info = {};
+	uint64_t err_offset = 0;
+	int ret;
+
+	if (!variant->use_gmem)
+		SKIP(return, "Attribute conversions only apply to guest_memfd");
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+
+	bar3_info.argsz = sizeof(bar3_info);
+	bar3_info.index = VFIO_PCI_BAR3_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar3_info);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * BAR0 is an operational MMIO register space with no MSI-X table.
+	 * Converting its pages to private should succeed.
+	 */
+	ret = __gmem_set_private(self->gmem_fd, bar0_info.offset,
+				 bar0_info.size, &err_offset);
+	ASSERT_EQ(0, ret);
+
+	/* Converting back to shared should succeed. */
+	ret = __gmem_set_shared(self->gmem_fd, bar0_info.offset,
+				bar0_info.size, &err_offset);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * BAR3 contains the MSI-X vector table and PBA. Private conversion
+	 * must be rejected by check_set_attributes to prevent host kernel
+	 * writes to interrupt tables from triggering fatal protection faults.
+	 */
+	ret = __gmem_set_private(self->gmem_fd, bar3_info.offset,
+				 bar3_info.size, &err_offset);
+	ASSERT_LT(ret, 0);
+	ASSERT_EQ(EINVAL, errno);
+	ASSERT_EQ(bar3_info.offset, err_offset);
+
+	/* Shared conversion on BAR3 should be permitted. */
+	ret = __gmem_set_shared(self->gmem_fd, bar3_info.offset,
+				bar3_info.size, &err_offset);
+	ASSERT_EQ(0, ret);
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 85/88] KVM: selftests: Verify shared and private mmap access for VFIO BARs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify read and write access to VFIO device BAR memory
mmap-ed via guest_memfd across shared and private states from both host
and guest perspectives.

Map an operational device BAR (BAR0) through guest_memfd and configure
it into a KVM memslot for a software-protected VM.

In shared state, verify that host userspace reads the device status
register and performs read-write verification on the scratch receive
address low register (RAL0). Add and initialize the guest vCPU with
a guest loop synchronized via GUEST_SYNC to perform read and write
operations over the BAR GPA and confirm that changes made by the guest
are visible to the host.

Convert the BAR memory to private. Verify that host userspace reads and
writes to the mmap-ed region are blocked and trigger SIGBUS, as
unmediated host access to private memory is disallowed. Verify that the
guest vCPU continues to have full read and write access to the private
BAR registers without exiting.

Convert the BAR memory back to shared. Verify that host userspace
access is restored, that host reads observe updates made by the guest
while private, and that guest operations continue normally.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 130 +++++++++++++++++++++
 1 file changed, 130 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 18fab1a39041e..a520cf45fccb8 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -814,4 +814,134 @@ TEST_F(guest_memfd_vfio, conversions)
 	ASSERT_EQ(0, ret);
 }
 
+static uint32_t guest_write_val;
+
+static void guest_rw_test_code(uint64_t bar_gpa)
+{
+	uint32_t *ral0 = (uint32_t *)(bar_gpa + E1000_RAL0);
+	uint32_t *status = (uint32_t *)(bar_gpa + E1000_STATUS);
+	uint32_t val;
+
+	for (;;) {
+		uint32_t write_val = READ_ONCE(guest_write_val);
+
+		/* Read read-only device STATUS register */
+		val = READ_ONCE(*status);
+		GUEST_ASSERT(val != 0xFFFFFFFF);
+
+		/* Write and read-back read-write RAL0 register */
+		WRITE_ONCE(*ral0, write_val);
+		val = READ_ONCE(*ral0);
+		GUEST_ASSERT_EQ(val, write_val);
+
+		GUEST_SYNC(0);
+	}
+}
+
+TEST_F(guest_memfd_vfio, mmap_shared_private_rw)
+{
+	struct vfio_region_info bar0_info = {};
+	uint64_t bar_gpa = 0xc0000000;
+	struct kvm_vcpu *vcpu;
+	void *bar0_mmap;
+	uint32_t *h_ral0;
+	uint32_t *h_status;
+	uint32_t val;
+	int ret;
+
+	if (!variant->use_gmem)
+		SKIP(return, "Shared and private attribute rw test only applies to guest_memfd");
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	/* mmap the VFIO BAR through guest_memfd descriptor */
+	bar0_mmap = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE,
+			 MAP_SHARED, self->bar_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0_mmap);
+
+	h_status = (uint32_t *)((uintptr_t)bar0_mmap + E1000_STATUS);
+	h_ral0 = (uint32_t *)((uintptr_t)bar0_mmap + E1000_RAL0);
+
+	/* Configure KVM guest memslot backed by guest_memfd */
+	vcpu = vm_vcpu_add(self->vm, 0, guest_rw_test_code);
+	vm_set_user_memory_region2(self->vm, 1, KVM_MEM_GUEST_MEMFD,
+				   bar_gpa, bar0_info.size, bar0_mmap,
+				   self->gmem_fd, bar0_info.offset);
+	virt_map(self->vm, bar_gpa, bar_gpa, bar0_info.size >> self->vm->page_shift);
+
+	/*
+	 * Phase 1: Shared memory access from Host and Guest.
+	 * Initially, the entire guest_memfd resource is shared.
+	 */
+	/* Host read from STATUS and write/read-back to RAL0 */
+	val = READ_ONCE(*h_status);
+	ASSERT_NE(0xFFFFFFFF, val);
+	WRITE_ONCE(*h_ral0, 0x11223344);
+	val = READ_ONCE(*h_ral0);
+	ASSERT_EQ(0x11223344, val);
+
+	/* Guest read/write to shared BAR memory */
+	vcpu_args_set(vcpu, 1, bar_gpa);
+	guest_write_val = 0xaabbccdd;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	/* Verify host sees the value written by guest in shared mode */
+	val = READ_ONCE(*h_ral0);
+	ASSERT_EQ(0xaabbccdd, val);
+
+	/*
+	 * Phase 2: Convert BAR0 memory to Private.
+	 */
+	gmem_set_private(self->gmem_fd, bar0_info.offset, bar0_info.size);
+
+	/*
+	 * Host accesses to private guest_memfd memory must fail with SIGBUS.
+	 */
+	TEST_EXPECT_SIGBUS((void)READ_ONCE(*h_status));
+	TEST_EXPECT_SIGBUS(WRITE_ONCE(*h_ral0, 0x12345678));
+
+	/*
+	 * Guest read and write access to private BAR memory must succeed.
+	 */
+	guest_write_val = 0x55667788;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	/*
+	 * Phase 3: Convert BAR0 memory back to Shared.
+	 */
+	gmem_set_shared(self->gmem_fd, bar0_info.offset, bar0_info.size);
+
+	/* Host read/write access must now succeed again */
+	val = READ_ONCE(*h_status);
+	ASSERT_NE(0xFFFFFFFF, val);
+	/* Host sees the value written by the guest while private */
+	val = READ_ONCE(*h_ral0);
+	ASSERT_EQ(0x55667788, val);
+
+	WRITE_ONCE(*h_ral0, 0x99aabbcc);
+	val = READ_ONCE(*h_ral0);
+	ASSERT_EQ(0x99aabbcc, val);
+
+	/* Guest read/write access continues to succeed in shared mode */
+	guest_write_val = 0xdeedbeef;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	val = READ_ONCE(*h_ral0);
+	ASSERT_EQ(0xdeedbeef, val);
+
+	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 86/88] KVM: selftests: Expect successful conversion on MSI-X BARs in VFIO test

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Disallowing private mappings of MSI-X vector tables is now enforced
lazily by rejecting private fault resolution when backends report
shared-only physical frames, rather than eagerly rejecting software
attribute conversions.

Update the conversions subtest to expect successful attribute conversion
on BAR3 containing the MSI-X table, matching the lazy rejection model.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index a520cf45fccb8..325d5e0c66026 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -798,17 +798,19 @@ TEST_F(guest_memfd_vfio, conversions)
 	ASSERT_EQ(0, ret);
 
 	/*
-	 * BAR3 contains the MSI-X vector table and PBA. Private conversion
-	 * must be rejected by check_set_attributes to prevent host kernel
-	 * writes to interrupt tables from triggering fatal protection faults.
+	 * BAR3 contains the MSI-X vector table and PBA. Disallowing private
+	 * mappings of the MSI-X table is enforced lazily via NO_PRIVATE PFN
+	 * attributes rather than eagerly rejecting software attribute conversion.
+	 * Converting BAR3 attributes to private in guest_memfd succeeds, but
+	 * attempting to fault the MSI-X page into the guest as private is
+	 * rejected with -EFAULT in kvm_gmem_get_pfn(), causing KVM to exit with
+	 * KVM_EXIT_MEMORY_FAULT.
 	 */
 	ret = __gmem_set_private(self->gmem_fd, bar3_info.offset,
 				 bar3_info.size, &err_offset);
-	ASSERT_LT(ret, 0);
-	ASSERT_EQ(EINVAL, errno);
-	ASSERT_EQ(bar3_info.offset, err_offset);
+	ASSERT_EQ(0, ret);
 
-	/* Shared conversion on BAR3 should be permitted. */
+	/* Shared conversion on BAR3 should also succeed. */
 	ret = __gmem_set_shared(self->gmem_fd, bar3_info.offset,
 				bar3_info.size, &err_offset);
 	ASSERT_EQ(0, ret);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 87/88] QUESTION: KVM: guest_memfd: Disallow private mappings of shared-only PFNs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Host kernel drivers directly write to the PCI MSI-X vector table to
program and mask interrupts. On confidential computing platforms, allowing
hardware protection tables (such as RMP, Secure-EPT, or GPT) to mark MSI-X
pages private causes subsequent host kernel interrupt updates to trigger
fatal hardware protection faults.

I see 2 options here:

1. Prevent it at conversion time, and require INIT_SHARED
2. Let guest_memfd track all conversions to private as per normal, but
   block setting the MSI-X range as private at the platform level by
   blocking platform config for SNP, ARM CCA at fault time, and by blocking
   faults for TDX.

Implement (2) partially by introducing a PFN attribute bit indicating that
a physical frame cannot be mapped or populated as private, reporting that
attribute from VFIO PCI whenever a BAR page overlaps the MSI-X vector
table.

When a PFN carries this attribute, reject private stage-2 fault resolution,
private memory population, and SNP RMP fault / VMSA reload paths before any
platform private transition occurs, while still allowing shared mappings.

The missing half is that for ARM CCA, which (for now?) doesn't use the
.gmem_make_private() hook, will have to enforce in its GPT configuration
path to block to-private config for MSI-X ranges.

With (2), the attribute-conversion check callback is not longer required.

Pros of preventing platform updates via PFN attributes:
- Enforces the invariant at the single boundary where physical frames
  enter hardware confidential state, covering both initial private files
  and later conversions uniformly. (Though for Confidential IO, the
  guest_memfd would have started off as INIT_SHARED anyway, so we could
  require INIT_SHARED in the VFIO .attach() handler)
- Allows virtual machine monitors to initialize or convert an entire BAR
  at once without carving out the MSI-X range when a memslot hole already
  traps guest MSI-X accesses.

Cons of preventing platform updates via PFN attributes:
- Defers error reporting from ioctl conversion or creation time to fault
  or populate time if userspace maps a shared-only page into a memslot and
  accesses it as private, whereas rejecting conversions and requiring
  shared initialization fails eagerly and keeps software attributes
  aligned with hardware capability.
- Special architecture make-private paths (CCA only?, since SNP's
  make-private is handled from within kvm_gmem_get_pfn) need to block this

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 arch/x86/kvm/svm/sev.c           |  9 +++++++-
 drivers/vfio/pci/vfio_pci.c      |  1 -
 drivers/vfio/pci/vfio_pci_core.c | 49 ++++++++++++----------------------------
 drivers/vfio/vfio_main.c         | 16 -------------
 include/linux/guest_memfd.h      | 14 ++----------
 include/linux/vfio.h             |  3 ---
 include/linux/vfio_pci_core.h    |  3 ---
 virt/kvm/guest_memfd.c           | 26 +++++----------------
 8 files changed, 30 insertions(+), 91 deletions(-)

diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 526f2a457cfe0..4d19ce862c27d 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4100,7 +4100,8 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL, &pfn_attrs))
 		return;
 
-	if ((pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) ||
+	if ((pfn_attrs & (GUEST_MEMFD_PFN_ATTRS_READONLY |
+			  GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE)) ||
 	    !kvm_gmem_pfn_is_cacheable(pfn_attrs))
 		return;
 
@@ -5111,6 +5112,12 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
 		return;
 	}
 
+	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE) {
+		pr_warn_ratelimited("SEV: Unexpected RMP fault, shared-only backing page for GPA 0x%llx\n",
+				    gpa);
+		return;
+	}
+
 	if (!kvm_gmem_pfn_is_cacheable(pfn_attrs)) {
 		pr_warn_ratelimited("SEV: Unexpected RMP fault, non-cacheable backing page for GPA 0x%llx\n",
 				    gpa);
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index e5e2dbca152ec..6b137e009982c 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -173,7 +173,6 @@ static const struct vfio_device_ops vfio_pci_ops = {
 	.gmem_release		= vfio_pci_core_gmem_release,
 	.gmem_get_pfn		= vfio_pci_core_gmem_get_pfn,
 	.gmem_lookup_pfn_range	= vfio_pci_core_gmem_lookup_pfn_range,
-	.gmem_check_set_attributes = vfio_pci_core_gmem_check_set_attributes,
 	.gmem_read		= vfio_pci_core_gmem_read,
 	.gmem_write		= vfio_pci_core_gmem_write,
 #endif
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index c976574782663..ccaa24620112a 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -16,7 +16,6 @@
 #include <linux/eventfd.h>
 #include <linux/file.h>
 #include <linux/guest_memfd.h>
-#include <linux/kvm_host.h>
 #include <linux/interrupt.h>
 #include <linux/iommu.h>
 #include <linux/module.h>
@@ -2790,45 +2789,25 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 	if (max_order)
 		*max_order = 0;
 
-	if (pfn_attrs)
+	if (pfn_attrs) {
 		*pfn_attrs = GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC;
 
-	return 0;
-}
-EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_get_pfn);
-
-int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
-					    pgoff_t index, size_t nr_pages,
-					    u64 attributes)
-{
-	struct vfio_pci_core_device *vdev =
-		container_of(core_vdev, struct vfio_pci_core_device, vdev);
-	pgoff_t bar_pgoff;
-	size_t size;
-	loff_t pos;
-	int bar;
-
-	if (!(attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE))
-		return 0;
-
-	/*
-	 * Prevent private conversion of pages containing the MSI-X vector
-	 * table. The host kernel directly programs MSI-X table entries to
-	 * configure and mask interrupts; converting these pages to private
-	 * would cause host kernel writes to trigger fatal hardware protection
-	 * faults (e.g. RMP or GPT violations) on confidential compute platforms.
-	 */
-	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
-	bar_pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
-	pos = (loff_t)bar_pgoff << PAGE_SHIFT;
-	size = nr_pages << PAGE_SHIFT;
-
-	if (vfio_pci_range_overlaps_msix(vdev, bar, pos, size))
-		return -EINVAL;
+		/*
+		 * Disallow private mappings of pages overlapping the MSI-X
+		 * vector table. The host kernel directly programs MSI-X table
+		 * entries to configure and mask interrupts; marking these pages
+		 * private in hardware protection tables (e.g. RMP or GPT) would
+		 * cause host kernel writes to trigger fatal faults.
+		 */
+		if (vfio_pci_range_overlaps_msix(vdev, bar,
+						 (loff_t)pgoff << PAGE_SHIFT,
+						 PAGE_SIZE))
+			*pfn_attrs |= GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE;
+	}
 
 	return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_check_set_attributes);
+EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_get_pfn);
 
 int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
 					pgoff_t index, unsigned long *pfn,
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 96a768b5ec71b..770ae8ef7926f 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -1621,21 +1621,6 @@ static int vfio_device_gmem_mmap(void *provider, struct vm_area_struct *vma)
 	return vfio_device_mmap(df->device, vma);
 }
 
-static int vfio_device_gmem_check_set_attributes(void *provider, pgoff_t index,
-						 size_t nr_pages,
-						 u64 attributes)
-{
-	struct file *file = provider;
-	struct vfio_device_file *df = file->private_data;
-	struct vfio_device *device = df->device;
-
-	if (!device->ops->gmem_check_set_attributes)
-		return 0;
-
-	return device->ops->gmem_check_set_attributes(device, index, nr_pages,
-						      attributes);
-}
-
 static int vfio_device_gmem_lookup_pfn_range(void *provider, pgoff_t index,
 					     unsigned long *pfn,
 					     size_t *nr_pages)
@@ -1692,7 +1677,6 @@ static const struct guest_memfd_provider_operations vfio_device_gmem_provider_op
 	.read                 = vfio_device_gmem_read,
 	.write                = vfio_device_gmem_write,
 	.mmap                 = vfio_device_gmem_mmap,
-	.check_set_attributes = vfio_device_gmem_check_set_attributes,
 };
 
 int vfio_device_gmem_revoke(struct vfio_device *device, pgoff_t start,
diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
index c85594b58112a..f839ae936ebed 100644
--- a/include/linux/guest_memfd.h
+++ b/include/linux/guest_memfd.h
@@ -32,6 +32,8 @@ struct mempolicy;
 #define GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC	(2ULL << 1)
 #define GUEST_MEMFD_PFN_ATTRS_CACHE_WC		(3ULL << 1)
 
+#define GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE	BIT_ULL(3)
+
 static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
 {
 	return (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_CACHE_MASK) ==
@@ -88,16 +90,6 @@ static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
  * @mmap: Optional callback to set up user mappings. Called when userspace calls
  *        mmap() on guest_memfd. The provider can customize VMA flags, or reject
  *        the mapping.
- * @check_set_attributes: Optional callback to validate attribute changes on a
- *                        range of indices before they are committed. Called
- *                        with the exclusive filemap invalidate lock held. The
- *                        provider can inspect the requested attributes and
- *                        offset range and return 0 to allow the transition or
- *                        a negative error code (e.g. -EINVAL or -EPERM) to
- *                        reject it. Once @check_set_attributes succeeds, the
- *                        provider must ensure this result stays valid and
- *                        constant until guest_memfd stops using the range. Do
- *                        not take locks held across kvm_gmem_revoke().
  *
  * Used by filesystems and device drivers that provide memory for guest_memfd.
  */
@@ -117,8 +109,6 @@ struct guest_memfd_provider_operations {
 			loff_t *ppos);
 	ssize_t (*write)(void *provider, const void *buf, size_t count,
 			 loff_t *ppos);
-	int (*check_set_attributes)(void *provider, pgoff_t index,
-				    size_t nr_pages, u64 attributes);
 };
 
 #ifdef CONFIG_KVM_GUEST_MEMFD
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 012a233a375ab..c38bbc5568fee 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -125,9 +125,6 @@ struct vfio_device_ops {
 	int (*gmem_get_pfn)(struct vfio_device *device, pgoff_t index,
 			    unsigned long *pfn, int *max_order,
 			    u64 *pfn_attrs);
-	int (*gmem_check_set_attributes)(struct vfio_device *device,
-					 pgoff_t index, size_t nr_pages,
-					 u64 attributes);
 	int (*gmem_lookup_pfn_range)(struct vfio_device *device, pgoff_t index,
 				     unsigned long *pfn, size_t *nr_pages);
 	ssize_t (*gmem_read)(struct vfio_device *device, void *buf,
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 03cb0b41855dd..5bab69cb83c25 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -175,9 +175,6 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev);
 int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
 			       unsigned long *pfn, int *max_order,
 			       u64 *pfn_attrs);
-int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
-					    pgoff_t index, size_t nr_pages,
-					    u64 attributes);
 int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
 					pgoff_t index, unsigned long *pfn,
 					size_t *nr_pages);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 385b517b49ed5..cd484bdcb3f0c 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -245,16 +245,6 @@ static ssize_t gmem_provider_write(struct gmem_inode *gi,
 	return -EINVAL;
 }
 
-static int gmem_provider_check_set_attributes(struct gmem_inode *gi, pgoff_t index,
-					      size_t nr_pages, u64 attributes)
-{
-	if (gi->provider_ops && gi->provider_ops->check_set_attributes)
-		return gi->provider_ops->check_set_attributes(gi->provider, index,
-							      nr_pages, attributes);
-
-	return 0;
-}
-
 #define kvm_gmem_for_each_file(f, inode) \
 	list_for_each_entry(f, &GMEM_I(inode)->gmem_file_list, entry)
 
@@ -1128,12 +1118,6 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
 	if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs))
 		goto out;
 
-	r = gmem_provider_check_set_attributes(gi, start, nr_pages, attrs);
-	if (r) {
-		*err_index = start;
-		goto out;
-	}
-
 	mas_init(&mas, mt, start);
 	r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
 	if (r) {
@@ -1774,10 +1758,11 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 	/*
 	 * Private memory requires exclusive, writable guest ownership and
-	 * cannot be backed by a read-only frame.
+	 * cannot be backed by a read-only or shared-only frame.
 	 */
 	if (kvm_gmem_is_private_mem(file_inode(file), index) &&
-	    (*pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)) {
+	    (*pfn_attrs & (GUEST_MEMFD_PFN_ATTRS_READONLY |
+			   GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE))) {
 		r = -EFAULT;
 		goto out_put_folio;
 	}
@@ -1824,9 +1809,10 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 	/*
 	 * Populating pages initializes private memory, which cannot be
-	 * read-only or non-cacheable.
+	 * read-only, shared-only, or non-cacheable.
 	 */
-	if ((pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) ||
+	if ((pfn_attrs & (GUEST_MEMFD_PFN_ATTRS_READONLY |
+			  GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE)) ||
 	    !kvm_gmem_pfn_is_cacheable(pfn_attrs)) {
 		ret = -EPERM;
 		goto out_put_folio;

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC v3 88/88] KVM: selftests: Verify shared and private pread and pwrite for VFIO BARs

[Ackerley Tng via B4 Relay]

From: Ackerley Tng <ackerleytng@google.com>

Add a subtest to verify read and write operations on guest memory files
backed by a VFIO PCI device across shared and private states using
file descriptors directly via pread and pwrite.

Verify that in shared state, host userspace can read device status and
perform read and write verification on device registers through pread
and pwrite on the guest memory file descriptor, and that updates made by
a guest virtual CPU in shared mode are visible to host pread.

Convert the memory to private. Verify that host pread and pwrite
operations to private guest memory are rejected with -EFAULT, while the
guest virtual CPU continues to have full read and write access to the
private registers.

Convert the memory back to shared. Verify that host pread and pwrite
access is restored, that host reads observe updates made by the guest
while private, and that guest operations continue normally.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 .../selftests/kvm/x86/guest_memfd_vfio_test.c      | 143 +++++++++++++++++++++
 1 file changed, 143 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
index 325d5e0c66026..eb3a2ce0e6ed6 100644
--- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
+++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
@@ -946,4 +946,147 @@ TEST_F(guest_memfd_vfio, mmap_shared_private_rw)
 	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
 }
 
+TEST_F(guest_memfd_vfio, rw_shared_private)
+{
+	struct vfio_region_info bar0_info = {};
+	uint64_t bar_gpa = 0xc0000000;
+	struct kvm_vcpu *vcpu;
+	void *bar0_mmap;
+	ssize_t ret_bytes;
+	uint32_t val;
+	int ret;
+
+	if (!variant->use_gmem)
+		SKIP(return, "Shared and private attribute rw test only applies to guest_memfd");
+
+	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
+
+	bar0_info.argsz = sizeof(bar0_info);
+	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
+	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
+	ASSERT_EQ(0, ret);
+	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	/* mmap the VFIO BAR through guest_memfd descriptor for memslot setup */
+	bar0_mmap = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE,
+			 MAP_SHARED, self->bar_fd, bar0_info.offset);
+	ASSERT_NE(MAP_FAILED, bar0_mmap);
+
+	/* Configure KVM guest memslot backed by guest_memfd */
+	vcpu = vm_vcpu_add(self->vm, 0, guest_rw_test_code);
+	vm_set_user_memory_region2(self->vm, 1, KVM_MEM_GUEST_MEMFD, bar_gpa,
+				   bar0_info.size, bar0_mmap, self->gmem_fd,
+				   bar0_info.offset);
+	virt_map(self->vm, bar_gpa, bar_gpa,
+		 bar0_info.size >> self->vm->page_shift);
+
+	/*
+	 * Phase 1: Shared memory pread and pwrite access from Host and Guest.
+	 * Initially, the entire guest_memfd resource is shared.
+	 */
+	/* Host read from STATUS and write/read-back to RAL0 via pread/pwrite */
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_NE(0xFFFFFFFF, val);
+
+	val = 0x11223344;
+	ret_bytes = pwrite(self->gmem_fd, &val, sizeof(val),
+			   bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0x11223344, val);
+
+	/* Guest read/write to shared BAR memory */
+	vcpu_args_set(vcpu, 1, bar_gpa);
+	guest_write_val = 0xaabbccdd;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	/* Verify host pread sees the value written by guest in shared mode */
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0xaabbccdd, val);
+
+	/*
+	 * Phase 2: Convert BAR0 memory to Private.
+	 */
+	gmem_set_private(self->gmem_fd, bar0_info.offset, bar0_info.size);
+
+	/*
+	 * Host pread and pwrite to private guest_memfd memory must fail with -EFAULT.
+	 */
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(-1, ret_bytes);
+	ASSERT_EQ(EFAULT, errno);
+
+	val = 0x12345678;
+	ret_bytes = pwrite(self->gmem_fd, &val, sizeof(val),
+			   bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(-1, ret_bytes);
+	ASSERT_EQ(EFAULT, errno);
+
+	/*
+	 * Guest read and write access to private BAR memory must succeed.
+	 */
+	guest_write_val = 0x55667788;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	/*
+	 * Phase 3: Convert BAR0 memory back to Shared.
+	 */
+	gmem_set_shared(self->gmem_fd, bar0_info.offset, bar0_info.size);
+
+	/* Host pread and pwrite access must now succeed again */
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_STATUS);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_NE(0xFFFFFFFF, val);
+
+	/* Host sees the value written by the guest while private */
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0x55667788, val);
+
+	val = 0x99aabbcc;
+	ret_bytes = pwrite(self->gmem_fd, &val, sizeof(val),
+			   bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0x99aabbcc, val);
+
+	/* Guest read/write access continues to succeed in shared mode */
+	guest_write_val = 0xdeedbeef;
+	sync_global_to_guest(self->vm, guest_write_val);
+	vcpu_run(vcpu);
+	ASSERT_EQ(UCALL_SYNC, get_ucall(vcpu, NULL));
+
+	val = 0;
+	ret_bytes = pread(self->gmem_fd, &val, sizeof(val),
+			  bar0_info.offset + E1000_RAL0);
+	ASSERT_EQ(sizeof(val), ret_bytes);
+	ASSERT_EQ(0xdeedbeef, val);
+
+	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
+}
+
 TEST_HARNESS_MAIN

-- 
2.56.0.455.g6c5a503b38-goog

Re: [PATCH RFC v3 03/88] mm: shmem: Implement guest_memfd provider operations for tmpfs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] mm: shmem: shmem_gmem_alloc_folio() allocates unmovable folios
  as movable
- [High] mm: shmem: sbinfo->mpol refcount leak in
  shmem_gmem_alloc_folio()
- [Medium] mm: shmem: uncontextualized sb_mpol and ref leak in
  shmem_gmem_alloc_folio()
--

commit bbcb85338583bc4b9f5ee5e619a3f55d643e1098
Author: Ackerley Tng <ackerleytng@google.com>

mm: shmem: Implement guest_memfd provider operations for tmpfs

This commit adds a guest_memfd_provider_operations pointer to struct
file_operations and implements the callbacks in shmem so directory file
descriptors on a tmpfs mount can provide memory to guest_memfd.

> diff --git a/mm/shmem.c b/mm/shmem.c
> index 848316eaa7f4f..2a9faf8be7382 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c

[ ... ]

> @@ -5237,6 +5245,124 @@ static const struct inode_operations shmem_special_inode_operations = {
>  #endif
>  };
>  
> +#ifdef CONFIG_KVM_GUEST_MEMFD

[ ... ]

> +static struct folio *shmem_gmem_alloc_folio(void *provider, pgoff_t index,
> +					    struct mempolicy *mpol)
> +{
> +	struct mempolicy *sb_mpol = NULL;
> +	struct vfsmount *mnt = provider;
> +	struct shmem_sb_info *sbinfo;
> +	struct folio *folio;
> +
> +	sbinfo = SHMEM_SB(mnt->mnt_sb);
> +	if (sbinfo->max_blocks &&
> +	    !percpu_counter_limited_add(&sbinfo->used_blocks,
> +					sbinfo->max_blocks, 1))
> +		return ERR_PTR(-ENOSPC);
> +
> +	if (!mpol) {
> +		sb_mpol = shmem_get_sbmpol(sbinfo);
> +		mpol = sb_mpol;
> +	}

[Severity: Medium]
Can passing sb_mpol directly to folio_alloc_mpol() use an
uncontextualized nodemask?

mpol_parse_str() stores the parsed mount option nodemask in
new->w.user_nodemask without calling mpol_set_nodemask(), expecting it to
be cloned and contextualized later (as mpol_shared_policy_init() does for
regular tmpfs files).  Using sb_mpol directly bypasses
MPOL_F_RELATIVE_NODES translation and cpuset mems_allowed
contextualization during NUMA node selection.

> +	if (mpol)
> +		folio = folio_alloc_mpol(GFP_HIGHUSER_MOVABLE, 0, mpol, index,
> +					 numa_node_id());
> +	else
> +		folio = folio_alloc(GFP_HIGHUSER_MOVABLE, 0);

[Severity: High]
Should shmem_gmem_alloc_folio() use GFP_HIGHUSER instead of
GFP_HIGHUSER_MOVABLE?

Folios returned here are inserted into the guest_memfd mapping, which
kvm_gmem_init_inode() marks with mapping_set_inaccessible() (setting
AS_INACCESSIBLE and AS_UNEVICTABLE) and GFP_HIGHUSER because guest_memfd
memory is neither migratable nor swappable.  Allocating with
__GFP_MOVABLE allows the page allocator to place these unmovable folios
into ZONE_MOVABLE and MIGRATE_CMA pageblocks, causing CMA allocations to
fail with -EBUSY and memory hot-unplug to loop indefinitely trying to
migrate them.

> +
> +	mpol_cond_put(sb_mpol);

[Severity: High]
Does mpol_cond_put(sb_mpol) leak a reference on sbinfo->mpol on every
allocation when mpol is NULL?

shmem_get_sbmpol() unconditionally increments sbinfo->mpol->refcnt via
mpol_get(), whereas mpol_cond_put() only decrements refcnt when
MPOL_F_SHARED is set in pol->flags.  Because sbinfo->mpol is allocated by
mpol_parse_str() -> mpol_new() without MPOL_F_SHARED, mpol_cond_put() is
a no-op here, preventing sbinfo->mpol from being freed on remount or
unmount and eventually wrapping its atomic_t refcnt.  Should this call
mpol_put(sb_mpol) instead?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=3

Re: [PATCH RFC v3 04/88] KVM: guest_memfd: Support provider folio invalidation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] KVM: guest_memfd: AS_RELEASE_ALWAYS set without release_folio()
  in kvm_gmem_aops
--

commit 2a00824af3a696cc40b4c36a78f74b7c53a20eb3
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support provider folio invalidation

This commit adds a provider .invalidate_folio() callback to guest_memfd
and sets mapping_set_release_always() on the guest_memfd address_space so
truncate paths invoke kvm_gmem_invalidate_folio().

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 545c3e646e9c1..ee823f2ef0368 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c

[ ... ]

> @@ -942,10 +950,24 @@ static void kvm_gmem_free_folio(struct folio *folio)
>  }
>  #endif
>  
> +static void kvm_gmem_invalidate_folio(struct folio *folio, size_t offset,
> +				      size_t len)
> +{
> +	struct inode *inode = folio->mapping->host;
> +	struct gmem_inode *gi = GMEM_I(inode);
> +
> +	/* guest_memfd only allows truncating full folios. */
> +	if (WARN_ON_ONCE(offset != 0 || len != folio_size(folio)))
> +		return;
> +
> +	gmem_provider_invalidate_folio(gi, folio);
> +}
> +
>  static const struct address_space_operations kvm_gmem_aops = {
>  	.dirty_folio = noop_dirty_folio,
>  	.migrate_folio	= kvm_gmem_migrate_folio,
>  	.error_remove_folio = kvm_gmem_error_folio,
> +	.invalidate_folio = kvm_gmem_invalidate_folio,
>  #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
>  	.free_folio = kvm_gmem_free_folio,
>  #endif
> @@ -984,6 +1006,7 @@ static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags)
>  	 */
>  	mapping_set_inaccessible(inode->i_mapping);
>  	WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping));
> +	mapping_set_release_always(inode->i_mapping);

[Severity: High]
Does calling mapping_set_release_always() without a .release_folio
callback in kvm_gmem_aops cause filemap_release_folio() to fall back to
try_to_free_buffers() and evict guest_memfd folios without invalidation?

Because mapping_set_release_always() makes folio_needs_release() return
true, filemap_release_folio() runs on the folio and falls back to
try_to_free_buffers() when a_ops->release_folio is absent:

mm/filemap.c:filemap_release_folio() {
	if (mapping && mapping->a_ops->release_folio)
		return mapping->a_ops->release_folio(folio, gfp);
	return try_to_free_buffers(folio);
}

fs/buffer.c:try_to_free_buffers() {
	/* Misconfigured folio check */
	if (WARN_ON_ONCE(!folio_buffers(folio)))
		return true;
...
}

When soft_offline_page() -> soft_offline_in_use_page() ->
mapping_evict_folio() runs on a clean guest_memfd folio with no extra
references (for example, a private folio mapped only in the guest page
tables, as kvm_gmem_get_pfn() drops its folio reference before returning),
mapping_evict_folio() calls filemap_release_folio().  try_to_free_buffers()
triggers a kernel warning and returns true, allowing mapping_evict_folio()
to call remove_mapping() directly.  That removes the folio from
inode->i_mapping without calling kvm_gmem_invalidate_folio() (leaking the
provider's per-folio accounting) and without holding
filemap_invalidate_lock or zapping guest page tables.

Implementing a .release_folio callback in kvm_gmem_aops that returns false
would satisfy the mapping_set_release_always() contract and prevent
mapping_evict_folio() from evicting guest_memfd folios behind KVM's back.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=4

Re: [PATCH RFC v3 05/88] KVM: guest_memfd: Support using resource provider

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] KVM: guest_memfd: movable allocation of unmovable
  tmpfs-provided folios
- [Medium] KVM: guest_memfd: fget_raw() mismatch with f_op in
  kvm_gmem_attach_resource()
- [Medium] kvm: guest_memfd: kvm_gmem_create() checks resource_fd when
  flag is not set
--

commit 120a9139413a4330ef86fed3c561aeed63c5587f
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support using resource provider

This commit adds the GUEST_MEMFD_FLAG_USE_RESOURCE flag and resource_fd
field to struct kvm_create_guest_memfd so guest_memfd instances can attach
to and release an external memory resource provider.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index ee823f2ef0368..84877beed6701 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -85,15 +85,26 @@ static const struct guest_memfd_provider_operations gmem_native_provider_ops = {
>  static int kvm_gmem_attach_resource(struct inode *inode, int resource_fd)
>  {
>  	const struct guest_memfd_provider_operations *ops = NULL;
> +	void *provider = ERR_PTR(-EOPNOTSUPP);
>  	struct file *resource_file = NULL;
> -	void *provider;
>  
> -	if (resource_fd == GMEM_RESOURCE_FD_NATIVE)
> +	if (resource_fd == GMEM_RESOURCE_FD_NATIVE) {
>  		ops = &gmem_native_provider_ops;
> -	else
> -		return -EOPNOTSUPP;
> +	} else {
> +		resource_file = fget_raw(resource_fd);
> +		if (!resource_file)
> +			return -EBADF;
> +
> +		if (resource_file->f_op)
> +			ops = resource_file->f_op->gmem_provider_ops;
> +	}

[Severity: Medium]
Why does kvm_gmem_attach_resource() use fget_raw() and then read
resource_file->f_op->gmem_provider_ops?

fget_raw() only differs from fget() by accepting O_PATH file descriptors
(such as descriptors from fsmount(), open_tree(), or open(..., O_PATH)),
for which do_dentry_open() sets f->f_op to &empty_fops rather than
inode->i_fop.  As a result, any O_PATH descriptor accepted by fget_raw()
finds resource_file->f_op->gmem_provider_ops == NULL and fails with
-EOPNOTSUPP.

If O_PATH descriptors are meant to be supported, gmem_provider_ops needs
to be read from file_inode(resource_file)->i_fop; otherwise, fget() should
be used so O_PATH descriptors are rejected with -EBADF.

> +
> +	if (ops && ops->attach && ops->alloc_folio)
> +		provider = ops->attach(inode, resource_file);

[Severity: High]
Does attaching a tmpfs provider place non-migratable guest_memfd folios
into movable pageblocks and ZONE_MOVABLE?

kvm_gmem_init_inode() marks inode->i_mapping with
mapping_set_inaccessible() and GFP_HIGHUSER because guest_memfd folios
cannot be migrated: kvm_gmem_migrate_folio() returns -EINVAL and
move_to_new_folio() rejects mapping_inaccessible() folios with
-EOPNOTSUPP.  However, when attached to tmpfs via shmem_gmem_provider_ops,
kvm_gmem_get_folio() calls shmem_gmem_alloc_folio(), which allocates with
GFP_HIGHUSER_MOVABLE before inserting the folio into the inaccessible
guest_memfd mapping:

mm/shmem.c:shmem_gmem_alloc_folio() {
    ...
	if (mpol)
		folio = folio_alloc_mpol(GFP_HIGHUSER_MOVABLE, 0, mpol, index,
					 numa_node_id());
	else
		folio = folio_alloc(GFP_HIGHUSER_MOVABLE, 0);
    ...
}

Allocating non-migratable folios with __GFP_MOVABLE pins MIGRATE_MOVABLE,
MIGRATE_CMA, and ZONE_MOVABLE memory, breaking CMA allocations,
compaction, and memory hot-unplug.  shmem_gmem_alloc_folio() should omit
__GFP_MOVABLE (or honour mapping_gfp_mask(inode->i_mapping)) when
allocating folios for guest_memfd.

[ ... ]

> @@ -1111,6 +1128,7 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
>  
>  int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
>  {
> +	int resource_fd = GMEM_RESOURCE_FD_NATIVE;
>  	loff_t size = args->size;
>  	u64 flags = args->flags;
>  
> @@ -1120,7 +1138,15 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
>  	if (size <= 0 || !PAGE_ALIGNED(size))
>  		return -EINVAL;
>  
> -	return __kvm_gmem_create(kvm, size, flags);
> +	if (flags & GUEST_MEMFD_FLAG_USE_RESOURCE) {
> +		if (args->resource_fd < 0)
> +			return -EBADF;
> +		resource_fd = args->resource_fd;
> +	} else if (args->resource_fd) {
> +		return -EINVAL;
> +	}
> +
> +	return __kvm_gmem_create(kvm, size, flags, resource_fd);
>  }

[Severity: Medium]
Does rejecting a non-zero args->resource_fd when
GUEST_MEMFD_FLAG_USE_RESOURCE is not set break backwards compatibility for
KVM_CREATE_GUEST_MEMFD?

Since 1a2b3c4d5e6f... wait - since a7800aa80ea4 ("KVM: Add
KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory"),
kvm_gmem_create() never validated that args->reserved[0..5] was zero.
Carving resource_fd out of reserved[0] and requiring args->resource_fd == 0
when GUEST_MEMFD_FLAG_USE_RESOURCE is clear causes a regression (-EINVAL)
for existing userspace callers that leave reserved[] uninitialized on the
stack, or that initialize unused file descriptor fields to -1.  In
addition, args->pad and args->reserved[0..4] are still left unvalidated.

Should args->resource_fd only be inspected when
GUEST_MEMFD_FLAG_USE_RESOURCE is set (similar to how kvm_set_memory_region()
only inspects guest_memfd when KVM_MEM_GUEST_MEMFD is set)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=5

Re: [PATCH RFC v3 06/88] KVM: selftests: Add helper to create guest_memfd with a resource file

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] KVM: guest_memfd: missing release_folio() with
  mapping_set_release_always()
- [Medium] mm: shmem: unmovable guest_memfd folios allocated with
  GFP_HIGHUSER_MOVABLE
--

commit a3e5841f5c68c0e42d4540b30c82c6fced7d8d4f
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Add helper to create guest_memfd with a resource file

This commit syncs the KVM UAPI header for GUEST_MEMFD_FLAG_USE_RESOURCE and
adds selftest helpers to create a guest_memfd instance backed by a resource
file descriptor.

> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index e2618877619e1..eca296539db79 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -809,26 +809,39 @@ static inline bool is_smt_on(void)
>  
>  void vm_create_irqchip(struct kvm_vm *vm);
>  
> -static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
> -					  u64 flags)
> +static inline int __vm_create_guest_memfd_resource(struct kvm_vm *vm, u64 size,
> +						   u64 flags, int resource_fd)
>  {
>  	struct kvm_create_guest_memfd guest_memfd = {
>  		.size = size,
>  		.flags = flags,
> +		.resource_fd = resource_fd,
>  	};
>  
>  	return __vm_ioctl(vm, KVM_CREATE_GUEST_MEMFD, &guest_memfd);
>  }

[Severity: High]
In the underlying kernel support for provider folio invalidation added
earlier in this series in commit "KVM: guest_memfd: Support provider folio
invalidation", kvm_gmem_init_inode() calls
mapping_set_release_always(inode->i_mapping), but kvm_gmem_aops does not
implement .release_folio.  Can this cause mapping_evict_folio() to warn and
evict guest_memfd folios without invalidating KVM MMU mappings or provider
accounting?

Because AS_RELEASE_ALWAYS is set, folio_needs_release() is true in
filemap_release_folio(), which falls back to try_to_free_buffers() when
a_ops->release_folio is NULL:

fs/buffer.c:try_to_free_buffers() {
	/* Misconfigured folio check */
	if (WARN_ON_ONCE(!folio_buffers(folio)))
		return true;

When soft_offline_in_use_page() calls mapping_evict_folio() on a clean,
unmapped guest_memfd folio, filemap_release_folio() triggers that warning
and returns true.  mapping_evict_folio() then removes the folio from the
page cache without calling kvm_gmem_invalidate_folio() or notifying the
KVM MMU, leaving stage-2 page tables pointing to the freed page and leaking
tmpfs provider block accounting.

> -static inline int vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
> -					u64 flags)
> +static inline int __vm_create_guest_memfd(struct kvm_vm *vm, u64 size,
> +					  u64 flags)
> +{
> +	return __vm_create_guest_memfd_resource(vm, size, flags, 0);
> +}
> +
> +static inline int vm_create_guest_memfd_resource(struct kvm_vm *vm, u64 size,
> +						 u64 flags, int resource_fd)
>  {
> -	int fd = __vm_create_guest_memfd(vm, size, flags);
> +	int fd = __vm_create_guest_memfd_resource(vm, size, flags, resource_fd);
>  
>  	TEST_ASSERT(fd >= 0, KVM_IOCTL_ERROR(KVM_CREATE_GUEST_MEMFD, fd));
>  	return fd;
>  }

[Severity: Medium]
When a guest_memfd is created with a tmpfs resource_fd (via commit "mm:
shmem: Implement guest_memfd provider operations for tmpfs" earlier in this
series), does shmem_gmem_alloc_folio() place unmovable folios into
ZONE_MOVABLE or MIGRATE_CMA?

kvm_gmem_init_inode() sets GFP_HIGHUSER (without __GFP_MOVABLE) and marks
the mapping AS_INACCESSIBLE because guest_memfd folios cannot be migrated.
However, shmem_gmem_alloc_folio() allocates with GFP_HIGHUSER_MOVABLE:

mm/shmem.c:shmem_gmem_alloc_folio() {
	...
	if (mpol)
		folio = folio_alloc_mpol(GFP_HIGHUSER_MOVABLE, 0, mpol, index,
					 numa_node_id());
	else
		folio = folio_alloc(GFP_HIGHUSER_MOVABLE, 0);

Once kvm_gmem_get_folio() inserts the folio into the AS_INACCESSIBLE
guest_memfd mapping, migration is rejected, pinning the page in
ZONE_MOVABLE or CMA and breaking CMA allocations or memory hot-unplug.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=6

Re: [PATCH RFC v3 10/88] KVM: selftests: Test rejection of swap tmpfs mounts

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: selftests: missing CAP_SYS_ADMIN check in
  create_tmpfs_pool_fd()
--

commit 3ffb77e319f3b82f8e3aaa6439dea9dda37671d4
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Test rejection of swap tmpfs mounts

This commit adds a helper to create a tmpfs mount fd and tests that
guest_memfd creation fails with -EINVAL when given a swap-enabled tmpfs
resource pool fd.

> diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
> index d62ae23256083..205f822e10c6c 100644
> --- a/tools/testing/selftests/kvm/guest_memfd_test.c
> +++ b/tools/testing/selftests/kvm/guest_memfd_test.c

[ ... ]

> @@ -478,6 +480,73 @@ static void test_resource_fd_memfd(struct kvm_vm *vm)
>  	close(file_fd);
>  }
>  
> +static int create_tmpfs_pool_fd(const char *huge, bool noswap, size_t size)
> +{
> +	char size_str[32];
> +	char template[] = "/tmp/gmem_test_dir_XXXXXX";
> +	char *mount_dir;
> +	int fs_fd, mnt_fd, dir_fd;
> +
> +	TEST_ASSERT(size && IS_ALIGNED(size, page_size),
> +		    "Pool size '0x%zx' must be positive and page-aligned", size);
> +
> +	fs_fd = syscall(__NR_fsopen, "tmpfs", FSOPEN_CLOEXEC);
> +	TEST_ASSERT(fs_fd >= 0, "fsopen failed");
> +
> +	if (noswap)
> +		TEST_ASSERT(!syscall(__NR_fsconfig, fs_fd, FSCONFIG_SET_FLAG,
> +				     "noswap", NULL, 0), "fsconfig noswap failed");

[ ... ]

> @@ -539,6 +608,7 @@ static void test_guest_memfd(unsigned long vm_type)
>  	test_resource_fd_invalid(vm);
>  	test_resource_fd_unsupported_fs(vm);
>  	test_resource_fd_memfd(vm);
> +	test_resource_fd_tmpfs_swap(vm);
>  
>  	__test_guest_memfd(vm, 0);

[Severity: Medium]
Does running guest_memfd_test without CAP_SYS_ADMIN abort the entire test
binary here?

fsopen() fails with EPERM when the process lacks CAP_SYS_ADMIN (and setting
"noswap" via fsconfig() fails with EINVAL unless capable(CAP_SYS_ADMIN) in
init_user_ns), causing TEST_ASSERT() to exit with code 254 instead of
skipping.  Because test_resource_fd_tmpfs_swap() runs unconditionally
before __test_guest_memfd(vm, 0), unprivileged users with access to
/dev/kvm can no longer run the standard guest_memfd tests.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=10

Re: [PATCH RFC v3 11/88] KVM: selftests: Test rejection of hugepage tmpfs mounts

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: selftests: missing THP check in
  test_resource_fd_tmpfs_huge()
--

commit c298eb47e39bc0e14d17d7c71365382bc9f5859c
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Test rejection of hugepage tmpfs mounts

This commit adds a selftest verifying that guest_memfd creation fails with
-EINVAL when backed by a tmpfs resource pool mounted with huge=always.

> diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
> index 205f822e10c6c..541d8cdd0bbf7 100644
> --- a/tools/testing/selftests/kvm/guest_memfd_test.c
> +++ b/tools/testing/selftests/kvm/guest_memfd_test.c
> @@ -547,6 +547,21 @@ static void test_resource_fd_tmpfs_swap(struct kvm_vm *vm)
>  	close(pool_fd);
>  }
>  
> +static void test_resource_fd_tmpfs_huge(struct kvm_vm *vm)
> +{
> +	int pool_fd, fd;
> +
> +	pool_fd = create_tmpfs_pool_fd("always", true, page_size);

[Severity: Medium]
Should test_resource_fd_tmpfs_huge() check thp_configured() before
configuring a tmpfs mount with huge=always?

In shmem_parse_one(), setting huge=always returns -EINVAL when
CONFIG_TRANSPARENT_HUGEPAGE is disabled or has_transparent_hugepage() is
false.  Because create_tmpfs_pool_fd() wraps FSCONFIG_SET_STRING for "huge"
in TEST_ASSERT(), running guest_memfd_test on a system without transparent
hugepages aborts the test suite during setup instead of skipping this
subtest.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=11

Re: [PATCH RFC v3 17/88] KVM: selftests: Test guest execution with tmpfs-backed guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] KVM: selftests: undersized tmpfs pool in
  test_guest_memfd_guest_resource()
- [Low] KVM: selftests: dead check and duplication in
  test_guest_memfd_guest_resource()
--

commit 5b99efdf7e83feb9f2f28a33ce29eb94813f4546
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Test guest execution with tmpfs-backed guest_memfd

This commit adds a selftest verifying that a guest vCPU can execute and
access memory mapped through a guest_memfd backed by a tmpfs resource pool.

> diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c
> index 20de24ad535f1..8e9261db23354 100644
> --- a/tools/testing/selftests/kvm/guest_memfd_test.c
> +++ b/tools/testing/selftests/kvm/guest_memfd_test.c
> @@ -799,6 +799,61 @@ static void test_guest_memfd_guest(void)
>  	kvm_vm_free(vm);
>  }
>  
> +static void test_guest_memfd_guest_resource(void)
> +{
> +	const gpa_t gpa = SZ_4G;
> +	const int slot = 1;
> +	struct kvm_vcpu *vcpu;
> +	struct kvm_vm *vm;
> +	int pool_fd, fd, i;
> +	size_t size;
> +	u8 *mem;
> +
> +	if (!kvm_check_cap(KVM_CAP_GUEST_MEMFD_FLAGS))
> +		return;
> +
> +	pool_fd = create_tmpfs_pool_fd("never", true, page_size);
> +	if (pool_fd < 0)
> +		TEST_REQUIRE(false);
> +
> +	vm = __vm_create_shape_with_one_vcpu(VM_SHAPE_DEFAULT, &vcpu, 1, guest_code);
> +
> +	TEST_ASSERT(vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS) & GUEST_MEMFD_FLAG_MMAP,
> +		    "Default VM type should support MMAP, supported flags = 0x%x",
> +		    vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS));
> +	TEST_ASSERT(vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS) & GUEST_MEMFD_FLAG_INIT_SHARED,
> +		    "Default VM type should support INIT_SHARED, supported flags = 0x%x",
> +		    vm_check_cap(vm, KVM_CAP_GUEST_MEMFD_FLAGS));
> +
> +	size = max_t(size_t, vm->page_size, page_size);
> +	fd = __vm_create_guest_memfd_resource(vm, size,
> +					      GUEST_MEMFD_FLAG_MMAP |
> +					      GUEST_MEMFD_FLAG_INIT_SHARED |
> +					      GUEST_MEMFD_FLAG_USE_RESOURCE,
> +					      pool_fd);
> +	TEST_ASSERT(fd >= 0, "guest_memfd with tmpfs pool should succeed");

[Severity: Low]
This isn't a bug, but test_guest_memfd_guest_resource() duplicates almost the
entire body of test_guest_memfd_guest() rather than parameterizing the
existing helper.  In addition, create_tmpfs_pool_fd() asserts via
TEST_ASSERT() on every error path and never returns a negative value, making
the pool_fd < 0 check unreachable, and the __vm_create_guest_memfd_resource()
plus TEST_ASSERT(fd >= 0, ...) call open-codes the existing
vm_create_guest_memfd_resource() wrapper.

> +
> +	vm_set_user_memory_region2(vm, slot, KVM_MEM_GUEST_MEMFD, gpa, size, NULL, fd, 0);
> +
> +	mem = kvm_mmap(size, PROT_READ | PROT_WRITE, MAP_SHARED, fd);
> +	memset(mem, 0xaa, size);

[Severity: Medium]
Will this memset() crash with SIGBUS when vm->page_size is larger than the
host page_size (for example, on LoongArch where VM_MODE_DEFAULT uses 16K
guest pages on a 4K-page host)?

create_tmpfs_pool_fd() limits the backing tmpfs mount to page_size bytes
before the VM is created, whereas the guest_memfd is sized and faulted in
across size = max_t(size_t, vm->page_size, page_size) bytes.  When
vm->page_size > page_size, faulting past the first host page hits -ENOSPC in
shmem_gmem_alloc_folio(), which kvm_gmem_fault_user_mapping() turns into
VM_FAULT_SIGBUS.  Creating pool_fd after computing size (and passing size as
the capacity) avoids exhausting the pool.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=17

Re: [PATCH RFC v3 19/88] KVM: selftests: Test guest_memfd conversions with tmpfs backing

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] KVM: selftests: gmem_fd leak in gmem_conversions_do_setup()
--

commit 4496ec39f3ba447f6a2dfb084849079edae5c81f
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Test guest_memfd conversions with tmpfs backing

This commit moves create_tmpfs_pool_fd() into test_util.c and extends
guest_memfd_conversions_test with fixture variants to test both native
and tmpfs-backed guest_memfd instances.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
> index e1c84df0f0596..023df9cb9d4a6 100644
> --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
[ ... ]
> @@ -50,12 +69,27 @@ static void gmem_conversions_do_setup(test_data_t *t, int nr_pages,
>  	 */
>  	const gpa_t gpa = SZ_4G;
>  	const u32 slot = 1;
> +	size_t size = nr_pages * page_size;
>  	struct kvm_vm *vm;
> +	int gmem_fd = -1;
>  
>  	vm = __vm_create_shape_with_one_vcpu(shape, &t->vcpu, nr_pages, guest_do_rmw);
>  
> +	t->resource_fd = -1;
> +
> +	if (variant->create_resource) {
> +		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE;
> +
> +		t->resource_fd = variant->create_resource(size);
> +		TEST_ASSERT(t->resource_fd >= 0, "Failed to create backing resource");
> +
> +		gmem_fd = __vm_create_guest_memfd_resource(vm, size, flags,
> +							   t->resource_fd);
> +		TEST_ASSERT(gmem_fd >= 0, "Failed to create guest_memfd with resource");
> +	}
> +
>  	vm_mem_add(vm, VM_MEM_SRC_SHMEM, gpa, slot, nr_pages,
> -		   KVM_MEM_GUEST_MEMFD, -1, 0, gmem_flags);
> +		   KVM_MEM_GUEST_MEMFD, gmem_fd, 0, gmem_flags);
>  
>  	t->gmem_fd = kvm_slot_to_fd(vm, slot);
>  	t->mem = addr_gpa2hva(vm, gpa);

[Severity: Low]
Does gmem_conversions_do_setup() leak the local gmem_fd when
variant->create_resource is set?

When a caller passes a non-negative gmem_fd to vm_mem_add(), vm_mem_add()
duplicates it via kvm_dup() for the memslot region and leaves ownership
of the passed-in descriptor with the caller.  Because gmem_fd is never
closed here, the guest_memfd file, its tmpfs mount and folios, and the VM
instance remain pinned across kvm_vm_free() in
gmem_conversions_do_teardown(), including across each loop iteration in
GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED tests.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=19

Re: [PATCH RFC v3 25/88] KVM: selftests: Add basic test for sample memory provider

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: selftests: open_provider_dev() reimplements
  __open_path_or_exit()
--

commit 0532d9b657d56f90c6d83715d992908b2bbe987a
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Add basic test for sample memory provider

This commit adds a selftest for the sample memory provider to test file
creation across multiple sizes and verify initial statistics.

> diff --git a/tools/testing/selftests/kvm/mem_provider_sample_test.c b/tools/testing/selftests/kvm/mem_provider_sample_test.c
> new file mode 100644
> index 0000000000000..b39161f4933d3
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/mem_provider_sample_test.c
> @@ -0,0 +1,85 @@
[ ... ]
> +#define DEV_PATH "/dev/memory_provider"
> +
> +static int open_provider_dev(void)
> +{
> +	int fd = open(DEV_PATH, O_RDWR);
> +
> +	if (fd < 0) {
> +		TEST_REQUIRE(errno != ENOENT && errno != ENODEV);
> +		TEST_FAIL("Failed to open %s: %d (%s)", DEV_PATH, errno,
> +			  strerror(errno));
> +	}
> +	return fd;
> +}

[Severity: Medium]
Could open_provider_dev() use __open_path_or_exit() from kvm_util.h
instead of hand-rolling the open() error handling?

/dev/memory_provider is registered with default 0600 permissions, so
running the test as an unprivileged user fails with EACCES and hits
TEST_FAIL().  Inside test_assert(), test_dump_stack() prints an assertion
failure stack trace and calls system(), clobbering errno before the
EACCES check and exiting with status 254 instead of KSFT_SKIP:

tools/testing/selftests/kvm/lib/assert.c:test_assert() {
	if (!(exp)) {
		...
		test_dump_stack();
		...
		if (errno == EACCES) {
			print_skip("Access denied - Exiting");
			exit(KSFT_SKIP);
		}
		exit(254);
	}
}

Using __open_path_or_exit() skips cleanly via ksft_exit_skip() on both
EACCES and ENOENT, and prints the device path and prerequisite help text
instead of the literal TEST_REQUIRE() condition when the module is not
loaded.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=25

Re: [PATCH RFC v3 28/88] KVM: guest_memfd: Support reporting PFN attributes

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential
issues to consider:
- [Medium] KVM: arm64: gmem_abort() exits on L2 write when L1 S2 is
  read-only
- [Low] KVM: arm64: nv: unresolved TODO comment in kvm_translate_vncr()
- [Low] KVM: SVM: shadowed read-only PFN check in sev_handle_rmp_fault()
- [Low] KVM: arm64: nv: unaligned GPA passed with PAGE_SIZE in
  kvm_translate_vncr()
--

commit 0c51d556213ce4b6cb7adb6d1befd0cf237a2240
Author: Fred Griffoul <fgriffo@amazon.co.uk>

KVM: guest_memfd: Support reporting PFN attributes

This commit adds a pfn_attrs output parameter to kvm_gmem_get_pfn() and
__kvm_gmem_get_pfn(), defines GUEST_MEMFD_PFN_ATTRS_READONLY, and
handles read-only guest_memfd frames across x86 and arm64 fault paths.

> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 705f6d84770e9..8f0217185973c 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
[ ... ]
> @@ -1773,18 +1775,21 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd,
>  	/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
>  	smp_rmb();
>  
> -	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, NULL);
> -	if (ret) {
> +	ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, NULL,
> +			       &pfn_attrs);
> +	writable = !(s2fd->memslot->flags & KVM_MEM_READONLY) &&
> +		   !(pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY);
> +	if (ret || (write_fault && !writable)) {
>  		/* If result is non-NULL this is a synthetic fault. */
>  		if (!result)
>  			kvm_prepare_memory_fault_exit(s2fd->vcpu, gfn_to_gpa(canonical_gfn),
>  						      s2vi.vma_pagesize, write_fault, exec_fault,
>  						      false);
>  		kfree(mapping);
> -		return ret;
> +		return ret ?: -EFAULT;
>  	}

[Severity: Medium]
When L2 writes to an unmapped shadow Stage-2 IPA that L1 mapped
read-only in its own Stage-2 page tables, kvm_s2_handle_perm_fault()
returns 0 on the initial translation fault so that gmem_abort() installs
a read-only shadow Stage-2 PTE and the retried write traps as a
permission fault to be forwarded to L1:

arch/arm64/kvm/nested.c:kvm_s2_handle_perm_fault() {
	if (!kvm_vcpu_trap_is_permission_fault(vcpu))
		return 0;
    ...
}

Does checking (write_fault && !writable) without checking whether
s2fd->nested permits writes cause gmem_abort() to exit to L0 userspace
with KVM_EXIT_MEMORY_FAULT instead of installing the read-only shadow
Stage-2 mapping when the backing PFN has GUEST_MEMFD_PFN_ATTRS_READONLY
set?  Checking (!s2fd->nested || kvm_s2_trans_writable(s2fd->nested))
here would let the read-only shadow mapping be installed so the Stage-2
permission fault is delivered to L1.

[ ... ]

> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index 4d83281a7a28b..4519e92d89d66 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -1615,14 +1615,24 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
>  			return -EFAULT;
>  		}
>  	} else {
> -		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, NULL);
> +		u64 pfn_attrs;
> +
> +		ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, NULL,
> +				       &pfn_attrs);
>  		if (ret) {
>  			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
>  					      write_fault, false, false);
>  			return ret;
>  		}
>  
> -		writable = !(memslot->flags & KVM_MEM_READONLY);
> +		writable = !(memslot->flags & KVM_MEM_READONLY) &&
> +			   !(pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY);
> +		/* TODO: Explain why vt->wr.pw needs to be checked. */

[Severity: Low]
This isn't a bug, but this placeholder TODO comment was left in place
instead of explaining that !vt->wr.pw must return 0 so
kvm_handle_vncr_abort() injects a stage-1 permission fault into the
guest via handle_vncr_perm() rather than exiting to host userspace.

> +		if (write_fault && vt->wr.pw && !writable) {
> +			kvm_prepare_memory_fault_exit(vcpu, vt->wr.pa, PAGE_SIZE,
> +						      true, false, false);
> +			return -EFAULT;
> +		}
>  	}

[Severity: Low]
VNCR_EL2 and vt->wr.pa are 4KB-aligned regardless of the host page size.
On a 16KB or 64KB page size host, does passing vt->wr.pa with PAGE_SIZE
to kvm_prepare_memory_fault_exit() report an unaligned GPA range
[vt->wr.pa, vt->wr.pa + PAGE_SIZE) that straddles into gfn + 1 instead
of gfn_to_gpa(gfn)?

[ ... ]

> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index 6427db390b1cd..5e37ea4bc7937 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
[ ... ]
> @@ -5092,13 +5097,19 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
>  	mmu_seq = kvm->mmu_invalidate_seq;
>  	smp_rmb();
>  
> -	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order);
> +	ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order, &pfn_attrs);
>  	if (ret) {
>  		pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n",
>  				    gpa);
>  		return;
>  	}
>  
> +	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY) {
> +		pr_warn_ratelimited("SEV: Unexpected RMP fault, read-only backing page for GPA 0x%llx\n",
> +				    gpa);
> +		return;
> +	}

[Severity: Low]
Can this GUEST_MEMFD_PFN_ATTRS_READONLY check be reached for a private
GPA?  In kvm_gmem_get_pfn(), a private page with
GUEST_MEMFD_PFN_ATTRS_READONLY set already fails with -EFAULT, so the
if (ret) branch above catches it first and logs "no backing page for
private GPA" instead of "read-only backing page".

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=28

Re: [PATCH RFC v3 32/88] KVM: guest_memfd: Support user fault handling in PFN mode

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: guest_memfd: false SIGBUS in kvm_gmem_pfn_mkwrite()
  after mprotect()
--

commit f3ec1839826136ca55a20572dfcf8ef47f4e0d64
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support user fault handling in PFN mode

This commit adds user fault handling support for guest_memfd in PFN mode,
inserting PFNs with protections constrained by the provider's PFN
attributes and introducing kvm_gmem_pfn_vm_ops with a .pfn_mkwrite
callback.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 183e15775523f..e94f2359b7c9a 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -565,11 +565,40 @@ static bool kvm_gmem_supports_mmap(struct inode *inode)
>  	return GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_MMAP;
>  }
>  
> +/*
> + * In PFN mode, physical frame attributes are constant for the lifetime of the
> + * mapping. If a page was inserted read-only, it was because the provider
> + * reported a read-only PFN attribute, and write upgrades are never permitted.
> + * Write faults on such pages must fail with SIGBUS.
> + */
> +static vm_fault_t kvm_gmem_pfn_mkwrite(struct vm_fault *vmf)
> +{
> +	return VM_FAULT_SIGBUS;
> +}
> +
> +static vm_fault_t gmem_insert_pfn(struct vm_fault *vmf, kvm_pfn_t pfn,
> +				 u64 pfn_attrs)
> +{
> +	struct vm_area_struct *vma = vmf->vma;
> +	vm_flags_t flags = vma->vm_flags;
> +
> +	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)
> +		flags &= ~VM_WRITE;
> +
> +	if ((vmf->flags & FAULT_FLAG_WRITE) && !(flags & VM_WRITE))
> +		return VM_FAULT_SIGBUS;
> +
> +	return vmf_insert_pfn_prot(vma, vmf->address, pfn,
> +				   vm_get_page_prot(flags));
> +}

[Severity: Medium]
Can kvm_gmem_pfn_mkwrite() deliver a spurious SIGBUS when userspace writes
to a writable PFN after mprotect(PROT_READ | PROT_WRITE)?

If userspace maps a guest_memfd with PROT_READ and MAP_SHARED and faults
in a writable PFN via a read access, gmem_insert_pfn() installs a clean,
read-only PTE because vma->vm_flags lacks VM_WRITE.  Because
kvm_gmem_pfn_vm_ops sets .pfn_mkwrite, vma_wants_writenotify() is true for
the VMA and vma_set_page_prot() keeps vma->vm_page_prot read-only;
a subsequent mprotect(..., PROT_READ | PROT_WRITE) therefore leaves the
clean PTE read-only (can_change_shared_pte_writable() checks
pte_dirty(pte)) so that the next write traps into .pfn_mkwrite, where
kvm_gmem_pfn_mkwrite() unconditionally returns VM_FAULT_SIGBUS.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=32

Re: [PATCH RFC v3 39/88] KVM: guest_memfd: Support put_pfn_range provider op in PFN mode

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: guest_memfd: missing invalidate_lock in
  kvm_gmem_destroy_inode()
--

commit caed7fc7e8fd2f7ee37985c5b86064cab5aac2e9
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support put_pfn_range provider op in PFN mode

This commit introduces the .put_pfn_range callback in struct
guest_memfd_provider_operations and invokes it after kvm_gmem_pfn_reclaim()
during hole punching and inode destruction in PFN mode.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index b46907b480138..501c3e250fc98 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c

[ ... ]

> @@ -1730,8 +1738,12 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
>  	 * Skip if the provider was not yet set up, as iput() can reach this on
>  	 * error paths before provider_ops is initialized.
>  	 */
> -	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi))
> -		kvm_gmem_pfn_reclaim(gi, 0, i_size_read(inode) >> PAGE_SHIFT);
> +	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi)) {
> +		pgoff_t nr_pages = i_size_read(inode) >> PAGE_SHIFT;
> +
> +		kvm_gmem_pfn_reclaim(gi, 0, nr_pages);
> +		gmem_provider_put_pfn_range(gi, 0, nr_pages);
> +	}
>  
>  	gmem_provider_release(gi);

[Severity: Medium]
Should kvm_gmem_destroy_inode() hold filemap_invalidate_lock(inode->i_mapping)
across gmem_provider_put_pfn_range()?

The kernel-doc for struct guest_memfd_provider_operations documents that
.put_pfn_range() is called with the exclusive filemap invalidate lock held,
and kvm_gmem_destroy_inode() only takes filemap_invalidate_lock() later
around __mt_destroy(&gi->attributes):

virt/kvm/guest_memfd.c:kvm_gmem_destroy_inode() {
	...
	if (!mtree_empty(&gi->attributes)) {
		filemap_invalidate_lock(inode->i_mapping);
		__mt_destroy(&gi->attributes);
		filemap_invalidate_unlock(inode->i_mapping);
	}
}

Because the provider receives gmem_inode in .attach() and remains attached
until gmem_provider_release() runs after gmem_provider_put_pfn_range(), a
provider that asserts or relies on gmem_inode->i_mapping->invalidate_lock in
.put_pfn_range() will trigger a lockdep splat or race during inode teardown.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=39

Re: [PATCH RFC v3 42/88] KVM: guest_memfd: Export kvm_gmem_revoke for provider-driven revocation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] KVM: guest_memfd: race between kvm_gmem_revoke() and inode
  teardown
- [Low] KVM: guest_memfd: stray @mmap kernel-doc entry in guest_memfd.h
--

commit 3bc97f7ff873e4720a82787260ba8378c8ce9990
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Export kvm_gmem_revoke for provider-driven revocation

This commit introduces __kvm_gmem_revoke() and exports kvm_gmem_revoke()
so external memory providers can revoke physical memory ranges from
guest_memfd without calling back into .put_pfn_range().

> diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
> index 697f8f8c9b0ae..fb2279a001ebf 100644
> --- a/include/linux/guest_memfd.h
> +++ b/include/linux/guest_memfd.h
> @@ -28,8 +28,13 @@ struct mempolicy;
>   *           @pfn with uniform @pfn_attrs in @max_order. Once @get_pfn succeeds,
>   *           the provider must keep the @index-to-@pfn mapping and @pfn_attrs
>   *           constant, and the memory accessible, until guest_memfd stops using
> - *           this page. Returns a negative error code on failure. The provider
> - *           must define @lookup_pfn_range when @get_pfn is defined.
> + *           this page. Do not take locks held across kvm_gmem_revoke().
> + *           Returns a negative error code on failure. The provider must define
> + *           @lookup_pfn_range when @get_pfn is defined.
> + * @mmap: Optional callback to set up user mappings. Called when userspace calls
> + *        mmap() on guest_memfd. The provider can customize VMA flags,
> + *        cacheability/protections, or reject the mapping. If not defined,
> + *        guest_memfd falls back to default mapping behavior.

[Severity: Low]
Does this @mmap kernel-doc entry belong in this commit?
struct guest_memfd_provider_operations has no mmap field here, which
triggers a scripts/kernel-doc warning and leaves a duplicate @mmap entry
once commit "KVM: guest_memfd: Support mmap provider operation" adds its
own description later in the series.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 1b6fc44ba00bb..023bf095340be 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c

[ ... ]

> @@ -469,6 +482,45 @@ static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
>  	return 0;
>  }
>  
> +static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len)
> +{
> +	return __kvm_gmem_revoke(inode, offset >> PAGE_SHIFT, len >> PAGE_SHIFT,
> +				 NULL, true);
> +}
> +
> +/**
> + * kvm_gmem_revoke - Revoke and reclaim a range of memory in guest_memfd.
> + * @inode:    The guest_memfd inode.
> + * @start:    Starting page index within the file.
> + * @nr_pages: Number of pages to revoke.
> + * @pfn:      The physical address / frame number being revoked. Directly
> + *            reclaims this physical range without querying the provider's
> + *            .lookup_pfn_range().
> + *
> + * guest_memfd will do everything to make sure the range revoked is no longer in
> + * use across guest and host mappings, as well as reclaim any confidential
> + * architecture state.
> + *
> + * Before calling kvm_gmem_revoke(), the caller must ensure that future
> + * .get_pfn() calls for the revoked range will fail so new mappings cannot be
> + * faulted back in after kvm_gmem_revoke() releases the inode's invalidate_lock.
> + * Must be called before accessing, freeing, resetting, or powering down the
> + * backing resource, and the caller must keep the @start-to-PFN mapping valid
> + * and accessible until kvm_gmem_revoke() returns. .put_pfn_range() will not be
> + * called.
> + *
> + * Acquires @inode->i_mapping->invalidate_lock exclusively; callers must not
> + * hold any lock that is also acquired inside provider callbacks.
> + *
> + * Return: 0 on success, or a negative error code on failure.
> + */
> +int kvm_gmem_revoke(struct inode *inode, pgoff_t start, size_t nr_pages,
> +		    const unsigned long *pfn)
> +{
> +	return __kvm_gmem_revoke(inode, start, nr_pages, pfn, false);
> +}
> +EXPORT_SYMBOL_GPL(kvm_gmem_revoke);

[Severity: High]
Can kvm_gmem_revoke() race with inode teardown in
kvm_gmem_destroy_inode()?

virt/kvm/guest_memfd.c:kvm_gmem_destroy_inode() {
    ...
	if (gi->provider_ops && kvm_gmem_is_pfn_mode(gi)) {
		pgoff_t nr_pages = i_size_read(inode) >> PAGE_SHIFT;

		kvm_gmem_pfn_reclaim(gi, 0, nr_pages);
		gmem_provider_put_pfn_range(gi, 0, nr_pages);
	}

	gmem_provider_release(gi);
    ...
}

Because provider callbacks cannot take provider locks held across
kvm_gmem_revoke(), inode->i_mapping->invalidate_lock is the only lock
serializing kvm_gmem_revoke() against reclaim and .put_pfn_range(), yet
kvm_gmem_destroy_inode() runs both without holding
filemap_invalidate_lock() before gmem_provider_release() detaches the
provider:

CPU0                                  CPU1
iput(inode)
  evict()
    kvm_gmem_destroy_inode()
      kvm_gmem_pfn_reclaim()
        gmem_provider_lookup_pfn_range()
                                      /* provider marks range absent */
                                      kvm_gmem_revoke(inode, ..., &pfn)
                                        filemap_invalidate_lock()
                                        kvm_arch_gmem_reclaim(*pfn, ...)
                                        filemap_invalidate_unlock()
                                      /* provider frees or resets memory */
        kvm_arch_gmem_reclaim(pfn, ...)
          <-- concurrent reclaim / use-after-free
      gmem_provider_put_pfn_range()

Since kvm_gmem_destroy_inode() does not hold filemap_invalidate_lock(),
kvm_gmem_revoke() can run kvm_arch_gmem_reclaim() concurrently on the same
PFNs and return 0 to the provider while CPU0 is still reclaiming and
releasing the backing pages.  In folio mode, evict() also runs
truncate_inode_pages_final() and clear_inode() before
kvm_gmem_destroy_inode() calls gmem_provider_release(), allowing a
concurrent kvm_gmem_revoke() to call truncate_inode_pages_range() during
or after final inode truncation.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=42

Re: [PATCH RFC v3 44/88] samples/kvm: Implement attach and release provider ops in sample provider

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] samples/kvm: build failure when CONFIG_KVM_GUEST_MEMFD is
  disabled
--

commit 5fdcfe92c76b1d01309d2b39a0b5cac375eca5a7
Author: Ackerley Tng <ackerleytng@google.com>

samples/kvm: Implement attach and release provider ops in sample provider

This commit implements the .attach and .release guest_memfd provider
operations in the sample memory provider module and registers them on
mp_file_fops.

> diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
> index 81ac94ae1a4e9..6f86e3e3262e3 100644
> --- a/samples/kvm/memory_provider.c
> +++ b/samples/kvm/memory_provider.c

[ ... ]

> @@ -161,11 +164,45 @@ static long mp_file_ioctl(struct file *file, unsigned int cmd,
>  	}
>  }
>  
> +static void *mp_gmem_attach(struct inode *gmem_inode,
> +			    struct file *resource_file)
> +{
> +	struct mp_info *info = resource_file->private_data;
> +
> +	mutex_lock(&info->lock);
> +	if (info->gmem_inode) {
> +		mutex_unlock(&info->lock);
> +		return ERR_PTR(-EBUSY);
> +	}
> +	info->gmem_inode = gmem_inode;
> +	mutex_unlock(&info->lock);
> +
> +	return get_file(resource_file);
> +}
> +
> +static void mp_gmem_release(void *provider)
> +{
> +	struct file *resource_file = provider;
> +	struct mp_info *info = resource_file->private_data;
> +
> +	mutex_lock(&info->lock);
> +	info->gmem_inode = NULL;
> +	mutex_unlock(&info->lock);
> +
> +	fput(resource_file);
> +}
> +
> +static const struct guest_memfd_provider_operations mp_gmem_provider_ops = {
> +	.attach		= mp_gmem_attach,
> +	.release	= mp_gmem_release,
> +};
> +
>  static const struct file_operations mp_file_fops = {
>  	.owner		= THIS_MODULE,
>  	.release	= mp_file_release,
>  	.unlocked_ioctl	= mp_file_ioctl,
>  	.compat_ioctl	= compat_ptr_ioctl,
> +	.gmem_provider_ops = &mp_gmem_provider_ops,
>  };

[Severity: Low]
Does initializing .gmem_provider_ops on mp_file_fops break the build when
CONFIG_KVM_GUEST_MEMFD is disabled?

struct file_operations only defines gmem_provider_ops under
#ifdef CONFIG_KVM_GUEST_MEMFD, while SAMPLE_KVM_MEMORY_PROVIDER in
samples/Kconfig only depends on CONTIG_ALLOC && X86_64 && m at this point in
the series.  A later patch in the series, commit "samples/kvm: Support
dynamic revocation via kvm_gmem_revoke", adds the KVM_GUEST_MEMFD dependency
to SAMPLE_KVM_MEMORY_PROVIDER; moving that Kconfig dependency here avoids
breaking bisection when CONFIG_KVM_GUEST_MEMFD is not enabled.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=44

Re: [PATCH RFC v3 48/88] samples/kvm: Support dynamic revocation via kvm_gmem_revoke

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] samples/kvm: stale identifier names and patch references in
  commit message
--

commit 42b1d24c6caccfeaf35dd92048463c002a041484
Author: Ackerley Tng <ackerleytng@google.com>

> samples/kvm: Support dynamic revocation via kvm_gmem_revoke
>
> Support dynamic revocation in the sample memory provider module by
> calling kvm_gmem_revoke() when pages are marked absent via
> MPS_SET_PRESENT.
>
> Note down the attaching gmem inode during .attach() and disallow double
> attachment to the same provider file.
>
> Clear the revoked pages after kvm_gmem_revoke() using mps_zero_range()
> to ensure that stale data is wiped and cannot be observed when pages are
> later marked present or reused.
>
> Regarding locking and ordering:
> - In mps_set_present(), mark the absent bitmap first before revoking so
>   any concurrent page fault immediately sees the page as absent and
>   rejects the lookup with -EFAULT.
> - kvm_gmem_revoke() is called under info->lock to fully serialize
>   MPS_SET_PRESENT operations against each other and prevent races
>   between marking pages absent and present.
> - The PFN passed to kvm_gmem_revoke() is derived from info->base_pfn +
>   start. The mapping between page index and physical PFN is static and
>   immutable for the lifetime of the provider file, ensuring that the
>   pfn mapping remains completely stable across the revoke call.
> - Passing pfn directly to kvm_gmem_revoke() allows guest_memfd to
>   reclaim the physical range directly without calling back into the
>   provider. This keeps the lock hierarchy strictly top-down (info->lock
>   -> filemap_invalidate_lock -> mmu_lock) with no callbacks or deadlocks.
>
> For now, nothing needs to be done when setting a page present, since users
> should (re-)fault and (re-)obtain the PFN via .get_pfn(), to be added in a
> later patch.
>
> For users that don't support faulting, like the IOMMU, the counterpart to
> kvm_gmem_revoke() will be made available for the first user to indicate
> that the memory is now available. guest_memfd should then do the page table
> mappings as necessary, based on the shared/private status.
>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>

[Severity: Low]
This isn't a bug, but the commit message has a few stale references from an
earlier version of the series:

- MPS_SET_PRESENT, mps_zero_range(), and mps_set_present() were renamed to
  MP_SET_PRESENT, mp_zero_range(), and mp_set_present().
- Recording info->gmem_inode in .attach() and disallowing double attachment
  was already done in commit "samples/kvm: Implement attach and release
  provider ops in sample provider", not in this patch.
- .get_pfn() was already added earlier in the series in commit "samples/kvm:
  Implement get_pfn provider operation", rather than in a later patch.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=48

Re: [PATCH RFC v3 49/88] samples/kvm: Implement lookup_pfn_range provider operation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] samples/kvm: unexpected -EINVAL from
  mp_gmem_lookup_pfn_range() past npages
--

commit 547b053613c36c4e6a545e75a670dd8b4151e0c0
Author: Ackerley Tng <ackerleytng@google.com>

samples/kvm: Implement lookup_pfn_range provider operation

This commit implements the .lookup_pfn_range callback in the sample
memory provider so that it can act as a PFN-mode provider for
guest_memfd.

> diff --git a/samples/kvm/memory_provider.c b/samples/kvm/memory_provider.c
> index 926f3cf68ed49..7fb7365b65940 100644
> --- a/samples/kvm/memory_provider.c
> +++ b/samples/kvm/memory_provider.c
> @@ -249,10 +249,34 @@ static int mp_gmem_get_pfn(void *provider, pgoff_t index, unsigned long *pfn,
>  	return 0;
>  }
>  
> +static int mp_gmem_lookup_pfn_range(void *provider, pgoff_t index,
> +				     unsigned long *pfn, size_t *nr_pages)
> +{
> +	struct file *resource_file = provider;
> +	struct mp_info *info = resource_file->private_data;
> +
> +	if (index >= info->npages)
> +		return -EINVAL;

[Severity: Medium]
Does returning -EINVAL when index >= info->npages trigger a WARN_ONCE()
in kvm_gmem_pfn_reclaim() and kvm_gmem_make_shared()?

Neither kvm_gmem_attach_resource() nor mp_gmem_attach() rejects attaching
a guest_memfd whose inode size exceeds info->npages << PAGE_SHIFT.  When
closing such a guest_memfd, kvm_gmem_destroy_inode() ->
kvm_gmem_pfn_reclaim() walks up to i_size_read(inode) >> PAGE_SHIFT:

virt/kvm/guest_memfd.c:kvm_gmem_pfn_reclaim() {
	...
		r = gmem_provider_lookup_pfn_range(gi, index, &pfn, &nr_pages);
		if (!r || r == -ENOENT) {
			step = min_t(size_t, nr_pages, end - index);
			if (!r)
				kvm_arch_gmem_reclaim(pfn, step);
		} else {
			WARN_ONCE(1, "unexpected lookup_pfn_range error %d at index 0x%lx\n",
				  r, index);
		}
	...
}

Once index reaches info->npages, returning -EINVAL hits the WARN_ONCE()
and falls back to step = 1, iterating one page at a time across the
remaining unbacked range in an unkillable loop.  Should this return
-ENOENT with *nr_pages set to cover the unbacked tail (or should
mp_gmem_attach() reject oversized guest_memfd inodes)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=49

Re: [PATCH RFC v3 50/88] KVM: selftests: Test guest_memfd conversions with sample memory provider backing

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Low] KVM: selftests: file descriptor leak of gup_test_fd in
  __pin_pages()
- [Low] KVM: selftests: gmem_fd leak in gmem_conversions_do_setup()
- [Low] KVM: selftests: wrong relative include path to memory_provider.h
--

commit 60d8850de9774956f2ff25264d606f044e04bb8c
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Test guest_memfd conversions with sample memory provider backing

This commit adds a fixture variant to guest_memfd_conversions_test to test
guest_memfd conversions backed by the sample memory provider, and adds a
__pin_pages() helper so the elevated_refcount test can check that pinning
PFN-backed memory fails.

> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index d42e201eba763..9793818f54d23 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -674,7 +674,7 @@ int __pin_task_to_cpu(pthread_t task, int cpu)
>  
>  static int gup_test_fd = -1;
>  
> -void pin_pages(void *vaddr, uint64_t size)
> +int __pin_pages(void *vaddr, uint64_t size)
>  {
>  	const struct pin_longterm_test args = {
>  		.addr = (uint64_t)vaddr,
> @@ -685,7 +685,12 @@ void pin_pages(void *vaddr, uint64_t size)
>  	gup_test_fd = __open_path_or_exit("/sys/kernel/debug/gup_test", O_RDWR,
>  					  "Is CONFIG_GUP_TEST enabled?");
>  
> -	TEST_ASSERT_EQ(ioctl(gup_test_fd, PIN_LONGTERM_TEST_START, &args), 0);
> +	return ioctl(gup_test_fd, PIN_LONGTERM_TEST_START, &args);
> +}

[Severity: Low]
Does __pin_pages() leak gup_test_fd when the PIN_LONGTERM_TEST_START ioctl
fails?  In the elevated_refcount multipage test for the mp variant,
__pin_pages() is expected to fail and the caller returns early without
calling unpin_pages(), so each loop iteration overwrites gup_test_fd
without closing the previous descriptor.  Closing and resetting
gup_test_fd via kvm_free_fd(gup_test_fd) when the ioctl returns an error
would avoid the leak.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
> index 023df9cb9d4a6..d0cbb1353d7ee 100644
> --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c
> @@ -16,11 +16,13 @@
>  #include "kselftest_harness.h"
>  #include "test_util.h"
>  #include "ucall_common.h"
> +#include "../../../../samples/kvm/memory_provider.h"

[Severity: Low]
Should this relative path have five "../" components instead of four?
Files under tools/testing/selftests/kvm/x86/ sit five levels below the
repository root (like tools/testing/selftests/kvm/lib/kvm_util.c, which
includes "../../../../../mm/gup_test.h").  With four "../" components, the
preprocessor searches two levels above the kernel tree via
-I$(LINUX_TOOL_INCLUDE) before resolving the header through
-I$(LINUX_TOOL_ARCH_INCLUDE), which breaks file-relative lookup for tools
and risks picking up a file outside the checkout.

[ ... ]

> @@ -75,10 +109,12 @@ static void gmem_conversions_do_setup(test_data_t *t,
>  
>  	vm = __vm_create_shape_with_one_vcpu(shape, &t->vcpu, nr_pages, guest_do_rmw);
>  
> +	t->expect_pin_failure = variant->expect_pin_failure;
>  	t->resource_fd = -1;
>  
>  	if (variant->create_resource) {
> -		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE;
> +		uint64_t flags = gmem_flags | GUEST_MEMFD_FLAG_USE_RESOURCE |
> +				 variant->extra_flags;
>  
>  		t->resource_fd = variant->create_resource(size);
>  		TEST_ASSERT(t->resource_fd >= 0, "Failed to create backing resource");

[Severity: Low]
Does gmem_conversions_do_setup() leak the local gmem_fd created by
__vm_create_guest_memfd_resource() for resource-backed variants?
vm_mem_add() duplicates a caller-supplied gmem_fd via kvm_dup(gmem_fd)
rather than taking ownership of it:

tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c:gmem_conversions_do_setup() {
	if (variant->create_resource) {
		...
		gmem_fd = __vm_create_guest_memfd_resource(vm, size, flags,
							   t->resource_fd);
		TEST_ASSERT(gmem_fd >= 0, "Failed to create guest_memfd with resource");
	}

	vm_mem_add(vm, VM_MEM_SRC_SHMEM, gpa, slot, nr_pages,
		   KVM_MEM_GUEST_MEMFD, gmem_fd, 0, gmem_flags);
}

Because gmem_fd is never closed after vm_mem_add(), the guest_memfd file
and its attached backing resource (including the CMA pages allocated by
the memory_provider module in the mp variant) stay pinned across teardown
in multipage test loops until the process exits.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=50

Re: [PATCH RFC v3 51/88] KVM: guest_memfd: Support reporting PFN cacheability

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] KVM: guest_memfd: gmem_insert_pfn() ignores PFN cacheability
  attributes
--

commit f25760b5d4313e6d13e16717d7b61b6d2b33bbac
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support reporting PFN cacheability

This commit adds cacheability attributes to guest_memfd PFN attributes and
handles them in stage-2 page fault, nested VNCR, SEV-SNP, and guest_memfd
population paths.

> diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
> index fb2279a001ebf..8cdc334d1c9aa 100644
> --- a/include/linux/guest_memfd.h
> +++ b/include/linux/guest_memfd.h
> @@ -2,6 +2,7 @@
>  #ifndef _LINUX_GUEST_MEMFD_H
>  #define _LINUX_GUEST_MEMFD_H
>  
> +#include <linux/bits.h>
>  #include <linux/types.h>
>  
>  struct file;
> @@ -9,7 +10,33 @@ struct folio;
>  struct inode;
>  struct mempolicy;
>  
> -#define GUEST_MEMFD_PFN_ATTRS_READONLY	(1ULL << 0)
> +#define GUEST_MEMFD_PFN_ATTRS_READONLY		(1ULL << 0)
> +
> +/*
> + * Cacheability attributes for physical frames returned by providers
> + * (columns show x86 / arm64 memory types):
> + *
> + * Attribute                               | Host PTE       | Stage-2 PTE
> + * ----------------------------------------+----------------+---------------
> + * GUEST_MEMFD_PFN_ATTRS_CACHE_WB          | WB / Normal    | WB / Normal
> + * GUEST_MEMFD_PFN_ATTRS_CACHE_UC          | UC / Device    | UC / Device
> + * GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC | UC / Device    | UC / Normal-NC
> + * GUEST_MEMFD_PFN_ATTRS_CACHE_WC          | WC / Normal-NC | UC / Normal-NC
> + *
> + * GUEST_MEMFD_PFN_ATTRS_CACHE_WB is 0 so normal cacheable memory is the
> + * default when no cacheability bits are set.
> + */
> +#define GUEST_MEMFD_PFN_ATTRS_CACHE_MASK	GENMASK_ULL(2, 1)
> +#define GUEST_MEMFD_PFN_ATTRS_CACHE_WB		(0ULL << 1)
> +#define GUEST_MEMFD_PFN_ATTRS_CACHE_UC		(1ULL << 1)
> +#define GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC	(2ULL << 1)
> +#define GUEST_MEMFD_PFN_ATTRS_CACHE_WC		(3ULL << 1)

[Severity: High]
Should gmem_insert_pfn() apply GUEST_MEMFD_PFN_ATTRS_CACHE_MASK to the page
protection passed to vmf_insert_pfn_prot()?

When userspace faults on an mmap()ed PFN-mode guest_memfd,
kvm_gmem_fault_user_mapping() passes pfn_attrs to gmem_insert_pfn(), which
only checks GUEST_MEMFD_PFN_ATTRS_READONLY:

virt/kvm/guest_memfd.c:gmem_insert_pfn() {
	...
	if (pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)
		flags &= ~VM_WRITE;

	if ((vmf->flags & FAULT_FLAG_WRITE) && !(flags & VM_WRITE))
		return VM_FAULT_SIGBUS;

	return vmf_insert_pfn_prot(vma, vmf->address, pfn,
				   vm_get_page_prot(flags));
}

Because vm_get_page_prot(flags) returns normal write-back page protections
(and also overwrites any custom vma->vm_page_prot set during mmap), host
userspace PTEs for GUEST_MEMFD_PFN_ATTRS_CACHE_UC,
GUEST_MEMFD_PFN_ATTRS_CACHE_UC_ALLOW_WC, and GUEST_MEMFD_PFN_ATTRS_CACHE_WC
frames are installed as cacheable on arm64 (and on x86 whenever PAT does not
override the protection).  This creates mismatched attributes between host
stage-1 and guest stage-2 mappings and allows speculative cache fills or
write-combining on strict uncached MMIO regions.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=51

Re: [PATCH RFC v3 52/88] KVM: guest_memfd: Support mmap provider operation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] KVM: guest_memfd: gmem_insert_pfn() discards provider mmap
  vm_page_prot
- [Low] KVM: guest_memfd: duplicate @mmap kernel-doc in
  guest_memfd_provider_operations
- [Low] KVM: guest_memfd: missing vm_area_struct forward declaration in
  guest_memfd.h
--

commit 649219a71c7e85644f581e90c9dd003891e92c1c
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support mmap provider operation

This commit adds an optional .mmap callback to struct
guest_memfd_provider_operations and invokes it from kvm_gmem_mmap() so
memory providers can customize VMA setup when userspace maps a
guest_memfd file.

> diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
> index 8cdc334d1c9aa..a2af279cd3c34 100644
> --- a/include/linux/guest_memfd.h
> +++ b/include/linux/guest_memfd.h
> @@ -85,6 +85,9 @@ static inline bool kvm_gmem_pfn_is_cacheable(u64 pfn_attrs)
>   *                 available, the provider should skip the unbacked range. Not
>   *                 called on kvm_gmem_revoke(). Do not take locks held across
>   *                 kvm_gmem_revoke().
> + * @mmap: Optional callback to set up user mappings. Called when userspace calls
> + *        mmap() on guest_memfd. The provider can customize VMA flags, or reject
> + *        the mapping.

[Severity: Low]
Does this add a duplicate @mmap entry to the kernel-doc comment for
struct guest_memfd_provider_operations?

Earlier in the series, commit "KVM: guest_memfd: Export kvm_gmem_revoke
for provider-driven revocation" already added an @mmap description to
this comment block (right after @get_pfn), so scripts/kernel-doc will
warn about a duplicate section name.

>   *
>   * Used by filesystems and device drivers that provide memory for guest_memfd.
>   */
> @@ -99,6 +102,7 @@ struct guest_memfd_provider_operations {
>  	int (*lookup_pfn_range)(void *provider, pgoff_t index,
>  				unsigned long *pfn, size_t *nr_pages);
>  	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
> +	int (*mmap)(void *provider, struct vm_area_struct *vma);
>  };

[Severity: Low]
Should include/linux/guest_memfd.h also forward-declare struct
vm_area_struct alongside struct file, struct folio, struct inode, and
struct mempolicy?

Without a prior file-scope declaration, a translation unit that includes
linux/guest_memfd.h before struct vm_area_struct is declared will scope
the incomplete type to the parameter list of .mmap and trigger a
-Wvisibility warning.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 1fbb719f0e31f..ec908419e5faa 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
[ ... ]
> @@ -869,6 +878,10 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
>  		return -EINVAL;
>  	}
>  
> +	ret = gmem_provider_mmap(gi, vma);
> +	if (ret)
> +		return ret;
> +
>  	if (kvm_gmem_is_pfn_mode(gi))
>  		vm_flags_set(vma, VM_PFNMAP);

[Severity: High]
When a PFN-mode provider customizes vma->vm_page_prot in its .mmap
callback (such as vfio_pci_core_mmap() setting pgprot_noncached() and
pgprot_decrypted() in commit "vfio: Implement mmap provider op for
guest_memfd"), does gmem_insert_pfn() discard those protection bits on
fault?

virt/kvm/guest_memfd.c:gmem_insert_pfn() {
	...
	return vmf_insert_pfn_prot(vma, vmf->address, pfn,
				   vm_get_page_prot(flags));
}

gmem_insert_pfn() recomputes the page protection from scratch via
vm_get_page_prot(flags) instead of preserving the provider's attributes
in vma->vm_page_prot.  On arm64 (which does not select
__HAVE_PFNMAP_TRACKING, making pfnmap_setup_cachemode_pfn() a no-op),
device MMIO BARs will be mapped into host userspace as write-back normal
memory instead of device memory, and on x86 hosts with memory encryption
enabled, vm_get_page_prot(flags) will re-set the encryption bit and drop
pgprot_decrypted().

Deriving the fault protection with pgprot_modify(vma->vm_page_prot,
vm_get_page_prot(flags)) would preserve the provider's vma->vm_page_prot
customizations while still clearing VM_WRITE on read-only PFNs.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=52

Re: [PATCH RFC v3 54/88] KVM: guest_memfd: Support read and write operations and provider callbacks

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] KVM: guest_memfd: corrupt *ppos on copy_to_user() fault in
  kvm_gmem_rw()
- [Low] KVM: guest_memfd: missing kernel-doc for provider read and write
  callbacks
--

commit e75034915fb7c05a994b62162e0f8d89c6caa18a
Author: Ackerley Tng <ackerleytng@google.com>

KVM: guest_memfd: Support read and write operations and provider callbacks

This commit adds read and write file operations for guest_memfd descriptors,
using a stack bounce buffer to forward shared memory reads and writes to
provider callbacks.

> diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h
> index cd634576c9af8..c85594b58112a 100644
> --- a/include/linux/guest_memfd.h
> +++ b/include/linux/guest_memfd.h
> @@ -113,6 +113,10 @@ struct guest_memfd_provider_operations {
>  				unsigned long *pfn, size_t *nr_pages);
>  	void (*put_pfn_range)(void *provider, pgoff_t index, size_t nr_pages);
>  	int (*mmap)(void *provider, struct vm_area_struct *vma);
> +	ssize_t (*read)(void *provider, void *buf, size_t count,
> +			loff_t *ppos);
> +	ssize_t (*write)(void *provider, const void *buf, size_t count,
> +			 loff_t *ppos);
>  	int (*check_set_attributes)(void *provider, pgoff_t index,
>  				    size_t nr_pages, u64 attributes);
>  };

[Severity: Low]
struct guest_memfd_provider_operations has a kernel-doc comment block above
it, so adding read and write without @read and @write descriptions triggers
scripts/kernel-doc warnings.

Could the kernel-doc comment also document that these callbacks run under
the shared filemap invalidate lock (so they must not take locks held across
kvm_gmem_revoke()) and are expected to advance *ppos by the number of bytes
transferred?

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 61a3ff7fcd84c..385b517b49ed5 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c

[ ... ]

> @@ -1204,7 +1239,86 @@ static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl,
>  	}
>  }
>  
> +static ssize_t kvm_gmem_rw(struct file *file, void __user *buf, size_t count,
> +			   loff_t *ppos, bool is_write)
> +{

[ ... ]

> +	while (count) {
> +		u8 bounce[256];
> +		size_t chunk = min_t(size_t, count, sizeof(bounce));
> +		pgoff_t start = *ppos >> PAGE_SHIFT;
> +		pgoff_t end = (*ppos + chunk - 1) >> PAGE_SHIFT;
> +		pgoff_t nr_pages = end - start + 1;
> +		ssize_t ret;
> +
> +		if (is_write) {
> +			if (copy_from_user(bounce, (const char __user *)buf + total_done, chunk))
> +				return total_done ?: -EFAULT;
> +		}
> +
> +		filemap_invalidate_lock_shared(inode->i_mapping);
> +
> +		if (!kvm_gmem_range_has_attributes(inode, start, nr_pages, 0)) {
> +			filemap_invalidate_unlock_shared(inode->i_mapping);
> +			return total_done ?: -EFAULT;
> +		}
> +
> +		if (is_write)
> +			ret = gmem_provider_write(gi, bounce, chunk, ppos);
> +		else
> +			ret = gmem_provider_read(gi, bounce, chunk, ppos);
> +
> +		filemap_invalidate_unlock_shared(inode->i_mapping);
> +
> +		if (ret < 0)
> +			return total_done ?: ret;
> +
> +		if (!ret)
> +			break;
> +
> +		if (!is_write) {
> +			if (copy_to_user((char __user *)buf + total_done, bounce, ret))
> +				return total_done ?: -EFAULT;
> +		}

[Severity: Medium]
Does a copy_to_user() fault here leave *ppos advanced past the unread
chunk?

gmem_provider_read() updates *ppos by ret before copy_to_user() runs.  If a
read spanning multiple 256-byte chunks faults in copy_to_user() after an
earlier chunk succeeded (total_done > 0), kvm_gmem_rw() returns total_done
while *ppos has already been incremented by total_done + ret, so ksys_read()
commits the over-advanced offset to file->f_pos and the next sequential read
silently skips ret bytes.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=54

Re: [PATCH RFC v3 56/88] vfio: Support VFIO_DEVICE_BIND_FLAG_CTRL_ONLY

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] vfio: UAPI comment for VFIO_DEVICE_BIND_FLAG_CTRL_ONLY documents
  wrong errno for BAR read/write
--

commit 25f54ac99c18d430e01125c8ec278111ab821c3f
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Support VFIO_DEVICE_BIND_FLAG_CTRL_ONLY

This commit adds VFIO_DEVICE_BIND_FLAG_CTRL_ONLY to VFIO_DEVICE_BIND_IOMMUFD
to restrict bound device descriptors to control-plane ioctls and non-mappable
region reads and writes.

> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index e41437fa17ad0..ea62d4e72d3bd 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -926,12 +926,19 @@ struct vfio_device_feature {
>   * be provided the same token as VFIO_DEVICE_FEATURE_PCI_VF_TOKEN provided to
>   * the PF.
>   *
> + * VFIO_DEVICE_BIND_FLAG_CTRL_ONLY restricts the bound device fd to
> + * control-plane ioctls and non-mappable region read/write access (such as PCI
> + * configuration space), rejecting mmap(), read/write to mmap-supported BARs,
> + * and all other ioctls or device features with -EOPNOTSUPP. This should be used
> + * to prepare the device fd for attachment to guest_memfd.

[Severity: Low]
Does this comment document the wrong errno for rejected BAR reads and writes?
vfio_df_check_rw_access() rejects non-control-plane read/write accesses with
-EINVAL (as described in the commit message), whereas this comment states
that read/write to mmap-supported BARs is rejected with -EOPNOTSUPP.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=56

Re: [PATCH RFC v3 57/88] vfio: Dispatch guest_memfd provider operations via vfio_device_ops

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] vfio: stray forward declaration before vfio_device_ops breaks
  kernel-doc
--

commit cc851a01b26fbe73cfc6ac69613c43e8df1f4787
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Dispatch guest_memfd provider operations via vfio_device_ops

This commit implements guest_memfd provider operations on vfio_device_fops
and adds gmem_attach and gmem_release callbacks to struct vfio_device_ops.

> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index c81373ddf2a9c..65dc2cb523351 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h

[ ... ]

> @@ -114,7 +117,11 @@ struct vfio_device {
>   *             this device is attached to.
>   * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
>   */
> +struct guest_memfd_provider_operations;
> +
>  struct vfio_device_ops {
> +	int (*gmem_attach)(struct vfio_device *device);
> +	void (*gmem_release)(struct vfio_device *device);
>  	char	*name;
>  	int	(*init)(struct vfio_device *vdev);
>  	void	(*release)(struct vfio_device *vdev);

[Severity: Low]
Does placing struct guest_memfd_provider_operations between the kernel-doc
comment and struct vfio_device_ops break kernel-doc parsing for this
header?

scripts/kernel-doc stops collecting the prototype at the first top-level
semicolon after a kernel-doc comment, so it tries to parse the unused
forward declaration as the struct definition and fails with "error: Cannot
parse struct or union!".  In addition, @gmem_attach and @gmem_release are
not documented in the struct vfio_device_ops kernel-doc comment above.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=57

Re: [PATCH RFC v3 59/88] vfio: Dispatch guest_memfd get_pfn provider op via vfio_device_ops

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] vfio: missing kernel-doc entry for gmem_get_pfn in struct
  vfio_device_ops
--

commit aa1fa472e3013956533385442614d3f6291a86c8
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Dispatch guest_memfd get_pfn provider op via vfio_device_ops

This commit adds a gmem_get_pfn callback to struct vfio_device_ops and
wires up vfio_device_gmem_provider_ops.get_pfn to dispatch to it.

> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index 65dc2cb523351..51e04b980a381 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -122,6 +122,9 @@ struct guest_memfd_provider_operations;
>  struct vfio_device_ops {
>  	int (*gmem_attach)(struct vfio_device *device);
>  	void (*gmem_release)(struct vfio_device *device);
> +	int (*gmem_get_pfn)(struct vfio_device *device, pgoff_t index,
> +			    unsigned long *pfn, int *max_order,
> +			    u64 *pfn_attrs);
>  	char	*name;
>  	int	(*init)(struct vfio_device *vdev);
>  	void	(*release)(struct vfio_device *vdev);

[Severity: Low]
struct vfio_device_ops has a kernel-doc comment block right above it, but
@gmem_get_pfn (along with @gmem_attach and @gmem_release from commit "vfio:
Dispatch guest_memfd provider operations via vfio_device_ops") is not
documented there, and the struct guest_memfd_provider_operations forward
declaration placed between the comment and struct vfio_device_ops breaks
scripts/kernel-doc parsing for the struct.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=59

Re: [PATCH RFC v3 60/88] vfio/pci: Implement get_pfn provider op for guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] vfio/pci: sub-page mappable BARs rejected in
  vfio_pci_core_gmem_get_pfn()
- [Medium] vfio/pci: missing vfio_pci_core_get_iomap() check in
  vfio_pci_core_gmem_get_pfn()
--

commit 1661618156a57f40eb0f4fe51d0d5dccb4378f2f
Author: Ackerley Tng <ackerleytng@google.com>

vfio/pci: Implement get_pfn provider op for guest_memfd

This commit implements the .gmem_get_pfn callback for vfio-pci to translate
guest_memfd page offsets into physical PCI BAR frame numbers.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index c7b67fbce4cae..9551a067b5689 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -2731,6 +2731,44 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev)
>  	up_write(&vdev->memory_lock);
>  }
>  EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_release);
> +
> +int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
> +			       unsigned long *pfn, int *max_order,
> +			       u64 *pfn_attrs)
> +{
> +	struct vfio_pci_core_device *vdev =
> +		container_of(core_vdev, struct vfio_pci_core_device, vdev);
> +	struct pci_dev *pdev = vdev->pdev;
> +	unsigned int bar;
> +	u64 pgoff;
> +
> +	/*
> +	 * Lockless check: if memory decoding is disabled or the device is in
> +	 * low-power D3, fail immediately without blocking on memory_lock.
> +	 */
> +	if (!smp_load_acquire(&vdev->bars_available))
> +		return -EFAULT;
> +
> +	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
> +	pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
> +
> +	if (bar >= PCI_STD_NUM_BARS || !vdev->bar_mmap_supported[bar])
> +		return -EINVAL;

[Severity: Medium]
Does vfio_pci_core_gmem_get_pfn() also need to check
vfio_pci_core_get_iomap(vdev, bar) before returning the BAR PFN?

vfio_pci_probe_mmaps() sets vdev->bar_mmap_supported[bar] based on resource
flags and alignment alone, while vfio_pci_core_map_bars() allows
pci_request_selected_regions() and pci_iomap() to fail softly by recording an
ERR_PTR in vdev->barmap[bar]:

drivers/vfio/pci/vfio_pci_core.c:vfio_pci_core_map_bars() {
	...
		if (pci_request_selected_regions(pdev, 1 << bar, "vfio")) {
			pci_dbg(pdev, "Failed to reserve region %d\n", bar);
			vdev->barmap[bar] = IOMEM_ERR_PTR(-EBUSY);
			continue;
		}
	...
}

Unlike vfio_pci_core_mmap() and vfio_pci_core_feature_dma_buf(), which verify
!IS_ERR(vfio_pci_core_get_iomap()), skipping that check here lets guest_memfd
map physical BAR frames that VFIO failed to reserve (for example, when owned
by another host driver such as efifb).

> +
> +	if (pgoff >= (pci_resource_len(pdev, bar) >> PAGE_SHIFT))
> +		return -EINVAL;

[Severity: Medium]
Does shifting pci_resource_len(pdev, bar) without PAGE_ALIGN() reject
page-aligned sub-page BARs?

When a BAR is smaller than PAGE_SIZE (for example, a 4 KiB BAR on a 64 KiB
PAGE_SIZE kernel) and page-aligned, vfio_pci_probe_mmaps() reserves the rest
of the host page with a dummy resource and sets
vdev->bar_mmap_supported[bar] = true.  Unlike vfio_pci_core_mmap(), which
computes PAGE_ALIGN(pci_resource_len(pdev, index)),
pci_resource_len(pdev, bar) >> PAGE_SHIFT evaluates to 0 for these BARs and
causes even pgoff == 0 to fail with -EINVAL.  Because bar_mmap_supported[bar]
is true, vfio_pci_core_is_ctrl_plane() also returns false and blocks
read()/write() on a VFIO_DEVICE_BIND_FLAG_CTRL_ONLY descriptor, leaving the
BAR inaccessible.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=60

Re: [PATCH RFC v3 61/88] vfio: Implement mmap provider op for guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] vfio: uncached and decrypted vma->vm_page_prot lost in
  guest_memfd fault
- [Medium] vfio: vfio_device_fops_mmap() returns -EOPNOTSUPP instead of
  -EINVAL
--

commit 6c78c8834b1a41506b88de16d4e56626d4ac8609
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Implement mmap provider op for guest_memfd

This commit extracts a vfio_device_mmap() helper from
vfio_device_fops_mmap() and wires up the .mmap provider callback in
vfio_device_gmem_provider_ops to forward guest_memfd mappings to the bus
driver's .mmap handler.

> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
> index 93bd0a200ad22..979ed69c5ebf7 100644
> --- a/drivers/vfio/vfio_main.c
> +++ b/drivers/vfio/vfio_main.c
> @@ -1503,20 +1503,25 @@ static ssize_t vfio_device_fops_write(struct file *filep,
>  	return device->ops->write(device, buf, count, ppos);
>  }
>  
> +static int vfio_device_mmap(struct vfio_device *device,
> +			    struct vm_area_struct *vma)
> +{
> +	if (unlikely(!device->ops->mmap))
> +		return -EOPNOTSUPP;
> +
> +	return device->ops->mmap(device, vma);
> +}
> +
>  static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
>  {
>  	struct vfio_device_file *df = filep->private_data;
> -	struct vfio_device *device = df->device;
>  	int ret;
>  
>  	ret = vfio_df_check_full_access(df);
>  	if (ret)
>  		return ret;
>  
> -	if (unlikely(!device->ops->mmap))
> -		return -EINVAL;
> -
> -	return device->ops->mmap(device, vma);
> +	return vfio_device_mmap(df->device, vma);
>  }

[Severity: Medium]
Does extracting vfio_device_mmap() here cause a regression where
vfio_device_fops_mmap() returns -EOPNOTSUPP instead of -EINVAL when a bus
driver does not implement device->ops->mmap?

Direct mmap(2) calls on full-access VFIO device fds for drivers without an
.mmap callback (such as vfio-ccw or vfio-ap) previously returned -EINVAL,
matching vfio_device_fops_read(), vfio_device_fops_write(), and
vfio_device_fops_unl_ioctl().  Returning -EOPNOTSUPP changes the user-visible
errno and makes a missing driver .mmap callback indistinguishable from a
file descriptor bound with VFIO_DEVICE_BIND_FLAG_CTRL_ONLY.

> @@ -1604,10 +1609,23 @@ static int vfio_device_gmem_get_pfn(void *provider, pgoff_t index,
>  					 pfn_attrs);
>  }
>  
> +static int vfio_device_gmem_mmap(void *provider, struct vm_area_struct *vma)
> +{
> +	struct file *file = provider;
> +	struct vfio_device_file *df = file->private_data;
> +
> +	/*
> +	 * No control- vs. data-plane filtering is required because
> +	 * vfio_device_mmap() already only allows mmap-ing the data plane.
> +	 */
> +	return vfio_device_mmap(df->device, vma);
> +}
> +
>  static const struct guest_memfd_provider_operations vfio_device_gmem_provider_ops = {
>  	.attach  = vfio_device_gmem_attach,
>  	.release = vfio_device_gmem_release,
>  	.get_pfn = vfio_device_gmem_get_pfn,
> +	.mmap    = vfio_device_gmem_mmap,
>  };
>  #endif

[Severity: High]
When kvm_gmem_mmap() overrides vma->vm_ops with kvm_gmem_pfn_vm_ops after
vfio_device_gmem_mmap() returns, does the guest_memfd fault handler discard
the uncached and decrypted protections that vfio_pci_core_mmap() sets on
vma->vm_page_prot?

vfio_pci_core_mmap() configures vma->vm_page_prot with pgprot_noncached()
and pgprot_decrypted().  However, on a user fault,
kvm_gmem_fault_user_mapping() -> gmem_insert_pfn() recomputes page
protections from vma->vm_flags instead of using vma->vm_page_prot:

virt/kvm/guest_memfd.c:gmem_insert_pfn() {
    ...
	return vmf_insert_pfn_prot(vma, vmf->address, pfn,
				   vm_get_page_prot(flags));
}

Because vm_get_page_prot(flags) derives a fresh pgprot_t from
protection_map[], both pgprot_noncached() and pgprot_decrypted() are lost.
On architectures without PAT PFNMAP tracking in pfnmap_setup_cachemode_pfn()
(such as arm64), PCI BAR MMIO pages are mapped into host userspace as
write-back cacheable normal memory rather than Device memory and without
PROT_NS_SHARED, which can corrupt MMIO accesses or trigger synchronous
external aborts and SError.  On x86 systems with SME or SEV active,
vm_get_page_prot() sets _PAGE_ENC, causing host userspace MMIO accesses to
unencrypted device BARs to be performed with the encryption bit set.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=61

Re: [PATCH RFC v3 63/88] vfio/pci: Implement check_set_attributes provider op for guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] vfio/pci: missing BAR checks in
  vfio_pci_core_gmem_check_set_attributes()
- [High] vfio/pci: unguarded linux/kvm_host.h include breaks non-KVM
  arch builds
--

commit 2255a7dd0042c189f9db1a91b5da96def65f3e26
Author: Ackerley Tng <ackerleytng@google.com>

vfio/pci: Implement check_set_attributes provider op for guest_memfd

This commit implements the .gmem_check_set_attributes callback in vfio-pci
to reject private attribute conversions on ranges overlapping the MSI-X
vector table.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 9551a067b5689..894b1332f8a0c 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -16,6 +16,7 @@
>  #include <linux/eventfd.h>
>  #include <linux/file.h>
>  #include <linux/guest_memfd.h>
> +#include <linux/kvm_host.h>
>  #include <linux/interrupt.h>
>  #include <linux/iommu.h>
>  #include <linux/module.h>

[Severity: High]
Does including <linux/kvm_host.h> unconditionally break builds with
CONFIG_VFIO_PCI_CORE on architectures without KVM support (such as 32-bit
arm, sparc, parisc, or m68k)?

include/linux/kvm_host.h pulls in <asm/kvm.h> and <asm/kvm_host.h>, which
only exist on KVM-capable architectures; drivers/vfio/vfio_main.c guards
this include with #if IS_ENABLED(CONFIG_KVM).  (A later patch in this
series, commit "QUESTION: KVM: guest_memfd: Disallow private mappings of
shared-only PFNs", removes this include, but intermediate commits fail to
build on non-KVM architectures.)

> @@ -2769,6 +2770,39 @@ int vfio_pci_core_gmem_get_pfn(struct vfio_device *core_vdev, pgoff_t index,
>  	return 0;
>  }
>  EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_get_pfn);
> +
> +int vfio_pci_core_gmem_check_set_attributes(struct vfio_device *core_vdev,
> +					    pgoff_t index, size_t nr_pages,
> +					    u64 attributes)
> +{
> +	struct vfio_pci_core_device *vdev =
> +		container_of(core_vdev, struct vfio_pci_core_device, vdev);
> +	pgoff_t bar_pgoff;
> +	size_t size;
> +	loff_t pos;
> +	int bar;
> +
> +	if (!(attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE))
> +		return 0;
> +
> +	/*
> +	 * Prevent private conversion of pages containing the MSI-X vector
> +	 * table. The host kernel directly programs MSI-X table entries to
> +	 * configure and mask interrupts; converting these pages to private
> +	 * would cause host kernel writes to trigger fatal hardware protection
> +	 * faults (e.g. RMP or GPT violations) on confidential compute platforms.
> +	 */
> +	bar = index >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
> +	bar_pgoff = index & ((1ULL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
> +	pos = (loff_t)bar_pgoff << PAGE_SHIFT;
> +	size = nr_pages << PAGE_SHIFT;
> +
> +	if (vfio_pci_range_overlaps_msix(vdev, bar, pos, size))
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_check_set_attributes);
>  #endif

[Severity: High]
Does vfio_pci_core_gmem_check_set_attributes() miss checking that the BAR
is valid and supports mmap, and that [pos, pos + size) fits within
pci_resource_len(vdev->pdev, bar)?

The commit message states that this function verifies that the targeted BAR
is a standard memory BAR and that the range does not exceed the BAR's
physical resource size, as vfio_pci_core_gmem_get_pfn() does:

drivers/vfio/pci/vfio_pci_core.c:vfio_pci_core_gmem_get_pfn() {
    ...
	if (bar >= PCI_STD_NUM_BARS || !vdev->bar_mmap_supported[bar])
		return -EINVAL;

	if (pgoff >= (pci_resource_len(pdev, bar) >> PAGE_SHIFT))
		return -EINVAL;
    ...
}

Without those checks, non-mmappable regions and offsets past the end of a
BAR can be marked private.  Furthermore, because kvm_gmem_set_attributes()
only bounds the request against the inode size spanning the 40-bit region
offset space and bar is decoded solely from the starting index, a
KVM_SET_MEMORY_ATTRIBUTES2 range starting in an earlier BAR (bar <
vdev->msix_bar) and spanning across the 40-bit boundary into vdev->msix_bar
has bar != vdev->msix_bar in vfio_pci_range_overlaps_msix(), bypassing the
MSI-X overlap check and allowing the MSI-X table pages to be marked private
and later faulted in one page at a time via vfio_pci_core_gmem_get_pfn().
(Commit "QUESTION: KVM: guest_memfd: Disallow private mappings of
shared-only PFNs" later in the series replaces this callback, but these
checks are needed if the check_set_attributes approach is kept or when
bisecting.)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=63

Re: [PATCH RFC v3 64/88] vfio: Add vfio_device_gmem_revoke helper for guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] vfio: link failure in vfio_device_gmem_revoke() with
  CONFIG_VFIO=y, KVM=m
- [High] vfio: unsynchronized gmem_inode access in
  vfio_device_gmem_revoke()
- [Low] vfio: return 0 in !CONFIG_KVM_GUEST_MEMFD
  vfio_device_gmem_revoke() stub
--

commit 64f7b4c8e914dff002926c09d51e57c7fda9c500
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Add vfio_device_gmem_revoke helper for guest_memfd

This commit adds vfio_device_gmem_revoke() so VFIO bus drivers can revoke
and reclaim device memory ranges from an attached guest_memfd instance.

> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
> index e4e7bfe77d03b..fd5f71dfc44b7 100644
> --- a/drivers/vfio/vfio_main.c
> +++ b/drivers/vfio/vfio_main.c
> @@ -1643,6 +1643,16 @@ static const struct guest_memfd_provider_operations vfio_device_gmem_provider_op
>  	.mmap                 = vfio_device_gmem_mmap,
>  	.check_set_attributes = vfio_device_gmem_check_set_attributes,
>  };
> +
> +int vfio_device_gmem_revoke(struct vfio_device *device, pgoff_t start,
> +			    size_t nr_pages, const unsigned long *pfn)
> +{
> +	if (!device->gmem_inode)
> +		return 0;
> +
> +	return kvm_gmem_revoke(device->gmem_inode, start, nr_pages, pfn);
> +}
> +EXPORT_SYMBOL_GPL(vfio_device_gmem_revoke);
>  #endif

[Severity: High]
Does calling kvm_gmem_revoke() from drivers/vfio/vfio_main.c break linking
vmlinux when CONFIG_VFIO=y and CONFIG_KVM=m?

CONFIG_KVM_GUEST_MEMFD is a bool selected by CONFIG_KVM_X86 on x86_64 even
when CONFIG_KVM=m, which links virt/kvm/guest_memfd.o into kvm.ko while
vfio_main.o is built into vmlinux.  With CONFIG_VFIO=m and CONFIG_KVM=m, the
direct symbol reference also forces vfio.ko to depend on kvm.ko even when
VFIO is used without KVM.

[Severity: High]
Can vfio_device_gmem_revoke() race with vfio_device_gmem_release() and
cause a use-after-free or NULL pointer dereference on device->gmem_inode?

device->gmem_inode is set and cleared under device->dev_set->lock, outside
the bus driver's lock (such as vdev->memory_lock), while bus driver
revocation paths call vfio_device_gmem_revoke() without holding
device->dev_set->lock or pinning the inode:

CPU0                                  CPU1
kvm_gmem_destroy_inode()
  vfio_device_gmem_release()
    mutex_lock(&dev_set->lock)
    ops->gmem_release(device)
      down_write(&vdev->memory_lock)
      ...
      up_write(&vdev->memory_lock)
                                      vfio_pci_zap_bars()
                                        vfio_device_gmem_revoke()
                                          if (!device->gmem_inode)
    device->gmem_inode = NULL;
    mutex_unlock(&dev_set->lock)
  ...
  kvm_gmem_free_inode()
                                          kvm_gmem_revoke(device->gmem_inode,
                                                          ...)
                                            <-- use-after-free / NULL deref

Because device->gmem_inode is read twice without READ_ONCE(), clearing it
on CPU0 can also cause CPU1 to reload NULL as the inode argument to
kvm_gmem_revoke().  Similarly, in vfio_device_gmem_attach(),
ops->gmem_attach() finishes before device->gmem_inode is assigned, leaving
a window where a concurrent vfio_device_gmem_revoke() sees NULL and skips
revocation.

> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index ffb01e655735f..aade091a91fe8 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -394,6 +394,18 @@ bool vfio_file_is_valid(struct file *file);
>  bool vfio_file_enforced_coherent(struct file *file);
>  void vfio_file_set_kvm(struct file *vfio_file, struct file *kvm);
>  
> +#ifdef CONFIG_KVM_GUEST_MEMFD
> +int vfio_device_gmem_revoke(struct vfio_device *device, pgoff_t start,
> +			    size_t nr_pages, const unsigned long *pfn);
> +#else
> +static inline int vfio_device_gmem_revoke(struct vfio_device *device,
> +					  pgoff_t start, size_t nr_pages,
> +					  const unsigned long *pfn)
> +{
> +	return -EOPNOTSUPP;
> +}
> +#endif

[Severity: Low]
Should the !CONFIG_KVM_GUEST_MEMFD stub for vfio_device_gmem_revoke()
return 0 instead of -EOPNOTSUPP?

When CONFIG_KVM_GUEST_MEMFD is enabled and no guest_memfd is attached to
the device (!device->gmem_inode), vfio_device_gmem_revoke() is a no-op and
returns 0.  Returning -EOPNOTSUPP when CONFIG_KVM_GUEST_MEMFD is disabled
makes callers that check the return value fail even though no guest_memfd
can be attached.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=64

Re: [PATCH RFC v3 65/88] vfio/pci: Revoke guest_memfd memory on BAR invalidation and reset

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] vfio/pci: BARs marked available while pm_runtime_engaged is
  true
- [High] vfio/pci: UAF race on gmem_inode between vfio_pci_zap_bars()
  and release
- [Medium] vfio/pci: vfio_pci_core_runtime_resume() fails to restore
  bars_available
--

commit d6c318d12802531d9e53ee3087896bdf7d1f3349
Author: Ackerley Tng <ackerleytng@google.com>

vfio/pci: Revoke guest_memfd memory on BAR invalidation and reset

This commit revokes guest_memfd mappings in vfio_pci_zap_bars() when BAR
access is invalidated and marks BARs available again after memory decoding
and D0 power state are restored.

> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
> index 9914f3ac69aef..fae3980fb8d9c 100644
> --- a/drivers/vfio/pci/vfio_pci_config.c
> +++ b/drivers/vfio/pci/vfio_pci_config.c
> @@ -630,8 +630,10 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
>  		*virt_cmd &= cpu_to_le16(~mask);
>  		*virt_cmd |= cpu_to_le16(new_cmd & mask);
>  
> -		if (__vfio_pci_memory_enabled(vdev))
> +		if (__vfio_pci_memory_enabled(vdev)) {
>  			vfio_pci_dma_buf_move(vdev, false);
> +			vfio_pci_set_bars_available(vdev, true);
> +		}
>  		up_write(&vdev->memory_lock);
>  	}

[Severity: High]
Can this set vdev->bars_available to true while vdev->pm_runtime_engaged is
still set?

When userspace enters low-power mode via VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY,
vfio_pci_runtime_pm_entry() zaps BARs and sets vdev->pm_runtime_engaged =
true.  A later config write or reset ioctl wraps the access in
pm_runtime_resume_and_get() and pm_runtime_put(), so pdev->current_state is
temporarily PCI_D0 while vdev->pm_runtime_engaged remains true.  Because
__vfio_pci_memory_enabled() does not check vdev->pm_runtime_engaged, this
sets vdev->bars_available = true, and when pm_runtime_put() suspends the
device back into D3hot/D3cold without calling vfio_pci_zap_bars(),
vfio_pci_core_gmem_get_pfn() will map BAR MMIO pages of a powered-down
device.

The same applies to vfio_lock_and_set_power_state(), vfio_exp_config_write(),
vfio_af_config_write(), vfio_pci_ioctl_reset(), and
vfio_pci_dev_set_hot_reset(); unlike vfio_pci_core_gmem_attach() and
vfio_pci_vmf_insert_pfn(), none of them check !vdev->pm_runtime_engaged.

[ ... ]

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 894b1332f8a0c..2b29d0ec6a0c9 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -464,8 +464,10 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev)
>  	 */
>  	down_write(&vdev->memory_lock);
>  	__vfio_pci_runtime_pm_exit(vdev);
> -	if (__vfio_pci_memory_enabled(vdev))
> +	if (__vfio_pci_memory_enabled(vdev)) {
>  		vfio_pci_dma_buf_move(vdev, false);
> +		vfio_pci_set_bars_available(vdev, true);
> +	}
>  	up_write(&vdev->memory_lock);
>  }

[Severity: Medium]
Does waking from VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY_WITH_WAKEUP leave
vdev->bars_available set to false?

When the device resumes with a wakeup eventfd configured,
vfio_pci_core_runtime_resume() exits low-power mode by calling
__vfio_pci_runtime_pm_exit() directly rather than vfio_pci_runtime_pm_exit():

drivers/vfio/pci/vfio_pci_core.c:vfio_pci_core_runtime_resume() {
	down_write(&vdev->memory_lock);
	if (vdev->pm_wake_eventfd_ctx) {
		eventfd_signal(vdev->pm_wake_eventfd_ctx);
		__vfio_pci_runtime_pm_exit(vdev);
	}
	up_write(&vdev->memory_lock);
}

Because vfio_pci_set_bars_available(vdev, true) is only in
vfio_pci_runtime_pm_exit(), subsequent guest_memfd BAR faults in
vfio_pci_core_gmem_get_pfn() will keep failing with -EFAULT after wakeup.

[ ... ]

> @@ -1727,8 +1731,31 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev)
>  	loff_t start = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_BAR0_REGION_INDEX);
>  	loff_t end = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_ROM_REGION_INDEX);
>  	loff_t len = end - start;
> +	int i;
>  
>  	unmap_mapping_range(core_vdev->inode->i_mapping, start, len, true);
> +
> +#ifdef CONFIG_KVM_GUEST_MEMFD
> +	if (vfio_pci_get_bars_available(vdev)) {
> +		vfio_pci_set_bars_available(vdev, false);
> +
> +		for (i = 0; i < PCI_STD_NUM_BARS; i++) {
> +			pgoff_t gmem_start;
> +			unsigned long pfn;
> +			size_t nr_pages;
> +
> +			if (!vdev->bar_mmap_supported[i])
> +				continue;
> +
> +			gmem_start = VFIO_PCI_INDEX_TO_OFFSET(i) >> PAGE_SHIFT;
> +			nr_pages = pci_resource_len(vdev->pdev, i) >> PAGE_SHIFT;
> +			pfn = pci_resource_start(vdev->pdev, i) >> PAGE_SHIFT;
> +
> +			vfio_device_gmem_revoke(&vdev->vdev, gmem_start,
> +						nr_pages, &pfn);
> +		}
> +	}
> +#endif
>  }

[ ... ]

> @@ -2726,8 +2753,7 @@ void vfio_pci_core_gmem_release(struct vfio_device *core_vdev)
>  
>  	down_write(&vdev->memory_lock);
>  
> -	/* Pairs with smp_load_acquire() in provider operations */
> -	smp_store_release(&vdev->bars_available, false);
> +	vfio_pci_set_bars_available(vdev, false);
>  
>  	up_write(&vdev->memory_lock);
>  }

[Severity: High]
Can vfio_pci_zap_bars() race with guest_memfd release and cause a
use-after-free on device->gmem_inode?

CPU0                                  CPU1
kvm_gmem_destroy_inode(inode)
  ...
  vfio_device_gmem_release()
    mutex_lock(&dev_set->lock)
    vfio_pci_core_gmem_release()
      down_write(&vdev->memory_lock)
      vfio_pci_set_bars_available(false)
      up_write(&vdev->memory_lock)
                                      vfio_pci_ioctl_reset()
                                        down_write(&vdev->memory_lock)
                                        ...
                                        vfio_pci_set_bars_available(true)
                                        up_write(&vdev->memory_lock)
                                      vfio_pci_ioctl_reset()
                                        down_write(&vdev->memory_lock)
                                        vfio_pci_zap_bars()
                                          vfio_device_gmem_revoke()
                                            loads device->gmem_inode
    device->gmem_inode = NULL
    mutex_unlock(&dev_set->lock)
  mt_destroy(&gi->attributes)
kvm_gmem_free_inode(inode)
                                            kvm_gmem_revoke(gmem_inode, ...)
                                              <-- use-after-free

vfio_device_gmem_release() clears device->gmem_inode under dev_set->lock
after vfio_pci_core_gmem_release() has already dropped vdev->memory_lock,
and even before vfio_device_gmem_release() runs, kvm_gmem_destroy_inode()
begins tearing down the evicting inode while vdev->bars_available is still
true.  Because vfio_pci_zap_bars() and vfio_pci_set_bars_available() only
take vdev->memory_lock (and set vdev->bars_available = true unconditionally
even when no guest_memfd is attached), nothing prevents
vfio_device_gmem_revoke() from dereferencing device->gmem_inode while the
inode is being destroyed and freed.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=65

Re: [PATCH RFC v3 68/88] vfio/pci: Use iov_iter for MMIO read and write

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] vfio/pci: missing linux/uio.h include in vfio_pci_rdwr.c breaks
  build
--

commit 857abb2cc1a3ab7c6f488750b101680f17be6f5d
Author: Ackerley Tng <ackerleytng@google.com>

vfio/pci: Use iov_iter for MMIO read and write

This commit refactors the internal MMIO read and write helpers in
vfio_pci_rdwr.c to operate on struct iov_iter, while keeping
vfio_pci_core_do_io_rw() unchanged by wrapping user buffers with
import_ubuf().

> diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
> index e5f1f60513709..cc64722d1ab9a 100644
> --- a/drivers/vfio/pci/vfio_pci_rdwr.c
> +++ b/drivers/vfio/pci/vfio_pci_rdwr.c
> @@ -91,34 +91,34 @@ VFIO_IOREAD(32)
>  VFIO_IOREAD(64)
>  
>  #define VFIO_IORDWR(size)						\
> -static int vfio_pci_iordwr##size(struct vfio_pci_core_device *vdev,\
> -				bool iswrite, bool test_mem,		\
> -				void __iomem *io, char __user *buf,	\
> -				loff_t off, size_t *filled)		\
> -{									\
> -	u##size val;							\
> -	int ret;							\
> -									\
> -	if (iswrite) {							\
> -		if (copy_from_user(&val, buf, sizeof(val)))		\
> -			return -EFAULT;					\
> -									\
> -		ret = vfio_pci_core_iowrite##size(vdev, test_mem,	\
> -						  val, io + off);	\
> -		if (ret)						\
> -			return ret;					\
> -	} else {							\
> -		ret = vfio_pci_core_ioread##size(vdev, test_mem,	\
> -						 &val, io + off);	\
> -		if (ret)						\
> -			return ret;					\
> -									\
> -		if (copy_to_user(buf, &val, sizeof(val)))		\
> -			return -EFAULT;					\
> -	}								\
> -									\
> -	*filled = sizeof(val);						\
> -	return 0;							\
> +static int vfio_pci_iordwr##size(struct vfio_pci_core_device *vdev,		\
> +				bool iswrite, bool test_mem,			\
> +				void __iomem *io, struct iov_iter *iter,	\
> +				loff_t off, size_t *filled)			\
> +{										\
> +	u##size val;								\
> +	int ret;								\
> +										\
> +	if (iswrite) {								\
> +		if (!copy_from_iter_full(&val, sizeof(val), iter))		\
> +			return -EFAULT;						\
> +										\
> +		ret = vfio_pci_core_iowrite##size(vdev, test_mem,		\
> +						  val, io + off);		\
> +		if (ret)							\
> +			return ret;						\
> +	} else {								\
> +		ret = vfio_pci_core_ioread##size(vdev, test_mem,		\
> +						 &val, io + off);		\
> +		if (ret)							\
> +			return ret;						\
> +										\
> +		if (!copy_to_iter_full(&val, sizeof(val), iter))		\
> +			return -EFAULT;						\
> +	}									\
> +										\
> +	*filled = sizeof(val);							\
> +	return 0;								\
>  }									\

[ ... ]

> +ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
> +			       void __iomem *io, char __user *buf,
> +			       loff_t off, size_t count, size_t x_start,
> +			       size_t x_end, bool iswrite,
> +			       enum vfio_pci_io_width max_width)
> +{
> +	struct iov_iter iter;
> +	int ret;
> +
> +	ret = import_ubuf(iswrite ? ITER_SOURCE : ITER_DEST, buf, count, &iter);
> +	if (ret)
> +		return ret;
> +
> +	return __vfio_pci_core_do_io_rw(vdev, test_mem, io, &iter, off, count,
> +					x_start, x_end, iswrite, max_width);
> +}

[Severity: High]
Does drivers/vfio/pci/vfio_pci_rdwr.c need to include <linux/uio.h>?

struct iov_iter, ITER_SOURCE, ITER_DEST, import_ubuf(), iov_iter_advance(),
copy_from_iter_full(), and copy_to_iter_full() are defined in linux/uio.h,
while linux/fs.h only forward-declares struct iov_iter. On x86_64 with
CONFIG_IA32_EMULATION=y, linux/uio.h happens to be pulled in transitively
via asm/ia32.h -> linux/compat.h, but builds on architectures such as arm64
(or x86_64 with CONFIG_IA32_EMULATION=n) fail with an incomplete type for
struct iov_iter and implicit function declarations.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=68

Re: [PATCH RFC v3 69/88] vfio: Dispatch guest_memfd read and write provider ops via vfio_device_ops

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] vfio: missing kernel-doc for gmem_read and gmem_write in
  vfio_device_ops
--

commit 1e0b7bdc6352dbf69f404e2847472ddba12fa151
Author: Ackerley Tng <ackerleytng@google.com>

vfio: Dispatch guest_memfd read and write provider ops via vfio_device_ops

This commit implements the .read and .write callbacks on
vfio_device_gmem_provider_ops to dispatch data-plane reads and writes to
the bus driver's .gmem_read and .gmem_write callbacks on struct
vfio_device_ops.

> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index 8022adedc26a6..012a233a375ab 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -130,6 +130,10 @@ struct vfio_device_ops {
>  					 u64 attributes);
>  	int (*gmem_lookup_pfn_range)(struct vfio_device *device, pgoff_t index,
>  				     unsigned long *pfn, size_t *nr_pages);
> +	ssize_t (*gmem_read)(struct vfio_device *device, void *buf,
> +			     size_t count, loff_t *ppos);
> +	ssize_t (*gmem_write)(struct vfio_device *device, const void *buf,
> +			      size_t count, loff_t *ppos);
>  	char	*name;
>  	int	(*init)(struct vfio_device *vdev);
>  	void	(*release)(struct vfio_device *vdev);

[Severity: Low]
Should @gmem_read and @gmem_write be documented in the kernel-doc comment
above struct vfio_device_ops?  Without descriptions for these new members,
scripts/kernel-doc warns about undocumented struct members under W=1
builds.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=69

Re: [PATCH RFC v3 71/88] vfio/pci: Implement read and write provider ops for guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] vfio/pci: unserialized sub-page BAR access in
  vfio_pci_core_gmem_rw()
- [High] vfio/pci: D3 MMIO access in vfio_pci_core_gmem_rw() when
  runtime PM engaged
--

commit a60cfc9fb84848a43b5d6f8b77f7f250457c4495
Author: Ackerley Tng <ackerleytng@google.com>

vfio/pci: Implement read and write provider ops for guest_memfd

This commit implements the .gmem_read and .gmem_write provider operations
in vfio-pci to service guest_memfd read and write accesses to mappable
MMIO BARs.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index a9274d95e9066..c976574782663 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -2871,6 +2871,42 @@ int vfio_pci_core_gmem_lookup_pfn_range(struct vfio_device *core_vdev,
>  	return -ENOENT;
>  }
>  EXPORT_SYMBOL_GPL(vfio_pci_core_gmem_lookup_pfn_range);
> +
> +static ssize_t vfio_pci_core_gmem_rw(struct vfio_device *core_vdev,
> +				     void *kbuf, size_t count,
> +				     loff_t *ppos, bool iswrite)
> +{
> +	struct vfio_pci_core_device *vdev =
> +		container_of(core_vdev, struct vfio_pci_core_device, vdev);
> +	unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
> +
> +	if (!count)
> +		return 0;
> +
> +	if (index >= PCI_STD_NUM_BARS || !vdev->bar_mmap_supported[index])
> +		return -EINVAL;

[Severity: High]
Can a sub-page BAR access here race with a reset or power transition?

In vfio_pci_probe_mmaps(), a page-aligned BAR smaller than PAGE_SIZE has
 its remainder reserved via dummy_res and vdev->bar_mmap_supported[bar] set
to true.  Unlike vfio_pci_core_gmem_get_pfn() and
vfio_pci_core_gmem_lookup_pfn_range(), which reject BARs where
pci_resource_len(pdev, bar) >> PAGE_SHIFT is 0, vfio_pci_core_gmem_rw()
allows them and calls vfio_pci_bar_rw_kbuf() with test_mem = false.

Skipping vdev->memory_lock relies on vfio_pci_zap_bars() revoking the BAR
range in guest_memfd so __kvm_gmem_revoke() takes
filemap_invalidate_lock() and drains in-flight kvm_gmem_rw() calls.  For a
sub-page BAR, however, vfio_pci_zap_bars() passes nr_pages = 0:

drivers/vfio/pci/vfio_pci_core.c:vfio_pci_zap_bars() {
	...
			gmem_start = VFIO_PCI_INDEX_TO_OFFSET(i) >> PAGE_SHIFT;
			nr_pages = pci_resource_len(vdev->pdev, i) >> PAGE_SHIFT;
			pfn = pci_resource_start(vdev->pdev, i) >> PAGE_SHIFT;

			vfio_device_gmem_revoke(&vdev->vdev, gmem_start,
						nr_pages, &pfn);
	...
}

When nr_pages is 0, __kvm_gmem_revoke() returns early before acquiring
filemap_invalidate_lock(inode->i_mapping):

CPU0                                  CPU1
kvm_gmem_rw()
  filemap_invalidate_lock_shared()
  vfio_pci_core_gmem_rw()
    vfio_pci_get_bars_available()
                                      vfio_pci_ioctl_reset()
                                        down_write(&vdev->memory_lock)
                                        vfio_pci_zap_bars()
                                          vfio_pci_set_bars_available(false)
                                          __kvm_gmem_revoke(..., nr_pages=0)
                                            return 0;
                                        pci_try_reset_function()
    vfio_pci_bar_rw_kbuf(..., false)
      vfio_pci_core_iowrite()  <-- MMIO during reset

This allows MMIO reads and writes to run concurrently with a PCI reset, a
transition to PCI_D3hot, or clearing PCI_COMMAND_MEMORY, which can cause
Completer Abort / Unsupported Request errors or synchronous external
aborts.

> +
> +	/* Lockless check on availability of BARs. */
> +	if (!vfio_pci_get_bars_available(vdev))
> +		return -EIO;
> +
> +	return vfio_pci_bar_rw_kbuf(vdev, kbuf, count, ppos, iswrite, false);

[Severity: High]
Can vfio_pci_core_gmem_rw() issue MMIO accesses while the device is
runtime-suspended in D3?

Unlike vfio_pci_rw(), vfio_pci_core_gmem_rw() does not call
pm_runtime_resume_and_get() and passes test_mem = false to
vfio_pci_bar_rw_kbuf(), relying on vdev->bars_available to be false while
runtime PM low-power mode is engaged (vdev->pm_runtime_engaged).

When userspace enters low-power mode via
VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY, vfio_pci_runtime_pm_entry() zaps BARs
(setting bars_available = false), sets vdev->pm_runtime_engaged = true,
and drops the runtime PM reference.  If userspace then performs a config
write (or a reset ioctl), vfio_pci_rw() temporarily resumes the device to
D0 for the duration of the call.  Because vdev->pm_wake_eventfd_ctx is
NULL, vfio_pci_core_runtime_resume() leaves vdev->pm_runtime_engaged set
to true, but the config write or reset path re-enables bars_available
without checking !vdev->pm_runtime_engaged:

drivers/vfio/pci/vfio_pci_config.c:vfio_basic_config_write() {
	...
		if (__vfio_pci_memory_enabled(vdev)) {
			vfio_pci_dma_buf_move(vdev, false);
			vfio_pci_set_bars_available(vdev, true);
		}
	...
}

When the config write finishes and calls pm_runtime_put(),
vfio_pci_core_runtime_suspend() suspends the device back to D3 without
clearing bars_available.  Subsequent guest_memfd reads and writes see
vfio_pci_get_bars_available(vdev) == true and perform MMIO against the
device while it is in D3.

Conversely, when waking from VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY_WITH_WAKEUP,
vfio_pci_core_runtime_resume() calls __vfio_pci_runtime_pm_exit() without
restoring bars_available = true, leaving guest_memfd BAR accesses failing
with -EIO after wakeup.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=71

Re: [PATCH RFC v3 72/88] KVM: selftests: Add initial VFIO test verifying device info

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] KVM: selftests: missing cdev availability check in
  find_igb_cdev_path()
--

commit f49eda30798c7be85f256d365f245ff1c80a3c8c
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Add initial VFIO test verifying device info

This commit adds a standalone KVM selftest that opens an Intel 82576
VFIO cdev, binds it to iommufd, and verifies the device information
returned by VFIO_DEVICE_GET_INFO.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> new file mode 100644
> index 0000000000000..669acc6058b5b
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c

[ ... ]

> +static char *find_igb_cdev_path(void)
> +{

[ ... ]

> +		/* Found an Intel 82576 device. Check if bound to vfio-pci cdev. */
> +		snprintf(path, sizeof(path), "%s/%s/vfio-dev", sys_pci, dev_entry->d_name);
> +		vfio_dir = opendir(path);
> +		if (!vfio_dir)
> +			continue;
> +
> +		while ((vfio_entry = readdir(vfio_dir)) != NULL) {
> +			if (strncmp(vfio_entry->d_name, "vfio", 4) == 0) {
> +				snprintf(cdev_path, sizeof(cdev_path), "/dev/vfio/devices/%s",
> +					 vfio_entry->d_name);
> +				closedir(vfio_dir);
> +				closedir(dir);
> +				return strdup(cdev_path);
> +			}
> +		}
> +		closedir(vfio_dir);
> +	}
> +
> +	closedir(dir);
> +	return NULL;
> +}
> +
> +FIXTURE(guest_memfd_vfio) {
> +	char *cdev_path;
> +	int iommufd;
> +	int vfio_fd;
> +};
> +
> +FIXTURE_SETUP(guest_memfd_vfio)
> +{
> +	struct vfio_device_bind_iommufd bind_args = {};
> +	int ret;
> +
> +	self->cdev_path = find_igb_cdev_path();
> +	if (!self->cdev_path)
> +		SKIP(return, "Intel 82576 device bound to vfio not found\n");
> +
> +	self->iommufd = open("/dev/iommu", O_RDWR);
> +	if (self->iommufd < 0) {
> +		free(self->cdev_path);
> +		self->cdev_path = NULL;
> +		SKIP(return, "Failed to open /dev/iommu\n");
> +	}
> +
> +	self->vfio_fd = open(self->cdev_path, O_RDWR);
> +	ASSERT_GE(self->vfio_fd, 0);
> +
> +	bind_args.argsz = sizeof(bind_args);
> +	bind_args.iommufd = self->iommufd;
> +	ret = ioctl(self->vfio_fd, VFIO_DEVICE_BIND_IOMMUFD, &bind_args);
> +	ASSERT_EQ(0, ret);
> +}

[Severity: Low]
Can FIXTURE_SETUP(guest_memfd_vfio) fail the test instead of skipping or
trying the next matching port when /dev/vfio/devices/vfioX is absent or
already in use?

vfio_init_device() registers the device under the vfio-dev sysfs class
even when CONFIG_VFIO_DEVICE_CDEV is disabled, so
/sys/bus/pci/devices/<bdf>/vfio-dev/vfioX exists without a corresponding
/dev/vfio/devices/vfioX node.  In addition, Intel 82576 is a multi-port
NIC, and find_igb_cdev_path() unconditionally returns the first function
it sees in sysfs, so if that port is already bound by another process or
if the cdev node is missing or inaccessible, the test fails on
ASSERT_GE() or ASSERT_EQ() instead of trying remaining ports or skipping.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=72

Re: [PATCH RFC v3 78/88] KVM: selftests: Verify VFIO access directly and via guest_memfd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] KVM: selftests: missing SW_PROTECTED_VM cap check in
  guest_memfd_vfio_setup()
- [Low] KVM: selftests: misuse of TEST_REQUIRE() in
  guest_memfd_vfio_setup()
--

commit 228d91c74f669cf39c584f55bf33a2c0818a27c4
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Verify VFIO access directly and via guest_memfd

This commit extends the VFIO igb selftest with fixture variants to test BAR
access, mmap, and device reset both directly via the VFIO descriptor and via
a guest_memfd resource descriptor.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> index 91cba359e4392..03c324f7f618d 100644
> --- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
[ ... ]
> @@ -123,12 +145,50 @@ FIXTURE_SETUP(guest_memfd_vfio)
>  
>  	bind_args.argsz = sizeof(bind_args);
>  	bind_args.iommufd = self->iommufd;
> +	if (variant->use_gmem)
> +		bind_args.flags = VFIO_DEVICE_BIND_FLAG_CTRL_ONLY;
> +
>  	ret = ioctl(self->vfio_fd, VFIO_DEVICE_BIND_IOMMUFD, &bind_args);
>  	ASSERT_EQ(0, ret);
> +
> +	/*
> +	 * Create a SW_PROTECTED_VM with 1 runnable vCPU allocated for memslot0
> +	 * and stack/page table allocation, so guest_mapping can execute vCPUs
> +	 * and FIXTURE_TEARDOWN uniformly cleans up self->vm across both variants.
> +	 */
> +	self->vm = __vm_create((struct vm_shape){ .mode = VM_MODE_DEFAULT,
> +						  .type = KVM_X86_SW_PROTECTED_VM },
> +			       1, 0);

[Severity: Medium]
Does creating a KVM_X86_SW_PROTECTED_VM unconditionally here cause a
regression when the kernel is built with CONFIG_KVM_SW_PROTECTED_VM=n or
booted with TDP MMU disabled?

Without checking kvm_check_cap(KVM_CAP_VM_TYPES) &
BIT(KVM_X86_SW_PROTECTED_VM) first, KVM_CREATE_VM returns -EINVAL and
vm_open() aborts via TEST_ASSERT(), failing all tests in both the
guest_memfd and direct variants instead of skipping when the VM type is
unavailable.

> +
> +	if (variant->use_gmem) {
> +		uint64_t gmem_flags = GUEST_MEMFD_FLAG_USE_RESOURCE |
> +				      GUEST_MEMFD_FLAG_PFN_MODE |
> +				      GUEST_MEMFD_FLAG_MMAP |
> +				      GUEST_MEMFD_FLAG_INIT_SHARED;
> +		uint64_t gmem_size = (uint64_t)VFIO_PCI_NUM_REGIONS << VFIO_PCI_OFFSET_SHIFT;
> +
> +		/*
> +		 * guest_memfd providers (including VFIO) is only intended to be
> +		 * used with gmem_in_place_conversion.
> +		 */
> +		TEST_REQUIRE(kvm_has_gmem_attributes);

[Severity: Low]
Should this use SKIP(return, ...) instead of TEST_REQUIRE()?

Because kselftest_harness.h calls ksft_reset_state() in the forked child
before running FIXTURE_SETUP(), TEST_REQUIRE() calls ksft_exit_skip() with
ksft_plan == 0, which prints a nested "1..0 # SKIP" plan line in the middle
of the TAP stream and leaves _metadata->results->reason empty so the harness
reports "# SKIP unknown".

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=78

Re: [PATCH RFC v3 79/88] KVM: selftests: Verify BAR revocation and access gating in low power state

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: selftests: low_power_bar_revocation() fails on direct
  VFIO variant
--

commit de038c6f26d71384fc01f1fa33e4f547141e9831
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Verify BAR revocation and access gating in low power state

This commit adds a low_power_bar_revocation subtest to guest_memfd_vfio_test
to verify that entering D3hot revokes BAR mmap mappings and rejects
pread/pwrite accesses until returning to D0.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> index 03c324f7f618d..cf2d46a365504 100644
> --- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> @@ -443,4 +443,90 @@ TEST_F(guest_memfd_vfio, device_reset)
>  	ASSERT_EQ(0, munmap(bar0, bar0_info.size));
>  }
>  
> +TEST_F(guest_memfd_vfio, low_power_bar_revocation)
> +{
[ ... ]
> +	/* Enter low-power D3hot state */
> +	feat.argsz = sizeof(feat);
> +	feat.flags = VFIO_DEVICE_FEATURE_SET |
> +		     VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY;
> +	ret = ioctl(self->vfio_fd, VFIO_DEVICE_FEATURE, &feat);
> +	ASSERT_EQ(0, ret);
> +
> +	/*
> +	 * In low-power state, BARs are marked unavailable and mappings are
> +	 * revoked.
> +	 *
> +	 * - Host mmap accesses trigger SIGBUS because page tables were zapped
> +	 *   and get_pfn fails with -EFAULT while in low power state.
> +	 * - Host pread() and pwrite() on the BAR descriptor fail with -EIO.
> +	 */
> +	TEST_EXPECT_SIGBUS((void)READ_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0)));
> +	TEST_EXPECT_SIGBUS(WRITE_ONCE(*(uint32_t *)((char *)bar0 + E1000_RAL0), 0x12345678));
> +
> +	val = 0;
> +	ret_bytes = pread(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_STATUS);
> +	ASSERT_EQ(-1, ret_bytes);
> +	ASSERT_EQ(EIO, errno);
> +
> +	val = 0x12345678;
> +	ret_bytes = pwrite(self->bar_fd, &val, sizeof(val), bar0_info.offset + E1000_RAL0);
> +	ASSERT_EQ(-1, ret_bytes);
> +	ASSERT_EQ(EIO, errno);

[Severity: Medium]
Will these pread() and pwrite() assertions fail in the direct fixture
variant?

In the guest_memfd variant, self->bar_fd is self->gmem_fd, and
vfio_pci_core_gmem_rw() rejects the access with -EIO because
vdev->bars_available is false.  In the direct variant, however,
self->bar_fd is self->vfio_fd, and VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY
wakes the device on any descriptor access through the VFIO uAPI:

drivers/vfio/pci/vfio_pci_core.c:vfio_pci_rw() {
	...
	ret = pm_runtime_resume_and_get(&vdev->pdev->dev);
	if (ret) {
		pci_info_ratelimited(vdev->pdev, "runtime resume failed %d\n",
				     ret);
		return -EIO;
	}

	switch (index) {
	...
	case VFIO_PCI_BAR0_REGION_INDEX ... VFIO_PCI_BAR5_REGION_INDEX:
		ret = vfio_pci_bar_rw(vdev, buf, count, ppos, iswrite);
		break;
}

Because vfio_pci_rw() resumes the device to PCI_D0 before calling
vfio_pci_bar_rw(), and __vfio_pci_memory_enabled() only checks
pdev->current_state < PCI_D3hot and PCI_COMMAND_MEMORY, pread() and
pwrite() succeed and return sizeof(val) instead of failing with EIO.
Should the pread()/pwrite() EIO checks be gated on variant->use_gmem, or
should D3hot be entered via PCI_PM_CTRL in config space so both variants
stay in D3hot across descriptor accesses?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=79

Re: [PATCH RFC v3 81/88] KVM: selftests: Verify guest mapping for VFIO BAR

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] KVM: selftests: guest_mapping() checks stage-1 PTE via
  vm_get_pte()
--

commit 8a5370d54b044c85eadf5eb14da654ead8af47a1
Author: Ackerley Tng <ackerleytng@google.com>

KVM: selftests: Verify guest mapping for VFIO BAR

This commit adds a selftest that configures a VFIO BAR into a KVM memslot
and runs a guest vCPU to verify guest access to the BAR.

> diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> index 79a0001ca1733..0339794fb6dd3 100644
> --- a/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c
> +++ b/tools/testing/selftests/kvm/x86/guest_memfd_vfio_test.c

[ ... ]

> @@ -594,4 +595,60 @@ TEST_F(guest_memfd_vfio, host_vma_flags)
>  	ASSERT_EQ(0, munmap(bar0_mmap, bar0_info.size));
>  }
>  
> +static void guest_memslot_cacheability_code(uint64_t bar_gpa)
> +{
> +	uint32_t *reg = (uint32_t *)(bar_gpa + E1000_STATUS);
> +	uint32_t val;
> +
> +	val = READ_ONCE(*reg);
> +	GUEST_ASSERT(val != 0xFFFFFFFF);
> +
> +	GUEST_DONE();
> +}
> +
> +TEST_F(guest_memfd_vfio, guest_mapping)
> +{
> +	struct vfio_region_info bar0_info = {};
> +	uint64_t bar_gpa = 0xc0000000;
> +	struct kvm_vcpu *vcpu;
> +	void *bar0_mmap;
> +	uint64_t *pte;
> +	int ret;
> +
> +	vfio_pci_enable_memory_and_bus_master(_metadata, self->vfio_fd);
> +
> +	bar0_info.argsz = sizeof(bar0_info);
> +	bar0_info.index = VFIO_PCI_BAR0_REGION_INDEX;
> +	ret = ioctl(self->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, &bar0_info);
> +	ASSERT_EQ(0, ret);
> +	ASSERT_TRUE(bar0_info.flags & VFIO_REGION_INFO_FLAG_MMAP);
> +
> +	/* mmap the VFIO BAR directly into host userspace for memslot config */
> +	bar0_mmap = mmap(NULL, bar0_info.size, PROT_READ | PROT_WRITE,
> +			 MAP_SHARED, self->bar_fd, bar0_info.offset);
> +	ASSERT_NE(MAP_FAILED, bar0_mmap);
> +
> +	vcpu = vm_vcpu_add(self->vm, 0, guest_memslot_cacheability_code);
> +	if (variant->use_gmem)
> +		vm_set_user_memory_region2(self->vm, 1, KVM_MEM_GUEST_MEMFD,
> +					   bar_gpa, bar0_info.size, bar0_mmap,
> +					   self->gmem_fd, bar0_info.offset);
> +	else
> +		vm_set_user_memory_region(self->vm, 1, 0, bar_gpa,
> +					  bar0_info.size, bar0_mmap);
> +
> +	virt_map(self->vm, bar_gpa, bar_gpa, bar0_info.size >> self->vm->page_shift);
> +
> +	/* Run guest to fault in the page via stage 2, then verify S2 PTE */
> +	vcpu_args_set(vcpu, 1, bar_gpa);
> +	vcpu_run(vcpu);
> +	ASSERT_EQ(UCALL_DONE, get_ucall(vcpu, NULL));
> +
> +	pte = vm_get_pte(self->vm, bar_gpa);
> +	ASSERT_NE(NULL, pte);
> +	ASSERT_TRUE(is_present_pte(&self->vm->mmu, pte));

[Severity: Low]
Does vm_get_pte() check the guest stage-1 PTE rather than the stage-2 PTE?

vm_get_pte() walks self->vm->mmu (the guest GVA-to-GPA page tables in
guest memory), where virt_map() -> __virt_pg_map() already set
PTE_PRESENT_MASK(&self->vm->mmu) before vcpu_run(), so is_present_pte()
always succeeds without inspecting KVM's stage-2 page tables.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=81

Re: [PATCH RFC v3 87/88] QUESTION: KVM: guest_memfd: Disallow private mappings of shared-only PFNs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: guest_memfd: private attribute on MSI-X BAR page blocks
  VMM MMIO access
--

commit e4385565a5ed77222308f8b663cc4c2b0a672e04
Author: Ackerley Tng <ackerleytng@google.com>

QUESTION: KVM: guest_memfd: Disallow private mappings of shared-only PFNs

This commit replaces the check_set_attributes provider callback with the
GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE PFN attribute so that VFIO PCI MSI-X
pages are blocked from private stage-2 mappings and population at fault
time rather than during attribute conversion.

> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 385b517b49ed5..cd484bdcb3f0c 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
[ ... ]
> @@ -1128,12 +1118,6 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
>  	if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs))
>  		goto out;
>  
> -	r = gmem_provider_check_set_attributes(gi, start, nr_pages, attrs);
> -	if (r) {
> -		*err_index = start;
> -		goto out;
> -	}
> -
>  	mas_init(&mas, mt, start);
>  	r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
>  	if (r) {
> @@ -1774,10 +1758,11 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
>  
>  	/*
>  	 * Private memory requires exclusive, writable guest ownership and
> -	 * cannot be backed by a read-only frame.
> +	 * cannot be backed by a read-only or shared-only frame.
>  	 */
>  	if (kvm_gmem_is_private_mem(file_inode(file), index) &&
> -	    (*pfn_attrs & GUEST_MEMFD_PFN_ATTRS_READONLY)) {
> +	    (*pfn_attrs & (GUEST_MEMFD_PFN_ATTRS_READONLY |
> +			   GUEST_MEMFD_PFN_ATTRS_NO_PRIVATE))) {
>  		r = -EFAULT;
>  		goto out_put_folio;
>  	}

[Severity: Medium]
If userspace converts an entire BAR to KVM_MEMORY_ATTRIBUTE_PRIVATE
without carving out the MSI-X range, does that block the VMM from
accessing the MSI-X Pending Bit Array or other device registers sharing
the MSI-X page?

When a VFIO PCI device is attached to guest_memfd, it is bound with
VFIO_DEVICE_BIND_FLAG_CTRL_ONLY, which rejects direct mmap(), pread(), and
pwrite() on the VFIO device fd for mappable BARs.  Once
__kvm_gmem_set_attributes() marks the MSI-X page private in
gi->attributes, guest stage-2 faults are rejected here with -EFAULT, while
host VMM accesses through guest_memfd are also rejected because both
kvm_gmem_fault_user_mapping() and kvm_gmem_rw() require the page's
software attribute to be shared:

virt/kvm/guest_memfd.c:kvm_gmem_fault_user_mapping() {
    ...
	if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) {
		r = -EACCES;
    ...
}

virt/kvm/guest_memfd.c:kvm_gmem_rw() {
    ...
		if (!kvm_gmem_range_has_attributes(inode, start, nr_pages, 0)) {
			filemap_invalidate_unlock_shared(inode->i_mapping);
			return total_done ?: -EFAULT;
		}
    ...
}

As a result, even if the VMM places a memslot hole over the MSI-X range to
trap guest MMIO accesses, it has no way to read or write co-located
registers on that page unless the MSI-X range is kept shared in
gi->attributes.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-gmem-alternative-providers-v3-0-8484dde66a7e@google.com?part=87