[PATCH v3 0/3] vfio/pci: fix huge_fault write handling and add selftest

[PATCH v3 0/3] vfio/pci: fix huge_fault write handling and add selftest

[Mina Almasry]

Fix vfio_pci_vmf_insert_pfn() passing write=false to
vmf_insert_pfn_pmd() and vmf_insert_pfn_pud(), which causes an infinite
write page-fault loop when vma->vm_page_prot starts read-only (such as
after soft-dirty clearing via /proc/<pid>/clear_refs in CRIU or live
migration, or on arm64 without hardware DBM), and add a companion VFIO
PCI selftest covering both read-then-write upgrade and initial write
faults after clear_refs.

Changes in v3:
- Add Patch 2/3 introducing vfio_pci_find_mem_bar() in libvfio to locate
  a memory-backed BAR (or range within a BAR) from a table of known
  devices (starting with QEMU's pci-testdev 1b36:0005 BAR 2), per David
  Matlack.
- Update Patch 3/3 (bar_write_fault_after_clear_refs) to require a known
  memory-backed BAR >= SZ_2M via vfio_pci_find_mem_bar(), zap BAR
  mappings via PCI_COMMAND_MEMORY before both sub-tests instead of
  re-mmapping (addressing both David Matlack's and Sashiko's feedback),
  and leave PCI_COMMAND_MEMORY enabled afterward.
- Link to v2: https://lore.kernel.org/all/20261101062049.998160-1-almasrymina@google.com/

Changes in v2:
- Expand Patch 1/2 commit log with practical trigger scenarios (CRIU /
  clear_refs soft-dirty clearing and arm64 software dirty tracking) for
  stable backport evaluation, per Alex Williamson.
- Drop the Assisted-by tag from Patch 1/2.
- Add Patch 2/2 selftest in vfio_pci_device_test (bar_write_fault_after_clear_refs),
  as suggested by Alex Williamson.
- Link to v1: https://lore.kernel.org/all/20261024015540.538758-1-almasrymina@google.com/

Mina Almasry (3):
  vfio/pci: pass FAULT_FLAG_WRITE to vmf_insert_pfn_pmd/pud()
  vfio: selftests: Add vfio_pci_find_mem_bar() helper
  vfio: selftests: Test BAR write faults after clear_refs

 drivers/vfio/pci/vfio_pci_core.c              |  6 +-
 .../lib/include/libvfio/vfio_pci_device.h     | 10 +++
 .../selftests/vfio/lib/vfio_pci_device.c      | 67 +++++++++++++++++++
 .../selftests/vfio/vfio_pci_device_test.c     | 65 ++++++++++++++++++
 4 files changed, 146 insertions(+), 2 deletions(-)


base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
-- 
2.56.0.385.gac671c819a-goog

[PATCH v3 1/3] vfio/pci: pass FAULT_FLAG_WRITE to vmf_insert_pfn_pmd/pud()

[Mina Almasry]

vfio_pci_vmf_insert_pfn() is the only caller of vmf_insert_pfn_pmd() and
vmf_insert_pfn_pud() in the kernel that hardcodes write=false instead of
passing vmf->flags & FAULT_FLAG_WRITE (unlike drm, panthor, and mshv).

On x86_64, normal MAP_SHARED | PROT_WRITE VFIO BAR mappings usually mask
this bug because vma->vm_page_prot already has _PAGE_RW set. However,
when vma->vm_page_prot starts read-only, insert_pmd() and insert_pud()
with write=false install a read-only huge entry without calling
pmd_mkdirty()/maybe_pmd_mkwrite() (and similarly for PUD). Subsequent
write-protection faults enter wp_huge_pmd() or wp_huge_pud(), which call
vma->vm_ops->huge_fault() to upgrade the entry in place. Because
vfio_pci_vmf_insert_pfn() passes write=false, insert_pmd() and
insert_pud() see !pmd_none(*pmd) with !write, do nothing, and return
VM_FAULT_NOPAGE, causing an unrecoverable infinite write page-fault loop
and host CPU soft lockup on the next userspace or KVM guest MMIO store.

In practice, vma->vm_page_prot is read-only on a writable MAP_SHARED BAR
mapping in two situations:

1. Userspace soft-dirty tracking (CONFIG_MEM_SOFT_DIRTY) when a process
   such as CRIU (during pre-dump/checkpoint), a VMM live-migration loop,
   a memory profiler, or an unprivileged process holding a VFIO device
   fd writes "4" (CLEAR_REFS_SOFT_DIRTY) to /proc/<pid>/clear_refs. That
   clears VM_SOFTDIRTY on all VMAs (including VM_PFNMAP) and calls
   vma_set_page_prot(), stripping _PAGE_RW from vma->vm_page_prot for
   subsequent huge faults.

2. Architectures where shared writable mappings start read-only for
   software dirty tracking, such as arm64 without hardware DBM
   (CONFIG_ARM64_HW_AFDBM off or FEAT_HAFDBS absent), where _PAGE_SHARED
   sets PTE_RDONLY | PTE_WRITE and pgprot_noncached() preserves
   PTE_RDONLY on MT_DEVICE_nGnRnE mappings until pmd_mkdirty() clears
   it on a write fault.

This bug was found by code inspection while auditing huge_fault handlers
and reproduced with a VFIO PCI selftest on a 2 MiB-aligned BAR mapping
after writing "4" to /proc/self/clear_refs.

Pass vmf->flags & FAULT_FLAG_WRITE to vmf_insert_pfn_pmd() and
vmf_insert_pfn_pud() so write faults properly mark huge PFNMAP entries
dirty and writable.

Fixes: f9e54c3a2f5b ("vfio/pci: implement huge_fault support")
Cc: stable@vger.kernel.org
Cc: Ankit Agrawal <ankita@nvidia.com>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: David Matlack <dmatlack@google.com>
Signed-off-by: Mina Almasry <almasrymina@google.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index fcbb2da860e2..204d959086e9 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1745,6 +1745,8 @@ vm_fault_t vfio_pci_vmf_insert_pfn(struct vfio_pci_core_device *vdev,
 				   unsigned long pfn,
 				   unsigned int order)
 {
+	bool write = vmf->flags & FAULT_FLAG_WRITE;
+
 	lockdep_assert_held_read(&vdev->memory_lock);
 
 	if (vdev->pm_runtime_engaged || !__vfio_pci_memory_enabled(vdev))
@@ -1754,10 +1756,10 @@ vm_fault_t vfio_pci_vmf_insert_pfn(struct vfio_pci_core_device *vdev,
 		return vmf_insert_pfn(vmf->vma, vmf->address, pfn);
 
 	if (IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_PFNMAP) && order == PMD_ORDER)
-		return vmf_insert_pfn_pmd(vmf, pfn, false);
+		return vmf_insert_pfn_pmd(vmf, pfn, write);
 
 	if (IS_ENABLED(CONFIG_ARCH_SUPPORTS_PUD_PFNMAP) && order == PUD_ORDER)
-		return vmf_insert_pfn_pud(vmf, pfn, false);
+		return vmf_insert_pfn_pud(vmf, pfn, write);
 
 	return VM_FAULT_FALLBACK;
 }
-- 
2.56.0.385.gac671c819a-goog

[PATCH v3 2/3] vfio: selftests: Add vfio_pci_find_mem_bar() helper

[Mina Almasry]

Add vfio_pci_find_mem_bar() to libvfio to locate a BAR (or range within
a BAR) of at least min_size bytes that behaves like ordinary memory and
can be safely read and written without device-specific side effects.

Detection is backed by a table of known devices, starting with QEMU's
pci-testdev (1b36:0005) BAR 2 when configured with membar=<size> and
membar-backed=on.

Suggested-by: David Matlack <dmatlack@google.com>
Cc: David Matlack <dmatlack@google.com>
Cc: Matt Evans <matt@ozlabs.org>
Signed-off-by: Mina Almasry <almasrymina@google.com>
---
 .../lib/include/libvfio/vfio_pci_device.h     | 10 +++
 .../selftests/vfio/lib/vfio_pci_device.c      | 67 +++++++++++++++++++
 2 files changed, 77 insertions(+)

diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
index 007adc322c1e..19d458adba1a 100644
--- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
+++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
@@ -15,6 +15,13 @@ struct vfio_pci_bar {
 	void *vaddr;
 };
 
+struct vfio_pci_mem_bar {
+	struct vfio_pci_bar *bar;
+	void *vaddr;
+	u64 offset;
+	u64 size;
+};
+
 struct vfio_pci_device {
 	const char *bdf;
 	int fd;
@@ -162,6 +169,9 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,
 		(device_id == vfio_pci_config_readw(device, PCI_DEVICE_ID));
 }
 
+bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,
+			   struct vfio_pci_mem_bar *mem_bar);
+
 const char *vfio_pci_get_cdev_path(const char *bdf);
 
 void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf);
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 4063a0e2b3df..d5308fe61490 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -19,6 +19,7 @@
 #include <linux/log2.h>
 #include <linux/mman.h>
 #include <linux/overflow.h>
+#include <linux/pci_ids.h>
 #include <linux/sizes.h>
 #include <linux/types.h>
 #include <linux/vfio.h>
@@ -245,6 +246,72 @@ static void vfio_pci_bar_unmap_all(struct vfio_pci_device *device)
 	}
 }
 
+#define PCI_DEVICE_ID_REDHAT_TEST	0x0005
+
+struct vfio_pci_mem_bar_desc {
+	u16 vendor_id;
+	u16 device_id;
+	u8 bar_index;
+	u64 offset;
+	u64 size;
+};
+
+static const struct vfio_pci_mem_bar_desc mem_bar_descs[] = {
+	/* QEMU pci-testdev with membar=<size>,membar-backed=on */
+	{
+		.vendor_id = PCI_VENDOR_ID_REDHAT,
+		.device_id = PCI_DEVICE_ID_REDHAT_TEST,
+		.bar_index = 2,
+	},
+};
+
+bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,
+			   struct vfio_pci_mem_bar *mem_bar)
+{
+	const u32 rw_mmap_flags = VFIO_REGION_INFO_FLAG_MMAP |
+				  VFIO_REGION_INFO_FLAG_READ |
+				  VFIO_REGION_INFO_FLAG_WRITE;
+	size_t i;
+
+	for (i = 0; i < ARRAY_SIZE(mem_bar_descs); i++) {
+		const struct vfio_pci_mem_bar_desc *desc = &mem_bar_descs[i];
+		struct vfio_pci_bar *bar;
+		u64 size;
+
+		if (!vfio_pci_device_match(device, desc->vendor_id,
+					   desc->device_id))
+			continue;
+
+		VFIO_ASSERT_LT(desc->bar_index, PCI_STD_NUM_BARS);
+		bar = &device->bars[desc->bar_index];
+
+		if ((bar->info.flags & rw_mmap_flags) != rw_mmap_flags)
+			continue;
+
+		VFIO_ASSERT_NOT_NULL(bar->vaddr);
+
+		if (bar->info.size <= desc->offset)
+			continue;
+
+		size = bar->info.size - desc->offset;
+		if (desc->size)
+			size = min(size, desc->size);
+
+		if (size < min_size)
+			continue;
+
+		*mem_bar = (struct vfio_pci_mem_bar){
+			.bar = bar,
+			.vaddr = bar->vaddr + desc->offset,
+			.offset = desc->offset,
+			.size = size,
+		};
+		return true;
+	}
+
+	return false;
+}
+
 void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
 			    size_t config, size_t size, void *data)
 {
-- 
2.56.0.385.gac671c819a-goog

Re: [PATCH v3 2/3] vfio: selftests: Add vfio_pci_find_mem_bar() helper

[David Matlack]

On 2026-11-02 11:04 PM, Mina Almasry wrote:
> Add vfio_pci_find_mem_bar() to libvfio to locate a BAR (or range within
> a BAR) of at least min_size bytes that behaves like ordinary memory and
> can be safely read and written without device-specific side effects.
> 
> Detection is backed by a table of known devices, starting with QEMU's
> pci-testdev (1b36:0005) BAR 2 when configured with membar=<size> and
> membar-backed=on.
> 
> Suggested-by: David Matlack <dmatlack@google.com>
> Cc: David Matlack <dmatlack@google.com>
> Cc: Matt Evans <matt@ozlabs.org>
> Signed-off-by: Mina Almasry <almasrymina@google.com>

Reviewed-by: David Matlack <dmatlack@google.com>

Re: [PATCH v3 2/3] vfio: selftests: Add vfio_pci_find_mem_bar() helper

[Alex Williamson]

On Mon,  2 Nov 2026 23:04:44 +0000
Mina Almasry <almasrymina@google.com> wrote:

> Add vfio_pci_find_mem_bar() to libvfio to locate a BAR (or range within
> a BAR) of at least min_size bytes that behaves like ordinary memory and
> can be safely read and written without device-specific side effects.
> 
> Detection is backed by a table of known devices, starting with QEMU's
> pci-testdev (1b36:0005) BAR 2 when configured with membar=<size> and
> membar-backed=on.

Should we test for this?

We could toggle memory decode around writing a short, unique pattern
with read-back verification.  This might be unnecessary for a physical
device, but per the definition of this function it should also be
harmless.  For the pci-testdev, which is our only match in this
proposal, it validates the BAR is backed by real memory and not handled
as an unimplemented memory range.

For the use case in patch 3/ it doesn't matter, but for future use
cases I'd rather see that we generate a skip or failure based on the
test here rather than debug failures in the test case consumers.
Thanks,

Alex
 
> Suggested-by: David Matlack <dmatlack@google.com>
> Cc: David Matlack <dmatlack@google.com>
> Cc: Matt Evans <matt@ozlabs.org>
> Signed-off-by: Mina Almasry <almasrymina@google.com>
> ---
>  .../lib/include/libvfio/vfio_pci_device.h     | 10 +++
>  .../selftests/vfio/lib/vfio_pci_device.c      | 67 +++++++++++++++++++
>  2 files changed, 77 insertions(+)
> 
> diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
> index 007adc322c1e..19d458adba1a 100644
> --- a/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
> +++ b/tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
> @@ -15,6 +15,13 @@ struct vfio_pci_bar {
>  	void *vaddr;
>  };
>  
> +struct vfio_pci_mem_bar {
> +	struct vfio_pci_bar *bar;
> +	void *vaddr;
> +	u64 offset;
> +	u64 size;
> +};
> +
>  struct vfio_pci_device {
>  	const char *bdf;
>  	int fd;
> @@ -162,6 +169,9 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,
>  		(device_id == vfio_pci_config_readw(device, PCI_DEVICE_ID));
>  }
>  
> +bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,
> +			   struct vfio_pci_mem_bar *mem_bar);
> +
>  const char *vfio_pci_get_cdev_path(const char *bdf);
>  
>  void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf);
> diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
> index 4063a0e2b3df..d5308fe61490 100644
> --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
> +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
> @@ -19,6 +19,7 @@
>  #include <linux/log2.h>
>  #include <linux/mman.h>
>  #include <linux/overflow.h>
> +#include <linux/pci_ids.h>
>  #include <linux/sizes.h>
>  #include <linux/types.h>
>  #include <linux/vfio.h>
> @@ -245,6 +246,72 @@ static void vfio_pci_bar_unmap_all(struct vfio_pci_device *device)
>  	}
>  }
>  
> +#define PCI_DEVICE_ID_REDHAT_TEST	0x0005
> +
> +struct vfio_pci_mem_bar_desc {
> +	u16 vendor_id;
> +	u16 device_id;
> +	u8 bar_index;
> +	u64 offset;
> +	u64 size;
> +};
> +
> +static const struct vfio_pci_mem_bar_desc mem_bar_descs[] = {
> +	/* QEMU pci-testdev with membar=<size>,membar-backed=on */
> +	{
> +		.vendor_id = PCI_VENDOR_ID_REDHAT,
> +		.device_id = PCI_DEVICE_ID_REDHAT_TEST,
> +		.bar_index = 2,
> +	},
> +};
> +
> +bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,
> +			   struct vfio_pci_mem_bar *mem_bar)
> +{
> +	const u32 rw_mmap_flags = VFIO_REGION_INFO_FLAG_MMAP |
> +				  VFIO_REGION_INFO_FLAG_READ |
> +				  VFIO_REGION_INFO_FLAG_WRITE;
> +	size_t i;
> +
> +	for (i = 0; i < ARRAY_SIZE(mem_bar_descs); i++) {
> +		const struct vfio_pci_mem_bar_desc *desc = &mem_bar_descs[i];
> +		struct vfio_pci_bar *bar;
> +		u64 size;
> +
> +		if (!vfio_pci_device_match(device, desc->vendor_id,
> +					   desc->device_id))
> +			continue;
> +
> +		VFIO_ASSERT_LT(desc->bar_index, PCI_STD_NUM_BARS);
> +		bar = &device->bars[desc->bar_index];
> +
> +		if ((bar->info.flags & rw_mmap_flags) != rw_mmap_flags)
> +			continue;
> +
> +		VFIO_ASSERT_NOT_NULL(bar->vaddr);
> +
> +		if (bar->info.size <= desc->offset)
> +			continue;
> +
> +		size = bar->info.size - desc->offset;
> +		if (desc->size)
> +			size = min(size, desc->size);
> +
> +		if (size < min_size)
> +			continue;
> +
> +		*mem_bar = (struct vfio_pci_mem_bar){
> +			.bar = bar,
> +			.vaddr = bar->vaddr + desc->offset,
> +			.offset = desc->offset,
> +			.size = size,
> +		};
> +		return true;
> +	}
> +
> +	return false;
> +}
> +
>  void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
>  			    size_t config, size_t size, void *data)
>  {

[PATCH v3 3/3] vfio: selftests: Test BAR write faults after clear_refs

[Mina Almasry]

Add a regression test in vfio_pci_device_test to verify that huge_fault
on writable MAP_SHARED VFIO PCI BAR mappings handles write faults after
soft-dirty tracking clears VM_SOFTDIRTY and downgrades vma->vm_page_prot
to read-only via /proc/self/clear_refs ("4").

Use vfio_pci_find_mem_bar() to locate a memory-backed BAR of at least
2 MiB that is safe to read and write, and zap existing BAR mappings via
PCI_COMMAND_MEMORY before each sub-test to verify both:

1. Read-then-write fault sequence after clear_refs, exercising in-place
   huge PMD/PUD write-protection upgrade via wp_huge_pmd()/wp_huge_pud().
2. Direct initial write fault on an unpopulated BAR VMA after clear_refs,
   exercising vmf_insert_pfn_pmd()/vmf_insert_pfn_pud() with
   FAULT_FLAG_WRITE set.

Skip gracefully when the device has no known memory-backed BAR >= 2 MiB
or when /proc/self/clear_refs soft-dirty tracking (CONFIG_MEM_SOFT_DIRTY)
is unavailable.

Suggested-by: Alex Williamson <alex@shazbot.org>
Cc: David Matlack <dmatlack@google.com>
Cc: Matt Evans <matt@ozlabs.org>
Signed-off-by: Mina Almasry <almasrymina@google.com>
---
 .../selftests/vfio/vfio_pci_device_test.c     | 65 +++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 93c11fd5e081..323a316f59e7 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -1,10 +1,13 @@
 // SPDX-License-Identifier: GPL-2.0-only
+#include <errno.h>
 #include <fcntl.h>
 #include <stdlib.h>
+#include <unistd.h>
 
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 
+#include <linux/compiler.h>
 #include <linux/limits.h>
 #include <linux/pci_regs.h>
 #include <linux/sizes.h>
@@ -102,6 +105,68 @@ TEST_F(vfio_pci_device_test, validate_bars)
 	}
 }
 
+static void zap_bar_mappings(struct vfio_pci_device *device)
+{
+	vfio_pci_cmd_clear(device, PCI_COMMAND_MEMORY);
+	vfio_pci_cmd_set(device, PCI_COMMAND_MEMORY);
+}
+
+static int clear_soft_dirty_refs(void)
+{
+	ssize_t ret;
+	int fd;
+
+	fd = open("/proc/self/clear_refs", O_WRONLY);
+	if (fd < 0)
+		return -errno;
+
+	ret = write(fd, "4\n", 2);
+	if (ret < 0)
+		ret = -errno;
+	else
+		VFIO_ASSERT_EQ(ret, 2);
+
+	VFIO_ASSERT_EQ(close(fd), 0);
+	return ret < 0 ? ret : 0;
+}
+
+TEST_F(vfio_pci_device_test, bar_write_fault_after_clear_refs)
+{
+	struct vfio_pci_mem_bar mem_bar;
+	int ret;
+	u32 val;
+
+	if (!vfio_pci_find_mem_bar(self->device, SZ_2M, &mem_bar))
+		SKIP(return, "No memory-backed BAR >= 2M on %s\n", device_bdf);
+
+	zap_bar_mappings(self->device);
+
+	ret = clear_soft_dirty_refs();
+	if (ret == -ENOENT || ret == -EINVAL)
+		SKIP(return, "soft-dirty clear_refs unsupported (%d)\n", ret);
+	ASSERT_EQ(0, ret);
+
+	/*
+	 * clear_soft_dirty_refs() stripped write permissions from
+	 * vma->vm_page_prot on the unpopulated BAR mapping. Fault it read-only
+	 * first, then write to exercise in-place write-upgrade
+	 * (wp_huge_pmd/wp_huge_pud).
+	 */
+	val = READ_ONCE(*(u32 *)mem_bar.vaddr);
+	WRITE_ONCE(*(u32 *)mem_bar.vaddr, val);
+
+	/*
+	 * Zap the BAR PTEs/PMDs, clear soft-dirty again, and fault with an
+	 * immediate store to exercise initial write insertion
+	 * (vmf_insert_pfn_pmd/pud with write=true).
+	 */
+	zap_bar_mappings(self->device);
+	ASSERT_EQ(0, clear_soft_dirty_refs());
+
+	WRITE_ONCE(*(u32 *)mem_bar.vaddr, val);
+	(void)READ_ONCE(*(u32 *)mem_bar.vaddr);
+}
+
 FIXTURE(vfio_pci_irq_test) {
 	struct iommu *iommu;
 	struct vfio_pci_device *device;
-- 
2.56.0.385.gac671c819a-goog

Re: [PATCH v3 3/3] vfio: selftests: Test BAR write faults after clear_refs

[David Matlack]

On 2026-11-02 11:04 PM, Mina Almasry wrote:
> Add a regression test in vfio_pci_device_test to verify that huge_fault
> on writable MAP_SHARED VFIO PCI BAR mappings handles write faults after
> soft-dirty tracking clears VM_SOFTDIRTY and downgrades vma->vm_page_prot
> to read-only via /proc/self/clear_refs ("4").
> 
> Use vfio_pci_find_mem_bar() to locate a memory-backed BAR of at least
> 2 MiB that is safe to read and write, and zap existing BAR mappings via
> PCI_COMMAND_MEMORY before each sub-test to verify both:
> 
> 1. Read-then-write fault sequence after clear_refs, exercising in-place
>    huge PMD/PUD write-protection upgrade via wp_huge_pmd()/wp_huge_pud().
> 2. Direct initial write fault on an unpopulated BAR VMA after clear_refs,
>    exercising vmf_insert_pfn_pmd()/vmf_insert_pfn_pud() with
>    FAULT_FLAG_WRITE set.
> 
> Skip gracefully when the device has no known memory-backed BAR >= 2 MiB
> or when /proc/self/clear_refs soft-dirty tracking (CONFIG_MEM_SOFT_DIRTY)
> is unavailable.
> 
> Suggested-by: Alex Williamson <alex@shazbot.org>
> Cc: David Matlack <dmatlack@google.com>
> Cc: Matt Evans <matt@ozlabs.org>
> Signed-off-by: Mina Almasry <almasrymina@google.com>

Reviewed-by: David Matlack <dmatlack@google.com>

Re: [PATCH v3 3/3] vfio: selftests: Test BAR write faults after clear_refs

[Alex Williamson]

On Mon,  2 Nov 2026 23:04:45 +0000
Mina Almasry <almasrymina@google.com> wrote:

> Add a regression test in vfio_pci_device_test to verify that huge_fault
> on writable MAP_SHARED VFIO PCI BAR mappings handles write faults after
> soft-dirty tracking clears VM_SOFTDIRTY and downgrades vma->vm_page_prot
> to read-only via /proc/self/clear_refs ("4").
> 
> Use vfio_pci_find_mem_bar() to locate a memory-backed BAR of at least
> 2 MiB that is safe to read and write, and zap existing BAR mappings via
> PCI_COMMAND_MEMORY before each sub-test to verify both:
> 
> 1. Read-then-write fault sequence after clear_refs, exercising in-place
>    huge PMD/PUD write-protection upgrade via wp_huge_pmd()/wp_huge_pud().
> 2. Direct initial write fault on an unpopulated BAR VMA after clear_refs,
>    exercising vmf_insert_pfn_pmd()/vmf_insert_pfn_pud() with
>    FAULT_FLAG_WRITE set.
> 
> Skip gracefully when the device has no known memory-backed BAR >= 2 MiB
> or when /proc/self/clear_refs soft-dirty tracking (CONFIG_MEM_SOFT_DIRTY)
> is unavailable.
> 
> Suggested-by: Alex Williamson <alex@shazbot.org>
> Cc: David Matlack <dmatlack@google.com>
> Cc: Matt Evans <matt@ozlabs.org>
> Signed-off-by: Mina Almasry <almasrymina@google.com>
> ---
>  .../selftests/vfio/vfio_pci_device_test.c     | 65 +++++++++++++++++++
>  1 file changed, 65 insertions(+)
> 
> diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
> index 93c11fd5e081..323a316f59e7 100644
> --- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
> +++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
> @@ -1,10 +1,13 @@
>  // SPDX-License-Identifier: GPL-2.0-only
> +#include <errno.h>
>  #include <fcntl.h>
>  #include <stdlib.h>
> +#include <unistd.h>
>  
>  #include <sys/ioctl.h>
>  #include <sys/mman.h>
>  
> +#include <linux/compiler.h>
>  #include <linux/limits.h>
>  #include <linux/pci_regs.h>
>  #include <linux/sizes.h>
> @@ -102,6 +105,68 @@ TEST_F(vfio_pci_device_test, validate_bars)
>  	}
>  }
>  
> +static void zap_bar_mappings(struct vfio_pci_device *device)
> +{
> +	vfio_pci_cmd_clear(device, PCI_COMMAND_MEMORY);
> +	vfio_pci_cmd_set(device, PCI_COMMAND_MEMORY);
> +}
> +
> +static int clear_soft_dirty_refs(void)
> +{
> +	ssize_t ret;
> +	int fd;
> +
> +	fd = open("/proc/self/clear_refs", O_WRONLY);
> +	if (fd < 0)
> +		return -errno;
> +
> +	ret = write(fd, "4\n", 2);
> +	if (ret < 0)
> +		ret = -errno;
> +	else
> +		VFIO_ASSERT_EQ(ret, 2);
> +
> +	VFIO_ASSERT_EQ(close(fd), 0);
> +	return ret < 0 ? ret : 0;
> +}
> +
> +TEST_F(vfio_pci_device_test, bar_write_fault_after_clear_refs)
> +{
> +	struct vfio_pci_mem_bar mem_bar;
> +	int ret;
> +	u32 val;
> +
> +	if (!vfio_pci_find_mem_bar(self->device, SZ_2M, &mem_bar))
> +		SKIP(return, "No memory-backed BAR >= 2M on %s\n", device_bdf);
> +
> +	zap_bar_mappings(self->device);
> +
> +	ret = clear_soft_dirty_refs();
> +	if (ret == -ENOENT || ret == -EINVAL)
> +		SKIP(return, "soft-dirty clear_refs unsupported (%d)\n", ret);

My comment on v2 that this doesn't accurately test whether soft-dirty
is supported, potentially leading to a false pass still stands.

Please also have the courtesy to give reasonable time for review before
a respin.  Personally I would expect self review and testing to take
longer than the 80 minutes between David's feedback and v3.  Thanks,

Alex

> +	ASSERT_EQ(0, ret);
> +
> +	/*
> +	 * clear_soft_dirty_refs() stripped write permissions from
> +	 * vma->vm_page_prot on the unpopulated BAR mapping. Fault it read-only
> +	 * first, then write to exercise in-place write-upgrade
> +	 * (wp_huge_pmd/wp_huge_pud).
> +	 */
> +	val = READ_ONCE(*(u32 *)mem_bar.vaddr);
> +	WRITE_ONCE(*(u32 *)mem_bar.vaddr, val);
> +
> +	/*
> +	 * Zap the BAR PTEs/PMDs, clear soft-dirty again, and fault with an
> +	 * immediate store to exercise initial write insertion
> +	 * (vmf_insert_pfn_pmd/pud with write=true).
> +	 */
> +	zap_bar_mappings(self->device);
> +	ASSERT_EQ(0, clear_soft_dirty_refs());
> +
> +	WRITE_ONCE(*(u32 *)mem_bar.vaddr, val);
> +	(void)READ_ONCE(*(u32 *)mem_bar.vaddr);
> +}
> +
>  FIXTURE(vfio_pci_irq_test) {
>  	struct iommu *iommu;
>  	struct vfio_pci_device *device;