Kernel KVM virtualization development
 help / color / mirror / Atom feed
* [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages
@ 2026-11-19 18:56 Aaron Lewis
  2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
  To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis

vfio/type1 uses PageReserved(), via is_invalid_reserved_pfn(), to guess
whether a pfn was GUP-pinned or came from a VM_PFNMAP mapping. Reserved
pfns are assumed to be PFNMAP, so they are not accounted in locked_vm
and are never unpinned.

That guess is wrong for reserved pages that GUP does pin, such as
device-dax. ZONE_DEVICE struct pages are marked PG_reserved when they
are initialized, and device-dax, unlike fsdax, can be long-term pinned.
When such memory is DMA mapped, or pinned through vfio_pin_pages(), the
pins are never dropped.

Commit f6c84a52cc41 ("vfio/type1: optimize vfio_pin_pages_remote()")
assumed that long-term GUP never returns reserved pages, so patch 2 also
drops the comment that says so.

This series records how each pfn was obtained instead of inferring it:

- Patch 1 makes vpfn_pages() count only the externally pinned pfns that
  were actually accounted. It fixes a locked_vm imbalance on its own,
  and patch 2 depends on it.
- Patch 2 tracks GUP vs PFNMAP per vfio_dma for the DMA map/unmap path.
- Patch 3 tracks it per vfio_pfn for external page pinning.

Two things to call out:

- Reserved GUP-pinned pages are now charged against RLIMIT_MEMLOCK like
  any other long-term pinned page, so mappings that used to bypass the
  limit can now fail with -ENOMEM.
- A single DMA mapping that contains both PFNMAP and GUP-pinned pfns
  still falls back to the PageReserved() heuristic on unpin, because
  type1 does not record which pfn is which. That stays correct unless
  the GUP-pinned pages are reserved. In that case their pins still leak
  and, since they are now accounted, so does their locked_vm accounting.
  Closing that needs per-range tracking.

The DMA map/unmap path (patch 2) was tested on vfio/next by adding
device-dax and /dev/mem mappings locally to vfio_dma_mapping_test:

1. 1GiB (262,144 pages) device-dax mapping, checking /proc/vmstat (after
   flushing /proc/sys/vm/stat_refresh) before and after running:

     ./vfio_dma_mapping_test -v vfio_type1v2_iommu_devdax_1gb <BDF>

   Before fix:
     nr_foll_pin_acquired: 27471 -> 289615 (+262144)
     nr_foll_pin_released: 27471 -> 27471  (+0, leaked)

   With fix:
     nr_foll_pin_acquired: 18786 -> 280930 (+262144)
     nr_foll_pin_released: 18786 -> 280930 (+262144)

2. /dev/mem (VM_PFNMAP) mapping of a reserved physical page with a valid
   struct page (memmap=4M$0x340000000), confirming DMA map/unmap still
   succeeds with +0/+0 pin deltas and no unpin warnings.

If desirable, I can add the test updates to this series, though the
changes depend on host-specific device-dax paths and reserved physical
addresses.

Aaron Lewis (3):
  vfio/type1: Skip reserved PFNs in vpfn_pages()
  vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
  vfio/type1: Track GUP vs PFNMAP in external page pinning

 drivers/vfio/vfio_iommu_type1.c | 147 +++++++++++++++++++-------------
 1 file changed, 87 insertions(+), 60 deletions(-)


base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
-- 
2.56.1.708.g77570a6a1d-goog


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages()
  2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
  2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
  2026-11-19 18:56 ` [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning Aaron Lewis
  2 siblings, 0 replies; 5+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
  To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis

vpfn_pages() is used by vfio_pin_pages_remote() and
vfio_unpin_pages_remote() to count how many pages in an IOVA range are
already accounted in dma->locked_vm via external page pinning.

However, while vfio_pin_page_external() and
vfio_iommu_unmap_unpin_reaccount() skip invalid and reserved PFNs when
accounting dma->locked_vm, vpfn_pages() counts every entry in
dma->pfn_list indiscriminately. If a reserved PFN is externally pinned,
vpfn_pages() reports it as accounted in dma->locked_vm even though it
never was.

As a result, vfio_unpin_pages_remote() inflates locked_vm when it
unwinds a failed vfio_iommu_replay(). vfio_pin_pages_remote() would
under-account for the same reason, but today it only consults
vpfn_pages() for runs that start with a non-reserved PFN. A subsequent
patch removes that restriction and relies on vpfn_pages() being exact.

Check is_invalid_reserved_pfn() in vpfn_pages() so it only counts
externally pinned PFNs that were actually accounted.

Fixes: f6c84a52cc41 ("vfio/type1: optimize vfio_pin_pages_remote()")
Fixes: d10872050ffe ("vfio/type1: batch vfio_find_vpfn() in function vfio_unpin_pages_remote()")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
 drivers/vfio/vfio_iommu_type1.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index f7addfbe1aab..7bad0508dd21 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -641,7 +641,7 @@ static long vpfn_pages(struct vfio_dma *dma,
 {
 	dma_addr_t iova_end = iova_start + (nr_pages << PAGE_SHIFT);
 	struct vfio_pfn *top = vfio_find_vpfn_range(dma, iova_start, iova_end);
-	long ret = 1;
+	long ret = 0;
 	struct vfio_pfn *vpfn;
 	struct rb_node *prev;
 	struct rb_node *next;
@@ -649,20 +649,25 @@ static long vpfn_pages(struct vfio_dma *dma,
 	if (likely(!top))
 		return 0;
 
+	if (!is_invalid_reserved_pfn(top->pfn))
+		ret++;
+
 	prev = next = &top->node;
 
 	while ((prev = rb_prev(prev))) {
 		vpfn = rb_entry(prev, struct vfio_pfn, node);
 		if (vpfn->iova < iova_start)
 			break;
-		ret++;
+		if (!is_invalid_reserved_pfn(vpfn->pfn))
+			ret++;
 	}
 
 	while ((next = rb_next(next))) {
 		vpfn = rb_entry(next, struct vfio_pfn, node);
 		if (vpfn->iova >= iova_end)
 			break;
-		ret++;
+		if (!is_invalid_reserved_pfn(vpfn->pfn))
+			ret++;
 	}
 
 	return ret;
-- 
2.56.1.708.g77570a6a1d-goog


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
  2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
  2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
  2026-11-19 19:12   ` sashiko-bot
  2026-11-19 18:56 ` [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning Aaron Lewis
  2 siblings, 1 reply; 5+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
  To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis

Both vfio_pin_pages_remote() and vfio_unpin_pages_remote() use
PageReserved() as a heuristic to infer whether a page was PFNMAP'd or
GUP-pinned in vaddr_get_pfns(). That heuristic breaks down when the same
reserved struct pages can appear on both paths:

- DevDax pages mapped from /dev/dax* are reserved and GUP-pinned.
- Those same pages mapped from /dev/mem (PFNMAP) are also reserved,
  but they are *not* GUP-pinned.

Because of this, when unpinning the DMA region, vfio_unpin_pages_remote()
doesn't know whether those reserved pages were GUP-pinned and skips
unpinning them altogether, *leaking page pins and refcounts*.

Fix this by replacing has_rsvd and has_non_rsvd in struct vfio_dma with
has_pfnmap and has_gup_pinned, tracking which path vaddr_get_pfns()
actually used, and clearing them when a DMA range is unpinned. Since
PFNMAP pages bypass struct vfio_batch, this also fixes locked_vm
accounting and batch error-unwind unpinning for reserved GUP-pinned
pages.

This isn't an exhaustive fix because mixed mappings are still allowed.
For example, if a single DMA region contains both GUP-pinned and
PFNMAP pages, vfio_unpin_pages_remote() can't tell them apart. Handle
this by preserving the existing put_pfn() fallback, and add
a comment explaining the situation.

Lastly, clear the new flags once a vfio_dma is fully unpinned. A vfio_dma
can outlive its pins and be pinned again later, after
vfio_iommu_unmap_unpin_reaccount() or a failed vfio_iommu_replay(), by
which time the user may have remapped the range. A stale has_rsvd only
selected the slower unpin path, but a stale has_pfnmap would send a
mapping that is now entirely GUP-pinned down the wrong unpin path.

Fixes: 3565fce3a659 ("mm, x86: get_user_pages() for dax mappings")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
 drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++--------------
 1 file changed, 58 insertions(+), 45 deletions(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 7bad0508dd21..ee40ba942fd5 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -93,8 +93,8 @@ struct vfio_dma {
 	bool			iommu_mapped;
 	bool			lock_cap;	/* capable(CAP_IPC_LOCK) */
 	bool			vaddr_invalid;
-	bool			has_rsvd;	/* has 1 or more rsvd pfns */
-	bool			has_non_rsvd;	/* has 1 or more !rsvd pfns */
+	bool			has_pfnmap;	/* has 1 or more pfnmap pfns */
+	bool			has_gup_pinned;	/* has 1 or more GUP-pinned pfns */
 	struct task_struct	*task;
 	struct rb_root		pfn_list;	/* Ex-user pinned pfn list */
 	unsigned long		*bitmap;
@@ -416,6 +416,13 @@ static struct vfio_pfn *vfio_iova_get_vfio_pfn(struct vfio_dma *dma,
 	return vpfn;
 }
 
+static inline void put_gup_pfns(unsigned long start_pfn, unsigned long npage,
+				int prot)
+{
+	unpin_user_page_range_dirty_lock(pfn_to_page(start_pfn), npage,
+					 prot & IOMMU_WRITE);
+}
+
 static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn)
 {
 	int ret = 0;
@@ -523,12 +530,11 @@ static void vfio_batch_init_single(struct vfio_batch *batch)
 
 static void vfio_batch_unpin(struct vfio_batch *batch, struct vfio_dma *dma)
 {
-	while (batch->size) {
-		unsigned long pfn = page_to_pfn(batch->pages[batch->offset]);
-
-		put_pfn(pfn, dma->prot);
-		batch->offset++;
-		batch->size--;
+	if (batch->size) {
+		unpin_user_pages_dirty_lock(&batch->pages[batch->offset],
+					    batch->size,
+					    dma->prot & IOMMU_WRITE);
+		batch->size = 0;
 	}
 }
 
@@ -673,6 +679,15 @@ static long vpfn_pages(struct vfio_dma *dma,
 	return ret;
 }
 
+/*
+ * After a successful vaddr_get_pfns(), GUP populates batch->pages while
+ * VM_PFNMAP leaves the batch empty.
+ */
+static bool vfio_batch_is_pfnmap(const struct vfio_batch *batch)
+{
+	return batch->size == 0;
+}
+
 /*
  * Attempt to pin pages.  We really don't want to track all the pfns and
  * the iommu can only map chunks of consecutive pfns anyway, so get the
@@ -685,7 +700,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
 	unsigned long pfn;
 	struct mm_struct *mm = current->mm;
 	long ret, pinned = 0, lock_acct = 0;
-	bool rsvd;
+	bool pfnmap = false;
 	dma_addr_t iova = vaddr - dma->vaddr + dma->iova;
 
 	/* This code path is only user initiated */
@@ -696,7 +711,6 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
 		/* Leftover pages in batch from an earlier call. */
 		*pfn_base = page_to_pfn(batch->pages[batch->offset]);
 		pfn = *pfn_base;
-		rsvd = is_invalid_reserved_pfn(*pfn_base);
 	} else {
 		*pfn_base = 0;
 	}
@@ -721,12 +735,18 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
 
 			if (!*pfn_base) {
 				*pfn_base = pfn;
-				rsvd = is_invalid_reserved_pfn(*pfn_base);
+				pfnmap = vfio_batch_is_pfnmap(batch);
+			} else if (pfnmap != vfio_batch_is_pfnmap(batch)) {
+				/*
+				 * Handle PFNMAP and GUP pages separately; any
+				 * GUP pages remain in batch for the next call.
+				 */
+				goto out;
 			}
 
 			/* Handle pfnmap */
-			if (!batch->size) {
-				if (pfn != *pfn_base + pinned || !rsvd)
+			if (pfnmap) {
+				if (pfn != *pfn_base + pinned)
 					goto out;
 
 				pinned += ret;
@@ -742,32 +762,20 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
 		 * due to the fact that vaddr_get_pfns() needs to provide the
 		 * initial pfn for pfnmaps.  Therefore to reduce redundancy,
 		 * the next pfn is fetched at the end of the loop.
-		 * A PageReserved() page could still qualify as page backed
-		 * and rsvd here, and therefore continues to use the batch.
 		 */
 		while (true) {
-			long nr_pages, acct_pages = 0;
+			long nr_pages, acct_pages;
 
-			if (pfn != *pfn_base + pinned ||
-			    rsvd != is_invalid_reserved_pfn(pfn))
+			if (pfn != *pfn_base + pinned)
 				goto out;
 
-			/*
-			 * Using GUP with the FOLL_LONGTERM in
-			 * vaddr_get_pfns() will not return invalid
-			 * or reserved pages.
-			 */
 			nr_pages = num_pages_contiguous(
 					&batch->pages[batch->offset],
 					batch->size);
-			if (!rsvd) {
-				acct_pages = nr_pages;
-				acct_pages -= vpfn_pages(dma, iova, nr_pages);
-			}
+			acct_pages = nr_pages - vpfn_pages(dma, iova, nr_pages);
 
 			/*
-			 * Reserved pages aren't counted against the user,
-			 * externally pinned pages are already counted against
+			 * Externally pinned pages are already counted against
 			 * the user.
 			 */
 			if (acct_pages) {
@@ -796,29 +804,21 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
 	}
 
 out:
-	dma->has_rsvd |= rsvd;
-	dma->has_non_rsvd |= !rsvd;
 	ret = vfio_lock_acct(dma, lock_acct, false);
 
 unpin_out:
 	if (ret < 0) {
-		if (pinned && !rsvd) {
-			for (pfn = *pfn_base ; pinned ; pfn++, pinned--)
-				put_pfn(pfn, dma->prot);
-		}
+		if (pinned && !pfnmap)
+			put_gup_pfns(*pfn_base, pinned, dma->prot);
 		vfio_batch_unpin(batch, dma);
 
 		return ret;
 	}
 
-	return pinned;
-}
+	dma->has_pfnmap |= pfnmap;
+	dma->has_gup_pinned |= !pfnmap;
 
-static inline void put_valid_unreserved_pfns(unsigned long start_pfn,
-		unsigned long npage, int prot)
-{
-	unpin_user_page_range_dirty_lock(pfn_to_page(start_pfn), npage,
-					 prot & IOMMU_WRITE);
+	return pinned;
 }
 
 static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
@@ -827,16 +827,23 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
 {
 	long unlocked = 0, locked = vpfn_pages(dma, iova, npage);
 
-	if (dma->has_rsvd) {
-		if (dma->has_non_rsvd) {
+	if (dma->has_pfnmap) {
+		if (dma->has_gup_pinned) {
 			unsigned long i;
 
+			/*
+			 * Because there is no way to tell if this DMA region
+			 * was PFNMAP or GUP-pinned, fall back to the
+			 * heuristic in put_pfn() of skipping invalid and
+			 * reserved pfns, which fails to unpin reserved pages
+			 * that were GUP-pinned.
+			 */
 			for (i = 0; i < npage; i++)
 				if (put_pfn(pfn++, dma->prot))
 					unlocked++;
 		}
 	} else {
-		put_valid_unreserved_pfns(pfn, npage, dma->prot);
+		put_gup_pfns(pfn, npage, dma->prot);
 		unlocked = npage;
 	}
 	if (do_accounting)
@@ -1232,6 +1239,9 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
 					    &iotlb_gather);
 	}
 
+	dma->has_pfnmap = false;
+	dma->has_gup_pinned = false;
+
 	if (do_accounting) {
 		vfio_lock_acct(dma, -unlocked, true);
 		return 0;
@@ -1947,6 +1957,9 @@ static int vfio_iommu_replay(struct vfio_iommu *iommu,
 			vfio_unpin_pages_remote(dma, iova, phys >> PAGE_SHIFT,
 						size >> PAGE_SHIFT, true);
 		}
+
+		dma->has_pfnmap = false;
+		dma->has_gup_pinned = false;
 	}
 
 	vfio_batch_fini(&batch);
-- 
2.56.1.708.g77570a6a1d-goog


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning
  2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
  2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
  2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
  2 siblings, 0 replies; 5+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
  To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis

Like the DMA map/unmap path, external page pinning for emulated IOMMU
drivers (vfio_iommu_type1_pin_pages()) uses the same heuristic
(is_invalid_reserved_pfn()) to infer if a struct vfio_pfn was PFNMAP'd
or GUP-pinned in vaddr_get_pfns().

That heuristic falls apart for reserved GUP-pinned mappings such as
device-dax. They are treated as PFNMAP, so they are never unpinned,
which leaks page pins and refcounts. They are also left out of locked_vm
accounting.

Record in struct vfio_pfn whether vaddr_get_pfns() used the PFNMAP path,
and use that (vpfn->pfnmap) across all external pinning, unpinning, and
locked_vm accounting paths instead of inferring it.

As with the DMA map/unmap path, GUP-pinned reserved pages are now
charged against RLIMIT_MEMLOCK. When no IOMMU-backed domain is present,
vfio_pin_pages() on such memory can now fail with -ENOMEM where it
previously bypassed the limit.

Opportunistically, drop the forward declaration of put_pfn(), which is
no longer needed.

Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
 drivers/vfio/vfio_iommu_type1.c | 39 ++++++++++++++++++++-------------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index ee40ba942fd5..b9c67ac0b16e 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -130,6 +130,7 @@ struct vfio_pfn {
 	dma_addr_t		iova;		/* Device address */
 	unsigned long		pfn;		/* Host pfn */
 	unsigned int		ref_count;
+	bool			pfnmap;
 };
 
 struct vfio_regions {
@@ -152,8 +153,6 @@ struct vfio_regions {
 #define DIRTY_BITMAP_PAGES_MAX	 ((u64)INT_MAX)
 #define DIRTY_BITMAP_SIZE_MAX	 DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX)
 
-static int put_pfn(unsigned long pfn, int prot);
-
 static struct vfio_iommu_group*
 vfio_iommu_find_iommu_group(struct vfio_iommu *iommu,
 			    struct iommu_group *iommu_group);
@@ -384,7 +383,7 @@ static void vfio_unlink_pfn(struct vfio_dma *dma, struct vfio_pfn *old)
 }
 
 static int vfio_add_to_pfn_list(struct vfio_dma *dma, dma_addr_t iova,
-				unsigned long pfn)
+				unsigned long pfn, bool pfnmap)
 {
 	struct vfio_pfn *vpfn;
 
@@ -395,6 +394,7 @@ static int vfio_add_to_pfn_list(struct vfio_dma *dma, dma_addr_t iova,
 	vpfn->iova = iova;
 	vpfn->pfn = pfn;
 	vpfn->ref_count = 1;
+	vpfn->pfnmap = pfnmap;
 	vfio_link_pfn(dma, vpfn);
 	return 0;
 }
@@ -429,7 +429,10 @@ static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn)
 
 	vpfn->ref_count--;
 	if (!vpfn->ref_count) {
-		ret = put_pfn(vpfn->pfn, dma->prot);
+		if (!vpfn->pfnmap) {
+			put_gup_pfns(vpfn->pfn, 1, dma->prot);
+			ret = 1;
+		}
 		vfio_remove_from_pfn_list(dma, vpfn);
 	}
 	return ret;
@@ -655,7 +658,7 @@ static long vpfn_pages(struct vfio_dma *dma,
 	if (likely(!top))
 		return 0;
 
-	if (!is_invalid_reserved_pfn(top->pfn))
+	if (!top->pfnmap)
 		ret++;
 
 	prev = next = &top->node;
@@ -664,7 +667,7 @@ static long vpfn_pages(struct vfio_dma *dma,
 		vpfn = rb_entry(prev, struct vfio_pfn, node);
 		if (vpfn->iova < iova_start)
 			break;
-		if (!is_invalid_reserved_pfn(vpfn->pfn))
+		if (!vpfn->pfnmap)
 			ret++;
 	}
 
@@ -672,7 +675,7 @@ static long vpfn_pages(struct vfio_dma *dma,
 		vpfn = rb_entry(next, struct vfio_pfn, node);
 		if (vpfn->iova >= iova_end)
 			break;
-		if (!is_invalid_reserved_pfn(vpfn->pfn))
+		if (!vpfn->pfnmap)
 			ret++;
 	}
 
@@ -853,7 +856,8 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
 }
 
 static int vfio_pin_page_external(struct vfio_dma *dma, unsigned long vaddr,
-				  unsigned long *pfn_base, bool do_accounting)
+				  unsigned long *pfn_base, bool do_accounting,
+				  bool *pfnmap)
 {
 	struct vfio_batch batch;
 	struct mm_struct *mm;
@@ -869,12 +873,13 @@ static int vfio_pin_page_external(struct vfio_dma *dma, unsigned long vaddr,
 	if (ret != 1)
 		goto out;
 
+	*pfnmap = vfio_batch_is_pfnmap(&batch);
 	ret = 0;
 
-	if (do_accounting && !is_invalid_reserved_pfn(*pfn_base)) {
+	if (do_accounting && !*pfnmap) {
 		ret = vfio_lock_acct(dma, 1, false);
 		if (ret) {
-			put_pfn(*pfn_base, dma->prot);
+			put_gup_pfns(*pfn_base, 1, dma->prot);
 			if (ret == -ENOMEM)
 				pr_warn("%s: Task %s (%d) RLIMIT_MEMLOCK "
 					"(%ld) exceeded\n", __func__,
@@ -957,6 +962,7 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
 		unsigned long phys_pfn;
 		dma_addr_t iova;
 		struct vfio_pfn *vpfn;
+		bool pfnmap;
 
 		iova = user_iova + PAGE_SIZE * i;
 		dma = vfio_find_dma(iommu, iova, PAGE_SIZE);
@@ -978,7 +984,7 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
 
 		remote_vaddr = dma->vaddr + (iova - dma->iova);
 		ret = vfio_pin_page_external(dma, remote_vaddr, &phys_pfn,
-					     do_accounting);
+					     do_accounting, &pfnmap);
 		if (ret)
 			goto pin_unwind;
 
@@ -987,10 +993,13 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
 			goto pin_unwind;
 		}
 
-		ret = vfio_add_to_pfn_list(dma, iova, phys_pfn);
+		ret = vfio_add_to_pfn_list(dma, iova, phys_pfn, pfnmap);
 		if (ret) {
-			if (put_pfn(phys_pfn, dma->prot) && do_accounting)
-				vfio_lock_acct(dma, -1, true);
+			if (!pfnmap) {
+				put_gup_pfns(phys_pfn, 1, dma->prot);
+				if (do_accounting)
+					vfio_lock_acct(dma, -1, true);
+			}
 			goto pin_unwind;
 		}
 
@@ -2471,7 +2480,7 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu)
 			struct vfio_pfn *vpfn = rb_entry(p, struct vfio_pfn,
 							 node);
 
-			if (!is_invalid_reserved_pfn(vpfn->pfn))
+			if (!vpfn->pfnmap)
 				locked++;
 		}
 		vfio_lock_acct(dma, locked - unlocked, true);
-- 
2.56.1.708.g77570a6a1d-goog


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
  2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
@ 2026-11-19 19:12   ` sashiko-bot
  0 siblings, 0 replies; 5+ messages in thread
From: sashiko-bot @ 2026-11-19 19:12 UTC (permalink / raw)
  To: Aaron Lewis, kvm

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Critical] vfio/type1: page pin leak in vfio_pin_pages_remote() [1]
--

commit 7c0ecb4964283632e7393bd0af979809631123dd
Author: Aaron Lewis <aaronlewis@google.com>

vfio/type1: Track GUP vs PFNMAP in DMA map/unmap

This commit replaces has_rsvd and has_non_rsvd in struct vfio_dma with
has_pfnmap and has_gup_pinned to track whether pages were pinned via GUP
or mapped via PFNMAP in vaddr_get_pfns(), and clears these flags when the
DMA range is unpinned.

> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 7bad0508dd214..ee40ba942fd5e 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
[ ... ]
> @@ -685,7 +700,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
>  	unsigned long pfn;
>  	struct mm_struct *mm = current->mm;
>  	long ret, pinned = 0, lock_acct = 0;
> -	bool rsvd;
> +	bool pfnmap = false;
>  	dma_addr_t iova = vaddr - dma->vaddr + dma->iova;
>  
>  	/* This code path is only user initiated */
> @@ -696,7 +711,6 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
>  		/* Leftover pages in batch from an earlier call. */
>  		*pfn_base = page_to_pfn(batch->pages[batch->offset]);
>  		pfn = *pfn_base;
> -		rsvd = is_invalid_reserved_pfn(*pfn_base);
>  	} else {
>  		*pfn_base = 0;
>  	}
> @@ -721,12 +735,18 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
>  
>  			if (!*pfn_base) {
>  				*pfn_base = pfn;
> -				rsvd = is_invalid_reserved_pfn(*pfn_base);
> +				pfnmap = vfio_batch_is_pfnmap(batch);
> +			} else if (pfnmap != vfio_batch_is_pfnmap(batch)) {
> +				/*
> +				 * Handle PFNMAP and GUP pages separately; any
> +				 * GUP pages remain in batch for the next call.
> +				 */
> +				goto out;
>  			}

[Severity: Critical]
This problem wasn't introduced by this patch, but can this overwrite
*pfn_base and leak page pins [1] when the first page frame mapped in
the loop is PFN 0?

*pfn_base is initialized to 0 to indicate that no base frame has been set
yet. If the initial vaddr_get_pfns() call returns PFN 0, *pfn_base remains
0, so on the next batch refill if (!*pfn_base) evaluates to true again and
overwrites *pfn_base and pfnmap with the next batch's values. The
contiguity check (pfn != *pfn_base + pinned) then fails and returns the
overwritten *pfn_base to the caller, mapping the wrong physical pages into
the IOMMU, leaking the pin on PFN 0, and unpinning the wrong PFN range on
unmap. Checking !pinned instead of !*pfn_base would avoid treating PFN 0 as
uninitialized.

[1] https://sashiko.dev/#/bug/linux-8352e900-d371-44e3-bf35-a0a7fd55ac4b

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185659.889654-1-aaronlewis@google.com?part=2

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2026-11-19 19:12 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
2026-11-19 19:12   ` sashiko-bot
2026-11-19 18:56 ` [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning Aaron Lewis

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox