* [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages()
2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis
vpfn_pages() is used by vfio_pin_pages_remote() and
vfio_unpin_pages_remote() to count how many pages in an IOVA range are
already accounted in dma->locked_vm via external page pinning.
However, while vfio_pin_page_external() and
vfio_iommu_unmap_unpin_reaccount() skip invalid and reserved PFNs when
accounting dma->locked_vm, vpfn_pages() counts every entry in
dma->pfn_list indiscriminately. If a reserved PFN is externally pinned,
vpfn_pages() reports it as accounted in dma->locked_vm even though it
never was.
As a result, vfio_unpin_pages_remote() inflates locked_vm when it
unwinds a failed vfio_iommu_replay(). vfio_pin_pages_remote() would
under-account for the same reason, but today it only consults
vpfn_pages() for runs that start with a non-reserved PFN. A subsequent
patch removes that restriction and relies on vpfn_pages() being exact.
Check is_invalid_reserved_pfn() in vpfn_pages() so it only counts
externally pinned PFNs that were actually accounted.
Fixes: f6c84a52cc41 ("vfio/type1: optimize vfio_pin_pages_remote()")
Fixes: d10872050ffe ("vfio/type1: batch vfio_find_vpfn() in function vfio_unpin_pages_remote()")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
drivers/vfio/vfio_iommu_type1.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index f7addfbe1aab..7bad0508dd21 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -641,7 +641,7 @@ static long vpfn_pages(struct vfio_dma *dma,
{
dma_addr_t iova_end = iova_start + (nr_pages << PAGE_SHIFT);
struct vfio_pfn *top = vfio_find_vpfn_range(dma, iova_start, iova_end);
- long ret = 1;
+ long ret = 0;
struct vfio_pfn *vpfn;
struct rb_node *prev;
struct rb_node *next;
@@ -649,20 +649,25 @@ static long vpfn_pages(struct vfio_dma *dma,
if (likely(!top))
return 0;
+ if (!is_invalid_reserved_pfn(top->pfn))
+ ret++;
+
prev = next = &top->node;
while ((prev = rb_prev(prev))) {
vpfn = rb_entry(prev, struct vfio_pfn, node);
if (vpfn->iova < iova_start)
break;
- ret++;
+ if (!is_invalid_reserved_pfn(vpfn->pfn))
+ ret++;
}
while ((next = rb_next(next))) {
vpfn = rb_entry(next, struct vfio_pfn, node);
if (vpfn->iova >= iova_end)
break;
- ret++;
+ if (!is_invalid_reserved_pfn(vpfn->pfn))
+ ret++;
}
return ret;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
2026-11-19 19:12 ` sashiko-bot
2026-11-19 18:56 ` [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning Aaron Lewis
2026-11-19 20:27 ` [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Jason Gunthorpe
3 siblings, 1 reply; 6+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis
Both vfio_pin_pages_remote() and vfio_unpin_pages_remote() use
PageReserved() as a heuristic to infer whether a page was PFNMAP'd or
GUP-pinned in vaddr_get_pfns(). That heuristic breaks down when the same
reserved struct pages can appear on both paths:
- DevDax pages mapped from /dev/dax* are reserved and GUP-pinned.
- Those same pages mapped from /dev/mem (PFNMAP) are also reserved,
but they are *not* GUP-pinned.
Because of this, when unpinning the DMA region, vfio_unpin_pages_remote()
doesn't know whether those reserved pages were GUP-pinned and skips
unpinning them altogether, *leaking page pins and refcounts*.
Fix this by replacing has_rsvd and has_non_rsvd in struct vfio_dma with
has_pfnmap and has_gup_pinned, tracking which path vaddr_get_pfns()
actually used, and clearing them when a DMA range is unpinned. Since
PFNMAP pages bypass struct vfio_batch, this also fixes locked_vm
accounting and batch error-unwind unpinning for reserved GUP-pinned
pages.
This isn't an exhaustive fix because mixed mappings are still allowed.
For example, if a single DMA region contains both GUP-pinned and
PFNMAP pages, vfio_unpin_pages_remote() can't tell them apart. Handle
this by preserving the existing put_pfn() fallback, and add
a comment explaining the situation.
Lastly, clear the new flags once a vfio_dma is fully unpinned. A vfio_dma
can outlive its pins and be pinned again later, after
vfio_iommu_unmap_unpin_reaccount() or a failed vfio_iommu_replay(), by
which time the user may have remapped the range. A stale has_rsvd only
selected the slower unpin path, but a stale has_pfnmap would send a
mapping that is now entirely GUP-pinned down the wrong unpin path.
Fixes: 3565fce3a659 ("mm, x86: get_user_pages() for dax mappings")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++--------------
1 file changed, 58 insertions(+), 45 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 7bad0508dd21..ee40ba942fd5 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -93,8 +93,8 @@ struct vfio_dma {
bool iommu_mapped;
bool lock_cap; /* capable(CAP_IPC_LOCK) */
bool vaddr_invalid;
- bool has_rsvd; /* has 1 or more rsvd pfns */
- bool has_non_rsvd; /* has 1 or more !rsvd pfns */
+ bool has_pfnmap; /* has 1 or more pfnmap pfns */
+ bool has_gup_pinned; /* has 1 or more GUP-pinned pfns */
struct task_struct *task;
struct rb_root pfn_list; /* Ex-user pinned pfn list */
unsigned long *bitmap;
@@ -416,6 +416,13 @@ static struct vfio_pfn *vfio_iova_get_vfio_pfn(struct vfio_dma *dma,
return vpfn;
}
+static inline void put_gup_pfns(unsigned long start_pfn, unsigned long npage,
+ int prot)
+{
+ unpin_user_page_range_dirty_lock(pfn_to_page(start_pfn), npage,
+ prot & IOMMU_WRITE);
+}
+
static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn)
{
int ret = 0;
@@ -523,12 +530,11 @@ static void vfio_batch_init_single(struct vfio_batch *batch)
static void vfio_batch_unpin(struct vfio_batch *batch, struct vfio_dma *dma)
{
- while (batch->size) {
- unsigned long pfn = page_to_pfn(batch->pages[batch->offset]);
-
- put_pfn(pfn, dma->prot);
- batch->offset++;
- batch->size--;
+ if (batch->size) {
+ unpin_user_pages_dirty_lock(&batch->pages[batch->offset],
+ batch->size,
+ dma->prot & IOMMU_WRITE);
+ batch->size = 0;
}
}
@@ -673,6 +679,15 @@ static long vpfn_pages(struct vfio_dma *dma,
return ret;
}
+/*
+ * After a successful vaddr_get_pfns(), GUP populates batch->pages while
+ * VM_PFNMAP leaves the batch empty.
+ */
+static bool vfio_batch_is_pfnmap(const struct vfio_batch *batch)
+{
+ return batch->size == 0;
+}
+
/*
* Attempt to pin pages. We really don't want to track all the pfns and
* the iommu can only map chunks of consecutive pfns anyway, so get the
@@ -685,7 +700,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
unsigned long pfn;
struct mm_struct *mm = current->mm;
long ret, pinned = 0, lock_acct = 0;
- bool rsvd;
+ bool pfnmap = false;
dma_addr_t iova = vaddr - dma->vaddr + dma->iova;
/* This code path is only user initiated */
@@ -696,7 +711,6 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
/* Leftover pages in batch from an earlier call. */
*pfn_base = page_to_pfn(batch->pages[batch->offset]);
pfn = *pfn_base;
- rsvd = is_invalid_reserved_pfn(*pfn_base);
} else {
*pfn_base = 0;
}
@@ -721,12 +735,18 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
if (!*pfn_base) {
*pfn_base = pfn;
- rsvd = is_invalid_reserved_pfn(*pfn_base);
+ pfnmap = vfio_batch_is_pfnmap(batch);
+ } else if (pfnmap != vfio_batch_is_pfnmap(batch)) {
+ /*
+ * Handle PFNMAP and GUP pages separately; any
+ * GUP pages remain in batch for the next call.
+ */
+ goto out;
}
/* Handle pfnmap */
- if (!batch->size) {
- if (pfn != *pfn_base + pinned || !rsvd)
+ if (pfnmap) {
+ if (pfn != *pfn_base + pinned)
goto out;
pinned += ret;
@@ -742,32 +762,20 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
* due to the fact that vaddr_get_pfns() needs to provide the
* initial pfn for pfnmaps. Therefore to reduce redundancy,
* the next pfn is fetched at the end of the loop.
- * A PageReserved() page could still qualify as page backed
- * and rsvd here, and therefore continues to use the batch.
*/
while (true) {
- long nr_pages, acct_pages = 0;
+ long nr_pages, acct_pages;
- if (pfn != *pfn_base + pinned ||
- rsvd != is_invalid_reserved_pfn(pfn))
+ if (pfn != *pfn_base + pinned)
goto out;
- /*
- * Using GUP with the FOLL_LONGTERM in
- * vaddr_get_pfns() will not return invalid
- * or reserved pages.
- */
nr_pages = num_pages_contiguous(
&batch->pages[batch->offset],
batch->size);
- if (!rsvd) {
- acct_pages = nr_pages;
- acct_pages -= vpfn_pages(dma, iova, nr_pages);
- }
+ acct_pages = nr_pages - vpfn_pages(dma, iova, nr_pages);
/*
- * Reserved pages aren't counted against the user,
- * externally pinned pages are already counted against
+ * Externally pinned pages are already counted against
* the user.
*/
if (acct_pages) {
@@ -796,29 +804,21 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
}
out:
- dma->has_rsvd |= rsvd;
- dma->has_non_rsvd |= !rsvd;
ret = vfio_lock_acct(dma, lock_acct, false);
unpin_out:
if (ret < 0) {
- if (pinned && !rsvd) {
- for (pfn = *pfn_base ; pinned ; pfn++, pinned--)
- put_pfn(pfn, dma->prot);
- }
+ if (pinned && !pfnmap)
+ put_gup_pfns(*pfn_base, pinned, dma->prot);
vfio_batch_unpin(batch, dma);
return ret;
}
- return pinned;
-}
+ dma->has_pfnmap |= pfnmap;
+ dma->has_gup_pinned |= !pfnmap;
-static inline void put_valid_unreserved_pfns(unsigned long start_pfn,
- unsigned long npage, int prot)
-{
- unpin_user_page_range_dirty_lock(pfn_to_page(start_pfn), npage,
- prot & IOMMU_WRITE);
+ return pinned;
}
static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
@@ -827,16 +827,23 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
{
long unlocked = 0, locked = vpfn_pages(dma, iova, npage);
- if (dma->has_rsvd) {
- if (dma->has_non_rsvd) {
+ if (dma->has_pfnmap) {
+ if (dma->has_gup_pinned) {
unsigned long i;
+ /*
+ * Because there is no way to tell if this DMA region
+ * was PFNMAP or GUP-pinned, fall back to the
+ * heuristic in put_pfn() of skipping invalid and
+ * reserved pfns, which fails to unpin reserved pages
+ * that were GUP-pinned.
+ */
for (i = 0; i < npage; i++)
if (put_pfn(pfn++, dma->prot))
unlocked++;
}
} else {
- put_valid_unreserved_pfns(pfn, npage, dma->prot);
+ put_gup_pfns(pfn, npage, dma->prot);
unlocked = npage;
}
if (do_accounting)
@@ -1232,6 +1239,9 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
&iotlb_gather);
}
+ dma->has_pfnmap = false;
+ dma->has_gup_pinned = false;
+
if (do_accounting) {
vfio_lock_acct(dma, -unlocked, true);
return 0;
@@ -1947,6 +1957,9 @@ static int vfio_iommu_replay(struct vfio_iommu *iommu,
vfio_unpin_pages_remote(dma, iova, phys >> PAGE_SHIFT,
size >> PAGE_SHIFT, true);
}
+
+ dma->has_pfnmap = false;
+ dma->has_gup_pinned = false;
}
vfio_batch_fini(&batch);
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
@ 2026-11-19 19:12 ` sashiko-bot
0 siblings, 0 replies; 6+ messages in thread
From: sashiko-bot @ 2026-11-19 19:12 UTC (permalink / raw)
To: Aaron Lewis, kvm
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Critical] vfio/type1: page pin leak in vfio_pin_pages_remote() [1]
--
commit 7c0ecb4964283632e7393bd0af979809631123dd
Author: Aaron Lewis <aaronlewis@google.com>
vfio/type1: Track GUP vs PFNMAP in DMA map/unmap
This commit replaces has_rsvd and has_non_rsvd in struct vfio_dma with
has_pfnmap and has_gup_pinned to track whether pages were pinned via GUP
or mapped via PFNMAP in vaddr_get_pfns(), and clears these flags when the
DMA range is unpinned.
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 7bad0508dd214..ee40ba942fd5e 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
[ ... ]
> @@ -685,7 +700,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
> unsigned long pfn;
> struct mm_struct *mm = current->mm;
> long ret, pinned = 0, lock_acct = 0;
> - bool rsvd;
> + bool pfnmap = false;
> dma_addr_t iova = vaddr - dma->vaddr + dma->iova;
>
> /* This code path is only user initiated */
> @@ -696,7 +711,6 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
> /* Leftover pages in batch from an earlier call. */
> *pfn_base = page_to_pfn(batch->pages[batch->offset]);
> pfn = *pfn_base;
> - rsvd = is_invalid_reserved_pfn(*pfn_base);
> } else {
> *pfn_base = 0;
> }
> @@ -721,12 +735,18 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
>
> if (!*pfn_base) {
> *pfn_base = pfn;
> - rsvd = is_invalid_reserved_pfn(*pfn_base);
> + pfnmap = vfio_batch_is_pfnmap(batch);
> + } else if (pfnmap != vfio_batch_is_pfnmap(batch)) {
> + /*
> + * Handle PFNMAP and GUP pages separately; any
> + * GUP pages remain in batch for the next call.
> + */
> + goto out;
> }
[Severity: Critical]
This problem wasn't introduced by this patch, but can this overwrite
*pfn_base and leak page pins [1] when the first page frame mapped in
the loop is PFN 0?
*pfn_base is initialized to 0 to indicate that no base frame has been set
yet. If the initial vaddr_get_pfns() call returns PFN 0, *pfn_base remains
0, so on the next batch refill if (!*pfn_base) evaluates to true again and
overwrites *pfn_base and pfnmap with the next batch's values. The
contiguity check (pfn != *pfn_base + pinned) then fails and returns the
overwritten *pfn_base to the caller, mapping the wrong physical pages into
the IOMMU, leaking the pin on PFN 0, and unpinning the wrong PFN range on
unmap. Checking !pinned instead of !*pfn_base would avoid treating PFN 0 as
uninitialized.
[1] https://sashiko.dev/#/bug/linux-8352e900-d371-44e3-bf35-a0a7fd55ac4b
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185659.889654-1-aaronlewis@google.com?part=2
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning
2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
2026-11-19 18:56 ` [PATCH 1/3] vfio/type1: Skip reserved PFNs in vpfn_pages() Aaron Lewis
2026-11-19 18:56 ` [PATCH 2/3] vfio/type1: Track GUP vs PFNMAP in DMA map/unmap Aaron Lewis
@ 2026-11-19 18:56 ` Aaron Lewis
2026-11-19 20:27 ` [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Jason Gunthorpe
3 siblings, 0 replies; 6+ messages in thread
From: Aaron Lewis @ 2026-11-19 18:56 UTC (permalink / raw)
To: kvm; +Cc: alex, dmatlack, jgg, Aaron Lewis
Like the DMA map/unmap path, external page pinning for emulated IOMMU
drivers (vfio_iommu_type1_pin_pages()) uses the same heuristic
(is_invalid_reserved_pfn()) to infer if a struct vfio_pfn was PFNMAP'd
or GUP-pinned in vaddr_get_pfns().
That heuristic falls apart for reserved GUP-pinned mappings such as
device-dax. They are treated as PFNMAP, so they are never unpinned,
which leaks page pins and refcounts. They are also left out of locked_vm
accounting.
Record in struct vfio_pfn whether vaddr_get_pfns() used the PFNMAP path,
and use that (vpfn->pfnmap) across all external pinning, unpinning, and
locked_vm accounting paths instead of inferring it.
As with the DMA map/unmap path, GUP-pinned reserved pages are now
charged against RLIMIT_MEMLOCK. When no IOMMU-backed domain is present,
vfio_pin_pages() on such memory can now fail with -ENOMEM where it
previously bypassed the limit.
Opportunistically, drop the forward declaration of put_pfn(), which is
no longer needed.
Fixes: a54eb55045ae ("vfio iommu type1: Add support for mediated devices")
Signed-off-by: Aaron Lewis <aaronlewis@google.com>
---
drivers/vfio/vfio_iommu_type1.c | 39 ++++++++++++++++++++-------------
1 file changed, 24 insertions(+), 15 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index ee40ba942fd5..b9c67ac0b16e 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -130,6 +130,7 @@ struct vfio_pfn {
dma_addr_t iova; /* Device address */
unsigned long pfn; /* Host pfn */
unsigned int ref_count;
+ bool pfnmap;
};
struct vfio_regions {
@@ -152,8 +153,6 @@ struct vfio_regions {
#define DIRTY_BITMAP_PAGES_MAX ((u64)INT_MAX)
#define DIRTY_BITMAP_SIZE_MAX DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX)
-static int put_pfn(unsigned long pfn, int prot);
-
static struct vfio_iommu_group*
vfio_iommu_find_iommu_group(struct vfio_iommu *iommu,
struct iommu_group *iommu_group);
@@ -384,7 +383,7 @@ static void vfio_unlink_pfn(struct vfio_dma *dma, struct vfio_pfn *old)
}
static int vfio_add_to_pfn_list(struct vfio_dma *dma, dma_addr_t iova,
- unsigned long pfn)
+ unsigned long pfn, bool pfnmap)
{
struct vfio_pfn *vpfn;
@@ -395,6 +394,7 @@ static int vfio_add_to_pfn_list(struct vfio_dma *dma, dma_addr_t iova,
vpfn->iova = iova;
vpfn->pfn = pfn;
vpfn->ref_count = 1;
+ vpfn->pfnmap = pfnmap;
vfio_link_pfn(dma, vpfn);
return 0;
}
@@ -429,7 +429,10 @@ static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn)
vpfn->ref_count--;
if (!vpfn->ref_count) {
- ret = put_pfn(vpfn->pfn, dma->prot);
+ if (!vpfn->pfnmap) {
+ put_gup_pfns(vpfn->pfn, 1, dma->prot);
+ ret = 1;
+ }
vfio_remove_from_pfn_list(dma, vpfn);
}
return ret;
@@ -655,7 +658,7 @@ static long vpfn_pages(struct vfio_dma *dma,
if (likely(!top))
return 0;
- if (!is_invalid_reserved_pfn(top->pfn))
+ if (!top->pfnmap)
ret++;
prev = next = &top->node;
@@ -664,7 +667,7 @@ static long vpfn_pages(struct vfio_dma *dma,
vpfn = rb_entry(prev, struct vfio_pfn, node);
if (vpfn->iova < iova_start)
break;
- if (!is_invalid_reserved_pfn(vpfn->pfn))
+ if (!vpfn->pfnmap)
ret++;
}
@@ -672,7 +675,7 @@ static long vpfn_pages(struct vfio_dma *dma,
vpfn = rb_entry(next, struct vfio_pfn, node);
if (vpfn->iova >= iova_end)
break;
- if (!is_invalid_reserved_pfn(vpfn->pfn))
+ if (!vpfn->pfnmap)
ret++;
}
@@ -853,7 +856,8 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
}
static int vfio_pin_page_external(struct vfio_dma *dma, unsigned long vaddr,
- unsigned long *pfn_base, bool do_accounting)
+ unsigned long *pfn_base, bool do_accounting,
+ bool *pfnmap)
{
struct vfio_batch batch;
struct mm_struct *mm;
@@ -869,12 +873,13 @@ static int vfio_pin_page_external(struct vfio_dma *dma, unsigned long vaddr,
if (ret != 1)
goto out;
+ *pfnmap = vfio_batch_is_pfnmap(&batch);
ret = 0;
- if (do_accounting && !is_invalid_reserved_pfn(*pfn_base)) {
+ if (do_accounting && !*pfnmap) {
ret = vfio_lock_acct(dma, 1, false);
if (ret) {
- put_pfn(*pfn_base, dma->prot);
+ put_gup_pfns(*pfn_base, 1, dma->prot);
if (ret == -ENOMEM)
pr_warn("%s: Task %s (%d) RLIMIT_MEMLOCK "
"(%ld) exceeded\n", __func__,
@@ -957,6 +962,7 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
unsigned long phys_pfn;
dma_addr_t iova;
struct vfio_pfn *vpfn;
+ bool pfnmap;
iova = user_iova + PAGE_SIZE * i;
dma = vfio_find_dma(iommu, iova, PAGE_SIZE);
@@ -978,7 +984,7 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
remote_vaddr = dma->vaddr + (iova - dma->iova);
ret = vfio_pin_page_external(dma, remote_vaddr, &phys_pfn,
- do_accounting);
+ do_accounting, &pfnmap);
if (ret)
goto pin_unwind;
@@ -987,10 +993,13 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
goto pin_unwind;
}
- ret = vfio_add_to_pfn_list(dma, iova, phys_pfn);
+ ret = vfio_add_to_pfn_list(dma, iova, phys_pfn, pfnmap);
if (ret) {
- if (put_pfn(phys_pfn, dma->prot) && do_accounting)
- vfio_lock_acct(dma, -1, true);
+ if (!pfnmap) {
+ put_gup_pfns(phys_pfn, 1, dma->prot);
+ if (do_accounting)
+ vfio_lock_acct(dma, -1, true);
+ }
goto pin_unwind;
}
@@ -2471,7 +2480,7 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu)
struct vfio_pfn *vpfn = rb_entry(p, struct vfio_pfn,
node);
- if (!is_invalid_reserved_pfn(vpfn->pfn))
+ if (!vpfn->pfnmap)
locked++;
}
vfio_lock_acct(dma, locked - unlocked, true);
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages
2026-11-19 18:56 [PATCH 0/3] vfio/type1: Fix pin leaks for reserved GUP-pinned pages Aaron Lewis
` (2 preceding siblings ...)
2026-11-19 18:56 ` [PATCH 3/3] vfio/type1: Track GUP vs PFNMAP in external page pinning Aaron Lewis
@ 2026-11-19 20:27 ` Jason Gunthorpe
3 siblings, 0 replies; 6+ messages in thread
From: Jason Gunthorpe @ 2026-11-19 20:27 UTC (permalink / raw)
To: Aaron Lewis; +Cc: kvm, alex, dmatlack
On Thu, Nov 19, 2026 at 06:56:56PM +0000, Aaron Lewis wrote:
> vfio/type1 uses PageReserved(), via is_invalid_reserved_pfn(), to guess
> whether a pfn was GUP-pinned or came from a VM_PFNMAP mapping. Reserved
> pfns are assumed to be PFNMAP, so they are not accounted in locked_vm
> and are never unpinned.
Please just use iommufd if you are running into real cases where this
doesn't work right :\
Jason
^ permalink raw reply [flat|nested] 6+ messages in thread