From: Swarna Prabhu <sw.prabhu6@gmail.com>
To: akpm@linux-foundation.org, david@kernel.org, jgg@ziepe.ca,
jhubbard@nvidia.com, viro@zeniv.linux.org.uk
Cc: linux-mm@kvack.org, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, dave@stgolabs.net,
riel@surriel.com, yeoreum.yun@arm.com, hch@lst.de,
Swarna Prabhu <s.prabhu@samsung.com>
Subject: [RFC PATCH 0/5] gup: introduce folio-range interface for pinning user memory
Date: Wed, 18 Nov 2026 13:34:53 -0800 [thread overview]
Message-ID: <20261118213458.377258-1-sw.prabhu6@gmail.com> (raw)
From: Swarna Prabhu <s.prabhu@samsung.com>
This series is an RFC to gather design feedback to introduce a folio-range
interface for pinning user memory in the GUP fast path. Pinned memory is
returned as folio ranges instead of a page array: a run of consecutive PTEs
mapping consecutive pages of one folio is pinned once and handed back as a single
range. Block direct I/O converts each range into a single bvec, hence reducing the
per-page pin and merge work.
Base and dependencies
=====================
This series is rebased on the following two series applied:
- "mm: gup: batch PTE-mapped large folios in gup_fast_pte_range()"
v4 (Yeoreum Yun; originally posted by Yuan-Hao Hsu [3]), applied to
mm/core.git gup-7.5.ptebatch [4]. It batches the reference grabs for
PTE-mapped large folios in GUP-fast.
- "batch lookups in follow_page_mask()" v5 (Rik van Riel) [5]. It makes
follow_page_mask() return a page count and record pages through
gup_fill_pages().
Note: "[PATCH v5 1/8] mm/gup: break out gup_fill_pages() helper" was
posted just as testing for this RFC was done. To avoid delaying
feedback on the overall design, this RFC retains the tested base
linked in [5] and will be rebased onto the latest revision for next
version.
This series builds on both: the run detection in gup_fast_pte_range() and
the batched slow-path lookup are reused, and this series changes how the
result of a run is recorded and returned.
Background
==========
Kundan's LPC '24 talk "Transitioning get_user_pages to folio based mapping"
[1] proposed a struct folio_vec (based on bcachefs) passed in place of a
pages array. Building on that and on suggestions from the community [2],
this series exposes folio + index + nr_pages, where each range spans a
single folio:
struct folio_range {
struct folio *folio;
unsigned int idx;
unsigned int nr_pages;
};
Design
======
A "struct gup_out" descriptor in mm/gup.c holds either the page array or
the folio-range array. The fast and slow page-table walkers append to the
descriptor instead of advancing a pages[] pointer, so a single walk
serves both interfaces without duplicating the walkers.
The PTE batching in gup_fast_pte_range() is provided by Yeoreum Yun's
series [4], which this RFC builds on. This series reuses that batching
and its page-table revalidation logic. For the folio-range interface,
each pinned run is recorded directly as a folio range instead of being
expanded into individual page pointers in the output array.
pin_user_folio_ranges_fast() stops once the caller's range array is
full, so callers size the array in ranges, independently of the number
of pages requested. A single GUP-fast walk is additionally capped at
EXTRACT_MAX_PAGES (1024 pages) to bound the time spent with interrupts
disabled when the buffer is backed by small pages.
unpin_user_folio_ranges() walks the range array and calls gup_put_folio()
once per range, releasing the references pin_user_folio_ranges_fast()
took for it.
In iov_iter_extract_bvecs(), the pinned folio ranges are stored into the
free slot of the bvec array and converted in place, one bvec per range.
Block direct I/O picks this up through bio_iov_iter_get_pages() without
changes to the block layer.
Non-folio mappings
==================
Memory that is not backed by folios, such as pages a driver inserted
into a VM_MIXEDMAP VMA, makes pin_user_folio_ranges_fast() fail with
-EOPNOTSUPP when no pages were pinned; such iterators fall back to the
page-array path.
For folio-range output, gup_fast_folio_allowed() additionally requires
a non-NULL folio->mapping, except for hugetlb folios. If the mapping is
NULL, it falls back to the slow path, where the VMA can be checked.
The existing file-backed and secretmem restrictions remain unchanged.
FOLL_LONGTERM is not supported yet and is rejected.
Testing
=======
Measured the time spent in bio_iov_iter_get_pages() per I/O with
function_graph, on the baseline and the baseline plus this series.
Both kernels ran with the performance governor, boost disabled, all
C-states except POLL disabled, and fio pinned to one CPU:
trace-cmd record -p function_graph -g bio_iov_iter_get_pages \
--max-graph-depth 8 -O funcgraph-overhead -O nofuncgraph-irqs -F -c \
fio --name=io_uring_test --filename=/dev/nvme1n1 --ioengine=io_uring \
--rw=randwrite --direct=1 --bs=<bs> --iodepth=8 --iomem=mmap \
--iomem_align=2M --numjobs=1 --size=64M --cpus_allowed=23
Median of calls not interrupted during the walk:
bs buffer baseline patched result
128K 64 kB mTHP 13.08 us 13.09 us neutral
1M 64 kB mTHP 63.0 us 63.6 us neutral
4M 2 MB THP (PMD) 25.9 us 8.65 us ~3.0x faster
8M 2 MB THP (PMD) 49.4 us ~17.0 us ~2.9x faster
Note: the baseline is the upstream kernel with Yeoreum Yun's [4] and
Rik van Riel's [5] changes applied.
For larger I/O, the baseline's bio extraction path requests at max
256 pages per GUP call, limited by the available bvec slots. This
series instead uses those slots for folio ranges, each of which can
describe many pages, thereby allowing the series to request up to 1024
pages per call which accounts for the speed up observed in
'bio_iov_iter_get_pages' for 4MB and 8MB DIRECT I/Os.
Testing: Ran selftests gup_test, gup_longterm and cow on the patched kernel
and no regressions were observed.
Link [1]: https://lpc.events/event/18/contributions/1773/
Link [2]: https://lore.kernel.org/all/f2b5189f-10de-4685-97f3-6ee08d159743@kernel.org/
Link [3]: https://lore.kernel.org/all/20260917083747.786-1-aa9736195201@gmail.com/
Link [4]: https://lore.kernel.org/all/20261110-refactor_gup_fast-v4-0-e779a7674274@arm.com/
Link [5]: https://lore.kernel.org/all/20261102193026.3860700-1-riel@surriel.com/
Swarna Prabhu (5):
mm/gup: add struct gup_out output descriptor
mm/gup: make the slow path walkers record into struct gup_out
mm/gup: make the GUP-fast walkers record into struct gup_out
mm/gup: return pinned memory as folio ranges
iov_iter: extract bvecs of user-backed iters from folio ranges
include/linux/mm.h | 37 +++
include/linux/mm_types.h | 23 ++
lib/iov_iter.c | 100 ++++++
mm/gup.c | 695 +++++++++++++++++++++++++++++++--------
4 files changed, 717 insertions(+), 138 deletions(-)
--
2.43.0
next reply other threads:[~2026-11-18 21:36 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 21:34 Swarna Prabhu [this message]
2026-11-18 21:34 ` [PATCH 1/5] mm/gup: add struct gup_out output descriptor Swarna Prabhu
2026-11-18 21:34 ` [PATCH 2/5] mm/gup: make the slow path walkers record into struct gup_out Swarna Prabhu
2026-11-18 21:34 ` [PATCH 3/5] mm/gup: make the GUP-fast " Swarna Prabhu
2026-11-18 21:34 ` [PATCH 4/5] mm/gup: return pinned memory as folio ranges Swarna Prabhu
2026-11-18 21:34 ` [PATCH 5/5] iov_iter: extract bvecs of user-backed iters from " Swarna Prabhu
2026-11-18 23:28 ` [RFC PATCH 0/5] gup: introduce folio-range interface for pinning user memory David Hildenbrand (Arm)
2026-11-19 0:19 ` John Hubbard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118213458.377258-1-sw.prabhu6@gmail.com \
--to=sw.prabhu6@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=dave@stgolabs.net \
--cc=david@kernel.org \
--cc=hch@lst.de \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=riel@surriel.com \
--cc=s.prabhu@samsung.com \
--cc=viro@zeniv.linux.org.uk \
--cc=yeoreum.yun@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox