[PATCH 0/2] mm/gup: batch PTE-mapped large folios in follow_page_mask()

[Rik van Riel <riel@surriel.com>]

follow_page_mask() reports the size of the page it found with a page_mask:
a bitmask of the enclosing naturally-aligned huge page. That form can only
describe a power-of-two block aligned to its own size, so a PTE-mapped
large folio (mTHP) is walked one page at a time even though it maps a
contiguous run, while the PMD/PUD-mapped cases already return their whole
mapping in one step.

Patch 1 replaces the page_mask output with nr_pages (a plain count) and
adds an @end argument bounding the walk, with no functional change: the
huge PMD/PUD paths report the same stride as before, just as a count.

Patch 2 uses the new interface to batch the PTE case: follow_pte_batch()
counts consecutive present PTEs that map consecutive pages of the same
folio with a uniform write bit, bounded by the page table, @end, the VMA,
and the folio itself, then follow_page_pte() hands the whole run back in
one call.

Measured with mm/gup_test.c (PIN_LONGTERM_BENCHMARK, the slow
pin_user_pages() path) on a 256 MB MADV_HUGEPAGE anonymous region in a
4 CPU VM, median get time over 16 iterations. Each folio size was confirmed
through the per-size anon_fault_alloc counters (4096 folios for 64 kB, 128
for 2 MB):

  gup_test -L -m 256 -n 65536 -r 16 -t
                        before      after
  64 kB mTHP            3140 us      412 us   (7.6x)
  2 MB THP (control)      78 us       76 us
  4 kB base (control)   3010 us     3042 us

The PMD-mapped 2 MB THP already returns the whole mapping in one step, so
it stays fast and unchanged. The 4 kB baseline shows the per-page walk cost
that the 64 kB case paid before this change; only the PTE-mapped large
folio case improves.

Both patches apply standalone against mm-unstable; no other series is
required.

Rik van Riel (2):
  mm/gup: pass an end address to follow_page_mask() and return a page
    count
  mm/gup: batch contiguous PTE-mapped large folios in follow_page_mask()

 mm/gup.c | 139 ++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 97 insertions(+), 42 deletions(-)

base-commit: 7368ccdeff50c27809d3a8276c85bae7e402c96c
---
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
To: linux-kernel@vger.kernel.org
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Usama Arif <usamaarif642@gmail.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: kernel-team@meta.com
-- 
2.53.0-Meta