[PATCH v4 00/21] mm: expand mmap_prepare functionality and usage

Lorenzo Stoakes (Oracle)
2026-03-20T22:40:10Z

This series expands the mmap_prepare functionality, which is intended to
replace the deprecated f_op->mmap hook which has been the source of bugs
and security issues for some time.

This series starts with some cleanup of existing mmap_prepare logic, then
adds documentation for the mmap_prepare call to make it easier for
filesystem and driver writers to understand how it works.

It then importantly adds a vm_ops->mapped hook, a key feature that was
missing from mmap_prepare previously - this is invoked when a driver which
specifies mmap_prepare has successfully been mapped but not merged with
another VMA.

mmap_prepare is invoked prior to a merge being attempted, so you cannot
manipulate state such as reference counts as if it were a new mapping.

The vm_ops->mapped hook allows a driver to perform tasks required at this
stage, and provides symmetry against subsequent vm_ops->open,close calls.

The series uses this to correct the afs implementation which wrongly
manipulated reference count at mmap_prepare time.

It then adds an mmap_prepare equivalent of vm_iomap_memory() -
mmap_action_simple_ioremap(), then uses this to update a number of drivers.

It then splits out the mmap_prepare compatibility layer (which allows for
invocation of mmap_prepare hooks in an mmap() hook) in such a way as to
allow for more incremental implementation of mmap_prepare hooks.

It then uses this to extend mmap_prepare usage in drivers.

Finally it adds an mmap_prepare equivalent of vm_map_pages(), which lays
the foundation for future work which will extend mmap_prepare to DMA
coherent mappings.

v4:
* Added partial revert of AFS as per Vlasta. Labelled as hotfix.
* Updated subsequent afs patch to apply against this version of AFS.
* Reverted rmap_lock_held changes to util.c, mm.h mmap_action_complete()
etc. as per Vlasta.
* Added hotfix to fix issue with rmap lock held over munmap() as per
Vlasta. Labelled as hotfix.
* Force-disable the rmap lock hold feature in the compatbility layer
because being run under the mmap hook eliminates the need for it.
* Removed superfluous map->hold_file_rmap_lock field.
* Moved handling of rmap lock and unmapping to mmap_action_complete().
* Removed unmap_vma_locked() as previous added patches render it
unnecessary.
* Removed __compat_vma_mapped() from compatibility layer and
call_vma_mapped() from VMA layer and made it part of mmap_action_finish()
for all callers.
* Propagated changes to VMA tests.
* Updated mmap_action_map_kernel_pages[_full]() patch to add missing
mmap_complete() noop switch enum value as per Nathan.
* Fixed a doc issue in the mmap_prepare docs - reference
vma_desc_test_flags() rather than _any().
* Rearranged logic so the vm_ops->mapped hook is called before the success
hook, but this should have no impact.

v3:
* Propagated tags (thanks Suren, Richard!)
* Updated 12/16 to correctly clear the vm_area_desc data structure in
set_desc_from_vma() as per Joshua Hahn (thanks! :)
* Fixed type in 12/16 as per Suren (cheers!)
* Fixed up 6/16 to use mmap_action_ioremap_full() in simple_ioremap_prepare() as
suggested by Suren.
* Also fixed up 6/16 to call io_remap_pfn_range_prepare() direct rather than
mmap_action_prepare() as per Suren.
* Also fixed up 6/16 to pass vm_len rather than vm_[start, end] to
__simple_ioremap_prep() as per Suren (thanks for all the above! :)
* Fixed issue in rmap lock being held - we were referencing a vma->vm_file after
the VMA was unmapped, so UAF. Avoid that. Also do_munmap() relies on rmap lock
NOT being held or may deadlock, so extend functionality to ensure we drop it
when it is held on error paths.
* Updated 'area' -> 'vma' variable in 3/16 in VMA test dup.h.
* Fixed up reference to __compat_vma_mmap() in 12/16 commit message.
* Updated 1/16 to no longer duplicatively apply io_remap_pfn_range_pfn().
* Updated 1/16 to delegate I/O remap complete to remap complete logic.
* Fixed various typos in 12/16.
* Fixed stale comment typos in 13/16.
* Fixed commit msg and comment typos in 14/16.
* Removed accidental sneak peak to future functionality in 15/16 commit message
:).
* Fixed up field names to be identical in VMA tests + mm_types.h in 6/16,
15/16.
https://lore.kernel.org/all/cover.1773944114.git.ljs@kernel.org/

v2:
* Rebased on
https://lore.kernel.org/all/cover.1773665966.git.ljs@kernel.org/ to make
Andrew's life easier :)
* Folded all interim fixes into series (thanks Randy for many doc fixes!))
* As per Suren, removed a comment about allocations too small to fail.
* As per Randy, fixed up typo in documentation for vm_area_desc.
* Fixed mmap_action_prepare() not returning if invalid action->type
specified, as updated from Andrew's interim fix (thanks!) and also
reported by kernel test bot.
* Updated mmap_action_prepare() and specific prepare functions to only
pass vm_area_desc parameter as per Suren.
* Fixed up whitespace as per Suren.
* Updated vm_op->open comment in vm_operations_struct to reference forking
as per Suren.
* Added a commit to check that input range is within VMA on remap as per
Suren (this also covers I/O remap and all other cases already asserted).
* Updated AFS to not incorrectly reference count on mmap prepare as per
Usama.
* Also updated various static AFS functions to be consistent with each
other.
* Updated AFS commit message to reflect mmap_prepare being before any VMA
merging as per Suren.
* Updated __compat_vma_mapped() to check for NULL vm_ops as per Usama.
* Updated __compat_vma_mapped() to not reference an unmapped VMA's fields
as per Usama.
* Updated __vma_check_mmap_hook() to check for NULL vm_ops as per Usama.
* Dropped comment about preferring mmap_prepare as seems overly confusing,
as per Suren.
* Updated the mmap lock assert in unmap_vma_locked() to a write lock assert
as per Suren.
* Copied vm_ops->open comment over to VMA tests in appropriate patch as per
Suren.
* Updated mmap_prepare documentation to reflect the fact that no resources
should be allocated upon mmap_prepare.
* Updated mmap_prepare documentation to reference the vm_ops->mapped
callback.
* Fixed stray markdown '## How to use' in documentation.
* Fixed bug reported by kernel test bot re: overlooked
vma_desc_test_flags() -> vma_desc_test() in MTD driver for nommu.
https://lore.kernel.org/linux-mm/cover.1773695307.git.ljs@kernel.org/

v1:
https://lore.kernel.org/linux-mm/cover.1773346620.git.ljs@kernel.org/

Lorenzo Stoakes (Oracle) (21):
mm: various small mmap_prepare cleanups
mm: add documentation for the mmap_prepare file operation callback
mm: document vm_operations_struct->open the same as close()
mm: avoid deadlock when holding rmap on mmap_prepare error
mm: switch the rmap lock held option off in compat layer
mm/vma: remove superfluous map->hold_file_rmap_lock
mm: have mmap_action_complete() handle the rmap lock and unmap
mm: add vm_ops->mapped hook
fs: afs: revert mmap_prepare() change
fs: afs: restore mmap_prepare implementation
mm: add mmap_action_simple_ioremap()
misc: open-dice: replace deprecated mmap hook with mmap_prepare
hpet: replace deprecated mmap hook with mmap_prepare
mtdchar: replace deprecated mmap hook with mmap_prepare, clean up
stm: replace deprecated mmap hook with mmap_prepare
staging: vme_user: replace deprecated mmap hook with mmap_prepare
mm: allow handling of stacked mmap_prepare hooks in more drivers
drivers: hv: vmbus: replace deprecated mmap hook with mmap_prepare
uio: replace deprecated mmap hook with mmap_prepare in uio_info
mm: add mmap_action_map_kernel_pages[_full]()
mm: on remap assert that input range within the proposed VMA

Documentation/driver-api/vme.rst | 2 +-
Documentation/filesystems/index.rst | 1 +
Documentation/filesystems/mmap_prepare.rst | 168 ++++++++++++++
drivers/char/hpet.c | 12 +-
drivers/hv/hyperv_vmbus.h | 4 +-
drivers/hv/vmbus_drv.c | 31 ++-
drivers/hwtracing/stm/core.c | 31 ++-
drivers/misc/open-dice.c | 19 +-
drivers/mtd/mtdchar.c | 21 +-
drivers/staging/vme_user/vme.c | 20 +-
drivers/staging/vme_user/vme.h | 2 +-
drivers/staging/vme_user/vme_user.c | 51 +++--
drivers/target/target_core_user.c | 26 ++-
drivers/uio/uio.c | 10 +-
drivers/uio/uio_hv_generic.c | 11 +-
fs/afs/file.c | 36 ++-
include/linux/fs.h | 14 +-
include/linux/hyperv.h | 4 +-
include/linux/mm.h | 158 ++++++++++++-
include/linux/mm_types.h | 17 +-
include/linux/uio_driver.h | 4 +-
mm/internal.h | 46 +++-
mm/memory.c | 175 ++++++++++----
mm/util.c | 251 ++++++++++++++-------
mm/vma.c | 48 ++--
mm/vma.h | 2 +-
tools/testing/vma/include/dup.h | 134 +++++++----
tools/testing/vma/include/stubs.h | 8 +-
28 files changed, 956 insertions(+), 350 deletions(-)
create mode 100644 Documentation/filesystems/mmap_prepare.rst

--
2.53.0