From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3ADF14A619 for ; Mon, 20 Jul 2026 19:48:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784576909; cv=none; b=XhLyZoiZzdFLVW6ZQcjUaEpt0UbQbCLlD2KDH2sMEopTTmlm8kiXmOdtb82O5TqCEeKOPlr6Pp/4IdvzbaiupVRtVwLs4Xa3FYGtzSODfBQhEP3rrT6JKUm0R7nTZTGzyGJ0SJ5Vyvr3dNEub2+ZqTgm90gE6KAcMsT06YRkNNo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784576909; c=relaxed/simple; bh=A+58XOPXoPBOepPmS7emtTSU5G1ib5BrjKXcTU75f+g=; h=Date:To:From:Subject:Message-Id; b=RJN1wsExz2SPi7lPIgvFcSPRzCKUN6UMzCeRXmHs+GL/XvgD/LUqQgOa1KA1SY8vWnd29+oKJj8jBbTn/5sK7/c1H4wazwrcfNgqt9MJQ9i80ozQLiS+3Zc2HMEPVKFzRy2U2k7y77Q8BItgc66gD2BYkgPNJ8iyQj9Rhp34Ogg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=Gqb9Zc8Q; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="Gqb9Zc8Q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D42F1F000E9; Mon, 20 Jul 2026 19:48:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1784576907; bh=u7mRbOL+/uyZqA+NuTCEKf372F1cb5wcZ6dG80KxGw4=; h=Date:To:From:Subject; b=Gqb9Zc8QTJItDKaR48niWlHrGaHJpzFn9l7rfwL+DvKKU3cfm6GHw4HQEXYUvTHV7 bUrcWZwI3uloSozUFxSlmFoIQVw6yBf5PqPdPweexFxc6ypjLv/GE6j6py1yzISO4x DbkJKf14Lyq1/h6wr6XnzKYeSeJsBTTQ0a4fLMWI= Date: Mon, 20 Jul 2026 12:48:26 -0700 To: mm-commits@vger.kernel.org,ljs@kernel.org,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers.patch added to mm-new branch Message-Id: <20260720194827.4D42F1F000E9@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm/vma: introduce VMA virtual page offset field and add helpers has been added to the -mm mm-new branch. Its filename is mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: "Lorenzo Stoakes (ARM)" Subject: mm/vma: introduce VMA virtual page offset field and add helpers Date: Mon, 20 Jul 2026 15:38:27 +0100 Patch series "mm/rmap: index MAP_PRIVATE file-backed folios by virt pgoff", v2. In memory management we've managed to manufacture a great deal of confusion around the concept of anonymous memory. We have: 1. 'Pure anon' memory - anonymous VMAs whose folios are anonymous and swap-backed (thus for reclaim purposes, treated as anonymous). These are simple enough. 2. shmem - file-backed VMAs, file-backed folios (from rmap perspective) so present in the page cache and mapped by an address_space object, but whose folios are also swap-backed (thus treated as anonymous for reclaim purposes). 3. MAP_PRIVATE-mapped /dev/zero - a strange beast whose VMAs have vma->vm_file set, but whose mmap_prepare callback clears vma->vm_ops to satisfy vma_is_anonymous(), which results in VMAs that were mmap()'d referencing a file, but are in every other sense anonymous, including the folios. 4. Other MAP_PRIVATE-file backed mappings - These possess file-backed VMAs and have file-backed folios until CoW'd, at which point those CoW'd folios are anonymous. This series fixes issues 3 and 4. In order for us to traverse VMAs using the reverse mapping, we require two fields - folio->mapping and folio->index. The first tells the rmap code where to look for VMAs, and the second tells it at which offset the folio starts within the referenced object. For anonymous folios, folio->mapping points at an anon_vma object. For file-backed folios, it points at an address_space. And: * For file-backed folios folio->index is simply the page offset of the start of the folio within the file. * For anonymous folios belonging to pure anon mappings, folio->index is equal to the virtual page offset of the folio. * For anonymous folios belonging to file-backed mappings (i.e. CoW'd folios of a MAP_PRIVATE file-backed mapping), folio->index is equal to the file page offset. This series establishes a new virtual page offset property of VMAs to allow us to map anonymous folios at their virtual page offset, consistent with pure anon. The purpose of doing so is to lay the foundations for the scalable CoW work. This is necessary because scalable CoW looks in the maple tree for the VMA located at folio->index << PAGE_SHIFT, before falling back to looking up tracked remaps if necessary. The MAP_PRIVATE file-backed case means that folio indices will very often conflict with one another and this remap tracking becomes substantially more contended, and of course the fast path can never be used. This also makes it possible, in future, to unshare anonymously mapped folios with deep fork hierarchies on remap, eliminating the need for remap tracking in the vast majority of cases. Similar to page offset of pure anonymous VMAs, we update the virtual page offset of unfaulted file-backed VMAs on remap, but do not once CoW'd (i.e. vma->anon_vma is non-NULL). Overall, there is little impact on mergeability - for shared file-backed mappings, we treat the anonymous page offset as equal to the file-backed one (via vma_start_anon_pgoff()), so merge behaviour remains the same there. The only impact is on MAP_PRIVATE-mapped file-backed mappings, which must now match on virtual page offset as well as file page offset to be merged. To fail to merge like this would require CoW'ing the mapping, then finding another VMA with identical file and compatible page offset to remap next to. This is therefore very much an edge case that should have very little impact (and which scalable CoW may very well address in any case). We also address the outlier case of MAP_PRIVATE-/dev/zero mappings which have file page offset but satisfy vma_is_anonymous() by making them truly anonymous. This is low-risk as for anything meaningful these mappings behave anonymously, but it is very beneficial to do so as we eliminate an odd corner case, and those are notorious for attracting subtle bugs. This patch (of 15): This patch establishes fields within the vm_area_struct type to store the virtual page offset of VMAs. The virtual page offset of a VMA is equal to vma->vm_start >> PAGE_SHIFT if they are unfaulted or were not remapped, otherwise it is equal to this value at the point of first fault. Currently, anonymous folios belonging to CoW'd MAP_PRIVATE-mapped file-backed VMAs are tracked by their file offset. By adding virtual offset as a property of VMAs, we can now track them by their virtual page offset instead. By tracking this, we provide the means by which to eliminate this inconsistency, and more importantly lay the foundations for future work for the scalable CoW anonymous rmap rework. This patch simply adds the fields and some simple helpers. Subsequent patches will update mm code to make use of these fields correctly. The fields chosen are packed in the VMA such that, for 64-bit kernel builds, no additional space is taken up. The first field is present on cacheline 0 containing key VMA fields, and the second on cacheline 3, which contains file-backed reverse mapping fields. Given the relative time spent accessing reverse mapping fields as well as updating them, there shouldn't be any performance impact here from false sharing. Update the VMA userland tests to account for this change. No callsites are updated yet, so no functional change intended. Link: https://lore.kernel.org/20260720-b4-scalable-cow-virt-pgoff-v2-0-2d549757a76f@kernel.org Link: https://lore.kernel.org/20260720-b4-scalable-cow-virt-pgoff-v2-1-2d549757a76f@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Cc: Alistair Popple Cc: Arnd Bergmann Cc: Baolin Wang Cc: Barry Song Cc: Byungchul Park Cc: Chengming Zhou Cc: David Hildenbrand Cc: Dev Jain Cc: Greg Kroah-Hartman Cc: Gregory Price Cc: Harry Yoo Cc: "Huang, Ying" Cc: Jan Kara Cc: Jann Horn Cc: Joshua Hahn Cc: Kees Cook Cc: Lance Yang Cc: Liam R. Howlett Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Naoya Horiguchi Cc: Nico Pache Cc: Pedro Falcato Cc: Peter Xu Cc: Rakie Kim Cc: Rik van Riel Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton --- include/linux/mm.h | 59 ++++++++++++++++++++++++++++++ include/linux/mm_types.h | 12 ++++++ mm/vma.h | 14 +++++++ mm/vma_init.c | 1 tools/testing/vma/include/dup.h | 26 +++++++++++++ 5 files changed, 112 insertions(+) --- a/include/linux/mm.h~mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers +++ a/include/linux/mm.h @@ -4393,6 +4393,65 @@ static inline pgoff_t vma_last_pgoff(con return vma_end_pgoff(vma) - 1; } +/** + * vma_start_virt_pgoff() - Get the virtual page offset of the start of @vma + * @vma: The VMA whose virtual page offset is required. + * + * If unfaulted, then this is vma->vm_start >> PAGE_SHIFT, if faulted then the + * virtual page offset at the time of first fault. + * + * If the VMA is anonymous, this returns the same value as vma_start_pgoff(). + * + * This value is used for tracking MAP_PRIVATE file-backed mappings by their + * virtual page offset. + * + * Returns: The virtual page offset of the start of @vma. + */ +static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma) +{ + pgoff_t pgoff = 0; + +#ifdef CONFIG_64BIT + pgoff += vma->__vm_virt_pgoff_hi; + pgoff <<= 32; +#endif + pgoff += vma->__vm_virt_pgoff_lo; + return pgoff; +} + +/** + * vma_end_virt_pgoff() - Get the virtual page offset of the exclusive end of + * @vma. + * @vma: The VMA whose end virtual page offset is required. + * + * This returns the virtual exclusive end page offset of @vma, which is useful + * for expressing page offset ranges. + * + * See the description of vma_start_virt_pgoff() for a description of VMA + * virtual page offsets. + * + * Returns: The exclusive end virtual page offset of @vma. + */ +static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma) +{ + return vma_start_virt_pgoff(vma) + vma_pages(vma); +} + +/** + * vma_last_virt_pgoff() - Get the virtual page offset of the last page in + * @vma. + * @vma: The VMA whose last virtual page offset is required. + * + * See the description of vma_start_virt_pgoff() for a description of VMA + * virtual page offsets. + * + * Returns: The last virtual page offset of @vma. + */ +static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma) +{ + return vma_end_virt_pgoff(vma) - 1; +} + static inline unsigned long vma_desc_size(const struct vm_area_desc *desc) { return desc->end - desc->start; --- a/include/linux/mm_types.h~mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers +++ a/include/linux/mm_types.h @@ -968,6 +968,11 @@ struct vm_area_struct { unsigned int vm_lock_seq; #endif /* + * Low 32-bits of virtual page offset. + * See vma_start_virt_pgoff() comment for details. + */ + unsigned int __vm_virt_pgoff_lo; + /* * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma * list, after a COW of one of the file pages. A MAP_SHARED vma * can only be in the i_mmap tree. An anonymous MAP_PRIVATE, stack @@ -1042,6 +1047,13 @@ struct vm_area_struct { struct lockdep_map vmlock_dep_map; #endif #endif +#ifdef CONFIG_64BIT + /* + * High 32-bits of virtual page offset. + * See vma_start_virt_pgoff() comment for details. + */ + unsigned int __vm_virt_pgoff_hi; +#endif /* * For areas with an address space and backing store, * linkage into the address_space->i_mmap interval tree. --- a/mm/vma.h~mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers +++ a/mm/vma.h @@ -283,6 +283,20 @@ static inline void vma_set_pgoff(struct vma->vm_pgoff = pgoff; } +static inline void __vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff) +{ +#ifdef CONFIG_64BIT + vma->__vm_virt_pgoff_hi = pgoff >> 32; +#endif + vma->__vm_virt_pgoff_lo = pgoff & GENMASK(31, 0); +} + +static inline void vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff) +{ + vma_assert_can_modify(vma); + __vma_set_virt_pgoff(vma, pgoff); +} + static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta) { vma_assert_can_modify(vma); --- a/mm/vma_init.c~mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers +++ a/mm/vma_init.c @@ -51,6 +51,7 @@ static void vm_area_init_from(const stru dest->vm_end = src->vm_end; dest->anon_vma = src->anon_vma; dest->vm_pgoff = vma_start_pgoff(src); + __vma_set_virt_pgoff(dest, vma_start_virt_pgoff(src)); dest->vm_file = src->vm_file; dest->vm_private_data = src->vm_private_data; vm_flags_init(dest, src->vm_flags); --- a/tools/testing/vma/include/dup.h~mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers +++ a/tools/testing/vma/include/dup.h @@ -577,6 +577,7 @@ struct vm_area_struct { */ unsigned int vm_lock_seq; #endif + unsigned int __vm_virt_pgoff_lo; /* * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma @@ -613,6 +614,9 @@ struct vm_area_struct { /* Unstable RCU readers are allowed to read this. */ refcount_t vm_refcnt; #endif +#ifdef CONFIG_64BIT + unsigned int __vm_virt_pgoff_hi; +#endif /* * For areas with an address space and backing store, * linkage into the address_space->i_mmap interval tree. @@ -1320,6 +1324,28 @@ static inline pgoff_t vma_end_pgoff(cons return vma_start_pgoff(vma) + vma_pages(vma); } +static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma) +{ + pgoff_t pgoff = 0; + +#ifdef CONFIG_64BIT + pgoff += vma->__vm_virt_pgoff_hi; + pgoff <<= 32; +#endif + pgoff += vma->__vm_virt_pgoff_lo; + return pgoff; +} + +static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma) +{ + return vma_start_virt_pgoff(vma) + vma_pages(vma); +} + +static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma) +{ + return vma_end_virt_pgoff(vma) - 1; +} + static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc) { return file->f_op->mmap_prepare(desc); _ Patches currently in -mm which might be from ljs@kernel.org are mm-vmalloc-acquire-init_mm-lock-on-huge-vmap-to-avoid-ptdump-uaf.patch x86-mm-pat-acquire-init_mm-write-lock-on-collapse-to-avoid-uaf.patch x86-mm-pat-acquire-init_mm-read-lock-on-attribute-change-to-avoid-uaf.patch mm-ptdump-always-stabilise-against-page-table-freeing-using-init_mm.patch arm64-remove-redundant-concurrent-ptdump-uaf-mitigation.patch mm-move-alloc-tag-to-mm.patch mm-move-vma_start_pgoff-into-mmh-and-clean-up.patch mm-add-kdoc-comments-for-vma_start-last_pgoff.patch tools-testing-vma-use-vma_start_pgoff-in-merge-tests.patch mm-introduce-and-use-vma_end_pgoff.patch mm-rmap-update-mm-interval_treec-comments.patch mm-rmap-parameterise-vma_interval_tree_-by-address_space.patch mm-rmap-elide-unnecessary-static-inlines-in-interval_treec.patch mm-rmap-rename-vma_interval_tree_-to-mapping_rmap_tree_.patch mm-rmap-parameterise-anon_vma_interval_tree_-by-anon_vma.patch mm-rmap-rename-anon_vma_interval_tree_-params-and-use-pgoff_t.patch mm-rmap-rename-anon_vma_interval_tree_-to-anon_rmap_tree_.patch maintainers-move-mm-interval_treec-to-rmap-section.patch mm-vma-introduce-and-use-vmg_pages-vmg__pgoff.patch mm-vma-clean-up-anon_vma_compatible.patch mm-vma-refactor-vmg_adjust_set_range-for-clarity.patch mm-vma-minor-cleanup-of-expand_.patch mm-introduce-and-use-linear_page_delta.patch mm-vma-use-vma_start_pgoff-linear_page_index-in-mm-code.patch mm-prefer-vma__pgoff-to-vma-vm_pgoff-in-kernel.patch mm-vma-remove-duplicative-vma_pgoff_offset-helper.patch mm-use-linear_page_-consistently.patch mm-vma-introduce-vma_assert_can_modify.patch mm-vma-add-and-use-vma__pgoff.patch mm-vma-move-__install_special_mapping-to-vmac.patch mm-vma-make-vma_set_range-static-drop-insert_vm_struct-decl.patch mm-vma-update-vma_shrink-to-not-pass-start-pgoff-parameters.patch mm-vma-update-vmg_adjust_set_range-to-offset-pgoff-instead.patch mm-vma-slightly-rework-the-anonymous-check-in-__mmap_new_vma.patch mm-vma-introduce-and-use-vma_set_pgoff.patch mm-vma-correct-incorrect-vmah-inclusion.patch mm-vma-use-guard-clauses-in-can_vma_merge_.patch tools-testing-vma-default-vma-mm-flag-bits-to-64-bit.patch tools-testing-vma-output-compared-expression-on-assert_.patch mm-introduce-vma_flags_can_grow-and-vma_can_grow.patch mm-vma-update-do_mmap-to-use-vma_flags_t.patch mm-convert-__get_unmapped_area-to-use-vma_flags_t.patch mm-update-generic_get_unmapped_area-to-use-vma_flags_t.patch mm-prefer-mm-def_vma_flags-in-mm-logic.patch mm-vma-convert-vm_pgprot_modify-to-use-vma_flags_t-and-rename.patch mm-vma-rename-vma_get_page_prot-to-vma_flags_to_page_prot.patch mm-introduce-vma_get_page_prot-and-use-it.patch mm-vma-update-create_init_stack_vma-to-use-vma_flags_t.patch mm-vma-convert-miscellaneous-uses-of-vma-flags-in-core-mm.patch mm-mlock-convert-mlock-code-to-use-vma_flags_t.patch mm-mprotect-convert-mprotect-code-to-use-vma_flags_t.patch mm-mremap-convert-mremap-code-to-use-vma_flags_t.patch mm-mseal-remove-superfluous-comments-fix-confusion-around-mm.patch mm-mseal-limit-scope-of-mseal-address-zero-to-address-zero.patch mm-mseal-remove-further-superfluous-comments-do_mseal.patch mm-vma-introduce-vma-virtual-page-offset-field-and-add-helpers.patch mm-introduce-linear_virt_page_index.patch mm-abstract-vma_address-and-introduce-vma_anon_address.patch mm-update-print_bad_page_map-to-show-virtual-page-index.patch mm-introduce-and-use-vma_filebacked_address.patch mm-propagate-vma-virtual-page-offset-on-map-remap-split-merge.patch mm-rmap-track-whether-the-page-vma-mapped-walk-is-anonymous.patch mm-introduce-and-use-linear_folio_page_index.patch mm-rmap-use-virt-pgoff-for-map_private-file-backed-anon-folios.patch tools-testing-vma-expand-vma-merge-tests-to-assert-virt-pgoff.patch tools-testing-selftests-mm-test-virtual-page-offset-merge-behaviour.patch mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous.patch mm-vma-make-map_private-mapped-dev-zero-mappings-truly-anonymous.patch tools-testing-vma-add-test-to-assert-map_private-dev-zero-is-anon.patch tools-testing-selftests-mm-add-map_private-dev-zero-merge-tests.patch