Re: [PATCH v4 00/20] mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff

[Andrew Morton <akpm@linux-foundation.org>]

On Thu, 06 Aug 2026 21:21:25 +0100 "Lorenzo Stoakes (ARM)" <ljs@kernel.org> wrote:

> In memory management we've managed to manufacture a great deal of confusion
> around the concept of anonymous memory. We have:
> 
> 1. 'Pure anon' memory - anonymous VMAs whose folios are anonymous and
>    swap-backed (thus for reclaim purposes, treated as anonymous). These are
>    simple enough.
> 
> 2. shmem - file-backed VMAs, file-backed folios (from rmap perspective) so
>    present in the page cache and mapped by an address_space object, but
>    whose folios are also swap-backed (thus treated as anonymous for reclaim
>    purposes).
> 
> 3. MAP_PRIVATE-mapped /dev/zero - a strange beast whose VMAs have
>    vma->vm_file set, but which clears vma->vm_ops to satisfy
>    vma_is_anonymous(), resulting in VMAs that were mmap()'d referencing a
>    file, but are in every other sense anonymous, including the folios.
> 
> 4. Other MAP_PRIVATE-file backed mappings - These possess file-backed VMAs
>    and have file-backed folios until CoW'd, at which point those CoW'd
>    folios are anonymous.
> 
> This series fixes issues 3 and 4.

Thanks, I've updated mm.git's mm-unstable branch to this version.

It's about 450 patches away from head-of-queue, so it will be a
second-week-of-merge-window thing.  Three weeks to go.

> v4:
> - Updated tags (thanks everyone!)
> - Adjusted some prose as per David.
> - Adjusted whitespace to 2 tabs in 2/15 as per David.
> - Improved linear_page_index(), __linear_anon_page_index() to be more
>   succinct in 2/15 as per David and updated commit message to reflect it.
> - Added commit to provide vma_[flags_]is_cow_mapping() - nearly all callers
>   are calling is_cow_mapping() as is_cow_mapping(vma->vm_flags) so just
>   provide a helper to do this for them.
> - In the same commit update all is_cow_mapping() callers and remove the
>   now-unused function.
> - Change linear_anon_page_index() to assert on !CoW mapping using
>   vma_is_cow_mapping() in 2/15 as per David.
> - Updated 4/15 to output index differently depending on whether the VMA is
>   a CoW mapping or not or whether the file index differs from the anon
>   index as per Gregory.
> - Updated 6/15 to change the 'update page offset' logic in copy_vma() to
>   not be gated on CoW or VMA_SHARED_BIT as discussed with David.
> - Updated 6/15 to improve the 'faulted in anon vma' assert stuff. It was
>   very unclear so rename the variable sensibly and update the comment.
> - Added a separate commit to fix the mess that is the faulted_in_anon_vma
>   and the VM_WARN_ON_ONCE_VMA() assert in copy_vma() - make it actually
>   only update the vmap in cases where the VMA was replaced (backwards
>   remap), update the checks to reflect this in a way that's actually
>   understandable and improve the comment.
> - Updated needs_adjacent_anon_gpoff() in 9/15 to use
>   vma_flags_is_cow_mapping() as per David. Updated the comment to reflect
>   it.
> - Broke out changes to vma_address_end() into a separate patch as per David.
> - Eliminated pgoff in vma_address_end() as it adds confusion - just use
>   pgoff_end, which was what pgoff used to be (confusingly).
> - Placed more variables in vma_address_end() as constants at the start of
>   the function.
> - Dropped KSM comment in 9/15 as per David.
> - Dropped linear_folio_page_index() patch altogether as per
>   David.
> - Realised all off uffd is only anon so use linear_anon_page_index()
>   throughout there and also the huge memory case for the same reason.
> - Added a patch to make remove_migration_pmd() accept a folio instead of a
>   page.
> - Added a patch to calculate large folio index using PFN as suggested by
>   David, eliminating the need for linear page index lookup at all.
> - Added a patch to add self merge VMA userland tests.

Well this grew up.  Here's how v4 altered mm.git:


 arch/s390/mm/gmap_helpers.c                            |    2 
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c                |    4 
 drivers/gpu/drm/drm_gem_shmem_helper.c                 |    2 
 drivers/gpu/drm/panthor/panthor_gem.c                  |    2 
 drivers/gpu/drm/ttm/ttm_bo_vm.c                        |    2 
 drivers/gpu/drm/xe/xe_device.c                         |    2 
 fs/proc/task_mmu.c                                     |    2 
 include/linux/mm.h                                     |   61 ++---
 include/linux/mm_types.h                               |   12 -
 include/linux/pagemap.h                                |   61 +----
 include/linux/rmap.h                                   |    6 
 include/linux/swapops.h                                |    6 
 kernel/events/uprobes.c                                |    2 
 mm/gup.c                                               |    2 
 mm/huge_memory.c                                       |   27 +-
 mm/hugetlb.c                                           |    2 
 mm/internal.h                                          |   45 +--
 mm/interval_tree.c                                     |    4 
 mm/ksm.c                                               |    7 
 mm/memory.c                                            |   40 +--
 mm/mempolicy.c                                         |    2 
 mm/migrate.c                                           |   23 -
 mm/mremap.c                                            |    6 
 mm/rmap.c                                              |    8 
 mm/userfaultfd.c                                       |    6 
 mm/vma.c                                               |  118 +++++-----
 mm/vma.h                                               |   70 -----
 mm/vma_init.c                                          |    2 
 tools/testing/selftests/mm/merge.c                     |   10 
 tools/testing/selftests/proc/proc-self-map-files-001.c |    2 
 tools/testing/selftests/proc/proc-self-map-files-002.c |    2 
 tools/testing/vma/include/dup.h                        |   41 ++-
 tools/testing/vma/shared.c                             |    2 
 tools/testing/vma/tests/merge.c                        |   59 ++---
 tools/testing/vma/tests/mmap.c                         |    6 
 tools/testing/vma/tests/vma.c                          |   46 +++
 36 files changed, 337 insertions(+), 357 deletions(-)

--- a/arch/s390/mm/gmap_helpers.c~b
+++ a/arch/s390/mm/gmap_helpers.c
@@ -200,7 +200,7 @@ static int find_zeropage_pte_entry(pte_t
 		 * currently only works in COW mappings, which is also where
 		 * mm_forbids_zeropage() is checked.
 		 */
-		if (!is_cow_mapping(walk->vma->vm_flags))
+		if (!vma_is_cow_mapping(walk->vma))
 			return -EFAULT;
 
 		*found_addr = addr;
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c~b
+++ a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -377,9 +377,9 @@ static int amdgpu_gem_object_mmap(struct
 	/* Workaround for Thunk bug creating PROT_NONE,MAP_PRIVATE mappings
 	 * for debugger access to invisible VRAM. Should have used MAP_SHARED
 	 * instead. Clearing VM_MAYWRITE prevents the mapping from ever
-	 * becoming writable and makes is_cow_mapping(vm_flags) false.
+	 * becoming writable and makes vma_is_cow_mapping(vma) false.
 	 */
-	if (is_cow_mapping(vma->vm_flags) &&
+	if (vma_is_cow_mapping(vma) &&
 	    !(vma->vm_flags & VM_ACCESS_FLAGS))
 		vm_flags_clear(vma, VM_MAYWRITE);
 
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c~b
+++ a/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -753,7 +753,7 @@ int drm_gem_shmem_mmap(struct drm_gem_sh
 		return ret;
 	}
 
-	if (is_cow_mapping(vma->vm_flags))
+	if (vma_is_cow_mapping(vma))
 		return -EINVAL;
 
 	dma_resv_lock(shmem->base.resv, NULL);
--- a/drivers/gpu/drm/panthor/panthor_gem.c~b
+++ a/drivers/gpu/drm/panthor/panthor_gem.c
@@ -761,7 +761,7 @@ static int panthor_gem_mmap(struct drm_g
 		return ret;
 	}
 
-	if (is_cow_mapping(vma->vm_flags))
+	if (vma_is_cow_mapping(vma))
 		return -EINVAL;
 
 	if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) {
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c~b
+++ a/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -489,7 +489,7 @@ static const struct vm_operations_struct
 int ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo)
 {
 	/* Enforce no COW since would have really strange behavior with it. */
-	if (is_cow_mapping(vma->vm_flags))
+	if (vma_is_cow_mapping(vma))
 		return -EINVAL;
 
 	drm_gem_object_get(&bo->base);
--- a/drivers/gpu/drm/xe/xe_device.c~b
+++ a/drivers/gpu/drm/xe/xe_device.c
@@ -330,7 +330,7 @@ static int xe_pci_barrier_mmap(struct fi
 	if (vma->vm_end - vma->vm_start > SZ_4K)
 		return -EINVAL;
 
-	if (is_cow_mapping(vma->vm_flags))
+	if (vma_is_cow_mapping(vma))
 		return -EINVAL;
 
 	if (vma->vm_flags & (VM_READ | VM_EXEC))
--- a/fs/proc/task_mmu.c~b
+++ a/fs/proc/task_mmu.c
@@ -1693,7 +1693,7 @@ static inline bool pte_is_pinned(struct
 
 	if (!pte_write(pte))
 		return false;
-	if (!is_cow_mapping(vma->vm_flags))
+	if (!vma_is_cow_mapping(vma))
 		return false;
 	if (likely(!mm_flags_test(MMF_HAS_PINNED, vma->vm_mm)))
 		return false;
--- a/include/linux/mm.h~b
+++ a/include/linux/mm.h
@@ -2269,17 +2269,20 @@ void unpin_user_pages(struct page **page
 void unpin_user_folio(struct folio *folio, unsigned long npages);
 void unpin_folios(struct folio **folios, unsigned long nfolios);
 
-static inline bool is_cow_mapping(vm_flags_t flags)
+static inline bool vma_flags_is_cow_mapping(const vma_flags_t *flags)
 {
-	return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE;
+	return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
+		!vma_flags_test(flags, VMA_SHARED_BIT);
 }
 
-static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
+static inline bool vma_is_cow_mapping(const struct vm_area_struct *vma)
 {
-	const vma_flags_t *flags = &desc->vma_flags;
+	return vma_flags_is_cow_mapping(&vma->flags);
+}
 
-	return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
-		!vma_flags_test(flags, VMA_SHARED_BIT);
+static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
+{
+	return vma_flags_is_cow_mapping(&desc->vma_flags);
 }
 
 #ifndef CONFIG_MMU
@@ -4391,62 +4394,62 @@ static inline pgoff_t vma_last_pgoff(con
 }
 
 /**
- * vma_start_virt_pgoff() - Get the virtual page offset of the start of @vma
- * @vma: The VMA whose virtual page offset is required.
+ * vma_start_anon_pgoff() - Get the anonymous page offset of the start of @vma
+ * @vma: The VMA whose anonymous page offset is required.
  *
  * If unfaulted, then this is vma->vm_start >> PAGE_SHIFT, if faulted then the
- * virtual page offset at the time of first fault.
+ * anonymous page offset at the time of first fault.
  *
  * If the VMA is anonymous, this returns the same value as vma_start_pgoff().
  *
  * This value is used for tracking MAP_PRIVATE file-backed mappings by their
- * virtual page offset.
+ * anonymous page offset.
  *
- * Returns: The virtual page offset of the start of @vma.
+ * Returns: The anonymous page offset of the start of @vma.
  */
-static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
 {
 	pgoff_t pgoff = 0;
 
 #ifdef CONFIG_64BIT
-	pgoff += vma->__vm_virt_pgoff_hi;
+	pgoff += vma->__vm_anon_pgoff_hi;
 	pgoff <<= 32;
 #endif
-	pgoff += vma->__vm_virt_pgoff_lo;
+	pgoff += vma->__vm_anon_pgoff_lo;
 	return pgoff;
 }
 
 /**
- * vma_end_virt_pgoff() - Get the virtual page offset of the exclusive end of
+ * vma_end_anon_pgoff() - Get the anonymous page offset of the exclusive end of
  * @vma.
- * @vma: The VMA whose end virtual page offset is required.
+ * @vma: The VMA whose end anonymous page offset is required.
  *
- * This returns the virtual exclusive end page offset of @vma, which is useful
+ * This returns the anonymous exclusive end page offset of @vma, which is useful
  * for expressing page offset ranges.
  *
- * See the description of vma_start_virt_pgoff() for a description of VMA
- * virtual page offsets.
+ * See the description of vma_start_anon_pgoff() for a description of VMA
+ * anonymous page offsets.
  *
- * Returns: The exclusive end virtual page offset of @vma.
+ * Returns: The exclusive end anonymous page offset of @vma.
  */
-static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
 {
-	return vma_start_virt_pgoff(vma) + vma_pages(vma);
+	return vma_start_anon_pgoff(vma) + vma_pages(vma);
 }
 
 /**
- * vma_last_virt_pgoff() - Get the virtual page offset of the last page in
+ * vma_last_anon_pgoff() - Get the anonymous page offset of the last page in
  * @vma.
- * @vma: The VMA whose last virtual page offset is required.
+ * @vma: The VMA whose last anonymous page offset is required.
  *
- * See the description of vma_start_virt_pgoff() for a description of VMA
- * virtual page offsets.
+ * See the description of vma_start_anon_pgoff() for a description of VMA
+ * anonymous page offsets.
  *
- * Returns: The last virtual page offset of @vma.
+ * Returns: The last anonymous page offset of @vma.
  */
-static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_last_anon_pgoff(const struct vm_area_struct *vma)
 {
-	return vma_end_virt_pgoff(vma) - 1;
+	return vma_end_anon_pgoff(vma) - 1;
 }
 
 static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)
--- a/include/linux/mm_types.h~b
+++ a/include/linux/mm_types.h
@@ -968,10 +968,10 @@ struct vm_area_struct {
 	unsigned int vm_lock_seq;
 #endif
 	/*
-	 * Low 32-bits of virtual page offset.
-	 * See vma_start_virt_pgoff() comment for details.
+	 * Low 32-bits of anonymous page offset.
+	 * See vma_start_anon_pgoff() comment for details.
 	 */
-	unsigned int __vm_virt_pgoff_lo;
+	unsigned int __vm_anon_pgoff_lo;
 	/*
 	 * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
 	 * list, after a COW of one of the file pages.	A MAP_SHARED vma
@@ -1049,10 +1049,10 @@ struct vm_area_struct {
 #endif
 #ifdef CONFIG_64BIT
 	/*
-	 * High 32-bits of virtual page offset.
-	 * See vma_start_virt_pgoff() comment for details.
+	 * High 32-bits of anonymous page offset.
+	 * See vma_start_anon_pgoff() comment for details.
 	 */
-	unsigned int __vm_virt_pgoff_hi;
+	unsigned int __vm_anon_pgoff_hi;
 #endif
 	/*
 	 * For areas with an address space and backing store,
--- a/include/linux/pagemap.h~b
+++ a/include/linux/pagemap.h
@@ -1094,79 +1094,46 @@ static inline pgoff_t linear_page_delta(
 static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,
 					const unsigned long address)
 {
-	pgoff_t pgoff;
-
-	pgoff = linear_page_delta(vma, address);
-	pgoff += vma_start_pgoff(vma);
-	return pgoff;
+	return linear_page_delta(vma, address) + vma_start_pgoff(vma);
 }
 
-static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,
-					       const unsigned long address)
+static inline pgoff_t __linear_anon_page_index(const struct vm_area_struct *vma,
+		const unsigned long address)
 {
-	pgoff_t pgoff;
-
-	pgoff = linear_page_delta(vma, address);
-	pgoff += vma_start_virt_pgoff(vma);
-	return pgoff;
+	return linear_page_delta(vma, address) + vma_start_anon_pgoff(vma);
 }
 
 /**
- * linear_virt_page_index() - Determine the absolute virtual page offset of
+ * linear_anon_page_index() - Determine the absolute anonymous page offset of
  * @address within @vma.
  * @vma: An anonymous or MAP_PRIVATE file-backed VMA in which @address resides.
  * @address: The address whose absolute page offset is required.
  *
- * This returns the virtual page offset of @address, which is the page offset
+ * This returns the anonymous page offset of @address, which is the page offset
  * the address possessed at the time the VMA was first faulted.
  *
  * For anonymous mappings, this returns the same value as linear_page_index().
  *
- * For MAP_PRIVATE file-backed mappings, this returns the virtual page offset of
- * @address, which is the page offset the address possessed at the time the VMA
- * was first faulted.
+ * For MAP_PRIVATE file-backed mappings, this returns the anonymous page offset
+ * of @address, which is the page offset the address possessed at the time the
+ * VMA was first faulted.
  *
  * It is not valid to call this function for shared file-backed mappings.
  *
- * Returns: The absolute virtual page offset of @address within @vma.
+ * Returns: The absolute anonymous page offset of @address within @vma.
  */
-static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,
-					     const unsigned long address)
+static inline pgoff_t linear_anon_page_index(const struct vm_area_struct *vma,
+		const unsigned long address)
 {
-	const pgoff_t pgoff = __linear_virt_page_index(vma, address);
+	const pgoff_t pgoff = __linear_anon_page_index(vma, address);
 
-	VM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));
+	VM_WARN_ON_ONCE(!vma_is_cow_mapping(vma));
 	if (vma_is_anonymous(vma))
 		VM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));
 
 	return pgoff;
 }
 
-/**
- * linear_folio_page_index() - Determine the absolute page offset of
- * @address within @vma from @folio.
- * @folio: The folio whose linear page index is sought.
- * @vma: The VMA in which @address resides.
- * @address: The address whose absolute page offset is required.
- *
- * Determines whether to obtain the virtual linear page index based on whether
- * @folio is anonymous or not.
- *
- * See the descriptions of linear_virt_page_index() and linear_page_index() for
- * details of each.
- *
- * Returns: The absolute page offset of @address within @vma.
- */
-static inline pgoff_t linear_folio_page_index(const struct folio *folio,
-					      const struct vm_area_struct *vma,
-					      const unsigned long address)
-{
-	if (folio_test_anon(folio))
-		return linear_virt_page_index(vma, address);
-
-	return linear_page_index(vma, address);
-}
-
 struct wait_page_key {
 	struct folio *folio;
 	int bit_nr;
--- a/include/linux/rmap.h~b
+++ a/include/linux/rmap.h
@@ -864,14 +864,14 @@ struct page *make_device_exclusive(struc
 struct page_vma_mapped_walk {
 	unsigned long pfn;
 	unsigned long nr_pages;
-	pgoff_t pgoff;
+	pgoff_t pgoff;	/* Only meaningful if nr_pages > 1 and not a KSM walk */
 	struct vm_area_struct *vma;
 	unsigned long address;
 	pmd_t *pmd;
 	pte_t *pte;
 	spinlock_t *ptl;
 	unsigned int flags;
-	bool is_anon_walk;
+	bool pgoff_is_anon : 1;
 };
 
 #define DEFINE_FOLIO_VMA_WALK(name, _folio, _vma, _address, _flags)	\
@@ -882,7 +882,7 @@ struct page_vma_mapped_walk {
 		.vma = _vma,						\
 		.address = _address,					\
 		.flags = _flags,					\
-		.is_anon_walk = folio_test_anon(_folio),		\
+		.pgoff_is_anon = folio_test_anon(_folio),		\
 	}
 
 static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)
--- a/include/linux/swapops.h~b
+++ a/include/linux/swapops.h
@@ -325,8 +325,8 @@ struct page_vma_mapped_walk;
 extern int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
 		struct page *page);
 
-extern void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
-		struct page *new);
+void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
+		struct folio *folio);
 
 extern void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd);
 
@@ -346,7 +346,7 @@ static inline int set_pmd_migration_entr
 }
 
 static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
-		struct page *new)
+		struct folio *folio)
 {
 	BUILD_BUG();
 }
--- a/kernel/events/uprobes.c~b
+++ a/kernel/events/uprobes.c
@@ -513,7 +513,7 @@ int uprobe_write(struct arch_uprobe *aup
 
 	uprobe = container_of(auprobe, struct uprobe, arch);
 
-	if (WARN_ON_ONCE(!is_cow_mapping(vma->vm_flags)))
+	if (WARN_ON_ONCE(!vma_is_cow_mapping(vma)))
 		return -EINVAL;
 
 	/*
--- a/mm/gup.c~b
+++ a/mm/gup.c
@@ -1236,7 +1236,7 @@ static int check_vma_flags(struct vm_are
 			 * Anon pages in shared mappings are surprising: now
 			 * just reject it.
 			 */
-			if (!is_cow_mapping(vm_flags))
+			if (!vma_is_cow_mapping(vma))
 				return -EFAULT;
 		}
 	} else if (!(vm_flags & VM_READ)) {
--- a/mm/huge_memory.c~b
+++ a/mm/huge_memory.c
@@ -1681,7 +1681,7 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_
 	BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)));
 	BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) ==
 						(VM_PFNMAP|VM_MIXEDMAP));
-	BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags));
+	BUG_ON((vma->vm_flags & VM_PFNMAP) && vma_is_cow_mapping(vma));
 
 	pfnmap_setup_cachemode_pfn(pfn, &pgprot);
 
@@ -1789,7 +1789,7 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_
 	BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)));
 	BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) ==
 						(VM_PFNMAP|VM_MIXEDMAP));
-	BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags));
+	BUG_ON((vma->vm_flags & VM_PFNMAP) && vma_is_cow_mapping(vma));
 
 	pfnmap_setup_cachemode_pfn(pfn, &pgprot);
 
@@ -1931,7 +1931,7 @@ int copy_huge_pmd(struct mm_struct *dst_
 		 * applied special bit, or we made the PRIVATE mapping be
 		 * able to wrongly write to the backend MMIO.
 		 */
-		VM_WARN_ON_ONCE(is_cow_mapping(src_vma->vm_flags) && pmd_write(pmd));
+		VM_WARN_ON_ONCE(vma_is_cow_mapping(src_vma) && pmd_write(pmd));
 		goto set_pmd;
 	}
 
@@ -2052,7 +2052,7 @@ int copy_huge_pud(struct mm_struct *dst_
 	 * TODO: once we support anonymous pages, use
 	 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
 	 */
-	if (is_cow_mapping(vma->vm_flags) && pud_write(pud)) {
+	if (vma_is_cow_mapping(vma) && pud_write(pud)) {
 		pudp_set_wrprotect(src_mm, addr, src_pud);
 		pud = pud_wrprotect(pud);
 	}
@@ -2930,8 +2930,7 @@ int move_pages_huge_pmd(struct mm_struct
 		}
 
 		folio_move_anon_rmap(src_folio, dst_vma);
-		src_folio->index = linear_folio_page_index(src_folio, dst_vma,
-							   dst_addr);
+		src_folio->index = linear_anon_page_index(dst_vma, dst_addr);
 
 		_dst_pmd = folio_mk_pmd(src_folio, dst_vma->vm_page_prot);
 		/* Follow mremap() behavior and treat the entry dirty after the move */
@@ -5078,9 +5077,8 @@ int set_pmd_migration_entry(struct page_
 	return 0;
 }
 
-void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
+void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct folio *folio)
 {
-	struct folio *folio = page_folio(new);
 	struct vm_area_struct *vma = pvmw->vma;
 	struct mm_struct *mm = vma->vm_mm;
 	unsigned long address = pvmw->address;
@@ -5116,11 +5114,9 @@ void remove_migration_pmd(struct page_vm
 		swp_entry_t entry;
 
 		if (pmd_write(pmde))
-			entry = make_writable_device_private_entry(
-							page_to_pfn(new));
+			entry = make_writable_device_private_entry(folio_pfn(folio));
 		else
-			entry = make_readable_device_private_entry(
-							page_to_pfn(new));
+			entry = make_readable_device_private_entry(folio_pfn(folio));
 		pmde = softleaf_to_pmd(entry);
 
 		if (pmd_swp_soft_dirty(*pvmw->pmd))
@@ -5135,11 +5131,12 @@ void remove_migration_pmd(struct page_vm
 		if (!softleaf_is_migration_read(entry))
 			rmap_flags |= RMAP_EXCLUSIVE;
 
-		folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags);
+		folio_add_anon_rmap_pmd(folio, &folio->page, vma, haddr, rmap_flags);
 	} else {
-		folio_add_file_rmap_pmd(folio, new, vma);
+		folio_add_file_rmap_pmd(folio, &folio->page, vma);
 	}
-	VM_BUG_ON(pmd_write(pmde) && folio_test_anon(folio) && !PageAnonExclusive(new));
+	VM_WARN_ON_ONCE(pmd_write(pmde) && folio_test_anon(folio) &&
+			!PageAnonExclusive(&folio->page));
 	set_pmd_at(mm, haddr, pvmw->pmd, pmde);
 
 	/* No need to invalidate - it was non-present before */
--- a/mm/hugetlb.c~b
+++ a/mm/hugetlb.c
@@ -4898,7 +4898,7 @@ int copy_hugetlb_page_range(struct mm_st
 	pte_t *src_pte, *dst_pte, entry;
 	struct folio *pte_folio;
 	unsigned long addr;
-	bool cow = is_cow_mapping(src_vma->vm_flags);
+	bool cow = vma_is_cow_mapping(src_vma);
 	struct hstate *h = hstate_vma(src_vma);
 	unsigned long sz = huge_page_size(h);
 	unsigned long npages = pages_per_huge_page(h);
--- a/mm/internal.h~b
+++ a/mm/internal.h
@@ -240,19 +240,18 @@ static inline int mmap_file(struct file
 {
 	int err = vfs_mmap(file, vma);
 
-	/* Hooks cannot mark themselves anonymous. */
-	if (WARN_ON_ONCE(vma_is_anonymous(vma)))
-		err = -EINVAL;
-
-	if (likely(!err))
-		return 0;
-
 	/*
-	 * OK, we tried to call the file hook for mmap(), but an error
-	 * arose. The mapping is in an inconsistent state and we must not invoke
-	 * any further hooks on it.
+	 * Either we tried to call the file hook for mmap() and an error arose
+	 * or a driver set vma->vm_ops = NULL intending there to be no VMA
+	 * operations.
+	 *
+	 * In the former case the VMA is in an inconsistent state and we mustn't
+	 * invoke any further hooks on it, in the latter case the hook actually
+	 * wanted no further hooks to be invoked, so fix both by setting dummy
+	 * VMA ops.
 	 */
-	vma->vm_ops = &vma_dummy_vm_ops;
+	if (unlikely(err || !vma->vm_ops))
+		vma->vm_ops = &vma_dummy_vm_ops;
 
 	return err;
 }
@@ -950,7 +949,7 @@ folio_within_range(struct folio *folio,
 
 	pgoff_folio = folio_pgoff(folio);
 	pgoff_vma_start = folio_test_anon(folio) ?
-		vma_start_virt_pgoff(vma) : vma_start_pgoff(vma);
+		vma_start_anon_pgoff(vma) : vma_start_pgoff(vma);
 
 	if (start < vma->vm_start)
 		start = vma->vm_start;
@@ -1064,20 +1063,20 @@ static inline unsigned long vma_fileback
  * vma_anon_address - Find the virtual address an anonymous page range is mapped
  * at.
  * @vma: The vma which maps this object.
- * @pgoff_virt: The virtual page index belonging to the folio.
+ * @pgoff_anon: The anonymous page index belonging to the folio.
  * @nr_pages: The number of pages to consider.
  *
  * This is only valid for anonymous or MAP_PRIVATE-mapped file-backed VMAs.
  *
- * Returns: If any page in this range is mapped by this VMA, return the first address
- * where any of these pages appear. Otherwise, return -EFAULT.
+ * Returns: If any page in this range is mapped by this VMA, return the first
+ * address where any of these pages appear. Otherwise, return -EFAULT.
  */
 static inline unsigned long vma_anon_address(const struct vm_area_struct *vma,
-		pgoff_t pgoff_virt, unsigned long nr_pages)
+		pgoff_t pgoff_anon, unsigned long nr_pages)
 {
-	VM_WARN_ON_ONCE(!vma_is_anonymous(vma) && vma_test(vma, VMA_SHARED_BIT));
+	VM_WARN_ON_ONCE(!vma_is_cow_mapping(vma));
 
-	return __vma_address(vma, pgoff_virt, vma_start_virt_pgoff(vma), nr_pages);
+	return __vma_address(vma, pgoff_anon, vma_start_anon_pgoff(vma), nr_pages);
 }
 
 /*
@@ -1086,22 +1085,20 @@ static inline unsigned long vma_anon_add
  */
 static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)
 {
+	const pgoff_t pgoff_end = pvmw->pgoff + pvmw->nr_pages;
 	const struct vm_area_struct *vma = pvmw->vma;
-	const pgoff_t pgoff = pvmw->pgoff;
 	pgoff_t pgoff_vma_start;
 	unsigned long address;
-	pgoff_t pgoff_end;
 
 	/* Common case, plus ->pgoff is invalid for KSM */
 	if (pvmw->nr_pages == 1)
 		return pvmw->address + PAGE_SIZE;
 
-	if (pvmw->is_anon_walk)
-		pgoff_vma_start = vma_start_virt_pgoff(vma);
+	if (pvmw->pgoff_is_anon)
+		pgoff_vma_start = vma_start_anon_pgoff(vma);
 	else
 		pgoff_vma_start = vma_start_pgoff(vma);
 
-	pgoff_end = pgoff + pvmw->nr_pages;
 	address = vma->vm_start +
 		((pgoff_end - pgoff_vma_start) << PAGE_SHIFT);
 	/* Check for address beyond vma (or wrapped through 0?) */
@@ -1393,7 +1390,7 @@ static inline bool gup_must_unshare(stru
 		 * ... because we only care about writable private ("COW")
 		 * mappings where we have to break COW early.
 		 */
-		return is_cow_mapping(vma->vm_flags);
+		return vma_is_cow_mapping(vma);
 	}
 
 	/* Paired with a memory barrier in folio_try_share_anon_rmap_*(). */
--- a/mm/interval_tree.c~b
+++ a/mm/interval_tree.c
@@ -83,12 +83,12 @@ mapping_rmap_tree_iter_next(struct vm_ar
 
 static pgoff_t avc_start_pgoff(struct anon_vma_chain *avc)
 {
-	return vma_start_virt_pgoff(avc->vma);
+	return vma_start_anon_pgoff(avc->vma);
 }
 
 static pgoff_t avc_last_pgoff(struct anon_vma_chain *avc)
 {
-	return vma_last_virt_pgoff(avc->vma);
+	return vma_last_anon_pgoff(avc->vma);
 }
 
 INTERVAL_TREE_DEFINE(struct anon_vma_chain, rb, pgoff_t, rb_subtree_last,
--- a/mm/ksm.c~b
+++ a/mm/ksm.c
@@ -1625,8 +1625,7 @@ static int try_to_merge_with_ksm_page(st
 	 * stable_tree, break_cow() will clean it up.
 	 */
 	rmap_item->anon_vma = vma->anon_vma;
-	/* The VMA is always anon/MAP_PRIVATE-file backed so use anon index. */
-	rmap_item->linear_page_index = linear_virt_page_index(vma, rmap_item->address);
+	rmap_item->linear_page_index = linear_anon_page_index(vma, rmap_item->address);
 	get_anon_vma(vma->anon_vma);
 out:
 	mmap_read_unlock(mm);
@@ -3153,7 +3152,7 @@ struct folio *ksm_might_need_to_copy(str
 			return folio;	/* no need to copy it */
 	} else if (!anon_vma) {
 		return folio;		/* no need to copy it */
-	} else if (folio->index == linear_virt_page_index(vma, addr) &&
+	} else if (folio->index == linear_anon_page_index(vma, addr) &&
 			anon_vma->root == vma->anon_vma->root) {
 		return folio;		/* still no need to copy it */
 	}
@@ -3223,7 +3222,7 @@ again:
 		/*
 		 * Currently, KSM folios are always small folios, so it's
 		 * sufficient to search for a single page. We can simply use
-		 * the linear_virt_page_index of the original de-duplicate
+		 * the linear_anon_page_index of the original de-duplicate
 		 * anonymous page that we remembered in the rmap_item while
 		 * de-duplicating. Note that mremap() always de-duplicates KSM
 		 * folios: so if there was mremap() in our parent or our child,
--- a/mm/memory.c~b
+++ a/mm/memory.c
@@ -631,14 +631,14 @@ static void print_bad_page_map(struct vm
 {
 	struct address_space *mapping;
 	char entry_str[PTVAL_STR_MAX];
-	pgoff_t index, virt_index;
+	pgoff_t index, anon_index;
 
 	if (is_bad_page_map_ratelimited())
 		return;
 
 	mapping = vma->vm_file ? vma->vm_file->f_mapping : NULL;
 	index = linear_page_index(vma, addr);
-	virt_index = __linear_virt_page_index(vma, addr);
+	anon_index = __linear_anon_page_index(vma, addr);
 
 	ptval_bytes_to_hex_str(entry_str, sizeof(entry_str), entry, entry_size);
 	pr_alert("BUG: Bad page map in process %s  %s:%s", current->comm,
@@ -646,9 +646,14 @@ static void print_bad_page_map(struct vm
 	__print_bad_page_map_pgtable(vma->vm_mm, addr);
 	if (page)
 		dump_page(page, "bad page map");
-	pr_alert("addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx virt_index:%lx\n",
-		 (void *)addr, vma->vm_flags, vma->anon_vma, mapping, index,
-		 virt_index);
+	pr_alert("addr:%px vm_flags:%08lx anon_vma:%px mapping:%px",
+		 (void *)addr, vma->vm_flags, vma->anon_vma, mapping);
+	if (!vma_is_cow_mapping(vma) || index == anon_index) {
+		pr_cont(" index:%lx\n", index);
+	} else {
+		pr_cont(" index:%lx (file) %lx (anon)\n", index, anon_index);
+	}
+
 	pr_alert("file:%pD fault:%ps mmap:%ps mmap_prepare: %ps read_folio:%ps\n",
 		 vma->vm_file,
 		 vma->vm_ops ? vma->vm_ops->fault : NULL,
@@ -782,7 +787,7 @@ static inline struct page *__vm_normal_p
 				/* Only CoW'ed anon folios are "normal". */
 				if (pfn == index)
 					return NULL;
-				if (!is_cow_mapping(vma->vm_flags))
+				if (!vma_is_cow_mapping(vma))
 					return NULL;
 			}
 		}
@@ -1004,7 +1009,6 @@ copy_nonpresent_pte(struct mm_struct *ds
 		pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *dst_vma,
 		struct vm_area_struct *src_vma, unsigned long addr, int *rss)
 {
-	vm_flags_t vm_flags = dst_vma->vm_flags;
 	pte_t orig_pte = ptep_get(src_pte);
 	softleaf_t entry = softleaf_from_pte(orig_pte);
 	pte_t pte = orig_pte;
@@ -1028,7 +1032,7 @@ copy_nonpresent_pte(struct mm_struct *ds
 		rss[mm_counter(folio)]++;
 
 		if (!softleaf_is_migration_read(entry) &&
-				is_cow_mapping(vm_flags)) {
+				vma_is_cow_mapping(dst_vma)) {
 			/*
 			 * COW mappings require pages in both parent and child
 			 * to be set to read. A previously exclusive entry is
@@ -1069,7 +1073,7 @@ copy_nonpresent_pte(struct mm_struct *ds
 		 * save and restore device driver state).
 		 */
 		if (softleaf_is_device_private_write(entry) &&
-		    is_cow_mapping(vm_flags)) {
+		    vma_is_cow_mapping(dst_vma)) {
 			entry = make_readable_device_private_entry(
 							swp_offset(entry));
 			pte = swp_entry_to_pte(entry);
@@ -1084,7 +1088,7 @@ copy_nonpresent_pte(struct mm_struct *ds
 		 * exclusive entries currently only support private writable
 		 * (ie. COW) mappings.
 		 */
-		VM_BUG_ON(!is_cow_mapping(src_vma->vm_flags));
+		VM_BUG_ON(!vma_is_cow_mapping(src_vma));
 		if (try_restore_exclusive_pte(src_vma, addr, src_pte, orig_pte))
 			return -EBUSY;
 		return -ENOENT;
@@ -1183,7 +1187,7 @@ static __always_inline void __copy_prese
 	}
 
 	/* If it's a COW mapping, write protect it both processes. */
-	if (is_cow_mapping(src_vma->vm_flags) && writable) {
+	if (vma_is_cow_mapping(src_vma) && writable) {
 		wrprotect_ptes(src_mm, addr, src_pte, nr);
 		pte = pte_wrprotect(pte);
 	}
@@ -1604,9 +1608,9 @@ copy_page_range(struct vm_area_struct *d
 	 * We need to invalidate the secondary MMU mappings only when
 	 * there could be a permission downgrade on the ptes of the
 	 * parent mm. And a permission downgrade will only happen if
-	 * is_cow_mapping() returns true.
+	 * vma_is_cow_mapping() returns true.
 	 */
-	is_cow = is_cow_mapping(src_vma->vm_flags);
+	is_cow = vma_is_cow_mapping(src_vma);
 
 	if (is_cow) {
 		mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE,
@@ -2439,7 +2443,7 @@ static bool vm_mixed_zeropage_allowed(st
 	if (mm_forbids_zeropage(vma->vm_mm))
 		return false;
 	/* zeropages in COW mappings are common and unproblematic. */
-	if (is_cow_mapping(vma->vm_flags))
+	if (vma_is_cow_mapping(vma))
 		return true;
 	/* Mappings that do not allow for writable PTEs are unproblematic. */
 	if (!(vma->vm_flags & (VM_WRITE | VM_MAYWRITE)))
@@ -2890,7 +2894,7 @@ vm_fault_t vmf_insert_pfn_prot(struct vm
 	BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)));
 	BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) ==
 						(VM_PFNMAP|VM_MIXEDMAP));
-	BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags));
+	BUG_ON((vma->vm_flags & VM_PFNMAP) && vma_is_cow_mapping(vma));
 	BUG_ON((vma->vm_flags & VM_MIXEDMAP) && pfn_valid(pfn));
 
 	if (addr < vma->vm_start || addr >= vma->vm_end)
@@ -3302,7 +3306,7 @@ static int remap_pfn_range_prepare_vma(s
 				       unsigned long size)
 {
 	const unsigned long end = addr + PAGE_ALIGN(size);
-	const bool is_cow = is_cow_mapping(vma->vm_flags);
+	const bool is_cow = vma_is_cow_mapping(vma);
 	int err;
 
 	err = get_remap_pgoff(is_cow, addr, end, vma->vm_start, vma->vm_end,
@@ -6802,7 +6806,7 @@ static vm_fault_t sanitize_fault_flags(s
 		 * FAULT_FLAG_UNSHARE only applies to COW mappings. Let's
 		 * just treat it like an ordinary read-fault otherwise.
 		 */
-		if (!is_cow_mapping(vma->vm_flags))
+		if (!vma_is_cow_mapping(vma))
 			*flags &= ~FAULT_FLAG_UNSHARE;
 	} else if (*flags & FAULT_FLAG_WRITE) {
 		/* Write faults on read-only mappings are impossible ... */
@@ -6810,7 +6814,7 @@ static vm_fault_t sanitize_fault_flags(s
 			return VM_FAULT_SIGSEGV;
 		/* ... and FOLL_FORCE only applies to COW mappings. */
 		if (WARN_ON_ONCE(!(vma->vm_flags & VM_WRITE) &&
-				 !is_cow_mapping(vma->vm_flags)))
+				 !vma_is_cow_mapping(vma)))
 			return VM_FAULT_SIGSEGV;
 	}
 #ifdef CONFIG_PER_VMA_LOCK
--- a/mm/mempolicy.c~b
+++ a/mm/mempolicy.c
@@ -844,7 +844,7 @@ bool folio_can_map_prot_numa(struct foli
 		return false;
 
 	/* Also skip shared copy-on-write folios */
-	if (is_cow_mapping(vma->vm_flags) && folio_maybe_mapped_shared(folio))
+	if (vma_is_cow_mapping(vma) && folio_maybe_mapped_shared(folio))
 		return false;
 
 	/* Folios are pinned and can't be migrated */
--- a/mm/migrate.c~b
+++ a/mm/migrate.c
@@ -356,25 +356,18 @@ static bool remove_migration_pte(struct
 
 	while (page_vma_mapped_walk(&pvmw)) {
 		rmap_t rmap_flags = RMAP_NONE;
-		pte_t old_pte;
-		pte_t pte;
+		unsigned long idx = 0;
 		softleaf_t entry;
 		struct page *new;
-		unsigned long idx = 0;
-
-		/* pgoff is invalid for ksm pages, but they are never large */
-		if (folio_test_large(folio) && !folio_test_hugetlb(folio)) {
-			idx += linear_folio_page_index(folio, vma, pvmw.address);
-			idx -= pvmw.pgoff;
-		}
-		new = folio_page(folio, idx);
+		pte_t old_pte;
+		pte_t pte;
 
 #ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES
 		/* PMD-mapped THP migration entry */
 		if (!pvmw.pte) {
 			VM_BUG_ON_FOLIO(folio_test_hugetlb(folio) ||
 					!folio_test_pmd_mappable(folio), folio);
-			remove_migration_pmd(&pvmw, new);
+			remove_migration_pmd(&pvmw, folio);
 			continue;
 		}
 #endif
@@ -383,14 +376,18 @@ static bool remove_migration_pte(struct
 						pvmw.pte);
 		else
 			old_pte = ptep_get(pvmw.pte);
+
+		entry = softleaf_from_pte(old_pte);
+		if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+			idx = softleaf_to_pfn(entry) - pvmw.pfn;
+
 		if (rmap_walk_arg->map_unused_to_zeropage &&
 		    try_to_map_unused_to_zeropage(&pvmw, folio, old_pte, idx))
 			continue;
 
 		folio_get(folio);
+		new = folio_page(folio, idx);
 		pte = mk_pte(new, READ_ONCE(vma->vm_page_prot));
-
-		entry = softleaf_from_pte(old_pte);
 		if (!softleaf_is_migration_young(entry))
 			pte = pte_mkold(pte);
 		if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry))
--- a/mm/mremap.c~b
+++ a/mm/mremap.c
@@ -1266,8 +1266,8 @@ static int copy_vma_and_data(struct vma_
 			     struct vm_area_struct **new_vma_ptr)
 {
 	const pgoff_t new_pgoff = linear_page_index(vrm->vma, vrm->addr);
-	const pgoff_t new_virt_pgoff =
-		__linear_virt_page_index(vrm->vma, vrm->addr);
+	const pgoff_t new_anon_pgoff =
+		__linear_anon_page_index(vrm->vma, vrm->addr);
 	struct vm_area_struct *vma = vrm->vma;
 	struct vm_area_struct *new_vma;
 	unsigned long moved_len;
@@ -1275,7 +1275,7 @@ static int copy_vma_and_data(struct vma_
 	PAGETABLE_MOVE(pmc, NULL, NULL, vrm->addr, vrm->new_addr, vrm->old_len);
 
 	new_vma = copy_vma(&vma, vrm->new_addr, vrm->new_len, new_pgoff,
-			   new_virt_pgoff, &pmc.need_rmap_locks);
+			   new_anon_pgoff, &pmc.need_rmap_locks);
 	if (!new_vma) {
 		vrm_uncharge(vrm);
 		*new_vma_ptr = NULL;
--- a/mm/rmap.c~b
+++ a/mm/rmap.c
@@ -1240,7 +1240,7 @@ static bool mapping_wrprotect_range_one(
 		.vma		= vma,
 		.address	= address,
 		.flags		= PVMW_SYNC,
-		.is_anon_walk   = false,
+		.pgoff_is_anon	= false,
 	};
 
 	state->cleaned += page_vma_mkclean_one(&pvmw);
@@ -1318,7 +1318,7 @@ int pfn_mkclean_range(unsigned long pfn,
 		.pgoff		= pgoff,
 		.vma		= vma,
 		.flags		= PVMW_SYNC,
-		.is_anon_walk   = false,
+		.pgoff_is_anon	= false,
 	};
 
 	if (invalid_mkclean_vma(vma, NULL))
@@ -1485,7 +1485,7 @@ static void __folio_set_anon(struct foli
 	 */
 	anon_vma = (void *) anon_vma + FOLIO_MAPPING_ANON;
 	WRITE_ONCE(folio->mapping, (struct address_space *) anon_vma);
-	folio->index = linear_virt_page_index(vma, address);
+	folio->index = linear_anon_page_index(vma, address);
 }
 
 /**
@@ -1513,7 +1513,7 @@ static void __page_check_anon_rmap(const
 	VM_BUG_ON_FOLIO(folio_anon_vma(folio)->root != vma->anon_vma->root,
 			folio);
 	VM_BUG_ON_PAGE(page_pgoff(folio, page) !=
-		       linear_virt_page_index(vma, address), page);
+		       linear_anon_page_index(vma, address), page);
 }
 
 static __always_inline void __folio_add_anon_rmap(struct folio *folio,
--- a/mm/userfaultfd.c~b
+++ a/mm/userfaultfd.c
@@ -1352,8 +1352,7 @@ static long move_present_ptes(struct mm_
 		}
 
 		folio_move_anon_rmap(src_folio, dst_vma);
-		src_folio->index = linear_folio_page_index(src_folio, dst_vma,
-							   dst_addr);
+		src_folio->index = linear_anon_page_index(dst_vma, dst_addr);
 
 		orig_dst_pte = folio_mk_pte(src_folio, dst_vma->vm_page_prot);
 		/* Set soft dirty bit so userspace can notice the pte was moved */
@@ -1429,8 +1428,7 @@ static int move_swap_pte(struct mm_struc
 	 */
 	if (src_folio) {
 		folio_move_anon_rmap(src_folio, dst_vma);
-		src_folio->index = linear_folio_page_index(src_folio, dst_vma,
-							   dst_addr);
+		src_folio->index = linear_anon_page_index(dst_vma, dst_addr);
 	} else {
 		/*
 		 * Check if the swap entry is cached after acquiring the src_pte
--- a/mm/vma.c~b
+++ a/mm/vma.c
@@ -18,7 +18,7 @@ struct mmap_state {
 	unsigned long addr;
 	unsigned long end;
 	pgoff_t pgoff;
-	pgoff_t virt_pgoff;
+	pgoff_t anon_pgoff;
 	unsigned long pglen;
 	union {
 		vm_flags_t vm_flags;
@@ -47,22 +47,14 @@ struct mmap_state {
 	bool file_doesnt_need_get :1;
 };
 
-static inline pgoff_t map_anon_pgoff(const struct mmap_state *map)
-{
-	if (vma_flags_test(&map->vma_flags, VMA_SHARED_BIT))
-		return map->pgoff;
-
-	return map->virt_pgoff;
-}
-
-#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, virt_pgoff_, vma_flags_, file_) \
+#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, anon_pgoff_, vma_flags_, file_) \
 	struct mmap_state name = {					\
 		.mm = mm_,						\
 		.vmi = vmi_,						\
 		.addr = addr_,						\
 		.end = (addr_) + (len_),				\
 		.pgoff = pgoff_,					\
-		.virt_pgoff = virt_pgoff_,				\
+		.anon_pgoff = anon_pgoff_,				\
 		.pglen = PHYS_PFN(len_),				\
 		.vma_flags = vma_flags_,				\
 		.file = file_,						\
@@ -77,7 +69,7 @@ static inline pgoff_t map_anon_pgoff(con
 		.end = (map_)->end,					\
 		.vma_flags = (map_)->vma_flags,				\
 		.pgoff = (map_)->pgoff,					\
-		.anon_pgoff = map_anon_pgoff(map_),			\
+		.anon_pgoff = (map_)->anon_pgoff,			\
 		.file = (map_)->file,					\
 		.prev = (map_)->prev,					\
 		.middle = vma_,						\
@@ -93,11 +85,11 @@ static void __vma_set_range(struct vm_ar
 }
 
 static void vma_set_range(struct vm_area_struct *vma, unsigned long start,
-			  unsigned long end, pgoff_t pgoff, pgoff_t virt_pgoff)
+			  unsigned long end, pgoff_t pgoff, pgoff_t anon_pgoff)
 {
 	__vma_set_range(vma, start, end);
 	vma_set_pgoff(vma, pgoff);
-	vma_set_virt_pgoff(vma, virt_pgoff);
+	vma_set_anon_pgoff(vma, anon_pgoff);
 }
 
 /* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */
@@ -213,6 +205,25 @@ static void init_multi_vma_prep(struct v
 }
 
 /*
+ * Does this merge require that adjacent VMAs must have adjacent anonymous page
+ * offsets in addition to having adjacent vma->vm_pgoff?
+ *
+ * This is only required for MAP_PRIVATE-file backed mappings as the page offset
+ * for pure anonymous VMAs is equal to the anonymous page offset.
+ *
+ * Read-only shared mappings (with VMA_SHARED_BIT cleared) are always unfaulted
+ * so automatically have correct anonymous page offset (as it is always updated
+ * on remap).
+ *
+ * 'Special' mappings in the sense of VDSO, VVAR etc. have !file but would in
+ * any case not be candidates for merge nor be mergeable.
+ */
+static bool needs_adjacent_anon_pgoff(const struct vma_merge_struct *vmg)
+{
+	return vmg->file && vma_flags_is_cow_mapping(&vmg->vma_flags);
+}
+
+/*
  * Return true if we can merge this (vma_flags,anon_vma,file,vm_pgoff)
  * in front of (at a lower virtual address and file offset than) the vma.
  *
@@ -233,7 +244,8 @@ static bool can_vma_merge_before(struct
 		return false;
 	if (vmg_end_pgoff(vmg) != vma_start_pgoff(vmg->next))
 		return false;
-	if (vmg_end_anon_pgoff(vmg) != vma_start_anon_pgoff(vmg->next))
+	if (needs_adjacent_anon_pgoff(vmg) &&
+	    vmg_end_anon_pgoff(vmg) != vma_start_anon_pgoff(vmg->next))
 		return false;
 	return true;
 }
@@ -255,7 +267,8 @@ static bool can_vma_merge_after(struct v
 		return false;
 	if (vma_end_pgoff(vmg->prev) != vmg_start_pgoff(vmg))
 		return false;
-	if (vma_end_anon_pgoff(vmg->prev) != vmg_start_anon_pgoff(vmg))
+	if (needs_adjacent_anon_pgoff(vmg) &&
+	    vma_end_anon_pgoff(vmg->prev) != vmg_start_anon_pgoff(vmg))
 		return false;
 	return true;
 }
@@ -1930,27 +1943,27 @@ static int vma_link(struct mm_struct *mm
  */
 struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
 	unsigned long addr, unsigned long len, pgoff_t pgoff,
-	pgoff_t virt_pgoff, bool *need_rmap_locks)
+	pgoff_t anon_pgoff, bool *need_rmap_locks)
 {
 	struct vm_area_struct *vma = *vmap;
-	const bool is_shared = vma_test(vma, VMA_SHARED_BIT);
-	unsigned long vma_start = vma->vm_start;
+	unsigned long old_vma_start = vma->vm_start;
 	struct mm_struct *mm = vma->vm_mm;
 	struct vm_area_struct *new_vma;
-	bool faulted_in_anon_vma = true;
+	bool can_self_merge = false;
 	VMA_ITERATOR(vmi, mm, addr);
 	VMG_VMA_STATE(vmg, &vmi, NULL, vma, addr, addr + len);
 
 	/*
-	 * If a vma has not yet been faulted, update its virtual pgoff to match
-	 * the new location to increase its chance of merging.
+	 * If a vma has not yet been faulted, update its anonymous pgoff to
+	 * match the new location to increase its chance of merging.
 	 */
-	if (!vma->anon_vma && !is_shared) {
-		virt_pgoff = addr >> PAGE_SHIFT;
+	if (!vma->anon_vma) {
+		anon_pgoff = addr >> PAGE_SHIFT;
 
-		if (vma_is_anonymous(vma))
-			pgoff = virt_pgoff;
-		faulted_in_anon_vma = false;
+		if (vma_is_anonymous(vma)) {
+			pgoff = anon_pgoff;
+			can_self_merge = true;
+		}
 	}
 
 	/*
@@ -1966,39 +1979,35 @@ struct vm_area_struct *copy_vma(struct v
 		return NULL;	/* should never get here */
 
 	vmg.pgoff = pgoff;
-	vmg.anon_pgoff = is_shared ? pgoff : virt_pgoff;
+	vmg.anon_pgoff = anon_pgoff;
 	vmg.next = vma_iter_next_rewind(&vmi, NULL);
 	new_vma = vma_merge_copied_range(&vmg);
 
 	if (new_vma) {
-		/*
-		 * Source vma may have been merged into new_vma
-		 */
-		if (unlikely(vma_start >= new_vma->vm_start &&
-			     vma_start < new_vma->vm_end)) {
+		/* Self-merged and VMA replaced. */
+		if (unlikely(new_vma->vm_start < old_vma_start &&
+			     new_vma->vm_end > old_vma_start)) {
 			/*
-			 * The only way we can get a vma_merge with
-			 * self during an mremap is if the vma hasn't
-			 * been faulted in yet and we were allowed to
-			 * reset the dst vma->vm_pgoff to the
-			 * destination address of the mremap to allow
-			 * the merge to happen. mremap must change the
-			 * vm_pgoff linearity between src and dst vmas
-			 * (in turn preventing a vma_merge) to be
-			 * safe. It is only safe to keep the vm_pgoff
-			 * linear if there are no pages mapped yet.
+			 * The only way a VMA can both self-merge and be
+			 * replaced is if the remap places the new VMA
+			 * immediately prior to its old self ('next') and
+			 * immediately after another VMA ('prev') causing the
+			 * next to be removed and prev to be expanded to cover
+			 * the entire range.
+			 *
+			 * This should only be possible if the anonymous page
+			 * offset was updated, i.e. the VMA is unfaulted.
 			 */
-			VM_WARN_ON_ONCE_VMA(faulted_in_anon_vma, new_vma);
+			VM_WARN_ON_ONCE_VMA(!can_self_merge, new_vma);
 			*vmap = vma = new_vma;
 		}
 		*need_rmap_locks =
-			(vma_start_pgoff(new_vma) <= vma_start_pgoff(vma)) ||
-			(vma_start_anon_pgoff(new_vma) <= vma_start_anon_pgoff(vma));
+			(vma_start_pgoff(new_vma) <= vma_start_pgoff(vma));
 	} else {
 		new_vma = vm_area_dup(vma);
 		if (!new_vma)
 			goto out;
-		vma_set_range(new_vma, addr, addr + len, pgoff, virt_pgoff);
+		vma_set_range(new_vma, addr, addr + len, pgoff, anon_pgoff);
 		if (vma_dup_policy(vma, new_vma))
 			goto out_free_vma;
 		if (anon_vma_clone(new_vma, vma, VMA_OP_REMAP))
@@ -2066,7 +2075,7 @@ static int anon_vma_compatible(struct vm
 	/* Page offset must align. */
 	if (vma_end_pgoff(a) != vma_start_pgoff(b))
 		return false;
-	/* Anon page offset must align. */
+	/* Only reached from anon path, so either MAP_PRIVATE file or anon. */
 	if (vma_end_anon_pgoff(a) != vma_start_anon_pgoff(b))
 		return false;
 	return true;
@@ -2615,8 +2624,11 @@ static int __mmap_new_file_vma(struct mm
 static bool map_is_dev_zero(const struct mmap_state *map)
 {
 	const struct file *file = map->file;
-	const struct inode *inode = file_inode(file);
+	struct inode *inode;
 
+	if (!file)
+		return false;
+	inode = file_inode(file);
 	if (!S_ISCHR(inode->i_mode))
 		return false;
 	return imajor(inode) == MEM_MAJOR && iminor(inode) == DEVZERO_MINOR;
@@ -2671,7 +2683,7 @@ static int __mmap_new_vma(struct mmap_st
 	if (is_anon)
 		vma_set_anonymous(vma);
 
-	vma_set_range(vma, map->addr, map->end, map->pgoff, map->virt_pgoff);
+	vma_set_range(vma, map->addr, map->end, map->pgoff, map->anon_pgoff);
 	vma->flags = map->vma_flags;
 	vma->vm_page_prot = map->page_prot;
 
@@ -2869,8 +2881,8 @@ static unsigned long __mmap_region(struc
 	struct vm_area_struct *vma = NULL;
 	bool have_mmap_prepare = file && file->f_op->mmap_prepare;
 	VMA_ITERATOR(vmi, mm, addr);
-	const pgoff_t virt_pgoff = addr >> PAGE_SHIFT;
-	MMAP_STATE(map, mm, &vmi, addr, len, pgoff, virt_pgoff, vma_flags, file);
+	const pgoff_t anon_pgoff = addr >> PAGE_SHIFT;
+	MMAP_STATE(map, mm, &vmi, addr, len, pgoff, anon_pgoff, vma_flags, file);
 	struct vm_area_desc desc = {
 		.mm = mm,
 		.file = file,
@@ -3452,7 +3464,7 @@ int insert_vm_struct(struct mm_struct *m
 		WARN_ON_ONCE(vma->anon_vma);
 		vma_set_pgoff(vma, vma->vm_start >> PAGE_SHIFT);
 	}
-	vma_set_virt_pgoff(vma, vma->vm_start >> PAGE_SHIFT);
+	vma_set_anon_pgoff(vma, vma->vm_start >> PAGE_SHIFT);
 
 	if (vma_link(mm, vma)) {
 		if (vma_test(vma, VMA_ACCOUNT_BIT))
--- a/mm/vma.h~b
+++ a/mm/vma.h
@@ -291,86 +291,32 @@ static inline pgoff_t vmg_end_anon_pgoff
 	return vmg_start_anon_pgoff(vmg) + vmg_pages(vmg);
 }
 
-static inline void __vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
+static inline void __vma_set_anon_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
 {
 #ifdef CONFIG_64BIT
-	vma->__vm_virt_pgoff_hi = pgoff >> 32;
+	vma->__vm_anon_pgoff_hi = pgoff >> 32;
 #endif
-	vma->__vm_virt_pgoff_lo = pgoff & GENMASK(31, 0);
+	vma->__vm_anon_pgoff_lo = pgoff & GENMASK(31, 0);
 }
 
-static inline void vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
+static inline void vma_set_anon_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
 {
 	vma_assert_can_modify(vma);
-	__vma_set_virt_pgoff(vma, pgoff);
+	__vma_set_anon_pgoff(vma, pgoff);
 }
 
 static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta)
 {
 	vma_assert_can_modify(vma);
 	vma_set_pgoff(vma, vma_start_pgoff(vma) + delta);
-	vma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) + delta);
+	vma_set_anon_pgoff(vma, vma_start_anon_pgoff(vma) + delta);
 }
 
 static inline void vma_sub_pgoff(struct vm_area_struct *vma, pgoff_t delta)
 {
 	vma_assert_can_modify(vma);
 	vma_set_pgoff(vma, vma_start_pgoff(vma) - delta);
-	vma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) - delta);
-}
-
-/**
- * vma_anon_pgoff_addr() - Calculates the absolute anonymous page offset of
- * @address.
- * @vma: The VMA whose anonymous page offset is required.
- * @address: The address whose absolute page offset is required.
- *
- * If the VMA is a shared file-backed mapping, then the file-based page offset
- * is returned.
- *
- * Otherwise, the virtual page offset is returned.
- *
- * This means that shared file-backed mappings are correctly merged based on
- * their file page offset compatibility.
- *
- * Returns: The absolute anonymous page offset of @address within @vma.
- */
-static inline pgoff_t vma_anon_pgoff_addr(const struct vm_area_struct *vma,
-					  unsigned long address)
-{
-	if (vma_test(vma, VMA_SHARED_BIT))
-		return linear_page_index(vma, address);
-
-	return linear_virt_page_index(vma, address);
-}
-
-/**
- * vma_start_anon_pgoff() - Calculates the absolute anonymous page offset used
- * for purposes of merge compatibility.
- * @vma: The VMA whose anonymous page offset is required.
- *
- * See vma_anon_pgoff_addr().
- *
- * Returns: The absolute anonymous page offset of @vma for purposes of merging.
- */
-static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
-{
-	return vma_anon_pgoff_addr(vma, vma->vm_start);
-}
-
-/**
- * vma_end_anon_pgoff() - Calculates the absolute exclusive end anonymous page
- * offset used for purposes of merge compatibility.
- * @vma: The VMA whose anonymous end page offset is required.
- *
- * See vma_start_anon_pgoff().
- *
- * Returns: The absolute exclusive end anonymous page offset of @vma for
- * purposes of merging.
- */
-static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
-{
-	return vma_start_anon_pgoff(vma) + vma_pages(vma);
+	vma_set_anon_pgoff(vma, vma_start_anon_pgoff(vma) - delta);
 }
 
 #define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_, anon_pgoff_) \
@@ -396,7 +342,7 @@ static inline pgoff_t vma_end_anon_pgoff
 		.end = end_,						\
 		.vm_flags = vma_->vm_flags,				\
 		.pgoff = linear_page_index(vma_, start_),		\
-		.anon_pgoff = vma_anon_pgoff_addr(vma_, start_),	\
+		.anon_pgoff = __linear_anon_page_index(vma_, start_),	\
 		.file = vma_->vm_file,					\
 		.anon_vma = vma_->anon_vma,				\
 		.policy = vma_policy(vma_),				\
--- a/mm/vma_init.c~b
+++ a/mm/vma_init.c
@@ -51,7 +51,7 @@ static void vm_area_init_from(const stru
 	dest->vm_end = src->vm_end;
 	dest->anon_vma = src->anon_vma;
 	dest->vm_pgoff = vma_start_pgoff(src);
-	__vma_set_virt_pgoff(dest, vma_start_virt_pgoff(src));
+	__vma_set_anon_pgoff(dest, vma_start_anon_pgoff(src));
 	dest->vm_file = src->vm_file;
 	dest->vm_private_data = src->vm_private_data;
 	vm_flags_init(dest, src->vm_flags);
--- a/tools/testing/selftests/mm/merge.c~b
+++ a/tools/testing/selftests/mm/merge.c
@@ -1305,7 +1305,7 @@ TEST_F(merge, merge_vmas_with_mseal)
 	ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr + 2 * page_size);
 }
 
-TEST_F(merge, virt_and_page_offset_mismatch_memfd)
+TEST_F(merge, anon_and_page_offset_mismatch_memfd)
 {
 	struct procmap_fd *procmap = &self->procmap;
 	unsigned int page_size = self->page_size;
@@ -1314,7 +1314,7 @@ TEST_F(merge, virt_and_page_offset_misma
 	int fd;
 
 	/* Create a 10 page memfd descriptor. */
-	fd = memfd_create("virt_page_offset_test", MFD_CLOEXEC);
+	fd = memfd_create("anon_page_offset_test", MFD_CLOEXEC);
 	ASSERT_NE(fd, -1);
 	ASSERT_EQ(ftruncate(fd, 10 * page_size), 0);
 
@@ -1346,10 +1346,10 @@ TEST_F(merge, virt_and_page_offset_misma
 	 * | unfaulted |           | faulted |
 	 * |-----------|           |---------|
 	 *
-	 * Because virtual page offset of the faulted region is now
+	 * Because the anonymous page offset of the faulted region is now
 	 * &carveout[10 * page_size], despite the two regions being mergeable
-	 * due to file page offset, they are NOT mergeable due to virtual page
-	 * offset.
+	 * due to file page offset, they are NOT mergeable due to anonymous
+	 * page offset.
 	 */
 	ptr2 = sys_mremap(ptr2, 5 * page_size, 5 * page_size,
 			  MREMAP_MAYMOVE | MREMAP_FIXED,
--- a/tools/testing/selftests/proc/proc-self-map-files-001.c~b
+++ a/tools/testing/selftests/proc/proc-self-map-files-001.c
@@ -51,7 +51,7 @@ int main(void)
 	int fd;
 	unsigned long a, b;
 
-	fd = open("/dev/zero", O_RDONLY);
+	fd = open("/proc/self/exe", O_RDONLY);
 	if (fd == -1)
 		return 1;
 
--- a/tools/testing/selftests/proc/proc-self-map-files-002.c~b
+++ a/tools/testing/selftests/proc/proc-self-map-files-002.c
@@ -57,7 +57,7 @@ int main(void)
 	int fd;
 	unsigned long a, b;
 
-	fd = open("/dev/zero", O_RDONLY);
+	fd = open("/proc/self/exe", O_RDONLY);
 	if (fd == -1)
 		return 1;
 
--- a/tools/testing/vma/include/dup.h~b
+++ a/tools/testing/vma/include/dup.h
@@ -599,7 +599,7 @@ struct vm_area_struct {
 	 */
 	unsigned int vm_lock_seq;
 #endif
-	unsigned int __vm_virt_pgoff_lo;
+	unsigned int __vm_anon_pgoff_lo;
 
 	/*
 	 * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
@@ -637,7 +637,7 @@ struct vm_area_struct {
 	refcount_t vm_refcnt;
 #endif
 #ifdef CONFIG_64BIT
-	unsigned int __vm_virt_pgoff_hi;
+	unsigned int __vm_anon_pgoff_hi;
 #endif
 	/*
 	 * For areas with an address space and backing store,
@@ -1184,6 +1184,17 @@ static inline bool vma_is_shared_maywrit
 	return is_shared_maywrite(&vma->flags);
 }
 
+static inline bool vma_flags_is_cow_mapping(const vma_flags_t *flags)
+{
+	return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
+		!vma_flags_test(flags, VMA_SHARED_BIT);
+}
+
+static inline bool vma_is_cow_mapping(const struct vm_area_struct *vma)
+{
+	return vma_flags_is_cow_mapping(&vma->flags);
+}
+
 static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)
 {
 	/*
@@ -1346,26 +1357,26 @@ static inline pgoff_t vma_end_pgoff(cons
 	return vma_start_pgoff(vma) + vma_pages(vma);
 }
 
-static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
 {
 	pgoff_t pgoff = 0;
 
 #ifdef CONFIG_64BIT
-	pgoff += vma->__vm_virt_pgoff_hi;
+	pgoff += vma->__vm_anon_pgoff_hi;
 	pgoff <<= 32;
 #endif
-	pgoff += vma->__vm_virt_pgoff_lo;
+	pgoff += vma->__vm_anon_pgoff_lo;
 	return pgoff;
 }
 
-static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
 {
-	return vma_start_virt_pgoff(vma) + vma_pages(vma);
+	return vma_start_anon_pgoff(vma) + vma_pages(vma);
 }
 
-static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)
+static inline pgoff_t vma_last_anon_pgoff(const struct vm_area_struct *vma)
 {
-	return vma_end_virt_pgoff(vma) - 1;
+	return vma_end_anon_pgoff(vma) - 1;
 }
 
 static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)
@@ -1633,22 +1644,22 @@ static inline pgprot_t vma_get_page_prot
 	return vma_flags_to_page_prot(vma->flags);
 }
 
-static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,
+static inline pgoff_t __linear_anon_page_index(const struct vm_area_struct *vma,
 					       const unsigned long address)
 {
 	pgoff_t pgoff;
 
 	pgoff = linear_page_delta(vma, address);
-	pgoff += vma_start_virt_pgoff(vma);
+	pgoff += vma_start_anon_pgoff(vma);
 	return pgoff;
 }
 
-static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,
-					     const unsigned long address)
+static inline pgoff_t linear_anon_page_index(const struct vm_area_struct *vma,
+		const unsigned long address)
 {
-	const pgoff_t pgoff = __linear_virt_page_index(vma, address);
+	const pgoff_t pgoff = __linear_anon_page_index(vma, address);
 
-	VM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));
+	VM_WARN_ON_ONCE(!vma_is_cow_mapping(vma));
 	if (vma_is_anonymous(vma))
 		VM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));
 
--- a/tools/testing/vma/shared.c~b
+++ a/tools/testing/vma/shared.c
@@ -24,7 +24,7 @@ struct vm_area_struct *alloc_vma(struct
 	vma->vm_start = start;
 	vma->vm_end = end;
 	vma_set_pgoff(vma, pgoff);
-	vma_set_virt_pgoff(vma, start >> PAGE_SHIFT);
+	vma_set_anon_pgoff(vma, start >> PAGE_SHIFT);
 	vma->flags = vma_flags;
 	vma_assert_detached(vma);
 
--- a/tools/testing/vma/tests/merge.c~b
+++ a/tools/testing/vma/tests/merge.c
@@ -121,7 +121,7 @@ static bool test_simple_merge(void)
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x3000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_FLAGS_SAME_MASK(&vma->flags, vma_flags);
 
 	detach_free_vma(vma);
@@ -154,7 +154,7 @@ static bool test_simple_modify(void)
 	ASSERT_EQ(vma->vm_start, 0x1000);
 	ASSERT_EQ(vma->vm_end, 0x2000);
 	ASSERT_EQ(vma_start_pgoff(vma), 1);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 1);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 1);
 
 	/*
 	 * Now walk through the three split VMAs and make sure they are as
@@ -167,7 +167,7 @@ static bool test_simple_modify(void)
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x1000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 
 	detach_free_vma(vma);
 	vma_iter_clear(&vmi);
@@ -177,7 +177,7 @@ static bool test_simple_modify(void)
 	ASSERT_EQ(vma->vm_start, 0x1000);
 	ASSERT_EQ(vma->vm_end, 0x2000);
 	ASSERT_EQ(vma_start_pgoff(vma), 1);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 1);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 1);
 
 	detach_free_vma(vma);
 	vma_iter_clear(&vmi);
@@ -187,7 +187,7 @@ static bool test_simple_modify(void)
 	ASSERT_EQ(vma->vm_start, 0x2000);
 	ASSERT_EQ(vma->vm_end, 0x3000);
 	ASSERT_EQ(vma_start_pgoff(vma), 2);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 2);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 2);
 
 	detach_free_vma(vma);
 	mtree_destroy(&mm.mm_mt);
@@ -217,7 +217,7 @@ static bool test_simple_expand(void)
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x3000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 
 	detach_free_vma(vma);
 	mtree_destroy(&mm.mm_mt);
@@ -240,7 +240,7 @@ static bool test_simple_shrink(void)
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x1000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 
 	detach_free_vma(vma);
 	mtree_destroy(&mm.mm_mt);
@@ -353,7 +353,7 @@ static bool __test_merge_new(bool is_sti
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x5000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 3);
@@ -375,7 +375,7 @@ static bool __test_merge_new(bool is_sti
 	ASSERT_EQ(vma->vm_start, 0x6000);
 	ASSERT_EQ(vma->vm_end, 0x9000);
 	ASSERT_EQ(vma_start_pgoff(vma), 6);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 6);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 6);
 	ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 3);
@@ -396,7 +396,7 @@ static bool __test_merge_new(bool is_sti
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x9000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 2);
@@ -417,7 +417,7 @@ static bool __test_merge_new(bool is_sti
 	ASSERT_EQ(vma->vm_start, 0xa000);
 	ASSERT_EQ(vma->vm_end, 0xc000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0xa);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0xa);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0xa);
 	ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 2);
@@ -437,7 +437,7 @@ static bool __test_merge_new(bool is_sti
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0xc000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 1);
@@ -458,7 +458,7 @@ static bool __test_merge_new(bool is_sti
 		ASSERT_EQ(vma->vm_start, 0);
 		ASSERT_EQ(vma->vm_end, 0xc000);
 		ASSERT_EQ(vma_start_pgoff(vma), 0);
-		ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+		ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 		ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
 
 		detach_free_vma(vma);
@@ -655,7 +655,8 @@ static bool test_vma_merge_with_close(vo
 	ASSERT_EQ(vmg.state, VMA_MERGE_SUCCESS);
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x5000);
-	ASSERT_EQ(vma_prev->vm_pgoff, 0);
+	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 
 	ASSERT_EQ(cleanup_mm(&mm, &vmi), 2);
 
@@ -766,7 +767,8 @@ static bool test_vma_merge_with_close(vo
 	ASSERT_EQ(vmg.state, VMA_MERGE_SUCCESS);
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x5000);
-	ASSERT_EQ(vma_prev->vm_pgoff, 0);
+	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 
 	ASSERT_EQ(cleanup_mm(&mm, &vmi), 2);
 
@@ -821,7 +823,7 @@ static bool test_vma_merge_new_with_clos
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x5000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_EQ(vma->vm_ops, &vm_ops);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 2);
@@ -878,12 +880,12 @@ static bool __test_merge_existing(bool p
 	ASSERT_EQ(vma_next->vm_start, 0x3000);
 	ASSERT_EQ(vma_next->vm_end, 0x9000);
 	ASSERT_EQ(vma_start_pgoff(vma_next), 3);
-	ASSERT_EQ(vma_start_virt_pgoff(vma_next), 3);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_next), 3);
 	ASSERT_EQ(vma_next->anon_vma, &dummy_anon_vma);
 	ASSERT_EQ(vma->vm_start, 0x2000);
 	ASSERT_EQ(vma->vm_end, 0x3000);
 	ASSERT_EQ(vma_start_pgoff(vma), 2);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 2);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 2);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_TRUE(vma_write_started(vma_next));
 	ASSERT_EQ(mm.map_count, 2);
@@ -913,7 +915,8 @@ static bool __test_merge_existing(bool p
 	ASSERT_EQ(vmg.state, VMA_MERGE_SUCCESS);
 	ASSERT_EQ(vma_next->vm_start, 0x2000);
 	ASSERT_EQ(vma_next->vm_end, 0x9000);
-	ASSERT_EQ(vma_next->vm_pgoff, 2);
+	ASSERT_EQ(vma_start_pgoff(vma_next), 2);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_next), 2);
 	ASSERT_EQ(vma_next->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma_next));
 	ASSERT_EQ(mm.map_count, 1);
@@ -946,12 +949,12 @@ static bool __test_merge_existing(bool p
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x6000);
 	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 	ASSERT_EQ(vma_prev->anon_vma, &dummy_anon_vma);
 	ASSERT_EQ(vma->vm_start, 0x6000);
 	ASSERT_EQ(vma->vm_end, 0x7000);
 	ASSERT_EQ(vma_start_pgoff(vma), 6);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 6);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 6);
 	ASSERT_TRUE(vma_write_started(vma_prev));
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 2);
@@ -983,7 +986,7 @@ static bool __test_merge_existing(bool p
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x7000);
 	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 	ASSERT_EQ(vma_prev->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma_prev));
 	ASSERT_EQ(mm.map_count, 1);
@@ -1016,7 +1019,7 @@ static bool __test_merge_existing(bool p
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x9000);
 	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 	ASSERT_EQ(vma_prev->anon_vma, &dummy_anon_vma);
 	ASSERT_TRUE(vma_write_started(vma_prev));
 	ASSERT_EQ(mm.map_count, 1);
@@ -1146,7 +1149,8 @@ static bool test_anon_vma_non_mergeable(
 	ASSERT_EQ(vmg.state, VMA_MERGE_SUCCESS);
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x7000);
-	ASSERT_EQ(vma_prev->vm_pgoff, 0);
+	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 	ASSERT_TRUE(vma_write_started(vma_prev));
 	ASSERT_FALSE(vma_write_started(vma_next));
 
@@ -1177,7 +1181,8 @@ static bool test_anon_vma_non_mergeable(
 	ASSERT_EQ(vmg.state, VMA_MERGE_SUCCESS);
 	ASSERT_EQ(vma_prev->vm_start, 0);
 	ASSERT_EQ(vma_prev->vm_end, 0x7000);
-	ASSERT_EQ(vma_prev->vm_pgoff, 0);
+	ASSERT_EQ(vma_start_pgoff(vma_prev), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma_prev), 0);
 	ASSERT_TRUE(vma_write_started(vma_prev));
 	ASSERT_FALSE(vma_write_started(vma_next));
 
@@ -1439,7 +1444,7 @@ static bool test_merge_extend(void)
 	ASSERT_EQ(vma->vm_start, 0);
 	ASSERT_EQ(vma->vm_end, 0x4000);
 	ASSERT_EQ(vma_start_pgoff(vma), 0);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(mm.map_count, 1);
 
@@ -1480,7 +1485,7 @@ static bool test_expand_only_mode(void)
 	ASSERT_EQ(vma->vm_start, 0x3000);
 	ASSERT_EQ(vma->vm_end, 0x9000);
 	ASSERT_EQ(vma_start_pgoff(vma), 3);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 3);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 3);
 	ASSERT_TRUE(vma_write_started(vma));
 	ASSERT_EQ(vma_iter_addr(&vmi), 0x3000);
 	vma_assert_attached(vma);
--- a/tools/testing/vma/tests/mmap.c~b
+++ a/tools/testing/vma/tests/mmap.c
@@ -74,7 +74,7 @@ static bool test_pure_anon_dev_zero(void
 
 	/*
 	 * Map a MAP_PRIVATE-/dev/zero mapping at address 0x300000 with a page
-	 * offset of 0x10, which we expect to be reset to the virtual page
+	 * offset of 0x10, which we expect to be reset to the anonymous page
 	 * offset.
 	 */
 	addr = __mmap_region(&file, 0x300000, 0x3000, vma_flags, 0x10, NULL);
@@ -86,9 +86,9 @@ static bool test_pure_anon_dev_zero(void
 	ASSERT_TRUE(vma_is_anonymous(vma));
 	ASSERT_EQ(vma->vm_file, NULL);
 	ASSERT_EQ(vma->vm_private_data, NULL);
-	/* Expect virtual page offsets. */
+	/* Expect anonymous page offsets. */
 	ASSERT_EQ(vma->vm_pgoff, 0x300);
-	ASSERT_EQ(vma_start_virt_pgoff(vma), 0x300);
+	ASSERT_EQ(vma_start_anon_pgoff(vma), 0x300);
 
 	cleanup_mm(&mm, &vmi);
 	return true;
--- a/tools/testing/vma/tests/vma.c~b
+++ a/tools/testing/vma/tests/vma.c
@@ -33,7 +33,51 @@ static bool test_copy_vma(void)
 	struct mm_struct mm = {};
 	bool need_locks = false;
 	VMA_ITERATOR(vmi, &mm, 0);
-	struct vm_area_struct *vma, *vma_new, *vma_next;
+	struct vm_area_struct *vma, *vma_prev, *vma_new, *vma_next, *vma_orig;
+
+	/* Move forwards, adjacent to old self - self-merge. */
+
+	vma = alloc_and_link_vma(&mm, 0x1000, 0x2000, 1, vma_flags);
+	vma_set_anonymous(vma);
+	vma_orig = vma;
+	vma_new = copy_vma(&vma, 0x2000, 0x1000, 1, 1, &need_locks);
+	ASSERT_EQ(vma_new, vma_orig);
+	ASSERT_EQ(vma, vma_orig);
+	ASSERT_EQ(vma_new->vm_start, 0x1000);
+	ASSERT_EQ(vma_new->vm_end, 0x3000);
+
+	cleanup_mm(&mm, &vmi);
+
+	/* Move backwards, adjacent to old self - self-merge. */
+
+	vma = alloc_and_link_vma(&mm, 0x2000, 0x3000, 2, vma_flags);
+	vma_set_anonymous(vma);
+	vma_orig = vma;
+	vma_new = copy_vma(&vma, 0x1000, 0x1000, 2, 2, &need_locks);
+	ASSERT_EQ(vma_new, vma_orig);
+	ASSERT_EQ(vma, vma_orig);
+	ASSERT_EQ(vma_new->vm_start, 0x1000);
+	ASSERT_EQ(vma_new->vm_end, 0x3000);
+
+	cleanup_mm(&mm, &vmi);
+
+	/*
+	 * Move backwards between prior VMA and old self - self-merge and vma
+	 * updated to a new VMA.
+	 */
+
+	vma_prev = alloc_and_link_vma(&mm, 0x1000, 0x2000, 1, vma_flags);
+	vma_set_anonymous(vma_prev);
+	vma = alloc_and_link_vma(&mm, 0x3000, 0x4000, 3, vma_flags);
+	vma_set_anonymous(vma);
+	vma_orig = vma;
+	vma_new = copy_vma(&vma, 0x2000, 0x1000, 3, 3, &need_locks);
+	ASSERT_NE(vma_new, vma_orig);
+	ASSERT_EQ(vma_new, vma);
+	ASSERT_EQ(vma_new->vm_start, 0x1000);
+	ASSERT_EQ(vma_new->vm_end, 0x4000);
+
+	cleanup_mm(&mm, &vmi);
 
 	/* Move backwards and do not merge. */
 
_