[PATCH] mm/vma: use vmg->target to specify target VMA for new VMA merge

[PATCH] mm/vma: use vmg->target to specify target VMA for new VMA merge

[Lorenzo Stoakes]

In commit 3a75ccba047b ("mm: simplify vma merge structure and expand
comments") we introduced the vmg->target field to make the merging of
existing VMAs simpler - clarifying precisely which VMA would eventually
become the merged VMA once the merge operation was complete.

New VMA merging did not get quite the same treatment, retaining the rather
confusing convention of storing the target VMA in vmg->middle.

This patch corrects this state of affairs, utilising vmg->target for this
purpose for both vma_merge_new_range() and also for vma_expand().

We retain the WARN_ON for vmg->middle being specified in
vma_merge_new_range() as doing so would make no sense, but add an
additional debug assert for setting vmg->target.

This patch additionally updates VMA userland testing to account for this
change.

Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
---
 mm/vma.c                | 36 +++++++++++++++++++-----------------
 mm/vma_exec.c           |  2 +-
 tools/testing/vma/vma.c |  6 +++---
 3 files changed, 23 insertions(+), 21 deletions(-)

diff --git a/mm/vma.c b/mm/vma.c
index bfbc56df190c..a58b39b4eeb1 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -1028,6 +1028,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)

 	mmap_assert_write_locked(vmg->mm);
 	VM_WARN_ON_VMG(vmg->middle, vmg);
+	VM_WARN_ON_VMG(vmg->target, vmg);
 	/* vmi must point at or before the gap. */
 	VM_WARN_ON_VMG(vma_iter_addr(vmg->vmi) > end, vmg);

@@ -1043,13 +1044,13 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
 	/* If we can merge with the next VMA, adjust vmg accordingly. */
 	if (can_merge_right) {
 		vmg->end = next->vm_end;
-		vmg->middle = next;
+		vmg->target = next;
 	}

 	/* If we can merge with the previous VMA, adjust vmg accordingly. */
 	if (can_merge_left) {
 		vmg->start = prev->vm_start;
-		vmg->middle = prev;
+		vmg->target = prev;
 		vmg->pgoff = prev->vm_pgoff;

 		/*
@@ -1071,10 +1072,10 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
 	 * Now try to expand adjacent VMA(s). This takes care of removing the
 	 * following VMA if we have VMAs on both sides.
 	 */
-	if (vmg->middle && !vma_expand(vmg)) {
-		khugepaged_enter_vma(vmg->middle, vmg->flags);
+	if (vmg->target && !vma_expand(vmg)) {
+		khugepaged_enter_vma(vmg->target, vmg->flags);
 		vmg->state = VMA_MERGE_SUCCESS;
-		return vmg->middle;
+		return vmg->target;
 	}

 	return NULL;
@@ -1086,27 +1087,29 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
  * @vmg: Describes a VMA expansion operation.
  *
  * Expand @vma to vmg->start and vmg->end.  Can expand off the start and end.
- * Will expand over vmg->next if it's different from vmg->middle and vmg->end ==
- * vmg->next->vm_end.  Checking if the vmg->middle can expand and merge with
+ * Will expand over vmg->next if it's different from vmg->target and vmg->end ==
+ * vmg->next->vm_end.  Checking if the vmg->target can expand and merge with
  * vmg->next needs to be handled by the caller.
  *
  * Returns: 0 on success.
  *
  * ASSUMPTIONS:
- * - The caller must hold a WRITE lock on vmg->middle->mm->mmap_lock.
- * - The caller must have set @vmg->middle and @vmg->next.
+ * - The caller must hold a WRITE lock on vmg->target->mm->mmap_lock.
+ * - The caller must have set @vmg->target and @vmg->next.
  */
 int vma_expand(struct vma_merge_struct *vmg)
 {
 	struct vm_area_struct *anon_dup = NULL;
 	bool remove_next = false;
-	struct vm_area_struct *middle = vmg->middle;
+	struct vm_area_struct *target = vmg->target;
 	struct vm_area_struct *next = vmg->next;

+	VM_WARN_ON_VMG(!target, vmg);
+
 	mmap_assert_write_locked(vmg->mm);

-	vma_start_write(middle);
-	if (next && (middle != next) && (vmg->end == next->vm_end)) {
+	vma_start_write(target);
+	if (next && (target != next) && (vmg->end == next->vm_end)) {
 		int ret;

 		remove_next = true;
@@ -1117,19 +1120,18 @@ int vma_expand(struct vma_merge_struct *vmg)
 		 * In this case we don't report OOM, so vmg->give_up_on_mm is
 		 * safe.
 		 */
-		ret = dup_anon_vma(middle, next, &anon_dup);
+		ret = dup_anon_vma(target, next, &anon_dup);
 		if (ret)
 			return ret;
 	}

 	/* Not merging but overwriting any part of next is not handled. */
 	VM_WARN_ON_VMG(next && !remove_next &&
-		       next != middle && vmg->end > next->vm_start, vmg);
+		       next != target && vmg->end > next->vm_start, vmg);
 	/* Only handles expanding */
-	VM_WARN_ON_VMG(middle->vm_start < vmg->start ||
-		       middle->vm_end > vmg->end, vmg);
+	VM_WARN_ON_VMG(target->vm_start < vmg->start ||
+		       target->vm_end > vmg->end, vmg);

-	vmg->target = middle;
 	if (remove_next)
 		vmg->__remove_next = true;

diff --git a/mm/vma_exec.c b/mm/vma_exec.c
index 2dffb02ed6a2..922ee51747a6 100644
--- a/mm/vma_exec.c
+++ b/mm/vma_exec.c
@@ -54,7 +54,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)
 	/*
 	 * cover the whole range: [new_start, old_end)
 	 */
-	vmg.middle = vma;
+	vmg.target = vma;
 	if (vma_expand(&vmg))
 		return -ENOMEM;

diff --git a/tools/testing/vma/vma.c b/tools/testing/vma/vma.c
index 238acd4e20fd..61a67aa6977c 100644
--- a/tools/testing/vma/vma.c
+++ b/tools/testing/vma/vma.c
@@ -400,7 +400,7 @@ static bool test_simple_expand(void)
 	VMA_ITERATOR(vmi, &mm, 0);
 	struct vma_merge_struct vmg = {
 		.vmi = &vmi,
-		.middle = vma,
+		.target = vma,
 		.start = 0,
 		.end = 0x3000,
 		.pgoff = 0,
@@ -1318,7 +1318,7 @@ static bool test_dup_anon_vma(void)
 	vma_next->anon_vma = &dummy_anon_vma;

 	vmg_set_range(&vmg, 0, 0x5000, 0, flags);
-	vmg.middle = vma_prev;
+	vmg.target = vma_prev;
 	vmg.next = vma_next;

 	ASSERT_EQ(expand_existing(&vmg), 0);
@@ -1501,7 +1501,7 @@ static bool test_vmi_prealloc_fail(void)
 	vma->anon_vma = &dummy_anon_vma;

 	vmg_set_range(&vmg, 0, 0x5000, 3, flags);
-	vmg.middle = vma_prev;
+	vmg.target = vma_prev;
 	vmg.next = vma;

 	fail_prealloc = true;
--
2.49.0

Re: [PATCH] mm/vma: use vmg->target to specify target VMA for new VMA merge

[Vlastimil Babka]

On 6/13/25 20:48, Lorenzo Stoakes wrote:
> In commit 3a75ccba047b ("mm: simplify vma merge structure and expand
> comments") we introduced the vmg->target field to make the merging of
> existing VMAs simpler - clarifying precisely which VMA would eventually
> become the merged VMA once the merge operation was complete.
> 
> New VMA merging did not get quite the same treatment, retaining the rather
> confusing convention of storing the target VMA in vmg->middle.
> 
> This patch corrects this state of affairs, utilising vmg->target for this
> purpose for both vma_merge_new_range() and also for vma_expand().
> 
> We retain the WARN_ON for vmg->middle being specified in
> vma_merge_new_range() as doing so would make no sense, but add an
> additional debug assert for setting vmg->target.
> 
> This patch additionally updates VMA userland testing to account for this
> change.
> 
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>

Reviewed-by: Vlastimil Babka <vbabka@suse.cz>

Nit below:

> @@ -1086,27 +1087,29 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
>   * @vmg: Describes a VMA expansion operation.
>   *
>   * Expand @vma to vmg->start and vmg->end.  Can expand off the start and end.
> - * Will expand over vmg->next if it's different from vmg->middle and vmg->end ==
> - * vmg->next->vm_end.  Checking if the vmg->middle can expand and merge with
> + * Will expand over vmg->next if it's different from vmg->target and vmg->end ==
> + * vmg->next->vm_end.  Checking if the vmg->target can expand and merge with
>   * vmg->next needs to be handled by the caller.
>   *
>   * Returns: 0 on success.
>   *
>   * ASSUMPTIONS:
> - * - The caller must hold a WRITE lock on vmg->middle->mm->mmap_lock.
> - * - The caller must have set @vmg->middle and @vmg->next.
> + * - The caller must hold a WRITE lock on vmg->target->mm->mmap_lock.

The assert uses vmg->mm so maybe the comment should do the same? (IIRC mm
was added only later to vmg?)

> + * - The caller must have set @vmg->target and @vmg->next.
>   */
>  int vma_expand(struct vma_merge_struct *vmg)
>  {
>  	struct vm_area_struct *anon_dup = NULL;
>  	bool remove_next = false;
> -	struct vm_area_struct *middle = vmg->middle;
> +	struct vm_area_struct *target = vmg->target;
>  	struct vm_area_struct *next = vmg->next;
> 
> +	VM_WARN_ON_VMG(!target, vmg);
> +
>  	mmap_assert_write_locked(vmg->mm);
> 
> -	vma_start_write(middle);
> -	if (next && (middle != next) && (vmg->end == next->vm_end)) {
> +	vma_start_write(target);
> +	if (next && (target != next) && (vmg->end == next->vm_end)) {
>  		int ret;
> 
>  		remove_next = true;

Re: [PATCH] mm/vma: use vmg->target to specify target VMA for new VMA merge

[Lorenzo Stoakes]

On Thu, Jun 19, 2025 at 03:44:21PM +0200, Vlastimil Babka wrote:
> On 6/13/25 20:48, Lorenzo Stoakes wrote:
> > In commit 3a75ccba047b ("mm: simplify vma merge structure and expand
> > comments") we introduced the vmg->target field to make the merging of
> > existing VMAs simpler - clarifying precisely which VMA would eventually
> > become the merged VMA once the merge operation was complete.
> >
> > New VMA merging did not get quite the same treatment, retaining the rather
> > confusing convention of storing the target VMA in vmg->middle.
> >
> > This patch corrects this state of affairs, utilising vmg->target for this
> > purpose for both vma_merge_new_range() and also for vma_expand().
> >
> > We retain the WARN_ON for vmg->middle being specified in
> > vma_merge_new_range() as doing so would make no sense, but add an
> > additional debug assert for setting vmg->target.
> >
> > This patch additionally updates VMA userland testing to account for this
> > change.
> >
> > Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
>
> Reviewed-by: Vlastimil Babka <vbabka@suse.cz>

Thanks!

>
> Nit below:
>
> > @@ -1086,27 +1087,29 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
> >   * @vmg: Describes a VMA expansion operation.
> >   *
> >   * Expand @vma to vmg->start and vmg->end.  Can expand off the start and end.
> > - * Will expand over vmg->next if it's different from vmg->middle and vmg->end ==
> > - * vmg->next->vm_end.  Checking if the vmg->middle can expand and merge with
> > + * Will expand over vmg->next if it's different from vmg->target and vmg->end ==
> > + * vmg->next->vm_end.  Checking if the vmg->target can expand and merge with
> >   * vmg->next needs to be handled by the caller.
> >   *
> >   * Returns: 0 on success.
> >   *
> >   * ASSUMPTIONS:
> > - * - The caller must hold a WRITE lock on vmg->middle->mm->mmap_lock.
> > - * - The caller must have set @vmg->middle and @vmg->next.
> > + * - The caller must hold a WRITE lock on vmg->target->mm->mmap_lock.
>
> The assert uses vmg->mm so maybe the comment should do the same? (IIRC mm
> was added only later to vmg?)

Haha you are the master of spotting stuff like this :)

Yeah you're right, this was actually updated for vma_merge_new_range() and
vma_merge_existing_range(), but missed this one.

Since we're updating it we may as well fix it, will send a fix-patch at
top-level :>)

>
> > + * - The caller must have set @vmg->target and @vmg->next.

Re: [PATCH] mm/vma: use vmg->target to specify target VMA for new VMA merge

[Lorenzo Stoakes]

Hi Andrew,

Small fix-patch to correct a comment here as per Vlastimil's feedback.

Thanks, Lorenzo

----8<----
From 0a84d237b05e6cf801933267488c8353ced7913b Mon Sep 17 00:00:00 2001
From: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Date: Thu, 19 Jun 2025 15:32:57 +0100
Subject: [PATCH] mm: make comment consistent in vma_expand()

The VMA merge structure contains a pointer to mm_struct so we don't need to
determine it from an input VMA's vm_mm field. Update the comment to reflect
this and make it consistent with vma_merge_new_range() and
vma_merge_existing_range().

Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
---
 mm/vma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vma.c b/mm/vma.c
index 13794a0ac5fe..4abed296d882 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -1094,7 +1094,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
  * Returns: 0 on success.
  *
  * ASSUMPTIONS:
- * - The caller must hold a WRITE lock on vmg->target->mm->mmap_lock.
+ * - The caller must hold a WRITE lock on the mm_struct->mmap_lock.
  * - The caller must have set @vmg->target and @vmg->next.
  */
 int vma_expand(struct vma_merge_struct *vmg)
--
2.49.0