[PATCH v2] mm: nommu: free unused resources when mremap shrinks the vma

[Hajime Tazaki <thehajime@gmail.com>]

When shrinking a VMA via mremap, the bounds are modified directly:
mm/nommu.c:do_mremap() {
    ...
    vma->vm_end = vma->vm_start + new_len;
    ...
}
This shrink the VMA without updating its bounds in the maple tree.
If the maple tree (mm->mm_mt) still contains the old bounds, a user
process could access the freed portion. The stale maple tree would
incorrectly return the shrunk VMA for an address past its new vm_end.

This commit fixes this issue by calling vmi_shrink_vma() when shrink
happens.  Additionally, if a file-backed map is to be shrunk, it reports
-EINVAL like do_munmap() does.

The issue is reported by Sashiko review, linked below.

Link: https://sashiko.dev/#/patchset/20260702012546.665383-1-thehajime@gmail.com
Link: https://sashiko.dev/#/patchset/20260710021028.892645-1-thehajime%40gmail.com
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>

--

v1 -> v2:
- handle error when vmi_shrink_vma() failed (reported by Sashiko)
- prevents mremap() with being shrunk for file-backed one like munmap()

v1: https://lore.kernel.org/linux-mm/20260710021028.892645-1-thehajime@gmail.com/
---
 mm/nommu.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index 852ec9bd0505..c8effbffa0f0 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1597,8 +1597,26 @@ static unsigned long do_mremap(unsigned long addr,
 	if (new_len > vma->vm_region->vm_end - vma->vm_region->vm_start)
 		return (unsigned long) -ENOMEM;
 
+	/* like do_munmap(), we're allowed to shrink an anonymous VMA but not a file-backed one */
+	if (vma->vm_file)
+		return (unsigned long) -EINVAL;
+
 	/* all checks complete - do it */
-	vma->vm_end = vma->vm_start + new_len;
+	if (new_len < old_len) {
+		/* shrink only happens addr + new_len and old_len are in different pages */
+		VMA_ITERATOR(vmi, current->mm, addr);
+		int ret;
+
+		/* vmi_shrink_vma() needs from/to pointers to be removed,
+		 * (mainly used in munmap) so, specify them.
+		 */
+		ret = vmi_shrink_vma(&vmi, vma, addr + new_len, addr + old_len);
+		if (ret < 0)
+			return (unsigned long) ret;
+	} else {
+		/* when there are no shrink, update vma.  */
+		vma->vm_end = vma->vm_start + new_len;
+	}
 	return vma->vm_start;
 }
 
-- 
2.43.0