[RFC PATCH 2/6] mm: nommu: use vma_is_anonymous() to check if vmas are anonymous

[Hajime Tazaki <thehajime@gmail.com>]

Private file mappings (like those from /dev/zero) can have vma->vm_file
set but remain structurally anonymous since they lack vm_ops. Testing
vma->vm_file instead of vma_is_anonymous(vma) might cause mremap to
return a spurious -EINVAL when userspace attempts to shrink these
mappings.

This commit fixes this issue by using vma_is_anonymous() instead of
testing vma->vm_file to address the case of /dev/zero.

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: linux-mm@kvack.org
Closes: https://sashiko.dev/#/patchset/20260710054648.924005-1-thehajime%40gmail.com
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 mm/nommu.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index 89444ee2aca6..e40990e15831 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1196,7 +1196,7 @@ unsigned long do_mmap(struct file *file,
 	add_nommu_region(region);
 
 	/* clear anonymous mappings that don't ask for uninitialized data */
-	if (!vma->vm_file &&
+	if (vma_is_anonymous(vma) &&
 	    (!IS_ENABLED(CONFIG_MMAP_ALLOW_UNINITIALIZED) ||
 	     !(flags & MAP_UNINITIALIZED)))
 		memset((void *)region->vm_start, 0,
@@ -1328,7 +1328,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
 
 	/* we're only permitted to split anonymous regions (these should have
 	 * only a single usage on the region) */
-	if (vma->vm_file)
+	if (!vma_is_anonymous(vma))
 		return -ENOMEM;
 
 	mm = vma->vm_mm;
@@ -1484,7 +1484,7 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, struct list
 	}
 
 	/* we're allowed to split an anonymous VMA but not a file-backed one */
-	if (vma->vm_file) {
+	if (!vma_is_anonymous(vma)) {
 		do {
 			if (start > vma->vm_start)
 				return -EINVAL;
@@ -1617,7 +1617,7 @@ static unsigned long do_mremap(unsigned long addr,
 		/* like do_munmap(), we're allowed to shrink an anonymous VMA but not
 		 * a file-backed one
 		 */
-		if (vma->vm_file)
+		if (!vma_is_anonymous(vma))
 			return (unsigned long) -EINVAL;
 
 		/* vmi_shrink_vma() needs from/to pointers to be removed,
-- 
2.43.0