From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C07039BFF1; Fri, 10 Jul 2026 20:28:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783715325; cv=none; b=gems00VxMrL+Cq33B6LHPGoTCZnLwHTsLpcwt1dQ1e0jdbVj7XfPByTsJFRDdfVt6frtz7j9Ct/7HhCkDCZbbk0bJ2mUt4wVkbplA5sTuBqOE3kgfxnruUY0eC5JDfzN13VyjZDSXLmdY0I0bFbXSccEA3GUY0u3a1hhhm0JVn8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783715325; c=relaxed/simple; bh=lsNQUYSqY8ftQ/S4LD1hHSt9yR5KwVcyJgXwScgbugs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MfwerTzhPNW3suNLU3c5NsKBOdpZ0ft/WIDx7j+vyeHpxAxZYRpq4O1E1DPzEIP2ma18QZrju9UYeAw5yyYEZ2IRNURr/HOi5W6w2JNXRFkHr6nzYUCIAjBibDuI5K5NIeMySwXQTnWRFARc6VX1MrD4TYAOSGntc0zTh9H2IYQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UJnlrTyq; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UJnlrTyq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3DCC91F00A3A; Fri, 10 Jul 2026 20:28:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783715323; bh=2+OkEViyUBUTI9uYLEX4a4/L0I2mHL2JL8LovbGGiMs=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=UJnlrTyqmIX76dhyuIJH+ZWeuwY2uSvegVNMHFmEwJS5ocaNZYISJS5JyWmBkSzbH UKl7cX8yUe6IAwGKYMBvQ1AtAheCCYRuKTz3ZwBjHtWknVjpRaA2L1umW1nd+gocfN uYSLyXTLS9g/m49T3bKQBwJ6bVtk02YqDXiOTC3Z0Ae1w7haCafZThtvQIsOsq/xD2 TyOEZJuaBvJm3nFlQnmL5EMx7WPZf7duGBLnLNoh4wgb9e/im1OGNIk33fMzJZFzeH 8IMw/hMYKsEbQCSkfHU9sEt4oreIes1GlIljfxrfCHA321k3WfSaKpA6UXcsWBD0jS PXToqOuC7BRlA== From: Lorenzo Stoakes Date: Fri, 10 Jul 2026 21:17:10 +0100 Subject: [PATCH v2 29/33] mm/vma: introduce and use vma_set_pgoff() Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260710-b4-pre-scalable-cow-v2-29-2a5aa403d977@kernel.org> References: <20260710-b4-pre-scalable-cow-v2-0-2a5aa403d977@kernel.org> In-Reply-To: <20260710-b4-pre-scalable-cow-v2-0-2a5aa403d977@kernel.org> To: Andrew Morton , David Hildenbrand , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Jann Horn , Lance Yang , Pedro Falcato , Russell King , Dinh Nguyen , Simon Schuster , "James E.J. Bottomley" , Helge Deller , Alexander Viro , Christian Brauner , Jan Kara , Dan Williams , Matthew Wilcox , Muchun Song , Oscar Salvador , Masami Hiramatsu , Oleg Nesterov , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , James Clark , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Miaohe Lin , Naoya Horiguchi , Xu Xin , Chengming Zhou , SJ Park , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Hugh Dickins , Peter Xu , Kees Cook , Marek Szyprowski , Robin Murphy , Andrey Konovalov , Alexander Potapenko , Dmitry Vyukov , Steven Rostedt , Mathieu Desnoyers , Jarkko Sakkinen , Dave Hansen , Thomas Gleixner , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , Ian Abbott , H Hartley Sweeten , Lucas Stach , Christian Gmeiner , David Airlie , Simona Vetter , Patrik Jakobsson , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Rob Clark , Dmitry Baryshkov , Abhinav Kumar , Jessica Zhang , Sean Paul , Marijn Suijten , Tomi Valkeinen , Thierry Reding , Mikko Perttunen , Jonathan Hunter , Christian Koenig , Huang Rui , Matthew Auld , Jason Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Ankit Agrawal , Alex Williamson , Paolo Bonzini , Shakeel Butt , Usama Arif Cc: Lorenzo Stoakes , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-parisc@vger.kernel.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-perf-users@vger.kernel.org, linux-trace-kernel@vger.kernel.org, damon@lists.linux.dev, iommu@lists.linux.dev, kasan-dev@googlegroups.com, linux-sgx@vger.kernel.org, etnaviv@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, freedreno@lists.freedesktop.org, linux-tegra@vger.kernel.org, kvm@vger.kernel.org, Russell King X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6848; i=ljs@kernel.org; h=from:subject:message-id; bh=lsNQUYSqY8ftQ/S4LD1hHSt9yR5KwVcyJgXwScgbugs=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLICg21rKw7tulL8I+ZRxy4Dt53VciHnrd73X9PobOva3 V6pJ2fbUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgIlYvGX4Xz5p+j7GExfu3Dws dUiu/h7j+1myno/eP78s28f/8YCc51SG/4VH5p2X8H/x3WJ6GqdZs/vz0HfMsw8mG4jETHd4vdf pKRsA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 In order to lay the foundation for work that permits us to track the virtual page offset of MAP_PRIVATE file-backed mappings, we abstract the assignment of vma->vm_pgoff to vma_set_pgoff(). We additionally add a lock check here using the newly introduced vma_assert_can_modify(). This asserts the VMA write lock if the VMA is attached. We also assert that, if this is an anonymous VMA and unfaulted, that its (virtual) page offset is equal to the page offset of the VMA's address. We must be careful about MAP_PRIVATE-/dev/zero which violates fundamental assumptions about anonymous memory, so we check for !vma->vm_file after using vma_is_anonymous() which these mappings satisfy. Additionally, we only perform the assert if CONFIG_MMU is defined, as nommu does not set vma->vm_pgoff = addr >> PAGE_SHIFT. This isn't really relevant to rmap as it has no anon rmap (nor needs it), but we must avoid it asserting falsely. All of this logic is kept in assert_sane_pgoff() to keep things clear. In order to maintain correctness given this assert, we also update __install_special_mapping() to invoke vma_set_range() after it's set vma->vm_ops (which determine whether the VMA is anonymous or not). We do not use vma_set_pgoff() in vm_area_init_from(), as at the point of forking, we don't necessarily have correct locking state. Updating vma_set_range() covers most cases, but in addition to this we also update insert_vm_struct(), compat_set_vma_from_desc() and nommu callers. We also update vma_add_pgoff() and vma_sub_pgoff() to use vma_set_pgoff(). While we're here, we drop a BUG_ON() and update insert_vm_struct()'s comment to reflect the fact anonymous mappings can be added here. Finally, we update the CONFIG_MMU, CONFIG_PER_VMA_LOCK defines in the VMA userland tests so IS_ENABLED() will work correctly with them. No functional change intended. Reviewed-by: Pedro Falcato Signed-off-by: Lorenzo Stoakes --- mm/nommu.c | 2 +- mm/vma.c | 14 +++++++------- mm/vma.h | 35 ++++++++++++++++++++++++++++++++--- tools/testing/vma/vma_internal.h | 4 ++-- 4 files changed, 42 insertions(+), 13 deletions(-) diff --git a/mm/nommu.c b/mm/nommu.c index 2a0136f6081d..21cbe8b093fc 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -1062,7 +1062,7 @@ unsigned long do_mmap(struct file *file, region->vm_pgoff = pgoff; vm_flags_init(vma, vm_flags); - vma->vm_pgoff = pgoff; + vma_set_pgoff(vma, pgoff); if (file) { region->vm_file = get_file(file); diff --git a/mm/vma.c b/mm/vma.c index ec64e179d2f9..7c75dd78edef 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -81,7 +81,7 @@ static void vma_set_range(struct vm_area_struct *vma, unsigned long start, unsigned long end, pgoff_t pgoff) { __vma_set_range(vma, start, end); - vma->vm_pgoff = pgoff; + vma_set_pgoff(vma, pgoff); } /* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */ @@ -3347,9 +3347,9 @@ int __vm_munmap(unsigned long start, size_t len, bool unlock) return ret; } -/* Insert vm structure into process list sorted by address - * and into the inode's i_mmap tree. If vm_file is non-NULL - * then i_mmap_rwsem is taken here. +/* + * Insert vm structure into process list sorted by address + * and into the inode's i_mmap tree if file-backed. */ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma) { @@ -3375,8 +3375,8 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma) * Similarly in do_mmap and in do_brk_flags. */ if (vma_is_anonymous(vma)) { - BUG_ON(vma->anon_vma); - vma->vm_pgoff = vma->vm_start >> PAGE_SHIFT; + WARN_ON_ONCE(vma->anon_vma); + vma_set_pgoff(vma, vma->vm_start >> PAGE_SHIFT); } if (vma_link(mm, vma)) { @@ -3422,7 +3422,6 @@ struct vm_area_struct *__install_special_mapping( if (unlikely(vma == NULL)) return ERR_PTR(-ENOMEM); - vma_set_range(vma, addr, addr + len, 0); vm_flags |= mm->def_flags | VM_DONTEXPAND; if (pgtable_supports_soft_dirty()) vm_flags |= VM_SOFTDIRTY; @@ -3431,6 +3430,7 @@ struct vm_area_struct *__install_special_mapping( vma->vm_ops = ops; vma->vm_private_data = priv; + vma_set_range(vma, addr, addr + len, 0); ret = insert_vm_struct(mm, vma); if (ret) diff --git a/mm/vma.h b/mm/vma.h index 40effaa3ebe4..58f48609ce22 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -247,16 +247,45 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg) return vmg_start_pgoff(vmg) + vmg_pages(vmg); } +static inline void assert_sane_pgoff(struct vm_area_struct *vma, pgoff_t pgoff) +{ + /* nommu doesn't set a virtual pgoff for anon VMAs. */ + if (!IS_ENABLED(CONFIG_MMU)) + return; + /* + * File-backed VMAs have arbitrary page offset (either page offset into + * file or for pfnmap the PFN of the start of the range or drivers may + * set arbitrary page offset). + */ + if (!vma_is_anonymous(vma)) + return; + /* MAP_PRIVATE-/dev/zero is anon, non-NULL vm_file, but has file pgoff. */ + if (vma->vm_file) + return; + /* If faulted in, could have been remapped. */ + if (vma->anon_vma) + return; + /* OK this is really an anon VMA - expect virtual page offset. */ + VM_WARN_ON_ONCE(pgoff != vma->vm_start >> PAGE_SHIFT); +} + +static inline void vma_set_pgoff(struct vm_area_struct *vma, pgoff_t pgoff) +{ + vma_assert_can_modify(vma); + assert_sane_pgoff(vma, pgoff); + vma->vm_pgoff = pgoff; +} + static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta) { vma_assert_can_modify(vma); - vma->vm_pgoff += delta; + vma_set_pgoff(vma, vma_start_pgoff(vma) + delta); } static inline void vma_sub_pgoff(struct vm_area_struct *vma, pgoff_t delta) { vma_assert_can_modify(vma); - vma->vm_pgoff -= delta; + vma_set_pgoff(vma, vma_start_pgoff(vma) - delta); } #define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ @@ -331,7 +360,7 @@ static inline void compat_set_vma_from_desc(struct vm_area_struct *vma, */ /* Mutable fields. Populated with initial state. */ - vma->vm_pgoff = desc->pgoff; + vma_set_pgoff(vma, desc->pgoff); if (desc->vm_file != vma->vm_file) vma_set_file(vma, desc->vm_file); vma->flags = desc->vma_flags; diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index e12ab2c80f95..4f6c5666ac07 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -14,8 +14,8 @@ #include -#define CONFIG_MMU -#define CONFIG_PER_VMA_LOCK +#define CONFIG_MMU 1 +#define CONFIG_PER_VMA_LOCK 1 #ifdef __CONCAT #undef __CONCAT -- 2.55.0