* Re: [PATCH v2 00/33] mm: make VMA page offset handling more consistent
From: Lorenzo Stoakes (ARM) @ 2026-07-16 13:31 UTC (permalink / raw)
To: Christian König
Cc: linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King,
Ackerley Tng, Kai Huang
In-Reply-To: <c2fd5114-74d0-4996-8035-311ba976b3eb@amd.com>
On Thu, Jul 16, 2026 at 12:03:41PM +0200, Christian König wrote:
>
> I only skimmed over the set but should there be any changes required to DMA-buf, amdgpu or radeon driver feel free to add
>
> Acked-by: Christian König <christian.koenig@amd.com>
>
> and merge upstream through any branch you feel fit.
>
> Regards,
> Christian.
Thanks :)
Cheers, Lorenzo
^ permalink raw reply
* Re: [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping
From: Will Deacon @ 2026-07-16 13:24 UTC (permalink / raw)
To: Pu Hu
Cc: mhiramat@kernel.org, ada.coupriediaz@arm.com,
catalin.marinas@arm.com, davem@davemloft.net, Hongyan Xia,
Jiazi Li, linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
naveen@kernel.org, yang@os.amperecomputing.com
In-Reply-To: <20260710063242.228714-3-hupu@transsion.com>
On Fri, Jul 10, 2026 at 06:32:55AM +0000, Pu Hu wrote:
> From: Pu Hu <hupu@transsion.com>
>
> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
> can happen when tracing or perf code runs from the debug exception path
> while the first kprobe is preparing or executing its out-of-line
> single-step instruction.
I don't understand this part. The single-step runs with debug exceptions
disabled (kprobes_save_local_irqflag() sets PSTATE.D) so how do we end
up taking one?
Will
^ permalink raw reply
* Re: [RFC] tracing: Try user copies with page faults disabled first
From: Usama Arif @ 2026-07-16 12:05 UTC (permalink / raw)
To: Steven Rostedt
Cc: linux-kernel, linux-trace-kernel, mathieu.desnoyers, mhiramat,
leitao
In-Reply-To: <20260715151120.0c7b80cb@gandalf.local.home>
On 15/07/2026 20:11, Steven Rostedt wrote:
> On Wed, 15 Jul 2026 08:54:54 -0700
> Usama Arif <usama.arif@linux.dev> wrote:
>
>> trace_user_fault_read() is called with preemption disabled to copy user
>> memory into a per-cpu scratch buffer. The existing implementation enables
>> preemption around the copy because faulting user memory can sleep. That
>> opens a window where another task can run on the same CPU and clobber the
>> per-cpu buffer, so the copy is wrapped in a retry loop: sample
>> nr_context_switches_cpu(), do the preempt-enabled copy, and retry if the
>> counter changed. If this fails to complete 100 times, the function gives up
>> with a warning.
>>
>> nr_context_switches_cpu() reads rq->nr_switches. That counter increments
>> for every context switch on the CPU, not only for switches to tasks that
>> use this tracing scratch buffer. On a heavily loaded system, unrelated
>> scheduler activity can move the counter during every preempt-enabled copy
>> attempt, exhaust the retry guard, and trigger the warning.
>>
>> This is showing up across the Meta fleet around 100 times a day since the
>> kernel began upgrading to 7.1, mostly on arm servers:
>>
>> Error: Too many tries to read user space
>> WARNING: kernel/trace/trace.c:6244 at trace_user_fault_read+0x284/0x2c8, CPU#28: Collection-18/677527
>> CPU: 28 UID: 0 PID: 677527 Comm: Collection-18 Kdump: loaded Not tainted 7.1.0-.... #1 PREEMPTLAZY
>> Hardware name: Quanta Java Island MP 29F0EMA08CH/Java Island, BIOS F0EJ3A16 03/12/2026
>> Call trace:
>> trace_user_fault_read+0x284/0x2c8 (P)
>> syscall_get_data+0x144/0x2c0
>> perf_syscall_enter+0xc0/0x2d8
>> syscall_trace_enter+0x1a0/0x270
>> do_el0_svc+0x54/0xb8
>> el0_svc+0x44/0x268
>> el0t_64_sync_handler+0x7c/0x120
>> el0t_64_sync+0x17c/0x180
>> ---[ end trace 0000000000000000 ]---
>>
>> The retry loop is only needed when preemption must be enabled for the user
>> copy. If the user pages are already resident, the copy can complete without
>> fault handling that sleeps, and preemption can stay disabled throughout.
>>
>> Add a fast path that first tries the copy with page faults disabled. For
>> the plain copy_from_user case, use __copy_from_user_inatomic(). If the
>> probe faults, the architecture exception-table fixup returns a non-zero
>> not-copied count and trace_user_fault_read() falls back to the existing
>> preempt-enabled slow path.
>>
>> Custom copy callbacks need the same behavior. Update the syscall argument
>> copy callbacks to report a non-zero return only when the pagefault-disabled
>> probe faults. With page faults enabled, keep their previous behavior:
>> record the syscall event and omit only the individual user argument that
>> still cannot be copied.
>>
>> The slow path remains in place for nonresident pages and permanent copy
>> failures. nr_context_switches_cpu() still overcounts, but the retry loop is
>> now avoided for the common resident-page case that does not need fault
>> handling.
>
> This was reported also under memory load.
>
> I have a patch that will only do a retry if another *user* task schedules
> in, and will not be bothered by kernel tasks (which may be scheduled in due
> to the copy from user to begin with).
>
> Can you see if that works too?
>
> https://lore.kernel.org/all/20260710083357.49e05ff6@gandalf.local.home/
>
Thanks, I will try and put this in the next kernel, although it will take sometime
for the new kernel to roll in the fleet and to see if the warnings disappear.
^ permalink raw reply
* Re: [PATCH v3 06/11] mm/cma: Allow dynamically creating CMA areas
From: Marek Szyprowski @ 2026-07-16 10:43 UTC (permalink / raw)
To: Thierry Reding
Cc: David Hildenbrand (Arm), Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Jonathan Hunter, Mikko Perttunen, Yury Norov,
Rasmus Villemoes, Russell King, Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Andrew Morton,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Robin Murphy, Sumit Semwal,
Benjamin Gaignard, Brian Starkey, John Stultz, T.J. Mercier,
Christian König, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Catalin Marinas, Will Deacon, devicetree,
linux-tegra, linux-kernel, dri-devel, linux-media,
linux-arm-kernel, linux-s390, linux-mm, iommu, linaro-mm-sig,
linux-trace-kernel
In-Reply-To: <ak_E4UxfnFJywJa7@orome>
On 09.07.2026 17:59, Thierry Reding wrote:
> On Thu, Jul 09, 2026 at 07:56:45AM +0200, Marek Szyprowski wrote:
>> On 08.07.2026 10:35, David Hildenbrand (Arm) wrote:
>>> On 7/7/26 12:02, Marek Szyprowski wrote:
>>>> On 01.07.2026 18:08, Thierry Reding wrote:
>>>>> From: Thierry Reding <treding@nvidia.com>
>>>>>
>>>>> There is no technical reason why there should be a limited number of CMA
>>>>> regions, so extract some code into helpers and use them to create extra
>>>>> functions (cma_create() and cma_free()) that allow creating and freeing,
>>>>> respectively, CMA regions dynamically at runtime.
>>>> Well, the technical reason for not creating cma regions dynamically at
>>>> runtime is that on some architectures (like 32bit ARM) the early fixup
>>>> for the region is needed to make it functional for DMA.
>>> Can you point me at the code that does that? Thanks!
>> Check dma_contiguous_early_fixup() and dma_contiguous_remap() in
>> arch/arm/mm/dma-mapping.c. Those functions ensures that the CPU mappings for
>> the CMA reserved region in linear map are remapped with 4k pages instead
>> of the 1M sections, so later, it will be possible to alter the mappings and
>> change them to coherent when needed (altering 1M sections is not possible,
>> because each process has it's own level-1 array even for the kernel linear
>> mapping).
>>
>>
>>
>> However, in the use case in this patchset the reserved region is only shared
>> with buddy allocator by using the CMA infrastructure, not registered to the
>> regular DMA-mapping API, so it would work fine. I'm not convinced that this
>> is the right API to use for this though.
> Are you saying you're not convinced that CMA is the right API to use for
> this? Or something else?
I read this again and indeed CMA seems to be right solution. I only wonder
why do You want to create the CMA areas dynamically? Imho it would work if
You just create large enough CMA area on boot, what would automatically
share the memory with buddy allocator and then allocate dynamic VPR regions
with cma_alloc(), potentially unmapping or marking the allocated region as
reserved in linear kernel mapping to avoid any potential speculative access
to the protected memory.
In both cases You will probably won't need the DMA-mapping API on top of
it, although it might be even possible to partially use with by
registering custom dma_ops for the devices using the protected region
(assuming that it would support only DMA_ATTR_NO_KERNEL_MAPPING
allocations).
> I certainly don't think we want to get the DMA-mapping API involved for
> this because that always implies that we perform cache operations, which
> we specifically don't want for this memory.
>
> Thierry
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
^ permalink raw reply
* Re: [PATCH v2 00/33] mm: make VMA page offset handling more consistent
From: Christian König @ 2026-07-16 10:03 UTC (permalink / raw)
To: Lorenzo Stoakes
Cc: linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King,
Ackerley Tng, Kai Huang
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-0-2a5aa403d977@kernel.org>
On 7/10/26 22:16, Lorenzo Stoakes wrote:
> This series performs a series of cleanups and improvements around how the
> vma->vm_pgoff field is used.
>
> Folios belonging to file-backed mappings are simply indexed by the page
> offset within the file they map (excepting pfnmap and driver mappings which
> sometimes do things different).
>
> However, anonymous folios belonging to pure anonymous mappings are indexed
> by their "virtual" page offset, which is equal to addr >> PAGE_SHIFT at the
> time at which the VMA was first faulted in.
>
> The page offset of a VMA is stored in vma->vm_pgoff and indicates the page
> offset of the start of the VMA range, whether it be file-backed or
> anonymous.
>
> The work here both cleans up how we reference this field, as well as laying
> the foundations for a future series which addresses the inconsistency of
> CoW'd folios in MAP_PRIVATE-file backed mappings, which are indexed as if
> they were file-backed but behave as if they were anonymous.
>
> This future series will make it such that all anonymous folios are indexed
> by virtual page offset whether belonging to VMAs who satisfy
> vma_is_anonymous() or MAP_PRIVATE-mapped file-backed mappings.
>
> This series:
>
> * Exposes vma_start_pgoff() and updates the kernel to use it consistently.
> * Adds and uses the useful vma_end_pgoff() helper.
> * Parameterises the file-backed mapping helpers vma_interval_tree_*()
> by address_space rather than rb_root_cached.
> * Renames: the misleadingly-named vma_interval_tree_*() helpers to
> mapping_rmap_tree_*() to be consistent with the renamed
> anon_rmap_tree_*().
> * Parameterises anon_rmap_tree_*() by anon_vma.
> * Moves mm/interval_tree.c to the rmap section.
> * Adds vmg_*() helpers for page offset.
> * Clarifies the confusing vmg_adjust_set_range() function.
> * Introduces linear_page_delta() to provide relative pgoff within a VMA.
> * Replaces open-coded versions of linear_page_delta() and
> linear_page_index() with invocations of these functions.
> * Introduces and uses vma_assert_can_modify() to account for whether a VMA
> can be modified (detached or write locked).
> * Adds and uses vma_[add,sub]_pgoff() to adjust VMA page offset.
> * Moves __install_special_mapping() to vma.c.
> * Makes vma_set_range() static and internal to vma.c.
> * Introduces and makes use of vma_set_pgoff().
> * Fixes incorrect vma.h header inclusion.
> * Defaults VMA userland tests to 64-bit vma flags size.
> * Updates VMA userland tests to give better output on failure.
> * Various smaller cleanups.
> * Updated the vma_set_pgoff() assert to account for MAP_PRIVATE /dev/zero
> correctly.
>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
I only skimmed over the set but should there be any changes required to DMA-buf, amdgpu or radeon driver feel free to add
Acked-by: Christian König <christian.koenig@amd.com>
and merge upstream through any branch you feel fit.
Regards,
Christian.
> ---
> v2:
> * Rebased on mm-unstable.
> * Updated tags (thanks to everyone and b4 trailers -F -u! ;)
> * Updated 1/30 to include the vma_start_pgoff() helper in VMA userland tests,
> which was previously causing a bisection hazard on the VMA userland tests.
> * Updated 2/30 to reference the pfnmap vma->vm_pgoff PFN abomination as per
> David.
> * Updated 4/30 to include vma_end_pgoff() helper in VMA userland tests at the
> appropriate point.
> * Updated 8/30 from mapping_interval_tree_*() to mapping_rmap_tree_*() and add
> commit to rename anon_vma_interval-tree_*() to anon_rmap_tree_*() as per
> Pedro.
> * Fixed issue in 8/30 where the internal helpers were still called
> __vma_interval_tree_*() rather than __mapping_rmap_tree_*().
> * Updated 9/30 to separate out the rename and change of parameter type for
> clarity as per Gregory.
> * Updated 17/30 to put perf_mmap_aux() variables in reverse Christmas tree order
> as per Pedro.
> * Updated 19/30 to remove incorrectly placed vma_start_pgoff() invocation as per
> Thomas/Gregory.
> * Updated 20/30, 21/30 to put the vma_assert_can_modify() VMA userland test stub
> in the right place as per Pedro.
> * Confined the vma_set_pgoff() assert to CONFIG_MMU, as the ever helpful
> nommu of course violates all assumptions in this regard.
> * Fixed issue in improved ASSERT_[TRUE, FALSE, EQ, NE] test runners with
> re-evaluation of input expressions.
> * Updated the vma_start_pgoff() comment further to discuss the MAP_PRIVATE
> /dev/zero edge case.
> * Update the vma_set_pgoff() assert to handle the mmap_region() newly
> established file mapping case.
> * Updated the linear_page_index() comment to explicitly defer to
> vma_start_pgoff() to avoid duplicate information and future bitrot.
> * Updated vma_shrink() to no longer accept the useless start parameter, as
> it is invoked from relocate_vma_down() only, which has already expanded
> the VMA to cover the range [new_start, old_end) thus always shrinks the
> tail.
> * Moved sanity checks from vma_set_pgoff() to new helper
> assert_sane_pgoff() and documented it clearly.
> * Fixed issue with nascent file-backed VMAs hitting vma_set_pgoff() assert
> by checking for vma_is_anonymous().
> * Moved vma_set_anonymous() earlier so vma_set_pgoff() works with nascent
> anonymous VMAs.
> * Updated the VMA userland tests so CONFIG_MMU, CONFIG_PER_VMA_LOCK work
> with IS_ENABLED().
>
> v1:
> https://patch.msgid.link/cover.1782735110.git.ljs@kernel.org
>
> To: Andrew Morton <akpm@linux-foundation.org>
> To: David Hildenbrand <david@kernel.org>
> To: "Liam R. Howlett" <liam@infradead.org>
> To: Vlastimil Babka <vbabka@kernel.org>
> To: Mike Rapoport <rppt@kernel.org>
> To: Suren Baghdasaryan <surenb@google.com>
> To: Michal Hocko <mhocko@suse.com>
> To: Rik van Riel <riel@surriel.com>
> To: Harry Yoo <harry@kernel.org>
> To: Jann Horn <jannh@google.com>
> To: Lance Yang <lance.yang@linux.dev>
> To: Pedro Falcato <pfalcato@suse.de>
> To: Russell King <linux@armlinux.org.uk>
> To: Dinh Nguyen <dinguyen@kernel.org>
> To: Simon Schuster <schuster.simon@siemens-energy.com>
> To: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
> To: Helge Deller <deller@gmx.de>
> To: Alexander Viro <viro@zeniv.linux.org.uk>
> To: Christian Brauner <brauner@kernel.org>
> To: Jan Kara <jack@suse.cz>
> To: Dan Williams <djbw@kernel.org>
> To: Matthew Wilcox <willy@infradead.org>
> To: Muchun Song <muchun.song@linux.dev>
> To: Oscar Salvador <osalvador@suse.de>
> To: Masami Hiramatsu <mhiramat@kernel.org>
> To: Oleg Nesterov <oleg@redhat.com>
> To: Peter Zijlstra <peterz@infradead.org>
> To: Ingo Molnar <mingo@redhat.com>
> To: Arnaldo Carvalho de Melo <acme@kernel.org>
> To: Namhyung Kim <namhyung@kernel.org>
> To: Mark Rutland <mark.rutland@arm.com>
> To: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> To: Jiri Olsa <jolsa@kernel.org>
> To: Ian Rogers <irogers@google.com>
> To: Adrian Hunter <adrian.hunter@intel.com>
> To: James Clark <james.clark@linaro.org>
> To: Zi Yan <ziy@nvidia.com>
> To: Baolin Wang <baolin.wang@linux.alibaba.com>
> To: Nico Pache <npache@redhat.com>
> To: Ryan Roberts <ryan.roberts@arm.com>
> To: Dev Jain <dev.jain@arm.com>
> To: Barry Song <baohua@kernel.org>
> To: Miaohe Lin <linmiaohe@huawei.com>
> To: Naoya Horiguchi <nao.horiguchi@gmail.com>
> To: Xu Xin <xu.xin16@zte.com.cn>
> To: Chengming Zhou <chengming.zhou@linux.dev>
> To: SJ Park <sj@kernel.org>
> To: Matthew Brost <matthew.brost@intel.com>
> To: Joshua Hahn <joshua.hahnjy@gmail.com>
> To: Rakie Kim <rakie.kim@sk.com>
> To: Byungchul Park <byungchul@sk.com>
> To: Gregory Price <gourry@gourry.net>
> To: Ying Huang <ying.huang@linux.alibaba.com>
> To: Alistair Popple <apopple@nvidia.com>
> To: Hugh Dickins <hughd@google.com>
> To: Peter Xu <peterx@redhat.com>
> To: Kees Cook <kees@kernel.org>
> To: Marek Szyprowski <m.szyprowski@samsung.com>
> To: Robin Murphy <robin.murphy@arm.com>
> To: Andrey Konovalov <andreyknvl@gmail.com>
> To: Alexander Potapenko <glider@google.com>
> To: Dmitry Vyukov <dvyukov@google.com>
> To: Steven Rostedt <rostedt@goodmis.org>
> To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> To: Jarkko Sakkinen <jarkko@kernel.org>
> To: Dave Hansen <dave.hansen@linux.intel.com>
> To: Thomas Gleixner <tglx@kernel.org>
> To: Borislav Petkov <bp@alien8.de>
> To: x86@kernel.org
> To: "H. Peter Anvin" <hpa@zytor.com>
> To: Ian Abbott <abbotti@mev.co.uk>
> To: H Hartley Sweeten <hsweeten@visionengravers.com>
> To: Lucas Stach <l.stach@pengutronix.de>
> To: Christian Gmeiner <christian.gmeiner@gmail.com>
> To: David Airlie <airlied@gmail.com>
> To: Simona Vetter <simona@ffwll.ch>
> To: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
> To: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> To: Maxime Ripard <mripard@kernel.org>
> To: Thomas Zimmermann <tzimmermann@suse.de>
> To: Rob Clark <robin.clark@oss.qualcomm.com>
> To: Dmitry Baryshkov <lumag@kernel.org>
> To: Abhinav Kumar <abhinav.kumar@linux.dev>
> To: Jessica Zhang <jesszhan0024@gmail.com>
> To: Sean Paul <sean@poorly.run>
> To: Marijn Suijten <marijn.suijten@somainline.org>
> To: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> To: Thierry Reding <thierry.reding@kernel.org>
> To: Mikko Perttunen <mperttunen@nvidia.com>
> To: Jonathan Hunter <jonathanh@nvidia.com>
> To: Christian Koenig <christian.koenig@amd.com>
> To: Huang Rui <ray.huang@amd.com>
> To: Matthew Auld <matthew.auld@intel.com>
> To: Jason Gunthorpe <jgg@ziepe.ca>
> To: Yishai Hadas <yishaih@nvidia.com>
> To: Shameer Kolothum <skolothumtho@nvidia.com>
> To: Kevin Tian <kevin.tian@intel.com>
> To: Ankit Agrawal <ankita@nvidia.com>
> To: Alex Williamson <alex@shazbot.org>
> To: Paolo Bonzini <pbonzini@redhat.com>
> To: Shakeel Butt <shakeel.butt@linux.dev>
> To: Usama Arif <usama.arif@linux.dev>
> Cc: ljs@kernel.org
> Cc: linux-mm@kvack.org
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-parisc@vger.kernel.org
> Cc: linux-fsdevel@vger.kernel.org
> Cc: nvdimm@lists.linux.dev
> Cc: linux-perf-users@vger.kernel.org
> Cc: linux-trace-kernel@vger.kernel.org
> Cc: damon@lists.linux.dev
> Cc: iommu@lists.linux.dev
> Cc: kasan-dev@googlegroups.com
> Cc: linux-sgx@vger.kernel.org
> Cc: etnaviv@lists.freedesktop.org
> Cc: dri-devel@lists.freedesktop.org
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-tegra@vger.kernel.org
> Cc: kvm@vger.kernel.org
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
>
> ---
> Lorenzo Stoakes (33):
> mm: move vma_start_pgoff() into mm.h and clean up
> mm: add kdoc comments for vma_start/last_pgoff()
> tools/testing/vma: use vma_start_pgoff() in merge tests
> mm: introduce and use vma_end_pgoff()
> mm/rmap: update mm/interval_tree.c comments
> mm/rmap: parameterise vma_interval_tree_*() by address_space
> mm/rmap: elide unnecessary static inline's in interval_tree.c
> mm/rmap: rename vma_interval_tree_*() to mapping_rmap_tree_*()
> mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
> mm/rmap: rename anon_vma_interval_tree_*() params and use pgoff_t
> mm/rmap: rename anon_vma_interval_tree_*() to anon_rmap_tree_*()
> MAINTAINERS: Move mm/interval_tree.c to rmap section
> mm/vma: introduce and use vmg_pages(), vmg_[start, end]_pgoff()
> mm/vma: clean up anon_vma_compatible()
> mm/vma: refactor vmg_adjust_set_range() for clarity
> mm/vma: minor cleanup of expand_[upwards, downwards]()
> mm: introduce and use linear_page_delta()
> mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
> mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
> mm/vma: remove duplicative vma_pgoff_offset() helper
> mm: use linear_page_[index, delta]() consistently
> mm/vma: introduce vma_assert_can_modify()
> mm/vma: add and use vma_[add/sub]_pgoff()
> mm/vma: move __install_special_mapping() to vma.c
> mm/vma: make vma_set_range() static, drop insert_vm_struct() decl
> mm/vma: update vma_shrink() to not pass start, pgoff parameters
> mm/vma: update vmg_adjust_set_range() to offset pgoff instead
> mm/vma: slightly rework the anonymous check in __mmap_new_vma()
> mm/vma: introduce and use vma_set_pgoff()
> mm/vma: correct incorrect vma.h inclusion
> mm/vma: use guard clauses in can_vma_merge_[before, after]()
> tools/testing/vma: default VMA, mm flag bits to 64-bit
> tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
>
> MAINTAINERS | 2 +-
> arch/arm/mm/fault-armv.c | 4 +-
> arch/arm/mm/flush.c | 2 +-
> arch/nios2/mm/cacheflush.c | 2 +-
> arch/parisc/kernel/cache.c | 2 +-
> arch/x86/kernel/cpu/sgx/virt.c | 3 +-
> drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 +-
> drivers/gpu/drm/gma500/gem.c | 2 +-
> drivers/gpu/drm/msm/msm_gem.c | 3 +-
> drivers/gpu/drm/omapdrm/omap_gem.c | 5 +-
> drivers/gpu/drm/tegra/gem.c | 3 +-
> drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +-
> drivers/vfio/pci/nvgrace-gpu/main.c | 3 +-
> drivers/vfio/pci/vfio_pci_core.c | 3 +-
> fs/dax.c | 2 +-
> fs/hugetlbfs/inode.c | 15 +-
> include/linux/huge_mm.h | 1 +
> include/linux/hugetlb.h | 3 +-
> include/linux/mm.h | 118 ++++++++++----
> include/linux/mmap_lock.h | 8 +
> include/linux/pagemap.h | 35 +++-
> kernel/dma/coherent.c | 7 +-
> kernel/dma/direct.c | 6 +-
> kernel/dma/mapping.c | 8 +-
> kernel/dma/ops_helpers.c | 4 +-
> kernel/events/core.c | 20 +--
> kernel/events/uprobes.c | 13 +-
> kernel/kcov.c | 2 +-
> kernel/trace/ring_buffer.c | 3 +-
> mm/damon/vaddr.c | 5 +-
> mm/debug.c | 2 +-
> mm/filemap.c | 7 +-
> mm/huge_memory.c | 2 +-
> mm/hugetlb.c | 15 +-
> mm/internal.h | 33 ++--
> mm/interval_tree.c | 111 ++++++++-----
> mm/khugepaged.c | 7 +-
> mm/ksm.c | 7 +-
> mm/madvise.c | 6 +-
> mm/mapping_dirty_helpers.c | 2 +-
> mm/memory-failure.c | 10 +-
> mm/memory.c | 33 ++--
> mm/mempolicy.c | 13 +-
> mm/mmap.c | 41 +----
> mm/mmu_notifier.c | 2 +-
> mm/mremap.c | 12 +-
> mm/msync.c | 4 +-
> mm/nommu.c | 22 +--
> mm/pagewalk.c | 4 +-
> mm/rmap.c | 14 +-
> mm/shmem.c | 9 +-
> mm/userfaultfd.c | 4 +-
> mm/util.c | 4 +-
> mm/vma.c | 293 ++++++++++++++++++++++------------
> mm/vma.h | 80 ++++++++--
> mm/vma_exec.c | 12 +-
> mm/vma_init.c | 6 +-
> mm/vma_internal.h | 4 +-
> tools/testing/vma/Makefile | 2 +-
> tools/testing/vma/include/dup.h | 41 ++++-
> tools/testing/vma/include/stubs.h | 16 +-
> tools/testing/vma/shared.c | 9 --
> tools/testing/vma/shared.h | 43 +++--
> tools/testing/vma/tests/merge.c | 40 ++---
> tools/testing/vma/vma_internal.h | 4 +-
> virt/kvm/guest_memfd.c | 2 +-
> 66 files changed, 751 insertions(+), 459 deletions(-)
> ---
> base-commit: 4a6c88e10a0791d9c3b0677ce984365815c2ab4f
> change-id: 20260710-b4-pre-scalable-cow-24309015dee4
>
> Cheers,
^ permalink raw reply
* Re: [PATCH v2 18/33] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:46 UTC (permalink / raw)
To: Vlastimil Babka (SUSE)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <9469d484-f736-4a2d-b1e9-a7ba1c381d18@kernel.org>
On Thu, Jul 16, 2026 at 11:43:35AM +0200, Vlastimil Babka (SUSE) wrote:
> On 7/16/26 11:27, Lorenzo Stoakes (ARM) wrote:
> > On Wed, Jul 15, 2026 at 05:23:09PM +0200, Vlastimil Babka (SUSE) wrote:
> >> On 7/10/26 22:16, Lorenzo Stoakes wrote:
> >> > There are many instances in which linear_page_index() (as well as
> >> > linear_page_delta()) is open-coded, which is confusing and inconsistent.
> >> >
> >> > Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is
> >> > the page offset of the start of the VMA range.
> >> >
> >> > Doing so also aids greppability.
> >> >
> >> > So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and
> >> > linear_page_index() where we can.
> >> >
> >> > This also lays the ground for future changes which will add an anonymous
> >> > page offset in order to be able to index MAP_PRIVATE-file backed anon
> >> > folios in terms of their virtual page offset.
> >> >
> >> > No functional change intended.
> >> >
> >> > Reviewed-by: Gregory Price <gourry@gourry.net>
> >> > Reviewed-by: SJ Park <sj@kernel.org>
> >> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> >> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
> >>
> >> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> >
> > Thanks!
> >
> >>
> >> Nit:
> >>
> >> > diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
> >> > index 2058db9c01d5..d10b8042adb5 100644
> >> > --- a/mm/damon/vaddr.c
> >> > +++ b/mm/damon/vaddr.c
> >> > @@ -10,6 +10,7 @@
> >> > #include <linux/mman.h>
> >> > #include <linux/mmu_notifier.h>
> >> > #include <linux/page_idle.h>
> >> > +#include <linux/pagemap.h>
> >> > #include <linux/pagewalk.h>
> >> > #include <linux/sched/mm.h>
> >> >
> >> > @@ -623,8 +624,8 @@ static void damos_va_migrate_dests_add(struct folio *folio,
> >> > }
> >> >
> >> > order = folio_order(folio);
> >> > - ilx = vma->vm_pgoff >> order;
> >> > - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
> >> > + ilx = vma_start_pgoff(vma) >> order;
> >> > + ilx += linear_page_delta(vma, addr) >> order;
> >>
> >> Could these be combined to linear_page_index(vma, addr) >> order?
> >
> > So this is:
> >
> > (x >> order) + (y >> order)
> >
> > And linear_page_index(vma, addr) >> order is:
> >
> > (x + y) >> order
> >
> > Consider:
> >
> > order = 1, x = 0b11, y = 0b01
> >
> > (x >> order) + (y >> order) =
> > (0b11 >> 1 ) + (0b01 >> 1 ) =
> > 0b1 + 0 =
> > 1
> >
> > (x + y ) >> order =
> > (0b11 + 0b01) >> 1 =
> > (0b100 ) >> 1 =
> > 2
> >
> > They are not equivalent - the carry bit changes the result.
>
> Right, thanks. Didn't consider it would be actually working with unaligned
> addresses but seems like it can.
Yeah, I initially did it the way you suggested then ran it by AI review and it
moaned at me so ;)
>
> >>
> >> >
> >> > for (i = 0; i < dests->nr_dests; i++)
> >> > weight_total += dests->weight_arr[i];
> >>
> >> > diff --git a/mm/filemap.c b/mm/filemap.c
> >> > index b39111abdc4b..1dbb4c6f824e 100644
> >> > --- a/mm/filemap.c
> >> > +++ b/mm/filemap.c
> >> > @@ -3411,8 +3411,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
> >> > * of memory.
> >> > */
> >> > struct vm_area_struct *vma = vmf->vma;
> >> > - unsigned long start = vma->vm_pgoff;
> >> > - unsigned long end = start + vma_pages(vma);
> >> > + const unsigned long start = vma_start_pgoff(vma);
> >> > + const unsigned long end = vma_end_pgoff(vma);
> >> > unsigned long ra_end;
> >> >
> >> > ra->order = exec_folio_order();
> >> > @@ -3930,7 +3930,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
> >> > goto out;
> >> > }
> >> >
> >> > - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) << PAGE_SHIFT);
> >> > + addr = vma->vm_start +
> >> > + ((start_pgoff - vma_start_pgoff(vma)) << PAGE_SHIFT);
> >>
> >> This could be linear_page_index(vma, start_pgoff) <<
>
> Thanks for pretending you didn't notice this one, which was even more
> obviously incorrect :)
Again I screwed this up when I first did it so was far more terribly wrong
myself :P All hail AI review I suppose?
>
> >> > vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, addr, &vmf->ptl);
> >> > if (!vmf->pte) {
> >> > folio_unlock(folio);
> >>
> >> > --- a/mm/mempolicy.c
> >> > +++ b/mm/mempolicy.c
> >> > @@ -2049,8 +2049,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
> >> > pol = get_task_policy(current);
> >> > if (pol->mode == MPOL_INTERLEAVE ||
> >> > pol->mode == MPOL_WEIGHTED_INTERLEAVE) {
> >> > - *ilx += vma->vm_pgoff >> order;
> >> > - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
> >> > + *ilx += vma_start_pgoff(vma) >> order;
> >> > + *ilx += linear_page_delta(vma, addr) >> order;
> >>
> >> Also?
> >
> > See above.
> >
> >>
> >> > }
> >> > return pol;
> >> > }
> >>
> >> > --- a/mm/nommu.c
> >> > +++ b/mm/nommu.c
> >> > @@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,
> >> > /* read the contents of a file into the copy */
> >> > loff_t fpos;
> >> >
> >> > - fpos = vma->vm_pgoff;
> >> > + fpos = vma_start_pgoff(vma);
> >> > fpos <<= PAGE_SHIFT;
> >> >
> >> > ret = kernel_read(vma->vm_file, base, len, &fpos);
> >> > @@ -1378,7 +1378,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> >> > delete_nommu_region(vma->vm_region);
> >> > if (new_below) {
> >> > vma->vm_region->vm_start = vma->vm_start = addr;
> >> > - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages;
> >>
> >> yuck, good to remove that ugly construct
> >> it's not the only occurence in this function
> >> oh well
> >
> > Yup at least improved it here :)
>
> And a later patch removes the other occurence.
A game of 2 halves!
>
> >>
> >> > + vma->vm_pgoff += npages;
> >> > + vma->vm_region->vm_pgoff = vma_start_pgoff(vma);
> >> > } else {
> >> > vma->vm_region->vm_end = vma->vm_end = addr;
> >> > vma->vm_region->vm_top = addr;
> >
> > Cheers, Lorenzo
>
^ permalink raw reply
* Re: [PATCH v2 01/33] mm: move vma_start_pgoff() into mm.h and clean up
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:44 UTC (permalink / raw)
To: Liam R. Howlett
Cc: Andrew Morton, David Hildenbrand, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <uqfnbb7knn5gs2symluouczdbxehpwuubek3ypziavv776jncb@l7r4zlatslxd>
On Wed, Jul 15, 2026 at 02:05:42PM -0400, Liam R. Howlett wrote:
> On 26/07/10 09:16PM, Lorenzo Stoakes wrote:
> > vma_last_pgoff() already lives there, so it's a bit odd to keep
> > vma_start_pgoff() in mm/interval_tree.c. Move them together.
> >
> > These each return unsigned long, which pgoff_t is typedef'd to. Make this
> > consistent and have these functions return pgoff_t instead.
> >
> > Additionally, express vma_last_pgoff() in terms of vma_start_pgoff(), since
> > we wrap the vma->vm_pgoff access, we may as well use it here.
> >
> > Also while we're here, const-ify the VMA and cleanup a bit.
> >
> > Also update the VMA userland tests to reflect the change.
>
> This patch could have been 01/35? Disappointing, but I will try to get
> over it.
;) well I have a series I've not sent yet that maybe could grow a bit to achieve
these aims? :>)
>
> >
> > No functional change intended.
> >
> > Reviewed-by: Gregory Price <gourry@gourry.net>
> > Acked-by: David Hildenbrand (Arm) <david@kernel.org>
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>
> Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Thanks!
>
> > ---
> > include/linux/mm.h | 9 +++++++--
> > mm/interval_tree.c | 5 -----
> > tools/testing/vma/include/dup.h | 5 +++++
> > 3 files changed, 12 insertions(+), 7 deletions(-)
> >
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index 25e669632d2c..09b06d8fea74 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -4307,9 +4307,14 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)
> > return (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
> > }
> >
> > -static inline unsigned long vma_last_pgoff(struct vm_area_struct *vma)
> > +static inline pgoff_t vma_start_pgoff(const struct vm_area_struct *vma)
> > {
> > - return vma->vm_pgoff + vma_pages(vma) - 1;
> > + return vma->vm_pgoff;
> > +}
> > +
> > +static inline pgoff_t vma_last_pgoff(const struct vm_area_struct *vma)
> > +{
> > + return vma_start_pgoff(vma) + vma_pages(vma) - 1;
> > }
> >
> > static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)
> > diff --git a/mm/interval_tree.c b/mm/interval_tree.c
> > index 32bcfbfcf15f..344d1f5946c7 100644
> > --- a/mm/interval_tree.c
> > +++ b/mm/interval_tree.c
> > @@ -10,11 +10,6 @@
> > #include <linux/rmap.h>
> > #include <linux/interval_tree_generic.h>
> >
> > -static inline unsigned long vma_start_pgoff(struct vm_area_struct *v)
> > -{
> > - return v->vm_pgoff;
> > -}
> > -
> > INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb,
> > unsigned long, shared.rb_subtree_last,
> > vma_start_pgoff, vma_last_pgoff, /* empty */, vma_interval_tree)
> > diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
> > index bf26b3f48d3a..668650067c7c 100644
> > --- a/tools/testing/vma/include/dup.h
> > +++ b/tools/testing/vma/include/dup.h
> > @@ -1301,6 +1301,11 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)
> > return (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
> > }
> >
> > +static inline pgoff_t vma_start_pgoff(const struct vm_area_struct *vma)
> > +{
> > + return vma->vm_pgoff;
> > +}
> > +
> > static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)
> > {
> > return file->f_op->mmap_prepare(desc);
> >
> > --
> > 2.55.0
> >
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 18/33] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
From: Vlastimil Babka (SUSE) @ 2026-07-16 9:43 UTC (permalink / raw)
To: Lorenzo Stoakes (ARM)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <aligtF3WwRam1rMd@lucifer>
On 7/16/26 11:27, Lorenzo Stoakes (ARM) wrote:
> On Wed, Jul 15, 2026 at 05:23:09PM +0200, Vlastimil Babka (SUSE) wrote:
>> On 7/10/26 22:16, Lorenzo Stoakes wrote:
>> > There are many instances in which linear_page_index() (as well as
>> > linear_page_delta()) is open-coded, which is confusing and inconsistent.
>> >
>> > Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is
>> > the page offset of the start of the VMA range.
>> >
>> > Doing so also aids greppability.
>> >
>> > So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and
>> > linear_page_index() where we can.
>> >
>> > This also lays the ground for future changes which will add an anonymous
>> > page offset in order to be able to index MAP_PRIVATE-file backed anon
>> > folios in terms of their virtual page offset.
>> >
>> > No functional change intended.
>> >
>> > Reviewed-by: Gregory Price <gourry@gourry.net>
>> > Reviewed-by: SJ Park <sj@kernel.org>
>> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
>> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>>
>> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
>
> Thanks!
>
>>
>> Nit:
>>
>> > diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
>> > index 2058db9c01d5..d10b8042adb5 100644
>> > --- a/mm/damon/vaddr.c
>> > +++ b/mm/damon/vaddr.c
>> > @@ -10,6 +10,7 @@
>> > #include <linux/mman.h>
>> > #include <linux/mmu_notifier.h>
>> > #include <linux/page_idle.h>
>> > +#include <linux/pagemap.h>
>> > #include <linux/pagewalk.h>
>> > #include <linux/sched/mm.h>
>> >
>> > @@ -623,8 +624,8 @@ static void damos_va_migrate_dests_add(struct folio *folio,
>> > }
>> >
>> > order = folio_order(folio);
>> > - ilx = vma->vm_pgoff >> order;
>> > - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
>> > + ilx = vma_start_pgoff(vma) >> order;
>> > + ilx += linear_page_delta(vma, addr) >> order;
>>
>> Could these be combined to linear_page_index(vma, addr) >> order?
>
> So this is:
>
> (x >> order) + (y >> order)
>
> And linear_page_index(vma, addr) >> order is:
>
> (x + y) >> order
>
> Consider:
>
> order = 1, x = 0b11, y = 0b01
>
> (x >> order) + (y >> order) =
> (0b11 >> 1 ) + (0b01 >> 1 ) =
> 0b1 + 0 =
> 1
>
> (x + y ) >> order =
> (0b11 + 0b01) >> 1 =
> (0b100 ) >> 1 =
> 2
>
> They are not equivalent - the carry bit changes the result.
Right, thanks. Didn't consider it would be actually working with unaligned
addresses but seems like it can.
>>
>> >
>> > for (i = 0; i < dests->nr_dests; i++)
>> > weight_total += dests->weight_arr[i];
>>
>> > diff --git a/mm/filemap.c b/mm/filemap.c
>> > index b39111abdc4b..1dbb4c6f824e 100644
>> > --- a/mm/filemap.c
>> > +++ b/mm/filemap.c
>> > @@ -3411,8 +3411,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
>> > * of memory.
>> > */
>> > struct vm_area_struct *vma = vmf->vma;
>> > - unsigned long start = vma->vm_pgoff;
>> > - unsigned long end = start + vma_pages(vma);
>> > + const unsigned long start = vma_start_pgoff(vma);
>> > + const unsigned long end = vma_end_pgoff(vma);
>> > unsigned long ra_end;
>> >
>> > ra->order = exec_folio_order();
>> > @@ -3930,7 +3930,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
>> > goto out;
>> > }
>> >
>> > - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) << PAGE_SHIFT);
>> > + addr = vma->vm_start +
>> > + ((start_pgoff - vma_start_pgoff(vma)) << PAGE_SHIFT);
>>
>> This could be linear_page_index(vma, start_pgoff) <<
Thanks for pretending you didn't notice this one, which was even more
obviously incorrect :)
>> > vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, addr, &vmf->ptl);
>> > if (!vmf->pte) {
>> > folio_unlock(folio);
>>
>> > --- a/mm/mempolicy.c
>> > +++ b/mm/mempolicy.c
>> > @@ -2049,8 +2049,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
>> > pol = get_task_policy(current);
>> > if (pol->mode == MPOL_INTERLEAVE ||
>> > pol->mode == MPOL_WEIGHTED_INTERLEAVE) {
>> > - *ilx += vma->vm_pgoff >> order;
>> > - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
>> > + *ilx += vma_start_pgoff(vma) >> order;
>> > + *ilx += linear_page_delta(vma, addr) >> order;
>>
>> Also?
>
> See above.
>
>>
>> > }
>> > return pol;
>> > }
>>
>> > --- a/mm/nommu.c
>> > +++ b/mm/nommu.c
>> > @@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,
>> > /* read the contents of a file into the copy */
>> > loff_t fpos;
>> >
>> > - fpos = vma->vm_pgoff;
>> > + fpos = vma_start_pgoff(vma);
>> > fpos <<= PAGE_SHIFT;
>> >
>> > ret = kernel_read(vma->vm_file, base, len, &fpos);
>> > @@ -1378,7 +1378,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
>> > delete_nommu_region(vma->vm_region);
>> > if (new_below) {
>> > vma->vm_region->vm_start = vma->vm_start = addr;
>> > - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages;
>>
>> yuck, good to remove that ugly construct
>> it's not the only occurence in this function
>> oh well
>
> Yup at least improved it here :)
And a later patch removes the other occurence.
>>
>> > + vma->vm_pgoff += npages;
>> > + vma->vm_region->vm_pgoff = vma_start_pgoff(vma);
>> > } else {
>> > vma->vm_region->vm_end = vma->vm_end = addr;
>> > vma->vm_region->vm_top = addr;
>
> Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 30/33] mm/vma: correct incorrect vma.h inclusion
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:42 UTC (permalink / raw)
To: Vlastimil Babka (SUSE)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <a273faf8-452d-499a-90e1-b3c9dc2395d0@kernel.org>
On Thu, Jul 16, 2026 at 10:38:00AM +0200, Vlastimil Babka (SUSE) wrote:
> On 7/10/26 22:17, Lorenzo Stoakes wrote:
> > The only files which should be including vma.h are the implementation files
> > for the core VMA logic - vma.c, vma_init.c, and vma_exec.c.
> >
> > This is in order to allow for userland testing of core VMA logic. In this
> > cases, vma_internal.h and vma.h are included, providing both the
> > dependencies upon which the core VMA logic requires and its declarations.
> >
> > Userland testable VMA logic is achieved by having separate vma_internal.h
> > implementations for userland and kernel.
> >
> > Callers other than the core VMA implementation should include internal.h
> > instead. This header does not need to include vma_internal.h as it only
> > contains the vma.h declarations, for which the includes already present
> > suffice.
> >
> > Update code to reflect this, update comments to reflect the fact there are
> > 3 VMA implementation files and document things more clearly.
> >
> > While we're here, slightly improve the language of the comment describing
> > vma_exec.c.
> >
> > No functional change intended.
> >
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Reviewed-by: Gregory Price <gourry@gourry.net>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Thanks!
>
> But given Mike's efforts [1] I wonder if there's a way to change the vma
> header system in the future to stop requiring users of vma functions to
> include the whole internal.h, while of course still allow the userland
> testability.
>
> [1]
> https://lore.kernel.org/all/20260709-internal-h-v1-0-956d5092782d@kernel.org/
Yeah we can always put a (correctly formed) vma.h include in any split off headers.
There's also Pedro's suggestion (my reply to that at [0]) which I plan to do as
a follow up :)
[0]:https://lore.kernel.org/all/akzWq08xwXKqwwtq@lucifer/
>
>
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 25/33] mm/vma: make vma_set_range() static, drop insert_vm_struct() decl
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:36 UTC (permalink / raw)
To: Vlastimil Babka (SUSE)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <b470eb19-d59c-4dd6-bf57-648b51281691@kernel.org>
On Wed, Jul 15, 2026 at 07:45:59PM +0200, Vlastimil Babka (SUSE) wrote:
> On 7/10/26 22:17, Lorenzo Stoakes wrote:
> > With __install_special_mapping() moved to vma.c, vma_set_range() can be
> > made into a static function there and is now completely isolated from the
> > rest of mm.
>
> Pedantic nit: it was already "static", but in a .h file
> Yeah it was also "inline" but that's not the important part of this change.
>
> So I'd say it was rather moved to vma.c, but you can ignore this nit if you
> want.
That is pretty pedantic ;) I mean to say bare 'static' here obvs. I can fix if
any respin but I think ok as-is for now.
>
> > While we're here, we can also remove the insert_vm_struct() declaration
> > from mm.h - the function is implemented in vma.c and already declared in
> > vma.h, and has no users outside of mm.
> >
> > Also update the VMA userland tests to reflect this change.
> >
> > No functional change intended.
> >
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Reviewed-by: Gregory Price <gourry@gourry.net>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Thanks! :)
>
> > ---
> > include/linux/mm.h | 1 -
> > mm/internal.h | 9 ---------
> > mm/vma.c | 8 ++++++++
> > tools/testing/vma/shared.c | 9 ---------
> > tools/testing/vma/shared.h | 5 -----
> > 5 files changed, 8 insertions(+), 24 deletions(-)
> >
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index 3d69b597b9b1..762313b47301 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -4130,7 +4130,6 @@ void anon_rmap_tree_verify(struct anon_vma_chain *avc);
> >
> > /* mmap.c */
> > extern int __vm_enough_memory(const struct mm_struct *mm, long pages, int cap_sys_admin);
> > -extern int insert_vm_struct(struct mm_struct *, struct vm_area_struct *);
> > extern void exit_mmap(struct mm_struct *);
> > bool mmap_read_lock_maybe_expand(struct mm_struct *mm, struct vm_area_struct *vma,
> > unsigned long addr, bool write);
> > diff --git a/mm/internal.h b/mm/internal.h
> > index 01a762bcc2b2..2c650d280e90 100644
> > --- a/mm/internal.h
> > +++ b/mm/internal.h
> > @@ -1540,15 +1540,6 @@ extern bool mirrored_kernelcore;
> > bool memblock_has_mirror(void);
> > void memblock_free_all(void);
> >
> > -static __always_inline void vma_set_range(struct vm_area_struct *vma,
> > - unsigned long start, unsigned long end,
> > - pgoff_t pgoff)
> > -{
> > - vma->vm_start = start;
> > - vma->vm_end = end;
> > - vma->vm_pgoff = pgoff;
> > -}
> > -
> > static inline bool vma_soft_dirty_enabled(struct vm_area_struct *vma)
> > {
> > /*
> > diff --git a/mm/vma.c b/mm/vma.c
> > index 5308aa5a8c91..cc0e449ad0be 100644
> > --- a/mm/vma.c
> > +++ b/mm/vma.c
> > @@ -70,6 +70,14 @@ struct mmap_state {
> > .state = VMA_MERGE_START, \
> > }
> >
> > +static void vma_set_range(struct vm_area_struct *vma, unsigned long start,
> > + unsigned long end, pgoff_t pgoff)
> > +{
> > + vma->vm_start = start;
> > + vma->vm_end = end;
> > + vma->vm_pgoff = pgoff;
> > +}
> > +
> > /* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */
> > static bool vma_is_fork_child(struct vm_area_struct *vma)
> > {
> > diff --git a/tools/testing/vma/shared.c b/tools/testing/vma/shared.c
> > index 2565a5aecb80..bea9ea6db02a 100644
> > --- a/tools/testing/vma/shared.c
> > +++ b/tools/testing/vma/shared.c
> > @@ -120,12 +120,3 @@ unsigned long rlimit(unsigned int limit)
> > {
> > return (unsigned long)-1;
> > }
> > -
> > -void vma_set_range(struct vm_area_struct *vma,
> > - unsigned long start, unsigned long end,
> > - pgoff_t pgoff)
> > -{
> > - vma->vm_start = start;
> > - vma->vm_end = end;
> > - vma->vm_pgoff = pgoff;
> > -}
> > diff --git a/tools/testing/vma/shared.h b/tools/testing/vma/shared.h
> > index 8b9e3b11c3cb..ca4f1238f1c7 100644
> > --- a/tools/testing/vma/shared.h
> > +++ b/tools/testing/vma/shared.h
> > @@ -125,8 +125,3 @@ void __vma_set_dummy_anon_vma(struct vm_area_struct *vma,
> > /* Provide a simple dummy VMA/anon_vma dummy setup for testing. */
> > void vma_set_dummy_anon_vma(struct vm_area_struct *vma,
> > struct anon_vma_chain *avc);
> > -
> > -/* Helper function to specify a VMA's range. */
> > -void vma_set_range(struct vm_area_struct *vma,
> > - unsigned long start, unsigned long end,
> > - pgoff_t pgoff);
> >
>
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 23/33] mm/vma: add and use vma_[add/sub]_pgoff()
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:33 UTC (permalink / raw)
To: Vlastimil Babka (SUSE)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <fb8d7fbb-1b4f-4fec-a087-8b1b9f55a167@kernel.org>
On Wed, Jul 15, 2026 at 07:38:38PM +0200, Vlastimil Babka (SUSE) wrote:
> On 7/10/26 22:17, Lorenzo Stoakes wrote:
> > Add helpers for adding or subtracting to a VMA's page offset, exposed
> > internally for VMA users within mm in mm/vma.h.
> >
> > This is to lay the foundations for tracking anonymous page offset for
> > MAP_PRIVATE file-backed mappings, where adding and subtracting from this
> > value must be reflected in both the file and anonymous offsets.
> >
> > These are used on VMA split and downward stack expansion.
> >
> > No functional change intended.
> >
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Thanks!
>
> Nits:
>
> > ---
> > mm/nommu.c | 6 ++++--
> > mm/vma.c | 6 +++---
> > mm/vma.h | 12 ++++++++++++
> > 3 files changed, 19 insertions(+), 5 deletions(-)
> >
> > diff --git a/mm/nommu.c b/mm/nommu.c
> > index c0a0869cd0d6..2a0136f6081d 100644
> > --- a/mm/nommu.c
> > +++ b/mm/nommu.c
> > @@ -41,6 +41,7 @@
> > #include <asm/tlbflush.h>
> > #include <asm/mmu_context.h>
> > #include "internal.h"
> > +#include "vma.h"
> >
> > unsigned long highest_memmap_pfn;
> > int heap_stack_gap = 0;
> > @@ -1361,7 +1362,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> > region->vm_top = region->vm_end = new->vm_end = addr;
> > } else {
> > region->vm_start = new->vm_start = addr;
> > - region->vm_pgoff = new->vm_pgoff += npages;
>
> Ah that takes care of the other case of ugliness I noticed earlier, good.
Churn for good ;)
>
> > + vma_add_pgoff(new, npages);
> > + region->vm_pgoff = vma_start_pgoff(new);
> > }
> >
> > vma_iter_config(vmi, new->vm_start, new->vm_end);
> > @@ -1378,7 +1380,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> > delete_nommu_region(vma->vm_region);
> > if (new_below) {
> > vma->vm_region->vm_start = vma->vm_start = addr;
> > - vma->vm_pgoff += npages;
> > + vma_add_pgoff(vma, npages);
> > vma->vm_region->vm_pgoff = vma_start_pgoff(vma);
> > } else {
> > vma->vm_region->vm_end = vma->vm_end = addr;
> > diff --git a/mm/vma.c b/mm/vma.c
> > index 7aa0149f076c..bdd99ba56b4d 100644
> > --- a/mm/vma.c
> > +++ b/mm/vma.c
> > @@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> > new->vm_end = addr;
> > } else {
> > new->vm_start = addr;
> > - new->vm_pgoff += linear_page_delta(vma, addr);
> > + vma_add_pgoff(new, linear_page_delta(vma, addr));
> > }
> >
> > err = -ENOMEM;
> > @@ -556,7 +556,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> >
> > if (new_below) {
> > vma->vm_start = addr;
> > - vma->vm_pgoff += (addr - new->vm_start) >> PAGE_SHIFT;
> > + vma_add_pgoff(vma, (addr - new->vm_start) >> PAGE_SHIFT);
>
> Hm isn't this also a case for using linear_page_delta(addr, new)?
> (I guess in patch 21/33)
> Would be like the hunk above.
Ah, yeah :) I can always rearrange which commit does what on any respin.
For now, Andrew - could you make a little edit here?
- vma_add_pgoff(vma, (addr - new->vm_start) >> PAGE_SHIFT);
+ vma_add_pgoff(vma, linear_page_delta(new, addr));
Thanks!
>
> > } else {
> > vma->vm_end = addr;
> > }
> > @@ -3305,7 +3305,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)
> > vm_stat_account(mm, vma->vm_flags, grow);
> > anon_rmap_tree_pre_update_vma(vma);
> > vma->vm_start = address;
> > - vma->vm_pgoff -= grow;
> > + vma_sub_pgoff(vma, grow);
> > /* Overwrite old entry in mtree. */
> > vma_iter_store_overwrite(&vmi, vma);
> > anon_rmap_tree_post_update_vma(vma);
> > diff --git a/mm/vma.h b/mm/vma.h
> > index 2342516ce00e..47fe35e5307e 100644
> > --- a/mm/vma.h
> > +++ b/mm/vma.h
> > @@ -247,6 +247,18 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)
> > return vmg_start_pgoff(vmg) + vmg_pages(vmg);
> > }
> >
> > +static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta)
> > +{
> > + vma_assert_can_modify(vma);
> > + vma->vm_pgoff += delta;
> > +}
> > +
> > +static inline void vma_sub_pgoff(struct vm_area_struct *vma, pgoff_t delta)
> > +{
> > + vma_assert_can_modify(vma);
> > + vma->vm_pgoff -= delta;
> > +}
> > +
> > #define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \
> > struct vma_merge_struct name = { \
> > .mm = mm_, \
> >
>
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 18/33] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
From: Lorenzo Stoakes (ARM) @ 2026-07-16 9:27 UTC (permalink / raw)
To: Vlastimil Babka (SUSE)
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <e98110da-f26a-46ca-8360-fc84fc159bd8@kernel.org>
On Wed, Jul 15, 2026 at 05:23:09PM +0200, Vlastimil Babka (SUSE) wrote:
> On 7/10/26 22:16, Lorenzo Stoakes wrote:
> > There are many instances in which linear_page_index() (as well as
> > linear_page_delta()) is open-coded, which is confusing and inconsistent.
> >
> > Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is
> > the page offset of the start of the VMA range.
> >
> > Doing so also aids greppability.
> >
> > So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and
> > linear_page_index() where we can.
> >
> > This also lays the ground for future changes which will add an anonymous
> > page offset in order to be able to index MAP_PRIVATE-file backed anon
> > folios in terms of their virtual page offset.
> >
> > No functional change intended.
> >
> > Reviewed-by: Gregory Price <gourry@gourry.net>
> > Reviewed-by: SJ Park <sj@kernel.org>
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Thanks!
>
> Nit:
>
> > diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
> > index 2058db9c01d5..d10b8042adb5 100644
> > --- a/mm/damon/vaddr.c
> > +++ b/mm/damon/vaddr.c
> > @@ -10,6 +10,7 @@
> > #include <linux/mman.h>
> > #include <linux/mmu_notifier.h>
> > #include <linux/page_idle.h>
> > +#include <linux/pagemap.h>
> > #include <linux/pagewalk.h>
> > #include <linux/sched/mm.h>
> >
> > @@ -623,8 +624,8 @@ static void damos_va_migrate_dests_add(struct folio *folio,
> > }
> >
> > order = folio_order(folio);
> > - ilx = vma->vm_pgoff >> order;
> > - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
> > + ilx = vma_start_pgoff(vma) >> order;
> > + ilx += linear_page_delta(vma, addr) >> order;
>
> Could these be combined to linear_page_index(vma, addr) >> order?
So this is:
(x >> order) + (y >> order)
And linear_page_index(vma, addr) >> order is:
(x + y) >> order
Consider:
order = 1, x = 0b11, y = 0b01
(x >> order) + (y >> order) =
(0b11 >> 1 ) + (0b01 >> 1 ) =
0b1 + 0 =
1
(x + y ) >> order =
(0b11 + 0b01) >> 1 =
(0b100 ) >> 1 =
2
They are not equivalent - the carry bit changes the result.
>
> >
> > for (i = 0; i < dests->nr_dests; i++)
> > weight_total += dests->weight_arr[i];
>
> > diff --git a/mm/filemap.c b/mm/filemap.c
> > index b39111abdc4b..1dbb4c6f824e 100644
> > --- a/mm/filemap.c
> > +++ b/mm/filemap.c
> > @@ -3411,8 +3411,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
> > * of memory.
> > */
> > struct vm_area_struct *vma = vmf->vma;
> > - unsigned long start = vma->vm_pgoff;
> > - unsigned long end = start + vma_pages(vma);
> > + const unsigned long start = vma_start_pgoff(vma);
> > + const unsigned long end = vma_end_pgoff(vma);
> > unsigned long ra_end;
> >
> > ra->order = exec_folio_order();
> > @@ -3930,7 +3930,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
> > goto out;
> > }
> >
> > - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) << PAGE_SHIFT);
> > + addr = vma->vm_start +
> > + ((start_pgoff - vma_start_pgoff(vma)) << PAGE_SHIFT);
>
> This could be linear_page_index(vma, start_pgoff) <<
>
> > vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, addr, &vmf->ptl);
> > if (!vmf->pte) {
> > folio_unlock(folio);
>
> > --- a/mm/mempolicy.c
> > +++ b/mm/mempolicy.c
> > @@ -2049,8 +2049,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
> > pol = get_task_policy(current);
> > if (pol->mode == MPOL_INTERLEAVE ||
> > pol->mode == MPOL_WEIGHTED_INTERLEAVE) {
> > - *ilx += vma->vm_pgoff >> order;
> > - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);
> > + *ilx += vma_start_pgoff(vma) >> order;
> > + *ilx += linear_page_delta(vma, addr) >> order;
>
> Also?
See above.
>
> > }
> > return pol;
> > }
>
> > --- a/mm/nommu.c
> > +++ b/mm/nommu.c
> > @@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,
> > /* read the contents of a file into the copy */
> > loff_t fpos;
> >
> > - fpos = vma->vm_pgoff;
> > + fpos = vma_start_pgoff(vma);
> > fpos <<= PAGE_SHIFT;
> >
> > ret = kernel_read(vma->vm_file, base, len, &fpos);
> > @@ -1378,7 +1378,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
> > delete_nommu_region(vma->vm_region);
> > if (new_below) {
> > vma->vm_region->vm_start = vma->vm_start = addr;
> > - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages;
>
> yuck, good to remove that ugly construct
> it's not the only occurence in this function
> oh well
Yup at least improved it here :)
>
> > + vma->vm_pgoff += npages;
> > + vma->vm_region->vm_pgoff = vma_start_pgoff(vma);
> > } else {
> > vma->vm_region->vm_end = vma->vm_end = addr;
> > vma->vm_region->vm_top = addr;
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v2 31/33] mm/vma: use guard clauses in can_vma_merge_[before, after]()
From: Vlastimil Babka (SUSE) @ 2026-07-16 8:39 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-31-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> Rather than combining a bunch of conditionals in a single expression,
> simplify by inverting the mergeability requirements into guard clauses.
>
> that is - instead of checking what must be true for the conditions to be
> met, instead check the inverse of the requirements and return false if any
> are true, defaulting to true.
>
> No functional change intended.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 30/33] mm/vma: correct incorrect vma.h inclusion
From: Vlastimil Babka (SUSE) @ 2026-07-16 8:38 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-30-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> The only files which should be including vma.h are the implementation files
> for the core VMA logic - vma.c, vma_init.c, and vma_exec.c.
>
> This is in order to allow for userland testing of core VMA logic. In this
> cases, vma_internal.h and vma.h are included, providing both the
> dependencies upon which the core VMA logic requires and its declarations.
>
> Userland testable VMA logic is achieved by having separate vma_internal.h
> implementations for userland and kernel.
>
> Callers other than the core VMA implementation should include internal.h
> instead. This header does not need to include vma_internal.h as it only
> contains the vma.h declarations, for which the includes already present
> suffice.
>
> Update code to reflect this, update comments to reflect the fact there are
> 3 VMA implementation files and document things more clearly.
>
> While we're here, slightly improve the language of the comment describing
> vma_exec.c.
>
> No functional change intended.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
But given Mike's efforts [1] I wonder if there's a way to change the vma
header system in the future to stop requiring users of vma functions to
include the whole internal.h, while of course still allow the userland
testability.
[1]
https://lore.kernel.org/all/20260709-internal-h-v1-0-956d5092782d@kernel.org/
^ permalink raw reply
* Re: [PATCH v2 29/33] mm/vma: introduce and use vma_set_pgoff()
From: Vlastimil Babka (SUSE) @ 2026-07-16 8:28 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-29-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> In order to lay the foundation for work that permits us to track the
> virtual page offset of MAP_PRIVATE file-backed mappings, we abstract the
> assignment of vma->vm_pgoff to vma_set_pgoff().
>
> We additionally add a lock check here using the newly introduced
> vma_assert_can_modify(). This asserts the VMA write lock if the VMA is
> attached.
>
> We also assert that, if this is an anonymous VMA and unfaulted, that its
> (virtual) page offset is equal to the page offset of the VMA's address.
>
> We must be careful about MAP_PRIVATE-/dev/zero which violates fundamental
> assumptions about anonymous memory, so we check for !vma->vm_file after
> using vma_is_anonymous() which these mappings satisfy.
>
> Additionally, we only perform the assert if CONFIG_MMU is defined, as nommu
> does not set vma->vm_pgoff = addr >> PAGE_SHIFT. This isn't really relevant
> to rmap as it has no anon rmap (nor needs it), but we must avoid it
> asserting falsely.
>
> All of this logic is kept in assert_sane_pgoff() to keep things clear.
>
> In order to maintain correctness given this assert, we also update
> __install_special_mapping() to invoke vma_set_range() after it's set
> vma->vm_ops (which determine whether the VMA is anonymous or not).
>
> We do not use vma_set_pgoff() in vm_area_init_from(), as at the point of
> forking, we don't necessarily have correct locking state.
>
> Updating vma_set_range() covers most cases, but in addition to this we also
> update insert_vm_struct(), compat_set_vma_from_desc() and nommu callers.
>
> We also update vma_add_pgoff() and vma_sub_pgoff() to use vma_set_pgoff().
>
> While we're here, we drop a BUG_ON() and update insert_vm_struct()'s
> comment to reflect the fact anonymous mappings can be added here.
>
> Finally, we update the CONFIG_MMU, CONFIG_PER_VMA_LOCK defines in the VMA
> userland tests so IS_ENABLED() will work correctly with them.
>
> No functional change intended.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 28/33] mm/vma: slightly rework the anonymous check in __mmap_new_vma()
From: Vlastimil Babka (SUSE) @ 2026-07-16 8:22 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-28-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> Determine if the VMA is anonymous early on, so we separate the logic which
> invokes callbacks from the rest of the logic.
>
> This is in order that a subsequent commit which asserts correct virtual
> page offset for anonymous mappings correctly asserts this for new anonymous
> mappings.
>
> No functional change intended.
>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 27/33] mm/vma: update vmg_adjust_set_range() to offset pgoff instead
From: Vlastimil Babka (SUSE) @ 2026-07-16 8:20 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-27-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> We are calculating the pgoff as an offset, since we have vma_add_pgoff()
> and vma_sub_pgoff() available, just offset this value directly and use
> __vma_set_range() for vma->vm_[start, end] values.
>
> We take care to update the range before offsetting the page offset, so the
> adjusted VMA's vm_start and vm_pgoff are mutually consistent at the point
> the page offset helpers operate - this matters once vma_set_pgoff() comes
> to assert invariants which relate the two.
>
> Doing so lays the foundation for future work which allows for use of
> virtual page offsets for MAP_PRIVATE-file backed mappings.
>
> No functional change intended.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> ---
> mm/vma.c | 15 ++++-----------
> 1 file changed, 4 insertions(+), 11 deletions(-)
>
> diff --git a/mm/vma.c b/mm/vma.c
> index a474920a25b3..0699fa07c40f 100644
> --- a/mm/vma.c
> +++ b/mm/vma.c
> @@ -714,9 +714,6 @@ void validate_mm(struct mm_struct *mm)
> */
> static void vmg_adjust_set_range(struct vma_merge_struct *vmg)
> {
> - struct vm_area_struct *adjust;
> - pgoff_t pgoff;
> -
> if (vmg->__adjust_middle_start) {
> /*
> * vmg->start vmg->end
> @@ -735,8 +732,8 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)
> struct vm_area_struct *middle = vmg->middle;
> const unsigned long delta = vmg->end - middle->vm_start;
>
> - pgoff = vma_start_pgoff(middle) + (delta >> PAGE_SHIFT);
> - adjust = middle;
> + __vma_set_range(middle, vmg->end, middle->vm_end);
> + vma_add_pgoff(middle, delta >> PAGE_SHIFT);
> } else if (vmg->__adjust_next_start) {
> /*
> * Originally:
> @@ -764,13 +761,9 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)
> struct vm_area_struct *next = vmg->next;
> const unsigned long delta = next->vm_start - vmg->end;
>
> - pgoff = vma_start_pgoff(next) - (delta >> PAGE_SHIFT);
> - adjust = next;
> - } else {
> - return;
> + __vma_set_range(next, vmg->end, next->vm_end);
> + vma_sub_pgoff(next, delta >> PAGE_SHIFT);
> }
> -
> - vma_set_range(adjust, vmg->end, adjust->vm_end, pgoff);
> }
>
> /*
>
^ permalink raw reply
* Re: [PATCH] tracing: Propagate errors from remote event bulk updates
From: Vincent Donnefort @ 2026-07-16 7:55 UTC (permalink / raw)
To: Steven Rostedt
Cc: Jackie Liu, mhiramat, mathieu.desnoyers, linux-trace-kernel
In-Reply-To: <20260715141956.6baf0134@gandalf.local.home>
On Wed, Jul 15, 2026 at 02:19:56PM -0400, Steven Rostedt wrote:
> On Wed, 15 Jul 2026 08:25:09 +0100
> Vincent Donnefort <vdonnefort@google.com> wrote:
>
> > > I checked the v2 series. Patch 02/18 handles registration failure cleanup,
> > > and patch 03/18 changes the boolean parsing in
> > > remote_events_dir_enable_write(), but the loop still ignores errors from
> > > trace_remote_enable_event().
> > >
> > > Would you prefer this fix to be folded into your series, or should I resend
> > > a v2 rebased on top of it?
> >
> > Ha yes my bad. Then no, no need to send it separately.
>
> So should I take this patch and send it to Linus now?
There's a v2 here [1] with my Reviewed-by tag.
[1] https://lore.kernel.org/all/20260715074455.3897-1-liu.yun@linux.dev/
>
> -- Steve
^ permalink raw reply
* Re: [PATCH v2 26/33] mm/vma: update vma_shrink() to not pass start, pgoff parameters
From: Vlastimil Babka (SUSE) @ 2026-07-16 7:47 UTC (permalink / raw)
To: Lorenzo Stoakes, Andrew Morton, David Hildenbrand,
Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif
Cc: linux-mm, linux-kernel, linux-parisc, linux-fsdevel, nvdimm,
linux-perf-users, linux-trace-kernel, damon, iommu, kasan-dev,
linux-sgx, etnaviv, dri-devel, linux-arm-msm, freedreno,
linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-26-2a5aa403d977@kernel.org>
On 7/10/26 22:17, Lorenzo Stoakes wrote:
> vma_shrink() is only used by relocate_vma_down() to shrink the tail of a
> VMA. Therefore neither the start nor the pgoff parameters make any sense.
>
> It seemed we were passing the pgoff parameter solely to satisfy
> vma_set_range()'s requirement for pgoff being specified.
>
> Since vma_set_range() is now isolated to vma.c, we can simply introduce
> __vma_set_range() which sets only vma->vm_[start, end], and invoke this
> instead, removing start and pgoff from vma_shrink() altogether.
>
> No functional change intended.
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [BUG] tracing: Too many tries to read user space
From: Jeongho Choi @ 2026-07-16 6:27 UTC (permalink / raw)
To: Steven Rostedt, mhiramat
Cc: jh1012.choi, linux-trace-kernel, linux-kernel, ji2yoon.jo,
minki.jang, hajun.sung
In-Reply-To: <20260713102343.74baba5b@gandalf.local.home>
[-- Attachment #1: Type: text/plain, Size: 1949 bytes --]
On Mon, Jul 13, 2026 at 10:23:43AM -0400, Steven Rostedt wrote:
> On Mon, 13 Jul 2026 12:16:59 +0900
> Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
>
> > > I also kept this around in case it was needed. Looks like it may be needed.
> >
> > Hmm, it's not clear whether this is the result of a scheduling storm
> > caused by kernel threads. But maybe worth to try.
>
> I don't see what else it could be.
>
> >
> > Jeongho, can you try this patch? Or/and you maybe also try using
> > mlock the LMKD process memory which is passed to trace_marker.
>
> Yes, Jeongho, can you please test this patch and also see if mlocking the
> memory used by trace_marker would help too?
>
> -- Steve
>
Hi Steven adn Masami,
Sorry for the delayed response.
I would like to share the current situation.
This issue occurred during the verification process of our latest Exynos
chip development.
Unfortunately, this problem is not being further reproduced.
We have continued testing the unmodified kernel on about 40 devices for
roughly 10 hours per day for approximately one week.
Because the baseline kernel no longer reproduces the problem at a
measurable rate, I do not think that a test with the proposed patch, or
with the LMKD buffer mlocked, would currently allow us to determine
whether either change improves the issue. A non-reproduction with the
patch would be indistinguishable from the current baseline result.
Versions that can be deployed for verification with a large
number of devices are limited to official release-ready versions.
I can only test the test patches you shared by building them
individually and testing them on the few devices I have.
Therefore, at this point I cannot provide a conclusive test result for
the proposed patch. I can still perform limited build and functional
testing on a small number of devices, but I would not be able to verify
whether the patch changes the reproduction rate.
Thanks,
Jeongho
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* [PATCH v8 9/9] selftests: ftrace: Add wprobe trigger testcase
From: Masami Hiramatsu (Google) @ 2026-07-16 2:53 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <178417033089.209165.16717079876036408877.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add a testcase for checking wprobe trigger. This sets set_wprobe and
clear_wprobe triggers on fprobe events to watch dentry access.
So this depends on both wprobe and fprobe.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v8:
- Use TMPDIR for making a test file.
- Ensure the fprobe and target functions exists, if not, exit
with UNRESOLVED (== skip).
Changes in v7:
- Add a newline at the end of file.(style fix)
- Use dentry_kill instead of __dentry_kill.
Changes in v5:
- Enable CONFIG_WPROBE_TRIGGERS in the config for ftrace test.
Changes in v3:
- Newly added.
---
tools/testing/selftests/ftrace/config | 1
.../ftrace/test.d/trigger/trigger-wprobe.tc | 65 ++++++++++++++++++++
2 files changed, 66 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index d2f503722020..ecdee77f360f 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -28,3 +28,4 @@ CONFIG_TRACER_SNAPSHOT=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
CONFIG_WPROBE_EVENTS=y
+CONFIG_WPROBE_TRIGGERS=y
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
new file mode 100644
index 000000000000..5ffe43f886d1
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
@@ -0,0 +1,65 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README events/sched/sched_process_fork/trigger
+
+echo 0 > tracing_on
+rm -f $TMPDIR/hoge
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@0:8 address=$addr value=+0($addr)' > dynamic_events
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "do_truncate" /proc/kallsyms; then
+ echo "UNRESOLVED: do_truncate not found"
+ exit_unresolved
+fi
+if ! grep -wq "dentry_kill" /proc/kallsyms; then
+ echo "UNRESOLVED: dentry_kill not found"
+ exit_unresolved
+fi
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+
+:;: "Add wprobe triggers" ;:
+echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+cat events/fprobes/truncate/trigger | grep ^set_wprobe
+cat events/fprobes/dentry_kill/trigger | grep ^clear_wprobe
+
+:;: "Ensure wprobe is still disabled" ;:
+cat events/wprobes/watch/enable | grep 0
+
+:;: "Enable events for triggers" ;:
+echo 1 >> events/fprobes/truncate/enable
+echo 1 >> events/fprobes/dentry_kill/enable
+
+:;: "Start test workload" ;:
+echo 1 >> tracing_on
+
+echo aaa > $TMPDIR/hoge
+echo bbb > $TMPDIR/hoge
+echo ccc > $TMPDIR/hoge
+rm $TMPDIR/hoge
+
+:;: "Check trace results" ;:
+cat trace | grep watch
+
+:;: "Ensure wprobe becomes disabled again" ;:
+cat events/wprobes/watch/enable | grep 0
+
+:;: "Remove wprobe triggers" ;:
+echo '!set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+echo '!clear_wprobe:watch' >> events/fprobes/dentry_kill/trigger
+! grep ^set_wprobe events/fprobes/truncate/trigger
+! grep ^clear_wprobe events/fprobes/dentry_kill/trigger
+
+exit 0
^ permalink raw reply related
* [PATCH v8 8/9] tracing: wprobe: Add wprobe event trigger
From: Masami Hiramatsu (Google) @ 2026-07-16 2:53 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <178417033089.209165.16717079876036408877.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add wprobe event trigger to set and clear the watch event dynamically.
This allows us to set an watchpoint on a given local variables and
a slab object instead of static objects.
The trigger syntax is below:
- set_wprobe:WPROBE:FIELD[+OFFSET] [if FILTER]
- clear_wprobe:WPROBE[:FIELD[+OFFSET]] [if FILTER]
set_wprobe sets the address pointed by FIELD[+offset] to the WPROBE
event. The FIELD is the field name of trigger event.
clear_wprobe clears the watch address of WPROBE event. If the FIELD
option is specified, it clears only if the current watch address is
same as the given FIELD[+OFFSET] value.
The set_wprobe trigger does not change the type and length, these
must be set when creating a new wprobe.
Also, the WPROBE event must be disabled when setting the new trigger
and it will be busy afterwards. Recommended usage is to add a new
wprobe at NULL address and keep disabled.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v8:
- Redesign wprobe_trigger() to be safe in NMI/hardirq contexts by
deferring register updates to a workqueue via irq_work.
- Skip trigger execution and increment an atomic missed count
(tw->missed) if the tracepoint runs in NMI context to prevent
recursive spinlock deadlocks. (this is currently hidden counter)
- Prohibit attaching wprobe triggers to kprobe_events by checking
TRACE_EVENT_FL_KPROBE in wprobe_trigger_cmd_parse().
- Use call_rcu() in trigger deactivation path and add
synchronize_rcu() in parse failure path to ensure safe RCU
lifetime cleanup.
- Acquire the target tracepoint's module reference via
trace_event_try_get_ref() to prevent module refcount underflows.
- Fix event_trigger_data memory leak by properly freeing the
initial refcount in wprobe_trigger_cmd_parse() on success.
- Call on_each_cpu() with wait=true in wprobe_work_func() to
prevent use-after-free during trigger unregistration.
- Synchronize concurrent wprobe triggers on the same event by
using a shared raw spinlock (tw->lock).
- Drop the support of kprobe events (that should be done later).
Changes in v7:
- Use kzalloc_obj().
- Update sample code in document.
Changes in v6:
- Update according to the latest change of trigger ops.
Changes in v5:
- Following the suggestions, the documentation was revised to suit rst.
Changes in v3:
- Add FIELD option support for clear_wprobe and update document.
- Fix to unregister/free event_trigger_data on file correctly.
- Fix syntax comments.
Changes in v2:
- Getting local cpu perf_event from trace_wprobe directly.
- Remove trace_wprobe_local_perf() because it is conditionally unused.
- Make CONFIG_WPROBE_TRIGGERS a hidden config.
---
Documentation/trace/wprobetrace.rst | 93 +++++++
include/linux/trace_events.h | 1
kernel/trace/Kconfig | 10 +
kernel/trace/trace.h | 1
kernel/trace/trace_wprobe.c | 437 +++++++++++++++++++++++++++++++++++
5 files changed, 542 insertions(+)
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
index eb4f10607530..a4c0f0e676fd 100644
--- a/Documentation/trace/wprobetrace.rst
+++ b/Documentation/trace/wprobetrace.rst
@@ -68,3 +68,96 @@ Here is an example to add a wprobe event on a variable `jiffies`.
<idle>-0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
You can see the code which writes to `jiffies` is `tick_do_update_jiffies64()`.
+
+Combination with trigger action
+-------------------------------
+The event trigger action can extend the utilization of this wprobe.
+
+- set_wprobe:WPEVENT:FIELD[+|-ADJUST]
+- clear_wprobe:WPEVENT[:FIELD[+|-]ADJUST]
+
+Set these triggers to the target event, then the WPROBE event will be
+setup to trace the memory access at FIELD[+|-ADJUST] address.
+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
+forcibly cleared. If FIELD[[+|-]ADJUST] is set, it clears WPEVENT only
+if its watching address is the same as the FIELD[[+|-]ADJUST] value.
+
+Notes:
+The set_wprobe trigger does not change the type and length, these
+must be set when creating a new wprobe.
+
+The WPROBE event must be disabled when setting the new trigger
+and it will be busy afterwards. Recommended usage is to add a new
+wprobe at NULL address and keep disabled.
+
+Wprobe triggers are not supported on kprobe_events, because kprobes
+themselves can use software breakpoints which conflicts with wprobe
+operation.
+
+
+For example, trace the first 8 bytes of the dentry data structure passed
+to do_truncate() until it is deleted by dentry_kill().
+(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
+::
+
+ # echo 'w:watch rw@0:8 address=$addr value=+0($addr)' >> dynamic_events
+ # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+ # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+ # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+ # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+ # echo 1 >> events/fprobes/truncate/enable
+ # echo 1 >> events/fprobes/dentry_kill/enable
+
+ # echo aaa > /tmp/hoge
+ # echo bbb > /tmp/hoge
+ # echo ccc > /tmp/hoge
+ # rm /tmp/hoge
+
+Then, the trace data will show::
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 32/32 #P:8
+ #
+ # _-----=> irqs-off/BH-disabled
+ # / _----=> need-resched
+ # | / _---=> hardirq/softirq
+ # || / _--=> preempt-depth
+ # ||| / _-=> migrate-disable
+ # |||| / delay
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
+ sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
+ sh-107 [004] ..... 9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
+ sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..... 9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
+ sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
+ sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
+ rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
+ rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
+ sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
+ sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+
+You can see the watch event is correctly configured on the dentry.
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index d1e5ab71d928..e6f3bbcbb9af 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -729,6 +729,7 @@ enum event_trigger_type {
ETT_EVENT_HIST = (1 << 4),
ETT_HIST_ENABLE = (1 << 5),
ETT_EVENT_EPROBE = (1 << 6),
+ ETT_EVENT_WPROBE = (1 << 7),
};
extern int filter_match_preds(struct event_filter *filter, void *rec);
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index d9b6fa5c35d9..af570fa2a882 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -876,6 +876,16 @@ config WPROBE_EVENTS
Those events can be inserted wherever hardware breakpoints can be
set, and record accessed memory address and values.
+config WPROBE_TRIGGERS
+ depends on WPROBE_EVENTS
+ depends on HAVE_REINSTALL_HW_BREAKPOINT
+ bool
+ default y
+ help
+ This adds an event trigger which will set the wprobe on a specific
+ field of an event. This allows user to trace the memory access of
+ an address pointed by the event field.
+
config BPF_EVENTS
depends on BPF_SYSCALL
depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 2f07c5c4ffc8..b4f964839350 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1982,6 +1982,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,
void *private_data);
extern void trigger_data_free(struct event_trigger_data *data);
extern int event_trigger_init(struct event_trigger_data *data);
+extern void event_trigger_free(struct event_trigger_data *data);
extern int trace_event_trigger_enable_disable(struct trace_event_file *file,
int trigger_enable);
extern void update_cond_flag(struct trace_event_file *file);
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
index 942ad2810d36..69ad48ac210c 100644
--- a/kernel/trace/trace_wprobe.c
+++ b/kernel/trace/trace_wprobe.c
@@ -6,7 +6,9 @@
*/
#define pr_fmt(fmt) "trace_wprobe: " fmt
+#include <linux/atomic.h>
#include <linux/compiler.h>
+#include <linux/errno.h>
#include <linux/hw_breakpoint.h>
#include <linux/kallsyms.h>
#include <linux/list.h>
@@ -15,11 +17,16 @@
#include <linux/perf_event.h>
#include <linux/rculist.h>
#include <linux/security.h>
+#include <linux/spinlock.h>
#include <linux/tracepoint.h>
#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <linux/irq_work.h>
+#include <linux/preempt.h>
#include <asm/ptrace.h>
+#include "trace.h"
#include "trace_dynevent.h"
#include "trace_probe.h"
#include "trace_probe_kernel.h"
@@ -50,6 +57,10 @@ struct trace_wprobe {
int len;
int type;
const char *symbol;
+ raw_spinlock_t lock;
+ struct irq_work irq_work;
+ struct work_struct work;
+ atomic_t missed;
struct trace_probe tp;
};
@@ -197,9 +208,42 @@ static void __unregister_trace_wprobe(struct trace_wprobe *tw)
}
}
+static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned long addr)
+{
+ struct perf_event *bp = *this_cpu_ptr(tw->bp_event);
+ struct perf_event_attr attr = bp->attr;
+
+ attr.bp_addr = addr;
+ return modify_wide_hw_breakpoint_local(bp, &attr);
+}
+
+static void wprobe_irq_work_func(struct irq_work *irq_work)
+{
+ struct trace_wprobe *tw = container_of(irq_work, struct trace_wprobe, irq_work);
+
+ schedule_work(&tw->work);
+}
+
+static void wprobe_smp_update_func(void *info)
+{
+ struct trace_wprobe *tw = info;
+ unsigned long addr = READ_ONCE(tw->addr);
+
+ trace_wprobe_update_local(tw, addr);
+}
+
+static void wprobe_work_func(struct work_struct *work)
+{
+ struct trace_wprobe *tw = container_of(work, struct trace_wprobe, work);
+
+ on_each_cpu(wprobe_smp_update_func, tw, true);
+}
+
static void free_trace_wprobe(struct trace_wprobe *tw)
{
if (tw) {
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
trace_probe_cleanup(&tw->tp);
kfree(tw->symbol);
kfree(tw);
@@ -231,6 +275,10 @@ static struct trace_wprobe *alloc_trace_wprobe(const char *group,
tw->addr = addr;
tw->len = len;
tw->type = type;
+ raw_spin_lock_init(&tw->lock);
+ init_irq_work(&tw->irq_work, wprobe_irq_work_func);
+ INIT_WORK(&tw->work, wprobe_work_func);
+ atomic_set(&tw->missed, 0);
ret = trace_probe_init(&tw->tp, event, group, false, nargs);
if (ret < 0)
@@ -707,3 +755,392 @@ static __init int init_wprobe_trace(void)
}
fs_initcall(init_wprobe_trace);
+#ifdef CONFIG_WPROBE_TRIGGERS
+
+static int wprobe_trigger_global_enabled;
+
+#define SET_WPROBE_STR "set_wprobe"
+#define CLEAR_WPROBE_STR "clear_wprobe"
+#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)&wprobe_trigger_global_enabled)
+
+struct wprobe_trigger_data {
+ struct rcu_head rcu;
+ struct trace_event_file *file;
+ struct trace_wprobe *tw;
+ unsigned int offset;
+ long adjust;
+ const char *field;
+ bool clear;
+};
+
+static void wprobe_trigger(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long addr, flags;
+ bool changed = false;
+
+ addr = *(unsigned long *)(rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+
+ if (in_nmi()) {
+ atomic_inc(&tw->missed);
+ return;
+ }
+
+ raw_spin_lock_irqsave(&tw->lock, flags);
+
+ if (!wprobe_data->clear) {
+ if (tw->addr == WPROBE_DEFAULT_CLEAR_ADDRESS) {
+ tw->addr = addr;
+ changed = true;
+ clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ }
+ } else {
+ if (tw->addr != WPROBE_DEFAULT_CLEAR_ADDRESS) {
+ if (!wprobe_data->field || tw->addr == addr) {
+ tw->addr = WPROBE_DEFAULT_CLEAR_ADDRESS;
+ changed = true;
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ }
+ }
+ }
+
+ if (changed)
+ irq_work_queue(&tw->irq_work);
+
+ raw_spin_unlock_irqrestore(&tw->lock, flags);
+}
+
+static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_data)
+{
+ if (wprobe_data) {
+ kfree(wprobe_data->field);
+ kfree(wprobe_data);
+ }
+}
+
+/* RCU callback to free wprobe_trigger_data after grace period */
+static void wprobe_trigger_data_rcu_free(struct rcu_head *rcu)
+{
+ struct wprobe_trigger_data *wprobe_data =
+ container_of(rcu, struct wprobe_trigger_data, rcu);
+ free_wprobe_trigger_data(wprobe_data);
+}
+DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_wprobe_trigger_data(_T));
+
+static int wprobe_trigger_print(struct seq_file *m,
+ struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+
+ if (wprobe_data->clear) {
+ seq_printf(m, "%s:%s", CLEAR_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call));
+ if (wprobe_data->field) {
+ seq_printf(m, ":%s%+ld",
+ wprobe_data->field, wprobe_data->adjust);
+ }
+ } else
+ seq_printf(m, "%s:%s:%s%+ld", SET_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call),
+ wprobe_data->field, wprobe_data->adjust);
+
+ if (data->filter_str)
+ seq_printf(m, " if %s\n", data->filter_str);
+ else
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static struct wprobe_trigger_data *
+wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *file,
+ bool clear)
+{
+ struct wprobe_trigger_data *wprobe_data;
+
+ wprobe_data = kzalloc_obj(*wprobe_data);
+ if (!wprobe_data)
+ return NULL;
+
+ wprobe_data->tw = tw;
+ wprobe_data->clear = clear;
+ wprobe_data->file = file;
+
+ return wprobe_data;
+}
+
+static void wprobe_trigger_free(struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ data->ref--;
+ if (!data->ref) {
+ /* Remove the SOFT_MODE flag */
+ trace_event_enable_disable(wprobe_data->file, 0, 1);
+ trace_event_put_ref(wprobe_data->file->event_call);
+ trigger_data_free(data);
+
+ /* Wait for RCU readers (tracepoint executions) to finish */
+ call_rcu(&wprobe_data->rcu, wprobe_trigger_data_rcu_free);
+ }
+}
+
+static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd,
+ char *param_and_filter)
+{
+ /*
+ * set_wprobe:EVENT:FIELD[+OFFS]
+ * clear_wprobe:EVENT[:FIELD[+OFFS]]
+ */
+ struct wprobe_trigger_data *wprobe_data __free(free_wprobe_trigger_data) = NULL;
+ struct event_trigger_data *trigger_data __free(kfree) = NULL;
+ struct ftrace_event_field *field = NULL;
+ struct trace_event_file *wprobe_file;
+ struct trace_array *tr = file->tr;
+ struct trace_event_call *event;
+ char *event_str, *field_str;
+ bool remove, clear = false;
+ struct trace_wprobe *tw;
+ char *param, *filter;
+ int ret;
+
+ remove = event_trigger_check_remove(glob);
+
+ if (!strcmp(cmd, CLEAR_WPROBE_STR))
+ clear = true;
+
+ if (event_trigger_empty_param(param_and_filter))
+ return -EINVAL;
+
+ ret = event_trigger_separate_filter(param_and_filter, ¶m, &filter, true);
+ if (ret)
+ return ret;
+
+ if (file->event_call->flags & TRACE_EVENT_FL_KPROBE)
+ return -EOPNOTSUPP;
+
+ event_str = strsep(¶m, ":");
+
+ /* Find target wprobe */
+ tw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);
+ if (!tw)
+ return -ENOENT;
+ /* The target wprobe must not be used (unless clear) */
+ if (!remove && !clear && trace_probe_is_enabled(&tw->tp))
+ return -EBUSY;
+
+ wprobe_file = find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str);
+ if (!wprobe_file)
+ return -EINVAL;
+
+ wprobe_data = wprobe_trigger_alloc(tw, wprobe_file, clear);
+ if (!wprobe_data)
+ return -ENOMEM;
+
+ /* Find target field, which must be equivarent to "void *" */
+ field_str = strsep(¶m, ":");
+ /* trigger removing or clear_wprobe does not need field. */
+ if (!remove && !clear && !field_str)
+ return -EINVAL;
+
+ if (field_str) {
+ char *offs;
+
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0)
+ return -EINVAL;
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ event = file->event_call;
+ field = trace_find_event_field(event, field_str);
+ if (!field)
+ return -ENOENT;
+
+ if (field->size != sizeof(void *))
+ return -ENOEXEC;
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+ }
+
+ trigger_data = trigger_data_alloc(cmd_ops, cmd, param, wprobe_data);
+ if (!trigger_data)
+ return -ENOMEM;
+
+ /* Up the trigger_data count to make sure nothing frees it on failure */
+ event_trigger_init(trigger_data);
+
+ if (remove) {
+ event_trigger_unregister(cmd_ops, file, glob+1, trigger_data);
+ return 0;
+ }
+
+ ret = event_trigger_parse_num(param, trigger_data);
+ if (ret)
+ return ret;
+
+ ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data);
+ if (ret < 0)
+ return ret;
+
+ /* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */
+ if (!trace_event_try_get_ref(wprobe_file->event_call))
+ return -ENODEV;
+
+ WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS);
+ ret = trace_event_enable_disable(wprobe_file, 1, 1);
+ if (ret < 0) {
+ trace_event_put_ref(wprobe_file->event_call);
+ event_trigger_reset_filter(cmd_ops, trigger_data);
+ return ret;
+ }
+ ret = event_trigger_register(cmd_ops, file, glob, trigger_data);
+ if (ret) {
+ event_trigger_reset_filter(cmd_ops, trigger_data);
+ trace_event_enable_disable(wprobe_file, 0, 1);
+ trace_event_put_ref(wprobe_file->event_call);
+ synchronize_rcu();
+ return ret;
+ }
+ /* Make it NULL to avoid freeing trigger_data and wprobe_data by __free() */
+ wprobe_data = NULL;
+ event_trigger_free(trigger_data);
+ trigger_data = NULL;
+
+ return 0;
+}
+
+/* Return event_trigger_data if there is a trigger which points the same wprobe */
+static struct event_trigger_data *
+wprobe_trigger_find_same(struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct wprobe_trigger_data *test_wprobe_data = test->private_data;
+ struct wprobe_trigger_data *wprobe_data;
+ struct event_trigger_data *iter;
+
+ list_for_each_entry(iter, &file->triggers, list) {
+ wprobe_data = iter->private_data;
+ if (!wprobe_data ||
+ iter->cmd_ops->trigger_type !=
+ test->cmd_ops->trigger_type)
+ continue;
+ if (wprobe_data->tw == test_wprobe_data->tw)
+ return iter;
+ }
+ return NULL;
+}
+
+static int wprobe_register_trigger(char *glob,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ int ret = 0;
+
+ lockdep_assert_held(&event_mutex);
+
+ /* The same wprobe is not accept on the same file (event) */
+ if (wprobe_trigger_find_same(data, file))
+ return -EEXIST;
+
+ if (data->cmd_ops->init) {
+ ret = data->cmd_ops->init(data);
+ if (ret < 0)
+ return ret;
+ }
+
+ list_add_rcu(&data->list, &file->triggers);
+
+ update_cond_flag(file);
+ ret = trace_event_trigger_enable_disable(file, 1);
+ if (ret < 0) {
+ list_del_rcu(&data->list);
+ update_cond_flag(file);
+ }
+ return ret;
+}
+
+static void wprobe_unregister_trigger(char *glob,
+ struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct event_trigger_data *data;
+
+ lockdep_assert_held(&event_mutex);
+
+ data = wprobe_trigger_find_same(test, file);
+ if (!data)
+ return;
+
+ list_del_rcu(&data->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+}
+
+static struct event_command trigger_wprobe_set_cmd = {
+ .name = SET_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = event_trigger_count,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static struct event_command trigger_wprobe_clear_cmd = {
+ .name = CLEAR_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = event_trigger_count,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static __init int init_trigger_wprobe_cmds(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_wprobe_set_cmd);
+ if (WARN_ON(ret < 0))
+ return ret;
+ ret = register_event_command(&trigger_wprobe_clear_cmd);
+ if (WARN_ON(ret < 0))
+ unregister_event_command(&trigger_wprobe_set_cmd);
+
+ if (!ret)
+ wprobe_trigger_global_enabled = 1;
+
+ return ret;
+}
+fs_initcall(init_trigger_wprobe_cmds);
+#endif /* CONFIG_WPROBE_TRIGGERS */
^ permalink raw reply related
* [PATCH v8 7/9] HWBP: Add modify_wide_hw_breakpoint_local() API
From: Masami Hiramatsu (Google) @ 2026-07-16 2:53 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <178417033089.209165.16717079876036408877.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add modify_wide_hw_breakpoint_local() arch-wide interface which allows
hwbp users to update watch address on-line. This is available if the
arch supports CONFIG_HAVE_REINSTALL_HW_BREAKPOINT.
Note that this allows to change the type only for compatible types,
because it does not release and reserve the hwbp slot based on type.
For instance, you can not change HW_BREAKPOINT_W to HW_BREAKPOINT_X.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v8:
- Parse new attributes into a temporary struct arch_hw_breakpoint
copy to prevent in-place mutation and corruption on parse error paths.
- Synchronize logical perf_event attributes by updating
bp->attr.bp_type and bp->attr.bp_len in modify_wide_hw_breakpoint_local().
Changes in v7:
- Update bp->attr.bp_attr so that we can correctly check the
address on it.
- Use -EOPNOTSUPP instead of -ENOSYS.
Changes in v4:
- Update kerneldoc comment about modify_wide_hw_breakpoint_local
according to Randy's comment.
Changes in v2:
- Check type compatibility by checking slot. (Thanks Jinchao!)
---
arch/Kconfig | 10 ++++++++++
arch/x86/Kconfig | 1 +
include/linux/hw_breakpoint.h | 6 ++++++
kernel/events/hw_breakpoint.c | 43 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 60 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index 959aee9568ff..4a87e843bb4c 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -467,6 +467,16 @@ config HAVE_POST_BREAKPOINT_HOOK
Select this option if your arch implements breakpoints overflow
handler hooks after the target memory is modified.
+config HAVE_REINSTALL_HW_BREAKPOINT
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Depending on the arch implementation of hardware breakpoints,
+ some of them are able to update the breakpoint configuration
+ without release and reserve the hardware breakpoint register.
+ What configuration is able to update depends on hardware and
+ software implementation.
+
config HAVE_USER_RETURN_NOTIFIER
bool
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 6b7e14ef8cfb..588218da8f41 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -247,6 +247,7 @@ config X86
select HAVE_GCC_PLUGINS
select HAVE_HW_BREAKPOINT
select HAVE_POST_BREAKPOINT_HOOK
+ select HAVE_REINSTALL_HW_BREAKPOINT
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64
select HAVE_IRQ_TIME_ACCOUNTING
diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index db199d653dd1..6754ffbee9ed 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context);
+extern int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr);
+
extern int register_perf_hw_breakpoint(struct perf_event *bp);
extern void unregister_hw_breakpoint(struct perf_event *bp);
extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);
@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context) { return NULL; }
static inline int
+modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr) { return -EOPNOTSUPP; }
+static inline int
register_perf_hw_breakpoint(struct perf_event *bp) { return -ENOSYS; }
static inline void unregister_hw_breakpoint(struct perf_event *bp) { }
static inline void
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index 789add0c185a..f4709c892d67 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -888,6 +888,49 @@ void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events)
}
EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint);
+/**
+ * modify_wide_hw_breakpoint_local - update breakpoint config for local CPU
+ * @bp: the hwbp perf event for this CPU
+ * @attr: the new attribute for @bp
+ *
+ * This does not release and reserve the slot of a HWBP; it just reuses the
+ * current slot on local CPU. So the users must update the other CPUs by
+ * themselves.
+ * Also, since this does not release/reserve the slot, this can not change the
+ * type to incompatible type of the HWBP.
+ * Return err if attr is invalid or the CPU fails to update debug register
+ * for new @attr.
+ */
+#ifdef CONFIG_HAVE_REINSTALL_HW_BREAKPOINT
+int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr)
+{
+ struct arch_hw_breakpoint info;
+ int ret;
+
+ if (find_slot_idx(bp->attr.bp_type) != find_slot_idx(attr->bp_type))
+ return -EINVAL;
+
+ ret = hw_breakpoint_arch_parse(bp, attr, &info);
+ if (ret)
+ return ret;
+
+ *counter_arch_bp(bp) = info;
+ bp->attr.bp_addr = attr->bp_addr;
+ bp->attr.bp_type = attr->bp_type;
+ bp->attr.bp_len = attr->bp_len;
+
+ return arch_reinstall_hw_breakpoint(bp);
+}
+#else
+int modify_wide_hw_breakpoint_local(struct perf_event *bp,
+ struct perf_event_attr *attr)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+EXPORT_SYMBOL_GPL(modify_wide_hw_breakpoint_local);
+
/**
* hw_breakpoint_is_used - check if breakpoints are currently used
*
^ permalink raw reply related
* [PATCH v8 6/9] x86/hw_breakpoint: Add arch_reinstall_hw_breakpoint
From: Masami Hiramatsu (Google) @ 2026-07-16 2:53 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <178417033089.209165.16717079876036408877.stgit@devnote2>
From: Jinchao Wang <wangjinchao600@gmail.com>
The new arch_reinstall_hw_breakpoint() function can be used in an
atomic context, unlike the more expensive free and re-allocation path.
This allows callers to efficiently re-establish an existing breakpoint.
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
arch/x86/include/asm/hw_breakpoint.h | 2 ++
arch/x86/kernel/hw_breakpoint.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index aa6adac6c3a2..c22cc4e87fc5 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -21,6 +21,7 @@ struct arch_hw_breakpoint {
enum bp_slot_action {
BP_SLOT_ACTION_INSTALL,
+ BP_SLOT_ACTION_REINSTALL,
BP_SLOT_ACTION_UNINSTALL,
};
@@ -65,6 +66,7 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_reinstall_hw_breakpoint(struct perf_event *bp);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index c323c2aab2af..5821bb4650ce 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -100,6 +100,10 @@ static int manage_bp_slot(struct perf_event *bp, enum bp_slot_action action)
old_bp = NULL;
new_bp = bp;
break;
+ case BP_SLOT_ACTION_REINSTALL:
+ old_bp = bp;
+ new_bp = bp;
+ break;
case BP_SLOT_ACTION_UNINSTALL:
old_bp = bp;
new_bp = NULL;
@@ -189,6 +193,11 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
return arch_manage_bp(bp, BP_SLOT_ACTION_INSTALL);
}
+int arch_reinstall_hw_breakpoint(struct perf_event *bp)
+{
+ return arch_manage_bp(bp, BP_SLOT_ACTION_REINSTALL);
+}
+
void arch_uninstall_hw_breakpoint(struct perf_event *bp)
{
arch_manage_bp(bp, BP_SLOT_ACTION_UNINSTALL);
^ permalink raw reply related
* [PATCH v8 5/9] x86/hw_breakpoint: Unify breakpoint install/uninstall
From: Masami Hiramatsu (Google) @ 2026-07-16 2:53 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <178417033089.209165.16717079876036408877.stgit@devnote2>
From: Jinchao Wang <wangjinchao600@gmail.com>
Consolidate breakpoint management to reduce code duplication.
The diffstat was misleading, so the stripped code size is compared instead.
After refactoring, it is reduced from 11976 bytes to 11448 bytes on my
x86_64 system built with clang.
This also makes it easier to introduce arch_reinstall_hw_breakpoint().
In addition, including linux/types.h to fix a missing build dependency.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v8:
- Add missing barrier() on the disable path of setup_hwbp() to prevent
the compiler from reordering this_cpu_write(cpu_dr7, ...) before
set_debugreg(dr7, 7).
- Clear existing slot control and enable bits in cpu_dr7 inside
setup_hwbp() using __encode_dr7() before setting new ones to prevent
register state corruption on update/reinstall.
---
arch/x86/include/asm/hw_breakpoint.h | 6 +
arch/x86/kernel/hw_breakpoint.c | 144 +++++++++++++++++++---------------
2 files changed, 86 insertions(+), 64 deletions(-)
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index 0bc931cd0698..aa6adac6c3a2 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -5,6 +5,7 @@
#include <uapi/asm/hw_breakpoint.h>
#define __ARCH_HW_BREAKPOINT_H
+#include <linux/types.h>
/*
* The name should probably be something dealt in
@@ -18,6 +19,11 @@ struct arch_hw_breakpoint {
u8 type;
};
+enum bp_slot_action {
+ BP_SLOT_ACTION_INSTALL,
+ BP_SLOT_ACTION_UNINSTALL,
+};
+
#include <linux/kdebug.h>
#include <linux/percpu.h>
#include <linux/list.h>
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index f846c15f21ca..c323c2aab2af 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -49,7 +49,6 @@ static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]);
*/
static DEFINE_PER_CPU(struct perf_event *, bp_per_reg[HBP_NUM]);
-
static inline unsigned long
__encode_dr7(int drnum, unsigned int len, unsigned int type)
{
@@ -86,96 +85,113 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *len, unsigned *type)
}
/*
- * Install a perf counter breakpoint.
- *
- * We seek a free debug address register and use it for this
- * breakpoint. Eventually we enable it in the debug control register.
- *
- * Atomic: we hold the counter->ctx->lock and we only handle variables
- * and registers local to this cpu.
+ * We seek a slot and change it or keep it based on the action.
+ * Returns slot number on success, negative error on failure.
+ * Must be called with IRQs disabled.
*/
-int arch_install_hw_breakpoint(struct perf_event *bp)
+static int manage_bp_slot(struct perf_event *bp, enum bp_slot_action action)
{
- struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long *dr7;
- int i;
-
- lockdep_assert_irqs_disabled();
+ struct perf_event *old_bp;
+ struct perf_event *new_bp;
+ int slot;
+
+ switch (action) {
+ case BP_SLOT_ACTION_INSTALL:
+ old_bp = NULL;
+ new_bp = bp;
+ break;
+ case BP_SLOT_ACTION_UNINSTALL:
+ old_bp = bp;
+ new_bp = NULL;
+ break;
+ default:
+ return -EINVAL;
+ }
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+ for (slot = 0; slot < HBP_NUM; slot++) {
+ struct perf_event **curr = this_cpu_ptr(&bp_per_reg[slot]);
- if (!*slot) {
- *slot = bp;
- break;
+ if (*curr == old_bp) {
+ *curr = new_bp;
+ return slot;
}
}
- if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
- return -EBUSY;
+ if (old_bp) {
+ WARN_ONCE(1, "Can't find matching breakpoint slot");
+ return -EINVAL;
+ }
- set_debugreg(info->address, i);
- __this_cpu_write(cpu_debugreg[i], info->address);
+ WARN_ONCE(1, "No free breakpoint slots");
+ return -EBUSY;
+}
- dr7 = this_cpu_ptr(&cpu_dr7);
- *dr7 |= encode_dr7(i, info->len, info->type);
+static void setup_hwbp(struct arch_hw_breakpoint *info, int slot, bool enable)
+{
+ unsigned long dr7;
+
+ set_debugreg(info->address, slot);
+ __this_cpu_write(cpu_debugreg[slot], info->address);
+
+ dr7 = this_cpu_read(cpu_dr7);
+ dr7 &= ~(__encode_dr7(slot, 0xc, 0x3) |
+ (DR_LOCAL_ENABLE << (slot * DR_ENABLE_SIZE)));
+ if (enable)
+ dr7 |= encode_dr7(slot, info->len, info->type);
/*
- * Ensure we first write cpu_dr7 before we set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
+ * Enabling:
+ * Ensure we first write cpu_dr7 before we set the DR7 register.
+ * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
*/
+ if (enable)
+ this_cpu_write(cpu_dr7, dr7);
+
barrier();
- set_debugreg(*dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(info->mask, i);
+ set_debugreg(dr7, 7);
+
+ amd_set_dr_addr_mask(enable ? info->mask : 0, slot);
- return 0;
+ barrier();
+
+ /*
+ * Disabling:
+ * Ensure the write to cpu_dr7 is after we've set the DR7 register.
+ * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
+ */
+ if (!enable)
+ this_cpu_write(cpu_dr7, dr7);
}
/*
- * Uninstall the breakpoint contained in the given counter.
- *
- * First we search the debug address register it uses and then we disable
- * it.
- *
- * Atomic: we hold the counter->ctx->lock and we only handle variables
- * and registers local to this cpu.
+ * find suitable breakpoint slot and set it up based on the action
*/
-void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+static int arch_manage_bp(struct perf_event *bp, enum bp_slot_action action)
{
- struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long dr7;
- int i;
+ struct arch_hw_breakpoint *info;
+ int slot;
lockdep_assert_irqs_disabled();
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
-
- if (*slot == bp) {
- *slot = NULL;
- break;
- }
- }
+ slot = manage_bp_slot(bp, action);
+ if (slot < 0)
+ return slot;
- if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
- return;
+ info = counter_arch_bp(bp);
+ setup_hwbp(info, slot, action != BP_SLOT_ACTION_UNINSTALL);
- dr7 = this_cpu_read(cpu_dr7);
- dr7 &= ~__encode_dr7(i, info->len, info->type);
-
- set_debugreg(dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(0, i);
+ return 0;
+}
- /*
- * Ensure the write to cpu_dr7 is after we've set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
+int arch_install_hw_breakpoint(struct perf_event *bp)
+{
+ return arch_manage_bp(bp, BP_SLOT_ACTION_INSTALL);
+}
- this_cpu_write(cpu_dr7, dr7);
+void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+{
+ arch_manage_bp(bp, BP_SLOT_ACTION_UNINSTALL);
}
static int arch_bp_generic_len(int x86_len)
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox