* Re: [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping
From: Hongyan Xia @ 2026-07-18 3:17 UTC (permalink / raw)
To: Will Deacon
Cc: Pu Hu, mhiramat@kernel.org, ada.coupriediaz@arm.com,
catalin.marinas@arm.com, davem@davemloft.net, Jiazi Li,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
naveen@kernel.org, yang@os.amperecomputing.com
In-Reply-To: <alpuL10h7-OK2hFb@willie-the-truck>
On 7/18/2026 2:02 AM, Will Deacon wrote:
> On Fri, Jul 17, 2026 at 11:31:31AM +0000, Hongyan Xia wrote:
>> On 7/17/2026 7:01 PM, Will Deacon wrote:
>>> Thanks. So perf is run synchronously from the debug exception entry path,
>>
>> Yes, exactly.
>>
>>> rather than because of a second exception taking place. Got it. But then
>>> it sounds like we should really make the debug exception handling path (at
>>> least, the part that runs for handling the kprobe step) noinstr to avoid
>>> getting into this state to begin with. Is that practical?
>>
>> Not sure about making the whole path noinstr (@Masami might have a
>> better opinion on this than me). Personally I don't mind either
>> disallowing it or making it correct.
>>
>> But it might be a good idea not to diverge too much between ISAs. This
>> patch is pretty much mirroring what the x86 side handles this situation.
>
> Ok, so how about this. I'll take these fixes for now, but let's try to
> make these paths noinstr in the future? That's a much bigger job, but I
> do worry that we're going to otherwise end up adding special logic every
> time we run into an unexpected re-entrant case.
Sure, thanks. In our case, surely if someone wants to trace certain
events like preempt_on/off (which is what we are doing with perf),
events outside debug exceptions should be enough. He or she might not be
so keen on events inside debug exceptions, and noinstr shouldn't hurt.
We will give it a thought and see if we can come up with some RFCs later.
>
> Will
^ permalink raw reply
* Re: [PATCH v9 00/10] tracing: wprobe: x86: Add wprobe for watchpoint
From: Borislav Petkov @ 2026-07-18 2:51 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Steven Rostedt, Peter Zijlstra, Ingo Molnar, x86, Jinchao Wang,
Mathieu Desnoyers, Thomas Gleixner, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users
In-Reply-To: <20260718084741.c2ae6e41f2a1510ffa06d12e@kernel.org>
On July 17, 2026 11:47:41 PM UTC, Masami Hiramatsu <mhiramat@kernel.org> wrote:
>On Fri, 17 Jul 2026 09:28:57 -0700
>Borislav Petkov <bp@alien8.de> wrote:
>
>> Hi,
>>
>> On Fri, Jul 17, 2026 at 11:19:30PM +0900, Masami Hiramatsu (Google) wrote:
>> > Here is the 9th version of the series for adding new wprobe (watch probe)
>>
>> any chance you won't blast out your patchset every day?
>
>Ah, sorry about that. I'll rate limit myself this weekend on this series.
Once a week is perfectly fine.
--
Small device. Typos and formatting crap
^ permalink raw reply
* Re: [PATCH] ring-buffer: serialize read-page order with subbuffer resize
From: Yousef Alhouseen @ 2026-07-18 12:20 UTC (permalink / raw)
To: rostedt
Cc: mhiramat, mathieu.desnoyers, petr.pavlu, linux-trace-kernel,
linux-kernel
In-Reply-To: <20260707103619.43b8b553@gandalf.local.home>
Not yet. The mutex-based draft isn't suitable; I still need to
implement and test the resize-in-progress flag design you requested.
I'll send v2 only after that is done.
Thanks,
Yousef
On Tue, 7 Jul 2026 10:36:19 -0400, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Tue, 30 Jun 2026 14:16:48 -0700
> Yousef Alhouseen <alhouseenyousef@gmail.com> wrote:
>
> > Agreed. I’ll add an explicit resize-in-progress flag, set it around
> > the order transition, and make the external read-page
> > allocation/free/read paths reject work while it is set. I’ll check the
> > flag under the locks that serialize each path so it cannot race the
> > transition, then compile and test the resulting v2.
>
> Did you ever get around to sending a v2?
>
> -- Steve
^ permalink raw reply
* [PATCH v2] dmaengine: fsl-edma: tracing: no ptr dereference during log output
From: Martin Kaiser @ 2026-07-18 12:58 UTC (permalink / raw)
To: Frank Li, Vinod Koul
Cc: Steven Rostedt, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, imx, dmaengine, Martin Kaiser, stable
The fsl edma events store a pointer to a struct fsl_edma_engine in the
ringbuffer and dereference it when a log entry is printed. At this time,
the pointer may no longer be valid.
Event injection can be used to trigger a crash:
$ cd /sys/kernel/tracing
$ echo 'value = 0' > events/fsl_edma/edma_writeb/inject
$ cat trace
The log output needs only edma->membase. Add a membase field at the end
of the event and use the new field for log output. Keep the existing
fields for backward compatibility.
Fixes: 11102d0c343b ("dmaengine: fsl-edma: add trace event support")
Cc: stable@vger.kernel.org
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
---
v2:
- dmaengine prefix in the subject line
- Cc stable
- Steven's Reviewed tag
drivers/dma/fsl-edma-trace.h | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/dma/fsl-edma-trace.h b/drivers/dma/fsl-edma-trace.h
index d3541301a247..45d964a3726d 100644
--- a/drivers/dma/fsl-edma-trace.h
+++ b/drivers/dma/fsl-edma-trace.h
@@ -19,14 +19,16 @@ DECLARE_EVENT_CLASS(edma_log_io,
__field(struct fsl_edma_engine *, edma)
__field(void __iomem *, addr)
__field(u32, value)
+ __field(void __iomem *, membase)
),
TP_fast_assign(
__entry->edma = edma;
__entry->addr = addr;
__entry->value = value;
+ __entry->membase = edma->membase;
),
TP_printk("offset %08x: value %08x",
- (u32)(__entry->addr - __entry->edma->membase), __entry->value)
+ (u32)(__entry->addr - __entry->membase), __entry->value)
);
DEFINE_EVENT(edma_log_io, edma_readl,
--
2.43.7
^ permalink raw reply related
* Re: [PATCH v2] dmaengine: fsl-edma: tracing: no ptr dereference during log output
From: Frank Li @ 2026-07-18 14:52 UTC (permalink / raw)
To: Martin Kaiser
Cc: Frank Li, Vinod Koul, Steven Rostedt, Masami Hiramatsu,
linux-kernel, linux-trace-kernel, imx, dmaengine, stable
In-Reply-To: <20260718130024.341243-1-martin@kaiser.cx>
On Sat, Jul 18, 2026 at 02:58:48PM +0200, Martin Kaiser wrote:
> The fsl edma events store a pointer to a struct fsl_edma_engine in the
> ringbuffer and dereference it when a log entry is printed. At this time,
> the pointer may no longer be valid.
>
> Event injection can be used to trigger a crash:
>
> $ cd /sys/kernel/tracing
> $ echo 'value = 0' > events/fsl_edma/edma_writeb/inject
> $ cat trace
>
> The log output needs only edma->membase. Add a membase field at the end
> of the event and use the new field for log output. Keep the existing
> fields for backward compatibility.
>
> Fixes: 11102d0c343b ("dmaengine: fsl-edma: add trace event support")
> Cc: stable@vger.kernel.org
> Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
> Signed-off-by: Martin Kaiser <martin@kaiser.cx>
> ---
Reviewed-by: Frank Li <Frank.Li@nxp.com>
> v2:
> - dmaengine prefix in the subject line
> - Cc stable
> - Steven's Reviewed tag
>
> drivers/dma/fsl-edma-trace.h | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/dma/fsl-edma-trace.h b/drivers/dma/fsl-edma-trace.h
> index d3541301a247..45d964a3726d 100644
> --- a/drivers/dma/fsl-edma-trace.h
> +++ b/drivers/dma/fsl-edma-trace.h
> @@ -19,14 +19,16 @@ DECLARE_EVENT_CLASS(edma_log_io,
> __field(struct fsl_edma_engine *, edma)
> __field(void __iomem *, addr)
> __field(u32, value)
> + __field(void __iomem *, membase)
> ),
> TP_fast_assign(
> __entry->edma = edma;
> __entry->addr = addr;
> __entry->value = value;
> + __entry->membase = edma->membase;
> ),
> TP_printk("offset %08x: value %08x",
> - (u32)(__entry->addr - __entry->edma->membase), __entry->value)
> + (u32)(__entry->addr - __entry->membase), __entry->value)
> );
>
> DEFINE_EVENT(edma_log_io, edma_readl,
> --
> 2.43.7
>
^ permalink raw reply
* Re: [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping
From: Masami Hiramatsu @ 2026-07-19 5:51 UTC (permalink / raw)
To: Hongyan Xia
Cc: Will Deacon, Pu Hu, mhiramat@kernel.org, ada.coupriediaz@arm.com,
catalin.marinas@arm.com, davem@davemloft.net, Jiazi Li,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
naveen@kernel.org, yang@os.amperecomputing.com
In-Reply-To: <be182a18-4cea-445d-a319-1ed3a6c17598@transsion.com>
On Fri, 17 Jul 2026 11:31:31 +0000
Hongyan Xia <hongyan.xia@transsion.com> wrote:
> On 7/17/2026 7:01 PM, Will Deacon wrote:
> > On Fri, Jul 17, 2026 at 01:51:12AM +0000, Hongyan Xia wrote:
> >> On 7/16/2026 11:20 PM, Will Deacon wrote:
> >>> On Thu, Jul 16, 2026 at 02:38:58PM +0000, Pu Hu wrote:
> >>>> On 7/16/2026 9:24 PM, Will Deacon wrote:
> >>>>> On Fri, Jul 10, 2026 at 06:32:55AM +0000, Pu Hu wrote:
> >>>>>> From: Pu Hu <hupu@transsion.com>
> >>>>>>
> >>>>>> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
> >>>>>> can happen when tracing or perf code runs from the debug exception path
> >>>>>> while the first kprobe is preparing or executing its out-of-line
> >>>>>> single-step instruction.
> >>>>>
> >>>>> I don't understand this part. The single-step runs with debug exceptions
> >>>>> disabled (kprobes_save_local_irqflag() sets PSTATE.D) so how do we end
> >>>>> up taking one?
> >>>>
> >>>> You are right that the single-step runs with debug exceptions disabled.
> >>>> However, the case I was referring to is not a hardware breakpoint or a
> >>>> software-step exception, but another Breakpoint Instruction exception
> >>>> generated by executing a BRK instruction. A BRK instruction exception is
> >>>> not masked by PSTATE.D, so it can still be taken while handling a kprobe.
> >>>>
> >>>> As far as I understand the architecture, there are two different cases here:
> >>>>
> >>>> - Breakpoint Instruction exceptions, generated by executing a BRK
> >>>> instruction.
> >>>> - Breakpoint exceptions, generated by the debug logic, for example by
> >>>> programmed breakpoint registers.
> >>>>
> >>>> PSTATE.D masks debug exceptions such as hardware breakpoints,
> >>>> watchpoints and software-step exceptions, but it does not mask
> >>>> Breakpoint Instruction exceptions generated by BRK. This also seems
> >>>> consistent with the pseudocode for BRK,
> >>>> Arch64.SoftwareBreakpoint(imm16), which does not appear to check
> >>>> PSTATE.D before taking the exception.
> >>>>
> >>>> Therefore, even if kprobes_save_local_irqflag() sets PSTATE.D while
> >>>> handling the first kprobe, if the code executed from that path reaches
> >>>> another instruction patched with BRK, it can still take a Breakpoint
> >>>> Instruction exception. In other words, the nested case I mentioned is
> >>>> another kprobe BRK being hit, not a hardware debug exception or a
> >>>> software-step exception.
> >>>
> >>> Yes, that's correct, but if we're doing the out-of-line step, how do we
> >>> end up executing a BRK? Or are you saying that it's the kprobes
> >>> BRK64_OPCODE_KPROBES_SS instruction that we use to implement the
> >>> single-step that is the problem? If so, how does taking that exception
> >>> result in us executing tracing or perf code?
> >>>
> >>> Sorry for all the questions, I just haven't understood what's going on
> >>> here from the commit message.
> >>
> >> The key is that, when you use 'perf --call-graph dwarf' to sample
> >> certain events, kernel perf code will sample a piece of user stack each
> >> time those events are hit, and copy_to/from_user() triggers page faults.
> >> Say you are profiling preempt_enable events:
> >>
> >> 1st BRK -> preempt_disable() -> debug_exception() -> set SS state ->
> >> preempt_enable() -> triggers perf -> perf_sample() -> sample user stack
> >> using copy_to/from_user() -> page fault or 2nd BRK on the page fault path.
> >>
> >> The key is perf sampling the user stack while the 1st BRK is still
> >> running. When a page fault is hit, a can of worms is released, including
> >> a possible 2nd BRK.
> >
> > Thanks. So perf is run synchronously from the debug exception entry path,
>
> Yes, exactly.
>
> > rather than because of a second exception taking place. Got it. But then
> > it sounds like we should really make the debug exception handling path (at
> > least, the part that runs for handling the kprobe step) noinstr to avoid
> > getting into this state to begin with. Is that practical?
>
> Not sure about making the whole path noinstr (@Masami might have a
> better opinion on this than me). Personally I don't mind either
> disallowing it or making it correct.
Yeah I agree with making it noinstr.
>
> But it might be a good idea not to diverge too much between ISAs. This
> patch is pretty much mirroring what the x86 side handles this situation.
I don't mind modifying how it is handled in each architecture, since
this part is too much depending on the ISA.
Thanks,
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply
* Re: [linus:master] [tracing] 206b25c090: kselftests-bpf.ftrace.ftracetest-ktap.Generic_dynamic_event_-_add/remove_eprobe_events.fail
From: Martin Kaiser @ 2026-07-19 10:04 UTC (permalink / raw)
To: kernel test robot
Cc: oe-lkp, lkp, linux-kernel, Masami Hiramatsu, linux-trace-kernel
In-Reply-To: <202607151010.b68428e1-lkp@intel.com>
Hi,
Thus wrote kernel test robot (oliver.sang@intel.com):
> kernel test robot noticed "kselftests-bpf.ftrace.ftracetest-ktap.Generic_dynamic_event_-_add/remove_eprobe_events.fail" on:
> commit: 206b25c09080cc20fd4c2bea12d59df4b7ba2121 ("tracing: eprobe: read the complete FILTER_PTR_STRING pointer")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> [test failed on linus/master 3b029c035b34bbc693405ddf759f0e9b920c27f1]
> [test failed on linux-next/master 49362394dad7df66c274c867a271394c10ca2bb8]
> in testcase: kselftests-bpf
> version:
> with following parameters:
> group: ftrace
> config: x86_64-rhel-9.4-bpf
> compiler: gcc-14
> test machine: 16 threads Intel(R) Core(TM) i7-13620H (Raptor Lake) with 32G memory
> (please refer to attached dmesg/kmsg for entire log/backtrace)
[...]
> # # ++ echo 0
> # # +++ grep '^ *ls-' trace
> # # +++ grep file=
> # # ++ content=' ls-5520 [006] ...1. 335.361454: eprobe_open: (syscalls.sys_enter_openat) file="/usr/libexec/coreutils/libstdbuf.so"
> # # ls-5520 [006] ...1. 335.361620: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v3/libselinux.so.1"
> # # ls-5520 [006] ...1. 335.361649: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v2/libselinux.so.1"
[...]
> # # ls-5520 [006] ...1. 335.364212: eprobe_open: (syscalls.sys_enter_openat) file="/usr/lib/locale/C.UTF-8/LC_NUMERIC"
> # # ls-5520 [006] ...1. 335.364239: eprobe_open: (syscalls.sys_enter_openat) file="/usr/lib/locale/C.utf8/LC_NUMERIC"
> # # ls-5520 [006] ...1. 335.364265: eprobe_open: (syscalls.sys_enter_openat) file="/usr/lib/locale/C.UTF-8/LC_CTYPE"
> # # ls-5520 [006] ...1. 335.364292: eprobe_open: (syscalls.sys_enter_openat) file="/usr/lib/locale/C.utf8/LC_CTYPE"
> # # ls-5520 [006] ...1. 335.364341: eprobe_open: (syscalls.sys_enter_openat) file="."'
> # # +++ grep '^ *ls-' trace
> # # +++ grep file=
> # # +++ grep -v -e '"/' -e '"."' -e '(fault)'
> # # ++ nocontent=' ls-5520 [006] ...1. 335.361620: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v3/libselinux.so.1"
> # # ls-5520 [006] ...1. 335.361649: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v2/libselinux.so.1"
> # # ls-5520 [006] ...1. 335.361676: eprobe_open: (syscalls.sys_enter_openat) file="libselinux.so.1"
> # # ls-5520 [006] ...1. 335.362051: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v3/libcap.so.2"
> # # ls-5520 [006] ...1. 335.362080: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v2/libcap.so.2"
> # # ls-5520 [006] ...1. 335.362107: eprobe_open: (syscalls.sys_enter_openat) file="libcap.so.2"
> # # ls-5520 [006] ...1. 335.362288: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v3/libc.so.6"
> # # ls-5520 [006] ...1. 335.362314: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v2/libc.so.6"
> # # ls-5520 [006] ...1. 335.362340: eprobe_open: (syscalls.sys_enter_openat) file="libc.so.6"
> # # ls-5520 [006] ...1. 335.362552: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v3/libpcre2-8.so.0"
> # # ls-5520 [006] ...1. 335.362581: eprobe_open: (syscalls.sys_enter_openat) file="glibc-hwcaps/x86-64-v2/libpcre2-8.so.0"
> # # ls-5520 [006] ...1. 335.362607: eprobe_open: (syscalls.sys_enter_openat) file="libpcre2-8.so.0"'
This list must be empty for the test to pass. Files with absolute paths and
pointers that can't be dereferenced "(fault)" are filtered.
My patch fixed the dereferencing, some of the faults are now readable
filenames. And for some of those, openat used a relative path.
IMHO, the test is not correct, relative paths should be allowed. (In fact, any
name of an existing file should be ok.)
I'll think about this a bit more and submit a patch to fix the test.
Thanks,
Martin
^ permalink raw reply
* [RFC PATCH 2/7] mm: dispatch __mmput() to an mm_reaper kthread via llist
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
Introduce a single dedicated, freezable, sleepable kthread (mm_reaper)
modeled on oom_reaper, and the machinery to route the deferrable unit of
teardown -- __mmput() -- to it through a lock-free llist plus waitqueue.
mmput_exit() does llist_add() and wakes the kthread, which drains with
llist_del_all() and runs __mmput() per entry with cond_resched(). The
drain loop also calls try_to_freeze() after each entry, so a freeze
request is honoured mid-batch rather than only at the wait point.
Nothing calls mmput_exit() yet: exit_mm() is not routed to it, and there
is no eligibility check or gate, so it would queue every final-drop mm
unconditionally if invoked. This changes no behavior -- the function is
built but unreferenced, the kthread receives no work, and __mmput() runs
inline on the exiting CPU exactly as today. The exit_mm() hook and the
eligibility/gating policy follow in later patches.
llist (rather than oom_reaper's spinlock+list) avoids a global lock on
the exit path under many-core parallel exits. llist_for_each_entry_safe()
caches the next pointer before __mmput(), so iteration is safe even when
the final mmdrop() inside __mmput() frees mm.
Lifetime: the mm_count reference that mm_users>0 stands for is released
only by that final mmdrop() inside __mmput(). Deferring __mmput() keeps
the reference held, so the mm_struct and its embedded async_reap_node
stay alive until the kthread runs. No extra mmgrab() is needed, for the
same reason mmput_async() needs none.
The kthread is started at subsys_initcall and set to nice 19 to minimize
competition with other EEVDF tasks. Its affinity is expressed as a
preference via kthread_affine_preferred() over the HK_TYPE_KTHREAD
housekeeping mask, so it keeps off isolcpus= and cpuset-isolated CPUs
without the hard binding kthread_bind_mask() would impose --
PF_NO_SETAFFINITY would otherwise leave admins unable to re-affine it.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
kernel/fork.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/kernel/fork.c b/kernel/fork.c
index f0e2e131a..884e4e767 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -112,6 +112,7 @@
#include <linux/unwind_deferred.h>
#include <linux/pgalloc.h>
#include <linux/uaccess.h>
+#include <linux/sched/isolation.h>
#include <asm/mmu_context.h>
#include <asm/cacheflush.h>
@@ -3407,3 +3408,56 @@ static int __init init_fork_sysctl(void)
}
subsys_initcall(init_fork_sysctl);
+
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+static LLIST_HEAD(mm_reaper_list);
+static DECLARE_WAIT_QUEUE_HEAD(mm_reaper_wait);
+
+static void async_mm_teardown_queue(struct mm_struct *mm)
+{
+ if (llist_add(&mm->async_reap_node, &mm_reaper_list))
+ wake_up(&mm_reaper_wait);
+}
+
+static int mm_reaper(void *unused)
+{
+ set_freezable();
+ while (true) {
+ struct llist_node *batch;
+ struct mm_struct *mm, *n;
+
+ wait_event_freezable(mm_reaper_wait, !llist_empty(&mm_reaper_list));
+ batch = llist_del_all(&mm_reaper_list);
+ llist_for_each_entry_safe(mm, n, batch, async_reap_node) {
+ __mmput(mm); /* may free mm via mmdrop */
+ cond_resched();
+ try_to_freeze();
+ }
+ }
+ return 0;
+}
+
+void mmput_exit(struct mm_struct *mm)
+{
+ might_sleep();
+ if (!atomic_dec_and_test(&mm->mm_users))
+ return;
+ async_mm_teardown_queue(mm);
+}
+
+static int __init mm_reaper_init(void)
+{
+ struct task_struct *th;
+
+ th = kthread_create(mm_reaper, NULL, "mm_reaper");
+ if (IS_ERR(th)) {
+ pr_err("mm_reaper: failed to start kthread: %ld\n", PTR_ERR(th));
+ return PTR_ERR(th);
+ }
+ set_user_nice(th, 19); /* minimize competition with other fair class tasks */
+ kthread_affine_preferred(th, housekeeping_cpumask(HK_TYPE_KTHREAD));
+ wake_up_process(th);
+ return 0;
+}
+subsys_initcall(mm_reaper_init);
+#endif
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 6/7] mm: add tracepoints and vmstat counters for async teardown
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
Add observability for the deferred teardown path:
- vmstat (CONFIG_ASYNC_MM_TEARDOWN, in /proc/vmstat):
async_mm_teardown_queued - deferred to the mm_reaper kthread
async_mm_teardown_sync - torn down inline on the exit path:
feature off, mm below the RSS
threshold, MMF_OOM_SKIP or
MMF_OOM_TARGETED set, or rejected by
backpressure. Counts only teardowns
reaching mmput_exit(); final drops via
mmput()/mmput_async() elsewhere in the
kernel are inline but not counted here.
async_mm_teardown_rejected - subset of the above that was eligible
but lost to async_mm_teardown_reserve()
(counted for both this and sync, so
sync stays the total for the exit path)
- tracepoints in a new include/trace/events/mm_reaper.h:
mm_async_teardown_queue at enqueue, and
mm_async_teardown_reap when the reaper picks an mm up, just before
__mmput(). queue carries the mm, its RSS,
and the exiting task's pid/comm, captured
while current is still that task, plus the
node of the exiting task. reap carries the
mm, the charged RSS (mm->async_reap_rss, the
amount mmput_exit() reserved and what the
pending-pages accounting releases), a fresh
live RSS read taken just before __mmput(),
and the node of the reaper. The gap between
charged and live measures how much reclaim
ate out of the queue before the reaper got
to it.
reap is emitted before __mmput() because the final mmdrop() inside
__mmput() may free the mm.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
include/linux/vm_event_item.h | 5 +++
include/trace/events/mm_reaper.h | 73 ++++++++++++++++++++++++++++++++
kernel/fork.c | 14 +++++-
mm/vmstat.c | 5 +++
4 files changed, 95 insertions(+), 2 deletions(-)
create mode 100644 include/trace/events/mm_reaper.h
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index 2628ccda0..de4dc20b7 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -179,6 +179,11 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
NRSWPIN,
NRSWPOUT,
#endif /* CONFIG_SWAP */
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+ ASYNC_MM_TEARDOWN_QUEUED,
+ ASYNC_MM_TEARDOWN_SYNC,
+ ASYNC_MM_TEARDOWN_REJECTED,
+#endif
NR_VM_EVENT_ITEMS
};
diff --git a/include/trace/events/mm_reaper.h b/include/trace/events/mm_reaper.h
new file mode 100644
index 000000000..315bced7c
--- /dev/null
+++ b/include/trace/events/mm_reaper.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM mm_reaper
+
+#if !defined(_TRACE_MM_REAPER_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_MM_REAPER_H
+
+#include <linux/tracepoint.h>
+#include <linux/sched.h>
+#include <linux/topology.h>
+
+TRACE_EVENT(mm_async_teardown_queue,
+
+ TP_PROTO(struct mm_struct *mm, unsigned long rss),
+
+ TP_ARGS(mm, rss),
+
+ TP_STRUCT__entry(
+ __field(struct mm_struct *, mm)
+ __field(int, pid)
+ __array(char, comm, TASK_COMM_LEN)
+ __field(unsigned long, rss)
+ __field(int, node)
+ ),
+
+ TP_fast_assign(
+ __entry->mm = mm;
+ __entry->pid = current->pid;
+ memcpy(__entry->comm, current->comm, TASK_COMM_LEN);
+ __entry->rss = rss;
+ __entry->node = numa_node_id();
+ ),
+
+ TP_printk("mm=%p pid=%d comm=%s rss=%lukB node=%d",
+ __entry->mm,
+ __entry->pid,
+ __entry->comm,
+ __entry->rss << (PAGE_SHIFT - 10),
+ __entry->node
+ )
+);
+
+TRACE_EVENT(mm_async_teardown_reap,
+
+ TP_PROTO(struct mm_struct *mm, unsigned long charged_rss, unsigned long live_rss),
+
+ TP_ARGS(mm, charged_rss, live_rss),
+
+ TP_STRUCT__entry(
+ __field(struct mm_struct *, mm)
+ __field(unsigned long, charged_rss)
+ __field(unsigned long, live_rss)
+ __field(int, node)
+ ),
+
+ TP_fast_assign(
+ __entry->mm = mm;
+ __entry->charged_rss = charged_rss;
+ __entry->live_rss = live_rss;
+ __entry->node = numa_node_id();
+ ),
+
+ TP_printk("mm=%p charged_rss=%lukB live_rss=%lukB node=%d",
+ __entry->mm,
+ __entry->charged_rss << (PAGE_SHIFT - 10),
+ __entry->live_rss << (PAGE_SHIFT - 10),
+ __entry->node
+ )
+);
+
+#endif /* _TRACE_MM_REAPER_H */
+
+#include <trace/define_trace.h>
diff --git a/kernel/fork.c b/kernel/fork.c
index 0976770db..4bdffa188 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -126,6 +126,9 @@
#define CREATE_TRACE_POINTS
#include <trace/events/task.h>
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+#include <trace/events/mm_reaper.h>
+#endif
#include <kunit/visibility.h>
@@ -3461,8 +3464,10 @@ static bool async_mm_teardown_reserve(unsigned long rss)
return true;
}
-static void async_mm_teardown_queue(struct mm_struct *mm)
+static void async_mm_teardown_queue(struct mm_struct *mm, unsigned long rss)
{
+ count_vm_event(ASYNC_MM_TEARDOWN_QUEUED);
+ trace_mm_async_teardown_queue(mm, rss);
if (llist_add(&mm->async_reap_node, &mm_reaper_list))
wake_up(&mm_reaper_wait);
}
@@ -3478,6 +3483,9 @@ static int mm_reaper(void *unused)
batch = llist_del_all(&mm_reaper_list);
llist_for_each_entry_safe(mm, n, batch, async_reap_node) {
unsigned long pages = mm->async_reap_rss;
+ unsigned long live = get_mm_rss(mm);
+
+ trace_mm_async_teardown_reap(mm, pages, live);
__mmput(mm); /* may free mm via mmdrop */
atomic_long_sub(pages, &mm_reaper_pending_pages);
@@ -3515,12 +3523,14 @@ void mmput_exit(struct mm_struct *mm)
if (async_mm_teardown_reserve(rss)) {
mm->async_reap_rss = rss;
- async_mm_teardown_queue(mm);
+ async_mm_teardown_queue(mm, rss);
return;
}
+ count_vm_event(ASYNC_MM_TEARDOWN_REJECTED);
}
}
+ count_vm_event(ASYNC_MM_TEARDOWN_SYNC);
__mmput(mm);
}
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 4e26e5fd6..e3ff30027 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1494,6 +1494,11 @@ const char * const vmstat_text[] = {
[I(NRSWPIN)] = "nrswpin",
[I(NRSWPOUT)] = "nrswpout",
#endif /* CONFIG_SWAP */
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+ [I(ASYNC_MM_TEARDOWN_QUEUED)] = "async_mm_teardown_queued",
+ [I(ASYNC_MM_TEARDOWN_SYNC)] = "async_mm_teardown_sync",
+ [I(ASYNC_MM_TEARDOWN_REJECTED)] = "async_mm_teardown_rejected",
+#endif
#undef I
#endif /* CONFIG_VM_EVENT_COUNTERS */
};
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 7/7] exit: route exit_mm()'s final mmput() through mmput_exit()
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
This makes the series operative. So far, nothing called mmput_exit().
Switch exit_mm() from mmput() to mmput_exit(), so the exiting task's final
reference drop can defer the address-space teardown to mm_reaper
instead of running exit_mmap() on the exiting CPU.
Only the exit path is routed. Every other mmput() caller (get_task_mm()
users like ptrace and /proc, kthread_unuse_mm(), etc.) still tears down
inline, and if one of those ends up holding the actual last reference,
behavior is unchanged -- the deferral applies only when the exiting
task's own drop is the final one.
The deferral triggers only when all of the following hold, and falls
back to inline __mmput() otherwise:
- vm.async_mm_teardown=1 (static key, default off)
- RSS >= vm.async_mm_teardown_thresh_pages (default 64MB)
- the mm is not an OOM target (MMF_OOM_TARGETED) and has not already
been reaped (MMF_OOM_SKIP)
- charging the RSS to the pending budget stays under
vm.async_mm_teardown_max_pending_pages
Results, bare metal, 32 CPUs / 32GB RAM (31 GiB usable), single
NUMA node, performance governor, medians of 10 reps, THP off unless noted,
vm.async_mm_teardown_max_pending_pages raised to 3/4 of RAM for the
async runs (the RAM/4 default would reject a single 16GB mm on this
box; see the cover letter's open question on the cap default):
reap latency (SIGKILL -> waitpid() returns), async off -> on:
1GB 18.01 ms -> 0.14 ms
4GB 73.71 ms -> 0.14 ms
16GB 282.16 ms -> 0.15 ms
16GB 27.68 ms -> 0.13 ms (THP on)
voluntary exit (_exit() -> reap) matches within noise:
16GB 271.27 ms -> 0.12 ms
redis-server with a 16GB populated dataset (real heap, VmRSS ~16.2GB):
292.97 ms -> 0.24 ms
32 simultaneous 0.5GB exits, tail until all reaped:
157.42 ms -> 0.32 ms
below-threshold 30MB process with the feature enabled:
0.56 ms -> 0.60 ms (sync path, within noise)
A CONFIG_ASYNC_MM_TEARDOWN=n build of the same tree reproduces the
async-off column within noise (18.01 vs 18.01 ms at 1GB, 282.14 vs
282.16 ms at 16GB), so the config itself costs nothing when dark.
The work is moved, not eliminated: with async on, the 16GB region is
fully back in the buddy allocator ~240 ms after the kill (vs ~283 ms
when torn down inline), and the reaper spends ~1.18x the inline CPU on
the same teardowns (nice-19 kthread, cache-cold on another CPU). The
cover letter carries the full matrix: freeing-latency checks,
backpressure fallback, OOM-killer interaction (an OOM victim is never
queued, tracepoint-verified, including the CLONE_VM-sharer case),
tmpfs inode eviction in reaper context, sysctl toggling under load,
and a freezer cycle with a queued backlog.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
kernel/exit.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/exit.c b/kernel/exit.c
index 1056422bc..6f97d204f 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -607,7 +607,7 @@ static void exit_mm(void)
task_unlock(current);
mmap_read_unlock(mm);
mm_update_next_owner(mm);
- mmput(mm);
+ mmput_exit(mm);
if (test_thread_flag(TIF_MEMDIE))
exit_oom_victim();
}
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 1/7] mm: add CONFIG_ASYNC_MM_TEARDOWN scaffolding for off-CPU exit teardown
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
Address-space teardown on process exit runs synchronously in the dying
task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks
page tables, updates rmap, frees the RSS and drops file refs, all on the
exiting CPU.
Add the scaffolding to later move __mmput() off-CPU for large exiting
address spaces, with no functional change in this patch:
- new config ASYNC_MM_TEARDOWN (bool, depends on MMU, default n)
- struct mm_struct::async_reap_node (llist_node), CONFIG-gated, with
an explicit <linux/llist.h> include (it was only ever available
transitively via spinlock.h)
- mmput_exit() declaration, with a static inline fallback that is a
plain mmput
When the config is disabled mmput_exit() compiles to mmput() with zero
delta, so nothing changes until the kthread and dispatch land.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
include/linux/mm_types.h | 4 ++++
include/linux/sched/mm.h | 6 ++++++
mm/Kconfig | 11 +++++++++++
3 files changed, 21 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 939b5ea8c..fffa285ef 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -7,6 +7,7 @@
#include <linux/auxvec.h>
#include <linux/kref.h>
#include <linux/list.h>
+#include <linux/llist.h>
#include <linux/spinlock.h>
#include <linux/rbtree.h>
#include <linux/maple_tree.h>
@@ -1367,6 +1368,9 @@ struct mm_struct {
atomic_long_t hugetlb_usage;
#endif
struct work_struct async_put_work;
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+ struct llist_node async_reap_node;
+#endif /* CONFIG_ASYNC_MM_TEARDOWN */
#ifdef CONFIG_IOMMU_MM_DATA
struct iommu_mm_data *iommu_mm;
diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
index 10be8a54b..20aeb1d7e 100644
--- a/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -147,6 +147,12 @@ extern void mmput(struct mm_struct *);
void mmput_async(struct mm_struct *);
#endif
+#ifdef CONFIG_ASYNC_MM_TEARDOWN
+void mmput_exit(struct mm_struct *);
+#else
+static inline void mmput_exit(struct mm_struct *mm) { mmput(mm); }
+#endif
+
/* Grab a reference to a task's mm, if it is not already going away */
extern struct mm_struct *get_task_mm(struct task_struct *task);
/*
diff --git a/mm/Kconfig b/mm/Kconfig
index c52ab6afc..296b16ad4 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1500,6 +1500,17 @@ config LAZY_MMU_MODE_KUNIT_TEST
If unsure, say N.
+config ASYNC_MM_TEARDOWN
+ bool "Async off-CPU address space teardown on exit"
+ depends on MMU
+ default n
+ help
+ Defer the address space teardown (exit_mmap()) of large exiting processes
+ to a kernel thread instead of running it on the exiting CPU. The feature
+ is off by default.
+
+ If unsure, say N.
+
source "mm/damon/Kconfig"
endmenu
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 5/7] mm: add runtime toggle and sysctls for async teardown
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
Add the userspace interface that arms and tunes off-CPU exit teardown,
and the static key that gates it.
Introduce async_mm_teardown_key (static key, default false) and gate the
deferral path in mmput_exit() on it via static_branch_unlikely(); when
off, mmput_exit() runs __mmput() inline. The get_mm_rss() read moves
inside the static_branch_unlikely() block, alongside eligibility and
exit_aio(), so the default (key off) path costs nothing beyond the
branch itself -- no RSS scan for every exiting process on kernels or
boots that never enable the feature. Under CONFIG_SYSCTL, register
three knobs under vm. with register_sysctl_init():
- async_mm_teardown: write 1/0 to enable/disable. A custom handler
flips the static key, serialized by a mutex so the key state always
matches the last value written.
- async_mm_teardown_thresh_pages: RSS threshold (default ~64MB).
- async_mm_teardown_max_pending_pages: backpressure cap.
The threshold defaults are set in the initcall outside any CONFIG_SYSCTL
guard, since eligible()/reserve() read them whenever the key is on; only
the toggle variable, its lock, the handler and the table depend on
CONFIG_SYSCTL. With CONFIG_SYSCTL=n there is no path to enable the key,
so the feature stays off and the thresholds are inert.
This patch is still dormant: nothing calls mmput_exit() yet, so flipping
async_mm_teardown on has no observable effect. exit_mm() is routed
through mmput_exit() later in the series; only then does enabling the key
defer teardown. The toggle and gate are introduced here so that, once
routing lands, the feature defaults off and an explicit opt-in is
required.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
Documentation/admin-guide/sysctl/vm.rst | 38 +++++++++++
kernel/fork.c | 90 ++++++++++++++++++++-----
mm/Kconfig | 3 +-
3 files changed, 114 insertions(+), 17 deletions(-)
diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index 5b318d17a..ccda6c75f 100644
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst
@@ -22,6 +22,9 @@ the writeout of dirty data to disk.
Currently, these files are in /proc/sys/vm:
- admin_reserve_kbytes
+- async_mm_teardown
+- async_mm_teardown_max_pending_pages
+- async_mm_teardown_thresh_pages
- compact_memory
- compaction_proactiveness
- compact_unevictable_allowed
@@ -108,6 +111,41 @@ On x86_64 this is about 128MB.
Changing this takes effect whenever an application requests memory.
+async_mm_teardown
+=================
+
+Available only when CONFIG_ASYNC_MM_TEARDOWN is set. Controls whether the
+address-space teardown (exit_mmap()) of large exiting processes is deferred
+to the mm_reaper kernel thread instead of running on the exiting CPU.
+
+Writing 1 enables deferral, 0 disables it. Disabling makes mmput_exit() tear
+down inline again; it does not drain mms already queued. Defaults to 0.
+
+Deferred teardown consumes CPU in the mm_reaper kernel thread, which runs in
+the root cgroup: the teardown work of a task in a CPU-limited cgroup is not
+charged to that cgroup, similar to kswapd and the oom_reaper. Consider this
+before enabling the feature on systems that rely on strict per-cgroup CPU
+accounting.
+
+
+async_mm_teardown_max_pending_pages
+===================================
+
+Available only when CONFIG_ASYNC_MM_TEARDOWN is set. Backpressure cap, in
+pages, on the total RSS of mms queued for the mm_reaper but not yet torn
+down. An exit that would push the total over this cap tears down
+synchronously instead of queuing. Defaults to totalram_pages() / 4.
+
+
+async_mm_teardown_thresh_pages
+==============================
+
+Available only when CONFIG_ASYNC_MM_TEARDOWN is set. Minimum RSS, in pages,
+for an exiting mm to be eligible for deferral to the mm_reaper. Exiting mms
+below this threshold are always torn down synchronously. Defaults to 64MB
+worth of pages.
+
+
compact_memory
==============
diff --git a/kernel/fork.c b/kernel/fork.c
index d45418e66..0976770db 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -3414,8 +3414,13 @@ subsys_initcall(init_fork_sysctl);
static LLIST_HEAD(mm_reaper_list);
static DECLARE_WAIT_QUEUE_HEAD(mm_reaper_wait);
static atomic_long_t mm_reaper_pending_pages;
+static DEFINE_STATIC_KEY_FALSE(async_mm_teardown_key);
static unsigned long sysctl_async_mm_teardown_thresh_pages;
static unsigned long sysctl_async_mm_teardown_max_pending_pages;
+#ifdef CONFIG_SYSCTL
+static u8 sysctl_async_mm_teardown_enabled;
+static DEFINE_MUTEX(async_mm_teardown_lock);
+#endif
static bool async_mm_teardown_eligible(struct mm_struct *mm, unsigned long rss)
{
@@ -3485,8 +3490,6 @@ static int mm_reaper(void *unused)
void mmput_exit(struct mm_struct *mm)
{
- unsigned long rss;
-
might_sleep();
/*
* Fully ordered RMW: combined with exit_mm() (or exec_mmap(), the
@@ -3498,27 +3501,80 @@ void mmput_exit(struct mm_struct *mm)
if (!atomic_dec_and_test(&mm->mm_users))
return;
- rss = get_mm_rss(mm);
+ if (static_branch_unlikely(&async_mm_teardown_key)) {
+ unsigned long rss = get_mm_rss(mm);
- if (async_mm_teardown_eligible(mm, rss)) {
- /*
- * exit_aio() can block indefinitely. Run it here so a stuck
- * AIO only hangs this task (same as how it would happen in
- * case of synchronous mmput()) instead of stranding every
- * teardown queued behind this mm
- */
- exit_aio(mm);
+ if (async_mm_teardown_eligible(mm, rss)) {
+ /*
+ * exit_aio() can block indefinitely. Run it here so a stuck
+ * AIO only hangs this task (same as how it would happen in
+ * case of synchronous mmput()) instead of stranding every
+ * teardown queued behind this mm
+ */
+ exit_aio(mm);
- if (async_mm_teardown_reserve(rss)) {
- mm->async_reap_rss = rss;
- async_mm_teardown_queue(mm);
- return;
+ if (async_mm_teardown_reserve(rss)) {
+ mm->async_reap_rss = rss;
+ async_mm_teardown_queue(mm);
+ return;
+ }
}
}
__mmput(mm);
}
+#ifdef CONFIG_SYSCTL
+static int async_mm_teardown_enabled_handler(const struct ctl_table *table,
+ int write, void *buffer,
+ size_t *lenp, loff_t *ppos)
+{
+ int ret;
+
+ /*
+ * Serialize concurrent writers so the static key state always matches
+ * the last value written to the variable.
+ */
+ guard(mutex)(&async_mm_teardown_lock);
+
+ ret = proc_dou8vec_minmax(table, write, buffer, lenp, ppos);
+ if (ret || !write)
+ return ret;
+
+ if (sysctl_async_mm_teardown_enabled)
+ static_branch_enable(&async_mm_teardown_key);
+ else
+ static_branch_disable(&async_mm_teardown_key);
+ return 0;
+}
+
+static const struct ctl_table async_mm_teardown_table[] = {
+ {
+ .procname = "async_mm_teardown",
+ .data = &sysctl_async_mm_teardown_enabled,
+ .maxlen = sizeof(u8),
+ .mode = 0644,
+ .proc_handler = async_mm_teardown_enabled_handler,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
+ },
+ {
+ .procname = "async_mm_teardown_thresh_pages",
+ .data = &sysctl_async_mm_teardown_thresh_pages,
+ .maxlen = sizeof(unsigned long),
+ .mode = 0644,
+ .proc_handler = proc_doulongvec_minmax,
+ },
+ {
+ .procname = "async_mm_teardown_max_pending_pages",
+ .data = &sysctl_async_mm_teardown_max_pending_pages,
+ .maxlen = sizeof(unsigned long),
+ .mode = 0644,
+ .proc_handler = proc_doulongvec_minmax,
+ },
+};
+#endif /* CONFIG_SYSCTL */
+
static int __init mm_reaper_init(void)
{
struct task_struct *th;
@@ -3533,7 +3589,9 @@ static int __init mm_reaper_init(void)
sysctl_async_mm_teardown_thresh_pages = SZ_64M >> PAGE_SHIFT;
sysctl_async_mm_teardown_max_pending_pages = totalram_pages() / 4; /* TODO: placeholder */
wake_up_process(th);
-
+#ifdef CONFIG_SYSCTL
+ register_sysctl_init("vm", async_mm_teardown_table);
+#endif
return 0;
}
subsys_initcall(mm_reaper_init);
diff --git a/mm/Kconfig b/mm/Kconfig
index 296b16ad4..a9d925406 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1507,7 +1507,8 @@ config ASYNC_MM_TEARDOWN
help
Defer the address space teardown (exit_mmap()) of large exiting processes
to a kernel thread instead of running it on the exiting CPU. The feature
- is off by default.
+ is off by default and is enabled at runtime via the vm.async_mm_teardown
+ sysctl.
If unsure, say N.
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 4/7] mm: gate async teardown on RSS with pending-pages backpressure
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
Deferring every exit would churn small processes for no benefit and let
unreaped memory grow without bound. Split the defer decision into a
pure predicate and a budget reservation:
- async_mm_teardown_eligible() (side-effect free): defer only when
RSS >= async_mm_teardown_thresh_pages and both MMF_OOM_SKIP and
MMF_OOM_TARGETED are clear:
* MMF_OOM_SKIP -- the mm is hidden from the OOM killer and
reaper. Usually that means it has already been reaped or run
through exit_mmap() and has no RSS left worth deferring; in the
mm-pinned-by-init case (__oom_kill_process()) the flag is set at
kill time with RSS intact, and keeping such an mm synchronous is
equally what we want;
* MMF_OOM_TARGETED -- the mm is the target of an in-flight OOM
kill (added in the preceding patch). Such a victim is torn
down synchronously so its memory is released promptly rather
than queued behind the low-priority, housekeeping-confined
mm_reaper, which would work against the oom_reaper during the
exact pressure that triggered the kill. The flag is set while
some task still holds a live reference to the mm, and this
check is reached only after mm_users has dropped to 0 -- which
requires that task's own exit_mm() (or exec_mmap(), the other
path that sheds ->mm) to have already cleared ->mm under that
same task_lock() -- so marking and checking can never overlap in
time. Whichever thread performs the final decrement is therefore
guaranteed to observe the flag already set, per the preceding
patch.
- async_mm_teardown_reserve(): charge RSS against a pending-pages
budget with atomic_long_add_return(); if the result exceeds
async_mm_teardown_max_pending_pages it subtracts back and returns
false, so the caller falls back to synchronous __mmput().
mmput_exit() reads RSS once and checks eligibility before reservation,
so the budget is only touched for processes that pass the cheap
checks. On success it stores the charged amount in mm->async_reap_node's
companion field async_reap_rss before enqueueing, and the reaper
releases exactly that stored charge after __mmput(). The charge cannot
be re-derived at reap time: RSS keeps shrinking after mm_users reaches
zero -- rmap-based reclaim can still swap out anon pages and truncation
can still unmap file pages, neither of which requires mm_users -- and
get_mm_rss() is an approximate per-CPU counter read besides. Two
independent reads at enqueue and reap would systematically
under-release, drifting the pending counter upward until the cap
permanently forces synchronous fallback.
An eligible mm would otherwise have its entire __mmput() -- including
exit_aio() -- deferred onto the single mm_reaper kthread.
exit_aio() can block indefinitely waiting for in-flight AIO to drain,
so mmput_exit() calls it eagerly, on the exiting task's own context,
before reservation: a stuck AIO context then only stalls this task,
same as a synchronous mmput() would, instead of blocking mm_reaper and
stranding every other mm already queued behind it. exit_aio() is
idempotent -- it is a no-op once mm->ioctx_table has been torn down --
so __mmput() calling it again unconditionally is harmless, whether that
second call lands on the reaper or -- when the reservation fails --
back on the exiting task itself.
Because reservation is add-then-check rather than read-then-add,
committed pending never stays above the cap: an exit that would exceed
it tears down synchronously instead. Under concurrent exits a failing
reservation can transiently inflate the counter between its add and
roll-back, which may cause another exiter to fall back to sync
conservatively -- it can over-reject but never over-admit.
The cap is a coarse safety bound on unreaped memory, not a reclaim
mechanism: pages queued here are invisible to reclaim until the reaper
frees them. Integrating the queue with reclaim (unmap-first draining,
a shrinker) is future work; see the cover letter.
The RSS threshold defaults to a fixed 64MB at init; the pending cap
defaults to totalram_pages() / 4 as a placeholder pending tuning. Both
are static module variables here and not yet user-tunable. The static
key that gates the whole path, and sysctl registration, follow in the
next patch.
This patch is dormant: nothing calls mmput_exit() yet (exit_mm() is
routed to it later in the series), so no teardown is deferred.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
include/linux/mm_types.h | 1 +
kernel/fork.c | 79 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 79 insertions(+), 1 deletion(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 3240c029f..ee6c0ac84 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1370,6 +1370,7 @@ struct mm_struct {
struct work_struct async_put_work;
#ifdef CONFIG_ASYNC_MM_TEARDOWN
struct llist_node async_reap_node;
+ unsigned long async_reap_rss;
#endif /* CONFIG_ASYNC_MM_TEARDOWN */
#ifdef CONFIG_IOMMU_MM_DATA
diff --git a/kernel/fork.c b/kernel/fork.c
index 884e4e767..d45418e66 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -113,6 +113,7 @@
#include <linux/pgalloc.h>
#include <linux/uaccess.h>
#include <linux/sched/isolation.h>
+#include <linux/sizes.h>
#include <asm/mmu_context.h>
#include <asm/cacheflush.h>
@@ -3412,6 +3413,48 @@ subsys_initcall(init_fork_sysctl);
#ifdef CONFIG_ASYNC_MM_TEARDOWN
static LLIST_HEAD(mm_reaper_list);
static DECLARE_WAIT_QUEUE_HEAD(mm_reaper_wait);
+static atomic_long_t mm_reaper_pending_pages;
+static unsigned long sysctl_async_mm_teardown_thresh_pages;
+static unsigned long sysctl_async_mm_teardown_max_pending_pages;
+
+static bool async_mm_teardown_eligible(struct mm_struct *mm, unsigned long rss)
+{
+ if (rss < READ_ONCE(sysctl_async_mm_teardown_thresh_pages))
+ return false;
+ if (mm_flags_test(MMF_OOM_SKIP, mm)) /* reaped, or hidden from the reaper */
+ return false;
+ /*
+ * MMF_OOM_TARGETED is set by the OOM killer while some task still
+ * holds a live reference to this mm (see mark_oom_victim() and
+ * __oom_kill_process()), and this eligibility check is reached only
+ * from mmput_exit(), after mm_users has dropped to 0 -- so marking
+ * and this check can never overlap in time. Whichever thread
+ * performs the final decrement is therefore guaranteed to observe
+ * the flag already set, regardless of whether that thread was
+ * itself ever passed to mark_oom_victim() (a CLONE_VM-sharing
+ * process in another thread group never is, but still observes
+ * this flag on the shared mm).
+ */
+ if (mm_flags_test(MMF_OOM_TARGETED, mm))
+ return false;
+ return true;
+}
+
+/*
+ * Charge rss against the pending-teardown budget. Returns true if it fits
+ * under the cap, in which case the caller enqueues and the reaper releases
+ * the same rss after __mmput(). On overshoot nothing stays charged and the
+ * caller must tear down synchronously.
+ */
+static bool async_mm_teardown_reserve(unsigned long rss)
+{
+ if (atomic_long_add_return(rss, &mm_reaper_pending_pages) >
+ READ_ONCE(sysctl_async_mm_teardown_max_pending_pages)) {
+ atomic_long_sub(rss, &mm_reaper_pending_pages);
+ return false;
+ }
+ return true;
+}
static void async_mm_teardown_queue(struct mm_struct *mm)
{
@@ -3429,7 +3472,10 @@ static int mm_reaper(void *unused)
wait_event_freezable(mm_reaper_wait, !llist_empty(&mm_reaper_list));
batch = llist_del_all(&mm_reaper_list);
llist_for_each_entry_safe(mm, n, batch, async_reap_node) {
+ unsigned long pages = mm->async_reap_rss;
+
__mmput(mm); /* may free mm via mmdrop */
+ atomic_long_sub(pages, &mm_reaper_pending_pages);
cond_resched();
try_to_freeze();
}
@@ -3439,10 +3485,38 @@ static int mm_reaper(void *unused)
void mmput_exit(struct mm_struct *mm)
{
+ unsigned long rss;
+
might_sleep();
+ /*
+ * Fully ordered RMW: combined with exit_mm() (or exec_mmap(), the
+ * other path that sheds ->mm) clearing ->mm under task_lock() before
+ * this call, it guarantees MMF_OOM_TARGETED set at any mark site is
+ * visible here -- see mark_oom_victim(). Any new caller of
+ * mmput_exit() outside exit_mm() must re-verify that argument.
+ */
if (!atomic_dec_and_test(&mm->mm_users))
return;
- async_mm_teardown_queue(mm);
+
+ rss = get_mm_rss(mm);
+
+ if (async_mm_teardown_eligible(mm, rss)) {
+ /*
+ * exit_aio() can block indefinitely. Run it here so a stuck
+ * AIO only hangs this task (same as how it would happen in
+ * case of synchronous mmput()) instead of stranding every
+ * teardown queued behind this mm
+ */
+ exit_aio(mm);
+
+ if (async_mm_teardown_reserve(rss)) {
+ mm->async_reap_rss = rss;
+ async_mm_teardown_queue(mm);
+ return;
+ }
+ }
+
+ __mmput(mm);
}
static int __init mm_reaper_init(void)
@@ -3456,7 +3530,10 @@ static int __init mm_reaper_init(void)
}
set_user_nice(th, 19); /* minimize competition with other fair class tasks */
kthread_affine_preferred(th, housekeeping_cpumask(HK_TYPE_KTHREAD));
+ sysctl_async_mm_teardown_thresh_pages = SZ_64M >> PAGE_SHIFT;
+ sysctl_async_mm_teardown_max_pending_pages = totalram_pages() / 4; /* TODO: placeholder */
wake_up_process(th);
+
return 0;
}
subsys_initcall(mm_reaper_init);
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
Address-space teardown on process exit runs synchronously in the dying
task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks
page tables, updates rmap, frees the RSS and drops file references, all
on the exiting CPU. For a multi-GB process that is hundreds of
milliseconds of exit-path latency, paid by whoever is waiting on the
death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an
orchestrator reaping a fleet of workers. On the test box below, killing
a 16GB process costs ~280ms before the parent's waitpid() returns.
This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path
that defers __mmput() of large exiting processes to a dedicated kernel
thread (mm_reaper), so the exiting CPU is released as soon as the task
is reaped and the teardown runs off to the side.
Design
======
- A single freezable kthread, mm_reaper, modeled on oom_reaper, fed
through a lock-free llist + waitqueue. llist rather than oom_reaper's
spinlock+list so many-core parallel exits do not serialize on a
global lock at enqueue. The kthread runs at nice 19; its affinity is
a preference (kthread_affine_preferred()) for the HK_TYPE_KTHREAD
housekeeping mask, so it keeps off isolcpus= and cpuset-isolated
CPUs without the hard bind of kthread_bind_mask() --
PF_NO_SETAFFINITY would leave admins unable to re-affine it.
- mmput_exit(), called only from exit_mm()'s final reference drop
(patch 7). Every other mmput() caller (get_task_mm() users such as
ptrace and /proc, kthread_unuse_mm(), ...) is unchanged and still
tears down inline if it happens to hold the last reference.
- Deferral only when ALL of the following hold, else inline __mmput():
* vm.async_mm_teardown=1 (static key, default off)
* RSS >= vm.async_mm_teardown_thresh_pages (default 64MB)
* the mm is not an OOM-kill target (MMF_OOM_TARGETED, patch 3) and
not already reaped (MMF_OOM_SKIP)
* charging the RSS to a pending-pages budget stays under
vm.async_mm_teardown_max_pending_pages (add-then-check: can
over-reject under concurrency, never over-admit)
- exit_aio() is pulled forward: mmput_exit() runs it eagerly on the
exiting task before reserving/queuing. exit_aio() can block
indefinitely waiting for in-flight AIO to drain; run eagerly, a
stuck AIO context stalls only that task -- exactly what a
synchronous mmput() would do -- instead of blocking the single
reaper and stranding every teardown queued behind it. It is
idempotent (a no-op once ioctx_table is gone), so __mmput() calling
it again later is harmless.
- OOM exclusion: an OOM victim's memory must come back promptly, not
sit behind a nice-19 kthread working against the oom_reaper.
MMF_OOM_TARGETED is mm-scoped (not per-task) because the final
mm_users drop can come from a CLONE_VM sharer in another thread
group that mark_oom_victim() never sees; it is set before any
SIGKILL for the kill event is dispatched, and the marking and the
eligibility check cannot overlap in time (see patch 3 for the
ordering argument). That argument is belt-and-suspenders, not a
correctness dependency: a missed flag would only defer the
victim's teardown (a latency/priority-inversion regression), never
corrupt anything, and the oom_reaper reaps the victim
independently either way. An LKMM litmus test for the
CLONE_VM-sharer case is available on request.
- No extra mm refcounting: deferring __mmput() keeps the mm_count
reference that mm_users > 0 stands for, exactly as mmput_async()
does; the mm and its embedded llist node stay alive until the
kthread runs.
- Observability: three vmstat counters (queued / sync / rejected) and
two tracepoints. queue carries pid/comm/rss/node of the exiting
task; reap carries the charged RSS vs a fresh RSS read plus the
reaper's node, so the gap measures how much reclaim ate out of the
queue, and queue-vs-reap node mismatches expose NUMA-remote
teardown.
The work is moved, not eliminated. Measured checks (same box
as below):
- Freeing latency: with async on, a killed 16GB region is freed on
the same timescale as inline teardown -- nr_free_pages recovers
within ~300ms of the kill in both configurations (single-rep
watcher readings of 240ms async vs 283ms inline, within run-to-run
variance; not a claim that async frees faster). Deferral does not
delay the return of memory; it makes the exit path stop waiting
for it.
- CPU conservation: 5 consecutive 4GB inline teardowns cost 372ms of
exit-path CPU; mm_reaper spends 440ms of kthread CPU draining the
same five, read from its schedstat over a fully drained window
(ratio 1.18, nice-19 kthread, cache-cold on another CPU).
- Pages queued for teardown are invisible to reclaim until the reaper
frees them. The pending cap bounds that exposure coarsely; it is not
a reclaim mechanism. Reclaim integration (unmap-first draining, a
shrinker) is future work, and is also an answer to the cap
sizing question below.
Scope of the claims: what this series claims today is the common-case
exit-latency win under a modest threshold and a conservative cap. It
deliberately does not claim the multi-GB-teardown-under-memory-pressure
regime: until the queue is visible to reclaim, that regime is exactly
where the cap must stay conservative, and the 16GB+ numbers below are
capability demonstrations, run with the cap deliberately raised on an
otherwise idle machine -- not a recommended production configuration.
Open question 1 is where this either gets fixed (reclaim integration)
or stays scoped.
Numbers
=======
Bare metal, 32 CPUs / 32GB RAM (31 GiB usable), single NUMA node,
performance governor, medians, THP off unless noted. reap = SIGKILL -> waitpid()
returns. For all async-on runs vm.async_mm_teardown_max_pending_pages
was raised to 3/4 of RAM so the measurements exercise the reaper, not
the backpressure fallback: with the RAM/4 default, a single 16GB mm on
this box (8GB cap) -- or the 64GB guest case -- exceeds the cap and
falls back to sync teardown by design. See open question 1.
kill, async off -> on:
1GB 18.01 ms -> 0.14 ms
4GB 73.71 ms -> 0.14 ms
16GB 282.16 ms -> 0.15 ms
16GB 27.68 ms -> 0.13 ms (THP on)
voluntary exit (_exit() -> reap), 16GB:
271.27 ms -> 0.12 ms
M simultaneous 0.5GB exits, tail until ALL reaped:
M=8 40.64 ms -> 0.15 ms
M=16 79.99 ms -> 0.17 ms
M=32 157.42 ms -> 0.32 ms
below-threshold 30MB process, feature ON vs OFF:
0.56 ms -> 0.60 ms (sync path, within noise)
file-backed (tmpfs) 4GB mapping, kill:
35.34 ms -> 0.12 ms
real workload -- redis-server with a 16GB populated dataset
(DEBUG POPULATE, VmRSS ~16.2GB), kill:
292.97 ms -> 0.24 ms
The redis point (5 reps/pass, sync spread 288.5-309.2ms, async
0.12-0.32ms) lands within ~3% of the synthetic 16GB per-GB rate, so
the benchmark's memset region is a fair proxy for a real heap, and
the async side carries over unchanged.
CONFIG=n control: the same driver on a =n build of the same tree
(interface verified fully absent: no sysctls, no vmstat counters, no
kthread, no tracepoints) reproduces the =y async-off column within
noise -- 18.01 vs 18.01 ms at 1GB kill, 282.14 vs 282.16 ms at 16GB
kill -- so with the key off, the config costs nothing measurable.
Single-reaper drain throughput: 16.4GB backlog (32 x 0.5GB) returned in
650ms, ~25GB/s on this box. For scale, a 32-vCPU/88GB QEMU guest,
pinned to one NUMA node of its host (vCPUs and preallocated guest RAM
both bound node-local; guest numbers, same kernel/config): 16GB kill
426.15ms -> 0.13ms, a 64GB backlog (32 x 2GB) drained in ~2.1s
(~31GB/s), a 16GB shmem inode evicted in reaper context, and the
pm_test freezer cycle passed with the full 64GB backlog queued.
Robustness testing (driver script, all sections pass, dmesg clean)
==================================================================
- Backpressure: with the cap forced to 1GB, 8 parallel 0.5GB exits
produce queued += 4, sync fallbacks += 12; the cap is respected and
fallback is graceful.
- OOM killer firing while 8GB of teardowns are queued: 3/3 reps clean,
backlog drains in ~650ms alongside the kill.
- Targeted MMF_OOM_TARGETED probes, verified per-pid against the
tracepoints (a positive control first proves a plain large exit of
the same binary IS seen as queued): a ~512MB memcg OOM victim is
never queued; a CLONE_VM pair sharing one mm, swept by the same
kill, produces no queue event from either thread group. Two claims
are documented as untestable from userspace rather than tested:
task_will_free_mem() fast-path marking, and flag stickiness/fork
non-inheritance (no userspace process can survive owning a targeted
mm: oom_score_adj writes propagate across mm sharers, and the
__oom_kill_process() sweep kills every sharer unconditionally).
- unlink-while-mapped tmpfs file: the VMA's fput at teardown is the
last reference, so the final iput -> shmem_evict_inode of a 4GB
shmem inode runs in reaper context; tmpfs is empty 212ms after the
reap. Note: a kthread's fput cannot use task_work, so the evict
actually executes in the (bound) delayed_fput workqueue -- see open
question 4.
- Toggling: disabling mid-drain still drains the backlog (8.0GB in
670ms) while new exits take the sync path; 50 rapid static-key
flip cycles under concurrent exits, clean.
- pm_test freezer cycle with a 16GB backlog queued: completes, and
the async path works after thaw. Caveat in open question 6.
NOT tested: NUMA. I have no bare-metal access to a multi-node machine;
all bare-metal numbers above are single-node, and the QEMU guests
present a single node too. The driver has a ready-to-run NUMA section
(reaper pinned to node 0, victim on node 0 vs node 1, kthread_tail
compared, tracepoints capture queue-vs-reap node) that skips on
single-node systems. Numbers from a multi-node box would be very
welcome and directly feed open question 2.
Open questions, input wanted
============================
1. The default for vm.async_mm_teardown_max_pending_pages. It is
currently totalram_pages()/4, an explicitly marked placeholder.
A static fraction of RAM is arbitrary; but making it a dynamic
function of available RAM is wrong in a different way: high
occupancy is not load. A box full of page cache or running a
legitimately large in-memory workload is not necessarily under
pressure, and those are exactly the boxes that want async teardown.
Under real pressure the cap is the wrong tool anyway, because the
queued pages are invisible to reclaim regardless of the cap value;
the correct fix there is reclaim integration (above).
The default also cuts against the feature: RAM/4 rejects any
single mm larger than a quarter of RAM from deferral entirely --
the exact processes with the worst exit latency are the first ones
sent back to the sync path. Every async number above needed the
cap raised to 3/4 of RAM to be measurable at all.
And the loose end of the range bites in practice: in a large
guest with the cap at that same 3/4 of RAM, a stress run that
started a new round of large exits before the previous round had
drained ran the box into OOM kills. That is what the design
predicts: queued mms are invisible to reclaim and unreachable by
the OOM killer -- their tasks are already dead, so they appear in
no task dump and the oom_reaper has no task to reap; only
mm_reaper can free them. The kills do resolve the pressure (OOM
victims are MMF_OOM_TARGETED, so their teardown is sync), but
tasks die on a box that is not really out of memory -- the memory
is in the queue. With the RAM/4 default the same pattern
self-limits: reservation fails and exits fall back to sync
teardown, which is its own backpressure.
Note the motivating workload (kill-and-respawn) is exactly the
pattern that re-faults immediately after reap, so transient demand
approaches old+new RSS; the cap bounds the old half. So: is RAM/4
acceptable as a coarse safety bound for now? Should the cap react
to watermarks or PSI instead? Should out_of_memory() drain or
expedite the pending queue before picking a victim, the way it
already coordinates with the oom_reaper? That last one is
attractive because every queued mm is already dead: draining is
guaranteed-progress reclaim with no victim to choose. But an
unbounded synchronous drain inside the allocation path is its own
stall (seconds for a large backlog), so it would need a budget.
Or stay dumb until the shrinker exists?
The shape I have in mind for the real fix: a shrinker whose
count_objects() reports the pending pages and whose scan path
kicks the reaper and switches it to unmap-first draining, so
memory returns ahead of the slow metadata teardown. Running full
__mmput()s inside direct reclaim would trade one latency cliff for
another, so the shrinker's job is to accelerate and prioritize the
drain, not to execute it in the caller's context. If the consensus
is that reclaim visibility must land before exit_mm() is routed
(patch 7), I would make that the centerpiece of v2.
2. One reaper vs per-node reapers vs an unbound workqueue. One thread
sustains ~25GB/s on this box and a 64GB backlog still drains in
~2.1s in the big guest, but all of that is node-local. The tracepoint
node fields exist precisely to evaluate this once multi-node
numbers exist.
3. CPU accounting. Teardown CPU moves from the exiting task (charged
to its cgroup) to a root-cgroup kthread. That is the same class of
escape as kswapd or the oom_reaper, but this extends it to routine
process exit. Patch 5's admin docs state the escape explicitly so
nobody enables this blind; the open question is whether that
suffices for an opt-in feature or whether cgroup-aware charging of
the reaper's CPU is a prerequisite.
4. The delayed_fput hop. Because mm_reaper is a kthread, the final
fput() it drops cannot use task_work and is punted to the bound
delayed_fput workqueue -- so a multi-GB shmem inode evict runs on a
bound system_wq worker (the workqueue watchdog already complains:
"delayed_fput hogged CPU for >13333us, consider switching to
WQ_UNBOUND"). Should the reaper drain its own fput backlog inline,
or is this an argument for making delayed_fput unbound generally?
5. memcg zombies. Deferring teardown also defers the final uncharge,
so a dying memcg can be pinned a little longer by a queued mm
(bounded by the pending cap and measured drain times, i.e.
normally milliseconds). I believe this is acceptable.
6. Freezing. The drain loop calls try_to_freeze() after every entry,
so a freeze request is honoured between mms mid-batch, and the
pm_test freezer cycle passes with a 16GB backlog queued. What
remains is that a single in-flight __mmput() cannot freeze
mid-teardown, so one sufficiently large mm could still exceed the
freeze timeout -- but that is equally true of the same exit_mmap()
running synchronously in the exiting task today. Is per-mm
granularity enough?
Related work
============
- "Two alternatives for mm async teardown" (Claudio Imbrenda, 2021,
https://lore.kernel.org/all/20211111095008.264412-1-imbrenda@linux.ibm.com/
): an earlier RFC with the same
goal, motivated by huge s390 KVM guest address spaces. It offered
either an arch hook in exit_mm() or a process_mmput_async syscall
that runs the teardown in another process's context, plus an OOM
notifier to shield the teardown from the OOM killer. Neither
variant was merged; the s390 protected-guest case was later
addressed with KVM-specific asynchronous destruction. This series
is arch-independent, needs no syscall or userspace agent (kernel
kthread plus an eligibility gate), and integrates with the OOM
killer by exclusion (MMF_OOM_TARGETED keeps victims synchronous)
rather than by blocking it.
- oom_reaper: same kthread pattern, but it strips memory from a
victim that cannot exit; this series is about not making healthy
exits pay the teardown latency.
- process_mrelease(2): userspace-driven reaping of a killed process.
It needs a daemon to issue it per-kill, only helps the kill case
(not voluntary exit), and does not remove the exit-path work from
the dying task -- it races it. This series needs no userspace
agent and covers both exit flavors.
- mmput_async(): existing deferral of __mmput() to a workqueue for
contexts that cannot sleep; this series reuses its lifetime
argument but adds gating, backpressure, OOM exclusion and a
dedicated low-priority thread, none of which mmput_async() has.
Patches
=======
1 mm: add CONFIG_ASYNC_MM_TEARDOWN scaffolding for off-CPU exit
teardown -- config, mm_struct::async_reap_node, mmput_exit()
fallback; no functional change.
2 mm: dispatch __mmput() to an mm_reaper kthread via llist -- the
kthread and queue; built but unreferenced.
3 mm/oom_kill: mark an OOM victim's mm with MMF_OOM_TARGETED -- the
mm-scoped victim flag and its visibility/ordering argument;
nothing reads it yet.
4 mm: gate async teardown on RSS with pending-pages backpressure --
eligibility predicate + budget reservation; charge memoized at
enqueue because RSS keeps shrinking after mm_users hits zero;
exit_aio() pulled forward onto the exiting task.
5 mm: add runtime toggle and sysctls for async teardown -- static
key, three vm.* sysctls, admin-guide docs.
6 mm: add tracepoints and vmstat counters for async teardown.
7 exit: route exit_mm()'s final mmput() through mmput_exit() -- the
switch that makes 1-6 operative.
Patches 1-6 are individually inert (each is either scaffolding or
dead code until patch 7), so the series bisects cleanly with the
feature dark until the last commit, and the runtime default is off
even then.
The benchmark (exit_latency.c) and the driver that produced every
number above (sanity/headline/wait-free/CPU-conservation/reuse/
parallel/backpressure/numa/smallproc/oom/oom-targeted/file-backed/
toggle/freezer sections, ~2k lines total), plus the standalone redis
driver behind the real-workload number, are available on request; if
there is interest I am happy to rework them into
tools/testing/selftests/mm for a later revision.
Based on mm-unstable, commit 890f8c4e827c
("mm/secretmem: don't allow highmem folios").
Aditya Sharma (7):
mm: add CONFIG_ASYNC_MM_TEARDOWN scaffolding for off-CPU exit teardown
mm: dispatch __mmput() to an mm_reaper kthread via llist
mm/oom_kill: mark an OOM victim's mm with MMF_OOM_TARGETED
mm: gate async teardown on RSS with pending-pages backpressure
mm: add runtime toggle and sysctls for async teardown
mm: add tracepoints and vmstat counters for async teardown
exit: route exit_mm()'s final mmput() through mmput_exit()
Documentation/admin-guide/sysctl/vm.rst | 38 +++++
include/linux/mm_types.h | 15 ++
include/linux/sched/mm.h | 6 +
include/linux/vm_event_item.h | 5 +
include/trace/events/mm_reaper.h | 73 +++++++++
kernel/exit.c | 2 +-
kernel/fork.c | 199 ++++++++++++++++++++++++
mm/Kconfig | 12 ++
mm/oom_kill.c | 61 ++++++++
mm/vmstat.c | 5 +
10 files changed, 415 insertions(+), 1 deletion(-)
create mode 100644 include/trace/events/mm_reaper.h
base-commit: 890f8c4e827c918dac668a12eaf63180ba8a9e6d
--
2.34.1
^ permalink raw reply
* [RFC PATCH 3/7] mm/oom_kill: mark an OOM victim's mm with MMF_OOM_TARGETED
From: Aditya Sharma @ 2026-07-19 18:54 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R . Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda, Aditya Sharma
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
A later part of this series defers the address-space teardown of large
exiting processes to a kernel thread (mm_reaper) instead of running it
inline on the exiting CPU. That deferral must never be applied to an
OOM victim: its memory has to be released promptly to relieve the
pressure that triggered the kill, and queuing it behind the
low-priority, housekeeping-CPU-confined mm_reaper would work directly
against the oom_reaper. The teardown path therefore needs a cheap
"is this mm an OOM victim?" test.
The eventual consumer runs from mmput_exit() on the exit path, after
exit_mm() has already cleared current->mm, so the only handle it has
is the mm pointer itself. Existing OOM state does not answer the
question well from an mm alone:
- signal->oom_mm and TIF_MEMDIE are per-task / per-signal and are
set only on the single task passed to mark_oom_victim(). A sibling
thread, or a CLONE_VM-sharing process in another thread group
(which __oom_kill_process() SIGKILLs but never marks), can be the
task that drops the last mm_users reference and runs the teardown.
A per-task check misses those.
- MMF_OOM_SKIP is mm-scoped but does not mean "OOM victim".
exit_mmap() sets it on every exiting mm once the memory is freed,
and dup_mmap() sets it on the fork-failure cleanup path, so it
produces false positives on ordinary exits. It is also not reliably
early: on a real victim it is normally set only once exit_mmap() or
oom_reap_task_mm() has already released the memory -- the exception
being the mm-pinned-by-init case in __oom_kill_process(), which sets
it at kill time with RSS fully intact. Wrong in both directions, so
it cannot serve as the predicate this path needs.
Add MMF_OOM_TARGETED, an mm-scoped flag, and set it in two places:
- in mark_oom_victim(), covering every caller uniformly (this is
the only marking for the two task_will_free_mem() fast paths in
oom_kill_process() and out_of_memory());
- early in __oom_kill_process(), before any SIGKILL for the kill
event is dispatched, to victim's own thread group or to other
thread groups sharing the mm.
Observing this flag is not load-bearing for correctness. Should it
ever be missed, the consequence is bounded: the victim's mm is queued
to mm_reaper instead of being torn down inline, i.e. a latency and
priority-inversion regression against the oom_reaper -- not
corruption, use-after-free or a leak. The mm is still fully torn down
and the oom_reaper still reaps the victim independently. The
visibility argument below is therefore belt-and-suspenders.
Ordering and coverage: this flag is reliably visible to whichever
thread ends up dropping the last reference in mmput_exit(), because
marking and queuing for async teardown can never overlap in time.
Every mark site runs while some task's ->mm still points at this mm --
either tsk is current, marking itself in program order (the
task_will_free_mem() fast paths), or tsk is task_lock()ed by the
caller (oom_kill_process()'s fast path, and __oom_kill_process() via
find_lock_task_mm()) -- while mmput_exit(), the only path that queues
for teardown, is reached only after mm_users hits 0, which requires
that same task's own exit_mm() (or exec_mmap(), the other path that
sheds ->mm) to have already cleared ->mm under that same task_lock().
Program order, or task_lock() release/acquire, therefore orders the
flag store before that task's own eventual mmput_exit(). This is stated
for userspace tasks: any userspace task's ->mm holds an mm_users
reference, whereas kthread_use_mm() borrows an mm under mmgrab() alone
-- kthreads are not signal-killable and are never OOM victims, so the
borrowed-mm case does not arise here.
If a different CLONE_VM sharer in another thread group performs the
actual final decrement instead -- a task mark_oom_victim() never marks
directly -- the ordering still carries, though not from the flag store
itself. mm_flags_set() is set_bit(), a non-value-returning RMW, so it
is unordered and contributes nothing on its own. The ordering comes
from the marking task's own atomic_dec_and_test(), a value-returning
RMW and therefore fully ordered (Documentation/atomic_t.txt: an
smp_mb() before and after). Decrements are totally ordered in
mm_users' modification order with the zeroing one last, so the general
barriers on both sides make the flag store visible to whichever thread
performs it.
The redundant set in mark_oom_victim() on the __oom_kill_process()
path is harmless and keeps the invariant "mark_oom_victim() implies
MMF_OOM_TARGETED" independent of future code motion.
The flag is intentionally sticky: it is never cleared. That is fine
because a marked mm is dying: __oom_kill_process() SIGKILLs every
process sharing the victim's mm -- only global init (whose mm also
gets MMF_OOM_SKIP set on the spot, keeping it off the async path
anyway) and kthreads are exempt from the sweep, and a kthread's
kthread_unuse_mm() drop is a plain mmput(), never mmput_exit(). So
no task that could reach the async path survives owning a marked mm.
Should some future path leave a survivor, the consequence is only
conservative: that mm keeps tearing down synchronously. The flag is
structurally outside legacy-mask inheritance: mm_init() clears the
whole flags bitmap and re-seeds only word 0 from
MMF_INIT_LEGACY_MASK, so flags above bit 31 are never copied on
fork(). NUM_MM_FLAG_BITS is a fixed 64 on all architectures, so bit 32
exists on 32-bit builds as well.
This commit only defines and sets the flag; nothing reads it yet. The
write is gated by IS_ENABLED(CONFIG_ASYNC_MM_TEARDOWN) and compiles
out entirely on kernels that will not consume it. The reader
(async_mm_teardown_eligible()) is added in the next patch, and the
deferred teardown is not routed onto the exit path until later in the
series, so this change is inert on its own.
Note: a bit named MMF_OOM_VICTIM existed until 2022 and was removed
as dead state once exit_mmap()'s special-case reap-before-teardown
was replaced by mmap_lock/MMF_OOM_SKIP-based concurrency between
exit_mmap() and oom_reap_task_mm(). MMF_OOM_TARGETED is a
deliberately distinct name for a new consumer -- deferred exit-path
teardown -- and does not revive the old exit_mmap() behaviour.
Signed-off-by: Aditya Sharma <adi.sharma@zohomail.in>
---
include/linux/mm_types.h | 10 +++++++
mm/oom_kill.c | 61 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 71 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index fffa285ef..3240c029f 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1989,6 +1989,16 @@ enum {
#define MMF_TOPDOWN 31 /* mm searches top down by default */
#define MMF_TOPDOWN_MASK BIT(MMF_TOPDOWN)
+/*
+ * mm is the target of an in-flight OOM kill. Only written under
+ * CONFIG_ASYNC_MM_TEARDOWN today. Sticky: never cleared -- the mm is
+ * dying (__oom_kill_process() kills every process sharing it), and a
+ * hypothetical surviving sharer would merely keep tearing down
+ * synchronously. Above bit 31, so it is outside MMF_INIT_LEGACY_MASK's
+ * domain entirely and is never inherited on fork.
+ */
+#define MMF_OOM_TARGETED 32
+
#define MMF_INIT_LEGACY_MASK (MMF_DUMP_FILTER_MASK |\
MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK |\
MMF_VM_MERGE_ANY_MASK | MMF_TOPDOWN_MASK)
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 5f372f6e2..b5490d1b1 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -754,6 +754,40 @@ static void mark_oom_victim(struct task_struct *tsk)
struct mm_struct *mm = tsk->mm;
WARN_ON(oom_killer_disabled);
+
+ /*
+ * Mark the mm itself, not just this task/signal, as an OOM target,
+ * so the deferred-teardown path can test it from the mm alone. Set
+ * ahead of the TIF_MEMDIE test-and-set below so that every call
+ * marks the mm, including a repeat mark on a task that is already
+ * TIF_MEMDIE.
+ *
+ * This is reliably visible to whichever thread ends up dropping the
+ * last reference in mmput_exit(): marking and queuing for async
+ * teardown can never overlap in time. Every caller reaches this with
+ * tsk->mm still set -- either tsk is current, marking itself in
+ * program order, or tsk is task_lock()ed by the caller (the
+ * task_will_free_mem() fast path in oom_kill_process()) -- while
+ * mmput_exit(), the only path that queues for teardown, is reached
+ * only after mm_users hits 0, which requires tsk's own exit_mm() (or
+ * exec_mmap(), the other path that sheds ->mm) to have already
+ * cleared ->mm under that same task_lock(). Program order or
+ * task_lock() release/acquire therefore orders this store before
+ * tsk's own eventual mmput_exit().
+ *
+ * If a different CLONE_VM sharer performs the actual final decrement
+ * instead, the ordering does not come from this store:
+ * mm_flags_set() is set_bit(), a non-value-returning RMW, so it is
+ * unordered and contributes nothing on its own. It comes from the
+ * marking task's own atomic_dec_and_test(), a value-returning RMW
+ * and therefore fully ordered (an smp_mb() before and after).
+ * Decrements are totally ordered in mm_users' modification order
+ * with the zeroing one last, so the general barriers on both sides
+ * make this store visible to whichever thread performs it.
+ */
+ if (IS_ENABLED(CONFIG_ASYNC_MM_TEARDOWN))
+ mm_flags_set(MMF_OOM_TARGETED, mm);
+
/* OOM killer might race with memcg OOM */
if (test_and_set_tsk_thread_flag(tsk, TIF_MEMDIE))
return;
@@ -931,6 +965,33 @@ static void __oom_kill_process(struct task_struct *victim, const char *message)
mm = victim->mm;
mmgrab(mm);
+ /*
+ * Set this before any SIGKILL for this kill event goes out below,
+ * while task_lock(victim) (held since find_lock_task_mm() above) is
+ * still held. This is reliably visible to whichever thread ends up
+ * running mmput_exit() and dropping the last reference to this mm --
+ * victim itself, a sibling, or a CLONE_VM sharer in another thread
+ * group (which mark_oom_victim() never marks, but which still
+ * observes this store on the shared mm). Marking and queuing for
+ * async teardown can never overlap: mmput_exit() is reached only
+ * after mm_users hits 0, and that requires victim's own exit_mm()
+ * (or exec_mmap(), the other path that sheds ->mm) to have cleared
+ * ->mm under this same task_lock() first, which release/acquire
+ * orders after the store above.
+ *
+ * If some other sharer performs the actual final decrement instead,
+ * the ordering does not come from this store: mm_flags_set() is
+ * set_bit(), a non-value-returning RMW, so it is unordered and
+ * contributes nothing on its own. It comes from the marking task's
+ * own atomic_dec_and_test(), a value-returning RMW and therefore
+ * fully ordered (an smp_mb() before and after). Decrements are
+ * totally ordered in mm_users' modification order with the zeroing
+ * one last, so the general barriers on both sides make this store
+ * visible to whichever thread performs it.
+ */
+ if (IS_ENABLED(CONFIG_ASYNC_MM_TEARDOWN))
+ mm_flags_set(MMF_OOM_TARGETED, mm);
+
/* Raise event before sending signal: task reaper must see this */
count_vm_event(OOM_KILL);
memcg_memory_event_mm(mm, MEMCG_OOM_KILL);
--
2.34.1
^ permalink raw reply related
* Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
From: Liam R . Howlett @ 2026-07-19 22:28 UTC (permalink / raw)
To: Aditya Sharma
Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
On 26/07/20 12:24AM, Aditya Sharma wrote:
> Address-space teardown on process exit runs synchronously in the dying
> task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks
> page tables, updates rmap, frees the RSS and drops file references, all
> on the exiting CPU. For a multi-GB process that is hundreds of
> milliseconds of exit-path latency, paid by whoever is waiting on the
> death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an
> orchestrator reaping a fleet of workers. On the test box below, killing
> a 16GB process costs ~280ms before the parent's waitpid() returns.
>
> This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path
> that defers __mmput() of large exiting processes to a dedicated kernel
> thread (mm_reaper), so the exiting CPU is released as soon as the task
> is reaped and the teardown runs off to the side.
This reads like it was written by an AI, but there is no assisted-by
tags. Which AI did you use to generate these patches and/or cover
letter?
I do not see a benefit to running the task cleanup 'off the side'. In
modern computers, that may mean switching core types that run at
different speeds.
Regardless of speed, the CPU that will be doing the work is remote to
the workload by design, (or may be, depending on the scheduling?). That
is, you are tasking another CPU to do cleanup of local CPU-aware
information.. The ideal CPU to do a task exit is the one that's doing it
already.
There are hidden costs to your approach and I don't really understand
how this could possibly speed things up - even parallelized tasks will
take more CPU time in total, even if locking issues are avoided.
Sure, the task waiting for cleanup to complete may think it's done and
continue to do the Next Thing - but if it really needs the cleanup to be
completed then you've just made it impossible to know when it has
happened.
Also, you've just made this extremely hard to debug if something is
missed.
The work that is done in teardown is necessary. Your time (and tokens?)
would be better spent trying to improve what we have to do instead of
shifting the work around.
Thanks,
Liam
^ permalink raw reply
* Re: [PATCH v2 12/33] MAINTAINERS: Move mm/interval_tree.c to rmap section
From: Liam R. Howlett @ 2026-07-19 23:27 UTC (permalink / raw)
To: Lorenzo Stoakes
Cc: Andrew Morton, David Hildenbrand, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Rik van Riel, Harry Yoo,
Jann Horn, Lance Yang, Pedro Falcato, Russell King, Dinh Nguyen,
Simon Schuster, James E.J. Bottomley, Helge Deller,
Alexander Viro, Christian Brauner, Jan Kara, Dan Williams,
Matthew Wilcox, Muchun Song, Oscar Salvador, Masami Hiramatsu,
Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Zi Yan, Baolin Wang, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin,
Chengming Zhou, SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim,
Byungchul Park, Gregory Price, Ying Huang, Alistair Popple,
Hugh Dickins, Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <20260710-b4-pre-scalable-cow-v2-12-2a5aa403d977@kernel.org>
On 26/07/10 09:16PM, Lorenzo Stoakes wrote:
> This file implements code for the interval trees used by the file and anon
> rmap implementation, so belongs in the rmap section.
>
> Acked-by: Pedro Falcato <pfalcato@suse.de>
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Liam R. Howlett (Oracle) <liam@infradead.org>
> ---
> MAINTAINERS | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c3e535ca4bef..ef168b583325 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -17208,6 +17208,7 @@ R: Lance Yang <lance.yang@linux.dev>
> L: linux-mm@kvack.org
> S: Maintained
> F: include/linux/rmap.h
> +F: mm/interval_tree.c
> F: mm/page_vma_mapped.c
> F: mm/rmap.c
> F: tools/testing/selftests/mm/rmap.c
> @@ -17313,7 +17314,6 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> F: include/trace/events/mmap.h
> F: fs/proc/task_mmu.c
> F: fs/proc/task_nommu.c
> -F: mm/interval_tree.c
> F: mm/mincore.c
> F: mm/mlock.c
> F: mm/mmap.c
>
> --
> 2.55.0
>
^ permalink raw reply
* Re: [PATCH v6 2/9] mm/page_owner: add MR_NEVER to enum migrate_reason and use it for last_migrate_reason
From: Ye Liu @ 2026-07-20 2:39 UTC (permalink / raw)
To: David Hildenbrand (Arm), Vlastimil Babka (SUSE), Andrew Morton,
Steven Rostedt, Masami Hiramatsu, Jan Kiszka, Kieran Bingham
Cc: Zi Yan, Matthew Brost, Joshua Hahn, Rakie Kim, Byungchul Park,
Gregory Price, Ying Huang, Alistair Popple, Mathieu Desnoyers,
Suren Baghdasaryan, Michal Hocko, Brendan Jackman,
Johannes Weiner, linux-mm, linux-kernel, linux-trace-kernel
In-Reply-To: <86ce288c-f82f-4d73-ba51-4d4f4a6ef1e6@kernel.org>
在 2026/7/14 22:44, David Hildenbrand (Arm) 写道:
> On 7/14/26 15:49, Vlastimil Babka (SUSE) wrote:
>> On 7/14/26 15:22, David Hildenbrand (Arm) wrote:
>>> On 7/14/26 15:09, Vlastimil Babka (SUSE) wrote:
>>>>
>>>> It's not used ever as a reason that would be actually passed to migration.
>>>
>>> Right, it's a placeholder for "there is no migrate reason because it is unset"
>>>
>>>> So I think the name is more descriptive this way.
>>>
>>> Not sure I agree. The usual translation of -1 -> unset is NONE or UNSET.
>>
>> (note it's no longer -1 after the patch.)
>>
>>> At least I was confused by "NEVER".
>>
>> I won't bikeshed this, so whatever.
>
> Just to be clear: if everybody here agrees that MR_NEVER is the right thing to
> use, fine with me.
>
> I just stumbled over it and it caught my attention.
>
>>
>> Just to avoid another extra respin, please also say how the following line
>> should change, as if we only rename MR_NEVER to MR_NONE, it will be:
>>
>> + EMe(MR_NONE, "never_migrated")
>
> I would probably just have use "not set".
>
> And if "last_migrate_reason == MR_NONE" that would imply "never".
>
> Ye Liu, feel free to keep it as is if you agree that using NEVER is better here.
>
>
Let's keep things as they are.
--
Thanks,
Ye Liu
^ permalink raw reply
* Re: [PATCH v2 08/33] mm/rmap: rename vma_interval_tree_*() to mapping_rmap_tree_*()
From: Lorenzo Stoakes (ARM) @ 2026-07-20 7:20 UTC (permalink / raw)
To: Zi Yan
Cc: Andrew Morton, David Hildenbrand, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Rik van Riel, Harry Yoo, Jann Horn, Lance Yang, Pedro Falcato,
Russell King, Dinh Nguyen, Simon Schuster, James E.J. Bottomley,
Helge Deller, Alexander Viro, Christian Brauner, Jan Kara,
Dan Williams, Matthew Wilcox, Muchun Song, Oscar Salvador,
Masami Hiramatsu, Oleg Nesterov, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Namhyung Kim, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Ian Rogers, Adrian Hunter,
James Clark, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Miaohe Lin, Naoya Horiguchi, Xu Xin, Chengming Zhou,
SJ Park, Matthew Brost, Joshua Hahn, Rakie Kim, Byungchul Park,
Gregory Price, Ying Huang, Alistair Popple, Hugh Dickins,
Peter Xu, Kees Cook, Marek Szyprowski, Robin Murphy,
Andrey Konovalov, Alexander Potapenko, Dmitry Vyukov,
Steven Rostedt, Mathieu Desnoyers, Jarkko Sakkinen, Dave Hansen,
Thomas Gleixner, Borislav Petkov, x86, H. Peter Anvin, Ian Abbott,
H Hartley Sweeten, Lucas Stach, Christian Gmeiner, David Airlie,
Simona Vetter, Patrik Jakobsson, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Tomi Valkeinen,
Thierry Reding, Mikko Perttunen, Jonathan Hunter,
Christian Koenig, Huang Rui, Matthew Auld, Jason Gunthorpe,
Yishai Hadas, Shameer Kolothum, Kevin Tian, Ankit Agrawal,
Alex Williamson, Paolo Bonzini, Shakeel Butt, Usama Arif,
linux-mm, linux-kernel, linux-arm-kernel, linux-parisc,
linux-fsdevel, nvdimm, linux-perf-users, linux-trace-kernel,
damon, iommu, kasan-dev, linux-sgx, etnaviv, dri-devel,
linux-arm-msm, freedreno, linux-tegra, kvm, Russell King
In-Reply-To: <DK1CWBDK5J60.3VPRP45IS7L7E@nvidia.com>
On Fri, Jul 17, 2026 at 10:54:24PM -0400, Zi Yan wrote:
> On Fri Jul 10, 2026 at 4:16 PM EDT, Lorenzo Stoakes wrote:
> > The family of vma_interval_tree_() functions manipulate the
> > address_space (which, of course, is generally referred to as 'mapping')
> > reverse mapping, but are named the 'VMA' interval tree.
> >
> > VMAs may be mapped by an anon_vma, an address_space, or both. Therefore
> > calling the mapping interval tree a 'VMA' interval tree is rather
> > confusing.
> >
> > This is also inconsistent with the anon_vma_interval_tree_*() functions
> > which explicitly reference the rmap object to which they pertain.
> >
> > Rename the vma_interval_tree_*() functions to mapping_rmap_tree_*() to
> > correct this.
> >
> > We will rename the anon rmap functions similarly in a subsequent patch.
> >
> > No functional change intended.
> >
> > Reviewed-by: Gregory Price <gourry@gourry.net>
> > Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
> > ---
> > arch/arm/mm/fault-armv.c | 2 +-
> > arch/arm/mm/flush.c | 2 +-
> > arch/nios2/mm/cacheflush.c | 2 +-
> > arch/parisc/kernel/cache.c | 2 +-
> > fs/dax.c | 2 +-
> > fs/hugetlbfs/inode.c | 4 ++--
> > include/linux/mm.h | 16 ++++++++--------
> > kernel/events/uprobes.c | 2 +-
> > mm/hugetlb.c | 4 ++--
> > mm/interval_tree.c | 22 +++++++++++-----------
> > mm/khugepaged.c | 4 ++--
> > mm/memory-failure.c | 6 +++---
> > mm/memory.c | 2 +-
> > mm/mmap.c | 2 +-
> > mm/nommu.c | 8 ++++----
> > mm/pagewalk.c | 2 +-
> > mm/rmap.c | 2 +-
> > mm/vma.c | 12 ++++++------
> > tools/testing/vma/include/stubs.h | 4 ++--
> > 19 files changed, 50 insertions(+), 50 deletions(-)
> >
>
> Just curious, why not file_rmap_tree_*()?
Just to avoid any confusion around address_space objects that are ostensibly
'not files' like devices or whatnot, but moreso for consistency with the rest of
the kernel in referring to this stuff as 'mapping'.
Naming is hard IOW :)
>
> One nice part of this renaming is that the length of each function does
> not chagne. That makes the patch easy to review.
>
> Reviewed-by: Zi Yan <ziy@nvidia.com>
Thanks!
>
> --
> Best Regards,
> Yan, Zi
>
Cheers, Lorenzo
^ permalink raw reply
* Re: [PATCH v4 04/17] verification/rvgen: Use pathlib instead of os.path
From: Nam Cao @ 2026-07-20 7:48 UTC (permalink / raw)
To: Gabriele Monaco, linux-kernel, linux-trace-kernel, Steven Rostedt,
Gabriele Monaco
Cc: Thomas Weissschuh, Tomas Glozar, John Kacur, Wen Yang
In-Reply-To: <20260717154638.220789-5-gmonaco@redhat.com>
Gabriele Monaco <gmonaco@redhat.com> writes:
> Migrate to the newer patlib library, bundled with python since 3.4 to
> increase readability over using os.path.
>
> Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
...
> + Path(__file__).resolve().parent / "templates" / self.template_dir
That is an interesting use of the division operator.
Reviewed-by: Nam Cao <namcao@linutronix.de>
^ permalink raw reply
* Re: [PATCH v4 05/17] verification/rvgen: Improve consistency in template files
From: Nam Cao @ 2026-07-20 7:49 UTC (permalink / raw)
To: Gabriele Monaco, linux-kernel, linux-trace-kernel, Steven Rostedt,
Gabriele Monaco
Cc: Thomas Weissschuh, Tomas Glozar, John Kacur, Wen Yang
In-Reply-To: <20260717154638.220789-6-gmonaco@redhat.com>
Gabriele Monaco <gmonaco@redhat.com> writes:
> Template files for rvgen had minor inconsistencies in their
> placeholders for default author and default tracepoint examples. The
> user needs to modify those anyway but keeping consistency may help in
> bulk editing or checking.
>
> Change default author from "dot2k: auto-generated" (for DA/containers)
> or /* TODO */ (for LTL) to the general "rvgen: auto-generated".
>
> Align the sample tracepoint handler name in LTL template to
> handle_example_event, consistently with the rest of the file.
>
> Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Reviewed-by: Nam Cao <namcao@linutronix.de>
^ permalink raw reply
* Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
From: David Hildenbrand (Arm) @ 2026-07-20 8:13 UTC (permalink / raw)
To: Aditya Sharma, Andrew Morton, Lorenzo Stoakes, Liam R . Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko
Cc: David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda
In-Reply-To: <20260719185409.409685-1-adi.sharma@zohomail.in>
On 7/19/26 19:54, Aditya Sharma wrote:
> Address-space teardown on process exit runs synchronously in the dying
> task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks
> page tables, updates rmap, frees the RSS and drops file references, all
> on the exiting CPU. For a multi-GB process that is hundreds of
> milliseconds of exit-path latency, paid by whoever is waiting on the
> death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an
> orchestrator reaping a fleet of workers. On the test box below, killing
> a 16GB process costs ~280ms before the parent's waitpid() returns.
>
> This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path
> that defers __mmput() of large exiting processes to a dedicated kernel
> thread (mm_reaper), so the exiting CPU is released as soon as the task
> is reaped and the teardown runs off to the side.
QEMU has an rather short implementation for async teardown using
clone(CLONE_VM), which is IIRC essentially the result of Claudios previous
kernel work you note below.
So nothing got merged because the problem was solvable in userspace.
Without something like CONFIG_ASYNC_MM_TEARDOWN in the kernel.
[1] https://github.com/qemu/qemu/blob/master/system/async-teardown.c
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH] tracing: Fix context switch counter truncation
From: Breno Leitao @ 2026-07-20 9:24 UTC (permalink / raw)
To: Usama Arif
Cc: linux-kernel, linux-trace-kernel, mathieu.desnoyers, mhiramat,
rostedt, stable
In-Reply-To: <20260717173252.3431565-1-usama.arif@linux.dev>
On Fri, Jul 17, 2026 at 10:32:52AM -0700, Usama Arif wrote:
> This affects long-running systems and workloads with high context-switch
> rates. A CPU switching 1,000 times per second takes about 50 days.
nr_context_switches_cpu() returns "unsigned long long"
and rq->nr_switches is u64, so the unsigned int cnt truncation is real
> Fixes: 64cf7d058a00 ("tracing: Have trace_marker use per-cpu data to read user space")
> Cc: stable@vger.kernel.org
> Reported-by: Breno Leitao <leitao@debian.org>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Breno Leitao <leitao@debian.org>
Thanks for the investigation and fix,
--breno
^ permalink raw reply
* [PATCH 0/4] tracing/probes: Fixes several bugs
From: Masami Hiramatsu (Google) @ 2026-07-20 10:12 UTC (permalink / raw)
To: Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers
Cc: linux-kernel, linux-trace-kernel
Hi,
Here are a patches to fix some bugs in the probe events and eprobe.
These are found during cleaning up the code. I made these as a split
series because those could be backported. Let me send another
series for cleaning up probe event code.
Thank you,
---
Masami Hiramatsu (Google) (4):
tracing/probes: Avoid temporary buffer truncation in trace_probe_match_command_args()
tracing/probes: Prevent out-of-bounds write in __trace_probe_log_err()
tracing/probes: Fix potential underflow in LEN_OR_ZERO macro
tracing/eprobe: Fix exact system name matching in eprobe_dyn_event_match()
kernel/trace/trace_eprobe.c | 3 ++-
kernel/trace/trace_probe.c | 13 +++++++------
2 files changed, 9 insertions(+), 7 deletions(-)
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply
* [PATCH 1/4] tracing/probes: Avoid temporary buffer truncation in trace_probe_match_command_args()
From: Masami Hiramatsu (Google) @ 2026-07-20 10:12 UTC (permalink / raw)
To: Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers
Cc: linux-kernel, linux-trace-kernel
In-Reply-To: <178454232006.290363.11301475407231432622.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
In trace_probe_match_command_args(), a stack buffer buf[MAX_ARGSTR_LEN + 1]
(256 bytes) is used to format "<name>=<comm>". However, since name can
be up to 32 bytes (MAX_ARG_NAME_LEN) and comm up to 255 bytes
(MAX_ARGSTR_LEN), the formatted string can exceed 256 bytes and get
truncated by snprintf(), causing spurious argument matching failures.
Instead of formatting into a temporary buffer on stack, compare the
argument name, the '=' delimiter, and the comm expression directly.
Fixes: eb5bf81330a7 ("tracing/kprobe: Add per-probe delete from event")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
kernel/trace/trace_probe.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index d17cfee77d9c..95e3d072321f 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -2338,16 +2338,17 @@ int trace_probe_compare_arg_type(struct trace_probe *a, struct trace_probe *b)
bool trace_probe_match_command_args(struct trace_probe *tp,
int argc, const char **argv)
{
- char buf[MAX_ARGSTR_LEN + 1];
int i;
if (tp->nr_args < argc)
return false;
for (i = 0; i < argc; i++) {
- snprintf(buf, sizeof(buf), "%s=%s",
- tp->args[i].name, tp->args[i].comm);
- if (strcmp(buf, argv[i]))
+ int len = strlen(tp->args[i].name);
+
+ if (strncmp(argv[i], tp->args[i].name, len) ||
+ argv[i][len] != '=' ||
+ strcmp(argv[i] + len + 1, tp->args[i].comm))
return false;
}
return true;
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox