* Re: [RFC PATCH v2 18/28] mm/damon: trace probe_hits
From: Steven Rostedt @ 2026-05-14 0:32 UTC (permalink / raw)
To: SeongJae Park
Cc: Andrew Morton, Masami Hiramatsu, Mathieu Desnoyers, damon,
linux-kernel, linux-mm, linux-trace-kernel
In-Reply-To: <20260514000611.147809-1-sj@kernel.org>
On Wed, 13 May 2026 17:06:10 -0700
SeongJae Park <sj@kernel.org> wrote:
> Btw, if you don't mind, may I ask your opinion about the name having '_v2'
> suffix? I chose that as an RFC phase temporal name that doesn't break the
> compatibility, planning to give it a better name later. But I start feeling
> just extending the original one might be another option because tracepoints are
> not strict stable ABI to my understanding, and the change of the TP_prink
> format should be simple enough (append the probe_hits= part) that the user
> space could reasonably deal with.
It's only a stable ABI if some useful userspace tooling depends on it.
Otherwise, feel free to change.
Nothing really should be parsing the TP_printk() format part as it is
really inefficient to do so. That's why I created libtraceevent and
libtracefs to do the parsing of the raw data for you.
-- Steve
^ permalink raw reply
* Re: [RFC PATCH v2 18/28] mm/damon: trace probe_hits
From: SeongJae Park @ 2026-05-14 0:06 UTC (permalink / raw)
To: Steven Rostedt
Cc: SeongJae Park, Andrew Morton, Masami Hiramatsu, Mathieu Desnoyers,
damon, linux-kernel, linux-mm, linux-trace-kernel
In-Reply-To: <20260513140732.2320c563@gandalf.local.home>
On Wed, 13 May 2026 14:07:32 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:
> On Tue, 12 May 2026 07:36:33 -0700
> SeongJae Park <sj@kernel.org> wrote:
>
> > Introduce a new tracepoint for exposing the per-region per-probe
> > positive sample count via tracefs.
> >
> > Signed-off-by: SeongJae Park <sj@kernel.org>
> > ---
> > include/trace/events/damon.h | 36 ++++++++++++++++++++++++++++++++++++
> > mm/damon/core.c | 7 +++++++
> > 2 files changed, 43 insertions(+)
> >
> > diff --git a/include/trace/events/damon.h b/include/trace/events/damon.h
> > index 7e25f4469b81b..d7b94c7640217 100644
> > --- a/include/trace/events/damon.h
> > +++ b/include/trace/events/damon.h
> > @@ -130,6 +130,42 @@ TRACE_EVENT(damon_monitor_intervals_tune,
> > TP_printk("sample_us=%lu", __entry->sample_us)
> > );
> >
> > +TRACE_EVENT(damon_aggregated_v2,
> > +
> > + TP_PROTO(unsigned int target_id, struct damon_region *r,
> > + unsigned int nr_regions, unsigned int nr_probes),
> > +
> > + TP_ARGS(target_id, r, nr_regions, nr_probes),
> > +
> > + TP_STRUCT__entry(
> > + __field(unsigned long, target_id)
> > + __field(unsigned long, start)
> > + __field(unsigned long, end)
> > + __field(unsigned int, nr_regions)
> > + __field(unsigned int, nr_accesses)
> > + __field(unsigned int, age)
> > + __dynamic_array(unsigned char, probe_hits, nr_probes)
> > + ),
> > +
> > + TP_fast_assign(
> > + __entry->target_id = target_id;
> > + __entry->start = r->ar.start;
> > + __entry->end = r->ar.end;
> > + __entry->nr_regions = nr_regions;
> > + __entry->nr_accesses = r->nr_accesses;
> > + __entry->age = r->age;
> > + memcpy(__get_dynamic_array(probe_hits), r->probe_hits,
> > + sizeof(*r->probe_hits) * nr_probes);
> > + ),
> > +
> > + TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u %u probe_hits=%s",
> > + __entry->target_id, __entry->nr_regions,
> > + __entry->start, __entry->end,
> > + __entry->nr_accesses, __entry->age,
> > + __print_hex(__get_dynamic_array(probe_hits),
> > + __get_dynamic_array_len(probe_hits)))
> > +);
> > +
> > TRACE_EVENT(damon_aggregated,
> >
> > TP_PROTO(unsigned int target_id, struct damon_region *r,
> > diff --git a/mm/damon/core.c b/mm/damon/core.c
> > index fe6c789f2cecb..14b15c9876516 100644
> > --- a/mm/damon/core.c
> > +++ b/mm/damon/core.c
> > @@ -1905,6 +1905,11 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
> > {
> > struct damon_target *t;
> > unsigned int ti = 0; /* target's index */
> > + unsigned int nr_probes = 0;
> > + struct damon_probe *probe;
> > +
> > + damon_for_each_probe(probe, c)
> > + nr_probes++;
>
> Is the above logic needed when the tracepoint isn't enabled? If not, then you could add:
>
> if (trace_damon_aggregated_v2_enabled()) {
> damon_for_each_probe(probe, c)
> nr_probes++;
> }
>
> And change the tracepoint to be a conditional tracepoint:
>
> TRACE_EVENT_CONDITION(damon_aggregated_v2,
>
> TP_PROTO(..),
>
> TP_ARGS(..),
>
> TP_CONDITION(nr_probes > 0),
>
> [..]
>
> And then the tracepoint is only triggered if nr_probes is greater than zero
> (to handle races between the tracepoint being enabled in between the above
> check and where it triggers).
It is not needed when the tracepoint isn't enabled. I will follow your
suggestion in the next revision. Thank you for the nice suggestion, Steven!
Btw, if you don't mind, may I ask your opinion about the name having '_v2'
suffix? I chose that as an RFC phase temporal name that doesn't break the
compatibility, planning to give it a better name later. But I start feeling
just extending the original one might be another option because tracepoints are
not strict stable ABI to my understanding, and the change of the TP_prink
format should be simple enough (append the probe_hits= part) that the user
space could reasonably deal with.
Thanks,
SJ
[...]
^ permalink raw reply
* [PATCH v2] perf/ftrace: Fix WARNING in __unregister_ftrace_function
From: Rik van Riel @ 2026-05-13 20:19 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, kernel-team
perf_ftrace_function_unregister() unconditionally calls
unregister_ftrace_function() without checking whether the ftrace_ops
was ever successfully registered. This triggers a WARN_ON in
__unregister_ftrace_function() when the ops doesn't have
FTRACE_OPS_FL_ENABLED set.
This can happen during perf_event_alloc() error cleanup when
perf_trace_destroy() is called via __free_event() on an event whose
ftrace_ops registration failed or was already torn down by
perf_try_init_event()'s err_destroy path.
The call path is:
perf_event_alloc() error cleanup
-> __free_event()
-> event->destroy() [tp_perf_event_destroy]
-> perf_trace_destroy()
-> perf_trace_event_close()
-> TRACE_REG_PERF_CLOSE
-> perf_ftrace_function_unregister()
-> unregister_ftrace_function()
-> __unregister_ftrace_function()
-> WARN_ON(!(ops->flags & FTRACE_OPS_FL_ENABLED))
Fix this by checking FTRACE_OPS_FL_ENABLED before attempting to
unregister. If the ops is not enabled, just free the filter and
return success.
Assisted-by: Claude:claude-opus-4.7 syzkaller
Signed-off-by: Rik van Riel <riel@surriel.com>
---
kernel/trace/trace_event_perf.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index a6bb7577e8c5..58e1b427b576 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -497,7 +497,11 @@ static int perf_ftrace_function_register(struct perf_event *event)
static int perf_ftrace_function_unregister(struct perf_event *event)
{
struct ftrace_ops *ops = &event->ftrace_ops;
- int ret = unregister_ftrace_function(ops);
+ int ret = 0;
+
+ if (ops->flags & FTRACE_OPS_FL_ENABLED)
+ ret = unregister_ftrace_function(ops);
+
ftrace_free_filter(ops);
return ret;
}
--
2.52.0
^ permalink raw reply related
* Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages
From: David Hildenbrand (Arm) @ 2026-05-13 20:10 UTC (permalink / raw)
To: Breno Leitao, Miaohe Lin, Andrew Morton, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513-ecc_panic-v7-1-be2e578e61da@debian.org>
On 5/13/26 17:39, Breno Leitao wrote:
> The first entry of error_states[],
>
> { reserved, reserved, MF_MSG_KERNEL, me_kernel },
>
> is unreachable. identify_page_state() has two callers, and neither
> one can dispatch a PG_reserved page to me_kernel():
>
> * memory_failure() reaches identify_page_state() only after
> get_hwpoison_page() returned 1. get_any_page() reaches that
> return only via __get_hwpoison_page(), which gates the refcount
> on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved
> pages, so they fail with -EBUSY/-EIO long before
> identify_page_state() runs.
You should clarify why they are rejected. There is no explicit check for
PG_reserved in there!
>
> * try_memory_failure_hugetlb() reaches identify_page_state() on
> the MF_HUGETLB_IN_USED branch, but the page is necessarily a
> hugetlb folio there. The first table entry that matches a
> hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so
> they dispatch to me_huge_page() before the (now-removed)
> reserved entry would have matched, regardless of whether
> PG_reserved happens to be set on the head page.
See hugetlb_folio_init_vmemmap(): we always clear PG_reserved for hugetlb folios
allocated from memblock.
>
> me_kernel() never executes and the entry exists only to be matched
> against by code that cannot see it.
>
> Drop the entry, the me_kernel() helper, and the now-unused
> "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it
> remains part of the tracepoint and pr_err() string tables, and
> follow-on work to classify unrecoverable kernel pages can reuse it
> without churning the user-visible enum.
>
> No functional change.
>
> Suggested-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
> mm/memory-failure.c | 14 --------------
> 1 file changed, 14 deletions(-)
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 866c4428ac7ef..49bcfbd04d213 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -992,17 +992,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,
> return false;
> }
>
> -/*
> - * Error hit kernel page.
> - * Do nothing, try to be lucky and not touch this instead. For a few cases we
> - * could be more sophisticated.
> - */
> -static int me_kernel(struct page_state *ps, struct page *p)
> -{
> - unlock_page(p);
> - return MF_IGNORED;
> -}
> -
> /*
> * Page in unknown state. Do nothing.
> * This is a catch-all in case we fail to make sense of the page state.
> @@ -1211,10 +1200,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)
> #define mlock (1UL << PG_mlocked)
> #define lru (1UL << PG_lru)
> #define head (1UL << PG_head)
> -#define reserved (1UL << PG_reserved)
>
> static struct page_state error_states[] = {
> - { reserved, reserved, MF_MSG_KERNEL, me_kernel },
> /*
> * free pages are specially detected outside this table:
> * PG_buddy pages only make a small fraction of all free pages.
> @@ -1246,7 +1233,6 @@ static struct page_state error_states[] = {
> #undef mlock
> #undef lru
> #undef head
> -#undef reserved
>
> static void update_per_node_mf_stats(unsigned long pfn,
> enum mf_result result)
>
Yes, I think this should work.
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()
From: David Hildenbrand (Arm) @ 2026-05-13 19:49 UTC (permalink / raw)
To: Breno Leitao, Miaohe Lin, Andrew Morton, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest,
linux-trace-kernel, kernel-team, Lance Yang
In-Reply-To: <20260513-ecc_panic-v7-4-be2e578e61da@debian.org>
On 5/13/26 17:39, Breno Leitao wrote:
> The previous patch already classifies PG_reserved pages as
> MF_MSG_KERNEL through the long path: get_hwpoison_page() calls
> __get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page()
> exhausts its shake_page() retry budget, and the resulting
> -ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The
> outcome is correct but the work in between is wasted: shake_page()
> cannot turn a reserved page into a handlable one.
If really required, can we just move the check right there, into get_any_page() etc?
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH v6 5/7] locking: Add contended_release tracepoint to qspinlock
From: Peter Zijlstra @ 2026-05-13 19:33 UTC (permalink / raw)
To: Dmitry Ilvokhin
Cc: Ingo Molnar, Will Deacon, Boqun Feng, Waiman Long,
Thomas Bogendoerfer, Juergen Gross, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Thomas Gleixner,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Arnd Bergmann,
Dennis Zhou, Tejun Heo, Christoph Lameter, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, linux-kernel, linux-mips,
virtualization, linux-arch, linux-mm, linux-trace-kernel,
kernel-team, Paul E. McKenney
In-Reply-To: <5d7ea75ffe74a785e6b234ada9f23c6373d4b4c1.1777999826.git.d@ilvokhin.com>
On Tue, May 05, 2026 at 05:09:34PM +0000, Dmitry Ilvokhin wrote:
> Use the arch-overridable queued_spin_release(), introduced in the
> previous commit, to ensure the tracepoint works correctly across all
> architectures, including those with custom unlock implementations (e.g.
> x86 paravirt).
>
> When the tracepoint is disabled, the only addition to the hot path is a
> single NOP instruction (the static branch). When enabled, the contention
> check, trace call, and unlock are combined in an out-of-line function to
> minimize hot path impact, avoiding the compiler needing to preserve the
> lock pointer in a callee-saved register across the trace call.
>
> Binary size impact (x86_64, defconfig):
> uninlined unlock (common case): +680 bytes (+0.00%)
> inlined unlock (worst case): +83659 bytes (+0.21%)
>
> The inlined unlock case could not be achieved through Kconfig options on
> x86_64 as PREEMPT_BUILD unconditionally selects UNINLINE_SPIN_UNLOCK on
> x86_64. The UNINLINE_SPIN_UNLOCK guards were manually inverted to force
> inline the unlock path and estimate the worst case binary size increase.
>
> In practice, configurations with UNINLINE_SPIN_UNLOCK=n have already
> opted against binary size optimization, so the inlined worst case is
> unlikely to be a concern.
This is not quite accurate. You add the (5byte) NOP for the static
branch, but then you also add another 5 bytes for the CALL and at least
another 2 bytes (possibly 5) for a JMP back into the previous stream.
That is 12-15 bytes added to what was a single MOV instruction.
That is quite ludicrous.
I disagree that UNINLINE_SPIN_UNLOCK=n opts against binary size. For x86
the unlock is smaller than a function call.
I really don't see how this is worth it.
^ permalink raw reply
* Re: [PATCH v6 0/7] locking: contended_release tracepoint instrumentation
From: Peter Zijlstra @ 2026-05-13 19:26 UTC (permalink / raw)
To: Dmitry Ilvokhin
Cc: Ingo Molnar, Will Deacon, Boqun Feng, Waiman Long,
Thomas Bogendoerfer, Juergen Gross, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Thomas Gleixner,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Arnd Bergmann,
Dennis Zhou, Tejun Heo, Christoph Lameter, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, linux-kernel, linux-mips,
virtualization, linux-arch, linux-mm, linux-trace-kernel,
kernel-team
In-Reply-To: <cover.1777999826.git.d@ilvokhin.com>
On Tue, May 05, 2026 at 05:09:29PM +0000, Dmitry Ilvokhin wrote:
> This series adds a contended_release tracepoint that fires on the
> holder side when a lock with waiters is released. This provides:
>
> - Hold time estimation: when the holder's own acquisition was
> contended, its contention_end (acquisition) and contended_release
> can be correlated to measure how long the lock was held under
> contention.
>
> - The holder's stack at release time, which may differ from what perf lock
> contention --lock-owner captures if the holder does significant work between
> the waiter's arrival and the unlock.
>
> Note: for reader/writer locks, the tracepoint fires for every reader
> releasing while a writer is waiting, not only for the last reader.
And for qspinlock.
I am really not sure this is worth the overhead for qspinlock / rwlock.
^ permalink raw reply
* [PATCH v2] tracing: Allow perf to read synthetic events
From: Steven Rostedt @ 2026-05-13 19:00 UTC (permalink / raw)
To: LKML, Linux Trace Kernel
Cc: Masami Hiramatsu, Mathieu Desnoyers, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim, Peter Zijlstra, Ian Rogers
From: Steven Rostedt <rostedt@goodmis.org>
Currently, perf can not enable synthetic events. When it does, it either
causes a warning in the kernel or errors with "no such device".
Add the necessary code to allow perf to also attach to synthetic events.
Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
Changes since v1: https://patch.msgid.link/20251217113920.50b56246@gandalf.local.home
- Forward ported to v7.1-rc2
kernel/trace/trace_events_synth.c | 121 +++++++++++++++++++++++-------
1 file changed, 94 insertions(+), 27 deletions(-)
diff --git a/kernel/trace/trace_events_synth.c b/kernel/trace/trace_events_synth.c
index 39ac4eba0702..e6871230bde9 100644
--- a/kernel/trace/trace_events_synth.c
+++ b/kernel/trace/trace_events_synth.c
@@ -499,28 +499,19 @@ static unsigned int trace_stack(struct synth_trace_event *entry,
return len;
}
-static void trace_event_raw_event_synth(void *__data,
- u64 *var_ref_vals,
- unsigned int *var_ref_idx)
+static __always_inline int get_field_size(struct synth_event *event,
+ u64 *var_ref_vals,
+ unsigned int *var_ref_idx)
{
- unsigned int i, n_u64, val_idx, len, data_size = 0;
- struct trace_event_file *trace_file = __data;
- struct synth_trace_event *entry;
- struct trace_event_buffer fbuffer;
- struct trace_buffer *buffer;
- struct synth_event *event;
- int fields_size = 0;
-
- event = trace_file->event_call->data;
-
- if (trace_trigger_soft_disabled(trace_file))
- return;
+ int fields_size;
fields_size = event->n_u64 * sizeof(u64);
- for (i = 0; i < event->n_dynamic_fields; i++) {
+ for (int i = 0; i < event->n_dynamic_fields; i++) {
unsigned int field_pos = event->dynamic_fields[i]->field_pos;
char *str_val;
+ int val_idx;
+ int len;
val_idx = var_ref_idx[field_pos];
str_val = (char *)(long)var_ref_vals[val_idx];
@@ -535,18 +526,18 @@ static void trace_event_raw_event_synth(void *__data,
fields_size += len;
}
+ return fields_size;
+}
- /*
- * Avoid ring buffer recursion detection, as this event
- * is being performed within another event.
- */
- buffer = trace_file->tr->array_buffer.buffer;
- guard(ring_buffer_nest)(buffer);
-
- entry = trace_event_buffer_reserve(&fbuffer, trace_file,
- sizeof(*entry) + fields_size);
- if (!entry)
- return;
+static __always_inline void write_synth_entry(struct synth_event *event,
+ struct synth_trace_event *entry,
+ u64 *var_ref_vals,
+ unsigned int *var_ref_idx)
+{
+ int data_size = 0;
+ int i, n_u64;
+ int val_idx;
+ int len;
for (i = 0, n_u64 = 0; i < event->n_fields; i++) {
val_idx = var_ref_idx[i];
@@ -587,10 +578,83 @@ static void trace_event_raw_event_synth(void *__data,
n_u64++;
}
}
+}
+
+static void trace_event_raw_event_synth(void *__data,
+ u64 *var_ref_vals,
+ unsigned int *var_ref_idx)
+{
+ struct trace_event_file *trace_file = __data;
+ struct synth_trace_event *entry;
+ struct trace_event_buffer fbuffer;
+ struct trace_buffer *buffer;
+ struct synth_event *event;
+ int fields_size;
+
+ event = trace_file->event_call->data;
+
+ if (trace_trigger_soft_disabled(trace_file))
+ return;
+
+ fields_size = get_field_size(event, var_ref_vals, var_ref_idx);
+
+ /*
+ * Avoid ring buffer recursion detection, as this event
+ * is being performed within another event.
+ */
+ buffer = trace_file->tr->array_buffer.buffer;
+ guard(ring_buffer_nest)(buffer);
+
+ entry = trace_event_buffer_reserve(&fbuffer, trace_file,
+ sizeof(*entry) + fields_size);
+ if (!entry)
+ return;
+
+ write_synth_entry(event, entry, var_ref_vals, var_ref_idx);
trace_event_buffer_commit(&fbuffer);
}
+#ifdef CONFIG_PERF_EVENTS
+static void perf_event_raw_event_synth(void *__data,
+ u64 *var_ref_vals,
+ unsigned int *var_ref_idx)
+{
+ struct trace_event_call *call = __data;
+ struct synth_trace_event *entry;
+ struct hlist_head *perf_head;
+ struct synth_event *event;
+ struct pt_regs *regs;
+ int fields_size;
+ size_t size;
+ int context;
+
+ event = call->data;
+
+ perf_head = this_cpu_ptr(call->perf_events);
+
+ if (!perf_head || hlist_empty(perf_head))
+ return;
+
+ fields_size = get_field_size(event, var_ref_vals, var_ref_idx);
+
+ size = ALIGN(sizeof(*entry) + fields_size, 8);
+
+ entry = perf_trace_buf_alloc(size, ®s, &context);
+
+ if (unlikely(!entry))
+ return;
+
+ write_synth_entry(event, entry, var_ref_vals, var_ref_idx);
+
+ perf_fetch_caller_regs(regs);
+
+ perf_trace_buf_submit(entry, size, context,
+ call->event.type, 1, regs,
+ perf_head, NULL);
+}
+#endif
+
static void free_synth_event_print_fmt(struct trace_event_call *call)
{
if (call) {
@@ -917,6 +981,9 @@ static int register_synth_event(struct synth_event *event)
call->flags = TRACE_EVENT_FL_TRACEPOINT;
call->class->reg = synth_event_reg;
call->class->probe = trace_event_raw_event_synth;
+#ifdef CONFIG_PERF_EVENTS
+ call->class->perf_probe = perf_event_raw_event_synth;
+#endif
call->data = event;
call->tp = event->tp;
--
2.53.0
^ permalink raw reply related
* Re: [RFC PATCH v2 18/28] mm/damon: trace probe_hits
From: Steven Rostedt @ 2026-05-13 18:07 UTC (permalink / raw)
To: SeongJae Park
Cc: Andrew Morton, Masami Hiramatsu, Mathieu Desnoyers, damon,
linux-kernel, linux-mm, linux-trace-kernel
In-Reply-To: <20260512143645.113201-19-sj@kernel.org>
On Tue, 12 May 2026 07:36:33 -0700
SeongJae Park <sj@kernel.org> wrote:
> Introduce a new tracepoint for exposing the per-region per-probe
> positive sample count via tracefs.
>
> Signed-off-by: SeongJae Park <sj@kernel.org>
> ---
> include/trace/events/damon.h | 36 ++++++++++++++++++++++++++++++++++++
> mm/damon/core.c | 7 +++++++
> 2 files changed, 43 insertions(+)
>
> diff --git a/include/trace/events/damon.h b/include/trace/events/damon.h
> index 7e25f4469b81b..d7b94c7640217 100644
> --- a/include/trace/events/damon.h
> +++ b/include/trace/events/damon.h
> @@ -130,6 +130,42 @@ TRACE_EVENT(damon_monitor_intervals_tune,
> TP_printk("sample_us=%lu", __entry->sample_us)
> );
>
> +TRACE_EVENT(damon_aggregated_v2,
> +
> + TP_PROTO(unsigned int target_id, struct damon_region *r,
> + unsigned int nr_regions, unsigned int nr_probes),
> +
> + TP_ARGS(target_id, r, nr_regions, nr_probes),
> +
> + TP_STRUCT__entry(
> + __field(unsigned long, target_id)
> + __field(unsigned long, start)
> + __field(unsigned long, end)
> + __field(unsigned int, nr_regions)
> + __field(unsigned int, nr_accesses)
> + __field(unsigned int, age)
> + __dynamic_array(unsigned char, probe_hits, nr_probes)
> + ),
> +
> + TP_fast_assign(
> + __entry->target_id = target_id;
> + __entry->start = r->ar.start;
> + __entry->end = r->ar.end;
> + __entry->nr_regions = nr_regions;
> + __entry->nr_accesses = r->nr_accesses;
> + __entry->age = r->age;
> + memcpy(__get_dynamic_array(probe_hits), r->probe_hits,
> + sizeof(*r->probe_hits) * nr_probes);
> + ),
> +
> + TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u %u probe_hits=%s",
> + __entry->target_id, __entry->nr_regions,
> + __entry->start, __entry->end,
> + __entry->nr_accesses, __entry->age,
> + __print_hex(__get_dynamic_array(probe_hits),
> + __get_dynamic_array_len(probe_hits)))
> +);
> +
> TRACE_EVENT(damon_aggregated,
>
> TP_PROTO(unsigned int target_id, struct damon_region *r,
> diff --git a/mm/damon/core.c b/mm/damon/core.c
> index fe6c789f2cecb..14b15c9876516 100644
> --- a/mm/damon/core.c
> +++ b/mm/damon/core.c
> @@ -1905,6 +1905,11 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
> {
> struct damon_target *t;
> unsigned int ti = 0; /* target's index */
> + unsigned int nr_probes = 0;
> + struct damon_probe *probe;
> +
> + damon_for_each_probe(probe, c)
> + nr_probes++;
Is the above logic needed when the tracepoint isn't enabled? If not, then you could add:
if (trace_damon_aggregated_v2_enabled()) {
damon_for_each_probe(probe, c)
nr_probes++;
}
And change the tracepoint to be a conditional tracepoint:
TRACE_EVENT_CONDITION(damon_aggregated_v2,
TP_PROTO(..),
TP_ARGS(..),
TP_CONDITION(nr_probes > 0),
[..]
And then the tracepoint is only triggered if nr_probes is greater than zero
(to handle races between the tracepoint being enabled in between the above
check and where it triggers).
-- Steve
>
> damon_for_each_target(t, c) {
> struct damon_region *r;
> @@ -1913,6 +1918,8 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
> int i;
>
> trace_damon_aggregated(ti, r, damon_nr_regions(t));
> + trace_damon_aggregated_v2(ti, r, damon_nr_regions(t),
> + nr_probes);
> damon_warn_fix_nr_accesses_corruption(r);
> r->last_nr_accesses = r->nr_accesses;
> r->nr_accesses = 0;
^ permalink raw reply
* Re: [PATCH v2] perf/ftrace: Fix WARNING in __unregister_ftrace_function
From: Steven Rostedt @ 2026-05-13 18:11 UTC (permalink / raw)
To: Rik van Riel
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513132445.24d8d9f6@fangorn>
On Wed, 13 May 2026 13:24:45 -0400
Rik van Riel <riel@surriel.com> wrote:
> From 9de86227b917c49315b7b67aac3a83afae8d792d Mon Sep 17 00:00:00 2001
> From: Rik van Riel <riel@meta.com>
> Date: Sat, 25 Apr 2026 03:33:54 -0700
> Subject: [PATCH] perf/ftrace: Fix WARNING in __unregister_ftrace_function
>
Can you resend this as a normal patch so that it can be picked up by patchwork.
Otherwise it will be ignored.
Thanks,
-- Steve
^ permalink raw reply
* Re: [RFC v7 6/7] ext4: fast commit: add lock_updates tracepoint
From: Steven Rostedt @ 2026-05-13 17:57 UTC (permalink / raw)
To: Li Chen
Cc: Zhang Yi, Theodore Ts'o, Andreas Dilger, Baokun Li, Jan Kara,
Ojaswin Mujoo, Ritesh Harjani (IBM), Zhang Yi, Masami Hiramatsu,
Mathieu Desnoyers, linux-ext4, linux-kernel, linux-trace-kernel
In-Reply-To: <20260511084304.1559557-7-me@linux.beauty>
On Mon, 11 May 2026 16:43:01 +0800
Li Chen <me@linux.beauty> wrote:
> @@ -1346,8 +1383,15 @@ static int ext4_fc_perform_commit(journal_t *journal)
> }
> ext4_fc_unlock(sb, alloc_ctx);
>
> - ret = ext4_fc_snapshot_inodes(journal, inodes, inodes_size);
> + ret = ext4_fc_snapshot_inodes(journal, inodes, inodes_size,
> + &snap_inodes, &snap_ranges, &snap_err);
> jbd2_journal_unlock_updates(journal);
> + if (trace_ext4_fc_lock_updates_enabled()) {
> + locked_ns = ktime_to_ns(ktime_sub(ktime_get(), lock_start));
> + trace_ext4_fc_lock_updates(sb, commit_tid, locked_ns,
> + snap_inodes, snap_ranges, ret,
> + snap_err);
Please change this to:
trace_call__ext4_fc_lock_updates(...)
As the "trace_ext4_fc_lock_updates_enabled()" already has the static
branch. No need to do it twice anymore. 7.1 introduced the
"trace_call__foo()" that will do a direct call to the tracepoints
registered, without the need for another static branch.
-- Steve
> + }
^ permalink raw reply
* Re: [PATCH v2] perf/ftrace: Fix WARNING in __unregister_ftrace_function
From: Rik van Riel @ 2026-05-13 17:24 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513123344.05b6bcfe@gandalf.local.home>
On Wed, 13 May 2026 12:33:44 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
>
> Instead of duplicating code, what about doing:
That is much nicer. Thank you!
---8<---
From 9de86227b917c49315b7b67aac3a83afae8d792d Mon Sep 17 00:00:00 2001
From: Rik van Riel <riel@meta.com>
Date: Sat, 25 Apr 2026 03:33:54 -0700
Subject: [PATCH] perf/ftrace: Fix WARNING in __unregister_ftrace_function
perf_ftrace_function_unregister() unconditionally calls
unregister_ftrace_function() without checking whether the ftrace_ops
was ever successfully registered. This triggers a WARN_ON in
__unregister_ftrace_function() when the ops doesn't have
FTRACE_OPS_FL_ENABLED set.
This can happen during perf_event_alloc() error cleanup when
perf_trace_destroy() is called via __free_event() on an event whose
ftrace_ops registration failed or was already torn down by
perf_try_init_event()'s err_destroy path.
The call path is:
perf_event_alloc() error cleanup
-> __free_event()
-> event->destroy() [tp_perf_event_destroy]
-> perf_trace_destroy()
-> perf_trace_event_close()
-> TRACE_REG_PERF_CLOSE
-> perf_ftrace_function_unregister()
-> unregister_ftrace_function()
-> __unregister_ftrace_function()
-> WARN_ON(!(ops->flags & FTRACE_OPS_FL_ENABLED))
Fix this by checking FTRACE_OPS_FL_ENABLED before attempting to
unregister. If the ops is not enabled, just free the filter and
return success.
Assisted-by: Claude:claude-opus-4.7 syzkaller
Signed-off-by: Rik van Riel <riel@surriel.com>
---
kernel/trace/trace_event_perf.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index a6bb7577e8c5..58e1b427b576 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -497,7 +497,11 @@ static int perf_ftrace_function_register(struct perf_event *event)
static int perf_ftrace_function_unregister(struct perf_event *event)
{
struct ftrace_ops *ops = &event->ftrace_ops;
- int ret = unregister_ftrace_function(ops);
+ int ret = 0;
+
+ if (ops->flags & FTRACE_OPS_FL_ENABLED)
+ ret = unregister_ftrace_function(ops);
+
ftrace_free_filter(ops);
return ret;
}
--
2.52.0
^ permalink raw reply related
* Re: [RFC PATCH v3] bpf: introduce TAINT_UNSAFE_BPF for mutating helpers
From: Steven Rostedt @ 2026-05-13 16:41 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Aaron Tomlin, Jonathan Corbet, Song Liu, KP Singh, Matt Bobrowski,
Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Eduard,
Kumar Kartikeya Dwivedi, Masami Hiramatsu, Shuah Khan, Jiri Olsa,
Martin KaFai Lau, Yonghong Song, Mathieu Desnoyers, Randy Dunlap,
neelx, sean, chjohnst, steve, mproche, nick.lange,
open list:DOCUMENTATION, LKML, bpf, linux-trace-kernel
In-Reply-To: <CAADnVQLw+_NaOVeaKabuf085wNo_-6MAv8w0EDO3fBz3KCQT5g@mail.gmail.com>
On Wed, 13 May 2026 09:35:29 -0700
Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> On Wed, May 13, 2026 at 8:23 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > On Wed, 13 May 2026 08:16:07 -0700
> > Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> >
> > > It's impossible to track all modifications.
> > > See what sched-ext is doing.
> > > What does it modify? Everything.
> >
> > What about just having a list of what BPF programs are loaded, what they
> > may be attached to, and what kfuncs they are calling?
>
> Ohh. These have been available forever.
> Just bpftool prog, bpftool link, bpftool prog dump xlated
Ah thanks. That is useful.
-- Steve
^ permalink raw reply
* Re: [RFC PATCH] trace: Introduce a new filter_pred "caller"
From: Steven Rostedt @ 2026-05-13 16:40 UTC (permalink / raw)
To: Masami Hiramatsu (Google)
Cc: Chen Jun, mathieu.desnoyers, linux-kernel, linux-trace-kernel
In-Reply-To: <20260512084750.c17a93d0ccdacddfd52d3d40@kernel.org>
On Tue, 12 May 2026 08:47:50 +0900
Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
> On Fri, 8 May 2026 20:26:23 +0800
> Chen Jun <chenjun102@huawei.com> wrote:
>
> > Low-level functions have many call paths, and sometimes
> > we only care about the calls on a specific call path.
> > Add a new filter to filter based on the call stack.
> >
> > Usage:
> > 1. echo 'caller=="$function_name"' > events/../filter
>
> Thanks for interesting idea :)
>
> BTW, we already have "stacktrace". Since this actually checks
> stacktrace, not caller, so I think we should reuse it.
> Also, I think OP_GLOB is more suitable for this case.
> (and more useful)
Actually, it's not a stack trace, it's a function that is called from other
functions. But since "caller" sounds like a direct called function (stack
trace of the first instance), I think perhaps it should be "called_within" or
something similar. :-/
Also, OP_GLOB can't work because it only works for a single function. At
the time of parsing, it finds the function (and should probably error out
if there's more than one function with a given name). It then records the
start and end address of the function so it only needs to find if one of
the entries in the stack trace is between the start and end of the function.
I don't think this is possible with GLOB. We don't want to do a search of
the functions when the event is triggered.
-- Steve
^ permalink raw reply
* Re: [RFC PATCH v3] bpf: introduce TAINT_UNSAFE_BPF for mutating helpers
From: Alexei Starovoitov @ 2026-05-13 16:35 UTC (permalink / raw)
To: Steven Rostedt
Cc: Aaron Tomlin, Jonathan Corbet, Song Liu, KP Singh, Matt Bobrowski,
Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Eduard,
Kumar Kartikeya Dwivedi, Masami Hiramatsu, Shuah Khan, Jiri Olsa,
Martin KaFai Lau, Yonghong Song, Mathieu Desnoyers, Randy Dunlap,
neelx, sean, chjohnst, steve, mproche, nick.lange,
open list:DOCUMENTATION, LKML, bpf, linux-trace-kernel
In-Reply-To: <20260513112307.53e77312@gandalf.local.home>
On Wed, May 13, 2026 at 8:23 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Wed, 13 May 2026 08:16:07 -0700
> Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
>
> > It's impossible to track all modifications.
> > See what sched-ext is doing.
> > What does it modify? Everything.
>
> What about just having a list of what BPF programs are loaded, what they
> may be attached to, and what kfuncs they are calling?
Ohh. These have been available forever.
Just bpftool prog, bpftool link, bpftool prog dump xlated
^ permalink raw reply
* Re: [PATCH] perf/ftrace: Fix WARNING in __unregister_ftrace_function
From: Steven Rostedt @ 2026-05-13 16:33 UTC (permalink / raw)
To: Rik van Riel
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513121607.14eab1f6@fangorn>
On Wed, 13 May 2026 12:16:07 -0400
Rik van Riel <riel@surriel.com> wrote:
> diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
> index a6bb7577e8c5..b9e33ae24867 100644
> --- a/kernel/trace/trace_event_perf.c
> +++ b/kernel/trace/trace_event_perf.c
> @@ -497,7 +497,14 @@ static int perf_ftrace_function_register(struct perf_event *event)
> static int perf_ftrace_function_unregister(struct perf_event *event)
> {
> struct ftrace_ops *ops = &event->ftrace_ops;
> - int ret = unregister_ftrace_function(ops);
> + int ret;
> +
> + if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) {
> + ftrace_free_filter(ops);
> + return 0;
> + }
> +
> + ret = unregister_ftrace_function(ops);
> ftrace_free_filter(ops);
> return ret;
> }
> --
Instead of duplicating code, what about doing:
static int perf_ftrace_function_unregister(struct perf_event *event)
{
struct ftrace_ops *ops = &event->ftrace_ops;
int ret = 0;
if (ops->flags & FTRACE_OPS_FL_ENABLED)
ret = unregister_ftrace_function(ops);
ftrace_free_filter(ops);
return ret;
}
?
-- Steve
^ permalink raw reply
* [PATCH] perf/ftrace: Fix WARNING in __unregister_ftrace_function
From: Rik van Riel @ 2026-05-13 16:16 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, kernel-team
perf_ftrace_function_unregister() unconditionally calls
unregister_ftrace_function() without checking whether the ftrace_ops
was ever successfully registered. This triggers a WARN_ON in
__unregister_ftrace_function() when the ops doesn't have
FTRACE_OPS_FL_ENABLED set.
This can happen during perf_event_alloc() error cleanup when
perf_trace_destroy() is called via __free_event() on an event whose
ftrace_ops registration failed or was already torn down by
perf_try_init_event()'s err_destroy path.
The call path is:
perf_event_alloc() error cleanup
-> __free_event()
-> event->destroy() [tp_perf_event_destroy]
-> perf_trace_destroy()
-> perf_trace_event_close()
-> TRACE_REG_PERF_CLOSE
-> perf_ftrace_function_unregister()
-> unregister_ftrace_function()
-> __unregister_ftrace_function()
-> WARN_ON(!(ops->flags & FTRACE_OPS_FL_ENABLED))
Fix this by checking FTRACE_OPS_FL_ENABLED before attempting to
unregister. If the ops is not enabled, just free the filter and
return success.
Assisted-by: Claude:claude-opus-4.7 syzkaller
Signed-off-by: Rik van Riel <riel@surriel.com>
---
kernel/trace/trace_event_perf.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index a6bb7577e8c5..b9e33ae24867 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -497,7 +497,14 @@ static int perf_ftrace_function_register(struct perf_event *event)
static int perf_ftrace_function_unregister(struct perf_event *event)
{
struct ftrace_ops *ops = &event->ftrace_ops;
- int ret = unregister_ftrace_function(ops);
+ int ret;
+
+ if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) {
+ ftrace_free_filter(ops);
+ return 0;
+ }
+
+ ret = unregister_ftrace_function(ops);
ftrace_free_filter(ops);
return ret;
}
--
2.53.0-Meta
^ permalink raw reply related
* Re: [PATCH v6 7/7] locking: Add contended_release tracepoint to qrwlock
From: Steven Rostedt @ 2026-05-13 15:43 UTC (permalink / raw)
To: Dmitry Ilvokhin
Cc: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng, Waiman Long,
Thomas Bogendoerfer, Juergen Gross, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Thomas Gleixner,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Arnd Bergmann,
Dennis Zhou, Tejun Heo, Christoph Lameter, Masami Hiramatsu,
Mathieu Desnoyers, linux-kernel, linux-mips, virtualization,
linux-arch, linux-mm, linux-trace-kernel, kernel-team
In-Reply-To: <b67fda8e847fff72da05eff7f799019f8d17ce21.1777999826.git.d@ilvokhin.com>
On Tue, 5 May 2026 17:09:36 +0000
Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> Extend the contended_release tracepoint to queued rwlocks, using the
> same out-of-line traced unlock approach as queued spinlocks.
>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> ---
> include/asm-generic/qrwlock.h | 22 ++++++++++++++++++++++
> kernel/locking/qrwlock.c | 16 ++++++++++++++++
> 2 files changed, 38 insertions(+)
>
> diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h
> index 4b627bafba8b..274c19006125 100644
> --- a/include/asm-generic/qrwlock.h
> +++ b/include/asm-generic/qrwlock.h
> @@ -14,6 +14,7 @@
> #define __ASM_GENERIC_QRWLOCK_H
>
> #include <linux/atomic.h>
> +#include <linux/tracepoint-defs.h>
> #include <asm/barrier.h>
> #include <asm/processor.h>
>
> @@ -35,6 +36,10 @@
> */
> extern void queued_read_lock_slowpath(struct qrwlock *lock);
> extern void queued_write_lock_slowpath(struct qrwlock *lock);
> +extern void queued_read_unlock_traced(struct qrwlock *lock);
> +extern void queued_write_unlock_traced(struct qrwlock *lock);
> +
> +DECLARE_TRACEPOINT(contended_release);
>
> /**
> * queued_read_trylock - try to acquire read lock of a queued rwlock
> @@ -115,6 +120,17 @@ static __always_inline void __queued_read_unlock(struct qrwlock *lock)
> */
> static inline void queued_read_unlock(struct qrwlock *lock)
> {
> + /*
> + * Trace and unlock are combined in the traced unlock variant so
> + * the compiler does not need to preserve the lock pointer across
> + * the function call, avoiding callee-saved register save/restore
> + * on the hot path.
> + */
> + if (tracepoint_enabled(contended_release)) {
> + queued_read_unlock_traced(lock);
Same issue here about duplicating the code.
> + return;
> + }
> +
> __queued_read_unlock(lock);
> }
>
> @@ -129,6 +145,12 @@ static __always_inline void __queued_write_unlock(struct qrwlock *lock)
> */
> static inline void queued_write_unlock(struct qrwlock *lock)
> {
> + /* See comment in queued_read_unlock(). */
> + if (tracepoint_enabled(contended_release)) {
> + queued_write_unlock_traced(lock);
And here.
> + return;
> + }
> +
> __queued_write_unlock(lock);
> }
>
> diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
> index d2ef312a8611..5ae4b0372719 100644
> --- a/kernel/locking/qrwlock.c
> +++ b/kernel/locking/qrwlock.c
> @@ -90,3 +90,19 @@ void __lockfunc queued_write_lock_slowpath(struct qrwlock *lock)
> trace_contention_end(lock, 0);
> }
> EXPORT_SYMBOL(queued_write_lock_slowpath);
> +
> +void __lockfunc queued_read_unlock_traced(struct qrwlock *lock)
> +{
> + if (queued_rwlock_is_contended(lock))
> + trace_call__contended_release(lock);
Just have this trace and not actually do any locking.
> + __queued_read_unlock(lock);
> +}
> +EXPORT_SYMBOL(queued_read_unlock_traced);
> +
> +void __lockfunc queued_write_unlock_traced(struct qrwlock *lock)
> +{
> + if (queued_rwlock_is_contended(lock))
> + trace_call__contended_release(lock);
Ditto.
-- Steve
> + __queued_write_unlock(lock);
> +}
> +EXPORT_SYMBOL(queued_write_unlock_traced);
^ permalink raw reply
* Re: [PATCH v6 6/7] locking: Factor out __queued_read_unlock()/__queued_write_unlock()
From: Steven Rostedt @ 2026-05-13 15:41 UTC (permalink / raw)
To: Dmitry Ilvokhin
Cc: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng, Waiman Long,
Thomas Bogendoerfer, Juergen Gross, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Thomas Gleixner,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Arnd Bergmann,
Dennis Zhou, Tejun Heo, Christoph Lameter, Masami Hiramatsu,
Mathieu Desnoyers, linux-kernel, linux-mips, virtualization,
linux-arch, linux-mm, linux-trace-kernel, kernel-team,
Paul E. McKenney
In-Reply-To: <8e88613c73f0603c4440ba3a62eb604a5dddc57b.1777999826.git.d@ilvokhin.com>
On Tue, 5 May 2026 17:09:35 +0000
Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> This is a preparatory refactoring for the next commit, which adds
Same thing about using "next commit" in change logs.
-- Steve
> contended_release tracepoint instrumentation and needs to call the
> unlock from both traced and non-traced paths.
>
> No functional change.
>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> Acked-by: Paul E. McKenney <paulmck@kernel.org>
> ---
> include/asm-generic/qrwlock.h | 20 +++++++++++++++-----
> 1 file changed, 15 insertions(+), 5 deletions(-)
^ permalink raw reply
* Re: [PATCH v6 5/7] locking: Add contended_release tracepoint to qspinlock
From: Steven Rostedt @ 2026-05-13 15:41 UTC (permalink / raw)
To: Dmitry Ilvokhin
Cc: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng, Waiman Long,
Thomas Bogendoerfer, Juergen Gross, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Thomas Gleixner,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Arnd Bergmann,
Dennis Zhou, Tejun Heo, Christoph Lameter, Masami Hiramatsu,
Mathieu Desnoyers, linux-kernel, linux-mips, virtualization,
linux-arch, linux-mm, linux-trace-kernel, kernel-team,
Paul E. McKenney
In-Reply-To: <5d7ea75ffe74a785e6b234ada9f23c6373d4b4c1.1777999826.git.d@ilvokhin.com>
On Tue, 5 May 2026 17:09:34 +0000
Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> Use the arch-overridable queued_spin_release(), introduced in the
> previous commit, to ensure the tracepoint works correctly across all
Remove the ", introduced in the previous commit," That's useless in git
change logs.
> architectures, including those with custom unlock implementations (e.g.
> x86 paravirt).
>
> When the tracepoint is disabled, the only addition to the hot path is a
> single NOP instruction (the static branch). When enabled, the contention
> check, trace call, and unlock are combined in an out-of-line function to
> minimize hot path impact, avoiding the compiler needing to preserve the
> lock pointer in a callee-saved register across the trace call.
>
> Binary size impact (x86_64, defconfig):
> uninlined unlock (common case): +680 bytes (+0.00%)
> inlined unlock (worst case): +83659 bytes (+0.21%)
>
> The inlined unlock case could not be achieved through Kconfig options on
> x86_64 as PREEMPT_BUILD unconditionally selects UNINLINE_SPIN_UNLOCK on
> x86_64. The UNINLINE_SPIN_UNLOCK guards were manually inverted to force
> inline the unlock path and estimate the worst case binary size increase.
>
> In practice, configurations with UNINLINE_SPIN_UNLOCK=n have already
> opted against binary size optimization, so the inlined worst case is
> unlikely to be a concern.
>
> Architectures with fully custom qspinlock implementations (e.g.
> PowerPC) are not covered by this change.
>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> Acked-by: Paul E. McKenney <paulmck@kernel.org>
> ---
> include/asm-generic/qspinlock.h | 18 ++++++++++++++++++
> kernel/locking/qspinlock.c | 8 ++++++++
> 2 files changed, 26 insertions(+)
>
> diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinlock.h
> index df76f34645a0..915a4c2777f6 100644
> --- a/include/asm-generic/qspinlock.h
> +++ b/include/asm-generic/qspinlock.h
> @@ -41,6 +41,7 @@
>
> #include <asm-generic/qspinlock_types.h>
> #include <linux/atomic.h>
> +#include <linux/tracepoint-defs.h>
>
> #ifndef queued_spin_is_locked
> /**
> @@ -129,12 +130,29 @@ static __always_inline void queued_spin_release(struct qspinlock *lock)
> }
> #endif
>
> +DECLARE_TRACEPOINT(contended_release);
> +
> +extern void queued_spin_release_traced(struct qspinlock *lock);
> +
> /**
> * queued_spin_unlock - unlock a queued spinlock
> * @lock : Pointer to queued spinlock structure
> + *
> + * Generic tracing wrapper around the arch-overridable
> + * queued_spin_release().
> */
> static __always_inline void queued_spin_unlock(struct qspinlock *lock)
> {
> + /*
> + * Trace and release are combined in queued_spin_release_traced() so
> + * the compiler does not need to preserve the lock pointer across the
> + * function call, avoiding callee-saved register save/restore on the
> + * hot path.
> + */
> + if (tracepoint_enabled(contended_release)) {
> + queued_spin_release_traced(lock);
> + return;
Get rid of the "return;". What does it save you? It just makes it that you
need to duplicate the code. Even though it's a one liner, it can cause bugs
in the future if this changes. You could call the function:
do_trace_queued_spin_release_traced(lock);
> + }
> queued_spin_release(lock);
> }
>
> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> index af8d122bb649..649fdca69288 100644
> --- a/kernel/locking/qspinlock.c
> +++ b/kernel/locking/qspinlock.c
> @@ -104,6 +104,14 @@ static __always_inline u32 __pv_wait_head_or_lock(struct qspinlock *lock,
> #define queued_spin_lock_slowpath native_queued_spin_lock_slowpath
> #endif
>
> +void __lockfunc queued_spin_release_traced(struct qspinlock *lock)
> +{
> + if (queued_spin_is_contended(lock))
> + trace_call__contended_release(lock);
> + queued_spin_release(lock);
And then remove the duplicate call of "queued_spin_release()" here.
-- Steve
> +}
> +EXPORT_SYMBOL(queued_spin_release_traced);
> +
> #endif /* _GEN_PV_LOCK_SLOWPATH */
>
> /**
^ permalink raw reply
* [PATCH v7 6/6] Documentation: document panic_on_unrecoverable_memory_failure sysctl
From: Breno Leitao @ 2026-05-13 15:39 UTC (permalink / raw)
To: Miaohe Lin, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett,
Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, Breno Leitao,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513-ecc_panic-v7-0-be2e578e61da@debian.org>
Add documentation for the new vm.panic_on_unrecoverable_memory_failure
sysctl, describing which failures trigger a panic (kernel-owned pages
the handler cannot recover) and which are intentionally left out
(transient allocator races and unclassified pages).
Signed-off-by: Breno Leitao <leitao@debian.org>
---
Documentation/admin-guide/sysctl/vm.rst | 80 +++++++++++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index 97e12359775c9..452c2ab25b35e 100644
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst
@@ -67,6 +67,7 @@ Currently, these files are in /proc/sys/vm:
- page-cluster
- page_lock_unfairness
- panic_on_oom
+- panic_on_unrecoverable_memory_failure
- percpu_pagelist_high_fraction
- stat_interval
- stat_refresh
@@ -925,6 +926,85 @@ panic_on_oom=2+kdump gives you very strong tool to investigate
why oom happens. You can get snapshot.
+panic_on_unrecoverable_memory_failure
+======================================
+
+When a hardware memory error (e.g. multi-bit ECC) hits a kernel page
+that cannot be recovered by the memory failure handler, the default
+behaviour is to ignore the error and continue operation. This is
+dangerous because the corrupted data remains accessible to the kernel,
+risking silent data corruption or a delayed crash when the poisoned
+memory is next accessed.
+
+When enabled, this sysctl triggers a panic on memory failure events
+hitting reserved (``PageReserved``) memory: firmware reservations,
+the kernel image, vDSO, the zero page, and similar memblock-reserved
+regions. These are owned by the kernel, are not managed by the page
+allocator, and cannot be recovered by the memory failure handler.
+
+Other unrecoverable kernel-owned populations (slab, vmalloc, page
+tables, kernel stacks, ...) are not currently covered by this
+sysctl. The handler cannot reliably distinguish them from a
+userspace folio temporarily off the LRU during migration or
+compaction, and the cost of a false-positive panic on a recoverable
+userspace page is too high. Such pages still go through the
+standard MF_MSG_GET_HWPOISON path: ``PG_hwpoison`` is set on them
+and a delayed crash on the next access remains possible. Coverage
+may grow in the future as the handler gains stronger
+kernel-ownership signals.
+
+Recoverable failure paths are also intentionally left out: in-flight
+buddy allocations and other transient races with the page allocator
+can reach the same diagnostic, and panicking on them would risk
+killing the box for a page destined for userspace where the standard
+SIGBUS recovery path applies. Pages whose state could not be
+classified at all are not covered either, since an unknown state is
+not a sound basis for a panic decision.
+
+For many environments it is preferable to panic immediately with a clean
+crash dump that captures the original error context, rather than to
+continue and face a random crash later whose cause is difficult to
+diagnose.
+
+Use cases
+---------
+
+This option is most useful in environments where unattributed crashes
+are expensive to debug or where data integrity must take precedence
+over availability:
+
+* Large fleets, where multi-bit ECC errors on kernel pages are observed
+ regularly and post-mortem analysis of an unrelated downstream crash
+ (often seconds to minutes after the original error) consumes
+ significant engineering effort.
+
+* Systems configured with kdump, where panicking at the moment of the
+ hardware error produces a vmcore that still contains the faulting
+ address, the affected page state, and the originating MCE/GHES
+ record — context that is typically lost by the time a delayed crash
+ occurs.
+
+* High-availability clusters that rely on fast, deterministic node
+ failure for failover, and prefer an immediate panic over silent data
+ corruption propagating to replicas or persistent storage.
+
+* Kernel and platform developers reproducing hwpoison issues with
+ tools such as ``mce-inject`` or error-injection debugfs interfaces,
+ where panicking on the unrecoverable path makes regressions
+ immediately visible instead of surfacing as later, unrelated
+ failures.
+
+= =====================================================================
+0 Try to continue operation (default).
+1 Panic immediately. If the ``panic`` sysctl is also non-zero then the
+ machine will be rebooted.
+= =====================================================================
+
+Example::
+
+ echo 1 > /proc/sys/vm/panic_on_unrecoverable_memory_failure
+
+
percpu_pagelist_high_fraction
=============================
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v7 5/6] mm/memory-failure: add panic option for unrecoverable pages
From: Breno Leitao @ 2026-05-13 15:39 UTC (permalink / raw)
To: Miaohe Lin, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett,
Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, Breno Leitao,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513-ecc_panic-v7-0-be2e578e61da@debian.org>
Add a sysctl panic_on_unrecoverable_memory_failure that triggers a
kernel panic when memory_failure() encounters pages that cannot be
recovered. This provides a clean crash with useful debug information
rather than allowing silent data corruption or a delayed crash at an
unrelated code path.
Panic eligibility is intentionally narrow: only MF_MSG_KERNEL with
result == MF_IGNORED panics. After the previous patch, MF_MSG_KERNEL
covers PG_reserved pages and the kernel-owned pages promoted from
get_hwpoison_page() via -ENOTRECOVERABLE (slab, vmalloc, page tables,
kernel stacks, ...).
All other action types are excluded:
- MF_MSG_GET_HWPOISON and MF_MSG_KERNEL_HIGH_ORDER can be reached by
transient refcount races with the page allocator (an in-flight buddy
allocation has refcount 0 and is no longer on the buddy free list,
briefly), and panicking on them would risk killing the box for what
is actually a recoverable userspace page.
- MF_MSG_UNKNOWN means identify_page_state() could not classify the
page; that is precisely the wrong basis for a panic decision.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
mm/memory-failure.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 8ba3df21d1270..cb2965c0ec0b4 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;
static int sysctl_enable_soft_offline __read_mostly = 1;
+static int sysctl_panic_on_unrecoverable_mf __read_mostly;
+
atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0);
static bool hw_memory_failure __read_mostly = false;
@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = {
.proc_handler = proc_dointvec_minmax,
.extra1 = SYSCTL_ZERO,
.extra2 = SYSCTL_ONE,
+ },
+ {
+ .procname = "panic_on_unrecoverable_memory_failure",
+ .data = &sysctl_panic_on_unrecoverable_mf,
+ .maxlen = sizeof(sysctl_panic_on_unrecoverable_mf),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
}
};
@@ -1267,6 +1278,15 @@ static void update_per_node_mf_stats(unsigned long pfn,
++mf_stats->total;
}
+static bool panic_on_unrecoverable_mf(enum mf_action_page_type type,
+ enum mf_result result)
+{
+ if (!sysctl_panic_on_unrecoverable_mf || result != MF_IGNORED)
+ return false;
+
+ return type == MF_MSG_KERNEL;
+}
+
/*
* "Dirty/Clean" indication is not 100% accurate due to the possibility of
* setting PG_dirty outside page lock. See also comment above set_page_dirty().
@@ -1284,6 +1304,9 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,
pr_err("%#lx: recovery action for %s: %s\n",
pfn, action_page_types[type], action_name[result]);
+ if (panic_on_unrecoverable_mf(type, result))
+ panic("Memory failure: %#lx: unrecoverable page", pfn);
+
return (result == MF_RECOVERED || result == MF_DELAYED) ? 0 : -EBUSY;
}
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()
From: Breno Leitao @ 2026-05-13 15:39 UTC (permalink / raw)
To: Miaohe Lin, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett,
Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, Breno Leitao,
linux-trace-kernel, kernel-team, Lance Yang
In-Reply-To: <20260513-ecc_panic-v7-0-be2e578e61da@debian.org>
The previous patch already classifies PG_reserved pages as
MF_MSG_KERNEL through the long path: get_hwpoison_page() calls
__get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page()
exhausts its shake_page() retry budget, and the resulting
-ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The
outcome is correct but the work in between is wasted: shake_page()
cannot turn a reserved page into a handlable one.
Detect PG_reserved up front in memory_failure() and report
MF_MSG_KERNEL directly. put_ref_page() releases the caller's
reference when MF_COUNT_INCREASED is set, which is important on the
MADV_HWPOISON path where get_user_pages_fast() holds a reference
across the call.
Suggested-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
mm/memory-failure.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 4b3a5d4190a07..8ba3df21d1270 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2398,6 +2398,19 @@ int memory_failure(unsigned long pfn, int flags)
goto unlock_mutex;
}
+ /*
+ * PG_reserved pages are kernel-owned (memblock reservations,
+ * driver reservations, ...) and cannot be recovered. Skip the
+ * get_hwpoison_page() lifecycle dance and report MF_MSG_KERNEL
+ * straight away; HWPoisonHandlable() would just keep rejecting
+ * the page through the retry budget anyway.
+ */
+ if (PageReserved(p)) {
+ put_ref_page(pfn, flags);
+ res = action_result(pfn, MF_MSG_KERNEL, MF_IGNORED);
+ goto unlock_mutex;
+ }
+
/*
* We need/can do nothing about count=0 pages.
* 1) it's a free page, and therefore in safe hand:
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v7 3/6] mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages
From: Breno Leitao @ 2026-05-13 15:39 UTC (permalink / raw)
To: Miaohe Lin, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett,
Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, Breno Leitao,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513-ecc_panic-v7-0-be2e578e61da@debian.org>
The previous patch teaches get_any_page() to return -ENOTRECOVERABLE
for stable unhandlable kernel pages (PG_reserved, slab, vmalloc, page
tables, kernel stacks, ...). memory_failure() still folds every
negative return into MF_MSG_GET_HWPOISON, so callers that want to
react to the unrecoverable cases (a panic option, smarter logging)
cannot tell them apart from transient page-allocator races.
Turn the post-call branch into a switch over the get_hwpoison_page()
return code: map -ENOTRECOVERABLE to MF_MSG_KERNEL and any other
negative return to MF_MSG_GET_HWPOISON. case 0 keeps the existing
free-buddy / kernel-high-order handling and case 1 falls through to
the rest of memory_failure() unchanged.
The MF_MSG_KERNEL label and tracepoint string are kept as
"reserved kernel page" to avoid breaking userspace tools that match
on those literals; the enum value still adequately tags the failure
even though it now also covers slab, vmalloc, page tables and kernel
stack pages.
Suggested-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
mm/memory-failure.c | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index bae883df3ccb2..4b3a5d4190a07 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2410,7 +2410,8 @@ int memory_failure(unsigned long pfn, int flags)
* that may make page_ref_freeze()/page_ref_unfreeze() mismatch.
*/
res = get_hwpoison_page(p, flags);
- if (!res) {
+ switch (res) {
+ case 0:
if (is_free_buddy_page(p)) {
if (take_page_off_buddy(p)) {
page_ref_inc(p);
@@ -2429,7 +2430,19 @@ int memory_failure(unsigned long pfn, int flags)
res = action_result(pfn, MF_MSG_KERNEL_HIGH_ORDER, MF_IGNORED);
}
goto unlock_mutex;
- } else if (res < 0) {
+ case 1:
+ /* Got a refcount on a handlable page. */
+ break;
+ case -ENOTRECOVERABLE:
+ /*
+ * Stable unhandlable kernel-owned page (PG_reserved,
+ * slab, vmalloc, page tables, kernel stacks, ...).
+ * No recovery possible.
+ */
+ res = action_result(pfn, MF_MSG_KERNEL, MF_IGNORED);
+ goto unlock_mutex;
+ default:
+ /* Transient lifecycle race with the page allocator. */
res = action_result(pfn, MF_MSG_GET_HWPOISON, MF_IGNORED);
goto unlock_mutex;
}
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
From: Breno Leitao @ 2026-05-13 15:39 UTC (permalink / raw)
To: Miaohe Lin, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Shuah Khan, Naoya Horiguchi, Steven Rostedt, Masami Hiramatsu,
Mathieu Desnoyers, Jonathan Corbet, Shuah Khan, Liam R. Howlett,
Liam R. Howlett
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, Breno Leitao,
linux-trace-kernel, kernel-team
In-Reply-To: <20260513-ecc_panic-v7-0-be2e578e61da@debian.org>
get_any_page() collapses three different failure modes into a single
-EIO return:
* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of
__get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the
count_increased / put_page / shake_page retry loop.
The first is transient (the page is racing with the allocator). The
second can be either transient (a userspace folio briefly off LRU
during migration/compaction) or stable (slab/vmalloc/page-table/
kernel-stack pages). The third describes a stable kernel-owned page
that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep -EIO for both the put_page
race and the -EBUSY-after-retries branch (shake_page() cannot drag a
folio back from active migration, so we cannot prove the page is
permanently kernel-owned from there), keep -EBUSY for the allocation
race (unchanged), and return -ENOTRECOVERABLE only from the
count_increased-true HWPoisonHandlable() rejection that exhausts its
retries -- the caller's reference is structural evidence that the
page is owned by the kernel.
Extend the unhandlable-page pr_err() to fire for either errno and
update the get_hwpoison_page() kerneldoc.
memory_failure() still folds every negative return into
MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so
this patch is a no-op for users of memory_failure() and only changes
the errno that soft_offline_page() can propagate to its callers. A
follow-up wires the new return code through memory_failure() and
reports MF_MSG_KERNEL for the unrecoverable cases.
Suggested-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
mm/memory-failure.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 49bcfbd04d213..bae883df3ccb2 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
shake_page(p);
goto try_again;
}
+ /*
+ * Return -EIO rather than -ENOTRECOVERABLE: this
+ * branch is also reached for pages that are merely
+ * off-LRU transiently (e.g. a folio in the middle
+ * of migration or compaction), which shake_page()
+ * cannot drag back. The caller cannot prove the
+ * page is permanently kernel-owned from here, so
+ * keep it on the recoverable errno.
+ */
ret = -EIO;
goto out;
}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
goto try_again;
}
put_page(p);
- ret = -EIO;
+ ret = -ENOTRECOVERABLE;
}
out:
- if (ret == -EIO)
+ if (ret == -EIO || ret == -ENOTRECOVERABLE)
pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
return ret;
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
* -EIO for pages on which we can not handle memory errors,
* -EBUSY when get_hwpoison_page() has raced with page lifecycle
* operations like allocation and free,
- * -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ * -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ * -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ * cannot recover (PG_reserved, slab, vmalloc, page tables,
+ * kernel stacks, and similar non-LRU/non-buddy pages).
*/
static int get_hwpoison_page(struct page *p, unsigned long flags)
{
--
2.53.0-Meta
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox