* Re: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
From: Breno Leitao @ 2026-06-08 14:15 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Miaohe Lin, linux-mm, linux-kernel, linux-doc, linux-kselftest,
linux-trace-kernel, kernel-team, Lance Yang, Andrew Morton,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Shuah Khan, Naoya Horiguchi,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers,
Jonathan Corbet, Shuah Khan, Liam R. Howlett
In-Reply-To: <f1a742be-80cb-4256-b1f9-e50a0f83cb15@kernel.org>
On Fri, Jun 05, 2026 at 11:42:53AM +0200, David Hildenbrand (Arm) wrote:
> On 6/5/26 11:35, Breno Leitao wrote:
> > On Wed, Jun 03, 2026 at 10:33:04AM +0800, Miaohe Lin wrote:
> >> On 2026/6/2 17:41, David Hildenbrand (Arm) wrote:
> >>>
> >>> Races are fine. We might miss some pages, but that can happen on races either way.
> >>>
> >>>
> >>> I'd just do something like
> >>>
> >>> if (PageReserved(page))
> >>> return true;
> >>>
> >>> head = compound_head(page);
> >>
> >> If @head is split just after compound_head. And then @head is freed into buddy and re-allocated as slab
> >> page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were
> >> supposed to successfully handle @page. Or am I miss something?
> >
> > You're right that it is racy, but I think it is an acceptable race here.
> >
>
> I mean, any such races can currently already happen one way or the other?
>
> Really, the only way to not get races is to tryget the (compound)page,
> revalidate that the page is still part of the compound page.
>
> I'm not sure if that's really a good idea.
>
> But my memory is a bit vague in which scenarios we already hold a page reference
> here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned() runs at the very top of get_any_page(), before
try_again: and before __get_hwpoison_page(). The first refcount taken in
the whole path is the folio_try_get() inside __get_hwpoison_page(), which
runs *after* the short-circuit.
So get_any_page() itself never holds a reference at the check -- the only way
one exists is if the caller passed MF_COUNT_INCREASED (count_increased ==
true).
So on the MCE/GHES path -- the one this panic option exists for -- no
reference is held when HWPoisonKernelOwned() does its compound_head() +
PageSlab()/PageTable()/PageLargeKmalloc() checks.
Given that, I'd rather keep it racy and take no refcount than add a
tryget + revalidate purely for this check. As I've said earleir, an operator
who enabled it has chosen to crash rather than run on corrupted memory;
mis-attributing one such rare, genuinely-poisoned page is within that contract.
^ permalink raw reply
* [RFC PATCH 0/7] tracing/probes: Add more typecast features
From: Masami Hiramatsu (Google) @ 2026-06-08 14:24 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
Hi,
Here is a series of patches to introduce more typecast features
to probe events, which includes 1. expanding BTF typecast to
fprobe and kprobe events, 2. introducing container_of like typecst
option, 3. supporting nested typecast, 4. adding $current special
variable support, 5. adding per-cpu dereference support, 6. adding
a testcase to check typecasts.
Steve introduced BTF typecast feature for eprobe[1].
This series extends it and add more options:
1. Expanding BTF typecast to kprobe and fprobe.
(currently only function entry/exit)
2. Introduce container_of like typecast. This adds a "assigned
member" option to the typecast.
(STRUCT,MEMBER)VAR->ANOTHER_MEMBER
This casts VAR to STRUCT type but the VAR is as the address
of STRUCT.MEMBER. In C, it is:
container_of(VAR, STRUCT, MEMBER)->ANOTHER_MEMBER
3. Support nested typecast, e.g.
(STRUCT)((STRUCT2)VAR->MEMBER2)->MEMBER
the nest level must be smaller than 3.
4. Add $current variable to point "current" task_struct.
This is useful with typecast, e.g.
(task_struct)$current->pid
5. per-cpu dereference support.
+CPU(VAR) is the same as this_cpu_read(VAR), and
+PCPU(VAR) is the same as this_cpu_ptr(VAR).
Also, "this_cpu_ptr(VAR)" is available. This is good
with nesting expression.
(STRUCT)(this_cpu_ptr(VAR))->MEMBER
(However, it might be better to allow a special way to omit
parentheses for thi_cpu_ptr())
And added a test script to test part of them.
[1] https://lore.kernel.org/all/20260601130746.2139d926@gandalf.local.home/
---
Masami Hiramatsu (Google) (7):
tracing/probes: Support typecast for various probe events
tracing/probes: Support nested typecast
tracing/probes: Support field specifier option for typecast
tracing/probes: Add $current variable support
tracing/probes: Add +CPU() and +PCPU() dereference method to fetcharg
tracing/probes: Support reserved this_cpu_ptr() method
tracing/probes: Add a new testcase for BTF typecasts
Documentation/trace/eprobetrace.rst | 11 +
Documentation/trace/fprobetrace.rst | 11 +
Documentation/trace/kprobetrace.rst | 12 +
kernel/trace/trace.c | 6
kernel/trace/trace_probe.c | 312 +++++++++++++++-----
kernel/trace/trace_probe.h | 12 +
kernel/trace/trace_probe_tmpl.h | 33 ++
samples/trace_events/trace-events-sample.c | 38 ++
samples/trace_events/trace-events-sample.h | 34 ++
.../ftrace/test.d/dynevent/btf_probe_event.tc | 52 +++
10 files changed, 422 insertions(+), 99 deletions(-)
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc
--
Signature
^ permalink raw reply
* [RFC PATCH 1/7] tracing/probes: Support typecast for various probe events
From: Masami Hiramatsu (Google) @ 2026-06-08 14:24 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Support BTF typecast feature on other probe events (but only if it is
kernel function entry or return.)
To support other probe events, we just need to use last_struct type
when we find a function parameter in parse_btf_arg().
This also update <tracefs>/README file to show struct typecast.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/trace/fprobetrace.rst | 3 +++
Documentation/trace/kprobetrace.rst | 4 ++++
kernel/trace/trace.c | 2 +-
kernel/trace/trace_probe.c | 12 +++++++-----
4 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst
index b4c2ca3d02c1..7435ded2d66d 100644
--- a/Documentation/trace/fprobetrace.rst
+++ b/Documentation/trace/fprobetrace.rst
@@ -57,6 +57,9 @@ Synopsis of fprobe-events
(u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
(x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
and bitfield are supported.
+ (STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
+ a pointer to STRUCT and then derference the pointer defined by
+ ->MEMBER.
(\*1) This is available only when BTF is enabled.
(\*2) only for the probe on function entry (offs == 0). Note, this argument access
diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst
index 3b6791c17e9b..f73614997d52 100644
--- a/Documentation/trace/kprobetrace.rst
+++ b/Documentation/trace/kprobetrace.rst
@@ -61,6 +61,10 @@ Synopsis of kprobe_events
(x8/x16/x32/x64), VFS layer common type(%pd/%pD), "char",
"string", "ustring", "symbol", "symstr" and bitfield are
supported.
+ (STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
+ a pointer to STRUCT and then derference the pointer defined by
+ ->MEMBER. Note that this is available only when the probe is
+ on function entry.
(\*1) only for the probe on function entry (offs == 0). Note, this argument access
is best effort, because depending on the argument type, it may be passed on
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 6eb4d3097a4d..aa93e7b01146 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4325,7 +4325,7 @@ static const char readme_msg[] =
#ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API
"\t $stack<index>, $stack, $retval, $comm, $arg<N>,\n"
#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
- "\t <argname>[->field[->field|.field...]],\n"
+ "\t [(structname)]<argname>[->field[->field|.field...]],\n"
#endif
#else
"\t $stack<index>, $stack, $retval, $comm,\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index fd1caa1f9723..609b156986c5 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -759,7 +759,10 @@ static int parse_btf_arg(char *varname,
return -ENOENT;
found:
- type = btf_type_skip_modifiers(ctx->btf, tid, &tid);
+ if (ctx->struct_btf)
+ type = ctx->last_struct;
+ else
+ type = btf_type_skip_modifiers(ctx->btf, tid, &tid);
found_type:
if (!type) {
trace_probe_log_err(ctx->offset, BAD_BTF_TID);
@@ -836,10 +839,9 @@ static int handle_typecast(char *arg, struct fetch_insn **pcode,
char *tmp;
int ret;
- /* Currently this only works for eprobes */
- if (!(ctx->flags & TPARG_FL_TEVENT)) {
- trace_probe_log_err(ctx->offset, TYPECAST_NOT_EVENT);
- return -EINVAL;
+ if (!(tparg_is_function_entry(ctx->flags) || tparg_is_function_return(ctx->flags))) {
+ trace_probe_log_err(ctx->offset, NOSUP_BTFARG);
+ return -EOPNOTSUPP;
}
tmp = strchr(arg, ')');
^ permalink raw reply related
* [RFC PATCH 2/7] tracing/probes: Support nested typecast
From: Masami Hiramatsu (Google) @ 2026-06-08 14:24 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
When we hit an open parenthesis right after typecast closing
parenthesis, it means we have nested typecast. This allows us to
typecast a generic data member in a structure to a pointer to
another structure.
For example, to cast a DATA_MEMBER of VAR structure to STRUCT pointer
and get MEMBER value.
(STRUCT)(VAR->DATA_MEMBER)->MEMBER
Also, we can nest typecast.
(STRUCT1)((STRUCT2)$ARG->FIELD2)->FIELD1
Currently the max nest level is limited to 3.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/trace/eprobetrace.rst | 2 +
Documentation/trace/fprobetrace.rst | 2 +
Documentation/trace/kprobetrace.rst | 2 +
kernel/trace/trace.c | 1
kernel/trace/trace_probe.c | 76 ++++++++++++++++++++++++++++++++---
kernel/trace/trace_probe.h | 7 +++
6 files changed, 82 insertions(+), 8 deletions(-)
diff --git a/Documentation/trace/eprobetrace.rst b/Documentation/trace/eprobetrace.rst
index fe3602540569..cd0b4aa7f896 100644
--- a/Documentation/trace/eprobetrace.rst
+++ b/Documentation/trace/eprobetrace.rst
@@ -50,6 +50,8 @@ Synopsis of eprobe_events
a pointer to STRUCT and then derference the pointer defined by
->MEMBER. Note that when this is used, the FIELD name does not
need to be prefixed with a '$'.
+ (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
+ also be used with another FETCHARG instead of FIELD.
Types
-----
diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst
index 7435ded2d66d..6b8bb27bb62d 100644
--- a/Documentation/trace/fprobetrace.rst
+++ b/Documentation/trace/fprobetrace.rst
@@ -60,6 +60,8 @@ Synopsis of fprobe-events
(STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
a pointer to STRUCT and then derference the pointer defined by
->MEMBER.
+ (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
+ also be used with another FETCHARG instead of FIELD.
(\*1) This is available only when BTF is enabled.
(\*2) only for the probe on function entry (offs == 0). Note, this argument access
diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst
index f73614997d52..c4382765d5b2 100644
--- a/Documentation/trace/kprobetrace.rst
+++ b/Documentation/trace/kprobetrace.rst
@@ -65,6 +65,8 @@ Synopsis of kprobe_events
a pointer to STRUCT and then derference the pointer defined by
->MEMBER. Note that this is available only when the probe is
on function entry.
+ (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
+ also be used with another FETCHARG instead of FIELD.
(\*1) only for the probe on function entry (offs == 0). Note, this argument access
is best effort, because depending on the argument type, it may be passed on
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index aa93e7b01146..4f70318918c2 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4326,6 +4326,7 @@ static const char readme_msg[] =
"\t $stack<index>, $stack, $retval, $comm, $arg<N>,\n"
#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
"\t [(structname)]<argname>[->field[->field|.field...]],\n"
+ "\t [(structname)](fetcharg)->field[->field|.field...],\n"
#endif
#else
"\t $stack<index>, $stack, $retval, $comm,\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 609b156986c5..ddd9b1b63a17 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -832,10 +832,35 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
return 0;
}
+/* Find the matching closing parenthesis for a given opening parenthesis. */
+static char *find_matched_close_paren(char *s)
+{
+ char *p = s;
+ int count = 0;
+
+ while (*p) {
+ if (*p == '(')
+ count++;
+ else if (*p == ')') {
+ if (--count == 0)
+ return p;
+ }
+ p++;
+ }
+ return NULL;
+}
+
+static int
+parse_probe_arg(char *arg, const struct fetch_type *type,
+ struct fetch_insn **pcode, struct fetch_insn *end,
+ struct traceprobe_parse_context *ctx);
+
static int handle_typecast(char *arg, struct fetch_insn **pcode,
struct fetch_insn *end,
struct traceprobe_parse_context *ctx)
{
+ int orig_offset = ctx->offset;
+ bool nested = false;
char *tmp;
int ret;
@@ -850,19 +875,56 @@ static int handle_typecast(char *arg, struct fetch_insn **pcode,
DEREF_OPEN_BRACE);
return -EINVAL;
}
- *tmp = '\0';
- ret = query_btf_struct(arg + 1, ctx);
- *tmp = ')';
+ *tmp++ = '\0';
+
+ /* Handle the nested structure like (STRUCT)(VAR->FIELD)->... */
+ if (*tmp == '(') {
+ char *close = find_matched_close_paren(tmp);
+
+ ctx->offset += tmp - arg;
+ if (!close) {
+ trace_probe_log_err(ctx->offset, DEREF_OPEN_BRACE);
+ return -EINVAL;
+ }
+ /* We expect a field access for typecast */
+ if (close[1] != '-' || close[2] != '>') {
+ trace_probe_log_err(ctx->offset + close - tmp + 1,
+ TYPECAST_REQ_FIELD);
+ return -EINVAL;
+ }
+ ctx->nested_level++;
+ if (ctx->nested_level > TRACEPROBE_MAX_NESTED_LEVEL) {
+ trace_probe_log_err(ctx->offset, TOO_MANY_NESTED);
+ return -E2BIG;
+ }
+ *close = '\0';
+
+ ctx->offset += 1; /* for the '(' */
+ /* We need to parse the nested one */
+ ret = parse_probe_arg(tmp + 1, find_fetch_type(NULL, ctx->flags),
+ pcode, end, ctx);
+ if (ret < 0)
+ return ret;
+ ctx->nested_level--;
+ clear_struct_btf(ctx);
+
+ tmp = close + 1;
+ nested = true;
+ }
+
+ ret = query_btf_struct(arg + 1, ctx);
if (ret < 0) {
trace_probe_log_err(ctx->offset + 1, NO_PTR_STRCT);
return -EINVAL;
}
- tmp++;
-
- ctx->offset += tmp - arg;
- ret = parse_btf_arg(tmp, pcode, end, ctx);
+ ctx->offset = orig_offset + tmp - arg;
+ /* If it is nested, tmp points to the field name. */
+ if (nested)
+ ret = parse_btf_field(tmp, ctx->last_struct, pcode, end, ctx);
+ else
+ ret = parse_btf_arg(tmp, pcode, end, ctx);
return ret;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 15758cc11fc6..8dcc65e4e1db 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -430,8 +430,11 @@ struct traceprobe_parse_context {
struct trace_probe *tp;
unsigned int flags;
int offset;
+ int nested_level;
};
+#define TRACEPROBE_MAX_NESTED_LEVEL 3
+
extern int traceprobe_parse_probe_arg(struct trace_probe *tp, int i,
const char *argv,
struct traceprobe_parse_context *ctx);
@@ -566,7 +569,9 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(TOO_MANY_ARGS, "Too many arguments are specified"), \
C(TOO_MANY_EARGS, "Too many entry arguments specified"), \
C(EVENT_TOO_BIG, "Event too big (too many fields?)"), \
- C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"),
+ C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \
+ C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \
+ C(TOO_MANY_NESTED, "Too many nested typecasts/dereferences"),
#undef C
#define C(a, b) TP_ERR_##a
^ permalink raw reply related
* [RFC PATCH 3/7] tracing/probes: Support field specifier option for typecast
From: Masami Hiramatsu (Google) @ 2026-06-08 14:24 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add a field specifier option for the typecast. This works like
container_of() macro.
(STRUCT[,FIELD[.FIELD2...]])VAR
This is equivalent to :
container_of(VAR, struct STRUCT, FIELD[.FIELD2...])
For example:
echo "f tick_nohz_handler next_tick=(tick_sched,sched_timer)timer->next_tick" >> dynamic_events
This will trace tick_nohz_handler() with its tick_sched::next_tick which
is converted from @timer by contianer_of(tick, struct tick_sched, sched_timer).
So, if you enabkle both fprobes:tick_nohz_handler__entry and
timer:hrtimer_expire_entry events, we will see something like:
<idle>-0 [002] d.h1. 3778.087272: hrtimer_expire_entry: hrtimer=00000000d63db328 f
unction=tick_nohz_handler now=3777450051040
<idle>-0 [002] d.h1. 3778.087281: tick_nohz_handler__entry: (tick_nohz_handler+0x4
/0x140) next_tick=3777450000000
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/trace/eprobetrace.rst | 5 +
Documentation/trace/fprobetrace.rst | 8 +-
Documentation/trace/kprobetrace.rst | 8 +-
kernel/trace/trace.c | 4 -
kernel/trace/trace_probe.c | 169 +++++++++++++++++++++++------------
kernel/trace/trace_probe.h | 4 +
6 files changed, 131 insertions(+), 67 deletions(-)
diff --git a/Documentation/trace/eprobetrace.rst b/Documentation/trace/eprobetrace.rst
index cd0b4aa7f896..680e0af43d5d 100644
--- a/Documentation/trace/eprobetrace.rst
+++ b/Documentation/trace/eprobetrace.rst
@@ -49,7 +49,10 @@ Synopsis of eprobe_events
(STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
a pointer to STRUCT and then derference the pointer defined by
->MEMBER. Note that when this is used, the FIELD name does not
- need to be prefixed with a '$'.
+ need to be prefixed with a '$'. ASGN can be specified optionally.
+ If ASGN is specified, FIELD will be cast to the same offset
+ position as the ASGN member, rather than to the beginning of
+ the STRUCT.
(STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
also be used with another FETCHARG instead of FIELD.
diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst
index 6b8bb27bb62d..290a9e6f7491 100644
--- a/Documentation/trace/fprobetrace.rst
+++ b/Documentation/trace/fprobetrace.rst
@@ -57,10 +57,12 @@ Synopsis of fprobe-events
(u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
(x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
and bitfield are supported.
- (STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
+ (STRUCT[,ASGN])FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
a pointer to STRUCT and then derference the pointer defined by
- ->MEMBER.
- (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
+ ->MEMBER. ASGN can be specified optionally. If ASGN is specified,
+ FIELD will be cast to the same offset position as the ASGN member,
+ rather than to the beginning of the STRUCT.
+ (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
also be used with another FETCHARG instead of FIELD.
(\*1) This is available only when BTF is enabled.
diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst
index c4382765d5b2..a62707e6a9f2 100644
--- a/Documentation/trace/kprobetrace.rst
+++ b/Documentation/trace/kprobetrace.rst
@@ -61,11 +61,13 @@ Synopsis of kprobe_events
(x8/x16/x32/x64), VFS layer common type(%pd/%pD), "char",
"string", "ustring", "symbol", "symstr" and bitfield are
supported.
- (STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
+ (STRUCT[,ASGN])FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to
a pointer to STRUCT and then derference the pointer defined by
->MEMBER. Note that this is available only when the probe is
- on function entry.
- (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
+ on function entry. ASGN can be specified optionally. If ASGN
+ is specified, FIELD will be cast to the same offset position
+ as the ASGN member, rather than to the beginning of the STRUCT.
+ (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can
also be used with another FETCHARG instead of FIELD.
(\*1) only for the probe on function entry (offs == 0). Note, this argument access
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 4f70318918c2..0e36af853199 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4325,8 +4325,8 @@ static const char readme_msg[] =
#ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API
"\t $stack<index>, $stack, $retval, $comm, $arg<N>,\n"
#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
- "\t [(structname)]<argname>[->field[->field|.field...]],\n"
- "\t [(structname)](fetcharg)->field[->field|.field...],\n"
+ "\t [(structname[,field])]<argname>[->field[->field|.field...]],\n"
+ "\t [(structname[,field])](fetcharg)->field[->field|.field...],\n"
#endif
#else
"\t $stack<index>, $stack, $retval, $comm,\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index ddd9b1b63a17..ff0b619e9a90 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -574,6 +574,65 @@ static int split_next_field(char *varname, char **next_field,
return ret;
}
+/* Inner loop for solving dot operator ('.'). Return bit-offset of the given field */
+static int get_bitoffset_of_field(char **pfieldname, const struct btf_type **ptype,
+ struct traceprobe_parse_context *ctx)
+{
+ const struct btf_type *type = *ptype;
+ const struct btf_member *field;
+ struct btf *btf = ctx_btf(ctx);
+ char *fieldname = *pfieldname;
+ int bitoffs = 0;
+ u32 anon_offs;
+ char *next;
+ int is_ptr;
+ s32 tid;
+
+ do {
+ next = NULL;
+ is_ptr = split_next_field(fieldname, &next, ctx);
+ if (is_ptr < 0)
+ return is_ptr;
+
+ anon_offs = 0;
+ field = btf_find_struct_member(btf, type, fieldname,
+ &anon_offs);
+ if (IS_ERR(field)) {
+ trace_probe_log_err(ctx->offset, BAD_BTF_TID);
+ return PTR_ERR(field);
+ }
+ if (!field) {
+ trace_probe_log_err(ctx->offset, NO_BTF_FIELD);
+ return -ENOENT;
+ }
+ /* Add anonymous structure/union offset */
+ bitoffs += anon_offs;
+
+ /* Accumulate the bit-offsets of the dot-connected fields */
+ if (btf_type_kflag(type)) {
+ bitoffs += BTF_MEMBER_BIT_OFFSET(field->offset);
+ ctx->last_bitsize = BTF_MEMBER_BITFIELD_SIZE(field->offset);
+ } else {
+ bitoffs += field->offset;
+ ctx->last_bitsize = 0;
+ }
+
+ type = btf_type_skip_modifiers(btf, field->type, &tid);
+ if (!type) {
+ trace_probe_log_err(ctx->offset, BAD_BTF_TID);
+ return -EINVAL;
+ }
+
+ if (next)
+ ctx->offset += next - fieldname;
+ fieldname = next;
+ } while (!is_ptr && fieldname);
+
+ *pfieldname = fieldname;
+ *ptype = type;
+
+ return bitoffs;
+}
/*
* Parse the field of data structure. The @type must be a pointer type
* pointing the target data structure type.
@@ -583,16 +642,14 @@ static int parse_btf_field(char *fieldname, const struct btf_type *type,
struct traceprobe_parse_context *ctx)
{
struct fetch_insn *code = *pcode;
- const struct btf_member *field;
- u32 bitoffs, anon_offs;
- bool is_struct = ctx->struct_btf != NULL;
struct btf *btf = ctx_btf(ctx);
- char *next;
- int is_ptr;
+ bool is_first_field = true;
+ int bitoffs;
s32 tid;
do {
- if (!is_struct) {
+ /* For the first field of typecast, @type will be the target structure type. */
+ if (!(is_first_field && ctx->struct_btf)) {
/* Outer loop for solving arrow operator ('->') */
if (BTF_INFO_KIND(type->info) != BTF_KIND_PTR) {
trace_probe_log_err(ctx->offset, NO_PTR_STRCT);
@@ -606,60 +663,25 @@ static int parse_btf_field(char *fieldname, const struct btf_type *type,
return -EINVAL;
}
}
- /* Only the first type can skip being a pointer */
- is_struct = false;
-
- bitoffs = 0;
- do {
- /* Inner loop for solving dot operator ('.') */
- next = NULL;
- is_ptr = split_next_field(fieldname, &next, ctx);
- if (is_ptr < 0)
- return is_ptr;
-
- anon_offs = 0;
- field = btf_find_struct_member(btf, type, fieldname,
- &anon_offs);
- if (IS_ERR(field)) {
- trace_probe_log_err(ctx->offset, BAD_BTF_TID);
- return PTR_ERR(field);
- }
- if (!field) {
- trace_probe_log_err(ctx->offset, NO_BTF_FIELD);
- return -ENOENT;
- }
- /* Add anonymous structure/union offset */
- bitoffs += anon_offs;
-
- /* Accumulate the bit-offsets of the dot-connected fields */
- if (btf_type_kflag(type)) {
- bitoffs += BTF_MEMBER_BIT_OFFSET(field->offset);
- ctx->last_bitsize = BTF_MEMBER_BITFIELD_SIZE(field->offset);
- } else {
- bitoffs += field->offset;
- ctx->last_bitsize = 0;
- }
-
- type = btf_type_skip_modifiers(btf, field->type, &tid);
- if (!type) {
- trace_probe_log_err(ctx->offset, BAD_BTF_TID);
- return -EINVAL;
- }
-
- ctx->offset += next - fieldname;
- fieldname = next;
- } while (!is_ptr && fieldname);
+ bitoffs = get_bitoffset_of_field(&fieldname, &type, ctx);
+ if (bitoffs < 0)
+ return bitoffs;
if (++code == end) {
trace_probe_log_err(ctx->offset, TOO_MANY_OPS);
return -EINVAL;
}
code->op = FETCH_OP_DEREF; /* TODO: user deref support */
+ if (is_first_field && ctx->struct_btf) {
+ /* The first field can be typecasted with field option. */
+ bitoffs -= ctx->prefix_bitoffs;
+ }
code->offset = bitoffs / 8;
*pcode = code;
ctx->last_bitoffs = bitoffs % 8;
ctx->last_type = type;
+ is_first_field = false;
} while (fieldname);
return 0;
@@ -700,8 +722,7 @@ static int parse_btf_arg(char *varname,
/* TEVENT is only here via a typecast */
if (WARN_ON_ONCE(ctx->struct_btf == NULL))
return -EINVAL;
- type = ctx->last_struct;
- goto found_type;
+ goto found;
}
if (ctx->flags & TPARG_FL_RETURN && !strcmp(varname, "$retval")) {
@@ -763,7 +784,6 @@ static int parse_btf_arg(char *varname,
type = ctx->last_struct;
else
type = btf_type_skip_modifiers(ctx->btf, tid, &tid);
-found_type:
if (!type) {
trace_probe_log_err(ctx->offset, BAD_BTF_TID);
return -EINVAL;
@@ -832,6 +852,41 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
return 0;
}
+static int parse_btf_casttype(char *casttype, struct traceprobe_parse_context *ctx)
+{
+ char *field;
+ int ret;
+
+ /* Field option - evaluated later. */
+ field = strchr(casttype, ',');
+ if (field)
+ *field++ = '\0';
+
+ ret = query_btf_struct(casttype, ctx);
+ if (ret < 0) {
+ trace_probe_log_err(ctx->offset, NO_PTR_STRCT);
+ return -EINVAL;
+ }
+
+ if (field) {
+ struct btf_type *type = (struct btf_type *)ctx->last_struct;
+
+ ctx->offset += field - casttype;
+ ret = get_bitoffset_of_field(&field, &ctx->last_struct, ctx);
+ if (ret < 0)
+ return ret;
+ if (ret % 8) {
+ trace_probe_log_err(ctx->offset, TYPECAST_NOT_ALIGNED);
+ return -EINVAL;
+ }
+ ctx->prefix_bitoffs = ret;
+ /* Restore the original struct type (overwritten by get_bitoffset_of_field) */
+ ctx->last_struct = type;
+ }
+
+ return ret;
+}
+
/* Find the matching closing parenthesis for a given opening parenthesis. */
static char *find_matched_close_paren(char *s)
{
@@ -913,11 +968,10 @@ static int handle_typecast(char *arg, struct fetch_insn **pcode,
nested = true;
}
- ret = query_btf_struct(arg + 1, ctx);
- if (ret < 0) {
- trace_probe_log_err(ctx->offset + 1, NO_PTR_STRCT);
- return -EINVAL;
- }
+ ctx->offset = orig_offset + 1; /* for the '(' */
+ ret = parse_btf_casttype(arg + 1, ctx);
+ if (ret < 0)
+ return ret;
ctx->offset = orig_offset + tmp - arg;
/* If it is nested, tmp points to the field name. */
@@ -925,6 +979,7 @@ static int handle_typecast(char *arg, struct fetch_insn **pcode,
ret = parse_btf_field(tmp, ctx->last_struct, pcode, end, ctx);
else
ret = parse_btf_arg(tmp, pcode, end, ctx);
+ ctx->prefix_bitoffs = 0;
return ret;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 8dcc65e4e1db..b1a54da3c761 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -431,6 +431,7 @@ struct traceprobe_parse_context {
unsigned int flags;
int offset;
int nested_level;
+ int prefix_bitoffs; /* The bit offset of the prefix field of typecast */
};
#define TRACEPROBE_MAX_NESTED_LEVEL 3
@@ -571,7 +572,8 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(EVENT_TOO_BIG, "Event too big (too many fields?)"), \
C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \
C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \
- C(TOO_MANY_NESTED, "Too many nested typecasts/dereferences"),
+ C(TOO_MANY_NESTED, "Too many nested typecasts/dereferences"), \
+ C(TYPECAST_NOT_ALIGNED, "Typecast field option is not byte-aligned"),
#undef C
#define C(a, b) TP_ERR_##a
^ permalink raw reply related
* [RFC PATCH 4/7] tracing/probes: Add $current variable support
From: Masami Hiramatsu (Google) @ 2026-06-08 14:24 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Since we can use the BTF to cast value to a structure pointer type,
it is useful to introduce "$current" special variable support to
fetcharg.
User can define a fetcharg to access current task_struct properties
using BTF typecast (or dereference - but this may be complicated) e.g.
(task_struct)$current->cpus_ptr
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/trace/eprobetrace.rst | 1 +
Documentation/trace/fprobetrace.rst | 1 +
Documentation/trace/kprobetrace.rst | 1 +
kernel/trace/trace.c | 2 +-
kernel/trace/trace_probe.c | 6 ++++++
kernel/trace/trace_probe.h | 1 +
kernel/trace/trace_probe_tmpl.h | 3 +++
7 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/trace/eprobetrace.rst b/Documentation/trace/eprobetrace.rst
index 680e0af43d5d..dcf92d5b4175 100644
--- a/Documentation/trace/eprobetrace.rst
+++ b/Documentation/trace/eprobetrace.rst
@@ -38,6 +38,7 @@ Synopsis of eprobe_events
@ADDR : Fetch memory at ADDR (ADDR should be in kernel)
@SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
$comm : Fetch current task comm.
+ $current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst
index 290a9e6f7491..3392cab016b3 100644
--- a/Documentation/trace/fprobetrace.rst
+++ b/Documentation/trace/fprobetrace.rst
@@ -50,6 +50,7 @@ Synopsis of fprobe-events
$argN : Fetch the Nth function argument. (N >= 1) (\*2)
$retval : Fetch return value.(\*3)
$comm : Fetch current task comm.
+ $current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*4)(\*5)
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst
index a62707e6a9f2..81e4fe38791d 100644
--- a/Documentation/trace/kprobetrace.rst
+++ b/Documentation/trace/kprobetrace.rst
@@ -53,6 +53,7 @@ Synopsis of kprobe_events
$argN : Fetch the Nth function argument. (N >= 1) (\*1)
$retval : Fetch return value.(\*2)
$comm : Fetch current task comm.
+ $current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 0e36af853199..e185a006cb08 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4329,7 +4329,7 @@ static const char readme_msg[] =
"\t [(structname[,field])](fetcharg)->field[->field|.field...],\n"
#endif
#else
- "\t $stack<index>, $stack, $retval, $comm,\n"
+ "\t $stack<index>, $stack, $retval, $comm, $current\n"
#endif
"\t +|-[u]<offset>(<fetcharg>), \\imm-value, \\\"imm-string\"\n"
"\t kernel return probes support: $retval, $arg<N>, $comm\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index ff0b619e9a90..2c5deb1e1463 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -1235,6 +1235,12 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,
return 0;
}
+ /* $current returns the address of the current task_struct. */
+ if (strcmp(arg, "current") == 0) {
+ code->op = FETCH_OP_CURRENT;
+ return 0;
+ }
+
#ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API
len = str_has_prefix(arg, "arg");
if (len) {
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index b1a54da3c761..f2b31089779c 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -96,6 +96,7 @@ enum fetch_op {
FETCH_OP_FOFFS, /* File offset: .immediate */
FETCH_OP_DATA, /* Allocated data: .data */
FETCH_OP_EDATA, /* Entry data: .offset */
+ FETCH_OP_CURRENT, /* Current task_struct address */
// Stage 2 (dereference) op
FETCH_OP_DEREF, /* Dereference: .offset */
FETCH_OP_UDEREF, /* User-space Dereference: .offset */
diff --git a/kernel/trace/trace_probe_tmpl.h b/kernel/trace/trace_probe_tmpl.h
index f39b37fcdb3b..f630930288d2 100644
--- a/kernel/trace/trace_probe_tmpl.h
+++ b/kernel/trace/trace_probe_tmpl.h
@@ -112,6 +112,9 @@ process_common_fetch_insn(struct fetch_insn *code, unsigned long *val)
case FETCH_OP_DATA:
*val = (unsigned long)code->data;
break;
+ case FETCH_OP_CURRENT:
+ *val = (unsigned long)current;
+ break;
default:
return -EILSEQ;
}
^ permalink raw reply related
* [RFC PATCH 5/7] tracing/probes: Add +CPU() and +PCPU() dereference method to fetcharg
From: Masami Hiramatsu (Google) @ 2026-06-08 14:25 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
When tracing the kernel local variables, sometimes we need to get the
CPU local variables. To access it, current simple dereference is not
enough.
Thus, introduce a special +CPU() dereference to access per-cpu variable
for the current CPU (accessing other CPU variable may race with
updates on other CPUs). Also +PCPU() is for accessing per-cpu pointer.
+CPU(pcp)
is equal to
this_cpu_read(pcp)
And
+PCPU(pcp)
is equal to
this_cpu_ptr(pcp)
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/trace/eprobetrace.rst | 3 ++
Documentation/trace/fprobetrace.rst | 3 ++
Documentation/trace/kprobetrace.rst | 3 ++
kernel/trace/trace.c | 1 +
kernel/trace/trace_probe.c | 48 +++++++++++++++++++++--------------
kernel/trace/trace_probe.h | 2 +
kernel/trace/trace_probe_tmpl.h | 30 ++++++++++++++++++----
7 files changed, 65 insertions(+), 25 deletions(-)
diff --git a/Documentation/trace/eprobetrace.rst b/Documentation/trace/eprobetrace.rst
index dcf92d5b4175..0c7878df02f6 100644
--- a/Documentation/trace/eprobetrace.rst
+++ b/Documentation/trace/eprobetrace.rst
@@ -40,6 +40,9 @@ Synopsis of eprobe_events
$comm : Fetch current task comm.
$current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
+ +CPU(FETCHARG) : Fetch memory at FETCHARG address on the CPU specified by CPU.
+ This is useful for fetching per-CPU variables.
+ +PCPU(FETCHARG) : Fetch memory address at FETCHARG address on the per-CPU area.
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst
index 3392cab016b3..c851f98bb310 100644
--- a/Documentation/trace/fprobetrace.rst
+++ b/Documentation/trace/fprobetrace.rst
@@ -52,6 +52,9 @@ Synopsis of fprobe-events
$comm : Fetch current task comm.
$current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*4)(\*5)
+ +CPU(FETCHARG) : Fetch memory at FETCHARG address on the CPU specified by CPU.
+ This is useful for fetching per-CPU variables.
+ +PCPU(FETCHARG) : Fetch memory address at FETCHARG address on the per-CPU area.
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst
index 81e4fe38791d..bc806fd82a91 100644
--- a/Documentation/trace/kprobetrace.rst
+++ b/Documentation/trace/kprobetrace.rst
@@ -55,6 +55,9 @@ Synopsis of kprobe_events
$comm : Fetch current task comm.
$current : Fetch the address of the current task_struct.
+|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
+ +CPU(FETCHARG) : Fetch memory at FETCHARG address on the CPU specified by CPU.
+ This is useful for fetching per-CPU variables.
+ +PCPU(FETCHARG) : Fetch memory address at FETCHARG address on the per-CPU area.
\IMM : Store an immediate value to the argument.
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index e185a006cb08..2b8c8ac4036a 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4332,6 +4332,7 @@ static const char readme_msg[] =
"\t $stack<index>, $stack, $retval, $comm, $current\n"
#endif
"\t +|-[u]<offset>(<fetcharg>), \\imm-value, \\\"imm-string\"\n"
+ "\t +CPU(<fetcharg>), +PCPU(<fetcharg>)\n"
"\t kernel return probes support: $retval, $arg<N>, $comm\n"
"\t type: s8/16/32/64, u8/16/32/64, x8/16/32/64, char, string, symbol,\n"
"\t b<bit-width>@<bit-offset>/<container-size>, ustring,\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 2c5deb1e1463..fa6757222fe6 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -1396,26 +1396,36 @@ parse_probe_arg(char *arg, const struct fetch_type *type,
case '+': /* deref memory */
case '-':
- if (arg[1] == 'u') {
- deref = FETCH_OP_UDEREF;
- arg[1] = arg[0];
- arg++;
- }
- if (arg[0] == '+')
- arg++; /* Skip '+', because kstrtol() rejects it. */
- tmp = strchr(arg, '(');
- if (!tmp) {
- trace_probe_log_err(ctx->offset, DEREF_NEED_BRACE);
- return -EINVAL;
- }
- *tmp = '\0';
- ret = kstrtol(arg, 0, &offset);
- if (ret) {
- trace_probe_log_err(ctx->offset, BAD_DEREF_OFFS);
- break;
+ if (str_has_prefix(arg, "+CPU(")) {
+ deref = FETCH_OP_DEREF_CPU;
+ arg += 5;
+ ctx->offset += 5;
+ } else if (str_has_prefix(arg, "+PCPU(")) {
+ deref = FETCH_OP_CPU_PTR;
+ arg += 6;
+ ctx->offset += 6;
+ } else {
+ if (arg[1] == 'u') {
+ deref = FETCH_OP_UDEREF;
+ arg[1] = arg[0];
+ arg++;
+ }
+ if (arg[0] == '+')
+ arg++; /* Skip '+', because kstrtol() rejects it. */
+ tmp = strchr(arg, '(');
+ if (!tmp) {
+ trace_probe_log_err(ctx->offset, DEREF_NEED_BRACE);
+ return -EINVAL;
+ }
+ *tmp = '\0';
+ ret = kstrtol(arg, 0, &offset);
+ if (ret) {
+ trace_probe_log_err(ctx->offset, BAD_DEREF_OFFS);
+ break;
+ }
+ ctx->offset += (tmp + 1 - arg) + (arg[0] != '-' ? 1 : 0);
+ arg = tmp + 1;
}
- ctx->offset += (tmp + 1 - arg) + (arg[0] != '-' ? 1 : 0);
- arg = tmp + 1;
tmp = strrchr(arg, ')');
if (!tmp) {
trace_probe_log_err(ctx->offset + strlen(arg),
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index f2b31089779c..bec04bcc4226 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -100,6 +100,8 @@ enum fetch_op {
// Stage 2 (dereference) op
FETCH_OP_DEREF, /* Dereference: .offset */
FETCH_OP_UDEREF, /* User-space Dereference: .offset */
+ FETCH_OP_DEREF_CPU, /* Per-CPU Dereference for this CPU */
+ FETCH_OP_CPU_PTR, /* Per-CPU pointer for this CPU */
// Stage 3 (store) ops
FETCH_OP_ST_RAW, /* Raw: .size */
FETCH_OP_ST_MEM, /* Mem: .offset, .size */
diff --git a/kernel/trace/trace_probe_tmpl.h b/kernel/trace/trace_probe_tmpl.h
index f630930288d2..82d753decf48 100644
--- a/kernel/trace/trace_probe_tmpl.h
+++ b/kernel/trace/trace_probe_tmpl.h
@@ -129,25 +129,43 @@ process_fetch_insn_bottom(struct fetch_insn *code, unsigned long val,
struct fetch_insn *s3 = NULL;
int total = 0, ret = 0, i = 0;
u32 loc = 0;
- unsigned long lval = val;
+ unsigned long lval, llval = val;
stage2:
/* 2nd stage: dereference memory if needed */
do {
- if (code->op == FETCH_OP_DEREF) {
- lval = val;
+ lval = val;
+ switch (code->op) {
+ case FETCH_OP_DEREF:
ret = probe_mem_read(&val, (void *)val + code->offset,
sizeof(val));
- } else if (code->op == FETCH_OP_UDEREF) {
- lval = val;
+ break;
+ case FETCH_OP_UDEREF:
ret = probe_mem_read_user(&val,
(void *)val + code->offset, sizeof(val));
- } else
break;
+ case FETCH_OP_DEREF_CPU:
+ case FETCH_OP_CPU_PTR:
+ if (!is_kernel_percpu_address(val)) {
+ ret = -EFAULT;
+ break;
+ }
+ val = (unsigned long)this_cpu_ptr((void __percpu *)val);
+ if (code->op == FETCH_OP_DEREF_CPU)
+ ret = probe_mem_read(&val, (void *)val, sizeof(val));
+ else
+ ret = 0;
+ break;
+ default:
+ lval = llval;
+ goto out;
+ }
if (ret)
return ret;
+ llval = lval;
code++;
} while (1);
+out:
s3 = code;
stage3:
^ permalink raw reply related
* [RFC PATCH 6/7] tracing/probes: Support reserved this_cpu_ptr() method
From: Masami Hiramatsu (Google) @ 2026-06-08 14:25 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
The +PCPU() dereference operator was introduced in trace probes to
access a per-CPU pointer of a CPU local variable. However, kernel
developers are more familiar with the "this_cpu_ptr()" macro.
To make trace probe syntax more intuitive and aligned with standard
kernel macros, introduce support for "this_cpu_ptr(<fetcharg>)" as a
reserved method.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
kernel/trace/trace.c | 2 +-
kernel/trace/trace_probe.c | 7 +++++++
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 2b8c8ac4036a..60ab839d0867 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4332,7 +4332,7 @@ static const char readme_msg[] =
"\t $stack<index>, $stack, $retval, $comm, $current\n"
#endif
"\t +|-[u]<offset>(<fetcharg>), \\imm-value, \\\"imm-string\"\n"
- "\t +CPU(<fetcharg>), +PCPU(<fetcharg>)\n"
+ "\t +CPU(<fetcharg>), +PCPU(<fetcharg>), this_cpu_ptr(<fetcharg>)\n"
"\t kernel return probes support: $retval, $arg<N>, $comm\n"
"\t type: s8/16/32/64, u8/16/32/64, x8/16/32/64, char, string, symbol,\n"
"\t b<bit-width>@<bit-offset>/<container-size>, ustring,\n"
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index fa6757222fe6..27be0664cdf3 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -1315,6 +1315,7 @@ parse_probe_arg(char *arg, const struct fetch_type *type,
struct fetch_insn **pcode, struct fetch_insn *end,
struct traceprobe_parse_context *ctx)
{
+ static const char *THIS_CPU_PTR_STR = "this_cpu_ptr(";
struct fetch_insn *code = *pcode;
unsigned long param;
int deref = FETCH_OP_DEREF;
@@ -1426,6 +1427,7 @@ parse_probe_arg(char *arg, const struct fetch_type *type,
ctx->offset += (tmp + 1 - arg) + (arg[0] != '-' ? 1 : 0);
arg = tmp + 1;
}
+handle_deref:
tmp = strrchr(arg, ')');
if (!tmp) {
trace_probe_log_err(ctx->offset + strlen(arg),
@@ -1476,6 +1478,11 @@ parse_probe_arg(char *arg, const struct fetch_type *type,
ret = handle_typecast(arg, pcode, end, ctx);
break;
default:
+ if (str_has_prefix(arg, THIS_CPU_PTR_STR)) {
+ arg += strlen(THIS_CPU_PTR_STR);
+ deref = FETCH_OP_CPU_PTR;
+ goto handle_deref;
+ }
if (isalpha(arg[0]) || arg[0] == '_') { /* BTF variable */
if (!tparg_is_function_entry(ctx->flags) &&
!tparg_is_function_return(ctx->flags)) {
^ permalink raw reply related
* [RFC PATCH 7/7] tracing/probes: Add a new testcase for BTF typecasts
From: Masami Hiramatsu (Google) @ 2026-06-08 14:25 UTC (permalink / raw)
To: Steven Rostedt, Mathieu Desnoyers
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, linux-kernel,
linux-trace-kernel, linux-doc, linux-kselftest
In-Reply-To: <178092865666.163648.10457567771536160909.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
With the introduction of container_of-style BTF typecasting and
per-CPU variable access support in trace probes, we need a way to
verify their functionality and prevent regressions.
Add a new ftrace kselftest and update the trace event sample module to
test and validate these features.
Specifically, update the trace-events-sample module to set up a periodic
timer whose callback accesses a per-CPU counter. Introduce a new sample
trace event, foo_timer_fn, to trace this callback and log the current
counter value.
Then, add a new test case, btf_probe_event.tc, which defines a dynamic
probe on the timer callback. The probe uses BTF typecasting to recover
the parent structure from the timer argument and +CPU() to fetch the
per-CPU counter. The test verifies the integrity of the implementation
by ensuring the values recorded by the dynamic probe match those from
the static tracepoint.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
samples/trace_events/trace-events-sample.c | 38 ++++++++++++++-
samples/trace_events/trace-events-sample.h | 34 ++++++++++++-
.../ftrace/test.d/dynevent/btf_probe_event.tc | 52 ++++++++++++++++++++
3 files changed, 120 insertions(+), 4 deletions(-)
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc
diff --git a/samples/trace_events/trace-events-sample.c b/samples/trace_events/trace-events-sample.c
index ecc7db237f2e..770315812218 100644
--- a/samples/trace_events/trace-events-sample.c
+++ b/samples/trace_events/trace-events-sample.c
@@ -94,6 +94,20 @@ static int simple_thread_fn(void *arg)
static DEFINE_MUTEX(thread_mutex);
static int simple_thread_cnt;
+static struct foo_timer_data *foo_timer_data;
+
+static void sample_timer_cb(struct timer_list *t)
+{
+ struct foo_timer_data *data = container_of(t, struct foo_timer_data, timer);
+
+ get_cpu();
+ trace_foo_timer_fn(data);
+ (*this_cpu_ptr(data->counter))++;
+ put_cpu();
+
+ mod_timer(t, jiffies + HZ);
+}
+
int foo_bar_reg(void)
{
mutex_lock(&thread_mutex);
@@ -128,9 +142,27 @@ void foo_bar_unreg(void)
static int __init trace_event_init(void)
{
+ foo_timer_data = kzalloc_obj(*foo_timer_data, GFP_KERNEL);
+ if (!foo_timer_data)
+ return -ENOMEM;
+
+ foo_timer_data->name = "sample_timer_counter";
+ foo_timer_data->counter = alloc_percpu(int);
+ if (!foo_timer_data->counter) {
+ kfree(foo_timer_data);
+ return -ENOMEM;
+ }
+
+ timer_setup(&foo_timer_data->timer, sample_timer_cb, 0);
+ mod_timer(&foo_timer_data->timer, jiffies + HZ);
+
simple_tsk = kthread_run(simple_thread, NULL, "event-sample");
- if (IS_ERR(simple_tsk))
+ if (IS_ERR(simple_tsk)) {
+ timer_delete_sync(&foo_timer_data->timer);
+ free_percpu(foo_timer_data->counter);
+ kfree(foo_timer_data);
return -1;
+ }
return 0;
}
@@ -143,6 +175,10 @@ static void __exit trace_event_exit(void)
kthread_stop(simple_tsk_fn);
simple_tsk_fn = NULL;
mutex_unlock(&thread_mutex);
+
+ timer_delete_sync(&foo_timer_data->timer);
+ free_percpu(foo_timer_data->counter);
+ kfree(foo_timer_data);
}
module_init(trace_event_init);
diff --git a/samples/trace_events/trace-events-sample.h b/samples/trace_events/trace-events-sample.h
index 1a05fc153353..816848a456a2 100644
--- a/samples/trace_events/trace-events-sample.h
+++ b/samples/trace_events/trace-events-sample.h
@@ -247,12 +247,14 @@
*/
/*
- * It is OK to have helper functions in the file, but they need to be protected
- * from being defined more than once. Remember, this file gets included more
- * than once.
+ * It is OK to have helper functions and data structures in the file, but they
+ * need to be protected from being defined more than once. Remember, this file
+ * gets included more than once.
*/
#ifndef __TRACE_EVENT_SAMPLE_HELPER_FUNCTIONS
#define __TRACE_EVENT_SAMPLE_HELPER_FUNCTIONS
+#include <linux/timer.h>
+
static inline int __length_of(const int *list)
{
int i;
@@ -270,6 +272,13 @@ enum {
TRACE_SAMPLE_BAR = 4,
TRACE_SAMPLE_ZOO = 8,
};
+
+struct foo_timer_data {
+ const char *name;
+ struct timer_list timer;
+ int __percpu *counter;
+};
+
#endif
/*
@@ -595,6 +604,25 @@ TRACE_EVENT(foo_rel_loc,
__get_rel_bitmask(bitmask),
__get_rel_cpumask(cpumask))
);
+
+TRACE_EVENT(foo_timer_fn,
+
+ TP_PROTO(struct foo_timer_data *data),
+
+ TP_ARGS(data),
+
+ TP_STRUCT__entry(
+ __string( name, data->name )
+ __field( int, count )
+ ),
+
+ TP_fast_assign(
+ __assign_str(name);
+ __entry->count = *this_cpu_ptr(data->counter);
+ ),
+
+ TP_printk("name=%s count=%d", __get_str(name), __entry->count)
+);
#endif
/***** NOTICE! The #if protection ends here. *****/
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc b/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc
new file mode 100644
index 000000000000..f1980650dbe2
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc
@@ -0,0 +1,52 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: BTF event with typecast and percpu access
+# requires: dynamic_events " +CPU(<fetcharg>)":README "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+# Check if the sample module is loaded
+if ! lsmod | grep -q trace_events_sample; then
+ modprobe trace-events-sample || exit_unsupported
+fi
+
+echo 0 > events/enable
+echo > dynamic_events
+
+# The sample_timer_cb(struct timer_list *t) is called.
+# We want to check (STRUCT,FIELD)VAR typecast and +PCPU() dereference.
+# (foo_timer_data,timer)t converts t to struct foo_timer_data * using container_of.
+# data->counter is a per-cpu pointer to int.
+# +PCPU(data->counter) should give the per-cpu address of the counter.
+# *+PCPU(data->counter) should give the value of the counter.
+
+echo 'f:mysample/myevent sample_timer_cb name=(foo_timer_data,timer)t->name:string count=+CPU((foo_timer_data,timer)t->counter)' >> dynamic_events
+
+echo 1 > events/mysample/myevent/enable
+echo 1 > events/sample-trace/foo_timer_fn/enable
+
+sleep 2
+
+echo 0 > events/mysample/myevent/enable
+echo 0 > events/sample-trace/foo_timer_fn/enable
+
+# Compare the values.
+MATCH=0
+while read line; do
+ if echo $line | grep -q "foo_timer_fn:"; then
+ NAME=`echo $line | sed 's/.*name=\([^ ]*\) .*/\1/'`
+ COUNT=`echo $line | sed 's/.*count=\([^ ]*\).*/\1/'`
+ if grep -q "myevent:.*name=\"${NAME}\" count=$COUNT" trace; then
+ MATCH=$((MATCH+1))
+ fi
+ fi
+done < trace
+
+if [ $MATCH -eq 0 ]; then
+ echo "No matching events found"
+ exit_fail
+fi
+
+# Clean up
+echo 0 > events/mysample/myevent/enable
+echo 0 > events/sample-trace/foo_timer_fn/enable
+echo > dynamic_events
+clear_trace
^ permalink raw reply related
* Re: [PATCH v2 6/6] x86/setup: prepend embedded bootconfig cmdline before parse_early_param
From: Breno Leitao @ 2026-06-08 14:41 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Andrew Morton, Nathan Chancellor, paulmck, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, kernel-team
In-Reply-To: <20260608191928.d7d2dea899b94f05d397f891@kernel.org>
On Mon, Jun 08, 2026 at 07:19:28PM +0900, Masami Hiramatsu wrote:
> On Fri, 05 Jun 2026 05:03:37 -0700
> Breno Leitao <leitao@debian.org> wrote:
>
> > Call xbc_prepend_embedded_cmdline() in setup_arch() right after the
> > CONFIG_CMDLINE merge and before strscpy(command_line, ...) so the
> > build-time-rendered embedded bootconfig "kernel" subtree is part of
> > boot_command_line by the time parse_early_param() runs. early_param()
> > handlers (mem=, earlycon=, loglevel=, ...) now see values supplied via
> > CONFIG_BOOT_CONFIG_EMBED_FILE without parsing bootconfig at runtime.
> >
> > Gate the prepend on the bootconfig opt-in: only fold in the embedded
> > kernel.* keys when "bootconfig" is present on the command line, or
> > CONFIG_BOOT_CONFIG_FORCE is set. Applying the embedded cmdline
> > unconditionally would (a) diverge from how embedded init.* keys are
> > treated and (b) break fail-safe recovery: a malformed embedded
> > console=/mem= could panic the boot with no way for the admin to disable
> > it by dropping "bootconfig" from the bootloader cmdline.
> > cmdline_find_option_bool() runs before parse_early_param(), so the gate
> > is cheap and correctly ordered.
> >
> > Select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG so the user-visible
> > CONFIG_BOOT_CONFIG_EMBED_CMDLINE option becomes selectable on x86.
>
> This seems like a dummy config. what code does depend on this flag?
No C code reads ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG directly — it's
a silent gating symbol, the same ARCH_SUPPORTS_* idiom as
ARCH_SUPPORTS_CFI, ARCH_SUPPORTS_LTO_CLANG, etc.
Its only role is the depends on line of BOOT_CONFIG_EMBED_CMDLINE: an
arch selects it once its setup_arch() calls
xbc_prepend_embedded_cmdline(), and that makes the user-visible
BOOT_CONFIG_EMBED_CMDLINE selectable.
Right now, only x86 supports embedded bootconfig, thus, only x86 does
the following (last patch):
config X86
+ select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG
So, no other platform can see CONFIG_BOOT_CONFIG_EMBED_CMDLINE.
> > --- a/init/main.c
> > +++ b/init/main.c
> > @@ -378,12 +378,15 @@ static void __init setup_boot_config(void)
> > int pos, ret;
> > size_t size;
> > char *err;
> > + bool from_embedded = false;
> >
> > /* Cut out the bootconfig data even if we have no bootconfig option */
> > data = get_boot_config_from_initrd(&size);
> > /* If there is no bootconfig in initrd, try embedded one. */
> > - if (!data)
> > + if (!data) {
> > data = xbc_get_embedded_bootconfig(&size);
> > + from_embedded = true;
>
> Even from embedded bootconfig, if the arch set
> ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG=n, this must be applied to
> the cmdline as we are doing.
Right — that path is preserved. When the arch doesn't select
ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG, BOOT_CONFIG_EMBED_CMDLINE is
unselectable, so xbc_embedded_cmdline_applied() is the no-op stub
returning false.
> > strscpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE);
> > err = parse_args("bootconfig", tmp_cmdline, NULL, 0, 0, 0, NULL,
> > @@ -421,8 +424,17 @@ static void __init setup_boot_config(void)
> > } else {
> > xbc_get_info(&ret, NULL);
> > pr_info("Load bootconfig: %ld bytes %d nodes\n", (long)size, ret);
> > - /* keys starting with "kernel." are passed via cmdline */
> > - extra_command_line = xbc_make_cmdline("kernel");
> > + /*
> > + * keys starting with "kernel." are passed via cmdline. When
> > + * this bootconfig came from the embedded source and
> > + * setup_arch() already prepended the rendered "kernel" subtree
> > + * to boot_command_line, rendering again here would duplicate
> > + * the keys in saved_command_line and make accumulating handlers
> > + * (console=, earlycon=, ...) re-register the same value. Skip
> > + * only when the prepend really happened.
>
> Also, this should mention ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG=n case.
Ack, I will update
Thanks for the review,
--breno
^ permalink raw reply
* Re: [PATCH 2/2] selftests/ftrace: Account for 8-byte aligned trace_marker_raw events
From: Hui Wang @ 2026-06-08 14:51 UTC (permalink / raw)
To: Masami Hiramatsu (Google)
Cc: rostedt, mathieu.desnoyers, pjw, linux-trace-kernel, shuah,
wangfushuai, linux-kselftest
In-Reply-To: <20260608181716.726cb9c81d41d49095e7f3cf@kernel.org>
On 6/8/26 17:17, Masami Hiramatsu (Google) wrote:
> On Sun, 7 Jun 2026 15:24:31 +0800
> Hui Wang <hui.wang@canonical.com> wrote:
>
[...]
> + for config_file in \
> + /boot/config-$uname_r \
> + /lib/modules/$uname_r/config \
> + /lib/modules/$uname_r/build/.config
>
> Hmm, also I don't like this, because this highly depends on the environment.
> Instead, we can add CONFIG_IKCONFIG_PROC=y in tools/testing/selftests/ftrace/config.
>
> Thank you,
>
Thanks for the review. I'll address all other comments in v2.
I have a concern about this specific point. On Ubuntu kernels, both
CONFIG_IKCONFIG and CONFIG_IKCONFIG_PROC are disabled by default, so
/proc/config.gz does not exist. If we drop the /boot/config-$(uname -r)
lookup and rely solely on /proc/config.gz, this test would become
unresolved on every Ubuntu kernel — a regression, since it works on
those kernels today.
There is also existing precedent for the /boot/config-$(uname -r)
fallback: tools/testing/selftests/mm/va_high_addr_switch.sh checks
/proc/config.gz first and falls back to /boot/config-$(uname -r).
So how about we keep /boot/config-$(uname -r) as a fallback, but drop
the /lib/modules/... paths you objected to. And add ftrace/config as you
suggested here.
Thanks,
Hui.
>> + do
>> + if [ -f "$config_file" ]; then
>> + grep -Eq "^${config}=(y|m)$" "$config_file"
>> + return $?
>> + fi
>> + done
>> +
>> + return 2
>> +}
>> +
>> LOCALHOST=127.0.0.1
>>
>> yield() {
>> --
>> 2.43.0
>>
>>
>
^ permalink raw reply
* Re: [PATCH mm-unstable v19 11/14] mm/khugepaged: Introduce mTHP collapse support
From: David Hildenbrand (Arm) @ 2026-06-08 14:56 UTC (permalink / raw)
To: Lance Yang, npache
Cc: linux-doc, linux-kernel, linux-mm, linux-trace-kernel, aarcange,
akpm, anshuman.khandual, apopple, baohua, baolin.wang, byungchul,
catalin.marinas, cl, corbet, dave.hansen, dev.jain, gourry,
hannes, hughd, jack, jackmanb, jannh, jglisse, joshua.hahnjy, kas,
liam, ljs, mathieu.desnoyers, matthew.brost, mhiramat, mhocko,
peterx, pfalcato, rakie.kim, raquini, rdunlap, richard.weiyang,
rientjes, rostedt, rppt, ryan.roberts, shivankg, sunnanyong,
surenb, thomas.hellstrom, tiwai, usamaarif642, vbabka,
vishal.moola, wangkefeng.wang, will, willy, yang, ying.huang, ziy,
zokeefe
In-Reply-To: <20260606102800.26940-1-lance.yang@linux.dev>
On 6/6/26 12:28, Lance Yang wrote:
>
> On Fri, Jun 05, 2026 at 10:14:18AM -0600, Nico Pache wrote:
>> Enable khugepaged to collapse to mTHP orders. This patch implements the
>> main scanning logic using a bitmap to track occupied pages and the
>> algorithm to find optimal collapse sizes.
>>
>> Previous to this patch, PMD collapse had 3 main phases, a light weight
>> scanning phase (mmap_read_lock) that determines a potential PMD
>> collapse, an alloc phase (mmap unlocked), then finally heavier collapse
>> phase (mmap_write_lock).
>>
>> To enabled mTHP collapse we make the following changes:
>>
>> During PMD scan phase, track occupied pages in a bitmap. When mTHP
>> orders are enabled, we remove the restriction of max_ptes_none during the
>> scan phase to avoid missing potential mTHP collapse candidates. Once we
>> have scanned the full PMD range and updated the bitmap to track occupied
>> pages, we use the bitmap to find the optimal mTHP size.
>>
>> Implement mthp_collapse() to walk forward through the bitmap and
>> determine the best eligible order for each naturally-aligned region. The
>> algorithm starts at the beginning of the PMD range and, for each offset,
>> tries the highest order that fits the alignment. If the number of
>> occupied PTEs in that region satisfies the max_ptes_none threshold for
>> that order, a collapse is attempted. On failure, the order is
>> decremented and the same offset is retried at the next smaller size. Once
>> the smallest enabled order is exhausted (or a collapse succeeds), the
>> offset advances past the region just processed, and the next attempt
>> starts at the highest order permitted by the new offset's natural
>> alignment.
>>
>> The algorithm works as follows:
>> 1) set offset=0 and order=HPAGE_PMD_ORDER
>> 2) if the order is not enabled, go to step (5)
>> 3) count occupied PTEs in the (offset, order) range using
>> bitmap_weight_from()
>> 4) if the count satisfies the max_ptes_none threshold, attempt
>> collapse; on success, advance to step (6)
>> 5) if a smaller enabled order exists, decrement order and retry
>> from step (2) at the same offset
>> 6) advance offset past the current region and compute the next
>> order from the new offset's natural alignment via __ffs(offset),
>> capped at HPAGE_PMD_ORDER
>> 7) repeat from step (2) until the full PMD range is covered
>>
>> mTHP collapses reject regions containing swapped out or shared pages.
>> This is because adding new entries can lead to new none pages, and these
>> may lead to constant promotion into a higher order mTHP. A similar
>> issue can occur with "max_ptes_none > HPAGE_PMD_NR/2" due to a collapse
>> introducing at least 2x the number of pages, and on a future scan will
>> satisfy the promotion condition once again. This issue is prevented via
>> the collapse_max_ptes_none() function which imposes the max_ptes_none
>> restrictions above.
>>
>> We currently only support mTHP collapse for max_ptes_none values of 0
>> and HPAGE_PMD_NR - 1. resulting in the following behavior:
>>
>> - max_ptes_none=0: Never introduce new empty pages during collapse
>> - max_ptes_none=HPAGE_PMD_NR-1: Always try collapse to the highest
>> available mTHP order
>>
>> Any other max_ptes_none value will emit a warning and default mTHP
>> collapse to max_ptes_none=0. There should be no behavior change for PMD
>> collapse.
>>
>> Once we determine what mTHP sizes fits best in that PMD range a collapse
>> is attempted. A minimum collapse order of 2 is used as this is the lowest
>> order supported by anon memory as defined by THP_ORDERS_ALL_ANON.
>>
>> Currently madv_collapse is not supported and will only attempt PMD
>> collapse.
>>
>> We can also remove the check for is_khugepaged inside the PMD scan as
>> the collapse_max_ptes_none() function handles this logic now.
>>
>> Signed-off-by: Nico Pache <npache@redhat.com>
>> ---
>> mm/khugepaged.c | 146 +++++++++++++++++++++++++++++++++++++++++++++---
>> 1 file changed, 138 insertions(+), 8 deletions(-)
>>
>> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
>> index ec886a031952..430047316f43 100644
>> --- a/mm/khugepaged.c
>> +++ b/mm/khugepaged.c
>> @@ -99,6 +99,8 @@ static DEFINE_READ_MOSTLY_HASHTABLE(mm_slots_hash, MM_SLOTS_HASH_BITS);
>>
>> static struct kmem_cache *mm_slot_cache __ro_after_init;
>>
>> +#define KHUGEPAGED_MIN_MTHP_ORDER 2
>> +
>> struct collapse_control {
>> bool is_khugepaged;
>>
>> @@ -110,6 +112,9 @@ struct collapse_control {
>>
>> /* nodemask for allocation fallback */
>> nodemask_t alloc_nmask;
>> +
>> + /* Each bit represents a single occupied (!none/zero) page. */
>> + DECLARE_BITMAP(mthp_present_ptes, MAX_PTRS_PER_PTE);
>> };
>>
>> /**
>> @@ -1440,20 +1445,130 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s
>> return result;
>> }
>>
>> +/* Return the highest naturally aligned order that fits at @offset within a PMD. */
>> +static unsigned int max_order_from_offset(unsigned int offset)
>> +{
>> + if (offset == 0)
>> + return HPAGE_PMD_ORDER;
>> +
>> + return min_t(unsigned int, __ffs(offset), HPAGE_PMD_ORDER);
>> +}
>> +
>> +/*
>> + * mthp_collapse() consumes the bitmap that is generated during
>> + * collapse_scan_pmd() to determine what regions and mTHP orders fit best.
>> + *
>> + * Each bit in cc->mthp_present_ptes represents a single occupied (!none/zero)
>> + * page. We start at the PMD order and check if it is eligible for collapse;
>> + * if not, we check the left and right halves of the PTE page table we are
>> + * examining at a lower order.
>> + *
>> + * For each of these, we determine how many PTE entries are occupied in the
>> + * range of PTE entries we propose to collapse, then we compare this to a
>> + * threshold number of PTE entries which would need to be occupied for a
>> + * collapse to be permitted at that order (accounting for max_ptes_none).
>> + *
>> + * If a collapse is permitted, we attempt to collapse the PTE range into a
>> + * mTHP.
>> + */
>> +static enum scan_result mthp_collapse(struct mm_struct *mm,
>> + unsigned long address, int referenced, int unmapped,
>> + struct collapse_control *cc, unsigned long enabled_orders)
>> +{
>> + unsigned int nr_occupied_ptes, nr_ptes, max_ptes_none;
>> + enum scan_result last_result = SCAN_FAIL;
>> + int collapsed = 0;
>> + bool alloc_failed = false;
>> + unsigned long collapse_address;
>> + unsigned int offset = 0;
>> + unsigned int order = HPAGE_PMD_ORDER;
>> +
>> + while (offset < HPAGE_PMD_NR) {
>> + nr_ptes = 1UL << order;
>> +
>> + if (!test_bit(order, &enabled_orders))
>> + goto next_order;
>> +
>> + max_ptes_none = collapse_max_ptes_none(cc, NULL, order);
>> + nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
>> + offset + nr_ptes);
>> +
>> + if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
>
> Looks broken for swap PTEs in PMD collapse ...
>
> collapse_scan_pmd() allows them up to max_ptes_swap and record them in
> unmapped, but they don't get a bit in mthp_present_ptes. And then
> mthp_collapse() does the check above:
Right. I assumed this is implicitly handled by the optimization in collapse_scan_pmd:
if (enabled_orders != BIT(HPAGE_PMD_ORDER))
max_ptes_none = KHUGEPAGED_MAX_PTES_LIMIT;
But we perform the check a second time.
>
> nr_occupied_ptes >= nr_ptes - max_ptes_none
>
> So max_ptes_none=0 + 511 present PTEs + one allowed swap PTE won't even
> call collapse_huge_page() for PMD order.
>
> Shouldn't we account for them in the PMD-order check? Something like:
>
> if (is_pmd_order(order))
> nr_occupied_ptes += unmapped;
As an alternative, we could either 1) skip the check there for
pmd order (as the check was already done); or 2) introduce+maintain
a bitmap that tracks non-present PTEs.
@@ -1475,7 +1477,9 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
offset + nr_ptes);
- if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
+ /* Check was already done in the caller. */
+ if (is_pmd_order(order) ||
+ nr_occupied_ptes >= nr_ptes - max_ptes_none) {
enum scan_result ret;
collapse_address = address + offset * PAGE_SIZE;
2) would probably be cleanest long-term.
--
Cheers,
David
^ permalink raw reply
* [PATCH] tracing: Add "within" filter for call-stack-based event filtering
From: Chen Jun @ 2026-06-08 14:55 UTC (permalink / raw)
To: rostedt, mhiramat, mathieu.desnoyers, linux-kernel,
linux-trace-kernel
Cc: chenjun102
Low-level kernel functions are called from many different paths.
When debugging, it is often useful to filter trace events to only
those occurring within a specific call chain.
Add a "within" filter predicate that tests whether a given function
appears in the current call stack at event time. The function name
is resolved to its address range via kallsyms during filter setup;
at runtime, stack_trace_save() captures the call stack and compares
each return address against the stored range.
Example:
echo 'within == "vfs_read"' > events/sched/sched_switch/filter
Only "==" and "!=" operators are supported. The filter depends on
CONFIG_STACKTRACE.
Signed-off-by: Chen Jun <chenjun102@huawei.com>
---
Documentation/trace/events.rst | 12 +++++++++
include/linux/trace_events.h | 1 +
kernel/trace/trace.h | 3 ++-
kernel/trace/trace_events.c | 3 +++
kernel/trace/trace_events_filter.c | 41 ++++++++++++++++++++++++++++--
5 files changed, 57 insertions(+), 3 deletions(-)
diff --git a/Documentation/trace/events.rst b/Documentation/trace/events.rst
index 18d112963dec..6e3877d376a9 100644
--- a/Documentation/trace/events.rst
+++ b/Documentation/trace/events.rst
@@ -243,6 +243,18 @@ the function "security_prepare_creds" and less than the end of that function.
The ".function" postfix can only be attached to values of size long, and can only
be compared with "==" or "!=".
+The special field "within" can be used to filter events based on whether
+a specific function appears in the current call stack::
+
+ within == "function_name"
+ within != "function_name"
+
+For example, to only trace events where "vfs_read" is in the call stack::
+
+ # echo 'within == "vfs_read"' > events/sched/sched_switch/filter
+
+The within field supports only the "==" and "!=" operators.
+
Cpumask fields or scalar fields that encode a CPU number can be filtered using
a user-provided cpumask in cpulist format. The format is as follows::
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 40a43a4c7caf..9ed22c210add 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -851,6 +851,7 @@ enum {
FILTER_COMM,
FILTER_CPU,
FILTER_STACKTRACE,
+ FILTER_WITHIN,
};
extern int trace_event_raw_init(struct trace_event_call *call);
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 80fe152af1dd..a383da42badf 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1825,7 +1825,8 @@ static inline bool is_string_field(struct ftrace_event_field *field)
field->filter_type == FILTER_RDYN_STRING ||
field->filter_type == FILTER_STATIC_STRING ||
field->filter_type == FILTER_PTR_STRING ||
- field->filter_type == FILTER_COMM;
+ field->filter_type == FILTER_COMM ||
+ field->filter_type == FILTER_WITHIN;
}
static inline bool is_function_field(struct ftrace_event_field *field)
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index c46e623e7e0d..b7d681e55b0c 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -199,6 +199,9 @@ static int trace_define_generic_fields(void)
__generic_field(char *, comm, FILTER_COMM);
__generic_field(char *, stacktrace, FILTER_STACKTRACE);
__generic_field(char *, STACKTRACE, FILTER_STACKTRACE);
+#ifdef CONFIG_STACKTRACE
+ __generic_field(char *, within, FILTER_WITHIN);
+#endif
return ret;
}
diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c
index 609325f57942..34e1a7f0b3cd 100644
--- a/kernel/trace/trace_events_filter.c
+++ b/kernel/trace/trace_events_filter.c
@@ -72,6 +72,7 @@ enum filter_pred_fn {
FILTER_PRED_FN_CPUMASK,
FILTER_PRED_FN_CPUMASK_CPU,
FILTER_PRED_FN_FUNCTION,
+ FILTER_PRED_FN_WITHIN,
FILTER_PRED_FN_,
FILTER_PRED_TEST_VISITED,
};
@@ -1009,6 +1010,22 @@ static int filter_pred_function(struct filter_pred *pred, void *event)
return pred->op == OP_EQ ? ret : !ret;
}
+/* Filter predicate for within. */
+static int filter_pred_within(struct filter_pred *pred, void *event)
+{
+#ifdef CONFIG_STACKTRACE
+ unsigned long entries[16];
+ unsigned int nr_entries;
+ int i;
+
+ nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0);
+ for (i = 0; i < nr_entries; i++)
+ if (pred->val <= entries[i] && entries[i] < pred->val2)
+ return !pred->not;
+#endif
+ return pred->not;
+}
+
/*
* regex_match_foo - Basic regex callbacks
*
@@ -1617,6 +1634,8 @@ static int filter_pred_fn_call(struct filter_pred *pred, void *event)
return filter_pred_cpumask_cpu(pred, event);
case FILTER_PRED_FN_FUNCTION:
return filter_pred_function(pred, event);
+ case FILTER_PRED_FN_WITHIN:
+ return filter_pred_within(pred, event);
case FILTER_PRED_TEST_VISITED:
return test_pred_visited_fn(pred, event);
default:
@@ -2002,10 +2021,28 @@ static int parse_pred(const char *str, void *data,
} else if (field->filter_type == FILTER_DYN_STRING) {
pred->fn_num = FILTER_PRED_FN_STRLOC;
- } else if (field->filter_type == FILTER_RDYN_STRING)
+ } else if (field->filter_type == FILTER_RDYN_STRING) {
pred->fn_num = FILTER_PRED_FN_STRRELLOC;
- else {
+ } else if (field->filter_type == FILTER_WITHIN) {
+ unsigned long func;
+
+ if (op == OP_GLOB)
+ goto err_free;
+ pred->fn_num = FILTER_PRED_FN_WITHIN;
+ func = kallsyms_lookup_name(pred->regex->pattern);
+ if (!func) {
+ parse_error(pe, FILT_ERR_NO_FUNCTION, pos + i);
+ goto err_free;
+ }
+ /* Now find the function start and end address */
+ if (!kallsyms_lookup_size_offset(func, &size, &offset)) {
+ parse_error(pe, FILT_ERR_NO_FUNCTION, pos + i);
+ goto err_free;
+ }
+ pred->val = func - offset;
+ pred->val2 = pred->val + size;
+ } else {
if (!ustring_per_cpu) {
/* Once allocated, keep it around for good */
ustring_per_cpu = alloc_percpu(struct ustring_buffer);
--
2.43.0
^ permalink raw reply related
* [PATCH v3 1/6] bootconfig: fix NULL-pointer arithmetic in xbc_snprint_cmdline()
From: Breno Leitao @ 2026-06-08 16:23 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
xbc_snprint_cmdline() is meant to be called twice: first with
buf=NULL, size=0 to probe the rendered length, then with a real
buffer to fill it (the standard snprintf() two-pass pattern). The
probe call makes the function compute "buf + size" (NULL + 0) and,
on every iteration, advance "buf += ret" from that NULL base and
pass the result back into snprintf().
Pointer arithmetic on a NULL pointer is undefined behavior. It is
harmless in the in-kernel callers today, but the follow-up patches
run this same code in the userspace tools/bootconfig parser at kernel
build time, where host UBSan / FORTIFY_SOURCE abort the build.
Track a running written length (size_t) instead of mutating @buf, and
only form "buf + len" when @buf is non-NULL. snprintf(NULL, 0, ...)
is itself well defined and returns the would-be length, so the
two-pass "probe then fill" usage returns identical byte counts.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
lib/bootconfig.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/lib/bootconfig.c b/lib/bootconfig.c
index f445b7703fdd..2ed9ee3dc81c 100644
--- a/lib/bootconfig.c
+++ b/lib/bootconfig.c
@@ -427,10 +427,18 @@ static char xbc_namebuf[XBC_KEYLEN_MAX] __initdata;
int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root)
{
struct xbc_node *knode, *vnode;
- char *end = buf + size;
const char *val, *q;
+ size_t len = 0;
int ret;
+ /*
+ * Track the running written length rather than advancing @buf, so we
+ * never form "buf + size" or "buf += ret" while @buf is NULL (the
+ * size-probe call passes buf=NULL, size=0). NULL pointer arithmetic
+ * is undefined behavior and trips host UBSan / FORTIFY_SOURCE when
+ * this renderer runs at kernel build time. snprintf(NULL, 0, ...)
+ * itself is well defined and returns the would-be length.
+ */
xbc_node_for_each_key_value(root, knode, val) {
ret = xbc_node_compose_key_after(root, knode,
xbc_namebuf, XBC_KEYLEN_MAX);
@@ -439,10 +447,11 @@ int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root)
vnode = xbc_node_get_child(knode);
if (!vnode) {
- ret = snprintf(buf, rest(buf, end), "%s ", xbc_namebuf);
+ ret = snprintf(buf ? buf + len : NULL, rest(len, size),
+ "%s ", xbc_namebuf);
if (ret < 0)
return ret;
- buf += ret;
+ len += ret;
continue;
}
xbc_array_for_each_value(vnode, val) {
@@ -452,15 +461,15 @@ int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root)
* whitespace.
*/
q = strpbrk(val, " \t\r\n") ? "\"" : "";
- ret = snprintf(buf, rest(buf, end), "%s=%s%s%s ",
- xbc_namebuf, q, val, q);
+ ret = snprintf(buf ? buf + len : NULL, rest(len, size),
+ "%s=%s%s%s ", xbc_namebuf, q, val, q);
if (ret < 0)
return ret;
- buf += ret;
+ len += ret;
}
}
- return buf - (end - size);
+ return len;
}
#undef rest
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v3 0/6] bootconfig: embed kernel.* cmdline at build time
From: Breno Leitao @ 2026-06-08 16:23 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
The userspace pieces (xbc_snprint_cmdline() in lib/, tools/bootconfig -C)
already landed; this series wires the rendered cmdline into the kernel.
Motivation: today the embedded bootconfig is parsed at runtime, after
parse_early_param() has already run, so early_param() handlers can't
see embedded values. Folding the kernel.* subtree into the cmdline at
build time gives a CONFIG_CMDLINE-equivalent for embedded-bootconfig
users without forcing them to maintain two cmdline sources.
Behaviorally, the "kernel" subtree is rendered to a flat string at
build time and stashed in .init.rodata. setup_arch() prepends it to
boot_command_line before parse_early_param() runs. Overflow is a soft
error: the helper logs and leaves boot_command_line untouched rather
than panicking, so an oversized embedded bconf cannot brick a boot.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
Changes in v3:
- Patch 3: Move HOSTCC override to the kernel-side rule; tool keeps
$(CC) for standalone/cross builds.
- Patch 6: Drop the false fail-safe wording; document the
BOOT_CONFIG_FORCE=y default interaction.
- Link to v2:
https://lore.kernel.org/r/20260605-bootconfig_using_tools-v2-0-d309f544b5f7@debian.org
Changes in v2 (addressing review of v1):
- Split out a standalone fix for the NULL-pointer arithmetic in
xbc_snprint_cmdline() so the build-time render cannot trip host
UBSan/FORTIFY_SOURCE.
- Rework the leaf-root handling: instead of returning early, skip @root
inside the loop so a root carrying both a value and subkeys
(kernel = x together with kernel.foo = bar) still renders its
descendant keys.
- Build tools/bootconfig with $(HOSTCC) so cross-compiled (ARCH=...)
builds render the cmdline on the build host instead of failing with
"Exec format error".
- Mark the embedded cmdline section read-only (drop the "w" flag from
.init.rodata).
- Add a make-clean hook so tools/bootconfig artifacts are removed by
make clean.
- Gate the x86 prepend on "bootconfig" being present on the command
line (or CONFIG_BOOT_CONFIG_FORCE), matching the init.* opt-in
semantics documented in bootconfig.rst and preserving fail-safe
recovery: dropping "bootconfig" from the bootloader cmdline now also
disables the embedded kernel.* keys.
- Link to v1: https://patch.msgid.link/20260527-bootconfig_using_tools-v1-0-b6906a86e7d5@debian.org
---
Breno Leitao (6):
bootconfig: fix NULL-pointer arithmetic in xbc_snprint_cmdline()
bootconfig: render descendant keys when xbc_snprint_cmdline() root has a value
bootconfig: render embedded bootconfig as a kernel cmdline at build time
bootconfig: clean build-time tools/bootconfig from make clean
bootconfig: add xbc_prepend_embedded_cmdline() helper
x86/setup: prepend embedded bootconfig cmdline before parse_early_param
MAINTAINERS | 1 +
Makefile | 24 +++++++++-
arch/x86/Kconfig | 1 +
arch/x86/kernel/setup.c | 16 +++++++
include/linux/bootconfig.h | 9 ++++
init/Kconfig | 36 +++++++++++++++
init/main.c | 25 ++++++++--
lib/Makefile | 16 +++++++
lib/bootconfig.c | 112 ++++++++++++++++++++++++++++++++++++++++++---
lib/embedded-cmdline.S | 16 +++++++
tools/bootconfig/Makefile | 4 +-
11 files changed, 247 insertions(+), 13 deletions(-)
---
base-commit: e7e28506af98ce4e1059e5ec59334b335c00a246
change-id: 20260508-bootconfig_using_tools-cfa7aa9d6a5a
Best regards,
--
Breno Leitao <leitao@debian.org>
^ permalink raw reply
* [PATCH v3 2/6] bootconfig: render descendant keys when xbc_snprint_cmdline() root has a value
From: Breno Leitao @ 2026-06-08 16:23 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
xbc_node_for_each_key_value() walks to the first leaf under @root, and
when @root is itself a leaf it yields @root. That happens not only for
an empty "kernel {}" subtree, but also when @root carries both a value
and subkeys, e.g.
kernel = x
kernel.foo = bar
Here @root ("kernel") is a leaf because its first child is the value
node "x", so the iterator returns @root first. Feeding @root back into
xbc_node_compose_key_after(root, root) returns -EINVAL, which the only
in-kernel caller papers over with a "len <= 0" check -- but the
follow-up tools/bootconfig -C user propagates the error and turns such
a bootconfig into a build failure. Worse, short-circuiting the whole
call on a leaf @root would silently drop the valid "kernel.foo = bar"
descendant that the pre-existing code rendered.
Skip @root inside the loop instead of bailing out: the value-only entry
is dropped (it is rendered through the "kernel" cmdline path, not here),
while real descendant keys are still emitted. An entirely empty subtree
now renders nothing and returns 0 rather than -EINVAL, matching the
"nothing to render is not an error" semantics expected by the new
build-time caller.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
lib/bootconfig.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/lib/bootconfig.c b/lib/bootconfig.c
index 2ed9ee3dc81c..926094d97397 100644
--- a/lib/bootconfig.c
+++ b/lib/bootconfig.c
@@ -440,6 +440,17 @@ int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root)
* itself is well defined and returns the would-be length.
*/
xbc_node_for_each_key_value(root, knode, val) {
+ /*
+ * An empty or value-only @root (e.g. "kernel {}" or
+ * "kernel = x", possibly alongside "kernel.foo = bar")
+ * yields @root itself here. Skip it: composing a key for it
+ * would fail with -EINVAL, yet any real descendant keys must
+ * still be rendered. An entirely empty subtree then renders
+ * nothing and returns 0 rather than an error.
+ */
+ if (knode == root)
+ continue;
+
ret = xbc_node_compose_key_after(root, knode,
xbc_namebuf, XBC_KEYLEN_MAX);
if (ret < 0)
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v3 3/6] bootconfig: render embedded bootconfig as a kernel cmdline at build time
From: Breno Leitao @ 2026-06-08 16:24 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
Add the build-time pipeline that renders the "kernel" subtree of
CONFIG_BOOT_CONFIG_EMBED_FILE into a flat cmdline string and stashes
it in .init.rodata as embedded_kernel_cmdline[]. A follow-up patch
adds the runtime helper that prepends this string to boot_command_line
during early architecture setup so parse_early_param() sees the values.
The build wires up:
tools/bootconfig -C kernel - userspace tool already shared with
lib/bootconfig.c, used here in -C mode
to render a bootconfig file to a cmdline
lib/embedded-cmdline.S - .incbin's the rendered text plus a NUL
(listed under the EXTRA BOOT CONFIG
MAINTAINERS entry)
lib/Makefile rule - runs tools/bootconfig at build time
Makefile prepare dep - ensures tools/bootconfig is built first,
same pattern as tools/objtool and
tools/bpf/resolve_btfids
Drop the test target from tools/bootconfig/Makefile's default 'all'
recipe so that hooking the binary into the kernel build does not run
test-bootconfig.sh on every prepare. The tests stay available as
'make -C tools/bootconfig test', matching the convention of
tools/objtool and tools/bpf/resolve_btfids whose 'all' targets only
build the binary.
Require BOOT_CONFIG_EMBED_FILE to be non-empty before the new option
can be enabled, otherwise tools/bootconfig -C runs against an empty
file and prints a parse error on every kernel build.
The feature gates on CONFIG_ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG, a
silent symbol arches select once they've wired the prepend call into
setup_arch(). No arch selects it in this patch, so the user-visible
CONFIG_BOOT_CONFIG_EMBED_CMDLINE is not yet enableable; when an arch
later opts in, the runtime behavior is added by the follow-up patches.
tools/bootconfig also installs on target systems, so its own Makefile
keeps $(CC) and stays cross-buildable as a standalone tool. The kernel
build, which runs the tool on the build host during prepare, instead
forces CC=$(HOSTCC) from a dedicated tools/bootconfig rule, so the
executed binary is always a host binary -- plain $(CC) would
cross-compile it under ARCH=... and fail to exec ("Exec format error").
embedded-cmdline.S places the rendered string in .init.rodata with the
"a" (allocatable, read-only) flag and %progbits, not "aw": the data is
never written at runtime, so it must not land in a writable section.
A follow-up patch wires the build-time tools/bootconfig into the
top-level clean target.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
MAINTAINERS | 1 +
Makefile | 11 +++++++++++
init/Kconfig | 36 ++++++++++++++++++++++++++++++++++++
lib/Makefile | 16 ++++++++++++++++
lib/embedded-cmdline.S | 16 ++++++++++++++++
tools/bootconfig/Makefile | 2 +-
6 files changed, 81 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 4087b67bbc69..fb9314cbe344 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9845,6 +9845,7 @@ F: fs/proc/bootconfig.c
F: include/linux/bootconfig.h
F: lib/bootconfig-data.S
F: lib/bootconfig.c
+F: lib/embedded-cmdline.S
F: tools/bootconfig/*
F: tools/bootconfig/scripts/*
diff --git a/Makefile b/Makefile
index d59f703f9797..4a8ea7c90ca8 100644
--- a/Makefile
+++ b/Makefile
@@ -1543,6 +1543,17 @@ prepare: tools/bpf/resolve_btfids
endif
endif
+# tools/bootconfig renders the embedded bootconfig into a cmdline at build time.
+ifdef CONFIG_BOOT_CONFIG_EMBED_CMDLINE
+prepare: tools/bootconfig
+endif
+
+# tools/bootconfig is run on the build host during prepare, so force a host
+# binary here; its own Makefile keeps $(CC) for standalone and cross builds.
+tools/bootconfig: FORCE
+ $(Q)mkdir -p $(objtree)/tools
+ $(Q)$(MAKE) O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ bootconfig CC=$(HOSTCC)
+
# The tools build system is not a part of Kbuild and tends to introduce
# its own unique issues. If you need to integrate a new tool into Kbuild,
# please consider locating that tool outside the tools/ tree and using the
diff --git a/init/Kconfig b/init/Kconfig
index ca35184532dc..203b1187fde7 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1569,6 +1569,42 @@ config BOOT_CONFIG_EMBED_FILE
This bootconfig will be used if there is no initrd or no other
bootconfig in the initrd.
+config ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG
+ bool
+ help
+ Silent symbol; no C code reads it directly. Architectures
+ select it once their setup_arch() calls
+ xbc_prepend_embedded_cmdline() before parse_early_param().
+ Its only role is to gate the user-visible
+ BOOT_CONFIG_EMBED_CMDLINE option per-arch, the same
+ ARCH_SUPPORTS_* idiom used by ARCH_SUPPORTS_CFI, etc.
+
+config BOOT_CONFIG_EMBED_CMDLINE
+ bool "Render embedded bootconfig as kernel cmdline at build time"
+ depends on BOOT_CONFIG_EMBED
+ depends on BOOT_CONFIG_EMBED_FILE != ""
+ depends on ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG
+ default n
+ help
+ Render the "kernel" subtree of the embedded bootconfig file into a
+ flat cmdline string at kernel build time and prepend it to
+ boot_command_line during early architecture setup. This makes
+ early_param() handlers (e.g. mem=, earlycon=, loglevel=) see the
+ values supplied via the embedded bootconfig.
+
+ The runtime bootconfig parser is unaffected, so tree-structured
+ consumers such as ftrace boot-time tracing keep working.
+
+ Note: when an initrd also carries a bootconfig, its "kernel"
+ subtree is still parsed at runtime, but the embedded "kernel"
+ keys remain in boot_command_line for parse_early_param() and
+ end up later than the initrd keys in saved_command_line, so
+ parse_args() last-wins favors the embedded values. If you need
+ initrd to override embedded kernel.* keys, leave this option
+ off.
+
+ If unsure, say N.
+
config CMDLINE_LOG_WRAP_IDEAL_LEN
int "Length to try to wrap the cmdline when logged at boot"
default 1021
diff --git a/lib/Makefile b/lib/Makefile
index 6e72d2c1cce7..9de0ac7732a2 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -273,6 +273,22 @@ filechk_defbconf = cat $(or $(real-prereqs), /dev/null)
$(obj)/default.bconf: $(CONFIG_BOOT_CONFIG_EMBED_FILE) FORCE
$(call filechk,defbconf)
+obj-$(CONFIG_BOOT_CONFIG_EMBED_CMDLINE) += embedded-cmdline.o
+$(obj)/embedded-cmdline.o: $(obj)/embedded_cmdline.bin
+
+# Render the bootconfig "kernel" subtree to a flat cmdline string using
+# the userspace tools/bootconfig parser (-C mode). The runtime prepend
+# helper enforces COMMAND_LINE_SIZE at boot, so no build-time size
+# check is performed here (COMMAND_LINE_SIZE is an arch header
+# constant, not a Kconfig value).
+quiet_cmd_render_cmdline = BCONF2C $@
+ cmd_render_cmdline = \
+ $(objtree)/tools/bootconfig/bootconfig -C $< > $@
+
+targets += embedded_cmdline.bin
+$(obj)/embedded_cmdline.bin: $(obj)/default.bconf $(objtree)/tools/bootconfig/bootconfig FORCE
+ $(call if_changed,render_cmdline)
+
obj-$(CONFIG_RBTREE_TEST) += rbtree_test.o
obj-$(CONFIG_INTERVAL_TREE_TEST) += interval_tree_test.o
diff --git a/lib/embedded-cmdline.S b/lib/embedded-cmdline.S
new file mode 100644
index 000000000000..740d7ad2dc01
--- /dev/null
+++ b/lib/embedded-cmdline.S
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Embed the build-time-rendered bootconfig "kernel" subtree as a flat
+ * cmdline string. setup_arch() prepends this to boot_command_line on
+ * architectures that select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG.
+ *
+ * Copyright (c) 2026 Meta Platforms, Inc. and affiliates
+ * Copyright (c) 2026 Breno Leitao <leitao@debian.org>
+ */
+ .section .init.rodata, "a", %progbits
+ .global embedded_kernel_cmdline
+embedded_kernel_cmdline:
+ .incbin "lib/embedded_cmdline.bin"
+ .byte 0
+ .global embedded_kernel_cmdline_end
+embedded_kernel_cmdline_end:
diff --git a/tools/bootconfig/Makefile b/tools/bootconfig/Makefile
index 90eb47c9d8de..4e82fd9553cd 100644
--- a/tools/bootconfig/Makefile
+++ b/tools/bootconfig/Makefile
@@ -15,7 +15,7 @@ override CFLAGS += -Wall -g -I$(CURDIR)/include
ALL_TARGETS := bootconfig
ALL_PROGRAMS := $(patsubst %,$(OUTPUT)%,$(ALL_TARGETS))
-all: $(ALL_PROGRAMS) test
+all: $(ALL_PROGRAMS)
$(OUTPUT)bootconfig: main.c include/linux/bootconfig.h $(LIBSRC)
$(CC) $(filter %.c,$^) $(CFLAGS) $(LDFLAGS) -o $@
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v3 4/6] bootconfig: clean build-time tools/bootconfig from make clean
From: Breno Leitao @ 2026-06-08 16:24 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
The previous patch builds tools/bootconfig during 'make prepare' to
render the embedded bootconfig cmdline, but nothing removes it on
'make clean', leaving the compiled tool and its objects behind.
Wire a bootconfig_clean hook into the top-level clean target so the
compiled tool and its objects are removed by make clean, matching the
prepare-wired tools/objtool and tools/bpf/resolve_btfids.
The hook runs tools/bootconfig's Makefile via $(MAKE), which the kernel
build invokes with -rR (MAKEFLAGS += -rR). -rR drops the built-in $(RM)
variable, so the existing "$(RM) -f ..." clean recipe would expand to a
bare "-f ..." and fail. Spell the recipe with a literal "rm -f" so it
keeps working both standalone and when invoked from Kbuild.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
Makefile | 13 ++++++++++++-
tools/bootconfig/Makefile | 2 +-
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index 4a8ea7c90ca8..84ca047f0c10 100644
--- a/Makefile
+++ b/Makefile
@@ -1580,6 +1580,17 @@ ifneq ($(wildcard $(objtool_O)),)
$(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) $(patsubst objtool_%,%,$@)
endif
+PHONY += bootconfig_clean
+
+bootconfig_O = $(abspath $(objtree))/tools/bootconfig
+
+# tools/bootconfig is only built (via the prepare hook above) when
+# CONFIG_BOOT_CONFIG_EMBED_CMDLINE is set; skip its clean otherwise.
+bootconfig_clean:
+ifneq ($(wildcard $(bootconfig_O)),)
+ $(Q)$(MAKE) -sC $(srctree)/tools/bootconfig O=$(bootconfig_O) clean
+endif
+
tools/: FORCE
$(Q)mkdir -p $(objtree)/tools
$(Q)$(MAKE) O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/
@@ -1749,7 +1760,7 @@ vmlinuxclean:
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/link-vmlinux.sh clean
$(Q)$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) clean)
-clean: archclean vmlinuxclean resolve_btfids_clean objtool_clean
+clean: archclean vmlinuxclean resolve_btfids_clean objtool_clean bootconfig_clean
# mrproper - Delete all generated files, including .config
#
diff --git a/tools/bootconfig/Makefile b/tools/bootconfig/Makefile
index 4e82fd9553cd..3cb8066d5141 100644
--- a/tools/bootconfig/Makefile
+++ b/tools/bootconfig/Makefile
@@ -27,4 +27,4 @@ install: $(ALL_PROGRAMS)
install $(OUTPUT)bootconfig $(DESTDIR)$(bindir)
clean:
- $(RM) -f $(OUTPUT)*.o $(ALL_PROGRAMS)
+ rm -f $(OUTPUT)*.o $(ALL_PROGRAMS)
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v3 5/6] bootconfig: add xbc_prepend_embedded_cmdline() helper
From: Breno Leitao @ 2026-06-08 16:24 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
Add a helper that prepends the build-time-rendered embedded bootconfig
"kernel" subtree (embedded_kernel_cmdline[] from embedded-cmdline.S) to
a cmdline buffer with a separating space. Architectures call this from
setup_arch() before parse_early_param() so early_param() handlers
(mem=, earlycon=, loglevel=, ...) see values supplied via the embedded
bootconfig.
The in-place prepend (shift the existing string right, then drop the
embedded string in front) is factored into a small str_prepend() helper.
On overflow the helper logs an error and leaves the cmdline untouched
rather than panicking. Booting without the embedded values is better
than refusing to boot, and the error tells the user why their embedded
keys are missing.
The helper records whether it actually prepended, exposed via
xbc_embedded_cmdline_applied(). setup_boot_config() uses this to decide
whether the runtime "kernel" render would duplicate keys already folded
into boot_command_line.
When CONFIG_BOOT_CONFIG_EMBED_CMDLINE=n, the public declaration in
<linux/bootconfig.h> resolves to a no-op stub so callers compile
unchanged.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
include/linux/bootconfig.h | 9 ++++++
lib/bootconfig.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 87 insertions(+)
diff --git a/include/linux/bootconfig.h b/include/linux/bootconfig.h
index 1c7f3b74ffcf..c186137f87ac 100644
--- a/include/linux/bootconfig.h
+++ b/include/linux/bootconfig.h
@@ -308,4 +308,13 @@ static inline const char *xbc_get_embedded_bootconfig(size_t *size)
}
#endif
+/* Build-time-rendered bootconfig cmdline prepended in setup_arch() */
+#ifdef CONFIG_BOOT_CONFIG_EMBED_CMDLINE
+void __init xbc_prepend_embedded_cmdline(char *dst, size_t size);
+bool __init xbc_embedded_cmdline_applied(void);
+#else
+static inline void xbc_prepend_embedded_cmdline(char *dst, size_t size) { }
+static inline bool xbc_embedded_cmdline_applied(void) { return false; }
+#endif
+
#endif
diff --git a/lib/bootconfig.c b/lib/bootconfig.c
index 926094d97397..f66be0b2dc24 100644
--- a/lib/bootconfig.c
+++ b/lib/bootconfig.c
@@ -19,6 +19,7 @@
#include <linux/errno.h>
#include <linux/cache.h>
#include <linux/compiler.h>
+#include <linux/printk.h>
#include <linux/sprintf.h>
#include <linux/memblock.h>
#include <linux/string.h>
@@ -34,6 +35,83 @@ const char * __init xbc_get_embedded_bootconfig(size_t *size)
return (*size) ? embedded_bootconfig_data : NULL;
}
#endif
+
+#ifdef CONFIG_BOOT_CONFIG_EMBED_CMDLINE
+/* embedded_kernel_cmdline is defined in embedded-cmdline.S */
+extern __visible const char embedded_kernel_cmdline[];
+extern __visible const char embedded_kernel_cmdline_end[];
+
+/* Set once the embedded cmdline has actually been prepended. */
+static bool xbc_cmdline_applied __initdata;
+
+/*
+ * str_prepend() - Prepend @src in front of the string in @dst, in place
+ * @dst: NUL-terminated destination buffer, currently @dst_len bytes long
+ * @dst_len: length of the current @dst string (excluding its NUL)
+ * @src: bytes to prepend (not NUL-terminated)
+ * @src_len: number of bytes from @src to prepend
+ *
+ * The caller must guarantee @dst has room for src_len + dst_len + 1 bytes.
+ * Moving dst_len + 1 bytes carries @dst's NUL terminator too, so an empty
+ * @dst needs no special case.
+ */
+static void __init str_prepend(char *dst, size_t dst_len,
+ const char *src, size_t src_len)
+{
+ memmove(dst + src_len, dst, dst_len + 1);
+ memcpy(dst, src, src_len);
+}
+
+/**
+ * xbc_prepend_embedded_cmdline() - Prepend embedded bootconfig cmdline
+ * @dst: cmdline buffer to prepend into (must already contain a NUL byte)
+ * @size: total capacity of @dst in bytes
+ *
+ * Prepend the build-time-rendered "kernel" subtree of the embedded
+ * bootconfig to @dst. The rendered string already ends with a single
+ * space (the xbc_snprint_cmdline() invariant), which serves as the
+ * separator between the embedded keys and any existing content of @dst.
+ * On overflow, log an error and leave @dst untouched rather than
+ * silently truncating: booting without the embedded values is better
+ * than refusing to boot, and the error message tells the user why
+ * their embedded keys are missing.
+ *
+ * Intended to be called from setup_arch() before parse_early_param() so
+ * that early_param() handlers see the embedded values.
+ */
+void __init xbc_prepend_embedded_cmdline(char *dst, size_t size)
+{
+ size_t embed_len = embedded_kernel_cmdline_end - embedded_kernel_cmdline;
+ size_t dst_len;
+
+ if (!size || embed_len <= 1) /* trailing NUL only */
+ return;
+ embed_len--; /* exclude trailing NUL byte */
+
+ dst_len = strnlen(dst, size);
+ if (embed_len + dst_len + 1 > size) {
+ pr_err("embedded bootconfig cmdline (%zu bytes) does not fit in COMMAND_LINE_SIZE with %zu bytes already used; ignoring embedded values\n",
+ embed_len, dst_len);
+ return;
+ }
+
+ str_prepend(dst, dst_len, embedded_kernel_cmdline, embed_len);
+ xbc_cmdline_applied = true;
+}
+
+/**
+ * xbc_embedded_cmdline_applied() - Did the embedded cmdline get prepended?
+ *
+ * Return true if xbc_prepend_embedded_cmdline() actually prepended the
+ * embedded "kernel" subtree. setup_boot_config() uses this to avoid
+ * rendering the same keys a second time.
+ */
+bool __init xbc_embedded_cmdline_applied(void)
+{
+ return xbc_cmdline_applied;
+}
+#endif
+
#endif
/*
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v3 6/6] x86/setup: prepend embedded bootconfig cmdline before parse_early_param
From: Breno Leitao @ 2026-06-08 16:24 UTC (permalink / raw)
To: Masami Hiramatsu, Andrew Morton, Nathan Chancellor, paulmck,
Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, linux-kernel, linux-trace-kernel, linux-kbuild,
bpf, Breno Leitao, kernel-team
In-Reply-To: <20260608-bootconfig_using_tools-v3-0-4ddd079a0696@debian.org>
Call xbc_prepend_embedded_cmdline() in setup_arch() right after the
CONFIG_CMDLINE merge and before strscpy(command_line, ...) so the
build-time-rendered embedded bootconfig "kernel" subtree is part of
boot_command_line by the time parse_early_param() runs. early_param()
handlers (mem=, earlycon=, loglevel=, ...) now see values supplied via
CONFIG_BOOT_CONFIG_EMBED_FILE without parsing bootconfig at runtime.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
arch/x86/Kconfig | 1 +
arch/x86/kernel/setup.c | 16 ++++++++++++++++
init/main.c | 25 ++++++++++++++++++++++---
3 files changed, 39 insertions(+), 3 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f24810015234..f839795692b4 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -126,6 +126,7 @@ config X86
select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096
select ARCH_SUPPORTS_CFI if X86_64
+ select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG
select ARCH_USES_CFI_TRAPS if X86_64 && CFI
select ARCH_SUPPORTS_LTO_CLANG
select ARCH_SUPPORTS_LTO_CLANG_THIN
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 46882ce79c3a..003f8651db6c 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -6,6 +6,7 @@
* parts of early kernel initialization.
*/
#include <linux/acpi.h>
+#include <linux/bootconfig.h>
#include <linux/console.h>
#include <linux/cpu.h>
#include <linux/crash_dump.h>
@@ -36,6 +37,7 @@
#include <asm/bios_ebda.h>
#include <asm/bugs.h>
#include <asm/cacheinfo.h>
+#include <asm/cmdline.h>
#include <asm/coco.h>
#include <asm/cpu.h>
#include <asm/efi.h>
@@ -924,6 +926,20 @@ void __init setup_arch(char **cmdline_p)
builtin_cmdline_added = true;
#endif
+ /*
+ * Match the runtime bootconfig parser's opt-in: only fold the
+ * embedded kernel.* keys into the cmdline when "bootconfig" is
+ * present on the command line, or CONFIG_BOOT_CONFIG_FORCE is set.
+ * setup_boot_config() bails out under the same condition, so the
+ * early prepend stays in lockstep with what the late runtime parser
+ * would have applied. CONFIG_BOOT_CONFIG_FORCE defaults to y when
+ * BOOT_CONFIG_EMBED is set, so on the default config the embedded
+ * keys are applied unconditionally.
+ */
+ if (IS_ENABLED(CONFIG_BOOT_CONFIG_FORCE) ||
+ cmdline_find_option_bool(boot_command_line, "bootconfig"))
+ xbc_prepend_embedded_cmdline(boot_command_line, COMMAND_LINE_SIZE);
+
strscpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
*cmdline_p = command_line;
diff --git a/init/main.c b/init/main.c
index e363232b428b..2ecb6aa536dd 100644
--- a/init/main.c
+++ b/init/main.c
@@ -378,12 +378,15 @@ static void __init setup_boot_config(void)
int pos, ret;
size_t size;
char *err;
+ bool from_embedded = false;
/* Cut out the bootconfig data even if we have no bootconfig option */
data = get_boot_config_from_initrd(&size);
/* If there is no bootconfig in initrd, try embedded one. */
- if (!data)
+ if (!data) {
data = xbc_get_embedded_bootconfig(&size);
+ from_embedded = true;
+ }
strscpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE);
err = parse_args("bootconfig", tmp_cmdline, NULL, 0, 0, 0, NULL,
@@ -421,8 +424,24 @@ static void __init setup_boot_config(void)
} else {
xbc_get_info(&ret, NULL);
pr_info("Load bootconfig: %ld bytes %d nodes\n", (long)size, ret);
- /* keys starting with "kernel." are passed via cmdline */
- extra_command_line = xbc_make_cmdline("kernel");
+ /*
+ * keys starting with "kernel." are passed via cmdline. When
+ * this bootconfig came from the embedded source and
+ * setup_arch() already prepended the rendered "kernel" subtree
+ * to boot_command_line, rendering again here would duplicate
+ * the keys in saved_command_line and make accumulating handlers
+ * (console=, earlycon=, ...) re-register the same value. Skip
+ * only when the prepend really happened.
+ *
+ * On arches that do not select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG,
+ * CONFIG_BOOT_CONFIG_EMBED_CMDLINE is unselectable and
+ * xbc_embedded_cmdline_applied() collapses to a stub returning
+ * false, so this path still runs and the embedded "kernel"
+ * keys reach the cmdline via the runtime parser exactly as
+ * before this series.
+ */
+ if (!from_embedded || !xbc_embedded_cmdline_applied())
+ extra_command_line = xbc_make_cmdline("kernel");
/* Also, "init." keys are init arguments */
extra_init_args = xbc_make_cmdline("init");
}
--
2.53.0-Meta
^ permalink raw reply related
* Re: [PATCH mm-unstable v19 11/14] mm/khugepaged: Introduce mTHP collapse support
From: Lance Yang @ 2026-06-08 16:26 UTC (permalink / raw)
To: david
Cc: lance.yang, npache, linux-doc, linux-kernel, linux-mm,
linux-trace-kernel, aarcange, akpm, anshuman.khandual, apopple,
baohua, baolin.wang, byungchul, catalin.marinas, cl, corbet,
dave.hansen, dev.jain, gourry, hannes, hughd, jack, jackmanb,
jannh, jglisse, joshua.hahnjy, kas, liam, ljs, mathieu.desnoyers,
matthew.brost, mhiramat, mhocko, peterx, pfalcato, rakie.kim,
raquini, rdunlap, richard.weiyang, rientjes, rostedt, rppt,
ryan.roberts, shivankg, sunnanyong, surenb, thomas.hellstrom,
tiwai, usamaarif642, vbabka, vishal.moola, wangkefeng.wang, will,
willy, yang, ying.huang, ziy, zokeefe
In-Reply-To: <2553caae-9e0e-42a7-8b61-d1216f1e81fa@kernel.org>
On Mon, Jun 08, 2026 at 04:56:37PM +0200, David Hildenbrand (Arm) wrote:
>On 6/6/26 12:28, Lance Yang wrote:
>>
>> On Fri, Jun 05, 2026 at 10:14:18AM -0600, Nico Pache wrote:
>>> Enable khugepaged to collapse to mTHP orders. This patch implements the
>>> main scanning logic using a bitmap to track occupied pages and the
>>> algorithm to find optimal collapse sizes.
>>>
>>> Previous to this patch, PMD collapse had 3 main phases, a light weight
>>> scanning phase (mmap_read_lock) that determines a potential PMD
>>> collapse, an alloc phase (mmap unlocked), then finally heavier collapse
>>> phase (mmap_write_lock).
>>>
>>> To enabled mTHP collapse we make the following changes:
>>>
>>> During PMD scan phase, track occupied pages in a bitmap. When mTHP
>>> orders are enabled, we remove the restriction of max_ptes_none during the
>>> scan phase to avoid missing potential mTHP collapse candidates. Once we
>>> have scanned the full PMD range and updated the bitmap to track occupied
>>> pages, we use the bitmap to find the optimal mTHP size.
>>>
>>> Implement mthp_collapse() to walk forward through the bitmap and
>>> determine the best eligible order for each naturally-aligned region. The
>>> algorithm starts at the beginning of the PMD range and, for each offset,
>>> tries the highest order that fits the alignment. If the number of
>>> occupied PTEs in that region satisfies the max_ptes_none threshold for
>>> that order, a collapse is attempted. On failure, the order is
>>> decremented and the same offset is retried at the next smaller size. Once
>>> the smallest enabled order is exhausted (or a collapse succeeds), the
>>> offset advances past the region just processed, and the next attempt
>>> starts at the highest order permitted by the new offset's natural
>>> alignment.
>>>
>>> The algorithm works as follows:
>>> 1) set offset=0 and order=HPAGE_PMD_ORDER
>>> 2) if the order is not enabled, go to step (5)
>>> 3) count occupied PTEs in the (offset, order) range using
>>> bitmap_weight_from()
>>> 4) if the count satisfies the max_ptes_none threshold, attempt
>>> collapse; on success, advance to step (6)
>>> 5) if a smaller enabled order exists, decrement order and retry
>>> from step (2) at the same offset
>>> 6) advance offset past the current region and compute the next
>>> order from the new offset's natural alignment via __ffs(offset),
>>> capped at HPAGE_PMD_ORDER
>>> 7) repeat from step (2) until the full PMD range is covered
>>>
>>> mTHP collapses reject regions containing swapped out or shared pages.
>>> This is because adding new entries can lead to new none pages, and these
>>> may lead to constant promotion into a higher order mTHP. A similar
>>> issue can occur with "max_ptes_none > HPAGE_PMD_NR/2" due to a collapse
>>> introducing at least 2x the number of pages, and on a future scan will
>>> satisfy the promotion condition once again. This issue is prevented via
>>> the collapse_max_ptes_none() function which imposes the max_ptes_none
>>> restrictions above.
>>>
>>> We currently only support mTHP collapse for max_ptes_none values of 0
>>> and HPAGE_PMD_NR - 1. resulting in the following behavior:
>>>
>>> - max_ptes_none=0: Never introduce new empty pages during collapse
>>> - max_ptes_none=HPAGE_PMD_NR-1: Always try collapse to the highest
>>> available mTHP order
>>>
>>> Any other max_ptes_none value will emit a warning and default mTHP
>>> collapse to max_ptes_none=0. There should be no behavior change for PMD
>>> collapse.
>>>
>>> Once we determine what mTHP sizes fits best in that PMD range a collapse
>>> is attempted. A minimum collapse order of 2 is used as this is the lowest
>>> order supported by anon memory as defined by THP_ORDERS_ALL_ANON.
>>>
>>> Currently madv_collapse is not supported and will only attempt PMD
>>> collapse.
>>>
>>> We can also remove the check for is_khugepaged inside the PMD scan as
>>> the collapse_max_ptes_none() function handles this logic now.
>>>
>>> Signed-off-by: Nico Pache <npache@redhat.com>
>>> ---
>>> mm/khugepaged.c | 146 +++++++++++++++++++++++++++++++++++++++++++++---
>>> 1 file changed, 138 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
>>> index ec886a031952..430047316f43 100644
>>> --- a/mm/khugepaged.c
>>> +++ b/mm/khugepaged.c
>>> @@ -99,6 +99,8 @@ static DEFINE_READ_MOSTLY_HASHTABLE(mm_slots_hash, MM_SLOTS_HASH_BITS);
>>>
>>> static struct kmem_cache *mm_slot_cache __ro_after_init;
>>>
>>> +#define KHUGEPAGED_MIN_MTHP_ORDER 2
>>> +
>>> struct collapse_control {
>>> bool is_khugepaged;
>>>
>>> @@ -110,6 +112,9 @@ struct collapse_control {
>>>
>>> /* nodemask for allocation fallback */
>>> nodemask_t alloc_nmask;
>>> +
>>> + /* Each bit represents a single occupied (!none/zero) page. */
>>> + DECLARE_BITMAP(mthp_present_ptes, MAX_PTRS_PER_PTE);
>>> };
>>>
>>> /**
>>> @@ -1440,20 +1445,130 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s
>>> return result;
>>> }
>>>
>>> +/* Return the highest naturally aligned order that fits at @offset within a PMD. */
>>> +static unsigned int max_order_from_offset(unsigned int offset)
>>> +{
>>> + if (offset == 0)
>>> + return HPAGE_PMD_ORDER;
>>> +
>>> + return min_t(unsigned int, __ffs(offset), HPAGE_PMD_ORDER);
>>> +}
>>> +
>>> +/*
>>> + * mthp_collapse() consumes the bitmap that is generated during
>>> + * collapse_scan_pmd() to determine what regions and mTHP orders fit best.
>>> + *
>>> + * Each bit in cc->mthp_present_ptes represents a single occupied (!none/zero)
>>> + * page. We start at the PMD order and check if it is eligible for collapse;
>>> + * if not, we check the left and right halves of the PTE page table we are
>>> + * examining at a lower order.
>>> + *
>>> + * For each of these, we determine how many PTE entries are occupied in the
>>> + * range of PTE entries we propose to collapse, then we compare this to a
>>> + * threshold number of PTE entries which would need to be occupied for a
>>> + * collapse to be permitted at that order (accounting for max_ptes_none).
>>> + *
>>> + * If a collapse is permitted, we attempt to collapse the PTE range into a
>>> + * mTHP.
>>> + */
>>> +static enum scan_result mthp_collapse(struct mm_struct *mm,
>>> + unsigned long address, int referenced, int unmapped,
>>> + struct collapse_control *cc, unsigned long enabled_orders)
>>> +{
>>> + unsigned int nr_occupied_ptes, nr_ptes, max_ptes_none;
>>> + enum scan_result last_result = SCAN_FAIL;
>>> + int collapsed = 0;
>>> + bool alloc_failed = false;
>>> + unsigned long collapse_address;
>>> + unsigned int offset = 0;
>>> + unsigned int order = HPAGE_PMD_ORDER;
>>> +
>>> + while (offset < HPAGE_PMD_NR) {
>>> + nr_ptes = 1UL << order;
>>> +
>>> + if (!test_bit(order, &enabled_orders))
>>> + goto next_order;
>>> +
>>> + max_ptes_none = collapse_max_ptes_none(cc, NULL, order);
>>> + nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
>>> + offset + nr_ptes);
>>> +
>>> + if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
>>
>> Looks broken for swap PTEs in PMD collapse ...
>>
>> collapse_scan_pmd() allows them up to max_ptes_swap and record them in
>> unmapped, but they don't get a bit in mthp_present_ptes. And then
>> mthp_collapse() does the check above:
>
>Right. I assumed this is implicitly handled by the optimization in collapse_scan_pmd:
>
> if (enabled_orders != BIT(HPAGE_PMD_ORDER))
> max_ptes_none = KHUGEPAGED_MAX_PTES_LIMIT;
>
>But we perform the check a second time.
Note that once lower orders are enabled, the scan *relaxes* max_ptes_none
only so it can cover the whole PMD and build the bitmap ...
>>
>> nr_occupied_ptes >= nr_ptes - max_ptes_none
>>
>> So max_ptes_none=0 + 511 present PTEs + one allowed swap PTE won't even
>> call collapse_huge_page() for PMD order.
>>
>> Shouldn't we account for them in the PMD-order check? Something like:
>>
>> if (is_pmd_order(order))
>> nr_occupied_ptes += unmapped;
>As an alternative, we could either 1) skip the check there for
>pmd order (as the check was already done); or 2) introduce+maintain
Yeah, skipping the check would do the trick, since isolate will check
max_ptes_none again later :)
>a bitmap that tracks non-present PTEs.
>
>@@ -1475,7 +1477,9 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
> nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
> offset + nr_ptes);
>
>- if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
>+ /* Check was already done in the caller. */
This check is not quite redundant for PMD order, though. It avoids
entering collapse_huge_page() for a range that already exceeds
max_ptes_none for that order.
>+ if (is_pmd_order(order) ||
>+ nr_occupied_ptes >= nr_ptes - max_ptes_none) {
> enum scan_result ret;
>
> collapse_address = address + offset * PAGE_SIZE;
>
>2) would probably be cleanest long-term.
Yeah, Agreed.
^ permalink raw reply
* Re: [PATCH 2/2] selftests/ftrace: Account for 8-byte aligned trace_marker_raw events
From: Steven Rostedt @ 2026-06-08 16:50 UTC (permalink / raw)
To: Hui Wang
Cc: mhiramat, mathieu.desnoyers, pjw, linux-trace-kernel, shuah,
wangfushuai, linux-kselftest
In-Reply-To: <20260607072431.125633-3-hui.wang@canonical.com>
On Sun, 7 Jun 2026 15:24:31 +0800
Hui Wang <hui.wang@canonical.com> wrote:
> trace_marker_raw.tc assumes that the raw marker payload length
> reported in trace_pipe is the result of int((id + 3) / 4) * 4, but
> that is not true on kernels with CONFIG_HAVE_64BIT_ALIGNED_ACCESS
> enabled.
>
> With forced 8-byte alignment, the ring buffer event forces 8-byte
> alignment. The event length is stored in array[0], the payload data
> and id are placed in a struct raw_data_entry which is stored starting
> at array[1]. In this case, the printed payload data length is 8*N+4
> bytes.
>
> To make the testcase pass in this case, add a kconfig_enabled() helper
> and use it to detect CONFIG_HAVE_64BIT_ALIGNED_ACCESS so
> trace_marker_raw.tc can calculate the expected length correctly.
>
> Assisted-by: Copilot:gpt-5.5
> Signed-off-by: Hui Wang <hui.wang@canonical.com>
NACK
Let's not change the kernel for a broken test. Also this has already
been fixed but appears not to be applied yet.
Shuah, can you please apply the below fix.
https://lore.kernel.org/all/20260601023251.1916483-1-dtcccc@linux.alibaba.com/
-- Steve
^ permalink raw reply
* Re: [PATCH 1/2] ring-buffer: Fix event length with forced 8-byte alignment
From: Steven Rostedt @ 2026-06-08 16:52 UTC (permalink / raw)
To: Masami Hiramatsu (Google)
Cc: Hui Wang, mathieu.desnoyers, pjw, linux-trace-kernel, shuah,
wangfushuai, linux-kselftest
In-Reply-To: <20260608180245.09e083867a7d4d96058d7323@kernel.org>
On Mon, 8 Jun 2026 18:02:45 +0900
Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
> On Sun, 7 Jun 2026 15:24:30 +0800
> Hui Wang <hui.wang@canonical.com> wrote:
>
> > When RB_FORCE_8BYTE_ALIGNMENT is true, rb_calculate_event_length()
> > reserves the space of event->array[0] for placing the data length and
> > rb_update_event() stores the data length in event->array[0]
> > accordingly. As a result the whole event length will add extra 4 bytes
> > for sizeof(event.array[0]) unconditionally.
> >
> > But ring_buffer_event_length() only subtracts the
> > sizeof(event->array[0]) for events larger than RB_MAX_SMALL_DATA +
> > sizeof(event->array[0]). As a result, small events on architectures
> > with RB_FORCE_8BYTE_ALIGNMENT=true report a data length that is 4
> > bytes larger than expected.
> >
> > To fix it, add the RB_FORCE_8BYTE_ALIGNMENT as a condition to subtract
> > the size of that length field whenever RB_FORCE_8BYTE_ALIGNMENT is
> > true.
> >
> > This issue is observed in a riscv64 kernel with
> > CONFIG_HAVE_64BIT_ALIGNED_ACCESS set to y, when we run ftrace selftest
> > trace_marker_raw.tc, we get the weird log: for cases where the id is
> > 1..100, the number of data field is 8*N, but once id exceeds 100, the
> > number of data field becomes 8*N+4:
> > # 1 buf: 58 00 00 00 80 5e d1 63 (number of data field is 8*1)
> > ...
> > # a buf: 58 ... (number of data field is 8*2)
> > ...
> > # 64 buf: 58 ... (number of data field is 8*13)
> > # 65 buf: 58 ... (number of data field is 8*13+4)
> >
> > After applying this change, the number of data field keeps being 8*N+4
> > consistently.
> >
>
> Good catch!
>
> This looks good to me.
>
> Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
This is the patch I meant to reply to.
NACK as the test is broken and not the kernel.
There's a pending fix already:
https://lore.kernel.org/all/20260601023251.1916483-1-dtcccc@linux.alibaba.com/
-- Steve
^ permalink raw reply
* Re: [PATCHv4 05/13] uprobes/x86: Move optimized uprobe from nop5 to nop10
From: Andrii Nakryiko @ 2026-06-08 20:46 UTC (permalink / raw)
To: Jiri Olsa
Cc: Oleg Nesterov, Peter Zijlstra, Ingo Molnar, Masami Hiramatsu,
Andrii Nakryiko, bpf, linux-trace-kernel
In-Reply-To: <20260526205840.173790-6-jolsa@kernel.org>
On Tue, May 26, 2026 at 1:59 PM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Andrii reported an issue with optimized uprobes [1] that can clobber
> redzone area with call instruction storing return address on stack
> where user code may keep temporary data without adjusting rsp.
>
> Fixing this by moving the optimized uprobes on top of 10-bytes nop
> instruction, so we can squeeze another instruction to escape the
> redzone area before doing the call, like:
>
> lea -0x80(%rsp), %rsp
> call tramp
>
> Note the lea instruction is used to adjust the rsp register without
> changing the flags.
>
> We use nop10 and following transformation to optimized instructions
> above and back as suggested by Peterz [2].
>
> Optimize path (int3_update_optimize):
>
> 1) Initial state after set_swbp() installed the uprobe:
> cc 2e 0f 1f 84 00 00 00 00 00
>
> From offset 0 this is INT3 followed by the tail of the original
> 10-byte NOP.
>
> After a previous unoptimization bytes 5..9 may still contain the
> old call instruction, which remains valid for threads already there.
>
> 2) Rewrite the LEA tail and call displacement:
> cc [8d 64 24 80 e8 d0 d1 d2 d3]
>
> From offset 0 this traps on the uprobe INT3. Bytes 1..9 are not
> executable entry points while byte 0 is trapped.
>
> 3) Publish the first LEA byte:
> [48] 8d 64 24 80 e8 d0 d1 d2 d3
>
> From offset 0 this is:
> lea -0x80(%rsp), %rsp
> call <uprobe-trampoline>
>
> Unoptimize path (int3_update_unoptimize):
>
> 1) Initial optimized state:
> 48 8d 64 24 80 e8 d0 d1 d2 d3
> Same as 3) above.
>
> 2) Trap new entries before restoring the NOP bytes:
> [cc] 8d 64 24 80 e8 d0 d1 d2 d3
>
> From offset 0 this traps. A thread that had already executed the
> LEA can still reach the intact CALL at offset 5.
>
> 3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped
> and byte 5 as CALL.
> cc [2e 0f 1f 84] e8 d0 d1 d2 d3
>
> From offset 0 this still traps. Offset 5 is still the CALL for any
> thread that was already past the first LEA byte.
>
> 4) Publish the first byte of the original NOP:
> [66] 2e 0f 1f 84 e8 d0 d1 d2 d3
>
> From offset 0 this is the restored 10-byte NOP; the CALL opcode and
> displacement are now only NOP operands. Offset 5 still decodes as
> CALL for a thread that was already there.
>
> Tthere is only a single target uprobe-trampoline for the given nop10
> instruction address, so the CALL instruction will not be changed across
> unoptimization/optimization cycles.
> Therefore, any task that is preempted at the CALL instruction is guaranteed
> to observe that CALL and not anything else.
>
> Note as explained in [2] we need to use following nop10:
> PF1 PF2 ESC NOPL MOD SIB DISP32
> NOP10: 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00 -- cs nopw 0x00000000(%rax,%rax,1)
>
> which means we need to allow 0x2e prefix which maps to INAT_PFX_CS
> attribute in is_prefix_bad function.
>
> Also changing the uprobe syscall error when called out of uprobe
> trampoline to -EPROTO, so we are able to detect the fixed kernel.
>
> The optimized uprobe performance stays the same:
>
> uprobe-nop : 3.129 ± 0.013M/s
> uprobe-push : 3.045 ± 0.006M/s
> uprobe-ret : 1.095 ± 0.004M/s
> --> uprobe-nop10 : 7.170 ± 0.020M/s
> uretprobe-nop : 2.143 ± 0.021M/s
> uretprobe-push : 2.090 ± 0.000M/s
> uretprobe-ret : 0.942 ± 0.000M/s
> --> uretprobe-nop10: 3.381 ± 0.003M/s
> usdt-nop : 3.245 ± 0.004M/s
> --> usdt-nop10 : 7.256 ± 0.023M/s
>
> [1] https://lore.kernel.org/bpf/20260509003146.976844-1-andrii@kernel.org/
> [2] https://lore.kernel.org/bpf/20260518104306.GU3102624@noisy.programming.kicks-ass.net/#t
> Reported-by: Andrii Nakryiko <andrii@kernel.org>
> Closes: https://lore.kernel.org/bpf/20260509003146.976844-1-andrii@kernel.org/
> Fixes: ba2bfc97b462 ("uprobes/x86: Add support to optimize uprobes")
> Assisted-by: Codex:GPT-5.5
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
> arch/x86/kernel/uprobes.c | 255 ++++++++++++++++++++++++++++----------
> 1 file changed, 190 insertions(+), 65 deletions(-)
>
[...]
> @@ -943,13 +1026,31 @@ static int int3_update(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
> smp_text_poke_sync_each_cpu();
>
> /*
> - * Write first byte.
> + * 3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped
> + * and byte 5 as CALL:
> + * cc [2e 0f 1f 84] e8 d0 d1 d2 d3
> + */
> + ctx.expect = EXPECT_SWBP_OPTIMIZED;
> + err = uprobe_write(auprobe, vma, vaddr + 1, insn + 1,
> + LEA_INSN_SIZE - 1, verify_insn,
> + true /* is_register */, false /* do_update_ref_ctr */,
tbh, it's quite subtle and non-obvious why is_register should be set
to true first two times (and especially that is_register and
do_update_ref_ctr are implicitly connected), not sure how to make it
cleaner, but maybe leave a short comment explaining this twice
register, once unregister sequence?
> + &ctx);
> + if (err)
> + return err;
> +
> + smp_text_poke_sync_each_cpu();
[...]
^ permalink raw reply
* Re: [PATCHv4 00/13] uprobes/x86: Fix red zone issue for optimized uprobes
From: Andrii Nakryiko @ 2026-06-08 20:48 UTC (permalink / raw)
To: Jiri Olsa, Peter Zijlstra, Ingo Molnar, Masami Hiramatsu
Cc: Oleg Nesterov, Andrii Nakryiko, bpf, linux-trace-kernel
In-Reply-To: <aiEiP54zktDqAZpG@krava>
On Wed, Jun 3, 2026 at 11:59 PM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Tue, May 26, 2026 at 10:58:27PM +0200, Jiri Olsa wrote:
> > hi,
> > Andrii reported an issue with optimized uprobes [1] that can clobber
> > redzone area with call instruction storing return address on stack
> > where user code may keep temporary data without adjusting rsp.
> >
> > Fixing this by moving the optimized uprobes on top of 10-bytes nop
> > instruction, so we can squeeze another instruction to escape the
> > redzone area before doing the call.
> >
> > Note we need upstream update first for patch 3 (github.com/libbpf/usdt),
> > if we decide to take this change.
> >
> > thanks,
> > jirka
> >
> >
> > v1: https://lore.kernel.org/bpf/20260514135342.22130-1-jolsa@kernel.org/
> > v2: https://lore.kernel.org/bpf/20260518105957.123445-1-jolsa@kernel.org/
> > v3: https://lore.kernel.org/bpf/20260521124411.31133-1-jolsa@kernel.org/
> >
> > v4 changes:
> > - do not use 2nd int3 (ont +5 offset) because the call instruction
> > is allways the same for the given nop10 address [Andrii/Peter]
> > - unmap unused trampoline vma after unsuccesfull optimization [sashiko]
> > - small change to patch#2 moved user_64bit_mode earlier in the path
> > and pass/use mm_struct pointer directly from arch_uprobe_optimize
> > instead of gettting current->mm
> > Andrii, keeping your ack, please shout otherwise
>
> hi,
> I think bots did not find anything substantial, I have just small
> selftests changes queued for v5
>
> any other feedback/review would be great
>
one small nit on only, otherwise LGTM.
Peter, Masami, Ingo, should this go through tip tree or should we
route this through bpf-next tree? I think we are fine either way, but
might be more convenient to route through bpf-next given libbpf and
BPF selftest changes.
If so, I'd appreciate another look at first 5 patches by Peter, if
that's ok. Thanks!
> thanks,
> jirka
>
>
> >
> > v3 changes:
> > - use nop10 update suggested by Peter in [2]
> > - remove struct uprobe_trampoline object, use vma objects directly instead
> > - selftests fixes [sashiko]
> > - ack from Andrii
> >
> > v2 changes:
> > - several selftest fixes [sashiko]
> > - consolidate is_lea_insn and is_call_insn insto single check [Jakub Sitnicki]
> > - use proper mm_struct object in __in_uprobe_trampoline check [sashiko]
> > - allow to copy uprobe trampolines vma objects on fork [sashiko]
> > - change uprobe syscall detection error from -ENXIO to -EPROTO [Andrii]
> > - added fork/clone tests
> > - I kept the selftest changes and nop5->nop10 changes in separate
> > commits for easier review, we can squash them later if we want to keep
> > bisect working properly
> >
> >
> > [1] https://lore.kernel.org/bpf/20260509003146.976844-1-andrii@kernel.org/
> > [2] https://lore.kernel.org/bpf/20260518104306.GU3102624@noisy.programming.kicks-ass.net/#t
> > ---
> > Andrii Nakryiko (1):
> > selftests/bpf: Add tests for uprobe nop10 red zone clobbering
> >
> > Jiri Olsa (12):
> > uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
> > uprobes/x86: Remove struct uprobe_trampoline object
> > uprobes/x86: Allow to copy uprobe trampolines on fork
> > uprobes/x86: Unmap trampoline vma object in case it's unused
> > uprobes/x86: Move optimized uprobe from nop5 to nop10
> > libbpf: Change has_nop_combo to work on top of nop10
> > libbpf: Detect uprobe syscall with new error
> > selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch
> > selftests/bpf: Change uprobe syscall tests to use nop10
> > selftests/bpf: Change uprobe/usdt trigger bench code to use nop10
> > selftests/bpf: Add reattach tests for uprobe syscall
> > selftests/bpf: Add tests for forked/cloned optimized uprobes
> >
> > arch/x86/kernel/uprobes.c | 379 +++++++++++++++++++++++++++++++++++++++++++-----------------------------
> > include/linux/uprobes.h | 5 -
> > kernel/events/uprobes.c | 10 --
> > kernel/fork.c | 1 -
> > tools/lib/bpf/features.c | 4 +-
> > tools/lib/bpf/usdt.c | 16 +--
> > tools/testing/selftests/bpf/bench.c | 20 ++--
> > tools/testing/selftests/bpf/benchs/bench_trigger.c | 38 ++++----
> > tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh | 2 +-
> > tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 307 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
> > tools/testing/selftests/bpf/prog_tests/usdt.c | 74 ++++++++++++--
> > tools/testing/selftests/bpf/progs/test_usdt.c | 25 +++++
> > tools/testing/selftests/bpf/usdt.h | 2 +-
> > tools/testing/selftests/bpf/usdt_2.c | 15 ++-
> > 14 files changed, 653 insertions(+), 245 deletions(-)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox