* Re: [PATCH v6 0/3] tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast
From: Steven Rostedt @ 2026-01-27 2:39 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, bpf
Cc: Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Paul E. McKenney, Sebastian Andrzej Siewior, Alexei Starovoitov
In-Reply-To: <20260126231145.728172709@kernel.org>
On Mon, 26 Jan 2026 18:11:45 -0500
Steven Rostedt <rostedt@kernel.org> wrote:
> The current use of guard(preempt_notrace)() within __DECLARE_TRACE()
> to protect invocation of __DO_TRACE_CALL() means that BPF programs
> attached to tracepoints are non-preemptible. This is unhelpful in
> real-time systems, whose users apparently wish to use BPF while also
> achieving low latencies.
>
> Change the protection of tracepoints to use fast_srcu() instead.
> This will allow the callbacks to be able to be preempted. This also
> means that the callbacks themselves need to be able to handle this
> new found preemption ability.
>
> For perf, add a guard(preempt) inside its handler too keep the old behavior
> of perf events being called with preemption disabled.
>
> For BPF, add a migrate_disable() to its handler. Actually, just replace
> the rcu_read_lock() with rcu_read_lock_dont_migrate() and make it
> cover more of the BPF callback handler.
My tests just triggered this, so I'm removing them from my queue for now.
-- Steve
[ 204.194772] ------------[ cut here ]------------
[ 204.194789] WARNING: kernel/rcu/srcutree.c:792 at __srcu_check_read_flavor+0x5c/0xb0, CPU#1: swapper/1/0
[ 204.194800] Modules linked in:
[ 204.194817] CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 6.19.0-rc7-test-00018-g2c774d6ad074-dirty #32 PREEMPT(voluntary)
[ 204.194821] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
[ 204.194824] RIP: 0010:__srcu_check_read_flavor+0x5c/0xb0
[ 204.194829] Code: 84 c9 74 19 39 f1 74 45 0f 0b 85 c0 74 2e 39 c1 74 45 0f 0b 39 f0 75 3f c3 cc cc cc cc 85 c0 74 16 83 fe 04 75 ee 0f 0b eb ea <0f> 0b 8d 46 ff 85 f0 74 ba 0f 0b eb b6 83
fe 04 74 3a 31 c0 f0 0f
[ 204.194832] RSP: 0018:fffffe4c48325b50 EFLAGS: 00010002
[ 204.194835] RAX: 0000000000000001 RBX: ffffffff8791e5a0 RCX: 0000000000000000
[ 204.194836] RDX: 00000000ffffffff RSI: 0000000000000004 RDI: ffffffff879f1180
[ 204.194838] RBP: ffff8e6453fd2000 R08: 0000000000000001 R09: 0000000000000000
[ 204.194839] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000001
[ 204.194840] R13: fffffe4c48325ef8 R14: ffffffff85eeae93 R15: ffff8e6453906900
[ 204.194842] FS: 0000000000000000(0000) GS:ffff8e6533593000(0000) knlGS:0000000000000000
[ 204.194844] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 204.194845] CR2: 000055d1e7cf8cc0 CR3: 000000010b0cc004 CR4: 0000000000172ef0
[ 204.194850] Call Trace:
[ 204.194866] <NMI>
[ 204.194868] lock_release+0x215/0x320
[ 204.194886] ? arch_perf_update_userpage+0x6c/0xf0
[ 204.195214] perf_event_update_userpage+0x158/0x2e0
[ 204.195538] x86_perf_event_set_period+0xc1/0x180
[ 204.195811] handle_pmi_common+0x1ac/0x450
[ 204.198605] ? __get_next_timer_interrupt+0x185/0x370
[ 204.198914] intel_pmu_handle_irq+0x10e/0x510
[ 204.199032] ? nmi_handle.part.0+0x30/0x270
[ 204.199197] ? __get_next_timer_interrupt+0x185/0x370
[ 204.199404] perf_event_nmi_handler+0x34/0x60
[ 204.199523] nmi_handle.part.0+0xc9/0x270
^ permalink raw reply
* [PATCH] tracing: kprobe-event: Return directly when trace kprobes is empty
From: sunliming @ 2026-01-27 2:01 UTC (permalink / raw)
To: ostedt, mhiramat
Cc: mathieu.desnoyers, linux-kernel, linux-trace-kernel, sunliming
From: sunliming <sunliming@kylinos.cn>
In enable_boot_kprobe_events(), it returns directly when trace kprobes is
empty, thereby reducing the function's execution time. This function may
otherwise wait for the event_mutex lock for tens of milliseconds on certain
machines, which is unnecessary when trace kprobes is empty.
Signed-off-by: sunliming <sunliming@kylinos.cn>
---
kernel/trace/trace_kprobe.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 9953506370a5..95f2c42603d5 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -82,6 +82,7 @@ static struct trace_kprobe *to_trace_kprobe(struct dyn_event *ev)
#define for_each_trace_kprobe(pos, dpos) \
for_each_dyn_event(dpos) \
if (is_trace_kprobe(dpos) && (pos = to_trace_kprobe(dpos)))
+#define trace_kprobe_list_empty() list_empty(&dyn_event_list)
static nokprobe_inline bool trace_kprobe_is_return(struct trace_kprobe *tk)
{
@@ -1982,6 +1983,9 @@ static __init void enable_boot_kprobe_events(void)
struct trace_kprobe *tk;
struct dyn_event *pos;
+ if (trace_kprobe_list_empty())
+ return;
+
guard(mutex)(&event_mutex);
for_each_trace_kprobe(tk, pos) {
list_for_each_entry(file, &tr->events, list)
--
2.25.1
^ permalink raw reply related
* Re: [RESEND][PATCH 3/5] perf: Use current->flags & PF_KTHREAD|PF_USER_WORKER instead of current->mm == NULL
From: Guenter Roeck @ 2026-01-27 1:32 UTC (permalink / raw)
To: Steven Rostedt
Cc: linux-kernel, linux-trace-kernel, linux-perf-users,
Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Alexander Shishkin, Jiri Olsa, Ian Rogers,
Adrian Hunter, Thomas Gleixner
In-Reply-To: <20260126121803.4a90c959@gandalf.local.home>
On 1/26/26 09:18, Steven Rostedt wrote:
> On Mon, 26 Jan 2026 12:05:53 -0500
> Steven Rostedt <rostedt@kernel.org> wrote:
>
>> I guess we need to also test for !current->mm because the flags set for an
>> exiting task is done when we can still do callchains. Thus, the only way to
>> know if it is safe to do a callchain when a task is exiting is via task->mm
>> and not task->flags :-/
>
> Can you test this patch?
>
Still crashing, though not as often and with a slightly different backtrace.
I added the backtrace to the bug report @ Google.
Guenter
> -- Steve
>
> diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c
> index 1f6589578703..c82d61d73bd8 100644
> --- a/kernel/events/callchain.c
> +++ b/kernel/events/callchain.c
> @@ -246,7 +246,14 @@ get_perf_callchain(struct pt_regs *regs, bool kernel, bool user,
>
> if (user && !crosstask) {
> if (!user_mode(regs)) {
> - if (current->flags & (PF_KTHREAD | PF_USER_WORKER))
> + /*
> + * Testing current->mm is not enough as some kernel threads
> + * may have one set. But testing the flags is not enough
> + * either as this can be called after a user task
> + * frees its mm just before it exits.
> + */
> + if (!current->mm ||
> + (current->flags & (PF_KTHREAD | PF_USER_WORKER)))
> goto exit_put;
> regs = task_pt_regs(current);
> }
^ permalink raw reply
* Re: [PATCH v2 1/3] tracing: Rename `eval_map_wq` and export it for asynchronous use by other modules
From: Yaxiong Tian @ 2026-01-27 1:23 UTC (permalink / raw)
To: Steven Rostedt
Cc: axboe, mhiramat, mathieu.desnoyers, linux-block, linux-kernel,
linux-trace-kernel
In-Reply-To: <20260126093646.3922b905@gandalf.local.home>
在 2026/1/26 22:36, Steven Rostedt 写道:
> On Mon, 26 Jan 2026 10:43:16 +0800
> Yaxiong Tian <tianyaxiong@kylinos.cn> wrote:
>
> Hi,
>
> Some of your terminology is a little confusing. The subject says "modules"
> but no module will use it (the term module means loadable code at runtime
> into the Linux kernel). A better subject would be:
>
>
> tracing: Rename `eval_map_wq` and allow other parts of tracing use it
>
>
>> The eval_map_work_func() function, though queued in eval_map_wq,
>> holds the trace_event_sem read-write lock for a long time during
>> kernel boot. This causes blocking issues for other functions.
>>
>> Rename eval_map_wq to trace_init_wq and export it, thereby allowing
> Also saying "export" for a function means something like "EXPORT_SYMBOLE()"
> which again deals with Linux modules. A better term is "make it global".
>
>> other modules to schedule work on this queue asynchronously and
> .. other parts of tracing ..
>
>> avoiding blockage of the main boot thread.
>>
>> Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
> Other than that, this patch looks good.
>
> -- Steve
Yes, those statements were not quite accurate. I have addressed them
in Version 3.
^ permalink raw reply
* [PATCH v3 3/3] blktrace: Make init_blk_tracer() asynchronous
From: Yaxiong Tian @ 2026-01-27 1:23 UTC (permalink / raw)
To: rostedt, axboe, mhiramat, mathieu.desnoyers
Cc: linux-block, linux-kernel, linux-trace-kernel, Yaxiong Tian
In-Reply-To: <20260127012016.76713-1-tianyaxiong@kylinos.cn>
The init_blk_tracer() function causes significant boot delay as it
waits for the trace_event_sem lock held by trace_event_update_all().
Specifically, its child function register_trace_event() requires
this lock, which is occupied for an extended period during boot.
To mitigate this contention, we have moved init_blk_tracer() to the
trace_init_wq workqueue, allowing it to execute asynchronously and
prevent blocking the main boot thread.
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
---
kernel/trace/blktrace.c | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/blktrace.c b/kernel/trace/blktrace.c
index d031c8d80be4..d611cd1f02ef 100644
--- a/kernel/trace/blktrace.c
+++ b/kernel/trace/blktrace.c
@@ -1832,7 +1832,9 @@ static struct trace_event trace_blk_event = {
.funcs = &trace_blk_event_funcs,
};
-static int __init init_blk_tracer(void)
+static struct work_struct blktrace_works __initdata;
+
+static int __init __init_blk_tracer(void)
{
if (!register_trace_event(&trace_blk_event)) {
pr_warn("Warning: could not register block events\n");
@@ -1852,6 +1854,25 @@ static int __init init_blk_tracer(void)
return 0;
}
+static void __init blktrace_works_func(struct work_struct *work)
+{
+ __init_blk_tracer();
+}
+
+static int __init init_blk_tracer(void)
+{
+ int ret = 0;
+
+ if (trace_init_wq) {
+ INIT_WORK(&blktrace_works, blktrace_works_func);
+ queue_work(trace_init_wq, &blktrace_works);
+ } else {
+ ret = __init_blk_tracer();
+ }
+
+ return ret;
+}
+
device_initcall(init_blk_tracer);
static int blk_trace_remove_queue(struct request_queue *q)
--
2.25.1
^ permalink raw reply related
* [PATCH v3 2/3] tracing/kprobes: Make setup_boot_kprobe_events() asynchronous
From: Yaxiong Tian @ 2026-01-27 1:23 UTC (permalink / raw)
To: rostedt, axboe, mhiramat, mathieu.desnoyers
Cc: linux-block, linux-kernel, linux-trace-kernel, Yaxiong Tian
In-Reply-To: <20260127012016.76713-1-tianyaxiong@kylinos.cn>
During kernel boot, the setup_boot_kprobe_events() function causes
significant delays, increasing overall startup time.
The root cause is a lock contention chain: its child function
enable_boot_kprobe_events() requires the event_mutex, which is
already held by early_event_add_tracer(). early_event_add_tracer()
itself is blocked waiting for the trace_event_sem read-write lock,
which is held for an extended period by trace_event_update_all().
To resolve this, we have moved the execution of
setup_boot_kprobe_events() to the trace_init_wq workqueue, allowing
it to run asynchronously.
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
---
kernel/trace/trace_kprobe.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 9953506370a5..4c6621f02696 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -2031,6 +2031,13 @@ static __init int init_kprobe_trace_early(void)
}
core_initcall(init_kprobe_trace_early);
+static struct work_struct kprobe_trace_work __initdata;
+
+static void __init kprobe_trace_works_func(struct work_struct *work)
+{
+ setup_boot_kprobe_events();
+}
+
/* Make a tracefs interface for controlling probe points */
static __init int init_kprobe_trace(void)
{
@@ -2048,7 +2055,12 @@ static __init int init_kprobe_trace(void)
trace_create_file("kprobe_profile", TRACE_MODE_READ,
NULL, NULL, &kprobe_profile_ops);
- setup_boot_kprobe_events();
+ if (trace_init_wq) {
+ INIT_WORK(&kprobe_trace_work, kprobe_trace_works_func);
+ queue_work(trace_init_wq, &kprobe_trace_work);
+ } else {
+ setup_boot_kprobe_events();
+ }
return 0;
}
--
2.25.1
^ permalink raw reply related
* [PATCH v3 1/3] tracing: Rename `eval_map_wq` and allow other parts of tracing use it
From: Yaxiong Tian @ 2026-01-27 1:23 UTC (permalink / raw)
To: rostedt, axboe, mhiramat, mathieu.desnoyers
Cc: linux-block, linux-kernel, linux-trace-kernel, Yaxiong Tian
In-Reply-To: <20260127012016.76713-1-tianyaxiong@kylinos.cn>
The eval_map_work_func() function, though queued in eval_map_wq,
holds the trace_event_sem read-write lock for a long time during
kernel boot. This causes blocking issues for other functions.
Rename eval_map_wq to trace_init_wq and make it global, thereby
allowing other parts of tracing to schedule work on this queue
asynchronously and avoiding blockage of the main boot thread.
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
---
kernel/trace/trace.c | 18 +++++++++---------
kernel/trace/trace.h | 2 ++
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index e18005807395..c61e30cb7339 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -10774,7 +10774,7 @@ int tracing_init_dentry(void)
extern struct trace_eval_map *__start_ftrace_eval_maps[];
extern struct trace_eval_map *__stop_ftrace_eval_maps[];
-static struct workqueue_struct *eval_map_wq __initdata;
+struct workqueue_struct *trace_init_wq __initdata;
static struct work_struct eval_map_work __initdata;
static struct work_struct tracerfs_init_work __initdata;
@@ -10790,15 +10790,15 @@ static int __init trace_eval_init(void)
{
INIT_WORK(&eval_map_work, eval_map_work_func);
- eval_map_wq = alloc_workqueue("eval_map_wq", WQ_UNBOUND, 0);
- if (!eval_map_wq) {
- pr_err("Unable to allocate eval_map_wq\n");
+ trace_init_wq = alloc_workqueue("trace_init_wq", WQ_UNBOUND, 0);
+ if (!trace_init_wq) {
+ pr_err("Unable to allocate trace_init_wq\n");
/* Do work here */
eval_map_work_func(&eval_map_work);
return -ENOMEM;
}
- queue_work(eval_map_wq, &eval_map_work);
+ queue_work(trace_init_wq, &eval_map_work);
return 0;
}
@@ -10807,8 +10807,8 @@ subsys_initcall(trace_eval_init);
static int __init trace_eval_sync(void)
{
/* Make sure the eval map updates are finished */
- if (eval_map_wq)
- destroy_workqueue(eval_map_wq);
+ if (trace_init_wq)
+ destroy_workqueue(trace_init_wq);
return 0;
}
@@ -10969,9 +10969,9 @@ static __init int tracer_init_tracefs(void)
if (ret)
return 0;
- if (eval_map_wq) {
+ if (trace_init_wq) {
INIT_WORK(&tracerfs_init_work, tracer_init_tracefs_work_func);
- queue_work(eval_map_wq, &tracerfs_init_work);
+ queue_work(trace_init_wq, &tracerfs_init_work);
} else {
tracer_init_tracefs_work_func(NULL);
}
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index de4e6713b84e..e52f259f8945 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -770,6 +770,8 @@ extern unsigned long nsecs_to_usecs(unsigned long nsecs);
extern unsigned long tracing_thresh;
+extern struct workqueue_struct *trace_init_wq __initdata;
+
/* PID filtering */
bool trace_find_filtered_pid(struct trace_pid_list *filtered_pids,
--
2.25.1
^ permalink raw reply related
* [PATCH v3 0/3] Tracing: Accelerate Kernel Boot by Asynchronizing
From: Yaxiong Tian @ 2026-01-27 1:20 UTC (permalink / raw)
To: rostedt, axboe, mhiramat, mathieu.desnoyers
Cc: linux-block, linux-kernel, linux-trace-kernel, Yaxiong Tian
On my ARM64 platform, I observed that certain tracing module
initializations run for up to 200ms—for example, init_kprobe_trace().
Analysis reveals the root cause: the execution flow eval_map_work_func()
→trace_event_update_with_eval_map()→trace_event_update_all()
is highly time-consuming. Although this flow is placed in eval_map_wq
for asynchronous execution, it holds the trace_event_sem lock, causing
other modules to be blocked either directly or indirectly.
To resolve this issue, I rename `eval_map_wq` and make it global and moved
other initialization functions under the tracing subsystem that are
related to this lock to run asynchronously on this workqueue. After this
optimization, boot time is reduced by approximately 200ms.
---
Changes in v2:
- Rename eval_map_wq to trace_init_wq.
Changes in v3:
- Opt PATCH 1/3 commit
Yaxiong Tian (3):
tracing: Rename `eval_map_wq` and allow other parts of tracing use it
tracing/kprobes: Make setup_boot_kprobe_events() asynchronous
blktrace: Make init_blk_tracer() asynchronous
kernel/trace/blktrace.c | 23 ++++++++++++++++++++++-
kernel/trace/trace.c | 18 +++++++++---------
kernel/trace/trace.h | 2 ++
kernel/trace/trace_kprobe.c | 14 +++++++++++++-
4 files changed, 46 insertions(+), 11 deletions(-)
--
2.25.1
^ permalink raw reply
* [PATCH] tracing: Documentation: Update histogram-design.rst for fn() handling
From: Steven Rostedt @ 2026-01-26 23:17 UTC (permalink / raw)
To: LKML, Linux Trace Kernel, linux-doc
Cc: Masami Hiramatsu, Mathieu Desnoyers, Tom Zanussi, Jonathan Corbet
From: Steven Rostedt <rostedt@goodmis.org>
The histogram documentation describes the old method of the histogram
triggers using the fn() field of the histogram field structure to process
the field. But due to Spectre mitigation, the function pointer to handle
the fields at runtime caused a noticeable overhead. It was converted over
to a fn_num and hist_fn_call() is now used to call the specific functions
for the fields via a switch statement based on the field's fn_num value.
Update the documentation to reflect this change.
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
Documentation/trace/histogram-design.rst | 20 +++++++++++++-------
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/Documentation/trace/histogram-design.rst b/Documentation/trace/histogram-design.rst
index ae71b5bf97c6..e92f56ebd0b5 100644
--- a/Documentation/trace/histogram-design.rst
+++ b/Documentation/trace/histogram-design.rst
@@ -69,7 +69,8 @@ So in this histogram, there's a separate bucket for each pid, and each
bucket contains a value for that bucket, counting the number of times
sched_waking was called for that pid.
-Each histogram is represented by a hist_data struct.
+Each histogram is represented by a hist_data struct
+(struct hist_trigger_data).
To keep track of each key and value field in the histogram, hist_data
keeps an array of these fields named fields[]. The fields[] array is
@@ -82,7 +83,7 @@ value or not, which the above histogram does not.
Each struct hist_field contains a pointer to the ftrace_event_field
from the event's trace_event_file along with various bits related to
-that such as the size, offset, type, and a hist_field_fn_t function,
+that such as the size, offset, type, and a hist field function,
which is used to grab the field's data from the ftrace event buffer
(in most cases - some hist_fields such as hitcount don't directly map
to an event field in the trace buffer - in these cases the function
@@ -241,28 +242,33 @@ it, event_hist_trigger() is called. event_hist_trigger() first deals
with the key: for each subkey in the key (in the above example, there
is just one subkey corresponding to pid), the hist_field that
represents that subkey is retrieved from hist_data.fields[] and the
-hist_field_fn_t fn() associated with that field, along with the
+hist field function associated with that field, along with the
field's size and offset, is used to grab that subkey's data from the
current trace record.
+Note, the hist field function use to be a function pointer in the
+hist_field stucture. Due to spectre mitigation, it was converted into
+a fn_num and hist_fn_call() is used to call the associated hist field
+function that corresponds to the fn_num of the hist_field structure.
+
Once the complete key has been retrieved, it's used to look that key
up in the tracing_map. If there's no tracing_map_elt associated with
that key, an empty one is claimed and inserted in the map for the new
key. In either case, the tracing_map_elt associated with that key is
returned.
-Once a tracing_map_elt available, hist_trigger_elt_update() is called.
+Once a tracing_map_elt is available, hist_trigger_elt_update() is called.
As the name implies, this updates the element, which basically means
updating the element's fields. There's a tracing_map_field associated
with each key and value in the histogram, and each of these correspond
to the key and value hist_fields created when the histogram was
created. hist_trigger_elt_update() goes through each value hist_field
-and, as for the keys, uses the hist_field's fn() and size and offset
+and, as for the keys, uses the hist_field's function and size and offset
to grab the field's value from the current trace record. Once it has
that value, it simply adds that value to that field's
continually-updated tracing_map_field.sum member. Some hist_field
-fn()s, such as for the hitcount, don't actually grab anything from the
-trace record (the hitcount fn() just increments the counter sum by 1),
+functions, such as for the hitcount, don't actually grab anything from the
+trace record (the hitcount function just increments the counter sum by 1),
but the idea is the same.
Once all the values have been updated, hist_trigger_elt_update() is
--
2.51.0
^ permalink raw reply related
* [PATCH v6 3/3] tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast
From: Steven Rostedt @ 2026-01-26 23:11 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, bpf
Cc: Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Paul E. McKenney, Sebastian Andrzej Siewior, Alexei Starovoitov
In-Reply-To: <20260126231145.728172709@kernel.org>
From: Steven Rostedt <rostedt@goodmis.org>
The current use of guard(preempt_notrace)() within __DECLARE_TRACE()
to protect invocation of __DO_TRACE_CALL() means that BPF programs
attached to tracepoints are non-preemptible. This is unhelpful in
real-time systems, whose users apparently wish to use BPF while also
achieving low latencies. (Who knew?)
One option would be to use preemptible RCU, but this introduces
many opportunities for infinite recursion, which many consider to
be counterproductive, especially given the relatively small stacks
provided by the Linux kernel. These opportunities could be shut down
by sufficiently energetic duplication of code, but this sort of thing
is considered impolite in some circles.
Therefore, use the shiny new SRCU-fast API, which provides somewhat faster
readers than those of preemptible RCU, at least on Paul E. McKenney's
laptop, where task_struct access is more expensive than access to per-CPU
variables. And SRCU-fast provides way faster readers than does SRCU,
courtesy of being able to avoid the read-side use of smp_mb(). Also,
it is quite straightforward to create srcu_read_{,un}lock_fast_notrace()
functions.
Link: https://lore.kernel.org/all/20250613152218.1924093-1-bigeasy@linutronix.de/
Co-developed-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
Changes since v5: https://patch.msgid.link/20260108220550.2f6638f3@fedora
- Just change from preempt_disable() to srcu_fast() always
Do not do anything different for PREEMPT_RT.
Now that BPF disables migration directly, do not have tracepoints
disable migration in its code.
include/linux/tracepoint.h | 9 +++++----
include/trace/trace_events.h | 4 ++--
kernel/tracepoint.c | 18 ++++++++++++++----
3 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h
index 8a56f3278b1b..22ca1c8b54f3 100644
--- a/include/linux/tracepoint.h
+++ b/include/linux/tracepoint.h
@@ -108,14 +108,15 @@ void for_each_tracepoint_in_module(struct module *mod,
* An alternative is to use the following for batch reclaim associated
* with a given tracepoint:
*
- * - tracepoint_is_faultable() == false: call_rcu()
+ * - tracepoint_is_faultable() == false: call_srcu()
* - tracepoint_is_faultable() == true: call_rcu_tasks_trace()
*/
#ifdef CONFIG_TRACEPOINTS
+extern struct srcu_struct tracepoint_srcu;
static inline void tracepoint_synchronize_unregister(void)
{
synchronize_rcu_tasks_trace();
- synchronize_rcu();
+ synchronize_srcu(&tracepoint_srcu);
}
static inline bool tracepoint_is_faultable(struct tracepoint *tp)
{
@@ -275,13 +276,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
return static_branch_unlikely(&__tracepoint_##name.key);\
}
-#define __DECLARE_TRACE(name, proto, args, cond, data_proto) \
+#define __DECLARE_TRACE(name, proto, args, cond, data_proto) \
__DECLARE_TRACE_COMMON(name, PARAMS(proto), PARAMS(args), PARAMS(data_proto)) \
static inline void __do_trace_##name(proto) \
{ \
TRACEPOINT_CHECK(name) \
if (cond) { \
- guard(preempt_notrace)(); \
+ guard(srcu_fast_notrace)(&tracepoint_srcu); \
__DO_TRACE_CALL(name, TP_ARGS(args)); \
} \
} \
diff --git a/include/trace/trace_events.h b/include/trace/trace_events.h
index 4f22136fd465..fbc07d353be6 100644
--- a/include/trace/trace_events.h
+++ b/include/trace/trace_events.h
@@ -436,6 +436,7 @@ __DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), PARAMS(tstruct), \
static notrace void \
trace_event_raw_event_##call(void *__data, proto) \
{ \
+ guard(preempt_notrace)(); \
do_trace_event_raw_event_##call(__data, args); \
}
@@ -447,9 +448,8 @@ static notrace void \
trace_event_raw_event_##call(void *__data, proto) \
{ \
might_fault(); \
- preempt_disable_notrace(); \
+ guard(preempt_notrace)(); \
do_trace_event_raw_event_##call(__data, args); \
- preempt_enable_notrace(); \
}
/*
diff --git a/kernel/tracepoint.c b/kernel/tracepoint.c
index 62719d2941c9..fd2ee879815c 100644
--- a/kernel/tracepoint.c
+++ b/kernel/tracepoint.c
@@ -34,9 +34,13 @@ enum tp_transition_sync {
struct tp_transition_snapshot {
unsigned long rcu;
+ unsigned long srcu_gp;
bool ongoing;
};
+DEFINE_SRCU_FAST(tracepoint_srcu);
+EXPORT_SYMBOL_GPL(tracepoint_srcu);
+
/* Protected by tracepoints_mutex */
static struct tp_transition_snapshot tp_transition_snapshot[_NR_TP_TRANSITION_SYNC];
@@ -46,6 +50,7 @@ static void tp_rcu_get_state(enum tp_transition_sync sync)
/* Keep the latest get_state snapshot. */
snapshot->rcu = get_state_synchronize_rcu();
+ snapshot->srcu_gp = start_poll_synchronize_srcu(&tracepoint_srcu);
snapshot->ongoing = true;
}
@@ -56,6 +61,8 @@ static void tp_rcu_cond_sync(enum tp_transition_sync sync)
if (!snapshot->ongoing)
return;
cond_synchronize_rcu(snapshot->rcu);
+ if (!poll_state_synchronize_srcu(&tracepoint_srcu, snapshot->srcu_gp))
+ synchronize_srcu(&tracepoint_srcu);
snapshot->ongoing = false;
}
@@ -112,10 +119,13 @@ static inline void release_probes(struct tracepoint *tp, struct tracepoint_func
struct tp_probes *tp_probes = container_of(old,
struct tp_probes, probes[0]);
- if (tracepoint_is_faultable(tp))
- call_rcu_tasks_trace(&tp_probes->rcu, rcu_free_old_probes);
- else
- call_rcu(&tp_probes->rcu, rcu_free_old_probes);
+ if (tracepoint_is_faultable(tp)) {
+ call_rcu_tasks_trace(&tp_probes->rcu,
+ rcu_free_old_probes);
+ } else {
+ call_srcu(&tracepoint_srcu, &tp_probes->rcu,
+ rcu_free_old_probes);
+ }
}
}
--
2.51.0
^ permalink raw reply related
* [PATCH v6 2/3] bpf: Have __bpf_trace_run() use rcu_read_lock_dont_migrate()
From: Steven Rostedt @ 2026-01-26 23:11 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, bpf
Cc: Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Paul E. McKenney, Sebastian Andrzej Siewior, Alexei Starovoitov,
Alexei Starovoitov
In-Reply-To: <20260126231145.728172709@kernel.org>
From: Steven Rostedt <rostedt@goodmis.org>
In order to switch the protection of tracepoint callbacks from
preempt_disable() to srcu_read_lock_fast() the BPF callback from
tracepoints needs to have migration prevention as the BPF programs expect
to stay on the same CPU as they execute. Put together the RCU protection
with migration prevention and use rcu_read_lock_dont_migrate() in
__bpf_trace_run(). This will allow tracepoints callbacks to be
preemptible.
Link: https://lore.kernel.org/all/CAADnVQKvY026HSFGOsavJppm3-Ajm-VsLzY-OeFUe+BaKMRnDg@mail.gmail.com/
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
kernel/trace/bpf_trace.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index fe28d86f7c35..abbf0177ad20 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -2062,7 +2062,7 @@ void __bpf_trace_run(struct bpf_raw_tp_link *link, u64 *args)
struct bpf_run_ctx *old_run_ctx;
struct bpf_trace_run_ctx run_ctx;
- cant_sleep();
+ rcu_read_lock_dont_migrate();
if (unlikely(this_cpu_inc_return(*(prog->active)) != 1)) {
bpf_prog_inc_misses_counter(prog);
goto out;
@@ -2071,13 +2071,12 @@ void __bpf_trace_run(struct bpf_raw_tp_link *link, u64 *args)
run_ctx.bpf_cookie = link->cookie;
old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);
- rcu_read_lock();
(void) bpf_prog_run(prog, args);
- rcu_read_unlock();
bpf_reset_run_ctx(old_run_ctx);
out:
this_cpu_dec(*(prog->active));
+ rcu_read_unlock_migrate();
}
#define UNPACK(...) __VA_ARGS__
--
2.51.0
^ permalink raw reply related
* [PATCH v6 1/3] tracing: perf: Have perf tracepoint callbacks always disable preemption
From: Steven Rostedt @ 2026-01-26 23:11 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, bpf
Cc: Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Paul E. McKenney, Sebastian Andrzej Siewior, Alexei Starovoitov
In-Reply-To: <20260126231145.728172709@kernel.org>
From: Steven Rostedt <rostedt@goodmis.org>
In preparation to convert protection of tracepoints from being protected
by a preempt disabled section to being protected by SRCU, have all the
perf callbacks disable preemption as perf expects preemption to be
disabled when processing tracepoints.
While at it, convert the perf system call callback preempt_disable() to a
guard(preempt).
Link: https://lore.kernel.org/all/20250613152218.1924093-1-bigeasy@linutronix.de/
Link: https://patch.msgid.link/20260108220550.2f6638f3@fedora
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
include/trace/perf.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/trace/perf.h b/include/trace/perf.h
index a1754b73a8f5..348ad1d9b556 100644
--- a/include/trace/perf.h
+++ b/include/trace/perf.h
@@ -71,6 +71,7 @@ perf_trace_##call(void *__data, proto) \
u64 __count __attribute__((unused)); \
struct task_struct *__task __attribute__((unused)); \
\
+ guard(preempt_notrace)(); \
do_perf_trace_##call(__data, args); \
}
@@ -85,9 +86,8 @@ perf_trace_##call(void *__data, proto) \
struct task_struct *__task __attribute__((unused)); \
\
might_fault(); \
- preempt_disable_notrace(); \
+ guard(preempt_notrace)(); \
do_perf_trace_##call(__data, args); \
- preempt_enable_notrace(); \
}
/*
--
2.51.0
^ permalink raw reply related
* [PATCH v6 0/3] tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast
From: Steven Rostedt @ 2026-01-26 23:11 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, bpf
Cc: Masami Hiramatsu, Mark Rutland, Mathieu Desnoyers, Andrew Morton,
Paul E. McKenney, Sebastian Andrzej Siewior, Alexei Starovoitov
The current use of guard(preempt_notrace)() within __DECLARE_TRACE()
to protect invocation of __DO_TRACE_CALL() means that BPF programs
attached to tracepoints are non-preemptible. This is unhelpful in
real-time systems, whose users apparently wish to use BPF while also
achieving low latencies.
Change the protection of tracepoints to use fast_srcu() instead.
This will allow the callbacks to be able to be preempted. This also
means that the callbacks themselves need to be able to handle this
new found preemption ability.
For perf, add a guard(preempt) inside its handler too keep the old behavior
of perf events being called with preemption disabled.
For BPF, add a migrate_disable() to its handler. Actually, just replace
the rcu_read_lock() with rcu_read_lock_dont_migrate() and make it
cover more of the BPF callback handler.
[ I would have sent this out earlier, but had a death in the family
which cause everything to be postponed ]
Changes since v5: https://patch.msgid.link/20260108220550.2f6638f3@fedora
- Add separate patch for perf to call preempt_disable()
- Add patch that has bpf call migrate_disable() directly.
- Just change from preempt_disable() to srcu_fast() always
Do not do anything different for PREEMPT_RT.
Now that BPF disables migration directly, do not have tracepoints
disable migration in its code.
Steven Rostedt (3):
tracing: perf: Have perf tracepoint callbacks always disable preemption
bpf: Have __bpf_trace_run() use rcu_read_lock_dont_migrate()
tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast
----
include/linux/tracepoint.h | 9 +++++----
include/trace/perf.h | 4 ++--
include/trace/trace_events.h | 4 ++--
kernel/trace/bpf_trace.c | 5 ++---
kernel/tracepoint.c | 18 ++++++++++++++----
5 files changed, 25 insertions(+), 15 deletions(-)
^ permalink raw reply
* Re: [PATCH] MAINTAINERS: add Rust files to STATIC BRANCH/CALL and TRACING
From: Alice Ryhl @ 2026-01-26 21:58 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Steven Rostedt, Miguel Ojeda, Masami Hiramatsu, Mathieu Desnoyers,
Josh Poimboeuf, Jason Baron, Ard Biesheuvel, linux-kernel,
linux-trace-kernel, rust-for-linux
In-Reply-To: <20260126213748.GR171111@noisy.programming.kicks-ass.net>
On Mon, Jan 26, 2026 at 10:37 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Jan 26, 2026 at 12:27:10PM +0000, Alice Ryhl wrote:
> > On Mon, Jan 12, 2026 at 11:54:08AM -0500, Steven Rostedt wrote:
> > > On Mon, 12 Jan 2026 15:10:31 +0000
> > > Alice Ryhl <aliceryhl@google.com> wrote:
> > >
> > > > > And have the M of the other sections be R here?
> > > >
> > > > Sure, we can do that.
> > > >
> > > > Are you still willing to pick up the patches? I think that is simpler in
> > > > case there are any series that touch both the C and Rust parts. (Such as
> > > > the initial tracepoint series did.)
> > >
> > > Yes. So I guess you can still add me with a 'M:'. But I wanted a separate
> > > section so that all the Rust expertise is still included.
> >
> > What about the STATIC BRANCH/CALL subsystem? Should I also leave you or
> > someone else as 'M:' there? It's unclear to me who usually picks up
> > patches for STATIC BRANCH/CALL when they are not a dependency to a patch
> > for somewhere else.
>
> I think that'd be me -- I typically do the static branch/call bits.
Ah, thanks for the clarification.
Are you ok with using the approach Steven suggested for STATIC
BRANCH/CALL subsystem too? That is, add a [RUST] entry below the
current one, list you and me as M:, and anyone else in the main entry
as R:, and patches land through the same tree as where they would have
landed if they were a C patch.
I'm open to whichever setup you prefer, but I think it'd be nice to
get these files into MAINTAINERS somewhere.
Alice
^ permalink raw reply
* Re: [PATCH] MAINTAINERS: add Rust files to STATIC BRANCH/CALL and TRACING
From: Peter Zijlstra @ 2026-01-26 21:37 UTC (permalink / raw)
To: Alice Ryhl
Cc: Steven Rostedt, Miguel Ojeda, Masami Hiramatsu, Mathieu Desnoyers,
Josh Poimboeuf, Jason Baron, Ard Biesheuvel, linux-kernel,
linux-trace-kernel, rust-for-linux
In-Reply-To: <aXddnjiHCX-XS5S9@google.com>
On Mon, Jan 26, 2026 at 12:27:10PM +0000, Alice Ryhl wrote:
> On Mon, Jan 12, 2026 at 11:54:08AM -0500, Steven Rostedt wrote:
> > On Mon, 12 Jan 2026 15:10:31 +0000
> > Alice Ryhl <aliceryhl@google.com> wrote:
> >
> > > > And have the M of the other sections be R here?
> > >
> > > Sure, we can do that.
> > >
> > > Are you still willing to pick up the patches? I think that is simpler in
> > > case there are any series that touch both the C and Rust parts. (Such as
> > > the initial tracepoint series did.)
> >
> > Yes. So I guess you can still add me with a 'M:'. But I wanted a separate
> > section so that all the Rust expertise is still included.
>
> What about the STATIC BRANCH/CALL subsystem? Should I also leave you or
> someone else as 'M:' there? It's unclear to me who usually picks up
> patches for STATIC BRANCH/CALL when they are not a dependency to a patch
> for somewhere else.
I think that'd be me -- I typically do the static branch/call bits.
^ permalink raw reply
* Re: [PATCH] tracing: Up the hist stacktrace size from 16 to 31
From: Tom Zanussi @ 2026-01-26 21:37 UTC (permalink / raw)
To: Steven Rostedt, LKML, Linux Trace Kernel
Cc: Masami Hiramatsu, Mathieu Desnoyers
In-Reply-To: <20260123105415.2be26bf4@gandalf.local.home>
Hi Steve,
On Fri, 2026-01-23 at 10:54 -0500, Steven Rostedt wrote:
> From: Steven Rostedt <rostedt@goodmis.org>
>
> Recording stacktraces is very useful, but the size of 16 deep is very
> restrictive. For example, in seeing where tasks schedule out in a non
> running state, the following can be used:
>
> ~# cd /sys/kernel/tracing
> ~# echo 'hist:keys=common_stacktrace:vals=hitcount if prev_state & 3' > events/sched/sched_switch/trigger
> ~# cat events/sched/sched_switch/hist
> [..]
> { common_stacktrace:
> __schedule+0xdc0/0x1860
> schedule+0x27/0xd0
> schedule_timeout+0xb5/0x100
> wait_for_completion+0x8a/0x140
> xfs_buf_iowait+0x20/0xd0 [xfs]
> xfs_buf_read_map+0x103/0x250 [xfs]
> xfs_trans_read_buf_map+0x161/0x310 [xfs]
> xfs_btree_read_buf_block+0xa0/0x120 [xfs]
> xfs_btree_lookup_get_block+0xa3/0x1e0 [xfs]
> xfs_btree_lookup+0xea/0x530 [xfs]
> xfs_alloc_fixup_trees+0x72/0x570 [xfs]
> xfs_alloc_ag_vextent_size+0x67f/0x800 [xfs]
> xfs_alloc_vextent_iterate_ags.constprop.0+0x52/0x230 [xfs]
> xfs_alloc_vextent_start_ag+0x9d/0x1b0 [xfs]
> xfs_bmap_btalloc+0x2af/0x680 [xfs]
> xfs_bmapi_allocate+0xdb/0x2c0 [xfs]
> } hitcount: 1
> [..]
>
> The above stops at 16 functions where knowing more would be useful. As the
> allocated storage for stacks is the same for strings, and that size is 256
> bytes, there is a lot of space not being used for stacktraces.
>
> 16 * 8 = 128
>
> Up the size to 31 (it requires the last slot to be zero, so it can't be 32).
>
Yeah, it can't be 32 because of the extra slot needed for saving the
number of elements. If the stacktrace doesn't take up the full 31
elements, it writes a 0 after the last one to mark the end; if it fills
it up, the 0 isn't written since the size of the array is known.
> Also change the BUILD_BUG_ON() to allow the size of the stacktrace storage
> to be equal to the max size. It already accounts for the empty slot via
> a "+ 1":
>
> BUILD_BUG_ON((HIST_STACKTRACE_DEPTH + 1) * sizeof(long) >= STR_VAR_LEN_MAX);
>
> Change that from ">=" to just ">", as now they are equal.
>
> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
The patch itself looks good to me, thanks!
Reviewed-by: Tom Zanussi <zanussi@kernel.org>
> ---
> kernel/trace/trace.h | 2 +-
> kernel/trace/trace_events_hist.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
> index 8888fc9335b6..69e7defba6c6 100644
> --- a/kernel/trace/trace.h
> +++ b/kernel/trace/trace.h
> @@ -128,7 +128,7 @@ enum trace_type {
>
> #define FAULT_STRING "(fault)"
>
> -#define HIST_STACKTRACE_DEPTH 16
> +#define HIST_STACKTRACE_DEPTH 31
> #define HIST_STACKTRACE_SIZE (HIST_STACKTRACE_DEPTH * sizeof(unsigned long))
> #define HIST_STACKTRACE_SKIP 5
>
> diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
> index 24441b30a1a4..dfb7c4754bf8 100644
> --- a/kernel/trace/trace_events_hist.c
> +++ b/kernel/trace/trace_events_hist.c
> @@ -3154,7 +3154,7 @@ static inline void __update_field_vars(struct tracing_map_elt *elt,
> u64 var_val;
>
> /* Make sure stacktrace can fit in the string variable length */
> - BUILD_BUG_ON((HIST_STACKTRACE_DEPTH + 1) * sizeof(long) >= STR_VAR_LEN_MAX);
> + BUILD_BUG_ON((HIST_STACKTRACE_DEPTH + 1) * sizeof(long) > STR_VAR_LEN_MAX);
>
> for (i = 0, j = field_var_str_start; i < n_field_vars; i++) {
> struct field_var *field_var = field_vars[i];
^ permalink raw reply
* Re: [PATCH v10 19/30] KVM: arm64: Add PKVM_DISABLE_STAGE2_ON_PANIC
From: Kalesh Singh @ 2026-01-26 21:31 UTC (permalink / raw)
To: Vincent Donnefort
Cc: rostedt, mhiramat, mathieu.desnoyers, linux-trace-kernel, maz,
oliver.upton, joey.gouly, suzuki.poulose, yuzenghui, kvmarm,
linux-arm-kernel, jstultz, qperret, will, aneesh.kumar,
kernel-team, linux-kernel
In-Reply-To: <20260126104419.1649811-20-vdonnefort@google.com>
On Mon, Jan 26, 2026 at 2:44 AM Vincent Donnefort <vdonnefort@google.com> wrote:
>
> On NVHE_EL2_DEBUG, when using pKVM, the host stage-2 is relaxed to grant
> the kernel access to the stacktrace, hypervisor bug table and text to
> symbolize addresses. This is unsafe for production. In preparation for
> adding more debug options to NVHE_EL2_DEBUG, decouple the stage-2
> relaxation into a separate option.
>
> While at it, rename PROTECTED_NVHE_STACKTRACE into PKVM_STACKTRACE,
> following the same naming scheme as PKVM_DISABLE_STAGE2_ON_PANIC.
>
> Cc: Kalesh Singh <kaleshsingh@google.com>
> Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Kalesh Singh <kaleshsingh@google.com>
Thanks,
Kalesh
>
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 4f803fd1c99a..6498dec00fe9 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -43,9 +43,27 @@ menuconfig KVM
>
> If unsure, say N.
>
> +if KVM
> +
> +config PTDUMP_STAGE2_DEBUGFS
> + bool "Present the stage-2 pagetables to debugfs"
> + depends on DEBUG_KERNEL
> + depends on DEBUG_FS
> + depends on ARCH_HAS_PTDUMP
> + select PTDUMP
> + default n
> + help
> + Say Y here if you want to show the stage-2 kernel pagetables
> + layout in a debugfs file. This information is only useful for kernel developers
> + who are working in architecture specific areas of the kernel.
> + It is probably not a good idea to enable this feature in a production
> + kernel.
> +
> + If in doubt, say N.
> +
> config NVHE_EL2_DEBUG
> bool "Debug mode for non-VHE EL2 object"
> - depends on KVM
> + default n
> help
> Say Y here to enable the debug mode for the non-VHE KVM EL2 object.
> Failure reports will BUG() in the hypervisor. This is intended for
> @@ -53,10 +71,23 @@ config NVHE_EL2_DEBUG
>
> If unsure, say N.
>
> -config PROTECTED_NVHE_STACKTRACE
> - bool "Protected KVM hypervisor stacktraces"
> - depends on NVHE_EL2_DEBUG
> +if NVHE_EL2_DEBUG
> +
> +config PKVM_DISABLE_STAGE2_ON_PANIC
> + bool "Disable the host stage-2 on panic"
> default n
> + help
> + Relax the host stage-2 on hypervisor panic to allow the kernel to
> + unwind and symbolize the hypervisor stacktrace. This however tampers
> + the system security. This is intended for local EL2 hypervisor
> + development.
> +
> + If unsure, say N.
> +
> +config PKVM_STACKTRACE
> + bool "Protected KVM hypervisor stacktraces"
> + depends on PKVM_DISABLE_STAGE2_ON_PANIC
> + default y
> help
> Say Y here to enable pKVM hypervisor stacktraces on hyp_panic()
>
> @@ -66,21 +97,6 @@ config PROTECTED_NVHE_STACKTRACE
>
> If unsure, or not using protected nVHE (pKVM), say N.
>
> -config PTDUMP_STAGE2_DEBUGFS
> - bool "Present the stage-2 pagetables to debugfs"
> - depends on KVM
> - depends on DEBUG_KERNEL
> - depends on DEBUG_FS
> - depends on ARCH_HAS_PTDUMP
> - select PTDUMP
> - default n
> - help
> - Say Y here if you want to show the stage-2 kernel pagetables
> - layout in a debugfs file. This information is only useful for kernel developers
> - who are working in architecture specific areas of the kernel.
> - It is probably not a good idea to enable this feature in a production
> - kernel.
> -
> - If in doubt, say N.
> -
> +endif # NVHE_EL2_DEBUG
> +endif # KVM
> endif # VIRTUALIZATION
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index cc7d5d1709cb..54aedf93c78b 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -539,7 +539,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr,
>
> /* All hyp bugs, including warnings, are treated as fatal. */
> if (!is_protected_kvm_enabled() ||
> - IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) {
> + IS_ENABLED(CONFIG_PKVM_DISABLE_STAGE2_ON_PANIC)) {
> struct bug_entry *bug = find_bug(elr_in_kimg);
>
> if (bug)
> diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S
> index eef15b374abb..3092befcd97c 100644
> --- a/arch/arm64/kvm/hyp/nvhe/host.S
> +++ b/arch/arm64/kvm/hyp/nvhe/host.S
> @@ -120,7 +120,7 @@ SYM_FUNC_START(__hyp_do_panic)
>
> mov x29, x0
>
> -#ifdef CONFIG_NVHE_EL2_DEBUG
> +#ifdef PKVM_DISABLE_STAGE2_ON_PANIC
> /* Ensure host stage-2 is disabled */
> mrs x0, hcr_el2
> bic x0, x0, #HCR_VM
> diff --git a/arch/arm64/kvm/hyp/nvhe/stacktrace.c b/arch/arm64/kvm/hyp/nvhe/stacktrace.c
> index 5b6eeab1a774..7c832d60d22b 100644
> --- a/arch/arm64/kvm/hyp/nvhe/stacktrace.c
> +++ b/arch/arm64/kvm/hyp/nvhe/stacktrace.c
> @@ -34,7 +34,7 @@ static void hyp_prepare_backtrace(unsigned long fp, unsigned long pc)
> stacktrace_info->pc = pc;
> }
>
> -#ifdef CONFIG_PROTECTED_NVHE_STACKTRACE
> +#ifdef CONFIG_PKVM_STACKTRACE
> #include <asm/stacktrace/nvhe.h>
>
> DEFINE_PER_CPU(unsigned long [NVHE_STACKTRACE_SIZE/sizeof(long)], pkvm_stacktrace);
> @@ -134,11 +134,11 @@ static void pkvm_save_backtrace(unsigned long fp, unsigned long pc)
>
> unwind(&state, pkvm_save_backtrace_entry, &idx);
> }
> -#else /* !CONFIG_PROTECTED_NVHE_STACKTRACE */
> +#else /* !CONFIG_PKVM_STACKTRACE */
> static void pkvm_save_backtrace(unsigned long fp, unsigned long pc)
> {
> }
> -#endif /* CONFIG_PROTECTED_NVHE_STACKTRACE */
> +#endif /* CONFIG_PKVM_STACKTRACE */
>
> /*
> * kvm_nvhe_prepare_backtrace - prepare to dump the nVHE backtrace
> diff --git a/arch/arm64/kvm/stacktrace.c b/arch/arm64/kvm/stacktrace.c
> index af5eec681127..9724c320126b 100644
> --- a/arch/arm64/kvm/stacktrace.c
> +++ b/arch/arm64/kvm/stacktrace.c
> @@ -197,7 +197,7 @@ static void hyp_dump_backtrace(unsigned long hyp_offset)
> kvm_nvhe_dump_backtrace_end();
> }
>
> -#ifdef CONFIG_PROTECTED_NVHE_STACKTRACE
> +#ifdef CONFIG_PKVM_STACKTRACE
> DECLARE_KVM_NVHE_PER_CPU(unsigned long [NVHE_STACKTRACE_SIZE/sizeof(long)],
> pkvm_stacktrace);
>
> @@ -225,12 +225,12 @@ static void pkvm_dump_backtrace(unsigned long hyp_offset)
> kvm_nvhe_dump_backtrace_entry((void *)hyp_offset, stacktrace[i]);
> kvm_nvhe_dump_backtrace_end();
> }
> -#else /* !CONFIG_PROTECTED_NVHE_STACKTRACE */
> +#else /* !CONFIG_PKVM_STACKTRACE */
> static void pkvm_dump_backtrace(unsigned long hyp_offset)
> {
> - kvm_err("Cannot dump pKVM nVHE stacktrace: !CONFIG_PROTECTED_NVHE_STACKTRACE\n");
> + kvm_err("Cannot dump pKVM nVHE stacktrace: !CONFIG_PKVM_STACKTRACE\n");
> }
> -#endif /* CONFIG_PROTECTED_NVHE_STACKTRACE */
> +#endif /* CONFIG_PKVM_STACKTRACE */
>
> /*
> * kvm_nvhe_dump_backtrace - Dump KVM nVHE hypervisor backtrace.
> --
> 2.52.0.457.g6b5491de43-goog
>
^ permalink raw reply
* [PATCHv2 bpf-next 6/6] selftests/bpf: Allow to benchmark trigger with stacktrace
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
Adding support to call bpf_get_stackid helper from trigger programs,
so far added for kprobe multi.
Adding the --stacktrace/-g option to enable it.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/testing/selftests/bpf/bench.c | 4 ++
tools/testing/selftests/bpf/bench.h | 1 +
.../selftests/bpf/benchs/bench_trigger.c | 1 +
.../selftests/bpf/progs/trigger_bench.c | 46 +++++++++++++++----
4 files changed, 43 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c
index bd29bb2e6cb5..8368bd3a0665 100644
--- a/tools/testing/selftests/bpf/bench.c
+++ b/tools/testing/selftests/bpf/bench.c
@@ -265,6 +265,7 @@ static const struct argp_option opts[] = {
{ "verbose", 'v', NULL, 0, "Verbose debug output"},
{ "affinity", 'a', NULL, 0, "Set consumer/producer thread affinity"},
{ "quiet", 'q', NULL, 0, "Be more quiet"},
+ { "stacktrace", 's', NULL, 0, "Get stack trace"},
{ "prod-affinity", ARG_PROD_AFFINITY_SET, "CPUSET", 0,
"Set of CPUs for producer threads; implies --affinity"},
{ "cons-affinity", ARG_CONS_AFFINITY_SET, "CPUSET", 0,
@@ -350,6 +351,9 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state)
case 'q':
env.quiet = true;
break;
+ case 's':
+ env.stacktrace = true;
+ break;
case ARG_PROD_AFFINITY_SET:
env.affinity = true;
if (parse_num_list(arg, &env.prod_cpus.cpus,
diff --git a/tools/testing/selftests/bpf/bench.h b/tools/testing/selftests/bpf/bench.h
index bea323820ffb..7cf21936e7ed 100644
--- a/tools/testing/selftests/bpf/bench.h
+++ b/tools/testing/selftests/bpf/bench.h
@@ -26,6 +26,7 @@ struct env {
bool list;
bool affinity;
bool quiet;
+ bool stacktrace;
int consumer_cnt;
int producer_cnt;
int nr_cpus;
diff --git a/tools/testing/selftests/bpf/benchs/bench_trigger.c b/tools/testing/selftests/bpf/benchs/bench_trigger.c
index 34018fc3927f..aeec9edd3851 100644
--- a/tools/testing/selftests/bpf/benchs/bench_trigger.c
+++ b/tools/testing/selftests/bpf/benchs/bench_trigger.c
@@ -146,6 +146,7 @@ static void setup_ctx(void)
bpf_program__set_autoload(ctx.skel->progs.trigger_driver, true);
ctx.skel->rodata->batch_iters = args.batch_iters;
+ ctx.skel->rodata->stacktrace = env.stacktrace;
}
static void load_ctx(void)
diff --git a/tools/testing/selftests/bpf/progs/trigger_bench.c b/tools/testing/selftests/bpf/progs/trigger_bench.c
index 2898b3749d07..4ea0422d1042 100644
--- a/tools/testing/selftests/bpf/progs/trigger_bench.c
+++ b/tools/testing/selftests/bpf/progs/trigger_bench.c
@@ -25,6 +25,34 @@ static __always_inline void inc_counter(void)
__sync_add_and_fetch(&hits[cpu & CPU_MASK].value, 1);
}
+volatile const int stacktrace;
+
+typedef __u64 stack_trace_t[128];
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, stack_trace_t);
+} stack_heap SEC(".maps");
+
+static __always_inline void do_stacktrace(void *ctx)
+{
+ if (!stacktrace)
+ return;
+
+ __u64 *ptr = bpf_map_lookup_elem(&stack_heap, &(__u32){0});
+
+ if (ptr)
+ bpf_get_stack(ctx, ptr, sizeof(stack_trace_t), 0);
+}
+
+static __always_inline void handle(void *ctx)
+{
+ inc_counter();
+ do_stacktrace(ctx);
+}
+
SEC("?uprobe")
int bench_trigger_uprobe(void *ctx)
{
@@ -81,21 +109,21 @@ int trigger_driver_kfunc(void *ctx)
SEC("?kprobe/bpf_get_numa_node_id")
int bench_trigger_kprobe(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
SEC("?kretprobe/bpf_get_numa_node_id")
int bench_trigger_kretprobe(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
SEC("?kprobe.multi/bpf_get_numa_node_id")
int bench_trigger_kprobe_multi(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
@@ -108,7 +136,7 @@ int bench_kprobe_multi_empty(void *ctx)
SEC("?kretprobe.multi/bpf_get_numa_node_id")
int bench_trigger_kretprobe_multi(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
@@ -121,34 +149,34 @@ int bench_kretprobe_multi_empty(void *ctx)
SEC("?fentry/bpf_get_numa_node_id")
int bench_trigger_fentry(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
SEC("?fexit/bpf_get_numa_node_id")
int bench_trigger_fexit(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
SEC("?fmod_ret/bpf_modify_return_test_tp")
int bench_trigger_fmodret(void *ctx)
{
- inc_counter();
+ handle(ctx);
return -22;
}
SEC("?tp/bpf_test_run/bpf_trigger_tp")
int bench_trigger_tp(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
SEC("?raw_tp/bpf_trigger_tp")
int bench_trigger_rawtp(void *ctx)
{
- inc_counter();
+ handle(ctx);
return 0;
}
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 5/6] selftests/bpf: Add stacktrace ips test for fentry/fexit
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
Adding test that attaches fentry/fexitand verifies the
ORC stacktrace matches expected functions.
The test is only for ORC unwinder to keep it simple.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
.../selftests/bpf/prog_tests/stacktrace_ips.c | 51 +++++++++++++++++++
.../selftests/bpf/progs/stacktrace_ips.c | 20 ++++++++
2 files changed, 71 insertions(+)
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
index 852830536109..da42b00e3d1f 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
@@ -183,6 +183,53 @@ static void test_stacktrace_ips_kprobe(bool retprobe)
stacktrace_ips__destroy(skel);
}
+static void test_stacktrace_ips_trampoline(bool retprobe)
+{
+ LIBBPF_OPTS(bpf_test_run_opts, topts);
+ struct stacktrace_ips *skel;
+
+ skel = stacktrace_ips__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
+ return;
+
+ if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+ test__skip();
+ goto cleanup;
+ }
+
+ if (retprobe) {
+ skel->links.fexit_test = bpf_program__attach_trace(skel->progs.fexit_test);
+ if (!ASSERT_OK_PTR(skel->links.fexit_test, "bpf_program__attach_trace"))
+ goto cleanup;
+ } else {
+ skel->links.fentry_test = bpf_program__attach_trace(skel->progs.fentry_test);
+ if (!ASSERT_OK_PTR(skel->links.fentry_test, "bpf_program__attach_trace"))
+ goto cleanup;
+ }
+
+ trigger_module_test_read(1);
+
+ load_kallsyms();
+
+ if (retprobe) {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 4,
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ } else {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 5,
+ ksym_get_addr("bpf_testmod_stacktrace_test"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ }
+
+cleanup:
+ stacktrace_ips__destroy(skel);
+}
+
static void __test_stacktrace_ips(void)
{
if (test__start_subtest("kprobe_multi"))
@@ -195,6 +242,10 @@ static void __test_stacktrace_ips(void)
test_stacktrace_ips_kprobe(false);
if (test__start_subtest("kretprobe"))
test_stacktrace_ips_kprobe(true);
+ if (test__start_subtest("fentry"))
+ test_stacktrace_ips_trampoline(false);
+ if (test__start_subtest("fexit"))
+ test_stacktrace_ips_trampoline(true);
}
#else
static void __test_stacktrace_ips(void)
diff --git a/tools/testing/selftests/bpf/progs/stacktrace_ips.c b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
index cae077a4061b..6830f2978613 100644
--- a/tools/testing/selftests/bpf/progs/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
@@ -53,4 +53,24 @@ int rawtp_test(void *ctx)
return 0;
}
+SEC("fentry/bpf_testmod_stacktrace_test")
+int fentry_test(struct pt_regs *ctx)
+{
+ /*
+ * Skip 2 bpf_program/trampoline stack entries:
+ * - bpf_prog_bd1f7a949f55fb03_fentry_test
+ * - bpf_trampoline_182536277701
+ */
+ stack_key = bpf_get_stackid(ctx, &stackmap, 2);
+ return 0;
+}
+
+SEC("fexit/bpf_testmod_stacktrace_test")
+int fexit_test(struct pt_regs *ctx)
+{
+ /* Skip 2 bpf_program/trampoline stack entries, check fentry_test. */
+ stack_key = bpf_get_stackid(ctx, &stackmap, 2);
+ return 0;
+}
+
char _license[] SEC("license") = "GPL";
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 4/6] selftests/bpf: Add stacktrace ips test for kprobe/kretprobe
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
Adding test that attaches kprobe/kretprobe and verifies the
ORC stacktrace matches expected functions.
The test is only for ORC unwinder to keep it simple.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
.../selftests/bpf/prog_tests/stacktrace_ips.c | 50 +++++++++++++++++++
.../selftests/bpf/progs/stacktrace_ips.c | 7 +++
2 files changed, 57 insertions(+)
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
index c93718dafd9b..852830536109 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
@@ -137,6 +137,52 @@ static void test_stacktrace_ips_raw_tp(void)
stacktrace_ips__destroy(skel);
}
+static void test_stacktrace_ips_kprobe(bool retprobe)
+{
+ LIBBPF_OPTS(bpf_kprobe_opts, opts,
+ .retprobe = retprobe
+ );
+ LIBBPF_OPTS(bpf_test_run_opts, topts);
+ struct stacktrace_ips *skel;
+
+ skel = stacktrace_ips__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
+ return;
+
+ if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+ test__skip();
+ goto cleanup;
+ }
+
+ skel->links.kprobe_test = bpf_program__attach_kprobe_opts(
+ skel->progs.kprobe_test,
+ "bpf_testmod_stacktrace_test", &opts);
+ if (!ASSERT_OK_PTR(skel->links.kprobe_test, "bpf_program__attach_kprobe_opts"))
+ goto cleanup;
+
+ trigger_module_test_read(1);
+
+ load_kallsyms();
+
+ if (retprobe) {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 4,
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ } else {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 5,
+ ksym_get_addr("bpf_testmod_stacktrace_test"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ }
+
+cleanup:
+ stacktrace_ips__destroy(skel);
+}
+
static void __test_stacktrace_ips(void)
{
if (test__start_subtest("kprobe_multi"))
@@ -145,6 +191,10 @@ static void __test_stacktrace_ips(void)
test_stacktrace_ips_kprobe_multi(true);
if (test__start_subtest("raw_tp"))
test_stacktrace_ips_raw_tp();
+ if (test__start_subtest("kprobe"))
+ test_stacktrace_ips_kprobe(false);
+ if (test__start_subtest("kretprobe"))
+ test_stacktrace_ips_kprobe(true);
}
#else
static void __test_stacktrace_ips(void)
diff --git a/tools/testing/selftests/bpf/progs/stacktrace_ips.c b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
index a96c8150d7f5..cae077a4061b 100644
--- a/tools/testing/selftests/bpf/progs/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
@@ -31,6 +31,13 @@ int unused(void)
__u32 stack_key;
+SEC("kprobe")
+int kprobe_test(struct pt_regs *ctx)
+{
+ stack_key = bpf_get_stackid(ctx, &stackmap, 0);
+ return 0;
+}
+
SEC("kprobe.multi")
int kprobe_multi_test(struct pt_regs *ctx)
{
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 3/6] selftests/bpf: Fix kprobe multi stacktrace_ips test
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
We now include the attached function in the stack trace,
fixing the test accordingly.
Fixes: c9e208fa93cd ("selftests/bpf: Add stacktrace ips test for kprobe_multi/kretprobe_multi")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
.../selftests/bpf/prog_tests/stacktrace_ips.c | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
index c9efdd2a5b18..c93718dafd9b 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
@@ -74,11 +74,20 @@ static void test_stacktrace_ips_kprobe_multi(bool retprobe)
load_kallsyms();
- check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 4,
- ksym_get_addr("bpf_testmod_stacktrace_test_3"),
- ksym_get_addr("bpf_testmod_stacktrace_test_2"),
- ksym_get_addr("bpf_testmod_stacktrace_test_1"),
- ksym_get_addr("bpf_testmod_test_read"));
+ if (retprobe) {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 4,
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ } else {
+ check_stacktrace_ips(bpf_map__fd(skel->maps.stackmap), skel->bss->stack_key, 5,
+ ksym_get_addr("bpf_testmod_stacktrace_test"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_3"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_2"),
+ ksym_get_addr("bpf_testmod_stacktrace_test_1"),
+ ksym_get_addr("bpf_testmod_test_read"));
+ }
cleanup:
stacktrace_ips__destroy(skel);
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 2/6] x86/fgraph,bpf: Switch kprobe_multi program stack unwind to hw_regs path
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Mahe Tardy, Peter Zijlstra, bpf, linux-trace-kernel, x86,
Yonghong Song, Song Liu, Andrii Nakryiko
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
Mahe reported missing function from stack trace on top of kprobe
multi program. The missing function is the very first one in the
stacktrace, the one that the bpf program is attached to.
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
('*' is used for kprobe_multi attachment)
The reason is that the previous change (the Fixes commit) fixed
stack unwind for tracepoint, but removed attached function address
from the stack trace on top of kprobe multi programs, which I also
overlooked in the related test (check following patch).
The tracepoint and kprobe_multi have different stack setup, but use
same unwind path. I think it's better to keep the previous change,
which fixed tracepoint unwind and instead change the kprobe multi
unwind as explained below.
The bpf program stack unwind calls perf_callchain_kernel for kernel
portion and it follows two unwind paths based on X86_EFLAGS_FIXED
bit in pt_regs.flags.
When the bit set we unwind from stack represented by pt_regs argument,
otherwise we unwind currently executed stack up to 'first_frame'
boundary.
The 'first_frame' value is taken from regs.rsp value, but ftrace_caller
and ftrace_regs_caller (ftrace trampoline) functions set the regs.rsp
to the previous stack frame, so we skip the attached function entry.
If we switch kprobe_multi unwind to use the X86_EFLAGS_FIXED bit,
we set the start of the unwind to the attached function address.
As another benefit we also cut extra unwind cycles needed to reach
the 'first_frame' boundary.
The speedup can be measured with trigger bench for kprobe_multi
program and stacktrace support.
- trigger bench with stacktrace on current code:
kprobe-multi : 0.810 ± 0.001M/s
kretprobe-multi: 0.808 ± 0.001M/s
- and with the fix:
kprobe-multi : 1.264 ± 0.001M/s
kretprobe-multi: 1.401 ± 0.002M/s
With the fix, the entry probe stacktrace:
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
__x64_sys_newuname+9
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
The return probe skips the attached function, because it's no longer
on the stack at the point of the unwind and this way is the same how
standard kretprobe works.
# bpftrace -e 'kretprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
Fixes: 6d08340d1e35 ("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"")
Reported-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
arch/x86/include/asm/ftrace.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index b08c95872eed..c56e1e63b893 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -57,7 +57,7 @@ arch_ftrace_get_regs(struct ftrace_regs *fregs)
}
#define arch_ftrace_partial_regs(regs) do { \
- regs->flags &= ~X86_EFLAGS_FIXED; \
+ regs->flags |= X86_EFLAGS_FIXED; \
regs->cs = __KERNEL_CS; \
} while (0)
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 1/6] x86/fgraph: Fix return_to_handler regs.rsp value
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
In-Reply-To: <20260126211837.472802-1-jolsa@kernel.org>
The previous change (Fixes commit) messed up the rsp register value,
which is wrong because it's already adjusted with FRAME_SIZE, we need
the original rsp value.
This change does not affect fprobe current kernel unwind, the !perf_hw_regs
path perf_callchain_kernel:
if (perf_hw_regs(regs)) {
if (perf_callchain_store(entry, regs->ip))
return;
unwind_start(&state, current, regs, NULL);
} else {
unwind_start(&state, current, NULL, (void *)regs->sp);
}
which uses pt_regs.sp as first_frame boundary (FRAME_SIZE shift makes
no difference, unwind stil stops at the right frame).
This change fixes the other path when we want to unwind directly from
pt_regs sp/fp/ip state, which is coming in following change.
Fixes: 20a0bc10272f ("x86/fgraph,bpf: Fix stack ORC unwind from kprobe_multi return probe")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
arch/x86/kernel/ftrace_64.S | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
index a132608265f6..62c1c93aa1c6 100644
--- a/arch/x86/kernel/ftrace_64.S
+++ b/arch/x86/kernel/ftrace_64.S
@@ -364,6 +364,9 @@ SYM_CODE_START(return_to_handler)
UNWIND_HINT_UNDEFINED
ANNOTATE_NOENDBR
+ /* Store original rsp for pt_regs.sp value. */
+ movq %rsp, %rdi
+
/* Restore return_to_handler value that got eaten by previous ret instruction. */
subq $8, %rsp
UNWIND_HINT_FUNC
@@ -374,7 +377,7 @@ SYM_CODE_START(return_to_handler)
movq %rax, RAX(%rsp)
movq %rdx, RDX(%rsp)
movq %rbp, RBP(%rsp)
- movq %rsp, RSP(%rsp)
+ movq %rdi, RSP(%rsp)
movq %rsp, %rdi
call ftrace_return_to_handler
--
2.52.0
^ permalink raw reply related
* [PATCHv2 bpf-next 0/6] x86/fgraph,bpf: Fix ORC stack unwind from kprobe_multi
From: Jiri Olsa @ 2026-01-26 21:18 UTC (permalink / raw)
To: Masami Hiramatsu, Steven Rostedt, Josh Poimboeuf
Cc: Peter Zijlstra, bpf, linux-trace-kernel, x86, Yonghong Song,
Song Liu, Andrii Nakryiko, Mahe Tardy
hi,
Mahe reported missing function from stack trace on top of kprobe multi
program. It turned out the latest fix [1] needs some more fixing.
v2 changes:
- keep the unwind same as for kprobes, attached function
is part of entry probe stacktrace, not kretprobe [Steven]
- several change in trigger bench [Andrii]
- added selftests for standard kprobes and fentry/fexit probes [Andrii]
Note I'll try to add similar stacktrace adjustment for fentry/fexit
in separate patchset to not complicate this change.
thanks,
jirka
[1] https://lore.kernel.org/bpf/20251104215405.168643-1-jolsa@kernel.org/
---
Jiri Olsa (6):
x86/fgraph: Fix return_to_handler regs.rsp value
x86/fgraph,bpf: Switch kprobe_multi program stack unwind to hw_regs path
selftests/bpf: Fix kprobe multi stacktrace_ips test
selftests/bpf: Add stacktrace ips test for kprobe/kretprobe
selftests/bpf: Add stacktrace ips test for fentry/fexit
selftests/bpf: Allow to benchmark trigger with stacktrace
arch/x86/include/asm/ftrace.h | 2 +-
arch/x86/kernel/ftrace_64.S | 5 +++-
tools/testing/selftests/bpf/bench.c | 4 ++++
tools/testing/selftests/bpf/bench.h | 1 +
tools/testing/selftests/bpf/benchs/bench_trigger.c | 1 +
tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c | 120 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
tools/testing/selftests/bpf/progs/stacktrace_ips.c | 27 ++++++++++++++++++++++
tools/testing/selftests/bpf/progs/trigger_bench.c | 46 +++++++++++++++++++++++++++++--------
8 files changed, 190 insertions(+), 16 deletions(-)
^ permalink raw reply
* [PATCH] tracing: Remove duplicate ENABLE_EVENT_STR and DISABLE_EVENT_STR macros
From: Steven Rostedt @ 2026-01-26 18:00 UTC (permalink / raw)
To: LKML, Linux Trace Kernel; +Cc: Masami Hiramatsu, Mathieu Desnoyers, Tom Zanussi
From: Steven Rostedt <rostedt@goodmis.org>
The macros ENABLE_EVENT_STR and DISABLE_EVENT_STR were added to trace.h so
that more than one file can have access to them, but was never removed
from their original location. Remove the duplicates.
Fixes: d0bad49bb0a09 ("tracing: Add enable_hist/disable_hist triggers")
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
kernel/trace/trace_events.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 166b0d59be3d..af6d1fe5cab7 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -4113,11 +4113,6 @@ void trace_put_event_file(struct trace_event_file *file)
EXPORT_SYMBOL_GPL(trace_put_event_file);
#ifdef CONFIG_DYNAMIC_FTRACE
-
-/* Avoid typos */
-#define ENABLE_EVENT_STR "enable_event"
-#define DISABLE_EVENT_STR "disable_event"
-
struct event_probe_data {
struct trace_event_file *file;
unsigned long count;
--
2.51.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox