* [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events
2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif
@ 2026-11-19 21:10 ` Usama Arif
2026-11-19 21:10 ` [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address Usama Arif
2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif
2 siblings, 0 replies; 4+ messages in thread
From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, linux-perf-users,
linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel,
shakeel.butt, kernel-team
Cc: Usama Arif, stable
do_perf_sw_event() passes one on-stack struct perf_sample_data to every
matching event. perf_prepare_sample() fills in the fields an event
samples, marks them in data->sample_flags and skips fields that are
already marked, and those marks survive from one event to the next.
So a sampling event later in the list emits the sample ID, timestamp,
pid/tid and callchain prepared for an earlier one. Its record size is
computed from the earlier event's dynamic fields as well and can come
out smaller than what is written, in which case the kernel writes past
its ring-buffer reservation over the records that follow.
Two "perf record -e page-faults,context-switches -c 1 -a" sessions
running at the same time hit this: the one started first gets the
other's sample IDs and fails with "Bad address".
Commit 1d1bfe30dad5 ("perf/core: Fix perf_sample_data not properly
initialized for different swevents in perf_tp_event()") fixed the same
problem for tracepoints. Do the same here: initialize the struct for
each matching event. While at it, drop the unused data argument of
perf_swevent_match().
Fixes: bb447c27a467 ("perf/core: Set data->sample_flags in perf_prepare_sample()")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
kernel/events/core.c | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 687bd398275ba3..1c1ffd103c74a0 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -11142,7 +11142,6 @@ int perf_exclude_event(struct perf_event *event, struct pt_regs *regs)
static int perf_swevent_match(struct perf_event *event,
enum perf_type_id type,
u32 event_id,
- struct perf_sample_data *data,
struct pt_regs *regs)
{
if (event->attr.type != type)
@@ -11207,11 +11206,10 @@ find_swevent_head(struct swevent_htable *swhash, struct perf_event *event)
}
static void do_perf_sw_event(enum perf_type_id type, u32 event_id,
- u64 nr,
- struct perf_sample_data *data,
- struct pt_regs *regs)
+ u64 nr, struct pt_regs *regs, u64 addr)
{
struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable);
+ struct perf_sample_data data;
struct perf_event *event;
struct hlist_head *head;
@@ -11221,8 +11219,10 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id,
goto end;
hlist_for_each_entry_rcu(event, head, hlist_entry) {
- if (perf_swevent_match(event, type, event_id, data, regs))
- perf_swevent_event(event, nr, data, regs);
+ if (perf_swevent_match(event, type, event_id, regs)) {
+ perf_sample_data_init(&data, addr, 0);
+ perf_swevent_event(event, nr, &data, regs);
+ }
}
end:
rcu_read_unlock();
@@ -11243,13 +11243,10 @@ void perf_swevent_put_recursion_context(int rctx)
void ___perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
{
- struct perf_sample_data data;
-
if (WARN_ON_ONCE(!regs))
return;
- perf_sample_data_init(&data, addr, 0);
- do_perf_sw_event(PERF_TYPE_SOFTWARE, event_id, nr, &data, regs);
+ do_perf_sw_event(PERF_TYPE_SOFTWARE, event_id, nr, regs, addr);
}
void __perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
--
2.53.1-Meta
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address
2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif
2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif
@ 2026-11-19 21:10 ` Usama Arif
2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif
2 siblings, 0 replies; 4+ messages in thread
From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, linux-perf-users,
linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel,
shakeel.butt, kernel-team
Cc: Usama Arif, stable
A BPF program attached to a perf event gets a struct
bpf_perf_event_data, and reading its addr field turns into a load from
the perf_sample_data the sample is being assembled in. The verifier
allows that load no matter which fields the event samples.
perf_sample_data_init() used to store the address unconditionally, so
the program read 0 when there was none. To touch fewer cachelines, it
now stores the address only when it is non-zero, and
perf_prepare_sample() fills it in only when the event samples it
(PERF_SAMPLE_ADDR, PHYS_ADDR or DATA_PAGE_SIZE). For all other events,
including context switches, cpu-clock and hardware counters like
cycles, the program reads whatever was left in that stack slot, often a
kernel pointer.
Kernels built with CONFIG_INIT_STACK_ALL_ZERO, the default when the
compiler supports it, mostly hide this by clearing the struct on
function entry. With INIT_STACK_ALL_PATTERN the program reads the fill
pattern instead.
Clear the address before running the program if nobody filled it in,
the same way the SIGTRAP path already checks PERF_SAMPLE_ADDR before
using it. Events without a BPF program are not affected.
Fixes: 7b0846301531 ("perf: Use sample_flags for addr")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
kernel/events/core.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 1c1ffd103c74a0..57ea7e3c01a025 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -10819,6 +10819,9 @@ static int bpf_overflow_handler(struct perf_event *event,
prog = READ_ONCE(event->prog);
if (prog) {
perf_prepare_sample(data, event, regs);
+ /* bpf_perf_event_data::addr is readable without PERF_SAMPLE_ADDR */
+ if (!(data->sample_flags & PERF_SAMPLE_ADDR))
+ data->addr = 0;
ret = bpf_prog_run(prog, &ctx);
}
rcu_read_unlock();
--
2.53.1-Meta
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event()
2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif
2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif
2026-11-19 21:10 ` [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address Usama Arif
@ 2026-11-19 21:10 ` Usama Arif
2 siblings, 0 replies; 4+ messages in thread
From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, linux-perf-users,
linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel,
shakeel.butt, kernel-team
Cc: Usama Arif
Kernels built with CONFIG_INIT_STACK_ALL_ZERO, the default when the
compiler supports it, clear every local variable on function entry. For
the 256-byte struct perf_sample_data in do_perf_sw_event() that is a
memset, and it runs on every entry, before the list of events is even
looked up.
do_perf_sw_event() is entered for a software event, e.g. a context
switch, as soon as one event of that kind is open anywhere in the
system. Counting events add to event->count and never read the struct.
For every matching event, perf_sample_data_init() resets sample_flags,
period and dyn_size and stores the address if there is one; for a
sampling event, perf_prepare_sample() then fills in what its
sample_type asks for.
A single context-switch counter, e.g. one kept open by a monitoring
daemon, is enough for every context switch on the machine to pay for
the clearing. On hosts running many small VMs, ___perf_sw_event() is
0.3% of all kernel cycles, and the memset about a seventh of that.
Mark the struct __uninitialized, which is what kernels without stack
auto-initialization get anyway. Because its contents are copied into a
ring buffer that userspace maps, every field that is read must have
been written first. That holds for perf_output_sample(), and with the
previous patch also for what a BPF program can read through
bpf_perf_event_data.
In a KVM guest on AMD Zen 5 with software counters open on every CPU,
this cuts the time they add to a context switch from 54 ns to 44 ns
(sched_yield() between two threads on one CPU, with
spec_rstack_overflow=off).
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
kernel/events/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 57ea7e3c01a025..fa10ad7cfd99af 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id,
u64 nr, struct pt_regs *regs, u64 addr)
{
struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable);
- struct perf_sample_data data;
+ struct perf_sample_data data __uninitialized;
struct perf_event *event;
struct hlist_head *head;
--
2.53.1-Meta
^ permalink raw reply related [flat|nested] 4+ messages in thread