From: Dapeng Mi <dapeng1.mi@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <ak@linux.intel.com>,
Eranian Stephane <eranian@google.com>,
Sandipan Das <sandipan.das@amd.com>,
Andrii Nakryiko <andrii@kernel.org>, Song Liu <song@kernel.org>,
Ravi Bangoria <ravi.bangoria@amd.com>,
Ananth Narayan <ananth.narayan@amd.com>
Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
bpf@vger.kernel.org, Dapeng Mi <dapeng1.mi@intel.com>,
Zide Chen <zide.chen@intel.com>,
Falcon Thomas <thomas.falcon@intel.com>,
Xudong Hao <xudong.hao@intel.com>,
Dapeng Mi <dapeng1.mi@linux.intel.com>,
Sashiko <sashiko-bot@kernel.org>, KP Singh <kpsingh@kernel.org>,
Matt Bobrowski <matt@bobrowski.net>, Jiri Olsa <jolsa@kernel.org>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
Daniel Xu <dxu@dxuuu.xyz>
Subject: [Patch v7 23/24] perf/core: Don't mutate shared branch stack depth
Date: Wed, 18 Nov 2026 11:14:52 +0800 [thread overview]
Message-ID: <20261118031453.1353625-24-dapeng1.mi@linux.intel.com> (raw)
In-Reply-To: <20261118031453.1353625-1-dapeng1.mi@linux.intel.com>
perf_sample_save_brstack() clamps the branch stack depth by directly
overwriting brs->nr with min(event->attr.sample_max_stack, brs->nr).
This is unsafe because the branch stack buffer is shared by all branch
events in the same PMI: once one event with a small sample_max_stack
truncates brs->nr, every subsequent event processed in that PMI sees
the shrunken depth and silently loses branch records it asked for.
Add perf_event_brstack_depth(), which computes the per-event depth
without modifying the shared branch stack. Use it in the relevant callers
so each event is limited only by its own sample_max_stack, without
clobbering the shared LBR state.
bpf_read_branch_records() also relied on the clamped brs->nr, so switch
it to perf_event_brstack_depth() to preserve the same per-event limit for
records returned to BPF programs.
Fixes: c53e14f1ea4a ("perf: Extend per event callchain limit to branch stack")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Cc: KP Singh <kpsingh@kernel.org>
Cc: Matt Bobrowski <matt@bobrowski.net>
Cc: Song Liu <song@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Daniel Xu <dxu@dxuuu.xyz>
---
include/linux/perf_event.h | 13 ++++++++++---
kernel/events/core.c | 8 ++++----
kernel/trace/bpf_trace.c | 6 ++++--
3 files changed, 18 insertions(+), 9 deletions(-)
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 19de3f1cbd6e..35d3cc4cc07e 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1424,12 +1424,19 @@ static inline bool has_branch_stack(struct perf_event *event)
return event->attr.sample_type & PERF_SAMPLE_BRANCH_STACK;
}
+static inline u64 perf_event_brstack_depth(struct perf_event *event,
+ struct perf_branch_stack *brs)
+{
+ return min_t(u16, event->attr.sample_max_stack, brs->nr);
+}
+
static inline void perf_sample_save_brstack(struct perf_sample_data *data,
struct perf_event *event,
struct perf_branch_stack *brs,
u64 *brs_cntr)
{
int size = sizeof(u64); /* nr */
+ u64 nr;
if (!has_branch_stack(event))
return;
@@ -1439,9 +1446,9 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data,
if (branch_sample_hw_index(event))
size += sizeof(u64);
- brs->nr = min_t(u16, event->attr.sample_max_stack, brs->nr);
+ nr = perf_event_brstack_depth(event, brs);
- size += brs->nr * sizeof(struct perf_branch_entry);
+ size += nr * sizeof(struct perf_branch_entry);
/*
* The extension space for counters is appended after the
@@ -1449,7 +1456,7 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data,
* of events of each branch.
*/
if (brs_cntr)
- size += brs->nr * sizeof(u64);
+ size += nr * sizeof(u64);
data->br_stack = brs;
data->br_stack_cntr = brs_cntr;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 457d75ea7142..8b80ca788523 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -8462,12 +8462,12 @@ void perf_output_sample(struct perf_output_handle *handle,
if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
if (data->br_stack) {
+ u64 nr = perf_event_brstack_depth(event, data->br_stack);
size_t size;
- size = data->br_stack->nr
- * sizeof(struct perf_branch_entry);
+ size = nr * sizeof(struct perf_branch_entry);
- perf_output_put(handle, data->br_stack->nr);
+ perf_output_put(handle, nr);
if (branch_sample_hw_index(event))
perf_output_put(handle, data->br_stack->hw_idx);
perf_output_copy(handle, data->br_stack->entries, size);
@@ -8476,7 +8476,7 @@ void perf_output_sample(struct perf_output_handle *handle,
* right after the struct perf_branch_stack.
*/
if (data->br_stack_cntr) {
- size = data->br_stack->nr * sizeof(u64);
+ size = nr * sizeof(u64);
perf_output_copy(handle, data->br_stack_cntr, size);
}
} else {
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 2aec56d7f3d3..7c42bbf05c2f 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -1549,6 +1549,7 @@ BPF_CALL_4(bpf_read_branch_records, struct bpf_perf_event_data_kern *, ctx,
static const u32 br_entry_size = sizeof(struct perf_branch_entry);
struct perf_branch_stack *br_stack = ctx->data->br_stack;
u32 to_copy;
+ u64 nr;
if (unlikely(flags & ~BPF_F_GET_BRANCH_RECORDS_SIZE))
return -EINVAL;
@@ -1559,13 +1560,14 @@ BPF_CALL_4(bpf_read_branch_records, struct bpf_perf_event_data_kern *, ctx,
if (unlikely(!br_stack))
return -ENOENT;
+ nr = perf_event_brstack_depth(ctx->event, br_stack);
if (flags & BPF_F_GET_BRANCH_RECORDS_SIZE)
- return br_stack->nr * br_entry_size;
+ return nr * br_entry_size;
if (!buf || (size % br_entry_size != 0))
return -EINVAL;
- to_copy = min_t(u32, br_stack->nr * br_entry_size, size);
+ to_copy = min_t(u32, nr * br_entry_size, size);
memcpy(buf, br_stack->entries, to_copy);
return to_copy;
--
2.34.1
next prev parent reply other threads:[~2026-11-18 3:24 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 3:14 [Patch v7 00/24] perf/x86: Fix multiple LBR sampling issues Dapeng Mi
2026-11-18 3:14 ` [Patch v7 01/24] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES Dapeng Mi
2026-11-18 3:50 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 02/24] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack Dapeng Mi
2026-11-18 3:48 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 03/24] perf/x86/amd/lbr: Limit branch snapshots to valid entries Dapeng Mi
2026-11-18 3:34 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 04/24] perf/x86/intel: " Dapeng Mi
2026-11-18 3:37 ` sashiko-bot
2026-11-18 4:10 ` bot+bpf-ci
2026-11-18 6:01 ` Mi, Dapeng
2026-11-18 3:14 ` [Patch v7 05/24] perf/x86: Add lbr_enabled to track LBR state Dapeng Mi
2026-11-18 3:58 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 06/24] perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled Dapeng Mi
2026-11-18 3:43 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 07/24] perf/x86/intel: Add helper to enable PMU without LBR Dapeng Mi
2026-11-18 3:28 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 08/24] perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled Dapeng Mi
2026-11-18 3:39 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 09/24] perf/x86/intel: Keep LBR frozen throughout PMI handling Dapeng Mi
2026-11-18 3:42 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 10/24] perf: Add counted-bounds tracking for perf_branch_stack Dapeng Mi
2026-11-18 3:35 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 11/24] perf/x86/intel: Allow LBR read helpers to operate on explicit stacks Dapeng Mi
2026-11-18 3:32 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 12/24] perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting Dapeng Mi
2026-11-18 3:43 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 13/24] perf/x86/intel: Always snapshot live hardware LBR state for BPF Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 14/24] perf/x86/intel: Account lbr_pebs_users for arch-PEBS events Dapeng Mi
2026-11-18 3:34 ` sashiko-bot
2026-11-18 4:10 ` bot+bpf-ci
2026-11-18 6:28 ` Mi, Dapeng
2026-11-18 3:14 ` [Patch v7 15/24] perf/x86/intel: Set PERF_EV_CAP_SIBLING for pure branch counter events Dapeng Mi
2026-11-18 4:05 ` sashiko-bot
2026-11-18 6:32 ` Mi, Dapeng
2026-11-18 3:14 ` [Patch v7 16/24] perf/x86/intel: Account LBR user counts " Dapeng Mi
2026-11-18 3:49 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 17/24] perf/x86/intel: Refresh LBR stack for non-PEBS LBR events when needed Dapeng Mi
2026-11-18 4:09 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 18/24] perf/x86/intel: Fix draining of un-overflowed PEBS buffer in PMI handler Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 19/24] perf/x86/intel: Drain PEBS buffer for branch events with PEBS LBR users Dapeng Mi
2026-11-18 3:52 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 20/24] perf/x86/intel: Clear raw LBR counters in the LBR stack Dapeng Mi
2026-11-18 3:50 ` sashiko-bot
2026-11-18 3:54 ` bot+bpf-ci
2026-11-18 6:48 ` Mi, Dapeng
2026-11-18 3:14 ` [Patch v7 21/24] perf/x86/intel: Refresh LBR stack for LBR counter events in PMI handler Dapeng Mi
2026-11-18 3:45 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 22/24] perf/x86/intel: Pass LBR stack size into intel_pmu_store_lbr() Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` Dapeng Mi [this message]
2026-11-18 3:51 ` [Patch v7 23/24] perf/core: Don't mutate shared branch stack depth sashiko-bot
2026-11-18 3:14 ` [Patch v7 24/24] perf/x86/intel: Fix incorrect PERF_METRICS overflow bit clearing Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118031453.1353625-24-dapeng1.mi@linux.intel.com \
--to=dapeng1.mi@linux.intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=ananth.narayan@amd.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=dapeng1.mi@intel.com \
--cc=dxu@dxuuu.xyz \
--cc=eranian@google.com \
--cc=ihor.solodrai@linux.dev \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=kpsingh@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=matt@bobrowski.net \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=sandipan.das@amd.com \
--cc=sashiko-bot@kernel.org \
--cc=song@kernel.org \
--cc=thomas.falcon@intel.com \
--cc=xudong.hao@intel.com \
--cc=zide.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.