From: Dapeng Mi <dapeng1.mi@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <ak@linux.intel.com>,
Eranian Stephane <eranian@google.com>,
Sandipan Das <sandipan.das@amd.com>,
Andrii Nakryiko <andrii@kernel.org>, Song Liu <song@kernel.org>,
Ravi Bangoria <ravi.bangoria@amd.com>,
Ananth Narayan <ananth.narayan@amd.com>
Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
bpf@vger.kernel.org, Dapeng Mi <dapeng1.mi@intel.com>,
Zide Chen <zide.chen@intel.com>,
Falcon Thomas <thomas.falcon@intel.com>,
Xudong Hao <xudong.hao@intel.com>,
Dapeng Mi <dapeng1.mi@linux.intel.com>
Subject: [Patch v7 00/24] perf/x86: Fix multiple LBR sampling issues
Date: Wed, 18 Nov 2026 11:14:29 +0800 [thread overview]
Message-ID: <20261118031453.1353625-1-dapeng1.mi@linux.intel.com> (raw)
This patch series fixes several x86 LBR sampling issues discovered during
review and debugging. Most of them are related to BPF LBR snapshotting
support.
The series originally targeted the BPF stack out-of-bounds access issue,
but additional problems were found during debugging and code review,
including:
- PMU and LBR being re-enabled unconditionally at BPF snapshotting exit,
even when they had already been disabled before BPF snapshotting on both
AMD and Intel.
- LBR not being fully frozen in the PMI handler on Intel, which allows
branches in the PMI path to pollute the BPF LBR snapshot.
- BPF snapshotting overwriting the cached LBR stack of PEBS LBR events on
Intel.
- PEBS LBR events overwriting the LBR stack of non-PEBS LBR events on
Intel when both overflow in the same PMI.
- LBR counter events reporting zero counters on Intel when multiple
LBR counter events overflow in the same PMI.
Patch layout:
- Patch 01: Add a sanity check to ensure the LBR stack depth never
exceeds MAX_LBR_ENTRIES.
- Patch 02: Validate that hardware has actually written LBR data into
the XSAVE area before parsing it.
- Patch 03-04: Fix a potential LBR stack out-of-bounds access on AMD
and Intel, respectively.
- Patch 05-08: Fix incorrect PMU and LBR re-enabling in
perf_snapshot_branch_stack() on AMD and Intel, respectively.
- Patch 09: Keep the LBR frozen throughout PMI processing so that all
LBR events observe a consistent LBR stack in the PMI handler.
- Patch 10: Add bounds tracking for the number of entries counted in
struct perf_branch_stack.
- Patch 11-12: Add a dedicated LBR stack for BPF LBR snapshots to avoid
clobbering the default LBR stack used by other LBR events.
- Patch 13: Make perf_snapshot_branch_stack() snapshot the live
hardware LBR data instead of the cached PEBS LBR stack.
- Patch 14-16: Account lbr_users/lbr_pebs_users for arch-PEBS and pure
branch counter events.
- Patch 17-19: Prevent the PEBS LBR stack from overwriting the non-PEBS
LBR stack in several corner cases.
- Patch 20-21: Ensure LBR counters are not lost when multiple LBR
counter events overflow in the same PMI.
- Patch 22: Clamp the parsed PEBS LBR stack depth to x86_pmu.lbr_nr to
avoid an LBR stack out-of-bounds access.
- Patch 23: Fix the shared LBR stack depth truncation issue.
- Patch 24: Fix incorrect clearing of the PERF_METRICS overflow bit.
Changes in v7:
- Patch 06/08: Set/clear the cpuc->enabled flag to track the PMU enabling
state in the *_snapshot_branch_stack() helpers. (Sashiko)
- Patch 10: Rename __lbr_stack[] to __bhrb_stack[] and the related
helpers accordingly, to stay consistent with the original powerpc code.
- Patch 14-16: New patches. Fix lbr_users/lbr_pebs_users not being
accounted correctly in several corner cases. (Sashiko)
- Patch 17-19: New patches. Fix the non-PEBS LBR stack being overwritten
by the PEBS LBR stack in two corner cases. (Sashiko)
- Patch 24: New patch. Fix the PERF_METRICS overflow bit not being
cleared correctly.
Changes in v6:
- Patch 01: Move LBR stack depth validation into common
init_hw_perf_events(). (Peter)
- Patch 10: New patch, add __counted_by() tracking for flexible array
entries[] of perf_branch_stack{}. (Peter)
- Patch 14: Refresh LBR stack only when LBR users are active for AMD.
(Peter)
- Patch 15: Refactor code to ensure no duplicated LBR stack reading.
(Peter)
- Patch 16: New patch, remove dead lbr_pebs_users.
Changes in v5:
- Patch 02: Clear xsave->header.xfeatures before calling the xsaves
instructuion to read LBR states.
- Patch 05-18: New patches to fix newly discovered issues described above.
Changes in v4:
- Patch 01: Disable LBR if parsed x86_pmu.lbr_nr exceeds
MAX_LBR_ENTRIES (Sashiko).
- Patch 05: New patch, which always snapshots live hardware LBR stack
for BPF path (Sashiko).
Changes in v3:
- Patch 01: A new patch to ensure x86_pmu.lbr_nr not to exclude
MAX_LBR_ENTRIES (Internal Sashiko).
- Patch 03: Clean cached LBR stack when reseting hardware LBR registers,
which prevents stale LBR stack leakage.
- Patch 04: Don't reset cached LBR stack if all LBR events are PEBS
events. This ensures PEBS captured LBR stack isn't cleared (Sashiko).
Changes in v2:
- Include a fix for Intel LBR.
- Add Tested-by from Dapeng to the Intel fix.
https://lore.kernel.org/all/eb6c7bda-0dcf-4fac-ad69-9f5b24abc21b@linux.intel.com/
History:
v6: https://lore.kernel.org/all/20261106043257.842864-1-dapeng1.mi@linux.intel.com/
v5: https://lore.kernel.org/all/20261029043914.1395147-1-dapeng1.mi@linux.intel.com/
v4: https://lore.kernel.org/all/20261021050711.4043703-1-dapeng1.mi@linux.intel.com/
v3: https://lore.kernel.org/all/20261019070836.2972403-1-dapeng1.mi@linux.intel.com/
v2: https://lore.kernel.org/all/cover.1792064123.git.sandipan.das@amd.com/
v1: https://lore.kernel.org/all/20261007181002.4005262-1-sandipan.das@amd.com/
Dapeng Mi (23):
perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES
perf/x86/intel: Validate LBR XSAVE state before updating LBR stack
perf/x86/intel: Limit branch snapshots to valid entries
perf/x86: Add lbr_enabled to track LBR state
perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled
perf/x86/intel: Add helper to enable PMU without LBR
perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled
perf/x86/intel: Keep LBR frozen throughout PMI handling
perf: Add counted-bounds tracking for perf_branch_stack
perf/x86/intel: Allow LBR read helpers to operate on explicit stacks
perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting
perf/x86/intel: Always snapshot live hardware LBR state for BPF
perf/x86/intel: Account lbr_pebs_users for arch-PEBS events
perf/x86/intel: Set PERF_EV_CAP_SIBLING for pure branch counter events
perf/x86/intel: Account LBR user counts for pure branch counter events
perf/x86/intel: Refresh LBR stack for non-PEBS LBR events when needed
perf/x86/intel: Fix draining of un-overflowed PEBS buffer in PMI
handler
perf/x86/intel: Drain PEBS buffer for branch events with PEBS LBR
users
perf/x86/intel: Clear raw LBR counters in the LBR stack
perf/x86/intel: Refresh LBR stack for LBR counter events in PMI
handler
perf/x86/intel: Pass LBR stack size into intel_pmu_store_lbr()
perf/core: Don't mutate shared branch stack depth
perf/x86/intel: Fix incorrect PERF_METRICS overflow bit clearing
Sandipan Das (1):
perf/x86/amd/lbr: Limit branch snapshots to valid entries
arch/powerpc/perf/core-book3s.c | 47 ++++--
arch/x86/events/amd/brs.c | 7 +-
arch/x86/events/amd/core.c | 25 +++-
arch/x86/events/amd/lbr.c | 48 ++++---
arch/x86/events/core.c | 8 +-
arch/x86/events/intel/core.c | 231 +++++++++++++++++++++++++-----
arch/x86/events/intel/ds.c | 31 ++--
arch/x86/events/intel/lbr.c | 169 +++++++++++++---------
arch/x86/events/perf_event.h | 52 ++++++-
arch/x86/include/asm/perf_event.h | 1 +
drivers/perf/arm_brbe.c | 5 +
include/linux/perf_event.h | 15 +-
kernel/events/core.c | 8 +-
kernel/trace/bpf_trace.c | 6 +-
14 files changed, 477 insertions(+), 176 deletions(-)
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.34.1
next reply other threads:[~2026-11-18 3:23 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 3:14 Dapeng Mi [this message]
2026-11-18 3:14 ` [Patch v7 01/24] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES Dapeng Mi
2026-11-18 3:50 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 02/24] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack Dapeng Mi
2026-11-18 3:48 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 03/24] perf/x86/amd/lbr: Limit branch snapshots to valid entries Dapeng Mi
2026-11-18 3:34 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 04/24] perf/x86/intel: " Dapeng Mi
2026-11-18 3:37 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 05/24] perf/x86: Add lbr_enabled to track LBR state Dapeng Mi
2026-11-18 3:58 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 06/24] perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled Dapeng Mi
2026-11-18 3:43 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 07/24] perf/x86/intel: Add helper to enable PMU without LBR Dapeng Mi
2026-11-18 3:28 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 08/24] perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled Dapeng Mi
2026-11-18 3:39 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 09/24] perf/x86/intel: Keep LBR frozen throughout PMI handling Dapeng Mi
2026-11-18 3:42 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 10/24] perf: Add counted-bounds tracking for perf_branch_stack Dapeng Mi
2026-11-18 3:35 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 11/24] perf/x86/intel: Allow LBR read helpers to operate on explicit stacks Dapeng Mi
2026-11-18 3:32 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 12/24] perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting Dapeng Mi
2026-11-18 3:43 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 13/24] perf/x86/intel: Always snapshot live hardware LBR state for BPF Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 14/24] perf/x86/intel: Account lbr_pebs_users for arch-PEBS events Dapeng Mi
2026-11-18 3:34 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 15/24] perf/x86/intel: Set PERF_EV_CAP_SIBLING for pure branch counter events Dapeng Mi
2026-11-18 4:05 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 16/24] perf/x86/intel: Account LBR user counts " Dapeng Mi
2026-11-18 3:49 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 17/24] perf/x86/intel: Refresh LBR stack for non-PEBS LBR events when needed Dapeng Mi
2026-11-18 3:14 ` [Patch v7 18/24] perf/x86/intel: Fix draining of un-overflowed PEBS buffer in PMI handler Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 19/24] perf/x86/intel: Drain PEBS buffer for branch events with PEBS LBR users Dapeng Mi
2026-11-18 3:52 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 20/24] perf/x86/intel: Clear raw LBR counters in the LBR stack Dapeng Mi
2026-11-18 3:50 ` sashiko-bot
2026-11-18 3:54 ` bot+bpf-ci
2026-11-18 3:14 ` [Patch v7 21/24] perf/x86/intel: Refresh LBR stack for LBR counter events in PMI handler Dapeng Mi
2026-11-18 3:45 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 22/24] perf/x86/intel: Pass LBR stack size into intel_pmu_store_lbr() Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 23/24] perf/core: Don't mutate shared branch stack depth Dapeng Mi
2026-11-18 3:51 ` sashiko-bot
2026-11-18 3:14 ` [Patch v7 24/24] perf/x86/intel: Fix incorrect PERF_METRICS overflow bit clearing Dapeng Mi
2026-11-18 3:46 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118031453.1353625-1-dapeng1.mi@linux.intel.com \
--to=dapeng1.mi@linux.intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=ananth.narayan@amd.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=dapeng1.mi@intel.com \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=sandipan.das@amd.com \
--cc=song@kernel.org \
--cc=thomas.falcon@intel.com \
--cc=xudong.hao@intel.com \
--cc=zide.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox