[Patch v5 00/18] perf/x86: Fix multiple LBR sampling issues

[Patch v5 00/18] perf/x86: Fix multiple LBR sampling issues

[Dapeng Mi]

This patch series fixes several x86 LBR sampling issues discovered during
review and debugging. Most of them are related to BPF LBR snapshotting
support.

The series originally targeted the BPF stack out-of-bounds access issue,
but additional problems were found during debugging and code review,
including:
- PMU and LBR being re-enabled unconditionally at BPF snapshotting exit,
  even when they had already been disabled before BPF snapshotting on both
  AMD and Intel.
- LBR not being fully frozen in the PMI handler on Intel, which allows
  branches in the PMI path to pollute the BPF LBR snapshot.
- BPF snapshotting overwriting the cached LBR stack of PEBS LBR events on
  Intel.
- PEBS LBR events overwriting the LBR stack of non-PEBS LBR events on
  Intel when both overflow in the same PMI.
- LBR counter events reporting zero counters on Intel when multiple
  LBR counter events overflow in the same PMI.


Patch layout:
- Patch 01: Add a sanity check to ensure the LBR stack does not exceed
  MAX_LBR_ENTRIES.
- Patch 02: Validate that hardware LBR data has actually been stored in
  the XSAVE area before parsing it.
- Patch 03-04: Fix the potential LBR stack out-of-bounds access issue on
  AMD and Intel separately.
- Patch 05-08: Fix the incorrect PMU and LBR re-enabling issue
  in perf_snapshot_branch_stack() for AMD and Intel separately.
- Patch 09: Freeze LBR throughout the PMI processing period and ensure
  all LBR events observe a consistent LBR stack in the PMI handler.
- Patch 10-11: Add a dedicated LBR stack for BPF LBR snapshotting to avoid
  overwriting the default LBR stack of other LBR events.
- Patch 12: Ensure perf_snapshot_branch_stack() snapshots the live
  hardware LBR data instead of the cached PEBS LBR stack.
- Patch 13: Prevent PEBS LBR data from overwriting non-PEBS LBR data when
  both overflow in the same PMI.
- Patch 14: Add the missing lbr_pebs_users accounting for arch-PEBS.
- Patch 15-16: Ensure LBR counters are not dropped when multiple LBR
  counter events overflow in the same PMI.
- Patch 17: Clamp the parsed PEBS LBR stack depth to x86_pmu.lbr_nr to
  avoid LBR stack out-of-bounds access.
- Patch 18: Fix the shared LBR stack depth truncation issue.


Changes in v5:
- Patch 02: Clear xsave->header.xfeatures before calling the xsaves
  instructuion to read LBR states.
- Patch 05-18: New patches to fix newly discovered issues described above.

Changes in v4:
- Patch 01: Disable LBR if parsed x86_pmu.lbr_nr exceeds
  MAX_LBR_ENTRIES (Sashiko).
- Patch 05: New patch, which always snapshots live hardware LBR stack
  for BPF path (Sashiko).

Changes in v3:
- Patch 01: A new patch to ensure x86_pmu.lbr_nr not to exclude
  MAX_LBR_ENTRIES (Internal Sashiko).
- Patch 03: Clean cached LBR stack when reseting hardware LBR registers,
  which prevents stale LBR stack leakage.
- Patch 04: Don't reset cached LBR stack if all LBR events are PEBS
  events. This ensures PEBS captured LBR stack isn't cleared (Sashiko).

Changes in v2:
- Include a fix for Intel LBR.
- Add Tested-by from Dapeng to the Intel fix.
  https://lore.kernel.org/all/eb6c7bda-0dcf-4fac-ad69-9f5b24abc21b@linux.intel.com/

History:
v4: https://lore.kernel.org/all/20261021050711.4043703-1-dapeng1.mi@linux.intel.com/
v3: https://lore.kernel.org/all/20261019070836.2972403-1-dapeng1.mi@linux.intel.com/
v2: https://lore.kernel.org/all/cover.1792064123.git.sandipan.das@amd.com/
v1: https://lore.kernel.org/all/20261007181002.4005262-1-sandipan.das@amd.com/

Dapeng Mi (17):
  perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES
  perf/x86/intel: Validate LBR XSAVE state before updating LBR stack
  perf/x86/intel/lbr: Limit branch snapshots to valid entries
  perf/x86: Add lbr_enabled to track LBR state
  perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled
  perf/x86/intel: Add helper to enable PMU without LBR
  perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled
  perf/x86/intel: Keep LBR frozen throughout PMI handling
  perf/x86/intel: Allow LBR read helpers to operate on explicit stacks
  perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting
  perf/x86/intel: Always snapshot live hardware LBR state for BPF
  perf/x86/intel: Refresh LBR stack for non-PEBS LBR events if needed
  perf/x86/intel: Account arch-PEBS in lbr_pebs_users
  perf/x86/intel: Clear raw LBR counters in the LBR stack
  perf/x86/intel: Refill LBR stack for LBR counter events in PMI handler
  perf/x86/intel: Pass LBR stack size into store_lbr()
  perf/core: Don't overwrite shared LBR stack depth

Sandipan Das (1):
  perf/x86/amd/lbr: Limit branch snapshots to valid entries

 arch/x86/events/amd/core.c   |  14 +++-
 arch/x86/events/amd/lbr.c    |  14 +++-
 arch/x86/events/intel/core.c | 135 ++++++++++++++++++++++++++-----
 arch/x86/events/intel/ds.c   |   8 +-
 arch/x86/events/intel/lbr.c  | 153 ++++++++++++++++++++++-------------
 arch/x86/events/perf_event.h |  37 +++++++--
 include/linux/perf_event.h   |  13 ++-
 kernel/events/core.c         |   8 +-
 8 files changed, 285 insertions(+), 97 deletions(-)


base-commit: f5a78ecccd98a04c47cea1cd63c2a2fdce8a4b48
-- 
2.34.1

[Patch v5 01/18] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

[Dapeng Mi]

Add a sanity check for x86_pmu.lbr_nr. If the hardware reports more LBR
entries than MAX_LBR_ENTRIES, disable LBR to avoid out-of-bounds access
in the cached LBR stack handling path.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/amd/lbr.c    | 8 +++++++-
 arch/x86/events/intel/core.c | 7 +++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index a55646fcb846..96c5b5e56053 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -433,7 +433,13 @@ __init int amd_pmu_lbr_init(void)
 	ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
 	x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz;
 
-	pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
+	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
+		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
+			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
+		x86_pmu.lbr_nr = 0;
+	} else {
+		pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
+	}
 
 	return 0;
 }
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 0a34d674df59..10c694db85ea 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -9023,6 +9023,13 @@ __init int intel_pmu_init(void)
 	 */
 	if (x86_pmu.lbr_tos && !check_msr(x86_pmu.lbr_tos, 0x3UL))
 		x86_pmu.lbr_nr = 0;
+
+	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
+		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
+			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
+		x86_pmu.lbr_nr = 0;
+	}
+
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		if (!(check_msr(x86_pmu.lbr_from + i, 0xffffUL) &&
 		      check_msr(x86_pmu.lbr_to + i, 0xffffUL)))
-- 
2.34.1

Re: [Patch v5 01/18] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] perf/x86/intel: incomplete Arch LBR disablement in intel_pmu_init()
- [Medium] perf/x86/amd: amd_pmu_lbr_init() returns 0 when LBR is disabled
--

commit 03c28823478ca7f089145cd93c93be2824eab310
Author: Dapeng Mi <dapeng1.mi@linux.intel.com>

perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

This commit adds a sanity check on x86_pmu.lbr_nr during AMD and Intel PMU
initialization to disable LBR if the hardware reports more entries than
MAX_LBR_ENTRIES.

> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
> index a55646fcb8465..96c5b5e56053b 100644
> --- a/arch/x86/events/amd/lbr.c
> +++ b/arch/x86/events/amd/lbr.c
> @@ -433,7 +433,13 @@ __init int amd_pmu_lbr_init(void)
>  	ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
>  	x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz;
>  
> -	pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> +		x86_pmu.lbr_nr = 0;
> +	} else {
> +		pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> +	}
>  
>  	return 0;
>  }

[Severity: Medium]
Should amd_pmu_lbr_init() return an error code such as -EOPNOTSUPP when
x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES?

Because amd_pmu_lbr_init() still returns 0 after setting x86_pmu.lbr_nr = 0,
its caller in arch/x86/events/amd/core.c treats LBR initialization as
successful and installs the LBR callbacks:

arch/x86/events/amd/core.c:amd_core_pmu_init() {
	...
	/* LBR and BRS are mutually exclusive features */
	if (!amd_pmu_lbr_init()) {
		/* LBR requires flushing on context switch */
		x86_pmu.sched_task = amd_pmu_lbr_sched_task;
		static_call_update(amd_pmu_branch_hw_config, amd_pmu_lbr_hw_config);
		static_call_update(amd_pmu_branch_reset, amd_pmu_lbr_reset);
		static_call_update(amd_pmu_branch_add, amd_pmu_lbr_add);
		static_call_update(amd_pmu_branch_del, amd_pmu_lbr_del);

		/* Only support branch_stack snapshot on perfmon v2 */
		if (x86_pmu.handle_irq == amd_pmu_v2_handle_irq)
			static_call_update(perf_snapshot_branch_stack, amd_pmu_v2_snapshot_branch_stack);
	}
	...
}

If perf_snapshot_branch_stack is later invoked (for example via
bpf_get_branch_snapshot()), amd_pmu_v2_snapshot_branch_stack() runs without
checking x86_pmu.lbr_nr and touches MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
MSR_AMD_DBG_EXTN_CFG, and MSR_IA32_DEBUGCTLMSR:

arch/x86/events/amd/core.c:amd_pmu_v2_snapshot_branch_stack() {
	...
	local_irq_save(flags);
	amd_pmu_core_disable_all();
	__amd_pmu_lbr_disable();

	cpuc = this_cpu_ptr(&cpu_hw_events);

	amd_pmu_lbr_read();
	cnt = min(cnt, x86_pmu.lbr_nr);
	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);

	amd_pmu_v2_enable_all(0);
	local_irq_restore(flags);
	...
}

> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> index 0a34d674df590..10c694db85ea3 100644
> --- a/arch/x86/events/intel/core.c
> +++ b/arch/x86/events/intel/core.c
> @@ -9023,6 +9023,13 @@ __init int intel_pmu_init(void)
>  	 */
>  	if (x86_pmu.lbr_tos && !check_msr(x86_pmu.lbr_tos, 0x3UL))
>  		x86_pmu.lbr_nr = 0;
> +
> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> +		x86_pmu.lbr_nr = 0;
> +	}

[Severity: Medium]
When X86_FEATURE_ARCH_LBR is enabled and CPUID leaf 28 enumerates a depth
greater than MAX_LBR_ENTRIES, does zeroing x86_pmu.lbr_nr here leave
Architectural LBR partially initialized and leak x86_pmu.task_ctx_cache and
per-CPU cpuc->lbr_xsave buffers?

In intel_pmu_init(), intel_pmu_arch_lbr_init() and
intel_pmu_check_event_constraints_all() are called before this check:

arch/x86/events/intel/core.c:intel_pmu_init() {
	...
	if (boot_cpu_has(X86_FEATURE_ARCH_LBR))
		intel_pmu_arch_lbr_init();

	intel_pmu_check_event_constraints_all(NULL);
	...
}

Inside arch/x86/events/intel/lbr.c:intel_pmu_arch_lbr_init(), lbr_nr is
computed as fls(eax.split.lbr_depth_mask) * 8. Because it does not bail out to
clear_arch_lbr when lbr_nr > MAX_LBR_ENTRIES, it already:

- writes lbr_nr to MSR_ARCH_LBR_DEPTH
- allocates x86_pmu.task_ctx_cache
- enables the x86_lbr_mispred, x86_lbr_cycles, and x86_lbr_type static
  branches
- sets x86_pmu.lbr_ctl_map (which keeps x86_pmu_has_lbr_callstack() true)
- sets PMU_FL_BR_CNTR and PMU_FL_DYN_CONSTRAINT in x86_pmu.flags
- leaves X86_FEATURE_ARCH_LBR set

As a result, when any hardware perf event is later opened,
reserve_lbr_buffers() still allocates cpuc->lbr_xsave across all possible CPUs
from task_ctx_cache:

arch/x86/events/intel/lbr.c:reserve_lbr_buffers() {
	...
	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR))
		return;

	for_each_possible_cpu(cpu) {
		cpuc = per_cpu_ptr(&cpu_hw_events, cpu);
		kmem_cache = x86_get_pmu(cpu)->task_ctx_cache;
		if (!kmem_cache || cpuc->lbr_xsave)
			continue;

		cpuc->lbr_xsave = kmem_cache_alloc_node(kmem_cache,
							GFP_KERNEL | __GFP_ZERO,
							cpu_to_node(cpu));
	}
}

In addition, lbr_is_visible() in arch/x86/events/intel/core.c still exposes
branch_counter_nr and branch_counter_width in sysfs because PMU_FL_BR_CNTR
remains set, and attach_perf_ctx_data() can still allocate task->perf_ctx_data
for events with PERF_SAMPLE_READ and PERF_SAMPLE_BRANCH_CALL_STACK.

Should intel_pmu_arch_lbr_init() check lbr_nr > MAX_LBR_ENTRIES before writing
MSR_ARCH_LBR_DEPTH and jump to clear_arch_lbr so X86_FEATURE_ARCH_LBR is
cleared cleanly?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=1

Re: [Patch v5 01/18] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

[Peter Zijlstra]

On Thu, Oct 29, 2026 at 12:38:57PM +0800, Dapeng Mi wrote:
> Add a sanity check for x86_pmu.lbr_nr. If the hardware reports more LBR
> entries than MAX_LBR_ENTRIES, disable LBR to avoid out-of-bounds access
> in the cached LBR stack handling path.

Cute, can this actually happen?

> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> ---
>  arch/x86/events/amd/lbr.c    | 8 +++++++-
>  arch/x86/events/intel/core.c | 7 +++++++
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
> index a55646fcb846..96c5b5e56053 100644
> --- a/arch/x86/events/amd/lbr.c
> +++ b/arch/x86/events/amd/lbr.c
> @@ -433,7 +433,13 @@ __init int amd_pmu_lbr_init(void)
>  	ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
>  	x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz;
>  
> -	pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> +		x86_pmu.lbr_nr = 0;
> +	} else {
> +		pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> +	}
>  
>  	return 0;
>  }
> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> index 0a34d674df59..10c694db85ea 100644
> --- a/arch/x86/events/intel/core.c
> +++ b/arch/x86/events/intel/core.c
> @@ -9023,6 +9023,13 @@ __init int intel_pmu_init(void)
>  	 */
>  	if (x86_pmu.lbr_tos && !check_msr(x86_pmu.lbr_tos, 0x3UL))
>  		x86_pmu.lbr_nr = 0;
> +
> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> +		x86_pmu.lbr_nr = 0;
> +	}
> +
>  	for (i = 0; i < x86_pmu.lbr_nr; i++) {
>  		if (!(check_msr(x86_pmu.lbr_from + i, 0xffffUL) &&
>  		      check_msr(x86_pmu.lbr_to + i, 0xffffUL)))

Would it make sense to have but one copy of this check in
init_hw_perf_events(), after calling into the micro-arch code?

Re: [Patch v5 01/18] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

[Mi, Dapeng]

On 10/29/2026 5:38 PM, Peter Zijlstra wrote:
> On Thu, Oct 29, 2026 at 12:38:57PM +0800, Dapeng Mi wrote:
>> Add a sanity check for x86_pmu.lbr_nr. If the hardware reports more LBR
>> entries than MAX_LBR_ENTRIES, disable LBR to avoid out-of-bounds access
>> in the cached LBR stack handling path.
> Cute, can this actually happen?

Currently it won't happen on real hardware since the maximum LBR stack
depth what hardware supports is 32. But it's better have this sanity check
assume the LBR stack depth could be extended to 64 or larger.


>
>> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
>> ---
>>  arch/x86/events/amd/lbr.c    | 8 +++++++-
>>  arch/x86/events/intel/core.c | 7 +++++++
>>  2 files changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
>> index a55646fcb846..96c5b5e56053 100644
>> --- a/arch/x86/events/amd/lbr.c
>> +++ b/arch/x86/events/amd/lbr.c
>> @@ -433,7 +433,13 @@ __init int amd_pmu_lbr_init(void)
>>  	ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
>>  	x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz;
>>  
>> -	pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
>> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
>> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
>> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
>> +		x86_pmu.lbr_nr = 0;
>> +	} else {
>> +		pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
>> +	}
>>  
>>  	return 0;
>>  }
>> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
>> index 0a34d674df59..10c694db85ea 100644
>> --- a/arch/x86/events/intel/core.c
>> +++ b/arch/x86/events/intel/core.c
>> @@ -9023,6 +9023,13 @@ __init int intel_pmu_init(void)
>>  	 */
>>  	if (x86_pmu.lbr_tos && !check_msr(x86_pmu.lbr_tos, 0x3UL))
>>  		x86_pmu.lbr_nr = 0;
>> +
>> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
>> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
>> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
>> +		x86_pmu.lbr_nr = 0;
>> +	}
>> +
>>  	for (i = 0; i < x86_pmu.lbr_nr; i++) {
>>  		if (!(check_msr(x86_pmu.lbr_from + i, 0xffffUL) &&
>>  		      check_msr(x86_pmu.lbr_to + i, 0xffffUL)))
> Would it make sense to have but one copy of this check in
> init_hw_perf_events(), after calling into the micro-arch code?

Yes, that's what the previous version does, but Sashiko complains it's too
late.

"

Does zeroing x86_pmu.lbr_nr here in init_hw_perf_events() happen too late,
after intel_pmu_init() and amd_pmu_init() have already configured hardware
and driver state using the unvalidated x86_pmu.lbr_nr value?

"

https://lore.kernel.org/all/sashiko-outbox-179814@kernel.org/

It makes sense in some ways, so move the sanity check into the vendor init
in this version. 

Thanks.

Re: [Patch v5 01/18] perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES

[Peter Zijlstra]

On Thu, Oct 29, 2026 at 05:56:15PM +0800, Mi, Dapeng wrote:
> 
> On 10/29/2026 5:38 PM, Peter Zijlstra wrote:
> > On Thu, Oct 29, 2026 at 12:38:57PM +0800, Dapeng Mi wrote:
> >> Add a sanity check for x86_pmu.lbr_nr. If the hardware reports more LBR
> >> entries than MAX_LBR_ENTRIES, disable LBR to avoid out-of-bounds access
> >> in the cached LBR stack handling path.
> > Cute, can this actually happen?
> 
> Currently it won't happen on real hardware since the maximum LBR stack
> depth what hardware supports is 32. But it's better have this sanity check
> assume the LBR stack depth could be extended to 64 or larger.
> 
> 
> >
> >> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> >> ---
> >>  arch/x86/events/amd/lbr.c    | 8 +++++++-
> >>  arch/x86/events/intel/core.c | 7 +++++++
> >>  2 files changed, 14 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
> >> index a55646fcb846..96c5b5e56053 100644
> >> --- a/arch/x86/events/amd/lbr.c
> >> +++ b/arch/x86/events/amd/lbr.c
> >> @@ -433,7 +433,13 @@ __init int amd_pmu_lbr_init(void)
> >>  	ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
> >>  	x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz;
> >>  
> >> -	pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> >> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> >> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> >> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> >> +		x86_pmu.lbr_nr = 0;
> >> +	} else {
> >> +		pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr);
> >> +	}
> >>  
> >>  	return 0;
> >>  }
> >> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> >> index 0a34d674df59..10c694db85ea 100644
> >> --- a/arch/x86/events/intel/core.c
> >> +++ b/arch/x86/events/intel/core.c
> >> @@ -9023,6 +9023,13 @@ __init int intel_pmu_init(void)
> >>  	 */
> >>  	if (x86_pmu.lbr_tos && !check_msr(x86_pmu.lbr_tos, 0x3UL))
> >>  		x86_pmu.lbr_nr = 0;
> >> +
> >> +	if (x86_pmu.lbr_nr > MAX_LBR_ENTRIES) {
> >> +		pr_warn("LBR entries %d exceeds max %d, disable LBR.\n",
> >> +			x86_pmu.lbr_nr, MAX_LBR_ENTRIES);
> >> +		x86_pmu.lbr_nr = 0;
> >> +	}
> >> +
> >>  	for (i = 0; i < x86_pmu.lbr_nr; i++) {
> >>  		if (!(check_msr(x86_pmu.lbr_from + i, 0xffffUL) &&
> >>  		      check_msr(x86_pmu.lbr_to + i, 0xffffUL)))
> > Would it make sense to have but one copy of this check in
> > init_hw_perf_events(), after calling into the micro-arch code?
> 
> Yes, that's what the previous version does, but Sashiko complains it's too
> late.
> 
> "
> 
> Does zeroing x86_pmu.lbr_nr here in init_hw_perf_events() happen too late,
> after intel_pmu_init() and amd_pmu_init() have already configured hardware
> and driver state using the unvalidated x86_pmu.lbr_nr value?
> 
> "
> 
> https://lore.kernel.org/all/sashiko-outbox-179814@kernel.org/
> 
> It makes sense in some ways, so move the sanity check into the vendor init
> in this version. 

Urgh. But there are no events yet that early, none of that can actually
happen.

If we're really worried about this, change the micro-arch init to spawn
in disabled state and we call x86_pmu_enable_all() at the end.

Also, this is before x86_pmu_static_call_update(), nothing really
'works' yet anyway.

[Patch v5 02/18] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack

[Dapeng Mi]

intel_pmu_arch_lbr_read_xsave() doesn't check if the LBR data is present
in the xsave area. If LBR data is in the INIT state, the LBR data may not
be really written into the xsave area.

Add an explicit LBR XSAVE state presence check before updating the cached
LBR stack. Also clear the cached LBR stack when intel_pmu_lbr_reset() is
called, in addition to resetting the hardware LBR register file, to prevent
stale LBR data from being reused after a context switch or similar
transition.

Fixes: c085fb877467 ("perf/x86/intel/lbr: Support XSAVES for arch LBR read")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/lbr.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index 0b446d8912e4..a2e4ba7e1a4f 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -193,6 +193,8 @@ void intel_pmu_lbr_reset(void)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
+	cpuc->lbr_stack.nr = 0;
+
 	if (!x86_pmu.lbr_nr)
 		return;
 
@@ -1010,9 +1012,15 @@ static void intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc)
 		intel_pmu_store_lbr(cpuc, NULL);
 		return;
 	}
+	xsave->header.xfeatures = 0;
 	xsaves(&xsave->xsave, XFEATURE_MASK_LBR);
 
-	intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
+	if (xsave->header.xfeatures & XFEATURE_MASK_LBR) {
+		intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
+	} else {
+		/* Clear the LBR stack to avoid leaking stale entries. */
+		cpuc->lbr_stack.nr = 0;
+	}
 }
 
 void intel_pmu_lbr_read(void)
-- 
2.34.1

Re: [Patch v5 02/18] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack

[sashiko-bot]

> intel_pmu_arch_lbr_read_xsave() doesn't check if the LBR data is present
> in the xsave area. If LBR data is in the INIT state, the LBR data may not
> be really written into the xsave area.
> 
> Add an explicit LBR XSAVE state presence check before updating the cached
> LBR stack. Also clear the cached LBR stack when intel_pmu_lbr_reset() is
> called, in addition to resetting the hardware LBR register file, to prevent
> stale LBR data from being reused after a context switch or similar
> transition.
> 
> Fixes: c085fb877467 ("perf/x86/intel/lbr: Support XSAVES for arch LBR read")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=2

Re: [Patch v5 02/18] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack

[Peter Zijlstra]

On Thu, Oct 29, 2026 at 12:38:58PM +0800, Dapeng Mi wrote:
> intel_pmu_arch_lbr_read_xsave() doesn't check if the LBR data is present
> in the xsave area. If LBR data is in the INIT state, the LBR data may not
> be really written into the xsave area.
> 
> Add an explicit LBR XSAVE state presence check before updating the cached
> LBR stack. Also clear the cached LBR stack when intel_pmu_lbr_reset() is
> called, in addition to resetting the hardware LBR register file, to prevent
> stale LBR data from being reused after a context switch or similar
> transition.
> 
> Fixes: c085fb877467 ("perf/x86/intel/lbr: Support XSAVES for arch LBR read")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> ---
>  arch/x86/events/intel/lbr.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
> index 0b446d8912e4..a2e4ba7e1a4f 100644
> --- a/arch/x86/events/intel/lbr.c
> +++ b/arch/x86/events/intel/lbr.c
> @@ -193,6 +193,8 @@ void intel_pmu_lbr_reset(void)
>  {
>  	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
>  
> +	cpuc->lbr_stack.nr = 0;
> +
>  	if (!x86_pmu.lbr_nr)
>  		return;
>  

This also clears the stack if there is no LBR; was that on purpose? The
Changelog is silent on this point.

> @@ -1010,9 +1012,15 @@ static void intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc)
>  		intel_pmu_store_lbr(cpuc, NULL);
>  		return;
>  	}
> +	xsave->header.xfeatures = 0;
>  	xsaves(&xsave->xsave, XFEATURE_MASK_LBR);
>  
> -	intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
> +	if (xsave->header.xfeatures & XFEATURE_MASK_LBR) {
> +		intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
> +	} else {
> +		/* Clear the LBR stack to avoid leaking stale entries. */
> +		cpuc->lbr_stack.nr = 0;
> +	}
>  }
>  
>  void intel_pmu_lbr_read(void)
> -- 
> 2.34.1
> 

Re: [Patch v5 02/18] perf/x86/intel: Validate LBR XSAVE state before updating LBR stack

[Mi, Dapeng]

On 10/29/2026 5:40 PM, Peter Zijlstra wrote:
> On Thu, Oct 29, 2026 at 12:38:58PM +0800, Dapeng Mi wrote:
>> intel_pmu_arch_lbr_read_xsave() doesn't check if the LBR data is present
>> in the xsave area. If LBR data is in the INIT state, the LBR data may not
>> be really written into the xsave area.
>>
>> Add an explicit LBR XSAVE state presence check before updating the cached
>> LBR stack. Also clear the cached LBR stack when intel_pmu_lbr_reset() is
>> called, in addition to resetting the hardware LBR register file, to prevent
>> stale LBR data from being reused after a context switch or similar
>> transition.
>>
>> Fixes: c085fb877467 ("perf/x86/intel/lbr: Support XSAVES for arch LBR read")
>> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
>> ---
>>  arch/x86/events/intel/lbr.c | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
>> index 0b446d8912e4..a2e4ba7e1a4f 100644
>> --- a/arch/x86/events/intel/lbr.c
>> +++ b/arch/x86/events/intel/lbr.c
>> @@ -193,6 +193,8 @@ void intel_pmu_lbr_reset(void)
>>  {
>>  	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
>>  
>> +	cpuc->lbr_stack.nr = 0;
>> +
>>  	if (!x86_pmu.lbr_nr)
>>  		return;
>>  
> This also clears the stack if there is no LBR; was that on purpose? The
> Changelog is silent on this point.

Yes, it's on purpose. The intent is to clear the cached software LBR stack
as well when resetting the hardware LBR stack. This is to avoid leak the
LBR stack information in some unexpected cases. In theory, it can be placed
after the !x86_pmu.lbr_nr, I hope the software LBR stack can always be
cleared regardless of there is LBR user or not.

My fault, I missed to describe this change in the change log. I would add
it in next version.

Thanks.


>
>> @@ -1010,9 +1012,15 @@ static void intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc)
>>  		intel_pmu_store_lbr(cpuc, NULL);
>>  		return;
>>  	}
>> +	xsave->header.xfeatures = 0;
>>  	xsaves(&xsave->xsave, XFEATURE_MASK_LBR);
>>  
>> -	intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
>> +	if (xsave->header.xfeatures & XFEATURE_MASK_LBR) {
>> +		intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
>> +	} else {
>> +		/* Clear the LBR stack to avoid leaking stale entries. */
>> +		cpuc->lbr_stack.nr = 0;
>> +	}
>>  }
>>  
>>  void intel_pmu_lbr_read(void)
>> -- 
>> 2.34.1
>>

[Patch v5 03/18] perf/x86/amd/lbr: Limit branch snapshots to valid entries

[Dapeng Mi]

From: Sandipan Das <sandipan.das@amd.com>

The branch snapshot callback used by bpf_get_branch_snapshot() copies
up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the
last LBR read kept fewer branches. The entries past lbr_stack.nr are
invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when
amd_pmu_lbr_read() skips reading the registers, so that a stale count
is never reused.

Fixes: a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-163130@kernel.org/
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index 49b6b8fce566..bf631f60757c 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index 96c5b5e56053..46bd3ce1d614 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -168,8 +168,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);
-- 
2.34.1

Re: [Patch v5 03/18] perf/x86/amd/lbr: Limit branch snapshots to valid entries

[sashiko-bot]

> From: Sandipan Das <sandipan.das@amd.com>
> 
> The branch snapshot callback used by bpf_get_branch_snapshot() copies
> up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the
> last LBR read kept fewer branches. The entries past lbr_stack.nr are
> invalid, but they end up reaching BPF programs.
> 
> Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when
> amd_pmu_lbr_read() skips reading the registers, so that a stale count
> is never reused.
> 
> Fixes: a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-163130@kernel.org/
> Signed-off-by: Sandipan Das <sandipan.das@amd.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=3

[Patch v5 04/18] perf/x86/intel/lbr: Limit branch snapshots to valid entries

[Dapeng Mi]

The branch snapshot callback used by bpf_get_branch_snapshot() always
copies x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the
last LBR read kept fewer branches. The entries past lbr_stack.nr hold
invalid branches which end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when
no active LBR events or LBR hardware is owned by guest. So the stale or
guest LBR stack data can't be leaked.

For the case where all LBR events are PEBS events, hardware LBR reads are
currently skipped; that can leave stale LBR stack data from previous PEBS
sampling in place and may be inconsistent with the intended semantics of
bpf_get_branch_snapshot(). This will be addressed in a follow-up patch.

Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-163130@kernel.org/
Co-developed-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 2 +-
 arch/x86/events/intel/lbr.c  | 9 +++++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 10c694db85ea..18cae4ac32b6 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	intel_pmu_lbr_read();
-	cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+	cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 	intel_pmu_enable_all(0);
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index a2e4ba7e1a4f..1ae853f7ac1a 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -1027,14 +1027,19 @@ void intel_pmu_lbr_read(void)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
+	if (!cpuc->lbr_users || vlbr_exclude_host()) {
+		/* Reset lbr_stack.nr to prevent stale data leakage. */
+		cpuc->lbr_stack.nr = 0;
+		return;
+	}
+
 	/*
 	 * Don't read when all LBRs users are using adaptive PEBS.
 	 *
 	 * This could be smarter and actually check the event,
 	 * but this simple approach seems to work for now.
 	 */
-	if (!cpuc->lbr_users || vlbr_exclude_host() ||
-	    cpuc->lbr_users == cpuc->lbr_pebs_users)
+	if (cpuc->lbr_users == cpuc->lbr_pebs_users)
 		return;
 
 	x86_pmu.lbr_read(cpuc);
-- 
2.34.1

Re: [Patch v5 04/18] perf/x86/intel/lbr: Limit branch snapshots to valid entries

[sashiko-bot]

> The branch snapshot callback used by bpf_get_branch_snapshot() always
> copies x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the
> last LBR read kept fewer branches. The entries past lbr_stack.nr hold
> invalid branches which end up reaching BPF programs.
> 
> Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when
> no active LBR events or LBR hardware is owned by guest. So the stale or
> guest LBR stack data can't be leaked.
> 
> For the case where all LBR events are PEBS events, hardware LBR reads are
> currently skipped; that can leave stale LBR stack data from previous PEBS
> sampling in place and may be inconsistent with the intended semantics of
> bpf_get_branch_snapshot(). This will be addressed in a follow-up patch.
> 
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-163130@kernel.org/
> Co-developed-by: Sandipan Das <sandipan.das@amd.com>
> Signed-off-by: Sandipan Das <sandipan.das@amd.com>
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=4

[Patch v5 05/18] perf/x86: Add lbr_enabled to track LBR state

[Dapeng Mi]

Add a lbr_enabled flag to track whether LBR is enabled. This will be
used by a subsequent change to avoid blindly re-enabling LBR in
perf_snapshot_branch_stack() without checking whether LBR has already
been disabled.

Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Fixes: a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/amd/lbr.c    | 2 ++
 arch/x86/events/intel/core.c | 5 +++++
 arch/x86/events/intel/lbr.c  | 2 ++
 arch/x86/events/perf_event.h | 9 +++++++++
 4 files changed, 18 insertions(+)

diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index 46bd3ce1d614..cf11e4815c39 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -399,6 +399,8 @@ void amd_pmu_lbr_enable_all(void)
 	if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
 		return;
 
+	cpuc->lbr_enabled = 1;
+
 	/* Set hardware branch filter */
 	if (cpuc->lbr_select) {
 		lbr_select = cpuc->lbr_sel->config & LBR_SELECT_MASK;
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 18cae4ac32b6..0ed0269189ba 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -4001,6 +4001,11 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 	 * It needs to be restored when leaving the handler.
 	 */
 	pmu_enabled = cpuc->enabled;
+
+	/* If pmu version < 4, LBR freeze disables LBR. */
+	if (x86_pmu.version < 4)
+		cpuc->lbr_enabled = 0;
+
 	/*
 	 * In general, the early ACK is only applied for old platforms.
 	 * For the big core starts from Haswell, the late ACK should be
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index 1ae853f7ac1a..ee724b28d90c 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -132,6 +132,8 @@ static void __intel_pmu_lbr_enable(bool pmi)
 	if (pmi && x86_pmu.version >= 4)
 		return;
 
+	cpuc->lbr_enabled = 1;
+
 	/*
 	 * No need to reprogram LBR_SELECT in a PMI, as it
 	 * did not change.
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index e274802ef062..a4aab8130e92 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -473,6 +473,7 @@ struct cpu_hw_events {
 	unsigned long		active_mask[BITS_TO_LONGS(X86_PMC_IDX_MAX)];
 	unsigned long		dirty[BITS_TO_LONGS(X86_PMC_IDX_MAX)];
 	int			enabled;
+	int			lbr_enabled;
 
 	int			n_events; /* the # of events in the below arrays */
 	int			n_added;  /* the # last events in the below arrays;
@@ -1676,8 +1677,11 @@ int amd_pmu_lbr_hw_config(struct perf_event *event);
 
 static __always_inline void __amd_pmu_lbr_disable(void)
 {
+	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 	u64 dbg_ctl, dbg_extn_cfg;
 
+	cpuc->lbr_enabled = 0;
+
 	rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
 	wrmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg & ~DBG_EXTN_CFG_LBRV2EN);
 
@@ -1829,13 +1833,18 @@ static __always_inline void __intel_pmu_pebs_disable_all(void)
 
 static __always_inline void __intel_pmu_arch_lbr_disable(void)
 {
+	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+
+	cpuc->lbr_enabled = 0;
 	wrmsrq(MSR_ARCH_LBR_CTL, 0);
 }
 
 static __always_inline void __intel_pmu_lbr_disable(void)
 {
+	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 	u64 debugctl;
 
+	cpuc->lbr_enabled = 0;
 	rdmsrq(MSR_IA32_DEBUGCTLMSR, debugctl);
 	debugctl &= ~(DEBUGCTLMSR_LBR | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
 	wrmsrq(MSR_IA32_DEBUGCTLMSR, debugctl);
-- 
2.34.1

Re: [Patch v5 05/18] perf/x86: Add lbr_enabled to track LBR state

[sashiko-bot]

> Add a lbr_enabled flag to track whether LBR is enabled. This will be
> used by a subsequent change to avoid blindly re-enabling LBR in
> perf_snapshot_branch_stack() without checking whether LBR has already
> been disabled.
> 
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Fixes: a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=5

[Patch v5 06/18] perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled

[Dapeng Mi]

amd_pmu_v2_snapshot_branch_stack() currently re-enables the PMU and LBR
unconditionally even when they were already disabled in PMI context
This can cause events such as cycles and instructions to be counted
while the PMI handler is running, which is not intended.

Only restore the PMU and LBR on the BPF path if they were originally
enabled.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
Fixes: a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/amd/core.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index bf631f60757c..851d8a938f52 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -932,22 +932,28 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 {
 	struct cpu_hw_events *cpuc;
 	unsigned long flags;
+	int pmu_enabled;
+	int lbr_enabled;
 
 	/*
 	 * The sequence of steps to freeze LBR should be completely inlined
 	 * and contain no branches to minimize contamination of LBR snapshot
 	 */
 	local_irq_save(flags);
+	cpuc = this_cpu_ptr(&cpu_hw_events);
+	pmu_enabled = cpuc->enabled;
+	lbr_enabled = cpuc->lbr_enabled;
 	amd_pmu_core_disable_all();
 	__amd_pmu_lbr_disable();
 
-	cpuc = this_cpu_ptr(&cpu_hw_events);
-
 	amd_pmu_lbr_read();
 	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
-	amd_pmu_v2_enable_all(0);
+	if (lbr_enabled)
+		amd_pmu_lbr_enable_all();
+	if (pmu_enabled)
+		amd_pmu_core_enable_all();
 	local_irq_restore(flags);
 
 	return cnt;
-- 
2.34.1

Re: [Patch v5 06/18] perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf/x86/amd: amd_pmu_v2_snapshot_branch_stack() leaves cpuc->enabled set while PMU is disabled
--

commit cb298524c7ea53df65b77e5a09ad097e570397f6
Author: Dapeng Mi <dapeng1.mi@linux.intel.com>

perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled

This commit updates amd_pmu_v2_snapshot_branch_stack() to save the current
PMU and LBR enabled state and only re-enable them after taking the LBR
snapshot if they were originally enabled. This avoids unconditionally
re-enabling the PMU and LBR when called from a PMI context where they were
already disabled.

> diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
> index bf631f60757cc..851d8a938f522 100644
> --- a/arch/x86/events/amd/core.c
> +++ b/arch/x86/events/amd/core.c
> @@ -932,22 +932,28 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
>  {
>  	struct cpu_hw_events *cpuc;
>  	unsigned long flags;
> +	int pmu_enabled;
> +	int lbr_enabled;
>  
>  	/*
>  	 * The sequence of steps to freeze LBR should be completely inlined
>  	 * and contain no branches to minimize contamination of LBR snapshot
>  	 */
>  	local_irq_save(flags);
> +	cpuc = this_cpu_ptr(&cpu_hw_events);
> +	pmu_enabled = cpuc->enabled;
> +	lbr_enabled = cpuc->lbr_enabled;
>  	amd_pmu_core_disable_all();
>  	__amd_pmu_lbr_disable();
>  
> -	cpuc = this_cpu_ptr(&cpu_hw_events);
> -
>  	amd_pmu_lbr_read();
>  	cnt = min(cnt, cpuc->lbr_stack.nr);
>  	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
>  
> -	amd_pmu_v2_enable_all(0);
> +	if (lbr_enabled)
> +		amd_pmu_lbr_enable_all();
> +	if (pmu_enabled)
> +		amd_pmu_core_enable_all();

[Severity: Medium]
Should amd_pmu_v2_snapshot_branch_stack() in arch/x86/events/amd/core.c also
clear cpuc->enabled = 0 while the core PMU is disabled, and restore
cpuc->enabled = pmu_enabled before re-enabling it?

While __amd_pmu_lbr_disable() sets cpuc->lbr_enabled = 0 and
amd_pmu_lbr_enable_all() sets cpuc->lbr_enabled = 1, amd_pmu_core_disable_all()
only clears the hardware MSR and leaves cpuc->enabled set to 1.

If a BPF program running in non-NMI context (such as a kprobe, tracepoint, or
fentry/fexit hook) calls bpf_get_branch_snapshot() ->
amd_pmu_v2_snapshot_branch_stack() while cpuc->enabled is 1, local_irq_save()
masks normal interrupts but does not mask NMIs. If an NMI arrives while
amd_pmu_v2_snapshot_branch_stack() is in amd_pmu_lbr_read() or memcpy(),
perf_event_nmi_handler() -> amd_pmu_v2_handle_irq() will observe
cpuc->enabled == 1:

arch/x86/events/amd/core.c:amd_pmu_v2_handle_irq() {
	pmu_enabled = cpuc->enabled;
	cpuc->enabled = 0;

	/* Stop counting but do not disable LBR */
	amd_pmu_core_disable_all();
	...
done:
	cpuc->enabled = pmu_enabled;

	/* Resume counting only if PMU is active */
	if (pmu_enabled)
		amd_pmu_core_enable_all();
}

Similarly, if a BPF program triggered inside an NMI calls
amd_pmu_v2_snapshot_branch_stack() re-entrantly during that window, it will
also see cpuc->enabled == 1.

In both cases, wouldn't the NMI path call amd_pmu_core_enable_all() on exit
and prematurely re-enable hardware PMU counting before the outer
amd_pmu_v2_snapshot_branch_stack() has finished?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=6

[Patch v5 07/18] perf/x86/intel: Add helper to enable PMU without LBR

[Dapeng Mi]

Currently, __intel_pmu_enable_all() also re-enables LBR in addition to
the PMU. Split out the PMU-only path into a new helper,
___intel_pmu_enable_all(), so the PMU can be enabled independently. This
helper will be used by a subsequent patch.

Also remove the unused added argument from __intel_pmu_enable_all().

No functional change intended.

Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 0ed0269189ba..5b44a4224d2c 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2783,13 +2783,11 @@ static __always_inline void intel_pmu_disable_all(void)
 	intel_pmu_lbr_disable_all();
 }
 
-static void __intel_pmu_enable_all(int added, bool pmi)
+static void ___intel_pmu_enable_all(void)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
 
-	intel_pmu_lbr_enable_all(pmi);
-
 	if (cpuc->fixed_ctrl_val != cpuc->active_fixed_ctrl_val) {
 		wrmsrq(MSR_ARCH_PERFMON_FIXED_CTR_CTRL, cpuc->fixed_ctrl_val);
 		cpuc->active_fixed_ctrl_val = cpuc->fixed_ctrl_val;
@@ -2809,10 +2807,16 @@ static void __intel_pmu_enable_all(int added, bool pmi)
 	}
 }
 
+static void __intel_pmu_enable_all(bool pmi)
+{
+	intel_pmu_lbr_enable_all(pmi);
+	___intel_pmu_enable_all();
+}
+
 static void intel_pmu_enable_all(int added)
 {
 	static_call_cond(x86_pmu_pebs_enable_all)();
-	__intel_pmu_enable_all(added, false);
+	__intel_pmu_enable_all(false);
 }
 
 static noinline int
@@ -4056,7 +4060,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 	/* Only restore PMU state when it's active. See x86_pmu_disable(). */
 	cpuc->enabled = pmu_enabled;
 	if (pmu_enabled)
-		__intel_pmu_enable_all(0, true);
+		__intel_pmu_enable_all(true);
 	intel_bts_enable_local();
 
 	/*
-- 
2.34.1

Re: [Patch v5 07/18] perf/x86/intel: Add helper to enable PMU without LBR

[sashiko-bot]

> Currently, __intel_pmu_enable_all() also re-enables LBR in addition to
> the PMU. Split out the PMU-only path into a new helper,
> ___intel_pmu_enable_all(), so the PMU can be enabled independently. This
> helper will be used by a subsequent patch.
> 
> Also remove the unused added argument from __intel_pmu_enable_all().
> 
> No functional change intended.
> 
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=7

[Patch v5 08/18] perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled

[Dapeng Mi]

__intel_pmu_snapshot_branch_stack() currently re-enables the PMU and LBR
unconditionally even when they were already disabled in PMI context.
This can cause events such as cycles and instructions to be counted
while the PMI handler is running, which is not intended.

Only restore the PMU and LBR on the BPF path if they were originally
enabled.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 38 ++++++++++++++++++++++++++++++++----
 1 file changed, 34 insertions(+), 4 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 5b44a4224d2c..15dcf8b78de9 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2821,7 +2821,8 @@ static void intel_pmu_enable_all(int added)
 
 static noinline int
 __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
-				  unsigned int cnt, unsigned long flags)
+				  unsigned int cnt, unsigned long flags,
+				  int pmu_enabled, int lbr_enabled)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
@@ -2829,7 +2830,22 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 	cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
-	intel_pmu_enable_all(0);
+
+	/*
+	 * Do not blindly call intel_pmu_enable_all() to re-enable LBR.
+	 * This path may be entered from the PMI handler through
+	 * bpf_overflow_handler().
+	 *
+	 * If PMU version < 4, LBR is disabled by the LBR freeze mechanism,
+	 * LBR can't be re-enabled here.
+	 * If PMU version >= 4, LBR is only frozen instead of disabled,
+	 * LBR should be re-enabled here.
+	 */
+	if (lbr_enabled)
+		intel_pmu_lbr_enable_all(false);
+	if (pmu_enabled)
+		___intel_pmu_enable_all();
+
 	local_irq_restore(flags);
 	return cnt;
 }
@@ -2837,27 +2853,41 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 static int
 intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries, unsigned int cnt)
 {
+	struct cpu_hw_events *cpuc;
 	unsigned long flags;
+	int pmu_enabled;
+	int lbr_enabled;
 
 	/* must not have branches... */
 	local_irq_save(flags);
+	cpuc = this_cpu_ptr(&cpu_hw_events);
+	pmu_enabled = cpuc->enabled;
+	lbr_enabled = cpuc->lbr_enabled;
 	__intel_pmu_disable_all(false); /* we don't care about BTS */
 	__intel_pmu_lbr_disable();
 	/*            ... until here */
-	return __intel_pmu_snapshot_branch_stack(entries, cnt, flags);
+	return __intel_pmu_snapshot_branch_stack(entries, cnt, flags,
+						 pmu_enabled, lbr_enabled);
 }
 
 static int
 intel_pmu_snapshot_arch_branch_stack(struct perf_branch_entry *entries, unsigned int cnt)
 {
+	struct cpu_hw_events *cpuc;
 	unsigned long flags;
+	int pmu_enabled;
+	int lbr_enabled;
 
 	/* must not have branches... */
 	local_irq_save(flags);
+	cpuc = this_cpu_ptr(&cpu_hw_events);
+	pmu_enabled = cpuc->enabled;
+	lbr_enabled = cpuc->lbr_enabled;
 	__intel_pmu_disable_all(false); /* we don't care about BTS */
 	__intel_pmu_arch_lbr_disable();
 	/*            ... until here */
-	return __intel_pmu_snapshot_branch_stack(entries, cnt, flags);
+	return __intel_pmu_snapshot_branch_stack(entries, cnt, flags,
+						 pmu_enabled, lbr_enabled);
 }
 
 /*
-- 
2.34.1

Re: [Patch v5 08/18] perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled

[sashiko-bot]

> __intel_pmu_snapshot_branch_stack() currently re-enables the PMU and LBR
> unconditionally even when they were already disabled in PMI context.
> This can cause events such as cycles and instructions to be counted
> while the PMI handler is running, which is not intended.
> 
> Only restore the PMU and LBR on the BPF path if they were originally
> enabled.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=8

[Patch v5 09/18] perf/x86/intel: Keep LBR frozen throughout PMI handling

[Dapeng Mi]

For the platforms which has 4+ PMU version, streamline LBR freeze is
supported. The PERF_GLOBAL_STATUS.LBR_FRZ bit is set to freeze LBR once
a PMU counter overflow. However, the current PMI handler clears the LBR
freeze bit too early, allowing branch operations that occur during PMI
handling to be recorded into the LBR stack.

This can pollute or overwrite the intended branch snapshot, especially
when bpf_get_branch_snapshot() is invoked from bpf_overflow_handler().
In that case, branches recorded while processing the PMI are mixed into
the original LBR data.

Defer unfreezing until after PMI handling completes so the LBR remains
frozen for the entire PMI path and no branch activity from the handler
is recorded.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
Fixes: 425507fa5f32 ("perf/x86/intel/lbr: Optimize v4 LBR unfreezing")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 15dcf8b78de9..a67297148bb6 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -4062,7 +4062,13 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 	loops = 0;
 again:
 	intel_pmu_lbr_read();
-	intel_pmu_ack_status(status);
+
+	/*
+	 * Do not clear the LBR frozen status here; keep LBR frozen
+	 * throughout PMI handling.
+	 */
+	intel_pmu_ack_status(status & ~GLOBAL_STATUS_LBRS_FROZEN);
+
 	if (++loops > 100) {
 		static bool warned;
 
@@ -4081,10 +4087,13 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 	 * Repeat if there is more work to be done:
 	 */
 	status = intel_pmu_get_status();
-	if (status)
+	if (status & ~GLOBAL_STATUS_LBRS_FROZEN)
 		goto again;
 
 done:
+	if (status & GLOBAL_STATUS_LBRS_FROZEN)
+		intel_pmu_ack_status(status & GLOBAL_STATUS_LBRS_FROZEN);
+
 	if (mid_ack)
 		apic_write(APIC_LVTPC, APIC_DM_NMI);
 	/* Only restore PMU state when it's active. See x86_pmu_disable(). */
-- 
2.34.1

Re: [Patch v5 09/18] perf/x86/intel: Keep LBR frozen throughout PMI handling

[sashiko-bot]

> For the platforms which has 4+ PMU version, streamline LBR freeze is
> supported. The PERF_GLOBAL_STATUS.LBR_FRZ bit is set to freeze LBR once
> a PMU counter overflow. However, the current PMI handler clears the LBR
> freeze bit too early, allowing branch operations that occur during PMI
> handling to be recorded into the LBR stack.
> 
> This can pollute or overwrite the intended branch snapshot, especially
> when bpf_get_branch_snapshot() is invoked from bpf_overflow_handler().
> In that case, branches recorded while processing the PMI are mixed into
> the original LBR data.
> 
> Defer unfreezing until after PMI handling completes so the LBR remains
> frozen for the entire PMI path and no branch activity from the handler
> is recorded.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
> Fixes: 425507fa5f32 ("perf/x86/intel/lbr: Optimize v4 LBR unfreezing")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=9

Re: [Patch v5 09/18] perf/x86/intel: Keep LBR frozen throughout PMI handling

[Peter Zijlstra]

On Thu, Oct 29, 2026 at 12:39:05PM +0800, Dapeng Mi wrote:
> For the platforms which has 4+ PMU version, streamline LBR freeze is
> supported. The PERF_GLOBAL_STATUS.LBR_FRZ bit is set to freeze LBR once
> a PMU counter overflow. However, the current PMI handler clears the LBR
> freeze bit too early, allowing branch operations that occur during PMI
> handling to be recorded into the LBR stack.
> 
> This can pollute or overwrite the intended branch snapshot, especially
> when bpf_get_branch_snapshot() is invoked from bpf_overflow_handler().
> In that case, branches recorded while processing the PMI are mixed into
> the original LBR data.
> 
> Defer unfreezing until after PMI handling completes so the LBR remains
> frozen for the entire PMI path and no branch activity from the handler
> is recorded.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
> Fixes: 425507fa5f32 ("perf/x86/intel/lbr: Optimize v4 LBR unfreezing")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> ---
>  arch/x86/events/intel/core.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> index 15dcf8b78de9..a67297148bb6 100644
> --- a/arch/x86/events/intel/core.c
> +++ b/arch/x86/events/intel/core.c
> @@ -4062,7 +4062,13 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
>  	loops = 0;
>  again:
>  	intel_pmu_lbr_read();
> -	intel_pmu_ack_status(status);
> +
> +	/*
> +	 * Do not clear the LBR frozen status here; keep LBR frozen
> +	 * throughout PMI handling.
> +	 */
> +	intel_pmu_ack_status(status & ~GLOBAL_STATUS_LBRS_FROZEN);
> +
>  	if (++loops > 100) {
>  		static bool warned;
>  
> @@ -4081,10 +4087,13 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
>  	 * Repeat if there is more work to be done:
>  	 */
>  	status = intel_pmu_get_status();
> -	if (status)
> +	if (status & ~GLOBAL_STATUS_LBRS_FROZEN)
>  		goto again;
>  
>  done:
> +	if (status & GLOBAL_STATUS_LBRS_FROZEN)
> +		intel_pmu_ack_status(status & GLOBAL_STATUS_LBRS_FROZEN);
> +
>  	if (mid_ack)
>  		apic_write(APIC_LVTPC, APIC_DM_NMI);
>  	/* Only restore PMU state when it's active. See x86_pmu_disable(). */

Given the whole PMU is disabled per __intel_pmu_disable_all(), all that
can really happen is for more bits to be set, right? I mean, any counter
cleared on the first ack_status, will never be re-set, it already fired
and its disabled. All that can happen is that the disable is 'delayed'
and a counter can maybe fire a little bit after the disable; getting
missed on the first go.

Can we really get more than 1 extra loop there?

I'm sure this code has changed lots since we added that loop thing
there.

Anyway, if we can never get more than 1 extra loop, we can probably move
the ack to the end, without having to do a second.

I suppose this isn't something you have to do now, but keep it on the
todo list or something?

[Patch v5 10/18] perf/x86/intel: Allow LBR read helpers to operate on explicit stacks

[Dapeng Mi]

The LBR read helpers currently assume the default stack,
cpuc->lbr_stack, and therefore cannot be reused for other LBR
stacks that may be in use. Extend the *_lbr_read() helpers to take
an explicit LBR stack argument, so they can operate on any LBR stack
instead of only the default one.

Update the related callers, including intel_pmu_store_lbr() and
intel_pmu_lbr_filter(), to pass the appropriate stack explicitly.

No function change intended.

Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c |   9 ++-
 arch/x86/events/intel/lbr.c  | 113 ++++++++++++++++++++---------------
 arch/x86/events/perf_event.h |  16 +++--
 3 files changed, 84 insertions(+), 54 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index a67297148bb6..5eda551cc9f2 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2826,10 +2826,12 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
-	intel_pmu_lbr_read();
+	intel_pmu_lbr_read(&cpuc->lbr_stack,
+			   cpuc->lbr_entries, cpuc->lbr_counters);
 	cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
-	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
+	memcpy(entries, cpuc->lbr_entries,
+	       sizeof(struct perf_branch_entry) * cnt);
 
 	/*
 	 * Do not blindly call intel_pmu_enable_all() to re-enable LBR.
@@ -4061,7 +4063,8 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 
 	loops = 0;
 again:
-	intel_pmu_lbr_read();
+	intel_pmu_lbr_read(&cpuc->lbr_stack,
+			   cpuc->lbr_entries, cpuc->lbr_counters);
 
 	/*
 	 * Do not clear the LBR frozen status here; keep LBR frozen
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index ee724b28d90c..63f0504f9d2a 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -105,7 +105,10 @@
 
 #define ARCH_LBR_CTL_MASK			0x7f000e
 
-static void intel_pmu_lbr_filter(struct cpu_hw_events *cpuc);
+static void intel_pmu_lbr_filter(struct cpu_hw_events *cpuc,
+				 struct perf_branch_stack *lbr_stack,
+				 struct perf_branch_entry *lbr_entries,
+				 u64 *lbr_counters);
 
 static __always_inline bool is_lbr_call_stack_bit_set(u64 config)
 {
@@ -741,10 +744,12 @@ void intel_pmu_lbr_disable_all(void)
 	}
 }
 
-void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
+void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc,
+			   struct perf_branch_stack *lbr_stack,
+			   struct perf_branch_entry *lbr_entries)
 {
 	unsigned long mask = x86_pmu.lbr_nr - 1;
-	struct perf_branch_entry *br = cpuc->lbr_entries;
+	struct perf_branch_entry *br = lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 
@@ -766,8 +771,8 @@ void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
 		};
 		br++;
 	}
-	cpuc->lbr_stack.nr = i;
-	cpuc->lbr_stack.hw_idx = tos;
+	lbr_stack->nr = i;
+	lbr_stack->hw_idx = tos;
 }
 
 /*
@@ -775,11 +780,13 @@ void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
  * is the same as the linear address, allowing us to merge the LIP and EIP
  * LBR formats.
  */
-void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
+void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc,
+			   struct perf_branch_stack *lbr_stack,
+			   struct perf_branch_entry *lbr_entries)
 {
 	bool need_info = false, call_stack = false;
 	unsigned long mask = x86_pmu.lbr_nr - 1;
-	struct perf_branch_entry *br = cpuc->lbr_entries;
+	struct perf_branch_entry *br = lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 	int out = 0;
@@ -862,8 +869,8 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
 		};
 		out++;
 	}
-	cpuc->lbr_stack.nr = out;
-	cpuc->lbr_stack.hw_idx = tos;
+	lbr_stack->nr = out;
+	lbr_stack->hw_idx = tos;
 }
 
 static DEFINE_STATIC_KEY_FALSE(x86_lbr_mispred);
@@ -904,8 +911,9 @@ static __always_inline u16 get_lbr_cycles(u64 info)
 
 static_assert((64 - PERF_BRANCH_ENTRY_INFO_BITS_MAX) > LBR_INFO_BR_CNTR_NUM * LBR_INFO_BR_CNTR_BITS);
 
-static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
-				struct lbr_entry *entries)
+static void intel_pmu_store_lbr(struct lbr_entry *entries,
+				struct perf_branch_stack *lbr_stack,
+				struct perf_branch_entry *lbr_entries)
 {
 	struct perf_branch_entry *e;
 	struct lbr_entry *lbr;
@@ -915,7 +923,7 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		lbr = entries ? &entries[i] : NULL;
-		e = &cpuc->lbr_entries[i];
+		e = &lbr_entries[i];
 
 		from = rdlbr_from(i, lbr);
 		/*
@@ -939,18 +947,18 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 			.cycles		= get_lbr_cycles(info),
 			.type		= get_lbr_br_type(info),
 			/*
-			 * Leverage the reserved field of
-			 * cpuc->lbr_entries[i] to temporarily store the
-			 * branch counters information. The later code will
-			 * decide what content can be disclosed to the perf
-			 * tool. Pleae see intel_pmu_lbr_counters_reorder().
+			 * Leverage the reserved field of lbr_entries[i] to
+			 * temporarily store the branch counters information.
+			 * The later code will decide what content can be
+			 * disclosed to the perf tool.
+			 * Please see intel_pmu_lbr_counters_reorder().
 			 */
 			.reserved	= (info >> LBR_INFO_BR_CNTR_OFFSET) &
 					  LBR_INFO_BR_CNTR_FULL_MASK,
 		};
 	}
 
-	cpuc->lbr_stack.nr = i;
+	lbr_stack->nr = i;
 }
 
 /*
@@ -1001,37 +1009,44 @@ void intel_pmu_lbr_save_brstack(struct perf_sample_data *data,
 	perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL);
 }
 
-static void intel_pmu_arch_lbr_read(struct cpu_hw_events *cpuc)
+static void intel_pmu_arch_lbr_read(struct cpu_hw_events *cpuc,
+				    struct perf_branch_stack *lbr_stack,
+				    struct perf_branch_entry *lbr_entries)
 {
-	intel_pmu_store_lbr(cpuc, NULL);
+	intel_pmu_store_lbr(NULL, lbr_stack, lbr_entries);
 }
 
-static void intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc)
+static void
+intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc,
+			      struct perf_branch_stack *lbr_stack,
+			      struct perf_branch_entry *lbr_entries)
 {
 	struct x86_perf_task_context_arch_lbr_xsave *xsave = cpuc->lbr_xsave;
 
 	if (!xsave) {
-		intel_pmu_store_lbr(cpuc, NULL);
+		intel_pmu_store_lbr(NULL, lbr_stack, lbr_entries);
 		return;
 	}
 	xsave->header.xfeatures = 0;
 	xsaves(&xsave->xsave, XFEATURE_MASK_LBR);
 
 	if (xsave->header.xfeatures & XFEATURE_MASK_LBR) {
-		intel_pmu_store_lbr(cpuc, xsave->lbr.entries);
+		intel_pmu_store_lbr(xsave->lbr.entries, lbr_stack, lbr_entries);
 	} else {
 		/* Clear the LBR stack to avoid leaking stale entries. */
-		cpuc->lbr_stack.nr = 0;
+		lbr_stack->nr = 0;
 	}
 }
 
-void intel_pmu_lbr_read(void)
+void intel_pmu_lbr_read(struct perf_branch_stack *lbr_stack,
+			struct perf_branch_entry *lbr_entries,
+			u64 *lbr_counters)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	if (!cpuc->lbr_users || vlbr_exclude_host()) {
-		/* Reset lbr_stack.nr to prevent stale data leakage. */
-		cpuc->lbr_stack.nr = 0;
+		/* Reset lbr_stack->nr to prevent stale data leakage. */
+		lbr_stack->nr = 0;
 		return;
 	}
 
@@ -1044,9 +1059,9 @@ void intel_pmu_lbr_read(void)
 	if (cpuc->lbr_users == cpuc->lbr_pebs_users)
 		return;
 
-	x86_pmu.lbr_read(cpuc);
+	x86_pmu.lbr_read(cpuc, lbr_stack, lbr_entries);
 
-	intel_pmu_lbr_filter(cpuc);
+	intel_pmu_lbr_filter(cpuc, lbr_stack, lbr_entries, lbr_counters);
 }
 
 /*
@@ -1229,7 +1244,10 @@ static const int arch_lbr_br_type_map[ARCH_LBR_BR_TYPE_MAP_MAX] = {
  * in PERF_SAMPLE_BRANCH_STACK sample may vary.
  */
 static void
-intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
+intel_pmu_lbr_filter(struct cpu_hw_events *cpuc,
+		     struct perf_branch_stack *lbr_stack,
+		     struct perf_branch_entry *lbr_entries,
+		     u64 *lbr_counters)
 {
 	u64 from, to;
 	int br_sel = cpuc->br_sel;
@@ -1241,11 +1259,11 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 	    ((br_sel & X86_BR_TYPE_SAVE) != X86_BR_TYPE_SAVE))
 		return;
 
-	for (i = 0; i < cpuc->lbr_stack.nr; i++) {
+	for (i = 0; i < lbr_stack->nr; i++) {
 
-		from = cpuc->lbr_entries[i].from;
-		to = cpuc->lbr_entries[i].to;
-		type = cpuc->lbr_entries[i].type;
+		from = lbr_entries[i].from;
+		to = lbr_entries[i].to;
+		type = lbr_entries[i].type;
 
 		/*
 		 * Parse the branch type recorded in LBR_x_INFO MSR.
@@ -1257,9 +1275,9 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 			to_plm = kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER;
 			type = arch_lbr_br_type_map[type] | to_plm;
 		} else
-			type = branch_type(from, to, cpuc->lbr_entries[i].abort);
+			type = branch_type(from, to, lbr_entries[i].abort);
 		if (type != X86_BR_NONE && (br_sel & X86_BR_ANYTX)) {
-			if (cpuc->lbr_entries[i].in_tx)
+			if (lbr_entries[i].in_tx)
 				type |= X86_BR_IN_TX;
 			else
 				type |= X86_BR_NO_TX;
@@ -1273,27 +1291,27 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 		 */
 		if (type == X86_BR_NONE || (br_sel & type) != type ||
 		    (!(br_sel & X86_BR_KERNEL) && (from_plm & X86_BR_KERNEL))) {
-			cpuc->lbr_entries[i].from = 0;
+			lbr_entries[i].from = 0;
 			compress = true;
 		}
 
 		if ((br_sel & X86_BR_TYPE_SAVE) == X86_BR_TYPE_SAVE)
-			cpuc->lbr_entries[i].type = common_branch_type(type);
+			lbr_entries[i].type = common_branch_type(type);
 	}
 
 	if (!compress)
 		return;
 
 	/* remove all entries with from=0 */
-	for (i = 0; i < cpuc->lbr_stack.nr; ) {
-		if (!cpuc->lbr_entries[i].from) {
+	for (i = 0; i < lbr_stack->nr; ) {
+		if (!lbr_entries[i].from) {
 			j = i;
-			while (++j < cpuc->lbr_stack.nr) {
-				cpuc->lbr_entries[j-1] = cpuc->lbr_entries[j];
-				cpuc->lbr_counters[j-1] = cpuc->lbr_counters[j];
+			while (++j < lbr_stack->nr) {
+				lbr_entries[j-1] = lbr_entries[j];
+				lbr_counters[j-1] = lbr_counters[j];
 			}
-			cpuc->lbr_stack.nr--;
-			if (!cpuc->lbr_entries[i].from)
+			lbr_stack->nr--;
+			if (!lbr_entries[i].from)
 				continue;
 		}
 		i++;
@@ -1311,8 +1329,9 @@ void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr)
 	else
 		cpuc->lbr_stack.hw_idx = intel_pmu_lbr_tos();
 
-	intel_pmu_store_lbr(cpuc, lbr);
-	intel_pmu_lbr_filter(cpuc);
+	intel_pmu_store_lbr(lbr, &cpuc->lbr_stack, cpuc->lbr_entries);
+	intel_pmu_lbr_filter(cpuc, &cpuc->lbr_stack,
+			     cpuc->lbr_entries, cpuc->lbr_counters);
 }
 
 /*
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index a4aab8130e92..2b942a445038 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -1192,7 +1192,9 @@ struct x86_pmu {
 	unsigned int	lbr_counters:4;
 
 	void		(*lbr_reset)(void);
-	void		(*lbr_read)(struct cpu_hw_events *cpuc);
+	void		(*lbr_read)(struct cpu_hw_events *cpuc,
+				    struct perf_branch_stack *lbr_stack,
+				    struct perf_branch_entry *lbr_entries);
 	void		(*lbr_save)(void *ctx);
 	void		(*lbr_restore)(void *ctx);
 
@@ -1992,11 +1994,17 @@ void intel_pmu_lbr_enable_all(bool pmi);
 
 void intel_pmu_lbr_disable_all(void);
 
-void intel_pmu_lbr_read(void);
+void intel_pmu_lbr_read(struct perf_branch_stack *lbr_stack,
+			struct perf_branch_entry *lbr_entries,
+			u64 *lbr_counters);
 
-void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc);
+void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc,
+			   struct perf_branch_stack *lbr_stack,
+			   struct perf_branch_entry *lbr_entries);
 
-void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc);
+void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc,
+			   struct perf_branch_stack *lbr_stack,
+			   struct perf_branch_entry *lbr_entries);
 
 void intel_pmu_lbr_save(void *ctx);
 
-- 
2.34.1

Re: [Patch v5 10/18] perf/x86/intel: Allow LBR read helpers to operate on explicit stacks

[sashiko-bot]

> The LBR read helpers currently assume the default stack,
> cpuc->lbr_stack, and therefore cannot be reused for other LBR
> stacks that may be in use. Extend the *_lbr_read() helpers to take
> an explicit LBR stack argument, so they can operate on any LBR stack
> instead of only the default one.
> 
> Update the related callers, including intel_pmu_store_lbr() and
> intel_pmu_lbr_filter(), to pass the appropriate stack explicitly.
> 
> No function change intended.
> 
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=10

[Patch v5 11/18] perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting

[Dapeng Mi]

BPF LBR snapshotting currently reuses the default LBR stack
(cpuc->lbr_stack) when storing sampled data. This is unsafe when the
BPF helper runs in PMI context via bpf_overflow_handler(): it can
overwrite the cached LBR state for active PEBS LBR events, causing
incorrect LBR data to be reported to userspace. When the PEBS and BPF
stacks have different depths, this can also lead to out-of-bounds reads
when the LBR stack is later exported.

Use a dedicated LBR stack for BPF snapshotting so the BPF path no
longer clobbers the shared LBR state used by other perf events.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 425507fa5f32 ("perf/x86/intel/lbr: Optimize v4 LBR unfreezing")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 5eda551cc9f2..482a148916bf 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2819,18 +2819,24 @@ static void intel_pmu_enable_all(int added)
 	__intel_pmu_enable_all(false);
 }
 
+struct lbr_snapshot {
+	struct perf_branch_stack stack;
+	struct perf_branch_entry entries[MAX_LBR_ENTRIES];
+	u64 counters[MAX_LBR_ENTRIES];
+};
+static DEFINE_PER_CPU(struct lbr_snapshot, lbr_snapshot);
+
 static noinline int
 __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 				  unsigned int cnt, unsigned long flags,
 				  int pmu_enabled, int lbr_enabled)
 {
-	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	struct lbr_snapshot *bpf = this_cpu_ptr(&lbr_snapshot);
 
-	intel_pmu_lbr_read(&cpuc->lbr_stack,
-			   cpuc->lbr_entries, cpuc->lbr_counters);
-	cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
+	intel_pmu_lbr_read(&bpf->stack, bpf->entries, bpf->counters);
+	cnt = min_t(unsigned int, cnt, bpf->stack.nr);
 
-	memcpy(entries, cpuc->lbr_entries,
+	memcpy(entries, bpf->entries,
 	       sizeof(struct perf_branch_entry) * cnt);
 
 	/*
-- 
2.34.1

Re: [Patch v5 11/18] perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting

[sashiko-bot]

> BPF LBR snapshotting currently reuses the default LBR stack
> (cpuc->lbr_stack) when storing sampled data. This is unsafe when the
> BPF helper runs in PMI context via bpf_overflow_handler(): it can
> overwrite the cached LBR state for active PEBS LBR events, causing
> incorrect LBR data to be reported to userspace. When the PEBS and BPF
> stacks have different depths, this can also lead to out-of-bounds reads
> when the LBR stack is later exported.
> 
> Use a dedicated LBR stack for BPF snapshotting so the BPF path no
> longer clobbers the shared LBR state used by other perf events.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Fixes: 425507fa5f32 ("perf/x86/intel/lbr: Optimize v4 LBR unfreezing")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=11

[Patch v5 12/18] perf/x86/intel: Always snapshot live hardware LBR state for BPF

[Dapeng Mi]

__intel_pmu_snapshot_branch_stack() is intended to capture the current
live hardware LBR state for the BPF path. However, that snapshot is
currently skipped when all LBR users are PEBS events.

BPF LBR snapshotting now uses its own dedicated LBR stack, so there is
no longer a risk of clobbering the shared default LBR stack. As a
result, always snapshot the live hardware LBR state for the BPF path.

Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 10 ++++++++--
 arch/x86/events/intel/lbr.c  |  9 ---------
 2 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 482a148916bf..110fff65bd50 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -4069,8 +4069,14 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 
 	loops = 0;
 again:
-	intel_pmu_lbr_read(&cpuc->lbr_stack,
-			   cpuc->lbr_entries, cpuc->lbr_counters);
+	/*
+	 * No need to read the LBR stack when all LBR users are PEBS events,
+	 * since the PEBS handler updates it.
+	 */
+	if (cpuc->lbr_users != cpuc->lbr_pebs_users) {
+		intel_pmu_lbr_read(&cpuc->lbr_stack,
+				   cpuc->lbr_entries, cpuc->lbr_counters);
+	}
 
 	/*
 	 * Do not clear the LBR frozen status here; keep LBR frozen
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index 63f0504f9d2a..790580f96040 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -1050,15 +1050,6 @@ void intel_pmu_lbr_read(struct perf_branch_stack *lbr_stack,
 		return;
 	}
 
-	/*
-	 * Don't read when all LBRs users are using adaptive PEBS.
-	 *
-	 * This could be smarter and actually check the event,
-	 * but this simple approach seems to work for now.
-	 */
-	if (cpuc->lbr_users == cpuc->lbr_pebs_users)
-		return;
-
 	x86_pmu.lbr_read(cpuc, lbr_stack, lbr_entries);
 
 	intel_pmu_lbr_filter(cpuc, lbr_stack, lbr_entries, lbr_counters);
-- 
2.34.1

Re: [Patch v5 12/18] perf/x86/intel: Always snapshot live hardware LBR state for BPF

[sashiko-bot]

> __intel_pmu_snapshot_branch_stack() is intended to capture the current
> live hardware LBR state for the BPF path. However, that snapshot is
> currently skipped when all LBR users are PEBS events.
> 
> BPF LBR snapshotting now uses its own dedicated LBR stack, so there is
> no longer a risk of clobbering the shared default LBR stack. As a
> result, always snapshot the live hardware LBR state for the BPF path.
> 
> Fixes: c22ac2a3d4bd ("perf: Enable branch record for software events")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=12

[Patch v5 13/18] perf/x86/intel: Refresh LBR stack for non-PEBS LBR events if needed

[Dapeng Mi]

PEBS and non-PEBS events currently share the same LBR stack,
cpuc->lbr_stack. In the PMI handler, the non-PEBS path refreshes the
LBR state before the PEBS buffer is drained.

When PEBS and non-PEBS LBR events overflow in the same PMI, draining the
PEBS buffer can overwrite the shared LBR stack with the PEBS snapshot.
This leaves stale LBR data from the PEBS record in place, and userspace
may observe the wrong branch stack instead of the live LBR state at PMI
entry.

Add a lbr_need_update flag to indicate when the LBR state must be
refreshed. If PEBS events and non-PEBS LBR events overflow in the same
PMI, set this flag so the non-PEBS LBR event refreshes the live LBR
state. Since the LBR state is frozen for the duration of PMI handling, it
only needs to be read once even when multiple non-PEBS LBR events
overflow in the same PMI.

Fixes: c22497f5838c ("perf/x86/intel: Support adaptive PEBS v4")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 110fff65bd50..37f304a1b48c 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3863,8 +3863,9 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 {
 	struct perf_sample_data data;
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	int bit;
+	bool lbr_need_update = false;
 	int handled = 0;
+	int bit;
 
 	inc_perf_irq_stat();
 
@@ -3929,6 +3930,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		if (cpuc->events[INTEL_PMC_IDX_FIXED_SLOTS] &&
 		    is_pebs_counter_event_group(cpuc->events[INTEL_PMC_IDX_FIXED_SLOTS]))
 			status &= ~GLOBAL_STATUS_PERF_METRICS_OVF_BIT;
+
+		lbr_need_update = true;
 	}
 
 	/*
@@ -3944,6 +3947,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		if (cpuc->events[INTEL_PMC_IDX_FIXED_SLOTS] &&
 		    is_pebs_counter_event_group(cpuc->events[INTEL_PMC_IDX_FIXED_SLOTS]))
 			status &= ~GLOBAL_STATUS_PERF_METRICS_OVF_BIT;
+
+		lbr_need_update = true;
 	}
 
 	/*
@@ -4002,8 +4007,14 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		 *
 		 * The PEBS buffer has to be drained before handling the A-PMI
 		 */
-		if (is_pebs_counter_event_group(event))
+		if (is_pebs_counter_event_group(event)) {
 			static_call(x86_pmu_drain_pebs)(regs, &data);
+			/*
+			 * Draining the PEBS buffer can overwrite the shared
+			 * LBR stack, so mark lbr_need_update as true.
+			 */
+			lbr_need_update = true;
+		}
 
 		last_period = event->hw.last_period;
 
@@ -4012,8 +4023,14 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 
 		perf_sample_data_init(&data, 0, last_period);
 
-		if (has_branch_stack(event))
+		if (has_branch_stack(event)) {
+			if (lbr_need_update) {
+				intel_pmu_lbr_read(&cpuc->lbr_stack,
+					cpuc->lbr_entries, cpuc->lbr_counters);
+				lbr_need_update = false;
+			}
 			intel_pmu_lbr_save_brstack(&data, cpuc, event);
+		}
 
 		x86_pmu_clear_perf_regs(regs);
 		x86_pmu_update_perf_regs(event, &data, regs, false);
-- 
2.34.1

Re: [Patch v5 13/18] perf/x86/intel: Refresh LBR stack for non-PEBS LBR events if needed

[sashiko-bot]

> PEBS and non-PEBS events currently share the same LBR stack,
> cpuc->lbr_stack. In the PMI handler, the non-PEBS path refreshes the
> LBR state before the PEBS buffer is drained.
> 
> When PEBS and non-PEBS LBR events overflow in the same PMI, draining the
> PEBS buffer can overwrite the shared LBR stack with the PEBS snapshot.
> This leaves stale LBR data from the PEBS record in place, and userspace
> may observe the wrong branch stack instead of the live LBR state at PMI
> entry.
> 
> Add a lbr_need_update flag to indicate when the LBR state must be
> refreshed. If PEBS events and non-PEBS LBR events overflow in the same
> PMI, set this flag so the non-PEBS LBR event refreshes the live LBR
> state. Since the LBR state is frozen for the duration of PMI handling, it
> only needs to be read once even when multiple non-PEBS LBR events
> overflow in the same PMI.
> 
> Fixes: c22497f5838c ("perf/x86/intel: Support adaptive PEBS v4")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=13

[Patch v5 14/18] perf/x86/intel: Account arch-PEBS in lbr_pebs_users

[Dapeng Mi]

lbr_pebs_users tracks how many PEBS events are currently using LBR.
It is updated for legacy PEBS events, but arch-PEBS events were missing
from that accounting.

Add the missing lbr_pebs_users accounting for arch-PEBS as well.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
Fixes: d243d0bb64af ("perf/x86/intel: Initialize architectural PEBS")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/lbr.c  | 4 ++--
 arch/x86/events/perf_event.h | 7 +++++++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index 790580f96040..2c9c9cf42e94 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -618,7 +618,7 @@ void intel_pmu_lbr_add(struct perf_event *event)
 	 * be 'new'. Conversely, a new event can get installed through the
 	 * context switch path for the first time.
 	 */
-	if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip > 0)
+	if (intel_pmu_has_pebs_lbr(event->pmu) && event->attr.precise_ip > 0)
 		cpuc->lbr_pebs_users++;
 	perf_sched_cb_inc(event->pmu);
 	if (!cpuc->lbr_users++ && !event->total_time_running)
@@ -689,7 +689,7 @@ void intel_pmu_lbr_del(struct perf_event *event)
 	if (event->hw.flags & PERF_X86_EVENT_LBR_SELECT)
 		cpuc->lbr_select = 0;
 
-	if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip > 0)
+	if (intel_pmu_has_pebs_lbr(event->pmu) && event->attr.precise_ip > 0)
 		cpuc->lbr_pebs_users--;
 	cpuc->lbr_users--;
 	WARN_ON_ONCE(cpuc->lbr_users < 0);
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index 2b942a445038..82e0c20e435d 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -2085,6 +2085,13 @@ static inline bool intel_pmu_has_pebs(void)
 	return x86_pmu.ds_pebs || x86_pmu.arch_pebs;
 }
 
+static inline bool intel_pmu_has_pebs_lbr(struct pmu *pmu)
+{
+	return x86_pmu.arch_pebs ?
+		!!(hybrid(pmu, arch_pebs_cap).caps & ARCH_PEBS_LBR) :
+		!!x86_pmu.intel_cap.pebs_baseline;
+}
+
 #else /* CONFIG_CPU_SUP_INTEL */
 
 static inline void reserve_ds_buffers(void)
-- 
2.34.1

Re: [Patch v5 14/18] perf/x86/intel: Account arch-PEBS in lbr_pebs_users

[sashiko-bot]

> lbr_pebs_users tracks how many PEBS events are currently using LBR.
> It is updated for legacy PEBS events, but arch-PEBS events were missing
> from that accounting.
> 
> Add the missing lbr_pebs_users accounting for arch-PEBS as well.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/sashiko-outbox-179823@kernel.org/
> Fixes: d243d0bb64af ("perf/x86/intel: Initialize architectural PEBS")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=14

[Patch v5 15/18] perf/x86/intel: Clear raw LBR counters in the LBR stack

[Dapeng Mi]

Raw LBR counters are currently stored in the reserved fields of the LBR
stack even when no LBR event needs them.

If an LBR counter event is present, it clears the raw LBR counters while
processing the stack, but those counters can be leaked to userspace when
no LBR event requires them. Clear the raw LBR counters when no LBR event
uses them.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 33744916196b ("perf/x86/intel: Support branch counters logging")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c |  1 +
 arch/x86/events/intel/lbr.c  | 14 ++++++++++++++
 arch/x86/events/perf_event.h |  3 +++
 3 files changed, 18 insertions(+)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 37f304a1b48c..e1834b329e37 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2834,6 +2834,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
 	struct lbr_snapshot *bpf = this_cpu_ptr(&lbr_snapshot);
 
 	intel_pmu_lbr_read(&bpf->stack, bpf->entries, bpf->counters);
+	intel_pmu_lbr_clear_counters(&bpf->stack, bpf->entries);
 	cnt = min_t(unsigned int, cnt, bpf->stack.nr);
 
 	memcpy(entries, bpf->entries,
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index 2c9c9cf42e94..e9336d570242 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -996,6 +996,19 @@ static void intel_pmu_lbr_counters_reorder(struct cpu_hw_events *cpuc,
 	}
 }
 
+void intel_pmu_lbr_clear_counters(struct perf_branch_stack *lbr_stack,
+				  struct perf_branch_entry *lbr_entries)
+{
+	int i;
+
+	if (!(x86_pmu.flags & PMU_FL_BR_CNTR))
+		return;
+
+	/* reserved field stores raw branch counters, clear it. */
+	for (i = 0; i < lbr_stack->nr; i++)
+		lbr_entries[i].reserved = 0;
+}
+
 void intel_pmu_lbr_save_brstack(struct perf_sample_data *data,
 				struct cpu_hw_events *cpuc,
 				struct perf_event *event)
@@ -1006,6 +1019,7 @@ void intel_pmu_lbr_save_brstack(struct perf_sample_data *data,
 		return;
 	}
 
+	intel_pmu_lbr_clear_counters(&cpuc->lbr_stack, cpuc->lbr_entries);
 	perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL);
 }
 
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index 82e0c20e435d..160df7b99621 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -1971,6 +1971,9 @@ void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr);
 
 void intel_pebs_init(void);
 
+void intel_pmu_lbr_clear_counters(struct perf_branch_stack *lbr_stack,
+				  struct perf_branch_entry *lbr_entries);
+
 void intel_pmu_lbr_save_brstack(struct perf_sample_data *data,
 				struct cpu_hw_events *cpuc,
 				struct perf_event *event);
-- 
2.34.1

Re: [Patch v5 15/18] perf/x86/intel: Clear raw LBR counters in the LBR stack

[sashiko-bot]

> Raw LBR counters are currently stored in the reserved fields of the LBR
> stack even when no LBR event needs them.
> 
> If an LBR counter event is present, it clears the raw LBR counters while
> processing the stack, but those counters can be leaked to userspace when
> no LBR event requires them. Clear the raw LBR counters when no LBR event
> uses them.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Fixes: 33744916196b ("perf/x86/intel: Support branch counters logging")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=15

[Patch v5 16/18] perf/x86/intel: Refill LBR stack for LBR counter events in PMI handler

[Dapeng Mi]

When handling an LBR counter event, the raw LBR counters in the reserved
field of each LBR entry are cleared after the stack is processed and
reported to userspace. This prevents raw counter data from leaking.

However, if multiple events overflow in the same PMI, an LBR counter
event may not be the first one processed. In that case, the raw LBR
counters may already have been cleared by the previous LBR event, so the
later LBR counter event cannot read the correct values.

Refill the LBR stack before processing an LBR counter event when multiple
events are handled in a single PMI. This restores the correct raw LBR
counter data for the event in this corner case.

This situation is rare, the performance impact from extra refill should
be negligible.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 33744916196b ("perf/x86/intel: Support branch counters logging")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index e1834b329e37..9267f2d7d21b 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3864,6 +3864,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 {
 	struct perf_sample_data data;
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	bool lbr_counter_cleared = false;
 	bool lbr_need_update = false;
 	int handled = 0;
 	int bit;
@@ -4025,12 +4026,20 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		perf_sample_data_init(&data, 0, last_period);
 
 		if (has_branch_stack(event)) {
-			if (lbr_need_update) {
+			/*
+			 * The raw LBR counters stored in the reserved field of
+			 * the LBR entries were cleared during processing of the
+			 * previous LBR event, so refresh the LBR stack again if
+			 * this event needs the LBR counters.
+			 */
+			if (lbr_need_update ||
+			    (lbr_counter_cleared && is_branch_counters_group(event))) {
 				intel_pmu_lbr_read(&cpuc->lbr_stack,
 					cpuc->lbr_entries, cpuc->lbr_counters);
 				lbr_need_update = false;
 			}
 			intel_pmu_lbr_save_brstack(&data, cpuc, event);
+			lbr_counter_cleared = true;
 		}
 
 		x86_pmu_clear_perf_regs(regs);
-- 
2.34.1

Re: [Patch v5 16/18] perf/x86/intel: Refill LBR stack for LBR counter events in PMI handler

[sashiko-bot]

> When handling an LBR counter event, the raw LBR counters in the reserved
> field of each LBR entry are cleared after the stack is processed and
> reported to userspace. This prevents raw counter data from leaking.
> 
> However, if multiple events overflow in the same PMI, an LBR counter
> event may not be the first one processed. In that case, the raw LBR
> counters may already have been cleared by the previous LBR event, so the
> later LBR counter event cannot read the correct values.
> 
> Refill the LBR stack before processing an LBR counter event when multiple
> events are handled in a single PMI. This restores the correct raw LBR
> counter data for the event in this corner case.
> 
> This situation is rare, the performance impact from extra refill should
> be negligible.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Fixes: 33744916196b ("perf/x86/intel: Support branch counters logging")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=16

[Patch v5 17/18] perf/x86/intel: Pass LBR stack size into store_lbr()

[Dapeng Mi]

intel_pmu_store_lbr() assumes the LBR stack always contains
x86_pmu.lbr_nr entries, but arch-PEBS can use a smaller configured depth.
This can result in out-of-bounds reads when the stack is shallower than
the hardware maximum.

Add a lbr_nr parameter to the helper so the code only processes the
available entries and avoids overrunning the LBR buffer.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: d21954c8a0ff ("perf/x86/intel: Process arch-PEBS records or record fragments")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/ds.c   |  8 ++++----
 arch/x86/events/intel/lbr.c  | 16 +++++++++-------
 arch/x86/events/perf_event.h |  2 +-
 3 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 8444670cee4a..ac452083de7a 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -2591,12 +2591,12 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event,
 
 	if (format_group & PEBS_DATACFG_LBRS) {
 		struct lbr_entry *lbr = next_record;
-		int num_lbr = ((format_group >> PEBS_DATACFG_LBR_SHIFT)
+		unsigned int num_lbr = ((format_group >> PEBS_DATACFG_LBR_SHIFT)
 					& 0xff) + 1;
 		next_record = next_record + num_lbr * sizeof(struct lbr_entry);
 
 		if (has_branch_stack(event)) {
-			intel_pmu_store_pebs_lbrs(lbr);
+			intel_pmu_store_pebs_lbrs(lbr, num_lbr);
 			intel_pmu_lbr_save_brstack(data, cpuc, event);
 		}
 	}
@@ -2777,7 +2777,7 @@ static void setup_arch_pebs_sample_data(struct perf_event *event,
 	if (header->lbr) {
 		struct arch_pebs_lbr_header *lbr_header = next_record;
 		struct lbr_entry *lbr;
-		int num_lbr;
+		unsigned int num_lbr;
 
 		next_record = lbr_header + 1;
 		lbr = next_record;
@@ -2788,7 +2788,7 @@ static void setup_arch_pebs_sample_data(struct perf_event *event,
 		next_record += num_lbr * sizeof(struct lbr_entry);
 
 		if (has_branch_stack(event)) {
-			intel_pmu_store_pebs_lbrs(lbr);
+			intel_pmu_store_pebs_lbrs(lbr, num_lbr);
 			intel_pmu_lbr_save_brstack(data, cpuc, event);
 		}
 	}
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index e9336d570242..f0098b0f16d3 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -911,7 +911,7 @@ static __always_inline u16 get_lbr_cycles(u64 info)
 
 static_assert((64 - PERF_BRANCH_ENTRY_INFO_BITS_MAX) > LBR_INFO_BR_CNTR_NUM * LBR_INFO_BR_CNTR_BITS);
 
-static void intel_pmu_store_lbr(struct lbr_entry *entries,
+static void intel_pmu_store_lbr(struct lbr_entry *entries, int lbr_nr,
 				struct perf_branch_stack *lbr_stack,
 				struct perf_branch_entry *lbr_entries)
 {
@@ -921,7 +921,7 @@ static void intel_pmu_store_lbr(struct lbr_entry *entries,
 	bool mispred;
 	int i;
 
-	for (i = 0; i < x86_pmu.lbr_nr; i++) {
+	for (i = 0; i < lbr_nr; i++) {
 		lbr = entries ? &entries[i] : NULL;
 		e = &lbr_entries[i];
 
@@ -1027,7 +1027,7 @@ static void intel_pmu_arch_lbr_read(struct cpu_hw_events *cpuc,
 				    struct perf_branch_stack *lbr_stack,
 				    struct perf_branch_entry *lbr_entries)
 {
-	intel_pmu_store_lbr(NULL, lbr_stack, lbr_entries);
+	intel_pmu_store_lbr(NULL, x86_pmu.lbr_nr, lbr_stack, lbr_entries);
 }
 
 static void
@@ -1038,14 +1038,15 @@ intel_pmu_arch_lbr_read_xsave(struct cpu_hw_events *cpuc,
 	struct x86_perf_task_context_arch_lbr_xsave *xsave = cpuc->lbr_xsave;
 
 	if (!xsave) {
-		intel_pmu_store_lbr(NULL, lbr_stack, lbr_entries);
+		intel_pmu_store_lbr(NULL, x86_pmu.lbr_nr, lbr_stack, lbr_entries);
 		return;
 	}
 	xsave->header.xfeatures = 0;
 	xsaves(&xsave->xsave, XFEATURE_MASK_LBR);
 
 	if (xsave->header.xfeatures & XFEATURE_MASK_LBR) {
-		intel_pmu_store_lbr(xsave->lbr.entries, lbr_stack, lbr_entries);
+		intel_pmu_store_lbr(xsave->lbr.entries, x86_pmu.lbr_nr,
+				    lbr_stack, lbr_entries);
 	} else {
 		/* Clear the LBR stack to avoid leaking stale entries. */
 		lbr_stack->nr = 0;
@@ -1323,7 +1324,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc,
 	}
 }
 
-void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr)
+void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr, unsigned int lbr_nr)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
@@ -1334,7 +1335,8 @@ void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr)
 	else
 		cpuc->lbr_stack.hw_idx = intel_pmu_lbr_tos();
 
-	intel_pmu_store_lbr(lbr, &cpuc->lbr_stack, cpuc->lbr_entries);
+	intel_pmu_store_lbr(lbr, min(lbr_nr, x86_pmu.lbr_nr),
+			    &cpuc->lbr_stack, cpuc->lbr_entries);
 	intel_pmu_lbr_filter(cpuc, &cpuc->lbr_stack,
 			     cpuc->lbr_entries, cpuc->lbr_counters);
 }
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index 160df7b99621..feceeebf0d06 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -1967,7 +1967,7 @@ void intel_pmu_pebs_late_setup(struct cpu_hw_events *cpuc);
 
 void intel_pmu_drain_pebs_buffer(void);
 
-void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr);
+void intel_pmu_store_pebs_lbrs(struct lbr_entry *lbr, unsigned int lbr_nr);
 
 void intel_pebs_init(void);
 
-- 
2.34.1

Re: [Patch v5 17/18] perf/x86/intel: Pass LBR stack size into store_lbr()

[sashiko-bot]

> intel_pmu_store_lbr() assumes the LBR stack always contains
> x86_pmu.lbr_nr entries, but arch-PEBS can use a smaller configured depth.
> This can result in out-of-bounds reads when the stack is shallower than
> the hardware maximum.
> 
> Add a lbr_nr parameter to the helper so the code only processes the
> available entries and avoids overrunning the LBR buffer.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Fixes: d21954c8a0ff ("perf/x86/intel: Process arch-PEBS records or record fragments")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=17

[Patch v5 18/18] perf/core: Don't overwrite shared LBR stack depth

[Dapeng Mi]

perf_sample_save_brstack() currently clamps the shared LBR stack depth
to min(event->attr.sample_max_stack, brs->nr). This is unsafe because
the global LBR stack is shared across all LBR events, and a small
sample_max_stack from one event can shrink the depth for all others in
the same PMI.

As a result, later events may output a shorter LBR stack than requested
even when they do not require a small sample depth. Do not overwrite the
shared LBR stack depth in perf_sample_save_brstack(); instead, keep the
per-event limit in a local variable so each event is constrained
independently without mutating global state.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: c53e14f1ea4a ("perf: Extend per event callchain limit to branch stack")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 include/linux/perf_event.h | 13 ++++++++++---
 kernel/events/core.c       |  8 ++++----
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 7797ce207555..1188aace3335 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1424,12 +1424,19 @@ static inline bool has_branch_stack(struct perf_event *event)
 	return event->attr.sample_type & PERF_SAMPLE_BRANCH_STACK;
 }
 
+static inline u64 max_lbr_stack_depth(struct perf_event *event,
+				      struct perf_branch_stack *brs)
+{
+	return min_t(u16, event->attr.sample_max_stack, brs->nr);
+}
+
 static inline void perf_sample_save_brstack(struct perf_sample_data *data,
 					    struct perf_event *event,
 					    struct perf_branch_stack *brs,
 					    u64 *brs_cntr)
 {
 	int size = sizeof(u64); /* nr */
+	u64 lbr_nr;
 
 	if (!has_branch_stack(event))
 		return;
@@ -1439,9 +1446,9 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data,
 	if (branch_sample_hw_index(event))
 		size += sizeof(u64);
 
-	brs->nr = min_t(u16, event->attr.sample_max_stack, brs->nr);
+	lbr_nr = max_lbr_stack_depth(event, brs);
 
-	size += brs->nr * sizeof(struct perf_branch_entry);
+	size += lbr_nr * sizeof(struct perf_branch_entry);
 
 	/*
 	 * The extension space for counters is appended after the
@@ -1449,7 +1456,7 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data,
 	 * of events of each branch.
 	 */
 	if (brs_cntr)
-		size += brs->nr * sizeof(u64);
+		size += lbr_nr * sizeof(u64);
 
 	data->br_stack = brs;
 	data->br_stack_cntr = brs_cntr;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index de05df65ab3d..ebc1cb417806 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -8442,12 +8442,12 @@ void perf_output_sample(struct perf_output_handle *handle,
 
 	if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
 		if (data->br_stack) {
+			u64 lbr_nr = max_lbr_stack_depth(event, data->br_stack);
 			size_t size;
 
-			size = data->br_stack->nr
-			     * sizeof(struct perf_branch_entry);
+			size = lbr_nr * sizeof(struct perf_branch_entry);
 
-			perf_output_put(handle, data->br_stack->nr);
+			perf_output_put(handle, lbr_nr);
 			if (branch_sample_hw_index(event))
 				perf_output_put(handle, data->br_stack->hw_idx);
 			perf_output_copy(handle, data->br_stack->entries, size);
@@ -8456,7 +8456,7 @@ void perf_output_sample(struct perf_output_handle *handle,
 			 * right after the struct perf_branch_stack.
 			 */
 			if (data->br_stack_cntr) {
-				size = data->br_stack->nr * sizeof(u64);
+				size = lbr_nr * sizeof(u64);
 				perf_output_copy(handle, data->br_stack_cntr, size);
 			}
 		} else {
-- 
2.34.1

Re: [Patch v5 18/18] perf/core: Don't overwrite shared LBR stack depth

[sashiko-bot]

> perf_sample_save_brstack() currently clamps the shared LBR stack depth
> to min(event->attr.sample_max_stack, brs->nr). This is unsafe because
> the global LBR stack is shared across all LBR events, and a small
> sample_max_stack from one event can shrink the depth for all others in
> the same PMI.
> 
> As a result, later events may output a shorter LBR stack than requested
> even when they do not require a small sample depth. Do not overwrite the
> shared LBR stack depth in perf_sample_save_brstack(); instead, keep the
> per-event limit in a local variable so each event is constrained
> independently without mutating global state.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Fixes: c53e14f1ea4a ("perf: Extend per event callchain limit to branch stack")
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029043914.1395147-1-dapeng1.mi@linux.intel.com?part=18