From: Dapeng Mi <dapeng1.mi@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Jiri Olsa <jolsa@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <ak@linux.intel.com>,
Eranian Stephane <eranian@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>,
broonie@kernel.org, Ravi Bangoria <ravi.bangoria@amd.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
Zide Chen <zide.chen@intel.com>,
Falcon Thomas <thomas.falcon@intel.com>,
Dapeng Mi <dapeng1.mi@intel.com>,
Xudong Hao <xudong.hao@intel.com>,
Dapeng Mi <dapeng1.mi@linux.intel.com>
Subject: [Patch v6 21/22] perf/x86/intel: Enable arch-PEBS based SIMD/eGPRs/SSP sampling
Date: Mon, 9 Feb 2026 15:20:46 +0800 [thread overview]
Message-ID: <20260209072047.2180332-22-dapeng1.mi@linux.intel.com> (raw)
In-Reply-To: <20260209072047.2180332-1-dapeng1.mi@linux.intel.com>
This patch enables arch-PEBS based SIMD/eGPRs/SSP registers sampling.
Arch-PEBS supports sampling of these registers, with all except SSP
placed into the XSAVE-Enabled Registers (XER) group with the layout
described below.
Field Name Registers Used Size
----------------------------------------------------------------------
XSTATE_BV XINUSE for groups 8 B
----------------------------------------------------------------------
Reserved Reserved 8 B
----------------------------------------------------------------------
SSER XMM0-XMM15 16 regs * 16 B = 256 B
----------------------------------------------------------------------
YMMHIR Upper 128 bits of YMM0-YMM15 16 regs * 16 B = 256 B
----------------------------------------------------------------------
EGPR R16-R31 16 regs * 8 B = 128 B
----------------------------------------------------------------------
OPMASKR K0-K7 8 regs * 8 B = 64 B
----------------------------------------------------------------------
ZMMHIR Upper 256 bits of ZMM0-ZMM15 16 regs * 32 B = 512 B
----------------------------------------------------------------------
Hi16ZMMR ZMM16-ZMM31 16 regs * 64 B = 1024 B
----------------------------------------------------------------------
Memory space in the output buffer is allocated for these sub-groups as
long as the corresponding Format.XER[55:49] bits in the PEBS record
header are set. However, the arch-PEBS hardware engine does not write
the sub-group if it is not used (in INIT state). In such cases, the
corresponding bit in the XSTATE_BV bitmap is set to 0. Therefore, the
XSTATE_BV field is checked to determine if the register data is actually
written for each PEBS record. If not, the register data is not outputted
to userspace.
The SSP register is sampled and placed into the GPRs group by arch-PEBS.
Additionally, the MSRs IA32_PMC_{GPn|FXm}_CFG_C.[55:49] bits are used to
manage which types of these registers need to be sampled.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
arch/x86/events/intel/core.c | 75 ++++++++++++++++++++++--------
arch/x86/events/intel/ds.c | 77 ++++++++++++++++++++++++++++---
arch/x86/include/asm/msr-index.h | 7 +++
arch/x86/include/asm/perf_event.h | 8 +++-
4 files changed, 142 insertions(+), 25 deletions(-)
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 1f063a1418fb..c57a70798364 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3221,6 +3221,21 @@ static void intel_pmu_enable_event_ext(struct perf_event *event)
if (pebs_data_cfg & PEBS_DATACFG_XMMS)
ext |= ARCH_PEBS_VECR_XMM & cap.caps;
+ if (pebs_data_cfg & PEBS_DATACFG_YMMHS)
+ ext |= ARCH_PEBS_VECR_YMMH & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_EGPRS)
+ ext |= ARCH_PEBS_VECR_EGPRS & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_OPMASKS)
+ ext |= ARCH_PEBS_VECR_OPMASK & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_ZMMHS)
+ ext |= ARCH_PEBS_VECR_ZMMH & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_H16ZMMS)
+ ext |= ARCH_PEBS_VECR_H16ZMM & cap.caps;
+
if (pebs_data_cfg & PEBS_DATACFG_LBRS)
ext |= ARCH_PEBS_LBR & cap.caps;
@@ -4418,6 +4433,34 @@ static void intel_pebs_aliases_skl(struct perf_event *event)
return intel_pebs_aliases_precdist(event);
}
+static inline bool intel_pebs_support_regs(struct perf_event *event, u64 regs)
+{
+ struct arch_pebs_cap cap = hybrid(event->pmu, arch_pebs_cap);
+ int pebs_format = x86_pmu.intel_cap.pebs_format;
+ bool supported = true;
+
+ /* SSP */
+ if (regs & PEBS_DATACFG_GP)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_GPR & cap.caps);
+ if (regs & PEBS_DATACFG_XMMS) {
+ supported &= x86_pmu.arch_pebs ?
+ ARCH_PEBS_VECR_XMM & cap.caps :
+ pebs_format > 3 && x86_pmu.intel_cap.pebs_baseline;
+ }
+ if (regs & PEBS_DATACFG_YMMHS)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_VECR_YMMH & cap.caps);
+ if (regs & PEBS_DATACFG_EGPRS)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_VECR_EGPRS & cap.caps);
+ if (regs & PEBS_DATACFG_OPMASKS)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_VECR_OPMASK & cap.caps);
+ if (regs & PEBS_DATACFG_ZMMHS)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_VECR_ZMMH & cap.caps);
+ if (regs & PEBS_DATACFG_H16ZMMS)
+ supported &= x86_pmu.arch_pebs && (ARCH_PEBS_VECR_H16ZMM & cap.caps);
+
+ return supported;
+}
+
static unsigned long intel_pmu_large_pebs_flags(struct perf_event *event)
{
unsigned long flags = x86_pmu.large_pebs_flags;
@@ -4427,24 +4470,20 @@ static unsigned long intel_pmu_large_pebs_flags(struct perf_event *event)
if (!event->attr.exclude_kernel)
flags &= ~PERF_SAMPLE_REGS_USER;
if (event->attr.sample_simd_regs_enabled) {
- u64 nolarge = PERF_X86_EGPRS_MASK | BIT_ULL(PERF_REG_X86_SSP);
-
- /*
- * PEBS HW can only collect the XMM0-XMM15 for now.
- * Disable large PEBS for other vector registers, predicate
- * registers, eGPRs, and SSP.
- */
- if (event->attr.sample_regs_user & nolarge ||
- fls64(event->attr.sample_simd_vec_reg_user) > PERF_X86_H16ZMM_BASE ||
- event->attr.sample_simd_pred_reg_user)
- flags &= ~PERF_SAMPLE_REGS_USER;
-
- if (event->attr.sample_regs_intr & nolarge ||
- fls64(event->attr.sample_simd_vec_reg_intr) > PERF_X86_H16ZMM_BASE ||
- event->attr.sample_simd_pred_reg_intr)
- flags &= ~PERF_SAMPLE_REGS_INTR;
-
- if (event->attr.sample_simd_vec_reg_qwords > PERF_X86_XMM_QWORDS)
+ if ((event_needs_ssp(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_GP)) ||
+ (event_needs_xmm(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_XMMS)) ||
+ (event_needs_ymm(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_YMMHS)) ||
+ (event_needs_egprs(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_EGPRS)) ||
+ (event_needs_opmask(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_OPMASKS)) ||
+ (event_needs_low16_zmm(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_ZMMHS)) ||
+ (event_needs_high16_zmm(event) &&
+ !intel_pebs_support_regs(event, PEBS_DATACFG_H16ZMMS)))
flags &= ~(PERF_SAMPLE_REGS_USER | PERF_SAMPLE_REGS_INTR);
} else {
if (event->attr.sample_regs_user & ~PEBS_GP_REGS)
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index ff8707885f74..2851622fbf0f 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -1732,11 +1732,22 @@ static u64 pebs_update_adaptive_cfg(struct perf_event *event)
((attr->config & INTEL_ARCH_EVENT_MASK) ==
x86_pmu.rtm_abort_event);
- if (gprs || (attr->precise_ip < 2) || tsx_weight)
+ if (gprs || (attr->precise_ip < 2) ||
+ tsx_weight || event_needs_ssp(event))
pebs_data_cfg |= PEBS_DATACFG_GP;
if (event_needs_xmm(event))
pebs_data_cfg |= PEBS_DATACFG_XMMS;
+ if (event_needs_ymm(event))
+ pebs_data_cfg |= PEBS_DATACFG_YMMHS;
+ if (event_needs_low16_zmm(event))
+ pebs_data_cfg |= PEBS_DATACFG_ZMMHS;
+ if (event_needs_high16_zmm(event))
+ pebs_data_cfg |= PEBS_DATACFG_H16ZMMS;
+ if (event_needs_opmask(event))
+ pebs_data_cfg |= PEBS_DATACFG_OPMASKS;
+ if (event_needs_egprs(event))
+ pebs_data_cfg |= PEBS_DATACFG_EGPRS;
if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
/*
@@ -2699,15 +2710,69 @@ static void setup_arch_pebs_sample_data(struct perf_event *event,
meminfo->tsx_tuning, ax);
}
- if (header->xmm) {
+ if (header->xmm || header->ymmh || header->egpr ||
+ header->opmask || header->zmmh || header->h16zmm) {
+ struct arch_pebs_xer_header *xer_header = next_record;
struct pebs_xmm *xmm;
+ struct ymmh_struct *ymmh;
+ struct avx_512_zmm_uppers_state *zmmh;
+ struct avx_512_hi16_state *h16zmm;
+ struct avx_512_opmask_state *opmask;
+ struct apx_state *egpr;
next_record += sizeof(struct arch_pebs_xer_header);
- ignore_mask |= XFEATURE_MASK_SSE;
- xmm = next_record;
- perf_regs->xmm_regs = xmm->xmm;
- next_record = xmm + 1;
+ if (header->xmm) {
+ ignore_mask |= XFEATURE_MASK_SSE;
+ xmm = next_record;
+ /*
+ * Only output XMM regs to user space when arch-PEBS
+ * really writes data into xstate area.
+ */
+ if (xer_header->xstate & XFEATURE_MASK_SSE)
+ perf_regs->xmm_regs = xmm->xmm;
+ next_record = xmm + 1;
+ }
+
+ if (header->ymmh) {
+ ignore_mask |= XFEATURE_MASK_YMM;
+ ymmh = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_YMM)
+ perf_regs->ymmh = ymmh;
+ next_record = ymmh + 1;
+ }
+
+ if (header->egpr) {
+ ignore_mask |= XFEATURE_MASK_APX;
+ egpr = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_APX)
+ perf_regs->egpr = egpr;
+ next_record = egpr + 1;
+ }
+
+ if (header->opmask) {
+ ignore_mask |= XFEATURE_MASK_OPMASK;
+ opmask = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_OPMASK)
+ perf_regs->opmask = opmask;
+ next_record = opmask + 1;
+ }
+
+ if (header->zmmh) {
+ ignore_mask |= XFEATURE_MASK_ZMM_Hi256;
+ zmmh = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_ZMM_Hi256)
+ perf_regs->zmmh = zmmh;
+ next_record = zmmh + 1;
+ }
+
+ if (header->h16zmm) {
+ ignore_mask |= XFEATURE_MASK_Hi16_ZMM;
+ h16zmm = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_Hi16_ZMM)
+ perf_regs->h16zmm = h16zmm;
+ next_record = h16zmm + 1;
+ }
}
if (header->lbr) {
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 6d1b69ea01c2..6c915781fdd3 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -350,6 +350,13 @@
#define ARCH_PEBS_LBR_SHIFT 40
#define ARCH_PEBS_LBR (0x3ull << ARCH_PEBS_LBR_SHIFT)
#define ARCH_PEBS_VECR_XMM BIT_ULL(49)
+#define ARCH_PEBS_VECR_YMMH BIT_ULL(50)
+#define ARCH_PEBS_VECR_EGPRS BIT_ULL(51)
+#define ARCH_PEBS_VECR_OPMASK BIT_ULL(53)
+#define ARCH_PEBS_VECR_ZMMH BIT_ULL(54)
+#define ARCH_PEBS_VECR_H16ZMM BIT_ULL(55)
+#define ARCH_PEBS_VECR_EXT_SHIFT 50
+#define ARCH_PEBS_VECR_EXT (0x3full << ARCH_PEBS_VECR_EXT_SHIFT)
#define ARCH_PEBS_GPR BIT_ULL(61)
#define ARCH_PEBS_AUX BIT_ULL(62)
#define ARCH_PEBS_EN BIT_ULL(63)
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 98fef9db0aa3..3665a0a2148e 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -148,6 +148,11 @@
#define PEBS_DATACFG_LBRS BIT_ULL(3)
#define PEBS_DATACFG_CNTR BIT_ULL(4)
#define PEBS_DATACFG_METRICS BIT_ULL(5)
+#define PEBS_DATACFG_YMMHS BIT_ULL(6)
+#define PEBS_DATACFG_OPMASKS BIT_ULL(7)
+#define PEBS_DATACFG_ZMMHS BIT_ULL(8)
+#define PEBS_DATACFG_H16ZMMS BIT_ULL(9)
+#define PEBS_DATACFG_EGPRS BIT_ULL(10)
#define PEBS_DATACFG_LBR_SHIFT 24
#define PEBS_DATACFG_CNTR_SHIFT 32
#define PEBS_DATACFG_CNTR_MASK GENMASK_ULL(15, 0)
@@ -545,7 +550,8 @@ struct arch_pebs_header {
rsvd3:7,
xmm:1,
ymmh:1,
- rsvd4:2,
+ egpr:1,
+ rsvd4:1,
opmask:1,
zmmh:1,
h16zmm:1,
--
2.34.1
next prev parent reply other threads:[~2026-02-09 7:26 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-09 7:20 [Patch v6 00/22] Support SIMD/eGPRs/SSP registers sampling for perf Dapeng Mi
2026-02-09 7:20 ` [Patch v6 01/22] perf/x86/intel: Restrict PEBS_ENABLE writes to PEBS-capable counters Dapeng Mi
2026-02-10 15:36 ` Peter Zijlstra
2026-02-11 5:47 ` Mi, Dapeng
2026-02-09 7:20 ` [Patch v6 02/22] perf/x86/intel: Enable large PEBS sampling for XMMs Dapeng Mi
2026-02-09 7:20 ` [Patch v6 03/22] perf/x86/intel: Convert x86_perf_regs to per-cpu variables Dapeng Mi
2026-02-09 7:20 ` [Patch v6 04/22] perf: Eliminate duplicate arch-specific functions definations Dapeng Mi
2026-02-09 7:20 ` [Patch v6 05/22] perf/x86: Use x86_perf_regs in the x86 nmi handler Dapeng Mi
2026-02-10 18:40 ` Peter Zijlstra
2026-02-11 6:26 ` Mi, Dapeng
2026-02-09 7:20 ` [Patch v6 06/22] perf/x86: Introduce x86-specific x86_pmu_setup_regs_data() Dapeng Mi
2026-02-09 7:20 ` [Patch v6 07/22] x86/fpu/xstate: Add xsaves_nmi() helper Dapeng Mi
2026-02-09 7:20 ` [Patch v6 08/22] x86/fpu: Ensure TIF_NEED_FPU_LOAD is set after saving FPU state Dapeng Mi
2026-02-11 19:39 ` Chang S. Bae
2026-02-11 19:55 ` Dave Hansen
2026-02-24 6:50 ` Mi, Dapeng
2026-02-25 13:02 ` Peter Zijlstra
2026-02-24 5:35 ` Mi, Dapeng
2026-02-24 19:13 ` Chang S. Bae
2026-02-25 0:35 ` Mi, Dapeng
2026-02-09 7:20 ` [Patch v6 09/22] perf: Move and rename has_extended_regs() for ARCH-specific use Dapeng Mi
2026-02-09 7:20 ` [Patch v6 10/22] perf/x86: Enable XMM Register Sampling for Non-PEBS Events Dapeng Mi
2026-02-15 23:58 ` Chang S. Bae
2026-02-24 7:11 ` Mi, Dapeng
2026-02-24 19:13 ` Chang S. Bae
2026-02-25 0:55 ` Mi, Dapeng
2026-02-25 1:11 ` Chang S. Bae
2026-02-25 1:36 ` Mi, Dapeng
2026-02-25 3:14 ` Chang S. Bae
2026-02-25 6:13 ` Mi, Dapeng
2026-02-09 7:20 ` [Patch v6 11/22] perf/x86: Enable XMM register sampling for REGS_USER case Dapeng Mi
2026-02-09 7:20 ` [Patch v6 12/22] perf: Add sampling support for SIMD registers Dapeng Mi
2026-02-10 20:04 ` Peter Zijlstra
2026-02-11 6:56 ` Mi, Dapeng
2026-02-09 7:20 ` [Patch v6 13/22] perf/x86: Enable XMM sampling using sample_simd_vec_reg_* fields Dapeng Mi
2026-02-09 7:20 ` [Patch v6 14/22] perf/x86: Enable YMM " Dapeng Mi
2026-02-09 7:20 ` [Patch v6 15/22] perf/x86: Enable ZMM " Dapeng Mi
2026-02-09 7:20 ` [Patch v6 16/22] perf/x86: Enable OPMASK sampling using sample_simd_pred_reg_* fields Dapeng Mi
2026-02-09 7:20 ` [Patch v6 17/22] perf: Enhance perf_reg_validate() with simd_enabled argument Dapeng Mi
2026-02-09 7:20 ` [Patch v6 18/22] perf/x86: Enable eGPRs sampling using sample_regs_* fields Dapeng Mi
2026-02-09 7:20 ` [Patch v6 19/22] perf/x86: Enable SSP " Dapeng Mi
2026-02-09 7:20 ` [Patch v6 20/22] perf/x86/intel: Enable PERF_PMU_CAP_SIMD_REGS capability Dapeng Mi
2026-02-09 7:20 ` Dapeng Mi [this message]
2026-02-09 7:20 ` [Patch v6 22/22] perf/x86: Activate back-to-back NMI detection for arch-PEBS induced NMIs Dapeng Mi
2026-02-09 8:48 ` [Patch v6 00/22] Support SIMD/eGPRs/SSP registers sampling for perf Mi, Dapeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260209072047.2180332-22-dapeng1.mi@linux.intel.com \
--to=dapeng1.mi@linux.intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=broonie@kernel.org \
--cc=dapeng1.mi@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=tglx@linutronix.de \
--cc=thomas.falcon@intel.com \
--cc=xudong.hao@intel.com \
--cc=zide.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox