From: Dapeng Mi <dapeng1.mi@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Jiri Olsa <jolsa@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <ak@linux.intel.com>,
Eranian Stephane <eranian@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>,
broonie@kernel.org, Ravi Bangoria <ravi.bangoria@amd.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
Zide Chen <zide.chen@intel.com>,
Falcon Thomas <thomas.falcon@intel.com>,
Dapeng Mi <dapeng1.mi@intel.com>,
Xudong Hao <xudong.hao@intel.com>,
Dapeng Mi <dapeng1.mi@linux.intel.com>
Subject: [Patch v10 20/23] perf/x86/intel: Support arch-PEBS based SIMD/eGPRs sampling
Date: Tue, 21 Jul 2026 14:25:03 +0800 [thread overview]
Message-ID: <20260721062506.3745816-21-dapeng1.mi@linux.intel.com> (raw)
In-Reply-To: <20260721062506.3745816-1-dapeng1.mi@linux.intel.com>
Support arch-PEBS based SIMD/eGPRs/SSP registers sampling.
Arch-PEBS supports sampling all of these registers. SIMD and eGPR
registers are placed into the XSAVE-Enabled Registers (XER) group with
the layout described below.
Field Name Registers Used Size
XSTATE_BV XINUSE for groups 8 B
Reserved Reserved 8 B
SSER XMM0-XMM15 16 regs * 16 B = 256 B
YMMHIR Upper 128 bits of YMM0-YMM15 16 regs * 16 B = 256 B
EGPR R16-R31 16 regs * 8 B = 128 B
OPMASKR K0-K7 8 regs * 8 B = 64 B
ZMMHIR Upper 256 bits of ZMM0-ZMM15 16 regs * 32 B = 512 B
Hi16ZMMR ZMM16-ZMM31 16 regs * 64 B = 1024 B
Memory space in the output buffer is allocated for these sub-groups as
long as the corresponding Format.XER[55:49] bits in the PEBS record
header are set. However, the arch-PEBS hardware engine does not write
the sub-group if it is not used (in INIT state). In such cases, the
corresponding bit in the XSTATE_BV bitmap is set to 0. Therefore, the
XSTATE_BV field is checked to determine if the register data is actually
written for each PEBS record. If not, the register data is not outputted
to userspace.
Additionally, the MSRs IA32_PMC_{GPn|FXm}_CFG_C.[55:49] bits are used to
manage which types of these registers need to be sampled.
Arch-PEBS based SIMD/eGPRs sampling will be enabled in a subsequent
patch that sets PERF_PMU_CAP_SIMD_REGS.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
arch/x86/events/core.c | 20 ++++++---
arch/x86/events/intel/core.c | 15 +++++++
arch/x86/events/intel/ds.c | 67 +++++++++++++++++++++++++++++--
arch/x86/include/asm/msr-index.h | 7 ++++
arch/x86/include/asm/perf_event.h | 8 +++-
5 files changed, 106 insertions(+), 11 deletions(-)
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index caa5a0fb3ed8..11799acd8c54 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -646,12 +646,20 @@ static int pebs_simd_regs_validate(struct perf_event *event)
if (event_needs_ssp(event) &&
!(x86_pmu.arch_pebs && (caps & ARCH_PEBS_GPR)))
return -EINVAL;
- /* PEBS does not support YMM/ZMM/OPMASK/eGPR registers sampling yet. */
- if (event_needs_ymm(event) ||
- event_needs_low16_zmm(event) ||
- event_needs_high16_zmm(event) ||
- event_needs_opmask(event) ||
- event_needs_egprs(event))
+ if (event_needs_ymm(event) &&
+ !(x86_pmu.arch_pebs && (caps & ARCH_PEBS_VECR_YMMH)))
+ return -EINVAL;
+ if (event_needs_egprs(event) &&
+ !(x86_pmu.arch_pebs && (caps & ARCH_PEBS_VECR_EGPRS)))
+ return -EINVAL;
+ if (event_needs_opmask(event) &&
+ !(x86_pmu.arch_pebs && (caps & ARCH_PEBS_VECR_OPMASK)))
+ return -EINVAL;
+ if (event_needs_low16_zmm(event) &&
+ !(x86_pmu.arch_pebs && (caps & ARCH_PEBS_VECR_ZMMH)))
+ return -EINVAL;
+ if (event_needs_high16_zmm(event) &&
+ !(x86_pmu.arch_pebs && (caps & ARCH_PEBS_VECR_H16ZMM)))
return -EINVAL;
return 0;
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 809ef2515b10..4eeb08352f4c 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3498,6 +3498,21 @@ static void intel_pmu_enable_event_ext(struct perf_event *event)
if (pebs_data_cfg & PEBS_DATACFG_XMMS)
ext |= ARCH_PEBS_VECR_XMM & cap.caps;
+ if (pebs_data_cfg & PEBS_DATACFG_YMMHS)
+ ext |= ARCH_PEBS_VECR_YMMH & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_EGPRS)
+ ext |= ARCH_PEBS_VECR_EGPRS & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_OPMASKS)
+ ext |= ARCH_PEBS_VECR_OPMASK & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_ZMMHS)
+ ext |= ARCH_PEBS_VECR_ZMMH & cap.caps;
+
+ if (pebs_data_cfg & PEBS_DATACFG_H16ZMMS)
+ ext |= ARCH_PEBS_VECR_H16ZMM & cap.caps;
+
if (pebs_data_cfg & PEBS_DATACFG_LBRS)
ext |= ARCH_PEBS_LBR & cap.caps;
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 3277bd72e0a8..ff6125ec8458 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -1733,6 +1733,16 @@ static u64 pebs_update_adaptive_cfg(struct perf_event *event)
if (sample_type & (PERF_SAMPLE_REGS_INTR | PERF_SAMPLE_REGS_USER)) {
if (event_needs_xmm(event))
pebs_data_cfg |= PEBS_DATACFG_XMMS;
+ if (x86_pmu.arch_pebs && event_needs_ymm(event))
+ pebs_data_cfg |= PEBS_DATACFG_YMMHS;
+ if (x86_pmu.arch_pebs && event_needs_low16_zmm(event))
+ pebs_data_cfg |= PEBS_DATACFG_ZMMHS;
+ if (x86_pmu.arch_pebs && event_needs_high16_zmm(event))
+ pebs_data_cfg |= PEBS_DATACFG_H16ZMMS;
+ if (x86_pmu.arch_pebs && event_needs_opmask(event))
+ pebs_data_cfg |= PEBS_DATACFG_OPMASKS;
+ if (x86_pmu.arch_pebs && event_needs_egprs(event))
+ pebs_data_cfg |= PEBS_DATACFG_EGPRS;
}
if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
@@ -2687,14 +2697,63 @@ static void setup_arch_pebs_sample_data(struct perf_event *event,
meminfo->tsx_tuning, ax);
}
- if (header->xmm) {
+ if (header->xmm || header->ymmh || header->egpr ||
+ header->opmask || header->zmmh || header->h16zmm) {
+ struct arch_pebs_xer_header *xer_header = next_record;
struct pebs_xmm *xmm;
+ struct ymmh_struct *ymmh;
+ struct avx_512_zmm_uppers_state *zmmh;
+ struct avx_512_hi16_state *h16zmm;
+ struct avx_512_opmask_state *opmask;
+ struct apx_state *egpr;
next_record += sizeof(struct arch_pebs_xer_header);
- xmm = next_record;
- perf_regs->xmm_regs = xmm->xmm;
- next_record = xmm + 1;
+ if (header->xmm) {
+ xmm = next_record;
+ /*
+ * Only output XMM regs to user space when arch-PEBS
+ * really writes data into xstate area.
+ */
+ if (xer_header->xstate & XFEATURE_MASK_SSE)
+ perf_regs->xmm_regs = xmm->xmm;
+ next_record = xmm + 1;
+ }
+
+ if (header->ymmh) {
+ ymmh = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_YMM)
+ perf_regs->ymmh = ymmh;
+ next_record = ymmh + 1;
+ }
+
+ if (header->egpr) {
+ egpr = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_APX)
+ perf_regs->egpr = egpr;
+ next_record = egpr + 1;
+ }
+
+ if (header->opmask) {
+ opmask = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_OPMASK)
+ perf_regs->opmask = opmask;
+ next_record = opmask + 1;
+ }
+
+ if (header->zmmh) {
+ zmmh = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_ZMM_Hi256)
+ perf_regs->zmmh = zmmh;
+ next_record = zmmh + 1;
+ }
+
+ if (header->h16zmm) {
+ h16zmm = next_record;
+ if (xer_header->xstate & XFEATURE_MASK_Hi16_ZMM)
+ perf_regs->h16zmm = h16zmm;
+ next_record = h16zmm + 1;
+ }
}
if (header->lbr) {
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 18c4be75e927..9c26d4075c5f 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -350,6 +350,13 @@
#define ARCH_PEBS_LBR_SHIFT 40
#define ARCH_PEBS_LBR (0x3ull << ARCH_PEBS_LBR_SHIFT)
#define ARCH_PEBS_VECR_XMM BIT_ULL(49)
+#define ARCH_PEBS_VECR_YMMH BIT_ULL(50)
+#define ARCH_PEBS_VECR_EGPRS BIT_ULL(51)
+#define ARCH_PEBS_VECR_OPMASK BIT_ULL(53)
+#define ARCH_PEBS_VECR_ZMMH BIT_ULL(54)
+#define ARCH_PEBS_VECR_H16ZMM BIT_ULL(55)
+#define ARCH_PEBS_VECR_EXT_SHIFT 49
+#define ARCH_PEBS_VECR_EXT (0x7full << ARCH_PEBS_VECR_EXT_SHIFT)
#define ARCH_PEBS_GPR BIT_ULL(61)
#define ARCH_PEBS_AUX BIT_ULL(62)
#define ARCH_PEBS_EN BIT_ULL(63)
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 4302ef39c42e..12f7db8c57b4 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -150,6 +150,11 @@
#define PEBS_DATACFG_LBRS BIT_ULL(3)
#define PEBS_DATACFG_CNTR BIT_ULL(4)
#define PEBS_DATACFG_METRICS BIT_ULL(5)
+#define PEBS_DATACFG_YMMHS BIT_ULL(6)
+#define PEBS_DATACFG_OPMASKS BIT_ULL(7)
+#define PEBS_DATACFG_ZMMHS BIT_ULL(8)
+#define PEBS_DATACFG_H16ZMMS BIT_ULL(9)
+#define PEBS_DATACFG_EGPRS BIT_ULL(10)
#define PEBS_DATACFG_LBR_SHIFT 24
#define PEBS_DATACFG_CNTR_SHIFT 32
#define PEBS_DATACFG_CNTR_MASK GENMASK_ULL(15, 0)
@@ -547,7 +552,8 @@ struct arch_pebs_header {
rsvd3:7,
xmm:1,
ymmh:1,
- rsvd4:2,
+ egpr:1,
+ rsvd4:1,
opmask:1,
zmmh:1,
h16zmm:1,
--
2.34.1
next prev parent reply other threads:[~2026-07-21 6:34 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-21 6:24 [Patch v10 00/23] Support SIMD/eGPRs/SSP registers sampling for perf Dapeng Mi
2026-07-21 6:24 ` [Patch v10 01/23] perf/x86: Move hybrid PMU initialization before x86_pmu_starting_cpu() Dapeng Mi
2026-07-21 6:24 ` [Patch v10 02/23] perf/x86/intel: Enable large PEBS sampling for XMMs Dapeng Mi
2026-07-21 6:24 ` [Patch v10 03/23] perf/x86/intel: Convert x86_perf_regs to per-cpu variables Dapeng Mi
2026-07-21 6:24 ` [Patch v10 04/23] perf: Eliminate duplicate arch-specific function definitions Dapeng Mi
2026-07-21 6:24 ` [Patch v10 05/23] perf/x86: Use x86_perf_regs in NMI handlers Dapeng Mi
2026-07-21 6:24 ` [Patch v10 06/23] x86/fpu/xstate: Add xsaves_nmi() helper Dapeng Mi
2026-07-21 6:24 ` [Patch v10 07/23] x86/fpu: Add update_fpu_state_and_flag() helper Dapeng Mi
2026-07-21 6:24 ` [Patch v10 08/23] perf: Move and enhance has_extended_regs() for arch-specific use Dapeng Mi
2026-07-21 6:24 ` [Patch v10 09/23] perf/x86/intel: Centralize PERF_PMU_CAP_EXTENDED_REGS updates Dapeng Mi
2026-07-21 6:24 ` [Patch v10 10/23] perf/x86: Enable XMM register sampling for non-PEBS events Dapeng Mi
2026-07-21 6:24 ` [Patch v10 11/23] perf/x86: Enable XMM register sampling for REGS_USER case Dapeng Mi
2026-07-21 6:24 ` [Patch v10 12/23] perf: Add sampling support for SIMD registers Dapeng Mi
2026-07-21 6:24 ` [Patch v10 13/23] perf/x86: Support XMM sampling using sample_simd_vec_reg_* fields Dapeng Mi
2026-07-21 6:24 ` [Patch v10 14/23] perf/x86: Support YMM " Dapeng Mi
2026-07-21 6:24 ` [Patch v10 15/23] perf/x86: Support ZMM " Dapeng Mi
2026-07-21 6:24 ` [Patch v10 16/23] perf/x86: Support OPMASK sampling using sample_simd_pred_reg_* fields Dapeng Mi
2026-07-21 6:25 ` [Patch v10 17/23] perf: Enhance perf_reg_validate() with simd_enabled argument Dapeng Mi
2026-07-21 6:25 ` [Patch v10 18/23] perf/x86: Support eGPRs sampling using sample_regs_* fields Dapeng Mi
2026-07-21 6:25 ` [Patch v10 19/23] perf/x86: Support SSP " Dapeng Mi
2026-07-21 6:25 ` Dapeng Mi [this message]
2026-07-21 6:25 ` [Patch v10 21/23] perf/x86/intel: Advertise PERF_PMU_CAP_SIMD_REGS capability Dapeng Mi
2026-07-21 6:25 ` [Patch v10 22/23] perf/x86: Activate back-to-back NMI detection for arch-PEBS induced NMIs Dapeng Mi
2026-07-21 6:25 ` [Patch v10 23/23] perf/x86/intel: Add sanity check for PEBS record/fragment size Dapeng Mi
2026-07-21 17:42 ` [Patch v10 00/23] Support SIMD/eGPRs/SSP registers sampling for perf Ian Rogers
2026-07-22 0:17 ` Mi, Dapeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260721062506.3745816-21-dapeng1.mi@linux.intel.com \
--to=dapeng1.mi@linux.intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=broonie@kernel.org \
--cc=dapeng1.mi@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=tglx@linutronix.de \
--cc=thomas.falcon@intel.com \
--cc=xudong.hao@intel.com \
--cc=zide.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox