From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94B8B3148BA; Wed, 15 Oct 2025 06:46:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.16 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760510793; cv=none; b=rleFlu4XdDGcIPFql6nQZfACv36OHC9eZi8kMkjql6zD2qNyKDhzDnmZytdSwJL+vh/VJH04fB3jP5DN6qf4/FXST4zofHgR+5nSYdz3Ov/e+p5GYWAZtlUXgGozTP4G8k4cBTlZMYj5l3rlGFyMK6HPAtRftdDIMnbRxR7vi8Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760510793; c=relaxed/simple; bh=lJgW3ny7iswoX3eKLhyR7tC3EaDfWjixdNifBZ0+mAM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GdVjTi8QkOw+GBtzZv8n0U+k8dKUUIpwnUkpG+ULrhF8fZG78u7ZB+RmWETLX5zbTjK+wi8/0JzL1qqoy+VUWZuJeviqCHuaZ8SC+EfpItnLYvTe8t7/wQCUrgM5dBiIw5M9ZfZ+M6cxj2Zd9r3UgQ4/WuRclZSAW0n0q6CX41k= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AdBdrh6a; arc=none smtp.client-ip=192.198.163.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AdBdrh6a" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1760510792; x=1792046792; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lJgW3ny7iswoX3eKLhyR7tC3EaDfWjixdNifBZ0+mAM=; b=AdBdrh6agBaHIzrPf6pyEOSiqvt+YouHyt46BWBetsuCvQUnWy5qiOVs uhCTTj6HDg45rAlH4VzaYuZDKgAqSq+fNZ6Ccyf6ZO9tua0BACmNvdhne gJsgcV16ZMBMbXqiByI+NqcYX4ode9mk/ZPM/tkjn/AQRVLags7164kh5 R+Jc8j0WfXV2BLMKaOOy1ILSshXffs/gcFc5V8JTIEanBMPvoT4nKSUPT yGbGvf1XilxxCbwEZKqPzQRchfkuo4HCvjz9qW1ZtPrnTLOboArZECsfV NJHU8LzxmCbNwDHI3dYOBXRfLh6mKJLoVaGJ9zKoPCBt4NuAfPBSueWqU w==; X-CSE-ConnectionGUID: oQd/at9rR6eXjBX7DOIPHQ== X-CSE-MsgGUID: JeiZ11ylQm+NpPROqDIL8Q== X-IronPort-AV: E=McAfee;i="6800,10657,11582"; a="50242484" X-IronPort-AV: E=Sophos;i="6.19,230,1754982000"; d="scan'208";a="50242484" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2025 23:46:31 -0700 X-CSE-ConnectionGUID: vwIOVjLOSTuFBgTAOijrJQ== X-CSE-MsgGUID: TGUS5Pu0Tzisp28YL8BSiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.19,230,1754982000"; d="scan'208";a="182029927" Received: from spr.sh.intel.com ([10.112.229.196]) by fmviesa006.fm.intel.com with ESMTP; 14 Oct 2025 23:46:27 -0700 From: Dapeng Mi To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Ian Rogers , Adrian Hunter , Alexander Shishkin , Andi Kleen , Eranian Stephane Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Dapeng Mi , Dapeng Mi , Kan Liang Subject: [Patch v8 06/12] perf/x86/intel/ds: Factor out PEBS record processing code to functions Date: Wed, 15 Oct 2025 14:44:16 +0800 Message-Id: <20251015064422.47437-7-dapeng1.mi@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20251015064422.47437-1-dapeng1.mi@linux.intel.com> References: <20251015064422.47437-1-dapeng1.mi@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Beside some PEBS record layout difference, arch-PEBS can share most of PEBS record processing code with adaptive PEBS. Thus, factor out these common processing code to independent inline functions, so they can be reused by subsequent arch-PEBS handler. Suggested-by: Kan Liang Signed-off-by: Dapeng Mi --- arch/x86/events/intel/ds.c | 101 ++++++++++++++++++++++++------------- 1 file changed, 66 insertions(+), 35 deletions(-) diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index a80881a20321..41acbf0a11c9 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -2629,6 +2629,64 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_d } } +static inline void __intel_pmu_handle_pebs_record(struct pt_regs *iregs, + struct pt_regs *regs, + struct perf_sample_data *data, + void *at, u64 pebs_status, + struct perf_event *events[], + short *counts, void **last, + setup_fn setup_sample) +{ + struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); + struct perf_event *event; + int bit; + + for_each_set_bit(bit, (unsigned long *)&pebs_status, X86_PMC_IDX_MAX) { + event = cpuc->events[bit]; + + if (WARN_ON_ONCE(!event) || + WARN_ON_ONCE(!event->attr.precise_ip)) + continue; + + if (counts[bit]++) + __intel_pmu_pebs_event(event, iregs, regs, data, + last[bit], setup_sample); + + last[bit] = at; + /* + * perf_event_overflow() called by below __intel_pmu_pebs_last_event() + * could trigger interrupt throttle and clear all event pointers of + * the group in cpuc->events[] to NULL. So snapshot the event[] before + * it could be cleared. This avoids the possible NULL event pointer + * access and PEBS record loss. + */ + if (counts[bit] && !events[bit]) + events[bit] = cpuc->events[bit]; + } +} + +static inline void +__intel_pmu_handle_last_pebs_record(struct pt_regs *iregs, struct pt_regs *regs, + struct perf_sample_data *data, u64 mask, + struct perf_event *events[], + short *counts, void **last, + setup_fn setup_sample) +{ + struct perf_event *event; + int bit; + + for_each_set_bit(bit, (unsigned long *)&mask, X86_PMC_IDX_MAX) { + if (!counts[bit]) + continue; + + event = events[bit]; + + __intel_pmu_pebs_last_event(event, iregs, regs, data, last[bit], + counts[bit], setup_sample); + } + +} + static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_data *data) { struct perf_event *events[INTEL_PMC_IDX_FIXED + MAX_FIXED_PEBS_EVENTS] = {NULL}; @@ -2639,9 +2697,7 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d struct x86_perf_regs perf_regs; struct pt_regs *regs = &perf_regs.regs; struct pebs_basic *basic; - struct perf_event *event; void *base, *at, *top; - int bit; u64 mask; if (!x86_pmu.pebs_active) @@ -2654,6 +2710,7 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d mask = hybrid(cpuc->pmu, pebs_events_mask) | (hybrid(cpuc->pmu, fixed_cntr_mask64) << INTEL_PMC_IDX_FIXED); + mask &= cpuc->pebs_enabled; if (unlikely(base >= top)) { intel_pmu_pebs_event_update_no_drain(cpuc, mask); @@ -2671,41 +2728,15 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d if (basic->format_size != cpuc->pebs_record_size) continue; - pebs_status = basic->applicable_counters & cpuc->pebs_enabled & mask; - for_each_set_bit(bit, (unsigned long *)&pebs_status, X86_PMC_IDX_MAX) { - event = cpuc->events[bit]; - - if (WARN_ON_ONCE(!event) || - WARN_ON_ONCE(!event->attr.precise_ip)) - continue; - - if (counts[bit]++) { - __intel_pmu_pebs_event(event, iregs, regs, data, last[bit], - setup_pebs_adaptive_sample_data); - } - last[bit] = at; - - /* - * perf_event_overflow() called by below __intel_pmu_pebs_last_event() - * could trigger interrupt throttle and clear all event pointers of - * the group in cpuc->events[] to NULL. So snapshot the event[] before - * it could be cleared. This avoids the possible NULL event pointer - * access and PEBS record loss. - */ - if (counts[bit] && !events[bit]) - events[bit] = cpuc->events[bit]; - } + pebs_status = mask & basic->applicable_counters; + __intel_pmu_handle_pebs_record(iregs, regs, data, at, + pebs_status, events, counts, last, + setup_pebs_adaptive_sample_data); } - for_each_set_bit(bit, (unsigned long *)&mask, X86_PMC_IDX_MAX) { - if (!counts[bit]) - continue; - - event = events[bit]; - - __intel_pmu_pebs_last_event(event, iregs, regs, data, last[bit], - counts[bit], setup_pebs_adaptive_sample_data); - } + __intel_pmu_handle_last_pebs_record(iregs, regs, data, mask, events, + counts, last, + setup_pebs_adaptive_sample_data); } static void __init intel_arch_pebs_init(void) -- 2.34.1