* [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver
@ 2026-08-11 8:38 Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw)
To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
Cc: Ian Rogers
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1948 ++++++++++++++++++++++++++++++++++++
3 files changed, 1959 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 10+ messages in thread* [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-11 8:38 ` Ian Rogers 2026-08-11 14:32 ` Uwe Kleine-König 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 10+ messages in thread From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw) To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel Cc: Ian Rogers Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1703 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1714 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..adb34aefb26e --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1703 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/mod_devicetable.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +#include <linux/version.h> + +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0) +static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer, + enum hrtimer_restart (*function)(struct hrtimer *), + clockid_t clock_id, + enum hrtimer_mode mode) +{ + hrtimer_init(timer, clock_id, mode); + timer->function = function; +} +#define hrtimer_setup rpi_axi_pmu_hrtimer_setup +#endif + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (config_to_monitor(config) == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + int bus, filter, bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus = pmu->monitor[mon].hw_events.monitored_bus[idx]; + filter = pmu->monitor[mon].hw_events.filter[idx]; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = true; +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + lockdep_assert_held(&pmu->lock); + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx]) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx); + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = true; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + if (!pmu->firmware) + return -EPROBE_DEFER; + } + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2, + &pmu->monitor[i].mailbox); + if (ret) { + dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret); + goto err_firmware_put; + } + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_firmware_put; + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_cpuhp; + } + return 0; + +err_cpuhp: + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + perf_pmu_unregister(&pmu->pmu); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + {}, +}; + +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 14:32 ` Uwe Kleine-König 0 siblings, 0 replies; 10+ messages in thread From: Uwe Kleine-König @ 2026-08-11 14:32 UTC (permalink / raw) To: Ian Rogers Cc: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel [-- Attachment #1: Type: text/plain, Size: 967 bytes --] Hello, On Tue, Aug 11, 2026 at 01:38:26AM -0700, Ian Rogers wrote: > +#include <linux/mod_devicetable.h> > [...] > +#include <linux/platform_device.h> Please rely on <linux/platform_device.h> to provide the definition of of_device_id and drop <linux/mod_devicetable.h>. The latter will go away soon. > +/* Devices matching this driver in Device Tree */ > +static const struct of_device_id rpi_axi_pmu_match[] = { > + { > + .compatible = "brcm,bcm2835-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + { > + .compatible = "brcm,bcm2711-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + {}, Please use { } as list terminator (that is, add a space and drop the comma). Also if the data is the same for all variants, maybe better drop the assigment to .data and hardcode the value accordingly. > +}; > + > +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); Usually there is no empty line between the device_id struct and the MODULE_DEVICE_TABLE macro. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 8:38 ` Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 0 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw) To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel Cc: Ian Rogers Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 4 +- drivers/perf/rpi_axi_pmu.c | 259 ++++++++++++++++++++++++++++++++++++- 2 files changed, 254 insertions(+), 9 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..b0c4eb0748dd 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU config RASPBERRYPI_AXI_PMU tristate "Raspberry Pi AXI PMU support" - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index adb34aefb26e..ec1327804c00 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -75,9 +77,11 @@ static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer, /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -129,6 +133,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -261,6 +303,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -483,16 +613,22 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (config >> 15 != 0) return false; - if (config_to_monitor(config) == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (config_to_monitor(config) == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -719,6 +855,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -875,12 +1049,79 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) */ + if (strstr(pmu_attr->event_str, "monitor=1")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "rp1_pcie")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1648,6 +1889,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, {}, }; -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers ` (2 more replies) 2 siblings, 3 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König This patch series adds an uncore Performance Monitoring Unit (PMU) driver for Broadcom AXI system and VideoCore VPU performance monitors found on Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Motivation & Background ------------------------ Currently, Linux lacks a standard perf-API compatible driver for the Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc kthreads, preventing integration with standard Linux perf tooling (`perf stat`, `perf list`, etc.). This driver implements standard `struct pmu` hardware uncore callbacks under `drivers/perf/`, exposing human-readable sysfs event aliases, unit scaling (`Bytes`), and bus filtering directly to user space. Key Architectural Improvements & Features ------------------------------------------ 1. Standard Linux Perf Integration: - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`. - Supports event sampling and hardware counter accumulation (`local64_add`), automatically managing 31-bit hardware counter wraparound across high-bandwidth interconnect transfers. 2. CPU Hotplug Support (`cpuhp`): - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`). - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online CPU core when a designated CPU goes offline, avoiding stale uncore state. 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture: - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns) directly mapped over ARM physical memory space (`MON__SYSTEM`). - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4), VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic sleeps or blocking in timer/interrupt context. 4. SoC Generation Support: - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4). - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1 Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3 interconnect monitoring. Hardware Validation ------------------- The driver has been validated on real hardware across multiple SoC generations: - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU firmware mailbox IPC performance counters. - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1 Southbridge transfers. Changes in v2 ------------- - Match Data Initialization: In rpi_axi_pmu__init(), set pmu->chip via of_device_get_match_data(dev) to ensure BCM2712 match data is correctly parsed. - Sysfs Event Visibility Filtering: Updated rpi_axi_pmu_events_is_visible() to properly filter all 7 BCM2712 master filtered event aliases (arm_, pcie0_, hvs_filtered, emmc0_) on BCM2835-BCM2711 systems, and hide legacy RPi 1-4 aliases on BCM2712 systems. - Unmapped Monitor Check: Added monitor base_address and initialization check to config_is_valid(). Events targeting uninitialized or unmapped monitors (e.g. missing memory resource or monitor=1 on BCM2712) are properly rejected with -EINVAL. - CPU Affinity Enforcement: Restrict uncore events in event_init() to event->cpu == pmu->cpu to prevent orphaned uncore events when a non-PMU CPU goes offline. - 31-bit Counter Wrap Delta: Mask counter accumulation with delta = (new_count - prev_count) & 0x7FFFFFFF in read(), timer_handler(), and vpu_work_handler() to prevent 2B+ event spikes on counter overflow. - Stopped Event Guard: Return early in rpi_axi_pmu_read() if event->hw.state & PERF_HES_STOPPED. - Lockless Race Fix: Pass bus and filter explicitly to rpi_axi_pmu_enable_bus_watcher() under spinlock to prevent lockless read races in vpu_work_handler(). - Slot Reuse Clean: Set enabled[idx] = false in rpi_axi_pmu_del() when refcount[idx] == 0 so re-allocated watcher slots properly enable hardware watchers. - Events Array Bounds: Return -ENOSPC in rpi_axi_pmu_add() if pmu->events array is full. - Timer Cancel Race Fix: Re-acquire pmu->lock after hrtimer_cancel() in rpi_axi_pmu_del() and re-verify active_events > 0 before returning. - Hotplug Timer Migration: Cancel and re-pin pmu->hrtimer to target CPU in rpi_axi_pmu_offline_cpu(). - Registration Order: Call perf_pmu_register() before cpuhp_state_add_instance() in probe, and cancel hrtimer before cancel_work_sync() and unregister in exit teardown. - Header & Style Cleanups: Removed <linux/mod_devicetable.h> and <linux/version.h>, updated rpi_axi_pmu_match[] terminator to { }, and fixed C variable declaration style. Ian Rogers (2): perf: Add Raspberry Pi BCM2835 AXI PMU driver perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2001 ++++++++++++++++++++++++++++++++++++ 3 files changed, 2012 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 0 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1739 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1750 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..cdf7d9a74f13 --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1739 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (mon >= MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = true; +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; + + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.enabled[idx] = false; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_PINNED); + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_PINNED); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = true; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + if (!pmu->firmware) + return -EPROBE_DEFER; + } + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2, + &pmu->monitor[i].mailbox); + if (ret) { + dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret); + goto err_firmware_put; + } + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_firmware_put; + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_pmu_unregister; + } + return 0; + +err_pmu_unregister: + perf_pmu_unregister(&pmu->pmu); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + perf_pmu_unregister(&pmu->pmu); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 0 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 4 +- drivers/perf/rpi_axi_pmu.c | 278 +++++++++++++++++++++++++++++++++++-- 2 files changed, 272 insertions(+), 10 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..b0c4eb0748dd 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU config RASPBERRYPI_AXI_PMU tristate "Raspberry Pi AXI PMU support" - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index cdf7d9a74f13..b13632fa8f84 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -60,9 +62,11 @@ /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -114,6 +118,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -246,6 +288,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) return false; - if (mon == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (mon != MON__SYSTEM) + return false; + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -867,12 +1043,94 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) + * and RPi 1-4 aliases + */ + if (strstr(pmu_attr->event_str, "monitor=1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1549,7 +1807,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd spin_lock_init(&pmu->lock); mutex_init(&pmu->vpu_mutex); - pmu->chip = CHIP_BCM2835; + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); pmu->pmu = (struct pmu) { .module = THIS_MODULE, @@ -1685,6 +1943,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2 siblings, 2 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will This patch series adds an uncore Performance Monitoring Unit (PMU) driver for Broadcom AXI system and VideoCore VPU performance monitors found on Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Motivation & Background ------------------------ Currently, Linux lacks a standard perf-API compatible driver for the Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc kthreads, preventing integration with standard Linux perf tooling (`perf stat`, `perf list`, etc.). This driver implements standard `struct pmu` hardware uncore callbacks under `drivers/perf/`, exposing human-readable sysfs event aliases, unit scaling (`Bytes`), and bus filtering directly to user space. Key Architectural Improvements & Features ------------------------------------------ 1. Standard Linux Perf Integration: - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`. - Supports event sampling and hardware counter accumulation (`local64_add`), automatically managing 31-bit hardware counter wraparound across high-bandwidth interconnect transfers. 2. CPU Hotplug Support (`cpuhp`): - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`). - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online CPU core when a designated CPU goes offline, avoiding stale uncore state. 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture: - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns) directly mapped over ARM physical memory space (`MON__SYSTEM`). - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4), VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic sleeps or blocking in timer/interrupt context. 4. PREEMPT_RT & Safety Hardening: - Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context, ensuring spinlock acquisitions are 100% PREEMPT_RT safe. - Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while active perf events exist. 5. SoC Generation Support: - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4). - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1 Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3 interconnect monitoring. Hardware Validation ------------------- The driver has been validated on real hardware across multiple SoC generations: - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU firmware mailbox IPC performance counters. - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1 Southbridge transfers. Changes in v3 ------------- - 64-bit Device Tree reg Parsing: Replaced of_property_read_u32_index with platform_get_resource(pdev, IORESOURCE_MEM, i) to retrieve mailbox offsets, supporting 64-bit DT layouts (#address-cells = <2>). - Firmware Phandle Fallback Safety: In rpi_axi_pmu__init(), if firmware node is missing or rpi_firmware_get() returns NULL, explicitly set MON__VPU use_mailbox_interface = false. config_is_valid() then returns -EINVAL for VPU events, preventing NULL pointer dereferences. - Driver Teardown Order: In rpi_axi_pmu__exit(), unregister perf_pmu_unregister(&pmu->pmu) first to block new event allocations before removing CPU hotplug callbacks, cancelling hrtimer, and syncing workqueue. - Probe Error Unwind Safety: Updated rpi_axi_pmu__init() error paths to call hrtimer_cancel() and cancel_work_sync() before returning probe failure. - Thread-Safe enabled[idx] Setting: Set enabled[idx] = true under pmu->lock after verifying event generation sequence in vpu_work_handler(), eliminating TOCTOU race conditions. - PREEMPT_RT Compatibility: Switched hrtimer setup and start to HRTIMER_MODE_REL_SOFT so timer callbacks run in softirq context, allowing spinlock_t acquisitions on PREEMPT_RT. - CPU Offline Timer Migration: In rpi_axi_pmu_offline_cpu(), restart hrtimer using unpinned HRTIMER_MODE_REL_SOFT so perf migrates timer execution to the target CPU. - Suppress Unsafe Sysfs Unbind: Set .suppress_bind_attrs = true in platform_driver. - Kconfig Cleanup: Removed non-existent ARCH_BCM2712 symbol from depends on in Kconfig. - Legacy Sysfs Alias Filtering: Updated rpi_axi_pmu_events_is_visible() to filter out all legacy RPi 1-4 System Monitor aliases (ccp2tx_, mphi_, h264_, v3d_, jpeg_, peripheral_, cpu_uc_, cpu_l2_, etc.) on BCM2712 systems. Ian Rogers (2): perf: Add Raspberry Pi BCM2835 AXI PMU driver perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2022 ++++++++++++++++++++++++++++++++++++ 3 files changed, 2033 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 1 sibling, 0 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1748 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1759 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..1923dc34cb3e --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1748 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (mon >= MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; + + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + pmu->monitor[MON__VPU].hw_events.enabled[idx] = true; + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true; + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.enabled[idx] = false; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, + HRTIMER_MODE_REL_SOFT); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + } + if (!pmu->firmware) + pmu->monitor[MON__VPU].use_mailbox_interface = false; + else + pmu->monitor[MON__VPU].use_mailbox_interface = true; + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) { + dev_err(dev, "Error reading mailbox resource %d\n", i); + ret = -EINVAL; + goto err_firmware_put; + } + pmu->monitor[i].mailbox = (u32)resource->start; + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_teardown; + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_pmu_unregister; + } + return 0; + +err_pmu_unregister: + perf_pmu_unregister(&pmu->pmu); +err_teardown: + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + perf_pmu_unregister(&pmu->pmu); + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + .suppress_bind_attrs = true, + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 1 sibling, 0 replies; 10+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 2 +- drivers/perf/rpi_axi_pmu.c | 290 ++++++++++++++++++++++++++++++++++++- 2 files changed, 283 insertions(+), 9 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..84ffa65c7b79 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index 1923dc34cb3e..478d58c7e4f4 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -60,9 +62,11 @@ /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -114,6 +118,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -246,6 +288,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) return false; - if (mon == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (mon != MON__SYSTEM) + return false; + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -867,12 +1043,106 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) + * and legacy RPi 1-4 System Monitor aliases + */ + if (strstr(pmu_attr->event_str, "monitor=1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system") || + strstr(attr->name, "ccp2tx_") || + strstr(attr->name, "mphi_") || + strstr(attr->name, "h264_") || + strstr(attr->name, "v3d_") || + strstr(attr->name, "jpeg_") || + strstr(attr->name, "peripheral_") || + strstr(attr->name, "cpu_uc_") || + strstr(attr->name, "cpu_l2_") || + strstr(attr->name, "l2_control_") || + strstr(attr->name, "system_control_") || + strstr(attr->name, "direct_control_") || + strstr(attr->name, "direct_data_")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1550,7 +1820,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd spin_lock_init(&pmu->lock); mutex_init(&pmu->vpu_mutex); - pmu->chip = CHIP_BCM2835; + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); pmu->pmu = (struct pmu) { .module = THIS_MODULE, @@ -1693,6 +1963,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-08-12 0:28 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 14:32 ` Uwe Kleine-König 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox