* [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver
@ 2026-08-11 8:38 Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw)
To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
Cc: Ian Rogers
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1948 ++++++++++++++++++++++++++++++++++++
3 files changed, 1959 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 20+ messages in thread* [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-11 8:38 ` Ian Rogers 2026-08-11 14:32 ` Uwe Kleine-König 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw) To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel Cc: Ian Rogers Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1703 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1714 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..adb34aefb26e --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1703 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/mod_devicetable.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +#include <linux/version.h> + +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0) +static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer, + enum hrtimer_restart (*function)(struct hrtimer *), + clockid_t clock_id, + enum hrtimer_mode mode) +{ + hrtimer_init(timer, clock_id, mode); + timer->function = function; +} +#define hrtimer_setup rpi_axi_pmu_hrtimer_setup +#endif + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (config_to_monitor(config) == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + int bus, filter, bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus = pmu->monitor[mon].hw_events.monitored_bus[idx]; + filter = pmu->monitor[mon].hw_events.filter[idx]; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = true; +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + lockdep_assert_held(&pmu->lock); + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx]) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = new_count - prev_count; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx); + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = true; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + if (!pmu->firmware) + return -EPROBE_DEFER; + } + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2, + &pmu->monitor[i].mailbox); + if (ret) { + dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret); + goto err_firmware_put; + } + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_firmware_put; + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_cpuhp; + } + return 0; + +err_cpuhp: + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + perf_pmu_unregister(&pmu->pmu); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + {}, +}; + +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 14:32 ` Uwe Kleine-König 0 siblings, 0 replies; 20+ messages in thread From: Uwe Kleine-König @ 2026-08-11 14:32 UTC (permalink / raw) To: Ian Rogers Cc: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel [-- Attachment #1: Type: text/plain, Size: 967 bytes --] Hello, On Tue, Aug 11, 2026 at 01:38:26AM -0700, Ian Rogers wrote: > +#include <linux/mod_devicetable.h> > [...] > +#include <linux/platform_device.h> Please rely on <linux/platform_device.h> to provide the definition of of_device_id and drop <linux/mod_devicetable.h>. The latter will go away soon. > +/* Devices matching this driver in Device Tree */ > +static const struct of_device_id rpi_axi_pmu_match[] = { > + { > + .compatible = "brcm,bcm2835-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + { > + .compatible = "brcm,bcm2711-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + {}, Please use { } as list terminator (that is, add a space and drop the comma). Also if the data is the same for all variants, maybe better drop the assigment to .data and hardcode the value accordingly. > +}; > + > +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); Usually there is no empty line between the device_id struct and the MODULE_DEVICE_TABLE macro. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 8:38 ` Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 0 replies; 20+ messages in thread From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw) To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel, linux-perf-users, linux-rpi-kernel Cc: Ian Rogers Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 4 +- drivers/perf/rpi_axi_pmu.c | 259 ++++++++++++++++++++++++++++++++++++- 2 files changed, 254 insertions(+), 9 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..b0c4eb0748dd 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU config RASPBERRYPI_AXI_PMU tristate "Raspberry Pi AXI PMU support" - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index adb34aefb26e..ec1327804c00 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -75,9 +77,11 @@ static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer, /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -129,6 +133,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -261,6 +303,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -483,16 +613,22 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (config >> 15 != 0) return false; - if (config_to_monitor(config) == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (config_to_monitor(config) == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -719,6 +855,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -875,12 +1049,79 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) */ + if (strstr(pmu_attr->event_str, "monitor=1")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "rp1_pcie")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1648,6 +1889,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, {}, }; -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers ` (2 more replies) 2 siblings, 3 replies; 20+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König This patch series adds an uncore Performance Monitoring Unit (PMU) driver for Broadcom AXI system and VideoCore VPU performance monitors found on Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Motivation & Background ------------------------ Currently, Linux lacks a standard perf-API compatible driver for the Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc kthreads, preventing integration with standard Linux perf tooling (`perf stat`, `perf list`, etc.). This driver implements standard `struct pmu` hardware uncore callbacks under `drivers/perf/`, exposing human-readable sysfs event aliases, unit scaling (`Bytes`), and bus filtering directly to user space. Key Architectural Improvements & Features ------------------------------------------ 1. Standard Linux Perf Integration: - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`. - Supports event sampling and hardware counter accumulation (`local64_add`), automatically managing 31-bit hardware counter wraparound across high-bandwidth interconnect transfers. 2. CPU Hotplug Support (`cpuhp`): - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`). - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online CPU core when a designated CPU goes offline, avoiding stale uncore state. 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture: - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns) directly mapped over ARM physical memory space (`MON__SYSTEM`). - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4), VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic sleeps or blocking in timer/interrupt context. 4. SoC Generation Support: - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4). - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1 Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3 interconnect monitoring. Hardware Validation ------------------- The driver has been validated on real hardware across multiple SoC generations: - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU firmware mailbox IPC performance counters. - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1 Southbridge transfers. Changes in v2 ------------- - Match Data Initialization: In rpi_axi_pmu__init(), set pmu->chip via of_device_get_match_data(dev) to ensure BCM2712 match data is correctly parsed. - Sysfs Event Visibility Filtering: Updated rpi_axi_pmu_events_is_visible() to properly filter all 7 BCM2712 master filtered event aliases (arm_, pcie0_, hvs_filtered, emmc0_) on BCM2835-BCM2711 systems, and hide legacy RPi 1-4 aliases on BCM2712 systems. - Unmapped Monitor Check: Added monitor base_address and initialization check to config_is_valid(). Events targeting uninitialized or unmapped monitors (e.g. missing memory resource or monitor=1 on BCM2712) are properly rejected with -EINVAL. - CPU Affinity Enforcement: Restrict uncore events in event_init() to event->cpu == pmu->cpu to prevent orphaned uncore events when a non-PMU CPU goes offline. - 31-bit Counter Wrap Delta: Mask counter accumulation with delta = (new_count - prev_count) & 0x7FFFFFFF in read(), timer_handler(), and vpu_work_handler() to prevent 2B+ event spikes on counter overflow. - Stopped Event Guard: Return early in rpi_axi_pmu_read() if event->hw.state & PERF_HES_STOPPED. - Lockless Race Fix: Pass bus and filter explicitly to rpi_axi_pmu_enable_bus_watcher() under spinlock to prevent lockless read races in vpu_work_handler(). - Slot Reuse Clean: Set enabled[idx] = false in rpi_axi_pmu_del() when refcount[idx] == 0 so re-allocated watcher slots properly enable hardware watchers. - Events Array Bounds: Return -ENOSPC in rpi_axi_pmu_add() if pmu->events array is full. - Timer Cancel Race Fix: Re-acquire pmu->lock after hrtimer_cancel() in rpi_axi_pmu_del() and re-verify active_events > 0 before returning. - Hotplug Timer Migration: Cancel and re-pin pmu->hrtimer to target CPU in rpi_axi_pmu_offline_cpu(). - Registration Order: Call perf_pmu_register() before cpuhp_state_add_instance() in probe, and cancel hrtimer before cancel_work_sync() and unregister in exit teardown. - Header & Style Cleanups: Removed <linux/mod_devicetable.h> and <linux/version.h>, updated rpi_axi_pmu_match[] terminator to { }, and fixed C variable declaration style. Ian Rogers (2): perf: Add Raspberry Pi BCM2835 AXI PMU driver perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2001 ++++++++++++++++++++++++++++++++++++ 3 files changed, 2012 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-12 0:04 ` sashiko-bot 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1739 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1750 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..cdf7d9a74f13 --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1739 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (mon >= MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = true; +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; + + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.enabled[idx] = false; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_PINNED); + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_PINNED); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = true; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + if (!pmu->firmware) + return -EPROBE_DEFER; + } + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2, + &pmu->monitor[i].mailbox); + if (ret) { + dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret); + goto err_firmware_put; + } + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_firmware_put; + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_pmu_unregister; + } + return 0; + +err_pmu_unregister: + perf_pmu_unregister(&pmu->pmu); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + perf_pmu_unregister(&pmu->pmu); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 0:04 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 0:04 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider: - [High] The probe function ignores a missing 'firmware' device tree phandle, leaving `pmu->firmware` NULL but leaving the VPU monitor enabled, causing a later NULL pointer dereference in the mailbox IPC. - [High] The driver fails to prevent sysfs unbinding (`.suppress_bind_attrs = true`), leading to a Use-After-Free if unbound while perf events are active. - [High] Incorrect manual parsing of the 'reg' property assumes fixed address and size cells, breaking on 64-bit platforms. - [High] Incorrect PMU teardown ordering in `rpi_axi_pmu__exit` and missing cleanup in probe error paths cause Use-After-Free vulnerabilities. - [High] The PMU driver uses a `spinlock_t` in a hardirq hrtimer callback, causing lockdep splats and sleeping in atomic context on PREEMPT_RT. - [Medium] An unprotected write to `hw_events.enabled[idx]` creates a race condition with event deletion, which can silently break hardware configuration for recycled bus watcher slots. - [Medium] The CPU hotplug offline callback erroneously restarts the polling hrtimer pinned to the dying CPU, sabotaging PMU migration. - [Low] Raspberry Pi 5 (BCM2712) support is partially implemented but completely unreachable because the chip type is hardcoded to BCM2835 and the device tree match table lacks BCM2712. -- --- Patch [1]: [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver --- commit 3567c05975da01f684191bbbfa826e0e8d8e9927 Author: Ian Rogers <irogers@google.com> Subject: perf: Add Raspberry Pi BCM2835 AXI PMU driver This commit introduces a new uncore PMU driver for Broadcom BCM2835-BCM2711 AXI system monitors. It supports tracking AXI traffic via MMIO for the system monitor and via mailbox IPC for the VPU monitor. > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > new file mode 100644 > index 0000000000000..cdf7d9a74f13e > --- /dev/null > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) > +{ [ ... ] > + pmu->chip = CHIP_BCM2835; [Severity: Low] Is BCM2712 hardware meant to be supported here? The chip type is unconditionally hardcoded to CHIP_BCM2835, which appears to make the BCM2712 definitions unreachable. > + > + pmu->pmu = (struct pmu) { [ ... ] > + if (pmu->chip == CHIP_BCM2835) { > + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; > + pmu->monitor[MON__VPU].use_mailbox_interface = true; > + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); > + if (fw_node) { > + pmu->firmware = rpi_firmware_get(fw_node); > + of_node_put(fw_node); > + if (!pmu->firmware) > + return -EPROBE_DEFER; > + } [Severity: High] Can this lead to a NULL pointer dereference in the mailbox IPC? If the device tree lacks the firmware property, fw_node is NULL and the driver continues with pmu->firmware left as NULL. Later, set_monitor_control() will call rpi_firmware_property() using this NULL pointer when VPU events are started. > + } else { [ ... ] > + for (int i = 0; i < MON__MAX; i++) { > + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); > + > + if (pmu->monitor[i].use_mailbox_interface) { > + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2, > + &pmu->monitor[i].mailbox); [Severity: High] Does this correctly parse the reg property on 64-bit platforms? Using of_property_read_u32_index() with a fixed index of i * 2 assumes that address and size cells are exactly 1. On platforms with 64-bit device tree layouts, could this fetch the wrong word and corrupt the mailbox address? > + if (ret) { [ ... ] > + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); > + if (ret) { > + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); > + goto err_pmu_unregister; > + } > + return 0; > + > +err_pmu_unregister: > + perf_pmu_unregister(&pmu->pmu); > +err_firmware_put: > + if (pmu->firmware) > + rpi_firmware_put(pmu->firmware); > + return ret; > +} [Severity: High] Could this regression cause a use-after-free if probe fails here? If cpuhp_state_add_instance() fails, the error path unregisters the PMU but does not cancel the hrtimer or vpu_work. Since the PMU is briefly visible in sysfs, might userspace race to open an event and arm the timer before the context is freed by devres? > +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) > +{ > + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); > + hrtimer_cancel(&pmu->hrtimer); > + cancel_work_sync(&pmu->vpu_work); > + if (pmu->firmware) > + rpi_firmware_put(pmu->firmware); > + perf_pmu_unregister(&pmu->pmu); > +} [Severity: High] Does this teardown ordering allow new events to race with driver removal? By cancelling the timer and workqueue before unregistering the PMU, there appears to be a window where new events can be scheduled that restart the timers while the context is being torn down. [ ... ] > +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, > + int idx, int bus, int filter) > +{ [ ... ] > + pmu->monitor[mon].hw_events.enabled[idx] = true; > +} [Severity: Medium] Is this state update safe from concurrent deletion? The array element is modified without holding pmu->lock. When rpi_axi_pmu_vpu_work_handler() drops the lock to call this function: rpi_axi_pmu_vpu_work_handler() spin_unlock_irq(&pmu->lock); rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); spin_lock_irq(&pmu->lock); A concurrent rpi_axi_pmu_del() could clear the slot and set enabled to false. If a new event immediately reuses this slot, wouldn't the unlocked write to true overwrite the hardware configuration state for the new event? [ ... ] > +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) > +{ > + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); > + unsigned long flags; > + > + spin_lock_irqsave(&pmu->lock, flags); [Severity: High] Will this lock acquisition panic on PREEMPT_RT kernels? The timer defaults to hardirq context, where acquiring a sleepable lock like spinlock_t triggers a panic on PREEMPT_RT. [ ... ] > +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) > +{ [ ... ] > + spin_lock_irqsave(&pmu->lock, spinflags); > + if (pmu->active_events > 0) { > + spin_unlock_irqrestore(&pmu->lock, spinflags); > + hrtimer_cancel(&pmu->hrtimer); > + spin_lock_irqsave(&pmu->lock, spinflags); > + if (pmu->active_events > 0) > + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, > + HRTIMER_MODE_REL_PINNED); > + } > + spin_unlock_irqrestore(&pmu->lock, spinflags); > + > + return 0; > +} [Severity: Medium] Does this logic unintentionally sabotage PMU migration during CPU offline? This callback runs in the context of the dying CPU. By passing HRTIMER_MODE_REL_PINNED here, the timer seems to be pinned to the offlining CPU instead of being successfully migrated to the target CPU. [ ... ] > +static int rpi_axi_pmu_probe(struct platform_device *pdev) > +{ > + struct rpi_axi_pmu *pmu; > + > + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); [ ... ] > +static const struct of_device_id rpi_axi_pmu_match[] = { > + { > + .compatible = "brcm,bcm2835-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + { > + .compatible = "brcm,bcm2711-axiperf", > + .data = (void *)CHIP_BCM2835, > + }, > + { } > +}; [Severity: Low] Could this device tree match table be missing a compatible string for BCM2712 hardware, preventing it from probing entirely? > +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); > + > +static struct platform_driver rpi_axi_pmu_driver = { > + .probe = rpi_axi_pmu_probe, > + .remove = rpi_axi_pmu_remove, > + .driver = { > + .name = PMU_NAME, > + .of_match_table = of_match_ptr(rpi_axi_pmu_match), > + }, > +}; [Severity: High] Can an administrator unbinding this driver via sysfs trigger a use-after-free? The pmu context is allocated with devm_kzalloc(), tying its memory lifetime to the device bind state. Since .suppress_bind_attrs = true is omitted, a sysfs unbind will free the memory while active perf events might still exist and try to access the freed context. -- Sashiko AI review · https://sashiko.dev/#/patchset/20260811235112.2267241-1-irogers@google.com?part=1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-11 23:51 ` Ian Rogers 2026-08-12 0:05 ` sashiko-bot 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, will, Uwe Kleine-König Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 4 +- drivers/perf/rpi_axi_pmu.c | 278 +++++++++++++++++++++++++++++++++++-- 2 files changed, 272 insertions(+), 10 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..b0c4eb0748dd 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU config RASPBERRYPI_AXI_PMU tristate "Raspberry Pi AXI PMU support" - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index cdf7d9a74f13..b13632fa8f84 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -60,9 +62,11 @@ /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -114,6 +118,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -246,6 +288,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) return false; - if (mon == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (mon != MON__SYSTEM) + return false; + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -867,12 +1043,94 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) + * and RPi 1-4 aliases + */ + if (strstr(pmu_attr->event_str, "monitor=1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1549,7 +1807,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd spin_lock_init(&pmu->lock); mutex_init(&pmu->vpu_mutex); - pmu->chip = CHIP_BCM2835; + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); pmu->pmu = (struct pmu) { .module = THIS_MODULE, @@ -1685,6 +1943,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 0:05 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 0:05 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Incorrect subsystem teardown ordering in both the exit and init error paths fails to cancel background tasks, leading to Use-After-Free. - [Medium] Incomplete hiding of legacy BCM2835-specific sysfs event aliases on BCM2712. - [Medium] Dropping the spinlock in the VPU work handler causes a TOCTOU race that corrupts hardware configurations. - [Low] Invalid Kconfig dependency on non-existent symbol `ARCH_BCM2712`. -- --- Patch [2]: [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support --- commit c6c37f9c3fb56c6bdbc0982c7918061dbe1d45cb Author: Ian Rogers <irogers@google.com> perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. > diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig > index 97c8bf34f4633..b0c4eb0748dd3 100644 > --- a/drivers/perf/Kconfig > +++ b/drivers/perf/Kconfig > @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU > > config RASPBERRYPI_AXI_PMU > tristate "Raspberry Pi AXI PMU support" > - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST > + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST [Severity: Low] Does this code add a dependency on an undefined symbol? ARCH_BCM2712 does not appear to be defined anywhere in the kernel tree. > select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 > help > Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom > - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). > + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). > > If compiled as a module, it will be called rpi_axi_pmu. > > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > index cdf7d9a74f13e..b13632fa8f84c 100644 > --- a/drivers/perf/rpi_axi_pmu.c > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > @@ -867,12 +1043,94 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { [ ... ] > +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, > + struct attribute *attr, int unused) > +{ > + struct device *dev = kobj_to_dev(kobj); > + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); > + struct perf_pmu_events_attr *pmu_attr; > + > + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); > + > + if (pmu->chip == CHIP_BCM2712) { > + /* > + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) > + * and RPi 1-4 aliases > + */ > + if (strstr(pmu_attr->event_str, "monitor=1") || > + strstr(attr->name, "cpu0_") || > + strstr(attr->name, "cpu1_") || > + strstr(attr->name, "dma0_") || > + strstr(attr->name, "dma1_") || > + strstr(attr->name, "v3d0_") || > + strstr(attr->name, "v3d1_") || > + strstr(attr->name, "hvs_system") || > + strstr(attr->name, "isp_system") || > + strstr(attr->name, "usb_system")) > + return 0; [Severity: Medium] Are there missing legacy aliases that should be hidden here? This hides VPU IPC events on RPi 5, but it looks like several legacy System Monitor aliases for diverged buses (such as ccp2tx, mphi_rx, h264, v3d, peripheral, cpu_uc, cpu_l2) are not included in this check. Opening them might measure the wrong hardware unit or fail with -EINVAL since their hardcoded bus indices exceed BCM2712_SB__MAX. > + } else { > + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ > + if (strstr(attr->name, "pcie_rp1") || > + strstr(attr->name, "a76_") || > + strstr(attr->name, "v3d7_") || > + strstr(attr->name, "hevc_dec") || > + strstr(attr->name, "arm_") || > + strstr(attr->name, "pcie0_") || > + strstr(attr->name, "hvs_filtered") || > + strstr(attr->name, "emmc0_")) > + return 0; > + } > + > + return attr->mode; > +} [ ... ] > @@ -1549,7 +1807,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd > spin_lock_init(&pmu->lock); > mutex_init(&pmu->vpu_mutex); > > - pmu->chip = CHIP_BCM2835; > + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); > > pmu->pmu = (struct pmu) { > .module = THIS_MODULE, [Severity: High] Can the error paths in rpi_axi_pmu__init() and the teardown in rpi_axi_pmu__exit() lead to a use-after-free? If cpuhp_state_add_instance() fails in rpi_axi_pmu__init(), the driver calls perf_pmu_unregister() but doesn't cancel the background tasks: rpi_axi_pmu__init() { ... err_pmu_unregister: perf_pmu_unregister(&pmu->pmu); err_firmware_put: if (pmu->firmware) rpi_firmware_put(pmu->firmware); return ret; } Since PMU memory is freed via devres upon returning the error, an active timer or workqueue (started by an event opened during the registration window) could fire and access freed memory. Similarly, in rpi_axi_pmu__exit(), the background tasks are cancelled before unregistering the PMU subsystem: rpi_axi_pmu__exit() { cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); hrtimer_cancel(&pmu->hrtimer); cancel_work_sync(&pmu->vpu_work); ... perf_pmu_unregister(&pmu->pmu); } A concurrent userspace action could start a new event in this window, restarting the timer and causing a use-after-free when the memory is subsequently freed. [Severity: Medium] Is there a time-of-check to time-of-use race condition in rpi_axi_pmu_vpu_work_handler()? When the lock is dropped, a concurrent event deletion and addition could potentially misconfigure the PMU watcher: rpi_axi_pmu_vpu_work_handler() { ... if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; spin_unlock_irq(&pmu->lock); rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); spin_lock_irq(&pmu->lock); } ... } Concurrently, rpi_axi_pmu_del() can delete the event and free the idx slot, which rpi_axi_pmu_add() can then reuse for a new event with different configurations. Because the old values were cached before dropping the lock, they might be forcefully written to the hardware, and enabled[idx] is modified outside the lock: rpi_axi_pmu_enable_bus_watcher() { ... pmu->monitor[mon].hw_events.enabled[idx] = true; ... } Could this silently misconfigure the PMU watcher for the new event? -- Sashiko AI review · https://sashiko.dev/#/patchset/20260811235112.2267241-1-irogers@google.com?part=2 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers ` (2 more replies) 2 siblings, 3 replies; 20+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will This patch series adds an uncore Performance Monitoring Unit (PMU) driver for Broadcom AXI system and VideoCore VPU performance monitors found on Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Motivation & Background ------------------------ Currently, Linux lacks a standard perf-API compatible driver for the Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc kthreads, preventing integration with standard Linux perf tooling (`perf stat`, `perf list`, etc.). This driver implements standard `struct pmu` hardware uncore callbacks under `drivers/perf/`, exposing human-readable sysfs event aliases, unit scaling (`Bytes`), and bus filtering directly to user space. Key Architectural Improvements & Features ------------------------------------------ 1. Standard Linux Perf Integration: - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`. - Supports event sampling and hardware counter accumulation (`local64_add`), automatically managing 31-bit hardware counter wraparound across high-bandwidth interconnect transfers. 2. CPU Hotplug Support (`cpuhp`): - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`). - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online CPU core when a designated CPU goes offline, avoiding stale uncore state. 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture: - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns) directly mapped over ARM physical memory space (`MON__SYSTEM`). - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4), VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic sleeps or blocking in timer/interrupt context. 4. PREEMPT_RT & Safety Hardening: - Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context, ensuring spinlock acquisitions are 100% PREEMPT_RT safe. - Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while active perf events exist. 5. SoC Generation Support: - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4). - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1 Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3 interconnect monitoring. Hardware Validation ------------------- The driver has been validated on real hardware across multiple SoC generations: - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU firmware mailbox IPC performance counters. - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1 Southbridge transfers. Changes in v3 ------------- - 64-bit Device Tree reg Parsing: Replaced of_property_read_u32_index with platform_get_resource(pdev, IORESOURCE_MEM, i) to retrieve mailbox offsets, supporting 64-bit DT layouts (#address-cells = <2>). - Firmware Phandle Fallback Safety: In rpi_axi_pmu__init(), if firmware node is missing or rpi_firmware_get() returns NULL, explicitly set MON__VPU use_mailbox_interface = false. config_is_valid() then returns -EINVAL for VPU events, preventing NULL pointer dereferences. - Driver Teardown Order: In rpi_axi_pmu__exit(), unregister perf_pmu_unregister(&pmu->pmu) first to block new event allocations before removing CPU hotplug callbacks, cancelling hrtimer, and syncing workqueue. - Probe Error Unwind Safety: Updated rpi_axi_pmu__init() error paths to call hrtimer_cancel() and cancel_work_sync() before returning probe failure. - Thread-Safe enabled[idx] Setting: Set enabled[idx] = true under pmu->lock after verifying event generation sequence in vpu_work_handler(), eliminating TOCTOU race conditions. - PREEMPT_RT Compatibility: Switched hrtimer setup and start to HRTIMER_MODE_REL_SOFT so timer callbacks run in softirq context, allowing spinlock_t acquisitions on PREEMPT_RT. - CPU Offline Timer Migration: In rpi_axi_pmu_offline_cpu(), restart hrtimer using unpinned HRTIMER_MODE_REL_SOFT so perf migrates timer execution to the target CPU. - Suppress Unsafe Sysfs Unbind: Set .suppress_bind_attrs = true in platform_driver. - Kconfig Cleanup: Removed non-existent ARCH_BCM2712 symbol from depends on in Kconfig. - Legacy Sysfs Alias Filtering: Updated rpi_axi_pmu_events_is_visible() to filter out all legacy RPi 1-4 System Monitor aliases (ccp2tx_, mphi_, h264_, v3d_, jpeg_, peripheral_, cpu_uc_, cpu_l2_, etc.) on BCM2712 systems. Ian Rogers (2): perf: Add Raspberry Pi BCM2835 AXI PMU driver perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2022 ++++++++++++++++++++++++++++++++++++ 3 files changed, 2033 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 2026-08-12 0:40 ` sashiko-bot 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1748 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1759 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..1923dc34cb3e --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1748 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/lockdep.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (mon >= MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + ret = tmp[2]; + } else { + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; + + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read (which can sleep) */ + spin_unlock_irq(&pmu->lock); + + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + + spin_lock_irq(&pmu->lock); + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + pmu->monitor[MON__VPU].hw_events.enabled[idx] = true; + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true; + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.enabled[idx] = false; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = raw_smp_processor_id(); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, + HRTIMER_MODE_REL_SOFT); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + } + if (!pmu->firmware) + pmu->monitor[MON__VPU].use_mailbox_interface = false; + else + pmu->monitor[MON__VPU].use_mailbox_interface = true; + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) { + dev_err(dev, "Error reading mailbox resource %d\n", i); + ret = -EINVAL; + goto err_firmware_put; + } + pmu->monitor[i].mailbox = (u32)resource->start; + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_teardown; + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_pmu_unregister; + } + return 0; + +err_pmu_unregister: + perf_pmu_unregister(&pmu->pmu); +err_teardown: + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + perf_pmu_unregister(&pmu->pmu); + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + .suppress_bind_attrs = true, + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 0:40 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 0:40 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [High] Calling hrtimer_cancel() inside rpi_axi_pmu_del() causes an unrecoverable deadlock if the PMU event is deleted via a cross-CPU IPI while the timer callback is running. - [High] Global monitor reset is asserted every time a single bus watcher is enabled, corrupting other active events. - [High] When MON__VPU falls back to MMIO, the vpu_work_handler drops the spinlock before accessing hardware, triggering lockdep assertions and unprotected MMIO writes. - [Medium] rpi_axi_pmu_read_counter() returns -1 (0xFFFFFFFF) when a mailbox IPC failure occurs, causing massive artificial spikes in the reported event counts. - [Medium] Race condition during probe allows the PMU to be permanently bound to an offline CPU. -- --- Patch [1]: [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver --- commit b18036baa6b636e4ea8f9af5eee5462724c801ef Author: Ian Rogers <irogers@google.com> perf: Add Raspberry Pi BCM2835 AXI PMU driver This commit adds an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > new file mode 100644 > index 0000000000000..1923dc34cb3e9 > --- /dev/null > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, > + int idx, int bus, int filter) > +{ > + int bus_control; > + > + if (pmu->monitor[mon].hw_events.enabled[idx]) > + return; > + > + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); > + if (filter) { > + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; > + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; > + } > + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); > + set_bus_watcher_control(pmu, mon, idx, bus_control); > + > + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); [Severity: High] Does this global reset corrupt other active events? It appears that whenever a new event is started, this unconditionally issues the reset bit to the global monitor control register. This might destroy the hardware state and clear the counters of all other currently running bus watchers, resulting in massive phantom spikes on their next read due to underflow. > + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); > +} [ ... ] > +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, > + enum counter counter) > +{ > + int watcher = watcher_offset(pmu, mon, idx); > + int offset = counter_offset(counter); > + u32 ret; > + > + if (!pmu->monitor[mon].hw_events.enabled[idx]) > + return 0; > + > + if (pmu->monitor[mon].use_mailbox_interface) { > + u32 tmp[3] = { > + pmu->monitor[mon].mailbox + watcher + offset, > + 1, -1 > + }; > + int err; > + > + might_sleep(); > + lockdep_assert_held(&pmu->vpu_mutex); > + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) > + return -1; > + > + err = rpi_firmware_property(pmu->firmware, > + RPI_FIRMWARE_GET_PERIPH_REG, > + tmp, sizeof(tmp)); > + > + if (err < 0 || tmp[1] != 1) > + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); > + ret = tmp[2]; [Severity: Medium] Is it safe to return the uninitialized buffer on an IPC failure? If the firmware mailbox call fails or times out, it looks like the response buffer tmp[2] remains -1, and that is returned directly. This could be treated as a huge jump (0xFFFFFFFF) in the delta calculation and result in massive artificial spikes in the reported event counts. > + } else { > + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; > + > + lockdep_assert_held(&pmu->lock); > + ret = readl(addr) & 0x7FFFFFFF; > + } > + return ret; > +} [ ... ] > +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) > +{ > + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); > + > + might_sleep(); > + mutex_lock(&pmu->vpu_mutex); > + spin_lock_irq(&pmu->lock); > + > + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { [ ... ] > + /* Drop spinlock during Mailbox IPC read (which can sleep) */ > + spin_unlock_irq(&pmu->lock); > + > + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); > + > + spin_lock_irq(&pmu->lock); [Severity: High] Could this drop of the spinlock trigger a lockdep warning and data race if the monitor falls back to MMIO? If the firmware is unavailable, the VPU monitor might fall back to direct MMIO (use_mailbox_interface = false). The work handler assumes it uses IPC and drops the lock. But the MMIO path in rpi_axi_pmu_read_counter() asserts lockdep_assert_held(&pmu->lock), and performs raw readl operations, potentially racing with synchronous MMIO reads in rpi_axi_pmu_read(). [ ... ] > +static void rpi_axi_pmu_del(struct perf_event *event, int flags) > +{ > + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); > + enum monitor mon = config_to_monitor(event->attr.config); > + unsigned long spinflags; > + int idx = event->hw.idx; > + > + if (idx < 0) > + return; > + > + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); > + > + spin_lock_irqsave(&pmu->lock, spinflags); [ ... ] > + event->hw.idx = -1; > + pmu->active_events--; > + if (mon == MON__VPU) > + pmu->active_vpu_events--; > + > + if (pmu->active_events == 0) { > + spin_unlock_irqrestore(&pmu->lock, spinflags); > + hrtimer_cancel(&pmu->hrtimer); > + spin_lock_irqsave(&pmu->lock, spinflags); [Severity: High] Can this sequence cause a deadlock? The rpi_axi_pmu_del() function can be invoked in hardirq context via an SMP cross-CPU IPI. Since the hrtimer is configured with HRTIMER_MODE_REL_SOFT, its callback runs in softirq context on the same CPU. If the IPI interrupts the executing softirq timer handler, synchronously calling hrtimer_cancel() will spin indefinitely waiting for the preempted callback to finish. > + if (pmu->active_events > 0) > + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, > + HRTIMER_MODE_REL_SOFT); > + spin_unlock_irqrestore(&pmu->lock, spinflags); > + return; > + } > + spin_unlock_irqrestore(&pmu->lock, spinflags); > +} [ ... ] > +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) > +{ > + struct device *dev = &pdev->dev; > + struct device_node *fw_node; > + int ret; > + > + spin_lock_init(&pmu->lock); > + mutex_init(&pmu->vpu_mutex); > + > + pmu->chip = CHIP_BCM2835; > + > + pmu->pmu = (struct pmu) { [ ... ] > + }; > + pmu->pdev = pdev; > + pmu->cpu = raw_smp_processor_id(); > + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, > + HRTIMER_MODE_REL_SOFT); > + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); [ ... ] > + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); > + if (ret) { > + dev_err(dev, "PMU register failed %d\n", ret); > + goto err_teardown; > + } > + > + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); [Severity: Medium] Is there a race condition here with CPU hotplug? The CPU is assigned using raw_smp_processor_id() in a preemptible context before cpuhp_state_add_instance() is called. If this CPU is taken offline before the hotplug instance is registered, the subsystem misses the transition. The teardown callback won't run, leaving the PMU permanently bound to an offline CPU. > + if (ret) { > + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); > + goto err_pmu_unregister; > + } > + return 0; -- Sashiko AI review · https://sashiko.dev/#/patchset/20260812002728.2283079-1-irogers@google.com?part=1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 0:27 ` Ian Rogers 2026-08-12 0:40 ` sashiko-bot 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 2 +- drivers/perf/rpi_axi_pmu.c | 290 ++++++++++++++++++++++++++++++++++++- 2 files changed, 283 insertions(+), 9 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..84ffa65c7b79 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index 1923dc34cb3e..478d58c7e4f4 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -60,9 +62,11 @@ /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -114,6 +118,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -246,6 +288,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) return false; - if (mon == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (mon != MON__SYSTEM) + return false; + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -867,12 +1043,106 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (pmu->chip == CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) + * and legacy RPi 1-4 System Monitor aliases + */ + if (strstr(pmu_attr->event_str, "monitor=1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system") || + strstr(attr->name, "ccp2tx_") || + strstr(attr->name, "mphi_") || + strstr(attr->name, "h264_") || + strstr(attr->name, "v3d_") || + strstr(attr->name, "jpeg_") || + strstr(attr->name, "peripheral_") || + strstr(attr->name, "cpu_uc_") || + strstr(attr->name, "cpu_l2_") || + strstr(attr->name, "l2_control_") || + strstr(attr->name, "system_control_") || + strstr(attr->name, "direct_control_") || + strstr(attr->name, "direct_data_")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -1550,7 +1820,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd spin_lock_init(&pmu->lock); mutex_init(&pmu->vpu_mutex); - pmu->chip = CHIP_BCM2835; + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); pmu->pmu = (struct pmu) { .module = THIS_MODULE, @@ -1693,6 +1963,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 0:40 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 0:40 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Implicit and fragile pointer cast from `struct pmu *` to `struct rpi_axi_pmu *` instead of using `container_of`. -- --- Patch [2]: [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support --- commit 313acedcd7c09e03b7e84ac26f169991353b78d9 Author: Ian Rogers <irogers@google.com> Subject: perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support This commit expands the Raspberry Pi AXI PMU driver to support the Broadcom BCM2712 SoC used in the Raspberry Pi 5. It adds new event definitions, bus enumerations, and filter aliases specifically for the new architecture. > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > index 1923dc34cb3e9..478d58c7e4f41 100644 > --- a/drivers/perf/rpi_axi_pmu.c > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > @@ -867,12 +1043,106 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { [ ... ] > +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, > + struct attribute *attr, int unused) > +{ > + struct device *dev = kobj_to_dev(kobj); > + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); [Severity: Low] Since the perf core sets the device's driver data to a struct pmu pointer via pmu_dev_alloc(), doesn't dev_get_drvdata(dev) return a struct pmu * here? While this implicit cast works right now because the pmu member is located at offset 0 in struct rpi_axi_pmu, would it be safer to use the existing pmu_to_rpi_axi_pmu() helper to avoid potential type confusion and memory corruption if the struct layout ever changes in the future? > + struct perf_pmu_events_attr *pmu_attr; > + > + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); > + > + if (pmu->chip == CHIP_BCM2712) { -- Sashiko AI review · https://sashiko.dev/#/patchset/20260812002728.2283079-1-irogers@google.com?part=2 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 5:24 ` Ian Rogers 2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers ` (2 more replies) 2 siblings, 3 replies; 20+ messages in thread From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will This patch series adds an uncore Performance Monitoring Unit (PMU) driver for Broadcom AXI system and VideoCore VPU performance monitors found on Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Motivation & Background ------------------------ Currently, Linux lacks a standard perf-API compatible driver for the Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc kthreads, preventing integration with standard Linux perf tooling (`perf stat`, `perf list`, etc.). This driver implements standard `struct pmu` hardware uncore callbacks under `drivers/perf/`, exposing human-readable sysfs event aliases, unit scaling (`Bytes`), and bus filtering directly to user space. Key Architectural Improvements & Features ------------------------------------------ 1. Standard Linux Perf Integration: - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`. - Supports event sampling and hardware counter accumulation (`local64_add`), automatically managing 31-bit hardware counter wraparound across high-bandwidth interconnect transfers. 2. CPU Hotplug Support (`cpuhp`): - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`). - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online CPU core when a designated CPU goes offline, avoiding stale uncore state. 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture: - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns) directly mapped over ARM physical memory space (`MON__SYSTEM`). - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4), VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic sleeps or blocking in timer/interrupt context. 4. PREEMPT_RT & Safety Hardening: - Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context, ensuring spinlock acquisitions are 100% PREEMPT_RT safe. - Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while active perf events exist. 5. SoC Generation Support: - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4). - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1 Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3 interconnect monitoring. Hardware Validation ------------------- The driver has been validated on real hardware across multiple SoC generations: - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU firmware mailbox IPC performance counters. - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1 Southbridge transfers. Changes in v4 ------------- - Global Monitor Reset Protection: In rpi_axi_pmu_enable_bus_watcher(), only issue global monitor resets (GEN_CTL_RESET_BIT) when num_monitored == 1. For subsequent watchers, reset only the individual bus watcher, keeping running watchers intact. - IPC Failure Handling: In rpi_axi_pmu_read_counter(), return 0 on rpi_firmware_property() failure instead of uninitialized -1, preventing 0xFFFFFFFF phantom counter spikes. - Lockdep & MMIO Safety: In rpi_axi_pmu_vpu_work_handler(), only unlock spinlock during counter reads if use_mailbox_interface is true, keeping MMIO reads lock-protected. - Hardirq IPI Deadlock Fix: In rpi_axi_pmu_del(), use hrtimer_try_to_cancel() when in_hardirq() to avoid infinite spinning if an SMP cross-CPU IPI interrupts a running softirq timer callback. - Timer Handler Early Return: In rpi_axi_pmu_timer_handler(), return HRTIMER_NORESTART early if active_events == 0. - CPU Hotplug Preemption Race Fix: Initialize pmu->cpu = cpumask_first(cpu_online_mask) in rpi_axi_pmu__init() to guarantee assignment to a valid online CPU core. - Type-Safe Driver Data Extraction: In rpi_axi_pmu_events_is_visible() and cpumask_show(), retrieve struct pmu * from dev_get_drvdata(dev) and convert via pmu_to_rpi_axi_pmu(). - Backward Compatibility: Added LINUX_VERSION_CODE check for hrtimer_setup fallback wrapper, allowing out-of-tree builds to compile cleanly on Linux 6.12 LTS kernels. Ian Rogers (2): perf: Add Raspberry Pi BCM2835 AXI PMU driver perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2049 ++++++++++++++++++++++++++++++++++++ 3 files changed, 2060 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers @ 2026-08-12 5:24 ` Ian Rogers 2026-08-12 5:36 ` sashiko-bot 2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Add an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 1773 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1784 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..97c8bf34f463 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RASPBERRYPI_AXI_PMU + tristate "Raspberry Pi AXI PMU support" + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 + help + Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + + If compiled as a module, it will be called rpi_axi_pmu. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..29d3915d2dbe 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..827656b14d85 --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1773 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON__SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence counters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include <linux/cpuhotplug.h> +#include <linux/cpumask.h> +#include <linux/hrtimer.h> +#include <linux/io.h> +#include <linux/version.h> + +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0) +static inline void rpi_hrtimer_setup(struct hrtimer *timer, + enum hrtimer_restart (*function)(struct hrtimer *), + clockid_t clock_id, enum hrtimer_mode mode) +{ + hrtimer_init(timer, clock_id, mode); + timer->function = function; +} +#define hrtimer_setup rpi_hrtimer_setup +#endif +#include <linux/lockdep.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> +#include <linux/spinlock.h> +#include <linux/sysfs.h> +#include <linux/vmalloc.h> +#include <linux/workqueue.h> + +#include <soc/bcm2835/raspberrypi-firmware.h> + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, +}; + +enum monitor { + MON__SYSTEM = 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 = 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 = 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS = 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE = 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->num_started = 0; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = BCM2835_FLT__NONE; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 15 != 0) + return false; + + if (mon >= MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) + return false; + + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] = { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may + * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a + * close (~95%) byte throughput approximation in perf stat output (accounting for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5"); + +/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5"); + +/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5"); + +/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5"); + +/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5"); + +/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5"); + +/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0"); + +/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5"); + +/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] = { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_events_group = { + .name = "events", + .attrs = rpi_axi_pmu_events_attrs, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + if (pmu->monitor[mon].hw_events.num_monitored == 1) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + } +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return; + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + + pmu->monitor[mon].hw_events.enabled[idx] = false; +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(pmu, mon, idx); + int offset = counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] = { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] != 1) { + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + return 0; + } + ret = tmp[2] & 0x7FFFFFFF; + } else { + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret = readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work). + * Direct read() syscalls return cached count immediately without blocking. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. + */ + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely. + * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__VPU) + continue; + + gen = pmu->event_gen[i]; + counter = config_to_counter(event->attr.config); + idx = event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; + + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } + + /* Drop spinlock during Mailbox IPC read only if using mailbox interface */ + if (pmu->monitor[MON__VPU].use_mailbox_interface) { + spin_unlock_irq(&pmu->lock); + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + spin_lock_irq(&pmu->lock); + } else { + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + } + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] == event && + pmu->event_gen[i] == gen && + !(event->hw.state & PERF_HES_STOPPED)) { + pmu->monitor[MON__VPU].hw_events.enabled[idx] = true; + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |= PERF_HES_UPTODATE; + } else { + prev_count = local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, flags); + return HRTIMER_NORESTART; + } + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON__SYSTEM) + continue; + + counter = config_to_counter(event->attr.config); + prev_count = local64_read(&event->hw.prev_count); + new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + local64_set(&event->hw.prev_count, new_count); + delta = (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + spin_unlock_irqrestore(&pmu->lock, flags); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state = 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon == MON__SYSTEM) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true; + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) { + rpi_axi_pmu_read(event); + event->hw.state |= PERF_HES_UPTODATE; + } + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state |= PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + if (mon == MON__SYSTEM) + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.enabled[idx] = false; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events == 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + if (in_hardirq()) + hrtimer_try_to_cancel(&pmu->hrtimer); + else + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip = CHIP_BCM2835; + + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = rpi_axi_pmu_attr_groups, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev = pdev; + pmu->cpu = cpumask_first(cpu_online_mask); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, + HRTIMER_MODE_REL_SOFT); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip == CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + fw_node = of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware = rpi_firmware_get(fw_node); + of_node_put(fw_node); + } + if (!pmu->firmware) + pmu->monitor[MON__VPU].use_mailbox_interface = false; + else + pmu->monitor[MON__VPU].use_mailbox_interface = true; + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON__VPU].use_mailbox_interface = false; + } + + for (int i = 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) { + dev_err(dev, "Error reading mailbox resource %d\n", i); + ret = -EINVAL; + goto err_firmware_put; + } + pmu->monitor[i].mailbox = (u32)resource->start; + } else { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_teardown; + } + + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_pmu_unregister; + } + return 0; + +err_pmu_unregister: + perf_pmu_unregister(&pmu->pmu); +err_teardown: + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + perf_pmu_unregister(&pmu->pmu); + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + hrtimer_cancel(&pmu->hrtimer); + cancel_work_sync(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = of_match_ptr(rpi_axi_pmu_match), + .suppress_bind_attrs = true, + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state = ret; + + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver 2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 5:36 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 5:36 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider: - [High] Transient IPC mailbox errors corrupt PMU counter values with massive positive spikes. - [High] Module unload triggers a Use-After-Free in CPU hotplug callback. - [High] Global hardware monitor is never enabled for batched or grouped event additions. - [High] Concurrent timer manipulation triggers WARN_ON_ONCE and timer failure. - [Medium] Lockless modification of event->hw.state overwrites PERF_HES_STOPPED flag. - [Medium] VPU hardware monitor events fail to collect data for up to 2 seconds due to missing explicit scheduling. - [Medium] Unconditional lock drop in worker thread causes lockdep splat and unlocked MMIO access on fallback paths. -- --- Patch [1]: [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver --- commit 8696502c11139b0621905345c84a98e3653345a3 Author: Ian Rogers <irogers@google.com> perf: Add Raspberry Pi BCM2835 AXI PMU driver This commit adds an uncore performance monitoring unit (PMU) driver for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox monitors found on Raspberry Pi 1 through 4. > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > new file mode 100644 > --- /dev/null > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) > +{ > + perf_pmu_unregister(&pmu->pmu); > + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); [Severity: High] The perf_pmu_unregister() call frees the PMU structures, but cpuhp_state_remove_instance() will subsequently invoke rpi_axi_pmu_offline_cpu() on active CPUs. In rpi_axi_pmu_offline_cpu(), we see: perf_pmu_migrate_context(&pmu->pmu, cpu, target); Could this cause a use-after-free by dereferencing pmu->pmu after it has been unregistered? Should the cpuhp instance be removed before unregistering the PMU? [ ... ] > +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, > + int idx, int bus, int filter) > +{ > + int bus_control; > + > + if (pmu->monitor[mon].hw_events.enabled[idx]) > + return; > + > + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F); > + if (filter) { > + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; > + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; > + } > + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); > + set_bus_watcher_control(pmu, mon, idx, bus_control); > + > + if (pmu->monitor[mon].hw_events.num_monitored == 1) { > + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); > + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); > + } > +} [Severity: High] If grouped events are added, rpi_axi_pmu_add() will increment num_monitored multiple times before rpi_axi_pmu_start() is called for the group. When start() invokes this function, num_monitored could already be greater than 1. Does this mean the global monitor control will never be enabled for grouped event additions, leaving the hardware disabled? [ ... ] > +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, > + enum counter counter) > +{ [ ... ] > + err = rpi_firmware_property(pmu->firmware, > + RPI_FIRMWARE_GET_PERIPH_REG, > + tmp, sizeof(tmp)); > + > + if (err < 0 || tmp[1] != 1) { > + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); > + return 0; > + } > + ret = tmp[2] & 0x7FFFFFFF; [Severity: High] If the mailbox communication fails, this function returns 0. Over in rpi_axi_pmu_vpu_work_handler(), this 0 is blindly used to calculate the new count: new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); ... prev_count = local64_read(&event->hw.prev_count); delta = (new_count - prev_count) & 0x7FFFFFFF; local64_add(delta, &event->count); Can this lead to a massive positive spike in the counter value when transient IPC errors occur, since 0 minus prev_count evaluates to a huge positive delta? [ ... ] > +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) > +{ [ ... ] > + /* If VPU bus watcher is not enabled on hardware, enable it in process context */ > + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { > + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; > + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx]; > + > + spin_unlock_irq(&pmu->lock); > + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); > + spin_lock_irq(&pmu->lock); > + } [Severity: Medium] This drops the spinlock unconditionally. However, if the VPU monitor is falling back to direct MMIO where use_mailbox_interface is false, rpi_axi_pmu_enable_bus_watcher() calls set_bus_watcher_control() which asserts that pmu->lock is held. Does this lock drop violate the lockdep assertion and risk data races on non-firmware platforms? [ ... ] > +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) > +{ [ ... ] > + if (pmu->active_vpu_events > 0) > + schedule_work(&pmu->vpu_work); > + spin_unlock_irqrestore(&pmu->lock, flags); > + > + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); > + return HRTIMER_RESTART; > +} [Severity: High] The spinlock is dropped before calling hrtimer_forward_now(). If a concurrent delete and add operation occurs on another CPU during this window, could hrtimer_start() be called while the timer is still processing? If hrtimer_start() enqueues the timer concurrently, won't hrtimer_forward_now() run on an already queued timer and trigger a warning in the hrtimer core? [ ... ] > +static void rpi_axi_pmu_start(struct perf_event *event, int flags) > +{ > + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); > + enum monitor mon = config_to_monitor(event->attr.config); > + enum counter counter = config_to_counter(event->attr.config); > + unsigned long spinflags; > + > + spin_lock_irqsave(&pmu->lock, spinflags); > + event->hw.state = 0; > + pmu->monitor[mon].hw_events.num_started++; > + if (mon == MON__SYSTEM) { > + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; [Severity: Medium] When mon is MON__VPU, the hardware enablement is delegated entirely to pmu->vpu_work. However, schedule_work() is not explicitly called here. Does this mean VPU monitoring relies entirely on the periodic 2-second hrtimer to start collecting data? If so, won't short-lived perf workloads report 0 counts for VPU events because the hardware takes up to 2 seconds to be enabled? [ ... ] > +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) > +{ [ ... ] > + if (event->hw.state & PERF_HES_STOPPED) > + return; > + > + if (flags & PERF_EF_UPDATE) { > + rpi_axi_pmu_read(event); > + event->hw.state |= PERF_HES_UPTODATE; > + } > + > + spin_lock_irqsave(&pmu->lock, spinflags); > + event->hw.state |= PERF_HES_STOPPED; [Severity: Medium] rpi_axi_pmu_stop() modifies event->hw.state to add PERF_HES_UPTODATE without holding the spinlock. Meanwhile, rpi_axi_pmu_vpu_work_handler() performs read-modify-write operations on event->hw.state under the lock, such as: event->hw.state |= PERF_HES_UPTODATE; If stop() sets PERF_HES_STOPPED under the lock shortly after the worker reads the old state, will the worker thread overwrite the state and accidentally clear the PERF_HES_STOPPED flag when it writes back? -- Sashiko AI review · https://sashiko.dev/#/patchset/20260812052417.2361040-1-irogers@google.com?part=1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers @ 2026-08-12 5:24 ` Ian Rogers 2026-08-12 5:34 ` sashiko-bot 2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon 2 siblings, 1 reply; 20+ messages in thread From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw) To: irogers, linux-perf-users, linux-rpi-kernel Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig, will Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. Assisted-by: Antigravity:gemini-3.6-flash Signed-off-by: Ian Rogers <irogers@google.com> --- drivers/perf/Kconfig | 2 +- drivers/perf/rpi_axi_pmu.c | 296 +++++++++++++++++++++++++++++++++++-- 2 files changed, 287 insertions(+), 11 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 97c8bf34f463..84ffa65c7b79 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU select RASPBERRYPI_FIRMWARE if ARCH_BCM2835 help Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711). + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712). If compiled as a module, it will be called rpi_axi_pmu. diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index 827656b14d85..02ab73be70b2 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,12 +6,13 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- * The Broadcom AXI performance hardware provides up to two independent monitors: * 1. System Monitor (MON__SYSTEM = 0): - * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS). + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). * Directly memory-mapped via ARM physical IO memory space (MMIO). * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). * @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer, /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 = 0, + CHIP_BCM2712, }; enum monitor { @@ -126,6 +130,44 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 = 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; + /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4) * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus @@ -258,6 +300,94 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE = 0, + BCM2712_FLT__VPU_UC0 = 1, + BCM2712_FLT__VPU_IC0 = 2, + BCM2712_FLT__VPU_DC0 = 3, + BCM2712_FLT__VPU_UC1 = 4, + BCM2712_FLT__VPU_IC1 = 5, + BCM2712_FLT__VPU_DC1 = 6, + BCM2712_FLT__VPU_L2 = 7, + BCM2712_FLT__DMA2 = 8, + BCM2712_FLT__VPU_DEBUG = 9, + BCM2712_FLT__ARM = 10, + BCM2712_FLT__DMA0 = 11, + BCM2712_FLT__DMA1 = 12, + BCM2712_FLT__RAAGA = 13, + BCM2712_FLT__BBSI = 14, + BCM2712_FLT__PCIE0 = 15, + BCM2712_FLT__PCIE1 = 16, + BCM2712_FLT__PCIE2 = 17, + BCM2712_FLT__UMR = 18, + BCM2712_FLT__SAGE = 19, + BCM2712_FLT__HVDP = 20, + BCM2712_FLT__BSP = 21, + BCM2712_FLT__HVS = 22, + BCM2712_FLT__HVS_WMK = 23, + BCM2712_FLT__MOP0 = 24, + BCM2712_FLT__MOP1 = 25, + BCM2712_FLT__MBVN = 26, + BCM2712_FLT__DSI = 27, + BCM2712_FLT__XPT = 28, + BCM2712_FLT__EMMC0 = 29, + BCM2712_FLT__GENET = 30, + BCM2712_FLT__USB = 31, + BCM2712_FLT__MAX = 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivalence */ +static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); + /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 #define GEN_CTL_ENABLE_BIT BIT(0) @@ -487,16 +617,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) return false; - if (mon == MON__SYSTEM) { - if (bus >= BCM2835_SB__MAX) + if (pmu->chip == CHIP_BCM2712) { + if (mon != MON__SYSTEM) + return false; + if (bus >= BCM2712_SB__MAX) + return false; + if (filter >= BCM2712_FLT__MAX) return false; } else { - if (bus >= VPU__MAX) + if (mon == MON__SYSTEM) { + if (bus >= BCM2835_SB__MAX) + return false; + } else { + if (bus >= VPU__MAX) + return false; + } + if (filter >= BCM2835_FLT__MAX) return false; } - if (filter >= BCM2835_FLT__MAX) - return false; - return counter < CNT__MAX; } @@ -723,6 +861,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24"); PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24"); +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=14, BCM2712_SB__CPU_L2) + */ +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29"); + static struct attribute *rpi_axi_pmu_events_attrs[] = { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -879,12 +1055,107 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events. + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + struct device *dev = kobj_to_dev(kobj); + struct pmu *pmu = dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (rpi_pmu->chip == CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) + * and legacy RPi 1-4 System Monitor aliases + */ + if (strstr(pmu_attr->event_str, "monitor=1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system") || + strstr(attr->name, "ccp2tx_") || + strstr(attr->name, "mphi_") || + strstr(attr->name, "h264_") || + strstr(attr->name, "v3d_") || + strstr(attr->name, "jpeg_") || + strstr(attr->name, "peripheral_") || + strstr(attr->name, "cpu_uc_") || + strstr(attr->name, "cpu_l2_") || + strstr(attr->name, "l2_control_") || + strstr(attr->name, "system_control_") || + strstr(attr->name, "direct_control_") || + strstr(attr->name, "direct_data_")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_")) + return 0; + } + + return attr->mode; +} + static const struct attribute_group rpi_axi_pmu_events_group = { .name = "events", .attrs = rpi_axi_pmu_events_attrs, + .is_visible = rpi_axi_pmu_events_is_visible, }; /** @@ -897,9 +1168,10 @@ static const struct attribute_group rpi_axi_pmu_events_group = { */ static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct rpi_axi_pmu *pmu = dev_get_drvdata(dev); + struct pmu *pmu = dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu); - return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu)); + return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu)); } static DEVICE_ATTR_RO(cpumask); @@ -1575,7 +1847,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd spin_lock_init(&pmu->lock); mutex_init(&pmu->vpu_mutex); - pmu->chip = CHIP_BCM2835; + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); pmu->pmu = (struct pmu) { .module = THIS_MODULE, @@ -1718,6 +1990,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = { .compatible = "brcm,bcm2711-axiperf", .data = (void *)CHIP_BCM2835, }, + { + .compatible = "brcm,bcm2712-axiperf", + .data = (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); -- 2.55.0.679.g6767b8d81c-goog ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support 2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 5:34 ` sashiko-bot 0 siblings, 0 replies; 20+ messages in thread From: sashiko-bot @ 2026-08-12 5:34 UTC (permalink / raw) To: Ian Rogers; +Cc: linux-perf-users Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: New issues: - [Medium] The BCM2712 visibility filter incorrectly hides the JPEG hardware acceleration bus PMU events. - [Medium] Missing and mismatched PMU event aliases for BCM2712 PERIPHERAL, CPU_UC, and CPU_L2 buses. Pre-existing issues: - [High] PMU is exposed to userspace before driver initialization completes, allowing a Use-After-Free if a subsequent step fails. -- --- Patch [2]: [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support --- commit ac1314c26eb484b245288019eb6f6293df532017 Author: Ian Rogers <irogers@google.com> perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712 (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder, and Cortex-A76 DSU L3 interconnects. > diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c > index 827656b14d85d..02ab73be70b25 100644 > --- a/drivers/perf/rpi_axi_pmu.c > +++ b/drivers/perf/rpi_axi_pmu.c [ ... ] > @@ -723,6 +861,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra [ ... ] > +/* > + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events > + * (bus=14, BCM2712_SB__CPU_L2) > + */ > +/* BCM2712 Master ID Filtered Events */ > +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10"); > +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10"); [Severity: Medium] Does this mapping correctly reflect the intended hardware bus? The comment states that the Cortex-A76 Cores L2 cache events correspond to bus 14 (BCM2712_SB__CPU_L2), but the event strings specify bus=5 (BCM2712_SB__SYSTEM_L2). Will this yield incorrect measurements for the CPU L2 cache when queried via sysfs? [ ... ] > @@ -879,12 +1055,107 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = { [ ... ] > +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, > + struct attribute *attr, int unused) > +{ [ ... ] > + if (rpi_pmu->chip == CHIP_BCM2712) { > + /* > + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) > + * and legacy RPi 1-4 System Monitor aliases > + */ > + if (strstr(pmu_attr->event_str, "monitor=1") || [ ... ] > + strstr(attr->name, "v3d_") || > + strstr(attr->name, "jpeg_") || [Severity: Medium] Should the jpeg_ aliases be hidden on BCM2712? The BCM2712 hardware definitions (BCM2712_SB__JPEG = 2) indicate that the JPEG hardware acceleration bus is supported, but this logic explicitly hides it from userspace on this platform. Does this incorrectly deny access to valid hardware events? > + strstr(attr->name, "peripheral_") || > + strstr(attr->name, "cpu_uc_") || > + strstr(attr->name, "cpu_l2_") || [Severity: Medium] Is it intentional to hide these legacy aliases without providing replacements for BCM2712? The aliases for peripheral_, cpu_uc_, and cpu_l2_ are hidden here, which limits functionality on the new platform since equivalents weren't added in the newly defined BCM2712 expanded event aliases. [ ... ] > @@ -1575,7 +1847,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd > spin_lock_init(&pmu->lock); > mutex_init(&pmu->vpu_mutex); > > - pmu->chip = CHIP_BCM2835; > + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); > > pmu->pmu = (struct pmu) { > .module = THIS_MODULE, [Severity: High] This is a pre-existing issue, but does this function expose the PMU to userspace before driver initialization is complete? Looking at rpi_axi_pmu__init(), the PMU is registered before ensuring all initializations succeed: drivers/perf/rpi_axi_pmu.c:rpi_axi_pmu__init() { ... ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); if (ret) { ... } ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); if (ret) { dev_err(dev, "Failed to add cpuhp instance %d\n", ret); goto err_pmu_unregister; } ... } If cpuhp_state_add_instance() fails, the error path calls perf_pmu_unregister(). However, if userspace opens the event during this window, it will retain a reference to the struct pmu memory. That memory gets freed by the devres framework when the probe fails, potentially causing a use-after-free when userspace interacts with the open event. -- Sashiko AI review · https://sashiko.dev/#/patchset/20260812052417.2361040-1-irogers@google.com?part=2 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers @ 2026-08-12 8:27 ` Will Deacon 2 siblings, 0 replies; 20+ messages in thread From: Will Deacon @ 2026-08-12 8:27 UTC (permalink / raw) To: Ian Rogers Cc: linux-perf-users, linux-rpi-kernel, linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig On Tue, Aug 11, 2026 at 10:24:15PM -0700, Ian Rogers wrote: > This patch series adds an uncore Performance Monitoring Unit (PMU) driver > for Broadcom AXI system and VideoCore VPU performance monitors found on > Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5). Why are you sending four versions of this patch series, at -rc7, each in reply to the previous one? Nobody is going to review that. Will ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2026-08-12 8:27 UTC | newest] Thread overview: 20+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-11 14:32 ` Uwe Kleine-König 2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 0:04 ` sashiko-bot 2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 0:05 ` sashiko-bot 2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 0:40 ` sashiko-bot 2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 0:40 ` sashiko-bot 2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers 2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers 2026-08-12 5:36 ` sashiko-bot 2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers 2026-08-12 5:34 ` sashiko-bot 2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.